You have a search engine's query log of the form:
where the userid is the IP address of the user and the timestamp is the time at which the action took place (in epoch).
The user either submits a query to the search engine (in which case query contains the query string submitted, otherwise this value is empty) or views a result list and clicks on a URL (stored in click, otherwise this value is empty). If the user clicked a URL, the log also shows the amount of time in seconds the user spent on the URL (the dwell-time).
A concrete toy example of a query log is the following:
324325,1417712111,tom jones singer,,,,
Here, user 312121 searches twice for Britney Spears (different spellings) and later clicks on the Wikipedia link and spends 25 seconds on it. The user 324325 searches for Tom Jones and visits Wikipedia for 1 second and then spends a lot of time on the official Tom Jones web site. The log is neither sorted by userid nor by time.
We want to know the number of unique queries submitted. To do this, we need to write a Hadoop job that contains: