At SIGIR 2005, Professor Thorsten Joachims and his coworkers have a very good paper, Accurately Interpreting Clickthrough Data as Implicit Feedback. In this work, they study how the user clickthrough data is reliable as the source of implicit feedback. They do two related user studies. One user study utilizes eye tracker to record user eye movement to infer the user browsing behavior (viewing, clicking and the relationship between them). The other user study asks the participants to explicitly judge relevance of search results so that the correlation between implicit feedback and explicit relevance feedback can be studied. The main finding of this work is that the relative relevance judgment (e.g., one search result is more relevant than another search result) rather than absolute relevance is more accurate.
This paper mainly conducts the user study to get some insight into the clickthrough data as implicit feedback. They have a related paper Optimizing Search Engines Using Clickthrough Data in SIGKDD 2002, which talks about using SVM to train a retrieval ranking function according to clickthrough data. In the SIGKDD 2002 paper, they already used the idea of using relative relevance instead of absolute relevance.
A follow-up paper of SIGIR 2005 paper is Query Chains: Learning to Rank from Implicit Feedback in SIGKDD 2005. They used clickthrough data in training a retrieval function of a search engine. The SIGKDD 2005 paper can be considered as a combination of the SIGKDD 2002 and SIGIR 2005 paper.
This work can also be considered as a finer-granularity study of how to make use of clickthrough data in contextual search. Our paper Context-Sensitive Information Retrieval Using Implicit Feedback in SIGIR 2005 is focused on how to model clickthrough data into the contextual search.
