A Non-Sequential Neural Click Model for Web Search

Getting a better understanding of click behavior is important for advancing information retrieval systems. A lot of work has already been done on click models that predict what documents a user will click on, but these models do not take order into account. The aim of this work is to create a click model that not only predicts what documents of a Search Engine Result Page (SERP) the user will click on, but also in what order. To do this two Deep Learning models are implemented and compared to the recently presented Neural Click Model. The Neural Click Model (Borisov et al., 2016) was recently introduced and is based on neural networks and managed to outperform existing click models that rely on the Probabilistic Graphical Model (PGM) framework. Both new models are adaptations of the Neural Click Model and use a bidirectional Long Short-Term Memory (LSTM) module to encode the information about the query and the documents. One of the models also uses an attention mechanism to give more information about the documents at the time of predicting the clicks. The results show that predicting the order of clicks in a sequence is a very complex task and that overall it is difficult to improve on models that assume a linear examination and click path. Still it is shown that it is possible to create a neural model that is able to produce non sequential click sequences and performs on par with models that assume a sequential click sequence. This makes the non sequential neural click model a promising subject for further research. The second contribution of this research is to introduce a method for evaluating the performance of click models on non sequential click sequences. Since frequently used metrics as the likelihood and perplexity offer an unfair advantage to models with outputs that have no information about the order, two new metrics are introduced: The Damerau-Levenshtein Distance and the Longest Common Subsequence. Both these metrics compare two ordered lists and calculate a score that represents how close one is to the other while keeping order into account. Acknowledgements Firstly, I would like to thank my daily supervisor Ilya Markov. Throughout the thesis, which took quite longer than anticipated, he continued to make time for me to give me advise and feedback. Even when he was not in the country he was there to answer my questions and go over my progress. A special thanks goes out to Alexey Borisov. Even though he was not officially my supervisor, he made time to help me out with this project. Apart from giving advice and brainstorming with Ilya and myself he provided the data without which this project would not have been possible. I would like to thank prof. Maarten de Rijke for taking the time to read my thesis and be a part of the defense committee on short notice. I would also like to thank my fellow students with whom I have spent countless hours working on projects and assignments. Especially Sébastien and Joost, working with you has been a pleasure and has helped make my time at the University fruitful and enjoyable. Lastly, I would like to thank those closest to me. My parents who have always supported me and giving me advice throughout my life. My girlfriend who has always encouraged me and has put up with my complaining when nothing seemed to work the way I intended it to. Also my sister and my closest friends, mostly for mocking me when I complain to much. Thank you all for helping me get to where I am today.