OpenRaiser/Intern-Atlas huggingface dataset:
Efficient and effective retrieval using selective pruning
Authors: Nicola Tonellotto, Craig Macdonald, Iadh Ounis
Venue: Conference WSDM 2013
Year: 2013

Retrieval can be made more efficient by deploying dynamic pruning strategies such as WAND, which do not degrade effectiveness up to a given rank. It is possible to increase the efficiency of such techniques by pruning more 'aggressively'. However, this may reduce effectiveness. In this work, we propose a novel selective framework that determines the appropriate amount of pruning aggressiveness on a per-query basis, thereby increasing overall efficiency without significantly reducing overall effectiveness. We postulate two hypotheses about the queries that should be pruned more aggressively, which generate two approaches within our framework, based on query performance predictors and query efficiency predictors, respectively. We thoroughly experiment to ascertain the efficiency and effectiveness impacts of the proposed approaches, as part of a search engine deploying state-of-the-art learning to rank techniques. Our results on 50 million documents of the TREC ClueWeb09 collection show that by using query efficiency predictors to target inefficient queries, we observe that a 36% reduction in mean response time and a 50% reduction of the response times experienced by the slowest 10% of queries can be achieved while still ensuring effectiveness.
Venue Appropriateness Prediction for Personalized Context-Aware Venue Suggestion
Authors: Mohammad Aliannejadi, Fabio Crestani
Venue: Conference SIGIR 2017
Year: 2017

Personalized context-aware venue suggestion plays a critical role in satisfying the users' needs on location-based social networks (LBSNs). In this paper, we present a set of novel scores to measure the similarity between a user and a candidate venue in a new city. The scores are based on user's history of preferences in other cities as well as user's context. We address the data sparsity problem in venue recommendation with the aid of a proposed approach to predict contextually appropriate places. Furthermore, we show how to incorporate different scores to improve the performance of recommendation. The experimental results of our participation in the TREC 2016 Contextual Suggestion track show that our approach beats state-of-the-art strategies.
Building a query log via crowdsourcing
Authors: Omar Alonso, Maria Stone
Venue: Conference SIGIR 2014
Year: 2014

A query log is a key asset in a commercial search engine. Everyday millions of users rely on search engines to find information on the Web by entering a few keywords on a simple search interface. Those queries represent a subset of user behavioral data which is used to mine and discover search patterns for improving the overall end user experience. While queries are very useful, it is not always possible to capture precisely what the user was looking for when the intent is not that clear. We explore a different alternative based on human computation to gather a bit more information from users and show the type of query log that would be possible to construct.
Searching on the Go: The Effects of Fragmented Attention on Mobile Web Search Tasks
Authors: Morgan Harvey, Matthew Pointon
Venue: Conference SIGIR 2017
Year: 2017

Smart phones and tablets are rapidly becoming our main method of accessing information and are frequently used to perform on-the-go search tasks. Mobile devices are commonly used in situations where attention must be divided, such as when walking down a street. Research suggests that this increases cognitive load and, therefore, may have an impact on performance. In this work we conducted a laboratory experiment with both device types in which we simulated everyday, common mobile situations that may cause fragmented attention, impact search performance and affect user perception.
Query type classification for web document retrieval
Authors: In-Ho Kang, Gil-Chang Kim
Venue: Conference SIGIR 2003
Year: 2003

The heterogeneous Web exacerbates IR problems and short user queries make them worse. The contents of web documents are not enough to find good answer documents. Link information and URL information compensates for the insufficiencies of content information. However, static combination of multiple evidences may lower the retrieval performance. We need different strategies to find target documents according to a query type. We can classify user queries as three categories, the topic relevance task, the homepage finding task, and the service finding task. In this paper, a user query classification scheme is proposed. This scheme uses the difference of distribution, mutual information, the usage rate as anchor texts, and the POS information for the classification. After we classified a user query, we apply different algorithms and information for the better results. For the topic relevance task, we emphasize the content information, on the other hand, for the homepage finding task, we emphasize the Link information and the URL information. We could get the best performance when our proposed classification method with the OKAPI scoring algorithm was used.
Mobile App Retrieval for Social Media Users via Inference of Implicit Intent in Social Media Text
Authors: Dae Hoon Park, Yi Fang, Mengwen Liu, ChengXiang Zhai
Venue: Conference CIKM 2016
Year: 2016

People often implicitly or explicitly express their needs in social media in the form of "user status text". Such text can be very useful for service providers and product manufacturers to proactively provide relevant services or products that satisfy people's immediate needs. In this paper, we study how to infer a user's intent based on the user's "status text" and retrieve relevant mobile apps that may satisfy the user's needs. We address this problem by framing it as a new entity retrieval task where the query is a user's status text and the entities to be retrieved are mobile apps. We first propose a novel approach that generates a new representation for each query. Our key idea is to leverage social media to build parallel corpora that contain implicit intention text and the corresponding explicit intention text. Specifically, we model various user intentions in social media text using topic models, and we predict user intention in a query that contains implicit intention. Then, we retrieve relevant mobile apps with the predicted user intention. We evaluate the mobile app retrieval task using a new data set we create. Experiment results indicate that the proposed model is effective and outperforms the state-of-the-art retrieval models.
Leveraging User Reviews to Improve Accuracy for Mobile App Retrieval
Authors: Dae Hoon Park, Mengwen Liu, ChengXiang Zhai, Haohong Wang
Venue: Conference SIGIR 2015
Year: 2015

Smartphones and tablets with their apps pervaded our everyday life, leading to a new demand for search tools to help users find the right apps to satisfy their immediate needs. While there are a few commercial mobile app search engines available, the new task of mobile app retrieval has not yet been rigorously studied. Indeed, there does not yet exist a test collection for quantitatively evaluating this new retrieval task. In this paper, we first study the effectiveness of the state-of-the-art retrieval models for the app retrieval task using a new app retrieval test data we created. We then propose and study a novel approach that generates a new representation for each app. Our key idea is to leverage user reviews to find out important features of apps and bridge vocabulary gap between app developers and users. Specifically, we jointly model app descriptions and user reviews using topic model in order to generate app representations while excluding noise in reviews. Experiment results indicate that the proposed approach is effective and outperforms the state-of-the-art retrieval models for app retrieval.
Building bridges for web query classification
Authors: Dou Shen, Jian-Tao Sun, Qiang Yang, Zheng Chen
Venue: Conference SIGIR 2006
Year: 2006

Web query classification (QC) aims to classify Web users' queries, which are often short and ambiguous, into a set of target categories. QC has many applications including page ranking in Web search, targeted advertisement in response to queries, and personalization. In this paper, we present a novel approach for QC that outperforms the winning solution of the ACM KDDCUP 2005 competition, whose objective is to classify 800,000 real user queries. In our approach, we first build a bridging classifier on an intermediate taxonomy in an offline mode. This classifier is then used in an online mode to map user queries to the target categories via the above intermediate taxonomy. A major innovation is that by leveraging the similarity distribution over the intermediate taxonomy, we do not need to retrain a new classifier for each new set of target categories, and therefore the bridging classifier needs to be trained only once. In addition, we introduce category selection as a new method for narrowing down the scope of the intermediate taxonomy based on which we classify the queries. Category selection can improve both efficiency and effectiveness of the online classification. By combining our algorithm with the winning solution of KDDCUP 2005, we made an improvement by 9.7% and 3.8% in terms of precision and F1 respectively compared with the best results of KDDCUP 2005.
From Queries to Cards: Re-ranking Proactive Card Recommendations Based on Reactive Search History
Authors: Milad Shokouhi, Qi Guo
Venue: Conference SIGIR 2015
Year: 2015

The growing accessibility of mobile devices has substantially reformed the way users access information. While the reactive search by query remains as common as before, recent years have witnessed the emergence of various proactive systems such as Google Now and Microsoft Cortana. In these systems, relevant content is presented to users based on their context without a query. Interestingly, despite the increasing popularity of such services, there is very little known about how users interact with them.
Mobile query reformulations
Authors: Milad Shokouhi, Rosie Jones, Umut Ozertem, Karthik Raghunathan, Fernando Diaz
Venue: Conference SIGIR 2014
Year: 2014

Users frequently interact with web search systems on their mobile devices via multiple modalities, including touch and speech. These interaction modes are substantially different from the user experience on desktop search. As a result, system designers have new challenges and questions around understanding the intent on these platforms. In this paper, we study the query reformulation patterns in mobile logs. We group query reformulations based on their input method into four categories; text-text, text-voice, voice-text and voice-voice. We discuss the unique characteristics of each of these groups by comparing them against each other and desktop logs. We also compare the distribution of reformulation types (e.g. adding/dropping words) against desktop logs and show that there are new classes of reformulations that are caused by errors in speech recognition. Our results suggest that users do not tend to switch between different input types (e.g. voice and text). Voice to text switches are largely caused by speech recognition errors, and text to voice switches are unlikely to be about the same intent.
Jobminer: A real-time system for mining job-related patterns from social media
Authors: Yu Cheng, Yusheng Xie, Zhengzhang Chen, Ankit Agrawal, Alok N. Choudhary, Songtao Guo
Venue: Conference KDD 2013
Year: 2013

The various kinds of booming social media not only provide a platform where people can communicate with each other, but also spread useful domain information, such as career and job market information. For example, LinkedIn publishes a large amount of messages either about people who want to seek jobs or companies who want to recruit new members. By collecting information, we can have a better understanding of the job market and provide insights to job-seekers, companies and even decision makers. In this paper, we analyze the job information from the social network point of view. We first collect the job-related information from various social media sources. Then we construct an inter-company job-hopping network, with the vertices denoting companies and the edges denoting flow of personnel between companies. We subsequently employ graphmining techniques to mine influential companies and related company groups based on the job-hopping network model. Demonstration on LinkedIn data shows that our system JobMiner can provide a better understanding of the dynamic processes and a more accurate identification of important entities in the job market.
Is it time for a career switch?
Authors: Jian Wang, Yi Zhang, Christian Posse, Anmol Bhasin
Venue: Conference WWW 2013
Year: 2013

Tenure is a critical factor for an individual to consider when making a job transition. For instance, software engineers make a job transition to senior software engineers in a span of 2 years on average, or it takes for approximately 3 years for realtors to switch to brokers. While most existing work on recommender systems focuses on finding what to recommend to a user, this paper places emphasis on when to make appropriate recommendations and its impact on the item selection in the context of a job recommender system. The approach we propose, however, is general and can be applied to any recommendation scenario where the decision-making process is dependent on the tenure (i.e., the time interval) between successive decisions.
Personalized interactive faceted search
Authors: Jonathan Koren, Yi Zhang, Xue Liu
Venue: Conference WWW 2008
Year: 2008

Faceted search is becoming a popular method to allow users to interactively search and navigate complex information spaces. A faceted search system presents users with key-value metadata that is used for query refinement. While popular in e-commerce and digital libraries, not much research has been conducted on which metadata to present to a user in order to improve the search experience. Nor are there repeatable benchmarks for evaluating a faceted search engine. This paper proposes the use of collaborative filtering and personalization to customize the search interface to each user's behavior. This paper also proposes a utility based framework to evaluate the faceted interface. In order to demonstrate these ideas and better understand personalized faceted search, several faceted search algorithms are proposed and evaluated using the novel evaluation methodology.
Towards Measuring and Inferring User Interest from Gaze
Authors: Yixuan Li, Pingmei Xu, Dmitry Lagun, Vidhya Navalpakkam
Venue: Conference WWW 2017
Year: 2017

How can we reliably infer web users' interest and evaluate the content relevance when lacking active user interaction such as click behavior? In this paper, we investigate the relationship between mobile users' implicit interest inferred from attention metrics, such as eye gaze or viewport time, and explicit interest expressed by users. We present the first quantitative gaze tracking study using front-facing camera of mobile devices instead of specialized, expensive eye-tracking devices. We focus on multi-column digital media pages in Google Play Store that display 30+ items per page belonging to diverse categories. In such pages, we find significantly different distribution of gaze metrics on items that users rate as interesting vs. not. We leverage this insight by building a prediction model that is able to infer a user's interest ratings from the the non-click actions of the user. Our model is able to attain AUC of 90.32% in predicting user interest at an individual item level. In addition, our experiments on collection item re-ranking show how user gaze and viewport signals can be used to personalize item ranking on the collection page. Beyond understanding users' attention behavior in novel contexts such as multi-column digital media pages in Google Play Store, the findings in this study have implications for the design of a novel personalization and recommendation mechanism by (1) prioritizing items that are most likely of interest to users based on historical attention signals, and (2) prioritizing positions receiving significant portion of gaze attention.
Understanding and Predicting Usefulness Judgment in Web Search
Authors: Jiaxin Mao, Yiqun Liu, Huan-Bo Luan, Min Zhang, Shaoping Ma, Hengliang Luo, Yuntao Zhang
Venue: Conference SIGIR 2017
Year: 2017

Usefulness judgment measures the user-perceived amount of useful information for the search task in the current search context. Understanding and predicting usefulness judgment are crucial for developing user-centric evaluation methods and providing contextualize results according to the search context. With a dataset collected in a laboratory user study, we systematically investigate the effects of a variety of content, context, and behavior factors on usefulness judgments and find that while user behavior factors are most important in determining usefulness judgments, content and context factors also have significant effects on it. We further adopt these factors as features to build prediction models for usefulness judgments. An AUC score of 0.909 in binary usefulness classification and a Pearson's correlation coefficient of 0.694 in usefulness regression demonstrate the effectiveness of our models. Our study sheds light on the understanding of the dynamics of the user-perceived usefulness of documents in a search session and provides implications for the evaluation and design of Web search engines.
Silence is also evidence: interpreting dwell time for recommendation from psychological perspective
Authors: Peifeng Yin, Ping Luo, Wang-Chien Lee, Min Wang
Venue: Conference KDD 2013
Year: 2013

Social media is a platform for people to share and vote content. From the analysis of the social media data we found that users are quite inactive in rating/voting. For example, a user on average only votes 2 out of 100 accessed items. Traditional recommendation methods are mostly based on users' votes and thus can not cope with this situation. Based on the observation that the dwell time on an item may reflect the opinion of a user, we aim to enrich the user-vote matrix by converting the dwell time on items into users' ``pseudo votes'' and then help improve recommendation performance. However, it is challenging to correctly interpret the dwell time since many subjective human factors, e.g. user expectation, sensitivity to various item qualities, reading speed, are involved into the casual behavior of online reading. In psychology, it is assumed that people have choice threshold in decision making. The time spent on making decision reflects the decision maker's threshold. This idea inspires us to develop a View-Voting model, which can estimate how much the user likes the viewed item according to her dwell time, and thus make recommendations even if there is no voting data available. Finally, our experimental evaluation shows that the traditional rate-based recommendation's performance is greatly improved with the support of VV model.
Heads and tails: studies of web search with common and rare queries
Authors: Doug Downey, Susan T. Dumais, Eric Horvitz
Venue: Conference SIGIR 2007
Year: 2007

A large fraction of queries submitted to Web search enginesoccur very infrequently. We describe search log studiesaimed at elucidating behaviors associated with rare andcommon queries. We present several analyses and discussresearch directions.
Active hashing with joint data example and tag selection
Authors: Qifan Wang, Luo Si, Zhiwei Zhang, Ning Zhang
Venue: Conference SIGIR 2014
Year: 2014

Similarity search is an important problem in many large scale applications such as image and text retrieval. Hashing method has become popular for similarity search due to its fast search speed and low storage cost. Recent research has shown that hashing quality can be dramatically improved by incorporating supervised information, e.g. semantic tags/labels, into hashing function learning. However, most existing supervised hashing methods can be regarded as passive methods, which assume that the labeled data are provided in advance. But in many real world applications, such supervised information may not be available.
Nonlinear Robust Discrete Hashing for Cross-Modal Retrieval
Authors: Zhan Yang 0001, Jun Long, Lei Zhu 0005, Wenti Huang
Venue: Conference SIGIR 2020
Year: 2020

Hashing techniques have recently been successfully applied to solve similarity search problems in the information retrieval field because of their significantly reduced storage and high-speed search capabilities. However, the hash codes learned from most recent cross-modal hashing methods lack the ability to comprehensively preserve adequate information, resulting in a less than desirable performance. To solve this limitation, we propose a novel method termed Nonlinear Robust Discrete Hashing (NRDH), for cross-modal retrieval. The main idea behind NRDH is motivated by the success of neural networks, i.e., nonlinear descriptors, in the field of representation learning, and the use of nonlinear descriptors instead of simple linear transformations is more in line with the complex relationships that exist between common latent representation and heterogeneous multimedia data in the real world. In NRDH, we first learn a common latent representation through nonlinear descriptors to encode complementary and consistent information from the features of the heterogeneous multimedia data. Moreover, an asymmetric learning scheme is proposed to correlate the learned hash codes with the common latent representation. Empirically, we demonstrate that NRDH is able to successfully generate a comprehensive common latent representation that significantly improves the quality of the learned hash codes. Then, NRDH adopts a linear learning strategy to fast learn the hash function with the learned hash codes. Extensive experiments performed on two benchmark datasets highlight the superiority of NRDH over several state-of-the-art methods.
Recent advances in conversational information retrieval
Authors: Jianfeng Gao 0001, Chenyan Xiong, Paul Bennett 0001
Venue: Conference SIGIR 2020
Year: 2020

Recent progress in deep learning has brought tremendous improvements in conversational AI, leading to a plethora of commercial conversational services that allow naturally spoken interactions, increasing the need for more human-centric interactions in IR. As a result, we have witnessed a resurgent interest in developing modern CIR systems in research communities and industry. This tutorial presents recent advances in CIR, focusing mainly on neural approaches and new applications developed in the past five years. Our goal is to provide a thorough and in-depth overview of the general definition of CIR, the components of CIR systems, new applications raised for its conversational aspects, and the (neural) techniques recently developed for it.
Beyond DCG: User Behavior as a Predictor of a Successful Search
Authors: Ahmed Hassan Awadallah, Rosie Jones, Kristina Lisa Klinkner
Venue: Conference WSDM 2010
Year: 2010

Web search engines are traditionally evaluated in terms of the relevance of web pages to individual queries. However, relevance of web pages does not tell the complete picture, since an individual query may represent only a piece of the user's information need and users may have different information needs underlying the same queries. In this work, we address the problem of predicting user search goal success by modeling user behavior. We show empirically that user behavior alone can give an accurate picture of the success of the user's web search goals, without considering the relevance of the documents displayed. In fact, our experiments show that models using user behavior are more predictive of goal success than those using document relevance. We build novel sequence models incorporating time distributions for this task and our experiments show that the sequence and time distribution models are more accurate than static models based on user behavior, or predictions based on document relevance.