OpenRaiser/Intern-Atlas huggingface dataset:
Enhanced Contextual Recommendation using Social Media Data
Authors: Anirban Chakraborty
Venue: SIGIR
Year: 2018

Making recommendations to a user which are more refined, personalized and contextually appropriate, has become an important problem in a variety of domains. Contextual Point of Interest (POI) recommendation is of particular interest for travel and tourism, to suggest places to visit for a user traveling to a new city. In this paper, I propose an approach which frames contextual POI recommendation as a traditional document ranking problem, where I represent each POI as a document. A significant aspect of this research investigates the generation of an enriched document representation for a POI, by harvesting information such as descriptions, users' reviews, and ratings from the web. I propose a log-linear retrieval model for document ranking using Kernel Density Estimation (KDE) which will eventually rank POIs. Additionally, I propose to use social media, such as Twitter, to supplement the recommendation process. People's spontaneous and unfiltered opinions about a POI give valuable information about that POI which is different from other online reviews. This unique type of user data will be used to enrich the document representation for POIs, and it is possible to incorporate the effect of social media into my retrieval model. I follow the TREC Contextual Suggestion track setup for my evaluation methodology.
CAN: Enhancing Sentence Similarity Modeling with Collaborative and Adversarial Network
Authors: Qin Chen, Qinmin Hu, Jimmy Xiangji Huang, Liang He
Venue: SIGIR
Year: 2018

The neural networks have attracted great attention for sentence similarity modeling in recent years. Most neural networks focus on the representation of each sentence, while the common features of a sentence pair are not well studied. In this paper, we propose a Collaborative and Adversarial Network (CAN), which explicitly models the common features between two sentences for enhancing sentence similarity modeling. To be specific, a common feature extractor is presented and embedded into our CAN model, which includes a generator and a discriminator playing a collaborative and adversarial game for common feature extraction. Experiments on three benchmark datasets, namely TREC-QA and WikiQA for answer selection and MSRP for paraphrase identification, show that our proposed model is effective to boost the performance of sentence similarity modeling. In particular, our proposed model outperforms the state-of-the-art approaches on TREC-QA without using any external resources or pre-training. For the other two datasets, our model is also comparable to if not better than the recent neural network approaches.
A User Study on Snippet Generation: Text Reuse vs. Paraphrases
Authors: Wei-Fan Chen, Matthias Hagen, Benno Stein, Martin Potthast
Venue: SIGIR
Year: 2018

The snippets in the result list of a web search engine are built with sentences from the retrieved web pages that match the query. Reusing a web page's text for snippets has been considered fair use under the copyright laws of most jurisdictions. As of recent, notable exceptions from this arrangement include Germany and Spain, where news publishers are entitled to raise claims under a so-called ancillary copyright. A similar legislation is currently discussed at the European Commission. If this development gains momentum, the reuse of text for snippets will soon incur costs, which in turn will give rise to new solutions for generating truly original snippets. A key question in this regard is whether the users will accept any new approach for snippet generation, or whether they will prefer the current model of "reuse snippets." The paper in hand gives a first answer. A crowdsourcing experiment along with a statistical analysis reveals that our test users exert no significant preference for either kind of snippet. Notwithstanding the technological difficulty, this result opens the door to a new snippet synthesis paradigm.
Ranking Without Learning: Towards Historical Relevance-based Ranking of Social Images
Authors: Min Min Chew, Sourav S. Bhowmick, Adam Jatowt
Venue: SIGIR
Year: 2018

Tag-based Social Image Retrieval (TagIR) aims to find relevant social images using keyword queries. State-of-the-art TagIR techniques typically rank query results based on relevance, temporal or popularity criteria. However, these criteria may not always be sufficient to match diverse search intents of users. In this paper, we present a novel ranking scheme that ranks query results (images) based on their historical relevance. Informally, an image is historically relevant if its visual content is relevant to the query and it depicts objects, scenes, or events that are related to human history. To this end, we propose a learning-agnostic technique that leverages Wikipedia to quantify historical relevance of images. We empirically demonstrate the effectiveness of our ranking scheme using Flickr dataset.
Lessons from Building a Large-scale Commercial IR-based Chatbot for an Emerging Market
Authors: Manoj Kumar Chinnakotla, Puneet Agrawal
Venue: SIGIR
Year: 2018

In this work, we highlight some interesting challenges faced when trying to build a large-scale commercial IR-based chatbot, Ruuh, for an emerging market like India which has unique characteristics such as high linguistic and cultural diversity, large section of young population and the second largest mobile market in the world. We set out to build a "human-like" AI agent which aspires to become the trusted friend of every Indian youth. To meet this objective, we realised that we need to think beyond the utilitarian notion of merely generating "relevant" responses and enable the agent to comprehend and meet a wider range of user social needs, like expressing happiness when user's favourite team wins, sharing a cute comment on showing the pictures of the user's pet and so on. The agent should also be well-versed with the informal language of the urban Indian youth which often includes slang and code-mixing across two or more languages (English and their native language). Finally, in order to be their trusted friend, the agent has to communicate with respect without offending their sentiments and emotions. Some of the above objectives pose significant research challenges in the areas of NLP, IR and AI. We take the audience through our journey of how we tackled some of the above challenges while building a large-scale commercial IR-based conversational agent. Our attempts to solve some of the above challenges have also resulted in some interesting research contributions in the form of publications and patents in the above areas. Our chat-bot currently has more than 1M users who have engaged in more than 70M conversations.
Universal Approximation Functions for Fast Learning to Rank: Replacing Expensive Regression Forests with Simple Feed-Forward Networks
Authors: Daniel Cohen, John Foley, Hamed Zamani, James Allan, W. Bruce Croft
Venue: SIGIR
Year: 2018

Learning to rank is a key component of modern information retrieval systems. Recently, regression forest models (i.e., random forests, LambdaMART and gradient boosted regression trees) have come to dominate learning to rank systems in practice, as they provide the ability to learn from large scale data while generalizing well to additional test queries. As a result, efficient implementations of these models is a concern in production systems, as evidenced by past work. We propose an alternate method for optimizing the execution of learned models: converting these expensive ensembles to a feed-forward neural network. This simple neural architecture is quite efficient to execute: we show that the resulting chain of matrix multiplies is quite efficient while maintaining the effectiveness of the original, more-expensive forest model. Our neural approach has the advantage of being easier to train than any direct neural models, since it can match the previously-learned regression rather than learn to generalize relevance judgments directly. We observe CPU document scoring speed improvements of up to 400x over traditional algorithms and up to 10x over state-of-the-art algorithms with no measurable loss in mean average precision. With a GPU available, our algorithm is able to score every document in a batch in parallel for another 10-100x improvement. While we are not the first work to observe that neural networks are efficient as well as being effective, our application of this observation to learning to rank is novel and will have large real-world impact.
Beyond Pooling
Authors: Gordon V. Cormack, Maura R. Grossman
Venue: SIGIR
Year: 2018

Dynamic Sampling is a novel, non-uniform, statistical sampling strategy in which documents are selected for relevance assessment based on the results of prior assessments. Unlike static and dynamic pooling methods that are commonly used to compile relevance assessments for the creation of information retrieval test collections, Dynamic Sampling yields a statistical sample from which substantially unbiased estimates of effectiveness measures may be derived. In contrast to static sampling strategies, which make no use of relevance assessments, Dynamic Sampling is able to select documents from a much larger universe, yielding superior test collections for a given budget of relevance assessments. These assertions are supported by simulation studies using secondary data from the TREC 2017 Common Core Track.
Ad Click Prediction in Sequence with Long Short-Term Memory Networks: an Externality-aware Model
Authors: Weiwei Deng, Xiaoliang Ling, Yang Qi, Tunzi Tan, Eren Manavoglu, Qi Zhang
Venue: SIGIR
Year: 2018

Ad click prediction is a task to estimate the click-through rate (CTR) in sponsored ads, the accuracy of which impacts user search experience and businesses' revenue. State-of-the-art sponsored search systems typically model it as a classification problem and employ machine learning approaches to predict the CTR per ad. In this paper, we propose a new approach to predict ad CTR in sequence which considers user browsing behavior and the impact of top ads quality to the current one. To the best of our knowledge, this is the first attempt in the literature to predict ad CTR by using Recurrent Neural Networks (RNN) with Long Short-Term Memory (LSTM) cells. The proposed model is evaluated on a real dataset and we show that LSTM-RNN outperforms DNN model on both AUC and RIG. Since the RNN inference is time consuming, a simplified version is also proposed, which can achieve more than half of the gain with the overall serving cost almost unchanged.
The Second Workshop on Knowledge Graphs and Semantics for Text Retrieval, Analysis, and Understanding (KG4IR)
Authors: Laura Dietz, Chenyan Xiong, Jeff Dalton, Edgar Meij
Venue: SIGIR
Year: 2018

Semantic technologies such as controlled vocabularies, thesauri, and knowledge graphs have been used throughout the history of information retrieval for a variety of tasks. Recent advances in knowledge acquisition, alignment, and utilization have given rise to a body of new approaches for utilizing knowledge graphs in text retrieval tasks and it is therefore time to consolidate the community efforts and study how such technologies can be employed in information retrieval systems in the most effective way. It is also time to start and deepen the dialogue between researchers and practitioners in order to ensure that breakthroughs, technologies, and algorithms in this space are widely disseminated. The goal of this workshop is to bring together and grow a community of researchers and practitioners who are interested in using, aligning, and constructing knowledge graphs and similar semantic resources for information retrieval applications.
CA-LSTM: Search Task Identification with Context Attention based LSTM
Authors: Cong Du, Peng Shu, Yong Li
Venue: SIGIR
Year: 2018

Search task identification aims to understand a user's information needs to improve search quality for applications such as query suggestion, personalized search, and advertisement retrieval. To properly identify the search task within long query sessions, it is important to partition these sessions into segments before further processing. In this paper, we present the first search session segmentation model that uses a long short-term memory (LSTM) network with an attention mechanism. This model considers sequential context knowledge, including temporal information, word and character, and essentially learns which parts of the input query sequence are relevant to the current segmentation task and determines the influence of these parts. This segmentation technique is also combined with an efficient clustering method using a novel query relevance metric for end-to-end search task identification. Using real-world datasets, we demonstrate that our segmentation technique improves task identification accuracy of existing clustering-based algorithms.
Convolution-based Memory Network for Aspect-based Sentiment Analysis
Authors: Chuang Fan, Qinghong Gao, Jiachen Du, Lin Gui, Ruifeng Xu, Kam-Fai Wong
Venue: SIGIR
Year: 2018

Memory networks have shown expressive performance on aspect based sentiment analysis. However, ordinary memory networks only capture word-level information and lack the capacity for modeling complicated expressions which consist of multiple words. Targeting this problem, we propose a novel convolutional memory network which incorporates an attention mechanism. This model sequentially computes the weights of multiple memory units corresponding to multi-words. This model may capture both words and multi-words expressions in sentences for aspect-based sentiment analysis. Experimental results show that the proposed model outperforms the state-of-the-art baselines.
Semantic Location in Email Query Suggestion
Authors: John Foley, Mingyang Zhang, Michael Bendersky, Marc Najork
Venue: SIGIR
Year: 2018

Mobile devices are pervasive, which means that users have access to web content and their personal documents at all locations, not just their home or office. Existing work has studied how locations can influence information needs, focusing on web queries. We explore whether or not location information can be helpful to users who are searching their own personal documents. We wish to study whether a users' location can predict their queries over their own personal data, so we focus on the task of query suggestion. While we find that using location directly can be helpful, it does not generalize well to novel locations. To improve this situation, we explore using semantic location: that is, rather than memorizing location-query associations, we generalize our location information to names of the closest point of interest. By using short, semantic descriptions of locations, we find that we can more robustly improve query completion and observe that users are already using locations to extend their own queries in this domain. We present a simple but effective model that can use location to predict queries for a user even before they type anything into a search box, and which learns effectively even when not all queries have location information.
Assessing the Readability of Web Search Results for Searchers with Dyslexia
Authors: Adam Fourney, Meredith Ringel Morris, Abdullah X. Ali, Laura Vonessen
Venue: SIGIR
Year: 2018

Standards organizations, (e.g., the World Wide Web Consortium), are placing increased importance on the cognitive accessibility of online systems, including web search. Previous work has shown an association between query-document relevance judgments, and query-independent assessments of document readability. In this paper we study the lexical and aesthetic features of web documents that may underlie this relationship. Leveraging a data set consisting of relevance and readability judgments for 200 web pages as assessed by 174 adults with dyslexia and 172 adults without dyslexia, we answer the following research questions: (1) Which web page features are most associated with readability? (2) To what extent are these features also associated with relevance? And, (3) are any features associated with the differences in readability/relevance judgments observed between dyslexic and non-dyslexic populations? Our findings have implications for improving the cognitive accessibility of search systems and web documents.
Efficiency-Effectiveness Trade-Offs in Machine Learned Models for Information Retrieval
Authors: Luke Gallagher
Venue: SIGIR
Year: 2018

Complex machine learning models are now an integral part of modern, large scale retrieval systems. However, collection size growth continues to outpace advances in efficiency improvements in the learning models which achieve the highest effectiveness. The current literature on cascade ranking does not address the issue of how state-of-the-art learning-to-rank (LTR) models can be adapted to a cascaded architecture. Secondly, complex neural networks for retrieval tasks are now in their infancy similar to LTR 15 years earlier and are attaining effective results. New research challenges arise with these models that require vast amounts of training data. Most of the current research is focused on constructing new models that improve effectiveness, leaving the questions of explainability and efficiency as open problems. This research proposes to focus on improving the efficiency-effectiveness trade-offs within machine learned models for late stage re-ranking and aims to: i) devise a cascade ranking architecture that can be used with current state-ofthe-art LTR algorithms; ii) study the robustness of current ad-hoc neural ranking models; iii) propose solutions for current efficiency challenges in neural ranking models and investigate how featuredriven machine learning and neural methods may be combined for ranking tasks. Cascade ranking has recently become an important point of emphasis within Information Retrieval. Growing collection sizes continue to curb the advancements of more sophisticated learning methods where effectiveness continues to improve. Wang et al. [4] provide one of the earliest works for cascade ranking designed specifically for IR. The key advantage of such an architecture is flexibility – an allowance for more fine grained control over the trade-off between efficiency and effectiveness. However, existing methods for cascade ranking focus on retrofitting the cascade approach into an existing algorithm and are not generalizable across
Neural Approaches to Conversational AI
Authors: Jianfeng Gao, Michel Galley, Lihong Li
Venue: SIGIR
Year: 2018

The present paper surveys neural approaches to conversational AI that have been developed in the last few years. We group conversational systems into three categories: (1) question answering agents, (2) task-oriented dialogue agents, and (3) chatbots. For each category, we present a review of state-of-the-art neural approaches, draw the connection between them and traditional approaches, and discuss the progress that has been made and challenges still being faced, using specific systems and models as case studies.
Data Science for Social Good and Public Policy: Examples, Opportunities, and Challenges
Authors: Rayid Ghani
Venue: SIGIR
Year: 2018

Can data science help reduce police violence and misconduct? Can it help increase retention of patients in care? Can it help prevent children from getting lead poisoning? Can it help cities better target limited resources to improve lives of citizens? We're all aware of the hype around data science and related buzzwords right now but turning this hype into social impact takes cross-disciplinary training, teams, and methods. In this talk, I'll discuss lessons learned from our work at University of Chicago while working on dozens of data science projects over the past few years with non-profits and governments on high-impact public policy and social challenges in criminal justice, public health, education, economic development, public safety, workforce training, and urban infrastructure. I'll highlight opportunities for IR researchers to get involved in these efforts as well as information retrieval challenges that are open research problems that need to be solved in order to increase the effectiveness of today's machine learning and data science algorithms in order to have social and policy impact in a fair and equitable manner.
Modeling Mobile User Actions for Purchase Recommendation using Deep Memory Networks
Authors: Djordje Gligorijevic, Jelena Gligorijevic, Aravindan Raghuveer, Mihajlo Grbovic, Zoran Obradovic
Venue: SIGIR
Year: 2018

Rapid expansion of mobile devices has brought an unprecedented opportunity for mobile operators and content publishers to reach many users at any point in time. Understanding usage patterns of mobile applications (apps) is an integral task that precedes advertising efforts of providing relevant recommendations to users. However, this task can be very arduous due to the unstructured nature of app data, with sparseness in available information. This study proposes a novel approach to learn representations of mobile user actions using Deep Memory Networks. We validate the proposed approach on millions of app usage sessions built from large scale feeds of mobile app events and mobile purchase receipts. The empirical study demonstrates that the proposed approach performed better compared to several competitive baselines in terms of recommendation precision quality. To the best of our knowledge this is the first study analyzing app usage patterns for purchase recommendation.
The Evolution of Content Analysis for Personalized Recommendations at Twitter
Authors: Ajeet Grewal, Jimmy Lin
Venue: SIGIR
Year: 2018

We present a broad overview of personalized content recommendations at Twitter, discussing how our approach has evolved over the years, represented by several generations of systems. Historically, content analysis of Tweets has not been a priority, and instead engineering efforts have focused on graph-based recommendation techniques that exploit structural properties of the follow graph and engagement signals from users. These represent "low hanging fruits" that have enabled high-quality recommendations using simple algorithms. As deployed systems have grown in maturity and our understanding of the problem space has become more refined, we have begun to look for other opportunities to further improve recommendation quality. We overview recent investments in content analysis, particularly named-entity recognition techniques built around recurrent neural networks, and discuss how they integrate with existing graph-based capabilities to open up the design space of content recommendation algorithms.
DATA: SEARCH'18 - Searching Data on the Web
Authors: Paul Groth, Laura Koesten, Philipp Mayr, Maarten de Rijke, Elena Simperl
Venue: SIGIR
Year: 2018

This half day workshop explores challenges in data search, with a particular focus on data on the web. We want to stimulate an interdisciplinary discussion around how to improve the description, discovery, ranking and presentation of structured and semi-structured data, across data formats and domain applications. We welcome contributions describing algorithms and systems, as well as frameworks and studies in human data interaction. The workshop aims to bring together communities interested in making the web of data more discoverable, easier to search and more user friendly.