OpenRaiser/Intern-Atlas huggingface dataset:
Informing the Design of Conversational IR Systems: Framework and Result Presentation
Authors: Souvick Ghosh
Venue: SIGIR
Year: 2019

Recent developments in conversational IR have raised questions about the nature of interactions which occur between the user and the system and the cognitive capabilities expected of such systems. In our research, we investigate the completeness of existing theoretical frameworks in explaining conversational search data propose modifications to such systems. The linear and transient nature of speech makes it cognitively challenging for the user to process a large amount of information. We propose a study to evaluate the users' preference of modalities when using conversational search systems. The study will help us to understand how results should be presented in a conversational search system. As we observe how users search using audio queries, interact with the intermediary, and process the results presented, we aim to develop an insight on how to present results more efficiently in a conversational search setting. We also plan on exploring the effectiveness and consistency of different media in a conversational search setting. Our observations will inform future designs and help to create a better understanding of such systems.
Ranking Robustness In Adversarial Retrieval Settings
Authors: Gregory Goren
Venue: SIGIR
Year: 2019

The adversarial retrieval setting over the Web entails many challenges. Some of these are due to, essentially, ranking competitions between document authors who modify them in response to rankings induced for queries of interest so as to rank-promote the documents. One unwarranted consequence is that rankings can rapidly change due to small, almost indiscernible changes, of documents. In recent work, we addressed the issue of ranking robustness under (adversarial) document manipulations for feature-based learning-to-rank approaches. We presented a formalism of different notions of ranking robustness that gave rise to a few theoretical findings. Empirical evaluation provided support to these findings. Our next goals are to devise learning-to-rank techniques for optimizing relevance and robustness simultaneously, study the connections between ranking robustness and fairness, and to devise additional testbeds for evaluating ranking robustness.
Mention Recommendation in Twitter with Cooperative Multi-Agent Reinforcement Learning
Authors: Tao Gui, Peng Liu, Qi Zhang, Liang Zhu, Minlong Peng, Yunhua Zhou, Xuanjing Huang
Venue: SIGIR
Year: 2019

In Twitter-like social networking services, the "@'' symbol can be used with the tweet to mention users whom the user wants to alert regarding the message. An automatic suggestion to the user of a small list of candidate names can improve communication efficiency. Previous work usually used several most recent tweets or randomly select historical tweets to make an inference about this preferred list of names. However, because there are too many historical tweets by users and a wide variety of content types, the use of several tweets cannot guarantee the desired results. In this work, we propose the use of a novel cooperative multi-agent approach to mention recommendation, which incorporates dozens of more historical tweets than earlier approaches. The proposed method can effectively select a small set of historical tweets and cooperatively extract relevant indicator tweets from both the user and mentioned users. Experimental results demonstrate that the proposed method outperforms state-of-the-art methods.
Deep Natural Language Processing for Search Systems
Authors: Weiwei Guo, Huiji Gao, Jun Shi, Bo Long
Venue: SIGIR
Year: 2019

Deep learning models have been very successful in many natural language processing tasks. Search engine works with rich natural language data, e.g., queries and documents, which implies great potential of applying deep natural language processing on such data to improve search performance. Furthermore, it opens an unprecedented opportunity to explore more advanced search experience, such as conversational search and chatbot. This tutorial offers an overview on deep learning based natural language processing for search systems from an industry perspective. We focus on how deep natural language processing powers search systems in practice. The tutorial introduces basic concepts, elaborates associated challenges, reviews the state-of-the-art approaches, covers end-to-end tasks in search systems with examples, and discusses the future trend.
Order-aware Embedding Neural Network for CTR Prediction
Authors: Wei Guo, Ruiming Tang, Huifeng Guo, Jianhua Han, Wen Yang, Yuzhou Zhang
Venue: SIGIR
Year: 2019

Product based models, which represent multi-field categorical data as embedding vectors of features, then model feature interactions in terms of vector product of shared embedding, have been extensively studied and have become one of the most popular techniques for CTR prediction. However, if the shared embedding is applied: (1) the angles of feature interactions of different orders may conflict with each other, (2) the gradients of feature interactions of high-orders may vanish, which result in learned feature interactions less effective. To solve these problems, we propose a novel technique named Order-aware Embedding (i.e., multi-embeddings are learned for each feature, and different embeddings are applied for feature interactions of different orders), which can be applied to various models and generates feature interactions more effectively. We further propose a novel order-aware embedding neural network (OENN) based on this embedding technique for CTR prediction. Extensive experiments on three publicly available datasets demonstrate the effectiveness of Order-aware Embedding and show that our OENN outperforms the state-of-the-art models.
FAQ Retrieval Using Attentive Matching
Authors: Sparsh Gupta, Vitor R. Carvalho
Venue: SIGIR
Year: 2019

The task of ranking question-answer pairs in response to an input query, aka FAQ (Frequently Asked Question) Retrieval, has traditionally been focused mostly on extracting relevance signals between query and questions based on extensive manual feature engineering. In this paper we propose multiple deep learning architectures designed for FAQ Retrieval that eliminate the need for feature engineering and are able to elegantly combine both query-question and query-answer similarities. We present experimental results showing that models that effectively combine both query-question and query-answer representations using attention mechanisms in a hierarchical manner yield the best results from all proposed models. We further verify the effectiveness of attention mechanisms for FAQ Retrieval by conducting experiments on a completely different attention-based architecture, originally designed for question duplicate detection tasks, and observing equally impressive experimental ranking results.
Searching for Communities: a Facebook Way
Authors: Viet Ha-Thuc, Srinath Aaleti, Rongda Zhu, Nade Sritanyaratana, Corey Chen
Venue: SIGIR
Year: 2019

Giving people the power to build community is central to Facebook's mission. Technically, searching for communities poses very different challenges compared to the standard IR problems. First, there is a vocabulary mismatch problem since most of the content of the communities is private. Second, the common labeling strategies based on human ratings and clicks do not work well due to limited public content available to third-party raters and users at search time. Finally, community search has a dual objective of satisfying searchers and growing the number of active communities. While A/B testing is a well known approach for assessing the former, it is an open question on how to measure progress on the latter. This talk discusses these challenges in depth and describes our solution.
Contextually Propagated Term Weights for Document Representation
Authors: Casper Hansen, Christian Hansen, Stephen Alstrup, Jakob Grue Simonsen, Christina Lioma
Venue: SIGIR
Year: 2019

Word embeddings predict a word from its neighbours by learning small, dense embedding vectors. In practice, this prediction corresponds to a semantic score given to the predicted word (or term weight). We present a novel model that, given a target word, redistributes part of that word's weight (that has been computed with word embeddings) across words occurring in similar contexts as the target word. Thus, our model aims to simulate how semantic meaning is shared by words occurring in similar contexts, which is incorporated into bag-of-words document representations. Experimental evaluation in an unsupervised setting against 8 state of the art baselines shows that our model yields the best micro and macro F1 scores across datasets of increasing difficulty.
Unsupervised Neural Generative Semantic Hashing
Authors: Casper Hansen, Christian Hansen, Jakob Grue Simonsen, Stephen Alstrup, Christina Lioma
Venue: SIGIR
Year: 2019

Fast similarity search is a key component in large-scale information retrieval, where semantic hashing has become a popular strategy for representing documents as binary hash codes. Recent advances in this area have been obtained through neural network based models: generative models trained by learning to reconstruct the original documents. We present a novel unsupervised generative semantic hashing approach, Ranking based Semantic Hashing (RBSH) that consists of both a variational and a ranking based component. Similarly to variational autoencoders, the variational component is trained to reconstruct the original document conditioned on its generated hash code, and as in prior work, it only considers documents individually. The ranking component solves this limitation by incorporating inter-document similarity into the hash code generation, modelling document ranking through a hinge loss. To circumvent the need for labelled data to compute the hinge loss, we use a weak labeller and thus keep the approach fully unsupervised. Extensive experimental evaluation on four publicly available datasets against traditional baselines and recent state-of-the-art methods for semantic hashing shows that RBSH significantly outperforms all other methods across all evaluated hash code lengths. In fact, RBSH hash codes are able to perform similarly to state-of-the-art hash codes while using 2-4x fewer bits.
Finding Camouflaged Needle in a Haystack?: Pornographic Products Detection via Berrypicking Tree Model
Authors: Guoxiu He, Yangyang Kang, Zhe Gao, Zhuoren Jiang, Changlong Sun, Xiaozhong Liu, Wei Lu, Qiong Zhang, Luo Si
Venue: SIGIR
Year: 2019

It is an important and urgent research problem for decentralized eCommerce services, e.g., eBay, eBid, and Taobao, to detect illegal products, e.g., unclassified pornographic products. However, it is a challenging task as some sellers may utilize and change camouflaged text to deceive the current detection algorithms. In this study, we propose a novel task to dynamically locate the pornographic products from very large product collections. Unlike prior product classification efforts focusing on textual information, the proposed model, BerryPIcking TRee MoDel (BIRD), utilizes both product textual content and buyers' seeking behavior information as berrypicking trees. In particular, the BIRD encodes both semantic information with respect to all branches sequence and the overall latent buyer intent during the whole seeking process. An extensive set of experiments have been conducted to demonstrate the advantage of the proposed model against alternative solutions. To facilitate further research of this practical and important problem, the codes and buyers' seeking behavior data have been made publicly available1.
Implicit Entity Recognition, Classification and Linking in Tweets
Authors: Hawre Hosseini
Venue: SIGIR
Year: 2019

Linking phrases to knowledge base entities is a process known as entity linking and has already been widely explored for various content types such as tweets. A major step in entity linking is to recognize and/or classify phrases that can be disambiguated and linked to knowledge base entities, i.e., Named Entity Recognition and Classification. Unlike common entity recognition and linking systems, however, we aim to recognize, classify, and link entities which are implicitly mentioned, and hence lack a surface form, to appropriate knowledge base entries. In other words, the objective of our work is to recognize and identify core entities of a tweet when those entities are not explicitly mentioned; this process is referred to as Implicit Named Entity Recognition and Linking.
Scalable Deep Multimodal Learning for Cross-Modal Retrieval
Authors: Peng Hu, Liangli Zhen, Dezhong Peng, Pei Liu
Venue: SIGIR
Year: 2019

Cross-modal retrieval takes one type of data as the query to retrieve relevant data of another type. Most of existing cross-modal retrieval approaches were proposed to learn a common subspace in a joint manner, where the data from all modalities have to be involved during the whole training process. For these approaches, the optimal parameters of different modality-specific transformations are dependent on each other and the whole model has to be retrained when handling samples from new modalities. In this paper, we present a novel cross-modal retrieval method, called Scalable Deep Multimodal Learning (SDML). It proposes to predefine a common subspace, in which the between-class variation is maximized while the within-class variation is minimized. Then, it trains m modality-specific networks for m modalities (one network for each modality) to transform the multimodal data into the predefined common subspace to achieve multimodal learning. Unlike many of the existing methods, our method can train different modality-specific networks independently and thus be scalable to the number of modalities. To the best of our knowledge, the proposed SDML could be one of the first works to independently project data of an unfixed number of modalities into a predefined common subspace. Comprehensive experimental results on four widely-used benchmark datasets demonstrate that the proposed method is effective and efficient in multimodal learning and outperforms the state-of-the-art methods in cross-modal retrieval.
Coarse-to-Fine Grained Classification
Authors: Yuqi Huo, Yao Lu, Yulei Niu, Zhiwu Lu, Ji-Rong Wen
Venue: SIGIR
Year: 2019

Fine-grained image classification and retrieval become topical in both computer vision and information retrieval. In real-life scenarios, fine-grained tasks tend to appear along with coarse-grained tasks when the observed object is coming closer. However, in previous works, the combination of fine-grained and coarse-grained tasks was often ignored. In this paper, we define a new problem called coarse-to-fine grained classification (C2FGC) which aims to recognize the classes of objects in multiple resolutions (from low to high). To solve this problem, we propose a novel Multi-linear Pooling with Hierarchy (MLPH) model. Specifically, we first design a multi-linear pooling module to include both trilinear and bilinear pooling, and then formulate the coarse-grained and fine-grained tasks within a unified framework. Experiments on two benchmark datasets show that our model achieves state-of-the-art results.
Identifying Entity Properties from Text with Zero-shot Learning
Authors: Wiradee Imrattanatrai, Makoto P. Kato, Masatoshi Yoshikawa
Venue: SIGIR
Year: 2019

We propose a method for identifying a set of entity properties from text. Identifying entity properties is similar to a relation extraction task that can be cast as a classification of sentences. Normally, this task can be achieved by distant supervised learning by automatically preparing training sentences for each property; however, it is impractical to prepare training sentences for every property. Therefore, we describe a zero-shot learning problem for this task and propose a neural network-based model that does not rely on a complete training set comprising training sentences for every property. To achieve this, we utilize embeddings of properties obtained from a knowledge graph embedding using different components of a knowledge graph structure. The embeddings of properties are combined with the model to enable identification of properties with no available training sentences. By using our newly constructed dataset as well as an existing dataset, experiments revealed that our model achieved a better performance for properties with no training sentences, relative to baseline results, even comparable to that achieved for properties with training sentences.
To Model or to Intervene: A Comparison of Counterfactual and Online Learning to Rank from User Interactions
Authors: Rolf Jagerman, Harrie Oosterhuis, Maarten de Rijke
Venue: SIGIR
Year: 2019

Learning to Rank (LTR) from user interactions is challenging as user feedback often contains high levels of bias and noise. At the moment, two methodologies for dealing with bias prevail in the field of LTR: counterfactual methods that learn from historical data and model user behavior to deal with biases; and online methods that perform interventions to deal with bias but use no explicit user models. For practitioners the decision between either methodology is very important because of its direct impact on end users. Nevertheless, there has never been a direct comparison between these two approaches to unbiased LTR. In this study we provide the first benchmarking of both counterfactual and online LTR methods under different experimental conditions. Our results show that the choice between the methodologies is consequential and depends on the presence of selection bias, and the degree of position bias and interaction noise. In settings with little bias or noise counterfactual methods can obtain the highest ranking performance; however, in other circumstances their optimization can be detrimental to the user experience. Conversely, online methods are very robust to bias and noise but require control over the displayed rankings. Our findings confirm and contradict existing expectations on the impact of model-based and intervention-based methods in LTR, and allow practitioners to make an informed decision between the two methodologies.
Family History Discovery through Search at Ancestry
Authors: Peng Jiang, Yingrui Yang, Gann Bierner, Fengjie Alex Li, Ruhan Wang, Azadeh Moghtaderi
Venue: SIGIR
Year: 2019

At Ancestry, we apply learning to rank algorithms to a new area to assist our customers in better understanding their family history. The foundation of our service is an extensive and unique collection of billions of historical records that we have digitized and indexed. Currently, our content collection includes 20 billion historical records. The record data consists of birth records, death records, marriage records, adoption records, census records, obituary records, among many others types. It is important for us to return relevant records from diversified record types in order to assist our customers to better understand their family history.
Video Dialog via Multi-Grained Convolutional Self-Attention Context Networks
Authors: Weike Jin, Zhou Zhao, Mao Gu, Jun Yu, Jun Xiao, Yueting Zhuang
Venue: SIGIR
Year: 2019

Video dialog is a new and challenging task, which requires an AI agent to maintain a meaningful dialog with humans in natural language about video contents. Specifically, given a video, a dialog history and a new question about the video, the agent has to combine video information with dialog history to infer the answer. And due to the complexity of video information, the methods of image dialog might be ineffectively applied directly to video dialog. In this paper, we propose a novel approach for video dialog called multi-grained convolutional self-attention context network, which combines video information with dialog history. Instead of using RNN to encode the sequence information, we design a multi-grained convolutional self-attention mechanism to capture both element and segment level interactions which contain multi-grained sequence information. Then, we design a hierarchical dialog history encoder to learn the context-aware question representation and a two-stream video encoder to learn the context-aware video representation. We evaluate our method on two large-scale datasets. Due to the flexibility and parallelism of the new attention mechanism, our method can achieve higher time efficiency, and the extensive experiments also show the effectiveness of our method.
Item Recommendation by Combining Relative and Absolute Feedback Data
Authors: Saikishore Kalloori, Tianyu Li, Francesco Ricci
Venue: SIGIR
Year: 2019

User preferences in the form of absolute feedback, s.a., ratings, are widely exploited in Recommender Systems (RSs). Recent research has explored the usage of preferences expressed with pairwise comparisons, which signal relative feedback. It has been shown that pairwise comparisons can be effectively combined with ratings, but, it is important to fine tune the technique that leverages both types of feedback. Previous approaches train a single model by converting ratings into pairwise comparisons, and then use only that type of data. However, we claim that these two types of preferences reveal different information about users interests and should be exploited differently. Hence, in this work, we develop a ranking technique that separately exploits absolute and relative preferences in a hybrid model. In particular, we propose a joint loss function which is computed on both absolute and relative preferences of users. Our proposed ranking model uses pairwise comparisons data to predict the user's preference order between pairs of items and uses ratings to push high rated (relevant) items to the top of the ranking. Experimental results on three different data sets demonstrate that the proposed technique outperforms competitive baseline algorithms on popular ranking-oriented evaluation metrics.
Why do Users Issue Good Queries?: Neural Correlates of Term Specificity
Authors: Lauri Kangassalo, Michiel M. A. Spapé, Giulio Jacucci, Tuukka Ruotsalo
Venue: SIGIR
Year: 2019

Despite advances in the past few decades in studying what kind of queries users input to search engines and how to suggest queries for the users, the fundamental question of what makes human cognition able to estimate goodness of query terms is largely unanswered. For example, a person searching information about "cats'' is able to choose query terms, such as "housecat'', "feline'', or "animal'' and avoid terms like "similar'', "variety'', and "distinguish''. We investigated the association between the specificity of terms occurring in documents and human brain activity measured via electroencephalography (EEG). We analyzed the brain activity data of fifteen participants, recorded in response to reading terms from Wikipedia documents. Term specificity was shown to be associated with the amplitude of evoked brain responses. The results indicate that by being able to determine which terms carry maximal information about, and can best discriminate between, documents, people have the capability to enter good query terms. Moreover, our results suggest that the effective query term selection process, often observed in practical search behavior studies, has a neural basis. We believe our findings constitute an important step in revealing the cognitive processing behind query formulation and evaluating informativeness of language in general.