OpenRaiser/Intern-Atlas huggingface dataset:
Multi-Level Matching Networks for Text Matching
Authors: Chunlin Xu, Zhiwei Lin, Shengli Wu, Hui Wang
Venue: SIGIR
Year: 2019

Text matching aims to establish the matching relationship between two texts. It is an important operation in some information retrieval related tasks such as question duplicate detection, question answering, and dialog systems. Bidirectional long short term memory (BiLSTM) coupled with attention mechanism has achieved state-of-the-art performance in text matching. A major limitation of existing works is that only high level contextualized word representations are utilized to obtain word level matching results without considering other levels of word representations, thus resulting in incorrect matching decisions for cases where two words with different meanings are very close in high level contextualized word representation space. Therefore, instead of making decisions utilizing single level word representations, a multi-level matching network (MMN) is proposed in this paper for text matching, which utilizes multiple levels of word representations to obtain multiple word level matching results for final text level matching decision. Experimental results on two widely used benchmarks, SNLI and Scaitail, show that the proposed MMN achieves the state-of-the-art performance.
Beyond Keyword Targeting: An End-to-End Ad Retrieval Framework for Sponsored Search
Authors: Xiao Yang, Zhi Guo, Zongyao Ding
Venue: SIGIR
Year: 2019

As the main revenue source for search engines, sponsored search system retrieves and allocates ads to display on the search result pages. Keyword targeting is widely adopted by most sponsored search systems as the basic model for expressing the advertiser's business and retrieving related ads. In this targeting model, the advertiser should cautiously select lots of keywords relevant to their business to optimize their campaigns, and the sponsored search system retrieves ads based on the relevance between queries and keywords. However, since there is a huge inventory of possible queries and the new queries grow dramatically, it is a great challenge for advertisers to identify and collect lots of relevant bid keywords for their ads, and it also takes great effort to select and maintain high-quality keywords and set corresponding match types for them. In the meantime, the keyword targeting leads to a multi-stage retrieval architecture as it contains the matching between query and keywords and the matching between keywords and ads. The retrieval funnel based on keyword targeting cannot achieve straightforward and optimal matching between search queries and ads. Consequently, traditional keyword targeting method gradually becomes the bottleneck of optimizing advertisers' campaigns and improving the monetization of search ads. In this paper, we present an end-to-end ad retrieval framework for sponsored search. This framework can break the limits of keyword targeting and can achieve direct matching from query to related ads. The framework has been deployed at Baidu's sponsored search system and the experimental result validates its effectiveness. We will describe the design and architecture of this framework, and hope that this framework can give some inspiration for the sponsored search industry.
Critically Examining the "Neural Hype": Weak Baselines and the Additivity of Effectiveness Gains from Neural Ranking Models
Authors: Wei Yang, Kuang Lu, Peilin Yang, Jimmy Lin
Venue: SIGIR
Year: 2019

Is neural IR mostly hype? In a recent SIGIR Forum article, Lin expressed skepticism that neural ranking models were actually improving ad hoc retrieval effectiveness in limited data scenarios. He provided anecdotal evidence that authors of neural IR papers demonstrate "wins" by comparing against weak baselines. This paper provides a rigorous evaluation of those claims in two ways: First, we conducted a meta-analysis of papers that have reported experimental results on the TREC Robust04 test collection. We do not find evidence of an upward trend in effectiveness over time. In fact, the best reported results are from a decade ago and no recent neural approach comes close. Second, we applied five recent neural models to rerank the strong baselines that Lin used to make his arguments. A significant improvement was observed for one of the models, demonstrating additivity in gains. While there appears to be merit to neural IR approaches, at least some of the gains reported in the literature appear illusory.
Local Matrix Approximation based on Graph Random Walk
Authors: Xuejiao Yang, Bang Wang
Venue: SIGIR
Year: 2019

How to decompose a large global matrix into many small local matrices has been recently researched a lot for matrix approximation. However, the distance computation in matrix decomposition is a challenging issue, as no prior knowledge about the most appropriate feature vectors and distance measures are available. In this paper, we propose a novel scheme for local matrix construction without involving distance computation. The basic idea is based on the application of convergence probabilities of graph random walk. At first, a user-item bipartite graph is constructed from the global matrix. After performing random walk on the bipartite graph, we select some user-item pairs as anchors. Then another random walk with restart is applied to construct the local matrix for each anchor. Finally, the global matrix approximation is obtained by averaging the prediction results of local matrices. Our experiments on the four real-world datasets show that the proposed solution outperforms the state-of-the-art schemes in terms of lower prediction errors and higher coverage ratios.
Adversarial Collaborative Neural Network for Robust Recommendation
Authors: Feng Yuan, Lina Yao, Boualem Benatallah
Venue: SIGIR
Year: 2019

Most of recent neural network(NN)-based recommendation techniques mainly focus on improving the overall performance, such as hit ratio for top-N recommendation, where the users' feedbacks are considered as the ground-truth. In real-world applications, those feedbacks are possibly contaminated by imperfect user behaviours, posing challenges on the design of robust recommendation methods. Some methods apply man-made noises on the input data to train the networks more effectively (e.g. the collaborative denoising auto-encoder). In this work, we propose a general adversarial training framework for NN-based recommendation models, improving both the model robustness and the overall performance. We apply our approach on the collaborative auto-encoder model, and show that the combination of adversarial training and NN-based models outperforms highly competitive state-of-the-art recommendation methods on three public datasets.
SAIN: Self-Attentive Integration Network for Recommendation
Authors: Seongjun Yun, Raehyun Kim, Miyoung Ko, Jaewoo Kang
Venue: SIGIR
Year: 2019

With the growing importance of personalized recommendation, numerous recommendation models have been proposed recently. Among them, Matrix Factorization (MF) based models are the most widely used in the recommendation field due to their high performance. However, MF based models suffer from cold start problems where user-item interactions are sparse. To deal with this problem, content based recommendation models which use the auxiliary attributes of users and items have been proposed. Since these models use auxiliary attributes, they are effective in cold start settings. However, most of the proposed models are either unable to capture complex feature interactions or not properly designed to combine user-item feedback information with content information. In this paper, we propose Self-Attentive Integration Network (SAIN) which is a model that effectively combines user-item feedback information and auxiliary information for recommendation task. In SAIN, a self-attention mechanism is used in the feature-level interaction layer to effectively consider interactions between multiple features, while the information integration layer adaptively combines content and feedback information. The experimental results on two public datasets show that our model outperforms the state-of-the-art models by 2.13%
Extracting, Mining and Predicting Users' Interests from Social Networks
Authors: Fattane Zarrinkalam, Hossein Fani, Ebrahim Bagheri
Venue: SIGIR
Year: 2019

The abundance of user generated content on social networks provides the opportunity to build models that are able to accurately and effectively extract, mine and predict users' interests with the hopes of enabling more effective user engagement, better quality delivery of appropriate services and higher user satisfaction. While traditional methods for building user profiles relied on AI-based preference elicitation techniques that could have been considered to be intrusive and undesirable by the users, more recent advances are focused on a non-intrusive yet accurate way of determining users' interests and preferences. In this tutorial, we cover five important aspects related to the effective mining of user interests: (1) we introduce the information sources that are used for extracting user interests, (2) various types of user interest profiles that have been proposed in the literature, (3) techniques that have been adopted or proposed for mining user interests, (4) the scalability and resource requirements of the state of the art methods, and finally (5) the evaluation methodologies that are adopted in the literature for validating the appropriateness of the mined user interest profiles. We also introduce existing challenges, open research question and exciting opportunities for further work.
Neural-Network Lexical Translation for Cross-lingual IR from Text and Speech
Authors: Rabih Zbib, Lingjun Zhao, Damianos G. Karakos, William Hartmann, Jay DeYoung, Zhongqiang Huang, Zhuolin Jiang, Noah Rivkin, Le Zhang, Richard M. Schwartz, John Makhoul
Venue: SIGIR
Year: 2019

We propose a neural network model to estimate word translation probabilities for Cross-Lingual Information Retrieval (CLIR). The model estimates better probabilities for word translations than automatic word alignments alone, and generalizes to unseen source-target word pairs. We further improve the lexical neural translation model (and subsequently CLIR), by incorporating source word context, and by encoding the character sequences of input source words to generate translations of out-of-vocabulary words. To be effective, neural network models typically need training on large amounts of data labeled directly on the final task, in this case relevance to queries. In contrast, our approach only requires parallel data to train the translation model, and uses an unsupervised model to compute CLIR relevance scores. We report results on the retrieval of text and speech documents from three morphologically complex languages with limited training data resources (Swahili, Tagalog, and Somali) and short English queries. Despite training on only about 2M words of parallel training data for each language, we obtain neural network translation models that are very effective for this task. We also obtain further improvements using (i) a modified relevance model, which uses the probability of occurrence of a translation of each query term in the source document, and (ii) confusion networks (instead of 1-best output) that encode multiple transcription alternatives in the output of an Automatic Speech Recognition (ASR) system. We achieve overall MAP relative improvements of up to 24% on Swahili, 50% on Tagalog, and 39% on Somali over the baseline probabilistic model, and larger improvements over monolingual retrieval from machine translation output.
Syntax-Aware Aspect-Level Sentiment Classification with Proximity-Weighted Convolution Network
Authors: Chen Zhang, Qiuchi Li, Dawei Song
Venue: SIGIR
Year: 2019

It has been widely accepted that Long Short-Term Memory (LSTM) network, coupled with attention mechanism and memory module, is useful for aspect-level sentiment classification. However, existing approaches largely rely on the modelling of semantic relatedness of an aspect with its context words, while to some extent ignore their syntactic dependencies within sentences. Consequently, this may lead to an undesirable result that the aspect attends on contextual words that are descriptive of other aspects. In this paper, we propose a proximity-weighted convolution network to offer an aspect-specific syntax-aware representation of contexts. In particular, two ways of determining proximity weight are explored, namely position proximity and dependency proximity. The representation is primarily abstracted by a bidirectional LSTM architecture and further enhanced by a proximity-weighted convolution. Experiments conducted on the SemEval 2014 benchmark demonstrate the effectiveness of our proposed approach compared with a range of state-of-the-art models is available at https://github.com/GeneZC/PWCN.
SIGIR 2019 Tutorial on Explainable Recommendation and Search
Authors: Yongfeng Zhang, Jiaxin Mao, Qingyao Ai
Venue: SIGIR
Year: 2019

Explainable recommendation and search attempt to develop models or methods that not only generate high-quality recommendation or search results, but also intuitive explanations of the results for users or system designers, which can help to improve the system transparency, persuasiveness, trustworthiness, and effectiveness, etc. This is even more important in personalized search and recommendation scenarios, where users would like to know why a particular product, web page, news report, or friend suggestion exists in his or her own search and recommendation lists. The tutorial focuses on the research and application of explainable recommendation and search algorithms, as well as their application in real-world systems such as search engine, e-commerce and social networks. The tutorial aims at introducing and communicating explainable recommendation and search methods to the community, as well as gathering researchers and practitioners interested in this research direction for discussions, idea communications, and research promotions.
EARS 2019: The 2nd International Workshop on ExplainAble Recommendation and Search
Authors: Yongfeng Zhang, Yi Zhang, Min Zhang, Chirag Shah
Venue: SIGIR
Year: 2019

Explainable recommendation and search attempt to develop models or methods that not only generate high-quality recommendation or search results, but also interpretability of the models or explanations of the results for users or system designers, which can help to improve the system transparency, persuasiveness, trustworthiness, and effectiveness, etc. This is even more important in personalized search and recommendation scenarios, where users would like to know why a particular product, web page, news report, or friend suggestion exists in his or her own search and recommendation lists. The workshop focuses on the research and application of explainable recommendation, search, and a broader scope of IR tasks. It will gather researchers as well as practitioners in the field for discussions, idea communications, and research promotions. It will also generate insightful debates about the recent regulations regarding AI interpretability, to a broader community including but not limited to IR, machine learning, AI, Data Science, and beyond.
Deep Distribution Network: Addressing the Data Sparsity Issue for Top-N Recommendation
Authors: Lei Zheng, Chaozhuo Li, Chun-Ta Lu, Jiawei Zhang, Philip S. Yu
Venue: SIGIR
Year: 2019

Existing recommendation methods mostly learn fixed vectors for users and items in a low-dimensional continuous space, and then calculate the popular dot-product to derive user-item distances. However, these methods suffer from two drawbacks: (1) the data sparsity issue prevents from learning high-quality representations; and (2) the dot-product violates the crucial triangular inequality and therefore, results in a sub-optimal performance. In this work, in order to overcome the two aforementioned drawbacks, we propose Deep Distribution Network (DDN) to model users and items via Gaussian distributions. We argue that, compared to fixed vectors, distribution-based representations are more powerful to characterize users' uncertain interests and items' distinct properties. In addition, we propose a Wasserstein-based loss, in which the critical triangular inequality can be satisfied. In experiments, we evaluate DDN and comparative models on standard datasets. It is shown that DDN significantly outperforms state-of-the-art models, demonstrating the advantages of the proposed distribution-based representations and wassertein loss.
Human Behavior Inspired Machine Reading Comprehension
Authors: Yukun Zheng, Jiaxin Mao, Yiqun Liu, Zixin Ye, Min Zhang, Shaoping Ma
Venue: SIGIR
Year: 2019

Machine Reading Comprehension (MRC) is one of the most challenging tasks in both NLP and IR researches. Recently, a number of deep neural models have been successfully adopted to some simplified MRC task settings, whose performances were close to or even better than human beings. However, these models still have large performance gaps with human beings in more practical settings, such as MS MARCO and DuReader datasets. Although there are many works studying human reading behavior, the behavior patterns in complex reading comprehension scenarios remain under-investigated. We believe that a better understanding of how human reads and allocates their attention during reading comprehension processes can help improve the performance of MRC tasks. In this paper, we conduct a lab study to investigate human's reading behavior patterns during reading comprehension tasks, where 32 users are recruited to take 60 distinct tasks. By analyzing the collected eye-tracking data and answers from participants, we propose a two-stage reading behavior model, in which the first stage is to search for possible answer candidates and the second stage is to generate the final answer through a comparison and verification process. We also find that human's attention distribution is affected by both question-dependent factors (e.g., answer and soft matching signal with questions) and question-independent factors (e.g., position, IDF and Part-of-Speech tags of words). We extract features derived from the two-stage reading behavior model to predict human's attention signals during reading comprehension, which significantly improves performance in the MRC task. Findings in our work may bring insight into the understanding of human reading and information seeking processes, and help the machine to better meet users' information needs.
Legal Intelligence for E-commerce: Multi-task Learning by Leveraging Multiview Dispute Representation
Authors: Xin Zhou, Yating Zhang, Xiaozhong Liu, Changlong Sun, Luo Si
Venue: SIGIR
Year: 2019

Various e-commerce platforms produce millions of transactions per day with many transaction disputes. This generates the demand for effective and efficient dispute resolutions for e-commerce transactions. This paper proposes a novel research task of Legal Dispute Judgment (LDJ) prediction for e-commerce transactions, which connects two yet isolated domains, e-commerce data mining and legal intelligence. Different from traditional legal intelligence with the focus on textual evidence of the dispute itself, the new research utilizes multiview information such as past behavior information of seller and buyer as well as textual evidence of the current transaction. The multiview dispute representation is integrated into an innovative multi-task learning framework for predicting the legal result. An extensive set of experiments with a large dispute case dataset collected from a world leading e-commerce platform shows that the proposed model can more accurately characterize a dispute case through buyer, seller, and transaction viewpoints for legal judgment prediction against several alternatives.
Simulacra and Selection: Clothing Set Recommendation at Stitch Fix
Authors: Kevin Zielnicki
Venue: SIGIR
Year: 2019

Choosing a set of items from an inventory is a common problem faced by customers of nearly any retailer. Modern retailers aid and influence customer decisions, using techniques like recommender systems and market basket analysis to deliver personalized and contextual item suggestions. While such methods typically just augment a traditional browsing experience, Stitch Fix goes a step further by exclusively delivering curated selections of items, via algorithmically-assisted stylist recommendations. Our understanding of the human-in-the-loop element in this selection process is crucial to building effective tools to assist stylists in delivering personalized items. When stylists at Stitch Fix select an assortment of clothing for a client, they consider many nuanced relationships between the items, as well as with the client. For example, a stylist may fill a client's request for a work outfit by choosing pants and a blouse that pair well together, as well as fitting the client's stated preferences and her inferred sense of style. This can be described as a contextual subset selection problem, where the goal is to assemble a conditionally optimal set of items, given inventory constraints and context. I will discuss our nonlinear factorization approach
Investigating the Interplay Between Searchers' Privacy Concerns and Their Search Behavior
Authors: Steven Zimmerman, Alistair Thorpe, Chris Fox, Udo Kruschwitz
Venue: SIGIR
Year: 2019

Privacy concerns are becoming a dominant focus in search applications, thus there is a growing need to understand implications of efforts to address these concerns. Our research investigates a search system with privacy warning labels, an approach inspired by decision making research on food nutrition labels. This approach is designed to alert users to potential privacy threats in their search for information as one possible avenue to address privacy concerns. Our primary goal is to understand the extent to which attitudes towards privacy are linked to behaviors that protect privacy. In the present study, participants were given a set of fact-based decision tasks from the domain of health search. Participants were rotated through variations of search engine results pages (SERPs) including a SERP with a privacy warning light system. Lastly, participants completed a survey to capture attitudes towards privacy, behaviors to protect privacy, and other demographic information. In addition to the comparison of interactive search behaviors of a privacy warning SERP with a control SERP, we compared self-report privacy measures with interactive search behaviors. Participants reported strong concerns around privacy of health information while simultaneously placing high importance on the correctness of this information. Analysis of our interactive experiment and self-report privacy measures indicate that 1) choice of privacy-protective browsers has a significant link to privacy attitudes and privacy-protective behaviors in a SERP and 2) there are no significant links between reported concerns towards privacy and recorded behavior in an information retrieval system with warnings that enable users to protect their privacy.
Efficient and Effective Text-Annotation through Active Learning
Authors: Markus Zlabinger
Venue: SIGIR
Year: 2019

The most commonly used active learning criterion is uncertainty sampling, where a supervised model is used to predict uncertain samples that should be labeled next by a human annotator. When using active learning, two problems are encountered: First, the supervised model has a cold-start problem in the beginning of the active learning process, and second, the human annotators might make labeling mistakes. In my Ph.D. research, I address the two problems with the development of an unsupervised method for the computation of informative samples. The informative samples are first manually labeled and then used for both: The training of the initial active learning model and the training of the human annotators in form of a learning-by-doing session. The planned unsupervised method will be based on word-embeddings and it will be limited to the area of text classification.