OpenRaiser/Intern-Atlas huggingface dataset:
How Deep Learning Works for Information Retrieval
Authors: Dacheng Tao
Venue: SIGIR
Year: 2020

Information retrieval (IR) is the science of search, the search of user query relevant pieces of information from a collection of unstructured resources. Information in this context includes text, imagery, audio, video, xml, program, and metadata. The journey of an IR process begins with a user query sent to the IR system which encodes the query, compares the query with the available resources, and returns the most relevant pieces of information. Thus, the system is equipped with the ability to store, retrieve and maintain information.
Improving Matching Models with Hierarchical Contextualized Representations for Multi-turn Response Selection
Authors: Chongyang Tao, Wei Wu 0014, Yansong Feng 0002, Dongyan Zhao 0001, Rui Yan 0001
Venue: SIGIR
Year: 2020

In this paper, we study context-response matching with pre-trained contextualized representations for multi-turn response selection in retrieval-based chatbots. Existing models, such as Cove and ELMo, are trained with limited context (often a single sentence or paragraph), and may not work well on multi-turn conversations, due to the hierarchical nature, informal language, and domain-specific words. To address the challenges, we propose pre-training hierarchical contextualized representations, including contextual word-level and sentence-level representations, by learning a dialogue generation model from large-scale conversations with a hierarchical encoder-decoder architecture. Then the two levels of representations are blended into the input and output layer of a matching model respectively. Experimental results on two benchmark conversation datasets indicate that the proposed hierarchical contextualized representations can bring significantly and consistently improvement to existing matching models for response selection.
Fair Classification with Counterfactual Learning
Authors: Maryam Tavakol
Venue: SIGIR
Year: 2020

Recent advances in machine learning have led to emerging new approaches to deal with different kinds of biases that exist in the data. On the one hand, counterfactual learning copes with biases in the policy used for sampling (or logging) the data in order to evaluate and learn new policies. On the other hand, fairness-aware learning aims at learning fair models to avoid discrimination against certain individuals or groups. In this paper, we design a counterfactual framework to model fairness-aware learning which benefits from counterfactual reasoning to achieve more fair decision support systems. We utilize a definition of fairness to determine the bandit feedback in the counterfactual setting that learns a classification strategy from the offline data, and balances classification performance versus fairness measure. In the experiments, we demonstrate that a counterfactual setting can be perfectly exerted to learn fair models with competitive results compared to a well-known baseline system.
Multi-grouping Robust Fair Ranking
Authors: Thibaut Thonet, Jean-Michel Renders
Venue: SIGIR
Year: 2020

Rankings are at the core of countless modern applications and thus play a major role in various decision making scenarios. When such rankings are produced by data-informed, machine learning-based algorithms, the potentially harmful biases contained in the data and algorithms are likely to be reproduced and even exacerbated. This motivated recent research to investigate a methodology for fair ranking, as a way to correct the aforementioned biases. Current approaches to fair ranking consider that the protected groups, i.e., the partition of the population potentially impacted by the biases, are known. However, in a realistic scenario, this assumption might not hold as different biases may lead to different partitioning into protected groups. Only accounting for one such partition (i.e., grouping) would still lead to potential unfairness with respect to the other possible groupings. Therefore, in this paper, we study the problem of designing fair ranking algorithms without knowing in advance the groupings that will be used later to assess their fairness. The approach that we follow is to rely on a carefully chosen set of groupings when deriving the ranked lists, and we empirically investigate which selection strategies are the most effective. An efficient two-step greedy brute-force method is also proposed to embed our strategy. As benchmark for this study, we adopted the dataset and setting composing the TREC 2019 Fair Ranking track.
Identifying Tasks from Mobile App Usage Patterns
Authors: Yuan Tian 0022, Ke Zhou 0003, Mounia Lalmas, Dan Pelleg
Venue: SIGIR
Year: 2020

Mobile devices have become an increasingly ubiquitous part of our everyday life. We use mobile services to perform a broad range of tasks (e.g. booking travel or office work), leading to often lengthy interactions within distinct apps and services. Existing mobile systems handle mostly simple user needs, where a single app is taken as the unit of interaction. To understand users' expectations and to provide context-aware services, it is important to model users' interactions in the task space. In this work, we first propose and evaluate a method for the automated segmentation of users' app usage logs into task units. We focus on two problems: (i) given a sequential pair of app usage logs, identify if there exists a task boundary, and (ii) given any pair of two app usage logs, identify if they belong to the same task. We model these as classification problems that use features from three aspects of app usage patterns: temporal, similarity, and log sequence. Our classifiers improve on traditional timeout segmentation, achieving over 89% performance for both problems. Secondly, we use our best task classifier on a large-scale data set of commercial mobile app usage logs to identify common tasks. We observe that users' performed common tasks ranging from regular information checking to entertainment and booking dinner. Our proposed task identification approach provides the means to evaluate mobile services and applications with respect to task completion.
Visual Intents vs. Clicks, Likes, and Purchases in E-commerce
Authors: Riku Togashi, Tetsuya Sakai
Venue: SIGIR
Year: 2020

In product-to-product search and recommendation, the product image often plays a pivotal role for the user to determine the relevance of that product. The present study investigates the relationship between the users' visual intents (in terms of colour, texture and material, and design) and the amount of user feedback (namely, clicks, likes, and purchases) using real product data and crowdsourcing. Through the analysis, we found that visual relevance (i.e., relevance of a target product with respect to a particular visual intent) correlates with the amount of user feedback, and that visual relevance can be the cause of user feedback.
Query/Task Satisfaction and Grid-based Evaluation Metrics Under Different Image Search Intents
Authors: Kosetsu Tsukuda, Masataka Goto
Venue: SIGIR
Year: 2020

People use web image search with various search intents: from serious demands for work to just passing time by browsing images of a favorite actor. Such a diversity of intents can influence user satisfaction and evaluation metrics, both of which are important factors for providing a better image search environment. In this paper, we investigate this influence by using a publicly available one-month field study dataset. With respect to satisfaction, we take into consideration both query-level and task-level satisfaction provided by search users. Regarding the evaluation metrics, we use grid-based evaluation metrics that incorporate user behavior specific to image search. The results of our analysis indicate that both query/task satisfaction and grid-based evaluation metrics are influenced by the image search intent. Based on the results, we show possibilities to support users' search processes according to their search intents. We also discuss that there is still room for improvement in evaluation metrics through the development of intent-aware evaluation metrics in image search.
Product Bundle Identification using Semi-Supervised Learning
Authors: Hen Tzaban, Ido Guy, Asnat Greenstein-Messica, Arnon Dagan, Lior Rokach, Bracha Shapira
Venue: SIGIR
Year: 2020

Many sellers on e-commerce platforms offer buyers product bundles, which package together two or more different items. The identification of such bundles is a necessary step to support a variety of related services, from recommendation to dynamic pricing. In this work, we present a comprehensive study of bundle identification on a large e-commerce website. Our analysis of bundle compared to non-bundle listed items reveals several key differentiating characteristics, spanning the listing's title, image, and attributes. Following, we experiment with a multi-modal classifier, which takes advantage of these characteristics as features. Our analysis also shows that a bundle indicator input by sellers tends to be highly noisy and carries only a weak signal. The bundle identification task therefore faces the challenge of having a small set of manually-labeled clean examples and a larger set of noisy-labeled examples, in conjunction with class imbalance due to the relative scarcity of bundles.
Generating Images Instead of Retrieving Them: Relevance Feedback on Generative Adversarial Networks
Authors: Antti Ukkonen, Pyry Joona, Tuukka Ruotsalo
Venue: SIGIR
Year: 2020

Finding images matching a user's intention has been largely based on matching a representation of the user's information needs with an existing collection of images. For example, using an example image or a written query to express the information need and retrieving images that share similarities with the query or example image. However, such an approach is limited to retrieving only images that already exist in the underlying collection. Here, we present a methodology for generating images matching the user intention instead of retrieving them. The methodology utilizes a relevance feedback loop between a user and generative adversarial neural networks (GANs). GANs can generate novel photorealistic images which are initially not present in the underlying collection, but generated in response to user feedback. We report experiments (N=29) where participants generate images using four different domains and various search goals with textual and image targets. The results show that the generated images match the tasks and outperform images selected as baselines from a fixed image collection. Our results demonstrate that generating new information can be more useful for users than retrieving it from a collection of existing information.
Cascade Model-based Propensity Estimation for Counterfactual Learning to Rank
Authors: Ali Vardasbi, Maarten de Rijke, Ilya Markov
Venue: SIGIR
Year: 2020

Unbiased counterfactual learning to rank (CLTR) requires click propensities to compensate for the difference between user clicks and true relevance of search results via inverse propensity scoring (IPS). Current propensity estimation methods assume that user click behavior follows the position-based click model (PBM) and estimate click propensities based on this assumption. However, in reality, user clicks often follow the cascade model (CM), where users scan search results from top to bottom and where each next click depends on the previous one. In this cascade scenario, PBM-based estimates of propensities are not accurate, which, in turn, hurts CLTR performance. In this paper, we propose a propensity estimation method for the cascade scenario, called cascade model-based inverse propensity scoring (CM-IPS). We show that CM-IPS keeps CLTR performance close to the full-information performance in case the user clicks follow the CM, while PBM-based CLTR has a significant gap towards the full-information. The opposite is true if the user clicks follow PBM instead of the CM. Finally, we suggest a way to select between CM- and PBM-based propensity estimation methods based on historical user clicks.
Studying Ranking-Incentivized Web Dynamics
Authors: Ziv Vasilisky, Moshe Tennenholtz, Oren Kurland
Venue: SIGIR
Year: 2020

The ranking incentives of many authors of Web pages play an important role in the Web dynamics. That is, authors who opt to have their pages highly ranked for queries of interest often respond to rankings for these queries by manipulating their pages; the goal is to improve the pages' future rankings. Various theoretical aspects of this dynamics have recently been studied using game theory. However, empirical analysis of the dynamics is highly constrained due to lack of publicly available datasets. We present an initial such dataset that is based on TREC's ClueWeb09 dataset. Specifically, we used the WayBack Machine of the Internet Archive to build a document collection that contains past snapshots of ClueWeb documents which are highly ranked by some initial search performed for ClueWeb queries. Temporal analysis of document changes in this dataset reveals that findings recently presented for small-scale controlled ranking competitions between documents' authors also hold for Web data. Specifically, documents' authors tend to mimic the content of documents that were highly ranked in the past, and this practice can result in improved ranking.
Data Poisoning Attacks against Differentially Private Recommender Systems
Authors: Soumya Wadhwa, Saurabh Agrawal, Harsh Chaudhari, Deepthi Sharma, Kannan Achan
Venue: SIGIR
Year: 2020

Recommender systems based on collaborative filtering are highly vulnerable to data poisoning attacks, where a determined attacker injects fake users with false user-item feedback, with an objective to either corrupt the recommender system or promote/demote a target set of items. Recently, differential privacy was explored as a defense technique against data poisoning attacks in the typical machine learning setting. In this paper, we study the effectiveness of differential privacy against such attacks on matrix factorization based collaborative filtering systems. Concretely, we conduct extensive experiments for evaluating robustness to injection of malicious user profiles by simulating common types of shilling attacks on real-world data and comparing the predictions of typical matrix factorization with differentially private matrix factorization.
Learning Discriminative Joint Embeddings for Efficient Face and Voice Association
Authors: Rui Wang, Xin Liu 0011, Yiu-ming Cheung, Kai Cheng, Nannan Wang 0001, Wentao Fan 0001
Venue: SIGIR
Year: 2020

Many cognitive researches have shown the natural possibility of face-voice association, and such potential association has attracted much attention in biometric cross-modal retrieval domain. Nevertheless, the existing methods often fail to explicitly learn the common embeddings for challenging face-voice association tasks. In this paper, we present to learn discriminative joint embedding for face-voice association, which can seamlessly train the face subnetwork and voice subnetwork to learn their high-level semantic features, while correlating them to be compared directly and efficiently. Within the proposed approach, we introduce bi-directional ranking constraint, identity constraint and center constraint to learn the joint face-voice embedding, and adopt bi-directional training strategy to train the deep correlated face-voice model. Meanwhile, an online hard negative mining technique is utilized to discriminatively construct hard triplets in a mini-batch manner, featuring on speeding up the learning process. Accordingly, the proposed approach is adaptive to benefit various face-voice association tasks, including cross-modal verification, 1:2 matching, 1:N matching, and retrieval scenarios. Extensive experiments have shown its improved performances in comparison with the state-of-the-art ones.
Global Context Enhanced Graph Neural Networks for Session-based Recommendation
Authors: Ziyang Wang, Wei Wei 0002, Gao Cong, Xiao-Li Li 0001, Xianling Mao, Minghui Qiu
Venue: SIGIR
Year: 2020

Session-based recommendation (SBR) is a challenging task, which aims at recommending items based on anonymous behavior sequences. Almost all the existing solutions for SBR model user preference only based on the current session without exploiting the other sessions, which may contain both relevant and irrelevant item-transitions to the current session. This paper proposes a novel approach, called Global Context Enhanced Graph Neural Networks (GCE-GNN) to exploit item transitions over all sessions in a more subtle manner for better inferring the user preference of the current session. Specifically, GCE-GNN learns two levels of item embeddings from session graph and global graph, respectively: (i) Session graph, which is to learn the session-level item embedding by modeling pairwise item-transitions within the current session; and (ii) Global graph, which is to learn the global-level item embedding by modeling pairwise item-transitions over all sessions. In GCE-GNN, we propose a novel global-level item representation learning layer, which employs a session-aware attention mechanism to recursively incorporate the neighbors' embeddings of each node on the global graph. We also design a session-level item representation learning layer, which employs a GNN on the session graph to learn session-level item embeddings within the current session. Moreover, GCE-GNN aggregates the learnt item representations in the two levels with a soft attention mechanism. Experiments on three benchmark datasets demonstrate that GCE-GNN outperforms the state-of-the-art methods consistently.
Next-item Recommendation with Sequential Hypergraphs
Authors: Jianling Wang, Kaize Ding, Liangjie Hong, Huan Liu 0001, James Caverlee
Venue: SIGIR
Year: 2020

There is an increasing attention on next-item recommendation systems to infer the dynamic user preferences with sequential user interactions. While the semantics of an item can change over time and across users, the item correlations defined by user interactions in the short term can be distilled to capture such change, and help in uncovering the dynamic user preferences. Thus, we are motivated to develop a novel next-item recommendation framework empowered by sequential hypergraphs. Specifically, the framework: (i) adopts hypergraph to represent the short-term item correlations and applies multiple convolutional layers to capture multi-order connections in the hypergraph; (ii) models the connections between different time periods with a residual gating layer; and (iii) is equipped with a fusion layer to incorporate both the dynamic item embedding and short-term user intent to the representation of each interaction before feeding it into the self-attention layer for dynamic user modeling. Through experiments on datasets from the ecommerce sites Amazon and Etsy and the information sharing platform Goodreads, the proposed model can significantly outperform the state-of-the-art in predicting the next interesting item for each user.
Match²: A Matching over Matching Model for Similar Question Identification
Authors: Zizhen Wang 0001, Yixing Fan, Jiafeng Guo, Liu Yang 0005, Ruqing Zhang 0001, Yanyan Lan, Xueqi Cheng, Hui Jiang, Xiaozhao Wang
Venue: SIGIR
Year: 2020

Community Question Answering (CQA) has become a primary means for people to acquire knowledge, where people are free to ask questions or submit answers. To enhance the efficiency of the service, similar question identification becomes a core task in CQA which aims to find a similar question from the archived repository whenever a new question is asked. However, it has long been a challenge to properly measure the similarity between two questions due to the inherent variation of natural language, i.e., there could be different ways to ask a same question or different questions sharing similar expressions. To alleviate this problem, it is natural to involve the existing answers for the enrichment of the archived questions. Traditional methods typically take aone-side usage, which leverages the answer as some expanded representation of the corresponding question. Unfortunately, this may introduce unexpected noises into the similarity computation since answers are often long and diverse, leading to inferior performance. In this work, we propose atwo-side usage, which leverages the answer as a bridge of the two questions. The key idea is based on our observation that similar questions could be addressed by similar parts of the answer while different questions may not. In other words, we can compare the matching patterns of the two questions over the same answer to measure their similarity. In this way, we propose a novel matching over matching model, namely Match2, which compares the matching patterns between two question-answer pairs for similar question identification. Empirical experiments on two benchmark datasets demonstrate that our model can significantly outperform previous state-of-the-art methods on the similar question identification task.
CKAN: Collaborative Knowledge-aware Attentive Network for Recommender Systems
Authors: Ze Wang, Guangyan Lin, Huobin Tan, Qinghong Chen, Xiyang Liu 0001
Venue: SIGIR
Year: 2020

Since it can effectively address the problem of sparsity and cold start of collaborative filtering, knowledge graph (KG) is widely studied and employed as side information in the field of recommender systems. However, most of existing KG-based recommendation methods mainly focus on how to effectively encode the knowledge associations in KG, without highlighting the crucial collaborative signals which are latent in user-item interactions. As such, the learned embeddings underutilize the two kinds of pivotal information and are insufficient to effectively represent the latent semantics of users and items in vector space.
Coding Electronic Health Records with Adversarial Reinforcement Path Generation
Authors: Shanshan Wang, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Jian-Yun Nie, Jun Ma 0001, Maarten de Rijke
Venue: SIGIR
Year: 2020

Electronic Health Record (EHR) coding is the task of assigning one or more International Classification of Diseases (ICD) codes to every EHR. Most previous work either ignores the hierarchical nature of the ICD codes or only focuses on parent-child relations. Moreover, existing EHR coding methods predict ICD codes from the leaf level with the greatest ICD number and the most fine-grained categories, which makes it difficult for models to make correct decisions. In order to address these problems, we model EHR coding as a path generation task. For this approach, we need to address two main challenges: (1) How to model relations between EHR and ICD codes, and relations between ICD codes? (2) How to evaluate the quality of generated ICD paths in order to obtain a signal that can be used to supervise the learning? We propose a coarse-to-fine ICD path generation framework, named Reinforcement Path Generation Network (RPGNet), that implements EHR coding with a Path Generator (PG) and a Path Discriminator (PD). We address challenge (1) by introducing a Path Message Passing (PMP) module in the PG to encode three types of relation: between EHRs and ICD codes, between parent-child ICD codes, and between sibling ICD codes. To address challgenge (2), we propose a PD component that estimates the reward for each ICD code in a generated path. RPGNet is trained with Reinforcement Learning (RL) in an adversarial manner. Experiments on the MIMIC-III benchmark dataset show that RPGNet significantly outperforms state-of-the-art methods in terms of micro-averaged F1 and micro-averaged AUC.