OpenRaiser/Intern-Atlas huggingface dataset:
Make It a Chorus: Knowledge- and Time-aware Item Modeling for Sequential Recommendation
Authors: Chenyang Wang 0003, Min Zhang 0006, Weizhi Ma, Yiqun Liu 0001, Shaoping Ma
Venue: SIGIR
Year: 2020

Traditional recommender systems mainly aim to model inherent and long-term user preference, while dynamic user demands are also of great importance. Typically, a historical consumption will have impacts on the user demands for its relational items. For instance, users tend to buy complementary items together (iPhone and Airpods) but not substitutive items (Powerbeats and Airpods), although substitutes of the bought one still cater to his/her preference. To better model the effects of history sequence, previous studies introduce the semantics of item relations to capture user demands for recommendation. However, we argue that the temporal evolution of the effects caused by different relations cannot be neglected. In the example above, user demands for headphones can be promoted after a long period when a new one is needed.
Immersive Search: Using Virtual Reality to Examine How a Third Dimension Impacts the Searching Process
Authors: Austin R. Ward, Robert G. Capra
Venue: SIGIR
Year: 2020

In this paper, we present results from an exploratory study to investigate users' behaviors and preferences for three different styles of search results presentation in a virtual reality (VR) head-mounted display (HMD). Prior work in 2D displays has suggested possible benefits of presenting information in ways that exploit users' spatial cognition abilities. We designed a VR system that displays search results in three different spatial arrangements: a list of 8 results, a 4x5 grid, and a 2x10 arc. These spatial display conditions were designed to differ in terms of the number of results displayed per page (8 vs 20) and the amount of head movement required to scan the results (list < grid < arc). Thirty-six participants completed 6 search trials in each display condition (18 total). For each trial, the participant was presented with a display of search results and asked to find a given target result or to indicate that the target was not present. We collected data about users' behaviors with and perceptions about the three display conditions using interaction data, questionnaires, and interviews. We explore the effects of display condition and target presence on behavioral measures (e.g., completion time, head movement, paging events, accuracy) and on users' perceptions (e.g., workload, ease of use, comfort, confidence, difficulty, and lostness). Our results suggest that there was no difference in accuracy among the display conditions, but that users completed tasks more quickly using the arc. However, users also expressed lower preferences for the arc, instead preferring the list and grid displays. Our findings extend prior research on visual search into to the area of 3-dimensional result displays for interactive information retrieval in VR HMD environments.
How Airbnb Tells You Will Enjoy Sunset Sailing in Barcelona? Recommendation in a Two-Sided Travel Marketplace
Authors: Liang Wu 0006, Mihajlo Grbovic
Venue: SIGIR
Year: 2020

A two-sided travel marketplace is an E-Commerce platform where users can both host tours or activities and book them as a guest. When a new guest visits the platform, given tens of thousands of available listings, a natural question is that what kind of activities or trips are the best fit. In order to answer the question, a recommender system needs to both understand characteristics of its inventories, and to know the preferences of each individual guest. In this work, we present our efforts on building a recommender system for Airbnb Experiences, a two-sided online marketplace for tours and activities. Traditional recommender systems rely on abundant user-listing interactions. Airbnb Experiences is an emerging business where many listings and guests are new to the platform. Instead of passively waiting for data to accumulate, we propose novel approaches to identify key features of a listing and estimate guest preference with limited data availability.
Investigating Reading Behavior in Fine-grained Relevance Judgment
Authors: Zhijing Wu 0001, Jiaxin Mao, Yiqun Liu 0001, Min Zhang 0006, Shaoping Ma
Venue: SIGIR
Year: 2020

A better understanding of users' reading behavior helps improve many information retrieval (IR) tasks, such as relevance estimation and document ranking. Existing research has already leveraged eye movement information to investigate user's reading process during document-level relevance judgments and the findings were adopted to build more effective ranking models. Recently, fine-grained (e.g., passage or sentence level) relevance judgments have been paid much attention to with the requirements in conversational search and QA systems. However, there is still a lack of thorough investigation on user's reading behavior during these kinds of interaction processes. To shed light on this research question, we investigate how users allocate their attention to passages of a document during the relevance judgment process. With the eye-tracking data collected in a laboratory study, we show that users pay more attention to the "key" passages which contain key useful information. Users tend to revisit these key passages several times to accumulate and verify the gathered information. With both content and user behavior features, we find that key passages can be predicted with supervised learning. We believe that this work contributes to better understanding users' reading behavior and may provide more explainability for relevance estimation.
Joint Item Recommendation and Attribute Inference: An Adaptive Graph Convolutional Network Approach
Authors: Le Wu, Yonghui Yang 0001, Kun Zhang 0015, Richang Hong, Yanjie Fu, Meng Wang 0001
Venue: SIGIR
Year: 2020

In many recommender systems, users and items are associated with attributes, and users show preferences to items. The attribute information describes users'(items') characteristics and has a wide range of applications, such as user profiling, item annotation, and feature-enhanced recommendation. As annotating user (item) attributes is a labor intensive task, the attribute values are often incomplete with many missing attribute values. Therefore, item recommendation and attribute inference have become two main tasks in these platforms. Researchers have long converged that user(item) attributes and the preference behavior are highly correlated. Some researchers proposed to leverage one kind of data for the remaining task, and showed to improve performance. Nevertheless, these models either neglected the incompleteness of user~(item) attributes or regarded the correlation of the two tasks with simple models, leading to suboptimal performance of these two tasks.
How UMass-FSD Inadvertently Leverages Temporal Bias
Authors: Dominik Wurzer, Yumeng Qin
Venue: SIGIR
Year: 2020

First Story Detection describes the task of identifying new events in a stream of documents. The UMass-FSD system is known for its strong performance in First Story Detection competitions. Recently, it has been frequently used as a high accuracy baseline in research publications. We are the first to discover that UMass-FSD inadvertently leverages temporal bias. Interestingly, the discovered bias contrasts previously known biases and performs significantly better. Our analysis reveals an increased contribution of temporally distant documents, resulting from an unusual way of handling incremental term statistics. We show that this form of temporal bias is also applicable to other well-known First Story Detection systems, where it improves the detection accuracy. To provide a more generalizable conclusion and demonstrate that the observed bias is not only an artefact of a particular implementation, we present a model that intentionally leverages a bias on temporal distance. Our model significantly improves the detection effectiveness of state-of-the-art First Story Detection systems.
Neural Hierarchical Factorization Machines for User's Event Sequence Analysis
Authors: Dongbo Xi, Fuzhen Zhuang, Bowen Song, Yongchun Zhu, Shuai Chen, Dan Hong, Tao Chen, Xi Gu, Qing He 0003
Venue: SIGIR
Year: 2020

Many prediction tasks of real-world applications need to model multi-order feature interactions in user's event sequence for better detection performance. However, existing popular solutions usually suffer two key issues: 1) only focusing on feature interactions and failing to capture the sequence influence; 2) only focusing on sequence information, but ignoring internal feature relations of each event, thus failing to extract a better event representation. In this paper, we consider a two-level structure for capturing the hierarchical information over user's event sequence: 1) learning effective feature interactions based event representation; 2) modeling the sequence representation of user's historical events. Experimental results on both industrial and public datasets clearly demonstrate that our model achieves significantly better performance compared with state-of-the-art baselines.
Multiplex Behavioral Relation Learning for Recommendation via Memory Augmented Transformer Network
Authors: Lianghao Xia, Chao Huang 0001, Yong Xu 0007, Peng Dai 0001, Bo Zhang, Liefeng Bo
Venue: SIGIR
Year: 2020

Capturing users' precise preferences is of great importance in various recommender systems (e.g., e-commerce platforms and online advertising sites), which is the basis of how to present personalized interesting product lists to individual users. In spite of significant progress has been made to consider relations between users and items, most of existing recommendation techniques solely focus on singular type of user-item interactions. However, user-item interactive behavior is often exhibited with multi-type (e.g., page view, add-to-favorite and purchase) and inter-dependent in nature. The overlook of multiplex behavior relations can hardly recognize the multi-modal contextual signals across different types of interactions, which limit the feasibility of current recommendation methods. To tackle the above challenge, this work proposes a Memory-Augmented Transformer Networks (MATN), to enable the recommendation with multiplex behavioral relational information, and joint modeling of type-specific behavioral context and type-wise behavior inter-dependencies, in a fully automatic manner. In our MATN framework, we first develop a transformer-based multi-behavior relation encoder, to make the learned interaction representations be reflective of the cross-type behavior relations. Furthermore, a memory attention network is proposed to supercharge MATN capturing the contextual signals of different types of behavior into the category-specific latent embedding space. Finally, a cross-behavior aggregation component is introduced to promote the comprehensive collaboration across type-aware interaction behavior representations, and discriminate their inherent contributions in assisting recommendations. Extensive experiments on two benchmark datasets and a real-world e-commence user behavior data demonstrate significant improvements obtained by MATN over baselines. Codes are available at: https://github.com/akaxlh/MATN.
Preference-based Evaluation Metrics for Web Image Search
Authors: Xiaohui Xie, Jiaxin Mao, Yiqun Liu 0001, Maarten de Rijke, Haitian Chen, Min Zhang 0006, Shaoping Ma
Venue: SIGIR
Year: 2020

Following the success of Cranfield-like evaluation approaches to evaluation in web search, web image search has also been evaluated with absolute judgments of (graded) relevance. However, recent research has found that collecting absolute relevance judgments may be difficult in image search scenarios due to the multi-dimensional nature of relevance for image results. Moreover, existing evaluation metrics based on absolute relevance judgments do not correlate well with search users' satisfaction perceptions in web image search.
Self-Supervised Reinforcement Learning for Recommender Systems
Authors: Xin Xin 0003, Alexandros Karatzoglou, Ioannis Arapakis, Joemon M. Jose
Venue: SIGIR
Year: 2020

In session-based or sequential recommendation, it is important to consider a number of factors like long-term user engagement, multiple types of user-item interactions such as clicks, purchases etc. The current state-of-the-art supervised approaches fail to model them appropriately. Casting sequential recommendation task as a reinforcement learning (RL) problem is a promising direction. A major component of RL approaches is to train the agent through interactions with the environment. However, it is often problematic to train a recommender in an on-line fashion due to the requirement to expose users to irrelevant recommendations. As a result, learning the policy from logged implicit feedback is of vital importance, which is challenging due to the pure off-policy setting and lack of negative rewards (feedback).
On Presuppositions of Machine Learning: A Meta Theory
Authors: Zongben Xu
Venue: SIGIR
Year: 2020

Machine learning (ML) has been run and applied by premising a series of presuppositions, which contributes both the great success of AI and the bottleneck of further development of ML. These presuppositions include (i) the independence assumption of loss function on dataset (Hypothesis I); (ii) the large capacity assumption on hypothesis space including solution (Hypothesis II); (iii) the completeness assumption of training data with high quality (Hypothesis III); and (iv) the Euclidean assumption on analysis framework and methodology (Hypothesis IV).
Label-Consistency based Graph Neural Networks for Semi-supervised Node Classification
Authors: Bingbing Xu 0001, Junjie Huang, Liang Hou, Huawei Shen, Jinhua Gao, Xueqi Cheng
Venue: SIGIR
Year: 2020

Graph neural networks (GNNs) achieve remarkable success in graph-based semi-supervised node classification, leveraging the information from neighboring nodes to improve the representation learning of target node. The success of GNNs at node classification depends on the assumption that connected nodes tend to have the same label. However, such an assumption does not always work, limiting the performance of GNNs at node classification. In this paper, we propose label-consistency based graph neural network (LC-GNN), leveraging node pairs unconnected but with the same labels to enlarge the receptive field of nodes in GNNs. Experiments on benchmark datasets demonstrate the proposed LC-GNN outperforms traditional GNNs in graph-based semi-supervised node classification. We further show the superiority of LC-GNN in sparse scenarios with only a handful of labeled nodes.
Deep Interest with Hierarchical Attention Network for Click-Through Rate Prediction
Authors: Weinan Xu, Hengxu He, Minshi Tan, Yunming Li, Jun Lang 0001, Dongbai Guo
Venue: SIGIR
Year: 2020

Deep Interest Network (DIN) is a state-of-the-art model which uses attention mechanism to capture user interests from historical behaviors. User interests intuitively follow a hierarchical pattern such that users generally show interests from a higher-level then to a lower-level abstraction. Modelling such interest hierarchy in an attention network can fundamentally improve the representation of user behaviors. We therefore propose an improvement over DIN to model arbitrary interest hierarchy: Deep Interest with Hierarchical Attention Network (DHAN). In this model, a multi-dimensional hierarchical structure is introduced on the first attention layer which attends to individual item, and the subsequent attention layers in the same dimension attend to higher-level hierarchy built on top of the lower corresponding layers. To enable modelling of multiple dimensional hierarchy, an expanding mechanism is introduced to capture one to many hierarchies. This design enables DHAN to attend different importance to different hierarchical abstractions thus can fully capture a user's interests at different dimensions (e.g. category, price or brand). To validate our model, a simplified DHAN is applied to Click-Through Rate (CTR) prediction and our experimental results on three public datasets with two levels of one-dimensional hierarchy only by category. It shows DHAN's superiority with significant AUC uplift from 12% to 21% over DIN. DHAN is also compared with another state-of-the-art model Deep Interest Evolution Network (DIEN), which models temporal interest. The simplified DHAN also gets slight AUC uplift from 1.0% to 1.7% over DIEN. A potential future work can be combination of DHAN and DIEN to model both temporal and hierarchical interests.
Correlated Features Synthesis and Alignment for Zero-shot Cross-modal Retrieval
Authors: Xing Xu 0001, Kaiyi Lin, Huimin Lu 0001, Lianli Gao, Heng Tao Shen
Venue: SIGIR
Year: 2020

The goal of cross-modal retrieval is to search for semantically similar instances in one modality by using a query from another modality. Existing approaches mainly consider the standard scenario that requires the source set for training and the target set for testing share the same scope of classes. However, they may not generalize well on zero-shot cross-modal retrieval (ZS-CMR) task, where the target set contains unseen classes that are disjoint with the seen classes in the source set. This task is more challenging due to 1) the absence of the unseen classes during training, 2) inconsistent semantics across seen and unseen classes, and 3) the heterogeneous multimodal distributions between the source and target set. To address these issues, we propose a novel Correlated Feature Synthesis and Alignment (CFSA) approach to integrate multimodal feature synthesis, common space learning and knowledge transfer for ZS-CMR. Our CFSA first utilizes class-level word embeddings to guide two coupled Wassertein generative adversarial networks (WGANs) to synthesize sufficient multimodal features with semantic correlation for stable training. Then the synthetic and true multimodal features are jointly mapped to a common semantic space via an effective distribution alignment scheme, where the cross-modal correlations of different semantic features are captured and the knowledge can be transferred to the unseen classes under the cycle-consistency constraint. Experiments on four benchmark datasets for image-text retrieval and two large-scale datasets for image-sketch retrieval show the remarkable improvements achieved by our CFAS method comparing with a bundle of state-of-the-art approaches.
Reinforcement Learning to Rank with Pairwise Policy Gradient
Authors: Jun Xu 0001, Zeng Wei, Long Xia, Yanyan Lan, Dawei Yin 0001, Xueqi Cheng, Ji-Rong Wen
Venue: SIGIR
Year: 2020

This paper concerns reinforcement learning~(RL) of the document ranking models for information retrieval~(IR). One branch of the RL approaches to ranking formalize the process of ranking with Markov decision process~(MDP) and determine the model parameters with policy gradient. Though preliminary success has been shown, these approaches are still far from achieving their full potentials. Existing policy gradient methods directly utilize the absolute performance scores (returns) of the sampled document lists in its gradient estimations, which may cause two limitations: 1) fail to reflect the relative goodness of documents within the same query, which usually is close to the nature of IR ranking; 2) generate high variance gradient estimations, resulting in slow learning speed and low ranking accuracy. To deal with the issues, we propose a novel policy gradient algorithm in which the gradients are determined using pairwise comparisons of two document lists sampled within the same query. The algorithm, referred to as Pairwise Policy Gradient (PPG), repeatedly samples pairs of document lists, estimates the gradients with pairwise comparisons, and finally updates the model parameters. Theoretical analysis shows that PPG makes an unbiased and low variance gradient estimations. Experimental results have demonstrated performance gains over the state-of-the-art baselines in search result diversification and text retrieval.
A Case Study of Multi-class Classification with Diversified Precision Recall Requirements for Query Disambiguation
Authors: Yingrui Yang, Christopher Miller 0003, Peng Jiang, Azadeh Moghtaderi
Venue: SIGIR
Year: 2020

We introduce a new metric for measuring the performance of multi-class classifiers. This metric is a generalization of the f1 score that is defined on binary classifiers, and offers significant improvement over other generalizations such as micro- and macro-averaging. In particular, one can select coefficients that weight the per-class precision and recall, as well as the overall class importance, with a robust mathematical interpretation. When certain parameters are selected our metric yields macro-averaged statistic as a special case. We demonstrate the efficacy of this metric on an application in genealogical search.
TADS: Learning Time-Aware Scheduling Policy with Dyna-Style Planning for Spaced Repetition
Authors: Zhengyu Yang 0002, Jian Shen 0003, Yunfei Liu 0002, Yang Yang 0001, Weinan Zhang 0001, Yong Yu 0001
Venue: SIGIR
Year: 2020

Spaced repetition technique aims at improving long-term memory retention for human students by exploiting repeated, spaced reviews of learning contents. The study of spaced repetition focuses on designing an optimal policy to schedule the learning contents. To the best of our knowledge, none of the existing methods based on reinforcement learning take into account the varying time intervals between two adjacent learning events of the student, which, however, are essential to determine real-world schedule. In this paper, we aim to learn a scheduling policy that fully exploits the varying time interval information with high sample efficiency. We propose the Time-Aware scheduler with Dyna-Style planning (TADS) approach: a sample-efficient reinforcement learning framework for realistic spaced repetition. TADS learns a Time-LSTM policy to select an optimal content according to the student's whole learning history and the time interval since the last learning event. Besides, Dyna-style planning is integrated into TADS to further improve the sample efficiency. We evaluate our approach on three environments built from synthetic data and real-world data based on well-recognized cognitive models. Empirical results demonstrate that TADS achieves superior performance against state-of-the-art algorithms.
A Knowledge-Enhanced Recommendation Model with Attribute-Level Co-Attention
Authors: Deqing Yang, Zengchun Song, Lvxin Xue, Yanghua Xiao
Venue: SIGIR
Year: 2020

Deep neural networks (DNNs) have been widely employed in recommender systems including incorporating attention mechanism for performance improvement. However, most of existing attention-based models only apply item-level attention on user side, restricting the further enhancement of recommendation performance. In this paper, we propose a knowledge-enhanced recommendation model ACAM, which incorporates item attributes distilled from knowledge graphs (KGs) as side information, and is built with a co-attention mechanism on attribute-level to achieve performance gains. Specifically, each user and item in ACAM are represented by a set of attribute embeddings at first. Then, user representations and item representations are augmented simultaneously through capturing the correlations between different attributes by a co-attention module. Our extensive experiments over two realistic datasets show that the user representations and item representations augmented by attribute-level co-attention gain ACAM's superiority over the state-of-the-art deep models.