OpenRaiser/Intern-Atlas huggingface dataset:
Booking.com Multi-Destination Trips Dataset
Authors: Dmitri Goldenberg, Pavel Levin
Venue: SIGIR
Year: 2021

We introduce a novel dataset of real multi-destination trips booked through Booking.com's online travel platform. The dataset consists of 1.5 million reservations representing 359,000 unique journeys made across 39,000 destinations. As such, the data is particularly well suited to model sequential recommendation and retrieval problems in a high cardinality target space. To preserve user privacy and protect business-sensitive statistics, the data is fully anonymized, sampled and limited to five user origin markets. Even so, the dataset is representative of the general travel purchase behavior and therefore presents a uniquely valuable resource for Machine Learning and information retrieval researchers. This work provides an overview of the dataset. It reports several benchmark results for relevant recommendation problems obtained as part of the recently held Booking.com data challenge during the WSDM WebTour workshop.
The Winner Takes it All: Geographic Imbalance and Provider (Un)fairness in Educational Recommender Systems
Authors: Elizabeth Gómez, Carlos Shui Zhang, Ludovico Boratto, Maria Salamó, Mirko Marras
Venue: SIGIR
Year: 2021

Educational recommender systems channel most of the research efforts on the effectiveness of the recommended items. While teachers have a central role in online platforms, the impact of recommender systems for teachers in terms of the exposure such systems give to the courses is an under-explored area. In this paper, we consider data coming from a real-world platform and analyze the distribution of the recommendations w.r.t. the geographical provenience of the teachers. We observe that data is highly imbalanced towards the United States, in terms of offered courses and of interactions. These imbalances are exacerbated by recommender systems, which overexpose the countryw.r.t. its representation in the data, thus generating unfairness for teachers outside that country. To introduce equity, we propose an approach that regulates the share of recommendations given to the items produced in a country (visibility) and the position of the items in the recommended list (exposure).
From Producer Success to Retention: A New Role of Search and Recommendation Systems on Marketplaces
Authors: Viet Ha-Thuc, Matthew Wood, Yunli Liu, Jagadeesan Sundaresan
Venue: SIGIR
Year: 2021

In online marketplaces, an increasing number of producers depend on search and recommendation systems to connect them with consumers to make a living. In this talk, we discuss how these systems will need to evolve from the traditional formulations by incorporating the producer value into their objectives. Jointly optimizing the ranking functions behind these systems on both consumer and producer values is a new direction and raises many technical challenges. To overcome these, we lay out an end-to-end solution and present the results of applying this solution on Facebook Marketplace.
Cross-Graph Attention Enhanced Multi-Modal Correlation Learning for Fine-Grained Image-Text Retrieval
Authors: Yi He, Xin Liu 0011, Yiu-Ming Cheung, Shu-Juan Peng, Jinhan Yi, Wentao Fan 0001
Venue: SIGIR
Year: 2021

Fine-grained Image-text retrieval is challenging but vital technology in the field of multimedia analysis. Existing methods mainly focus on learning the common embedding space of images (or patches) and sentences (or words), whereby their mapping features in such embedding space can be directly measured. Nevertheless, most existing image-text retrieval works rarely consider the shared semantic concepts that potentially correlated the heterogeneous modalities, which can enhance the discriminative power of learning such embedding space. Toward this end, we propose a Cross-Graph Attention model (CGAM) to explicitly learn the shared semantic concepts, which can be well utilized to guide the feature learning process of each modality and promote the common embedding learning. More specifically, we build semantic-embedded graph for each modality, and smooth the discrepancy between two modalities via cross-graph attention model to obtain shared semantic-enhanced features. Meanwhile, we reconstruct image and text features via the shared semantic concepts and original embedding representations, and leverage multi-head mechanism for similarity calculation. Accordingly, the semantic-enhanced cross-modal embedding between image and text is discriminatively obtained to benefit the fine-grained retrieval with high retrieval performance. Extensive experiments evaluated on benchmark datasets show the performance improvements in comparison with state-of-the-arts.
CINES: Explore Citation Network and Event Sequences for Citation Forecasting
Authors: Fang He 0002, Wang-Chien Lee, Tao-Yang Fu, Zhen Lei 0005
Venue: SIGIR
Year: 2021

Citations of scientific papers and patents reveal the knowledge flow and usually serve as the metric for evaluating their novelty and impacts in the field. Citation Forecasting thus has various applications in the real world. Existing works on citation forecasting typically exploit the sequential properties of citation events, without exploring the citation network. In this paper, we propose to explore both the citation network and the related citation event sequences which provide valuable information for future citation forecasting. We propose a novel \em Citation Network and Event Sequence (CINES) Model to encode signals in the citation network and related citation event sequences into various types of embeddings for decoding to the arrivals of future citations. Moreover, we propose atemporal network attention and three alternative designs of \em bidirectional feature propagation to aggregate the retrospective and prospective aspects of publications in the citation network, coupled with the citation event sequence embeddings learned by a \em two-level attention mechanism for the citation forecasting. We evaluate our models and baselines on both a U.S. patent dataset and a DBLP dataset. Experimental results show that our models outperform the state-of-the-art methods, i.e., RMTPP, CYAN-RNN, Intensity-RNN, and PC-RNN, reducing the forecasting error by 37.76% - 75.32%.
GilBERT: Generative Vision-Language Pre-Training for Image-Text Retrieval
Authors: Weixiang Hong, Kaixiang Ji, Jiajia Liu 0002, Jian Wang 0108, Jingdong Chen, Wei Chu
Venue: SIGIR
Year: 2021

Given a text/image query, image-text retrieval aims to find the relevant items in the database. Recently, visual-linguistic pre-training (VLP) methods have demonstrated promising accuracy on image-text retrieval and other visual-linguistic tasks. These VLP methods are typically pre-trained on a large amount of image-text pairs, then fine-tuned on various downstream tasks. Nevertheless, due to the natural modality incompleteness in image-text retrieval, i.e., the query is either image or text rather than an image-text pair, the naive application of VLP to image-text retrieval results in significant inefficiency. Moreover, existing VLP methods cannot extract comparable representations for a single-modal query and multi-modal database items. In this work, we propose a generative visual-linguistic pre-training approach, termed as GilBERT, to simultaneously learn generic representations of image-text data and complete the missing modality for incomplete pairs. In testing phase, the proposed GilBERT facilitates efficient vector-based retrieval by providing unified feature embedding for query and database items. Moreover, the generative training not only makes GilBERT compatible with non-parallel text/image corpus, but also enables GilBERT to model the image-text relationships without suffering massive randomly-sampled negative samples, leading to superior experimental performances. Extensive experiments demonstrate the advantages of GilBERT in image-text retrieval, in terms of both efficiency and accuracy.
Learning to Select Instance: Simultaneous Transfer Learning and Clustering
Authors: Zhaoxin Huan, Yulong Wang, Yong He 0009, Xiaolu Zhang, Chilin Fu, Weichang Wu, Jun Zhou 0011, Ke Ding, Liang Zhang, Linjian Mo
Venue: SIGIR
Year: 2021

Transfer learning leverages knowledge from a source domain with rich data to a target domain with sparse data. However, the difference between the source and target data distribution weakens the transferability. To bridge this gap, we focus on selecting source instances that are closely related to and have the same distribution as the target domain. In this paper, we propose a novel Adaptive Clustering Transfer Learning (ACTL) method to improve transferability. Specifically, we simultaneously train the instance selector and the transfer learning model. The selector adaptively conducts clustering on the training data and learns the weights for source instances. The weight will activate or inhibit the contribution of the corresponding source instance during transfer learning. Meanwhile, the transfer learning model guides the selector to learn the weight appropriately according to the objective function. To evaluate the effectiveness of our method, we conduct experiments on two different tasks including recommender system and text matching. Experimental results show that our method consistently outperforms competing methods and the selected source instances share a similar data distribution with the target domain.
Deep User Match Network for Click-Through Rate Prediction
Authors: Zai Huang, Mingyuan Tao, Bufeng Zhang
Venue: SIGIR
Year: 2021

Click-through rate (CTR) prediction is a crucial task in many applications (e.g. recommender systems). Recently deep learning based models have been proposed and successfully applied for CTR prediction by focusing on feature interaction or user interest based on the item-to-item relevance between user behaviors and candidate item. However, these existing models neglect the user-to-user relevance between the target user and those who like the candidate item, which can reflect the preference of target user. To this end, in this paper, we propose a novel Deep User Match Network (DUMN) which measures the user-to-user relevance for CTR prediction. Specifically, in DUMN, we design a User Representation Layer to learn a unified user representation which contains user latent interest based on user behaviors. Then, User Match Layer is designed to measure the user-to-user relevance by matching the target user and those who have interacted with candidate item and modeling their similarities in user representation space. Extensive experimental results on three public real-world datasets validate the effectiveness of DUMN compared with state-of-the-art methods.
IR for Children 2000-2020: Where Are We Now?
Authors: Theo Huibers, Monica Landoni, Emiliana Murgia, Maria Soledad Pera
Venue: SIGIR
Year: 2021

Over 20 years ago, Information Retrieval (IR) researchers began their quest for sound IR systems for children. The path was not straightforward. Challenges posed by interface design, relevance determination, diverse contexts, ethics, and many more, were taken up and explored from different perspectives. Large projects such as Puppy-IR and the International Children's Digital Library gave this field a certain boost; still, there is neither a sound solution for children in the search area in 2021 nor a roadmap to get there. What is the reason for this? Does the field cry out for specific IR solutions developed on a small scale for very small sub-fields and specific target groups? Are there some significant unforeseen barriers that hinder researchers? What about obstacles natural to areas of study such as this one that require a multidisciplinary approach or involve protected populations? With this workshop, we want to bring together as many key experts as possible from research and industry who focus on IR for children to understand why, unlike other IR areas, this one has not flourished and look for the biggest challenges for the next 10 years. We are not only thinking of traditional researchers and designers but also of those who develop and use IR systems for fields, such as in music, film, and education, as a way to push past this immobility and look at the problem from new, and perhaps more stimulating, perspectives.
RMBERT: News Recommendation via Recurrent Reasoning Memory Network over BERT
Authors: Qinglin Jia, Jingjie Li, Qi Zhang 0001, Xiuqiang He 0001, Jieming Zhu
Venue: SIGIR
Year: 2021

Personalized news recommendation aims to alleviate information overload and help users find news of their interests. Accurately matching candidate news and users' interests is the key to news recommendation. Most existing methods separately encode each user and news into vectors by news contents and then match the two vectors. However, a user's interest may differ in each news or each topic of one news. It's necessary to dynamically learn user and news vector and model their interaction. In this work, we present Recurrent Reasoning Memory Network over BERT (RMBERT) for news recommendation. Compared with other methods, our approach can leverage the ability of content modeling from BERT. Moreover, the recurrent reasoning memory network which performs a series of attention based reasoning steps can dynamically learn user and news vector and model their interaction in each step. As a result, our approach can better model user's interests. We conduct extensive experiments on a real-world news recommendation dataset and the results show that our approach significantly outperforms existing state-of-the-art methods.
MetaP: Meta Pattern Learning for One-Shot Knowledge Graph Completion
Authors: Zhiyi Jiang, Jianliang Gao, Xinqi Lv
Venue: SIGIR
Year: 2021

Knowledge Graphs (KGs) are widely used in various applications of information retrieval. Despite the large scale of KGs, they are still facing incomplete problems. Conventional approaches on Knowledge Graph Completion (KGC) require a large number of training instances for each relation. However, long-tail relations which only have a few related triples are ubiquitous in KGs. Therefore, it is very difficult to complete the long-tail relations. In this paper, we propose a meta pattern learning framework (MetaP) to predict new facts of relations under a challenging setting where there is only one reference for each relation. Patterns in data are representative regularities to classify data. Triples in KGs also conform to relation-specific patterns which can be used to measure the validity of triples. Our model extracts the patterns effectively through a convolutional pattern learner and measures the validity of triples accurately by matching query patterns with reference patterns. Extensive experiments demonstrate the effectiveness of our method. Besides, we build a few-shot KGC dataset of COVID-19 to assist the research process of the new coronavirus.
Hierarchical Cross-Modal Graph Consistency Learning for Video-Text Retrieval
Authors: Weike Jin, Zhou Zhao 0001, Pengcheng Zhang, Jieming Zhu, Xiuqiang He 0001, Yueting Zhuang
Venue: SIGIR
Year: 2021

Due to the popularity of video contents on the Internet, the information retrieval between videos and texts has attracted broad interest from researchers, which is a challenging cross-modal retrieval task. A common solution is to learn a joint embedding space to measure the cross-modal similarity. However, many existing approaches either pay more attention to textual information, video information, or cross-modal matching methods, but less to all three. We believe that a good video-text retrieval system should take into account all three points, fully exploiting the semantic information of both modalities and considering a comprehensive match. In this paper, we propose a Hierarchical Cross-Modal Graph Consistency Learning Network (HCGC) for video-text retrieval task, which considers multi-level graph consistency for video-text matching. Specifically, we first construct a hierarchical graph representation for the video, which includes three levels from global to local: video, clips and objects. Similarly, the corresponding text graph is constructed according to the semantic relationships among sentence, actions and entities. Then, in order to learn a better match between the video and text graph, we design three types of graph consistency (both direct and indirect): inter-graph parallel consistency, inter-graph cross consistency and intra-graph cross consistency. Extensive experimental results on different video-text datasets demonstrate the effectiveness of our approach on both text-to-video and video-to-text retrieval.
DiffIR: Exploring Differences in Ranking Models' Behavior
Authors: Kevin Martin Jose, Thong Nguyen 0004, Sean MacAvaney, Jeffrey Dalton 0001, Andrew Yates
Venue: SIGIR
Year: 2021

Understanding and comparing the behavior of retrieval models is a fundamental challenge that requires going beyond examining average effectiveness and per-query metrics, because these do not reveal key differences in how ranking models' behavior impacts individual results. DiffIR is a new open-source web tool to assist with qualitative ranking analysis by visually 'diffing' system rankings at the individual result level for queries where behavior significantly diverges. Using one of several configurable similarity measures, it identifies queries for which the rankings of models compared have important differences in individual rankings and provides a visual web interface to compare the rankings side-by-side. DiffIR additionally supports a model-specific visualization approach based on custom term importance weight files. These support studying the behavior of interpretable models, such as neural retrieval methods that produce document scores based on a similarity matrix or based on a single document passage. Observations from this tool can complement neural probing approaches like ABNIRML to generate quantitative tests. We provide an illustrative use case of DiffIR by studying the qualitative differences between recently developed neural ranking models on a standard TREC benchmark dataset.
Reinforcement Learning from Reformulations in Conversational Question Answering over Knowledge Graphs
Authors: Magdalena Kaiser, Rishiraj Saha Roy, Gerhard Weikum
Venue: SIGIR
Year: 2021

The rise of personal assistants has made conversational question answering (ConvQA) a very popular mechanism for user-system interaction. State-of-the-art methods for ConvQA over knowledge graphs (KGs) can only learn from crisp question-answer pairs found in popular benchmarks. In reality, however, such training data is hard to come by: users would rarely mark answers explicitly as correct or wrong. In this work, we take a step towards a more natural learning paradigm - from noisy and implicit feedback via question reformulations. A reformulation is likely to be triggered by an incorrect system response, whereas a new follow-up question could be a positive signal on the previous turn's answer. We present a reinforcement learning model, termed CONQUER, that can learn from a conversational stream of questions and reformulations. CONQUER models the answering process as multiple agents walking in parallel on the KG, where the walks are determined by actions sampled using a policy network. This policy network takes the question along with the conversational context as inputs and is trained via noisy rewards obtained from the reformulation likelihood. To evaluate CONQUER, we create and release ConvRef, a benchmark with about 11k natural conversations containing around 205k reformulations. Experiments show that CONQUER successfully learns from noisy reward signals, significantly improving over a state-of-the-art baseline.
ECOM'21: The SIGIR 2021 Workshop on eCommerce
Authors: Surya Kallumadi, Tracy Holloway King, Shervin Malmasi, Maarten de Rijke
Venue: SIGIR
Year: 2021

eCommerce Information Retrieval (IR) is receiving increasing attention in the academic literature and is an essential component of some of the world's largest web sites (e.g., Airbnb, Alibaba, Amazon, eBay, Facebook, Flipkart, Lowe's, Taobao, and Target). SIGIR has for several years seen sponsorship from eCommerce organisations, reflecting the importance of IR research to them. The purpose of this workshop is (1) to bring together researchers and practitioners of eCommerce IR to discuss topics unique to it, (2) to determine how to use eCommerce's unique combination of free text, structured data, and customer behavioral data to improve search relevance, and (3) to examine how to build datasets and evaluate algorithms in this domain. Since eCommerce customers often do not know exactly what they want to buy (i.e. navigational and spearfishing queries are rare), recommendations are valuable for inspiration and serendipitous discovery as well as basket building.
Precision Medicine Search for Paediatric Oncology
Authors: Bevan Koopman, Tracey Wright, Natacha Omer, Veronica McCabe, Guido Zuccon
Venue: SIGIR
Year: 2021

We present a search engine aimed to help clinicians find targeted treatments for children with cancer. Childhood cancer is a leading cause of death and clinicians increasingly seek treatments that are tailored to an individual patient, particularly their tumour genetics. Finding treatments that are specific to paediatrics and match individual genetics is a real challenge amongst the vast and growing body of medical literature and clinical trials. We aim to help clinicians through a search system tailored to this problem.
Revisiting the Tag Relevance Prediction Problem
Authors: Denis Kotkov, Alexandr V. Maslov, Mats Neovius
Venue: SIGIR
Year: 2021

Traditionally, recommender systems provide a list of suggestions to a user based on past interactions with items of this user. These recommendations are usually based on user preferences for items and generated with a delay. Critiquing recommender systems allow users to provide immediate feedback to recommendations with tags and receive a new set of recommendations in response. However, these systems often require rich item descriptions that contain relevance scores indicating the strength, with which a tag applies to an item. For example, this relevance score could indicate how violent the movie "The Godfather" is on a scale from 0 to 1. Retrieving these data is a very demanding process, as it requires users to explicitly indicate the degree to which a tag applies to an item. This process can be improved with machine learning methods that predict tag relevance. In this paper, we explore the dataset from a different study, where the authors collected relevance scores on movie-tag pairs. In particular, we define the tag relevance prediction problem, explore the inconsistency of relevance scores provided by users as a challenge of this problem and present a method, which outperforms the state-of-the-art method for predicting tag relevance. We found a moderate inconsistency of user relevance scores. We also found that users tend to disagree more on subjective tags, such as "good acting", "bad plot" or "quotable" than on objective tags, such as "animation", "cars" or "wedding", but the disagreement of users regarding objective tags is also moderate.