OpenRaiser/Intern-Atlas huggingface dataset:
Comprehensive Linguistic-Visual Composition Network for Image Retrieval
Authors: Haokun Wen, Xuemeng Song, Xin Yang 0008, Yibing Zhan, Liqiang Nie
Venue: SIGIR
Year: 2021

Composing text and image for image retrieval (CTI-IR) is a new yet challenging task, for which the input query is not the conventional image or text but a composition, i.e., a reference image and its corresponding modification text. The key of CTI-IR lies in how to properly compose the multi-modal query to retrieve the target image. In a sense, pioneer studies mainly focus on composing the text with either the local visual descriptor or global feature of the reference image. However, they overlook the fact that the text modifications are indeed diverse, ranging from the concrete attribute changes, like "change it to long sleeves", to the abstract visual property adjustments, e.g., "change the style to professional". Thus, simply emphasizing the local or global feature of the reference image for the query composition is insufficient. In light of the above analysis, we propose a Comprehensive Linguistic-Visual Composition Network (CLVC-Net) for image retrieval. The core of CLVC-Net is that it designs two composition modules: fine-grained local-wise composition module and fine-grained global-wise composition module, targeting comprehensive multi-modal compositions. Additionally, a mutual enhancement module is designed to promote local-wise and global-wise composition processes by forcing them to share knowledge with each other. Extensive experiments conducted on three real-world datasets demonstrate the superiority of our CLVC-Net. We released the codes to benefit other researchers.
Wiki-Reliability: A Large Scale Dataset for Content Reliability on Wikipedia
Authors: KayYen Wong, Miriam Redi, Diego Sáez-Trumper
Venue: SIGIR
Year: 2021

Wikipedia is the largest online encyclopedia, used by algorithms and web users as a central hub of reliable information on the web.The quality and reliability of Wikipedia content is maintained by a community of volunteer editors. Machine learning and information retrieval algorithms could help scale up editors' manual efforts around Wikipedia content reliability. However, there is a lack of large-scale data to support the development of such research. To fill this gap, in this paper, we propose Wiki-Reliability, the first dataset of English Wikipedia articles annotated with a wide set of content reliability issues. To build this dataset, we rely on Wikipedia "templates". Templates are tags used by expert Wikipedia editors to indicate content issues, such as the presence of "non-neutral point of view" or "contradictory articles", and serve as a strong signal for detecting reliability issues in a revision. We select the 10 most popular reliability-related templates on Wikipedia, and propose an effective method to label almost 1M samples of Wikipedia article revisions as positive or negative with respect to each template. Each positive/negative example in the dataset comes with the full article text and 20 features from the revision's metadata. We provide an overview of the possible downstream tasks enabled by such data, and show that Wiki-Reliability can be used to train large-scale models for content reliability prediction. We release all data and code for public use.
Deep Learning on Graphs for Natural Language Processing
Authors: Lingfei Wu 0001, Yu Chen 0022, Heng Ji 0001, Bang Liu 0003
Venue: SIGIR
Year: 2021

This tutorial of Deep Learning on Graphs for Natural Language Processing (DLG4NLP) will cover relevant and interesting topics on applying deep learning on graph techniques to NLP, including automatic graph construction for NLP, graph representation learning for NLP, advanced GNN based models (e.g., graph2seq, graph2tree, and graph2graph) for NLP, and the applications of GNNs in various NLP tasks (e.g., machine translation, natural language generation, information extraction and semantic parsing). In addition, a handson demonstration session will be included to help the audience gain practical experience on applying GNNs to solve challenging NLP problems using our recently developed open source library - Graph4NLP, the first library for researchers and practitioners for easy use of GNNs for various NLP tasks.
Pseudo Siamese Network for Few-shot Intent Generation
Authors: Congying Xia, Caiming Xiong, Philip S. Yu
Venue: SIGIR
Year: 2021

Few-shot intent detection is a challenging task due to the scare annotation problem. In this paper, we propose a Pseudo Siamese Network (PSN) to generate labeled data for few-shot intents and alleviate this problem. PSN consists of two identical subnetworks with the same structure but different weights: an action network and an object network. Each subnetwork is a transformer-based variational autoencoder that tries to model the latent distribution of different components in the sentence. The action network is learned to understand action tokens and the object network focuses on object-related expressions. It provides an interpretable framework for generating an utterance with an action and an object existing in a given intent. Experiments on two real-world datasets show that PSN achieves state-of-the-art performance for the generalized few shot intent detection task.
Transformer Reasoning Network for Personalized Review Summarization
Authors: Hongyan Xu 0001, Hongtao Liu 0008, Pengfei Jiao, Wenjun Wang 0002
Venue: SIGIR
Year: 2021

Review summarization aims to generate condensed text for online product reviews, and has attracted more and more attention in E-commerce platforms. In addition to the input review, the quality of generated summaries is highly related to the characteristics of users and products, e.g., their historical summaries, which could provide useful clues for the target summary generation. However, most previous works ignore the underlying interaction between the given input review and the corresponding historical summaries. Therefore, we aim to explore how to effectively incorporate the history information into the summary generation. In this paper, we propose a novel transformer-based reasoning framework for personalized review summarization. We design an elaborately adapted transformer network containing an encoder and a decoder, to fully infer the important and informative parts among the historical summaries in terms of the input review to generate more comprehensive summaries. In the encoder of our approach, we develop an inter- and intra-attention to involve the history information selectively to learn the personalized representation of the input review. In the decoder part, we propose to incorporate the constructed reasoning memory learning from historical summaries into the original transformer decoder, and design a memory-decoder attention module to retrieve more useful information for the final summary generation. Extensive experiments are conducted and the results show our approach could generate more reasonable summaries for recommendation, and outperform many competitive baseline methods.
AliMe DA: A Data Augmentation Framework for Question Answering in Cold-start Scenarios
Authors: Guohai Xu, Yan Shao, Chenliang Li 0003, Feng-Lin Li, Bin Bi, Ji Zhang 0011, Haiqing Chen
Venue: SIGIR
Year: 2021

Cold-start is the most difficult and time-consuming phase when building a question answering based chatbot for a new business scenario because of the collection of sufficient training data. In this paper, we propose AliMe DA, a practical data augmentation (DA) framework that consists of data production, denoising and consumption, to alleviate this problem. We show how our DA approach can be used to substantially enhance annotation productivity and also improve downstream model performance. More importantly, we provide best practices for data augmentation, including how to choose and employ appropriate methods at each stage of our framework, and share our observation on the applicable scene of data augmentation in the era of pre-trained language models.
Policy-Gradient Training of Fair and Unbiased Ranking Functions
Authors: Himank Yadav, Zhengxiao Du, Thorsten Joachims
Venue: SIGIR
Year: 2021

While implicit feedback (e.g., clicks, dwell times, etc.) is an abundant and attractive source of data for learning to rank, it can produce unfair ranking policies for both exogenous and endogenous reasons. Exogenous reasons typically manifest themselves as biases in the training data, which then get reflected in the learned ranking policy and often lead to rich-get-richer dynamics. Moreover, even after the correction of such biases, reasons endogenous to the design of the learning algorithm can still lead to ranking policies that do not allocate exposure among items in a fair way. To address both exogenous and endogenous sources of unfairness, we present the first learning-to-rank approach that addresses both presentation bias and merit-based fairness of exposure simultaneously. Specifically, we define a class of amortized fairness-of-exposure constraints that can be chosen based on the needs of an application, and we show how these fairness criteria can be enforced despite the selection biases in implicit feedback data. The key result is an efficient and flexible policy-gradient algorithm, called FULTR, which is the first to enable the use of counterfactual estimators for both utility estimation and fairness constraints. Beyond the theoretical justification of the framework, we show empirically that the proposed algorithm can learn accurate and fair ranking policies from biased and noisy feedback.
Enhanced Graph Learning for Collaborative Filtering via Mutual Information Maximization
Authors: Yonghui Yang 0001, Le Wu, Richang Hong, Kun Zhang 0015, Meng Wang 0001
Venue: SIGIR
Year: 2021

Neural graph based Collaborative Filtering (CF) models learn user and item embeddings based on the user-item bipartite graph structure, and have achieved state-of-the-art recommendation performance. In the ubiquitous implicit feedback based CF, users' unobserved behaviors are treated as unlinked edges in the user-item bipartite graph. As users' unobserved behaviors are mixed with dislikes and unknown positive preferences, the fixed graph structure input is missing with potential positive preference links. In this paper, we study how to better learn enhanced graph structure for CF. We argue that node embedding learning and graph structure learning can mutually enhance each other in CF, as updated node embeddings are learned from previous graph structure, and vice versa ~(i.e., newly updated graph structure are optimized based on current node embedding results). Some previous works provided approaches to refine the graph structure. However, most of these graph learning models relied on node features for modeling, which are not available in CF. Besides, nearly all optimization goals tried to compare the learned adaptive graph and the original graph from a local reconstruction perspective, whether the global properties of the adaptive graph structure are modeled in the learning process is still unknown. To this end, in this paper, we propose an enhanced graph learning network EGLN approach for CF via mutual information maximization. The key idea of EGLN is two folds: First, we let the enhanced graph learning module and the node embedding module iteratively learn from each other without any feature input. Second, we design a local-global consistency optimization function to capture the global properties in the enhanced graph learning process. Finally, extensive experimental results on three real-world datasets clearly show the effectiveness of our proposed model.
Pretrained Transformers for Text Ranking: BERT and Beyond
Authors: Andrew Yates, Rodrigo Nogueira 0001, Jimmy Lin
Venue: SIGIR
Year: 2021

The goal of text ranking is to generate an ordered list of texts retrieved from a corpus in response to a query. Although the most common formulation of text ranking is search, instances of the task can also be found in many natural language processing applications. This tutorial, based on a forthcoming book, provides an overview of text ranking with neural network architectures known as transformers, of which BERT is the best-known example. The combination of transformers and self-supervised pretraining has, without exaggeration, revolutionized the fields of natural language processing (NLP), information retrieval (IR), and beyond. We provide a synthesis of existing work as a single point of entry for both researchers and practitioners. Our coverage is grouped into two categories: transformer models that perform reranking in multi-stage ranking architectures and learned dense representations that perform ranking directly. Two themes pervade our treatment: techniques for handling long documents and techniques for addressing the tradeoff between effectiveness (result quality) and efficiency (query latency). Although transformer architectures and pretraining techniques are recent innovations, many aspects of their application are well understood. Nevertheless, there remain many open research questions, and thus in addition to laying out the foundations of pretrained transformers for text ranking, we also attempt to prognosticate the future.
Cross Interaction Network for Natural Language Guided Video Moment Retrieval
Authors: Xinli Yu, Mohsen Malmir, Xin He, Jiangning Chen, Tong Wang, Yue Wu, Yue Liu, Yang Liu 0004
Venue: SIGIR
Year: 2021

Natural language query grounding in videos is a challenging task that requires comprehensive understanding of the query, video and fusion of information across these modalities. Existing methods mostly emphasize on the query-to-video one-way interaction with a late fusion scheme, lacking effective ways to capture the relationship within and between query and video in a fine-grained manner. Moreover, current methods are often overly complicated resulting in long training time. We propose a self-attention together with cross interaction multi-head-attention mechanism in an early fusion scheme to capture video-query intra-dependencies as well as inter-relation from both directions (query-to-video and video-to-query). The cross-attention method can associate query words and video frames at any position and account for long-range dependencies in the video context. In addition, we propose a multi-task training objective that includes start/end prediction and moment segmentation. The moment segmentation task provides additional training signals that remedy the start/end prediction noise caused by annotator disagreement. Our simple yet effective architecture enables speedy training (within 1 hour on an AWS P3.2xlarge GPU instance) and instant inference. We showed that the proposed method achieves superior performance compared to complex state of the art methods, in particular surpassing the SOTA on high IoU metrics ([email protected], IoU=0.7) by 3.52% absolute (11.09% relative) on the Charades-STA dataset.
Heterogeneous Attention Network for Effective and Efficient Cross-modal Retrieval
Authors: Tan Yu, Yi Yang 0031, Yi Li, Lin Liu, Hongliang Fei, Ping Li 0001
Venue: SIGIR
Year: 2021

Traditionally, the task of cross-modal retrieval is tackled through joint embedding. However, the global matching used in joint embedding methods often fails to effectively describe matchings between local regions of the image and words in the text. Hence they may not be effective in capturing the relevance between the text and the image. In this work, we propose a heterogeneous attention network (HAN) for effective and efficient cross-modal retrieval. The proposed HAN represents an image by a set of bounding box features and a sentence by a set of word features. The relevance between the image and the sentence is determined by the set-to-set matching between the set of word features and the set of bounding box features. To enhance the matching effectiveness, we exploit the proposed heterogeneous attention layer to provide the cross-modal context for word features as well as bounding box features. Meanwhile, to optimize the metric more effectively, we propose a new soft-max triplet loss, which adaptively gives more attention to harder negatives and thus trains the proposed HAN in a more effective manner compared with the original triplet loss. Meanwhile, the proposed HAN is efficient, and its lightweight architecture only needs a single GPU card for training. Extensive experiments conducted on two public benchmarks demonstrate the effectiveness and efficiency of our HAN. This work has been deployed in production Baidu Search Ads and is part of the "PaddleBox'' platform.
NeurJudge: A Circumstance-aware Neural Framework for Legal Judgment Prediction
Authors: Linan Yue, Qi Liu 0003, Binbin Jin, Han Wu 0002, Kai Zhang 0038, Yanqing An, Mingyue Cheng 0004, Biao Yin, Dayong Wu
Venue: SIGIR
Year: 2021

Legal Judgment Prediction is a fundamental task in legal intelligence of the civil law system, which aims to automatically predict the judgment results of multiple subtasks, such as charge, law article, and term of penalty prediction. Existing studies mainly focus on the impact of the entire fact description on all subtasks. They ignore the practical judicial scenario, where judges adopt circumstances of crime (i.e., various parts of the fact) to decide judgment results. To this end, in this paper, we propose a circumstance-aware legal judgment prediction framework (i.e., NeurJudge) by exploring circumstances of crime. Specifically, NeurJudge utilizes the results of intermediate subtasks to separate the fact description into different circumstances and exploits them to make the predictions of other subtasks. In addition, considering the popularity of confusing verdicts (i.e., charges and law articles), we further extend NeurJudge to a more comprehensive framework which is denoted by NeurJudge+. Particularly, NeurJudge+ utilizes a label embedding method to incorporate the semantics of labels (i.e., charges and law articles) into facts to generate more expressive fact representations for confusing verdicts problems. Extensive experimental results on two real-world datasets clearly validate the effectiveness of our proposed frameworks.
Is Query Performance Prediction With Multiple Query Variations Harder Than Topic Performance Prediction?
Authors: Oleg Zendel, J. Shane Culpepper, Falk Scholer
Venue: SIGIR
Year: 2021

Accurately estimating the retrieval effectiveness of different queries representing distinct information needs is a problem in Information Retrieval (IR) that has been studied for over 20 years. Recent work showed that the problem can be significantly harder when multiple queries representing the same information need are used in prediction. By generalizing the existing evaluation framework of Query Performance Prediction (QPP) we explore the causes of these differences in prediction quality in the two scenarios. Our empirical analysis demonstrates that for most predictors, this difference is solely an artifact of the underlying differences in the query effectiveness distributions. Our detailed analysis also demonstrates key performance distribution properties under which (QPP) is most and least reliable.
PAN: Prototype-based Adaptive Network for Robust Cross-modal Retrieval
Authors: Zhixiong Zeng, Shuai Wang, Nan Xu, Wenji Mao
Venue: SIGIR
Year: 2021

In practical applications of cross-modal retrieval, test queries of the retrieval system may vary greatly and come from unknown category. Meanwhile, due to the cost and difficulty of data collection as well as other issues, the available data for cross-modal retrieval are often imbalanced over different modalities. In this paper, we address two important issues to increase the robustness of cross-modal retrieval system for real-world applications: handling test queries from unknown category and modality-imbalanced training data. The first issue has not been addressed by existing methods and the second issue was not well addressed in the related research. To tackle the above issues, we take the advantage of prototype learning, and propose a prototype-based adaptive network (PAN) for robust cross-modal retrieval. Our method leverages a unified prototype to represent each semantic category across modalities, which provides discriminative information of different categories and takes unified prototypes as anchors to learn cross-modal representations adaptively. Moreover, we propose a novel prototype propagation strategy to reconstruct balanced representations which preserves the semantic consistency and modality heterogeneity. Experimental results on the benchmark datasets demonstrate the effectiveness of our method compared to the SOTA methods, and further robustness tests show the superiority of our method in solving the above issues.
Interactive Information Retrieval: Models, Algorithms, and Evaluation
Authors: Chengxiang Zhai
Venue: SIGIR
Year: 2021

Since Information Retrieval (IR) is an interactive process in general, it is important to study Interactive Information Retrieval (IIR), where we would attempt to model and optimize an entire interactive retrieval process (rather than a single query) with consideration of many different ways a user can potentially interact with a search engine. This tutorial systematically reviews the progress of research in IIR with an emphasis on the most recent progress in the development of models, algorithms, and evaluation strategies for IIR, ending with a brief discussion of the major open challenges in IIR and some of the most promising future research directions.
CSR 2021: The 1st International Workshop on Causality in Search and Recommendation
Authors: Yongfeng Zhang 0003, Xu Chen 0017, Yi Zhang 0001, Xianjie Chen
Venue: SIGIR
Year: 2021

Most of the current machine learning approaches to IR---including search and recommendation tasks---are mostly designed based on the basic idea of matching, which work from the perceptual and similarity learning perspective. This include both the learning of features from data such as representation learning, and the learning of similarity matching functions from data such as neural function learning. Though many models have been widely used in practical ranking systems such as search and recommendation, their design philosophy limits the models to the correlative signals in data. However, advancing from correlative learning to causal learning in search and recommendation is an important problem, because causal modeling can help us to think outside of the observational data for representation learning and ranking. More specially, causal learning can bring benefits to the IR community on various dimensions, including but not limited to Explainable IR models, Unbiased IR models, Fairness-aware IR models, Robust IR models and Cognitive Reasoning IR models. This workshop focuses on the research and application of causal modeling in search, recommendation and a broader scope of IR tasks. The workshop will gather both researchers and practitioners in the field for discussions, idea communications, and research promotions. It will also generate insightful debates about the recent regulations on AI Ethics, to a broader community including but not limited to IR, machine learning, AI, Data Science, and beyond. Workshop homepage is available online at https://csr21.github.io/.