OpenRaiser/Intern-Atlas huggingface dataset:
Investigating Reference Dependence Effects on User Search Interaction and Satisfaction: A Behavioral Economics Perspective
Authors: Jiqun Liu, Fangyuan Han
Venue: SIGIR
Year: 2020

How users think, behave, and make decisions when interacting with information retrieval (IR) systems is a fundamental research problem in the area of Interactive IR. There is substantial evidence from behavioral economics and decision sciences demonstrating that in the context of decision-making under uncertainty, the carriers of value behind actions are gains and losses defined relative to a reference point, rather than the absolute final outcomes. This Reference Dependence Effect as a systematic cognitive bias was largely ignored by most formal interaction models built upon a series of unrealistic assumptions of user rationality. To address this gap, our work seeks to 1) understand the effects of reference points on search behavior and satisfaction at both query and session levels; 2) apply the knowledge of reference dependence in predicting users' search decisions and variations in level of satisfaction. Based on our experiments on three datasets collected from 1840 task-based search sessions (5225 query segments), we found that: 1) users' search satisfaction and many aspects of search behaviors and decisions are significantly associated with relative gains, losses and the associated reference points; 2) users' judgments of session-level satisfaction are significantly affected by peak and end reference moments; 3) compared to final-outcome-based baselines, models employing gain- and loss-based features often achieve significantly better performances in predicting search decisions and user satisfaction. The adaptation of behavioral economics perspective enables us to keep taking advantage of the collision of interdisciplinary insights in advancing IR research and also increase the explanatory power of formal search models by providing them with a more realistic behavioral and psychological foundation.
Joint-modal Distribution-based Similarity Hashing for Large-scale Unsupervised Deep Cross-modal Retrieval
Authors: Song Liu, Shengsheng Qian, Yang Guan, Jiawei Zhan, Long Ying
Venue: SIGIR
Year: 2020

Hashing-based cross-modal search which aims to map multiple modality features into binary codes has attracted increasingly attention due to its storage and search efficiency especially in large-scale database retrieval. Recent unsupervised deep cross-modal hashing methods have shown promising results. However, existing approaches typically suffer from two limitations: (1) They usually learn cross-modal similarity information separately or in a redundant fusion manner, which may fail to capture semantic correlations among instances from different modalities sufficiently and effectively. (2) They seldom consider the sampling and weighting schemes for unsupervised cross-modal hashing, resulting in the lack of satisfactory discriminative ability in hash codes.
Certifiable Robustness to Discrete Adversarial Perturbations for Factorization Machines
Authors: Yang Liu 0245, Xianzhuo Xia, Liang Chen 0001, Xiangnan He 0001, Carl Yang 0001, Zibin Zheng
Venue: SIGIR
Year: 2020

Factorization machines (FMs) have been widely adopted to model the discrete feature interactions in recommender systems. Despite their great success, currently there is no study of their robustness to discrete adversarial perturbations. Whether modifying a certain number of the discrete input features has a dramatic effect on the FM's prediction? Although there exist robust training methods for FMs, they neglect the discrete property of input features and lack of an effective mechanism to verify the model robustness.
CapableOf Reasoning: A Step Towards Commonsense Oracle
Authors: Jingping Liu, Yanghua Xiao, Ao Wang, Liang He 0010, Bin Shao
Venue: SIGIR
Year: 2020

Commonsense knowledge is fundamental to make machines reach human-level intelligence. However, conventional methods of commonsense extraction generally do not work well because commonsense by nature is usually not explicitly stated in texts or other data. Besides, commonsense knowledge graphs built in advance are difficult to cover all the knowledge required for practical tasks due to the incompleteness of knowledge graphs. In this paper, we propose an online commonsense oracle to achieve knowledge reasoning. Specifically, we focus on the on-demand inference of specific commonsense propositions. We use capableOf relation as an example due to its notable significance in daily life. For more effective capableOf reasoning, informative supporting features derived from an existing commonsense knowledge graph and a Web search engine are exploited. Finally, we conduct extensive experiments, and the results demonstrate the effectiveness of our approach.
Chinese Document Classification with Bi-directional Convolutional Language Model
Authors: Bin Liu 0022, Guosheng Yin
Venue: SIGIR
Year: 2020

By setting a typeface, each character of the Chinese text can be converted to a glyph pixel matrix. We propose to conduct text classification with such glyph features using bi-directional convolution. Although the pixel embedding can be applied to all languages, it is much more convenient to be used to represent Chinese scripts due to the square shape of Chinese characters. We extract both the forward and backward n-gram features of the text via bi-directional convolutional operations and then concatenate them. A subsequent 1-dimensional max-over-time pooling is applied to the bi-directional feature maps, and then three fully connected layers are used for conducting text classification. The proposed model has a light-weight architecture that only contains a single-layer convolutional neural network. Experiments on several Chinese text classification datasets demonstrate surprisingly excellent results for the training speed and superior performance of the proposed model in comparison with traditional methods.
Multi-Level Multimodal Transformer Network for Multimodal Recipe Comprehension
Authors: Ao Liu 0008, Shuai Yuan 0018, Chenbin Zhang, Congjian Luo, Yaqing Liao, Kun Bai, Zenglin Xu
Venue: SIGIR
Year: 2020

Multimodal Machine Comprehension ($\rm M^3C$) has been a challenging task that requires understanding both language and vision, as well as their integration and interaction. For example, the RecipeQA challenge, which provides several $\rm M^3C$ tasks, requires deep neural models to understand textual instructions, images of different steps, as well as the logic orders of food cooking. To address this challenge, we propose a Multi-Level Multi-Modal Transformer (MLMM-Trans) framework to integrate and understand multiple textual instructions and multiple images. Our model can conduct intensive attention mechanism at multiple levels of objects (e.g., step level and passage-image level) for sequences of different modalities. Experiments have shown that our model can achieve the state-of-the-art results on the three multimodal tasks of RecipeQA.
Interactive Entity Linking Using Entity-Word Representations
Authors: Pei-Chi Lo, Ee-Peng Lim
Venue: SIGIR
Year: 2020

To leverage on entity and word semantics in entity linking, embedding models have been developed to represent entities, words and their context such that candidate entities for each mention can be determined and ranked accurately using their embeddings. To leverage on entity and word semantics in entity linking, embedding models have been developed to represent entities, words and their context such that candidate entities for each mention can be determined and ranked accurately using their embeddings. In this paper, we leverage on human intelligence for embedding-based interactive entity linking. We adopt an active learning approach to select mentions for human annotation that can best improve entity linking accuracy at the same time updating the embedding model. We propose two mention selection strategies based on: (1) coherence of entities linked, and (2) contextual closeness of candidate entities with respect to mention. Our experiments show that our proposed interactive entity linking methods outperform their batch counterpart in all our experimented datasets with relatively small amount of human annotations.
Deep Natural Language Processing for Search and Recommendation
Authors: Bo Long, Jieping Ye, Zang Li, Huiji Gao, Sandeep Kumar Jha
Venue: SIGIR
Year: 2020

Search and recommender systems process rich natural language text data such as user queries and documents. Achieving high-quality search and recommendation results requires processing and understanding such information effectively and efficiently, where natural language processing (NLP) technologies are widely deployed. In recent years, the rapid development of deep learning technology has been proven successful for improving various NLP tasks, indicating their great potential of promoting search and recommender systems.
Query-level Early Exit for Additive Learning-to-Rank Ensembles
Authors: Claudio Lucchese, Franco Maria Nardini, Salvatore Orlando 0001, Raffaele Perego 0001, Salvatore Trani
Venue: SIGIR
Year: 2020

Search engine ranking pipelines are commonly based on large ensembles of machine-learned decision trees. The tight constraints on query response time recently motivated researchers to investigate algorithms to make faster the traversal of the additive ensemble or to early terminate the evaluation of documents that are unlikely to be ranked among the top-k. In this paper, we investigate the novel problem of query-level early exiting, aimed at deciding the profitability of early stopping the traversal of the ranking ensemble for all the candidate documents to be scored for a query, by simply returning a ranking based on the additive scores computed by a limited portion of the ensemble. Besides the obvious advantage on query latency and throughput, we address the possible positive impact on ranking effectiveness. To this end, we study the actual contribution of incremental portions of the tree ensemble to the ranking of the top-k documents scored for a given query. Our main finding is that queries exhibit different behaviors as scores are accumulated during the traversal of the ensemble and that query-level early stopping can remarkably improve ranking quality. We present a reproducible and comprehensive experimental evaluation, conducted on two public datasets, showing that query-level early exiting achieves an overall gain of up to 7.5% in terms of [email protected] with a speedup of the scoring process of up to 2.2x.
Segmenting Search Query Logs by Learning to Detect Search Task Boundaries
Authors: Luis Lugo, José G. Moreno 0001, Gilles Hubert 0001
Venue: SIGIR
Year: 2020

To fulfill their information needs, users submit sets of related queries to available search engines. Query logs record users' activities along with timestamps and additional search-related information. The analysis of those chronological query logs enables the modeling of search tasks from user interactions. Previous research works rely on clicked URLs and surrounding queries to determine if adjacent queries are part of the same search tasks to segment the query logs properly. However, waiting for clicked URLs or future adjacent queries could render the use of these methods unfeasible in user supporting applications that require model results on the fly. Therefore, we propose a model for sequential search log segmentation. The proposed model uses only query pairs and their time span, generating results suited for on the fly user supporting applications, with improved accuracy over existing search segmentation approaches. We also show the advantages of fine-tuning the proposed model for adjusting the architecture to a small annotated collection.
A Multilingual Approach for Unsupervised Search Task Identification
Authors: Luis Lugo, José G. Moreno 0001, Gilles Hubert 0001
Venue: SIGIR
Year: 2020

Users convert their information needs to search queries, which are then run on available search engines. Query logs registered by search engines enable the automatic identification of the search tasks that users perform to fulfill their information needs. Search engine logs contain queries in multiple languages, but most existing methods for search task identification are not multilingual. Some methods rely on search context training of custom embeddings or external indexed collections that support a single language, making it challenging to support the multiple languages of queries run in search engines. Other methods depend on supervised components and user identifiers to model search tasks. The supervised components require labeled collections, which are difficult and costly to get in multiple languages. Also, the need for user identifiers renders these methods unfeasible in user agnostic scenarios. Hence, we propose an unsupervised multilingual approach for search task identification. The proposed approach is user agnostic, enabling its use in both user-independent and personalized scenarios. Furthermore, the multilingual query representation enables us to address the existing trade-off when mapping new queries to the identified search tasks.
Spatial Object Recommendation with Hints: When Spatial Granularity Matters
Authors: Hui Luo 0001, Jingbo Zhou 0003, Zhifeng Bao, Shuangli Li, J. Shane Culpepper, Haochao Ying, Hao Liu 0026, Hui Xiong 0001
Venue: SIGIR
Year: 2020

Existing spatial object recommendation algorithms generally treat objects identically when ranking them. However, spatial objects often cover different levels of spatial granularity and thereby are heterogeneous. For example, one user may prefer to be recommended a region (say Manhattan), while another user might prefer a venue (say a restaurant). Even for the same user, preferences can change at different stages of data exploration. In this paper, we study how to support top-k spatial object recommendations at varying levels of spatial granularity, enabling spatial objects at varying granularity, such as a city, suburb, or building, as a Point of Interest (POI). To solve this problem, we propose the use of a POI tree, which captures spatial containment relationships between POIs. We design a novel multi-task learning model called MPR (short for Multi-level POI Recommendation), where each task aims to return the top-k POIs at a certain spatial granularity level. Each task consists of two subtasks: (i) attribute-based representation learning; (ii) interaction-based representation learning. The first subtask learns the feature representations for both users and POIs, capturing attributes directly from their profiles. The second subtask incorporates user-POI interactions into the model. Additionally, MPR can provide insights into why certain recommendations are being made to a user based on three types of hints: user-aspect, POI-aspect, and interaction-aspect. We empirically validate our approach using two real-life datasets, and show promising performance improvements over several state-of-the-art methods.
Network On Network for Tabular Data Classification in Real-world Applications
Authors: Yuanfei Luo, Hao Zhou, Wei-Wei Tu, Yuqiang Chen, Wenyuan Dai, Qiang Yang 0001
Venue: SIGIR
Year: 2020

Tabular data is the most common data format adopted by our customers ranging from retail, finance to E-commerce, and tabular data classification plays an essential role to their businesses. In this paper, we present Network On Network (NON), a practical tabular data classification model based on deep neural network to provide accurate predictions. Various deep methods have been proposed and promising progress has been made. However, most of them use operations like neural network and factorization machines to fuse the embeddings of different features directly, and linearly combine the outputs of those operations to get the final prediction. As a result, the intra-field information and the non-linear interactions between those operations (e.g. neural network and factorization machines) are ignored. Intra-field information is the information that features inside each field belong to the same field. NON is proposed to take full advantage of intra-field information and non-linear interactions. It consists of three components: field-wise network at the bottom to capture the intra-field information, across field network in the middle to choose suitable operations data-drivenly, and operation fusion network on the top to fuse outputs of the chosen operations deeply. Extensive experiments on six real-world datasets demonstrate NON can outperform the state-of-the-art models significantly. Furthermore, both qualitative and quantitative study of the features in the embedding space show NON can capture intra-field information effectively.
Towards Differentially Private Text Representations
Authors: Lingjuan Lyu, Yitong Li 0002, Xuanli He, Tong Xiao
Venue: SIGIR
Year: 2020

Most deep learning frameworks require users to pool their local data or model updates to a trusted server to train or maintain a global model. The assumption of a trusted server who has access to user information is ill-suited in many applications. To tackle this problem, we develop a new deep learning framework under an untrusted server setting, which includes three modules: (1) embedding module, (2) randomization module, and (3) classifier module. For the randomization module, we propose a novel local differentially private (LDP) protocol to reduce the impact of privacy parameter ε on accuracy, and provide enhanced flexibility in choosing randomization probabilities for LDP. Analysis and experiments show that our framework delivers comparable or even better performance than the non-private framework and existing LDP protocols, demonstrating the advantages of our LDP protocol.
Large-scale Image Retrieval with Sparse Binary Projections
Authors: Changyi Ma, Chonglin Gu, Wenye Li 0001, Shuguang Cui
Venue: SIGIR
Year: 2020

Inspired by the recent discoveries in neuroscience, the study of the sparse binary projection model started to attract people's attention, shedding new light on image retrieval. Different from the classical work that tries to reduce the dimension of the data for faster retrieval speed, the model projects dense input samples into a higher-dimensional space and outputs sparse binary data representations after winner-take-all competition. Following the work along this line, this paper designed a new algorithm which obtains a high-quality sparse binary projection matrix through unsupervised training. Simple as it is, the algorithm reported significantly improved results over the state-of-the-art methods in both search accuracy and retrieval speed in a series of empirical evaluations on large-scale image retrieval tasks, which exhibited its promising potential in industrial applications.
SPot: A Tool for Identifying Operating Segments in Financial Tables
Authors: Zhiqiang Ma, Steven Pomerville, Mingyang Di, Armineh Nourbakhsh
Venue: SIGIR
Year: 2020

In this paper we present SPot, an automated tool for detecting operating segments and their related performance indicators from earnings reports. Due to their company-specific nature, operating segments cannot be detected using taxonomy-based approaches. Instead, we train a bidirectional RNN classifier that can distinguish between common metrics such as "revenue" and company-specific metrics that are likely to be operating segments, such as "iPhone" or "cloud services". SPot surfaces the results in an interactive web interface that allows users to trace and adjust performance metrics for each operating segment. This facilitates credit monitoring, enables them to perform competitive benchmarking more effectively, and can be used for trend analysis at company and sector levels.
Learning Colour Representations of Search Queries
Authors: Paridhi Maheshwari, Manoj Ghuhan, Vishwa Vinay
Venue: SIGIR
Year: 2020

Image search engines rely on appropriately designed ranking features that capture various aspects of the content semantics as well as the historic popularity. In this work, we consider the role of colour in this relevance matching process. Our work is motivated by the observation that a significant fraction of user queries have an inherent colour associated with them. While some queries contain explicit colour mentions (such as 'black car' and 'yellow daisies'), other queries have implicit notions of colour (such as 'sky' and 'grass'). Furthermore, grounding queries in colour is not a mapping to a single colour, but a distribution in colour space. For instance, a search for 'trees' tends to have a bimodal distribution around the colours green and brown. We leverage historical clickthrough data to produce a colour representation for search queries and propose a recurrent neural network architecture to encode unseen queries into colour space. We also show how this embedding can be learnt alongside a cross-modal relevance ranker from impression logs where a subset of the result images were clicked. We demonstrate that the use of a query-image colour distance feature leads to an improvement in the ranker performance as measured by users' preferences of clicked versus skipped images.
CrossBERT: A Triplet Neural Architecture for Ranking Entity Properties
Authors: Jarana Manotumruksa, Jeff Dalton 0001, Edgar Meij, Emine Yilmaz
Venue: SIGIR
Year: 2020

Task-based Virtual Personal Assistants (VPAs) such as the Google Assistant, Alexa, and Siri are increasingly being adopted for a wide variety of tasks. These tasks are grounded in real-world entities and actions (e.g., book a hotel, organise a conference, or requesting funds). In this work we tackle the task of automatically constructing actionable knowledge graphs in response to a user query in order to support a wider variety of increasingly complex assistant tasks. We frame this as an entity property ranking task given a user query with annotated properties. We propose a new method for property ranking, CrossBERT. CrossBERT builds on the Bidirectional Encoder Representations from Transformers (BERT) and creates a new triplet network structure on cross query-property pairs that is used to rank properties. We also study the impact of using external evidence for query entities from textual entity descriptions. We perform experiments on two standard benchmark collections, the NTCIR-13 Actionable Knowledge Graph Generation (AKGG) task and Entity Property Identification (EPI) task. The results demonstrate that CrossBERT significantly outperforms the best performing runs from AKGG and EPI, as well as previous state-of-the-art BERT-based models. In particular, CrossBERT significantly improves Recall and NDCG by approximately 2-12% over the BERT models across the two used datasets.
Item Tagging for Information Retrieval: A Tripartite Graph Neural Network based Approach
Authors: Kelong Mao, Xi Xiao, Jieming Zhu, Biao Lu, Ruiming Tang, Xiuqiang He 0001
Venue: SIGIR
Year: 2020

Tagging has been recognized as a successful practice to boost relevance matching for information retrieval (IR), especially when items lack rich textual descriptions. A lot of research has been done for either multi-label text categorization or image annotation. However, there is a lack of published work that targets at item tagging specifically for IR. Directly applying a traditional multi-label classification model for item tagging is sub-optimal, due to the ignorance of unique characteristics in IR. In this work, we propose to formulate item tagging as a link prediction problem between item nodes and tag nodes. To enrich the representation of items, we leverage the query logs available in IR tasks, and construct a query-item-tag tripartite graph. This formulation results in a TagGNN model that utilizes heterogeneous graph neural networks with multiple types of nodes and edges. Different from previous research, we also optimize both full tag prediction and partial tag completion cases in a unified framework via a primary-dual loss mechanism. Experimental results on both open and industrial datasets show that our TagGNN approach outperforms the state-of-the-art multi-label classification approaches.