OpenRaiser/Intern-Atlas huggingface dataset:
LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation
Authors: Xiangnan He 0001, Kuan Deng, Xiang Wang 0010, Yan Li 0068, Yong-Dong Zhang 0001, Meng Wang 0001
Venue: SIGIR
Year: 2020

Graph Convolution Network (GCN) has become new state-of-the-art for collaborative filtering. Nevertheless, the reasons of its effectiveness for recommendation are not well understood. Existing work that adapts GCN to recommendation lacks thorough ablation analyses on GCN, which is originally designed for graph classification tasks and equipped with many neural network operations. However, we empirically find that the two most common designs in GCNs -- feature transformation and nonlinear activation -- contribute little to the performance of collaborative filtering. Even worse, including them adds to the difficulty of training and degrades recommendation performance.
Expressions of Style in Information Seeking Conversation with an Agent
Authors: Paul Thomas 0001, Daniel McDuff, Mary Czerwinski, Nick Craswell
Venue: SIGIR
Year: 2020

Past work in information-seeking conversation has demonstrated that people exhibit different conversational styles---for example, in word choice or prosody---that differences in style lead to poorer conversations, and that partners actively align their styles over time. One might assume that this would also be true for conversations with an artificial agent such as Cortana, Siri, or Alexa; and that agents should therefore track and mimic a user's style. We examine this hypothesis with reference to a lab study, where 24 participants carried out relatively long information-seeking tasks with an embodied conversational agent. The agent combined topical language models with a conversational dialogue engine, style recognition and alignment modules. We see that "style'' can be measured in human-to-agent conversation, although it looks somewhat different to style in human-to-human conversation and does not correlate with self-reported preferences. There is evidence that people align their style to the agent, and that conversations run more smoothly if the agent detects, and aligns to, the human's style as well.
CAsT-19: A Dataset for Conversational Information Seeking
Authors: Jeffrey Dalton 0001, Chenyan Xiong, Vaibhav Kumar, Jamie Callan
Venue: SIGIR
Year: 2020

CAsT-19 is a new dataset that supports research on conversational information seeking. The corpus is 38,426,252 passages from the TREC Complex Answer Retrieval (CAR) and Microsoft MAchine Reading COmprehension (MARCO) datasets. Eighty information seeking dialogues (30 train, 50 test) are an average of 9 to 10 questions long. A dialogue may explore a topic broadly or drill down into subtopics. Questions contain ellipsis, implied context, mild topic shifts, and other characteristics of human conversation that may prevent them from being understood in isolation. Relevance assessments are provided for 30 training topics and 20 test topics.
On Understanding Data Worker Interaction Behaviors
Authors: Lei Han 0003, Tianwa Chen, Gianluca Demartini, Marta Indulska, Shazia Sadiq
Venue: SIGIR
Year: 2020

Understanding how data workers interact with data and various pieces of information (e.g., code snippet examples) is key to design systems that can better support them in exploring a given dataset. To date, however, there is a paucity of research studying information seeking patterns and the strategies adopted by data workers as they carry out data curation activities. In this work, we aim at understanding the behaviors of data workers in discovering data quality issues, and how these behavioral observations relate to their performance. Specifically, we investigate how data workers use information resources and tools to support their task completion. To this end, we collect a multi-modal dataset through a data-driven experiment that relies on the use of eye-tracking technology with a purpose-designed platform built on top of iPython Notebook. The collected data reveals that: (i) searching in external resources is a prevalent action that can be leveraged to achieve better performance; (ii) 'copy-paste-modify' is a typical strategy for writing code to complete tasks; (iii) providing sample code within the system could help data workers to get started with their task; and (iv) surfacing underlying data is an effective way to support exploration. By investigating the behaviors prior to each search action, we also find that the most common reasons that trigger external search actions are the need to seek assistance in writing or debugging code and to search for relevant code to reuse. Our findings provide insights into patterns of interactions with various system components and information resources to perform data curation tasks. This bears implications on the design of domain-specific IR systems for data workers like code-base search.
Predicting Session Length for Product Search on E-commerce Platform
Authors: Shashank Gupta 0001, Subhadeep Maji
Venue: SIGIR
Year: 2020

Estimation of session duration for an e-commerce search engine is important for various downstream applications, including user satisfaction prediction, personalization, and diversification of search results. It has been shown in previous studies that search session length has a strong correlation with user's explore vs specific purchase intent. Based on previous work [14], we hypothesize that early prediction of session length distribution can be used to control the degree of explore vs exploit (loosely related to diversification v/s personalization) for Search Engine Result Pages (SERPs) in the user's session to follow. In this work, we try to early predict the user's session length, which will enable the control on explore v/s exploit of the search results. Towards this end, based on previous work and strong empirical evidence, we hypothesize session lengths are Weibull distributed and propose its parameters being modeled by a Recurrent Neural Network over actions in user's search sessions. Through experimentation, we demonstrate that our method performs better as compared to strong baselines for the same.
Multi-source Domain Adaptation for Sentiment Classification with Granger Causal Inference
Authors: Min Yang 0007, Ying Shen 0001, Xiaojun Chen 0006, Chengming Li 0004
Venue: SIGIR
Year: 2020

In this paper, we propose a multi-source domain adaptation method with a Granger-causal objective (MDA-GC) for cross-domain sentiment classification. Specifically, for each source domain, we build an expert model by using a novel sentiment-guided capsule network, which captures the domain invariant knowledge that bridges the knowledge gap between the source and target domains. Then, an attention mechanism is devised to assign importance weights to a mixture of experts, each of which specializes in a different source domain. In addition, we propose a Granger causal objective to make the weights assigned to individual experts correlate strongly with their contributions to the decision at hand. Experimental results on a benchmark dataset demonstrate that the proposed MDA-GC model significantly outperforms the compared methods.
Online Collective Matrix Factorization Hashing for Large-Scale Cross-Media Retrieval
Authors: Di Wang 0011, Quan Wang 0006, Yaqiang An, Xinbo Gao 0001, Yumin Tian
Venue: SIGIR
Year: 2020

Cross-modal hashing has been widely investigated recently for its efficiency in large-scale cross-media retrieval. However, most existing cross-modal hashing methods learn hash functions in a batch-based learning mode. Such mode is not suitable for large-scale data sets due to the large memory consumption and loses its efficiency when training streaming data. Online cross-modal hashing can deal with the above problems by learning hash model in an online learning process. However, existing online cross-modal hashing methods cannot update hash codes of old data by the newly learned model. In this paper, we propose Online Collective Matrix Factorization Hashing (OCMFH) based on collective matrix factorization hashing (CMFH), which can adaptively update hash codes of old data according to dynamic changes of hash model without accessing to old data. Specifically, it learns discriminative hash codes for streaming data by collective matrix factorization in an online optimization scheme. Unlike conventional CMFH which needs to load the entire data points into memory, the proposed OCMFH retrains hash functions only by newly arriving data points. Meanwhile, it generates hash codes of new data and updates hash codes of old data by the latest updated hash model. In such way, hash codes of new data and old data are well-matched. Furthermore, a zero mean strategy is developed to solve the mean-varying problem in the online hash learning process. Extensive experiments on three benchmark data sets demonstrate the effectiveness and efficiency of OCMFH on online cross-media retrieval.
Dynamic Clustering with Discrete Time Event Prediction
Authors: Karan Aggarwal, Georgios Theocharous, Anup B. Rao
Venue: SIGIR
Year: 2020

Discrete time event sequences from web users are commonly encountered in a variety of real-world domains, from digital marketing to recommender systems. Most of the literature in the area focuses on supervised prediction tasks of the user's next action or return time. However, little attention has been paid to the challenging task of modeling user evolution and behavior in a quantitative fashion. For example, progress of the users' knowledge in an online tutorial, user exploration, or engagement. We propose methods to perform a time-varying clustering along with event predictions in a unified domain agnostic framework. Our framework can help track the evolution of the users as they interact with the platform. We evaluate our methods on three real-world datasets and show that our method performs at par with the supervised baseline on the prediction tasks while providing meaningful clusters.
Automatic Generation of Topic Labels
Authors: Areej Alokaili, Nikolaos Aletras, Mark Stevenson 0001
Venue: SIGIR
Year: 2020

Topic modelling is a popular unsupervised method for identifying the underlying themes in document collections that has many applications in information retrieval. A topic is usually represented by a list of terms ranked by their probability but, since these can be difficult to interpret, various approaches have been developed to assign descriptive labels to topics. Previous work on the automatic assignment of labels to topics has relied on a two-stage approach: (1) candidate labels are retrieved from a large pool (e.g. Wikipedia article titles); and then (2) re-ranked based on their semantic similarity to the topic terms. However, these extractive approaches can only assign candidate labels from a restricted set that may not include any suitable ones. This paper proposes using a sequence-to-sequence neural-based approach to generate labels that does not suffer from this limitation. The model is trained over a new large synthetic dataset created using distant supervision. The method is evaluated by comparing the labels it generates to ones rated by humans.
Proposal and Comparison of Health Specific Features for the Automatic Assessment of Readability
Authors: Hélder Antunes, Carla Teixeira Lopes
Venue: SIGIR
Year: 2020

Looking for health information is one of the most popular activities online. However, the specificity of language on this domain is frequently an obstacle to comprehension, especially for the ones with lower levels of health literacy. For this reason, search engines should consider the readability of health content and, if possible, adapt it to the user behind the search. In this work, we explore methods to assess the readability of health content automatically. We propose features capable of measuring the specificity of a medical text and estimate the knowledge necessary to comprehend it. The features are based on information retrieval metrics and the log-likelihood of a text with lay and medico-scientific language models. To evaluate our methods, we built and used a dataset composed of health articles of Simple English Wikipedia and the respective documents in ordinary Wikipedia. We achieved a maximum accuracy of 88% in binary classifications (easy versus hard-to-read). We found out that the machine learning algorithm does not significantly interfere with performance. We also experimented and compared different features combinations. The features using the values of the log-likelihood of a text with lay and medico-scientific language models perform better than all the others.
Language Agnostic Hate Speech Detection
Authors: Aymé Arango
Venue: SIGIR
Year: 2020

The growth in social Web platforms in the past years has brought an increase in displays of online hate speech. This subject is considered as a critical matter in the Web community, since it can be related to potentially dangerous actions that affect individuals and groups in the physical world. The automatic detection of this type of expressions has been the center of several investigations over the past few years. However, most research on this subject has been done for the English language and on rather limited datasets. In addition, although some works approach the problem from a multilingual perspective, analyzing different language separately, across-lingual perspective of this problem has not been used so far.
Learning Efficient Representations of Mouse Movements to Predict User Attention
Authors: Ioannis Arapakis, Luis A. Leiva
Venue: SIGIR
Year: 2020

Tracking mouse cursor movements can be used to predict user attention on heterogeneous page layouts like SERPs. So far, previous work has relied heavily on handcrafted features, which is a time-consuming approach that often requires domain expertise. We investigate different representations of mouse cursor movements, including time series, heatmaps, and trajectory-based images, to build and contrast both recurrent and convolutional neural networks that can predict user attention to direct displays, such as SERP advertisements. Our models are trained over raw mouse cursor data and achieve competitive performance. We conclude that neural network models should be adopted for downstream tasks involving mouse cursor movements, since they can provide an invaluable implicit feedback signal for re-ranking and evaluation.
Choppy: Cut Transformer for Ranked List Truncation
Authors: Dara Bahri, Yi Tay, Che Zheng, Donald Metzler, Andrew Tomkins
Venue: SIGIR
Year: 2020

Work in information retrieval has traditionally focused on ranking and relevance: given a query, return some number of results ordered by relevance to the user. However, the problem of determining how many results to return, i.e. how to optimally truncate the ranked result list, has received less attention despite being of critical importance in a range of applications. Such truncation is a balancing act between the overall relevance, or usefulness of the results, with the user cost of processing more results. In this work, we propose Choppy, an assumption-free model based on the widely successful Transformer architecture, to the ranked list truncation problem. Needing nothing more than the relevance scores of the results, the model uses a powerful multi-head attention mechanism to directly optimize any user-defined IR metric. We show Choppy improves upon recent state-of-the-art methods.
Studying Ransomware Attacks Using Web Search Logs
Authors: Chetan Bansal, Pantazis Deligiannis, Chandra Shekhar Maddila, Nikitha Rao
Venue: SIGIR
Year: 2020

Cyber attacks are increasingly becoming prevalent and causing significant damage to individuals, businesses and even countries. In particular, ransomware attacks have grown significantly over the last decade. We do the first study on mining insights about ransomware attacks by analyzing query logs from Bing web search engine. We first extract ransomware related queries and then build a machine learning model to identify queries where users are seeking support for ransomware attacks. We show that user search behavior and characteristics are correlated with ransomware attacks. We also analyse trends in the temporal and geographical space and validate our findings against publicly available information. Lastly, we do a case study on 'Nemty', a popular ransomware, to show that it is possible to derive accurate insights about cyber attacks by query log analysis.
Multi-Document Answer Generation for Non-Factoid Questions
Authors: Valeriya Baranova-Bolotova
Venue: SIGIR
Year: 2020

The current research will be devoted to the challenging and under-investigated task of multi-source answer generation for complex non-factoid questions. We will start with experimenting with generative models on one particular type of non-factoid questions - instrumental/procedural questions which often start with "how-to". For this, a new dataset, comprised of more than 100,000 QA-pairs which were crawled from a dedicated web-resource where each answer has a set of references to the articles it was written upon, will be used. We will also compare different ways of model evaluation to choose a metric which better correlates with human assessment. To be able to do this, the way people evaluate answers to non-factoid questions and set some formal criteria of what makes a good quality answer is needed to be understood. Eye-tracking and crowdsourcing methods will be employed to study how users interact with answers and evaluate them, and how the answer features correlate with task complexity. We hope that our research will help to redefine the way users interact and work with search engines so as to transform IR finally into the answer retrieval systems that users have always desired.
Bayesian Inferential Risk Evaluation On Multiple IR Systems
Authors: Rodger Benham, Ben Carterette, J. Shane Culpepper, Alistair Moffat
Venue: SIGIR
Year: 2020

Information retrieval (IR) ranking models in production systems continually evolve in response to user feedback, insights from research, and new developments. Rather than investing all engineering resources to produce a single challenger to the existing system, a commercial provider might choose to explore multiple new ranking models simultaneously. However, even small changes to a complex model can have unintended consequences. In particular, the per-topic effectiveness profile is likely to change, and even when an overall improvement is achieved, gains are rarely observed for every query, introducing the risk that some users or queries may be negatively impacted by the new model if deployed into production.
Legal Data Analytics: Developing Assistive Tools for Legal Practitioners
Authors: Paheli Bhattacharya
Venue: SIGIR
Year: 2020

With the advancement of the Web and large number of legal documents being made available digitally, legal practitioners are now facing new challenges. It is now intractable for them to manually find relevant information (prior cases, related statutes, etc.) that would assist an ongoing case. From our discussions with law experts from India (faculty members from the Rajiv Gandhi School of Intellectual Property Law, India) as well as other countries like UK (Swansea University) and USA (Thomson Reuters), we understand that it is important to develop assistive tools for several tasks in the legal domain, e.g., identifying relevant documents, summarizing legal text, and so on. Though there exist legal information systems (e.g., Thomson Reuters WestLaw, LexisNexis), often law practitioners are not satisfied with the search results/summaries available on such systems. Developing assistive tools in the legal domain needs addressing certain research challenges. We outline the challenges and some initial progress made on each of them.
Hier-SPCNet: A Legal Statute Hierarchy-based Heterogeneous Network for Computing Legal Case Document Similarity
Authors: Paheli Bhattacharya, Kripabandhu Ghosh, Arindam Pal 0001, Saptarshi Ghosh 0001
Venue: SIGIR
Year: 2020

Computing similarity between two legal case documents is a challenging task, for which text-based and network-based measures have been proposed in literature. All prior network-based similarity methods considered a precedent citation network among case documents only (PCNet). However, this approach misses an important source of legal knowledge - the hierarchy of legal statutes that are applicable in a given legal jurisdiction (e.g., country). We propose to augment the PCNet with the hierarchy of legal statutes, to form a heterogeneous network Hier-SPCNet. Experiments over a set of Indian Supreme Court case documents show that Hier-SPCNet enables significantly better document similarity estimation, as compared to existing approaches using PCNet. We also show that the proposed network-based method can complement text-based measures for better estimation of legal document similarity.
A Transformer-based Embedding Model for Personalized Product Search
Authors: Keping Bi, Qingyao Ai, W. Bruce Croft
Venue: SIGIR
Year: 2020

Product search is an important way for people to browse and purchase items on E-commerce platforms. While customers tend to make choices based on their personal tastes and preferences, analysis of commercial product search logs has shown that personalization does not always improve product search quality. Most existing product search techniques, however, conduct undifferentiated personalization across search sessions. They either use a fixed coefficient to control the influence of personalization or let personalization take effect all the time with an attention mechanism. The only notable exception is the recently proposed zero-attention model (ZAM) that can adaptively adjust the effect of personalization by allowing the query to attend to a zero vector. Nonetheless, in ZAM, personalization can act at most as equally important as the query and the representations of items are static across the collection regardless of the items co-occurring in the user's historical purchases. Aware of these limitations, we propose a transformer-based embedding model (TEM) for personalized product search, which could dynamically control the influence of personalization by encoding the sequence of query and user's purchase history with a transformer architecture. Personalization could have a dominant impact when necessary and interactions between items can be taken into consideration when computing attention weights. Experimental results show that TEM outperforms state-of-the-art personalization product retrieval models significantly.