OpenRaiser/Intern-Atlas huggingface dataset:
Bridge the Gap: Industrial Data Pipeline oriented Recommendation Enhancement
Authors: Jie Dai
Venue: SIGIR
Year: 2021

In a world of overloading choices, recommender system has become indispensable as personalized information filter, which makes it of great interest to both industry and academia. Recently, inspiring insights are provided by deep learning based algorithms, and bring us many state-of-the-art recommender systems [1]. However, there exists a widening gap between researchers and engineers: academics usually prefer end-to-end recommender systems, whereas engineers tend to design a recommender system as a three stage pipeline, including preprocessing original data, recalling a candidate subset, and ranking recommendation results. Such pipeline can greatly benefit system development and project management, which has been proven by many Internet companies over recent years. To bridge the gap between end-to-end and pipeline, we propose three research directions and relevant methodologies, including label propagation based methods for preprocessing, graph neural network based negative sampling strategy for recall, and graph-based model of implicit interactions for ranking. In the following, we will describe them in detail.
MSSM: A Multiple-level Sparse Sharing Model for Efficient Multi-Task Learning
Authors: Ke Ding, Xin Dong 0012, Yong He 0009, Lei Cheng 0005, Chilin Fu, Zhaoxin Huan, Hai Li, Tan Yan, Liang Zhang, Xiaolu Zhang, Linjian Mo
Venue: SIGIR
Year: 2021

Multi-task learning(MTL) is an open and challenging problem in various real-world applications. The typical way of conducting multi-task learning is establishing some global parameter sharing mechanism across all tasks or assigning each task an individual set of parameters with cross-connections between tasks. However, for most existing approaches, all tasks just thoroughly or proportionally share all the features without distinguishing the helpfulness of them. By that, some tasks would be intervened by the unhelpful features that are useful for other tasks, leading to undesired negative transfer between tasks. In this paper, we design a novel architecture named the Multiple-level Sparse Sharing Model (MSSM), which can learn features selectively and share knowledge across all tasks efficiently. MSSM first employs a field-level sparse connection module (FSCM) to enable much more expressive combinations of feature fields to be learned for generalization across tasks while still allowing for task-specific features to be customized for each task. Furthermore, a cell-level sparse sharing module (CSSM) can recognize the sharing pattern through a set of coding variables that selectively choose which cells to route for a given task. Extensive experimental results on several real-world datasets show that MSSM outperforms SOTA models significantly in terms of AUC and LogLoss metrics.
Interpretable Graph Similarity Computation via Differentiable Optimal Alignment of Node Embeddings
Authors: Khoa D. Doan, Saurav Manchanda, Suchismit Mahapatra, Chandan K. Reddy
Venue: SIGIR
Year: 2021

Computing graph similarity is an important task in many graph-related applications such as retrieval in graph databases or graph clustering. While numerous measures have been proposed to capture the similarity between a pair of graphs, Graph Edit Distance (GED) and Maximum Common Subgraphs (MCS) are the two widely used measures in practice. GED and MCS are domain-agnostic measures of structural similarity between the graphs and define the similarity as a function of pairwise alignment of different entities (such as nodes, edges, and subgraphs) in the two graphs. The explicit explainability offered by the pairwise alignment provides transparency and justification of the similarity score, thus, GED and MCS have important practical applications. However, their exact computations are known to be NP-hard. While recently proposed neural-network based approximations have been shown to accurately compute these similarity scores, they have limited ability in providing comprehensive explanations compared to classical combinatorial algorithms, e.g., Beam search. This paper aims at efficiently approximating these domain-agnostic similarity measures through a neural network, and simultaneously learning the alignments (i.e., explanations) similar to those of classical intractable methods. Specifically, we formulate the similarity between a pair of graphs as the minimal "transformation" cost from one graph to another in the learnable node-embedding space. We show that, if node embedding is able to capture its neighborhood context closely, our proposed similarity function closely approximates both the alignment and the similarity score of classical methods. Furthermore, we also propose an efficient differentiable computation of our proposed objective for model training. Empirically, we demonstrate that the proposed method achieves up to 50%-100% reduction in the Mean Squared Error for the graph similarity approximation task and up to 20% improvement in the retrieval evaluation metrics for the graph retrieval task. The source code is available at https://github.com/khoadoan/GraphOTSim.
Legal Judgment Prediction via Relational Learning
Authors: Qian Dong, Shuzi Niu
Venue: SIGIR
Year: 2021

Given a legal case and all law articles, L egal J udgment P rediction (LJP) is to predict the case's violated articles, charges and term of penalty. Naturally, these labels are entangled among different tasks and within a task. For example, each charge is only logically or semantically related to some fixed articles. Ignoring these constraints, LJP methods would predict unreliable results. To solve this problem, we first formalize LJP as a node classification problem over a global consistency graph derived from the training set. In terms of node encoder, we utilize a masked transformer network to obtain case aware node representations consistent among tasks and discriminative within a task. In terms of node classifier, each node's label distribution is dependent on its neighbors' in this graph to achieve local consistency by relational learning. Both the node encoder and classifier are optimized by variational EM. Finally, we propose a novel measure to evaluate self-consistency of classification results. Experimental results on two benchmark datasets demonstrate that the F1 improvement of our method is about $4.8%$ compared with SOTA methods.
Semi-automatic Labelling of Scientific Articles using Deep Learning to Enlarge Benchmark Data for Scientific Summarization
Authors: Alaa El-Ebshihy
Venue: SIGIR
Year: 2021

Scientific article summarization is a challenging task not least due to the lack of large annotated corpora. In this research proposal, we present an approach to construct a large annotated corpus for scientific articles using semi-supervised/automatic annotation approaches. We intend to apply deep learning methods to increase a small seed of annotated corpus. Then, we will measure the quality of the annotated corpus on down stream informative summaries using various evaluation techniques.
Enabling Performance Prediction in Information Retrieval Evaluation
Authors: Guglielmo Faggioli
Venue: SIGIR
Year: 2021

How to model the performance of a retrieval system before its deploying has puzzled the Information Retrieval (IR)researchers for a long time. Currently, the evaluation of IR systems relies on empirical experiments. Empirical evaluation means that we need experimental collections: building them is expensive both in term of time and money. Exploiting already available collections to predict the performance of a system on new collections, would dramatically reduce such cost. With the research line described in this work,we plan to study the development of predictive models for the performance of the IR systems. In particular, the proposed research line will investigate Generalized Linear Mixed Models and Causal Inference. Furthermore, we highlight the importance of modelling the performance as distributions rather than point estimations.
Clustering-Based Online News Topic Detection and Tracking Through Hierarchical Bayesian Nonparametric Models
Authors: Wentao Fan 0001, Zhiyan Guo, Nizar Bouguila, Wenjuan Hou
Venue: SIGIR
Year: 2021

In this paper, we propose a clustering-based online news topic detection and tracking (TDT) approach based on hierarchical Bayesian nonparametric framework that allows topics to be shared across different news stories in a corpus. Our approach is formulated using the hierarchical Pitman-Yor process mixture model with the inverted Beta-Liouville (IBL) distribution as its component density, which has shown superior performance in modeling text data than the widely used Gaussian distribution. Moreover, we theoretically develop a convergence-guaranteed online learning algorithm that can effectively learn the proposed TDT model from a stream of news stories based on varational Bayes. The merits of our TDT approach are illustrated by comparing it with other well-defined clustering-based TDT approaches on different news data sets.
GemNN: Gating-enhanced Multi-task Neural Networks with Feature Interaction Learning for CTR Prediction
Authors: Hongliang Fei, Jingyuan Zhang, Xingxuan Zhou, Junhao Zhao, Xinyang Qi, Ping Li 0001
Venue: SIGIR
Year: 2021

Deep neural network (DNN) models have been widely used for click-through rate (CTR) prediction in online advertising. The training framework typically consists of embedding layers and multi-layer perceptions (MLP). At Baidu Search Ads (a.k.a. Phoenix Nest), the new generation of CTR training platform has become PaddleBox, a GPU-based parameter server system. In this paper, we present Baidu's recently updated CTR training framework, called Gating-enhanced Multi-task Neural Networks (GemNN). In particular, we develop a neural network based multi-task learning model to predict CTR in a coarse-to-fine manner, which gradually reduces ad candidates and allows parameter sharing from upstream tasks to downstream tasks to improve the training efficiency. Also, we introduce a gating mechanism between embedding layers and MLP to learn feature interactions and control the information flow fed to MLP layers. We have launched our solution in Baidu PaddleBox platform and observed considerable improvements in both offline and online evaluations. It is now part of the current production~system.
Hybrid Learning to Rank for Financial Event Ranking
Authors: Fuli Feng, Moxin Li, Cheng Luo 0001, Ritchie Ng, Tat-Seng Chua
Venue: SIGIR
Year: 2021

The financial markets are moved by events such as the issuance of administrative orders. The participants in financial markets (e.g., traders) thus pay constant attention to financial news relevant to the financial asset (e.g., oil) of interest. Due to the large scale of news stream, it is time and labor intensive to manually identify influential events that can move the price of the financial asset, pushing the financial participants to embrace automatic financial event ranking, which has received relatively little scrutiny to date. In this work, we formulate the financial event ranking task, which aims to score financial news (document) according to its influence to the given asset (query). To solve this task, we propose a Hybrid News Ranking framework that, from the asset perspective, evaluates the influence of news articles by comparing their contents; and from the event perspective, accesses the influence over all query assets. Moreover, we resolve the dilemma between the essential requirement of sufficient labels for training the framework and the unaffordable cost of hiring domain experts for labeling the news. In particular, we design a cost-friendly system for news labeling that leverages the knowledge within published financial analyst reports. In this way, we construct three financial event ranking datasets. Extensive experiments on the datasets validate the effectiveness of the proposed framework and the rationality of solving financial event ranking through learning to rank.
Estimating the Reliability of Health-related Search Results
Authors: Marcos Fernández-Pichel
Venue: SIGIR
Year: 2021

Determining reliability of online data is a challenge that has recently received increasing attention. In particular, unreliable health-related content has become pervasive during the COVID-19 pandemic. The main objective of this Ph.D. thesis is to study how end-users judge the correctness and credibility of online content and provide them with a series of tools to assist them in assessing content reliability. To that end, we need to determine which sources of evidence may help to better assess the reliability of health-related online content, and how to combine them learning. Finally, I will also study which presentation aspects might help end-users to better assess reliability since previous research has proved that the format and layout of the information items, combined with user-based biases, influence their final assessments.
Gazetteer Enhanced Named Entity Recognition for Code-Mixed Web Queries
Authors: Besnik Fetahu, Anjie Fang, Oleg Rokhlenko, Shervin Malmasi
Venue: SIGIR
Year: 2021

Named entity recognition (NER) for Web queries is very challenging. Queries often do not consist of well-formed sentences, and contain very little context, with highly ambiguous queried entities. Code-mixed queries, with entities in a different language than the rest of the query, pose a particular challenge in domains like e-commerce (e.g. queries containing movie or product names). This work tackles NER for code-mixed queries, where entities and non-entity query terms co-exist simultaneously in different languages. Our contributions are twofold. First, to address the lack of code-mixed NER data we create EMBER, a large-scale dataset in six languages with four different scripts. Based on Bing query data, we include numerous language combinations that showcase real-world search scenarios. Secondly, we propose a novel gated architecture that enhances existing multi-lingual Transformers with a Mixture-of-Experts model to dynamically infuse multi-lingual gazetteers, allowing it to simultaneously differentiate and handle entities and non-entity query terms in multiple languages. Experimental evaluation on code-mixed queries in several languages shows that our approach efficiently utilizes gazetteers to recognize entities in code-mixed queries with an F1=68%, an absolute improvement of +31% over a non-gazetteer baseline.
Identifying Queries in Instant Search Logs
Authors: Markus Fischer, Kristof Komlossy, Benno Stein 0001, Martin Potthast, Matthias Hagen
Venue: SIGIR
Year: 2021

Query logs of search engines with instant search functionality are challenging for log analysis, since the log entries represent interactions at the keystroke level, rather than at the query level. To enable log analyses at the query level, a user's logged sequence of keystroke-level interactions needs to be mapped to distinct queries. This problem bears strong parallels to session detection in "standard" query logs (i.e., forming groups of subsequent queries on the same topic), but there are salient differences. In this paper, we present a new approach to identifying interactions belonging to the same query in instant query logs. In an experimental comparison, our new approach achieves an F2 score of 0.93 compared to only 0.83 of a state-of-the-art cascading method for query log session detection.
SDG: A Simplified and Dynamic Graph Neural Network
Authors: Dongqi Fu, Jingrui He
Venue: SIGIR
Year: 2021

Graph Neural Networks (GNNs) have achieved state-of-the-art performance in many high-impact applications such as fraud detection, information retrieval, and recommender systems due to their powerful representation learning capabilities. Some nascent efforts have been concentrated on simplifying the structures of GNN models, in order to reduce the computational complexity. However, the dynamic nature of these applications requires GNN structures to be evolving over time, which has been largely overlooked so far. To bridge this gap, in this paper, we propose a simplified and dynamic graph neural network model, called SDG. It is efficient, effective, and provides interpretable predictions. In particular, in SDG, we replace the traditional message-passing mechanism of GNNs with the designed dynamic propagation scheme based on the personalized PageRank tracking process. We conduct extensive experiments and ablation studies to demonstrate the effectiveness and efficiency of our proposed SDG. We also design a case study on fake news detection to show the interpretability of SDG.
Automatic Form Filling with Form-BERT
Authors: Gilad Fuchs, Haggai Roitman, Matan Mandelbrod
Venue: SIGIR
Year: 2021

Digital-forms are commonly used for collecting structured information from users. However, filling digital-forms that include a large number of fields is tedious and error-prone. Auto-filling form fields for the user is highly beneficial for improving user experience and potentially collecting more valuable information (in cases where not all fields are mandatory). Online E-commerce marketplaces quite often utilize such forms to collect listing attributes from sellers. In this work, we describe Form-BERT -- a Transformer-based model which is optimized for auto-filling listing attributes given the following inputs: free-text, list of known attribute names, and zero or more attribute values. Form-BERT can be further used iteratively to leverage filled out attributes as the form filling progresses.
RCD: Relation Map Driven Cognitive Diagnosis for Intelligent Education Systems
Authors: Weibo Gao, Qi Liu 0003, Zhenya Huang, Yu Yin 0002, Haoyang Bi, Mu-Chun Wang, Jianhui Ma 0001, Shijin Wang 0001, Yu Su 0002
Venue: SIGIR
Year: 2021

Cognitive diagnosis (CD) is a fundamental issue in intelligent educational settings, which aims to discover the mastery levels of students on different knowledge concepts. In general, most previous works consider it as an inter-layer interaction modeling problem, e.g., student-exercise interactions in IRT or student-concept interactions in DINA, while the inner-layer structural relations, such as educational interdependencies among concepts, are still underexplored. Furthermore, there is a lack of comprehensive modeling for the student-exercise-concept hierarchical relations in CD systems. To this end, in this paper, we present a novel Relation map driven Cognitive Diagnosis (RCD) framework, uniformly modeling the interactive and structural relations via a multi-layer student-exercise-concept relation map. Specifically, we first represent students, exercises and concepts as individual nodes in a hierarchical layout, and construct three well-defined local relation maps to incorporate inter- and inner-layer relations, including a student-exercise interaction map, a concept-exercise correlation map and a concept dependency map. Then, we leverage a multi-level attention network to integrate node-level relation aggregation inside each local map and balance map-level aggregation across different maps. Finally, we design an extendable diagnosis function to predict students' performance and jointly train the networks. Extensive experimental results on real-world datasets clearly show the effectiveness and extendibility of our RCD in both diagnosis accuracy improvement and relation-aware representation learning.
Maximal Multipolarized Cliques Search in Signed Networks
Authors: Jie Gao, Fei Hao 0001, Geyong Min, Zhipeng Cai 0001
Venue: SIGIR
Year: 2021

The increasing of group polarization on social media seriously impacts on the health of public discourse and information dissemination. At present, detecting polarized structures in signed networks is well-motivated for studying the group polarization on social media. However, most studies restricted the number of polarized structures to only two, while neglecting the real-world scenario where signed networks consist of multiple polarized structures, that is an unreasonable assumption. To conquer the limitations of the existing work, in this paper, we present a novel cohesive subgraph model based on structural clusterable theory, named maximal multipolarized clique (MMC), which can be partitioned into k polarized subcliques such that the edges in subcliques are positive and the edges between subcliques are negative. This paper formulates the problem of Maximal Multipolarized Cliques Search (MMCS) in signed networks which is proved to be NP-hard. To address this problem, we first devise powerful pruning rules to reduce the signed network significantly and further develop an efficient algorithm to search all maximal multipolarized cliques in the reduced signed network. The experimental results on real-world signed networks demonstrate the efficiency and effectiveness of our algorithm.
Iterative Network Pruning with Uncertainty Regularization for Lifelong Sentiment Classification
Authors: Binzong Geng, Min Yang 0007, Fajie Yuan, Shupeng Wang, Xiang Ao 0001, Ruifeng Xu 0001
Venue: SIGIR
Year: 2021

Lifelong learning capabilities are crucial for sentiment classifiers to process continuous streams of opinioned information on the Web. However, performing lifelong learning is non-trivial for deep neural networks as continually training of incrementally available information inevitably results in catastrophic forgetting or interference. In this paper, we propose a novel i terative network p runing with uncertainty r egularization method for l ifelong s entiment classification (IPRLS), which leverages the principles of network pruning and weight regularization. By performing network pruning with uncertainty regularization in an iterative manner, IPRLS can adapt a single BERT model to work with continuously arriving data from multiple domains while avoiding catastrophic forgetting and interference. Specifically, we leverage an iterative pruning method to remove redundant parameters in large deep networks so that the freed-up space can then be employed to learn new tasks, tackling the catastrophic forgetting problem. Instead of keeping the old-tasks fixed when learning new tasks, we also use an uncertainty regularization based on the Bayesian online learning framework to constrain the update of old tasks weights in BERT, which enables positive backward transfer, i.e. learning new tasks improves performance on past tasks while protecting old knowledge from being lost. In addition, we propose a task-specific low-dimensional residual function in parallel to each layer of BERT, which makes IPRLS less prone to losing the knowledge saved in the base BERT network when learning a new task. Extensive experiments on 16 popular review corpora demonstrate that the proposed IPRLS method significantly outperforms the strong baselines for lifelong sentiment classification. For reproducibility, we submit the code and data at: \urlhttps://github.com/siat-nlp/IPRLS .
Putting the Role of Personalization into Context
Authors: Dmitri Goldenberg
Venue: SIGIR
Year: 2021

Personalization is omnipresent in our life, with applications ranging from entertainment and commercial uses to smart devices and medical treatments. The integration of personalization in various products turned rapidly from an unnecessary luxury to a commodity that is expected by customers. While different machine learning fields present state-of-the-art advances and super-human performance, personalization applications are often late-adopters of novel solutions due to their complex framing and multiple stakeholders' with different business goals. The role of personalisation applications is also ambiguous: it is unclear, for instance, whether models just predict a user's next action or proactively affect the user's selections. This talk focuses on examining the role of recommenders and their ability to adapt to customer feedback. Key topics such as causality and active exploration are depicted with real examples and demonstrated alongside business considerations and implementation challenges. It relies on recent advances in the field and on work conducted at Booking.com, where we implement personalization models on one of the world's leading online travel platform.