OpenRaiser/Intern-Atlas huggingface dataset:
Multiple Topics Community Detection in Attributed Networks
Authors: Chaobo He, Junwei Cheng, Guohua Chen, Yong Tang
Venue: SIGIR
Year: 2023

Since existing methods are often not effective to detect communities with multiple topics in attributed networks, we propose a method named SSAGCN via Autoencoder-style self-supervised learning. SSAGCN firstly designs an adaptive graph convolutional network (AGCN), which is treated as the encoder for fusing topology information and attribute information automatically, and then utilizes a dual decoder to simultaneously reconstruct network topology and attributes. By further introducing the modularity maximization and the joint optimization strategies, SSAGCN can detect communities with multiple topics in an end-to-end manner. Experimental results show that SSAGCN outperforms state-of-the-art approaches, and also can be used to conduct topic analysis well.
Do-GOOD: Towards Distribution Shift Evaluation for Pre-Trained Visual Document Understanding Models
Authors: Jiabang He, Yi Hu, Lei Wang, Xing Xu, Ning Liu, Hui Liu, Heng Tao Shen
Venue: SIGIR
Year: 2023

Numerous pre-training techniques for visual document understanding (VDU) have recently shown substantial improvements in performance across a wide range of document tasks. However, these pre-trained VDU models cannot guarantee continued success when the distribution of test data differs from the distribution of training data. In this paper, to investigate how robust existing pre-trained VDU models are to various distribution shifts, we first develop an out-of-distribution (OOD) benchmark termed Do-GOOD for the fine-Grained analysis on Document image-related tasks specifically. The Do-GOOD benchmark defines the underlying mechanisms that result in different distribution shifts and contains 9 OOD datasets covering 3 VDU related tasks, e.g., document information extraction, classification and question answering. We then evaluate the robustness and perform a fine-grained analysis of 5 latest VDU pre-trained models and 2 typical OOD generalization algorithms on these OOD datasets. Results from the experiments demonstrate that there is a significant performance gap between the in-distribution (ID) and OOD settings for document images, and that fine-grained analysis of distribution shifts can reveal the brittle nature of existing pre-trained VDU models and OOD generalization algorithms. The code and datasets for our Do-GOOD benchmark can be found at https://github.com/MAEHCM/Do-GOOD.
DMBIN: A Dual Multi-behavior Interest Network for Click-Through Rate Prediction via Contrastive Learning
Authors: Tianqi He, Kaiyuan Li, Shan Chen, Haitao Wang, Qiang Liu, Xingxing Wang, Dong Wang
Venue: SIGIR
Year: 2023

Click-through rate (CTR) prediction plays a critical role in various online applications, aiming to estimate the user's click probability. User interest modeling from various interactive behaviors(e.g., click, add-to-cart, order) is becoming a mainstream approach to CTR prediction. We argue that the various user behaviors contain two important intrinsic characteristics: 1) The discrepancy in various behaviors reveals different aspects of user's behavior-specific interests. For example, one may click out of need but pay more attention to the rating when purchasing. 2) The consistency of various behaviors contains user's behavior-invariant interest. For example, the user prefers interacted items rather than other items. Therefore, it is necessary to disentangle the discrepancy and consistency signals from the massive behavior information. Unfortunately, previous methods have yet to study this phenomenon well, which limits the recommendation performance. To tackle this challenge, we propose a novel Dual Multi-Behavior Interest Network (DMBIN for short) to disentangle behavior-specific and behavioral-invariant interests from various behaviors for a better recommendation. Specifically, DMBIN formalizethe discrepancy and consistency characteristics among various behaviors. Extensive experiments and empirical analysis on two real-world datasets demonstrate that DMBIN significantly outperforms the state-of-the-art methods. Moreover, DMBIN is also deployed in the online sponsored search advertising system in Meituan and achieves 2.11% and 2.76% improvement on CTR and CPM, respectively.s the dismantlement task as two contrastive learning tasks of multi-behavior interests extracted through the Multi-behavior Interest Module: Multi-behavior Interest Contrast(MIC) task and Multi-behavior Interest Alignment(MIA) task. These two tasks focus on extracting
Candidate-aware Graph Contrastive Learning for Recommendation
Authors: Wei He, Guohao Sun, Jinhu Lu, Xiu Susie Fang
Venue: SIGIR
Year: 2023

Recently, Graph Neural Networks (GNNs) have become a mainstream recommender system method, where it captures high-order collaborative signals between nodes by performing convolution operations on the user-item interaction graph to predict user preferences for different items. However, in real scenarios, the user-item interaction graph is extremely sparse, which means numerous users only interact with a small number of items, resulting in the inability of GNN in learning high-quality node embeddings. To alleviate this problem, the Graph Contrastive Learning (GCL)-based recommender system method is proposed. GCL improves embedding quality by maximizing the similarity of the positive pair and minimizing the similarity of the negative pair. However, most GCL-based methods use heuristic data augmentation methods, i.e., random node/edge drop and attribute masking, to construct contrastive pairs, resulting in the loss of important information. To solve the problems in GCL-based methods, we propose a novel method, Candidate-aware Graph Contrastive Learning for Recommendation, called CGCL. In CGCL, we explore the relationship between the user and the candidate item in the embedding at different layers and use similar semantic embeddings to construct contrastive pairs. By our proposed CGCL, we construct structural neighbor contrastive learning objects, candidate contrastive learning objects, and candidate structural neighbor contrastive learning objects to obtain high-quality node embeddings. To validate the proposed model, we conducted extensive experiments on three publicly available datasets. Compared with various state-of-the-art DNN-, GNN- and GCL-based methods, our proposed CGCL achieved significant improvements in all indicators.
MetroScope: An Advanced System for Real-Time Detection and Analysis of Metro-Related Threats and Events via Twitter
Authors: Jianfeng He, Syuan-Ying Wu, Abdulaziz Alhamadani, Chih-Fang Chen, Wen-Fang Lu, Chang-Tien Lu, David Solnick, Yanlin Li
Venue: SIGIR
Year: 2023

Metro systems are vital to our daily lives, but they face safety or reliability challenges, such as criminal activities or infrastructure disruptions, respectively. Real-time threat detection and analysis are crucial to ensure their safety and reliability. Although many existing systems use Twitter to detect metro-related threats or events in real-time, they have limitations in event analysis and system maintenance. Specifically, they cannot analyze event development, or prioritize events from numerous tweets. Besides, their users are required to continuously monitor system notifications, use inefficient content retrieval methods, and perform detailed system maintenance. We addressed those issues by developing the MetroScope system, a real-time threat/event detection system applied to Washington D.C. metro system. MetroScope can automatically analyze event development, prioritize events based on urgency, send emergency notifications via emails, provide efficient content retrieval, and self-maintain the system. Our MetroScope system is now available at http://orion.nvc.cs.vt.edu:5000/, with a video (https://www.youtube.com/watch?v=vKIK9M60-J8) introducing its features and instructions. MetroScope is a significant advancement in enhancing the safety and reliability of metro systems.
Multimodal Named Entity Recognition and Relation Extraction with Retrieval-Augmented Strategy
Authors: Xuming Hu
Venue: SIGIR
Year: 2023

Multimodal Named Entity Recognition (MNER) and Multimodal Relation Extraction (MRE) are tasks in information retrieval that aim to recognize entities and extract relations among them using information from multiple modalities, such as text and images. Although current methods have attempted a variety of modality fusion approaches to enhance the information in text, a large amount of readily available internet retrieval data has not been considered. Therefore, we attempt to retrieve real-world text related to images, objects, and entire sentences from the internet and use this retrieved text as input for cross-modal fusion to improve the performance of entity and relation extraction tasks in the text.
SelfLRE: Self-refining Representation Learning for Low-resource Relation Extraction
Authors: Xuming Hu, Junzhe Chen, Shiao Meng, Lijie Wen, Philip S. Yu
Venue: SIGIR
Year: 2023

Low-resource relation extraction (LRE) aims to extract potential relations from limited labeled corpus to handle the problem of scarcity of human annotations. Previous works mainly consist of two categories of methods: (1) Self-training methods, which improve themselves through the models' predictions, thus suffering from confirmation bias when the predictions are wrong. (2) Self-ensembling methods, which learn task-agnostic representations, therefore, generally do not work well for specific tasks. In our work, we propose a novel LRE architecture named SelfLRE, which leverages two complementary modules, one module uses self-training to obtain pseudo-labels for unlabeled data, and the other module uses self-ensembling learning to obtain the task-agnostic representations, and leverages the existing pseudo-labels to refine the better task-specific representations on unlabeled data. The two models are jointly trained through multi-task learning to iteratively improve the effect of LRE task. Experiments on three public datasets show that SelfLRE achieves 1.81% performance gain over the SOTA baseline. Source code is available at: https://github.com/THU-BPM/SelfLRE.
A Data-centric Solution to Improve Online Performance of Customer Service Bots
Authors: Sen Hu, Changlin Yang, Junjie Wang, Siye Liu, Teng Xu, Wangshu Zhang, Jing Zheng
Venue: SIGIR
Year: 2023

The online performance of customer service bots is often less than satisfactory because of the gap between limited training data and real-world user questions. As a straightforward way to improve online performance, model iteration and re-deployment are time consuming and labor-intensive, and therefore difficult to sustain. To fix badcases and improve online performance of chatbots in a timely and continuous manner, we propose a data-centric solution consisting of three main modules: badcase detection, bad case correction, and answer extraction. By making full use of online model signals, implicit user feedback and artificial customer service log, the proposed solution can fix online badcases automatically. Our solution has been deployed and bringing consistently positive impacts for hundreds of customer service bots used by Alipay app.
Dual Contrastive Transformer for Hierarchical Preference Modeling in Sequential Recommendation
Authors: Chengkai Huang, Shoujin Wang, Xianzhi Wang, Lina Yao
Venue: SIGIR
Year: 2023

Sequential recommender systems (SRSs) aim to predict the subsequent items which may interest users via comprehensively modeling users' complex preference embedded in the sequence of user-item interactions. However, most of existing SRSs often model users' single low-level preference based on item ID information while ignoring the high-level preference revealed by item attribute information, such as item category. Furthermore, they often utilize limited sequence context information to predict the next item while overlooking richer inter-item semantic relations. To this end, in this paper, we proposed a novel hierarchical preference modeling framework to substantially model the complex low- and high-level preference dynamics for accurate sequential recommendation. Specifically, in the framework, a novel dual-transformer module and a novel dual contrastive learning scheme have been designed to discriminatively learn users' low- and high-level preference and to effectively enhance both low- and high-level preference learning respectively. In addition, a novel semantics-enhanced context embedding module has been devised to generate more informative context embedding for further improving the recommendation performance. Extensive experiments on six real-world datasets have demonstrated both the superiority of our proposed method over the state-of-the-art ones and the rationality of our design.
RiverText: A Python Library for Training and Evaluating Incremental Word Embeddings from Text Data Streams
Authors: Gabriel Iturra-Bocaz, Felipe Bravo-Marquez
Venue: SIGIR
Year: 2023

Word embeddings have become essential components in various information retrieval and natural language processing tasks, such as ranking, document classification, and question answering. However, despite their widespread use, traditional word embedding models present a limitation in their static nature, which hampers their ability to adapt to the constantly evolving language patterns that emerge in sources such as social media and the web (e.g., new hashtags or brand names). To overcome this problem, incremental word embedding algorithms are introduced, capable of dynamically updating word representations in response to new language patterns and processing continuous data streams. This paper presents RiverText, a Python library for training and evaluating incremental word embeddings from text data streams. Our tool is a resource for the information retrieval and natural language processing communities that work with word embeddings in streaming scenarios, such as analyzing social media. The library implements different incremental word embedding techniques, such as Skip-gram, Continuous Bag of Words, and Word Context Matrix, in a standardized framework. In addition, it uses PyTorch as its backend for neural network training. We have implemented a module that adapts existing intrinsic static word embedding evaluation tasks for word similarity and word categorization to a streaming setting. Finally, we compare the implemented methods with different hyperparameter settings and discuss the results. Our open-source library is available at https://github.com/dccuchile/rivertext.
AttriBERT - Session-based Product Attribute Recommendation with BERT
Authors: Akshay Jagatap, Nikki Gupta, Sachin Farfade, Prakash Mandayam Comar
Venue: SIGIR
Year: 2023

Finding the right product on e-commerce websites with millions of products is a daunting task for a large set of customers. On the search page, product attribute filters a.k.a. "refinements" emerge as a convenient navigational option for customers to narrow down the search results along product attributes of their choice (e.g., Material:Cotton, Color:Black for 'shirt'). However, on mobile devices, refinements are not easily discoverable due to lack of screen space. To improve discoverability, contextually relevant refinements are suggested in-line on search page by refinement recommendation systems. Existing works on refinement recommendations primarily rely on the search context as input, and are trained using aggregated refinement preferences 'explicitly' expressed by customers. These solutions fail to capture 'implicit' preferences expressed during the customer shopping mission through in-session browsing activity. In this paper, we propose a session-based recommendation system (SBRS) which recommends refinements by inferring product attribute preferences of customers based on the sequence of products viewed earlier in the session. For the task of refinement recommendation, we propose a) AttriBERT, a model which extends BERT architecture to learn from the attribute values of products and b) a novel product representation strategy, which represents each product as a dictionary of attribute:value pairs (e.g., RAM Size:64GB). We evaluate our approach on RecSys 2022 Challenge and Amazon e-commerce datasets. Our approach consistently outperforms various state-of-the-art sequence models on the task of session-based refinement recommendation.
Interactive Recommendation System for Meituan Waimai
Authors: Chen Ji, Yacheng Li, Rui Li, Fei Jiang, Xiang Li, Wei Lin, Chenglong Zhang, Wei Wang, Shuyang Wang
Venue: SIGIR
Year: 2023

As the largest local retail & instant delivery platform in China, Meituan Waimai has deployed a personalized recommender system on server and recommend nearby stores to users through APP homepage. To capture real-time intention of users and flexibly adjust the recommendation results on the homepage, we further add an interactive recommender system. The existing interactive recommender systems in the industry mainly capture intention of users based on their feedback on a specific UI of questions. However, we find that it will undermine use fluency and increase use complexity by rashly inserting a new question UI when users browse the homepage. Therefore, we develop an Embedded Interactive Recommender System (EIRS) that directly infers users' intention according to their click behaviors on the homepage and dynamically inserts a new recommendation result into the homepage1. To demonstrate the effectiveness of EIRS, we conduct systematic online A/B Tests, where click-through & conversion rate of the inserted EIRS result is 132% higher than that of the initial result on the homepage, and the overall gross merchandise volume is effectively enhanced by 0.43%.
A Practical Online Allocation Framework at Industry-scale in Constrained Recommendation
Authors: Daohong Jian, Yang Bao, Jun Zhou, Hua Wu
Venue: SIGIR
Year: 2023

Online allocation is a critical challenge in constrained recommendation systems, where the distribution of goods, ads, vouchers, and other content to users with limited resources needs to be managed effectively. While the existing literature has made significant progress in improving recommendation algorithms for various scenarios, less attention has been given to developing and deploying industry-scale online allocation system in an efficient manner. To address this issue, this paper introduces an integrated and efficient learning framework in constrained recommendation scenarios at Alipay. The framework has been tested through experiments, demonstrating its superiority over other state-of-the-art methods.
Uncertainty Quantification for Extreme Classification
Authors: Jyun-Yu Jiang, Wei-Cheng Chang, Jiong Zhang, Cho-Jui Hsieh, Hsiang-Fu Yu
Venue: SIGIR
Year: 2023

Uncertainty quantification is one of the most crucial tasks to obtain trustworthy and reliable machine learning models for decision making. However, most research in this domain has only focused on problems with small label spaces and ignored eXtreme Multi-label Classification (XMC), which is an essential task in the era of big data for web-scale machine learning applications. Moreover, enormous label spaces could also lead to noisy retrieval results and intractable computational challenges for uncertainty quantification. In this paper, we aim to investigate general uncertainty quantification approaches for tree-based XMC models with a probabilistic ensemble-based framework. In particular, we analyze label-level and instance-level uncertainty in XMC, and propose a general approximation framework based on beam search to efficiently estimate the uncertainty with a theoretical guarantee under long-tail XMC predictions. Empirical studies on six large-scale real-world datasets show that our framework not only outperforms single models in predictive performance, but also can serve as strong uncertainty-based baselines for label misclassification and out-of-distribution detection, with significant speedup. Besides, our framework can further yield better state-of-the-art results based on deep XMC models with uncertainty quantification.
RCENR: A Reinforced and Contrastive Heterogeneous Network Reasoning Model for Explainable News Recommendation
Authors: Hao Jiang, Chuanzhen Li, Juanjuan Cai, Jingling Wang
Venue: SIGIR
Year: 2023

Existing news recommendation methods suffer from sparse and weak interaction data, leading to reduced effectiveness and explainability. Knowledge reasoning, which explores inferential trajectories in the knowledge graph, can alleviate data sparsity and provide explicitly recommended explanations. However, brute-force pre-processing approaches used in conventional methods are not suitable for fast-changing news recommendation. Therefore, we propose an explainable news recommendation model: the Reinforced and Contrastive Heterogeneous Network Reasoning Model for Explainable News Recommendation (RCENR), consisting of NHN-R2 and MR&CO frameworks. The NHN-R2 framework generates user/news subgraphs to enhance recommendation and extend the dimensions and diversity of reasoning. The MR&CO framework incorporates contrastive learning with a reinforcement-based strategy for self-supervised and efficient model training. Experiments on the MIND dataset show that RCENR is able to improve recommendation accuracy and provide diverse and credible explanations.
Limitations of Open-Domain Question Answering Benchmarks for Document-level Reasoning
Authors: Ehsan Kamalloo, Charles L. A. Clarke, Davood Rafiei
Venue: SIGIR
Year: 2023

Many recent QA models retrieve answers from passages, rather than whole documents, due to the limitations of deep learning models with limited context size. However, this approach ignores important document-level cues that can be crucial in answering questions. This paper reviews three open-domain QA benchmarks from a document-level perspective and finds that they are biased towards passage-level information. Out of 17,000 assessed questions, 82 were identified as requiring document-level reasoning and could not be answered by passage-based models. Document-level retrieval (BM25) outperformed both dense and sparse passage-level retrieval on these questions, highlighting the need for more evaluation of models' ability to understand documents, an often-overlooked challenge in open-domain QA.
MMEAD: MS MARCO Entity Annotations and Disambiguations
Authors: Chris Kamphuis, Aileen Lin, Siwen Yang, Jimmy Lin, Arjen P. de Vries, Faegheh Hasibi
Venue: SIGIR
Year: 2023

MMEAD, or MS MARCO Entity Annotations and Disambiguations, is a resource for entity links for the MS MARCO datasets. We specify a format to store and share links for both document and passage collections of MS MARCO. Following this specification, we release entity links to Wikipedia for documents and passages in both MS MARCO collections (v1 and v2). Entity links have been produced by the REL and BLINK systems. MMEAD is an easy-to-install Python package, allowing users to load the link data and entity embeddings effortlessly. Using MMEAD takes only a few lines of code. Finally, we show how MMEAD can be used for IR research that uses entity information. We show how to improve recall@1000 and MRR@10 on more complex queries on the MS MARCO v1 passage dataset by using this resource. We also demonstrate how entity expansions can be used for interactive search applications.