OpenRaiser/Intern-Atlas huggingface dataset:
Learning Query-aware Embedding Index for Improving E-commerce Dense Retrieval
Authors: Mingming Li, Chunyuan Yuan, Binbin Wang, Jingwei Zhuo, Songlin Wang, Lin Liu, Sulong Xu
Venue: SIGIR
Year: 2023

The embedding index has become an essential part of the dense retrieval (DR) system, which enables a fast search for billion of items in online E-commerce applications. To accelerate the retrieval process in industrial scenarios, most of the previous studies only utilize item embeddings. However, the product quantization process without query embeddings will lead to inconsistency between queries and items. A straightforward solution is to put query embedding into the product quantization process. But we found that the distance of the positive query and item embedding pairs is too large, which means the query and item embeddings learned by the two-tower are not fully aligned. This problem would lead to performance decay when directly putting query embeddings into the product quantization. In this paper, we propose a novel query-aware embedding Index framework, which aligns the query and item embedding space to reduce the distance between positive pairs, thereby mixing the query and item embeddings to learn better cluster centers for product quantization. Specifically, we first propose s symmetric loss to train a better two-tower to achieve space alignment. Subsequently, we propose a mixed quantization strategy to put the query embeddings into the product quantization process for bridging the gap between queries and compressed item embeddings. Extensive experiments show that our framework significantly outperforms previous models on a real-world dataset, which demonstrates the superiority and effectiveness of the framework.
Power Norm Based Lifelong Learning for Paraphrase Generations
Authors: Dingcheng Li, Peng Yang, Yue Zhang, Ping Li
Venue: SIGIR
Year: 2023

Lifelong seq2seq language generation models are trained with multiple domains in a lifelong learning manner, with data from each domain being observed in an online fashion. It is a well-known problem that lifelong learning suffers from the catastrophic forgetting (CF). To handle this challenge, existing works have leveraged experience replay or dynamic architecture to consolidate the past knowledge, which however result in incremental memory space or high computational cost. In this work, we propose a novel framework name "power norm based lifelong learning" (PNLLL), which aims to remedy the catastrophic forgetting issues with a power normalization on NLP transformer models. Specifically, PNLLL leverages power norm to achieve a better balance between past experience rehearsal and new knowledge acquisition. These designs enable the knowledge adaptation onto new tasks while memorizing the experience of past tasks. Our experiments on paraphrase generation tasks show that PNLLL not only outperforms SOTA models by a considerable margin and but also largely alleviates forgetting.
Matching Point of Interests and Travel Blog with Multi-view Information Fusion
Authors: Shuokai Li, Jingbo Zhou, Jizhou Huang, Hao Chen, Fuzhen Zhuang, Qing He, Dejing Dou
Venue: SIGIR
Year: 2023

The past few years have witnessed an explosive growth of user-generated POI-centric travel blogs, which can provide a comprehensive understanding of a POI for people. However, evaluating the quality of the POI-centric travel blogs and ranking the blogs is not a simple task without domain knowledge or actual travel experience on the target POI. Nevertheless, our insight is that the user search behavior related to the target POI on the online map service can partly valid the rationality of the POIs appearing in the travel blogs, which helps for travel blogs ranking. To this end, in this paper, we propose a novel end-to-end framework for travel blogs ranking, coined Matching POI and Travel Blogs with Multi-view InFormation (MOTIF). Concretely, we first construct two POI graphs as multi-view information: (1) the search-level POI graph which reflects the user behaviors on the online map service; and (2) the document-level POI graph which shows the POI co-occurrence frequency in travel blogs. Then, to better model the intrinsic correlation of the two graphs, we adopt Mutual Information Maximization to align the search-level and document-level semantic spaces. Moreover, we leverage a pair-wise ranking loss for POI-document relevance scoring. Extensive experiments on two real-world datasets demonstrate the superiority of our method.
Next Basket Recommendation with Intent-aware Hypergraph Adversarial Network
Authors: Ran Li, Liang Zhang, Guannan Liu, Junjie Wu
Venue: SIGIR
Year: 2023

Next Basket Recommendation (NBR) that recommends a basket of items to users has become a promising promotion artifice for online businesses. The key challenge of NBR is rooted in the complicated relations of items that are dependent on one another in a same basket with users' diverse purchasing intentions, which goes far beyond the pairwise item relations in traditional recommendation tasks, and yet has not been well addressed by existing NBR methods that mostly model the inter-basket item relations only. To that end, in this paper, we construct a hypergraph from basket-wise purchasing records and probe the inter-basket and intra-basket item relations behind the hyperedges. In particular, we combine the strength of HyperGraph Neural Network with disentangled representation learning to derive the intent-aware representations of hyperedges for characterizing the nuances of user purchasing patterns. Moreover, considering the information loss in traditional item-wise optimization, we propose a novel basket-wise optimization scheme via an adversarial network to generate high-quality negative baskets. Extensive experiments conducted on four different data sets demonstrate the superior performances over the state-of-the-art NBR methods. Notably, our method is shown to strike a good balance in recommending both repeated and explorative items as a basket.
TAML: Time-Aware Meta Learning for Cold-Start Problem in News Recommendation
Authors: Jingyuan Li, Yue Zhang, Xuan Lin, Xinxing Yang, Ge Zhou, Longfei Li, Hong Chen, Jun Zhou
Venue: SIGIR
Year: 2023

Meta-learning has become a widely used method for the user cold-start problem in recommendation systems, as it allows the model to learn from similar learning tasks and transfer the knowledge to new tasks. However, most existing meta-learning methods do not consider the temporal factor of users' preferences, which is crucial for news recommendation scenarios where news streams change dynamically over time. In this paper, we propose Time-Aware Meta-Learning (TAML), a novel framework that focuses on cold-start users in news recommendation systems. TAML factorizes user preferences into time-specifc and time-shift representations that jointly affect users' news preferences. These temporal factors are further incorporated into the meta-learning framework to achieve accurate and timely cold-start recommendations. Extensive experiments are conducted on two real-world datasets, demonstrating the superior performance of TAML over state-of-the-art methods.
PersonalTM: Transformer Memory for Personalized Retrieval
Authors: Ruixue Lian, Sixing Lu, Clint Solomon, Gustavo Aguilar, Pragaash Ponnusamy, Jialong Han, Chengyuan Ma, Chenlei Guo
Venue: SIGIR
Year: 2023

The Transformer Memory as a Differentiable Search Index (DSI) has been proposed as a new information retrieval paradigm, which aims to address the limitations of dual-encoder retrieval framework based on the similarity score. The DSI framework outperforms strong baselines by directly generating relevant document identifiers from queries without relying on an explicit index. The memorization power of DSI framework makes it suitable for personalized retrieval tasks. Therefore, we propose a Personal Transformer Memory (PersonalTM) architecture for personalized text retrieval. PersonalTM incorporates user-specific profiles and contextual user click behaviors, and introduces hierarchical loss in the decoding process to align with the hierarchical assignment of document identifier. Additionally, PersonalTM also employs an adapter architecture to improve the scalability for index updates and reduce computation costs, compared to the vanilla DSI. Experiments show that PersonalTM outperforms the DSI baseline, BM25, fine-tuned dual-encoder, and other personalized models in terms of precision at top 1st and 10th positions and Mean Reciprocal Rank (MRR). Specifically, PersonalTM improves p@1 by 58%, 49%, and 12% compared to BM25, Dual-encoder, and DSI, respectively.
On the "Rough Use" of Machine Learning Techniques
Authors: Chih-Jen Lin
Venue: SIGIR
Year: 2023

Machine learning is everywhere, but unfortunately, we are not experts of every method. Sometimes we "inappropriately'' use machine learning techniques. Examples include reporting training instead of test performance and comparing two methods without suitable hyper-parameter searches. However, the reality is that there are more sophisticated or more subtle examples, which we broadly call the "rough use'' of machine learning techniques. The setting may be roughly fine, but seriously speaking, is inappropriate. We briefly discuss two intriguing examples. - In the topic of graph representation learning, to evaluate the quality of the obtained representations, the multi-label problem of node classification is often considered. An unrealistic setting was used in almost the entire area by assuming that the number of labels of each test instance is known in the prediction stage. In practice, such ground truth information is rarely available. Details of this interesting story are in Lin et al. (2021). - In training deep neural networks, the optimization process often relies on the validation performance for termination or selecting the best epoch. Thus in many public repositories, training, validation, and test sets are explicitly provided. Many think this setting is standard in applying any machine learning technique. However, except that the test set should be completely independent, users can do whatever the best setting on all the available labeled data (i.e., training and validation sets combined). Through real stories, we show that many did not clearly see the relation between training, validation, and test sets. The rough use of machine learning methods is common and sometimes unavoidable. The reason is that nothing is called a perfect use of a machine learning method. Further, it is not easy to assess the seriousness of the situation. We argue that having high-quality and easy-to-use software is an important way to improve the practical use of machine learning techniques.
Gated Attention with Asymmetric Regularization for Transformer-based Continual Graph Learning
Authors: Hongxiang Lin, Ruiqi Jia, Xiaoqing Lyu
Venue: SIGIR
Year: 2023

Continual graph learning (CGL) aims to mitigate the topological-feature-induced catastrophic forgetting problem (TCF) in graph neural networks, which plays an essential role in the field of information retrieval. The TCF is mainly caused by the forgetting of node features of old tasks and the forgetting of topological features shared by old and new tasks. Existing CGL methods do not pay enough attention to the forgetting of topological features shared between different tasks. In this paper, we propose a transformer-based CGL method (Trans-CGL), thereby taking full advantage of the transformer's properties to mitigate the TCF problem. Specifically, to alleviate forgetting of node features, we introduce a gated attention mechanism for Trans-CGL based on parameter isolation that allows the model to be independent of each other when learning old and new tasks. Furthermore, to address the forgetting of shared parameters that store topological information between different tasks, we propose an asymmetric mask attention regularization module to constrain the shared attention parameters ensuring that the shared topological information is preserved. Comparative experiments show that the method achieves competitive performance on four real-world datasets.
Exploring the Spatiotemporal Features of Online Food Recommendation Service
Authors: Shaochuan Lin, Jiayan Pei, Taotao Zhou, Hengxu He, Jia Jia, Ning Hu
Venue: SIGIR
Year: 2023

Online Food Recommendation Service (OFRS) has remarkable spatiotemporal characteristics and the advantage of being able to conveniently satisfy users' needs in a timely manner. There have been a variety of studies that have begun to explore its spatiotemporal properties, but a comprehensive and in-depth analysis of the OFRS spatiotemporal features is yet to be conducted. Therefore, this paper studies the OFRS based on three questions: how spatiotemporal features play a role; why self-attention cannot be used to model the spatiotemporal sequences of OFRS; and how to combine spatiotemporal features to improve the efficiency of OFRS. Firstly, through experimental analysis, we systemically extracted the spatiotemporal features of OFRS, identified the most valuable features and designed an effective combination method. Secondly, we conducted a detailed analysis of the spatiotemporal sequences, which revealed the shortcomings of self-attention in OFRS, and proposed a more optimized spatiotemporal sequence method for replacing self-attention. In addition, we also designed a Dynamic Context Adaptation Model to further improve the efficiency and performance of OFRS. Through the offline experiments on two large datasets and online experiments for a week, the feasibility and superiority of our model were proven.
Hierarchical Type Enhanced Negative Sampling for Knowledge Graph Embedding
Authors: Zhenzhou Lin, Zishuo Zhao, Jingyou Xie, Ying Shen
Venue: SIGIR
Year: 2023

Knowledge graph embedding aims at modeling knowledge by projecting entities and relations into a low-dimensional semantic space. Most of the works on knowledge graph embedding construct negative samples by negative sampling as knowledge graphs typically only contain positive facts. Although substantial progress has been made by dynamic distribution based sampling methods, selecting plausible and prior information-engaged negative samples still poses many challenges. Inspired by type constraint methods, we propose Hierarchical Type Enhanced Negative Sampling (HTENS) which leverages hierarchical entity type information and entity-relation cooccurrence information to optimize the sampling probability distribution of negative samples. The experiments performed on the link prediction task demonstrate the effectiveness of HTENS. Additionally, HTENS shows its superiority in versatility and can be integrated into scalable systems with enhanced negative sampling.
The 1st International Workshop on Implicit Author Characterization from Texts for Search and Retrieval (IACT'23)
Authors: Marina Litvak, Irina Rabaev, Ricardo Campos, Alípio Mário Jorge, Adam Jatowt
Venue: SIGIR
Year: 2023

The first edition of the Implicit Author Characterization from Texts for Search and Retrieval (IACT'23) aims at bringing to the forefront the challenges involved in identifying and extracting from texts implicit information about authors (e.g., human or AI) and using it in IR tasks. The IACT workshop provides a common forum to consolidate multi-disciplinary efforts and foster discussions to identify the wide-ranging issues related to the task of extracting implicit author-related information from the textual content, including novel tasks and datasets. We will also discuss the ethical implications of implicit information extraction. In addition, we announce a shared task focused on automatically determining the literary epochs of written books.
Online Conversion Rate Prediction via Neural Satellite Networks in Delayed Feedback Advertising
Authors: Qiming Liu, Haoming Li, Xiang Ao, Yuyao Guo, Zhihong Dong, Ruobing Zhang, Qiong Chen, Jianfeng Tong, Qing He
Venue: SIGIR
Year: 2023

The delayed feedback is becoming one of the main obstacles in online advertising due to the pervasive deployment of the cost-per-conversion display strategy requesting a real-time conversion rate (CVR) prediction. It makes the observed data contain a large number of fake negatives that temporarily have no feedback but will convert later. Training on such biased data distribution would severely harm the performance of models. Prevailing approaches wait for a set period of time to see if samples convert before training on them, but solutions to guaranteeing data freshness remain under-explored by current research. In this work, we propose Delayed Feed-back modeling via neural Satellite Networks (DFSN for short) for online CVR prediction. It tackles the issue of data freshness to permit adaptive waiting windows. We first assign a long waiting window for our main model to cover most of conversions and greatly reduce fake negatives. Meanwhile, two kinds of satellite models are devised to learn from the latest data, and online transfer learning techniques are utilized to sufficiently exploit their knowledge. With information from satellites, our main model can deal with the issue of data freshness, achieving better performance than previous methods. Extensive experiments on two real-world advertising datasets demonstrate the superiority of our model.
Unsupervised Readability Assessment via Learning from Weak Readability Signals
Authors: Yuliang Liu, Zhiwei Jiang, Yafeng Yin, Cong Wang, Sheng Chen, Zhaoling Chen, Qing Gu
Venue: SIGIR
Year: 2023

Unsupervised readability assessment aims to evaluate the reading difficulty of text without any manually-labeled data for model training. This is a challenging task because the absence of labeled data makes it difficult for the model to understand what readability is. In this paper, we propose a novel framework to Learn a neural model from Weak Readability Signals (LWRS). Instead of relying on labeled data, LWRS utilizes a set of heuristic signals that specialize in describing text readability from different aspects to guide the model in outputting readability scores for ranking. Specifically, to effectively use multiple heuristic weak signals for model training, we build a multi-signal learning model that ranks the unlabeled texts from multiple readability-related aspects based on intra- and inter-signal learning. We also adopt the pairwise ranking paradigm to reduce the cascade coupling among partial-order pairs. Furthermore, we propose identifying the most representative signal based on the batch-level consensus distribution of all signals. This strategy helps identify the predicted signal that is most correlated with readability in the absence of ground-truth labels. We conduct experiments on three public readability assessment datasets. The experimental results demonstrate that our LWRS outperforms each heuristic signal and their combinations significantly, and can even perform comparably with some supervised methods. Additionally, our LWRS trained on one dataset can be effectively transferred to other datasets, including those in other languages, which indicates its good generalization and potential for wide application.
Bootstrapping Query Suggestions in Spotify's Instant Search System
Authors: Alva Liu, Humberto Jesús Corona Pampín, Enrico Palumbo
Venue: SIGIR
Year: 2023

Instant search systems present results to the user at every keystroke. This type of search system works best when the query ambiguity is low, the catalog is limited, and users know what they are looking for. However, Spotify's catalog is large and diverse, leading some users to struggle when formulating search intents. Query suggestions can be a powerful tool that helps users to express intents and explore content from the long-tail of the catalog. In this paper, we explain how we introduce query suggestions in Spotify's instant search system--a system that connects hundreds of millions of users with billions of items in our audio catalog. Specifically, we describe how we: (1) generate query suggestions from instant search logs, which largely contains in-complete prefix queries that cannot be directly applied as suggestions; (2) experiment with the generated suggestions in a specific UI feature, Related Searches; and (3) develop new metrics to measure whether the feature helps users to express search intent and formulate exploratory queries.
Calibration Learning for Few-shot Novel Product Description
Authors: Zheng Liu, Mingjing Wu, Bo Peng, Yichao Liu, Qi Peng, Chong Zou
Venue: SIGIR
Year: 2023

In the field of E-commerce, the rapid introduction of new products poses challenges for product description generation. Traditional approaches rely on large labelled datasets, which are often unavailable for novel products with limited data. To address this issue, we propose a calibration learning approach for few-shot novel product description. Our method leverages a small amount of labelled data for calibration and utilizes the novel product's semantic representation as prompts to generate accurate and informative descriptions. We evaluate our approach on three large-scale e-commerce datasets of novel products and demonstrate its effectiveness in significantly improving the quality of generated product descriptions compared to existing methods, especially when only limited data is available. We also conduct the analysis to understand the impact of different modules on the performance.
GammaGL: A Multi-Backend Library for Graph Neural Networks
Authors: Yaoqi Liu, Cheng Yang, Tianyu Zhao, Hui Han, Siyuan Zhang, Jing Wu, Guangyu Zhou, Hai Huang, Hui Wang, Chuan Shi
Venue: SIGIR
Year: 2023

Graph Neural Networks (GNNs) have shown their superiority in modeling graph-structured data, and gained much attention over the last five years. Though traditional deep learning frameworks such as TensorFlow and PyTorch provide convenient tools for implementing neural network algorithms, they do not support the key operations of GNNs well, e.g., the message passing computation based on sparse matrices. To address this issue, GNN libraries such as PyG are proposed by introducing rich Application Programming Interfaces (APIs) specialized for GNNs. However, most current GNN libraries only support a specific deep learning framework as the backend, e.g., PyG is tied up with PyTorch. In practice, users usually need to combine GNNs with other neural network components, which may come from their co-workers or open-source codes with different deep-learning backends. Consequently, users have to be familiar with various GNN libraries, and rewrite their GNNs with corresponding APIs. To provide a more convenient user experience, we present Gamma Graph Library (GammaGL), a GNN library that supports multiple deep learning frameworks as backends. GammaGL uses a framework-agnostic design that allows users to easily switch between deep learning backends on top of existing components with a single line of code change. Following the tensor-centric design idea, GammaGL splits the graph data into several key tensors, and abstracts GNN computational processes (such as message passing and graph mini-batch operations) into a few key functions. We develop many efficient operators in GammaGL for acceleration. So far, GammaGL has provided more than 40 GNN examples that can be applied to a variety of downstream tasks. GammaGL also provides tools for heterogeneous graph neural networks and recommendations to facilitate research in related fields. We present the performance of models implemented by GammaGL and the time consumption of our optimized operators to show the efficiency. Our library is available at https://github.com/BUPT-GAMMA/GammaGL.