OpenRaiser/Intern-Atlas huggingface dataset:
Integrating Knowledge Graph Data with Large Language Models for Explainable Inference
Authors: Carlos Efrain Quintero Narvaez, Raúl Monroy
Venue: WSDM
Year: 2024

We propose a method to enable Large Language Models to access Knowledge Graph (KG) data and justify their text generation by showing the specific graph data the model accessed during inference. For this, we combine Language Models with methods from Neurosymbolic Artificial Intelligence designed to answer queries on Knowledge Graphs. This is done by modifying the model so that at different stages of inference it outputs an Existential Positive First-Order (EPFO) query, which is then processed by an additional query appendix. In turn, the query appendix uses neural link predictors along with description aware embeddings to resolve these queries. After that, the queries are logged and used as an explanation of the inference process of the complete model. Lastly, we train the model using a Linear Temporal Logic (LTL) constraint-based loss function to measure the consistency of the queries among each other and with the final model output.
A Linguistic Grounding-Infused Contrastive Learning Approach for Health Mention Classification on Social Media
Authors: Usman Naseem, Jinman Kim, Matloob Khushi, Adam G. Dunn
Venue: WSDM
Year: 2024

Social media users use disease and symptoms words in different ways, including describing their personal health experiences figuratively or in other general discussions. The health mention classification (HMC) task aims to separate how people use terms, which is important in public health applications. Existing HMC studies address this problem using pretrained language models (PLMs). However, the remaining gaps in the area include the need for linguistic grounding, the requirement for large volumes of labelled data, and that solutions are often only tested on Twitter or Reddit, which provides limited evidence of the transportability of models. To address these gaps, we propose a novel method that uses a transformer-based PLM to obtain a contextual representation of target (disease or symptom) terms coupled with a contrastive loss to establish a larger gap between target terms' literal and figurative uses using linguistic theories. We introduce the use of a simple and effective approach for harvesting candidate instances from the broad corpus and generalising the proposed method using self-training to address the label scarcity challenge. Our experiments on publicly available health-mention datasets from Twitter (HMC2019) and Reddit (RHMD) demonstrate that our method outperforms the state-of-the-art HMC methods on both datasets for the HMC task. We further analyse the transferability and generalisability of our method and conclude with a discussion on the empirical and ethical considerations of our study.
Gaussian Graphical Model-Based Clustering of Time Series Data
Authors: Kohei Obata
Venue: WSDM
Year: 2024

Time series subsequence clustering is a useful tool for recognizing dynamic changes and uncovering interesting patterns in time series, and it can also be applied to downstream tasks. In addition to clustering the data, the interpretability of the cluster is crucial when analyzing the data, particularly because we frequently lack information about each cluster. The Gaussian Graphical Model (GGM) provides a clear explanation of the cluster as the inverse Gaussian covariance matrix (i.e., network) of the GGM encodes the conditional independence structure. In this study, we aim to enhance our understanding of the data by achieving GGM-based clustering for time series data of various types and structures. Furthermore, our objective is to utilize GGM in downstream tasks, including missing value imputation and forecasting, by taking advantage of the relationships between variables shown in the network. We aim to answer the following research questions: RQ1 : How can we obtain interpretable clusters for tensor time series? RQ2 : How can we deal with time series containing missing values? RQ3 : How can we effectively forecast a time series stream?
Effective and Efficient Transformer Models for Sequential Recommendation
Authors: Aleksandr V. Petrov
Venue: WSDM
Year: 2024

Sequential Recommender Systems use the order of user-item interactions to predict the next item in the sequence. This task is similar to Language Modelling, where the goal is to predict the next token based on the sequence of past tokens. Therefore, adaptations of language models, and, in particular, Transformer-based models, achieved state-of-the-art results for a sequential recommendation. However, despite similarities, the sequential recommendation problem poses a number of specific challenges not present in Language Modelling. These challenges include the large catalogue size of real-world recommender systems, which increases GPU memory requirements and makes the training and the inference of recommender models slow. Another challenge is that a good recommender system should focus not only on the accuracy of recommendation but also on additional metrics, such as diversity and novelty, which makes the direct adaptation of language model training strategies problematic. Our research focuses on solving these challenges. In this doctoral consortium abstract, we briefly describe the motivation and background for our work and then pose research questions and discuss current progress towards solving the described problems.
RecJPQ: Training Large-Catalogue Sequential Recommenders
Authors: Aleksandr V. Petrov, Craig Macdonald
Venue: WSDM
Year: 2024

Sequential Recommendation is a popular recommendation task that uses the order of user-item interaction to model evolving users' interests and sequential patterns in their behaviour. Current state-of-the-art Transformer-based models for sequential recommendation, such as BERT4Rec, generate sequence embeddings and compute scores for catalogue items, but the increasing catalogue size makes training these models costly. The Joint Product Quantisation (JPQ) method, originally proposed for passage retrieval, markedly reduces the size of the retrieval index with minimal effect on model effectiveness, by replacing passage embeddings with a limited number of shared sub-embeddings. This paper introduces RecJPQ, a novel adaptation of JPQ for sequential recommendations, which replaces each item embedding with a concatenation of a limited number of shared sub-embeddings and, therefore, limits the number of learnable model parameters. The main idea of RecJPQ is to split items into sub-item entities before training the main recommendation model, which is inspired by splitting words into tokens and training tokenizers in language models. We apply RecJPQ to SASRec, BERT4Rec, and GRU models on three large-scale sequential datasets. Our results showed that RecJPQ could notably reduce the model size (e.g., 48× reduction for the Gowalla dataset with no effectiveness degradation). RecJPQ can also improve model performance through a regularization effect (e.g. +0.96% NDCG@10 improvement on the Booking.com dataset). Overall, RecJPQ allows the training of state-of-the-art transformer models in industrial applications, where datasets with millions of items are common.
Accelerating Pharmacovigilance using Large Language Models
Authors: Mukkamala Venkata Sai Prakash, Ganesh Parab, Meghana Veeramalla, Siddartha Reddy, Varun V, Saisubramaniam Gopalakrishnan, Vishal Pagidipally, Vishal Vaddina
Venue: WSDM
Year: 2024

Pharmacovigilance is the science and practice of monitoring, assessing, and preventing adverse effects or any other drug-related problems. Pharmacovigilance ensures the post-market safety of pharmaceuticals and plays a crucial role in public health by enhancing drug safety. This discipline involves collecting, analyzing, and reporting data on adverse events, allowing for informed regulatory decisions. Manual systems face challenges in handling data volume, potentially leading to oversight and delays. Automation with advanced technologies can be a practical solution to mitigate these challenges and ensure efficient data management. In this talk, we explain the potential application of pre-trained Large Language Models (LLMs) in pharmacovigilance. We begin with a comprehensive overview of the process, covering all the stages of the lifecycle. We emphasize the pivotal process of scrutinizing documents for relevant adverse effects and elucidate the measures that enhance their effectiveness. We delineate our strategy for generating informative summaries, with a specific emphasis on adverse effects and their antecedent occurrences. We emphasize the imperative requirement for factual accuracy validation via the implementation of a fact-checking mechanism. We demonstrate the framework's efficacy with a focus on output fidelity and summary informativeness. We provide quantifiable evidence of the time-saving benefits of our method compared to manual methods, advocating for the adoption of our framework in pharmacovigilance, and conclude by addressing potential refinements for increased efficiency.
Integrity 2024: Integrity in Social Networks and Media
Authors: Lluís Garcia Pueyo, Symeon Papadopoulos, Prathyusha Senthil Kumar, Aristides Gionis, Panayiotis Tsaparas, Vasilis Verroios, Giuseppe Manco, Anton Andryeyev, Stefano Cresci, Timos Sellis, Anthony McCosker
Venue: WSDM
Year: 2024

Integrity 2024 is the fifth edition of the Workshop on Integrity in Social Networks and Media, held in conjunction with the ACM Conference on Web Search and Data Mining (WSDM) since the 2020 edition [1-4]. The goal of the workshop is to bring together academic and industry researchers working on integrity, fairness, trust and safety in social networks to discuss the most pressing risks and cutting-edge technologies to reliably measure and mitigate them. The event consists of invited talks from academic experts and industry leaders as well as peer-reviewed papers and posters through an open call-for-papers.
Scaling Up LLM Reviews for Google Ads Content Moderation
Authors: Wei Qiao, Tushar Dogra, Otilia Stretcu, Yu-Han Lyu, Tiantian Fang, Dongjin Kwon, Chun-Ta Lu, Enming Luo, Yuan Wang, Chih-Chun Chia, Ariel Fuxman, Fangzhou Wang, Ranjay Krishna, Mehmet Tek
Venue: WSDM
Year: 2024

Large language models (LLMs) are powerful tools for content moderation, but their inference costs and latency make them prohibitive for casual use on large datasets, such as the Google Ads repository. This study proposes a method for scaling up LLM reviews for content moderation in Google Ads. First, we use heuristics to select candidates via filtering and duplicate removal, and create clusters of ads for which we select one representative ad per cluster. We then use LLMs to review only the representative ads. Finally, we propagate the LLM decisions for the representative ads back to their clusters. This method reduces the number of reviews by more than 3 orders of magnitude while achieving a 2x recall compared to a baseline non-LLM model. The success of this approach is a strong function of the representations used in clustering and label propagation; we found that cross-modal similarity representations yield better results than uni-modal representations.
Intent Contrastive Learning with Cross Subsequences for Sequential Recommendation
Authors: Xiuyuan Qin, Huanhuan Yuan, Pengpeng Zhao, Guanfeng Liu, Fuzhen Zhuang, Victor S. Sheng
Venue: WSDM
Year: 2024

The user purchase behaviors are mainly influenced by their intentions (e.g., buying clothes for decoration, buying brushes for painting, etc.). Modeling a user's latent intention can significantly improve the performance of recommendations. Previous works model users' intentions by considering the predefined label in auxiliary information or introducing stochastic data augmentation to learn purposes in the latent space. However, the auxiliary information is sparse and not always available for recommender systems, and introducing stochastic data augmentation may introduce noise and thus change the intentions hidden in the sequence. Therefore, leveraging user intentions for sequential recommendation (SR) can be challenging because they are frequently varied and unobserved. In this paper, Intent contrastive learning with Cross Subsequences for sequential Recommendation (ICSRec) is proposed to model users' latent intentions. Specifically, ICSRec first segments a user's sequential behaviors into multiple subsequences by using a dynamic sliding operation and takes these subsequences into the encoder to generate the representations for the user's intentions. To tackle the problem of no explicit labels for purposes, ICSRec assumes different subsequences with the same target item may represent the same intention and proposes a coarse-grain intent contrastive learning to push these subsequences closer. Then, fine-grain intent contrastive learning is mentioned to capture the fine-grain intentions of subsequences in sequential behaviors. Extensive experiments conducted on four real-world datasets demonstrate the superior performance of the proposed ICSRec model compared with baseline methods.
Budgeted Embedding Table For Recommender Systems
Authors: Yunke Qu, Tong Chen, Quoc Viet Hung Nguyen, Hongzhi Yin
Venue: WSDM
Year: 2024

At the heart of contemporary recommender systems (RSs) are latent factor models that provide quality recommendation experience to users. These models use embedding vectors, which are typically of a uniform and fixed size, to represent users and items. As the number of users and items continues to grow, this design becomes inefficient and hard to scale. Recent lightweight embedding methods have enabled different users and items to have diverse embedding sizes, but are commonly subject to two major drawbacks. Firstly, they limit the embedding size search to optimizing a heuristic balancing the recommendation quality and the memory complexity, where the trade-off coefficient needs to be manually tuned for every memory budget requested. The implicitly enforced memory complexity term can even fail to cap the parameter usage, making the resultant embedding table fail to meet the memory budget strictly. Secondly, most solutions, especially reinforcement learning based ones derive and optimize the embedding size for each each user/item on an instance-by-instance basis, which impedes the search efficiency. In this paper, we propose Budgeted Embedding Table (BET), a novel method that generates table-level actions (i.e., embedding sizes for all users and items) that is guaranteed to meet pre-specified memory budgets. Furthermore, by leveraging a set-based action formulation and engaging set representation learning, we present an innovative action search strategy powered by an action fitness predictor that efficiently evaluates each table-level action. Experiments have shown state-of-the-art performance on two real-world datasets when BET is paired with three popular recommender models under different memory budgets.
Customer Understanding for Recommender Systems
Authors: Md. Mostafizur Rahman, Yu Hirate
Venue: WSDM
Year: 2024

Recommender systems are powerful tools for enhancing customer engagement and driving sales for Rakuten businesses. However, to achieve their full potential, these systems must possess a profound understanding of customer behaviors. This understanding can be gained from a variety of sources, including customer purchase history, customer feedback, and customer behavioral patterns. One of the most important aspects of customer understanding is the ability to identify lookalike customers, understand their behavioral patterns, and predict lifestyles, for example, whether a customer is married or unmarried, owns a car or plays golf, etc. Rakuten provides more than 70 different services and heavily relies on recommendations for many of its products. In our platforms, we can observe groups of customers who share similar interests, needs, or behaviors often end up being attracted to similar products or services. Customer preferences can change over time, so it is important for recommender systems to adapt those changes. This can be achieved by tracking customer behavior, static or dynamic environment changes around targeted customers, and their feedback. We utilize various graph and deep learning based models to address the customer understanding problem. The objective of this talk is to offer a comprehensive overview of customer understanding and modeling for complex recommender systems, leading to increased customer satisfaction, loyalty, and sales. We also present various empirical results on both Rakuten data as well as public benchmark datasets, providing evidence of the benefits of customer understanding on various downstream tasks. We also share insights to characterize the circumstances in which the graph-based and deep learning-based models offer the most significant improvements. We conclude this talk by criticizing the current models and discussing possible future developments and improvements.
"Maya"- A Conversational Shopping Assistant for Fashion at Myntra
Authors: Akhil Raj, Hrishikesh Ganu, Saikat Kumar Das, R. Sandeep, Satyajeet Singh, Sreekanth Vempati
Venue: WSDM
Year: 2024

Fashion Ecommerce is challenging because users have intents that could be very broad Eg: ''I want a dress to wear for my cousin's wedding''. Traditional single-shot retrieval mechanisms like search don't work very well for such intents. To solve this problem we developed ''Maya'', a conversational shopping assistant based on LLMs. Maya has been deployed in production and is available to select cohort of users at scale who come to our platform. It is arguably, one of the earliest production deployments of GenAI Fashion assistants at this scale. We describe the journey of building Maya and lessons post the production deployment.
CharmBana: Progressive Responses with Real-Time Internet Search for Knowledge-Powered Conversations
Authors: Revanth Gangi Reddy, Sharath Chandra Etagi Suresh, Hao Bai, Wentao Yao, Mankeerat Sidhu, Karan Aggarwal, Prathamesh Sonawane, ChengXiang Zhai
Venue: WSDM
Year: 2024

Chatbots are often hindered by the latency associated with integrating real-time web search results, compromising user experience. To overcome this, we present CharmBana, an innovative social chatbot that introduces the use of progressive response generation to effortlessly blend search results into the bot's responses, while ensuring low response latency. The use of progressive responses is especially beneficial for voice-based chatbots, where the preliminary response buys time for a detailed follow-up, ensuring a smooth user interaction. As a result, our method not only cuts down user waiting times by 50% but also generates more relevant, precise, and engaging search inquiries. When tested in the Alexa Prize Socialbot Grand Challenge 5, our chatbot employing progressive responses consistently received higher user ratings.
Future Timelines: Extraction and Visualization of Future-related Content From News Articles
Authors: Juwal Regev, Adam Jatowt, Michael Färber
Venue: WSDM
Year: 2024

In today's rapidly evolving world, maintaining a comprehensive overview of the future landscape is essential for staying competitive and making informed decisions. However, given the large volume of daily news, manually obtaining a thorough overview of an entity's future prospects is quite challenging. To address this, we present a system designed to automatically extract and summarize future-related information of a queried entity from news articles. Our approach utilizes a novel and publicly accessible multi-source dataset comprising 6,800 annotated sentences to fine-tune a language model to identify future-related sentences. We then use topic modeling to extract the main topics from the data and rank them by relevance as well as present them on an interactive timeline. User evaluations have shown that the timelines and summaries our system produces are useful. The system is available as a web application at: https://chronicle2050.regevson.com.
SSLRec: A Self-Supervised Learning Framework for Recommendation
Authors: Xubin Ren, Lianghao Xia, Yuhao Yang, Wei Wei, Tianle Wang, Xuheng Cai, Chao Huang
Venue: WSDM
Year: 2024

Self-supervised learning (SSL) has gained significant interest in recent years as a solution to address the challenges posed by sparse and noisy data in recommender systems. Despite the growing number of SSL algorithms designed to provide state-of-the-art performance in various recommendation scenarios (e.g., graph collaborative filtering, sequential recommendation, social recommendation, KG-enhanced recommendation), there is still a lack of unified frameworks that integrate recommendation algorithms across different domains. Such a framework could serve as the cornerstone for self-supervised recommendation algorithms, unifying the validation of existing methods and driving the design of new ones. To address this gap, we introduce SSLRec, a novel benchmark platform that provides a standardized, flexible, and comprehensive framework for evaluating various SSL-enhanced recommenders. The SSLRec framework features a modular architecture that allows users to easily evaluate state-of-the-art models and a complete set of data augmentation and self-supervised toolkits to help create SSL recommendation models with specific needs. Furthermore, SSLRec simplifies the process of training and evaluating different recommendation models with consistent and fair settings. Our SSLRec platform covers a comprehensive set of state-of-the-art SSL-enhanced recommendation models across different scenarios, enabling researchers to evaluate these cutting-edge models and drive further innovation in the field. Our implemented SSLRec framework is available at the source code repository https://github.com/HKUDS/SSLRec.
Preserving Heritage: Developing a Translation Tool for Indigenous Dialects
Authors: Melissa Robles, Cristian A. Martínez, Juan Camilo Prieto, Sara Palacios, Rubén Manrique
Venue: WSDM
Year: 2024

The preservation and understanding of indigenous languages emerge as crucial, given their substantial contribution to the cultural and linguistic heritage of communities. Despite their undeniable value, these languages are threatened by extinction due to a dwindling number of native speakers and the predominance of oral traditions over written forms. In this context, this study aims to contribute to the conservation of these languages through the development of a Spanish-indigenous language translator. This research employs neural machine translation technology, investigating three distinct approaches: a translation model based on transformers, finetuning with a Finnish translator, and finetuning with a multilingual translator. The results obtained from these methodologies are promising, demonstrating competitive viability when compared to the limited existing research in this field of study.