OpenRaiser/Intern-Atlas huggingface dataset:
WSDM 2023 Workshop on Interactive Recommender Systems
Authors: Yong Liu, Hao Zhang, Zhu Sun, Shoujin Wang, Jie Zhang
Venue: WSDM
Year: 2023

Interactive recommender systems have attracted increasingly research attentions from both academia and industry. This workshop is a half-day event, which provides a forum for researchers and practitioners to discuss recent research progress and novel research directions about interactive recommender systems. The program will include two keynotes and 6 to 8 research paper presentations. The objective of this workshop is to consolidate the recent technical progresses about interactive recommendation, which will be a promising research and development direction for future recommendation technologies. This workshop will attract the attention of researchers from both academia and industry. It aligns with WSDM's spirit of promoting the collaborations between academia and industry.
Graph Sequential Neural ODE Process for Link Prediction on Dynamic and Sparse Graphs
Authors: Linhao Luo, Gholamreza Haffari, Shirui Pan
Venue: WSDM
Year: 2023

Link prediction on dynamic graphs is an important task in graph mining. Existing approaches based on dynamic graph neural networks (DGNNs) typically require a significant amount of historical data (interactions over time), which is not always available in practice. The missing links over time, which is a common phenomenon in graph data, further aggravates the issue and thus creates extremely sparse and dynamic graphs. To address this problem, we propose a novel method based on the neural process, called Graph Sequential Neural ODE Process (GSNOP). Specifically, GSNOP combines the advantage of the neural process and neural ordinary differential equation that models the link prediction on dynamic graphs as a dynamic-changing stochastic process. By defining a distribution over functions, GSNOP introduces the uncertainty into the predictions, making it generalize to more situations instead of overfitting to the sparse data. GSNOP is also agnostic to model structures that can be integrated with any DGNN to consider the chronological and geometrical information for link prediction. Extensive experiments on three dynamic graph datasets show that GSNOP can significantly improve the performance of existing DGNNs and outperform other neural process variants.
Model-based Unbiased Learning to Rank
Authors: Dan Luo, Lixin Zou, Qingyao Ai, Zhiyu Chen, Dawei Yin, Brian D. Davison
Venue: WSDM
Year: 2023

Unbiased Learning to Rank(ULTR), i.e., learning to rank documents with biased user feedback data, is a well-known challenge in information retrieval. Existing methods in unbiased learning to rank typically rely on click modeling or inverse propensity weighting(IPW). Unfortunately, search engines face the issue of a severe long-tail query distribution, which neither click modeling nor IPW handles well. Click modeling usually requires that the same query-document pair appears multiple times for reliable inference, which makes it fall short for tail queries; IPW suffers from high variance since it is highly sensitive to small propensity score values. Therefore, a general debiasing framework that works well under tail queries is sorely needed. To address this problem, we propose a model-based unbiased learning-to-rank framework. Specifically, we develop a general context-aware user simulator to generate pseudo clicks for unobserved ranked lists to train rankers, which addresses the data sparsity problem. In addition, considering the discrepancy between pseudo clicks and actual clicks, we take the observation of a ranked list as the treatment variable and further incorporate inverse propensity weighting with pseudo labels in a doubly robust way. The derived bias and variance indicate that the proposed model-based method is more robust than existing methods. Extensive experiments on benchmark datasets, including simulated datasets and real click logs, demonstrate that the proposed model-based method consistently outperforms state-of-the-art methods in various scenarios. The code is available at https://github.com/rowedenny/MULTR.
Graph Summarization via Node Grouping: A Spectral Algorithm
Authors: Arpit Merchant, Michael Mathioudakis, Yanhao Wang
Venue: WSDM
Year: 2023

Graph summarization via node grouping is a popular method to build concise graph representations by grouping nodes from the original graph into supernodes and encoding edges into superedges such that the loss of adjacency information is minimized. Such summaries have immense applications in large-scale graph analytics due to their small size and high query processing efficiency. In this paper, we reformulate the loss minimization problem for summarization into an equivalent integer maximization problem. By initially allowing relaxed (fractional) solutions for integer maximization, we analytically expose the underlying connections to the spectral properties of the adjacency matrix. Consequently, we design an algorithm called SpecSumm that consists of two phases. In the first phase, motivated by spectral graph theory, we apply k-means clustering on the k largest (in magnitude) eigenvectors of the adjacency matrix to assign nodes to supernodes. In the second phase, we propose a greedy heuristic that updates the initial assignment to further improve summary quality. Finally, via extensive experiments on 11 datasets, we show that SpecSumm efficiently produces high-quality summaries compared to state-of-the-art summarization algorithms and scales to graphs with millions of nodes.
Learning to Infer Product Attribute Values From Descriptive Texts and Images
Authors: Pablo Montalvo, Aghiles Salah
Venue: WSDM
Year: 2023

Online marketplaces are able to offer a staggering array of products that no physical store can match. While this makes it more likely for customers to find what they want, in order for online providers to ensure a smooth and efficient user experience, they must maintain well-organized catalogs, which depends greatly on the availability of per-product attribute values such as color, material, brand, to name a few. Unfortunately, such information is often incomplete or even missing in practice, and therefore we have to resort to predictive models as well as other sources of information to impute missing attribute values. In this talk we present the deep learning-based approach that we have developed at Rakuten Group to extract attribute values from product descriptive texts and images. Starting from pretrained architectures to encode textual and visual modalities, we discuss several refinements and improvements that we find necessary to achieve satisfactory performance and meet strict business requirements, namely improving recall while maintaining a high precision (>= 95%). Our methodology is driven by a systematic investigation into several practical research questions surrounding multimodality, which we revisit in this talk. At the heart of our multimodal architecture, is a new method to combine modalities inspired by empirical cross-modality comparisons. We present the latter component in details, point out one of its major limitations, namely exacerbating the issue of modality collapse, i.e., when the model forgets one modality, and describe our mitigation to this problem based on a principled regularization scheme. We present various empirical results on both Rakuten data as well as public benchmark datasets, which provide evidence of the benefits of our approach compared to several strong baselines. We also share some insights to characterise the circumstances in which the proposed model offers the most significant improvements. We conclude this talk by criticising the current model and discussing possible future developments and improvements. Our model is successfully deployed in Rakuten Ichiba - a Rakuten marketplace - and we believe that our investigation into multimodal attribute value extraction for e-commerce will benefit other researchers and practitioners alike embarking on similar journeys.
Next-generation Challenges of Responsible Data Integration
Authors: Fatemeh Nargesian, Abolfazl Asudeh, H. V. Jagadish
Venue: WSDM
Year: 2023

Data integration has been extensively studied by the data management community and is a core task in the data pre-processing step of ML pipelines. When the integrated data is used for analysis and model training, responsible data science requires addressing concerns about data quality and bias. We present a tutorial on data integration and responsibility, highlighting the existing efforts in responsible data integration along with research opportunities and challenges. In this tutorial, we encourage the community to audit data integration tasks with responsibility measures and develop integration techniques that optimize the requirements of responsible data science. We focus on three critical aspects: (1) the requirements to be considered for evaluating and auditing data integration tasks for quality and bias; (2) the data integration tasks that elicit attention to data responsibility measures and methods to satisfy these requirements; and, (3) techniques, tasks, and open problems in data integration that help achieve data responsibility.
A Multimodal Framework for the Identification of Vaccine Critical Memes on Twitter
Authors: Usman Naseem, Jinman Kim, Matloob Khushi, Adam G. Dunn
Venue: WSDM
Year: 2023

Memes can be a useful way to spread information because they are funny, easy to share, and can spread quickly and reach further than other forms. With increased interest in COVID-19 vaccines, vaccination-related memes have grown in number and reach. Memes analysis can be difficult because they use sarcasm and often require contextual understanding. Previous research has shown promising results but could be improved by capturing global and local representations within memes to model contextual information. Further, the limited public availability of annotated vaccine critical memes datasets limit our ability to design computational methods to help design targeted interventions and boost vaccine uptake. To address these gaps, we present VaxMeme, which consists of 10,244 manually labelled memes. With VaxMeme, we propose a new multimodal framework designed to improve the memes' representation by learning the global and local representations of memes. The improved memes' representations are then fed to an attentive representation learning module to capture contextual information for classification using an optimised loss function. Experimental results show that our framework outperformed state-of-the-art methods with an F1-Score of 84.2%. We further analyse the transferability and generalisability of our framework and show that understanding both modalities is important to identify vaccine critical memes on Twitter. Finally, we discuss how understanding memes can be useful in designing shareable vaccination promotion, myth debunking memes and monitoring their uptake on social media platforms.
Considerations for Ethical Speech Recognition Datasets
Authors: Orestis Papakyriakopoulos, Alice Xiang
Venue: WSDM
Year: 2023

Speech AI Technologies are largely trained on publicly available datasets or by the massive web-crawling of speech. In both cases, data acquisition focuses on minimizing collection effort, without necessarily taking the data subjects' protection or user needs into consideration. This results to models that are not robust when used on users who deviate from the dominant demographics in the training set, discriminating individuals having different dialects, accents, speaking styles, and disfluencies. In this talk, we use automatic speech recognition as a case study and examine the properties that ethical speech datasets should possess towards responsible AI applications. We showcase diversity issues, inclusion practices, and necessary considerations that can improve trained models, while facilitating model explainability and protecting users and data subjects. We argue for the legal & privacy protection of data subjects, targeted data sampling corresponding to user demographics & needs, appropriate meta data that ensure explainability & accountability in cases of model failure, and the sociotechnical & situated model design. We hope this talk can inspire researchers & practitioners to design and use more human-centric datasets in speech technologies and other domains, in ways that empower and respect users, while improving machine learning models' robustness and utility.
Never Too Late to Learn: Regularizing Gender Bias in Coreference Resolution
Authors: Sunyoung Park, Kyuri Choi, Haeun Yu, Youngjoong Ko
Venue: WSDM
Year: 2023

Leveraging pre-trained language models (PLMs) as initializers for efficient transfer learning has become a universal approach for text-related tasks. However, the models not only learn the language understanding abilities but also reproduce prejudices for certain groups in the datasets used for pre-training. Recent studies show that the biased knowledge acquired from the datasets affects the model predictions on downstream tasks. In this paper, we mitigate and analyze the gender biases in PLMs with coreference resolution, which is one of the natural language understanding (NLU) tasks. PLMs exhibit two types of gender biases: stereotype and skew. The primary causes for the biases are the imbalanced datasets with more male examples and the stereotypical examples on gender roles. While previous studies mainly focused on the skew problem, we aim to mitigate both gender biases in PLMs while maintaining the model's original linguistic capabilities. Our method employs two regularization terms, Stereotype Neutralization (SN) and Elastic Weight Consolidation (EWC). The models trained with the methods show to be neutralized and reduce the biases significantly on the WinoBias dataset compared to the public BERT. We also invented a new gender bias quantification metric called the Stereotype Quantification (SQ) score. In addition to the metrics, embedding visualizations were used to interpret how our methods have successfully debiased the models.
Learning Stance Embeddings from Signed Social Graphs
Authors: John Pougué-Biyong, Akshay Gupta, Aria Haghighi, Ahmed El-Kishky
Venue: WSDM
Year: 2023

A challenge in social network analysis, is understanding the position, or stance, of people on a large set of topics. While past work has modeled (dis)agreement in social networks using signed graphs, these approaches have not modeled agreement patterns across a range of correlated topics. For instance, disagreement on one topic may make disagreement (or agreement) more likely for related topics. Recognizing topics influence agreement and disagreement, we propose the Stance Embeddings Model (SEM), which jointly learns embeddings for each user and topic in signed social graphs with distinct edge types for each topic. By jointly learning user and topic embeddings, SEM can perform cold-start topic stance detection, predicting the stance of a user on topics for which we have not observed their engagement. We demonstrate the effectiveness of SEM using two large-scale Twitter signed graph datasets that we open-source. One dataset, TwitterSG, labels (dis)agreements using engagements between users via tweets to derive topic-informed, signed edges. The other, BirdwatchSG, leverages community reports on misinformation and misleading content. On TwitterSG and BirdwatchSG, SEM shows a 39% and 26% error reduction respectively against strong topic-agnostic baselines.
Integrity 2023: Integrity in Social Networks and Media
Authors: Lluís Garcia Pueyo, Panayiotis Tsaparas, Prathyusha Senthil Kumar, Timos Sellis, Paolo Papotti, Sibel Adali, Giuseppe Manco, Tudor Trufinescu, Gireeja Ranade, James R. Verbus, Mehmet N. Tek, Anthony McCosker
Venue: WSDM
Year: 2023

Integrity 2023 is the fourth edition of the successful Workshop on Integrity in Social Networks and Media, held in conjunction with the ACM Conference on Web Search and Data Mining (WSDM) in the past three years. The goal of the workshop is to bring together researchers and practitioners to discuss content and interaction integrity challenges in social networks and social media platforms. The event consists of a combination of invited talks by reputed members of the Integrity community from both academia and industry and peer-reviewed contributed talks and posters solicited via an open call-for-papers.
Robust Training of Graph Neural Networks via Noise Governance
Authors: Siyi Qian, Haochao Ying, Renjun Hu, Jingbo Zhou, Jintai Chen, Danny Z. Chen, Jian Wu
Venue: WSDM
Year: 2023

Graph Neural Networks (GNNs) have become widely-used models for semi-supervised learning. However, the robustness of GNNs in the presence of label noise remains a largely under-explored problem. In this paper, we consider an important yet challenging scenario where labels on nodes of graphs are not only noisy but also scarce. In this scenario, the performance of GNNs is prone to degrade due to label noise propagation and insufficient learning. To address these issues, we propose a novel RTGNN (Robust Training of Graph Neural Networks via Noise Governance) framework that achieves better robustness by learning to explicitly govern label noise. More specifically, we introduce self-reinforcement and consistency regularization as supplemental supervision. The self-reinforcement supervision is inspired by the memorization effects of deep neural networks and aims to correct noisy labels. Further, the consistency regularization prevents GNNs from overfitting to noisy labels via mimicry loss in both the inter-view and intra-view perspectives. To leverage such supervisions, we divide labels into clean and noisy types, rectify inaccurate labels, and further generate pseudo-labels on unlabeled nodes. Supervision for nodes with different types of labels is then chosen adaptively. This enables sufficient learning from clean labels while limiting the impact of noisy ones. We conduct extensive experiments to evaluate the effectiveness of our RTGNN framework, and the results validate its consistent superior performance over state-of-the-art methods with two types of label noises and various noise rates.
DisenPOI: Disentangling Sequential and Geographical Influence for Point-of-Interest Recommendation
Authors: Yifang Qin, Yifan Wang, Fang Sun, Wei Ju, Xuyang Hou, Zhe Wang, Jia Cheng, Jun Lei, Ming Zhang
Venue: WSDM
Year: 2023

Point-of-Interest (POI) recommendation plays a vital role in various location-aware services. It has been observed that POI recommendation is driven by both sequential and geographical influences. However, since there is no annotated label of the dominant influence during recommendation, existing methods tend to entangle these two influences, which may lead to sub-optimal recommendation performance and poor interpretability. In this paper, we address the above challenge by proposing DisenPOI, a novel Disentangled dual-graph framework for POI recommendation, which jointly utilizes sequential and geographical relationships on two separate graphs and disentangles the two influences with self-supervision. The key novelty of our model compared with existing approaches is to extract disentangled representations of both sequential and geographical influences with contrastive learning. To be specific, we construct a geographical graph and a sequential graph based on the check-in sequence of a user. We tailor their propagation schemes to become sequence-/geo-aware to better capture the corresponding influences. Preference proxies are extracted from check-in sequence as pseudo labels for the two influences, which supervise the disentanglement via a contrastive loss. Extensive experiments on three datasets demonstrate the superiority of the proposed model.
Beyond-Accuracy Goals, Again
Authors: Maarten de Rijke
Venue: WSDM
Year: 2023

Improving the performance of information retrieval systems tends to be narrowly scoped. Often, better prediction performance is considered the only metric of improvement. As a result, work on improving information retrieval methods usually focuses on im- proving the methods' accuracy. Such a focus is myopic. Instead, as researchers and practitioners we should adopt a richer perspective measuring the performance of information retrieval systems. I am not the first to make this point (see, e.g., [4]), but I want to highlight dimensions that broaden the scope considered so far and offer a number of examples to illustrate what this would mean for our research agendas. First, trustworthiness is a prerequisite for people, organizations, and societies to use AI-based, and, especially, machine learning- based systems in general, and information retrieval systems in particular. Trust can be gained in an intrinsic manner by revealing the inner workings of an AI-based system, i.e., through explainability. Or it can be gained extrinsically by showing, in a principled or empirical manner, that a system upholds verifiable guarantees. Such guarantees should obtained for the following dimensions (at a minimum): (i) accuracy, including well-defined and explained contexts of usage; (ii) reliability, including exhibiting parity with respect to sensitive attributes; (iii) repeatable and reproducible results, including audit trails; (iv) resilience to adversarial examples, distributional shifts; and (v) safety, including privacy-preserving search and recommendation. Second, in information retrieval, our experiments are mostly conducted in controlled laboratory environments. Extrapolating this information to evaluate the real-world effects often remains a challenge. This is particularly true when measuring the impact of information retrieval systems across broader scales, both temporally and spatially. Conducting controlled experimental trials for evaluating real-world impacts of information retrieval systems can result in depicting a snapshot situation, where systems are tailored towards that specific environment. As society is constantly changing, the requirements set for information retrieval systems are changing as well, resulting in short-term and long-term feedback loops with interactions between society and information retrieval systems.
Self-supervision and Controlling Techniques to Improve Counter Speech Generation
Authors: Punyajoy Saha
Venue: WSDM
Year: 2023

Hate speech is a challenging problem in today's online social media. One of the current solutions followed by different social media platforms is detecting hate speech using human-in-the-loop approaches. After detection, they moderate such hate speech by deleting the posts or suspending the users. While this approach can be a short-term solution for reducing the spread of hate, many researchers argue that it stifles freedom of expression. An alternate strategy that does not hamper freedom of expression is counterspeech. Recently, many studies have tried to create generation models to assist counter speakers by providing counterspeech suggestions for combating the explosive proliferation of online hate. This pipeline has two major challenges 1) How to improve the performance of generation without a large-scale dataset since building the dataset is costly 2) How to add control in the counter speech generation to make it more personalized. In this paper, we present our published and proposed research aimed at solving these two challenges.