OpenRaiser/Intern-Atlas huggingface dataset:
Toward User Engagement Optimization in 2D Presentation
Authors: Liang Wu, Mihajlo Grbovic, Jundong Li
Venue: WSDM
Year: 2021

Given a collection of items to display, such as news, videos, or products, how can we optimize their presentation order to maximize user engagements, such as click-through rate, viewing time, and the number of purchases? The problem becomes more complicated when the items are displayed in a grid-based, 2-dimensional presentation on a widescreen. For example, many E-Commerce websites such as Amazon and Etsy are displaying their products in a grid-like format, and so are streaming services like Youtube and Netflix. Unlike 1-dimensional space, where products can be naturally ranked in a vertical order, the presentation in 2-dimensional space poses a novel challenge about how to find the best presentation order - should we put the best listing on the top left corner, or the central position on the second row? We are aware that many traditional methods can be applied to solve the problem, such as conducting an attention heatmap web test, or a randomization experiment by shuffling positions of listings. However, both tests are costly to perform and they may downgrade the quality of users' search experience. By contrast, we focus on utilizing existing search log data to reveal propensity of positions, which is readily available and ubiquitously abundant. In a nutshell, the study presents how we find an optimal way of presentation in a grid-based environment - more relevant content should be placed in a more noticeable position. The position noticeability is further quantified to help ranking models better understand the signal of relevance manifested in user feedbacks. Our investigation paves the way for an end-to-end item presentation framework that learns the optimal layout for optimizing user engagements. Experimental results based on real-world data show the superiority of the proposed approach over state-of-the-art methods.
Data-Sharing Economy: Value-Addition from Data meets Privacy
Authors: Piyush Bagad, Subrata Mitra, Sunny Dhamnani, Atanu R. Sinha, Raunak Gautam, Haresh Khanna
Venue: WSDM
Year: 2021

The need for improved segmentation, targeting, personalization fuel the practice of data sharing among companies. Concurrently, data sharing faces the headwind of new laws emphasizing users' privacy in data. Under the premise that sharing of data occurs from a provider to a recipient, we propose a practicable approach of generating representational data for sharing that achieves value-addition for the recipient's tasks while preserving privacy of users. Prior art shows that the mechanism to improve value-addition inevitably weakens privacy in the generated data. In a first of a kind contribution, our system offers tunable controls to adjust the extent of privacy desired by the provider and the extent of value-addition expected by the recipient. Our experiments on a public data show that under common organizational practice of data-sharing, data generation for value-addition is achievable while preserving privacy. Our demonstration starkly shows the trade-off between privacy-protection and value addition, through user-controlled knobs and offers a prototype of a platform for data sharing which is mindful of this trade-off.
Enhancing Neural Recommender Models through Domain-Specific Concordance
Authors: Ananth Balashankar, Alex Beutel, Lakshminarayanan Subramanian
Venue: WSDM
Year: 2021

Recommender models trained on historical observational data alone can be brittle when domain experts subject them to counterfactual evaluation. In many domains, experts can articulate common, high-level mappings or rules between categories of inputs (user's history) and categories of outputs (preferred recommendations). One challenge is to determine how to train recommender models to adhere to these rules. In this work, we introduce the goal of domain-specific concordance: the expectation that a recommender model follow a set of expert-defined categorical rules. We propose a regularization-based approach that optimizes for robustness on rule-based input perturbations. To test the effectiveness of this method, we apply it in a medication recommender model over diagnosis-medicine categories, and in movie and music recommender models, on rules over categories based on movie tags and song genres. We demonstrate that we can increase the category-based robustness distance by up to 126% without degrading accuracy, but rather increasing it by up to 12% compared to baseline models in the popular MIMIC-III, MovieLens-20M and Last.fm Million Song datasets.
GPR: Global Personalized Restaurant Recommender System Leveraging Billions of Financial Transactions
Authors: Mangesh Bendre, Mahashweta Das, Fei Wang, Hao Yang
Venue: WSDM
Year: 2021

In this paper, we demonstrate our Global Personalized Recommender (GPR) system for restaurants. GPR does not use any explicit reviews, ratings, or domain-specific metadata but rather leverages over 3 billion anonymized payment transactions to learn user and restaurant behavior patterns. The design and development of GPR have been challenging, primarily due to the scale and skew of the data. Our system supports over 450M cardholders from over 200 countries and 2.5M restaurants in over 35K cities worldwide, respectively. Additionally, GPR being a global recommender system, needs to account for the regional variations in people's food choices and habits. We address the challenges by combining three different recommendation algorithms instead of using a single revolutionary model in the backend. The individual recommendation models are scalable and adapt to varying data skew challenges to ensure high-quality personalized recommendations for any user anywhere in the world.
Advances in Bias-aware Recommendation on the Web
Authors: Ludovico Boratto, Mirko Marras
Venue: WSDM
Year: 2021

The goal of this tutorial is to provide the WSDM community with recent advances on the assessment and mitigation of data and algorithmic bias in recommender systems. We first introduce conceptual foundations, by presenting the state of the art and describing real-world examples of how bias can impact on recommendation algorithms from several perspectives (e.g., ethical and system objectives). The tutorial continues with a systematic showcase of algorithmic countermeasures to uncover, assess, and reduce bias along the recommendation design process. A practical part then provides attendees with implementations of pre-, in-, and post-processing bias mitigation algorithms, leveraging open-source tools and public datasets; in this part, tutorial participants are engaged in the design of bias countermeasures and in articulating impacts on stakeholders. We conclude the tutorial by analyzing emerging open issues and future directions in this rapidly evolving research area (Website: https://biasinrecsys.github.io/wsdm2021).
F-FADE: Frequency Factorization for Anomaly Detection in Edge Streams
Authors: Yen-Yu Chang, Pan Li, Rok Sosic, M. H. Afifi, Marco Schweighauser, Jure Leskovec
Venue: WSDM
Year: 2021

Edge streams are commonly used to capture interactions in dynamic networks, such as email, social, or computer networks. The problem of detecting anomalies or rare events in edge streams has a wide range of applications. However, it presents many challenges due to lack of labels, a highly dynamic nature of interactions, and the entanglement of temporal and structural changes in the network. Current methods are limited in their ability to address the above challenges and to efficiently process a large number of interactions. Here, we propose F-FADE, a new approach for detection of anomalies in edge streams, which uses a novel frequency-factorization technique to efficiently model the time-evolving distributions of frequencies of interactions between node-pairs. The anomalies are then determined based on the likelihood of the observed frequency of each incoming interaction. F-FADE is able to handle in an online streaming setting a broad variety of anomalies with temporal and structural changes, while requiring only constant memory. Our experiments on one synthetic and six real-world dynamic networks show that F-FADE achieves state of the art performance and may detect anomalies that previous methods are unable to find.
An Efficient and Effective Framework for Session-based Social Recommendation
Authors: Tianwen Chen, Raymond Chi-Wing Wong
Venue: WSDM
Year: 2021

In many applications of session-based recommendation, social networks are usually available. Since users' interests are influenced by their friends, recommender systems can leverage social networks to better understand their users' preferences and thus provide more accurate recommendations. However, existing methods for session-based social recommendation are not efficient. To predict the next item of a user's ongoing session, the methods need to process many additional sessions of the user's friends to capture social influences, while non-social-aware methods (i.e., those without using social networks) only need to process one single session. To solve the efficiency issue, we propose an efficient framework for session-based social recommendation. In the framework, first, a heterogeneous graph neural network is used to learn user and item representations that integrate the knowledge from social networks. Then, to generate predictions, only the user and item representations relevant to the current session are passed to a non-social-aware model. During inference, since the user and item representations can be precomputed, the overall model runs as fast as the original non-social-aware model, while it can achieve better performance by leveraging the knowledge from social networks. Apart from being efficient, our framework has two additional advantages. First, the framework is flexible because it is compatible with any existing non-social-aware models and can easily incorporate more knowledge other than social networks. Second, our framework can capture cross-session item transitions while existing methods can only capture intra-session item transitions. Extensive experiments conducted on three public datasets demonstrate the effectiveness and efficiency of the proposed framework. Our code is available at https://github.com/twchen/SEFrame.
Beyond Point Estimate: Inferring Ensemble Prediction Variation from Neuron Activation Strength in Recommender Systems
Authors: Zhe Chen, Yuyan Wang, Dong Lin, Derek Zhiyuan Cheng, Lichan Hong, Ed H. Chi, Claire Cui
Venue: WSDM
Year: 2021

Despite deep neural network (DNN)'s impressive prediction performance in various domains, it is well known now that a set of DNN models trained with the same model specification and the exact same training data could produce very different prediction results. People have relied on the state-of-the-art ensemble method to estimate prediction uncertainty. However, ensembles are expensive to train and serve for web-scale traffic systems. In this paper, we seek to advance the understanding of prediction variation estimated by the ensemble method. Through empirical experiments on two widely used benchmark datasets Movielens and Criteo in recommender systems, we observe that prediction variations come from various randomness sources, including training data shuffling, and random initialization. When we add more randomness sources to ensemble members, we see higher prediction variations among these ensemble members, and more accurate mean prediction. Moreover, we propose to infer prediction variation from neuron activation strength and demonstrate its strong prediction power. Our approach provides a simple way for prediction variation estimation, and opens up new opportunities for future work in many interesting areas (e.g., model-based reinforcement learning) without relying on serving expensive ensemble models.
Long-Term Effect Estimation with Surrogate Representation
Authors: Lu Cheng, Ruocheng Guo, Huan Liu
Venue: WSDM
Year: 2021

There are many scenarios where short- and long-term causal effects of an intervention are different. For example, low-quality ads may increase short-term ad clicks but decrease the long-term revenue via reduced clicks. This work, therefore, studies the the problem of long-term effect where the outcome of primary interest, orprimary outcome, takes months or even years to accumulate. The observational study of long-term effect presents unique challenges. First, the confounding bias causes large estimation error and variance, which can further accumulate towards the prediction of primary outcomes. Second, short-term outcomes are often directly used as the proxy of the primary outcome, i.e., thesurrogate. Nevertheless, this method entails the strong surrogacy assumption that is often impractical. To tackle these challenges, we propose to build connections between long-term causal inference and sequential models in machine learning. This enables us to learnsurrogate representations that account for thetemporal unconfoundedness and circumvent the stringent surrogacy assumption by conditioning on the inferred time-varying confounders. Experimental results show that the proposed framework outperforms the state-of-the-art.
WoMG: A Library for Word-of-Mouth Cascades Generation
Authors: Federico Cinus, Francesco Bonchi, Corrado Monti, André Panisson
Venue: WSDM
Year: 2021

Studying information propagation in social media is an important task with plenty of applications for business and science. Generating realistic synthetic information cascades can help the research community in developing new methods and applications, testing sociological hypotheses and different what-if scenarios by simply changing few parameters. We demonstrate womg, a synthetic data generator which combines topic modeling and a topic-aware propagation model to create realistic information-rich cascades, whose shape depends on many factors, including the topic of the item and its virality, the homophily of the social network, the interests of its users and their social influence.
Say No to the Discrimination: Learning Fair Graph Neural Networks with Limited Sensitive Attribute Information
Authors: Enyan Dai, Suhang Wang
Venue: WSDM
Year: 2021

Graph neural networks (GNNs) have shown great power in modeling graph structured data. However, similar to other machine learning models, GNNs may make predictions biased on protected sensitive attributes, e.g., skin color and gender. Because machine learning algorithms including GNNs are trained to reflect the distribution of the training data which often contains historical bias towards sensitive attributes. In addition, the discrimination in GNNs can be magnified by graph structures and the message-passing mechanism. As a result, the applications of GNNs in sensitive domains such as crime rate prediction would be largely limited. Though extensive studies of fair classification have been conducted on i.i.d data, methods to address the problem of discrimination on non-i.i.d data are rather limited. Furthermore, the practical scenario of sparse annotations in sensitive attributes is rarely considered in existing works. Therefore, we study the novel and important problem of learning fair GNNs with limited sensitive attribute information. FairGNN is proposed to eliminate the bias of GNNs whilst maintaining high node classification accuracy by leveraging graph structures and limited sensitive information. Our theoretical analysis shows that FairGNN can ensure the fairness of GNNs under mild conditions given limited nodes with known sensitive attributes. Extensive experiments on real-world datasets also demonstrate the effectiveness of FairGNN in debiasing and keeping high accuracy.
L2D 2021: First International Workshop on Enabling Data-Driven Decisions from Learning on the Web
Authors: Danilo Dessì, Tanja Käser, Mirko Marras, Elvira Popescu, Harald Sack
Venue: WSDM
Year: 2021

By offering courses and resources, learning platforms on the Web have been attracting lots of participants, and the interactions with these systems have generated a vast amount of learning-related data. Their collection, processing and analysis have promoted a significant growth of learning analytics and have opened up new opportunities for supporting and assessing educational experiences. To provide all the stakeholders involved in the educational process with a timely guidance, being able to understand student's behavior and enable models which provide data-driven decisions pertaining to the learning domain is a primary property of online platforms, aiming at maximizing learning outcomes. In this workshop, we focus on collecting new contributions in this emerging area and on providing a common ground for researchers and practitioners (Website: https://mirkomarras.github.io/l2d-wsdm2021).
Online Experimentation with Surrogate Metrics: Guidelines and a Case Study
Authors: Weitao Duan, Shan Ba, Chunzhe Zhang
Venue: WSDM
Year: 2021

A/B tests have been widely adopted across industries as the golden rule that guides decision making. However, the long-term true north metrics we ultimately want to drive through A/B test may take a long time to mature. In these situations, a surrogate metric which predicts the long-term metric is often used instead to conclude whether the treatment is effective. However, because the surrogate rarely predicts the true north perfectly, a regular A/B test based on surrogate metrics tends to have high false positive rate and the treatment variant deemed favorable from the test may not be the winning one. In this paper, we discuss how to adjust the A/B testing comparison to ensure experiment results are trustworthy. We also provide practical guidelines on the choice of good surrogate metrics. To provide a concrete example of how to leverage surrogate metrics for fast decision making, we present a case study on developing and evaluating the predicted confirmed hire surrogate metric in LinkedIn job marketplace.
Beyond Web Search: How Email Search is Different and Why it Matters
Authors: Susan T. Dumais
Venue: WSDM
Year: 2021

Web search has transformed how we access all kinds of information and has become a core fabric of everyday life. It is used to find information, buy things, plan travel, understand medical conditions, monitor events, etc. Search in other domains has not received nearly the same attention so our experiences in web search shape our thinking about search more generally even when the scenarios are quite different. This is especially true for email search. Although email was initially designed to facilitate asynchronous communication, it has also become a large repository of personal information. The volume of email continues to grow in both consumer and enterprise settings, and search plays a key role in getting back to needed information. Email search is, however, very different than Web search on many dimensions -- the content being sought is personal and private, metadata such as who sent it or when it was sent is plentiful and important, search intentions are different, people know a lot about what they are looking for, etc. Given these differences, new approaches are required. In this talk I will summarize research we have done using large-scale behavioral logs and complementary qualitative methods to characterize what are people looking for, what they know about what they are looking for, and how this interacts with email management practices. I will then describe several opportunities to help people articulate their information needs and design interfaces and interaction techniques to support this. Finally, I will conclude by pointing to new frontiers in email management and search.
SoMin.ai: Personality-Driven Content Generation Platform
Authors: Aleksandr Farseev, Qi Yang, Andrey Filchenkov, Kirill Lepikhin, Yu-Yi Chu-Farseeva, Daron-Benjamin Loo
Venue: WSDM
Year: 2021

In this technical demonstration, we showcase the World's first personality-driven marketing content generation platform, called SoMin.ai. The platform combines deep multi-view personality profiling framework and style generative adversarial networks facilitating the automatic creation of content that appeals to different human personality types. The platform can be used for enhancement of the social networking user experience as well as for content marketing routines. Guided by the MBTI personality type, automatically derived from a user social network content, SoMin.ai generates new social media content based on the preferences of other users with a similar personality type aiming at enhancing the user experience on social networking venues as well diversifying the efforts of marketers when crafting new content for digital marketing campaigns. The real-time user feedback to the platform via the platform's GUI fine-tunes the content generation model and the evaluation results demonstrate the promising performance of the proposed multi-view personality profiling framework when being applied in the content generation scenario. By leveraging content generation at a large scale, marketers will be able to execute more effective digital marketing campaigns at a lower cost.
Real-time Streaming of Gait Assessment for Parkinson's Disease
Authors: Cristopher Flagg, Ophir Frieder, Sean MacAvaney, Gholam Motamedi
Venue: WSDM
Year: 2021

Patients with progressive neurological disorders such as Parkinson's disease, Huntington's disease, and Amyotrophic Lateral Sclerosis (ALS) suffer both chronic and episodic difficulties with locomotion. Real-time assessment and visualization of sensor data can be valuable to physicians monitoring the progression of these conditions. We present a system that utilizes the attention based bi-directional recurrent neural network (RNN) presented in [2] to evaluate foot pressure sensor data streamed directly from a pair of sensors attached to a patient. The demonstration also supports indirect streaming from recorded sessions, such as those stored in a FHIR [1] enabled electronic medical records repository, for post-hoc evaluation and comparison of a patient's gait over time. The system evaluates and visualizes the streamed gait in a real time web interface to provide a personalized normality rating that highlights the strengths and weaknesses of a patient's gait.
WSDM 2021 Tutorial on Conversational Recommendation Systems
Authors: Zuohui Fu, Yikun Xian, Yongfeng Zhang, Yi Zhang
Venue: WSDM
Year: 2021

Recent years have witnessed the emerging of conversational systems, including both physical devices and mobile-based applications. Both the research community and industry believe that conversational systems will have a major impact on human-computer interaction, and specifically, the IR/DM/RecSys communities have begun to explore Conversational Recommendation Systems. Conversational recommendation aims at finding or recommending the most relevant information (e.g., web pages, answers, movies, products) for users based on textual- or spoken-dialogs, through which users can communicate with the system more efficiently using natural language conversations. Due to users' constant need to look for information to support both work and daily life, conversational recommendation system will be one of the key techniques towards an intelligent web. The tutorial focuses on the foundations and algorithms for conversational recommendation, as well as their applications in real-world systems such as search engine, e-commerce and social networks. The tutorial aims at introducing and communicating conversational recommendation methods to the community, as well as gathering researchers and practitioners interested in this research direction for discussions, idea communications, and research promotions.