OpenRaiser/Intern-Atlas huggingface dataset:
DAWAR: Diversity-aware web apis recommendation for mashup creation based on correlation graph
Authors: Wenwen Gong, Xuyun Zhang, Yifei Chen, Qiang He, Amin Beheshti, Xiaolong Xu, Chao Yan, Lianyong Qi
Venue: Conference SIGIR 2022
Year: 2022

With the ever-increasing popularity of microservice architecture, a considerable number of enterprises or organizations have encapsulated their complex business services into various lightweight functions as published them accessible APIs (Application Programming Interfaces). Through keyword search, a software developer could select a set of APIs from a massive number of candidates to implement the functions of a complex mashup, which reduces the development cost significantly. However, traditional keyword search methods for APIs often suffer from several critical issues such as functional compatibility and limited diversity in search results, which may lead to mashup creation failures and lower development productivity. To deal with these challenges, this paper designs DAWAR, a diversity-aware Web APIs recommendation approach that finds diversified and compatible APIs for mashup creation. Specifically, the APIs recommendation problem for mashup creating is modelled as a graph search problem that aims to find the minimal group Steiner trees in a correlation graph of APIs. DAWAR innovatively employs the determinantal point processes to diversify the recommended results. Empirical evaluation is performed on commonly-used real-world datasets, and the statistic results show that DAWAR is able to achieve significant improvements in terms of recommendation diversity, accuracy, and compatibility.
Lagging inference networks and posterior collapse in variational autoencoders
Authors: Junxian He, Daniel Spokoyny, Graham Neubig, Taylor Berg-Kirkpatrick
Venue: Conference ICLR 2019
Year: 2019

The variational autoencoder (VAE) is a popular combination of deep latent variable model and accompanying variational learning technique. By using a neural inference network to approximate the model's posterior on latent variables, VAEs efficiently parameterize a lower bound on marginal data likelihood that can be optimized directly via gradient methods. In practice, however, VAE training often results in a degenerate local optimum known as "posterior collapse" where the model learns to ignore the latent variable and the approximate posterior mimics the prior. In this paper, we investigate posterior collapse from the perspective of training dynamics. We find that during the initial stages of training the inference network fails to approximate the model's true posterior, which is a moving target. As a result, the model is encouraged to ignore the latent encoding and posterior collapse occurs. Based on this observation, we propose an extremely simple modification to VAE training to reduce inference lag: depending on the model's current mutual information between latent variable and observation, we aggressively optimize the inference network before performing each model update. Despite introducing neither new model components nor significant complexity over basic VAE, our approach is able to avoid the problem of collapse that has plagued a large amount of previous work. Empirically, our approach outperforms strong autoregressive baselines on text and image benchmarks in terms of held-out likelihood, and is competitive with more complex techniques for avoiding collapse while being substantially faster.
RecVAE: A New Variational Autoencoder for Top-N Recommendations with Implicit Feedback
Authors: Ilya Shenbin, Anton Alekseev, Elena Tutubalina, Valentin Malykh, Sergey I. Nikolenko
Venue: Conference WSDM 2020
Year: 2020

Recent research has shown the advantages of using autoencoders based on deep neural networks for collaborative filtering. In particular, the recently proposed Mult-VAE model, which used the multinomial likelihood variational autoencoders, has shown excellent results for top-N recommendations. In this work, we propose the Recommender VAE (RecVAE) model that originates from our research on regularization techniques for variational autoencoders. RecVAE introduces several novel ideas to improve Mult-VAE, including a novel composite prior distribution for the latent codes, a new approach to setting the beta hyperparameter for the beta-VAE framework, and a new approach to training based on alternating updates. In experimental evaluation, we show that RecVAE significantly outperforms previously proposed autoencoder-based models, including Mult-VAE and RaCT, across classical collaborative filtering datasets, and present a detailed ablation study to assess our new developments. Code and models are available at https://github.com/ilya-shenbin/RecVAE.
Distributionally-robust Recommendations for Improving Worst-case User Experience
Authors: Hongyi Wen, Xinyang Yi, Tiansheng Yao, Jiaxi Tang, Lichan Hong, Ed H. Chi
Venue: Conference WWW 2022
Year: 2022

Modern recommender systems have evolved rapidly along with deep learning models that are well-optimized for overall performance, especially those trained under Empirical Risk Minimization (ERM). However, a recommendation algorithm that focuses solely on the average performance may reinforce the exposure bias and exacerbate the “rich-get-richer” effect, leading to unfair user experience. In a simulation study, we demonstrate that such performance gap among various user groups is enlarged by an ERM-trained recommender in the long-term. To mitigate such amplification effects, we propose to optimize for the worst-case performance under the Distributionally Robust Optimization (DRO) framework, with the goal of improving long-term fairness for disadvantaged subgroups. In addition, we propose a simple-yet-effective streaming optimization improvement called Streaming-DRO (S-DRO), which effectively reduces loss variances for recommendation problems with sparse and long-tailed data distributions. Our results on two large-scale datasets suggest that (1) DRO is a flexible and effective technique for improving worst-case performance, and (2) Streaming-DRO outperforms vanilla DRO and other strong baselines by improving the worst-case and overall performance at the same time.
Invariant collaborative filtering to popularity distribution shift
Authors: An Zhang, Jingnan Zheng, Xiang Wang, Yancheng Yuan, Tat-Seng Chua
Venue: Conference WWW 2023
Year: 2023

Collaborative Filtering (CF) models, despite their great success, suffer from severe performance drops due to popularity distribution shifts, where these changes are ubiquitous and inevitable in real-world scenarios. Unfortunately, most leading popularity debiasing strategies, rather than tackling the vulnerability of CF models to varying popularity distributions, require prior knowledge of the test distribution to identify the degree of bias and further learn the popularity-entangled representations to mitigate the bias. Consequently, these models result in significant performance benefits in the target test set, while dramatically deviating the recommendation from users’ true interests without knowing the popularity distribution in advance. In this work, we propose a novel learning framework, Invariant Collaborative Filtering (InvCF), to discover disentangled representations that faithfully reveal the latent preference and popularity semantics without making any assumption about the popularity distribution. At its core is the distillation of unbiased preference representations (i.e., user preference on item property), which are invariant to the change of popularity semantics, while filtering out the popularity feature that is unstable or outdated. Extensive experiments on five benchmark datasets and four evaluation settings (i.e., synthetic long-tail, unbiased, temporal split, and out-of-distribution evaluations) demonstrate that InvCF outperforms the state-of-the-art baselines in terms of popularity generalization ability on real recommendations. Visualization studies shed light on the advantages of InvCF for disentangled representation learning. Our codes are available at https://github.com/anzhang314/InvCF.
A Model-Agnostic Popularity Debias Training Framework for Click-Through Rate Prediction in Recommender System
Authors: Fan Zhang, Qijie Shen
Venue: Conference SIGIR 2023
Year: 2023

Recommender system (RS) is widely applied in a multitude of scenarios to aid individuals obtaining the information they require efficiently. At the same time, the prevalence of popularity bias in such systems has become a widely acknowledged issue. To address this challenge, we propose a novel method named Model-Agnostic Popularity Debias Training Framework (MDTF). It consists of two basic modules including 1) General Ranking Model (GRM), which is model-agnostic and can be implemented as any ranking models; and 2) Popularity Debias Module (PDM), which estimates the impact of the competitiveness and popularity of candidate items on the CTR, by utilizing the feedback of cold-start users to re-weigh the loss in GRM. MDTF seamlessly integrates these two modules in an end-to-end multi-task learning framework. Extensive experiments on both real-world offline dataset and online A/B test demonstrate its superiority over state-of-the-art methods.
Google news personalization: scalable online collaborative filtering
Authors: Abhinandan Das, Mayur Datar, Ashutosh Garg, Shyamsundar Rajaram
Venue: Conference WWW 2007
Year: 2007

Several approaches to collaborative filtering have been studied but seldom have studies been reported for large (several millionusers and items) and dynamic (the underlying item set is continually changing) settings. In this paper we describe our approach to collaborative filtering for generating personalized recommendations for users of Google News. We generate recommendations using three approaches: collaborative filtering using MinHash clustering, Probabilistic Latent Semantic Indexing (PLSI), and covisitation counts. We combine recommendations from different algorithms using a linear model. Our approach is content agnostic and consequently domain independent, making it easily adaptable for other applications and languages with minimal effort. This paper will describe our algorithms and system setup in detail, and report results of running the recommendations engine on Google News.
Embedding-based News Recommendation for Millions of Users
Authors: Shumpei Okura, Yukihiro Tagami, Shingo Ono, Akira Tajima
Venue: Conference KDD 2017
Year: 2017

It is necessary to understand the content of articles and user preferences to make effective news recommendations. While ID-based methods, such as collaborative filtering and low-rank factorization, are well known for making recommendations, they are not suitable for news recommendations because candidate articles expire quickly and are replaced with new ones within short spans of time. Word-based methods, which are often used in information retrieval settings, are good candidates in terms of system performance but have issues such as their ability to cope with synonyms and orthographical variants and define "queries" from users' historical activities. This paper proposes an embedding-based method to use distributed representations in a three step end-to-end manner: (i) start with distributed representations of articles based on a variant of a denoising autoencoder, (ii) generate user representations by using a recurrent neural network (RNN) with browsing histories as input sequences, and (iii) match and list articles for users based on inner-product operations by taking system performance into consideration. The proposed method performed well in an experimental offline evaluation using past access data on Yahoo! JAPAN's homepage. We implemented it on our actual news distribution system based on these experimental results and compared its online performance with a method that was conventionally incorporated into the system. As a result, the click-through rate (CTR) improved by 23% and the total duration improved by 10%, compared with the conventionally incorporated method. Services that incorporated the method we propose are already open to all users and provide recommendations to over ten million individual users per day who make billions of accesses per month.
Collaborative topic modeling for recommending scientific articles
Authors: Chong Wang, David M. Blei
Venue: Conference KDD 2011
Year: 2011

Researchers have access to large online archives of scientific articles. As a consequence, finding relevant papers has become more difficult. Newly formed online communities of researchers sharing citations provides a new way to solve this problem. In this paper, we develop an algorithm to recommend scientific articles to users of an online community. Our approach combines the merits of traditional collaborative filtering and probabilistic topic modeling. It provides an interpretable latent structure for users and items, and can form recommendations about both existing and newly published articles. We study a large subset of data from CiteULike, a bibliography sharing service, and show that our algorithm provides a more effective recommender system than traditional collaborative filtering.
HDNR: A Hyperbolic-Based Debiased Approach for Personalized News Recommendation
Authors: Shicheng Wang, Shu Guo, Lihong Wang, Tingwen Liu, Hongbo Xu
Venue: Conference SIGIR 2023
Year: 2023

Personalized news recommendation aims to recommend candidate news to the target user, according to the clicked news history. The user-news interaction data exhibits power-law distribution, however, existing works usually learn representations in Euclidean space which makes inconsistent capacities between data space and embedding space, leading to severe representation distortion problem. Besides, the existence of conformity bias, a potential cause of power-law distribution, may introduce biased guidance to learn user representations. In this paper, we propose a novel debiased method based on hyperbolic space, named HDNR, to tackle the above problems. Specifically, first, we employ hyperboloid model with exponential growth capacity to conduct user and news modeling, in order to solve inconsistent space capacities problem and obtain low distortion representations. Second, we design a re-weighting aggregation module to further mitigate conformity bias in data distribution, through considering local importance of the clicked news among contextual history and its global popularity degree simultaneously. Finally, we calculate the relevance score between target user and candidate news representations. We conduct experiments on two real-world news recommendation datasets MIND-Large, MIND-Small and empirical results demonstrate the effectiveness of our approach from multiple perspectives.
Unsupervised visualization of image datasets using contrastive learning
Authors: Jan Niklas Böhm, Philipp Berens, Dmitry Kobak
Venue: Conference ICLR 2023
Year: 2023

Visualization methods based on the nearest neighbor graph, such as t-SNE or UMAP, are widely used for visualizing high-dimensional data. Yet, these approaches only produce meaningful results if the nearest neighbors themselves are meaningful. For images represented in pixel space this is not the case, as distances in pixel space are often not capturing our sense of similarity and therefore neighbors are not semantically close. This problem can be circumvented by self-supervised approaches based on contrastive learning, such as SimCLR, relying on data augmentation to generate implicit neighbors, but these methods do not produce two-dimensional embeddings suitable for visualization. Here, we present a new method, called t-SimCNE, for unsupervised visualization of image data. T-SimCNE combines ideas from contrastive learning and neighbor embeddings, and trains a parametric mapping from the high-dimensional pixel space into two dimensions. We show that the resulting 2D embeddings achieve classification accuracy comparable to the state-of-the-art high-dimensional SimCLR representations, thus faithfully capturing semantic relationships. Using t-SimCNE, we obtain informative visualizations of the CIFAR-10 and CIFAR-100 datasets, showing rich cluster structure and highlighting artifacts and outliers.
On Missing Labels, Long-Tails and Propensities in Extreme Multi-Label Classification
Authors: Erik Schultheis, Marek Wydmuch, Rohit Babbar, Krzysztof Dembczynski
Venue: Conference KDD 2022
Year: 2022

The propensity model introduced by Jain et al. 2016 has become a standard approach for dealing with missing and long-tail labels in extreme multi-label classification (XMLC). In this paper, we critically revise this approach showing that despite its theoretical soundness, its application in contemporary XMLC works is debatable. We exhaustively discuss the flaws of the propensity-based approach, and present several recipes, some of them related to solutions used in search engines and recommender systems, that we believe constitute promising alternatives to be followed in XMLC.
Distilsum: Distilling the knowledge for extractive summarization
Authors: Ruipeng Jia, Yanan Cao, Haichao Shi, Fang Fang, Yanbing Liu, Jianlong Tan
Venue: Conference CIKM 2020
Year: 2020

A popular choice for extractive summarization is to conceptualize it as sentence-level classification, supervised by binary labels. While the common metric ROUGE prefers to measure the text similarity, instead of the performance of classifier. For example, BERTSUMEXT, the best extractive classifier so far, only achieves a precision of 32.9% at the top 3 extracted sentences (P@3) on CNN/DM dataset. It is obvious that current approaches cannot model the complex relationship of sentences exactly with 0/1 targets. In this paper, we introduce DistilSum, which contains teacher mechanism and student model. Teacher mechanism produces high entropy soft targets at a high temperature. Our student model is trained with the same temperature to match these informative soft targets and tested with temperature of 1 to distill for ground-truth labels. Compared with large version of BERTSUMEXT, our experimental result on CNN/DM achieves a substantial improvement of 0.99 ROUGE-L score (text similarity) and 3.95 P@3 score (performance of classifier). Our source code will be available on Github.
A new generation of perspective api: Efficient multilingual character-level transformers
Authors: Alyssa Lees, Vinh Q. Tran 0002, Yi Tay, Jeffrey Sorensen, Jai Prakash Gupta 0001, Donald Metzler, Lucy Vasserman
Venue: Conference KDD 2022
Year: 2022

On the world wide web, toxic content detectors are a crucial line of defense against potentially hateful and offensive messages. As such, building highly effective classifiers that enable a safer internet is an important research area. Moreover, the web is a highly multilingual, cross-cultural community that develops its own lingo over time. As such, it is crucial to develop models that are effective across a diverse range of languages, usages, and styles. In this paper, we present the fundamentals behind the next version of the Perspective API from Google Jigsaw. At the heart of the approach is a single multilingual token-free Charformer model that is applicable across a range of languages, domains, and tasks. We demonstrate that by forgoing static vocabularies, we gain flexibility across a variety of settings. We additionally outline the techniques employed to make such a byte-level model efficient and feasible for productionization. Through extensive experiments on multilingual toxic comment classification benchmarks derived from real API traffic and evaluation on an array of code-switching, covert toxicity, emoji-based hate, human-readable obfuscation, distribution shift, and bias evaluation settings, we show that our proposed approach outperforms strong baselines. Finally, we present our findings from deploying this system in production.
Reliable decision from multiple subtasks through threshold optimization: Content moderation in the wild
Authors: Donghyun Son, Byounggyu Lew, Kwanghee Choi, Yongsu Baek, Seungwoo Choi, Beomjun Shin, Sungjoo Ha, Buru Chang
Venue: Conference WSDM 2023
Year: 2023

Social media platforms struggle to protect users from harmful content through content moderation. These platforms have recently leveraged machine learning models to cope with the vast amount of user-generated content daily. Since moderation policies vary depending on countries and types of products, it is common to train and deploy the models per policy. However, this approach is highly inefficient, especially when the policies change, requiring dataset re-labeling and model re-training on the shifted data distribution. To alleviate this cost inefficiency, social media platforms often employ third-party content moderation services that provide prediction scores of multiple subtasks, such as predicting the existence of underage personnel, rude gestures, or weapons, instead of directly providing final moderation decisions. However, making a reliable automated moderation decision from the prediction scores of the multiple subtasks for a specific target policy has not been widely explored yet. In this study, we formulate real-world scenarios of content moderation and introduce a simple yet effective threshold optimization method that searches the optimal thresholds of the multiple subtasks to make a reliable moderation decision in a cost-effective way. Extensive experiments demonstrate that our approach shows better performance in content moderation compared to existing threshold optimization methods and heuristics.
Learning structural node embeddings via diffusion wavelets
Authors: Claire Donnat, Marinka Zitnik, David Hallac, Jure Leskovec
Venue: Conference KDD 2018
Year: 2018

Nodes residing in different parts of a graph can have similar structural roles within their local network topology. The identification of such roles provides key insight into the organization of networks and can be used for a variety of machine learning tasks. However, learning structural representations of nodes is a challenging problem, and it has typically involved manually specifying and tailoring topological features for each node. In this paper, we develop GraphWave, a method that represents each node's network neighborhood via a low-dimensional embedding by leveraging heat wavelet diffusion patterns. Instead of training on hand-selected features, GraphWave learns these embeddings in an unsupervised way. We mathematically prove that nodes with similar network neighborhoods will have similar GraphWave embeddings even though these nodes may reside in very different parts of the network, and our method scales linearly with the number of edges. Experiments in a variety of different settings demonstrate GraphWave's real-world potential for capturing structural roles in networks, and our approach outperforms existing state-of-the-art baselines in every experiment, by as much as 137%.
Theoretical impediments to machine learning with seven sparks from the causal revolution
Authors: Judea Pearl
Venue: Conference WSDM 2018
Year: 2018

Current machine learning systems operate, almost exclusively, in a statistical, or model-free mode, which entails severe theoretical limits on their power and performance. Such systems cannot reason about interventions and retrospection and, therefore, cannot serve as the basis for strong AI. To achieve human level intelligence, learning machines need the guidance of a model of reality, similar to the ones used in causal inference tasks. To demonstrate the essential role of such models, I will present a summary of seven tasks which are beyond reach of current machine learning systems and which have been accomplished using the tools of causal modeling.