OpenRaiser/Intern-Atlas huggingface dataset:
Graph neural networks pretraining through inherent supervision for molecular property prediction
Authors: Roy Benjamin, Uriel Singer, Kira Radinsky
Venue: Conference CIKM 2022
Year: 2022

Recent global events have emphasized the importance of accelerating the drug discovery process. A way to deal with the issue is to use machine learning to increase the rate at which drugs are made available to the public. However, chemical labeled data for real-world applications is extremely scarce making traditional approaches less effective. A fruitful course of action for this challenge is to pretrain a model using related tasks with large enough datasets, with the next step being finetuning it for the desired task. This is challenging as creating these datasets requires labeled data or expert knowledge. To aid in solving this pressing issue, we introduce MISU - Molecular Inherent SUpervision, a unique method for pretraining graph neural networks for molecular property prediction. Our method leapfrogs past the need for labeled data or any expert knowledge by introducing three innovative components that utilize inherent properties of molecular graphs to induce information extraction at different scales, from the local neighborhood of an atom to substructures in the entire molecule. Our empirical results for six chemical-property-prediction tasks show that our method reaches state-of-the-art results compared to numerous baselines.
Incorporating interpretability into latent factor models via fast influence analysis
Authors: Weiyu Cheng, Yanyan Shen, Linpeng Huang, Yanmin Zhu
Venue: Conference KDD 2019
Year: 2019

Latent factor models (LFMs) such as matrix factorization have achieved the state-of-the-art performance among various collaborative filtering approaches for recommendation. Despite the high recommendation accuracy of LFMs, a critical issue to be resolved is their lack of interpretability. Extensive efforts have been devoted to interpreting the prediction results of LFMs. However, they either rely on auxiliary information which may not be available in practice, or sacrifice recommendation accuracy for interpretability. Influence functions, stemming from robust statistics, have been developed to understand the effect of training points on the predictions of black-box models. Inspired by this, we propose a novel explanation method named FIA (Fast Influence Analysis) to understand the prediction of trained LFMs by tracing back to the training data with influence functions. We present how to employ influence functions to measure the impact of historical user-item interactions on the prediction results of LFMs and provide intuitive neighbor-style explanations based on the most influential interactions. Our proposed FIA exploits the characteristics of two important LFMs, matrix factorization and neural collaborative filtering, and is capable of accelerating the overall influence analysis process. We provide a detailed complexity analysis for FIA over LFMs and conduct extensive experiments to evaluate its performance using real-world datasets. The results demonstrate the effectiveness and efficiency of FIA, and the usefulness of the generated explanations for the recommendation results.
Fedcdr: federated cross-domain recommendation for privacy-preserving rating prediction
Authors: Meihan Wu, Li Li, Chang Tao, Eric Rigall, Xiaodong Wang, Cheng-Zhong Xu
Venue: Conference CIKM 2022
Year: 2022

The cold-start problem, faced when providing recommendations to newly joined users with no historical interaction record existing in the platform, is one of the most critical problems that negatively impact the performance of a recommendation system. Fortunately, cross-domain recommendation~(CDR) is a promising approach for solving this problem, which can exploit the knowledge of these users from source domains to provide recommendations in the target domain. However, this method requires that the central server has the interaction behaviour data in both domains of all the users, which prevents users from participating due to privacy issues. In this work, we propose FedCDR, a federated learning based cross-domain recommendation system that effectively trains the recommendation model while keeping users' raw data and private user-specific parameters located on their own devices. Unlike existing CDR models, a personal module and a transfer module are designed to adapt to the extremely heterogeneous data on the participating devices. Specifically, the personal module extracts private user features for each user, while the transfer module is responsible for transferring the knowledge between the two domains. Moreover, in order to provide personalized recommendations with less storage and communication costs while effectively protecting privacy, we design a personalized update strategy for each client and a personalized aggregation strategy for the server. In addition, we conduct comprehensive experiments on the representative Amazon 5-cores datasets for three popular rating prediction tasks to evaluate the effectiveness of FedCDR. The results show that FedCDR outperforms the state-of-the-art methods in mean absolute error (MAE) and root mean squared error (RMSE). For example, in task Movie&Music, FedCDR can effectively improve the performance up to 65.83% and 55.45% on MAE and RMSE, respectively, when the new users are in the movie domain.
Improving the expressiveness of k-hop message-passing gnns by injecting contextualized substructure information
Authors: Tianjun Yao, Yingxu Wang, Kun Zhang 0001, Shangsong Liang
Venue: Conference KDD 2023
Year: 2023

Graph neural networks (GNNs) have become the de facto standard for representational learning in graphs, and have achieved state-of-the-art performance in many graph-related tasks; however, it has been shown that the expressive power of standard GNNs are equivalent maximally to 1-dimensional Weisfeiler-Lehman (1-WL) Test. Recently, there is a line of works aiming to enhance the expressive power of graph neural networks. One line of such works aim at developing K-hop message-passing GNNs where node representation is updated by aggregating information from not only direct neighbors but all neighbors within K-hop of the node. Another line of works leverages subgraph information to enhance the expressive power which is proven to be strictly more powerful than 1-WL test. In this work, we discuss the limitation of K-hop message-passing GNNs and propose substructure encoding function to uplift the expressive power of any K-hop message-passing GNN. We further inject contextualized substructure information to enhance the expressiveness of K-hop message-passing GNNs. Our method is provably more powerful than previous works on K-hop graph neural networks and 1-WL subgraph GNNs, which is a specific type of subgraph based GNN models, and not less powerful than 3-WL. Empirically, our proposed method set new state-of-the-art performance or achieves comparable performance for a variety of datasets. Our code is available at https://github.com/tianyao-aka/Expresive_K_hop_GNNs.
Boosted Decision Tree Regression Adjustment for Variance Reduction in Online Controlled Experiments
Authors: Alexey Poyarkov, Alexey Drutsa, Andrey Khalyavin, Gleb Gusev, Pavel Serdyukov
Venue: Conference KDD 2016
Year: 2016

Nowadays, the development of most leading web services is controlled by online experiments that qualify and quantify the steady stream of their updates achieving more than a thousand concurrent experiments per day. Despite the increasing need for running more experiments, these services are limited in their user traffic. This situation leads to the problem of finding a new or improving existing key performance metric with a higher sensitivity and lower variance. We focus on the problem of variance reduction for engagement metrics of user loyalty that are widely used in A/B testing of web services. We develop a general framework that is based on evaluation of the mean difference between the actual and the approximated values of the key performance metric (instead of the mean of this metric). On the one hand, it allows us to incorporate the state-of-the-art techniques widely used in randomized experiments of clinical and social research, but limitedly used in online evaluation. On the other hand, we propose a new class of methods based on advanced machine learning algorithms, including ensembles of decision trees, that, to the best of our knowledge, have not been applied earlier to the problem of variance reduction. We validate the variance reduction approaches on a very large set of real large-scale A/B experiments run at Yandex for different engagement metrics of user loyalty. Our best approach demonstrates $63\%$ average variance reduction (which is equivalent to 63% saved user traffic) and detects the treatment effect in $2$ times more A/B experiments.
FINAL: Factorized Interaction Layer for CTR Prediction
Authors: Jieming Zhu, Qinglin Jia, Guohao Cai, Quanyu Dai, Jingjie Li, Zhenhua Dong, Ruiming Tang, Rui Zhang
Venue: Conference SIGIR 2023
Year: 2023

Multi-layer perceptron (MLP) serves as a core component in many deep models for click-through rate (CTR) prediction. However, vanilla MLP networks are inefficient in learning multiplicative feature interactions, making feature interaction learning an essential topic for CTR prediction. Existing feature interaction networks are effective in complementing the learning of MLPs, but they often fall short of the performance of MLPs when applied alone. Thus, their integration with MLP networks is necessary to achieve improved performance. This situation motivates us to explore a better alternative to the MLP backbone that could potentially replace MLPs. Inspired by factorization machines, in this paper, we propose FINAL, a factorized interaction layer that extends the widely-used linear layer and is capable of learning 2nd-order feature interactions. Similar to MLPs, multiple FINAL layers can be stacked into a FINAL block, yielding feature interactions with an exponential degree growth. We unify feature interactions and MLPs into a single FINAL block and empirically show its effectiveness as a replacement for the MLP block. Furthermore, we explore the ensemble of two FINAL blocks as an enhanced two-stream CTR model, setting a new state-of-the-art on open benchmark datasets. FINAL can be easily adopted as a building block and has achieved business metric gains in multiple applications at Huawei. Our source code will be made available at MindSpore/models and FuxiCTR/model_zoo.
Inductive knowledge graph embedding via exploring interaction patterns of relations
Authors: Chong Mu, Lizong Zhang, Jinchuan Zhang, Qian Huang, Zhiguo Wang
Venue: Conference CIKM 2024
Year: 2024

Recent research in inductive reasoning has focused on predicting missing links between entities that are not observed during training. However, most approaches usually require that the relations are known at the inference time. In the real world, new entities and new relations usually emerge concurrently, which greatly challenges the model's generalization ability. In this paper, we propose a novel inductive knowledge graph embedding model that effectively handles unknown entities and relations by capturing their local structural features. Specifically, a relation graph is constructed to learn relation representations. In the relation graph, we employ a four-dimensional vector to represent the interaction patterns between nodes (relations), where each dimension corresponds to a specific type of interaction. For entity representations, our model dynamically initializes entity features using relation features and attentively aggregates neighboring features of entities to update entity features. By modeling interaction patterns between relations and incorporating structural information of entities, our model learns how to aggregate neighboring embeddings using attention mechanisms, thus generating high-quality embeddings for new entities and relations. Extensive experiments on benchmark datasets demonstrate that our model outperforms state-of-the-art methods, particularly in scenarios involving completely new relations.
Generative recommendation models: Progress and directions
Authors: Yupeng Hou, An Zhang, Leheng Sheng, Zhengyi Yang, Xiang Wang, Tat-Seng Chua, Julian J. McAuley
Venue: Conference WWW 2025
Year: 2025

Recommendation models typically follow a discriminative paradigm, predicting whether items should be retrieved. While effective, the expressive capabilities of these recommender systems are limited. Users can only passively browse the recommended items rather than actively express their needs and engage in an interactive experience. With recent advances in generative models such as large language models, a paradigm shift is happening in the study of recommender systems. Researchers propose building generative recommendation models either by aligning pre-trained generative models with user behaviors or designing recommendation models within a generative framework. These models enable the systems to receive and deliver more human-like content, such as natural language, images, and beyond. In this tutorial, we first provide an overview of the latest progress in generative recommendation models, covering approaches based on large language models, semantic IDs, diffusion models, and more. We then make an in-depth discussion on the challenges, open questions, and potential future directions in developing generative recommendation models.
Was this review helpful to you? it depends! context and voting patterns in online content
Authors: Ruben Sipos, Arpita Ghosh, Thorsten Joachims
Venue: Conference WWW 2014
Year: 2014

When a website hosting user-generated content asks users a straightforward question - "Was this content helpful?" with one "Yes" and one "No" button as the two possible answers - one might expect to get a straightforward answer. In this paper, we explore how users respond to this question and find that their responses are not quite straightforward after all. Using data from Amazon product reviews, we present evidence that users do not make absolute, independent voting decisions based on individual review quality alone. Rather, whether users vote at all, as well as the polarity of their vote for any given review, depends on the context in which they view it - reviews receive a larger overall number of votes when they are 'misranked', and the polarity of votes becomes more positive/negative when the review is ranked lower/higher than it deserves. We distill these empirical findings into a new probabilistic model of rating behavior that includes the dependence of rating decisions on context. Understanding and formally modeling voting behavior is crucial for designing learning mechanisms and algorithms for review ranking, and we conjecture that many of our findings also apply to user behavior in other online content-rating settings.
Adaptivegcn: Efficient gcn through adaptively sparsifying graphs
Authors: Dongyue Li, Tao Yang, Lun Du, Zhezhi He, Li Jiang
Venue: Conference CIKM 2021
Year: 2021

Graph Convolutional Networks (GCNs) have become the prevailing approach to efficiently learn representations from graph-structured data. Current GCN models adopt a neighborhood aggregation mechanism based on two primary operations, aggregation and combination. The workload of these two processes is determined by the input graph structure, making the graph input the bottleneck of processing GCN. Meanwhile, a large amount of task-irrelevant information in the graphs would hurt the model generalization performance. This brings the opportunity of studying how to remove the redundancy in the graphs. In this paper, we aim to accelerate GCN models by removing the task-irrelevant edges in the graph. We present AdaptiveGCN, an efficient and supervised graph sparsification framework. AdaptiveGCN adopts an edge predictor module to get edge selection strategies by learning the downstream task feedback signals for each GCN layer separately and adaptively in the training stage, then only inference with the selected edges in the test stage to speed up the GCN computation. The experimental results indicate that AdaptiveGCN could yield 43% (on CPU) and 39% (on GPU) GCN model speed-up averagely with comparable model performance on public graph learning benchmarks.
Contrastive Box Embedding for Collaborative Reasoning
Authors: Tingting Liang, Yuanqing Zhang, Qianhui Di, Congying Xia, Youhuizi Li, Yuyu Yin
Venue: Conference SIGIR 2023
Year: 2023

Most of the existing personalized recommendation methods predict the probability that one user might interact with the next item by matching their representations in the latent space. However, as a cognitive task, it is essential for an impressive recommender system to acquire the cognitive capacity rather than to decide the users' next steps by learning the pattern from the historical interactions through matching-based objectives. Therefore, in this paper, we propose to model the recommendation as a logical reasoning task which is more in line with an intelligent recommender system. Different from the prior works, we embed each query as a box rather than a single point in the vector space, which is able to model sets of users or items enclosed and logical operators (e.g., intersection) over boxes in a more natural manner. Although modeling the logical query with box embedding significantly improves the previous work of reasoning-based recommendation, there still exist two intractable issues including aggregation of box embeddings and training stalemate in critical point of boxes. To tackle these two limitations, we propose a Contrastive Box learning framework for Collaborative Reasoning (CBox4CR). Specifically, CBox4CR combines a smoothed box volume-based contrastive learning objective with the logical reasoning objective to learn the distinctive box representations for the user's preference and the logical query based on the historical interaction sequence. Extensive experiments conducted on four publicly available datasets demonstrate the superiority of our CBox4CR over the state-of-the-art models in recommendation task.
Boxcare: A box embedding model for disease representation and diagnosis prediction in healthcare data
Authors: Hang Lv, Zehai Chen, Yacong Yang, Guofang Ma, Yanchao Tan, Carl Yang
Venue: Conference WWW 2024
Year: 2024

Diagnosis prediction is becoming crucial to develop healthcare plans for patients based on Electronic Health Records (EHRs). Existing works usually enhance diagnosis prediction via learning accurate disease representation, where many of them try to capture inclusive relations based on the hierarchical structures of existing disease ontologies such as those provided by ICD-9 codes. However, they overlook exclusive relations that can reflect different and complementary perspectives of the ICD-9 structures, and thus fail to accurately represent relations among diseases and ICD-9 codes. To this end, we propose to project disease embeddings and ICD-9 code embeddings into boxes, where a box is an axis-aligned hyperrectangle with a geometric region and two boxes can clearly "include" or "exclude" each other. Upon box embeddings, we further obtain patient embeddings via aggregating the disease representations for diagnosis prediction. Extensive experiments on two real-world EHR datasets show significant performance gains brought by our proposed framework, yielding average improvements of 6.04% for diagnosis prediction over state-of-the-art competitors.
Learning probabilistic box embeddings for effective and efficient ranking
Authors: Lang Mei, Jiaxin Mao, Gang Guo, Ji-Rong Wen
Venue: Conference WWW 2022
Year: 2022

Ranking has been one of the most important tasks in information retrieval. With the development of deep representation learning, many researchers propose to encode both the query and items into embedding vectors and rank the items according to the inner product or distance measures in the embedding space. However, the ranking models based on vector embeddings may have shortages in effectiveness and efficiency. For effectiveness, they lack the intrinsic ability to model the diversity and uncertainty of queries and items in ranking. For efficiency, nearest neighbor search in a large collection of item vectors can be costly. In this work, we propose to use the recently proposed probabilistic box embeddings for effective and efficient ranking, in which queries and items are parameterized as high-dimensional axis-aligned hyper-rectangles. For effectiveness, we utilize probabilistic box embeddings to model the diversity and uncertainty with the overlapping relations of the hyper-rectangles, and prove that such overlapping measure is a kernel function which can be adopted in other kernel-based methods. For efficiency, we propose a box embedding-based indexing method, which can safely filter irrelevant items and reduce the retrieval latency. We further design a training strategy to increase the proportion of irrelevant items that can be filtered by the index. Experiments on public datasets show that the box embeddings and the box embedding-based indexing approaches are effective and efficient in two ranking tasks: ad hoc retrieval and product recommendation.
Test-time embedding normalization for popularity bias mitigation
Authors: Dain Kim, Jinhyeok Park, Dongwoo Kim
Venue: Conference CIKM 2023
Year: 2023

Popularity bias is a widespread problem in the field of recommender systems, where popular items tend to dominate recommendation results. In this work, we propose 'Test Time Embedding Normalization' as a simple yet effective strategy for mitigating popularity bias, which surpasses the performance of the previous mitigation approaches by a significant margin. Our approach utilizes the normalized item embedding during the inference stage to control the influence of embedding magnitude, which is highly correlated with item popularity. Through extensive experiments, we show that our method combined with the sampled softmax loss effectively reduces popularity bias compare to previous approaches for bias mitigation. We further investigate the relationship between user and item embeddings and find that the angular similarity between embeddings distinguishes preferable and non-preferable items regardless of their popularity. The analysis explains the mechanism behind the success of our approach in eliminating the impact of popularity bias. Our code is available at https://github.com/ml-postech/TTEN.
Eye Fixation and Early Vision: Kinetic Depth
Authors: Dana H. Ballard, Altan Ozcandarli
Venue: Conference ICCV 1988
Year: 1988

One aspect of primate intelligence is the ability to coordinate eye movements in the process of solving complex tasks. Primate eye movements have been studied in several disciplines but little work has been directed toward a computational theory that shows how the eye nouements can confer specific advantages in problem-solving behaviors, This paper outlines some of the elements of such a theory, emphasizing one point: the advantages of an active system in choosing an external frame of reference for the computations of early vision. These advantages are illustrated by the real-time computation of depth map with a monocular, fixating vision system.