OpenRaiser/Intern-Atlas huggingface dataset:
Vigil: Effective End-to-end Monitoring for Large-scale Recommender Systems at Glance
Authors: Priyansh Saxena, Manisha R
Venue: CIKM
Year: 2023

The success of large-scale recommender systems hinges upon their ability to deliver accurate and timely recommendations to a diverse user base. At Glance, we deliver snackable personalized content to the lock screens of 200M smartphones. In this context, continuous monitoring is paramount as it safeguards data integrity, detects drifts, addresses evolving user preferences, optimizes system downtime, and ultimately augments the system's effectiveness and user satisfaction. In this talk, we delve into Vigil, a set of monitoring practices developed to provide comprehensive end-to-end monitoring of recommender systems at Glance. These practices revolve around three key pillars: mitigating developer fatigue, ensuring precise predictions, and establishing a centralized monitoring framework. By adopting these practices, we have observed a 30% reduction in compute cost, a 26% drop in downtime, and a surge in developer productivity demonstrated by a 45% decrease in turnaround time.
Generative AI and the Future of Information Access
Authors: Chirag Shah
Venue: CIKM
Year: 2023

The prominent model of retrieving, evaluating, and using relevant information from databases, collections, and the web is going through a significant transformation. This is largely due to wide-scale availability of various generative AI systems that can take in natural language inputs and generate highly customized natural language text, images, audio, and videos. This transformation in how people seek and access information will have profound impacts on users, developers, and policymakers. It is already changing many sectors including education, health, and commerce. But the hopes and hypes of generative AI are often not clear as we get swept up by either the current capabilities and limitations of this technology in the short term or fear from speculative future in the long term. Instead, I believe we need to approach this area pragmatically and with scientific curiosity, scholarly rigor, and societal responsibility. In this talk, I will highlight some of the opportunities and challenges for information access stemming from recent advancements in generative AI. For instance, there are new possibilities now for addressing accessibility, low-resource domains, and bias in training data using generative AI tools. On the other hand, there are new challenges concerning hallucination, toxicity, and information provenance. It is clear that we want to benefit from what AI systems are capable of, but how do we do that while curbing some of these problems? I will argue that the solution is multifaceted and complex -- some will require technical advancements and others will call for policy changes. We will need to not only build information systems with fairness, transparency, and accountability in mind, but also train a new generation of developers, policymakers, and of course the users. The goal here is to cut through both hype and fear and think pragmatically about the future of information access.
Transferable Structure-based Adversarial Attack of Heterogeneous Graph Neural Network
Authors: Yu Shang, Yudong Zhang, Jiansheng Chen, Depeng Jin, Yong Li
Venue: CIKM
Year: 2023

Heterogeneous graph neural networks (HGNNs) have achieved remarkable development recently and exhibited superior performance in various tasks. However, recently HGNNs have been shown to have robustness weakness towards adversarial perturbations, which brings critical pitfalls for real applications, e.g. node classification and recommender systems. In particular, the transfer-based black-box attack is the most practical method to attack unknown models and poses a great threat to the reliability of HGNNs. In this work, we take the first step to explore the transferability of adversarial examples of HGNNs. Due to the overfitting of the source model, the adversarial perturbations generated by traditional methods usually exhibit unpromising transferability. To address this problem and boost adversarial transferability, we expect to seek common vulnerable directions of different models to attack. Inspired by the observation of the notable commonality of edge attention distribution between different HGNNs, we propose to guide the perturbation generation toward disrupting edge attention distribution. This edge attention-guided attack prioritizes the perturbation on edges that are more likely to be given common attention by different models, which benefits the transferability of adversarial perturbations. Finally, we develop two edge attention-guided attack methods towards heterogeneous relations tailored for HGNNs, called EA-FGSM and EA-PGD. Extensive experiments on six representative models and two datasets verify the effectiveness of our methods and form an unprecedented transfer robustness benchmark for HGNNs.
TOAK: A Topology-oriented Attack Strategy for Degrading User Identity Linkage in Cross-network Learning
Authors: Jiangli Shao, Yongqing Wang, Fangda Guo, Boshen Shi, Huawei Shen, Xueqi Cheng
Venue: CIKM
Year: 2023

Privacy concerns on social networks have received extensive attention in recent years. The task of user identity linkage (UIL), which aims to identify corresponding users across different social networks, poses a threat to privacy if applied unethically. Sensitive user information would be inferred with cross-network identity linkages. A feasible solution to this issue is to design an adversarial strategy that degrades the matching performance of UIL models. Nevertheless, most of the current adversarial attacks on graphs are tailored towards models working within a single network, failing to account for the challenges presented by cross-network learning tasks such as UIL. Also, in real-world scenarios, the adversarial strategy against UIL has more constraints as service providers can only add perturbations to their own networks. To tackle these challenges, this paper proposes a novel poisoning strategy to prevent nodes in a target network from being linked to other networks by UIL algorithms. Specifically, the UIL problem is formalized in the kernelized topology consistency perspective, and the objective is formulated as maximizing the structural variations in the target network before and after modifications. To achieve this, a novel graph kernel is defined based on earth mover's distance (EMD) in the edge-embedding space. In terms of efficiency, a fast attack strategy is proposed using greedy searching and a lower bound approximation of EMD. Results on three real-world datasets demonstrate that the proposed method outperforms six baselines and reaches a balance between effectiveness and imperceptibility while being efficient.
CANA: Causal-enhanced Social Network Alignment
Authors: Jiangli Shao, Yongqing Wang, Fangda Guo, Boshen Shi, Huawei Shen, Xueqi Cheng
Venue: CIKM
Year: 2023

Social network alignment is widely applied in web applications for identifying corresponding nodes across different networks, such as linking users across two social networks. Existing methods for social network alignment primarily rely on alignment consistency, assuming that nodes with similar attributes and neighbors are more likely to be aligned. However, distributional discrepancies in node attributes and neighbors across different networks would bring biases in alignment consistency, leading to inferior alignment performance. To address this issue, we conduct a causal analysis of alignment consistency. Based on this analysis, we propose a novel model called CANA that uses causal inference approaches to mitigate biases and enhance social network alignment. Firstly, we disentangle observed node attributes into endogenous features and exogenous features with multi-task learning. Only endogenous features are retained to overcome node attribute discrepancies. To eliminate biases caused by neighbors discrepancies, we propose causal-aware attention mechanisms and integrate them in graph neural network to reweight contributions of different neighbors in alignment consistency comparison. Additionally, backdoor adjustment is applied to reduce confounding effects and estimate unbiased alignment probability. Through experimental evaluation on four real-world datasets, the proposed method demonstrates superior performance in terms of alignment accuracy and top-k hits precision.
Representation Learning in Continuous-Time Dynamic Signed Networks
Authors: Kartik Sharma, Mohit Raghavendra, Yeon-Chang Lee, Anand Kumar M, Srijan Kumar
Venue: CIKM
Year: 2023

Signed networks allow us to model conflicting relationships and interactions, such as friend/enemy and support/oppose. These signed interactions happen in real-time. Modeling such dynamics of signed networks is crucial to understanding the evolution of polarization in the network and enabling effective prediction of the signed structure (i.e., link signs) in the future. However, existing works have modeled either (static) signed networks or dynamic (unsigned) networks but not dynamic signed networks. Since both sign and dynamics inform the graph structure in different ways, it is non-trivial to model how to combine the two features. In this work, we propose a new Graph Neural Network (GNN)-based approach to model dynamic signed networks, named SEMBA: Signed link's Evolution using Memory modules and Balanced Aggregation. Here, the idea is to incorporate the signs of temporal interactions using separate modules guided by balance theory and to evolve the embeddings from a higher-order neighborhood. Experiments on 4 real-world datasets and 3 different tasks demonstrate that SEMBA consistently and significantly outperforms the baselines by up to 80% on the tasks of predicting signs of future links while matching the state-of-the-art performance on predicting existence of these links in the future. We find that this improvement is due specifically to superior performance of SEMBA on the minority negative class. Code is made available at https://github.com/claws-lab/semba.
OnlineAutoClust: A Framework for Online Automated Clustering
Authors: Radwa El Shawi, Dmitri Rozgonjuk
Venue: CIKM
Year: 2023

Automated Machine Learning (AutoML) has been successful when the learning task is assumed to be static. However, it remains unclear whether AutoML methods can efficiently create online pipelines in dynamic environments. The current online AutoML frameworks primarily focus on supervised learning. However, unsupervised learning, particularly clustering, also requires AutoML solutions, especially with the ambiguity associated with evaluating clustering results. In this paper, we introduce OnlineAutoClust, a framework for online automated clustering for algorithm selection and hyperparameter tuning. OnlineAutoClust combines the inherent adaptation capabilities of online learners with automated pipeline optimization using Bayesian optimization. OnlineAutoClust develops a collaborative mechanism based on clustering ensemble to combine optimized pipelines based on different internal cluster validity indices. The proposed framework is based on River library and utilizes five clustering algorithms. Empirical evaluation on several real and synthetic data streams with varying types of concept drift demonstrates the effectiveness of the proposed approach compared to existing methods
DAE: Distribution-Aware Embedding for Numerical Features in Click-Through Rate Prediction
Authors: Bin Shen, Jingran Xu, Xu Min, Zeyu Ke, Yong He, Liang Zhang, Xin Dong, Linjian Mo
Venue: CIKM
Year: 2023

Numerical features are an important type of input for CTR prediction models. Recently, several discretization and numerical transformation methods have been proposed to deal with numerical features. However, existing approaches do not fully consider compatibility with different distributions. Here, we propose a novel numerical feature embedding framework, called Distribution-Aware Embedding (DAE), which is applicable to various numerical feature distributions. First, DAE efficiently approximates the cumulative distribution function by estimating the expectation of the order statistics. Then, the distribution information is applied to the embedding layer by nonlinear interpolation. Finally, to capture both local and global information, we aggregate the embeddings at multiple scales to obtain the final representation. Empirical results validate the effectiveness of DAE compared to the baselines, while demonstrating the adaptability to different CTR models and distributions.
STREAMS: Towards Spatio-Temporal Causal Discovery with Reinforcement Learning for Streamflow Rate Prediction
Authors: Paras Sheth, Ahmadreza Mosallanezhad, Kaize Ding, Reepal Shah, John Sabo, Huan Liu, K. Selçuk Candan
Venue: CIKM
Year: 2023

The capacity to anticipate streamflow is critical to the efficient functioning of reservoir systems as it gives vital information to reservoir operators about water release quantities as well as help quantify the impact of environmental factors on downstream water quality. Yet, streamflow modelling is difficult owing to the intricate interactions between different watershed outlets. In this paper, we argue that one possible solution to this problem is to identify the causal structure of these outlets, which would allow for the identification of crucial watershed outlets while capturing the spatiotemporally informed complex relationships leading to improved hydrological resource management. However, due to the inherent complexity of spatiotemporal causal learning problems, extending existing causal discovery methods to a whole basin is a major hurdle. To address these issues, we offer STREAMS, a new framework that uses Reinforcement Learning (RL) to optimize the search space for causal discovery and an LSTM-GCN based autoencoder to infer spatiotemporal causal features for streamflow rate prediction. We conduct extensive experiments on the Brazos river basin carried out within the scope of a US Army Corps of Engineers, Engineering With Nature Initiative project, including empirical studies of generalization performance to verify the nature of the inferred relationships.
Learning to Simulate Complex Physical Systems: A Case Study
Authors: Jiasheng Shi, Fu Lin, Weixiong Rao
Venue: CIKM
Year: 2023

Complex physical system simulation is important in many real world applications. We study the general simulation scenario to generate the response result when a physical object is applied by external factors. Traditional solvers on Partial Differential Equations (PDEs) suffer from significantly high computational cost. Many recent learning-based approaches focus on multivariate time series alike simulation prediction problem and do not work for our case. In this paper, we propose a novel two-level graph neural networks (GNNs) to learn the simulation result of a physical object applied by external factors. The key is a two-level graph structure where one fine mesh graph is mapped to multiple coarse one. Our preliminary evaluation on both synthetic and real datasets demonstrates that our work outperforms three state-of-the-arts by much lower errors.
Improving Graph Domain Adaptation with Network Hierarchy
Authors: Boshen Shi, Yongqing Wang, Fangda Guo, Jiangli Shao, Huawei Shen, Xueqi Cheng
Venue: CIKM
Year: 2023

Graph domain adaptation models have become instrumental in addressing cross-network learning problems due to their ability to transfer abundant label and structural knowledge from source graphs to target graphs. A crucial step in transfer involves measuring domain discrepancy, which refers to distribution shifts between graphs from source and target domains. While conventional models simply provide a node-level measurement, exploiting information from different levels of network hierarchy is intuitive. As each hierarchical level characterizes distinct and meaningful properties or functionalities of the original graph, integrating domain discrepancy based on such hierarchies should contribute to a more precise domain discrepancy measurement. Moreover, class conditional distribution shift is often overlooked in node classification tasks, which could potentially lead to sub-optimal performance. To address the above limitations, we propose a new graph domain adaptation model and apply it to cross-network node classification tasks. Specifically, a hierarchical pooling model to extract meaningful and adaptive hierarchical structures is designed, where both marginal and class conditional distribution shifts on each hierarchical level are jointly minimized. The effectiveness is demonstrated through theoretical analysis and experimental studies across various datasets.
OpenGDA: Graph Domain Adaptation Benchmark for Cross-network Learning
Authors: Boshen Shi, Yongqing Wang, Fangda Guo, Jiangli Shao, Huawei Shen, Xueqi Cheng
Venue: CIKM
Year: 2023

Graph domain adaptation models are widely adopted in cross-network learning tasks to transfer labeling or structural knowledge. Currently, there mainly exist two limitations in evaluating graph domain adaptation models. On one side, they are primarily tested for the specific cross-network node classification task, leaving tasks at edge-level and graph-level largely under-explored. Moreover, they are primarily examined in limited scenarios, such as social networks or citation networks, needing more validation in richer scenarios. As comprehensively assessing models could enhance model practicality in real-world applications, we propose a benchmark known as OpenGDA. It provides abundant pre-processed and unified datasets for different types of tasks (node, edge, graph). They originate from diverse scenarios, covering web information systems, urban systems and natural systems. Furthermore, it integrates state-of-the-art models with standardized and end-to-end pipelines. Overall, OpenGDA provides a user-friendly, scalable and reproducible benchmark for evaluating graph domain adaptation models. The benchmark experiments highlight the challenges of applying GDA models to real-world applications with consistent good performance, and they potentially provide insights to future research. As an emerging project, OpenGDA will be regularly updated with new datasets and models. It could be accessed from https://github.com/Skyorca/OpenGDA.
Intersectional Bias Mitigation in Pre-trained Language Models: A Quantum-Inspired Approach
Authors: Omid Shokrollahi
Venue: CIKM
Year: 2023

The growing criticality of contextualized language models has raised concerns about the perpetuation of biases. Current fairness research often concentrates on single aspects of social groups. However, our study brings a multifaceted approach to the table, reflecting the intricate realities of intersectionality. We propose a bias mitigation algorithm inspired by quantum theory to fine-tune pre-trained language models. Our method applies Jensen-Shannon divergence alongside an entanglement entropy measure to quantify the complexity of entwined identities. Moreover, we utilize an innovative entanglement embedding neural network to address emergent features from different intersectional groups. An Actor-Critic setup facilitates effective fine-tuning. Our approach broadens the scope of intersectional fairness beyond just statistical parity, providing a strategic tool to tackle complex, interrelated biases. We anticipate that this fresh approach to bias mitigation will substantially enhance fairness in a wide range of language model applications.
Combating Ad Fatigue via Frequency-Recency Features in Online Advertising Systems
Authors: Natalia Silberstein, Or Shoham, Assaf Klein
Venue: CIKM
Year: 2023

Online advertising is a driving force of Internet services today. One of the main challenges in advertising systems is finding the right balance between user experience and overall revenue. In this paper we address one of the problems that negatively impacts the user experience, specifically, the repeated display of identical ads to the same user. This problem leads to the phenomenon called "ad fatigue", characterized by diminished interest in the ad, resulting in a decrease in the click-through rate (CTR) as users encounter the same ad repeatedly. Naive solutions such as placing a hard limit on the number of times a specific ad is displayed to a specific user, usually come at the cost of reduced revenue. To address the ad fatigue problem, we introduce a new family of features, called FoRI (Frequency over Recent Intervals). FoRI features integrate information about frequency and recency of previous user-ad interactions within the CTR prediction model. This approach involves allocating these interactions to unevenly distributed time intervals, enabling a higher emphasis on more recent interactions. Furthermore, we introduce new metrics to assess ad fatigue in terms of the repetitiveness and novelty of the displayed ads. We conducted a comprehensive large-scale online evaluation which shows that integrating FoRI features into our CTR prediction model offers two-fold benefits. Firstly, it improves user experience by reducing the occurrence of repeated ads by 15%, and increasing the exposure to unseen ads by 5% (ads not previously displayed to the user), leading to a substantial boost in CTR. Secondly, it significantly increases revenue.
EmFore: Online Learning of Email Folder Classification Rules
Authors: Mukul Singh, José Cambronero, Sumit Gulwani, Vu Le, Gust Verbruggen
Venue: CIKM
Year: 2023

Modern email clients support predicate-based folder assignment rules that can automatically organize emails. Unfortunately, users still need to write these rules manually. Prior machine learning approaches have framed automatically assigning email to folders as a classification task and do not produce symbolic rules. Prior inductive logic programming (ILP) approaches, which generate symbolic rules, fail to learn efficiently in the online environment needed for email management. To close this gap, we present EmFORE, an online system that learns symbolic rules for email classification from observations. Our key insights to do this successfully are: (1) learning rules over a folder abstraction that supports quickly determining candidate predicates to add or replace terms in a rule, (2) ensuring that rules remain consistent with historical assignments, (3) ranking rule updates based on existing predicate and folder name similarity, and (4) building a rule suppression model to avoid surfacing low-confidence folder predictions while keeping the rule for future use. We evaluate on two popular public email corpora and compare to 13 baselines, including state-of-the-art folder assignment systems, incremental machine learning, ILP and transformer-based approaches. We find that EmFORE performs significantly better, updates four orders of magnitude faster, and is more robust than existing methods and baselines.