OpenRaiser/Intern-Atlas huggingface dataset:
DyGen: Learning from Noisy Labels via Dynamics-Enhanced Generative Modeling
Authors: Yuchen Zhuang, Yue Yu 0001, Lingkai Kong, Xiang Chen 0010, Chao Zhang 0014
Venue: KDD
Year: 2023

Learning from noisy labels is a challenge that arises in many real-world applications where training data can contain incorrect or corrupted labels. When fine-tuning language models with noisy labels, models can easily overfit the label noise, leading to decreased performance. Most existing methods for learning from noisy labels use static input features for denoising, but these methods are limited by the information they can provide on true label distributions and can result in biased or incorrect predictions. In this work, we propose the Dynamics-Enhanced Generative Model (DyGen), which uses dynamic patterns in the embedding space during the fine-tuning process of language models to improve noisy label predictions. DyGen uses the variational auto-encoding framework to infer the posterior distributions of true labels from noisy labels and training dynamics. Additionally, a co-regularization mechanism is used to minimize the impact of potentially noisy labels and priors. DyGen demonstrates an average accuracy improvement of 3.10% on two synthetic noise datasets and 1.48% on three real-world noise datasets compared to the previous state-of-the-art. Extensive experiments and analyses show the effectiveness of each component in DyGen. Our code is available for reproducibility on GitHub.
ReCTSi: Resource-efficient Correlated Time Series Imputation via Decoupled Pattern Learning and Completeness-aware Attentions
Authors: Zhichen Lai 0001, Dalin Zhang 0001, Huan Li 0003, Dongxiang Zhang, Hua Lu 0001, Christian S. Jensen
Venue: KDD
Year: 2024

Imputation of Correlated Time Series (CTS) is essential in data preprocessing for many tasks, particularly when sensor data is often incomplete. Deep learning has enabled sophisticated models that improve CTS imputation by capturing temporal and spatial patterns. However, deep models often incur considerable consumption of computational resources and thus cannot be deployed in resource-limited settings. This paper presents ReCTSi (Resource-efficient CTS imputation), a method that adopts a new architecture for decoupled pattern learning in two phases: (1) the Persistent Pattern Extraction phase utilizes a multi-view learnable codebook mechanism to identify and archive persistent patterns common across different time series, enabling rapid pattern retrieval during inference. (2) the Transient Pattern Adaptation phase introduces completeness-aware attention modules that allocate attention to the complete and hence more reliable data segments. Extensive experimental results show that ReCTSi achieves state-of-the-art imputation accuracy while consuming much fewer computational resources than the leading existing model, consuming only 0.004% of the FLOPs for inference compared to its closest competitor. The blend of high accuracy and very low resource consumption makes ReCTSi the currently best method for resource-limited scenarios. The related code is available at https://github.com/ryanlaics/RECTSI.
Revisiting Local PageRank Estimation on Undirected Graphs: Simple and Optimal
Authors: Hanzhi Wang 0001
Venue: KDD
Year: 2024

We propose a simple and optimal algorithm, BackMC, for local PageRank estimation in undirected graphs: given an arbitrary target node t in an undirected graph G comprising n nodes and m edges, BackMC accurately estimates the PageRank score of node t while assuring a small relative error and a high success probability. The worst-case computational complexity of BackMC is upper bounded by O(1/dmin ⋅ min(dt, m1/2)), where dmin denotes the minimum degree of G, and dt denotes the degree of t, respectively. Compared to the previously best upper bound of O(log n ⋅ min(dt, m1/2)) (VLDB '23), which is derived from a significantly more complex algorithm and analysis, our BackMC improves the computational complexity for this problem by a factor of Θ(log n/dmin) with a much simpler algorithm. Furthermore, we establish a matching lower bound of Ω(1/dmin ⋅ min(dt, m1/2)) for any algorithm that attempts to solve the problem of local PageRank estimation, demonstrating the theoretical optimality of our BackMC. We conduct extensive experiments on various large-scale real-world and synthetic graphs, where BackMC consistently shows superior performance.
Generative AI Day
Authors: Jie Tang 0001, Yuxiao Dong, Michalis Vazirgiannis
Venue: KDD
Year: 2024

The Generative AI (AIGC) Day at KDD'24 is a dedicated full-day event for generative AI at KDD. This is an opportunity to bring together researchers, practitioners, and startups to share the insights about the cutting-edge advancements and to discuss the potential societal impacts of LLMs and AIGC. It is exciting that this year, we have invited speakers from both industry (e.g., Amazon, Zhipu AI) and academia (e.g., USC, UCLA). The topics cover various perspectives of generative AI including foundation models, streaming LLMs, LLM training and inference. As demonstrated, data plays a crucial role in developing cutting-edge generative AI models. For example, the Gemini Team has found that "data quality is an important factor for highly-performing models...''. To date, there is still significant room to define design principles and develop methods for improved data collection, selection, and synthetic data generation for the pre-training and alignment of language, vision, and multi-modal models. Therefore, the Day will invite the speakers and KDD audience to discuss the challenges and opportunities for data mining researchers in the era of generative AI.
A Tutorial on Multi-Armed Bandit Applications for Large Language Models
Authors: Djallel Bouneffouf 0001, Raphaël Féraud
Venue: KDD
Year: 2024

This tutorial offers a comprehensive guide on using multi-armed bandit (MAB) algorithms to improve Large Language Models (LLMs). As Natural Language Processing (NLP) tasks grow, efficient and adaptive language generation systems are increasingly needed. MAB algorithms, which balance exploration and exploitation under uncertainty, are promising for enhancing LLMs.
23rd International Workshop on Data Mining in Bioinformatics (BIOKDD 2024)
Authors: Da Yan 0001, Ahmed Abdeen Hamed, Jake Y. Chen, Mohammed J. Zaki
Venue: KDD
Year: 2024

The goal of the 22nd International Workshop on Data Mining in Bioinformatics (BIOKDD 2023) is to encourage KDD researchers to solve the numerous problems and challenges in Bioinformatics using Data Mining technologies. Based on the organizers' expertise and communities, BIOKDD 2023 features the theme "Large-Scale Data-Driven Methods for Bioinformatics". This theme encourages the use of high-performance computing (HPC) to support the training of large machine learning models for problems in Bioinformatics and Computational Biology. The key goal is to accelerate the convergence between Data Mining and Bioinformatics communities to expedite discoveries in basic biology, medicine and healthcare.The goal of the 23rd International Workshop on Data Mining in Bioinformatics (BIOKDD 2024) is to encourage KDD researchers to solve the numerous problems and challenges in Bioinformatics using Data Mining technologies. Based on the organizers' expertise and communities, BIOKDD 2024 features the theme "Advancing Bioinformatics with LLMs and GenAI". This theme encourages the use of large language models and generative artificial intelligence to solve problems in Bioinformatics and Computational Biology. The key goal is to accelerate the convergence between Data Mining and Bioinformatics communities to expedite discoveries in basic biology, medicine and healthcare.
The Fourth International Workshop on Smart Data for Blockchain and Distributed Ledger (SDBD'24)
Authors: Feida Zhu 0001, Jian Pei 0001, Michael Zeller, Bingxue Zhang
Venue: KDD
Year: 2024

With the advent of Bitcoin, a cryptographically-enabled peer-to-peer digital payment system, blockchain together with a whole package of distributed ledger technologies, which serve as the underlying foundation of all the crypto-currencies, have been gaining attention from both academia and industry in the last fifteen years. The recent years have witnessed tremendous momentum in the development of blockchain and distributed ledger technologies, largely due to the impressive rise in the market capital of these digital tokens. More and more industries, from banking and insurance, to supply chain and e-commerce, are quickly realizing the great potential in blockchain technology in efficiency boost, process automation and secure data sharing across otherwise isolated data silos. Furthermore, as the recognition of the data value began to sink in, data assets has become an essential part of the development of enterprises and countries. Blockchain technology is regarded as the foundation of digital economy and provides an effective approach for data ownership, pricing and transactions, which are the core issues of data asset management. However, the potential implications of Blockchain technologies go far beyond their application as the technological backbone for cryptocurrencies. Web3.0, using blockchain as underlying technology, allow for various novel application scenarios, which are built upon distributed consensus and thus are hard to block or censor while providing public verifiability of peer-to-peer transactions without a trusted central party. Web3.0 are expected to become the main front for a plethora of highly expressive applications. To more thoroughly explore the potential of blockchain and web3.0 and promote their progress, SDBD'24 will provide a forum for the most recent blockchain and web3.0 research, innovations, and applications, bridging the gap between theory and practice in the design.
AutoXPCR: Automated Multi-Objective Model Selection for Time Series Forecasting
Authors: Raphael Fischer 0001, Amal Saadallah
Venue: KDD
Year: 2024

Automated machine learning (AutoML) streamlines the creation of ML models, but few specialized methods have approached the challenging domain of time series forecasting. Deep neural networks (DNNs) often deliver state-of-the-art predictive performance for forecasting data, however these models are also criticized for being computationally intensive black boxes. As a result, when searching for the "best" model, it is crucial to also acknowledge other aspects, such as interpretability and resource consumption. In this paper, we propose AutoXPCR - a novel method that produces DNNs for forecasting under consideration of multiple objectives in an automated and explainable fashion. Our approach leverages meta-learning to estimate any model's performance along PCR criteria, which encompass (P)redictive error, (C)omplexity, and (R)esource demand. Explainability is addressed on multiple levels, as AutoXPCR pro-vides by-product explanations of recommendations and allows to interactively control the desired PCR criteria importance and trade-offs. We demonstrate the practical feasibility AutoXPCR across 108 forecasting data sets from various domains. Notably, our method outperforms competing AutoML approaches - on average, it only requires 20% of computation costs for recommending highly efficient models with 85% of the empirical best quality.
Shopping Trajectory Representation Learning with Pre-training for E-commerce Customer Understanding and Recommendation
Authors: Yankai Chen 0001, Quoc-Tuan Truong, Xin Shen, Jin Li 0003, Irwin King
Venue: KDD
Year: 2024

Understanding customer behavior is crucial for improving service quality in large-scale E-commerce. This paper proposes C-STAR, a new framework that learns compact representations from customer shopping journeys, with good versatility to fuel multiple downstream customer-centric tasks. We define the notion of shopping trajectory that encompasses customer interactions at the level of product categories, capturing the overall flow of their browsing and purchase activities. C-STAR excels at modeling both inter-trajectory distribution similarity-the structural similarities between different trajectories, and intra-trajectory semantic correlation-the semantic relationships within individual ones. This coarse-to-fine approach ensures informative trajectory embeddings for representing customers. To enhance embedding quality, we introduce a pre-training strategy that captures two intrinsic properties within the pre-training data. Extensive evaluation on large-scale industrial and public datasets demonstrates the effectiveness of C-STAR across three diverse customer-centric tasks. These tasks empower customer profiling and recommendation services for enhancing personalized shopping experiences on our E-commerce platform.
Hypformer: Exploring Efficient Transformer Fully in Hyperbolic Space
Authors: Menglin Yang 0001, Harshit Verma, Delvin Ce Zhang, Jiahong Liu 0001, Irwin King, Rex Ying
Venue: KDD
Year: 2024

Hyperbolic geometry have shown significant potential in modeling complex structured data, particularly those with underlying tree-like and hierarchical structures. Despite the impressive performance of various hyperbolic neural networks across numerous domains, research on adapting the Transformer to hyperbolic space remains limited. Previous attempts have mainly focused on modifying self-attention modules in the Transformer. However, these efforts have fallen short of developing a complete hyperbolic Transformer. This stems primarily from: (i) the absence of well-defined modules in hyperbolic space, including linear transformation layers, LayerNorm layers, activation functions, dropout operations, etc. (ii) the quadratic time complexity of the existing hyperbolic self-attention module w.r.t the number of input tokens, which hinders its scalability. To address these challenges, we propose, Hypformer, a novel hyperbolic Transformer based on the Lorentz model of hyperbolic geometry. In Hypformer, we introduce two foundational blocks that define the essential modules of the Transformer in hyperbolic space. Furthermore, we develop a linear self-attention mechanism in hyperbolic space, enabling hyperbolic Transformer to process billion-scale graph data and long-sequence inputs for the first time. Our experimental results confirm the effectiveness and efficiency of \method across various datasets, demonstrating its potential as an effective and scalable solution for large-scale data representation and large models.
Systems for Scalable Graph Analytics and Machine Learning: Trends and Methods
Authors: Da Yan 0001, Lyuheng Yuan, Akhlaque Ahmad, Chenguang Zheng, Hongzhi Chen, James Cheng
Venue: KDD
Year: 2024

Graph-theoretic algorithms and graph machine learning models are essential tools for addressing many real-life problems, such as social network analysis and bioinformatics. To support large-scale graph analytics, graph-parallel systems have been actively developed for over one decade, such as Google's Pregel and Spark's GraphX, which (i) promote a think-like-a-vertex computing model and target (ii) iterative algorithms and (iii) those problems that output a value for each vertex. However, this model is too restricted for supporting the rich set of heterogeneous operations for graph analytics and machine learning that many real applications demand. In recent years, two new trends emerge in graph-parallel systems research: (1) a novel think-like-a-task computing model that can efficiently support the various computationally expensive problems of subgraph search; and (2) scalable systems for learning graph neural networks. These systems effectively complement the diversity needs of graph-parallel tools that can flexibly work together in a comprehensive graph processing pipeline for real applications, with the capability of capturing structural features. This tutorial will provide an effective categorization of the recent systems in these two directions based on their computing models and adopted techniques, and will review the key design ideas of these systems.
The 4th KDD Workshop on Deep Learning for Spatiotemporal Data, Applications, and Systems (DeepSpatial'24)
Authors: Zhe Jiang 0001, Liang Zhao 0002, Xun Zhou 0001, Junbo Zhang 0004, Shashi Shekhar 0001, Jieping Ye
Venue: KDD
Year: 2024

Over the last decades, a rapidly growing volume of spatiotemporal data has been collected from smartphones and GPS, terrestrial, seaborne, airborne, and spaceborne sensors, as well as computational simulations. Meanwhile, advances in deep learning technologies, especially the recent breakthroughs of generative AI and foundation models such as Large Language Models (LLMs) and Large Vision Models (LVMs), have achieved tremendous success in natural language processing and computer vision applications. There is growing anticipation of the same level of accomplishment of AI on spatiotemporal data in tackling grand societal challenges, such as national water resource management, monitoring coastal hazards, energy and food security, as well as mitigation and adaptation to climate change. When deep learning, especially emerging foundation models, intersects spatiotemporal data in scientific domains, it opens up new opportunities and challenges. The workshop aims to bring together academic researchers in both AI and scientific domains, government program managers, leaders from non-profit organizations, as well as industry executives to brainstorm and debate on the emerging opportunities and novel challenges of deep learning (foundation models) for spatiotemporal data inspired by real-world scientific applications.
Measuring an LLM's Proficiency at using APIs: A Query Generation Strategy
Authors: Ying Sheng 0002, Sudeep Gandhe, Bhargav Kanagal, Nick Edmonds, Zachary Fisher, Sandeep Tata, Aarush Selvan
Venue: KDD
Year: 2024

Connecting Large Language Models (LLMs) with the ability to leverage APIs (Web Search, Charting, Calculators, Calendar, Flight Search, Hotel Search, Data Lookup, etc. ) is likely to allow us to solve a variety of new hard problems. Several research efforts have made this observation and suggested recipes for LLMs to emit API calls, and proposed mechanisms by which they can generate additional text conditioned on the output for the API call. However, in practice, the focus has been on relatively simple slot-filling tasks that make an API call rather unlocking novel capabilities by combining different tools, reasoning over the response from a tool, making multiple invocations, or complex planning. In this paper, we pose the following question: what does it mean to say that an LLM is proficient at using a set of APIs? We answer this question in the context of structured APIs by defining seven capabilities for API-use. We provide an approach for generating synthetic tasks that exercise each of these capabilities given only the description of an API. We argue that this provides practitioners with a principled way to construct a dataset to evaluate an LLM's ability to use a given set of APIs. Through human evaluations, we show that our approach produces high-quality tasks for each of the seven capabilities. We also describe how we used this approach to on-board new API and create principled evaluation sets for multiple LLM-based products.
FAST: An Optimization Framework for Fast Additive Segmentation in Transparent ML
Authors: Brian Liu 0002, Rahul Mazumder
Venue: KDD
Year: 2024

We present FAST, an optimization framework for fast additive segmentation. FAST segments piecewise constant shape functions for each feature in a dataset to produce transparent additive models. The framework leverages a novel optimization procedure to fit these models ~2 orders of magnitude faster than existing state-of-the-art methods, such as explainable boosting machines[20]. We also develop new feature selection algorithms in the FAST framework to fit parsimonious models that perform well. Through experiments and case studies, we show that FAST improves the computational efficiency and interpretability of additive models.
Topology-aware Embedding Memory for Continual Learning on Expanding Networks
Authors: Xikun Zhang 0002, Dongjin Song, Yixin Chen 0001, Dacheng Tao
Venue: KDD
Year: 2024

Memory replay based techniques have shown great success for continual learning with incrementally accumulated Euclidean data. Directly applying them to continually expanding networks, however, leads to the potential memory explosion problem due to the need to buffer representative nodes and their associated topological neighborhood structures. To this end, we systematically analyze the key challenges in the memory explosion problem, and present a general framework,i.e., Parameter Decoupled Graph Neural Networks (PDGNNs) with Topology-aware Embedding Memory (TEM), to tackle this issue. The proposed framework not only reduces the memory space complexity from O (ndL) to O (n)1: memory budget, d: average node degree, L: the radius of the GNN receptive field, but also fully utilizes the topological information for memory replay. Specifically, PDGNNs decouple trainable parameters from the computation ego-subnetwork viaTopology-aware Embeddings (TEs), which compress ego-subnetworks into compact vectors (i.e., TEs) to reduce the memory consumption. Based on this framework, we discover a unique pseudo-training effect in continual learning on expanding networks and this effect motivates us to develop a novel coverage maximization sampling strategy that can enhance the performance with a tight memory budget. Thorough empirical studies demonstrate that, by tackling the memory explosion problem and incorporating topological information into memory replay, PDGNNs with TEM significantly outperform state-of-the-art techniques, especially in the challenging class-incremental setting.
Unveiling Vulnerabilities of Contrastive Recommender Systems to Poisoning Attacks
Authors: Zongwei Wang 0002, Junliang Yu, Min Gao 0001, Hongzhi Yin, Bin Cui 0001, Shazia Sadiq
Venue: KDD
Year: 2024

Contrastive learning (CL) has recently gained prominence in the domain of recommender systems due to its great ability to enhance recommendation accuracy and improve model robustness. Despite its advantages, this paper identifies a vulnerability of CL-based recommender systems that they are more susceptible to poisoning attacks aiming to promote individual items. Our analysis indicates that this vulnerability is attributed to the uniform spread of representations caused by the InfoNCE loss. Furthermore, theoretical and empirical evidence shows that optimizing this loss favors smooth spectral values of representations. This finding suggests that attackers could facilitate this optimization process of CL by encouraging a more uniform distribution of spectral values, thereby enhancing the degree of representation dispersion. With these insights, we attempt to reveal a potential poisoning attack against CL-based recommender systems, which encompasses a dual-objective framework: one that induces a smoother spectral value distribution to amplify the InfoNCE loss's inherent dispersion effect, named dispersion promotion; and the other that directly elevates the visibility of target items, named rank promotion. We validate the threats of our attack model through extensive experimentation on four datasets. By shedding light on these vulnerabilities, our goal is to advance the development of more robust CL-based recommender systems. The code is available at https://github.com/CoderWZW/ARLib.