OpenRaiser/Intern-Atlas huggingface dataset:
Synthetic Survey Data Generation and Evaluation
Authors: Yanru Jiang, Siyu Liang, Junwon Choi
Venue: KDD
Year: 2025

Survey data are common and invaluable in social science research for understanding population processes and supporting policymaking and planning. Depending on the nature and scale, survey data sharing comes with privacy risks, and data collectors and agencies are constrained by disclosure permissions, limiting usage across research groups and institutes. Previous methods for synthetic data generation and deidentification may not entirely prevent information disclosures, or they may sacrifice data quality and granularity. Using a large-scale national voter file at both national and state levels, this paper introduces an end-to-end pipeline to streamline synthetic data generation and evaluation for survey researchers. This study selects four generative approaches based on different statistical assumptions: the regression-based Synthpop, the generative deep learning-based CTGAN and TVAE, and the large language model-based REaLDTabFormer, and compares them to the baseline synthetic minority oversampling technique (SMOTE). We consider three key dimensions of evaluation (utility, fidelity, and privacy) to highlight the strengths and weaknesses of each approach, and systematically evaluate across various datasets and training sizes. The results reveal that Synthpop is optimized for general utility (i.e., fidelity), while TVAE excels in downstream applications (i.e., target-specific utility) but compromises on general utility and potentially risks data overfitting. REaLDTabFormer demonstrates a balanced performance in both general and target-specific utility, whereas CTGAN offers the best privacy protection. We recommend that future researchers select a generative method by considering the trade-offs between performance across various evaluation dimensions, training size, data type, and computational infrastructure.
Retrieval And Structuring Augmented Generation with Large Language Models
Authors: Pengcheng Jiang, Siru Ouyang, Yizhu Jiao, Ming Zhong, Runchu Tian, Jiawei Han
Venue: KDD
Year: 2025

Large Language Models (LLMs) have revolutionized natural language processing with their remarkable capabilities in text generation and reasoning. However, these models face critical challenges when deployed in real-world applications, including hallucination generation, outdated knowledge, and limited domain expertise. Retrieval And Structuring (RAS) Augmented Generation addresses these limitations by integrating dynamic information retrieval with structured knowledge representations. This survey (1) examines retrieval mechanisms including sparse, dense, and hybrid approaches for accessing external knowledge; (2) explore text structuring techniques such as taxonomy construction, hierarchical classification, and information extraction that transform unstructured text into organized representations; and (3) investigate how these structured representations integrate with LLMs through prompt-based methods, reasoning frameworks, and knowledge embedding techniques. It also identifies technical challenges in retrieval efficiency, structure quality, and knowledge integration, while highlighting research opportunities in multimodal retrieval, cross-lingual structures, and interactive systems. This comprehensive overview provides researchers and practitioners with insights into RAS methods, applications, and future directions.
HiBench: Benchmarking LLMs Capability on Hierarchical Structure Reasoning
Authors: Zhuohang Jiang, Pangjing Wu, Ziran Liang, Peter Q. Chen, Xu Yuan, Ye Jia, Jiancheng Tu, Chen Li, Peter H. F. Ng, Qing Li
Venue: KDD
Year: 2025

Structure reasoning is a fundamental capability of large language models (LLMs), enabling them to reason about structured commonsense and answer multi-hop questions. However, existing benchmarks for structure reasoning mainly focus on horizontal and coordinate structures (e.g. graphs), overlooking the hierarchical relationships within them. Hierarchical structure reasoning is crucial for human cognition, particularly in memory organization and problem-solving. It also plays a key role in various real-world tasks, such as information extraction and decision-making. To address this gap, we propose HiBench, the first framework designed to systematically benchmark the hierarchical reasoning capabilities of LLMs from initial structure generation to final proficiency assessment. It encompasses six representative scenarios, covering both fundamental and practical aspects, and consists of 30 tasks with varying hierarchical complexity, totaling 39,519 queries. To evaluate LLMs comprehensively, we develop five capability dimensions that depict different facets of hierarchical structure understanding. Through extensive evaluation of 20 LLMs from 10 model families, we reveal key insights into their capabilities and limitations: 1) existing LLMs show proficiency in basic hierarchical reasoning tasks; 2) they still struggle with more complex structures and implicit hierarchical representations, especially in structural modification and textual reasoning. Based on these findings, we create a small yet well-designed instruction dataset, which enhances LLMs' performance on HiBench by an average of 88.84% (Llama-3.1-8B) and 31.38% (Qwen2.5-7B) across all tasks. The HiBench dataset and toolkit are available at https://github.com/jzzzzh/HiBench to encourage evaluation.
PipeRAG: Fast Retrieval-Augmented Generation via Adaptive Pipeline Parallelism
Authors: Wenqi Jiang, Shuai Zhang, Boran Han, Jie Wang, Bernie Wang, Tim Kraska
Venue: KDD
Year: 2025

Retrieval-augmented generation (RAG) can enhance the generation quality of large language models (LLMs) by incorporating external token databases. However, retrievals from large databases can constitute a substantial portion of the overall generation time, particularly when retrievals are periodically performed to align the retrieved content with the latest states of generation. In this paper, we introduce PipeRAG, a novel algorithm-system co-design approach to reduce generation latency and enhance generation quality. PipeRAG integrates (1) pipeline parallelism to enable concurrent retrieval and generation processes, (2) flexible retrieval intervals to maximize the efficiency of pipeline parallelism, and (3) a performance model to automatically balance retrieval quality and latency based on the generation states and underlying hardware. Our evaluation shows that, by combining the three aforementioned methods, PipeRAG achieves up to 2.6× speedup in end-to-end generation latency while improving generation quality. These promising results showcase the effectiveness of co-designing algorithms with underlying systems, paving the way for the adoption of PipeRAG in future RAG systems.
Urban Region Pre-training and Prompting: A Graph-based Approach
Authors: Jiahui Jin, Yifan Song, Dong Kan, Haojia Zhu, Xiangguo Sun, Zhicheng Li, Xigang Sun, Jinghui Zhang
Venue: KDD
Year: 2025

Urban region representation is crucial for various urban downstream tasks. However, despite the proliferation of methods and their success, acquiring general urban region knowledge and adapting to different tasks remains challenging. Existing work pays limited attention to the fine-grained functional layout semantics in urban regions, limiting their ability to capture transferable knowledge across regions. Further, inadequate handling of the unique features and relationships required for different downstream tasks may also hinder effective task adaptation. In this paper, we propose a Graph-based Urban Region Pre-training and Prompting framework (GURPP) for region representation learning. Specifically, we first construct an urban region graph and develop a subgraph-centric urban region pre-training model to capture the heterogeneous and transferable patterns of entity interactions. This model pre-trains knowledge-rich region embeddings using contrastive learning and multi-view learning methods. To further refine these representations, we design two graph-based prompting methods: a manually-defined prompt to incorporate explicit task knowledge and a task-learnable prompt to discover hidden knowledge, which enhances the adaptability of these embeddings to different tasks. Extensive experiments on various urban region prediction tasks and different cities demonstrate the superior performance of our framework.
Experience Speaks Louder: Black-box Hard-label Adversarial Attack through Reinforcement Learning
Authors: Yilun Jin, Kun Zhu, Feng Tang, Jiaxuan Shi, Yong Chen
Venue: KDD
Year: 2025

Black-box hard-label adversarial attack on text can be used to evaluate the robustness of text classification models (TCMs), allowing for a more thorough examination of potential security flaws before deployment. Research on this problem is still in the embryonic stage and only a few methods are available. Existing adversarial attack methods often treat each adversarial text generation as an independent task, failing to leverage past adversarial experience to facilitate subsequent generation. Consequently, these methods suffer from high query costs (i.e., the number of times TCMs need to be accessed to generate adversarial text) and pose challenges in generating adversarial texts with high semantic similarity and low perturbation rates under a query-limited setup. To achieve query-efficient adversarial text generation, we propose EGAttack, an experience-guided black-box hard-label attack method based on reinforcement learning. Specifically, we models adversarial text generation as a sequential decision process and propose a new adversarial environment to capture the intricate relationships among word perturbations, text semantic, and the target model's outputs for agent learning. Meanwhile, we extend the classical reinforcement framework DDQN into adversarial DDQN (ADDQN) through designing a new adversarial agent and a new dual-objective constraint, driving EGAttack to learn from past adversarial experience while identifying the vulnerabilities of TCMs, thus generating high-quality adversarial texts. Experimental results across various target models and datasets demonstrate that EGAttack generates adversarial texts with higher semantic similarity, lower perturbation rates, and competitive attack success rate, making it an efficient and effective tool for evaluating the robustness of TCMs.
Why Not Together? A Multiple-Round Recommender System for Queries and Items
Authors: Jiarui Jin, Xianyu Chen, Weinan Zhang, Yong Yu, Jun Wang
Venue: KDD
Year: 2025

A fundamental technique of recommender systems involves modeling user preferences, where queries and items are widely used as symbolic representations of user interests. Queries delineate user needs at an abstract level, providing a high-level description, whereas items operate on a more specific and concrete level, representing the granular facets of user preference. While practical, both query and item recommendations encounter the challenge of sparse user feedback. To this end, we propose a novel approach named Multiple-round Auto Guess-and-Update System (MAGUS) that capitalizes on the synergies between both types, allowing us to leverage both query and item information to form user interests. This integrated system introduces a recursive framework that could be applied to any recommendation method to exploit queries and items in historical interactions and to provide recommendations for both queries and items in each interaction round. Concretely, MAGUS first represents queries and items through combinations of categorical words, and then constructs a relational graph to capture the interconnections and dependencies among these individual words and word combinations. In response to each user request, MAGUS employs an offline tuned recommendation model to assign estimated scores to words representing items; and these scores are subsequently disseminated throughout the graph, impacting each individual word or combination of words. Through multiple-round interactions, MAGUS initially guesses user interests by formulating meaningful word combinations and presenting them as potential queries or items. Subsequently, MAGUS is updated based on user feedback, enhancing its recommendations iteratively. Empirical results from testing 12 different recommendation methods demonstrate that integrating queries into item recommendations via MAGUS significantly enhances the efficiency, with which users can identify their preferred items during multiple-round interactions.
Large Vison-Language Foundation Model in Baidu AIGC Image Advertising
Authors: Zhipeng Jin, Wen Tao, Yafei Li, Yi Yang, Cong Han, Shuanglong Li, Lin Liu
Venue: KDD
Year: 2025

Recent advances in generative artificial intelligence have revolutionized information retrieval and content generation, opening up new opportunities for the e-commerce industry. Alignment learning between small models and parallel corpora cannot meet current needs. The success of ChatGPT demonstrates that large models need to first establish a fundamental understanding, and then utilize high-quality corpora for generation. Having a large model foundation is indispensable. In this paper, we establish a fundamental 10B multimodal model foundation for multimodal generation tasks and propose a scene-based alignment learning approach called conditional sample supervised fine-tuning for downstream generation tasks. Meanwhile, diffusion models are known to be vulnerable to outliers in training data. To address this, we utilize an alternative diffusion loss function that preserves the high quality of generated data like the original squared L2 loss while being robust to outliers.In practical test sets, the multimodal foundation fully demonstrates its alignment and comprehension abilities for graphic and textual content. Additionally, conditional fine-tuning and the design of the loss function significantly enhance the quality of generated content. The quality rate of images has increased by 34.3 percentage points, and prompt control has improved by 19.8 percentage points. The application of our framework in Baidu Search Ads has led to significant revenue growth. For instance, ads with generated image creatives have achieved a 29% higher click-through rate (CTR), resulting in a daily consumption of 3 million yuan.
On Measuring Unnoticeability of Graph Adversarial Attacks: Observations, New Measure, and Applications
Authors: Hyeonsoo Jo, Hyunjin Hwang, Fanchen Bu, Soo Yong Lee, Chanyoung Park, Kijung Shin
Venue: KDD
Year: 2025

Adversarial attacks are allegedly unnoticeable. Prior studies have designed attack noticeability measures on graphs, primarily using statistical tests to compare the topology of original and (possibly) attacked graphs. However, we observe two critical limitations in the existing measures. First, because the measures rely on simple rules, attackers can readily enhance their attacks to bypass them, reducing their attack ''noticeability'' and, yet, maintaining their attack performance. Second, because the measures naively leverage global statistics, such as degree distributions, they may entirely overlook attacks until severe perturbations occur, letting the attacks be almost ''totally unnoticeable.'' To address the limitations, we introduce HideNSeek, a learnable measure for graph attack noticeability. First, to mitigate the bypass problem, HideNSeek learns to distinguish the original and (potential) attack edges using a learnable edge scorer (LEO), which scores each edge on its likelihood of being an attack. Second, to mitigate the overlooking problem, HideNSeek conducts imbalance-aware aggregation of all the edge scores to obtain the final noticeability score. Using six real-world graphs, we empirically demonstrate that HideNSeek effectively alleviates the observed limitations, and LEO (i.e., our learnable edge scorer) outperforms eleven competitors in distinguishing attack edges under five different attack methods. For an additional application, we show that LEO boost the performance of robust GNNs by removing attack-like edges.
EMBER2024 - A Benchmark Dataset for Holistic Evaluation of Malware Classifiers
Authors: Robert J. Joyce, Gideon Miller, Phil Roth, Richard Zak, Elliott Zaresky-Williams, Hyrum S. Anderson, Edward Raff, James Holt
Venue: KDD
Year: 2025

A lack of accessible data has historically restricted malware analysis research, and practitioners have relied heavily on datasets provided by industry sources to advance. Existing public datasets are limited by narrow scope - most include files targeting a single platform, have labels supporting just one type of malware classification task, and make no effort to capture the evasive files that make malware detection difficult in practice. We present EMBER2024, a new dataset that enables holistic evaluation of malware classifiers. Created in collaboration with the authors of EMBER2017 and EMBER2018, the EMBER2024 dataset includes hashes, metadata, feature vectors, and labels for more than 3.2 million files from six file formats. Our dataset supports the training and evaluation of machine learning models on seven malware classification tasks, including malware detection, malware family classification, and malware behavior identification. EMBER2024 is the first to include a collection of malicious files that initially went undetected by a set of antivirus products, creating a ''challenge'' set to assess classifier performance against evasive malware. This work also introduces EMBER feature version 3, with added support for several new feature types. We are releasing the EMBER2024 dataset to promote reproducibility and empower researchers in the pursuit of new malware research topics.
Revisiting Self-attention for Cross-domain Sequential Recommendation
Authors: Clark Mingxuan Ju, Leonardo Neves, Bhuvesh Kumar, Liam Collins, Tong Zhao, Yuwei Qiu, Qing Dou, Sohail Nizam, Sen Yang, Neil Shah
Venue: KDD
Year: 2025

Sequential recommendation is a popular paradigm in modern recommender systems. In particular, one challenging problem in this space is cross-domain sequential recommendation (CDSR), which aims to predict future behaviors given user interactions across multiple domains. Existing CDSR frameworks are mostly built on the self-attention transformer and seek to improve by explicitly injecting additional domain-specific components (e.g. domain-aware module blocks). While these additional components help, we argue they overlook the core self-attention module already present in the transformer, a naturally powerful tool to learn correlations among behaviors. In this work, we aim to improve the CDSR performance for simple models from a novel perspective of enhancing the self-attention. Specifically, we introduce a Pareto-optimal self-attention and formulate the cross-domain learning as a multi-objective problem, where we optimize the recommendation task while dynamically minimizing the cross-domain attention scores. Our approach automates knowledge transfer in CDSR (dubbed as AutoCDSR) - it not only mitigates negative transfer but also encourages complementary knowledge exchange among auxiliary domains. Based on the idea, we further introduce AutoCDSR+, a more performant variant with slight additional cost. Our proposal is easy to implement and works as a plug-and-play module that can be incorporated into existing transformer-based recommenders. Besides flexibility, it is practical to deploy because it brings little extra computational overheads without heavy hyper-parameter tuning. We conduct experiments over both large-scale production recommender data as well as academic benchmarks, where AutoCDSR consistently enhances the performance of base transformers, enabling simple models to perform on par with state-of-the-art with less overhead (e.g., 4x faster than state-of-the-art CDSR models). AutoCDSR on average improves Recall@10 for SASRec and Bert4Rec by 9.8% and 16.0% and NDCG@10 by 12.0% and 16.7%, respectively. Code is available at https://github.com/snap-research/AutoCDSR.
Harnessing Influence Function in Explaining Graph Neural Networks
Authors: Heesoo Jung, Chanyong Kim, Geonhee Han, Hogun Park
Venue: KDD
Year: 2025

Explaining graphs and their target Graph Neural Networks (GNNs) has gained attention with the growing use of GNNs. Most existing explainable AI (XAI) methods for GNNs focus on extracting an explanation subgraph and assume the target GNN is supervised with accessible class probabilities. However, the growing prevalence of GNN models in unsupervised settings underscores the necessity for task-irrelevant explanations. Moreover, most existing studies scarcely explore whether identifying edges absent from the original graph can improve explanation quality. To this end, we propose HINT-G (Harnessing INfluence function for Task-irrelevant explanation on Graph neural networks), a method that uses influence functions to explain models across diverse learning paradigms and considers edges beyond the given graph. The influence of an edge can be determined directly or by aggregating the influence scores of its constituent nodes, while the influence of a non-existent edge can also be determined. Furthermore, this method is task-irrelevant, since the influence score can be obtained whenever the loss function of the target model is differentiable. Experimental results on several datasets consistently demonstrate that HINT-G effectively explains graphs through the influence function framework. Our implementation code is available at https://github.com/cycy-kim/HINT-G.
Simplicial SMOTE: Oversampling Solution to the Imbalanced Learning Problem
Authors: Oleg Kachan, Andrey V. Savchenko, Gleb Gusev
Venue: KDD
Year: 2025

SMOTE (Synthetic Minority Oversampling Technique) is the established geometric approach to random oversampling to balance classes in the imbalanced learning problem, followed by many extensions. Its idea is to introduce synthetic data points of the minor class, with each new point being the convex combination of an existing data point and one of its k-nearest neighbors. In this paper, by viewing SMOTE as sampling from the edges of a geometric neighborhood graph and borrowing tools from the topological data analysis, we propose a novel technique, Simplicial SMOTE, that samples from the simplices of a geometric neighborhood simplicial complex. A new synthetic point is defined by the barycentric coordinates w.r.t. a simplex spanned by an arbitrary number of data points being sufficiently close rather than a pair. Such a replacement of the geometric data model results in better coverage of the underlying data distribution compared to existing geometric sampling methods and allows the generation of synthetic points of the minority class closer to the majority class on the decision boundary. We experimentally demonstrate that our Simplicial SMOTE outperforms several popular geometric sampling methods, including the original SMOTE. Moreover, we show that simplicial sampling can be easily integrated into existing SMOTE extensions. We generalize and evaluate simplicial extensions of the classic Borderline SMOTE, Safe-level SMOTE, and ADASYN algorithms, all of which outperform their graph-based counterparts.