OpenRaiser/Intern-Atlas huggingface dataset:
Unbiased Locally Private Estimator for Polynomials of Laplacian Variables
Authors: Quentin Hillebrand, Vorapong Suppakitpaisarn, Tetsuo Shibuya
Venue: KDD
Year: 2023

This work presents a mechanism to debias polynomial functions computed from locally differentially private data. Local differential privacy is a widely used privacy notion where users add Laplacian noise to their information before submitting it to a central server. That, however, causes bias when we calculate non-linear functions based on those noisy information. Our proposed recursive algorithm debiases these functions, with a calculation time of O(r n log n), where r is the polynomial degree and n is the number of users. We evaluate our method on the problems of k-star counting and variance estimation, comparing results with state-of-the-art algorithms. The results show that our method not only eliminates bias, but also provides at least 100 times more accuracy than previous works.
International Workshop on Federated Learning for Distributed Data Mining
Authors: Junyuan Hong, Zhuangdi Zhu, Lingjuan Lyu, Yang Zhou 0001, Vishnu Naresh Boddeti, Jiayu Zhou
Venue: KDD
Year: 2023

The past decade has witnessed wide applications of machine learning to various domains for decision-making, including crime detection, urban planning, drug discovery, and health monitoring, which benefited from surging data resources. As data collection in real-world applications is often done in different locations, being able to mine and discover knowledge from distributed data sources is an essential requirement for building powerful predictive models. However, directly uploading all data sources to an untrustworthy centralized data server for learning will lead to risks of privacy leakage. Federated Learning (FL) emerges as a decentralized learning framework that aggregates knowledge from distributed data without centralizing them, hence mitigating privacy risks. By hosting this workshop, we aim to attract a broad spectrum of audiences, including researchers and practitioners from academia and industry interested in the latest advances in FL. As an effort to advance the fundamental development of FL in data mining, this workshop will encourage ideas exchange on the trustworthiness, scalability, robustness, and broad applications of FL.
Expert Knowledge-Aware Image Difference Graph Representation Learning for Difference-Aware Medical Visual Question Answering
Authors: Xinyue Hu 0002, Lin Gu 0003, Qiyuan An, Mengliang Zhang, Liangchen Liu, Kazuma Kobayashi, Tatsuya Harada, Ronald M. Summers, Yingying Zhu 0003
Venue: KDD
Year: 2023

To contribute to automating the medical vision-language model, we propose a novel Chest-Xray Different Visual Question Answering (VQA) task. Given a pair of main and reference images, this task attempts to answer several questions on both diseases and, more importantly, the differences between them. This is consistent with the radiologist's diagnosis practice that compares the current image with the reference before concluding the report. We collect a new dataset, namely MIMIC-Diff-VQA, including 700,703 QA pairs from 164,324 pairs of main and reference images. Compared to existing medical VQA datasets, our questions are tailored to the Assessment-Diagnosis-Intervention-Evaluation treatment procedure used by clinical professionals. Meanwhile, we also propose a novel expert knowledge-aware graph representation learning model to address this task. The proposed baseline model leverages expert knowledge such as anatomical structure prior, semantic, and spatial knowledge to construct a multi-relationship graph, representing the image differences between two images for the image difference VQA task. The dataset and code can be found at https://github.com/Holipori/MIMIC-Diff-VQA. We believe this work would further push forward the medical vision language model.
Graph Learning in Physical-informed Mesh-reduced Space for Real-world Dynamic Systems
Authors: Yeping Hu, Bo Lei 0004, Victor M. Castillo
Venue: KDD
Year: 2023

Recent machine learning approaches have demonstrated their ability to extract information from data that could be translated into knowledge about the underlying dynamic systems. However, these learning-based models suffer from scalability issues when training on high-dimensional and high-resolution simulation data generated for real-world applications. In this work, we aim to tackle this challenge by deliberately prioritizing certain aspects of dynamic systems, while allocating relatively less attention and computational resources to others. Specifically, we concentrate on improving the predictive accuracy of crucial properties or regions that significantly impact these dynamic systems, while comparatively reducing emphasis on the remaining aspects. By employing graph learning schemes and custom-designed modules, we have developed a two-stage prediction model that incorporates prior knowledge of the systems. This approach enables us to place a heightened emphasis on the region of interest (ROI) during the learning process where the model operates in a reduced-dimensional mesh space, resulting in reduced computational costs while preserving crucial physical properties. To test and evaluate our method, we utilized two simulation datasets: lid-driven cavity and cylinder flow. The results show that even under reduced operational space, our method still achieves desirable performance on accuracy and generalizability of both prediction and physical consistency over region of interest.
ST-iFGSM: Enhancing Robustness of Human Mobility Signature Identification Model via Spatial-Temporal Iterative FGSM
Authors: Mingzhi Hu, Xin Zhang 0098, Yanhua Li, Xun Zhou 0001, Jun Luo 0007
Venue: KDD
Year: 2023

The Human Mobility Signature Identification (HuMID) problem aims at determining whether the incoming trajectories were generated by a claimed agent from the historical movement trajectories of a set of individual human agents such as pedestrians and taxi drivers. The HuMID problem is significant, and its solutions have a wide range of real-world applications, such as criminal identification for police departments, risk assessment for auto insurance providers, driver verification in ride-sharing services, and so on. Though Deep neural networks (DNN) based HuMID models on spatial-temporal mobility fingerprint similarity demonstrate remarkable performance in effectively identifying human agents' mobility signatures, it is vulnerable to adversarial attacks as other DNN-based models. Therefore, in this paper, we propose a Spatial-Temporal iterative Fast Gradient Sign Method with L0 regularization - ST-iFGSM - to detect the vulnerability and enhance the robustness of HuMID models. Extensive experiments with real-world taxi trajectory data demonstrate the efficiency and effectiveness of our ST-iFGSM algorithm. We tested our method on both the ST-SiameseNet and an LSTM-based HuMID classification model. It shows that ST-iFGSM can generate successful attacks to fool the HuMID models with only a few steps of attack in a small portion of the trajectories. The generated attacks can be used as augmented data to update and improve the HuMID model accuracy significantly from 47.36% to 76.18% on testing samples after the attack(86.25% on the original testing samples).
Leveraging Relational Graph Neural Network for Transductive Model Ensemble
Authors: Zhengyu Hu, Jieyu Zhang 0001, Haonan Wang, Siwei Liu 0001, Shangsong Liang
Venue: KDD
Year: 2023

Traditional methods of pre-training, fine-tuning, and ensembling often overlook essential relational data and task interconnections. To address this gap, our study presents a novel approach to harnessing this relational information via a relational graph-based model. We introduce Relational grAph Model ensemBLE model, abbreviated as RAMBLE. This model distinguishes itself by performing class label inference simultaneously across all data nodes and task nodes, employing the relational graph in a transductive manner. This fine-grained approach allows us to better comprehend and model the intricate interplay between data and tasks. Furthermore, we incorporate a novel variational information bottleneck-guided scheme for embedding fusion and aggregation. This innovative technique facilitates the creation of an informative fusion embedding, honing in on embeddings beneficial for the intended task while simultaneously filtering out potential noise-laden embeddings. Our theoretical analysis, grounded in information theory, confirms that the use of relational information for embedding fusion allows us to achieve higher upper and lower bounds on our target task's accuracy. We thoroughly assess our proposed model across eight diverse datasets, and the experimental results demonstrate the model's effective utilization of relational knowledge derived from all pre-trained models, thereby enhancing its performance on our target tasks.
Learning Discrete Document Representations in Web Search
Authors: Rong Huang, Danfeng Zhang, Weixue Lu, Han Li, Meng Wang, Daiting Shi, Jun Fan, Zhicong Cheng, Simiu Gu, Dawei Yin 0001
Venue: KDD
Year: 2023

Product quantization (PQ) has been usually applied to dense retrieval (DR) of documents thanks to its competitive time, memory efficiency and compatibility with other approximate nearest search (ANN) methods. Originally, PQ was learned to minimize the reconstruction loss, i.e., the distortions between the original dense embeddings and the reconstructed embeddings after quantization. Unfortunately, such an objective is inconsistent with the goal of selecting ground-truth documents for the input query, which may cause a severe loss of retrieval quality. Recent research has primarily concentrated on jointly training the biencoders and PQ to ensure consistency for improved performance. However, it is still difficult to design an approach that can cope with challenges like discrete representation collapse, mining informative negatives, and deploying effective embedding-based retrieval (EBR) systems in a real search engine.
Generative Perturbation Analysis for Probabilistic Black-Box Anomaly Attribution
Authors: Tsuyoshi Idé, Naoki Abe
Venue: KDD
Year: 2023

We address the task of probabilistic anomaly attribution in the black-box regression setting, where the goal is to compute the probability distribution of the attribution score of each input variable, given an observed anomaly. The training dataset is assumed to be unavailable. This task differs from the standard XAI (explainable AI) scenario, since we wish to explain the anomalous deviation from a black-box prediction rather than the black-box model itself.
Parameter-free Spikelet: Discovering Different Length and Warped Time Series Motifs using an Adaptive Time Series Representation
Authors: Makoto Imamura, Takaaki Nakamura
Venue: KDD
Year: 2023

Over the last two decades, time series motif discovery has emerged as a useful primitive for many downstream analytical tasks, including clustering, classification, rule discovery, segmentation, and summarization. In parallel, it has long been known that Dynamic Time Warping (DTW) is superior to other similarity measures such as Euclidean Distance under most settings. Recently an algorithm to allow scalable DTW motif discovery was proposed; however, it is limited to finding pairs of subsequences whose subsequence lengths are the same. Moreover, that length must be provided by the user ahead of time. In this work, we propose a novel method to discover "warped" motifs whose lengths may differ. Moreover, our method allows input parameters that are not fixed lengths but rather just bounds on the maximum length of motifs to find. This allows us to quickly find different-length motifs without the burdensome trial-and-error of conventional methods. With extensive empirical work, we show that our method is scalable enough for real-world datasets and enables us to find variable-length and "warped" motifs that would otherwise escape the attention of conventional algorithms.
Hierarchical Proxy Modeling for Improved HPO in Time Series Forecasting
Authors: Arindam Jati, Vijay Ekambaram, Shaonli Pal, Brian Quanz, Wesley M. Gifford, Pavithra Harsha, Stuart Siegel, Sumanta Mukherjee, Chandra Narayanaswami
Venue: KDD
Year: 2023

Selecting the right set of hyperparameters is crucial in time series forecasting. The classical temporal cross-validation framework for hyperparameter optimization (HPO) often leads to poor test performance because of a possible mismatch between validation and test periods. To address this test-validation mismatch, we propose a novel technique, H-Pro to drive HPO via test proxies by exploiting data hierarchies often associated with time series datasets. Since higher-level aggregated time series often show less irregularity and better predictability as compared to the lowest-level time series which can be sparse and intermittent, we optimize the hyperparameters of the lowest-level base-forecaster by leveraging the proxy forecasts for the test period generated from the forecasters at higher levels. H-Pro can be applied on any off-the-shelf machine learning model to perform HPO. We validate the efficacy of our technique with extensive empirical evaluation on five publicly available hierarchical forecasting datasets. Our approach outperforms existing state-of-the-art methods in Tourism, Wiki, and Traffic datasets, and achieves competitive result in Tourism-L dataset, without any model-specific enhancements. Moreover, our method outperforms the winning method of the M5 forecast accuracy competition.
Community-based Dynamic Graph Learning for Popularity Prediction
Authors: Shuo Ji, Xiaodong Lu, Mingzhe Liu 0002, Leilei Sun, Chuanren Liu, Bowen Du 0001, Hui Xiong 0001
Venue: KDD
Year: 2023

Popularity prediction, which aims to forecast how many users would like to interact with a target item or online content in the future, can help online shopping or social media platforms to identify popular items or digital contents. Many efforts have been made to study how the multi-faceted factors, such as item features, user preferences, and social influence, affect user-item interactions, but little work has focused on the evolutionary dynamics of these factors for individuals or groups. In that light, this paper develops a community-based dynamic graph learning method for popularity prediction. First, a dynamic graph learning framework is proposed to maintain a dynamic representation for each item or user entity and update the representations according to the newly observed user-item interactions. Second, a community detection module is designed to capture the evolving community structures and identify the most influential nodes. More importantly, our framework leverages a community-level message passing during the learning process to balance local and global information propagation. Finally, we predict the popularity of the target item or online content based on the learned representations. Our experimental results based on three real-world datasets demonstrate that the proposed method achieves better performance than the baselines. Our method could not only model the changes in a user's preferences, but also capture how the communities evolve over time.
GetPt: Graph-enhanced General Table Pre-training with Alternate Attention Network
Authors: Ran Jia, Haoming Guo, Xiaoyuan Jin, Chao Yan, Lun Du, Xiaojun Ma 0001, Tamara Stankovic, Marko Lozajic, Goran Zoranovic, Igor Ilic, Shi Han, Dongmei Zhang 0001
Venue: KDD
Year: 2023

Tables are widely used for data storage and presentation due to their high flexibility in layout. The importance of tables as information carriers and the complexity of tabular data understanding attract a great deal of research on large-scale pre-training for tabular data. However, most of the works design models for specific types of tables, such as relational tables and tables with well-structured headers, neglecting tables with complex layouts. In real-world scenarios, there are many such tables beyond their target scope that cannot be well supported. In this paper, we propose GetPt, a unified pre-training architecture for general table representation applicable even to tables with complex structures and layouts. First, we convert a table to a heterogeneous graph with multiple types of edges to represent the layout of the table. Based on the graph, a specially designed transformer is applied to jointly model the semantics and structure of the table. Second, we devise the Alternate Attention Network (AAN) to better model the contextual information across multiple granularities of a table including tokens, cells, and the table. To better support a wide range of downstream tasks, we further employ three pre-training objectives and pre-train the model on a large table dataset. We fine-tune and evaluate GetPt model on two representative tasks, table type classification, and table structure recognition. Experiments show that GetPt outperforms existing state-of-the-art methods on these tasks.
Semantic Dissimilarity Guided Locality Preserving Projections for Partial Label Dimensionality Reduction
Authors: Yuheng Jia, Jiahao Jiang, Yongheng Wang
Venue: KDD
Year: 2023

Partial label learning (PLL) is a significant weakly supervised learning framework, where each training example corresponds to a set of candidate labels among which only one is the ground-truth label. Existing works on partial label dimensionality reduction only exploit the disambiguated labels, but overlook the available semantic dissimilarity relationship hidden in the disambiguated labeling confidence, i.e., the smaller the inner product of the labeling confidences of two instances, the less likely they have the same ground-truth label. By combining such global dissimilarity relationship with local neighborhood information, we propose a novel partial label dimensionality reduction method named SDLPP, which employs an alternating procedure including candidate label disambiguation, semantic dissimilarity generation and dimensionality reduction. The labeling confidences of candidate labels and semantic dissimilarity relationship are constantly updated through the alternating procedure, where the processes in each iteration are based on the low-dimensional data obtained in the previous iteration. After the alternating procedure, SDLPP maps the original data to a pre-specified low-dimensional feature space. Comprehensive experiments on both synthetic and real-world data sets validate that SDLPP can improve the generalization performance of different PLL algorithms, and outperform state-of-the-art partial label dimensionality reduction methods. The codes can be publicly accessible on the link https://github.com/jhjiangSEU/SDLPP.
Enhancing Node-Level Adversarial Defenses by Lipschitz Regularization of Graph Neural Networks
Authors: Yaning Jia, Dongmian Zou, Hongfei Wang, Hai Jin 0001
Venue: KDD
Year: 2023

Graph neural networks (GNNs) have shown considerable promise for graph-structured data. However, they are also known to be unstable and vulnerable to perturbations and attacks. Recently, the Lipschitz constant has been adopted as a control on the stability of Euclidean neural networks, but calculating the exact constant is also known to be difficult even for very shallow networks. In this paper, we extend the Lipschitz analysis to graphs by providing a systematic scheme for estimating upper bounds of the Lipschitz constants of GNNs. We also derive concrete bounds for widely used GNN architectures including GCN, GraphSAGE and GAT. We then use these Lipschitz bounds for regularized GNN training for improved stability. Our numerical results on Lipschitz regularization of GNNs not only illustrate enhanced test accuracy under random noise, but also show consistent improvement for state-of-the-art defense methods against adversarial attacks.
Heterformer: Transformer-based Deep Node Representation Learning on Heterogeneous Text-Rich Networks
Authors: Bowen Jin, Yu Zhang 0044, Qi Zhu 0008, Jiawei Han 0001
Venue: KDD
Year: 2023

Representation learning on networks aims to derive a meaningful vector representation for each node, thereby facilitating downstream tasks such as link prediction, node classification, and node clustering. In heterogeneous text-rich networks, this task is more challenging due to (1) presence or absence of text: Some nodes are associated with rich textual information, while others are not; (2) diversity of types: Nodes and edges of multiple types form a heterogeneous network structure. As pretrained language models (PLMs) have demonstrated their effectiveness in obtaining widely generalizable text representations, a substantial amount of effort has been made to incorporate PLMs into representation learning on text-rich networks. However, few of them can jointly consider heterogeneous structure (network) information as well as rich textual semantic information of each node effectively. In this paper, we propose Heterformer, a Heterogeneous Network-Empowered Transformer that performs contextualized text encoding and heterogeneous structure encoding in a unified model. Specifically, we inject heterogeneous structure information into each Transformer layer when encoding node texts. Meanwhile, Heterformer is capable of characterizing node/edge type heterogeneity and encoding nodes with or without texts. We conduct comprehensive experiments on three tasks (i.e., link prediction, node classification, and node clustering) on three large-scale datasets from different domains, where Heterformer outperforms competitive baselines significantly and consistently. The code can be found at https://github.com/PeterGriffinJin/Heterformer.
Multimodal Indoor Localisation in Parkinson's Disease for Detecting Medication Use: Observational Pilot Study in a Free-Living Setting
Authors: Ferdian Jovan, Catherine Morgan, Ryan McConville, Emma L. Tonkin, Ian Craddock, Alan L. Whone
Venue: KDD
Year: 2023

Parkinson's disease (PD) is a slowly progressive, debilitating neurodegenerative disease which causes motor symptoms including gait dysfunction. Motor fluctuations are alterations between periods with a positive response to levodopa therapy ("on") and periods marked by re-emergency of PD symptoms ("off") as the response to medication wears off. These fluctuations often affect gait speed and they increase in their disabling impact as PD progresses. To improve the effectiveness of current indoor localisation methods, a transformer-based approach utilising dual modalities which provide complementary views of movement, Received Signal Strength Indicator (RSSI) and accelerometer data from wearable devices, is proposed. A sub-objective aims to evaluate whether indoor localisation, including its in-home gait speed features (i.e. the time taken to walk between rooms), could be used to evaluate motor fluctuations by detecting whether the person with PD is taking levodopa medications or withholding them. To properly evaluate our proposed method, we use a free-living dataset where the movements and mobility are greatly varied and unstructured as expected in real-world conditions. 24 participants lived in pairs (consisting of one person with PD, one control) for five days in a smart home with various sensors. Our evaluation on the resulting dataset demonstrates that our proposed network outperforms other methods for indoor localisation. The sub-objective evaluation shows that precise room-level localisation predictions, transformed into in-home gait speed features, produce accurate predictions on whether the PD participant is taking or withholding their medications.