GenAI4ELab/papercli-papers huggingface dataset:
ExPERT: Modeling Human Behavior Under External Stimuli Aware Personalized MTPP
Authors: Subhendu Khatuya; Ritvik Vij; Paramita Koley; Samik Datta; Niloy Ganguly
Venue: AAAI 2025 (main)

Abstract: Marked Temporal Point Process (MTPP) -- the de-facto sequence model for continuous-time event sequences -- historically employed for modeling human-generated action sequences, lack awareness of external stimuli. In this study, we propose a novel framework developed over Transformer Hawkes Process (THP) to incorporate external stimuli in a domain-agnostic manner. Furthermore, we integrate personalization into our framework by employing language model-based representations of user and event descriptions, which is essential for modeling human-generated action sequences. Towards evaluating the efficacy, we put together a comprehensive benchmark comprising 5 datasets (2 novel additions, and 3 repurposed from existing open datasets) harvested from several domains, spanning education, e-commerce, online payment, and discussion forum. On average, we achieve 9.35% gain in type-prediction accuracy and 7.38% reduction in time-prediction RMSE across all datasets over SOTA MTPP baselines. We demonstrate the superior performance of our proposed model through extensive ablations and showcasing its ability to capture complex combinations of external stimuli in a synthetic set up.
Learning Structural Causal Models from Ordering: Identifiable Flow Models
Authors: Minh Khoa Le; Kien Do; Truyen Tran
Venue: AAAI 2025 (main)

Abstract: In this study, we address causal inference when only observational data and a valid causal ordering from the causal graph are available. We introduce a set of flow models that can recover component-wise, invertible transformation of exogenous variables. Our flow-based methods offer flexible model design while maintaining causal consistency regardless of the number of discretization steps. We propose design improvements that enable simultaneous learning of all causal mechanisms and reduce abduction and prediction complexity to linear O(n) relative to the number of layers, independent of the number of causal variables. Empirically, we demonstrate that our method outperforms previous state-of-the-art approaches and delivers consistent performance across a wide range of structural causal models in answering observational, interventional, and counterfactual questions. Additionally, our method achieves a significant reduction in computational time compared to existing diffusion-based techniques, making it practical for large structural causal models.
Configuring Data Augmentations to Reduce Variance Shift in Positional Embedding of Vision Transformers
Authors: Bum Jun Kim; Sang Woo Kim
Venue: AAAI 2025 (main)

Abstract: Vision transformers (ViTs) have demonstrated remarkable performance in a variety of vision tasks. Despite their promising capabilities, training a ViT requires a large amount of diverse data. Several studies empirically found that using rich data augmentations, such as Mixup, Cutmix, and random erasing, is critical to the successful training of ViTs. Now, the use of rich data augmentations has become a standard practice in the current state. However, we report a vulnerability to this practice: Certain data augmentations such as Mixup cause a variance shift in the positional embedding of ViT, which has been a hidden factor that degrades the performance of ViT during the test phase. We claim that achieving a stable effect from positional embedding requires a specific condition on the image, which is often broken for the current data augmentation methods. We provide a detailed analysis of this problem as well as the correct configuration for these data augmentations to remove the side effects of variance shift. Experiments showed that adopting our guidelines improves the performance of ViTs compared with the current configuration of data augmentations.
T-MDML: Triplet-based Multiple Distance Metric Learning for Multi-Instance Multi-Label Classification with Label Correlation
Authors: Dongyeon Kim; Yejin Kan; Gangman Yi
Venue: AAAI 2025 (main)

Abstract: The multi-instance multi-label (MIML) problem is a new supervised learning paradigm that has emerged to efficiently represent complex data. Therefore, various similarity-based algorithms have been proposed, but existing algorithms commonly measure similarity by considering only the structural relationships in the feature space without utilizing information from the label space. As these approaches do not adequately reflect the complex properties of MIML data, it is essential to improve the accuracy of MIML classification by utilizing information from both feature and label spaces. Thus, we propose a new algorithm, T-MDML: triplet-based multiple distance metric learning for MIML. T-MDML defines a distance metric by learning a global property shared by the entire label space and a label-specific property for each label. In addition, we simultaneously consider the structural characteristics of features and label space to extract label correlation and incorporate it into the optimization process. In experiments, we demonstrate the efficiency of our label correlation estimation method and verify its performance by applying it to MIMLkNN. We also demonstrate T-MDML’s relative superiority over existing MIML algorithms, as well as its scalability when applied to similarity-based MIML methods.
Singular Value Scaling: Efficient Generative Model Compression via Pruned Weights Refinement
Authors: Hyeonjin Kim; Jaejun Yoo
Venue: AAAI 2025 (main)

Abstract: While pruning methods effectively maintain model performance without extra training costs, they often focus solely on preserving crucial connections, overlooking the impact of pruned weights on subsequent fine-tuning or distillation, leading to inefficiencies. Moreover, most compression techniques for generative models have been developed primarily for GANs, tailored to specific architectures like StyleGAN, and research into compressing Diffusion models has just begun. Even more, these methods are often applicable only to GANs or Diffusion models, highlighting the need for approaches that work across both model types. In this paper, we introduce Singular Value Scaling (SVS), a versatile technique for refining pruned weights, applicable to both model types. Our analysis reveals that pruned weights often exhibit dominant singular vectors, hindering fine-tuning efficiency and leading to suboptimal performance compared to random initialization. Our method enhances weight initialization by minimizing the disparities between singular values of pruned weights, thereby improving the fine-tuning process. This approach not only guides the compressed model toward superior solutions but also significantly speeds up fine-tuning. Extensive experiments on StyleGAN2, StyleGAN3 and DDPM demonstrate that SVS improves compression performance across model types without additional training costs.
Battling the Non-stationarity in Time Series Forecasting via Test-time Adaptation
Authors: HyunGi Kim; Siwon Kim; Jisoo Mok; Sungroh Yoon
Venue: AAAI 2025 (main)

Abstract: Deep Neural Networks have spearheaded remarkable advancements in time series forecasting (TSF), one of the major tasks in time series modeling. Nonetheless, the non-stationarity of time series undermines the reliability of pre-trained source time series forecasters in mission-critical deployment settings. In this study, we introduce a pioneering test-time adaptation framework tailored for TSF (TSF-TTA). TAFAS, the proposed approach to TSF-TTA, flexibly adapts source forecasters to continuously shifting test distributions while preserving the core semantic information learned during pre-training. The novel utilization of partially-observed ground truth and gated calibration module enables proactive, robust, and model-agnostic adaptation of source forecasters. Experiments on diverse benchmark datasets and cutting-edge architectures demonstrate the efficacy and generality of TAFAS, especially in long-term forecasting scenarios that suffer from significant distribution shifts.
Accurate Link Prediction for Edge-Incomplete Graphs via PU Learning
Authors: Junghun Kim; Ka Hyun Park; Hoyoung Yoon; U Kang
Venue: AAAI 2025 (main)

Abstract: Given an edge-incomplete graph, how can we accurately find its missing links?
The problem aims to discover the missing relations between entities when their relationships are represented as a graph.
Edge-incomplete graphs are prevalent in real-world due to practical limitations, such as not checking all users when adding friends in a social network.
Addressing the problem is crucial for various tasks, including recommending friends in social networks and finding references in citation networks.
However, previous approaches rely heavily on the given edge-incomplete (observed) graph, making it challenging to consider the missing (unobserved) links.

In this paper, we propose PULL, an accurate link prediction method based on the positive-unlabeled (PU) learning.
PULL treats the observed edges in the training graph as positive examples, and the unconnected node pairs as unlabeled ones.
PULL effectively prevents the link predictor from blindly trusting the observed graph by proposing latent variables for every edge, and leveraging the expected graph structure with respect to these variables.
Extensive experiments on real- world datasets show that PULL consistently outperforms the baselines for predicting links in edge-incomplete graphs.
To Predict or Not to Predict? Proportionally Masked Autoencoders for Tabular Data Imputation
Authors: Jungkyu Kim; Kibok Lee; Taeyoung Park
Venue: AAAI 2025 (main)

Abstract: Masked autoencoders (MAEs) have recently demonstrated effectiveness in tabular data imputation. However, due to the inherent heterogeneity of tabular data, the uniform random masking strategy commonly used in MAEs can disrupt the distribution of missingness, leading to suboptimal performance. To address this, we propose a proportional masking strategy for MAEs. Specifically, we first compute the statistics of missingness based on the observed proportions in the dataset, and then generate masks that align with these statistics, ensuring that the distribution of missingness is preserved after masking. Furthermore, we argue that simple MLP-based token mixing offers competitive or often superior performance compared to attention mechanisms while being more computationally efficient, especially in the tabular domain with the inherent heterogeneity. Experimental results validate the effectiveness of the proposed proportional masking strategy across various missing data patterns in tabular datasets.
Salient Frequency-aware Exemplar Compression for Resource-constrained Online Continual Learning
Authors: Junsu Kim; Suhyun Kim
Venue: AAAI 2025 (main)

Abstract: Online Class-Incremental Learning (OCIL) enables a model to learn new classes from a data stream. Since data stream samples are seen only once and the capacity of storage is constrained, OCIL is particularly susceptible to Catastrophic Forgetting (CF). While exemplar replay methods alleviate CF by storing representative samples, the limited capacity of the buffer inhibits capturing the entire old data distribution, leading to CF. In this regard, recent papers suggest image compression for better memory usage. However, existing methods raise two concerns: computational overhead and compression defects. On one hand, computational overhead can limit their applicability in OCIL settings, as models might miss learning opportunities from the current streaming data if computational resources are budgeted and preoccupied with compression. On the other hand, typical compression schemes demanding low computational overhead, such as JPEG, introduce noise detrimental to training. To address these issues, we propose Salient Frequency-aware Exemplar Compression (SFEC), an efficient and effective JPEG-based compression framework. SFEC exploits saliency information in the frequency domain to reduce negative impacts from compression artifacts for learning. Moreover, SFEC employs weighted sampling for exemplar elimination based on the distance between raw and compressed data to mitigate artifacts further. Our experiments employing the baseline OCIL method on benchmark datasets such as CIFAR-100 and Mini-ImageNet demonstrate the superiority of SFEC over previous exemplar compression methods in streaming scenarios.
Diffusion-Based Active Learning for Distributed Client Manifolds
Authors: Kwang In Kim
Venue: AAAI 2025 (main)

Abstract: In privacy-preserving distributed learning environments, data stored on local clients cannot be shared with other clients or servers. We consider a new active learning problem setup for these environments, where the server aims to build a centralized model by distributing labeling budgets across different clients. Our algorithm identifies which clients and their data points warrant annotation by estimating the global impact of the resulting labels. We evaluate this impact by embedding the clients into the manifold of learner parameters, formed by the task learner's predictions on unlabeled data, and diffusing the reduction in predictive uncertainties caused by labeling. The algorithm effectively selects clients with high estimated impact while achieving diversity in client selection, all without accessing local client data. In experiments, our approach demonstrates substantial advancements when compared to  adaptations of existing active learning algorithms.
A Stochastic Approach to Bi-Level Optimization for Hyperparameter Optimization and Meta Learning
Authors: Minyoung Kim; Timothy Hospedales
Venue: AAAI 2025 (main)

Abstract: We tackle the general differentiable meta learning problem that is ubiquitous in modern deep learning, including hyperparameter optimization, loss function learning, few-shot learning and more. These problems are often formalized as Bi-Level Optimizations (BLO). We introduce a novel perspective by turning a given BLO problem into a stochastic optimization, where the inner loss function becomes a smooth probability distribution, and the outer loss becomes an expected loss over the inner distribution. To solve this stochastic optimization, we adopt Stochastic Gradient Langevin Dynamics (SGLD) MCMC to sample inner distribution, and propose a recurrent algorithm to compute the MC-estimated hypergradient. Our derivation is similar to forward-mode differentiation, but we introduce a new first-order approximation that makes it feasible for large models without needing to store huge Jacobian matrices. The main benefits are two fold: i) Our stochastic formulation takes into account uncertainty, which makes the method robust to suboptimal inner optimization or non-unique multiple inner minima due to overparametrization; ii) Compared to existing methods that often exhibit unstable behavior and hyperparameter sensitivity in practice, our method leads to considerably more reliable solutions. We demonstrate that the new approach achieves promising results on diverse meta learning problems and easily scales to learning 87M hyper-parameters in the case of Vision Transformers.
Super-Class Guided Transformer for Zero-Shot Attribute Classification
Authors: Sehyung Kim; Chanhyeong Yang; Jihwan Park; Taehoon Song; Hyunwoo J. Kim
Venue: AAAI 2025 (main)

Abstract: Attribute classification is crucial for identifying specific characteristics within image regions.
Vision-Language Models (VLMs) have been effective in zero-shot tasks by leveraging their general knowledge from large-scale datasets.
Recent studies demonstrate that transformer-based models with class-wise queries can effectively address zero-shot multi-label classification.
However, poor utilization of the relationship between seen and unseen attributes makes the model lack generalizability.
Additionally, attribute classification generally involves many attributes, making maintaining the model’s scalability difficult.
To address these issues, we propose Super-class guided transFormer (SugaFormer), a novel framework that leverages super-classes to enhance scalability and generalizability for zero-shot attribute classification.
SugaFormer employs Super-class Query Initialization (SQI) to reduce the number of queries, utilizing common semantic information from super-classes, and incorporates Multi-context Decoding (MD) to handle diverse visual cues.
To strengthen generalizability, we introduce two knowledge transfer strategies that utilize VLMs. 
During training, Super-class guided Consistency Regularization (SCR) aligns model’s features with VLMs using super-class guided prompts, and during inference, Zero-shot Retrieval-based Score Enhancement (ZRSE) refines predictions for unseen attributes.
Extensive experiments demonstrate that SugaFormer achieves state-of-the-art performance across three widely-used attribute classification benchmarks under zero-shot, and cross-dataset transfer settings.
SAP: Corrective Machine Unlearning with Scaled Activation Projection for Label Noise Robustness
Authors: Sangamesh Kodge; Deepak Ravikumar; Gobinda Saha; Kaushik Roy
Venue: AAAI 2025 (main)

Abstract: Label corruption, where training samples are mislabeled due to non-expert annotation or adversarial attacks, significantly degrades model performance. Acquiring large, perfectly labeled datasets is costly, and retraining models from scratch is computationally expensive. To address this, we introduce Scaled Activation Projection (SAP), a novel SVD (Singular Value Decomposition)-based corrective machine unlearning algorithm. SAP mitigates label noise by identifying a small subset of trusted samples using cross-entropy loss and projecting model weights onto a clean activation space estimated using SVD on these trusted samples. This process suppresses the noise introduced in activations due to the mislabeled samples. In our experiments, we demonstrate SAP’s effectiveness on synthetic noise with different settings and real-world label noise. SAP applied to the CIFAR dataset with 25% synthetic corruption show upto 6% generalization improvements. Additionally, SAP can improve the generalization over noise robust training approaches on CIFAR dataset by ∼ 3.2% on average. Further, we observe generalization improvements of 2.31% for a Vision Transformer model trained on naturally corrupted Clothing1M.
Sample-aware Adaptive Structured Pruning for Large Language Models
Authors: Jun Kong; Xinge Ma; Jin Wang; Xuejie Zhang
Venue: AAAI 2025 (main)

Abstract: Large language models (LLMs) have achieved outstanding performance in natural language processing, but enormous model sizes and high computational costs limit their practical deployment. Structured pruning can effectively reduce the resource demands for deployment by removing redundant model parameters. However, the randomly selected calibration data and fixed single importance estimation metrics in existing structured pruning methods lead to degraded performance of pruned models. This study introduces AdaPruner, a sample-aware adaptive structured pruning framework for LLMs, aiming to optimize the calibration data and importance estimation metrics in the structured pruning process. Specifically, AdaPruner effectively removes redundant parameters from LLMs by constructing a structured pruning solution space and then employing Bayesian optimization to adaptively search for the optimal calibration data and importance estimation metrics. Experimental results show that the AdaPruner outperforms existing structured pruning methods on a family of LLMs with varying pruning ratios, demonstrating its applicability and robustness. Remarkably, at a 20% pruning ratio, the model pruned with AdaPruner maintains 97% of the performance of the unpruned model.
Certification of Speaker Recognition Models to Additive Perturbations
Authors: Dmitrii Korzh; Elvir Karimov; Mikhail Pautov; Oleg Y. Rogov; Ivan Oseledets
Venue: AAAI 2025 (main)

Abstract: Speaker recognition technology is applied to various tasks, from personal virtual assistants to secure access systems. However, the robustness of these systems against adversarial attacks, particularly to additive perturbations, remains a significant challenge. In this paper, we pioneer applying robustness certification techniques to speaker recognition, initially developed for the image domain. Our work covers this gap by transferring and improving randomized smoothing certification techniques against norm-bounded additive perturbations for classification and few-shot learning tasks to speaker recognition. We demonstrate the effectiveness of these methods on VoxCeleb 1 and 2 datasets for several models. We expect this work to improve the robustness of voice biometrics and accelerate the research of certification methods in the audio domain.
FCOM: A Federated Collaborative Online Monitoring Framework via Representation Learning
Authors: Tanapol Kosolwattana; Huazheng Wang; Raed Al Kontar; Ying Lin
Venue: AAAI 2025 (main)

Abstract: Monitoring a large population of dynamic processes with limited resources presents a significant challenge across various industrial sectors. This is due to 1) the inherent disparity between the available monitoring resources and the extensive number of processes to be monitored and 2) the unpredictable and heterogeneous dynamics inherent in the progression of these processes. Online learning approaches, commonly referred to as bandit methods, have demonstrated notable potential in addressing this issue by dynamically allocating resources and effectively balancing the exploitation of high-reward processes and the exploration of uncertain ones. However, most online learning algorithms are designed for 1) a centralized setting that requires data sharing across processes for accurate predictions or 2) a homogeneity assumption that estimates a single global model from decentralized data. To overcome these limitations and enable online learning in a heterogeneous population under a decentralized setting, we propose a federated collaborative online monitoring method. Our approach utilizes representation learning to capture the latent representative models within the population and introduces a novel federated collaborative UCB algorithm to estimate these models from sequentially observed decentralized data. This strategy facilitates informed monitoring of resource allocation. The efficacy of our method is demonstrated through theoretical analysis, simulation studies, and its application to decentralized cognitive degradation monitoring in Alzheimer’s disease.