GenAI4ELab/papercli-papers huggingface dataset:
In-depth Analysis of Low-rank Matrix Factorisation in a Federated Setting
Authors: Constantin Philippenko; Kevin Scaman; Laurent Massoulié
Venue: AAAI 2025 (main)

Abstract: This work presents a novel approach to low-rank matrix factorization in a federated learning context, where multiple clients collaboratively solve a matrix decomposition problem without sharing their local data. The algorithm introduces a power initialization technique for the global factorization matrix and combines it with local gradient descent updates to achieve strong theoretical and practical guarantees. Considering this power initialization, we rewrite the previous smooth non-convex problem into a smooth strongly-convex problem that we solve using a parallel Nesterov gradient descent potentially requiring a single step of communication at the initialization step. We provide a linear rate of convergence of the excess loss, our results improve the rates of convergence given in the literature. We provide an upper bound on the Frobenius-norm error of reconstruction under the power initialization strategy. We complete our analysis with experiments on both synthetic and real data.
TinySubNets: An Efficient and Low Capacity Continual Learning Strategy
Authors: Marcin Pietron; Kamil Faber; Dominik Żurek; Roberto Corizzo
Venue: AAAI 2025 (main)

Abstract: Continual Learning (CL) is a highly relevant setting gaining traction in recent machine learning research. Among CL
works, architectural and hybrid strategies are particularly effective due to their potential to adapt the model architecture as
new tasks are presented. However, many existing solutions do not efficiently exploit model sparsity, and are prone to capacity 
saturation due to their inefficient use of available weights, which limits the number of learnable tasks. In this paper, we
propose TinySubNets (TSN), a novel architectural CL strategy that addresses the issues through the unique combination of
pruning with different sparsity levels, adaptive quantization, and weight sharing. Pruning identifies a subset of weights that
preserve model performance, making less relevant weights available for future tasks. Adaptive quantization allows a single 
weight to be separated into multiple parts which can be assigned to different tasks. Weight sharing between tasks boosts
the exploitation of capacity and task similarity, allowing for the identification of a better trade-off between model accuracy
and capacity. These features allow TSN to efficiently leverage the available capacity, enhance knowledge transfer, and reduce
computational resources consumption. Experimental results involving common benchmark CL datasets and scenarios show
that our proposed strategy achieves better results in terms of accuracy than existing state-of-the-art CL strategies. Moreover, 
our strategy is shown to provide a significantly improved model capacity exploitation.
Denoising Diffusion Variational Inference: Diffusion Models as Expressive Variational Posteriors
Authors: Wasu Top Piriyakulkij; Yingheng Wang; Volodymyr Kuleshov
Venue: AAAI 2025 (main)

Abstract: We propose denoising diffusion variational inference (DDVI), a black-box variational inference algorithm for latent variable models which relies on diffusion models as flexible approximate posteriors. Specifically, our method introduces an expressive class of diffusion-based variational posteriors that perform iterative refinement in latent space; we train these posteriors with a novel regularized evidence lower bound (ELBO) on the marginal likelihood inspired by the wake-sleep algorithm. Our method is easy to implement (it fits a regularized extension of the ELBO), is compatible with black-box variational inference, and outperforms alternative classes of approximate posteriors based on normalizing flows or adversarial networks. We find that DDVI improves inference and learning in deep latent variable models across common benchmarks as well as on a motivating task in biology---inferring latent ancestry from human genomes---where it outperforms strong baselines on the Thousand Genomes dataset.
Iterative Counterfactual Data Augmentation
Authors: Mitchell Plyler; Min Chi
Venue: AAAI 2025 (main)

Abstract: Counterfactual data augmentation (CDA) is a method for controlling information or biases in training datasets by generating a complementary dataset with typically opposing biases. Prior work often either relies on hand-crafted rules or algorithmic CDA methods which can leave unwanted information in the augmented dataset. In this work, we show iterative CDA (ICDA) with initial, high-noise interventions can converge to a state with significantly lower noise. Our ICDA procedure produces a dataset where one target signal in the training dataset maintains high mutual information with a corresponding label and the information of spurious signals are reduced. We show training on the augmented datasets produces rationales on documents that better align with human annotation. Our experiments include six human produced datasets and two large-language model generated datasets.
On Corruption-Robustness in Performative Reinforcement Learning
Authors: Vasilis Pollatos; Debmalya Mandal; Goran Radanovic
Venue: AAAI 2025 (main)

Abstract: In performative Reinforcement Learning (RL), an agent faces a policy-dependent environment: the reward and transition functions depend on the agent's policy. Prior work on performative RL has studied the convergence of repeated retraining approaches to a performatively stable policy. In the finite sample regime, these approaches repeatedly solve for a saddle point of a convex-concave objective, which estimates the Lagrangian of a regularized version of the reinforcement learning problem. In this paper, we aim to extend such repeated retraining approaches, enabling them to operate under corrupted data. More specifically, we consider Huber's ε-contamination model, where an ε fraction of data points is corrupted by arbitrary adversarial noise. We propose a repeated retraining approach based on convex-concave optimization under corrupted gradients and a novel problem-specific robust mean estimator for the gradients. We prove that our approach exhibits last-iterate convergence to an approximately stable policy, with the approximation error linear in √ε. We experimentally demonstrate the importance of accounting for corruption in performative reinforcement learning.
Federated Unsupervised Domain Generalization Using Global and Local Alignment of Gradients
Authors: Farhad Pourpanah; Mahdiyar Molahasani; Milad Soltany; Michael Greenspan; Ali Etemad
Venue: AAAI 2025 (main)

Abstract: We address the problem of federated domain generalization in an unsupervised setting for the first time. We first theoretically establish a connection between domain shift and alignment of gradients in unsupervised federated learning and show that aligning the gradients at both client and server levels can facilitate the generalization of the model to new (target) domains. Building on this insight, we propose a novel method named FedGaLA, which performs gradient alignment at the client level to encourage clients to learn domain-invariant features, as well as global gradient alignment at the server to obtain a more generalized aggregated model. To empirically evaluate our method, we perform various experiments on four commonly used multi-domain datasets, PACS, OfficeHome, DomainNet, and TerraInc. The results demonstrate the effectiveness of our method which outperforms comparable baselines. Ablation and sensitivity studies demonstrate the impact of different components and parameters in our approach.
Enhancing SQL Query Generation with Neurosymbolic Reasoning
Authors: Henrijs Princis; Cristina David; Alan Mycroft
Venue: AAAI 2025 (main)

Abstract: We propose a neurosymbolic architecture aimed at boosting the performance of any Language Model (LM) for SQL query generation. This approach leverages symbolic reasoning to guide the LM's exploration of the search space by considering multiple paths, symbolically evaluating choices at each decision point to choose the next step, with the added novel ability to backtrack. A key innovation is the use of symbolic checks on both partially and fully generated SQL queries, enabling early truncation of unsuccessful search paths. Input consists of textual requirements on the desired query, along with optional example tuples to be selected by the query. Experiments on Xander, our open-source implementation, show it both reduces runtime and increases accuracy of the generated SQL. A specific result is an LM using Xander outperforming a four-times-larger LM.
Robust Self-Paced Hashing for Cross-Modal Retrieval with Noisy Labels
Authors: Ruitao Pu; Yuan Sun; Yang Qin; Zhenwen Ren; Xiaomin Song; Huiming Zheng; Dezhong Peng
Venue: AAAI 2025 (main)

Abstract: Cross-modal hashing (CMH) has appeared as a popular technique for cross-modal retrieval due to its low storage cost and high computational efficiency in large-scale data. Most existing methods implicitly assume that multi-modal data is correctly labeled, which is expensive and even unattainable due to the inevitable imperfect annotations (i.e., noisy labels) in real-world scenarios. Inspired by human cognitive learning, a few methods introduce self-paced learning to gradually train the model from easy to hard samples, which is often used to mitigate the effects of feature noise or outliers. It is a less-touched problem that how to utilize SPL to alleviate the misleading of noisy labels on the hash model. To tackle this problem, we propose a new cognitive cross-modal retrieval method called Robust Self-paced Hashing with Noisy Labels (RSHNL), which can mimic the human cognitive process to identify the noise while embracing robustness against noisy labels. Specifically, we first propose a contrastive hashing learning (CHL) scheme to improve multi-modal consistency, thereby reducing the inherent semantic gap. Afterward, we propose center aggregation learning (CAL) to mitigate the intra-class variations. Finally, we propose Noise-tolerance Self-paced Hashing (NSH) that dynamically estimates the learning difficulty for each instance and distinguishes noisy labels through the difficulty level. For all estimated clean pairs, we further adopt a self-paced regularizer to gradually learn hash codes from easy to hard. Extensive experiments demonstrate that the proposed RSHNL performs remarkably well over the state-of-the-art CMH methods.
Leveraging Group Classification with Descending Soft Labeling for Deep Imbalanced Regression
Authors: Ruizhi Pu; Gezheng Xu; Ruiyi Fang; Bing-Kun Bao; Charles Ling; Boyu Wang
Venue: AAAI 2025 (main)

Abstract: Deep imbalanced regression (DIR), where the target values have a highly skewed distribution and are also continuous, is an intriguing yet under-explored problem in machine learning. 
  While recent works have already shown that incorporating various classification-based regularizers can produce enhanced outcomes, the role of classification remains elusive in DIR.
  Moreover, such regularizers (e.g., contrastive penalties) merely focus on learning discriminative features of data, which inevitably results in ignorance of either continuity or similarity across the data.
  To address these issues, we first bridge the connection between the objectives of DIR and classification from a Bayesian perspective. 
  Consequently, this motivates us to decompose the objective of DIR into a combination of classification and regression tasks, which naturally guides us toward a divide-and-conquer manner to solve the DIR problem.
  Specifically, by aggregating the data at nearby labels into the same groups, we introduce an ordinal group-aware contrastive learning loss along with a multi-experts regressor to tackle the different groups of data thereby maintaining the data continuity.
  Meanwhile, considering the similarity between the groups, we also propose a symmetric descending soft labeling strategy to exploit the intrinsic similarity across the data, which allows classification to facilitate regression more effectively.
  Extensive experiments on real-world datasets also validate the effectiveness of our method.
Cross-Silo Feature Space Alignment for Federated Learning on Clients with Imbalanced Data
Authors: Zhuang Qi; Lei Meng; Zhaochuan Li; Han Hu; Xiangxu Meng
Venue: AAAI 2025 (main)

Abstract: Data imbalance across clients in federated learning often leads to different local feature space partitions, harming the global model's generalization ability. Existing methods either employ knowledge distillation to guide consistent local training or performs procedures to calibrate local models before aggregation. However, they overlook the ill-posed model aggregation caused by imbalanced representation learning. To address this issue, this paper presents a cross-silo feature space alignment method (FedFSA), which learns a unified feature space for clients to bridge inconsistency. Specifically, FedFSA consists of two modules, where the in-silo prototypical space learning (ISPSL) module uses predefined text embeddings to regularize representation learning, which can improve the distinguishability of representations on imbalanced data. Subsequently, it introduces a variance transfer approach to construct the prototypical space, which aids in calibrating minority classes feature distribution and provides necessary information for the cross-silo feature space alignment (CSFSA) module. Moreover, the CSFSA module utilizes augmented features learned from the ISPSL module to learn a generalized mapping and align these features from different sources into a common space, which mitigates the negative impact caused by imbalanced factors. Experimental results from three datasets verified that FedFSA improves the consistency between diverse spaces on imbalanced data, which results in superior performance compared to existing methods.
DHAKR: Learning Deep Hierarchical Attention-Based Kernelized Representations for Graph Classification
Authors: Feifei Qian; Lu Bai; Lixin Cui; Ming Li; Ziyu Lyu; Hangyuan Du; Edwin Hancock
Venue: AAAI 2025 (main)

Abstract: Graph-based representations are powerful tools for analyzing structured data. In this paper, we propose a novel model to learn Deep Hierarchical Attention-based Kernelized Representations (DHAKR) for graph classification. To this end, we commence by learning an assignment matrix to hierarchically map the substructure invariants into a set of composite invariants, resulting in hierarchical kernelized representations for graphs. Moreover, we introduce the feature-channel attention mechanism to capture the interdependencies between different substructure invariants that will be converged into the composite invariants, addressing the shortcoming of discarding the importance of different substructures arising in most existing R-convolution graph kernels. We show that the proposed DHAKR model can adaptively compute the kernel-based similarity between graphs, identifying the common structural patterns over all graphs. Experiments demonstrate the effectiveness of the proposed DHAKR model.
Advancing Retrosynthesis with Retrieval-Augmented Graph Generation
Authors: Anjie Qiao; Zhen Wang; Jiahua Rao; Yuedong Yang; Zhewei Wei
Venue: AAAI 2025 (main)

Abstract: Diffusion-based molecular graph generative models have achieved significant success in template-free, single-step retrosynthesis prediction. However, these models typically generate reactants from scratch, often overlooking the fact that the scaffold of a product molecule typically remains unchanged during chemical reactions. To leverage this useful observation, we introduce a retrieval-augmented molecular graph generation framework. Our framework comprises three key components: a retrieval component that identifies similar molecules for the given product, an integration component that learns valuable clues from these molecules about which part of the product should remain unchanged, and a base generative model that is prompted by these clues to generate the corresponding reactants. We explore various design choices for critical and under-explored aspects of this framework and instantiate it as the Retrieval-Augmented RetroBridge (RARB). RARB demonstrates state-of-the-art performance on standard benchmarks, achieving a 14.8% relative improvement in top-1 accuracy over its base generative model, highlighting the effectiveness of retrieval augmentation. Additionally, RARB excels in handling out-of-distribution molecules, and its advantages remain significant even with smaller models or fewer denoising steps. These strengths make RARB highly valuable for real-world retrosynthesis applications, where extrapolation to novel molecules and high-throughput prediction are essential.
Federated t-SNE and UMAP for Distributed Data Visualization
Authors: Dong Qiao; Xinxian Ma; Jicong Fan
Venue: AAAI 2025 (main)

Abstract: High-dimensional data visualization is crucial in big data era and these techniques such as t-SNE and UMAP have been widely used in science and engineering. Big data, however, is often distributed across multiple data centers and subject to security and privacy concerns, which leads to difficulties for the standard algorithms of t-SNE and UMAP. To tackle the challenge, this work proposes Fed-tSNE and Fed-UMAP, which provide high-dimensional data visualization under the framework of federated learning, without exchanging data across clients or sending data to the central server. The main idea of Fed-tSNE and Fed-UMAP is implicitly learning the distribution information of data in a manner of federated learning and then estimating the global distance matrix for t-SNE and UMAP. To further enhance the protection of data privacy, we propose Fed-tSNE+ and Fed-UMAP+. We also extend our idea to federated spectral clustering, yielding algorithms of clustering distributed data. In addition to these new algorithms, we offer theoretical guarantees of distance and similarity estimation and analyze the property of differential privacy. Experiments on multiple datasets demonstrate that, compared to the original algorithms, the accuracy drops of our federated algorithms are tiny.
Certified Causal Defense with Generalizable Robustness
Authors: Yiran Qiao; Yu Yin; Chen Chen; Jing Ma
Venue: AAAI 2025 (main)

Abstract: While machine learning models have proven effective across various scenarios, it is widely acknowledged that many models are vulnerable to adversarial attacks. Recently, numerous efforts have emerged in adversarial defense. Among them, certified defense is well known for its theoretical guarantees against arbitrary adversarial perturbations on input within a certain range. However, most existing works in this line struggle to generalize their certified robustness in other data domains with distribution shifts. This issue is rooted in the difficulty of eliminating the negative impact of spurious correlations on robustness in different domains. To address this problem, in this work, we propose a novel certified defense framework GLEAN, which incorporates a causal perspective into the generalization problem in certified defense. More specifically, our framework integrates a certifiable causal factor learning component to disentangle the causal relations and spurious correlations between input and label, thereby excluding the negative effect of spurious correlations on defense. On top of that, we design a causally certified defense strategy to handle adversarial attacks on latent causal factors. In this way, our framework is not only robust against malicious noises on data in the training distribution but also can generalize its robustness across domains with distribution shifts. Extensive experiments on benchmark datasets validate the superiority of our framework in certified robustness generalization in different data domains.
SUMO: Search-Based Uncertainty Estimation for Model-Based Offline Reinforcement Learning
Authors: Zhongjian Qiao; Jiafei Lyu; Kechen Jiao; Qi Liu; Xiu Li
Venue: AAAI 2025 (main)

Abstract: The performance of offline reinforcement learning (RL) suffers from the limited size and quality of static datasets. Model-based offline RL addresses this issue by generating synthetic samples through a dynamics model to enhance overall performance. To evaluate the reliability of the generated samples, uncertainty estimation methods are often employed. However, model ensemble, the most commonly used uncertainty estimation method, is not always the best choice. In this paper, we propose a Search-based Uncertainty estimation method for Model-based Offline RL (SUMO) as an alternative. SUMO characterizes the uncertainty of synthetic samples by measuring their cross entropy against the in-distribution dataset samples, and uses an efficient search-based method for implementation. In this way, SUMO can achieve trustworthy uncertainty estimation. We integrate SUMO into several model-based offline RL algorithms including MOPO and Adapted MOReL (AMOReL), and provide theoretical analysis for them. Extensive experimental results on D4RL datasets demonstrate that SUMO can provide accurate uncertainty estimation and boost the performance of base algorithms. These indicate that SUMO could be a better uncertainty estimator for model-based offline RL when used in either reward penalty or trajectory truncation.
Scalable One-Pass Incomplete Multi-View Clustering by Aligning Anchors
Authors: Yalan Qin; Guorui Feng; Xinpeng Zhang
Venue: AAAI 2025 (main)

Abstract: Multi-view clustering has gained increasing attention by utilizing the complementary and consensus information across views. To alleviate the computation cost for the existing multi-view clustering approaches on datasets with large scales, studies based on anchor have been presented. Although extensively adopted in the real scenarios, most of these works ignore to learn an integral subspace revealing the cluster structure with anchors from different views being aligned, where the centroid and cluster assignment matrix can be directly achieved based on the integral subspace. Moreover, these works neglect to perform the alignment among anchors and integral subspace learning in a unified model on the incomplete multi-view dataset. Then the mutual improvements among aligning anchors and learning integral subspace are not guaranteed in optimizing the objective function, which inevitably limit the representation ability of the model and result in the suboptimal clustering performance. In this paper, we propose a novel anchor learning method for incomplete multi-view dataset termed Scalable One-pass incomplete Multi-view clustEring by Aligning anchorS (SOME-AS). Specifically, we capture the complementary information among multiple views by building the anchor graph for each view on the incomplete dataset. The integral subspace reflecting the cluster structure is learned with the alignment among anchors from different views being considered. We build the cluster assignment and centroid representation with orthogonal constraint to approximate the integral subspace. Then the subspace itself and the partition are simultaneously taken into account in this manner. Besides, the mutual improvements among aligning anchors and learning integral subspace are able to be ensured. Experiments on several incomplete multi-view datasets validate the efficiency and effectiveness of SOME-AS.