GenAI4ELab/papercli-papers huggingface dataset:
Long-Tailed Out-of-Distribution Detection via Normalized Outlier Distribution Adaptation
Authors: Wenjun Miao; Guansong Pang; Jin Zheng; Xiao Bai
Venue: NeurIPS 2024 (NeurIPS 2024 poster)

Abstract: One key challenge in Out-of-Distribution (OOD) detection is the absence of ground-truth OOD samples during training. One principled approach to address this issue is to use samples from external datasets as outliers ($\textit{i.e.}$, pseudo OOD samples) to train OOD detectors.
  However, we find empirically that the outlier samples often present a distribution shift compared to the true OOD samples, especially in Long-Tailed Recognition (LTR) scenarios, where ID classes are heavily imbalanced, $\textit{i.e.}$, the true OOD samples exhibit very different probability distribution to the head and tailed ID classes from the outliers.
  In this work, we propose a novel approach, namely $\textit{normalized outlier distribution adaptation}$ (AdaptOD), to tackle this distribution shift problem.
  One of its key components is $\textit{dynamic outlier distribution adaptation}$ that effectively adapts a vanilla outlier distribution based on the outlier samples to the true OOD distribution by utilizing the OOD knowledge in the predicted OOD samples during inference.
  Further, to obtain a more reliable set of predicted OOD samples on long-tailed ID data, a novel $\textit{dual-normalized energy loss}$ is introduced in AdaptOD, which leverages class- and sample-wise normalized energy to enforce a more balanced prediction energy on imbalanced ID samples. This helps avoid bias toward the head samples and learn a substantially better vanilla outlier distribution than existing energy losses during training. It also eliminates the need of manually tuning the sensitive margin hyperparameters in energy losses.
  Empirical results on three popular benchmarks for OOD detection in LTR show the superior performance of AdaptOD over state-of-the-art methods.
Code is available at https://github.com/mala-lab/AdaptOD.
Geometric-Averaged Preference Optimization for Soft Preference Labels
Authors: Hiroki Furuta; Kuang-Huei Lee; Shixiang Shane Gu; Yutaka Matsuo; Aleksandra Faust; Heiga Zen; Izzeddin Gur
Venue: NeurIPS 2024 (NeurIPS 2024 poster)

Abstract: Many algorithms for aligning LLMs with human preferences assume that human preferences are binary and deterministic.
However, human preferences can vary across individuals, and therefore should be represented distributionally.
In this work, we introduce the distributional soft preference labels and improve Direct Preference Optimization (DPO) with a weighted geometric average of the LLM output likelihood in the loss function.
This approach adjusts the scale of learning loss based on the soft labels such that the loss would approach zero when the responses are closer to equally preferred.
This simple modification can be easily applied to any DPO-based methods and mitigate over-optimization and objective mismatch, which prior works suffer from.
Our experiments simulate the soft preference labels with AI feedback from LLMs and demonstrate that geometric averaging consistently improves performance on standard benchmarks for alignment research. 
In particular, we observe more preferable responses than binary labels and significant improvements where modestly-confident labels are in the majority.
Predicting Future Actions of Reinforcement Learning Agents
Authors: Stephen Chung; Scott Niekum; David Krueger
Venue: NeurIPS 2024 (NeurIPS 2024 poster)

Abstract: As reinforcement learning agents become increasingly deployed in real-world scenarios, predicting future agent actions and events during deployment is important for facilitating better human-agent interaction and preventing catastrophic outcomes. This paper experimentally evaluates and compares the effectiveness of future action and event prediction for three types of RL agents: explicitly planning, implicitly planning, and non-planning. We employ two approaches: the inner state approach, which involves predicting based on the inner computations of the agents (e.g., plans or neuron activations), and a simulation-based approach, which involves unrolling the agent in a learned world model. Our results show that the plans of explicitly planning agents are significantly more informative for prediction than the neuron activations of the other types. Furthermore, using internal plans proves more robust to model quality compared to simulation-based approaches when predicting actions, while the results for event prediction are more mixed. These findings highlight the benefits of leveraging inner states and simulations to predict future agent actions and events, thereby improving interaction and safety in real-world deployments.
Learning to Merge Tokens via Decoupled Embedding for Efficient Vision Transformers
Authors: Dong Hoon Lee; Seunghoon Hong
Venue: NeurIPS 2024 (NeurIPS 2024 poster)

Abstract: Recent token reduction methods for Vision Transformers (ViTs) incorporate token merging, which measures the similarities between token embeddings and combines the most similar pairs.
However, their merging policies are directly dependent on intermediate features in ViTs, which prevents exploiting features tailored for merging and requires end-to-end training to improve token merging.
In this paper, we propose Decoupled Token Embedding for Merging (DTEM) that enhances token merging through a decoupled embedding learned via a continuously relaxed token merging process.
Our method introduces a lightweight embedding module decoupled from the ViT forward pass to extract dedicated features for token merging, thereby addressing the restriction from using intermediate features.
The continuously relaxed token merging, applied during training, enables us to learn the decoupled embeddings in a differentiable manner.
Thanks to the decoupled structure, our method can be seamlessly integrated into existing ViT backbones and trained either modularly by learning only the decoupled embeddings or end-to-end by fine-tuning. 
We demonstrate the applicability of DTEM on various tasks, including classification, captioning, and segmentation, with consistent improvement in token merging.
Especially in the ImageNet-1k classification, DTEM achieves a 37.2\% reduction in FLOPs while maintaining a top-1 accuracy of 79.85\% with DeiT-small.
Can LLMs Learn by Teaching for Better Reasoning? A Preliminary Study
Authors: Xuefei Ning; Zifu Wang; Shiyao Li; Zinan Lin; Peiran Yao; Tianyu Fu; Matthew B. Blaschko; Guohao Dai; Huazhong Yang; Yu Wang
Venue: NeurIPS 2024 (NeurIPS 2024 poster)

Abstract: Teaching to improve student models (e.g., knowledge distillation) is an extensively studied methodology in LLMs. However, in human education, teaching enhances not only the students but also the teachers by fostering more rigorous and clearer reasoning, as well as deeper knowledge building. We ask: Can LLMs also learn by teaching (LbT) for better reasoning? If the answer is yes, we can potentially unlock the possibility of continuously advancing the models without solely relying on human-produced data or stronger models. In this paper, we provide a preliminary exploration of this question. We show that LbT ideas can be incorporated into existing LLM training/prompting pipelines and bring improvements. Specifically, we design three methods, each mimicking one of the three levels of LbT: observing students' feedback, learning from the feedback, and learning iteratively, with the goal of improving answer accuracy without training or improving models' inherent capability with fine-tuning. We reveal some findings: (1) Teaching materials that make it easier for students to learn (via in-context learning) have clearer and more accurate logic; (2) Weak-to-strong generalization: LbT might help improve strong models by teaching weak models; (3) Diversity in students might help: teaching multiple students could be better than teaching a single student or the teacher alone. We hope that our exploration can inspire future research on LbT and, more broadly, the adoption of advanced education techniques to improve LLMs. The code and website are at https://github.com/imagination-research/lbt and https://sites.google.com/view/llm-learning-by-teaching.
Measuring Goal-Directedness
Authors: Matt MacDermott; James Fox; Francesco Belardinelli; Tom Everitt
Venue: NeurIPS 2024 (NeurIPS 2024 spotlight)

Abstract: We define maximum entropy goal-directedness (MEG), a formal measure of goal-
directedness in causal models and Markov decision processes, and give algorithms
for computing it. Measuring goal-directedness is important, as it is a critical
element of many concerns about harm from AI. It is also of philosophical interest,
as goal-directedness is a key aspect of agency. MEG is based on an adaptation of
the maximum causal entropy framework used in inverse reinforcement learning. It
can measure goal-directedness with respect to a known utility function, a hypothesis
class of utility functions, or a set of random variables. We prove that MEG satisfies
several desiderata and demonstrate our algorithms with small-scale experiments.
MoGU: A Framework for Enhancing Safety of LLMs While Preserving Their Usability
Authors: Yanrui Du; Sendong Zhao; Danyang Zhao; Ming Ma; Yuhan Chen; Liangyu Huo; Qing Yang; Dongliang Xu; Bing Qin
Venue: NeurIPS 2024 (NeurIPS 2024 poster)

Abstract: Large Language Models (LLMs) are increasingly deployed in various applications. As their usage grows, concerns regarding their safety are rising, especially in maintaining harmless responses when faced with malicious instructions. Many defense strategies have been developed to enhance the safety of LLMs. However, our research finds that existing defense strategies lead LLMs to predominantly adopt a rejection-oriented stance, thereby diminishing the usability of their responses to benign instructions. To solve this problem, we introduce the MoGU framework, designed to enhance LLMs' safety while preserving their usability. Our MoGU framework transforms the base LLM into two variants: the usable LLM and the safe LLM, and further employs dynamic routing to balance their contribution. When encountering malicious instructions, the router will assign a higher weight to the safe LLM to ensure that responses are harmless. Conversely, for benign instructions, the router prioritizes the usable LLM, facilitating usable and helpful responses. On various open-sourced LLMs, we compare multiple defense strategies to verify the superiority of our MoGU framework. Besides, our analysis provides key insights into the effectiveness of MoGU and verifies that our designed routing mechanism can effectively balance the contribution of each variant by assigning weights. Our work released the safer Llama2, Vicuna, Falcon, Dolphin, and Baichuan2.
Graph Classification via Reference Distribution Learning: Theory and Practice
Authors: Zixiao Wang; Jicong Fan
Venue: NeurIPS 2024 (NeurIPS 2024 poster)

Abstract: Graph classification is a challenging problem owing to the difficulty in quantifying the similarity between graphs or representing graphs as vectors, though there have been a few methods using graph kernels or graph neural networks (GNNs). Graph kernels often suffer from computational costs and manual feature engineering, while GNNs commonly utilize global pooling operations, risking the loss of structural or semantic information. This work introduces Graph Reference Distribution Learning (GRDL), an efficient and accurate graph classification method. GRDL treats each graph's latent node embeddings given by GNN layers as a discrete distribution, enabling direct classification without global pooling, based on maximum mean discrepancy to adaptively learned reference distributions. To fully understand this new model (the existing theories do not apply) and guide its configuration (e.g., network architecture, references' sizes, number, and regularization) for practical use, we derive generalization error bounds for GRDL and verify them numerically. More importantly, our theoretical and numerical results both show that GRDL has a stronger generalization ability than GNNs with global pooling operations. Experiments on moderate-scale and large-scale graph datasets show the superiority of GRDL over the state-of-the-art, emphasizing its remarkable efficiency, being at least 10 times faster than leading competitors in both training and inference stages.
PURE: Prompt Evolution with Graph ODE for Out-of-distribution Fluid Dynamics Modeling
Authors: Hao Wu; Changhu Wang; Fan Xu; Jinbao Xue; Chong Chen; Xian-Sheng Hua; Xiao Luo
Venue: NeurIPS 2024 (NeurIPS 2024 poster)

Abstract: This work studies the problem of out-of-distribution fluid dynamics modeling. Previous works usually design effective neural operators to learn from mesh-based data structures. However, in real-world applications, they would suffer from distribution shifts from the variance of system parameters and 
temporal evolution of the dynamical system. In this paper, we propose a novel approach named \underline{P}rompt Evol\underline{u}tion with G\underline{r}aph OD\underline{E} (\method{}) for out-of-distribution fluid dynamics modeling. The core of our \method{} is to learn time-evolving prompts using a graph ODE to adapt spatio-temporal forecasting models to different scenarios. In particular, our \method{} first learns from historical observations and system parameters in the frequency domain to explore multi-view context information, which could effectively initialize prompt embeddings. More importantly, we incorporate the interpolation of observation sequences into a graph ODE, which can capture the temporal evolution of prompt embeddings for model adaptation. These time-evolving prompt embeddings are then incorporated into basic forecasting models to overcome temporal distribution shifts. We also minimize the mutual information between prompt embeddings and observation embeddings to enhance the robustness of our model to different distributions. Extensive experiments on various benchmark datasets validate the superiority of the proposed \method{} in comparison to various baselines.
On the Complexity of Identification in Linear Structural Causal Models
Authors: Julian Dörfler; Benito van der Zander; Markus Bläser; Maciej Liskiewicz
Venue: NeurIPS 2024 (NeurIPS 2024 poster)

Abstract: Learning the unknown causal parameters of a linear structural causal 
model is a fundamental task in causal analysis. The task, known as the 
problem of identification, asks to estimate the parameters of the model from a
combination of assumptions on the graphical structure of the model and 
observational data, represented as a non-causal covariance matrix.
In this paper, we give a new sound and complete algorithm for generic 
identification which runs in polynomial space. By a standard simulation 
result, namely $\mathsf{PSPACE} \subseteq \mathsf{EXP}$,
this algorithm has exponential running time which vastly improves 
the state-of-the-art double exponential time method using a Gröbner basis 
approach. The paper also presents evidence that parameter identification 
is computationally hard in general. In particular, we prove, that the task
asking whether, for a given feasible correlation matrix, there 
are exactly one or two or more parameter sets explaining the observed 
matrix, is hard for $\forall \mathbb{R}$, the co-class of the existential theory 
of the reals. In particular, this problem is $\mathsf{coNP}$-hard.
To our best knowledge, this is the first hardness result for some notion 
of identifiability.
AlchemistCoder: Harmonizing and Eliciting Code Capability by Hindsight Tuning on Multi-source Data
Authors: Zifan Song; Yudong Wang; Wenwei Zhang; Kuikun Liu; Chengqi Lyu; Demin Song; Qipeng Guo; Hang Yan; Dahua Lin; Kai Chen; Cairong Zhao
Venue: NeurIPS 2024 (NeurIPS 2024 poster)

Abstract: Open-source Large Language Models (LLMs) and their specialized variants, particularly Code LLMs, have recently delivered impressive performance. However, previous Code LLMs are typically fine-tuned on single-source data with limited quality and diversity, which may insufficiently elicit the potential of pre-trained Code LLMs. In this paper, we present AlchemistCoder, a series of Code LLMs with enhanced code generation and generalization capabilities fine-tuned on multi-source data. To achieve this, we pioneer to unveil inherent conflicts among the various styles and qualities in multi-source code corpora and introduce data-specific prompts with hindsight relabeling, termed AlchemistPrompts, to harmonize different data sources and instruction-response pairs. Additionally, we propose incorporating the data construction process into the fine-tuning data as code comprehension tasks, including instruction evolution, data filtering, and code review. Extensive experiments demonstrate that AlchemistCoder holds a clear lead among all models of the same size (6.7B/7B) and rivals or even surpasses larger models (15B/33B/70B), showcasing the efficacy of our method in refining instruction-following capabilities and advancing the boundaries of code intelligence. Source code and models are available at https://github.com/InternLM/AlchemistCoder.
The Implicit Bias of Gradient Descent toward Collaboration between Layers: A Dynamic Analysis of Multilayer Perceptions
Authors: Zheng Wang; Geyong Min; Wenjie Ruan
Venue: NeurIPS 2024 (NeurIPS 2024 poster)

Abstract: The implicit bias of gradient descent has long been considered the primary mechanism explaining the superior generalization of over-parameterized neural networks without overfitting, even when the training error is zero. However, the implicit bias toward adversarial robustness has rarely been considered in the research community, although it is crucial for the trustworthiness of machine learning models. To fill this gap, in this paper, we explore whether consecutive layers collaborate to strengthen adversarial robustness during gradient descent. By quantifying this collaboration between layers using our proposed concept, co-correlation, we demonstrate a monotonically increasing trend in co-correlation, which implies a decreasing trend in adversarial robustness during gradient descent. Additionally, we observe different behaviours between narrow and wide neural networks during gradient descent. We conducted extensive experiments that verified our proposed theorems.
Optimal Flow Matching: Learning Straight Trajectories in Just One Step
Authors: Nikita Maksimovich Kornilov; Petr Mokrov; Alexander Gasnikov; Alexander Korotin
Venue: NeurIPS 2024 (NeurIPS 2024 poster)

Abstract: Over the several recent years, there has been a boom in development of Flow Matching (FM) methods for generative modeling. One intriguing property pursued by the community is the ability to learn flows with straight trajectories which realize the Optimal Transport (OT) displacements. Straightness is crucial for the fast integration (inference) of the learned flow's paths. Unfortunately, most existing flow straightening methods are based on non-trivial iterative FM procedures which accumulate the error during training or exploit heuristics based on minibatch OT. To address these issues, we develop and theoretically justify the novel Optimal Flow Matching approach which allows recovering the straight OT displacement for the quadratic transport in just one FM step. The main idea of our approach is the employment of vector field for FM which are parameterized by convex functions. The code of our OFM implementation and the conducted experiments is available at https://github.com/Jhomanik/Optimal-Flow-Matching
VeXKD: The Versatile Integration of Cross-Modal Fusion and Knowledge Distillation for 3D Perception
Authors: JI Yuzhe; Yijie CHEN; Liuqing Yang; Rui Ding; Meng Yang; Xinhu Zheng
Venue: NeurIPS 2024 (NeurIPS 2024 poster)

Abstract: Recent advancements in 3D perception have led to a proliferation of network architectures, particularly those involving multi-modal fusion algorithms. While these fusion algorithms improve accuracy, their complexity often impedes real-time performance. This paper introduces VeXKD, an effective and Versatile framework that integrates Cross-Modal Fusion with Knowledge Distillation. VeXKD applies knowledge distillation exclusively to the Bird's Eye View (BEV) feature maps, enabling the transfer of cross-modal insights to single-modal students without additional inference time overhead. It avoids volatile components that can vary across various 3D perception tasks and student modalities, thus improving versatility. The framework adopts a modality-general cross-modal fusion module to bridge the modality gap between the multi-modal teachers and single-modal students. Furthermore, leveraging byproducts generated during fusion, our BEV query guided mask generation network identifies crucial spatial locations across different BEV feature maps in a data-driven manner, significantly enhancing the effectiveness of knowledge distillation. Extensive experiments on the nuScenes dataset demonstrate notable improvements, with up to 6.9\%/4.2\% increase in mAP and NDS for 3D detection tasks and up to 4.3\% rise in mIoU for BEV map segmentation tasks, narrowing the performance gap with multi-modal models.
Watermarking Makes Language Models Radioactive
Authors: Tom Sander; Pierre Fernandez; Alain Oliviero Durmus; Matthijs Douze; Teddy Furon
Venue: NeurIPS 2024 (NeurIPS 2024 spotlight)

Abstract: We investigate the radioactivity of text generated by large language models (LLM), \ie whether it is possible to detect that such synthetic input was used to train a subsequent LLM.
Current methods like membership inference or active IP protection either work only in settings where the suspected text is known or do not provide reliable statistical guarantees.
We discover that, on the contrary, it is possible to reliably determine if a language model was trained on synthetic data if that data is output by a watermarked LLM.
Our new methods, specialized for radioactivity, detects with a provable confidence weak residuals of the watermark signal in the fine-tuned LLM.
We link the radioactivity contamination level to the following properties: the watermark robustness, its proportion in the training set, and the fine-tuning process.
For instance, if the suspect model is open-weight, we demonstrate that training on watermarked instructions can be detected with high confidence ($p$-value $< 10^{-5}$) even when as little as $5\%$ of training text is watermarked.
ADOPT: Modified Adam Can Converge with Any $\beta_2$ with the Optimal Rate
Authors: Shohei Taniguchi; Keno Harada; Gouki Minegishi; Yuta Oshima; Seongcheol Jeong; Go Nagahara; Tomoshi Iiyama; Masahiro Suzuki; Yusuke Iwasawa; Yutaka Matsuo
Venue: NeurIPS 2024 (NeurIPS 2024 poster)

Abstract: Adam is one of the most popular optimization algorithms in deep learning. However, it is known that Adam does not converge in theory unless choosing a hyperparameter, i.e., $\beta_2$, in a problem-dependent manner. There have been many attempts to fix the non-convergence (e.g., AMSGrad), but they require an impractical assumption that the gradient noise is uniformly bounded. In this paper, we propose a new adaptive gradient method named ADOPT, which achieves the optimal convergence rate of $\mathcal{O} ( 1 / \sqrt{T} )$ with any choice of $\beta_2$ without depending on the bounded noise assumption. ADOPT addresses the non-convergence issue of Adam by removing the current gradient from the second moment estimate and changing the order of the momentum update and the normalization by the second moment estimate. We also conduct intensive numerical experiments, and verify that our ADOPT achieves superior results compared to Adam and its variants across a wide range of tasks, including image classification, generative modeling, natural language processing, and deep reinforcement learning. The implementation is available at https://github.com/iShohei220/adopt.