GenAI4ELab/papercli-papers huggingface dataset:
HALL-E: Hierarchical Neural Codec Language Model for Minute-Long Zero-Shot Text-to-Speech Synthesis
Authors: Yuto Nishimura; Takumi Hirose; Masanari Ohi; Hideki Nakayama; Nakamasa Inoue
Venue: ICLR 2025 (ICLR 2025 Poster)

Abstract: Recently, Text-to-speech (TTS) models based on large language models (LLMs)
that translate natural language text into sequences of discrete audio tokens have
gained great research attention, with advances in neural audio codec (NAC) mod-
els using residual vector quantization (RVQ). However, long-form speech synthe-
sis remains a significant challenge due to the high frame rate, which increases the
length of audio tokens and makes it difficult for autoregressive language models
to generate audio tokens for even a minute of speech. To address this challenge,
this paper introduces two novel post-training approaches: 1) Multi-Resolution Re-
quantization (MReQ) and 2) HALL-E. MReQ is a framework to reduce the frame
rate of pre-trained NAC models. Specifically, it incorporates multi-resolution
residual vector quantization (MRVQ) module that hierarchically reorganizes dis-
crete audio tokens through teacher-student distillation. HALL-E is an LLM-based
TTS model designed to predict hierarchical tokens of MReQ. Specifically, it incor-
porates the technique of using MRVQ sub-modules and continues training from a
pre-trained LLM-based TTS model. Furthermore, to promote TTS research, we
create MinutesSpeech, a new benchmark dataset consisting of 40k hours of filtered
speech data for training and evaluating speech synthesis ranging from 3s up to
180s. In experiments, we demonstrated the effectiveness of our approaches by ap-
plying our post-training framework to VALL-E. We achieved the frame rate down
to as low as 8 Hz, enabling the stable minitue-long speech synthesis in a single
inference step. Audio samples, dataset, codes and pre-trained models are available
at https://yutonishimura-v2.github.io/HALL-E_DEMO.
Zeroth-Order Policy Gradient for Reinforcement Learning from Human Feedback without Reward Inference
Authors: Qining Zhang; Lei Ying
Venue: ICLR 2025 (ICLR 2025 Poster)

Abstract: Reward inference (learning a reward model from human preferences) is a critical intermediate step in the Reinforcement Learning from Human Feedback (RLHF) pipeline for fine-tuning Large Language Models (LLMs). In practice, RLHF faces fundamental challenges such as distribution shift, reward model overfitting, and problem misspecification. An alternative approach is direct policy optimization without reward inference, such as Direct Preference Optimization (DPO), which provides a much simpler pipeline and has shown empirical success in LLM applications. However, DPO utilizes the closed-form expression between the optimal policy and the reward function, which is only suitable under the bandit setting or deterministic MDPs. This paper develops two RLHF algorithms without reward inference for general RL problems beyond bandits and deterministic MDPs, and general preference models beyond the Bradley-Terry model. The key idea is to estimate the local value function difference from human preferences and then approximate the policy gradient with a zeroth-order gradient approximator. For both algorithms, we establish polynomial convergence rates in terms of the number of policy gradient iterations, the number of trajectory samples, and human preference queries per iteration. Numerical experiments in stochastic environments validate the performance of our proposed algorithms, outperforming popular RLHF baselines such as DPO and PPO. Our paper shows there exist provably efficient methods to solve general RLHF problems without reward inference.
Zigzag Diffusion Sampling: Diffusion Models Can Self-Improve via Self-Reflection
Authors: Bai LiChen; Shitong Shao; zikai zhou; Zipeng Qi; zhiqiang xu; Haoyi Xiong; Zeke Xie
Venue: ICLR 2025 (ICLR 2025 Poster)

Abstract: Diffusion models, the most popular generative paradigm so far, can inject conditional information into the generation path to guide the latent towards desired directions. However, existing text-to-image diffusion models often fail to maintain high image quality and high prompt-image alignment for those challenging prompts. To mitigate this issue and enhance existing pretrained diffusion models, we mainly made three contributions in this paper. First, we propose **diffusion self-reflection** that alternately performs denoising and inversion and demonstrate that such diffusion self-reflection can leverage the guidance gap between denoising and inversion to capture prompt-related semantic information with theoretical and empirical evidence. Second, motivated by theoretical analysis, we derive Zigzag Diffusion Sampling (Z-Sampling), a novel self-reflection-based diffusion sampling method that leverages the guidance gap between denosing and inversion to accumulate semantic information step by step along the sampling path, leading to improved sampling results. Moreover, as a plug-and-play method, Z-Sampling can be generally applied to various diffusion models (e.g., accelerated ones and Transformer-based ones) with very limited coding and computational costs. Third, our extensive experiments demonstrate that Z-Sampling can generally and significantly enhance generation quality across various benchmark datasets, diffusion models, and performance evaluation metrics. For example, DreamShaper with Z-Sampling can self-improve with the HPSv2 winning rate up to **94%** over the original results. Moreover, Z-Sampling can further enhance existing diffusion models combined with other orthogonal methods, including Diffusion-DPO. The code is publicly available at
[github.com/xie-lab-ml/Zigzag-Diffusion-Sampling](https://github.com/xie-lab-ml/Zigzag-Diffusion-Sampling).
Discrete Latent Plans via Semantic Skill Abstractions
Authors: Haobin Jiang; Jiangxing Wang; Zongqing Lu
Venue: ICLR 2025 (ICLR 2025 Poster)

Abstract: Skill learning from language instructions is a critical challenge in developing intelligent agents that can generalize across diverse tasks and follow complex human instructions. Hierarchical methods address this by decomposing the learning problem into multiple levels, where the high-level and low-level policies are mediated through a latent plan space. Effective modeling and learning of this latent plan space are key to enabling robust and interpretable skill learning. In this paper, we introduce LADS, a hierarchical approach that learns language-conditioned discrete latent plans through semantic skill abstractions. Our method decouples the learning of the latent plan space from the language-conditioned high-level policy to improve training stability. First, we incorporate a trajectory encoder to learn a discrete latent space with the low-level policy, regularized by language instructions. Next, we model the high-level policy as a categorical distribution over these discrete latent plans to capture the multi-modality of the dataset. Through experiments in simulated control environments, we demonstrate that LADS outperforms state-of-the-art methods in both skill learning and compositional generalization.
Neuralized Markov Random Field for Interaction-Aware Stochastic Human Trajectory Prediction
Authors: Zilin Fang; David Hsu; Gim Hee Lee
Venue: ICLR 2025 (ICLR 2025 Poster)

Abstract: Interactive human motions and the continuously changing nature of intentions pose significant challenges for human trajectory prediction. In this paper, we present a neuralized Markov random field (MRF)-based motion evolution method for probabilistic interaction-aware human trajectory prediction. We use MRF to model each agent's motion and the resulting crowd interactions over time, hence is robust against noisy observations and enables group reasoning. We approximate the modeled distribution using two conditional variational autoencoders (CVAEs) for efficient learning and inference. Our proposed method achieves state-of-the-art performance on ADE/FDE metrics across two dataset categories: overhead datasets ETH/UCY, SDD, and NBA, and ego-centric JRDB. Furthermore, our approach allows for real-time stochastic inference in bustling environments, making it well-suited for a 30FPS video setting. We open-source our codes at: https://github.com/AdaCompNUS/NMRF_TrajectoryPrediction.git
A General Framework for Producing Interpretable Semantic Text Embeddings
Authors: Yiqun Sun; Qiang Huang; Yixuan Tang; Anthony Kum Hoe Tung; Jun Yu
Venue: ICLR 2025 (ICLR 2025 Poster)

Abstract: Semantic text embedding is essential to many tasks in Natural Language Processing (NLP). While black-box models are capable of generating high-quality embeddings, their lack of interpretability limits their use in tasks that demand transparency. Recent approaches have improved interpretability by leveraging domain-expert-crafted or LLM-generated questions, but these methods rely heavily on expert input or well-prompt design, which restricts their generalizability and ability to generate discriminative questions across a wide range of tasks. To address these challenges, we introduce \algo{CQG-MBQA} (Contrastive Question Generation - Multi-task Binary Question Answering), a general framework for producing interpretable semantic text embeddings across diverse tasks. Our framework systematically generates highly discriminative, low cognitive load yes/no questions through the \algo{CQG} method and answers them efficiently with the \algo{MBQA} model, resulting in interpretable embeddings in a cost-effective manner. We validate the effectiveness and interpretability of \algo{CQG-MBQA} through extensive experiments and ablation studies, demonstrating that it delivers embedding quality comparable to many advanced black-box models while maintaining inherently interpretability. Additionally, \algo{CQG-MBQA} outperforms other interpretable text embedding methods across various downstream tasks. The source code is available at \url{https://github.com/dukesun99/CQG-MBQA}.
Unifying Causal Representation Learning with the Invariance Principle
Authors: Dingling Yao; Dario Rancati; Riccardo Cadei; Marco Fumero; Francesco Locatello
Venue: ICLR 2025 (ICLR 2025 Poster)

Abstract: Causal representation learning (CRL) aims at recovering latent causal variables from high-dimensional observations to solve causal downstream tasks, such as predicting the effect of new interventions or more robust classification. 
  A plethora of methods have been developed, each tackling carefully crafted problem settings that lead to different types of identifiability. 
  These different settings are widely assumed to be important because they are often linked to different rungs of Pearl's causal hierarchy, even though this correspondence is not always exact.
    This work shows that instead of strictly conforming to this hierarchical mapping, *many causal representation learning approaches methodologically align their representations with inherent data symmetries.*
  Identification of causal variables is guided by invariance principles that are not necessarily causal. 
  This result allows us to unify many existing approaches in a single method that can mix and match different assumptions, including non-causal ones, based on the invariance relevant to the problem at hand. 
  It also significantly benefits applicability, which we demonstrate by improving treatment effect estimation on real-world high-dimensional ecological data. Overall, this paper clarifies the role of causal assumptions in the discovery of causal variables and shifts the focus to preserving data symmetries.
A Multiscale Frequency Domain Causal Framework for Enhanced Pathological Analysis
Authors: Xiaoyu Cui; Weixing Chen; Jiandong Su
Venue: ICLR 2025 (ICLR 2025 Poster)

Abstract: Multiple Instance Learning (MIL) in digital pathology Whole Slide Image (WSI) analysis has shown significant progress. However, due to data bias and unobservable confounders, this paradigm still faces challenges in terms of performance and interpretability. Existing MIL methods might identify patches that do not have true diagnostic significance, leading to false correlations, and experience difficulties in integrating multi-scale features and handling unobservable confounders. To address these issues, we propose a new Multi-Scale Frequency Domain Causal framework (MFC). This framework employs an adaptive memory module to estimate the overall data distribution through multi-scale frequency-domain information during training and simulates causal interventions based on this distribution to mitigate confounders in pathological diagnosis tasks. The framework integrates the Multi-scale Spatial Representation Module (MSRM), Frequency Domain Structure Representation Module (FSRM), and Causal Memory Intervention Module (CMIM) to enhance the model's performance and interpretability. Furthermore, the plug-and-play nature of this framework allows it to be broadly applied across various models. Experimental results on Camelyon16 and TCGA-NSCLC dataset show that, compared to previous work, our method has significantly improved accuracy and generalization ability, providing a new theoretical perspective for medical image analysis and potentially advancing the field further. The code will be released at https://github.com/WissingChen/MFC-MIL.
WeatherGFM: Learning a Weather Generalist Foundation Model via In-context Learning
Authors: Xiangyu Zhao; Zhiwang Zhou; zhangwenlong; Yihao Liu; Xiangyu Chen; Junchao Gong; Hao Chen; Ben Fei; Shiqi Chen; Wanli Ouyang; Xiao-Ming Wu; LEI BAI
Venue: ICLR 2025 (ICLR 2025 Poster)

Abstract: The Earth's weather system involves intricate weather data modalities and diverse weather understanding tasks, which hold significant value to human life. 
Existing data-driven models focus on single weather understanding tasks (e.g., weather forecasting). 
While these models have achieved promising results, they fail to tackle various complex tasks within a single and unified model. 
Moreover, the paradigm that relies on limited real observations for a single scenario hinders the model's performance upper bound.
Inspired by the in-context learning paradigm from visual foundation models and large language models, in this paper, we introduce the first generalist weather generalist foundation model (WeatherGFM) to address weather understanding tasks in a unified manner. 
Specifically, we first unify the representation and definition for diverse weather understanding tasks.
Subsequently, we design weather prompt formats to handle different weather data modalities, including single, multiple, and temporal modalities. 
Finally, we adopt a visual prompting question-answering paradigm for the training of unified weather understanding tasks. 
Extensive experiments indicate that our WeatherGFM can effectively handle up to 12 weather understanding tasks, including weather forecasting, super-resolution, weather image translation, and post-processing. Our method also showcases generalization ability on unseen tasks. The source code is available at https://github.com/xiangyu-mm/WeatherGFM.
Following the Human Thread in Social Navigation
Authors: Luca Scofano; Alessio Sampieri; Tommaso Campari; Valentino Sacco; Indro Spinelli; Lamberto Ballan; Fabio Galasso
Venue: ICLR 2025 (ICLR 2025 Spotlight)

Abstract: The success of collaboration between humans and robots in shared environments relies on the robot's real-time adaptation to human motion. Specifically, in Social Navigation, the agent should be close enough to assist but ready to back up to let the human move freely, avoiding collisions. Human trajectories emerge as crucial cues in Social Navigation, but they are partially observable from the robot's egocentric view and computationally complex to process.

We present the first Social Dynamics Adaptation model (SDA) based on the robot's state-action history to infer the social dynamics. We propose a two-stage Reinforcement Learning framework: the first learns to encode the human trajectories into social dynamics and learns a motion policy conditioned on this encoded information, the current status, and the previous action. Here, the trajectories are fully visible, i.e.,  assumed as privileged information. In the second stage, the trained policy operates without direct access to trajectories. Instead, the model infers the social dynamics solely from the history of previous actions and statuses in real-time.
Tested on the novel Habitat 3.0 platform, SDA sets a novel state-of-the-art (SotA) performance in finding and following humans. 

The code can be found at https://github.com/L-Scofano/SDA.
Provable Robust Overfitting Mitigation in Wasserstein Distributionally Robust Optimization
Authors: Shuang Liu; Yihan Wang; Yifan Zhu; Yibo Miao; Xiao-Shan Gao
Venue: ICLR 2025 (ICLR 2025 Poster)

Abstract: Wasserstein distributionally robust optimization (WDRO) optimizes against worst-case distributional shifts within a specified uncertainty set, leading to enhanced generalization on unseen adversarial examples, compared to standard adversarial training which focuses on pointwise adversarial perturbations. However, WDRO still suffers fundamentally from the robust overfitting problem, as it does not consider statistical error. We address this gap by proposing a novel robust optimization framework under a new uncertainty set for adversarial noise via Wasserstein distance and statistical error via Kullback-Leibler divergence, called the Statistically Robust WDRO. We establish a robust generalization bound for the new optimization framework, implying that out-of-distribution adversarial performance is at least as good as the statistically robust training loss with high probability. Furthermore, we derive conditions under which Stackelberg and Nash equilibria exist between the learner and the adversary, giving an optimal robust model in certain sense.Finally, through extensive experiments, we demonstrate that our method significantly mitigates robust overfitting and enhances robustness within the framework of WDRO.
Don't flatten, tokenize! Unlocking the key to SoftMoE's efficacy in deep RL
Authors: Ghada Sokar; Johan Samir Obando Ceron; Aaron Courville; Hugo Larochelle; Pablo Samuel Castro
Venue: ICLR 2025 (ICLR 2025 Spotlight)

Abstract: The use of deep neural networks in reinforcement learning (RL) often suffers from performance degradation as model size increases. While soft mixtures of experts (SoftMoEs) have recently shown promise in mitigating this issue for online RL, the reasons behind their effectiveness remain largely unknown. In this work we provide an in-depth analysis identifying the key factors driving this performance gain. We discover the surprising result that tokenizing the encoder output, rather than the use of multiple experts, is what is behind the efficacy of SoftMoEs. Indeed, we demonstrate that even with an appropriately scaled single expert, we are able to maintain the performance gains, largely thanks to tokenization.
Direct Distributional Optimization for Provable Alignment of Diffusion Models
Authors: Ryotaro Kawata; Kazusato Oko; Atsushi Nitanda; Taiji Suzuki
Venue: ICLR 2025 (ICLR 2025 Poster)

Abstract: We introduce a novel alignment method for diffusion models from distribution optimization perspectives while providing rigorous convergence guarantees.
We first formulate the problem as a generic regularized loss minimization over probability distributions and directly optimize the distribution using the Dual Averaging method.
Next, we enable sampling from the learned distribution by approximating its score function via Doob's $h$-transform technique.
The proposed framework is supported by rigorous convergence guarantees and an end-to-end bound on the sampling error, which imply that when the original distribution's score is known accurately, the complexity of sampling from shifted distributions is independent of isoperimetric conditions.
This framework is broadly applicable to general distribution optimization problems, including alignment tasks in Reinforcement Learning with Human Feedback (RLHF), Direct Preference Optimization (DPO), and Kahneman-Tversky Optimization (KTO). We empirically validate its performance on synthetic and image datasets using the DPO objective.
MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific Hypotheses
Authors: Zonglin Yang; Wanhao Liu; Ben Gao; Tong Xie; Yuqiang Li; Wanli Ouyang; Soujanya Poria; Erik Cambria; Dongzhan Zhou
Venue: ICLR 2025 (ICLR 2025 Poster)

Abstract: Scientific discovery contributes largely to the prosperity of human society, and recent progress shows that LLMs could potentially catalyst the process. However, it is still unclear whether LLMs can discover novel and valid hypotheses in chemistry. In this work, we investigate this main research question: whether LLMs can automatically discover novel and valid chemistry research hypotheses, given only a research question? With extensive discussions with chemistry experts, we adopt the assumption that a majority of chemistry hypotheses can be resulted from a research background question and several inspirations. With this key insight, we break the main question into three smaller fundamental questions. In brief, they are: (1) given a background question, whether LLMs can retrieve good inspirations; (2) with background and inspirations, whether LLMs can lead to hypothesis; and (3) whether LLMs can identify good hypotheses to rank them higher. To investigate these questions, we construct a benchmark consisting of 51 chemistry papers published in Nature or a similar level in 2024 (all papers are only available online since 2024). Every paper is divided by chemistry PhD students into three components: background, inspirations, and hypothesis. The goal is to rediscover the hypothesis given only the background and a large chemistry literature corpus consisting the ground truth inspiration papers, with LLMs trained with data up to 2023. We also develop an LLM-based multi-agent framework that leverages the assumption, consisting of three stages reflecting the more smaller questions. The proposed method can rediscover many hypotheses with very high similarity with the ground truth ones, covering the main innovations.