On the Challenges and Opportunities in Generative AI
Authors: Laura Manduchi
Clara Meister
Kushagra Pandey
Robert Bamler
Ryan Cotterell
Sina Däubener
Sophie Fellenz
Asja Fischer
Thomas Gärtner
Matthias Kirchler
Marius Kloft
Yingzhen Li
Christoph Lippert
Gerard de Melo
Eric Nalisnick
Björn Ommer
Rajesh Ranganath
Maja Rudolph
Karen Ullrich
Guy Van den Broeck
Julia E Vogt
Yixin Wang
Florian Wenzel
Frank Wood
Stephan Mandt
Vincent Fortuin
Venue: Accepted by TMLR ()

Abstract: The field of deep generative modeling has grown rapidly in the last few years. With the availability of massive amounts of training data coupled with advances in scalable unsupervised learning paradigms, recent large-scale generative models show tremendous promise in synthesizing high-resolution images and text, as well as structured data such as videos and molecules. However, we argue that current large-scale generative AI models exhibit several fundamental shortcomings that hinder their widespread adoption across domains. In this work, our objective is to identify these issues and highlight key unresolved challenges in modern generative AI paradigms that should be addressed to further enhance their capabilities, versatility, and reliability. By identifying these challenges, we aim to provide researchers with insights for exploring fruitful research directions, thus fostering the development of more robust and accessible generative AI solutions.

Published in Transactions on Machine Learning Research (08/2025)
On the Challenges and Opportunities in Generative Al
Laura Manduchi;' Clara M Kushagra Pandey;? Robert Bamler, 3 Ryan Cotterell,
Sina Däubener, 4 Sophie Fellenz, 5 Asja Fischer,' Thomas Gärtner, Matthias Kirchler," Marius Kloft, 5 Yingzhen Li, Christoph Lippert, Gerard de Melo, ?9 Eric Nalisnick," 10 Björn Ommer," Rajesh Ranganath," !2 Maja Waldron, 13 Karen Ulrrich, Guy Van den Broeck, 15 Julia E Vogt, Yixin Wang, Florian Wenzel, Frank Wood, 18
Stephan Mandt, Vincent Fortuin 19
'ETH Zürich; UC Irvine; UUniversity of Tübingen: 'Ruhr-Universin University Bochum: "RPTU Kaiserslautern- Landau; TTU Wien; HHsso Plattner Institute; Imperial College London; UUniversity Potsdam; 10 Johns Hopkins University; "LMU Munich; 'eNew York University; UnUiieerstty of Wisconsin-Maddion; 14Meta AI; UCLA: Universiy of Michigan; Mirrll AI; Univverryy of British Columbia; Helmmhottz AI;
T0 Munich
Reviewed on OpenReview: http:p//ppaaariiieeeeeeo//////ia
Abstract
The field of deep generative modeling has grown rapidly in the last few years. With the availability of massive amounts training data coupled with advances in scalable unsuper- vised learning paradigms, recent large-scale generative models show tremendous promise in synthesizing high-resolution images and text, well as structured data such as videos and molecules. However, we argue that current large-scale generative AI models exhibit several fundamental shortcomings that hinder their widespread adoption across domains. In this work, our objective is to identify these issues and highlight key unresolved challenges in mod- ern generative Al paradigms that should be addressed to further enhance their capabilities, versatility, and reliability. By identifying these challenges, we aim to provide researchers with insights for exploring fruitful research directions, thus fostering the development of more robust and accessible generative AI solutions.
1 Introduction
The past few years have demonstrated the immense potential large-scale generative models to create powerful AT tools capable of impacting society profoundly. Large Language Models (LLMs) (Brown et al. 2020 Chowdhery al. 2022 OpenAT, 2023 Rac et al. 2021) and their dialogue agents, such as ChatGPT OpenAl 2023) and Llama Grattafiori et al. 2024) have enabled the development of highly effective text generation systems that produce coherent, contextually relevant, and user-tailored outputs across wide range of use cases. Similarly, advancements in diffusion models (Sohl-Dickstein et al. 2015 Song et al.
2020 Ho et al. 2020 have led to groundbreaking advancements in image synthesis tasks, such as large-scale
texi-to-image generation (Ramesh et al., 2022 Rombach et al. 2022 Saharia et al.
2022 Esser et al.
2024
These successes show that highly effective Al systems can be built using a relatively straightforward
recipe:
combining simple genorative modcling paradigms (Larochelle & Murray] [001]] Sohl-Dickstein al. 2015
with successful network archilectures (Vaswani et al.) 2017 Dosovitskiy et. al. 2020 Ronneberger el. al.
2015
training large-scale datasets, and the incorporation of preferences via human feedback (Ouyang
et al.
2022 Ziegler et al. 2019 The impact of generative Al has not been limited to text and image
Eqqaal contribution. Correspondence laura. manduchi@inf....mmmmmmmmm ethz.
| Equal contribution.

---

Published in Transactions on Machine Learning Research (08/2025)
generation applications. It has fueled accelerated progress across variety of research fields and practical applications, spanning from biology (Jumper et al. 2021 to weather forecasting (Ravuri et al. 2021| code
generation (Chen et al. 2021b Li et al. 2022b), video creation (Yang et al. 2023c Ho et al. 2022b Singer
cct al. 2022 Brooks ct al. 2024]. audio synthesis
Borsos et al. 2023
Liu ct al. 2023a and even artistic
and musical composition Huang el al. 2023b
With the current advancements and excitement surrounding generative AI, question naturally arises: Are we on the brink of an Al utopia? Are we close to developing what we might call a perfect generative model? For the purpose of this survey, we define such model as single system that (i) can approximate the joint data distribution of any modality, (ii) provides calibrated uncertainty assessments and demonstrates causal consistency, and (iii) delivers controllable outputs that satisfy stringent requirements on robustness, safety, efficiency and societal alignment. We argue in this paper that the answer is a resounding no; rather, the realization of such a model, one that would fundamentally transform the field of AI is still hampered by substantial thoorctical, practical, and ethical challenges, and incremental advances alone are unlikely to close the gap in the near term.
Amidst the excitement and anticipation surrounding this new wave of Deep Generative Models (DGMs) it is easy lo overlook the new set of challenges they introduce. Unlike many of the traditional machine learning models, DGMs generate outputs in very high-dimensionall spaces, which introduces several techni- cal complexities. These include significantly increased computational demands, a need for larger datasets to accurately capture the underlying data distribution, and challenges in effectively evaluating and interpreting the generated outputs (Tong et al.) 2024). And while significant progress has been made in improving inter- pretability and computational efficiency for traditional models (Mareinkevies & Vogt, 2020), these existing methods are frequently ill-suited for DGMs (Singh et al. 2024). at least in part because of the complex and high-dimensional nature of their outputs. Consequently, there is pressing need for the development of new set of techniques and tools tailored to these models, particularly to enable efficient inference, in- terpretability and quantization. These challenges lead us to conclude that scaling up current paradigms is not in isolation the ultimate path towards a perfect generative model. While increasing model size and training data can enhance performance on benchmarks (Hoffmann et al. 2022), it does little to address the fundamental shortcomings of DGMs, such as their inefficiency, lack of inclusivity, limited transparency, and barriers to bacelity-pppticularr/y in high-stakes domains where reliability and fairness are paramount.
This work offers a collection of views and opinions from different communities about these key unresolved challenges in generative AI, with the ultimate goal of guiding future research toward what we perceive are the most critical and promising areas. Concretely, we discuss key challenges in (a) broadening the scope and adaptability of DGMs, ie., their ability to robustly generalize across different domains and modalities (Section 2: (b) improving their efficiency and resource utilization, i.c., lower the memory and computational requirements and enhance accessibility and sustainability in their adoption (Section 3 and, finally, (c) addressing ethical and societal concerns that are crucial for responsible deployment (Section
By presenting a broader roadmap of the current state and open challenges in generative AI, this paper offers an integrated entry point for researchers and practitioners. Although many existing surveys offer in-depth reviews of specific subfields, such as robustness, causal modeling, or modality-specific techniques, they are often narrowly focused, making it difficult to grasp the collective landscape. In contrast, we put forward high-level insights across domains, highlight emerging research directions, and guide readers to foundational and topic-specifie work. Tn doing so, our goal is to foster the development of generative AT systems that are more robust, inclusive, and accessible.
This paper emerged as result of the Dagstuhl Seminar on Challenges and Perspectives in Deep Generative Modeling held in Spring 2023.
IIn this paper, take the term Generative Al refer systems whose core component is large-scale DGM. For brevity, refer this class and its instances simply as DGMs. /https://wwe www.wddg.sttddeeooo de/23072
2

---

Published in Transactions on Machine Learning Research (08/2025)
2 Expanding Scope and Adaptability
Sub- challenge
Typical failure mode
Promising mitigation
Reference materials
Research-question candidates
avenues
Robust generalisation to OOD inputs ($2.1)
Large performance
Retrieval-sugmenter
Datasets:
Q1: Can retrieval-augmente
drop unseen domains
generation (RAG):
WILDS ImageNet-CC
generators help close the WILDS gap without model retraining?
Group DRO training
Surveys:
(c.g.. Sagawa" al.
Shen al
2021
Q2: Can provide an inductive bias for foundation models via architectural
2020
Yang
2023b
Li
et
2023c
modifications or training objectives that predisposes them to accurately capture tail events?
Resilience to adversarial perturbations (62.1)
Imperceptible noise fools the model
Gaussian noise smoothing; Certified defense mothods; Adversarial finctuning for diffusion
Benchmarks:
Q1: Will certifiably-robust diffusion sampling algorithms scale to large models, e.g,, ones trained on ImageNet?
RobustBench
Leaderboard NSPW
Adversarial Benchmark
Q2: How can we unify adversarial training across different modalitics for multi-modal models in a single framework?
Surveys:
Sun et. al.
2023b
Mitigating learning of spurious correlations (52.1 2.2)
Model prodicts based on background cues, not capturing meaningful relationships
Counterfactual data augmentation; Invariant Risk Minimization-
Datasets:
Q1: How can reliably detect and quantify hidden spurious cues encoded by foundation-model features? Q8: Which causal probes best expose hidden biases in high-dimensional latent spaces?
Waterbirds
Colored
NST
Surveys:
approaches
et al.
2024
Capturing causal dependencies ($2.2)
Models generate statistically plansible but causally impossible outcomes
SCM-guided loss
Benchmarks:
Qf: How can causal invariance objectives be integrated into generative model training to improve robustness to distribution shifts?
functions; Interventional training objectives
CausalBench
Surveys:
Komanduri et al.
2024
Q2: Can tractable surrogate objectives approximate interventional likelihond, enabling scalable causal DGM training?
Accounting for implicit. assumptions ($2.3)
(Implicitly) assumed characteristics of data-generating distribution do not persist under domain shift
Domain-expert
Q1: How can we quantify the degree of modeling assumption modeling-assumptinn trained generative model before deploying it?
knowledge
violations in
integration; Statistical
assumption testing (e.g., independence, stationarity checks)
Q2: What learning objectives most robust when the true data-generate settraeeenaiing process lics outside the model family?
Cross-modal transfer in specialized domains ($2.4)
Failure/ /inabiit to link signals across modalities (e.g. ECG o notes)
Contrastive
Datasets:
Qf: How can physiological constraints be incorporated into multi-modal pre-training objectives medical domains?
multimodal
MMIST-CCRCC
pre-training (e.g.
GMALMMBench
Raghu et 2022
Surveys:
Shaik al. 2024
Q2: Which training/fie-ttnin /finetuninngggggg methods can achieve sufficient cross-modal alignment in low-resuurce clinical settings?
Table 1: Research challenges summary table Expanding Scope and Adaptability
State-of-tho-art leaderboard rankings show the remarkable progress model performance that has been made by scaling DGMs to massive datasets and model sizes (for instance, in text and high-resolution image synthesis). However, automatic evaluations of model success (Bender et al.) 2021); such evaluations often fail to capture the nuanced limitations of DGMs, such as potential biases, inabilities to generalize to inputs from underrepresented or specialized distributions, and difficulties with aligning outputs with specific domain requirements. Understanding these inherent and often hidden constraints essential for ensuring that DGMs can be reliably applied to various real-world tasks, where data characteristics, domain-spocific constraints, and measures of success may differ significantly from those in standardized benchnarks (Durall et al., 2020 Daunhawer al. 2022 Xu et al. 2024). This section analyzes some of these challenges in the context of large-scale DGMs from the lens
popular benchmark datasets cannot be our only measure
3

---

Published in Transactions on Machine Learning Research (08/2025)
of their generalization capabilities (Section 2.1 and the lack of transparency in their underlying modeling assumptions (Section 2.3 We examine these fundamental challenges and provide research directions that could broaden the adaptability of DGMs to promote long-term progress in the field. We also discuss two promising avenues that have the potential to greatly enhance the scope of gencrative models: (i) integrating causal learning (Section 2.2 and (ii) the development of a versatile, generalist agent capable of handling heterogencous data types (Section 2.4
2.1 Generalization and Robustness
To ensure reliability across various domains, DGMs must generalize effectively under shifts to the data- generating distribution of inputs, often referred to as ouf-of-distribution (OOD) robustness, and be resilient to minor variations in the imput, a necessary component of the broader notion of adversarial robustness. Without proper generalization, generative models may produce unrealistic or biased outputs limiting their practical utility and trustworthiness in rcal-world applications.
While large-scale generalive models show some promise in achieving OOD robustness Wang et al. 2023a these models still face challenges in accurately capturing rare events or responding to adversarial inputs (Zhu et al. 2024), a difficulty that lies in effectively modcling the tail of information (Kandpal et al., 2023 i.e., the information that appears rarely or only once in the dataset used to (pre)train the model. This limitation indicates a gap in their ability to fully represent the vast. and diverse spectrum of real-world scenarios, especially those that are less common but equally significant. Retriievalauugeented language models represent a promising approach for integrating rare specialized knowledge into model outputs, effectively addressing challenges that cannot. be resolved solely by scaling up training datasets (Kandpal )023). In the vision domain, test-time approaches, such as Generalized Diffusion Adaptation (Tsai
lt al.
ee al.
2024). present a promising avenue Lowards attaining OOD robustness.
DGMs are also prone to adversarial vulnerability, often due to the presence of highly predictive but non- robust featurcs that are used as shorteuts for prediction (Du et al. 2023a; Puli et al., 2023 Webson & Pavlick 2022). This behavior poses significant threat to various downstream scenarios, especially those of safety-critical applications (Poursaeed et al.. 2021 Wang et al. 2023a Several approaches to mitigate the
effect of shortcut lcarning are basod on model refinement or on dataset refinement, also known as data-centric
approaches (Whang et al. 2021 Zha et al.;
2025). In the former, work has been done towards improving robustness via adversarial training (Zou et al. 2023b Choi et al.) 2025). feature masking during training
(Asgari et al. 2022). ensembling (Clark et al. 2019), contrastive learning (Choi et al. 2022), and the direct integration of prior knowledge (Ilyas et al. 2019p. The latter includes improving the quality of the data used
by large-scale models during training, such through augmentation (Zhang al.] 2018), labeling (Kutlu
ct al. 2020g) and inference techniques for example, employing prompt engineering (Wallace et al. 2019 or data slicing (Chung et al. 2019).
However, in most applications, foundation models are often adapted to specific tasks and downstream datasots. Standard fino-tuning techniques often overemphasize the target task, leading to catastrophic for- getting (Thanh-Turee & Tran 2020) and a loss in the general robustness of the upstream model (Suprem & Pu 2022). Therefore, a significant challenge is to develop robust adaptation methods that adequately solve the target task but still maintain the beneficial robustness properties of the upstream model (e.g., robustness to distribution shifts of the target dataset) (Balaji et al. 2020 Du et al. 2023a Han et al. 2021 Liu ct al. 2020). These same issues come into play when developing smaller and more efficient models for the sake of economization of DGM inference and memory costs- which we discuss in greater detail in Section 3 In this context, it is important to develop robust distillation methods that do not sacrifice the robustness of the model (Du et al. 2023b Zi el al., 2021). We argue that two particularly promising approaches to oblaining robust and interpretable models are embedding cansal structure and explicitly encoding human priors into the training process topics we examine in the following sections.
Herr, we the terminology biased outputs to refer to systematic deviations in model outputs caused by imbalances or inaccuracies in the training data and/or modeling process. These outputs then do not accurately reflect the true underlying data distribution or are skewed ways that perpetuate inacouracies, stereotypes, or unfair conceptions about certain outcomes.
4

---

Published in Transactions on Machine Learning Research (08/2025)
2.2 Causal Generative Models
Going beyond learning mere statistical correlations and understanding how underlying factors influence the generative process is the main objective of learning a causal structure of data (Pearl & Mackenzie, 2018). Structural Causal Models (SCMs) provide the mathematical foundation for this endcavor, representing causal relationships through directed acyelic graphs paired with structural equations that encode how variables causally influence one another. Such knowledge can be used to reason about hypothetical scenarios in the world, understand the effect of interventions, and perform counterfactuals (Pearl) 2019), thus facilitating informed docision-making. Although there have been attempts to develop methods for learning the optimal generative structure of deep latent, variable models from data (He et al.] 2019 Manduchi et. al.] 2023) current generative models often neglect the underlying causal dependencies in their generative processes. making them prone to shortcut learning and spurious associations |Gururangan et al., 2018 McCoy et al. 2019
Causal generative models have the potential offer distribution-shift robustness, fairness, and interpretabil- ity (Schölkopf et al. 2021 Wang & Jordan 2021). They are either focused on causal representation learning, which discovers causally related latent variables, or controllable counterfactual generation, which, instead, fo- cuses on learning a mapping between data and known causal variables. For a detailed review of the topic, we refer to (Komanduri et al. 2024). Current open challenges include but are not limited to, scalable and robust causal discovery from observational data (Reizinger et. al., 2023 Zhou et al. 2022 Montagna et al. 2024).
identifiability of DGMs under weaker forms of supervision (Ahuja et al.) 2023 Locatello et al. [0200] von
Kügelgen et al.] (0024), lack of benchmark datasets and metries to evaluate counterfactual quality (Montciro
et al.] 2023), strong assumptions that are oflen violated in real-world applications (Komanduri el al.) 2024).
and, finally, the integration of diffusion models, field that is currently under-cxplored but has tremendous growth potential (Mittal et al. 2021 Pandey et al. 2022 Sanchez & Tsaftaris 2022 Sanchez et al. 2023).
We suggest that the integration of causal principles in DGMs could pave the way for the development of more robust, interpretable, and actionable generative AT systems (Zhou et al. 2023
2.3 Accounting for Implicit Assumptions
Silent Assumptions. Current generative models often make use of implicit assumptions and inductive biases. Many of these, such as translational equivariance in CNNs or locality in audio diffusion models, are principled and empirically validated. Others, however, persist mainly for computational convenience 2 or
historical precedent, even when their validity for specific applications remains unexamined or is blatantly known to be wrong (Zhao et al., 2018). As one example, the algorithms used in machine learning often assume that data are drawn independently. In reality, data points are often correlated, such as in time-series data or through repeated measurements from the same individual (Jiang & Nguyen 2007 Kirchler et al. 2023). As another example, most generative models assume that latent distributions can be modeled on simple topological structures. However, latent distributions typically benefit from more expressive approaches (Slimper et al. 2022), suggesting the assumptions of their simplicity may be ill-founded.
We argue that convenience should not be the driving factor behind modeling assumptions. While the impact of model misspecifications on downstream applications in DGMs are not yet well understood, we have preliminary evidence suggesting their elfects are undesirable: In traditional statistical analyses, such misspecifications are observed to have immense impacts (Cardon & Palmer 2003); more recently, empirical studies have revealed systematic biases in DGMs that may stem from inadequate modeling assumptions (Zhao lt al.) 2018), and models that rely heavily on the training data distribution have been observed to exhibit bias and decreased performance if not properly corrected by meaningful modeling assumptions (Fortuin 2022).
There has been some progress towards developing methods that allow practitioners to encode more precise and complex modeling assumptions. As concrete examples, random effects (Jiang & Nguyen 2007) the paradigm used by traditional statistical methods to model data dependencies have been adapted to work with neural models (Simchoni& & Rosset 2023). ). In normalizing flows, data dependencies can be incorporated
4 By "convenience" we mean design choices adopted because they are easy implement or tractable optimize, not because they have been shown to match the true structure of the data.
5

---

Published in Transactions on Machine Learning Research (08/2025)
directly into the likelihood objective (Kirchler et al., 2023 an approach that might be extended to other probabilistic approaches such as VA and diffusson models (Sutter et al. 2023) Causal models can also be integrated to directly model data dependencies and perform counterfactual inference (Pawlowski et al. 2020 which we discuss in more detail in Section 2.2 Notably, these methods have yet to achieve widespread adoption, despite addressing issues that are prevalent and influential in many applications. We believe that further rescarch into the effects of implicit modeling assumptions and methods that allow a wider range of modeling assumptions are promising and impactful directions for the field.
Incorporation of Prior Knowledge, Recent breakthroughs in DGMs have primarily been achieved in settings where models could be trained on internet-scale data (OpenAl 2023; Rombach et al. 2022). How- ever, many real-world applications, such as drug design (Vamathevan et al. 2019), material engineering (Wei
et al. 2019h, personalized medicine MMacFachern & Forkert 2021), and protein biochemistry (Bonetta. & Valentino 2020 often have much smaller datasets due to the high cost of data generation. In these areas, domain experts often possess extensive prior knowledge, which could potentially be leveraged to enable more data-efficient learning in generative Al models. Indeed, it has been shown in the context of VAEs that incorporating domain prior knowledge can significantly improve model performance (Fortuin et al. 2020:
Jazbec al. 2021 | and even unlock their use for tasks that were previously impossible (Fortuin et al.,
2019
Manduchi et al. 2021 )022).
There are multiple routes via which prior knowledge can be encoded in generative Al systems (Dash et al. 2022). One straight forward way to incorporate domain knowledge is in Bayesian settings through the choice of prior distribution; such distributiions can explicitly encode known properties of the larget data. For example, an informed prior can reflect physiological constraints in medicine or chemical properties in materials science (aa ct al.] 2024), taking a step towards ensuring that the learned model aligns with real-world principles. Similarly, recent work in diffusion models employs heavy-tailed priors to model extreme or rare events Pandey et al. 2025). Therefore, we need generative models that can learn mappings between domain-specific priors and the observed data distributions more flexibly (Albergo et al. 2023 Lipman et al. 2023 Beyond priors, domain knowledge can guide architectural design by suggesting specialized network components or hierarchical structures that reflect known relationships within the data (Andreas et al.]
2016b Shen et al.
2019 Bronstein et al. 202I) or can encourage models to process data in a more human- like manner for the sake of interpretability (Andreas et al.] 2016a; McCoy et al., 2020; Vu et al. 2023 Lu ct al. 2023 Finally, constraints embedded in either the model specification or the training algorithm can further ground generative models in real-world processos, leading to improved performance and trustworthiness (Raissi et al., 2019 Ren et al. 2020 Dash et al. 2021 Mohan et al. (002).. Each of these approaches can equip our models with helpful inductive biases that aid data-efficient learning.
While designing future models with domain-informed inductive biases holds great promise, it also presents several challenges that must be carefully considered (Battaglia et al. 2018 Bronstein el al. 2021 For example, while VAFs are Bayesian models and, therefore, offer a natural paradigm for specifying a prior distribution over their latent space, many other DGMs lack such explicit mechanisms for encoding prior information. We consider diffusion models as concrete example. At first, it might seem that the diffusion process's 's Gaussian sampling distribution is comparable to the Gaussian latent prior in VAE, suggestinn a straightforward route for specifying priors for these models. However, this property of the diffusion process arises from the central limit theorem rather than from precise knowledge about the nature of the underlying lata-generating distribution. Recent works have attempted to enhance the space of diffusion priors through auxiliary dimensions (Pandey & Mandi| 2023 Singhal et al.] 2023), and through alternative structured or learned priors TTippe et al.] 20233 Wu et al. 2024). Unfortunately, those approaches do not offer the same degree of flexibility of prior specification provided by the Bayesian priors in VAEs' latent space, highlighting the need for contimued research into priors for diffusion models. More broadly speaking, by definition, biases constrain or push our models towards certain solutions (Mitchell 1980). If the underlying bias does not capture every facet of the real-world process an especially common concern in areas like biology, where core mechanisms remain poorly understood it may inadvertently limit the model's expressivity or lead to systematic crrors. That is, models may fail to learn important patterns that fall outside the imposed structure (Ghassemi et al. (020)). Adding constraints also often introduces computational challenges: physically or biologically inspired restrictions might be non-differentiable or otherwise difficult to incorporate into standard
6

---

Published in Transactions on Machine Learning Research (08/2025)
training pipelines, leading to more complex optimization procedures increasing computational overhead. for example when enforcing PDE/ODE constrainis or discrete combinatorial rules (Cranmer et al.] 2020 Li
ett al. 2023e). Thus, while biasing models with domain knowledge can significantly improve data efficiency and performance, careful considcration of both the correctness of those biases and the technical fcasibility of their implementation is essential.
2.4 Foundation Models for Domain- and Heterogeneous Modalities
While there has been tremendous progress in large-scale foundation models for modalities like text and images, as the scope of their application widens to encompass a broader range of data modalities, a variety of challenges surrounding cross-modal aligument, data interoperability, privacy, and evaluation emergee These challenges are particularly pronounced in specialized fields such as healthcare and chemistry (Raghupathi &
Raghupathi 2014: Korshunova et al.) 2022 Busch et al. 2025).
In healthcare, generation based on diverse data lypes including imaging, health records, and genomics poses challenges in interoperability, data privacy, and security (Moor et al. 2023a]. Time series generation, in particular, requires addressing irregularly sampled data, missing values, scasonality, and long-term de- pendencies (Steinberg et al., 2021). D. In chemistry, physics, and chemical engineering, generative models have huge potential, not just for molecule, drug, and material design, but also in data augmentation, property prediction, and reaction prediction (Winter al., 2019]] Ahmad et al., 2022 Castro Nascimento Pimentel 2023 Hu et al.) 2020). Data in these fields are often sparse, heterogencous and correlated. On the other hand, they provide a vast. body of physical and chemical domain knowledge, ranging from (striet) laws of nature and boundary conditions to (soft) empirical correlations and human experience. Therefore, develop- ing hybrid (ML + domain knowledge) foundation models is particular challenge. While there has been some recent progress toward this goal, e.g., in the realms of physics (Jirasek et al.] 2022: Jirasek & Hasse 2023 Howard et al.] 2022) and medicine (Raghu et al. 2022. Moor et al. 2023b Xia et al.] 2024), there is still much work to be done (Venkatasuurammaiia (2019).
We argue that an overarching goal of the generative modeling field is to build general models that can scamlessly integrate information from diverse sources and understand complex relationships across different types of data (Li et al. 2023a; Reed et al. 2022 Driess et al. 2023). This multi-modal integration challenge becomes particularly evident in embodied AT applications, where generative models must serve multiple functions simultaneously. We can thus view embodied agents as a natural testing ground for truly general gencrative AI bocause of their broad requirements: (1) generative world modcling to predict future states and simulate outcomes of potential actions, (2) multi-modal generation to produce coherent plans that span language instructions, visual predictions, and motor commands, and (3) conditional generation that respects physical constraints while taking into account dynamics of the environment. Unlike purely digital applications, where errors may be aesthetic or semantic, embodied systems expose fundamental limitations in our generative models through physical failure, e.g., a robot failing to grasp objects or navigate spaces. However, developing these systems faces a significant hurdle; data scarcity. While datasets for natural language or images are relatively accessible (Hausknecht et al.] 2020 Li et al. (2023b 2024[), multi-task, multi- enviromenv datasets of control trajectories (states, actions, and outcomes] remain comparatively rare, hindering progress on the development of generalist embodied agents. Generative simulation is one route we
identify to achieve this potential use case for cross-domain generative models (Xian et al. 2023 Fan et al. 2022
3 Optimizing Efficiency and Resource Utilization
Efforts to scale DGMs for tasks like language modeling and text-to-image synthesis often involve training large models with billions of parameters, which demands significant computational resources. This leads to practical issues such as high energy costs (Wu et al. 2022) and expensive inference, limiting access for many users. This further raises environmental concerns due to the energy consumption required to fuel modern tensor processing hardware (Strubell et al.) 2020). Training PaLM leads to 271 tons of CO2c effective emissions (Chowdhery et al.) (2022) and training GPT-3 emits roughly twice as much under comparable accounting assumptions (Patterson al. 2022). Therefore, there is a clear need to reduce the memory and
7

---

Published in Transactions on Machine Learning Research (08/2025)
Sub- challenge
Typical failure mode
Promising
Reference materials
Research-question candidates
mitigation avenues
Efficient attention
Long contexts needed in certain settings and context length limited computational demands
Hardware-awar
Datasets:
Can (non-auttoregressive) sclective-state SSMs (c.g. Mamba) achieve the same performance as transformers in text generation tasks?
QI:
attention computations (e.g.,
LongBench
mechanisms ($3.1)
Surveys:
+lashAttention-2); Sub-quadratic attention alternatives
Tay et al.
2022a
Q8: Can better retrieval methods RAG systems mitigate the need for
longer context windows in LLMs?
Low-bit quantization without quality loss ($3.1)
Sharp accuracy drop when roducing PP precision to less than 4- 4-bits
Activation-awarc weight quantization; Quantization-aware training and fine-tuning
Surveys:
Q1: Do activation- aware quantization methods preserve model calibration after proference-bused fine-tuning (c.g., RLHFF)?
Zeng et al. 2025
Q2: What theoretical limits bound post-training quantization of diffusion models?
Fast sampling for diffusion models ($3.1)
Hundreds of network evaluations needed per sample
Progressive Distillation; Consistency Models;
Benchmarks:
Q1: Can we design models that achieve one-step generation while maintaining diffusion models' training stability and sample quality?
FTD/IS
CIFAR-10/ ITmage-e
Model quantization
Surveys:
Shen al.
2025
Reliahle quality metrics ($3.2)
Automatic evaluation metrics do not correlate
Generative models for quality assessment (e-g.. L.M-assJudge, Auto-J); Sample- based metrics (e.g., Teature-Lkkelihood
Benchmarks:
Qf: How can learned reward models be made reproducible enough to serve as public benchmarkk?
JudgeBench
Surveys:
with human perception quality
[eetzalll et al.
Q2: Is unified multi-modal MAUVE variant feasible?
2024)
Q3: How reliable are L..M-as- scores in OOD settings?
Score)
Compute- efficient model selection ($3.2)
Brute force rrid-search approaches to model selection prohibitive for today's large models
Scaling-law
Benchmarks:
Q1: Can information- iowoioottthhheiieee complexity measures be used during model architecture search to reliahly rule out entire classes of models? Q2: Can zero-cost proxies and scaling law extrapolations be effectively combined provide stronger indication of optimal models than their individual signals?
extrapolation; Zero-cost, proxies
NAS- MAS--Bench-ll 101
NATS-Bench
Surveys:
Li et al. (2024
White et al. 2022
Table 2: Research challenges summary table Optimizing Efficiency and Resource Utilization
computational requirements of large-scale DGMs enhance accessibility and sustainability (Bender et al. 2021).
In this context, we discuss the efficiency-related challenges in current DGMs. We focus on minimizing training and inference costs (3.1, well as highlighting challenges in designing evaluation metrics for DGMs d3.2 which greatly affect the computational resources needed for model selection and tuning.
3.1 Efficient Training and Inference
Network Architecture. Optimizing the network architecture, which forms the backbone of modern ma- chine learning, is crucial for efficient training and inference in DGMs. While we have seen recent improve- ments in model quality (OpenAl, 2023: Touvron al., 2023 Peebles & Xie 2023), there is still dearth of systematic comparative studies of architectural components' contributions to generative model performance. For instance, several popular LLMs like PaLM (Chowdhery et al. 2022) and Llama (Tovron et al. 2023 still largely reuse the original transformer architecture from Vaswani ct al. (2017) with some additional mod- ifications (Shazeer 2020 Su et al.] 2024 Zhang & Semnrich 2019). A modification of particular importance has been that the self-attention (Bahdanau et al.) 2015) mechanism; in the original architecture, this operation incurred a computational cost that scaled quadratically in the context length. This made infer- ence computationally expensive, especially for long-context modeling. Several recent works have proposed attention variants that provide faster inference times
(Tay et al. 2022b). For example, Flash Attention (Dao

---

Published in Transactions on Machine Learning Research (08/2025)
et al. 2022) employs hardware optimizations and efficient memory management techniques to reduce the effective computational overhead of attention from quadratic to linear in the context length; Flash Atten- tion 2 (Dao) 2024) takes these optimizations a step further, bringing attention computations close to the achiovable bounds on fast matrix multiplication. Grouped Query Attention (Ainslic et al. 2023 proposes a structural change to the standard attention mechanism, where querie are divided into distinct groups that are then processed independently and simultancously. We see several promising research directions for reducing the computational needs of large-scale generative models, including specialized methods that make popular network architectures more computationally efficient (e.g., the attention variants discussed above), early-exit designs that allow models to make predictions without running the forward pass through the full network (Chen et al. 2024b) and alternative autoregressive sequence-modeling frameworks with favorable properties like scalability and linear complexity in the context. length (Gu & Dao 2023 Gu et al. 2021
Similarly, several popular large-scale text-to-image diffusion models like DALL-E 2 (Ramesh et al. and Stable Diffusion (Rombach et al.) 2022) largely reuse the popular UNet (Ronneberger et al. 2015)
2022)
backbone from Ho et al. (2020), which has high memory costs. Therefore, we believe that principled st study of the impact of different network components in large-scale generative models is crucial for efficient training
and inference. Some recent works (Hoogcboom ct al. 2023 Karras ct al. 2023 Peebles & Xie 2023 Podell ct al. training dynamics while enabling faster inference with a lower memory footprint.
2024) already explore architectural design choices for reducing diffusion model sizes, thereby improving
Model Quantization. The goal of model quantization is to reduce the precision of model weights and activations, enabling faster, memory-efficient training and inference, idcally without losing performance on downstream tasks. The most common quantization approaches are Post-Training Quantization (PTQ), which applies quantization to a pre-trained large model to enable faster and memory- inference, and Quantization-Awar Training (QAT), which involves training quantized model from scratch (Krishnamoor- thi 2018).
Despite some progress in developing PTQ and QAT methods for LLMs (Dettmers et al. 2022 Liu et al. 2023b; Xiao et al. 2023 Yao et al. 2022 Dettmers et al. 2023) and large-scale text-to-image diffusion models Li et al.j 2023d). the existing methods are far from perfect. For instance, OPTQ (Frantar et al.
2023 a PTQ--based approach, can perform inference for a quantized LLM (in this case OPT (Zhang el al. 2022 with 175B parameters on single A100 GPU with 80GB of memory without degradation in accuracy. Though impressive, even this quantized model would likely have limited utility on a consumer-grade GPU device, let alone standard edge devices. Similarly, QaT-based approaches can often achieve lower bitrates but trade off additional training for this efficiency. This can be a major computational bottleneck for large generative models. While some recent work suggests preliminary success in this direction şLin et al. 2024a we believe that investigating the impact of model quantization at low bitrates in large-scale generative models is a crucial direction for the practical deployment of these models.
Design Challenges. The current dominant modeling paradigms in generative AT, such as diffusion mod- els (Ho et al.) 2020) and LLMs (OpenAl, 2023), demonstrate remarkable sample quality. However, the design of the generative processos in these approaches can cause significant challenges. Diffusion models, for instance, rely on an iterative, multi-stage denoising process, which slows down inference considerably. Gen- erating high-quality samples often requires hundreds to thousands of network function evaluations (NFFs) (Ho et al., 2020: Song et al. 2020). Similarly, LLMs employ an autoregressive structure that generates tokens sequentially, resulting in slow inference due to the left-to-right generation process. These challenges contrast with alternative generative models like VAFs and GANs, which require only a single NFE for sample gencration. However, these models suffer from other drawbacks, such as blurry sample generation in VAEs (Dosovitskiy & Brox 2016) and mode collapse in GANs (Arjovsky et al. 2017).
To address the inefficiencies in diffusion models, researchers have explored multiple complementary ap- proaches to speed up inference. Some notable approaches include: developing training-free samplers (Song
et al. 2021: Liu et al. 2022al La el al. 2022 Zhang & Chen 2023 Karras et. al. 2022 Pandey et al.
2024), designing better diffusion processes Singhal et al. 2023 Dockhorn et al. 2022 Pandey & Mandt
TTn the attention mechanism, a query transformed vector representation, typically derived from input tokens.

---

Published in Transactions on Machine Learning Research (08/2025)
2023
Karras et al.
2022), and combining other model families with diffusion models (Pandyy et al. 2022
Zheng et al. 2023b Xiao et al. 2022 Yang & Mandt 2023). Additionally, training a diffuusion model in the latent space of lossy transform (ahddat et al., 2021 Rombach et al. 2022) not only improves memory requirements and sampling efficiency but also provides access to a more interpretable low-dimensional latent representation. A lossy transform (such as VQ-GAN (Esser el al. 2021 can drastically reduce data dimen- sionality while retaining the perceptually relevant details of high-resolution images. Designing more efficient lossy compression operations in the context of diffuusion models has received less attention in the community and is an important direction for further work (Yang et al. 2023d Havasi et al. 2019 Yang et al., 2020). Despite these ad advances, sampling from diffusion models remains computationally challenging, typically re- quiring 25-50 NFEs to generate high-quality samples. While approaches based on progressive cistillation (Salimans& & Ho. 2022 Meng et al. 2023) can further speed up inference, they trade off additional training for faster sampling. Therefore, there is a need for DGMs that inherit all the advantages of diffusion models while supporting onc-step sample gencration by design (c.g., see consistency models (Song et al.. 2023 & Dhariwal 2024) for recent. work in this direction).
Song
In the case of LLMs, in addition to an expensive self-attention operation in transformer- cansformer-based autoregressive models, sequential token generation in left-to-right fashion in these models makes inference more expensive. Indeed, it is one reason that (sub)word tokenization the pre-processing of text into pre-defined units is still an essential part of these pipelines. Notably, tokenization itself introduces a strong inductive bias into language modeling: the model is constrained to work with the predefined units set by the tokenization scheme. The representation of the data that the model Iearns is inherently shaped by these units, limiting the model's flexibility. Token-free approaches have been proposed to allow for the joint optimization of text segmentation alongside other parameters, but in practice, they are often computationally infcasible with attention-based archilectures because handling raw character sequences at scale magnifies the already expensive attention mechanism (Xue et al.) 2022). Dynamic tokenization schemes (Ahia et al. 2024] e.g..) present an interesting research direction, as they allow for predicting token boundaries at inference time, but they likewise can introduce significant computational overhead. Tokenization remains a key design choice that shapes both the performance and efficiency of modern generative models and whose further optimization is constrained by needs for efficiency (Rust al. 2021 Toraman et al., 2023 Ali et al. 2024). Methods such speculative decoding (Leviathan et al.) 2022) are one approach that can help roduce the bottleneck caused by left-to-right generation. Non-auuloregressive models also offer an interesting alternative for sequence modeling. For example, diffusion models amortize the computational cost of generating sequences across all tokens simultaneously (Dieleman et al.) 2022 Wu et al. 2023; Li et al. 2022a). However, these models inherently lack the inductive bias for contextual generation, which has been shown to work well empirically for sequential modeling tasks. This affects their performance in downstream tasks that might require long- context modeling, such as video synthesis (Yang et al.] 2023c] Ho et al., 2022a). While diffusion models can be incorporated within the autoregressive framework for such tasks, the resulting models can be very expensive during inference (due to the cost of synthesizing a single token using diffusion across multiple tokens). Therefore, we identify a potential tradcoff between long context modeling and efficient inference, with the diffusion and autoregressive modeling paradigms falling on the opposite ends of this tradeoff. Hence, designing generative modeling paradigms that can optimally balance this tradeoff remains challenging.
3.2 Evaluation Metrics
Evaluation metrics are crucial in guiding research, as the conclusions derived from empirical studies depend greatly on the chosen metries. In modern MT. evaluation metrics are additionally a key component in hyperparameter tuning and model selection; their design thus affects computational resources required during large-scale training. However, designing robust and meaningful evaluation metrics for DGMs is challenging for several reasons.
Evaluation Metric Design. Many generative models are probabilistic, making likelihood-based metrics a seemingly natural choice for evaluating their performance. These metrics have been widely utilized in the literature due to their alignment with the probabilistic franeworks of such models. However, empirical evidence suggests that likelihood- based metrics often do not provide an accurate assessment of generation quality (Theis et al. 2016). In particular, they often fail to correlate with human judgments of sample
10

---

Published in Transactions on Machine Learning Research (08/2025)
quality (Kolchinski et al., 2019 Pimentel et al. 2023 Moreover, many popular generative models do not even allow for tractable likelihood computation. Other automatic evaluation metrics are thus necessary for evaluating the quality of generated samples.
Several notable evaluation metrics for generative models follow the general paradigm of comparing the distribution of generated samples to that of train/test samples (Sajjadi et al., 2018 Pillutla et al. 2024:
Jiralerspong et al.) 2023). J For instance, the Fréchet inception distance (Heusel et al.) 2017|, which is widely used for evaluating image synthesis models, takes this approach (Salimans et al., 2016 Binkowski et al, 2018). However, these metrics are far from perfect. First, robust computations of these metrics require large set of samples (around 50k for image generation models). This can be computationally demanding for generative models with a sequential inference process, like diffusion and autoregressive models. Even given large sample sizes, these metrics have still demonstrated issues with robustness. For example, FID can be
sensitive to minor perturbations in the input data (Parmar et al. 2022) (see Chong Forsyth (2020) for additional discussion on sources of bias associated with FID and Borji (2022) for more related evaluation metrics). Second, these methods typically rely on an external pretrained model, e-g., the GPT-2 family
of language models (Radford et al.) 2019)) or classifier network trained on ImageNet (Deng et al. 2009).
This property makes the metric effective for evaluating sample quality within the domain of the pretrained model's data but seemingly causes it to overlook significant features or overemphasize arbitrary ones in other
domains Kynkäänniemi et al. 2023 Pimentel et al. 2023
Recent works have attempted to improve upon the above-mentioned shortcomings. For example, Jayasumanal et al. (2024) propose the use of embeddings from the CLIP model (Radford et al., (021), b, which aligns images and text in shared embedding space, in order to make a more robust evaluation metric for image synthesis models. Several evaluation metrics for text generation systems use LLMs to score rank samples,
either via prompting (Li et al.) 2024g) or explicit fine-tuning on the task of evaluation (Li et al.) 2024e). These metrics correlate remarkabie bly well with human judgments (Koemi & Federmann 2023) and offer more fino-grained assessments of text generation
-aceindiiroviinngg feodback at the individual sample level and taking into account user-specified criteria (Jiang el al. 2024a). These methods make progress towards broader applicability across domains and greater aligument with human evaluations but also demonstrate increased reliance on generative models for the evaluation of other generative models. This circular dependency introduces the risk of amplifying existing model biases (Fang et al. 2024 and narrowing the
diversity of model-generated content (Doshi & Hauser) 2024 Gambetta et al. 2024). as the underlying paradigm of such evaluation metrics should lead to the favoring of outputs that align with the characteristics and biases of the models used for assessment. Some works have proposed (either explicitly or implicitly) rewarding sample diversity in evaluation metrics (Zhu el al. 2018 Alihosseini el al. 2019 Jiralerspong et al., 2023), which would alleviate the latter problem. However, there is often a quality-diversity trade-off (Caccia) et al.) 2019 Zhang et al.] 2021 Naeem et al.) 2020), where a model that generates high-quality samples might have low diversity across its samples and vice versa. Further, the quantification of diversity is in itself difficult task (Tevet & Berant
2021
Subjective aspects in generation. A major challenge underlying the evaluation of generation quality is the subjective nature of sample attributes, such as realism, fluency, and style. While human inspection is typically the gold standard for evaluating generative models (Denton et al. 2015 Zhou et al. 2019 Saharia ct al.] 2022), in many cases, human judges disagroe over several attributes, such as which samples have better quality (Clark et al., 2021] or what is considered realistic in the larget. domain (e-g, medical images or industrial optical inspection). This challenge is even present for conditional synthesis tasks when the set of suitable outputs is limited by constraints from the input. For example, in toxt-to-image generation (Ramesh) et al. 2021 Rombach et al. 2022 Saharia et al. 2022), human evaluators may have different opinions on how closely a generated image aligns with the provided description.
For this reason, a common approach is to collect numerous human judgments and set up benchmarks based on these collective scores, e.g., the Open Parti Prompts Leaderboard for image generator evaluation or TURIGGBENC (Uchendu et al., 2021) for language generator evaluation. While such approaches-
along with attempts to standardize human evaluation practices [Elangovan et al. 2024
help make human
11

---

Published in Transactions on Machine Learning Research (08/2025)
evaluations more reliable signal for guiding the development of generative models, there are several other issues with human evaluation. These include its monetary costs, the general inconsistency of human raters (Clark et al. 2021; Belz et al. 2023), and its focus on individual samples, overlooking how well the generative model reflects the data distribution as a whole.
Evaluating Model Uncertainty and Calibration. Model uncertainty and calibration have become quantities of interest because of their implications for the reliability, interpretability, and safety of gener- ative AI systems. Here, use the term model uncertainty to refer to the degree of confidence model has in its outputs; model calibration refers to the degree to which a model's estimated probability of an event is consistent with that event's true probability of occurring - As concrete examples of the importance of these metrics, high model uncertainty in language generation tasks has been linked with the occurrence of hallucinations- insttances where the model produces outputs that are implausible or factually incorrect (van der Poel et al., 2022 Zhang et al. 2023c); autonomous driving systems increasingly use generative models to predict the trajectories of other vehicles, cyclists, and pedestrians (Yuan et al., 2020), and miscal- ibration of the probabilities of such trajectories can lead to severe accidents.
Historically, relatively simple metrics have been employed for measuring uncertainty and calibration in
machine learning. For example, Shannon entropy (Shannon 1951 has been a common metric for quantifying
total model uncertainty (Houlsby et al. 2011 Depeweg 2019 expected calibration error (Pakdaman Naeini
et al. 2020)
2015 ECE) has often been employed for assessing model calibration (Guo et al. 2017: Dormann ssee Abdar et al. (2021) and [Wang (2023) for detailed surveys on uncertainty and calibration in deep
learning, respectively). While these metrics are well-suited to models for simpler classification problems, their extension to generative models is non-trivial. For example, language models operate over countably infinite output space (i.e., the set of all possible strings), making exact computation of metrics like entropy or ECE infeasible. Consequently, key aspect of research on these characteristics in generative models has
been on defining metrics that are suited to them (Zhao et al., 2021 Ran et al., 2022 Luo et al. 2023 Zhao
ct al. 2023b
Fei et al., 2023
To complicate matters further, there debate regarding which definitions of these metrics actually provide useful insights about a generative model. Model uncertainty, for instance, can come from multiple sources, such alcatoric uncortainty (intrinsic noise in the data) epistemic uncortainty (uncertainty in the model parameters) (Hüllermeier & Waegeman 2021 Wimmer et al. 2023). Depending on the specific use case for a model, one may be interested in the contribution of only one of these sources rather than in total model uncertainty (Osband et al., 2023 Giulianelli et al. 2023 Kuhn et al.] 2023). With respect to calibration, it is unclear exactly which distribution a generative model should be calibrated to (Koevering & Kleinberg, 2024); often times, we are more interested in modeling the distribution of high-quality outputs than the data-generating distributions (Ouyang et al., 2022). albeit the data from the latter is what models are often trained on (Kalai & Vempala 2023). These choices must be carefully and thoughtfully considered, as they play critical role in shaping the development of methods to quantify these metrics or address poor model performance in terms of these metrics.
Model Selection. Model selection, i.e., identifying which model configuration or set of hyperparameters will perform best on a given task, is essential in training large-scale generative models. Evaluation metrics play a critical role here. Using knowledge of scaling laws (Kaplan et al. 20200: Henighan et al. 2020 evaluation metrics can be used to predict early on in a training run whether model is likely to be successful (OpenAT, 2023). Recent work has shown that these predictions can be done quite precisely (Ruan et al. 2024)
potentially reducing the need to train numerous large neural networks in the search for single good model. Zero-shot proxies (Abdelfattah et al. 2021)-metrics metrics computed on an untrained or minimally initialized network that approximate its final task performance before any training is done- are another promising research direction for compute- efficient model selection. We also believe that more effort should be invested in analyzing models' performance- compleeziy tradeoff, an important yet muder-investigate
measure for real- world applications at scale. This tradeoff refers to the balance between model performance and computational complexity. We argue that model selection and evaluation should perhaps shift towards identifying the model families that lie in the associated Pareto set that optimizes this tradeoff (Devroye 2010
Braverman 2005
We note that her definitions for the terms have been used; we employ this one as it the most relevant for our exposition.
12

---

Published in Transactions on Machine Learning Research (08/2025)
Chen et al 2022 Braverman 2023 as optimizing for these characteristics in isolation does not account for well-performing models in each of the model classes
real-world constraints. The naive under consideration and computing their respective computational complexities is time- and resource- intensive. We posit that alternative assessments of complexity from information and learning theory (c.g.. Xu et al. 2020) could provide the basis for more efficient metrics in these types of evaluations.
Looking Forward. The multi-faceted nature of what defines a high-quality generative model makes de- signing robust and meaningful evaluation metrics a particularly challenging task. Instead of relying on human priors about what constitutes a good quantitative metric of model quality, developers have increas- ingly turned to the strategy of learning reward functions directly from human preferences (Ouyang et al. 2022). This approach should allow for evaluation metrics that are more aligned with human judgment, the reward functions are directly informed by human feedback rather than predefined criteria. These reward functions could serve as the foundation for new evaluation frameworks for generative models, and we hope they will be open-sourced to enable the development of publicly accessible benchmarks.
Evaluation metrics can help us understand and ultimately mitigate model shortcomings. While this approach has been embraced for improving model quality, it also has significant potential to enhance model fairness, safcty, and reliability. For instance, metrics designed to quantify various forms of bias can aid in identifying and addressing model unfairness. While such metrics exist for classification or regression models (e.g.. demographic parity equalized odds), their extension to generative models is non-trivial. Research is thus needed to develop and refine metrics that can effectively quantify biases in the complex outputs of generative models M This includes creating frameworks that account for the nuanced and context-dependent nature of generated content, ensuring these models are not only high-quality but also fair and aligned with ethical standards [Ray] 2023). Unfortunately, to be effective, these metrics must also be adaptable to closed-source
gencrative models since parametors and logits of most commercial models are not publicly available (Zhao
et al. 2023a Sun et al., 2023a Laszkiewicz et al. 2024
Despite the availability of more advanced evaluation metries, some domains continue to rely heavily out- dated automatic evaluation methods. For instance, BLEU (Papineni et al. 2002) and ROUGE (Lin 2004)
metrics based on n-gram matching that are known to empirically correlate poorly with human judgments 1122)-remain -remain extremely prominent in the evaluation of machine translation and abstractive summarization systems. The continued reliance on these inaccurate metrics may ultimately impede the advancement of generative AI, as they provide weak signals for model improvement and fail to guide the development of systems that truly align with human expectations and real-world applications. A shift towards new evaluation metrics requires critical mass of adoption within the community. Therefore, we must encourage practitioners to move beyond the convenience of outdaled metrics and embrace this new generation of improved metrics.
[Reiter 2018 Deutsch et al. )
4 Ethical Deployment and Societal Impact
With the current excitement around the scope and application of large-scale generative models, we are also witnessing a growing apprehension, fueled by media reports, of adverse outcomes surrounding the rapid advancement generative AI. These concerns add to the conceptual and practical considerations discussed so far and encompass range of issues, including the spread of misinformation, the absence of regulatory frameworks (Meskó & Topol 2023), unintended harm (Greenfield & Bhavnani] 2023), and debates over open-
source versus closed-source technologics (Chen et al. 2023). among others. Here identify key challenges concerning the responsible deployment of large-scale DGMs. More specifically, we discuss several aspects, including the dissemination of misinformation (4.1) violation of privacy and copyright (422), presence of
biases (4.) lack of interpretability (4.4), and constraint satisfaction (Section 4.5)
Many aspects of fairness cannot be captured by quantitative metrics. Further, definitions of fairness differ amongst different people and groups, and these definitions may evolve over time. However, they can still provide insights into whether models achieve a certain level of fairness in specific aspects.
13

---

Published in Transactions on Machine Learning Research (08/2025)
Sub- challenge
Typical failure mode
Promising mitigation avenues
Reference materials
Research-question candidates
Misinformation & synthetic media detection (64.1)
Deepfakes bypass detectors
Model-rooted watermarks (e.g.; Tree-Ring Watermarks)
Datasets:
Q1: Can diffusion-time watermarking survive multimodal adversarial attacks?
Decpfake Detection
Challenge
Q2: Can uncertainty-aware abstention policies mitigate hallucinated facts DGM outputs?
WaterBench
Surveys:
Rana ct
2022
Privacy violations, copyright infringement ($4.2) & PIT leakage
Model regenerates training data ver batim or uear-verbatim
Differential Privacy
Datasets:
Q1: What
learning techniques
CPDM
Priv PrivLM- Bench
fine-tuning strategies remain feasible for
(c.g., DP-SCD);
Surveys:
2 100B-parameter models under practical compute budgets?
Machine Unicarning
Yao ct al
2024
methods
Kibriya et al.
2024
Fairness across languages ($4.3)
Worse compression + more tokens needed higher cost for low-resource
Dynamic tokenization
Surveys:
Q1: Can subword-free sequence modeling architectures (o.g., SSMs) eliminate tolemizaatinninnurees performance disparities languages?
Xu et al. 2025
schemes; Vocabulary transfer methods
Qin et al. 2025
languages
Bias & dis- crimination generated content ($4.3)
Models reflect and propagate bias and discrimination present training data
Counterfactual
Datasets:
Q1: How do watermarking methods interact with appearance demographic biases in model outputs?
evaluation benchmarks;
HolisticBias/
OpenBias
Surveys:
Fairness metric (e.g-
Gallegos et al.
2024
demographic parity) incorporation into training objectives
Interpretability DGM parameters are uninterpretable, making them black boxes for and developers
Automated circuit discovery; Causal tracing
Datasets:
Q1: Which confidence metrics hest
& transparency ($4.4)
GPT-2
predict whether mechanistir explanation truly modulates model behavior?
neuron-explanation
dataset
Surveys:
Q2: Can mechanistie insights be transferred model sizes?
Marcinkevics & Vogt
(2020
Constraint satisfaction ($4.5)
Outputs violate hard rules (e.g. won't compile)
Context Free Grammar-guide guided
Benchmarks:
Q1: How hard-constraint decoding generalised from code text and images?
HumanEval
dccoding algorithms;
Big ligCodeBench
Coustrained RLHF; Constraint- embedded model architectures
Surveys:
Q2: constraint-aware training methods (e.g., constrained RLHF) achieve better safety-performance trade-offs than inference-time constraint enforcement (i.c., constrained decoding)?
Zhang et
2023a
Table 3: Research challenges and mitigation strategies
Ethical Deployment and Societal Impact
4.1 Misinformation and Uncertainty
As the quality generated data synthesized using large-scale generative models increases, it can become more and more difficult. to distinguish between real and generated content, especially for uninformed consumers (Fann et al., 2024). This indistinguishabiiit facilitates the spread of misinformation (e.g., by deepfakes [Helmus] 2022 To ensure the trustworthiness of information, par with the advances in generative models and allow us to robustly detect and mark synthetic data. Numcrous models for diffcrentiating machinc-gone generetedeedd from real content have been proposed over the last years [Rana et al. 2022), but the increasing quality of generative model outputs has decreased their accuracy. Watermarking is another approach in which there has recently been increased interest. The goal of these methods is to manipulate generated sample (c.g., an image or piece of text) such that signature can be detected in downstream tasks, albeit with minimal effects on sample quality. There have been several approaches to watermark synthetic data generated from L.MSs (Kirchenbauer et al. 2023 Dathathri et. al.
need algorithmic solutions that are
14

---

Published in Transactions on Machine Learning Research (08/2025)
2024 Zhao et al. 2024] and image generation models (Zhao et al. 2023c Wen et al.
2023
Jiang et al,
2024b). However, current watermarking methods are far from robust (Saerii et al. often possible by small manipulations (like paraphrasing or pixel perturbations) (Jiang et al. 2023) and the injecting of watermarks into content can be inefficient (Liu et al. 2024a]. Recent work has demonstrated that model-integrated watermarks i.e., signalures embedded in a model's 's sampling process rather than post-hoc in its outputs-au are a promising route forward, as they can survive a range of real-world corruptions (e.g.. paraphrasing attacks in text (Krishma et al. 2023) and latent-trajectory perturbations in images (Wen al, 2023 Fernandez et al., 2023 Nonetheless, the anformation-thherrett limits on detectability, false-positive rate, and adversarial removability for watermarking remain poorly understood, and initial negative results suggest unavoidable trade-offs between robustness, the quality of the altered sample, and watermark payload
2023
Evasion is
Kirchenbauer et. al. 2024 Yoo el. al. 2024).
Notably, misinformation can emerge even without malicious intent. Tools like ChatGPT are increasingly expocted to serve as universal question-answering engines, even though their core objective-to estimate the likelihood of the next token in sequence is traditionally designed to assess the linguistic plausibility of strings (Kalai & Vempala 2023), rather than their factual accuracy. This distinction between the two objectives is evinced by the discrepancies observed between the probability a model explicitly assigns to a statement when prompted vs. the underlying log probability it assigns (Hu & Levy 2023). models' tendencies to hallucinate (Huang et al. 2024a), and their difficulty in achieving probabilistic consistency [Flazar et al. 2021 e.g,, ensuring logical predictions between a statement and its negation.
Some works have turned to model uncertainty estimates (e.g., those discussed in 3.2 as indicators of model re- liability, developing methods to enhance the trustworthiness of AT systems based these estimates (Edupu- ganti et al., 2021 Yang et al., 2023e). For example, Ren et al. (2023 propose selective generation approach, where models abstain from providing a response in the face of high uncertainty; Kuhn et al. (2023) use a The
notion of a model's semantic uncertainty to predict the correctness of its answer in question-answeringg development of methods that explicitly account for uncertainty represents an interpretable approach toward ensuring model reliability, offering a stralegy that also has grounding in a well-studied concept in machine
learning (Malinin & Gales 2018
Abdar et al. 2021 Gawlikowski et al. 2023).
Encouragingly, some rescarch suggests that larger LMs actually are well-calibrated in terms of their world knowledge, i.e., their predicted likelihoods reflect the probability that statement is true (Srivastava el al.
2022 Zhu et al. 2023 Yu et al., 2024). Further, recent studies show that language models often do possess the ability to assess the truthfulness of their own statements {Lin et al. 2022: Kadavath et al. 2022 Xiong ct al. 2024). However, tinettnnng or RLHF, which are frequently applied to these models, have been shown to hurt calibration; rather, they have been widely observed to exhibit overconfidence the tendency of a model to assign excessively high probabilities to its predictions regardless of their correctness (Kadavath
ct al. 2022 Tian et al.] 2023 OpenAl 2023 Xiong et al., 2024). There has been some work on mitigating
miscalibration issues for fine-tuning (e.g., Wang et al. 2023b) and RLHF (e.g.. Tian et al.) 2023 Zhang
ct al. 2024), and there is an increasing focus on systems where evidence can be brought in from external
knowledge sources (Blattmann et al. 2022 Pan et al. 2023 Gao et al., 2023), grounding model responses to
reliable knowledge sources. Such research along with benchmarks to assess model factuality (e.g., KoLA;
Yu et al. 2024 is a critical step towards ensuring the trustworthiness and reliability of generative models.
4.2 Security, Privacy and Copyright Infringement
While modern generative models like LLMs are deployed practically for many applications, this also exposes them to potential malicious attacks, which can have significant costs for downstream applications or users. One class of malicious attacks on LLMs is the so-called "backdoor attacks" where the main idea is to train the model using poisoned data and then trigger specific output response from the model corresponding to specific prompts. For instance, Yang et al. (2023a) discusses backdoor attacks on LLMs in the context of communication networks (see Zhou et. al. 2025 for a more in-depth exploration of backdoor allacks). Another class of attacks known as "jail-breaking" involves designing adversarial prompts to generate mali- cious outputs from the model while bypassing guardrails employed to comply with usage policies. There has been a good deal of research in the context of LLMs exploring techniques for mail-breaking and for guarding
against, jail-breaking (Shen et al. 2024 Yi et al. 2024 Jin et al. 2024
15

---

Published in Transactions on Machine Learning Research (08/2025)
Beyond malicious attacks, another critical risk associated with generative AI models is their tendency to memorize and unintentionally reproduce training data, raising serious concerns about privacy and data leakage. A variety of works have shown that publicly available LLMs and large-scale text-to-image models can implicitly "memorize" " training data fvan den Burg & Williams 2021), to the point that samples from
the dataset can be (almost exactly) reconstructed (Carlini el al. 2023a b Somepalli el al. 2023 Nasr
et al., 2023). ). This behavior potentially infringes on data privacy, underscoring the importance of detecting whether private information has been leaked into an LLM's training data (Kim et al., 2023) and exploring whether generative models can be trained while safeguarding sensitive information. Differential privacy (DP) constraints, which can be enforced during generative model training, offer an attractive theoretical framework to ensure privacy Dwork & Roth (2014); Li et al. (2021); Dockhorn et al.] (2023). However, DP-based approaches have several shortcomings. They sulfer from a trade-olf between privacy and utility Cummings et a 2024). There are different DP formulations, each based on different assumptions about trust, data access, and the point at which noise is introduced. The meanings of the canonical DP paramcters are thus not consistent, making comparison of models produced using different approaches difficult {Li et al., 2024d). Recent work has instead focused on generative DP synthetic data, where foundation models are only used as black boxes (Lin et al. 2024b). Building privacy constraints into the training of large-scale generative models can be a promising direction for further research.
Another byproduct of memorization in generative models is that it can lead to unauthorized distribution or replication of training data, resulting in copyright infringement liabilities Current efforts towards preventing such behavior have approached the problem from different angles. Some focus on filtering training data: techniques such as contractual licensing filters and hash-based de-duplication allow developers to identify
and exclude protected material before training (Carr & Jeffrey 2022 Duarte et al., 2024 Plagiarism
style-clone detectors allow intervention downstream. flagging generated samples that are substantially similar to copyrighted works (Li et al. 2024c Kim et al. 2021). Other works have proposed training methods, which can be broadly categorized under two complementary strategies: (i) imitationn-eesistad training objectives that discourage verbatim memorization (Liang et al.
2023 Zhao et al. 2023d), and (ii) machine unlearning methods that aim to eliminate the influence of certain da datapoints (e.g., copyrighted or private material)
2024b 2025). There is still much open rescarch in this domain, including provenance tracking that can scale to work with modern, massive datascts and copyright infringement risk metrics that encompass legal concerns.
from a model's predictions after training (Li et al. 2024b Liu et al.)
Beyond privacy and copyright concerns, indiscriminate memorization has other undesirable effects. Many of the current target applications for generative models-sutee as creative writing or graphics generation- demand novelty and user-specific adaptation; when a model merely regurgitates training data, it does not provide the desired diversity, originality and personalization. This behavior potentially worsens user experi- ence and limits the practical value of generative models. Further, excessive memorization can lead to biases in generated content, where certain perspectives or styles dominate because they were overrepresented in the training data; we discuss this last issue in more detail in the next section. Ultimately, deploying DGMs safely and uscfully will require turning the request of "don't 't copy the training set" into an explicit design and auditing objective, which necessitates continued research into how to measure, control, and (when necessary) unlearn memorized examples (Chen et al. 2024a Lesci et al. 2024).
4.3 Fairness
Large-scale generative models are often trained on massive datasets containing billions of samples scraped from the internet. While preprocossing such large datasets often involves tagging or removing toxic content, variely of other societal biases are often harder to detect. Consequently, the trained models can rellect biases and produce outputs that may be deemed toxic or harmful (Pagano et al. 2023 Gallegos et al.) 2024 Zhou et al. 2024). For instance, WWidinger et al. (2021) outline a series of harms that can result from using LLMs that produce discriminatory or exclusionary language, e.g., the amplification of stereotypes or exclusionary norms. Multimodal models may exhibit. biases about gender, ethnicity, and religion, among others (Janghorbani & De Melo 2023) that have similar negative effects on society.
16

---

Published in Transactions on Machine Learning Research (08/2025)
Not all unfair behaviors exhibited by generative models are overtly harmful generating toxic content. Models can show more subtle biases towards certain subpopulations, such as allocation bias-when AT systems extend or withhold opportunities, resources, or information or quality-of-service bias when Al systems work better for people in some subpopulations than others; these behaviors should not be downplayod as they can perpetuate systemic inequalities. A main culprit of such behaviors sterss from the fact that the data used in the training of most generative models is disproportionately from certain countries and languages. Such models therefore might not work well for languages or images outside of these rainstream groups. For example, multilingual language models typically perform substantially better on English tasks compared to tasks in other languages (Tai et al.) 2023: Huang et al. 2023a). Further, because the tokenizers for such models have also been trained disproportionately using English data, the compression rate for English lexts is much higher than for lexts in low-resource languages (Pelrov et al. 2023 Ahia el al.) 2023) Consequently, services that charge based on token counts impose higher costs for a query made in a low- resource language compared to a query with the same undorlying meaning made in English. This exacerbates accessibility challenges for speakers of underrepresented languages.
Numerous approaches have been proposed to mitigate the biases of generative models in post-hoc manner (Bai ct al. 2022 Glaese ct al. 2022 Ferrara 2024 Olmos et al. 2024). However, the achieved changes are often merely superficial, leaving the possibility of remnant biases. For example, Gonen & Goldberg (2019)
demonstrate that word embeddings still cluster based on gender stereotypes even after bias mitigation tech- niques, effectively "hiding" rather than eliminating the issue. In the vision domain, post-hoe bias mitigation strategies have been observed to work poorly in the face of test-time distribution shift (Kong et al. 2023).
Moreover, most. evaluations assess only a single fairness axis-for instance, gender in English or skin-tone in photos-and residual biases along other dimensions can remain undetected. Some key rescarch arcas that we identify as needing further atlention include: (i) joint evaluation across multiple, potentially interact- ing fairness criteria (e.g., gender x dialect); (ii) stress-testing mitigation strategies under data- distriiittiittion shifts; and (iii) training-time interventions that prevent harmful biases from emerging in the first place. Some areas that we identify as needing further research are combined evaluation with respect to multiple forms of fairness criteria, robust assessments across multiple domains and training methods that can more robustly mitigate the learning of harmful biases. Promising steps in these directions include multilingual, multi-attribute benchmark suites such as XCOPA-Bias de-biasing schedules that adjust sampling weights during pre-training (Kim et al. 2024
Goldfarb-Tarrane et al. 2023 and gradient- based
Ultimately, assessing and ensuring fairness in technology applications is a complex challenge. Aside from the aforementioned issue of differing (and potentially dynamic) qualitative definitions of fairness (3.2 different notions of fairness often cannot be fully satisfied simultaneously (Ferrara (024).. Therefore, is essential for the builders of generative AI tools to carefully evaluate the various dimensions of fairness and make deliberate trade-offs appropriate for the specific use case.
4.4 Interpretability and Transparency
In high-stakes applications such healthcare and legal domains, it is critical to understand the logic and influencing factors behind generative models' outputs. In other words, we need to be able to interpre! how a generative model produces its results, with its decision-making process being transparent, i.e., accessible and understandable This is particularly true in safety-critcall domains such as healthcare (Chen et al., 2021a) or finance applications- but is also important across general AI use cases, where interpretability is essential for diagnosing errors and fostering user trust. These nceds are not new and have been present since the start of publicly available Al-based products and tools {Confalonieri et al. 2021]. There has thus been sizable amount of research in neural network interpretability methods. However, these methods are not always feasible for use with large-scale DGMs. For example, interpretability methods, such as SHAP, LIME, or Intograted Gradients (Lundberg & Lee 2017) Ribciro et al., 2016 (Sundararajan et al. 2017) struggle to scale effectively with the complexity and size of large models; many interpretability mel hods work by attempting to understand concepts encoded in models' latent representations (Crabbe et al.) 2021 Esser et al. 2020), but this becomes more difficult in the high-dimensional latent spaces used by DGMs. Further, while one might hope that explanations derived from interpreting smaller models could be used for understanding their larger counterparts, scaling up generative models gives rise to unpredictable effects.
17

---

Published in Transactions on Machine Learning Research (08/2025)
e.g.; models demonstrating unexpectedly advanced capabilities (Wei et al. 2022 Schaeffer et al., 2024 conclusions drawn when using small models therefore may not apply to today's s larger models.
The fundamental challenge is to develop explanation methods for DGMs that are both well-understood
by humans and faithful to the underlying model behaviors (Schut et al. 2023 Gurnee & Tegmark 2024)
Mechanistic interpretability is a field that attempts to achieve this goal by reverse-engineering neural network
decisions, translating them to numea--nterpretablle recision-making processes (Bereska & Gavves 2024
This is specifically done by analyzing models at the level of their internal computations, representations, and structural components, e.g., identifying minimal subnetworks (referred to circuits) that implement specific computation. A large appeal of mechanistic interpretability is that it provides causal explanations for models' outputs, i.e., a decision or prediction can be attributed in a causal manner to some component of the input. For example, causal tracing, a prominent tool in this line of work, perturbs internal activations in a manner that allows us to determine whether they cause a change the output. This allows researchers to move beyond correlation-base explanations and instead understand the actual computational mechanisms driving observed model behavior, which in turn enables more precise debugging, bias detection, and control over generative outputs. Mechanistic interpretability research has uncovered number of interesting and useful properties of DGMs. For example, specific neurons or layers in GANs and VAEs encode "disentangled" (i.e., distinct and interpretable) features, such as shape, texture, or pose in images (Shen et al. 2020: Mita et al. 2021 Other works have found that certain attention heads in transformers correspond to meaningful linguistic patterns, e.g., some might focus on syntactic structure while others might capture semantic information (Vig & Belinkov] 22199 Elhage et al. 2021]. Such properties not only help practitioners better understand DGMs, they also enable them to control generation some extent (Härkönen et al. 2020;
Fetty et al. 2020).
However, the reliability and comprehensiveness of mechanistic interpretability methods remain subject of debate (Golechha & Dao 2024a Sharkey et al., 2025). A central criticism is that, while these techniques aim to identify causal relationships between model components and outputs, they may only provide partial or even misleading insights into the actual computational processes at work. For instance, attention patterns which have been used by various methods to attribute model predictions to certain tokens (Xu et al. 2015; Choi et al.) 2016. inter alia) do not always faithfully reflect how why certain tokens influence the final prediction JJin & Wallace, 2019 Liu et al.) 2022b). In some cases, they may merely highlight correlations rather than reveal deeper causal structures. Similar concerns have been raised about other methods for ex- plaining model behaviors from mechanistic interpretability. Further, the new wave of large-scale generative models makes the application of some of these mothods more difficult: the larger a model becomes, the (arguably) more difficult it becomes to fully reverse-engineer a prediction, both from a theoretical and com- putational standpoint. Every muance of these models' decision-making may not be deducible from a subset of neurons, layers, or attention heads, and interpretations derived from one subset of model components may overlook equally critical interactions elsewhere in the large network.
Representation engineering [Zo et al. 2023a) and the use of sparse autoencoders (Bricken et al. 2023) are lines of research in mechauistic interpretability that potentially address the former issues, offering inter- pretable explanations even for large-scale models. Efficient methods for circuit identification have started to address the latter issue (Hsu et al., 2025). However, we are still in need of validation methods that can confirm whether the identifiod "mechanisms" " truly govern a model's outputs, or whether they merely reflect convenient, yel. incomplete, narratives about its internal workings.
Going forward, researchers should continually assess user needs for explainability to ensure that the appro- priate objectives are guiding the development of interpretability methods (Liao et al., 2020: Wang & Yin 2021 Poursabzi-Sangdeh et al. 2021). Attention must also be paid to the relevance and effectiveness of the metrics and evaluation frameworks used to assess these methods (Ross et al.) 2021 Jethani et al.) 2021 Another important research direction is enhancing the robustness of explainable meth such as counter- factual explanations (Wachter et al. 2017 Slack et al.) (2021). Further research is also needed for the new wave of multimodal models, as existing explainability methods may not be equipped to offer explanations in the face of modal interactions.
18

---

Published in Transactions on Machine Learning Research (08/2025)
4.5 Constraint Satisfaction
Generative models such as ChatGPT are used by millions of people and deployed across diverse use cases. Many applications require generative models to satisfy domain-spocific constraints To In some cases, these merely stem from desire to have a more controlled form of generation, such as when a generated image is conditioned on given depth map (Zhang et al. 2023b). In other cases, ethical and safety considerations are key concerns. For instance, in fields like engineering design, generative model outputs must meet engineering standards and adhere to laws of nature (i.e., physics). More generally, there are widespread calls for generative models to avoid toxicity, mitigate bias and prevent other outputs that may lead to harmful effects (Weidinger et al. 2021), e.g., by refraining from responding in ways that could pose a risk to the mental health of human interlocutors and by refusing to carry out tasks related to illegal activities. While reinforcement lcarning from human feedback (RLHF; Ouyang et al.) 2022)- and particularly constrained RLHF (Moskovitz et al,, 2024)- has offered initial step towards these goals, enabling companies and users to provide models with soft constraints within their queries, such constraints can be circumvented (Shen et al. 2023 Ultimately, methods that allow us to place hard constraints on model outputs are necessary.
In language generation, decoding methods that allow for arbitrary constraints (both hard and soft) have been a focus area of the research community (Kumar et al., 2021 2022). There are several prominent challenges in the development of such methods, inchuding: efficiently enforcing constraints without significantly increasing computational costs, maintaining fluency and coherence while adhering to constraints, and handling multiple constraints, which may result in conflicting requirements. The discrete nature of text presents particular difficulty, small changes to loken sequences can drastically alter meaning, making il difficult to oplimize decoding methods (Beurer- Kellner et al.) 2024 Ugare et al. 2024) address these issues by guaranteeing that every generated sequence conforms to a user-supplied context-free free grammar, achieving hard constraints with only a modest runtime overhead. Such methods have started to gain traction other domains, e.g., in healthcare (Golechha & Dao 2024b). Meanwhile, research on controllable image generation has likewise gained momentum (Deng et al. 2020 Huang et al.) 2024b), with various approaches aiming to regulate attributes such style, composition, specific content elements. Methods range from applying spatial constraints (e.g., bounding boxes, masks, or layout specifications (Zheng et al. 2023a)) ) to enforcing semantic conditions (c.g., ensuring that certain objects or visual features are present (Pavllo et al. 2020)). Technical challenges arise in this domain as well, such as the need for more complex conditioning mechanisms and heavier computational demands- especially when constraints must be integrated at each step of the generative process in e.g., diffusion models. Code generation stands out as domain where constraint enforcement has long been a primary focus (Poesia et al. 2022 Dong et al. 2023). Here, the constraints- such as following a language's syntax and producing compilable code arguably more well-defined and straightforward to verify. Concepts and methods from this field could conccivably help rescarch in enforcement of hard constraints in other generative AI fields, e.g. in the application of generative AT to the physical domain, where laws of nature must be satisfied. Overall, methods to ensure effective constraint adherence can substantially improve control over generative models, which is crucial for ensuring their safe and reliable deployment (Regenwetter et al. 2024). Despite progress across various generative AI fields, the development of scalable and generalizable techniques capable of handling the diverse and often conflicting demands of real-world constraints remains an open challenge. We encourage further research in this area to bridge this gap and advance the controllability of generative models across different domains.
for constraints from a computational perspective. Recent rrammar-constraiin
5 Conclusion
Generative Al has achieved remarkable progress in recent years, pushing the boundaries of what computa- tional systems can model and generate. Yet, this rapid evolution also comes with wide array of technical, ethical, and societal challenges that demand careful altention. Thus, the goal of achieving perfect genera- tive model remains far from reality, As we have outlined throughout this paper, current generative Al models
Here we focus on constraints specified at inference time. We discuss constraints that must be integrated into the model during training in 23
19

---

Published in Transactions on Machine Learning Research (08/2025)
struggle with robustness under distribution shift, remain resource-intensive,, and often exhibit behaviors that are difficult to interpret or control, particularly in high-stakes and specialized domains.
Crucially, the path forward is not merely a matter of scaling existing paradigms. Despite gains in per- formance from ever-larger models and training corpora, core limitations persist. Addressing them requires (re)consideration of foundational assumptions. We have emphasized the importance of integrating causal reasoning into model architectures to overcome spurious correlations and improve generalization, especially under distributional shifts. Likewise, embedding domain knowledge and inductive biases can enable more data-efficient learning in fields like healthcare, chemistry, and the physical sciences, where data is often limited but expert insight is abundant.
Efficieney and accessibility are also central concerns for facilitating the widespread use of DGMs. Today's DGMs are not only expensive to train and deploy but also have created a growing barrier to entry for researchers and practitioners without access to large-scale compute. Optimizing inference efficiency through quantization, architectural innovations, and fast sampling methods is essential for democratizing generative AI and minimizing its environmental footprint. At the same time, robust evaluation remains an open challenge: current metries often fail to align with human judgments or to meaningfully assess qualities such as diversity, calibration, or ethical compliance. The development of more principled and context- dependent evaluation frameworks potenniillly learned from human preferences will be vital to guiding model development and ensuring model trustworthiness.
Beyond technical concerns, we must also address the broader societal and ethical impacts that generative AT can have. Interpretability, fairness, and safety must. become a priority in design considerations, not afterthought. These systems have alrcady demonstrated the ability to generate misinformation, propagate social biases, infringe privacy, and produce unsafe or unreliable outputs. While recently-developed mitiga- tion strategies, from watermarking and uncertainty-awaal abstention to differential privacy and constraint- aware decoding, appear to offer promising solutions, the methods alone are insufficient without regulatory frameworks for ensuring their usage.
We believe that addressing this range of issues will require deeper cross- discipiisyyyyyaary collaboration, new benchmarks that reflect real-world complexity, and an intentional shift from optimizing for performance to optimizing for positive societal impact. By confronting the limitations discussed here, we can transform DGMs from data replicators (Bender et al. 2021 to tools with transformative capabilities across various domains. We hope that the information and opinions presented here will point to directions that ultimately contribute to these goals.
References
Moloud Abdar, Farhad Pourpanah, Sadiq Hussain, Dana Rezazadegan, Li Liu, Mohammad Ghavamzadeh, Paul Fieguth, Xiaochun Cao, Abbas Khosravi, U. Rajendra Acharya, Vladimir Makarenkov, and Saeid Nahavandi. A review of uncertainty quantification in deep learning: Techniques, applications and chal-
lenges. Inf. Fusion, 76(C):243-297, December 2021. ISSN 1566-2535. doi: (0.1066j.inffus.2222 2021.05.00. URL pttpp://ddo.orgg/// http://1..1116...fffff...
2021.05.00
Mohamed S. Abdelfattah, Abhinav Mehrotra, Fukasz Dudziak, and Nicholas D. Lane. Zero-Cost Proxies for Lightweight NAS. In International Conference on Learning Representations (ICLR), 2021.
Orevaoghene Ahia, Sachin Kumar, Hila Gonen, Jungo Kasai, David Mortensen, Noah Smith, and Yulia Tsvetkov. Do all languages cost the same? tokenization in the era of commercial language models. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empéri- cal Melhods in Natural Language Processing, pp. 9904-99229 9923, Singapore, December 2023. Association for Computational Linguistics. doi: 10. 10.118/611/2000 2wm22......mmm-mmm.emm www.p-mmin..666.6mmmmmm 614. URL https://aclantholll org/
2023. 2023.emmlp wwnne-nnnnmmmmmee maip. 614/
Orevaoghene Ahia, Sachin Kumar, Hila Gonen, Valentin Hofmann, Tomasz Limisiewicz, Yulia Tsvetkov, and Noah A. Smith. MAGNET: Improving the multilingual fairness of language models with adaptive gradient- based tokenization. In The eiirtt-eighth Annual Conference on Neural Information Processing Systems, 2024. URL /ttpss://oenrrevie wwt..e//tooruumnnd1eeeeossssss
20

---

Published in Transactions on Machine Learning Research (08/2025)
Walid Ahmad, Elana Simon, Seyone Chithrananda, Gabriel Grand, and Bharath Ramsundar. Chemberta-2; Towards chemical foundation models. orv preprinl or arxi:22009000 01712, 2022.
Kartik Ahuja, Divyat Mahajan, Yixin Wang, and Yoshua Bengio. Interventional causal representation learning. In International conference on machine learning, pp. 372 407. PMLR, 2023.
Joshua Ainslic, James Lco-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebron, and Sumit Sang- hai. GQA: Training generalized multi-query transformer models from multi-head checkpoints. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference Empirical Meth- ods in Natural Language Processing, pp. 4895 4901, Singapore, December 2023. Association for Compu- tational Linguistics. doi: 10.18653/v1/2022 monnl-mminn.... URL https:// 2rgpaa/ttoooooooooo/2000323 emnlp-main. /
Michael S. Albergo, Nicholas M. Boffi, and Eric Vanden-Fiindden Stochastic interpolants: unifying frame- work for flows and diffusions, 2023. URL
08797
Mchdi Ali, Michael Fromm, Klaudia Thellmann, Richard Rutmann, Max Lübbering, Johannes Level- ing, Katrin Klug, Jan Ebert, Niclas Doll, Jasper Buschhoff, Charvi Jain, Alexander Weber, Lena Ju- rkschat, Hammam Abdelwahab, Chelsea John, Pedro Ortiz Suarez, Malte Ostendorff, Samuel Weinbach, Rafet Sifa, Stefan Kesselheim, and Nicolas Flores-Herr. Tokenizer choice for LLM training: Negligi- ble or crucial? In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Findings of the Associ- ation for Compulational Linguislies: NAACL 2024, pp. Association for Computational Linguisties. doi: 10.18653/v1. //aclanthology.oo wtt.aaatoooooooooo/224/...f.. wwtp////.....niinnoooooeo. wwndinggs-naccl..... 247/
Mexico City, Mexico, June 2024. muddings-naacl.25 URL https:
Danial Alihosseini, Ehsan Montahaei, and Mahdieh Soleymani Baghshah. Jointly measuring diversity and quality in text generation models. In Antoine Bosselut, Asli Celikyilmaz, Marjan Ghazvininejad, Srinivasan Tyer, Urvashi Khandelwal, Hannah Rashkin, and Thomas Wolf (ods.), Proceedings of the Workshop on Methods for Optimizing and Evaluating Neural Language Generation, pp. 90-98, Minncapolis, Minnesota, URI. https://
June 2019. Association for Computational Linguistics. doi: (4.1663/v1/W199 aclanthology wwceloggy.o.g///oo W19-2311/
Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein. Learning to compose neural networks for question answering. In Kevin Knight, Ani Nenkova, and Owen Rambow (eds.), Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1 1545-1554 San Diego, California, June 2016a. Association for Computational Linguistics. doi: 10.18653/v1111
1181. URL https://acclantooooggy wttp://ttoooooooooo///----1 org/N16-1181/
Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein. Neural module networks. In 2016 IEEE Conference on Compuler Vision and Pallern Recognition (CVPR), pp. 39-48, 2016b. doi: (0.109/CVPP) 2016.12.
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp. 214.223. PMLR, 06-11 Aug 2017. URL https::/prooceeding: .ml. wttp:///v////ooovsyyyy/.r
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arxiv, 2019.
Saeid Asgari, Aliasghar Khani, Pereshte Khani, Ali Gholami, Linh Tran, Ali Mahdavi-Amiri, and Ghassan Hamarneh, Masktune: Mitigating spurious correlations by forcing to explore. In Alice H. Oh, Alekh Agar- wal, Danielle Belgrave, and Kyunghyun Cho (ods.), Advances in Neural Information Processing Systems, 2022. URL http:///ppppreeeeeeeeeeeorrrrrra www.:/ooooridddannnsaaoaaaao
Dzmitry Bahdanau, Kyung Hyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In 3rd International Conference on Learning Regresentations, TCLR 2015, 2015.
21

---

Published in Transactions on Machine Learning Research (08/2025)
Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitational ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073 2022.
Yogesh Balaji, Rama Chellappa, and Soheil Feizi. Robust optimal transport with applications in generative modeling and domain adaptation. Aduances in Neural Information Processing Systems, 13:12934-12944 2020.
Peter Battaglia, Jessica Blake Chandler Hamrick, Victor Bapst, Alvaro Sanchez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, Caglar Gulcehre, Francis Song, Andy Ballard, Justin Gilmer, George E. Dahl, Ashish Vaswani, Kelsey Allen, Charles Nash, Vic- toria Jayne Langston, Chris Dyer, Nicolas Heess, Daan Wierstra, Pushmeet Kohli, Matt Botvinick, Oriol Vinyals, Yujia Li, and Razvan Pascanu. Relational inductive biases, deep learning, and graph networks. CoRR, abs/1806.01 1///11806012211 2018. URL https://arxiv: http:///aaxxxv..gpp///////s 010pp//....000.....1.11 01261.
Anya Belz, Craig Thomson, Ehud Reiter, and Simon Mille. Non-repeatable experiments and non-reproducible results: The reproducibility crisis in human evaluation in NLP. In Anna Rogers, Jordan Goyd-Graber and Naoaki Okazaki (eds.), Findings of the Association for Computational Linguisties: ACL 2023, pp. 3676 3687, Toronto, Canada, July 2023. Association for Compulational Linguistios. doi: 10.18653/v1/ 2023. findiininng.c.... ac1.226. URL https: ptttps://aclanthoooogy 2tg/2023 findings-acl..222 226/
Emily M. Bender, Timnit Gebru, Angelina McMillan Major, and Shmargaret Shmitchell. On the dangers of stochastic parrots: Can language models be too big? Proceedings of the 2021 ACM Conference on Fairness, Accounlabilily, and Transporency, 2021.
Leonard Bereska and Stratis Gavves. Mechanistic interpretability for AT safety - a review. Transac- tions on Machine Learning Research, 2024. ISSN 2835-8856. URL https://openreview.nee id=ePUVetPKu6 Survey Certification, Expert Certification.
Eyal Betzalel, Coby Penso, and Ethan Fetaya. Evaluation metrics for generative models: An empirical study, Machine Learning and Knowledge Extraction, 6(3):1531 1544, 2024. ISSN 2504-4990. doi: 10. 6390/make6000073. URL https: http://wwwwwwaeee wtw.ppp..coo//54444999/5555
Luca Beurer-Kellner, Mare Fischer, and Martin Vechev. Guiding 1lms the right way: fast, non-invasive con- strained generation. In Proceedings of the 41st International Conference on Machine Learning, TCMT/24.
Jog,.org, 2024.
Mikolaj Binkowski, Dougal J. Sutherland, Michael Arbel, and Arthur Gretton. Demystifying MMD GANs. In International Conference on Learning Representations, 2018. URL https://ppeerrvvee..ntttttt www.:eewwweeefforerreeo id=r 1100WWW
Andreas Blattmann, Robin Rombach, Kaan Oktay, Jonas Müller, and Björn Ommer. Retrievallangmented diffusion models. Advances in Neural Information Processing Systems, 35:15309 15324, 2022.
Rosalin Bonetta and Gianluca Valentino. Machine learning techniques for protein function prediction. Pro- teins: Structure, Function, and Bioinformaties, 88(3):397-44 411,,411,440 2020.
Ali Borji. Pros and cons of gan evaluation measures: New developments. Computer Vision and Image Understanding. 215:103329, 2022.
Zalán Borsos, Raphaël Marinier, Damien Vincent, Eugene Kharitonov, Olivier Pietquin, Matt Sharifi, Do-
minik Roblek, Olivier Teboul, David Grangier, Marco Tagliasacchi, et al. Audiolm: a language modeling approach to audio generation. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2023.
Mark Braverman. On the complexity of real functions. In 46th Annual IEFE Symposium on Foundations of Computer Science (FOCS 05), pp. 155-164. TEEE, 2005.
22

---

Published in Transactions on Machine Learning Research (08/2025)
Mark Braverman. Communication and information complerity, pp. 284 320. EMS Press, December 2023. ISBN 9783985475599. doi: /0.4111//cm2222/// /208. URL. http://dx. dow..ddoi..... org/ 2/g//0.0o 41171///mmm2222/208
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravee, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac HatfieldDoodds, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, and Christopher Olah. Towards monose- manticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2023. http://ransfoomrrrrrrrtttt pub/2023, monosemannic- featuressttteeeeeeeeee hww.uureeeiiee...ee /tt///indx.tmmtm html.
Michael M. Bronstein, Joan Bruna, Taco Cohen, and Petar Veličkovié. Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges. arXiv preprint 2rXiv:2104.13478, 2021.
Tim Brooks, Bill Peebles, Connor Homes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. URL https://openaa: http::///poaai.com/rreeaccch com/research/
Video generation models as world simulators. 2024.
video satto---ennrrttt---oooll-------- aarllls--aas-ooddddsmmmattrtro
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Nee- lakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christo- pher Berner, Sam McCandlish, Alec Radford, 1lya Sutskever, and Dario Amodei. Language models are few-shot Iearners. In Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (cds.), Advances in Newral Information Processing Systems, volume 33, pp.
Curran Associates, Inc., 2020.
Felix Busch, Lena Hoffmann, Christopher Rueger, Elon van Dijk, Rawen Kader, Esteban Ortiz-Prado, Marcus Makowski, Luca Saba, Martin Hadamitzky, Jakob Kather, Daniel Truhn, Renato Cuocolo, Lisa Adams, and Keno Bressem. Current applications and challenges in large language models for patient care: systematic review. Communications Medicine, 5, 01 2025. doi: 10.1038/s44 -43856-024-000111
Massimo Caccia, Lucas Caccia, William Fedus, Hugo Larochelle, Joelle Pineau, and Laurent Charlin. Lan- guage gans falling short. In International Conference on Learning Representations, 2019.
Lon R Cardon and Lyle J Palmer. Population stratification and spurious allelic association. The Lancet, (9357)::5998-- 604, 2003.
Nicholas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramer, Borja Balle, Daphne Ippolito, and Eric Wallace. Extracting training data from diffusion models. In Proceedings of the 32nd USENIX Conference on Security Symposium, SEC '2, 2023a.
Nicolas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramer, Borja Balle, Daphne Ippolito, and Eric Wallace. Extracting training data from diffusion models. In 32nd USENIX Security Symposium (USENIX Securily 23), pp. 5253 5270, 2023b.
SR. Carr and N. Jeffrey. Class Action Complaint. Sarah Anderson, et al., Stability AI LTD., et al, 2022.
Cayque Monteiro Castro Nascimento and André Silva Pimentel. Do large language models understand chemistry? conversation with chatgpt. Journal of Chemical Information and Modeling, 63(6):1649 1655, 2023. doi: 10.1021 10.1021/acs.jehe jam..aa...m..3000255.. URL http:://///i..org///.00/0/////// 3ttp://.....00//.......m.0000000 3c0003 PMID: 36926868.
Chen Chen, Daochang Liu, and Chang Xu. Towards memorization-free diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8425 8434, June 2024a.
Hailin Chen, Fangkai Jiao, Xingxuan Li, Chengwei Qin, Mathicu Ravaut, Ruochen Zhao, Caiming Xiong, and Shafiq R. Joty. Chatgpt's one-year anniversary: Are open-source large language models catching up? Arxiv, abs/2311. 16989, 2023.
23

---

Published in Transactions on Machine Learning Research (08/2025)
Irene Y Chen, Shalmali Joshi, Marzych Ghassemi, and Rajesh Ranganath. Probabilistic machine learning for healthcare. Annual review of biomedical dala science, 4:393-414 1008,-11, 2021a.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pondé de Oliveira Pinto, Jared Kaplan, Har- rison Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Fe- lipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Flizabeth Barnes, Ariel Herbert- Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Joshua Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Nya. Sutskever, and Wojciech Zaremba. Evaluating large language models trained on code. ColR,, abs/2107.0331 03374, 2021b.
Achiam,
Sitan Chen, Jerry Li, and Yuanzhi Li. Learning (very) simple generative models hard. Advances in Neural Information Processing Systems, 35:35143 35155, 2022.
Yanxi Chen, Xuchen Pan, Yaliang Li, Bolin Ding, and Jingren Zhou. Ee-llm: large-scale training and inference of carly-exit large language models with 3d parallelism. In Proceedings of the 41st International Conference on Machine Learning, ICMT 24. JML..orrg, 2024b.
Edward Choi, Mohammad Taha Bahadori, Joshua A. Kulas, Andy Schuetz, Walter F. Stewart, and Jimeng Sun. RETAIN: an interpretable predictive model for healthcare using reverse time attention mechanism. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS 16, pp. 3512-3520 Red Hook, NY, USA, 2016. Curran Associates Inc. ISBN 9781510838819.
June Suk Choi, Kyungmin Lee, Jongheon Jeong, Saining Xie, Jinwoo Shin, and Kimin Lee. Diffusionguard: A robust defense against malicious diffusion- based image editing. In The Thirteenth International Conference on Learning Representations, 2025. URL htttt::// wtw.eearrriiieeeee/rrrruur///eee
Seungtaek Choi, Myeongho Jeong, Hojae Han, and Seung-won Hwang. C21: Causally contrastive learning for robust text classification. Proceedings of the AAAI Conference on Artificial Intelligence, 36(10)110226- 10534, Jun. 2022.
Min Jin Chong and David Forsyth. Effectively unbiased fid and inception score and where to find them. In Proceedings of the IEEE/CVE /CFF conference on computer vision and pattern recognition, pp. 6070-6079, 2020.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam M. Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Benton C. Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier García, Vodant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayana Pillai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Sacta, Mark Díaz, Orhan Firat, Michele Catasta, Jason Wei, Kathloen S. Meier- Hellstern, Douglas Eck, Jeff Dean, Slav Petrov, and Noah Fiedel. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research,
2022.
Yeounoh Chung, Tim Kraska, Neoklis Polyzotis, Ki Hyun Tae, and Steven Fuijong Whang. Slice finder: Automated data slicing for model validation. In 2019 IEEE 35th International Conference on Data Engi- neering (ICDE), pp. 1550-1553, 2019. doi:
Christopher Clark, Mark Yatskar, and Luke Zettlemoyer. Don't take the easy way out: Ensemble basod methods for avoiding known dataset biases. In Kentaro Imui, Jing Jiang, Vincent Ng, and Xiaojun Wan (eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and
24

---

Published in Transactions on Machine Learning Research (08/2025)
the 9th International Joint Conference Natural Language Processing, pp. 4067-4080. Association for Computational Linguistics, 2019.
Elizabeth Clark, Tal August, Sofia. Serrano, Nikita Haduong, Suchin Gururangan, and Noah A. Smith. All that's 'human' is not gold: Evaluating human evaluation of generated text. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.); Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 1182-72900 Online, August 2021. Association for Computational Linguistics. doi: 10. 10.18653/v1/2022 http://11/22...a-ooo....... 565. URL httpps://aclantholooo
565/
Roberto Confalonieri, Ludovik Coba, Benedikt Wagner, and Tarek R. Besold. A historical perspective of explainable artificial intelligence. WTREs Data Mining and Knowledge Discovery, 11(1) Jee331 2021. doi: https:///ooi.org/ 11110/0///mmm//m/m 1391. URL https://wind onlinelibrary wiley..wwee................ com /oi/////bb////////////////
10. 10.10w//wddm 1391
Jonathan Crabbe, Zhaozhi Qian, Fergus Imrie, and Mihaela van der Schaar. Explaining latent representa- tions with corpus of examples. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Newral Information Processing Systems, volume 34, pp. 12154-12166. Cur- ran Associates, Inc., 2021. URL https: wetggs..eurip..oo nowrip..ccppppprrr http://s.ccpppppo...ee/////////p fitpe//ppppppc/222//eeee
pdf
Miles D. Cranmer; Sam Greydanus, Stephan Hoyer, Peter W. Battaglia, David N. Spergel, and Shirley Ho. Lagrangian neural networks. CoRR, abs/2003.04555 2020. URL http://ablpp.uii uii-trier .et.t.rrreerdrdd////e
journalsccort/com cor.r///oor2030.o00 #ttt::#aab200000445
Rachel Cummings, Damien Desfontaines, David Evans, Roxana Geambasu, Yangsibo Huang, Matthew Jagielski, Peter Kairouz, Gautam Kamath, Sewoong Oh, Olga Ohrimenko, Nicolas Papernot, Ryan Rogers, Milan Shen, Shuang Song, Weijie Su, Andreas Terzis, Abhradoep Thakurta, Sergei Vassilvitskii, Yu-Xiang Wang, Li Xiong, Sergey Yekhanin, Da Yu, Huanyu Zhang, and Wanrong Zhang. Advancing Differential Privacy: Where We Are Now and Future Directions for Real-Worke Deployment. Harvard Data Science Review, 6(1), jan 16 2024. https:/ http://ds..mtipreee mit. mwt.eduuttttee cadu/pub/s/hi Sg.gh.
Tri Dao. Flashattention-2: Faster attention with better parallelism and work partitioning. In The Twelfth International Conference on Learning Represenlations, 2024. URL httpp::/opeareriie..nee///e m2n2Xyh9Ec
Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. FlashAttention: Fast and memory- efficient exact attention with 10-awareness. In Advances in Neural Information Processing Systems, 2022.
Tirtharaj Dash, Ashwin Srinivasan, and Lovekesh Vig. Incorporating symbolic domain knowledge into graph neural networks. Machine Learning, 11097))669--1636, Jul 2021. ISSN 1573-0565. doi: 10.1007/ s10994-021-00966-- URL http::://ai.orrg////////// wtt.:://......01711111119--0--10
Tirtharaj Dash, Sharad Chitlangia, Aditya Abuja, and Ashwin Srinivasan. A review of some techniques for inclusion of domain-knowledde into deep neural networks. Scientific Reports, 12(11)1040, Jan 2022. ISSN 2045-2322. doi: 10111////1110000011-050500000000 URL https:://oot: http://0..00rr/0000110/////11111
Sumanth Dathathri, Abigail See, Sumedh Ghaisas, Po-Sen Huang, Rob McAdam, Johannes Welbl, Vandana Bachani, Alex Kaskasoli, Robert. Stanforth, Tatiana Matejovicova, Jamie Hayes, Nidhi Vyas, Majd Al Merey, Jonah Brown-Cohen, Rudy Bunel, Borja Balle, Taylan Cemgil, Zahra Ahmed, Kitty Stacpoole, Ilia Shumailov, Ciprian Baetu, Sven Gowal, Demis Hassabis, and Pushmeet Kohli. Scalable watermarking for identifying large language model outputs. Nature, 634 634(8035):818 823, Oct 2024. ISSN 1476-4687. doi: (0.1038/s41586--2224
URL
Imant Daunhawer, Thomas M. Sutter, Kieran Chin-Chcong, Emanuele Palumbo, and Julia E Vogt. On the limitations of multimodal VAEs. In International Conference on Learning Representations, 2022. URL
https://openreeview.ct wtwp:///wwwwwe...e.e//.////tt/aa
25

---

Published in Transactions on Machine Learning Research (08/2025)
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pallern recognilion, pp. Teee, 2009.
Yu Deng, Jiaolong Yang, Dong Chen, Fang Wen, and Xin Tong. Disentangled and controllable face image gen- cration via 3d imitative-contrastiie learning. 2020 IEEE/CVE Conference on Computer Vision and Pat- lern Recognition (CVPR), pp. 5153 5162, 2020. URL http://pp...emanticcccooo.ll.. http:/cccccooaaoooororrrruoaaa
216144588
Emily L Denton, Soumith Chintala, Rob Fergus, et al. Deep generative image models using a laplacian pyramid of adversarial networks. Advances in neural information processing systems, 28, 2015.
Stefan Depeweg, Modeling Epistemic and Aleatoric Uncertainty with Bayesian Newral Networks and Latent Variables. PhD thesis, Technische Universität München, 2019. URL luttps://mediatum. ub. tum. de/
1482483
Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. Gpt3. int8 (): 8-bit matrix multiplication for transformers at scale. Advances in Neural Information Processing Systems, 35:30318 13:30388-3033,, 2022.
Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. QLORA: efficient finetuning of
quantized LLMs. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS '23, Red Hook, NY, USA, 2023. Curran Associates Inc.
Daniel Deutsch, Rotem Dror, and Dan Roth. Re-examining system-level correlations of automatic sum- marization evaluation metrics. In Marine Carpuat, Maric-Catherine de Marneffe, and Ivan Vladimir Meza. Ruiz (eds.), Proceedings of the 2022 Conference of the North American Chapler of the Association for Computational Linguistics: Human Language Technologies, pp. 6038 6052, Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022 naacl maacl-main.4ee 442..442. URL https://aclaathoooog.oor///////
naacl main.. 442/
Lue Devroye. Complexity questions in non-uniform random variate generation. In Proceedings of COMP- 2011 19th International Conference on Computational StatisticsParia France, August 22-27, 2010 Keynote, Invited and Contributed Papers, pp. 3-18. Springer, 2010.
Sander Dieleman, Laurent Sartran, Arman Roshannai, Nikolay Savinov, Yaroslav Ganin, Pierre H Richemond, Arnaud Doucet, Robin Strudel, Chris Dyer, Conor Durkan, et al. Continuous diffusion for categorical data. arXiv preprint (.Xix:2211.15000 2022.
Tim Dockhorn, Arash Vahdat, and Karsten Krcis. Scoro-based generative modeling with priically-dampel langevin diffuusion. In International Conference on Learning Represenlations, 2022.
Tim Dockhorn, Tianshi Cao, Arash Vahdat, and Karsten Kreis. Differentially private diffusion models. Transactions on Machine Learning Research, 2023. ISSN 2835-8856.
Yihong Dong, Ge Li, and Zhi Jin. Codep: Grammatical seq2seq model for general-purpose code generation. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2023, pp. 188-198, New York, NY, USA, 2023. Association for Computing Machinery. ISBN
9798400702211. doi: 10.1145/3597926..55 3599048 URL
1tt.://..rr.g1111111133//9999990 3598048
Carsten F. Dormann. Calibration of probability predictions from machine-learning and statistical models. Global Ecology and Biogeography, 76(4):600-765 2020. doi: https:/ 1ob.orr/10.1111//e.133077 URL
13070
Anil R. Doshi and Oliver P. Hauser. Generative ai enhances individual creativity but reduces the collective diversity of novel content. Science Advances, 10(28) saddn5290, 2024. doi: 00.118//ciadvv.adn2222 URL add22999...
https://ww..ccinns wwwwwwwwwwww.ceeoow..t.ww.. org/ hts/doo//ae hath///s/10... /12//scciaii adn5290
Alexey Dosovitskiy and Thomas Brox. Generating images with perceptual similarity metrics based on deep networks. Advances in neural information processing systems, 29, 2016.
26

---

Published in Transactions on Machine Learning Research (08/2025)
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Un- terthiner, Mostafa. Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Rep- resentations, 2020.
Danny Dricss, F. Xia, Mchdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Ho Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussaint, Klaus Greff, Andy Zeng, Igor Mordatch, and Peter R. Florence. Palm-e: An embodicd multimodal language model. In International Conference on Machine Learning, 2023.
Menguan Du, Fengxiang He, Na Zou, Dacheng Tao, and Xia Hu. Shortcut learning of large language models in natural language understanding. Communications of the ACM, 6711:111-- 2023a.
Mengnan Du, Subhabrata Mukherjee, Yu Chong, Milad Shokouhi, Xia Hu, and Ahmed Hassan. Robustness challenges in model distillation and pruming for natural language understanding. In Proceedings of the 1 17th Conference of the European Chapter of the Association for Computational Linguistics, pp. 1758-1770, 2023b.
André Vicente Duarte, Xuandong Zhao, Arlindo Oliveira, and Lei Li. DE-COP: content in language models training data. In Forty-first International Conference on Machine Learning, 2024. URL httpa:///penreevie.. wttp:/iiww..eteooorrre/eddAAdAa.
Detecting copyrighted
Ricard Durall, Margret Keuper, and Janis Keuper. Watch your up-convolution: CNN based generative deep neural networks are failing to reproduce spectral distributions. 2020 IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 7887-7896, 2020. Utthttps:://ee httts:///pppi semanticscholar.
org/CorpusID 211988660
Cynthia Dwork and Aaron Roth. The algorithmic foundations of differential privacy. Found. Trends Theor, Comput. Sci., 9(3 9(3-4):211-407 August 2014. ISSN 1551-305X. doi: //doi. org/10.0010 1561
URL https:
Vineet Edupuganti, Morteza Mardani, Shreyas Vasanawala, and John Pauly, Uncertainty quantification in deep mri reconstruction. IEEE Transactions on Medical Imaging, 40(1):239-25) 250, 2021. doi: 10.1109/TMM 2020.3025065.
A Aparna Elangovan, Ling Liu, Lei Xu, Sravan Babu Bodapati, and Dan Roth. ConSiDERS-the-hune evaluation framework: Rethinking human evaluation for generative large language models. In Lun-We Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Compulational Linguistics (Volume 1: Long Pupers), pp. 1137 1160, Bangkok, Thailand, August URL https://
2024. Association for Computational Linguistics. doi: 10.18653/vv aclanthology org/2024. 2.a.thology.rr//2224...... acl-long.666. 16/
Yanai Flazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, and Yoav Goldberg. Measuring and Improving Consistency in Pretrained Language Models. Transactions of the Association for Computational Linguisties, 9:1012 1031, 12 2021. ISSN 2307-387X. doi: 10.1162/ a 00410. URL
Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, et al. A mathematical framework for transformer circuits. Trans- former Circuits Thread, 11)::12, 2021.
Patrick Faser, Robin Rombach, and Bjorn Ommer. A disentangling invertible interpretation network for explaining latent representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pallern Recognition (CVPR), June 2020.
Patrick Esscr, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11873- 12883, 2021.
27

---

Published in Transactions on Machine Learning Research (08/2025)
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, Kyle Lacey, Alex Goodwin, Yannik Marek, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthesis, 2024. URL httts://aarat awwiv..xivvv... http:///arxxv.ooraa//////// wwwp/b/////...o.ooo////
Linxi Fan, Guanzhi Wang, Yunfan Jiang, Ajay Mandlekar, Yuncong Yang, Haoyi Zhu, Andrew Tang, De- An Huang, Yuke Zhu, and Anima Anandkumar. Minedojo: Building open-ended embodied agents with internet-scale knowledge. In Thirty-sirth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022.
Xiao Fang, Shangkun Che, Minjia Mao, Hongzhe Zhang, Ming Zhao, and Xiaohang Zhao. Bias of ai- generated content: an examination of news produced by large language models. Scienlific Reports, 14(1): 5224, Mar 2024. ISSN 2045-2322.
doi: 10.1038 00.1038//41598.0 028-880024-5588- URL https://doi. wtw.oo..//./..0.111 1038/
051598-024-556862
Yu Fei, Yifan Hou, Zeriing Chen, and Antoine Bosselut. Mitigating label biases for in-context learning. In Anna Rogers, Jordan Boyd-Grabes and Naoaki Okazaki (eds.). Proceedings of the 61st Annual Meeting of the Associalion for Compulational Linguistics ( (Volume 1: Long Pupers), pp. 14014-1403) Toronto, Canada, July 2023. Association for Computational Linguistics. doi:
long. 783. URL
https::/aclanthology.ooom www.://.................. acl- acl-long. 783/
Pierre Fernandez, Guillaume Couairon, Hervé Jégou, Matthijs Douze, and Teddy Furon. The stable signature: Rooting watermarks in latent diffusion models. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 22409 22420, 2023. doi: 10. 10.100//CCCV510072222...000
Emilio Ferrara. Fairness and bias in artificial intelligence: A brief survey of sources, impacts, and mitigation strategics. Sci, 6(1), 2024. ISSN 2413-4155. doi: 10.3390/sci0010003. URL htttps:/wwk http///wwwpppp....///////// com/
2413-4155/6//3
Lukas Fetty, Mikael Bylund, Peter Kuess, Gerd Heilemann, Tufve Nyholm, Dietmar Georg, and Tommy Löf- stedt. Lalent space manipulation for high-resolution medical image synthesis via the sty stylegan. Zeilschrifi für Medizinische Physik, 30(4):305 314, 2020. ISSN 0939-3889. doi: btttps://doi.orr //tp.//////....10/0/.ezeeee zemedi.
2020.05.001. URL https: /typs:://ww... www. sciencedirect. coom//ccence//rrtic/// htt.a////..oaaaarrraa///////////
Vincent Fortuin. Priors in Bayesian deep learning: A review. International Statistical Review, 90(3):563-59) 591, 2022.
Vincent Fortuin, Matthias Hüser, Francesco Locatello, Heiko Strathmann, and Gunnar Rätsch. SOM-VAE: Interpretable discrete representation learning on time series. In International Conference on Learning Representations, 2019.
Vincent Fortuin, Dmitry Baranchuk, Gunnar Rätsch, and Stephan Mandt. GP-VAE: Deep probabilistic time series imputation. In International Conference on Artificial Intelligence and Statistics, pp. 1651 1661. PMLR, 2020.
Joel Frank, Franziska Herbert, Jonas Ricker, Lea Schönherr, Thorsten Eisenhofer, Asja Fischer, Markus Dürmuth, and Thorsten Holz. A representative study human detection of artificially generated media across countries. In 2024 IEEE Symposium on Security and Privacy (SP), pp. 159-159. IEEE Computer Society, 2024.
Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. OPTQ: Accurate quantization for gen- erative pre-trained transformers. In The Eleventh International Conference on Learning Represenlations, 2023. URL httppe://oeenreviee
Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K. Ahmed. Bias and fairness in large language models: A survey. Computational Linguistics, 1097),1097 09 2024. ISSN 0891-2017. doi: 10.1162/coli 00524. URL http:://abb..rrr//00116///..... _
28

---

Published in Transactions on Machine Learning Research (08/2025)
Danicle Gambetta, Gizem Gezici, Fosca. Giannotti, Dino Pedreschi, Alistair Knott, and Luca Pappalardo. A linguistic analysis of undesirable outcomes in the era of generative AT, 2024. URL https://arxiv.or abs/2410. 12341
Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Qianyu Guo, Meng Wang, and Haofen Wang. Reurieval-augmmented generation for large language models: A survey. Arxiv, abs/2312. 109977 2023. URL :ttt::::/app..emmatticceee http:///wwaaaaaaooooooooooooooor wrtp:aooooooossoee233399909
Jakob Gawlikowski, Cedrique Rovile Njieutcheu Tassi, Mohsin Ali, Jongseok Lee, Matthias Humt, Jianxiang Feng, Anna Kruspe, Rudolph Tricbel, Peter Jung, Ribana Roscher, Muhammad Shahzad, Wen Yang, Richard Bamlerr and Xiao Xiang Zhu. A survey of uncertainty in deep neural net nerksrks. Artificial In- telligence Review, 56(1):1513- Oct 2023. ISSN 1573-7462. doi: 10.1007/s10462-000 10562-9. URL ottp:///o...aoo///..0000/0000000 10562-9
Marzych Ghassemi, Tristan Naumann, Peter Schulam, Androw Beam, Irene Y Chen, and Rajesh Ran- ganath. A review of challenges and opportunities in machine learning for health. AMIA JL Summiils Transl Sci Proc, 2020:191 200, May 2020.
Mario Giulianelli, Joris Baan, Wilker Aziz, Raquel Fernández, and Barbara Plank. What comes next? evalu- ating uncertainty in neural text generators against human production variability. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.); Proceedings of the 2023 Conference on Empirical Melhods in Natural Lan- guage Processing, pp. 14349-1493 Singapore, December 2023. Association for Computational Linguistics. URL huttps://allatthol.aa
doi: 10.18653/v1 000653//112023.c//// wa0.022.mmlpmmmi.........
org/2023 mainp-mminn...
Amelia Glaese, Nat McAleese, Maja Trebacz, John Aslanides, Vlad Firoiu, Timo Ewalds, Maribeth Rauh, Laura Weidinger, Martin Chadwick, Phoebe Thacker, et al. Improving alignment of dialogue agents via targeted human judgements. arXiv preprint arXiv:2209 14335,, 2022.
Scraphina Goldfarb-Tarran Adam Lopez, Roi Blanco, and Diego Marcheggiani. Bias beyond English: Counterfactual tests for bias in sentiment analysis in four languages. In Anna Rogers, Jordan Boyd- Graber, and Naoaki Okazaki (eds.), Findings of the Association for Computational Linguistics: ACL 2023, pp. 5558-446,, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10. 18653/v1/2022 www../....innnnaaalea..... acl.n27. URL
findings-dddnddn 272/
Satvik Golechha and James Dao. Challenges in mechanistically interpreting model representations. ICML 2024 Workshop on Mechanistic Interpretability, 2024a. URL https://pperreviie..et//ttt
Id=wfemKUcgoB
Satvik Golechha and James Dao. Challenges in mechanistically interpreting model representations. In ICML 2024 Workshop on Mechanistic Interpretabllity, 2024b. URL https://openrevieu wttp:rewwwii..ee/fuurrrr.. net/forum?
id=wfenKUcgoß
Hila Gonen and Yoav Goldberg. Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.),
Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pp. 609 614, Minneapolis, Minncsota, June 2019. Association for Computational Linguistics. doi: 10.18653 /v1 11/119-1661 URL
http:://acclatthooogg..oo////
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, and Abhishek Kadian et al. The Ilama 3 herd of models, 2024. URL https: org/ abg///b5/2407 21783
David Greenfield and Shivan Bhavnani. Social media: generative Al could harm mental health. Nature, 617 (7962):676 676, May 2023. doi: 10.1033//A41556-023--0110
Albert Gu and Tri Dao. Mamba: Linear-time time sequence modeling with selective state spaces, arXio preprint arXiv:2312.0075. 2023.
29

---

Published in Transactions on Machine Learning Research (08/2025)
Albert Gu, Karan Goel, and Christopher Re. Efficiently modeling long sequences with structured state spaces. In International Conference on Learning Regresenlations, 2021.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning, pp. 1321 1330. PMLR, 2017.
Wes Gurnee and Max Tegmark. Language models represent space and time. In The Tweljth International Conference Learning Representations, 2024. URL https://openrevieer hot.:ee.nnee//oomrr1ddddd/aaaa
Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. Annotation artifacts in natural language inference data, In Marilyn Walker, Heng Ji, and Amanda Stent (eds.). Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pp. 107-112, New Orleans, Louisiana, June 2018. Association for Compulational Linguistics. doi: 10.18653/v1/N188 URL
Org/N18-2000
Wenjuan Han, Bo Pang, and Ying Nian Wu. Robust transfer learning with pretrained language models through adapters. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Association for Compulational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 2: Short Papers), Virtual Event, August 1-6, 2021, pp. 854 861. Association for Computational Linguistics, 2021. doi: 10.18653/
"1/2021AACLSSHORT. 108. URL httt::///od..oggg//..816//////// 18611001121113/1/22222 act aaleiaacl-shiie sot.sshhot...00
Matthew Hausknecht, Prithviraj Ammanabrolu, Marc-Alexane Alexandre Côté, and Xingdi Yuan. Interactive fiction games: A colossal adventure. Proceedings of the AAAI Conference Artificial Intelligence, 34(05): 7903-7910, Apr. 2020.
Marton Havasi, Robert Peharz, and José Miguel Hernández-Lobato. Minimal random code learning: Getting bits back from compressed model parameters. In 7th International Conference on Learning Representa- lions, ICLR 2019, 2019.
Jiawei He, Yu Gong, Joseph Marino, Greg Mori, and Andreas Lehrmann. Variational autoencoders with jointly optimized latent dependency structure. In International Conference on Learning Representations, 2019. URL https////pppparrreeoeooeorrrrrrn
Todd C. Helmus. Artificial Intelligence, Deepfakes, and Disinformation: A Primer. RAND Corporation, Santa Monica, CA, 2022. doi: 10.7249 11249/PEA114--1.
Tom Henighan, Jared Kaplan, Mor Katz, Mark Chen, Christopher Hesse, Jacob Jackson, Heewoo Jun, Tom B. Brown, Prafulla Dhariwal, Scott Gray, Chris Hallacy, Benjamin Mann, Alec Radford, Aditya Ramesh, Nick Ryder, Daniel M. Ziegler, John Schulman, Dario Amodei, and Sam McCandlish. Scaling laws for autoregressive generative modeling. CoRR, 2b112211.,4 2020. URL abs/2010 14701
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural infor- malion processing systems, 30, 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 6840 Curran Associates, Inc., 2020. URt/ttpe://prooeedug.ooo
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. In Advances in Neural Information Processing Systems, volume 35, pp. 8633- 8646. Curran Associates, Inc., 2022a.
30

---

Published in Transactions on Machine Learning Research (08/2025)
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. In Advances in Neural Information Processing Systems, volume 35, pp. 8633- 8646, 2022b.
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Thomas Hennigan, Eric Noland, Katherine Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karén Simonyan, Erich Elsen, Oriol Vinyals, Jack Rae, and Laurent Sifre. An empirical analysis of compute- optimal large language model training. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 3001 300000 Cur- ran Associates, Inc., 2022. URL htttt::://pproeeeeee c122faff6f6688770511110 1tw.09551114ebb04a3eeee Paper- Conf wwonferenne.... pdf
neurips. neurips.cc/pppprr httpp/////pppppr...ee//p///////a /itpp///i//////pppr/22///////
Emiel Hoogcboom, Jonathan Heck, and Tim Salimans. simple diffusion: End-to-cnd diffusion for high resolution images. arXiw preprint arXiv:230..11093, 2023.
Neil Houlsby, Ferene Huszár, Zoubin Ghahramani, and Máté Lengvel. Bayesian active learning for classifi- cation and preference learning. ArXiv, abs/1112.57745, 2011. URL https://apl semanticscholar.
Corpus ID: 13612582
Jessica N Howard, Stephan Mandt, Daniel Whitcson, and Yibo Yang. Learning to simulate high energy particle collisions from unlabeled data. Scientifie Reports, 12(1):7567, 2022.
Aliyah R. Hsu, Georgia Zhou, Yeshwanth Cherapanamjeri, Yaxuan Huang, Anobel Odisho, Peter R. Carroll, and Bin Yu. Efficient automated circuit discovery in transformers using contextual docomposition. In The Thirleenth International Conference on Learning Regresentations, 2025. URL https://openreeieew.eet
?orum?id=41 71-44111BY151
Jennifer Hu and Roger Levy. Prompting is not a substitute for probability measurements in large language models. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 5040-5060, Singapore, December 2023. Association for Computational Linguistios. doi: 10. 18613/v6// org/2023.
URL. httaps://aclaatholloy/
emm.2222m.mmmmppmmmmmmmmmmmmmmmm wwweeemnnllleeeeeeeeee 306/
Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. Strategies for pre-training graph neural networks. In International Conference on Learning Representa- tions, 2020.
Haoyang Huang, Tianyi Tang, Dongdong Zhang, Xin Zhao, Ting Song, Yan Xia, and Furu Wei. Not all languages are created equal in LLMs: Improving multilingual capability by gros-linguallthoughh prompt- ing. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computa- tional Linguistics: EMNLP 2023, pp. 12365-12394, Singapore, December 2023a. Association for Compu- tational Linguistics. doi: 10. 10.18653/v 11////1///1/ /2023 findings-emnlp..e URI httthttts///ccanthaee htwpptthooooooooo//222022222 org/2023. madingg-emmlp.8255
Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Tiu. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Trans. Inf. Syst., November 2024a. ISSN 1046-8188. doi: 10.1145/3703155. URL http:::t///// //dps://doi..o org/ 10. 1145/3703155 Just Accepted.
Qingqing Huang, Daniel S Park, Tao Wang, Timo I Denk, Andy Ly, Nanxin Chen, Zhengdong Zhang, Zhishuai Zhang, Jiahui Yu, Christian Frank, et al. Noise2music: Text-conniitooee music generation with diffusion models. arXio preprint Xiw:2302.00911, 2023b.
Shanshan Huang, Yuanhao Wang, Zhili Gong, Jun Liao, Shu Wang, and Li Liu. Controllable image generation based on causal representation learning. Frontiers Inf. Technol. Electron. Eng., 25(1):1335 148, January 2024b. URL htt..//ppp///ooo..rr/......////. 230030
31

---

Published in Transactions on Machine Learning Research (08/2025)
Eyke Hüillereeirr and Willem Waegeman. Alcatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods. Machine Learning, 110(3):457 506, Mar 2021. ISSN 1573-0565. doi: 10.10077/1109944211--99466
URL /ttp://////////////////////////0
Erik Härkönen, Aaron Hertzmann, Jaakko Lehtinen, and Sylvain Paris. Ganspace: Discovering interpretable gan controls. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan- Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NewrIPS 2020, December 6-12, 2020, virtual, 2020. URL httpss// wwttee-bbsttrtttttaaea wwtp:enwsss....ppeeeeeeeeeeeeeee http://////ee///////eeeeee111100
Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Alcksander Madry. Adversarial examples are not bugs, they are features. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' d'Alché- Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
Sarthak Jain and Byron C. Wallace. Attention is not Explanation. In Jill Burstein, Christy Doran, and Thamar Solorio (cds.), Proceedings of the 2019 Conference of the North American Chapter of the Asso- cialion for Compulalional Linguislics: Human Language Technologies, Volume 1 (Long and Short Pa- pers), pp. 3543 3556, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10.18653/ /11/6653////// 1 111/9919115555 1357. URL
Sepehr Janghorbani and Gerard De Melo. Multi-modal bias: Introducing a framework for stereotypical bias assessment beyond gender and race in languae models. In Andreas Vlachos and Isabelle Augenstein (eds.), Proceedings of the 17th Conference of the European Chapler of the Association for Compulational Linguistics, pp. 1725 1735, Dubrovnik, Croatia, May 2023. Association for Computational Linguistics.
Sadeep Jayasumana, Srikumar Ramalingam, Andreas Veit, Daniel Glasner, Ayan Chakrabarti, and Sanjiv Kumar. Rethinking fid: Towards a better evaluation metric for image generation. In CVPR, pp. 9307 9315, 2024. URL https:://o1. /ttp:::///oioor//1/// (1w19//CVP52233.2202 2024
Metod Jazbec, Malt Ashman, Vincent Fortuin, Michael Pearce, Stephan Mandt, and Gunnar Rätsch. Scal- able Gaussian process variational autoencoders. In International Conference on Artificial Intelligence and Statistics, pp. 3511-3519. PMLR, 2021.
Neil Jethani, Mukund Sudarshan, Yindalon Aphinyanaphongs, and Rajesh Ranganath. Have we learned to explain?: How interpretability methods can learn to encode predictions in their interpretations. In International Conference on Artificial Intelligence and Statistics, pp. 1459-1467. PMIR, 2021.
Dongfu Jiang, Yishan Li, Ge Zhang, Wenhao Huang, Bill Yuchen Lin, and Wenhu Chen. TIGERScore: Towards building explainable metric for all text generation tasks. Transactions on Machine Learning Research, 2024a. ISSN 2835-8856. URL https:://openrevier http://perrvviee..eeeee/ofrrrr// mett/foroummTTdddECCCCCCS
Jiming Jiang and Thuan Nguyen. Linear and generalized linear mixed models and their applications, volume 1. Springer, 2007.
Zhengyuan Jiang, Jinghuai Zhang, and Neil Zhenqiang Gong. Evading watermark based detection of ai- generated content. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communica- tions Security, CCS '23, pp. 1168-1181, New York, NY, USA, 2023. Association for Computing Machin- URL htti::://doi.orgg// 114///000114//5799155
ery. ISBN 9798400700507. doi: 1411453557611536222 3623189
Zhengyuan Jiang, Moyang Guo, Yuepeng Hu, Jinyuan Jia, and Neil Zhenqiang Gong. Certifiably robust image watermark. In ECCV (77), pp. 427 443, 2024b. URL 0ttp://000...g/0..11000////
978-3-031-72980-5 25
Haibo Jin, Leyang Hu, Xinuo Li, Peiyan Zhang, Chonghan Chen, Jun Zhuang, and Haohan Wang. Jail- breakzoo: Survey, landscapes, and horizons in jailbreaking large language and lanion-language models, 2024. URL CtR./tttss:/ http///axiv.oo/aa/////000/
32

---

Published in Transactions on Machine Learning Research (08/2025)
Marco Jiralerspong, Jocy Bose, Ian Gemp, Chongli Qin, Yoram Bachrach, and Gauthier Gidel. Feature likelihood score: Evaluating the generalization of generative models using samples. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL ittps://openrevierr hwtppppwwwiiweeeeeeeeeeeeeeeeeee wwt.forunfff
12VKZko1T7
Fabian Jirasck and Hans Hassc. Combining machine learning with physical knowledge in thermody- namic modeling of fluid mixtures. Annual Review of Chemical and Biomolecular Engineering, 14 (1):31-51, 2023. doi: 10.1146 10.1146/annurev-
URL https://doi. oto/10..g111..0 org/11.1146/
annurev-chember www.ebboong-00220000---
PMID: 36944250.
Fabian Jirasek, Robert Bamler, Sophie Fellenz, Michael Bortz, Marius Kloft, Stephan Mandt, and Hans Hasse. Making thermodynamic models of mixtures predictive by machine learning; matrix completion of pair interactions. Chemical Science, 13(17):4854-490) 2022.
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Zidek, Anna Potapenko, Alex Bridgland, Clemens Meyer, Simon A. A. Kohl, Andrew J. Ballard, Andrew Cowie, Bernardino Romera-Paredes, Stanislav Nikolov, Rishub Jain, Jonas Adler, Trevor Back, Stig Petersen, David Reiman, Ellen Clancy, Michal Zielinski, Martin Steinegger, Michalina Pacholska, Tamas Berghammer,, Sebastian Bodenstein, David Silver, Oriol Vinyals, Andrew W. Senior, Koray Kavukcuoglu, Pushmoet Kohli, and Demis Hassabis. Highly accurate protein structure prediction with alphafold. Nature, 596((873):583 589, July 2021. ISSN 1476-4687. doi: 10.1038/ 041586-001-038199 2. URL http://dxx org..orrg/00 10. 1000/0011000000000 841586--211111 021-03819-2
Saurav Kadavath, Tom Conerly, Amanda Askell, T. J. Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zachary Dodds, Nova DasSarma, Eli Tran-Johnson, Scott Johnston, Sheer Fl-Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav Fort, Deep Gan- guli, Danny Hernandez, Josh Jacobson, John Kernion, Shauna Kravee, Liane Lovitt, Kamal Ndlusse, Catherine Olsson, Sam Ringer, Dario Amodei, Tom B. Brown, Jack Clark, Nicholas Joseph, Benjamin Mann, Sam McCandlish, Christopher Olah, and Jared Kaplan. Language models (mostly) know what they know. ArXiv, abs/2207 220/222000522,, 2022. URL httpps//aapp wwwanti wtw.aaa......g/rrpsssss......... org/CorpusID :5045111251011111
Adam Tauman Kalai and Santosh S Vempala. Calibrated language models must hallucinate. ar preprint arXiv:231.116448, 2023.
Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. Large language models struggle to learn long-tail knowledge. In Andreas Kranse, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 15696-15707. PMLR, 23 29 Jul 2023.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. CoRR, abs/2001.0882 000001,,8361.....1.0 2020. URL https: httpss://axxvvvr/////e www//bs//////////eee/..//eeeee// wttpa////...0000000 08361
Tero Karras, Mika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- generative models. Advances in Neural Information Processing Systems, 35:26565-2657, 2022.
Tero Karras, Milka Aittala, Jaakko Lchtinen, Janne Hellsten, Timo Aila, and Samuli Laine. Analyzing and improving the training dynamics of diffusion models. arXie preprinl ar XXix:2312.02696 2023.
Harcem Kibriya, Wazir Zada Khan, Ayesha Siddiqa, and Muhammad Khurram Khan. Privacy issues in large language models: A survey, Computers and Electrical Engineering, 120:109698, 2024. ISSN 0045- 7906. doi: https///dbb.ogg/10000///cooooooo
2024410999.22 10240000 URL httpp:://www.scieeeddreetttttttt
Doyoung Kim, Suwoong Heo, Jiwoo Kang, Hogab Kang, and Sanghoon Loc. photo identification framework prevent copyright infringement with manipulations. Applied Sciences, 1191)), 2021. ISSN 2076-3417. doi: 10.3390/app1119991 URI.
33

---

Published in Transactions on Machine Learning Research (08/2025)
Siwon Kim, Sangdoo Yun, Hwaran Lee, Martin Gubri, Sungroh Yoon, and Scong Joon Oh. ProPILE: probing privacy leakage in large language models. In Proceedings of the 37th International Conference on Newral Information Processing Systems, NIPS '23, Red Hook, NY, USA, 2023. Curran Associates Inc.
Yeongmin Kim, Byeonghu Na, Minsang Park, JoonHo Jang, Dongjun Kim, Wanmo Kang, and Il chul Moon. Training unbiased diffusion models from biased dataset. In The Twelfth International Conference on Learning Representations, 2024. URL httts::/
John Kirchenbauer, Jonas Gciping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watcrmark for large language models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 17061 17084. PMLR, 23 29 Jul 2023. URL htttps://proceeeeeeee
mlr. www..erresssvee.. wwtp//22////2enaee www.hhhwwwwbbba.........
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Manli Shu, Khalid Saifullah, Kezhi Kong, Kasun Fernando, Aniruddha Saha, Micah Goldblum, and Tom Goldstein. On the reliability of watermarks for large language models. In The Twelfth International Conference on Learning Represenlations, 2024. URL https:// openreview. wwtprevviewne//ffrrruee
Matthias Kirchler, Christoph Lippert, and Marius Kloft. Training normalizing flows from dependent data. In International Conference on Machine Learning, pp. 17105 17121. PMLR, 2023.
Tom Koemi and Christian Federmann. Large language models are state-of-the-art evaluators of transla- tion quality, In Mary Nurminen, Judith Brenner, Maarit Koponen, Sirkku Latomaa, Mikhail Mikhailov, Frederike Schierl, Tharindu Ranasinghe, Eva Vanmassenhove, Sergi Alvarez Vidal, Nora Aranberri, Mara Nunziatini, Carla Parra Escartín, Mikel Forcada, Maja Popovic, Carolina Scarton, and Helena Moniz (eds.), Proceedings of the 24th Annual Conference of the European Association for Machine Transla- tion, pp. 193 203, Tampere, Finland, June 2023. European Association for Machine Translation. URL https://aclanthooogg. org/2023. 2aatt02....mmm1m..mm eamt-1.19/
Katherine Van Koevering and Jon Kleinberg. How random is random? evaluating the randomness and humaness of Ilms' coin flips, 2024. URL https: httpe//////vvo.o///bbbbb
Y. Alex Kolchinski, Sharon Zhou, Shengjia Zhao, Mitchell Gordon, and Stefano Ermon. Approximating human judgment of generated image quality. arXiv preprint arxiv:1900 arxiv. hariv..rgg/abs/1922.111
2019. URL https://
Aneesh Komanduri, Xintao Wu, Yongkai Wu, and Feng Chen. From identifiable causal representations to controllable counterfactual generation: survey on causal generative modeling. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL
Fanjie Kong, Shuai Yuan, Weituo Hao, and Ricardo Henao. Mitigating Lest-time bias for fair im- age retrieval. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL wet.//oofrrntteMMMoooooI
Maria Korshunova, Niles Huang, Stephen Capuzzi, Dmytro S. Radchenko, Olena Savych, Yuriy S. Moroz, Carrow I. Wells, Timothy M. Willson, Alexander Tropsha, and Olexandr Isayev. Generative and reinforce- ment learning approaches for the automated de novo design of bioactive compounds. Communicatiins Chemistry, 5(1):129, October 2022. ISSN 2399-3669.
Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Frederick Wicting, and Mohit Iyyer. Paraphrasing evades detectors of Al-generated text, but retrieval an effective defense. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL httppp///pppaaaoooe....c/////ots
Raghuraman Krishnamoorthi. Quantizing deep convolutional networks for efficient inference: A whitepaper. Ar Arxiv, 088//1806.0842,, 2018.
34

---

Published in Transactions on Machine Learning Research (08/2025)
Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. Semantic uncertainty: Linguistic invariances for uncer- tainty estimation in natural language generation. In The Fleventh International Conference on Learning Representations, 2023. URL
www.:eeettooooe http://ppearreeeeee.te/etrrrrrre AYtPodve
Sachin Kumar, Eric Malmi, Aliaksei Severyn, and Yulia Tsvetkov. Controlled text generation continuous optimization with multiple constraints. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vanghan (eds.), Advances in Neural Information Processing Systems, 2021. URL https://openrewieet net/forum?
id Id=kly7bbm-44
Sachin Kumar, Biswajit Paria, and Yulia Tsvetkov. Gradient-based constrained sampling from language models. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 2251 2277, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10. 10.18653/v1/ (1/653/112222..o//// 0000022mmmlmmmmem..mm.e matp. URL https://aclanthology/ org.orgg/// emnlp-main. main. 144/
Mucahid Kutlu, Tyler McDomnell, Matthew Lease, and Tamer Elsayed. Annotator rationales for labeling tasks in crowdsourcing. Journal of Artificial Intelligence Research, 69:143 189, September 2020. ISSN 1076-9757. doi: 1.611/jair.1.120111 CRL // doi. 0t0.i..0//0...113///.1 1613/ 12012
Tuomas Kynkäänniemi, Tero Karras, Miika Aittala, Timo Aila, and Jaakko Lehtinen. The role of imagenet classes in fréchet inception distance. International Conference on Learning Representations, 2023.
Viet Dac Lai, Nghia Ngo, Amir Pouran Ben Veyseh, Hieu Man, Franck Dernoncourt, Trung Bui, and Thien Huu Nguyen. ChatGPT beyond English: Towards a comprehensive evaluation of large language models in multilingual learning. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Compulational Linguisties: EMNLP 2023, pp. 13171-13189, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653 1 120223 windiiiinnnnnnnnnne emnll.878. URL https: findings-
//aclanthology.org
Hugo Larochelle and Iain Murray. The neural autoregressive distribution estimator. In Gcoffrey Gordon,
David Dunson, and Miroslav Dudik (eds.), Proceedings of the Fourleenth International Conference
Artificial Intelligence and Statistics, volume 15 of Proceedings of Machine Learning Research, pp. 29- 37, Fort Lauderdale, FL, USA, 11-13 Apr 2011. PMLR. URL
larochellella.hmm
Mike Laszkiewicz, Imant Daunhawer, Julia E Vogt, Asja Fischer, and Johannes Lederer. Benchmarking the fairness of image upsampling methods. arXiv preprint 2rXiv:2401.1 18555, 2024.
Pictro Lesci, Clara Meister, Thomas Hofmann, Andreas Vlachos, and Tiago Pimentel. Causal estimation of memorisation profiles. In Lun- Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics ( (Volume 1: Long Papers), pp. 15616 15635, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. 10.18653/v1/ 2024. 2024.acl-long.834 834. URL https:://cclatthat ottp:/oooooooorg/2.2a..a.aa org/ acl- acl-long........... 8844/
Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative de- coding. In International Conference on Muchine Learning, 2022. URL
Chunyuan Li, Zhe Gan, Zhengyuan Yang, Jianwei Yang, Linjie Li, Lijuan Wang, and Jianfeng Gao. Multi- modal foundation models: From specialists to general-purpose assistants. ArXio, abs/2309. 10020, 2023a.
Guihong Li, Duc Hoang, Kartikeya Bhardwaj. Ming Lin, Zhangyang Wang, and Radu Marculescu. Zero-shot neural architecture search: Challenges, solutions, and opportunities. IEFE Trans. Pattern Anal. Mach. Intell,, 46(12):7618-76355 December 2024a. ISSN 0162-8828. doi: 1A1109/TPAM..20244 https://doi.oon org/ 11199/0111119PP/M111111 TT2.T..2.............. 3395443
URL
Guihong Li, Hsiang Hsu, Chun-Fu Chen, and Radu Marculescu. Machine unlcarning for imago-to-image generative models. CoRR, abs/ hta//242.00000 00051,, 2024b. doi: 10.48550/ARXII 240200131.. URL https:
wtt.://///....5///rrra.... 2402. 2402.00351
35

---

Published in Transactions on Machine Learning Research (08/2025)
Haodong Li, Gelei Deng, Yi Liu, Kailong Wang, Yuekang Li, Tianwei Zhang, Yang Liu, Guoai Xu, Guosheng Xu, and Haoyu Wang. Digger: Detecting copyright content mis-usage in large language model training. ArXiu, abs/ 000/2401.00666,, 2024c. htt.ltrppe//p....eennocooooooooo hwtpaarrrrrrrrrrrpua//26666683
semanticscholar
Haoran Li, Dadi Guo, Doughao Li, Wei Fan, Qi Hu, Xin Liu, Chunkit Chan, Duanyi Yao, Yuan Yao, and Yangqiu Song. PrivLM-bench: A multi-level privacy evaluation benchmark for language models. In Lun-Wer Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Lènguistics (Volume 1: Long Papers), pp. 54-73, Bangkok, Thailand, URL https:
August 2024d. Association for Computational Linguisties. doi: 1116333/112224.ccllblo //aclanthology..ook wrg.
Junlong Li, Shichao Sun, Weizhe Yuan, Run-Ze Fan, hai thao, and Pengfei Liu. Generative judge for evaluating alignment. In The Twelfth International Conference on Learning Representations, 2024e. URL
httzps://ppnreeviou net/ferun?meee
Weichen Li, Rati Devidze, and Sophie Fellenz. Learning to play text-based adventure games with maximum entropy reinforcement learning. In Machine Learning and Knowledge Discovery in Databases (ECML PKDD), 2023b.
Weichen Li, Rati Devidze, Waleed Mustafa, and Sophie Fellenz. Ethics in action: Training reinforcement learning agent for moral decision-making in text-based adventure games. In Proceedings of International Conference on Artificial Intelligence and Statistics (A (AISTATS), 2024f.
Xiang Lisa Li, John Thickstun, Ishaan Guirajani, Percy Liang and Tatsunori Hashimoto. Diffusion-LM improves controllable text generation. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022a.
Xinzhe Li, Ming Liu, Shang Gao, and Wray Buntine. A survey on out-of-distribution evaluation of neural nlp models. In Edith Elkind (ed.), Proceedings of the Thirly-Second International Joint Conference on Artificial Intelligence, IJCAI-23, pp. 6683 6691. International Joint Conferences on Artificial Intelligence Organization, 8 2023c. doi: 10. 3300633/jjai222223 /749. Survey Track.
2023/7449
Xiuyu Li, Yijiang Liu, Long Lian, Huanrui Yang, Zhen Dong, Daniel Kang, Shanghang Zhang, and Kurt Keutzer. Q-diffusion: Quantizing diffusion models. In Proceedings of the IEEE/CVF International Con- ference on Compuler Vision, pp. 17535 17545, 2023d.
Xuechen Li, Florian Tramer, Percy Liang, and Tatsunori Hashimoto. Large language models can be strong differentially private learners. In International Conference on Learning Representations, 2021.
Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Fe- cles, James Koeling, Felix Gimeno, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Mas- son d' d'Autume, Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov, James Mollay, Daniel J. Mankowitz, Esme Sutherland Robson, Pushmeet Kohli, Nando de Freitas, Koray Kavukcuoglu, and Oriol Vinyals. Competition-lovel code generation with alphacode. Science, 376(642)):1092 1097, 2022b.
Zhen Li, Xiaohan Xu, Tao Shen, Can Xu, Ca-C Gu, Yuxuan Lai, Chongyang Tao, and Shuai Ma. Leveraging large language models for NI. evaluation: Advances and challenges. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empérical Methods in Natural Language Processing, pp. 16028 16045, Miami, Florida, USA, November 2024g. Association for Computational Linguistics. doi: 10. 10.118/3/v1/200 http:/22222mmm//. maulp-mmin..800 896. URL https:://claatthllog// org/
2024. www.eennalma.e.. mwt.p.am/nn./.
Zongyi Li, Daniel Zhengyu Huang, Burigede Liu, and Anima Anandkumar. Fourier neural operator with learned deformations for pdes on general geometries. J. Mach. Learn. Res., 24(1), Jamuary 2023c. ISSN 1532-4435.
36

---

Published in Transactions on Machine Learning Research (08/2025)
Chumeng Liang, Xiaoyu Wu, Yang Hua, Jiaru Zhang, Yiming Xue, Tao Song, Zhengui Xue, Ruhui Ma, and Haibing Guan. Adversarial example does good: Preventing painting imitation from diffusion models via adversarial examples. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (ods.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 20763 20786. PMLR, 23 29 Jul 2023.
Q Vera Liao, Daniel Gruen, and Sarah Miller. Questioning the ai: informing design practices for explainable ai user experiences. In Proceedings of the 2020 CHI conference on human factors in computing systems, pp. 1-15, 2020.
Chin- Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pp. 74-81, 2004.
Ji Lin, Jiaming Tang, Haotian Tang. Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. Awq: Activation-aware weight quantization for on-device Ilm compression and acceleration. In P. Gibbons, G. Pekhimenko, and C. De Sa (eds.), Proceedings of Machine Learning and Systems, volume 6, pp. 87 100, 2024a. URL https://proceeddii mly......mm.....m..mmmmmmmm..m.. aalsys.co . paper paper_files/ files/ www.ttewweeeeeeeeeeeeeaaaeeeeeee wwwaper/2224/// /atp:///2444aaaaaeeaee htt.://caa..aaeeeeeeaaaaaaeeeeee Paper Coceerr-oooffeeeeee pdf
Stephanie Lin, Jacob Hilton, and Owain Evans. Teaching models to express their uncertainty in words. Transactions on Machine Learning Research, 2022. ISSN 2835-8856. URL httpps://pperrevier.
forum?1d=8s8K217G177
Zinan Lin, Sivakanth Gopi, Janardhan Kulkarni, Harsha Nori, and Sergey Yekhanin. Differentially private synthetic data via foundation model APIs 1: Images. In The Twelfth International Conference on Learning Representations, 2024b. URL
Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling, 2023. URL http:::///aaxvv../// abs/ 02747
Aiwei Liu, Leyi Pan, Yijian Lu, Jingjing Li, Xuming Hu, Xi Zhang, Lijie Wen, Irwin King, Hui Xiong, and Philip Yu. A survey of text watermarking in the era of large language models. ACM Comput. Surv., (2), November 2024a. ISSN 0360-0300. doi: 10.1145/369)
URL https://doii oropii..rg/1... 10///.0.110 1145//666666
Haohe Liu, Zehua Chen, Yi Yuan, Xinhao Mei, Xubo Liu, Danilo Mandic, Wenwu Wang, and Mark D Plumbley. AudioLDM: Text-lo-audio generation with latent diffusion models. In Andreas Krause, Emna Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 21450 21474. PMLR, 23 29 Jul 2023a. URL liu23f. html
mlr, press//222/
Luping Liu, Yi Ren, Zhijic Lin, and Zhou Zhao. Pscudo numerical methods for diffusion models on manifolds. In International Conference on Learning Representations, 2022a.
Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalie Baracaldo, Peter Hase, Yuguang Yao, Chris Yuhao Liu, Xiaojun Xu, Hang Li, Kush R. Varshney, Mohit Bansal, Sanmi Koyejo, and Yang Liu. Rethinking machine unlcarning for large language models. Nature Machine Intelligence, 7(2):181- 194, Feb 2025. ISSN 2522-5839. doi: 10. 0t/5:/442226-02--090555 URL https://doi. http///oi..r///1.10000.03
842256--02-00988--0
Xiaodong Liu, Hao Cheng, Pengcheng He, Weizhu Chen, Yu Wang, Hoifung Poon, and Jianfeng Gao. Adversarial training for large neural language models. ar preprint or orXim:2004.08094 2020.
Yibing Liu, Haoliang Li, Yangyang Guo, Chenqi Kong, Jing Li, and Shiqi Wang. Rethinking attention-model explainability through faithfulness violation test. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szopesvari, Gang Niu, and Sivan Sabato (cds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 13807-13824. PMLR, 17-23 Jul 2022b. URL httpps//proccediing.. mlr. press//162//iu22: html
37

---

Published in Transactions on Machine Learning Research (08/2025)
Zechun Liu, Barlas Oguz, Changsheng Zhao, Ernie Chang, Pierre Stock, Yashar Mehdad, Yangyang Shi, Raghuraman Krishnamoorthi, and Vikas Chandra. Llm-qat: Data-free quantization aware training for large language models. arxiv preprint arXiv:2305.1788 2023b.
Zheyuan Liu, Guangyao Dou, Zhaoxuan Tan, Yijun Tian, and Meng Jiang. Machine unlearning in generative ai: A survey. CoRR, 20000//44072201111 2024b. URL http://dblp... uni umi-triir.deddb// www.d/////wwaa/////eeeooo /oorr Sorr2407.htmlHabse htw.hhtaamaaaooitmmmee 240-722202220
Francesco Locatello, Ben Poole, Gunnar Rätsch, Bernhard Schölkopf, Olivier Bachem, and Michael Tschan- nen. Weakly-supervised disentanglement without compromises. In International Conference on Machine PMLR, 2020.
Learning, pp.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems,
2022.
Pan Lu, Baolin Peng, Hao Cheng, Michel Galley, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, and Jianfeng Gao. Chameleon: Plug-and-play compositional reasoning with large lan- guage models. In A. Oh, T. Naumann, A. Globerson, K. Sacnko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 43447 43478. Curran Associates, Inc., 2023. URL https::/ http:///pppooedddnnne.oo......p. neurips hewrips.ccpppppppperrree wwtppppee.illee/ppppepeeeeeeeeee ftlee//ppprr/222///ie BT1el056774a1cctee6666ee22222222
Conferenee
Scolt M. Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NTPS'17, pp. 4768 4777, Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 9781510560964.
Guanxiong Luo, Moritz Blumenthal, Martin Heide, and Martin Uecker. Bayesian mri reconstruction with joint uncertainty cstimation using diffusion models. Magnetic Resonance in Medicine, 90(1):295-311, 2023. doi: holps://doi.orgg / 10. 10.1002/ 10010000.2220/// 29624. m9224 URL https:///omliineiibbaayy wttp:///eee///////////////0.0..0
mrm. 29624
Sarah J MacEachern and Nils D Forkert. Machine learning for precision medicine. Genome, 64(4):416 425, 2021.
Andrey Malinin and Mark Gales. Predictive uncertainty estimation via prior networks. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchii, and R. Garnett (eds.), Advances in Newral Information Processing Systems, volume 31. Curran Associates, Inc., 2018 URL httpp://prooeeednggg oatta///eeeeebeeeeeeeeeeeeeeeeea
.pff
Laura Manduchi, Matthias Hüser, Martin Faltys, Julia Vogt, Gunnar Rätsch, and Vincent. Fortuin. T- DPSOM: An interpretable clustering method for unsupervised lcarning of patient health states. In Pro- ceedings of the Conference on Health, Inference, and Learning, pp. 236 245, 2021.
Laura Manduchi, Ričards Marcinkevičs, Michela C. Massi, Thomas Weikert, Alexander Sauter, Verena Gotta, Timothy Müller, Flavio Vasella, Marian C. Neidert, Marc Pfister, Bram Stieltjes, and Julia E Vogt. A deep variational approach to clustering survival data. In International Conference on Learning Representations, 2022. URL httpss://ppenrevieet www.:ewww.eeee.eeeeeeeeeeeee net/forumm 1d0002222-1000222220000
Laura Manduchi, Moritz Vandenhirlz, Alain Ryser, and Julia E Vogt. Tree variational autoencoders. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https: het//forumiddraqqootte
Ricards Marcinkevics and Julia E. Vogt. Interpretability and explainability: A machine learning 200 mini- tour. ArXio, abs/2012.01105, 2020. URL
org/CorpusID httpr//aaaccoooooooooororroaaaaa 227254760
R. Thomas McCoy, Robert Frank, and Tal Linzen. Does syntax need to grow on trees? sources of hi- crarchical inductive bias in sequence-to-scquenee networks. Transactions of the Association for Com-
putational Lönguistics, 8:125-14h 140, 01 2020. ISSN 2307-387X. doi: 10.1162/tacl a 00304. URL
https://dbi.org//// wttp:///.....////..... 00304
38

---

Published in Transactions on Machine Learning Research (08/2025)
Tom McCoy, Ellie Pavlick, and Tal Linzen. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In Anna Korhonen, David Traum, and Lluís Märquez (eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 3428 3448, Florence, Italy, July 2019. Association for Computational Linguistics.
Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14297-14306, 2023.
Bertalan Meskó and Eric J. Topol. The imperative for regulatory oversight of large language models (or generative ai) in healthcare. NPJ Digital Medicine, 6, 2023.
Graziano Mita, Maurizio Filipponc, and Pictro Michiardi. An identifiable double vac for disentangled rep- resentations. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 18-24 Jul 2021. URL httts::/ppocceddiggg mlr www.mmrr...ss.e.. htt...ppress//13mmmma22222
PMLR,
Tom M Mitchell. The need for biases in learning generalizations (rutgers computer science tech. rept. cbm- tr-117). Rulgers Universily, 1980.
Sarthak Mittal, Korbinian Abstreiter, Stefan Bauer, Bernhard Schölkopf, and Arash Mchrjon. Diffusion based representation learning. In International Conference on Machine Learning, 2021.
Arvind Mohan, Nicholas Lubbers, Misha Chertkov, and Daniel Livescu. Embedding hard physical constraints in neural network coarso-graining of three-dimensional turbulence. Physical Review Fluids, 8, 01 2023. doi: 10.1103 10.1103/PhysSeee Beee//hhyRPeluuiiii sodids.8.01400 8.014604.
Francesco Montagna, Atalanti Mastakouri, Elias Eulig, Nicoletta Noceti, Lorenzo Rosasco, Dominik Janzing, Bryon Aragam, and Francesco Locatello. Assumption violations in causal discovery and the robustness of score matching. Advances in Neural Information Processing Systems, 36, 2024.
Miguel Monteiro, Fabio De Sousa Ribeiro, Nick Pawlowski, Daniel C. Castro, and Ben Glocker, Measur- ing axiomatic soundness of counterfactual image models. In The Fleventh International Conference on Learning Representations, 2023.
Michael Moor, Oishi Banerjee, Zahra Shakeri Hossein Abad, Harlan M. Krumholz, Jure Leskovec, Eric Topol, and Pranav Rajpurkar. Foundation models for generalist medical artificial intelligence. Nature 616, 259 265, 2023a.
Michael Moor, Qian Huang, Shirley Wu, Michihiro Yasunaga, Yash Dalmia, Jure Leskovec, Cyril Zakka, Eduardo Pontes Reis, and Pranav Rajpurkar. Med-flamingo- a multimodal medical few-shot learner. In Stefan Hegselmann, Antonio Parziale, Divya Shanmugam, Shengpu Tang, Mercy Nyamewaa Asiedu, Serina Chang, Tom Hartvigsen, and Harvineet Singh (eds.), Proceedings of the 3rd Machine Learning for Health Symposium, volume 225 of Proceedings of Machine Learning Research, pp. 353-367 PMLR, 10 Dec 2023b. URL https://procceedingg mlr, ppess/v225 http:///222mmmoo223... mo.r23a. html
Ted Moskovitz, Aaditya K Singh, DJ Strouse, Tuomas Sandholm, Ruslan Salakhutdinov, Anca Dragan, and Stephen Marcus McAleer. Confronting reward model overoptimization with constrained RLHF In The Twelfth International Conference on Learning Representations, 2024. URL forum? Porguiid=gKfUUnOOOL
net/
Muhammad Ferjad Naeem, Seong Joon Oh, Youngjung Uh, Yunjey Choi, and Jaejun Yoo. Reliable fidelity and diversity metrics for generative models. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Re- search, pp. 7176-7185. PMLR, 13-18 Jul 2020. URL httphtts//prcccedinggg mlr wtw..press//11//aaaom//// html
39

---

Published in Transactions on Machine Learning Research (08/2025)
Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A. Feder Cooper, Daphne Ippolito, Christopher A. Choquetie-Choo, Eric Wallace, Florian Tramer, and Katherine Lee. Scalable extraction of training data from (production) language models. arXiv preprint arzio:2311.17035 2023.
C. Lopez Olmos, A. Ncophytou, S. Sengupta, and D. P. Papadopoulos. Latent directions: A simple pathway bias mitigation in generative ai. Proceedings of the CVPR Conference at ReGenAl: First Workshop on Responsible Generalive AT, February 2024.
OpenAl. Gpt-4 technical report. ArXiu, abs/2333.000 230..00774 2023.
Ian Osband, Zheng Wen, Seyed Mohammad Asghari, Vikranth Dwaracherla, Morteza Ibrahimi, Xiuyuan Lu, and Benjamin Van Roy. Epistemic neural networks. In Therth- Conference on Newral Information Processing Systems, 2023. URL https:// www.:nooooornddddeeeeeeo
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe. Training language models follow instructions with human feedback. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 27730 27744. Curran Associates, Inc., 2022.
Tiago P. Pagano, Ralael B. Loureiro, Fernanda V. N. Lisboa, Rodrigo M. Peixoto, Guilherme A. S. Guimarães, Gustavo O. R. Cruz, Maira M. Araujo, Lucas L. Santos, Marco A. S. Cruz, Ewerton L. S. Oliveira, Ingrid Winkler, and Erick G. S. Nascimento. Bias and unfairness in machine learning mod- els: A systematic review on datasets, tools, fairness metrics, and identification and mitigation meth- ods. Big Data and Cognitive Computing, 7(1), 2023. ISSN 2504-2289. doi: 1..3390/bccc7010 httpp://wwi http://ww.mppi.ccoom htt.pp..coocm/442222///////
URL
Mahdi Pakdaman Nacini, Gregory Cooper, and Milos Hauskrocht. Obtaining well calibrated probabilities using bayesian binning. Proceedings of the AAAI Conference Artificial Intelligence, 29(1), Feb. 2015.
doi: 10.1609/aaai, http:/aaii.vi1....... v2911.9602. URL attps://oj] http://sssaaaaaaaaaaaaaaaaaa..aa org/index. phtp:/aA//arrtiil////eeee///e/
Jeff Z. Pan, Simon Razniewski, Jan-Christoph Kalo, Sneha Singhania, Jiaoyan Chen, Stefan Dietze, Hajira Jabeen, Janna Omeliyanenko, Wen Zhang, Matteo Lissandrini, Russa Biswas, Gerard de Melo, Angela Bonifati, Edlira Vakaj, Mauro Dragoni, and Damien Graux. Large Language Models and Knowledge Graphs: Opportunities and Challenges. Thansactions on Graph Data and Knowledge, 1(1):2:1-22) 2:38, 2023. doi: (0.423//TGDK.LJ. URL https://áropa: dagstubl de/ /ttee/ttts/ttcueeetluuieeeeeeeee http://cmmmmmmt/0000200000/0 4230/TCDK. 1.
Kushagra Pandey and Stephan Mandt. A complete recipe for diffusion generative models. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp.
2023.
Kushagra Pandey, Avideep Mukherjee, Piyush Rai, and Abhishek Kumar. Diffuse Efficient, controllable and
generation from low-dimensional latents. Transactions on Machine Leurning Research, 2022. ISSN 2835-8856.
K Kushagra Pandey, Maja Rudolph, and Stephan Mandt. Efficient. integrators for diffusion generative models. In The Twelfth International Conference on Learning Representations, 2024. URL https:://ppeerrevee
Kushagra Pandey, Jaideep Pathak, Yilun Xu, Stephan Mandt, Michael Pritchard, Arash Vahdat, and Morteza Mardani. Heavy-tailed diffusion models. In The Thirleenth International Conference on Learning Representations, 2025. URL http::/pppnneeeveeeeeettttooaanp
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 311-318, 2002.
40

---

Published in Transactions on Machine Learning Research (08/2025)
Gaurav Parmar, Richard Zhang, and Jun- Yan Zhu. On aliased resizing and surprising subtleties in gan evaluation. In Proceedings of the IEEE/CVE Conference on Computer Vision and Paltern Recognation, pp. 11410 11420, 2022.
David A. Patterson, Joseph Gonzalez, Urs Holzle, Quoe V. Le, Chen Liang, Lhuís-Miquel Munguía, Daniel Rothchild, David R. So, Mand Texier, and Jeff Dean. The carbon footprint of machine learning training will plateau, then shrink. Compuler, 55:18 28, 2022.
Dario Pavllo, Aurelien Lucchi, and Thomas Hofmann. Controlling style and semantics in weakly- supervised image generation. In Computer Vision ECCV 2020: 16th European Conference, Glas- gou; UK, August 23-28, 2020, Proceedings, Part VI, pp. 482-499, Berlin, Heidelberg, 2020. Springer- Verlag. ISBN 978-3-030-58838-- doi:
29.
/tt////i...rg/00110007///
978-3-080-68539-6 29
Nick Pawlowski, Daniel Coelho de Castro, and Ben Glocker. Deep structural causal models for tractable counterfactual inference. Advances in Neural Information Processing Systems, 33:857-869, 2020.
Judea Pearl. The seven tools causal inference, with reflections on machine learning. Commun. ACM, 62 (3):54-60, feb 2019. ISSN 0001-0782.
Judea Pearl and Dana Mackenzie. The Book of Why: The New Science of Cause and Effeet.. Basic Books, Inc., USA, 1st edition, 2018. ISBN 046509760X.
William Peobles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 4195-4205, 2023.
Aleksandar Petrov, Emanuele La Malfa, Philip Torr, and Adel Bibi. Language model tokenizers intro- duce unfairness between languages. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 36963 36990. Cur- ran Associates, Inc., 2023. URL https://proceed neurrnig.n.n.nnn..ooeee.mrrrr.is mwwr.ps.cc.papprrr httppr/ppp.cppppp....efe////pppp fillss/paper22222/11ee 7b5224ce8SAAae222834444eeeearrr0
Krishna Pillutla, Lang Liu, John Thickstun, Scan Welleck, Swabha Swayamdipta, Rowan Zellers, Scwoong Oh, Yejin Choi, and Zaid Harchaoui. MAUVE scores for generative models: theory and practice. Journal of Machine Learning Research, 24(1), March 2024. ISSN 1532-4435
Tiago Pimentel, Clara Meister, and Ryan Cotterell. On the usefulness of embeddings, clusters and strings for text generation evaluation. In The Eleventh International Conference on Learning Representations, 2023. URL httpps://ppenrrvveer
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for high-resolution image synthesis. In The Twelfth International Conference on Learning Representations, 2024. URL /httt::// htttpe://operrevie..o
Gabriel Poesia, Alex Polozov, Vu Le, Ashish Tiwari, Gustavo Soares, Christopher Meek, and Sumit Gulwani. Synchromesh: Reliable code generation from pre-trained language modols. In International Conference on Learning Representations, 2022. URL httppp///geeaaaeeeeeeeeerooooeer htt.://orrumm..dd.m////
Forough Poursabxi-Sanglee, Daniel G Goldstein, Jake M Hofman, Jennifer Wortman Wortman Vaughan, and Hanna Wallach. Manipulating and measuring model interpretability. In Proceedings of the 2021 CHI conference on human factors in computing systems, pp. 1-52, 2021.
Omid Poursaced, Tianxing Jiang, Harry Yang, Serge J. Belongie, and Ser-Nam Lim. Robustness and gen- eralization via generative adversarial training. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 15691-15700, 2021.
Aahlad Manas Puli, Lily Zhang, Yoav Wald, and Rajesh Ranganath. Don't blame dataset shift! shorteut learning due gradients and cross entropy. Advances in Newral Information Processing Systems, 36, 2023.
41

---

Published in Transactions on Machine Learning Research (08/2025)
Libo Qin, Qiguang Chen, Yuhang Zhou, Zhi Chen, Yinghui Li, Lizi Liao, Min Li, Wanxiang Che, and Philip S. Yu. A survey of multilingual large language models. Palterns, 6(1):101118, 2025. ISSN
2666-3899. doi: http:///loiorr//11100//...p sctanc//arrrcce//////66663222000
http://0....00.1ptttt..222.....0 2024.101118. URL http:///www.cieeeedrre...oo.
Alec Radford, Jeffrey Wu; Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language
models are unsupervised multitask learners, 2019. URL https://ddmucffpsssw..cck wlo.:ywwwww.oodoorrt.ooo....m... net/
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramosh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Πlaa Sutskever. Learning transferable visual models from natural language supervision. CoRR, abs/2103.0000 2021. URL https:
//arxiv.org/abs/2222 00020
Jack W Rac, Scbastian Borgeaud, Trevor Cai, Katic Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, et al. Scaling language models: Methods, analysis & insights from training gopher. arXiv preprint arXiv:2112. 11446, 2021.
Aniruddh Raghu, Payal Chandak, Ridwan Alam, John Guttag, and Collin Stultz. Contrastive pre-training for multinodal medical time series. In NewrIPS 2022 Workshop on Learning from Time Series for Health, 2022. URL htttps://ppenrevieee www.:neetoommidddn-n-no.m.
Wullianallur Raghupathi and Viju Raghupathi. Big data analytics in healthcare: promise and potential. Health Information Science and Systems, 2, 2014.
Maziar Raissi, Paris Perdikaris, and George Karniadakis. Physics-informed neural networks: deep learn- ing framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686-77 707, 2019.
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and llya Sutskever. Zero-shot text-to-image generation. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learn- ing Research, pp. ramesh21a.htm
mlr. press/v139/ PMLR, 18-24 Jul 2021. URL httpp::/ /
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint 2rX4v:2204.06195 2022.
Xuming Ran, Mingkun Xu, Lingrui Mei, Qi Xu, and Quanying Liu. Detecting out-of-distribution samples via variational auto-encoder with reliable uncertainty estimation. Newral Networks, 145:199 208, 2022. ISSN 0893-6080. doi: http::///d0o..rg/10000///...o. ott/://choooo/rrraaaa//////66010
URL https://www.stt httpp::/www..cceeeedreeee.
Md Shohel Rana, Mohammad Nur Nobi, Beddhu Murali, and Andrew H. Sung. Deepfake detection: A systematic literature review. IEEE Access, 10:25494 25513, 2022. doi: TOD///AACCSSSS222..1555555
Suman Ravuri, Karel Lenc, Matthow Willson, Dmitry Kangin, Remi Lam, Piotr Mirowski, Megan Fitzsi- mons, Maria Athanassiadou, Sheleem Kashem, Sam Madge, Rachel Prudden, Amol Mandhane, Aidan Clark, Andrew Brock, Karen Simonyan, Raia Hadsell, Niall Robinson, Ellen Clancy, Alberto Arribas, and Shakir Mohamed. Skilful precipitation nowcasting using deep generative models of radar. Na- ture, 597(7878) September 2021. ISSN 1476-4687. doi: 1410088/411888--21--38888 http://dx. doi. wtt.el/gg//....11.111 100.10080/44188-021--00000
URL
Partha Pratim Ray. Benchmarking, ethical alignment, and evaluation framework for conversational ai: Advancing responsible development of chatgpt. BenchCouncil Transactions on Benchmarks, Standards and Evaluations, 3(3): 2023. ISSN 2772-4859. doi: https://doi.orr/100.0010 htt.://......been...220.0.......
42

---

Published in Transactions on Machine Learning Research (08/2025)
Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gomez Colmenarejo, Alexander Novikov, Gabriel Barth- Maron, Mai Gimenez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, Tom Eccles, Jake Bruce, Ali Razavi, Ashley D. Edwards, Nicolas Manfred Otto Heess, Yutian Chen, Raia Hadsell, Oriol Vinyals, Mahyar Bordbar, and Nando de Freitas. A generalist agent. Trans. Mach. Learn. Res., 2022, 2022.
Lyle Regenwetter, Giorgio Giannone, Akash Srivastava, Dan Gutfreund, and Facz Ahmed. Constraining generalive models for engineering design with negalive data. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL Inttps:/ net/formm?id-FWBB2reee
Ehud Reiter. A structured review of the validity of BLEU. Computational Linguistics, 44(3):393 401, September 2018. doi: 10.1162/coli a c0322___0_022... URL http:://aclaattboog..oo///1-1
Patrik Reizinger, Yash Sharma, Matthias Bethge, Bernhard Schölkopf, Ferenc Huszár, and Wieland Brendel. Jacobian-basoo causal discovery with nonlincar ICA. Transactions on Machine Learning Research, 2023.
ISSN 2835-8856.
Jie Ren, Jiaming Luo, Yao Zhao, Kundan Krishna, Mohammad Saleh, Balaji Lakshminarayanan and Peter J Liu. Outto--distribution detection and selective generation for conditional language models. In The Eleventh International Conference on Learning Representations, 2023. URL https::/openreviier net/
Zekun Ren, Felipe Oviedo, Maung Thway, Siyu I. P. Tian, Yue Wang, Hansong Xue, Jose Dario Perea, Mariya Layurova, Thomas Heumueller, Erik Birgersson, Armin G. Aberle, Christoph J. Brabec, Rolf Stangl, Qianxiao Li, Shijing Sun, Fen Lin, Ian Marius Peters, and Tonio Buonassisi. Embedding physics domain knowledge into baycsian network enables layer-byllayer process innovation for photovoltaics. vpj Compulational Materials, 6(1):9, Jan 2020. ISSN 2057-3960. doi: 10.1038/s111224000000777 URL
attps://dbi.org// 10t////0..000///52-00000000
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 'why should i trust you?': Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Dato Mining, KDD '16, pp. 1135 1144, New York, NY, USA, 2016. As- sociation for Computing Machinery. ISBN 9781450342322. doi: 10. 10.1155/0930722233373. URL
https: htt.//oo.oo/////////////oo 10. 1145/2939672 2939778
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10684-10695, 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Nassir Navab, Joachim Hornegger, William M. Wells, and Alejandro F. Frangi (eds.). Medical Image Computing and Computer-Assistee Intervention -IICAI 2015, pp. 234-241 Cham, 2015. Springer International Publishing. ISBN 978-3-319-2457444.
Andrew Ross, Nina Chen, Elisa Zhao Hang, Elena L. Glassman, and Finale Doshi-Veles Evaluating the interpretability of generative models by interactive reconstruction. In Proceedings of the 2021 CHI Con- ference on Human Factors in Computing Systems, CHI 221 New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450380966.
Yangiun Ruan, Chris J. Maddison, and Tatsunori Hashimoto. Observational scaling laws and the pre- dictability langauge model performance. In The Thirty- -iggtt Annual Conference on Neural Information Processing Systems, 2024. URL /tttps://operreview www.ee...eeeeeeee http:fforumndddnS5i.e
Phillip Rust, Jonas Pfciffer, Ivan Vulić, Scbastian Ruder, and Iryna Gurevych. How good is your tokenizer? on the monolingual performance of multilingual language models. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 3118 3135, Online, August 2021, Association for Computational Linguistics. doi: 10.18653/ v1 w1/2021 acb-llacl-oog 243. URL https://aclanthollgy www.pe/..o............. argg/221.accl1oom. llng- 243/
43

---

Published in Transactions on Machine Learning Research (08/2025)
Mehrdad Saberi, Vinu Sankar Sadasivan, Keivan Rezaci, Aounon Kumar, Atoosa Chegini, Wenxiao Wang, and Soheil Feizi. Robustness of ai-image detectors: Fundamental limits and practical attacks. In The Twelfth International Conference on Learning Representations, 2023.
Shiori Sagawa*, Pang Wei Koh*, Tatsunori B. Hashimoto, and Percy Liang, Distributionally robust neural networks. In International Conference on Learning Representations, 2020. URL https://openreviee. net/forum'id=ryzzn aet/forrum?derryyGGFFFFF
Chitwan Saharia, William Chan, Saurabh Saxena, Tala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Raphael Gontijo-Lopes, Bureu Karagol Ayan, Tim Salimans, Jonathan Ho, David J. Fleet, and Mohammad Norouzi. Photorcalistic tcxt-to-image image diffusion models with doep language understand- ing. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL https://openrevier. wttpreewwwwweeeeeouunuuum.eme net:/forur1dd-001-1552222
Mehdi SM Sajjadi, Olivier Bachem, Mario Lucic, Olivier Bousquet, and Sylvain Gelly. Assessing generative models via precision and recall. Advonces in newral information processing systems, 31, 2018.
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In International Conference on Learning Representations, 2022. URL
Tim Salimans, Ian Goodfellow, Wojeiech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. Advances in neural information processing systems, 29, 2016.
Dylan Sam, Rattana Pukdee, Daniel P. Jeong, Yewon Byun, and J. Zico Kolter. Bayesian neural networks with domain knowledge priors. ArXiv, abs abs/2440.134400, 2024. URL https://api. sea.aaicccaaat.cccccar....rr org/
CD2PUSIDD267770645
Pedro Sanchez and Sotirios A. Tsaftaris. Diffusion causal models for counterfactual estimation. In First Conference Causal Learning and Reasoning, 2022.
Pedro Sanchez, Xiao Liu, Alison Q O'Neil, and Sotírios A. Tsaftaris. Diffusion models for causal discovery via topological ordering. In The Eleventh International Conference on Learning Representations, 2023.
Rylan Schaeffer, Brando Miranda, and Sanmi Koyejo. Are emergent abilities of large language models a mirage? Advances Neural Information Processing Systems, 36, 2024.
Lisa Schut, Nenad Tomašev, Tom McGrath, Demis Hassabis, Ulrich Paquet, and Been Kim. Bridging the human-ai knowledge gap: Concept discovery and transfer in alphazero. ArXiv, abs/2230.11410, 2023.
URL https://api. http::://pp..seetatiieaaaaeeeee semanticscholaa org/CorpusID 264451628
Bernhard Schölkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. Toward causal representation learning. Proceedings of the IEEE, 109(:): 612 634, 2021.
Thanveer Shaik, Xiaohui Tao, Lin Li, Haoran Xie, and Juan D. Velásquez. A survey of multimodal information fusion for smart healthcare: Mapping the journcy from data to wisdom. Information Fusion, 102:102040, 2024. ISSN 1566-2535. doi: attps://wwit www.::wwweeeeeeee..ooo..... wtt.a//www........ooo.....aaaaaa
20w.iiff..22..11....0 202...022..0........ URL
C. E. Shannon. Prediction and entropy of printed english. The Bell System Technical Journal, 30(1):50- 64, 1951. doi: 10.1002/j.15338-7305 115.000..1111...113600...
Lee Sharkey, Bilal Chughtai, Joshua Batson, Jack Lindsey, Jeff Wu, Lucius Bushnaq. Nicholas Goldowsky- Dill, Stefan Heimersheim, Alejandro Oriega, Joseph Bloom, Stella Biderman, Adria Carriga-Alons Arthur Conmy, Neel Nanda, Jessica Rumbelow, Martin Wattenberg, Nandi Schoots, Joseph Miller, Eric J. Michaud, Stephen Casper, Max Tegmark, William Saunders, David Bau, Eric Todd, Atticus Geiger, Mor Geva, Jesse Hoogland, Daniel Murfet, and Tom McGrath. Open problems in mechanistic interpretability, 2025. URL htttps://arss www.av..rrgabb//////// 16496 16496
44

---

Published in Transactions on Machine Learning Research (08/2025)
Noam Shazeer. Glu variants improve transformer. arXiv preprint arXiv:2002.05200 2020.
Hui Shen, Jingxuan Zhang, Boning Xiong, Rui Hu, Shoufa Chen, Zhongwei Wan, Xin Wang, Yu Zhang, Zixuan Gong, Guangyin Bao, Chaofan Tao, Yongleng Huang, Ye Yuan, and Mi Zhang. Efficient. diffusion models: A survey. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. URL https: //openreview.o http://wwww.ot....orrr///dddtaaa
Survey Certification.
Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models. arXiv preprint arXiv:2308.088: 03882 2023.
Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. 'do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models, 2024. URL https://arxiv.org. abs/2308. 03825
Yikang Shen, Shawn Tan, Alessandro Sordoni, and Aaron Courville. Ordered neurons: Integrating tree structures into recurrent neural networks. In International Conference on Learning Representations, 2019. URL httts:://pperreiieet wwt./eorommddd-illeeeeee
Yujun Shen, Jinjin Gu, Xiaoou Tang, and Bolei Zhou. Interpreting the latent space of gans for semantic face editing. In CVPR, 2020.
Zheyan Shen, Jiashuo Liu, Yue He, Xingxuan Zhang, Renzhe Xu, Han Yu, and Peng Cui. To- wards out-of-distribution generalization: A survey. ArXie, abs abs/2108.13626 2021. URL https://api:
semanticscholar. www.:olllarrro/oooooooooooooo Patg/orpuuIDD.. :twp//ooppDDDD:23341122 237364121
Giora Simchoni and Saharon Rosset. Integrating random effects in deep neural networks. Journal of Machine Learning Research, 24(156):11555 2023. URL. http://jmlr.... http:paperss//22222-00000 htmml
Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. In The Eleventh International Conference on Learning Representations, 2022.
Chandan Singh, Jeevana Priya Inala, Michel Galley, Rich Caruana, and Jianfeng Gao. Rethinking inter- pretability in the era of large language models. orXiv preprint arXiv:2402.0170 017611 2024.
Raghav Singhal, Mark Goldstein, and Rajesh Ranganath. Where to diffuse, how to diffuse, and how to get back: Automated learning for multivariate diffusions. In The Eleventh International Conference Learning Representations, 2023.
Dylan Slack, Anna Hilgard, Himabindu Lakkaraju, and Samcer Singh. Counterfactual explanations can be manipulated. Advances in neural information processing systems, 34:62 75, 2021.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learn- ing using nonequilibrium thermodynamics. In Francis Bach and David Blei (eds.), Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Re- Lille, France, 07-09 Jul 2015. PMLR. URL
search, pp. -37/sohl-diccsteein15
Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Diffusion art or digital forgery? investigating data replication in diffusion models. In Proceedings of the IEFE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6048 6058, 2023.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021.
Yang Song and Prafulla Dhariwal. Improved techniques for training consistency models. In The Twelfth International Conference on Learning Representations, 2024. URL http:///pppnrrriieeeeeeeooooroee MNzy9bRDvC
45

---

Published in Transactions on Machine Learning Research (08/2025)
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. orXio preprinl arXiv:2011113450, 2020.
Yang Song, Prafulla Dhariwal, Mark Chen, and llya Sutskever. Consistency models. arXiv preprint arXiv:2303.01469, 2023.
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupla, Adrià Garriga-Alonso, et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXie preprint arXiv:2206.0400 2022.
Ethan Steinberg, Ken Jung, Jason A. Fries, Conor K. Corbin, Stephen R. Pfohl, and Nigam H. Shah. Language models are an effective representation learning technique for electronic health record data. Journal of Biomedical Informalics, 113:103637, 2021. ISSN 1532-0464.
Vincent Stimper, Bernhard Schölkopf, and José Miguel Hernández-Lobato Resampling base distributions of normalizing flows. In International Conference on Artificial Intelligence and Statistics, pp. 4915 4936. PMLR, 2022.
Emma Strubell, Ananya Ganesh, and Andrew McCallum. Energy and policy considcrations for modern doep
learning research. Proceedings of the AAAI Conference on Artificial Intelligence, 34(09))13693- 1369113363331666,,0- Apr. 2020. doi: 00.1009/aaai.v34400.777 7123
URL https://ojs. aaai wrwg/indg....ooeoo htt.p//..pp/p/...tttttaaaaaa/aaa
Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Nearocomputing, 568: 167:12, 2024.
Hao Sun, Zhexin Zhang, Jiawen Deng, Jiale Cheng, and Minlie Huang. Safety assessment of chinese large language models. arXiv preprint 1041:23044004330 2023a.
Hui Sun, Tianqing Zhu, Zhiqiu Zhang, Dawei Jin, Ping Xiong, and Wanlei Zhou. Adversarial attacks against deep generative models on data: A survey, IEEE Trans. on Knowl. and Data Eng., 35(4):3367 3388, April 2023b. ISSN 1041-4347. doi: 00.109/TKDE.202..130000 URL https:///61..or/10..11 TKDE. 2121..
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference Machine Learning Volume 70, ICML'17, pp. 3319-3328 JMLR.org,
2017.
Abhijit Suprem and Calton Pu. Exaluating generalizability of fine-tuned models for fake news detection. ArXiu, abs/2205.07110 2022.
Thomas M. Sutter, Laura Manduchi, Alain Ryser, and Julia Vogt. Learning group importance using the diffcrentiable hypergoometric distribution. In The Eleventh International Conference Learning Represenlations, 2023. URL https:://oeenrvview wtt.:/peeeeeeeeeeeeeeooororiiiie
L4oY
Yi Tay, Mostafa. Dehghani, Dara Bahri, and Donald Metzler. Efficient transformers: A survey. ACM Compul. Surv., 56))), December 2022a. ISSN 0360-0300. doi: 10.1145/3530811. URL httth:://///ot http:////ooooooo.../.. 1145/ 3530811
Yi Tay, Mostafa Dehghani, Dara Bahri, and Donald Metzler. Efficient transformers: A survey, arXiv preprint arXiv:2009.0677 2022b.
Guy Tevet and Jonathan Berant. Evaluating the evaluation of diversity in natural language generation. Paola Merlo, Jorg Tiedemann, and Reut Tsarfaty (eds.), Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguisties: Main Volume, pp. 326-346, Online, April 2021. Association for Computational Linguistics. doi: 10. 10653v1// 2000///v1/22111// //1/222.cceeeeee waw.-mmain........ 25. URL https: V/aclanthologg.orgg/// www.:/orgg/22122 www..eeallllmml.2...
46

---

Published in Transactions on Machine Learning Research (08/2025)
Hoang Thanh- Thanh-Tung and Truyen Tran. Catastrophic forgetting and mode collapse in gans. In 2020 interna- tional joint conference on neural nelworks (ijenn), pp. 1-10. IEFE, 2020.
L. Theis, A. van den Oord, and M. Bethge. A note on the evaluation of generative models. In International Conference on Learning Representations, Apr 2016. URL
http://wwww..orga/////////// 01844
Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher D Manning. Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback. arXiv preprint arxi:22305.144 14975, 2023.
Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. Eyes wide shut? exploring the visual shortcomings of multimodal lims. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9568-9578, 2024.
Cagri Toraman, Eyup Halit Yilmaz, Furkan Sahimuc, and Oguzhan Ozcelik. Impact of tokenization on language models: An analysis for turkish. ACM Trans. Asian Low- Resour. Lang. Inf. Process., 22(4), March 2023, ISSN 2375-4699. doi: 10.1145/3578707. URL https: /t:ps://dof org/10. 1145/3578707
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bash- lykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Fer- rer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Tsabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anrs Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuming Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nic, Andrew Poulton, Jeremy Reizenstein, Rashi Rungla, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subra- manian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, lliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models, 2023.
Brian L. Trippe, Jason Yim, Doug Tischer, David Baker, Tamara Broderick, Regina Barzilay, and Tommi Jaakkola. Diffusson probabilistie modeling of protein backbones in 3d for the motif-scaffolding problem. In International Conference on Learning Representations, 2023. URL http:://opennevview.neffroooo id=6TxBxqMME1Y
Yun-Yun Tsai, Fu-Chen Chen, Albert Y. C. Chen, Junfeng Yang, Che-Chun Su, Min Sun, and Cheng-Hap Kuo. Gda: Generalized diffusion for robust test-time adaplation. In CVPR, pp. 23242 23251, 2024. URL https://doi. org/ 10.
2024. 02193
Adaku Uchendu, Zeyu Ma, Thai Le, Rui Zhang, and Dongwon Lee. TURINGBENCH: A benchmark environment for Turing test in the age of neural text generation. In Maric-Francins Moens, Xuan- jing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Findings of the Association for Compula- tional Linguistics: EMNLP 2021, pp. 2001 2016, Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10.18653/v1/200 http://////222..nnnsssooo/oo andings-cmnlp.112 URL https:
//aclanthology.oo www.a//oooooyooo//22....o wtg/ findings emllp.
Shubham Ugare, Tarun Suresh, Hangoo Kang, Sasa Misailovic, and Gagandeep Singh.
Syncode: Llm
generation with grammar augmentation, 2024.
Arash Vahdat, Karsten Kreis, and Jan Kautz. Score-based generative modeling in latent space. Advances in Neural Information Processing Systems, 34:11287 11302, 2021.
Jessica Vamathevan, Dominic Clark, Paul Czodrowski, Ian Dunham, Edgardo Ferran, George Loe, Bin Li, Anant Madabhushi, Parantu Shah, Michaela Spitzer, et al. Applications of machine learning in drug discovery and development. Nature reviews Drug discovery, 18(6):463 -47,, 2019.
47

---

Published in Transactions on Machine Learning Research (08/2025)
Gerrit van den Burg and Chris Williams. On memorization in probabilistic deep generative mod- els. In M. Ranzato, A. Beygekzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 27916 27928. Curran Associates, Inc., 2021. URL Bae pdf
Liam van der Poel, Ryan Cotterell, and Clara Meister. Mutual information alleviates hallucinations in abstractive summarization. In Yoav Goldberg, Zornitsa Kozarova, and Yue Zhang (ods.), Proceedings of the 2022 Conference on Empirical Methods in Naturul Language Processing, pp. 5956 5965, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10. 10.18653/v1/ 2022. www.emmmmmlmmmm09000. URL attps://acllantholooy wrw.:/22....222222.2 emnlp-main 399/
Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Usækoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NIPS, 2017.
Venkat Venkatasuurramanian. The promise of artificial intelligence in chemical engineering: Is it here, finally?
AIChE Journal, 65(2):466-478, 2019.
Jesse Vig and Yonatan Belinkov. Analyzing the structure of attention in a transformer language model. In Tal Linzen, Grzegorz Chrupala, Yonatan Belinkov, and Dieuwke Hupkes (eds.), Proceedings of the 2019 ACL Workshop BlackbozNLP: Analyzing and Interpreting Neural Networks for NLP, pp. 63-76, Florence, Italy, August 2019. Association for Computational Linguisties. doi: 10.18653/w1/W199440 URL
Julius von Kügglgen, Michel Besserve, Liang Wendong, Luigi Gresele, Armin Kekić, Elias Barcinboim, David Blei, and Bernhard Schölkopf. Nonparametric identifiability of causal representations from unknown interventions. Advonces in Neural Information Processing Systems, 36, 2024.
Mai Ha Vu, Rahmad Akbar, Philippe A. Robert, Bartlomiej Swiatczak, Geir Kjetil Sandve, Victor Greiff, and Dag Trygve Truslew Haug. Linguistically inspired roadmap for building biologically reliable protein language models. Nature Machine Intelligence, 5(5):485 496, May 2023. ISSN 2522-5839. doi: 10.1038/ 112256-023-006377 URL https://///oi org/
Sandra Wachter, Brent Mittelstadt, and Chris Russell. Counterfcctul explanations without opening the black box: Automated decisions and the gdpr. Harv. JL & Tech. 31:841, 2017.
Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. Universal adversarial triggers for attacking and analyzing nlp. arXiv preprint arXiv:1908.0710 2019.
Cheng Wang. Calibration in deep learning: A survey of the state-of-the-art ArXiv, abs/2238800122 2023. Prrg/CorpusID::0e 260377914
URL http::://app..smsttiscsssee.
Jindong Wang, Xixu HU, Wenxin Hou, Hao Chen, Runkai Zheng, Yidong Wang, Linyi Yang, Wei Ye, Haojun Huang, Xiubo Geng, Binxing Jiao, Yue Zhang, and Xing Xie. On the robustness of chatGPT: An adversarial and out-of-distribation perspective. In ICLR 2023 Workshop on Trustworthy and Reliable Large-Scale Machine Learning Models, 2023a.
Xi Wang, Laurence Aitchison, and Maja Rudolph. Lora ensembles for large language model fine-tuning. arXiv preprint orXic:2310.0003: 2023b.
Xinru Wang and Ming Yin. Are explanations helpful? a comparative study of the effects of explanations in ai-assisted decision-making. In 26th international conference intelligent user interfaces, pp. 318 3288, 2021.
Yixin Wang and Michael I Jordan. Desiderata for representation learning: A causal perspective. arXiv preprint XrXi::2109 03795, 2021.
48

---

Published in Transactions on Machine Learning Research (08/2025)
Albert Webson and Ellie Pavlick. Do prompt-based models really understand the meaning of their prompts? In Marine Carpuat, Marie-Catherine de Marneffe, and Tvan Vladimir Meza Ruiz (eds.), Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2300-2344, Seattle, United States, July 2022. Association for Com- putational Linguistics. doi: 10. 10.18653/v ///0//v1/////
mainncc-main.6... URL https:///clantholl/ www.gy....gg.///////////
167
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Tiang, Jeff Dean, and William Fedus. Emergent abilities of large language models. Transactions on Ma- chine Learning Research, 2022. ISSN 2835-8856. URL https://openreviir: http:://pppaaueeeee.eee/e//rrrrp .ett:/orrummraxyyyysssuuuzu Survey Certification.
Jing Wei, Xuan Chu, Xiang- Yu Sun, Kun Xu, Hui-Xiong Deng, Jigen Chen, Zhongming Wei, and Ming Lei. Machine learning in materials science. InfoMat, 1(3):338-33) 2019.
Laura Weidinger, John F. J. Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, Zachary Kenton, Sande Minnich Brown, William T. Hawkins, Tom Stepleton, Courtney Biles, Abeba Birhane, Julia Haas, Taura Rimell, Lisa Anne Hendricks, William S. Isaac, Sean Legassick, Geoffrey Irving, and lason Gabriel. Ethical and social risks of harm from language models. ArXiv, Abs/2112.04439, 2021.
Yuxin Wen, John Kirchenbauer, Jonas Geiping, and Tom Goldstein. Tree-rings watermarks: Invisible fin- gerprints for diffusion images. In Thirty-sewenth Conference on Neural Information Processing Systems, 2023. URL http://ppeneeeieeeeeeeftrorrmmee Jrmubl
Steven Euijong Whang, Yuji Roh, Hwanjun Song, and Jac-Gil Lee. Data collection and quality challenges in deep learning: data-centric ai perspective. The VLDB Journal, 32:791-813, 2021.
Colin White, Mikhail Khodak, Renbo Tu, Shital Shah, Sébastien Bubeck, and Debadeepta Dey. A deeper look at zero-cost proxies for lightweight nas. In ICLR Blog Track, 2022. htttp::///crrrrbbbo blog track. 2ttthh/..//0002//////////rroorrp proxies/.
https:/ http://ccrrrbbbe www---looootrrreee wtw.ttrracctaiiia.. www.ui.ggtuhbbbhhb.ee. coosooo-
Lisa Wimmer, Yusuf Sale, Paul Hofman, Bernd Bischl, and Eyke Hüllermeier. Quantifying aleatoric and epistemic uncertainty in machine learning: Are conditional entropy and mutual information appropriate measures? In Robin J. Evans and Πya Shpitser (eds.), Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence, volume 216 of Proceedings of Machine Learning Research, pp. 2282 2292. PMLR, 31 Jul 04 Aug 2023. URL https: htttps://proceeednngg mmr...g..mm.....mm..mm.......... prrsss/v11//vimmerr/// http:///g...........as//////////
html
Robin Winter, Floriane Montanari, Frank Noé, and Djork- Clevert. Learning continuous and data-driven molecular descriptors by translating equivalent chemical representations. Chem. Sci., 10:1692-1701, 2019. doi: 10.1039 C8SC04175J. URL http: / /xx doi. www//0...0./....ooo 1039///0411/3
Carole-Jean WI, Ramya Raghavendra, Udit Gupta, Bilge Acun, Newsha Ardalani, Kiwan Maeng, Gloria Chang, Fiona Aga, Jinshi Huang, Charles Bai, Michael Gschwind, Anurag Gupta, Myle Ott, Anastasia Melnikov, Salvatore Candido, David Brooks, Geeta Chauhan, Benjamin Lee, Hsien-Hsin Lee, Bugra Aky- ildiz, Maximilian Balandat, Joe Spisak, Ravi Jain, Mike Rabbat, and Kim Hazelwood. Sustainable ai: Environmental implications, challenges and opportunities. In D. Marculescu, Y. Chi, and C. Wu (eds.). Proceedings of Machine Learning and Systems, volume 4, pp. 795-813, 2022. URL U/L/http::/prroeedingg mlsys. org/ paper_ wwtp//per..p.p.r.cm
.dff
Tong Wu, Zhihao Fan, Xiao Liu, Hai- Zheng, Yeyun Gong, yelong shen, Jian Jiao, Juntao Li, zhongyu wei, Jian Guo, Nan Duan, and Weizhu Chen. AR-diffusion: Auto-regressive diffusion model for text generation. In seirth- Conference on Neural Information Processing Systems, 2023. URL https: (woggareriivv.ee/fffommitteeeeee
49

---

Published in Transactions on Machine Learning Research (08/2025)
Zongliang Wu, Ruiying Lu, Ying Fu, and Xin Yuan. Latent diffusion prior enhanced deep unfolding for snapshot spectral compressive imaging. In Computer Vision FCCV 2024: 18th European Conference, Milan, Italy, September 29 October 4, 2024, Proceedings, Part XXXIII, pp. 164 181, Berlin, Heidelberg, 2024. Springer- ISBN 978-9--01--731113.. doi: 10.100779988---081-778888
10. URL /RLttt/::://do
Peng Xia, Kangyu Zhu, Haoran Li, Tianze Wang, Weijia Shi, Sheng Wang, Linjun Zhang, James Zou, and Huaxiu Yao. MMed-RAG: Versatile multimodal RAG system for medical vision language mod- els. In Neurips Safe Generative Al Workshop 2024, 2024. URL sttps://openreviee.net// DjUumZhV3s
Zhou Xian, Theophile Gervel, Zhenjia Xu, Yi-Ling Qiao, Tsun-Hsuan Wang, and Yian Wang. Towards generalist robots: A promising paradigm via generative simulation. arXiv preprint arXiv:2305.10455, 2023.
Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. Smoothquant: Accurate and efficient post-training quantization for large language models. In International Conference on Machine Learning, pp. 38087 -8099. PMLR, 2023.
Zhisheng Xiao, Karsten Kreis, and Arash Vahdat. Tackling the generative learning trilemmaa with de- noising diffusion GANs. In International Conference on Learning Representations, 2022. URL https:
Miao Xiong, Zhiyuan Hu, Xinyang Lu, YIFEI LI, Jie Fu, Junxian He, and Bryan Hooi. Can LLMs express their uncertainty? an empirical evaluation of confidence elicitation in LLMs. In The Twelfth International Conference on Learning Representations, 2024. URL https:/ net:/foummiddggjieeeeoooo
Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio. Show, attend and tell: Neural image caption generation with visual attention. In Francis Bach and David Blei (eds.), Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pp. 2048 2048-2057, Lille, France, 07-09 Jul 2015. PMLR. URL https://pproceedinng mlr. wwwppppsssssseeeeepee wwt.///1./1..... html
Yilun Xu, Shengjia Zhao, Jiaming Song, Russell Stewart, and Stefano Ermon. A theory of usable information under computational constraints. In International Conference Learning Representations, 2020. URL
https://openrrevie: wttp::/wwww.....ooooooooroeoeo//
Yucmci Xu, Ling Hu, Jiayi Zhao, Zihan Qiu, Kexin Xu, Yuqi Ye, and Hanwen Gu. A survey on multilingual large language models: corpora, alignment, and bias. Froni. Compul. Sci., 19(11), April 2025. ISSN 2095-2228. doi: 1111007/111704400444007777
URL
Ziwei Xu, Sanjay Jain, and Mohan S. Kankanhalli. Hallucination is inevitable: An innate limitation of large language models. ArXiv, abs/ abs/2400.118117 2024. URL
267069207
Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, and Colin Raffel. ByT5: Towards token-free future with pre- pre-trained byte-to-byte models. Transactions of the Association for Computational Linguistics, 10:291-306, 2022. doi: 10.1162/tacl a 00461. URL
https://aclanthoolggy www.oorg//20 202g/2022.c00 1tt.:o/......022222..1111.......
Haomiao Yang, Kunlan Xiang, Mengyu Ge, Hongwei Li, Rongxing Lu, and Shui Yu. A comprehensive overview of backdoor attacks in large language models within communication networks, 2023a. URL
https://arxiv. www./oo//////aeeeeeeoeee 14367
Linyi Yang, Yaoxian Song, Xuan Ren, Chenyang Lyu, Yidong Wang, Jingming Zhuo, Lingqiao Liu, Jindong Wang, Jennifer Foster, and Yue Zhang. Out-of-distribution generalization in natural language processing: Past, present, and future. In The 2023 Conference on Empirical Methods in Notural Language Processing. 2023b. URL httpap///ppppeeeeee.eee//////s/s
50

---

Published in Transactions on Machine Learning Research (08/2025)
Ruihan Yang and Stephan Mandt. Lossy image compression with conditional diffusion models. In Neural Information Processing Systems, 2023.
Ruihan Yang. Prakhar Srivastava, and Stephan Mandt. Diffusion probabilistic modeling for video generation. Entropy, (5(10):1469, 2023c.
Yibo Yang, Robert Bamler, and Stephan Mandt. Variational Bayesian quantization. In International Con- ference on Machine Learning, pp. 10670-10680. PMLR, 2020.
Yibo Yang, Stephan Mandt, and Lucas Theis. An introduction to neural data compression. Foundations and Trendx0 in Computer Graphics and Vision, 15(2):113-200 2023d.
Yuchen Yang, Houqiang Li, Yanfeng Wang, and Yu Wang. Improving the reliability of large language models by leveraging uncertainty-awant in-context learning, 2023e. URL Intpps///arxiv httpp:://axxx..orrr//////20.0000 www..o//aab//21..0078...
Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Zhibo Sun, and Yue Zhang. A survey on large language model (Ilm) security and privacy: The good, the bad, and the ugly. High-Confidens Computing, 4(2): 100211, 2024. ISSN 2667-2952.
doi: https:///bii.rrg/ 10. hhc.
URL https://www.
sciencedirect, @cieeneeirett.coom/ccieiee//taae http://sceeeeeeaatttraaaaa//////
Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu, Conglong Li, and Yuxiong He. Zeroquant: Efficient and affordable post-training quantization for large-scale transformers. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Aduances in Neural Information Processing Systems, volume 35, pp. 27168-27188 27183. Curran Associates, Inc., 2022.
Wenqian Ye, Guangtao Zheng, Xu Cao, Yunsheng Ma, and Aidong Zhang. Spurious correlations in machine learning: A survey, 2024. URL attps://aaxxxv.orr/bbb///e 12715.
Sibo Yi, Yule Liu, Zhen Sun, Tianshuo Cong, Xinlei He, Jiaxing Song, Ke Xu, and Qi Li. Jailbreak attcks and defenses against large language models: A survey, 2024. URL https: wwtpa//wwww..p////////....../
KiYoon Yoo, Wonhyuk Ahn, and Nojun Kwak. Advancing beyond identification: Multi-bit bit watermark for large language models. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 4031-4055, Mexico City, Mexico, June 2024. Association for Computational Linguistics. doi: 10. / //aclanthology org/2024 haacl-long 224/
URL. https:
Jifan Yu, Xiaozhi Wang, Shangqing Tu, Shulin Cao, Daniel Zhang-Li, Xin Lv, Hao Peng, Zijun Yao, Xiaohan Zhang, Hanming Li, Chunyang Li, Zheyuan Zhang, Yushi Bai, Yantao Liu, Amy Xin, Kaifeng Yun, Linlu GONG, Nianyi Lin, Jianhui Chen, Zhili Wu, Yunjia Qi, Weikai Li, Yong Guan, Kaisheng Zeng, Ji Qi, Hailong Jin, Jinxin Liu, Yu Gu, Yuan Yao, Ning Ding, Lei Hou, Zhiyuan Liu, Xu Bin, Jie Tang, and Juanzi Li. KoLA: Carefully benchmarking world knowledge of large language models. In The Twelfth International Conference Learning Representations, 2024. URL https:// hett/forrum/dd-gg230g0pr
Kaiwen Yuan, Zhenyu Guo, and Z. Jane Wang. RGGNet: Tolerance aware LiDAR-camera online calibration with goometric deep lcarning and generative model. IEEE Robotics and Automation Letters, 5(4):6956- 6963, 2020. doi: 10.1109 1..1099/RRAA www.......022.00000
Qian Zeng, Chenggong Hu, Mingli Song, and Jie Song. Diffusson model quantization: A review, 2025. URL
https://arxiv.org/ abw055 05215
Daochen Zha, Zaid Pervaiz Bhat, Kwei-Herny Lai, Fan Yang, Zhimeng Jiang, Shaochen Zhong, and Xia Hu. Data-centric artificial intelligence: A survey. ACM Comput. Surv., 57(5), January 2025. ISSN 0360-0300. doi: 10.1145/371111
URL https::///oo/ http://doo.orgg100 1tt.o//////....////11111.. 1145/3711188
Biao Zhang and Rico Sennrich. Root mean square layer normalization. Advances in Neural Information Processing Systems, 32, 2019.
51

---

Published in Transactions on Machine Learning Research (08/2025)
Hanqing Zhang, Haolin Song, Shaoyu Li, Ming Zhou, and Dawei Song. A survey of controllable text generation using transformer-based pre-trained language models. ACM Compul. Surv., 56(3), October 2023a. ISSN 0360-0300. doi: 10.1145/3617666 URL httpa:///doo.o//////// 111111.11361119890
Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization, 2018. URL httpp://arrav.oorr/bb//111.... 09412
Hugh Zhang, Daniel Duckworth; Daphne Ippolito, and Arvind Neelakantan. Trading off diversity and quality in natural language generation. In Anya Belz, Shubham Agarwal, Yvette Graham, Ehud Reiter, and Anastasia Shimorina (eds.), Proceedings of the Workshop on Human Evaluction of NLP Systems (HumEval), pp. 25-33, Online, April 2021. Association for Computational Linguisties. URL https: //aclanthology. org/2021 htt.2.lhmeall1133///
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV), pp. 3836-3844 3847, October 2023b.
Mozhi Zhang, Mianqiu Huang, Rundong Shi, Linsen Guo, Chong Peng, Peng Yan, Yaqian Zhou, and Xipeng Qiu. Calibrating the confidence of large language models by eliciting fidelity. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 2959 2979, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10. 18813/86111// 11863//112024..mm/ppp /2024.emmppmmminn117. URL htttps:/aclaathoologg org/ 2024. emnlp-main 173/
Qinsheng Zhang and Yongxin Chen. Fast sampling of diffusion models with exponential integrator. In The Eleventh International Conference on Learning Representations, 2023.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint Xi::::200..0 01068, 2022.
Tianhang Zhang, Lin Qiu, Qipeng Guo, Cheng Deng, Yue Zhang, Zheng Zhang, Chenghu Zhou, Xin- bing Wang, and Luoyi Fu. Enhancing uncertainty-based hallucination delection with stronger focus. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 915 932, Singapore, December 2023c. Association for Com- org/e//3..g22222
putational Linguistics. doi: 10.18653/v1/22 www.220022...2.2. 58. URL emnlp-main.65 58/
Jiaxu Zhao, Meng Fang, Shirui Pan, Wenpeng Yin, and Mykola Pechenizkiy. Gptbias: A comprehensive framework for evaluating bias in large language models. arXiv preprint XrXiv:2812, 2. 2.3115, 2023a.
Shengjia Zhao, Hongyu Ren, Arianna Yuan, Jiaming Song, Noah Goodman, and Stefano Ermon. Bias and generalization in deep generative models: An empirical study. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018
Xuandong Zhao, Prabhanjan Vijendra Ananth, Lei Li, and Yu-Xiang Wang. Provable robust. watermarking for Al-generated text. In The Twelfth International Conference on Learning Representations, 2024. URL
met//forum?id=SSemraaa555
Yao Zhao, Rishabh Joshi, Tianqi Liu, Misha Khalman, Mohammad Saleh, and Peter J. Liu. SLIC-HF: Sequence likelihood calibration with human feedback, 2023b. URL htttts:////xxii org/ aa/////////2//// 10425.
Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Ngai-Man Cheung, and Min Lin. A recipe for water- marking diffusion models. arXiv preprint arXi::2308 10137, 2023c.
Zhengyue Zhao, Jinhao Duan, Xing Hu, Kaidi Xu, Chenan Wang, Rui Zhang, Zidong Du, Qi Guo, and Yunji Chen. Unlearnable examples for diffusion models: Protect data from unauthorized exploitation. arXiv preprint or Xiv::2300.0011 01900, 2023d.
52

---

Published in Transactions on Machine Learning Research (08/2025)
Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. Calibrate before use: Improving few- shot performance of language models. In Marina Meila and Tong Zhang (eds.), Proceedlings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 12697-12706. PMLR, 18-24 Jul 2021. URL htttp:://pooeeet
mlr www..press/vii. presss/v39/2hao2cc..oo
zhao2ic. html
Guangcong Zheng, Xianpan Zhou, Xuewei Li, Zhongang Qi, Ying Shan, and Xi Li. Layoutdiffusion: Con- trollable diffusion model for layout-to-image generation. In 2023 TEFE/CH CVF Conference on Computer Vision and Pattern Recognation (CVPR), pp. 2290022499, 2023a. doi: 10.1109//CV /CtCC//222.222..2222222222
Huangjie Zheng, Pengcheng He, Weizhu Chen, and Mingyuan Zhou. Truncated difTusion probabilistic models and diffusion-based adversarial auto-encoders. In The Fleventh International Conference on Learning Representations, 2023b. URL https://openrevien wttprrevvieeeeeeffrrrueee hett/forrumiicctxxaak.ettt
Fangling Zhou, Kejun He, and Yang Ni. Causal discovery with heterogeneous observational data. In James Cussens and Kun Zhang (eds.), Proceedings of the Nhirty-Eighth Conference on Uncertainty in Artificial Intelligence, volume 180 of Proceedings of Machine Learning Research, pp. 2883. PMLR, 01-05 Aug 2022.
Guanglin Zhou, Shaoan Xie, Guangyuan Hao, Shiming Chen, Biwei Huang, Xiwei Xu, Chen Wang, Liming Zhu, Lina Yao, and Kun Zhang. Emerging synergies in causality and deep generative models: A survey, arXiv preprint ar Xiv:23001. 12351, 2023.
Mi Zhou, Vibhanshu Abhishek, Timothy Derdenger, Jaymo Kim, and Kannan Srinivasan. Bias generative ai. Papers, arXiv.org, 2024. URL https:///Eco/ppperrrrepppe ortpeeeerrr/PPPP.aaaa....a papp::a 240..02726
Sharon Zhou, Mitchell Gordon, Ranjay Krishna, Austin Narcomey, Li Fei-Fei, and Michael Bernstein. Hype: A benchmark for human eye perceptual evaluation of generative models. Advonces in neural information processing systems, 32, 2019.
Yihe Zhou, Tao Ni, Wei-Bin Loe, and Qingchuan Zhao. survey on backdoor threats in large language models (Imss): Atackk,, defenses, and evaluations, 2025. URL
Chiwei Zhu, Benfeng Xu, Quan Wang, Yongdong Zhang, and Zhendong Mao. On the calibration of large language models and alignment. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Associalion for Compulational Linguistics: EMNLP 2023, pp. 9778 9795, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2022 http///........nn...e... settiiigseemmlp.6....... URL https:// aclanthology org/2023. 2023. f Findings-emiin em41
Kaijie Zhu, Jindong Wang, Jiaheng Zhou, Zichen Wang, Hao Chen, Yidong Wang, Linyi Yang, Wei Ye, Yue Zhang, Neil Gong, and Xing Xie. Promptrobust: Towards evaluating the robustness of large language models on adversarial prompts. In Proceedings of the 1st ACM Workshop on Large Al Systems and Models with Privacy and Safely Analysis, LAMPS '24, pp. 57 68, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400712098, doi: 01L1145/669227..388 10. 10.1145/3689211 3690621
URL htttps://dot org/
Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. Texygen: A benchmarking platform for text goneration models. In The 41st international ACM SIGIR conference on research & development in information retrieval, pp. 1097 1100, 2018
Bojia Zi, Shihao Zhao, Xingjun Ma, and Yu-Gang Jiang. Revisiting adversarial robustness distillation: Robust soft labels make student better. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 16443 16452, 2021.
Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Chris- tiano, and Geoffrey Irving. Fino-tuning language models from human preferences. arXiv preprint arXix:1909008500 2019.
53

---

Published in Transactions on Machine Learning Research (08/2025)
Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, An-KKathri Dombrowski, Shashwat, Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Troy Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, Zico Kolter, and Dan Hendrycks. Reprosentation engineering: A top-down approach to ai transparency. ArXiv, abs/2310.014055, 2023a. URL https://api. semanticscholar :tt.:aarrrrrrrrrrrrrai::::::00 263366661
Andy Zou, Zifan Wang, J. Zico Kolter, and Malt Fredrikson. Universal and transferable adversarial atlacks on aligned language models. ArXiv, abs/2307.15044, 2023b.
54


Reviews:
[{"review_id": "nrBUXiJi67", "reviewer": "Reviewer_HKXq", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\nThe paper is very well written and easy to read. It includes a diverse number of aspects related to existing challenges in generative models, provides relevant references and, in some cases, highlights possible paths or hints to address the mentioned challenges.\n\n**Weaknesses**\nAs the paper explores many diverse topics in a relatively small number of pages it keeps the discussion on an high level, without delving into much details for most of the challenges it mentions. While this makes it a useful and comprehensive read for someone approaching the field, I worry much of the provided information might be already known to researchers in the area (at least at this depth of a discussion level).\n\nMoreover there is no direct discussion regarding the surveys that already exist about the topic or sub-topics. On one hand this makes it hard to objectively evaluate the usefulness of this paper compared to the existing literature. On the other it is a lost opportunity to reference, in a structured and concise way, existing studies that, while maybe more limited in scope, delve deeper into the details of specific sub-areas related to generative models.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a comprehensive overview of the current shortcomings of recent generative AI methodologies, highlighting the related challenges and respective areas where further research is needed. In this regard the authors explore the topics of generalization to specific domains, robustness to malicious or out-of-distribution inputs, prior knowledge incorporation, causal modeling, handling of heterogenous data, other than text and images. Moreover also topics related to computational efficiency and model evaluation are explored, as well as ethical and societal aspects.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- I would suggest that the authors state more clearly what differentiates their work from other surveys that discuss similar topics.\n- I think it would meaningfully improve the paper if the authors could concisely provide references to topic-specific surveys (or the most meaningful works, in their absence); for instance by means of a table. As the scope of the discussion is very broad, this could provide a structured point of aggregation for the reader that wants to read further on specific topics, even though these references are already provided organically along the document.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ZKMGeahhuu", "reviewer": "Reviewer_pxE1", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths\n- Coverage is fairly comprehensive\n- The manuscript is well structured and clearly written. I never had trouble understanding what the authors meant\n- A lot of attention is given to evaluation metrics, which I find justified, given their importance for the progress of the field\n- Discussion is detailed and well contextualized within the current literature\n\n# Weaknesses\n- The discussion of robustness does not cover security concerns, e.g. jailbreaking\n- There is no real discussion of environmental impact of models", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper covers a wide range of currently open research problems in the field of generative AI. These include the areas of generalization, robustness, efficiency, evaluation, interpretability, fairness, and societal impacts, among others. In addition to reporting on these challenges and recent research attempting to solve them, the authors highlight certain directions which they believe would particularly benefit the community if solved. The result is a comprehensive review of the current challenges in generative AI, which should aid researchers looking to help solve these problems.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "# Security concerns\n\nSecurity concerns such as jailbreaking and backdoor attacks could significantly affect adoption in critical systems with a generative AI component, particularly when using LLMs. A discussion of the research in this area would strengthen the work. \n\nFor example, the following are useful starting points for references for backdoor attacks: \n- \"A Comprehensive Overview of Backdoor Attacks in Large Language Models within Communication Networks\" \n- \"A Survey on Backdoor Threats in Large Language Models (LLMs): Attacks, Defenses, and Evaluations\"\n\nAnd for jailbreaking:\n- \"Jailbreak Attacks and Defenses Against Large Language Models: A Survey\"\n- \"\"Do Anything Now\": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models\"\n\n# Environmental impact\n\nIt would be useful to have an indication of the environmental impact of large models, beyond high energy costs. What are the estimated carbon emissions of these models? What percentage of electricity usage does generative AI currently make up and how much is it projected to make up in the future? Answers to these questions would improve the submission, since environmental impact is also a social cost and a major challenge going forward.\n\n# Minor mistake\n\nOn page 10, the reference to Xu et al. 2020 is incorrectly formatted. The \"e.g.\" should come before the reference or be dropped.", "broader_impact_concerns": "No concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "oqHDL5sQXL", "reviewer": "Reviewer_pcoR", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The set of problems addressed in the paper are important and very timely.\n- The paper gives a broad (although somewhat superficial) overview of several important topics within deep generative models.\n- The paper includes many (around 375) references, and may serve as a good resource to navigate the literature.\n- The writing is clear and accessible.\n\n\nWeaknesses:\n- The paper does not present any new findings, novel insights or methodologies, but only summarizes known results.\n- The content selection in the review does not appear to be selected using a systematic approach.\n- The contents could be structured better, perhaps by summarizing key points, providing overviews in the form of tables or figures, or by providing clear, actionable recommendations.\n- The centrally defined \"perfect generative model\" in the introduction remains elusive, being either under-specified or, at face value, seemingly impossible.\n- The paper has a very broad focus, exemplified primarily through language and image models. It is not clear what the intended audience is.\n- Overall, the discussion remains quite abstract, which can occasionally make it challenging to understand how the discussed challenges could be addressed in a practical context. As a result, the reader gains more of an overview than a deep understanding.\n\nMany of the conclusions throughout the paper remain very general, which makes it difficult for the reader to contextualize. One example is:\n\n\"We argue...explicity encode human priors into the training process.\" How that could be done in a practical setting is not clear. Perhaps it would be more clear with some well chosen examples.\n\n\nThere are some statements throughout that seem overly general and perhaps not sufficiently substantiated. I will just mention a few examples:\n\n\"...inductive biases, largely driven by convenience or convention. (...) We argue that convenience should not be the driving factor...\" This seems to me to be a too strong statement. Many inductive biases are well thought through and well understood, such as translational equivariance in CNN's etc. What exactly is meant by convenience here? I would argue that practicality (i.e. what works) is a stronger driving factor than convenience or convention.\n\n\"The overarching goal is (...) applications (...) across different types of data (...) embodied agents must integrate perception...\" I do not follow the argument that applying DGM across domains necessitates embodied agents and addressing control tasks.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper discusses a range of challenges in deep generative models, including the following\n- generalization and adversarial robustness, \n- implicit and explicit assumptions, priors, and inductive biases\n- integration with causal models\n- cross-modality, integration of domain knowledge\n- efficiency and quantization\n- meaningful metrics, aligned with human evaluation\n- uncertainty quantification and hallucination\n- privacy, fairness, explainability, and constraint satisfaction\n\nAcross all these topics, the paper summarizes key challenges and provides extensive references to the literature.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I would suggest that the authors consider the points raised under Weaknesses and make any necessary revisions. In its current form, I believe the paper satisfies the criteria for publication, and would be useful to some readers looking to get an overview of the field and a curated set of references. \n\n\nA few typos etc.\n\n\"...(Grattefiori et al., 2024). have enabled...\" dot after parenthesis\n\n\"... resilient to minor variations...\" Is this a precise definition of adversarial robustness?\n\n\"...with the provided description. .\" two dots\n\n\"For example, (Ren et al., 2023)...\" -> \"For example, Ren et al. (2023)...\"\n\n\"(Kuhn et al. 2023) use a...\" -> \"Kuhn et al. (2023) use a...\"", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)