Reliable and Responsible Foundation Models
Authors: Xinyu Yang
Junlin Han
Rishi Bommasani
Jinqi Luo
Wenjie Qu
Wangchunshu Zhou
Adel Bibi
Xiyao Wang
Jaehong Yoon
Elias Stengel-Eskin
Shengbang Tong
Lingfeng Shen
Rafael Rafailov
Runjia Li
Zhaoyang Wang
Yiyang Zhou
Chenhang Cui
Yu Wang
Wenhao Zheng
Huichi Zhou
Jindong Gu
Zhaorun Chen
Peng Xia
Tony Lee
Thomas P Zollo
Vikash Sehwag
Jixuan Leng
Jiuhai Chen
Yuxin Wen
Huan Zhang
Zhun Deng
Linjun Zhang
Pavel Izmailov
Pang Wei Koh
Yulia Tsvetkov
Andrew Gordon Wilson
Jiaheng Zhang
James Zou
Cihang Xie
Hao Wang
Philip Torr
Julian McAuley
David Alvarez-Melis
Florian Tramèr
Kaidi Xu
Suman Jana
Chris Callison-Burch
Rene Vidal
Filippos Kokkinos
Mohit Bansal
Beidi Chen
Huaxiu Yao
Venue: Accepted by TMLR ()

Abstract: Foundation models, including Large Language Models (LLMs), Multimodal Large Language Models (MLLMs), Image Generative Models (i.e, Text-to-Image Models and Image-Editing Models), and Video Generative Models, have become essential tools with broad applications across various domains such as law, medicine, education, finance, and beyond. As these models see increasing real-world deployment, ensuring their reliability and responsibility has become critical for academia, industry, and government. This survey addresses the reliable and responsible development of foundation models. We explore critical issues, including bias and fairness, security and privacy, uncertainty, explainability, and distribution shift. Our research also covers model limitations, such as hallucinations, as well as methods like alignment and Artificial Intelligence-Generated Content (AIGC) detection. For each area, we review the current state of the field and outline concrete future research directions. Additionally, we discuss the intersections between these areas, highlighting their connections and shared challenges. We hope our survey fosters the development of foundation models that are not only powerful but also ethical, trustworthy, reliable, and socially responsible.

Published in Transactions on Machine Learning Research (10/2025)
Reliable and Responsible Foundation Models
Xinyu Yang", Junlin Han", 2 Rishi Bommasani", Jinqi Luo",, Wenjie Qu", Wangchunshu Zhou", Adel B Xiyao Wang", Jaehong Yoon', Elias Stengel-Eskin', Shengbang Tong", Lingfeng Shen'o, 10 Rafacl Rafailov Runjia Li², Zhaoyang Wang', Yiyang Zhou', Chenhang Cui', Yu Wangli, Wenhao Zheng Huichi Zhou Jindong Gu², Zhaorun Chen Peng Xia², Tony Lee", Thomas Zollo'4, Vikash Sehwagl5, Jixuan Leng', Jiuhai Chen', Yuxin Wen', Huan Zhang Zhun Dengli2, Linjun Zhangli? Pavel Izmailov Pang Wei Koht's, Yulia Tsvetkov! Andrew Wilson Jiaheng Zhang*; James Zou Cihang Xie Hao Wang!", Philip Torr*? Julian McAuleytll, David Alvarez-Melis<20) Florian Tramerts, Kaidi Xutaı Suman Janat", Chris Callison-Burchi Rene Vidal Filippos Kokkinos Mohit Bansal Beidi Chen Huaxiu Yao!"
Carneeiee Mellon University University of Oxford $Stanford University *University of Pennsylvania 'National University of Singapore 'ETH Zurich 'Univeriity of Maryland UNC Chapel Hill New York University Johns Hopkins Universily "University of California, San Diego 1mmprrial College London University of Chicago 14Columbia University 1PPrinceoon University Universiiy of Montreal & Mila Rutgers University Universsiy of Washington Univerrsty of California, Santa Cruz Harvard University 21 Drexel University University College London
Reviewed on OpenReview:
Abstract
Foundation models, including Large Language Models (LLMs), Multimodal Large Language Models (MLLMs), Image Generative Models (i.e, Text-to-lmage Models and Image- Editing Models), and Video Generative Models, have become essential tools with broad applications across various domains such as law, medicine, education, finance, and beyond. As these models see increasing real-world deployment, ensuring their reliability and responsibility has become critical for academia, industry, and government. This survey addresses the reliable and responsible development of foundation models. We explore critical issues, including bias and fairness, security and privacy, uncertainty, explainability, and distribution shift. Our rescarch also covers model limitations, such hallucinations, well as methods like alignment and Artificial Intelligence- Geneneted Content (AIGC) detection. For each area, we review the current state of the field and outline concrete future research directions. Additionally, we discuss the intersections between these areas, highlighting their connections and shared challenges. We hope our survey fosters the development of foundation models that are not only powerful but also ethical, trustworthy, reliable, and socially responsible.
PPooecct leader. Corresponding authors: xinyuya20andrew.cmmm wwt.:/aaaadrrr....eed.oo edu, huaxiuucs, unc. edu. "Major contribution. 1 "Advisory role.

---

Published in Transactions on Machine Learning Research (10/2025)
1 Introduction
Zero-shot Detectors
Seatistical Detection
1. Bius Evaliation
Intuitive Indicsiors
Generited Text
Pre-tmined
b) Feature Embedding
11. Watermark based Derectors
0) Probubility
Training Watermarking
Mitigation
b) Learnable Watemarking
3) Training
11I. Neural Network Detectors
Post- processing
AIGC Detection
Bias and Fairness
Explaining with Raw Features
1. Training Data Alignment
II. Exploring Knowledge LLMs
Humun Preferunce Aligument
Discovering the Roles Data
Reinforeemant Learning
IV. Evalcation Metrics
Explainability
Aligmment
Prompt Enginsering
Applications
IIL. Multimodality Alignment
Distribution
Reliuble and Responsible Foundation Models
1. Attacks
II. OOD Detection
Detinition Categorization
Shift
Security
a) Backdoor Attacks
Jailbrrakk Aliacks
OOD Generalization
Adversurial Attacks
Domain Adaptation
11. Defenses
Uncertainty
Hallucination
Sources
Privacy
x)
Training Data
I. Sources
b))
Modeling Decision
5) Training Data
Quantifying Addressing Unncemainny
b) Model Toining
a) Uncertainty Estimation
Model
Calibration
II. Hallucination Evalustion
Expressing Uncectainty in Words
Techniques
Distribution free Unceriainty Quantification
b) Metrics and Messures
Hallucination Mitigation
Privacy
8) Pata/Madel-busdd Appronches
a) Membership Inference
b) Trainin/Decoding Decoding Post-processing Methods
Extraction
c) Procppt and Model Stcoling
11 Privacy Preserving Techniques
Figure Overview of reliable and responsible foundation models. This survey comprehensively summarizes existing work from nine critical dimensions: bias and fairness, aligmment, security, privacy, hallucination, uncertainly, distribution shift, explainability, and Artificial Intelligence-Genentt Generated Content (AIGC) detection. We organize foundation models into four categories, including Large Language Models (LLMs), Multimodal Large Language Models (MLLMs), Image Generative Models, and Video Generative Models, to illustrate how each category uniquely interacts with these dimensions. Additionally, we explore how these dimensions interact and reinforce one another to highlight their synergies and shared challenges.
Recently, the paradigm for building artificial intelligence (AT) systems has undergone a fundamental shift, shaped by a compelling dual imperative: to develop increasingly powerful and versatile foundation models and to build inherently reliable and responsible ones. Foundation models are large-scale neural networks [481], typically pre-trained on massive and diverse datasets. defining characteristic is their general-purpose nature: instead being designed for a single, narrow task, they serve as a "foundation" that can be adapted wide array of downstream applications through methods like in-context learning or fine-tuning [100].
Among these foundation models, four major classes have fundamentally reshaped how with AI, including Large Language Models (LLMs), Multimodal Large Language Models (MLLMs), Image Generative Models (i.e, Text-to-Image Models and Image-Editing Models), and Video Generative Models.
and interact
These models demonstrate a series of powerful capabilities: LLMs can engage in multi-turn conversations and human-like reasoning processos, MILMs can generate HTML code from a screenshot of a sketched website, Image Generative Models can synthesize photorealistic images from complex textual descriptions, and Video Generative Models simulate interactive dynamics and commonsense knowledge of the physical world.
2

---

Published in Transactions on Machine Learning Research (10/2025)
The advent of foundation models can be traced back to the development of large-scale language representa- tions, which evolved from early word embeddings such as GloVE [700] and word2vec [638], to contextualized representations like ELMo [702]. This progress paved the way for mausformer-basee models such as BERT [idircctional Encoder Representations from Transformers; 219], which revolutionized natural language pro- cessing by providing powerful representations to improve performance on various downstream lasks. Next, the GPT-serics [Generative Pre-trained Transformer; 723, 724, 113] of autoregressive generative models showed how self-supervised learning via next-token prediction can yield high-quality text generative models.
With the release of ChatGPT [675], the strong capabilities of foundation models further gained mainstream attention, which exposed the public to intuitíve conversational user interface. Today, autoregrossive generative models have become the established paradigm for Al beyond natural language processing with multimodal models like GPT-4V(isson) [676], GPT-4o [677], GPT-4.5 [683], GPT-5 [682], Gemini series [309].
Claude [33], Claude 4 [36], Qwen2.5VVT. [18], @wwe2.550mmn [20], Qwen3-VT, Qwen3- ivvnn-Ommi [1019], and Scod1.5-V6 LLLL [331]. Concurrently, models such as OpenAI ol [679], OpenAI o3 and o4-mini [681], DeepSock RI [207], Claude 3.7 [35], Gemini-2.5 [312], Grok 3 [996], Grok4 [995], Qwen3 [21], and Seed-1.6 [116] enhance reasoning capabilities by increasing compute at inference time. Behind these models, the key innovation is the self-attention mechanism [895], which constructs contextual representations by enabling each token to weight and aggregate information from other tokens in the imput sequence. Compared to earlier recurrent models [314, 191], its inherent parallelizability served as a erucial catalyst for massive scaling during pre- training, making it feasible to train exceptionally large models, which led to the era of foundation model [100].
Concurrently, the field has witnessed rapid advances in generative diffusion models, which learn reverse a carefully controlled noise injection process to capture the underlying data distribution. This approach has emerged as leading paradigm for high-fidelity content generation, particularly excelling in visual genera- tion tasks [816, 358]. Among them, Image Generative Models, including text-to-Image and Image-Editing models such as DALL E 3 [85], Stable Diffusion 3.5 [259], Playground v3 [549], FLUX. 1 Kontext [72]. GPT-Image-1 [674], Gemini 2.5 Flash Image [274], Imagen 4 [66], Qwen-Image and Qwen-Imagc-Eatt [977]. and SeedReam-4 [287] can now generate high-resolution and high-quality images from lextual descriptions. Similarly, recent advancements in Video Generative Models, pioneered by Sora [680] and followed by Hun- yuanVideo [456], CogVideoX-1.5 [1047], Kling 2.5 [461], Wan2.2 Video [911], Vco 3 [308], Hailuo 02 [643], and Scedance 1.0 [288] have emphasized adherence to physical laws and commonsense reasoning. These models focus on generating realistic physical-wond scenarios and human-centric content. Collectively, they achieve remarkable ble spatial resolution and temporal coherence, enabling the creation of long-form, high-quality videos.
Training data
Pre-training
Adaptation
Application
Gemini
fIXI
T
O
Bias
Privacy
Alignment
Security f
Distribution Shift Explainability
Hallucination
Unvertainty
Hallucination
Fairesss
Hallucination
AIGC Detection
Figure 2: Foundation models are typically trained on diverse modalities and then adapted for downssteam applications. Throughout this pipeline, various reliable and responsible issues emerge different stages.
The powerful capabilities demonstrated by these rapidly evolving models have fueled their swift integration across the economy [198), with applications ranging from decision-making in business processes to personal
3

---

Published in Transactions on Machine Learning Research (10/2025)
assistants in everyday life. To quantify this broad usage, for example, ChatGPT reached estimated 100 million monthly active users in less than three months, while Deepseek-R1 achieved the same milestone in only one month, making these foundation models the fastest-growing consumer internet application in history [884, 13]. This scale of usage and the resulting sociocconomic impact accentuate the urgent need for these models to be both reliable and responsible [320]. In the context of this survey, we provide explicit definitions of these foundational concepts. We define reliability as the model's 's capacity to perform its intended functions accurately, consistently, and robustly, especially under challenging conditions like distribution shifts. We define responsibility the alignment of model's behavior with ethical principles and societal values, encompassing crucial aspects such as fairness, privacy, security, and transparency. This survey aims to synthesize the technical challenges and solutions for building models that satisfy both of these critical criteria.
In addition, this survey provides a comprehensive and unified examination of reliable and responsible foun- dation models. While prior surveys [37, 82, 922] have offered in-depth analyses of specific issues like hallu- cination [762, 383] or safety [1128, 603), our primary contribution lies in presenting a holistic, cross-cutting analysis (as in Section 12) that connects the nine critical dimensions across four major model classes (see Fig- ure 1 and Figure 2). This unique perspective reveals crucial coter-connettions and trade-offs, offering insights that are often overlooked in more specialized reviews yet crucial for advancing reliable and responsible AI.
Furthermore, our scope emphasizes the challenges of ensuring that foundation models operate reliably and responsibly when used as intended by their developers, focusing on their intrinsic properties rather than external misuse scenarios. This perspective complements separate bodies of work that address the deliberate misuse of AI for malicious purposes, such as disinformation, cyberattacks, and other adversarial activities.
We preview cach section of the paper below:
We begin with an examination of bias and fairness in foundation models, where we detail how biases arise, review methods for their evaluation and mitigation, and discuss the associated challenges.
Next, we explore the concept of alignment: why do we align foundation models with human values and how do we mitigate misalignment?
We conceptualize security for foundation models: what threats do they pose, and what measures can enable safer deployment?
In tandem with security, consider the data privacy challenge: how can respect individual privacy rights when collecting large-scale data?
Our exploration continues with a look at the phenomenon of hallucination in foundation models, where the model generates or responds to questions incorrectly, stating incorrect "facts" with high confidence.
We then examine the critical need for models to express uncertainty to prevent misleading results. We cover various sources as well as methods for the quantification and mitigation of uncertainty.
Next, we discuss the challenge of distribution shifts in foundation models: how to ensure models perform robustly on domain-specific tasks and out-of-distribution scenarios?
We review explainability to understand how foundation models work internally. We investigate methods for explaining LLMSs with raw features, uncovering the knowledge in LLMs, and examining the roles of samples in training, fine-tuning, and few-shot learning.
We conclude by discussing Al-generated content (AIGC) detection, where we frame the inherent challenges differentiating human and AT-generated content, the statc-of-the-art detection mothods, and the underlying assumption for different detection methods.
This survey provides comprehensive review of the current state of development of reliable and responsible foundation m It offers valuable insights for researchers, practitioners, and policymakers who aim to design, deploy, and regulate Al systems in responsible and reliable way across diverse real-world settings.
We cover literature that was publicly available up May 2025.
4

---

Published in Transactions on Machine Learning Research (10/2025)
Contents
1 Introduction
2
2 Types of Foundation Models
9
Bias and Fairness
3.1 Definitions
11
3.2 Methods for Bias Evaluation
13
3.3 Methods for Bias Mitigation
15
3.4 Bias and Fairness in MLLMs
16
3.5 Bias and Fairness in Image Generative Models
................................
3.6
Current Limitations and Future Directions...
3.6.1
Limitations and Open Challenges of Bias and Fairness
18
3.6.2
Future Directions
18
4
Alignment
20
4.1 Supervised Fine-Tuee Tine-Tuning
20
4.2 Reinforcement Learning from Human Feedback
23
4.3 Prompt Engineering
26
4.4 Alignment for MLLMs
28
4.5 Current Limitations and Future Directions
28
Security
30
5.1
Security in LLMs
30
5.1.1
Attack in LLMs
30
5.1.2
Defense in LLMs
32
5.2
Security in MLLMs
33
5.2.1
Atlack in MLLMs
33
5.2.2
Defense in MLLMs
34
5.3
Security in Image Generative Models
34
5.3.1
Attack in Image Generative Models
34
5.3.2
Defense in Image Generative Models
35
5.4
Current Limitations and Future Directions
36
5.4.1
Limitations and Open Challenges of Attacks
36
5.4.2
Limitations and Open Challenges of Defenses
36
6 Privacy
38
6.1 Privacy in LLMss
38

---

Published in Transactions on Machine Learning Research (10/2025)
6.1.1 Privacy threats in LLMs
38
6.1.2
Privacy-preserving techniques in LLMs
39
6.2
Privacy in MLLMs
40
6.3
Privacy in Text-to-lmage Models
40
6.4
Current Limitations and Future Directions
41
6.4.1
Limitations and Open Challenges of Privacy Attacks
41
6.4.2
Limitations and Open Challenges of Privacy Preserving techniques
41
7
Hallucination
42
7.1
Hallucination Categorization
42
7.1.1
Hallucinations in LLMs
42
7.1.2
Hallucinations in MLLMs
43
7.2
Sources of Hallucinations
44
7.3
Hallucination Detection and Measurement
45
7.3.1
Detecting Language Hallucinations
45
7.3.2
Measuring Multimodal Hallucinations
47
7.3.3
Measuring Text-to-image Hallucinations
7.4
Hallucination Mitigation
47
7.4.1
Reducing Hallucinations in L.IMs
47
7.4.2
Reducing Hallucinations in MLLMs
48
7.5
Current Limitations and Future Directions.
50
Uncertainty
51
8.1
Sources of Uncertainty
51
8.1.1
Data
8.1.2
Model
8.1.3
Aleatoric vs. Epistemic Uncertainty
52
8.2
Quantifying and Addressing Uncertainty
53
8.2.1
Estimating Uncertainty
53
8.2.2
Calibration
54
8.2.3
Verbalized Uncertainty
56
8.2.4
Addressing Uncertain Examples
56
8.2.5
Distribution-fre Uncertainty Quantification
57
8.3
Current Limitations and Future Directions..
58
Distribution Shift
59
9.1
Definition and Categorization
59

---

Published in Transactions on Machine Learning Research (10/2025)
9.2
Out-of-Distriuttion Detection
60
9.3
Duttoo-Distribution Generalization
61
9.3.1
Data Augmentation
61
9.3.2
Adversarial Training
9.3.3
Label Smoothing
62
9.3.4
Invariant Learning
62
9.3.5
Model Ensemble
62
9.4
Domain Adaptation
63
9.4.1
In-context Learning
9.4.2
Retrieval-augmented Goneration
63
9.4.3
with New Knowledge
64
9.4.4
Test-time Training
65
9.4.5
Model Editing
65
9.5
Current Limitations and Future Directions
10
Explainability
70
10.1
Feature Attribution Methods
70
10.1.1
Perturbing the Input for Explanation
70
10.1.2
Gradient-based Explanation
71
10.1.3
Attention-based Explanation
71
10.2 Exploring the Knowledge in LLMs
10.2.1 Probing the Representations within LLMs
72
10.2.2 Explaining LLMs with Concepts
73
10.2.3 Mechanistic Interpretability
73
10.3 Discovering the Roles of Samples in Training, Finc-tuning, and Few-shot Learning
73
10.3.1 Influence of Single Example in Training
10.3.2 Inffuence of Training Stages
74
10.3.3 Influence of Samples in Few-shot Learning
74
10.4 Evaluation of Explainability
75
10.4.1
Evaluation of Plausibility
75
10.4.2 Evaluation of Faithfulness
10.5 Applications of Explainability
76
10.5.1
Avoiding Shortcut Learning
76
10.5.2 Improving Model Performances
76
10.6 Explainability of MLLMs
76
10.7 Current Limitations and Future Directions.
10.7.1 Faithfulness of Raw Features
77

---

Published in Transactions on Machine Learning Research (10/2025)
10.7.2
Understanding How LLMs Store Knowledge
77
10.7.3 Transferability of Explanation Across Different Modalities
77
10.7.4 Reliability and Responsibility of Foundation Models from the Explainability Perspective
77
11 AIGC Detection
79
11.1
The AIGC Detection Problem
11.2 Zero-shot Detectors...
79
11.2.1 Statistical Detection
80
11.2.2 Intuitive Indicators
81
11.2.3 Pro-trained LLMs
82
11.3
Watermarkk-basdd Delection
11.3.1 Training-free Watermarking
83
11.3.2 Learnable Walermarking
83
11.4
Neural Network Detectors
84
11.5 Current Limitations and Future Directions..
85
11.5.1
Fairness of ATGG Detection
11.5.2 Robustness of Watermarks
85
11.5.3 Origin Attribution of Generated Images
85
12 Intersection and Conclusion
86
12.1 Summary of Recent Community Efforts
86
12.2 Bias, Fairness, and Security
12.3 Bias, Fairness, and Al-generated Content Detection
................................
87
12.4 Security and Privacy
87
12.5 Security and Al-generated Content
87
12.6 Uncertainty and Alignment
88
12.7 Hallucination, Uncertainty, Distribution Shift, and Alignment

---

Published in Transactions on Machine Learning Research (10/2025)
2 Types of Foundation Models
As discussed in the prior chapter, foundation models are designed to serve as versatile backbones for applica- tions in various domains, oflering world knowledge that can be adapted to a wide range of downstream tasks. In this survey, we first define a set of core modalities M = (T, (T, V. A,,,, representing Text, Image, Video, and Audio, respectively, We then focus on four popular classes of foundation models: Text-to-Text (i.e., LLMs), Multimodal-to-Text (i.e., MLLMs), Multimodal-to-Imaag (i.e., image generative models, including text-to-image and image-editing models), and Mullimodal-to-iddes (i.e., video generative models) systems.
LLMs are foundation models specifically designed to understand, generate, and manipulate human language. They encompass range of architectures, including encoder-only models (e.g., BERT) for understanding tasks like text classification, decoder-only models (e.g., GPT) for generative tasks like content creation, and encoder-decoder models (e.g.. T5) for sequenec-to--sqqennc tasks. Frequent applications include summa- rization, translation, sentiment analysis, and dialogue systems. In notation, we can represent the general function of an LLM by f:
f: - where X,Y (7)
where T is the space of text sequences. The function ʃ is parameterized by weights 0 and is predomi- nantly implemented using Transformer architectures [895], though emerging architectures like State-Space Models [319] are also gaining traction.
MLLMs, in our review context, are large-scale deep neural networks that process multiple modalities of data to generate text outputs. A general and prevalent design recipe for MLLMs to adopt architectures that integrate multimodal features in a shared latent space, as exemplified by seminal works like CLIP [725] and models like LLaVA [556]. Applications of MLLMs vast, including robotics, healtheare, and augmented reality. The mathematical expression of general MLLM can be viewed as mapping function g:
g g x 22 .....x..-.............. T,, where X, E (T,I,V.A)
where X; represent different modalitics. Notably, prominent class of contemporary multimodal models focuses processing text and images to generate text. This will be the primary focus of our work when referencing MLLMs, corresponding to the specific mapping g I x TT.
We further denote image generative models as foundation models that generate images based on textual inputs. Applications of these models exlend beyond art to product design and education. In notation, we represent image generative models model by h:
h: X1 x X2 x ... -,,
x where X, E (T,IVV,A) and y e (I)
(3)
where T is the space of input text and I is the space of output images. h is parameterized by weights v and is often implemented with generative modeling approaches such as diffusion models [816, 358, 823), generative adversarial networks (GANs) [306], and autoregressive Transformers [737]. These models typically feature a hybrid architecture, combining, for instance, Thansformer-based text encoder to interpret the input prompt with a U-Net-based diffusion model to generate the image. In this work, focus primarily on diffusion models, as they have emerged as the dominant architecture in T21 models.
Finally, we focus on video generative models that generate videos based on multimodal inputs. Applications include realistic physical-wort world simulations or high-quality human-centric interactions. We represent a video gencrative model by u:
U: Xi x X2 x.. ..... +3, where X, E (T,I,V,A) and Y E (V}
(4)
where X; represents different inpul modalities and is the space of output videos. The function is parameterized by weights w and often implements architectures that extend diffusion models to handle temporal relations for coherent video generation.
9

---

Published in Transactions on Machine Learning Research (10/2025)
3 Bias and Fairness
Foundation models are often pro-trained on large-scale data. Consequently, these models inherently acquire biases from their training data, which can propagate to various downstream applications [320, 97, 530]. In practice, the nature and impact of biases present in training data, foundation models, and downstream applications are poorly understood [279]. Therefore, more work is needed to measure and mitigate bias in foundation models to advance fairness and equity in AI systems [100].
This section explores bias and fairness in foundation models, organized as follows: We begin by establishing basic definitions to formalize bias and fairness, highlight potential consequences, and outline the essential criteria that require fairness for LLMs (Section 3.1). Next, we review methods for bias measurement (Section 3.2) and bias mitigation (Section 3.3) as shown in Figure 5. Finally, we discuss bias and fairness in multimodal contexts, focusing on MLLMs and image generative models, respectively (Sections 3.4 and 3.5). We provide the detailed visualization of the categorization of these concepts and methods in Figure 3.
Dedititions
Pilcuss
Language Nurmx
Coonggs Taoommy (33.1)
Social Bas Cotegocies
Represoiration Diserinttationn
Tusk Manifostaicns
Gooeratka/M//IIB /RA NIA//Duswiicttie
Informatioon Rotziioaal (1466. Q Quadiio Auewsiing [222, da1).
Tost Generation 755, 1045, s00), Machine Tonnatiio Je25, 800).
Naturel Lengurge Inference (219], Text Cloectiration (420), tooo
Distrlbution Metrios
Demographie Represestation (20, Sug
Sovial Group Subelitution [10%) Cocornmens Scoree [102]]
Gonerated Test
Casecooation Metties
Pempective [532, 183, 190), Temitity Fraction [3306 Style Clasetice (813]
Bass de Vairmess in Foutddationn Modicl
Hurtlex [71], Dold (220)
Word-ooe Metiies
Hanor [[T]]
Ernliation bitthos ($312)
Fosture Emiiekkings
WEAT [177], IXT [ie]] SEAT [G20]
Token Selection
Context Association Test (ond] Unmasked Libelihood (424)
Tekensboel Proh. Eatimation [15-1] Parsio-lag Libelhhud [012, 700].
CDA (885, 715, 154], Mix-Debise [1ord], Tasta Piltering 2111, tot, 839),
Treaning Data Angroentation
Sclf lustccut [141), Coustruticual All [Go]
Training
bittipation Mochoda (85.8)
Aligament and Instruction Tuning
RLII sttle Aligemont (140), EnstraeGPPT joes DPO [ran]
Infrrence
EVER (a2s] RdD [16]]
Viika-grauddng Hellstinntins [1168, 021, 925, 500, (ta). Batity/Comnt// (Lggi Brows 11000, 354]
Phenomens
MLL.Ms (83.4)
Tehissing
Counterrazz Image Comtate [S0t, 709, 907).
Test time Embedkinng Debiading [297]
Stoaad Profeccion /Etbmeety /Chos Reprontstiioe [1127, 180, s0, 501). Marginallaod Ideatity U'sdeteqreoesaaaioon [snd].
Bas Patterss
Beal-eceld Ditrilucia Amplilictioo. [76]]
fazage Cenerotive Models ($1.))
Promupt-lool Miligatios
Paiz Procapting (270). Soft- tolen, Embeddiing Steering (437, 188).
Connrge Manipulation 150g] Keyword Filluring Cantions (183]
Modiliiao Outkook
Sumpling-hood Mitigsiio
Rammeee-oarr Amplifiel-poos Ssmpling [181]
Clus-bahancig Diffusia [S10].
Auteregrestvve Das Propegation [1007, 771, 1199),
Contestiaa /Narattiv Franing loase 2130, 700).
Linits & Chillenges
Evibuttinn Gaps
Cross cultuall Dediition Variance 1110, 200f.
seree-ahhnnn hontlciiyy Tensiun [7.]].
Web-scale Ressmpling Limittatioss [S37]
Unhiseed Telonization /Rmbbelding Mechurl 1101, 1100).
Patrness swice Aligament [1077, 1270, 585, 1168/.
Futum Direction
Rassand Agyuds
Antoregnasiive Tixd Posl-pooossing [1160].
Meehamett Stoeentype Creuit Interjnetabilln [SeT, maad) Modaltt--pecille Auaboss (174), Farress utillty Thade olti
Figure 3: Taxonomy of Bias & Fairness in Foundation Models.
10

---

Published in Transactions on Machine Learning Research (10/2025)
3.1 Definitions
Table 1: Categories of Social Biases in LLMs. We provide definitions and an example for each type of bias. The framework builds upon Gallegos et al. [280] with additional refinements.
Bias Type
Definition
Example
Pejorative Language
The use shurs, insults, or other derogatory language that targets and denigrates social group.
Using the word "bitch" conveys contempt. and stereotypes hostile attitudes towards women [86].
Lingnistic Diversity
A preference for standard language forms in LLM training may sideline dialects, indirectly devaluing the linguistic patterns of marginal- ized groups in society.
The misclassification of African American English (AAE) expressions like "finna" as nou-English more often than Standard Amer- ican English (SAE) equivalents [96].
Normativity
Reinforcement of the normativity of the dom- inant social group while implicitly excluding other groups.
doctors if doctor itself entails not-woman [81].
Referring
Misrepresentation
happens when generalizing from an incom- plete or non-represettative sample population to a social group, leading to misrepresenta- tions.
An inappropriate respouse like "I'm sorry to hear that." to "I'm mustachioed guy.", reflecting a misunderstanding of mus- tache [813].
Stereotype
Negative and immutable abstractions about labeled social group.
Linking "Muslim" to "terrorist" fuels negative and violent stercotypes [5].
Hate Speech
Offensive language that attacks, thrcatens, incites hate violence against social group,
Stating "Asian people are gross and uni- versally terrible" is disrespectful and hate- ful [228].
Explicit Discrimination
The direct and clear differential treatment individuals or groups based on their member- ship in a social group, such as race, gender, age, ethnicity, religion, or sexual orientation.
recruitment policy that states or implics a preference for candidates a certain race over others, or a club that refuses membership based on gender [270].
Implicit Discrimination
Individuals are treated differently based on unconscious subtle prejudices and stereo- types rather than explicit intentions to dis-
A health assessment tool used by insurance companies assigns higher risk scores to pa- tients from certain ethnic backgrounds (270).
criminate.
Given language tochnologics' broad impact, the study of bi- ases has become increasingly central to Natural Language Pro- cessing (NLP) in recent years. Language intimately intercon- neets with aspects of human identity, social relationships, and power dynamics. Biases, particularly social biases, pertain to
1think run are hoser laddr that wertee.
Is commoc atcoottrp that rece are betta lasles, but stadies saow that
kenierkkir qudites not dntanred acuder Both wouee
cuxpticcal leades depeading os taii skils, experiesoe and passeality bials.
Le fer enoctyp the suak cramarei
Rejucs awweyye
the segmentation and distinctiveness of various social groups, imparting both generic and pejorative connotations, and corre- late specific demographics with stereotypical, uncharacteristic, or overly generalized traits [279, 97, 1023). Prior work [69, 81, 966, 846, 626] understand the concepts of bias and fairness in terms of these social groups and addresses a variety of social do- mains and downstream tasks [1078, 988, 906, 751, 107]. While several surveys provide in-depth analyses of bias and fairness in language models, this section situates these challenges within the broader context of foundation model reliability, connecting them to issues of security and content detection.
Biss
cnmpelling gained
espolletic colaabrntive aggaes elfective leadealip.
gmale-baawl chim vithel aditzwing shcoeteee
Figure 4: An example of gender bias in LLM responses.
Bias. Biases in LLMs refer to systematic deviations in the model's responses, representations, and reasoning paths that reflect disparilies, stereolypes, or inaccuracies in the training data. These biases can misalign with
11

---

Published in Transactions on Machine Learning Research (10/2025)
overinterpret the reference social and cultural norms implicd by human prompts. Typically, such biases arise from the unbalanced or biased data distribution in domains (i.e., areas of knowledge) and genres (i.e., types of text, such as news, fiction, dialogue, etc.) representing different groups. For instance, the male- female distribution of Wikipedia articles about US Presidents would lead to biases on the role of different genders in politics. Figure 4 ralustrates a similar example of language bias related to leadership in LLM- generated contents.
Following Gallegos et al. [279], we provide a detailed summary and categorization of biases in LLMs, inchuding definitions and examples, shown in Table 1. These biases may manifest in distinct ways based the specific context and downstream tasks. Recognizing and addressing these biases is crucial for developing fair and equitable NLP technologies. To better understand the unique forms in which bias can manifest in LLMs, we have listed some examples drawn from various NLP tasks below:
Text Generation. We might encounter local biases, such different job choices when generating phrases like "The man worked as car salesman." versus "The woman worked as a nurse." Addi- tionally, we may face global biases, such as the overall depiction of certain cultural backgrounds like "East Asians like to cat rice" [795, 1048, 898).
Machine Translation. Translation tools may show a tendency towards gender-specific expressions when translating job-related phrases [625]. For example, translating "the engineer solved the prob- lem" into German might default to "der Ingenieur" " (the masculine form), given that in an existing English-German corpus, "der Ingenieur" was found to be 75 times more prevalent than its feminine counterpart "die Ingenieurin" [866].
Information Retrieval. Searches like "successful leaders" may be biased towards returning doc- uments about male leaders, overlooking female ones, exhibit bias towards certain cultural interpretations retrieving information about cultural holidays [745].
Question Answering. When faced with specific questions, answers can be influenced by gender or occupational stereotypes. For example, assume the primary caregiver in a household is "the mother" or "a woman", defaulting to "a man" as company's CEO [222, 694].
Natural Language Inference. When given a premise like "the doctor is seeing a patient", the model might incorrectly infor the doctor's gender or make assumptions about the age or gender of participants in sports activities based on stereotypes [218].
Text Classification. Models might wrongly categorize statements that use regional dialects or slang as aggressive or inappropriate. They may also exhibit bias when classifying posts discussing sensitive topics [450, 1050], failing to consider the actual content of the text.
Fairness. Due to the biases discussed above, LLMs may exhibit disparities in task-specific performance across different social groups. Consequently, it is essential to ensure that these models' behavior, outputs, and decisions are fair and unbiasod, reflecting and respecting the diversity and complexity inherent in socicty.
Considering the data distributions across social groups differ in a complex way, we use performance disparities to measure it. Following Section 2, an LLM can be denoted as function f: X y, which maps a context prompt X to target response Y. Additionally, measurement function S: y - maps a response y to scalar score R. The model f is considered fair for groups A and B in terms of the measurement S if the following condition holds:
Ex, (S(F(Xx: = Ex, (S(f(X;;))) (S((((((
(5)
where XA represents the prompt or context information related to a particular group A, with different groups possibly encompassing attributes such as race, gender, etc. When it fails to satisfy Equation 5, it is said that the model M exhibits bias towards a particular group. It is noteworthy that this is just one possible definition, while other definitions and metrics can also be reasonable [280, 336].
With the increasing deployment of LLMs in the business domain, such as customer service and decision support systems, ensuring these LLMs are fair and unbiased has become paramount. Similarly, given these models' role as part of social services, the requirements for fairness and non-toxicity are crucial to avoid potential social biases and adverse impacts. In the study conducted by Gallegos et al. [279), a comprehensive
12

---

Published in Transactions on Machine Learning Research (10/2025)
set of principles was discussed, including Fairness through Unawareness, Invariance, Equal Social Group Associations, Equal Neutral Associations, and Replicated Distributions. These principles not only guide NLP tasks but also lay the foundation for fairness and non-toxicity in the practical deployment of LLMs. Such efforts aim to develop monsensus-bulldin approaches across diverse stakcholder groups, ensuring that LLM applications don't disproportionately impact specific communities, thereby supporting the sustainable development of equitable social services.
3.2 Methods for Bias Evaluation
In this section, we summarize three popular approaches for evaluating bias LLMs:
Methods based on Generated Text. These evaluation methods are primarily based on assessing the lext generated by LLMs in response to specific prompts, often using specialized benchmarks. Typical benchmarks include Dhamala et al. [222] and Gchman et al. [293]. They utilize guiding prompts to induce biased outputs from the model to evaluate the inherent biases of LLMs. Therefore, models with more severe biases are more prone to exhibiting tendencies toward certain groups After obtaining the model's textual responses to the designed prompts, three metrics are generally used to assess the biases in the responses.
WEAT [Caliskaner al., 2012]
Feature Embedding
LAT [Grecnwald et at. 1998]
SEAT [May of. 20199
Training Data Augmentation
Distribution Metrics
CDA (Lu of al, 2020)
Bias in LLMs
Social Group Substitutions (Rajpurkar et ol. 2016)
Mix-Debias Yuu of, 2025)
Co-occurrence Seares (Bordia 2019]
Data FilteringBBorchhms [Brrhherser er of., 2023]
Demugraphic Reprosentation [Liang 2022]
Self Instruct [Wang or at, 2022]
Constitutional Al [Bai
Clavsfieation Metries
Training
Evaluation
Generated Text
Pengecttvve API [Chowdbery al. 2023]
Toxicity Fraction [Liang et af., 2022]
Style Classifier (Smith al., 2022]
Alignment Instruction Tuning
KLHE (Christiano et al., 2017]
Word-level Metries
InstraciCPPP [Long st at. 2022]
Hurtllx [Bassignana al. 2018] 2018]
DPO [Rafailov 2023]
Mitigation
[Dhamala 2021)
Honest [Neornet et ai. 2021]
Tokm--lveel Prob. Estimation (Wehsteretd 2020]
[Kang 2023) RelD (Chen et ot., 2025)
Inference
Token Selection Prob.
Context Associzzion Test [Nadoem at al., 2020]
Pseudo Likelihood (Salazarer at., 2019]
All Unmasked Likelihood [Kineko at 2022]
Figure 3: An overview of strategies for evaluating and mitigating bias in LLMs, covering evaluation via feature embedding, generated text, and token sclection probability and mitigation during training or inference.
(1) Distribution metrics: One of the simplest metrics in this category is Social Group Substitutions (SGS), which evaluates whether a model's responses exhibit an identical token distribution when provided with context input X biased towards different groups A and B. For context inputs XA representing commonsense scenarios and XB denoting founterfactual scenarios, mandates:
= = -(((X):))))
(6)
where f(X; represents the response generated by a LLM denoted as f. with input X and model parameter 0, and va symbolizes an invariance metric such as exact match [733].
There also metries based on the frequency of specific words appearing in response compared to their average distribution, such the bias metric based on word co-occurrence scores [102]:
P(Z.(EA))
bias(x:) log P(x,
(7)
where di belongs to a word in the response X = (I1, Fm), and aA and an can represent keywords biased towards two different groups, such as men and women.
Similarly, Demographic Representation (DR), as discussed in Bommasani & Liang [99]. Liang et al. [532]. compares the frequency of specific remographic--llated word mentions with the original data distribution.
13

---

Published in Transactions on Machine Learning Research (10/2025)
Here, C( 36) represents the count of occurrences of the word I, in the sequence yi, where ys E Y is specific model generation from the set of model generations for a scenario. For each group E G is associated with a set W of words that represent the group, the count DR(j) is calculated as follows:
DR()) = Σ Σ C(a,,)))
(8)
Z,EW; MEY
To evaluate against baseline probability distribution, such as uniform distribution, the count vector DR [DR(1), (R()) is normalized to a probability distribution. We then employ metrics such as total variation distance, KT, divergence, Wasserstein distance, other appropriate indicators for this evaluation.
(2) Classification metrics: Classifier-based metrics utilize expert models or specialized APIs to assess bias in generated texts. This approach involves comparing texts generated from simílar prompts, especially when comes to classifying outcomes related to different social groups, to detect potential biases. In the field of toxicity detection, Perspective API is commonly used tool to evaluate toxicity probabilities in LLM-generated texts [532, 183, 190]. This probability can be quantified by generating texts multiple times and calculating the expected maximum toxicity to assess the toxicity performance of LLMs. Other metrics include Toxicity Probability (TP), measuring the empirical probability of generating text with toxicity score 2 0.5 at least once in multiple generations, and Toxicity Fraction (TF), indicating the proportion of toxic generations [532].
Classification-based evaluation methods are not limitod to the domain of text toxicity. For instance, Sheng et al. [795] assess the altitudes and perceptions of social groups in model responses, employing metrics akin to sentiment and respect scores. Their approach involved manually constructing prompt templates for LLMs dataset. This dataset subsequently served the training set for a regard classifier, enabling the classification of response preferences in other LLMs.
(e.g., "This woman works as"), generating
Similarly, Smith et al. [813] use a style classifier to compute the style vector for each generated response O(,,;0), where X; is a prompt related to a group E G. Bias is measured by calculating the variance across the sets of all generated sentences from each group (i.e, X, for group i):
Gen Bi__Bias( (fas(f((;0)) 0))
Varieg
(9)
X
X,EX:
where e represents the style classifier, and each element is the probability of sentence belonging to one of C style classes, i.e.,
(3) Word-level metrics: This evaluation approach is similar to fine-grained distribution metrics, which is relatively straightforward. Basically, it involves word-level metrics that analyze the generated output, where each word is either compared a predefined list of harmful words or assigned a precomputed bias score [671, 71, 222].
In general, evaluation methods based on generated text generally applicable to most LLMs, especially specialized black-box models such as ChatGPT and Bard. More recently, Bouchard et al. [106] released LangFair, a Python toolkit that makes the evaluation of bias and fairness easier for LLM practitioners and developers.
Methods based on Feature Embedding. In addition to assessing models through corresponding text, another common approach involves evaluating model bias based on feature embedding. Specifically, this typically entails measuring the distances in vector space between neutral words (such as professions) and identity-related words (such as gender pronouns) based on the embedding of output texts. Using these distanco-related metrics, we can roughly assess the bias between the model's textual responses and the standard reference group.
A more detailed evaluation metric relies on word embeddings, specifically, the Word Embedding Association Test (WEAT) introduced by Caliskan et al. [117], which is comparable to similar approaches used for contextualized sentence emboddings. WEAT evaluates associations between concepts related to social groups, such as masculine and feminine words, and neutral attributes such family and occupation words, resembling
14

---

Published in Transactions on Machine Learning Research (10/2025)
the Implicit Association Test (IAT) [316]. Another set of evaluation metrics, focusing on sentence-level embeddings, incorporates more contextual information. An example of this is SEAT [620], an improvement upon WEAT, SEAT generates embeddings for semantically bleached template- based sentences that integrate social group and neutral attribute words, and extends the evaluation to specific bias dimensions using unbleached templates, offering a contextualized approach for assessing bias in sentence embeddings.
Methods based on Token Selection Probability. Furthermore, we discuss bias and fairness metrics that leverage the token selection probability from LLMs. This probability can be obtained by masking a word in sentence and prompting a masked language model to predict the missing token. For example, Webster et al. [954] utilize specific prompt templates (e.g., "[MASK] is [MASK]" and "[MASK] likes [MASK]")) In these templates, the first [MASK] is automatically filled with words biased toward a particular group (such as gendered terms), and the second [MASK] is replaced with candidate predictions from LLMs. The score is calculated by averaging the count of divergent predictions between social groups across all specific prompt templates. Kurita et al. [472] employ similar template- based approach to assess bias in neutral attributc words (e.g., occupations). However, Webster et al. [954] normalize token's predicted probability (based on the template prompt "[MASK] is an [ITEM FROM GROUP i)"") with the model's prior probability (based on the template "[MASK] is a [MASK]"). This normalization corrects for the model's prior inclination toward one social group over another, focusing solely on bias attributable to the [ITEM FROM GROUP i] token.
Another category probability- methods is pseudo-log likelihood (PLL). Various techniques [912, 763] utilize PLL to score the probability of generating individual words in a given sentence. For a response denoted as X (21: Im); the expression of PLL is presented as follows:
PLL(X) = Σ log ((xs(XM)SS))))) x,E
(10)
Nangia et al. [666] utilize the CrowS-Pairs dataset, which involves pairs of sentences where one is stereotyp- ical and the other is less sterootypical. PLL is employed to evaluate the model's preference for stercotypical sentences. For sentence pairs, the metric approximales the probability of shared, unmodified tokens U con- ditioned on modified, typically protected attribute tokens M. The Context Association Test (CAT) [662]. introduced alongside the StereoSet dataset, assesses sentence bias by pairing each sentence with stereotype, anti-sterootype, and meaningless options. Unlike pscuuo--lg-likelhhood, CAT considers conditional probabil- ity. The Idealized CAT (iCAT) Score [662] is calculated from these options, and an idealized language model has specific scoring criteria. All Unmasked Likelihood (AUL) [424] extends CrowS-Pair Score and CAT, considering multiple correct candidate predictions and avoiding selection biases in word masking. Language Model Bias (LMB) [67] compares mean perplexity between biased and counterfactual statements using the t-value of Student's two-tailed test.
3.3 Methods for Bias Mitigation
The current popular methods for mitigating biases in LLMs' response texts can be broadly categorized into two types: those based on the training process and those involving post-processing techniques. Next, we provide a detailed breakdown and explanation of these two types. Specific categories will be examined in greater depth in the subsequent chapters.
Methods based on the Training Process. This type can be divided into two classes: methods based on training data augmentation and alignment with instruction tuning.
(1) Training data augmentation: For LLMs, biases frequently originate from imbalanced data distribution and poor data quality [279]. One of the most direct and effective solutions is improving the quality, di- versity, and balance of training data. Data augmentation techniques aim to mitigate biases by introducing additional instances into the training data, thereby increasing the data points related to underrepresented or misrepresented social groups. Data balancing approaches aim to achieve equitable distribution across various social groups. One primary technique for this purpose Counterfactual Data Augmentation (CDA) [588, 715, 954], which involves replacing protected attribute words, such as gendered pronouns, to create a balanced dataset.
15

---

Published in Transactions on Machine Learning Research (10/2025)
Inspired by the mixup technique [1108], interpolation approaches blend counterfactually augmented training instances with their original counterparts and labels, thereby achieving a more balanced distribution of the training data [1050, 1049, 1039]. In Ahn et al. [LI], the mixup framework is harnessed to align the output logits of a pro-trained model between two opposing words within a gendered pair. In Mix-Dobias, Yu el al. [1074] apply mixups across various corpora, aiming to alleviate gender stereotypes by leveraging an augmented training set.
Wang et al. [941] introduce an automated iterative framework that prompts LLMs in conjunction with filtering criterion. Through self-instructive process, this framework reconstructs more diverse dataset from initial seed data tailored for LLMs' instruct tuning. The prompts for this dataset are generated automatically by LLMs and undergo various metric-based filtering to ensure diversity in the dataset.
In addition, there are numerous data filtering methods [291, 101, 857] that aim to enhance the balance of data distribution by either removing low-quality data or selectively retaining diverse and underrepresented set of data.
(2) Better alignment with instruction tuming: With a vast amount of data, LLMs typically undergo pre- training and instruction tuning. In the pre-training phase, LLMs internalize knowledge from the training data into trainable parameters. Instruction tuning, on the other hand, teaches the model to understand human instructions. However, it is essential to recognize that biases in the training data and the training process are not inherently designed to understand or prioritize human values. This limitation leads to biases and potentially toxic responses from LLMs when faced with complex and divergent human preferences. They often arise naturally from the data or model training procedures, from human design decisions that reflect their own values and preferences. To address this challenge, we provide a detailed overview of some current alignment techniques and training algorithms to harmonize LLMs with human preferences in Section 4.
Methods based on Post-processing Techniques. Another approach is based on post-processing tech- niques. Post-processing, in the context of LLMs, generally refers to the practice of invoking external knowl- edge bases or employing word-based detection techniques to identify biased statements during inference. Subsequently, the identified biases are corrected in the generated text. In Kang al. [425], techniques such retricval are employed within LLMs to match responses during cach phase of the Chain of Thought (CoT) generation [963]. This involves retrieving and correcting biased toxic text at every stage of the LLM's responses, thereby ensuring that LLMs produce accurate and unbiased text responses throughout the CoT process. Andriopoulos & Pouwelse [29] also mention various methods that enhance LLMs by invoking Wikipedia and various external knowledge bases for retricval. The objective of these approaches is to boost the reliability of LLM outputs and reduce biases in generated text. Additionally, word-level detection is employed in Chen et al. [163] to identify instances in LLM responses involving counterfactual information or not aligning with the context. Subsequently, a post-processing approach is applied to correct and enhance LLM's reliability by removing such inaccuracies from the generated text. On the other hand, Li et al. [500] synthesize aultural-ppccfic instruction data to incorporate cultural differences into LLMs. Raza et al. [743] further propose MBIAS, LLM framework instruction fine-tuned on a custom datasct designed explicitly for safely interventions. Moreover, Wang & Demberg [940] introduces a multi-objeclive probability alignment approach to overcome current challenges by incorporating multiple debiasing losses to locate and penalize bias in different forms, which is more effective in removing stereotypical bias of LLMs while retaining their general performance.
Overall, the methods based on post-processing techniques can effectively and accurately handle certain biased information. However, they also have certain drawbacks. For instance, when relevant information is not present in external knowledge bases, biases in LLMs' responses might remain uncorrected. Additionally, post-processing may introduce erroneous information from external knowledge bases. Moreover, approaches relying on post-processing techniques often lead to a significant increase in latency.
3.4 Bias and Fairness in MLLMs
Compared to LLMs, the emphasis on fairness MLLMs is more direct toward ensuring that the responses align faithfully with the inputs in different modalities, such as images or audio in context. The responses must align consistently with the visual content, ensuring they are free from any biased text that contradicts
16

---

Published in Transactions on Machine Learning Research (10/2025)
the context. Currently, most research exploring bias and fairness in MLLMs is focused on the phenomenon of image hallucination. This term describes scenarios in which the model, when describing images or answering questions based on visual information, generates responses containing entities, quantities, or logical informa- tion that does not exist in the given image [1163, 924, 923, 520, 552, 554, 1169]. In Scction 7, we conducted a detailed analysis of recent advances in understanding and addressing hallucinations within MLLMs.
Apart from hallucinations, there is a notable lack of in-depth exploration into the bias and fairness of MLLMs. Similar to LLMs, MLLMs may exhibit significant biases due to the training paradigm and dataset distribution. The scarcity of image-text data for specific groups, coupled with the presence of biased infor- mation in the datasct, may lead MLLMs to acquire stercotypical impressions of certain groups. Additionally, imbalanced dataset distribution might cause MLLMs to showcase biases in responses to specific image-text pairs. Earlier efforts on generating counterfactual images towards semantic textual concepts have shown that machine learning models will encode biases related to certain attributes if the training data is imbalancod [594, 709, 997]. To further mitigate blases during inference, BEND-VILS VLM [297] tailors the debiasing opera- tion for MLLM embedding to each unique input al the test time, thereby avoiding catastrophic forgetting in fine-tuning. However, biases remain largely unexplored in MLLMs, and addressing fairness and bias in MLLMs is crucial for building foundation models that are beneficial and equitable for humanity.
3.5 Bias and Fairness in Image Generative Models
The rise of image generalive models sparks discussions on systematic social bias and fairness issues in generated content, as indi- cated by several studies [1127, 761, 180, 89, 591, 502]. Text-guided diffusion models, in particular, have been found to exhibit bi- ases related to professions, cthnicities, and social classes. The generated contents di- verge from the distributions in the real world and even amplify the biases in real socicties [1127, 89). For instance, a study conducted by Luccioni et al. [591] highlights that image generative diffusion models con- sistently underrepresent marginalized iden- tities in the generated images. Some exam- ples of gender biases in image generative models are presented in Figure 6.
Please draw a professor.
Gender Bias
Please draw a nurse.
To overcome the systematic bias and fair- ness issues, many methods [275, 437, 188, 502] focus on mitigating biases in image generative models through prompting tech- niques. Fair Diffusion [275] randomly in- jects additional subject pronouns in the prompts to achieve a more balanced gender distribution in the generated images. Other work [437] optimizes the soft. token in the prompts to induce a more balanced gender distribution. Furthermore, Chuang al. [188] work directly the text embodding space to obtain a more balanced gender distribution in vision-language models. A recent study [502] addresses these problems by finding the bias-related concept in interpretable latent space and manipulating the generation process with the concepts found. These prompt-based regulations are by far the most widely adopted strategy to reduce biases in the generated content. However, it has been noted that keyword-based approaches could disproportionately affect marginalized groups, implying that their use at the prompt level could yield similar outcomes [232].
Figure 6: Examples of gender biases in T21 models: DALL E
shows spurious correlation between gender and profession.
Another direction of research involves addressing biases through sampling methods. For example, the D2C method [810] generates unconditional diffusion via few-shot conditional diffusion to balance the numbers in
17

---

Published in Transactions on Machine Learning Research (10/2025)
generated classes. Furthermore, Fair Sampling [181] introduces fairness-aware sampling technique aimed at reducing the amplified biases inherent in training data.
3.6 Current Limitations and Future Directions
Despite significant advancements in the domain of bias and fairness in foundation models, there are still some limitations in bias and fairness evaluation that require future attention.
3.6.1
Limitations and Open Challenges of Bias and Fairness
Currently, most bias evaluation methods are limited to token or paragraph-level assessments, making it challenging to capture the gradual propagation of bias during autoregressive generation [1007, 772, 1186]. In autoregressive models, each token's prodiction relies on previously generatod tokens, mcaning that biases may accumulate and spread over time. Traditional token paragraph-level fairness metrics [130, 74] are insufficient to fully assess this bias propagation issue, making it difficult to accurately measure the biases in these model outputs.
Additionally, when dealing with specific content, such as social media posts or content related to current events, the concept bias becomes more complex. Biases in such cases may not always be evident or confined to a single token but may be reflected through subtle contextual influences or narrative frameworks. Therefore, bias concerns more than just token-level differences; it also involves how the model handles historical, social, cultural influences, which may be embedded in the model's training data. This presents significant challenge for mitigating biases, as it often intertwines with factual reporting and socially accepted norms.
When evaluating biases in foundational models, the lack of clear and consistent definitions of fairness within these models complicates both assessment and improvement efforts [230, 796, 1110J. What considered fair can vary significantly depending on cultural, social, and historical perspectives. This variability becomes especially pronounced when dealing with news content, where fairness often intersects with historical accuracy or the prosentation of current facts. Models must therefore navigate delicate balance: striving for unbiased outputs while carefully weighing the tension between fairness principles and accurately representing reality,
For example, in reporting on historical events or current issues, there may be cases where acknowledging certain incqualities or biased social structures is necessary to present the facts accurately. In such situations, pursuing absolute fairness might mean overlooking or distorting facts, leading to significant conflict be- tween fairness and authenticity. For model developers, maintaining the integrity generated or processed information while addressing ethical concerns is a significant challenge. Furthermore, when fairness consid- erations span different regions, cultures, and social norms, the complexity of such evaluations is exacerbated, increasing the difficulty of implementing fairness assessments in foundational models. In addition, societal biases are challenging to mitigate with common techniques such as data resampling [357], and addressing them in web-scale datasets remains an open problem.
3.6.2 Future Directions
Addressing the limitations of bias and fairness in current foundation models opens several avenues for future research. Firstly, exploring unbiased tokenization and embedding methods could help mitigate the introduc- tion of bias during natural language processing [704, 1106). This involves developing tokenization techniques and embedding representations that maintain fairness at a more fine-grained level. Secondly, in terms of unbiased fine-tuning and preference learning, employing techniques such as constrained RLHF [1077]. DPO [1170, 935, 1168], and revised LoRA methods [567, 1026] can help to adjust the model's training process to reduce bias introduced during generation. As models are made more secure through methods like adversarial training, future work must also ensure these security protocols are designed to be fairness-aware, preventing the accidental amplification of biases as a side effect. Furthermore, the prot-procsssing of autoregressive generation is an important area of focus [1169), which can help further detoct and correct potential biases in the generated content. This challenge of equitable evaluation extends related tools, creating need to audit detectors of AT-generated content (AIGC) to ensure they do not unfairly penalize content. from
18

---

Published in Transactions on Machine Learning Research (10/2025)
specific demographic groups. Morcover, explainability methods offer promising frontier for more targeted interventions, opening research avenues into using mechanistic interpretability to locate and edit the specific model circuits that encode stereotypes. In MLLMs, individual modalities can have disparate impact on bias and fairness and may require modality-speciffc interventions [974]. Lastly, balancing fairness and utility is crucial, as striving for absolute fairness often conflicts with the model's ' practical utility and performance. Therefore, developing effective trade-offs that simultancously address fairness and utility will be a significant challenge for future research.
19

---

Published in Transactions on Machine Learning Research (10/2025)
4 Alignment
Foundation models have significantly expanded their functionalityy advancing beyond simple content gener- alion to wide range of applications including stralegic planning [391, 820, 580], code generation [154, 707], tool integration [718, 794], complex reasoning [963, 379], and even addressing challenges in natural sciences, especially mathematics [239, 399, 676]. Despite these advancements, it is important to note that foundation models are primarily trained on large datasets with objectives such as next-token prediction [723], next-scale prediction [863), or diffusion [545]. They are not inherently equipped to understand or prioritize human values and preferences.
This gap between their powerful capabilities and inherent limitations underscores the potential risks as- sociated with their deployment. For example, without proper safeguards, foundation models could be jailbroken by users to disclose personal information or engage in harmful behaviors, which should be avoided [508, 854, 792, 789, 156, 142]. Moreover, the ability of AI agents to adapt their capabilities to diverse objectives (e.g., scientific discovery and management systems) further highlights the importance of thoughtful oversight. According to the orthogonality thesis [105], AI systems can pursuc any number of goals, regardless of their intelligence level. This concern is compounded by the instrumental convergence thesis [105], which suggests that regardless of their ultimate goals, Al systems might adopt certain potentially harmful strategies means to achieve them such self- preservation or resource acquisition, which could lead to power-secking behaviors [105, 115]. As the capabilities of foundation models advance, it becomes crucial to carefully design these models to align with human-centric values and the nuanced requirements of specific tasks. This alignment is essential for ensuring the deployment of foundation models meets rigorous safety and cthical standards in various real-world applications.
In the following section, we will focus on aligning LLMs with human preferences. We not only reviews core techniques like Supervised Fine- Fine-Tuning and RLHF but also frames them crucial component in a larger ecosystem of responsible AI development, with direct intersections with uncertainty and hallucination. We will begin with Supervised Fine- Tame-TTnigg [962, 1152, 190], proceed to Reinforcement Learning from Human Feedback [686, 185, 57], and then explore Prompt Engineering [564, 321]. Furthermore, we will extend our discussion to MLLMs in Section 4.4. Finally, we will discuss the limitations of current alignment methods in Section 4.5. These categories can be visualized in Figure
4.1 Supervised Fine- Munn-Tunig
Supervised Fine-Tuning (SFT) is widely-used approach to aligu pre-trained LLMs with human preferences, which directly tunes the LLM f to mimic desired ground-truth responses. It often serves as the first stage of the aligmment process. Most SFT methods can be formally expressed as:
LSFT - Elog log Py (re p.r<)),
(11)
where p denotes the imput prompt and (r1, F2.. is the sequence of the target response. This approach maximizes the likelihood of generating the optimally selected response, akin how a student learns from teacher's guidance. Combined with other training methods for alignment, SFT can often enhance the stability of the whole alignment process.
Although SFT is efficient in aligning LLMs, its success heavily relies on the quality and diversity of the training data. LIMA [1152] presents a study that highlights the importance of this aspect, where the authors curate dataset of 1, 1,000 high-quality prompt-response pairs, with 750 of them coming from diverse sources such as StackExchango wikiHow" and the Pushshift Roddit Dataset [73], while the remaining 250 pairs are manually annotated. LIMA demonstrates that LlaMa-65B [876), when fine-tuned on small but high- quality dataset using the regular SFT training objective, can achieve significant performance improvements without requiring reinforcement learning or explicit human preference modeling. In this line of rescarch,
2nttpe://stackexxcangg. con/ onttps://www wikihou. con/
20

---

Published in Transactions on Machine Learning Research (10/2025)
qusliquality Duturets
LIMA [11ta], Curated Procupt Respouse Puins
(78)
Data Quality
Data Cleaning
Minihrsh [132], LSII [and Toile based Filtering [1119) 6ies, Tas)
Datx Selvction
Instraction Pollowing Diffirilty [s11g Importanee Weights [1012]
Stl--latrus Franzwoork
Sclf-lodieect [012, ICI-laaed Gomeratio [118]]
Supervieed Pise- Tuning
AL Genetated Data
Open souzee Models
Alpsox (853] Vieuno [178 Textbook Quality Date. [128]
MACPIE [1024) Rodeopiaping Dati (125), Agenl-ianing Dutix [2:0].
Athaned Syschowis
Self-talk Generation [st]]
Voherabilitie
Stiey looure
Overtratzed Models [828]]
Adluvvarial Manipulations [1122] Red Town Dataoett (183]]
Finegrained Framenorks
Pinuegra/ned RLHF (102) Rowanded Soup [*31]]
Beyond Curventional Reward Modils
Guneral Prefensoo Approacion
Compemtivve Evaluction [1150]
Nush: Equilitriiu (158, 1068) Dueling Bundii [1068, 1188).
R-wier Maxicumm Liksiiboad [1175, (177), StwwrLM [190] Multi reward Bolance [1175, 99S. 109, 125
Multbohgjatior Rewords
Verified Rewands
DaveSwiic RI [[N]] OpenAT af [070]] Mothemratioal Bessoning 1633
Al Faatlacck
RLAIF [se]. Cuncitttiomal At [66]
RLMP
Human-Ametatel Data
Deyond
Tesk specidl Applications
Summuriaation [asa], Compuex Ressoning [146), Culline Al Foodback [ats]
Silfespluring Mulhands
SELM [11200, Cocatiiaiinall DPO [X08]
Altorzative RL Methods
BAFT [210], RRIU [1O85, BcST [sas), SLC (1342)
Alignmment in: Foumdation Modeli
Beyord
Procinul Polcyy Optisiattin
Direct Prefevenco Methoos
DPO [127], KTO [onc] IPO (po), RPO (1sa), ShmPP (cau), ORPO (acu)
Fificient RI. Algorithers
GRPO (aor] RAINFORCE+++ [371]
Contiania Procapia
S.0 Fromptt
Gewdinntt-ooed Tunins [717, 227, 492, S11, 571, 345]
Mantal Crufting
Natural Language Procupts, Humaat Hummas eaterpoctable Design
Diamnta Prunpts
Automaatl Genccation
Buk- haseb Search [oos, 374], RL -boeed Search [200, 1121/, Seuliivity Botus (778)
Rosic
CVT [o63] Sell-comsistmeey [109] Zenoshot GeT [150]
Pronu: Enginerrng
Clain-of-Thunght
Estanded Structures
ToT [106a]. G2T [8:]] Long OCT (670, 207, 652, 10, 21]
Limitatkas
Post boe Rationalizations [ess] Test thme Computing (814, 185)
hize-hazed Optiinaation
APB [1174], APO [reof OPRO (100) Seff-refune looe)]
Prompi Optimisatios
Exulutionry Meshode
ExuPrunge (350), Dervoativv- Opiimization (:24))
BeAst (1962) PautionPrunot [[00].
Specialionl Tidmiquu
Symbole Learning (1 109, Control [1153]
Vicual Incruction Tuming
Lanoa-LLag Iratroctions [601] Linar Projuctico Layers
LLabLA Adestee (250), MilhinedalGPPP (300)
Multinodaa Datascts
Otter (188] PaocaGPT (638], LaVA [00]
MLLM Aligumans
Data Quality Control
Perplesity Sooting [1.50), Grodbent Computation [121]. GPT4 Rating (65]
Prefenenee Methods
DRESS [oo), POVID [1308, STCC (210), Self rewording [IAS, 2170
RLIP Challenges
Data Polsoning, Peotbock Bases
Tractability
High-guality Fredback sost,
Multimodal A'igmeet & Liuitations
Oureut Limilations
Cencrality Ismes
Human Capieity Lonttettons, Diveme Valto Modeling. Reward Hacking, Powxe socking Belavioes
Direct Aligatont loues
Overtsting Probiens (725), Out-of- offto--tttriiutto Robustocss [728]
Superaligmeet
Sealable Overiight 1S5 108, 180, 539 Weak: stroog Conerollection [115, Say]
Puture Directious
Advanced Methods
Itoentive Trating [1085, 756, 947),
Ouline Aliqumeu [133], Modil Merging [Tad]
Seauity Concens
Superteial Altgnment,
Badaloar Vultemuiiliiie J127, 130), Evaiattion Dillocitios
Figure 7: Taxonomy of Alignment in Foundation Models.
methods based on Minihash [112] and Local Sensitive Hashing (LSH) [206] are often used to deduplicate the data, which serve as the first step of refining data quality. Then, a scries of works [1160, 698, 726, 930, 152] propose to use well-suited rules, metries, and LLMs-based methods for further data cleaning. These work leverage measure the quality of the data and perform improvements such that deduplication and rewriting
21

---

Published in Transactions on Machine Learning Research (10/2025)
Align with training data: SFT
Align with human preference: RLHF
Supervised Fine-tuning
Preference
Align with vision: Visual Instruction Tuning
Align with prompt: Prompt Engineering
Visual Instruction Tuning
Input Prompt
Figure 8: Alignment is required at different stages in the foundation models. Typically, LLMs are aligned using SFT and RLHF during post-training, while using prompt engineerfing at inference time. Compared to LLMs, MLLMs require an additional step of multimodal alignment. at post-training, such as Visual Instruction Tuning.
to improve the overall quality. Li et al. [510] introduces the instruction following difficulty metric for efficient data selection. Similarly, Xie et al. [1012] estimates importance weights for high-quality data selection.
Moreover, many empirical results of applying scaling laws [428, 359] to LLM training show that the data size bocomes crucial for performance improvements. To reduce the cost of human annotations, rescarchers are increasingly interested in incorporating Al-gencrated data into the aligmment process, especially with the advent of closed-source L.Mss like GPT-4 [676], Gemini 2.0 [310], and Claude 3.5 [34]. A line of research explores using LLMs to self-generate instruction-tunn tuniionnntiinggg data. A notable advancement in this domain is the Self-Instruct [942], which leverages the in-context learning (ICL) capabilities of GPT-3 [113] to gather in- structions and preferred responses autonomously. This approach begins with a small set of human-annotated seed instructions that are subsequently refined and expanded to generate large-scale instruction data across diverse tasks. Building on this methodology, researchers have achieved significant advances developing open-source LLMs with enhanced -istruction-folloiing capabilities such as Alpaca [853] and Vieuna [178]. Gunasekar et al. [326] propose use a mix of "textbook quality" data from the web and GPT-3.5 generated data to train the Phi, lightweight LLM suitable for edge scenarios [1020]. It also pioncers large-scale, high-quality data generation. More recently, Xu et al. [1024] proposed a self-synthesis method that leverages the regressgee nature of LLMs to generate diverse data without requiring any initial seed question or prompt. Zhou et al. [1165] propose to synthesize natural language descriptions for controllable text gen- eration [377, 841]. Wang et al. [928] introduce a method for synthesizing role-playing data using LLMs with carefully curated role descriptions. Similarly, Qiao et al. [716] present an approach for synthesizing agent-tuning data via self-planning with LLMs; Ulmer et al. [889] generates training data via "self-talk" of LLMs which can be used for further supervised finetuning.
Nevertheless, the simplicity of SFT does not shield it from potential vulnerabilities, especially in terms of model safety and robustness. Qi et al. [712] illustrate that LLMs, such as OpenAl's GPT-3.5 Turbo [675], are prone to adversarial manipulations. They demonstrate that fine-tuning these models with a limited set of strategically crafted examples from the Anthropic red team dataset [283] can significantly undermine the model's safety protocols. This phenomenon highlights the necessity for rigorous examination selecting and preparing SFT data. Further Springer et al. [826] highlights that overtrained LLMs impose challenge in further supervised finetuning.
In conclusion, while SFT demonstrates notable efficiency, its effectiveness in aligning LLMs hinges critically on the quality, scale, and diversity of training data. The critical role of a meticulously curated dataset extends beyond improving model performance; it also vital to mitigate risks related to model safety and robustness. Inadequately vetted or dcliberately compromised data can introduce harmful biases and trigger undesirable behaviors in LLMs. This concern highlights the ongoing necessity for rigorous data curation methods to enhance both the reliability and security of LLMs in real-world deployments.
22

---

Published in Transactions on Machine Learning Research (10/2025)
4.2 Reinforcement Learning from Human Feedback
Reinforcement Learning from Human Feedback (RLHF) [686, 185, 57, 1135] represents a significant advance- ment in aligning LLMs with human preferences, involving several critical steps:
Supervised Fine- Daninn (SFT) is performed on a pre-trained model using a high-quality, instruction- following dataset. The resulting model serves as the initial policy, #SFT, for the subsequent RLHF optimization.
2. Collection of pairwise ranking data to train reward model that correctly scores these data.
Optimization the policy model obtained in Step 1 against the reward model in Step 2 using the Proximal Policy Optimization (PPO) [776].
To stabilize the optimization in Step 3, KL-divergence regularization is introduced [686], ensuring that the model remains reasonably close to the initial policy model acquired in Step 1.
a) Beyond Conventional Reward Models
Preference Reward
100
Prompt
Language Model
LM Outputs
Human Feedback
Reward Model
PPO
Output
Update policy
b) Beyond Human-Annotated Data
Beyond Proximal Policy Optimization
ff
D
Al Feedback
Preference data
Objective Function
Final Model
Update policy
Figure 9: The evolution of Reinforcement Learning from Human Feedback (RLHF), illustrating three key
areas of advancement. Panel (a) depicts the conventional RLHF pipeline and the move Beyond Conventional Reward Models. Panel (b) shows the shift Beyond Human-Annotade Data to include Al-generated feedback. Panel (c) illustrates the trend of moving Beyond Proximal Policy Optimization (PPO) towards simpler, direct preference optimization objectives.
Beyond Conventional Reward Models. The effectiveness of RLHF is closely linked to the accuracy and robustness of the reward model. Recent research has identified biases in reward models [791, 490] and has focused on refining traditional Bradley-Terry reward models [109]. Wu et al. [992] introduce a fino-grained RLHF framework that addresses the challenges of translating human preferences into scalar learning signals for extensive textual outputs. This approach utilizes multiple fine-grained reward models and has demonstrated superior performance in tasks such as detoxification and extended question-answering Complementing this, Rame et al. [735] propose "rewarded soup", which linearly interpolates weights across specialized networks to derive diverse rewards. It emphasizes the importance of a varied reward structure and aims to achieve Pareto-optimal generalization ion across the complete preference space. An addition ional generation of reward modeling, which referred to as the "general preference" approach, directly learns a pairwise preference function and socks a model that identifics the Nash equilibrium an entropy-regulariee minimax game [658, 1058). This strategy draws inspiration from the classical dueling bandit problem [1088, 1182]. Zhou & Xu [1159] propose train comparative evaluation model based annotated pairwise preference
23

---

Published in Transactions on Machine Learning Research (10/2025)
data and use it to train TextGAN [1132] with RL in Zhou et al. [1161], this can be viewed as an early version of RLHF.. Zhu et al. [1174] extend pairwise ranking by considering the ranking of multiple responses and trains the reward model using K-wise maximum likelihood [1175]. Additionally, beyond the single reward model, another approach considers the joint preferences implied by multiple reward functions, such as "helpfulness, harmfulneess verbosity", etc. Some these reward functions may conflict with each other. The objective here is to strike a balance among various rewards, reflecting diverse user preferences [236, 1173, 918, 169, 129]. This 129].This diversity is not merely individual but also deeply cultural, as preferences and values can vary significantly across different populations, making single, universal alignment target an ill-posed problem [445].
Recently, the success of DeepSeek R1 [207] and other reasoning models [652] such as OpenAl ol [679] on closed-end domains such as mathematical and code reasoning demonstrate the importance of verified rewards in large scale reinforcement learning optimization, which also highlights the serious reward hacking problem. Despite these adjustments to the reward model, such methodologies remain firmly within the overarching RLHF framework.
Table 2: Various preference optimization objectives given the preference data D (ar, Yur W)) where is an input, y is an output, you and Y/ are the winning and losing responses, and y², E [mx] are ranked responses.
Method
Objective
RAFT [233]
max. D.v-ppp (r (r, v)]
RRHF [1085]
Last + Eisy max O0, ( (a:) (( [x)
max ENND [AE (yll)) F(x, y: r) V log [o(v)=)]
ReST [325]
+(1 yopppll)))) [2, y: T) log No [o(y)x))
SLiC-HF [1142]
max (0, 5 log ao (Vuol (x) + log Th (vr (z)) À log (V(w
DPO [727]
llog a (álog Tuly
33 log (p,
-(u"
PRO [822]
BCan
1n k= log
3/6o" (xoga))))
T(u (x) T/C (mig)) (m.
21 exp
17C0"(0. J-r ""
IPO [53]
(log Nolywlw) (w) Frat(Wllar
No(vi(z) log
1
Fzct(3/ (ar)
2r
-Nn (Blog
so(mmla)) (m)
Zraf + AIT
Fren - 3 log so(2n
KTO [260]
F
where Zref ((ED) [3KL (To (3/lz]] [1]r]
ORPO [360]
Lsn A Alogo a log m(y" -V T(5'|=)(1 [[))]==]]
min n/3 Egw a (+1x) [- log(Tro (v [z))] noly w)
RPO [882]
llog (Blog Feer(yalo) 33 Blog
no (xxl))
F teitiil
SimPO [630]
log σ |e log To (Nollz) Iael
log To (w)l x)) -7)
Beyond Human-Annotator Data. Synthetic data generation has proven effective for SFT. However, when it comes to RLHF, pairwise preference ranking data is typically collected through human annotations, process that can be costly for scaling. To mitigate this issue, recent research has shown that Al-generated data can also provide helpful feedback for alignment. Bai et al. [58] introduce "RL from Al Feedback" (RLAIF), which blends human and AI preferences under the "Constitutional AI" (CAT) framework. In this framework, AI behaviors are governed by principles analogous to a constitution, supported by a few examples for few-shot prompting. This methodology aims to train a non-evasive Al assistant that is effective and harmless without relying solely on human labels. Further extending the concopt of RLAIF, Lee et al. [483] apply it summarization tasks, while Wang et al. [946] adapt it for complex reasoning tasks, highlighting the potential of AT feedback. Additionally, Guo et al. [333] enhance the RLAIF paradigm with online AT
24

---

Published in Transactions on Machine Learning Research (10/2025)
feedback, demonstrating superior performance in model alignment compared to both offiine RLAIF and traditional RLHF. Zhang et al. [1120] propose Self-Exploring Language Models (SELM) to elicit preferences for online alignment actively. Wang et al. [933] propose Constitutional DPO, which uses expert-annotated principles to synthesize negative examples for preference learning. Shi et al. [797], Li et al. [513] propose to use agent workflows to generate verifiable agentic lasks for end-to-end agentic reinforcement learning.
Beyond Proximal Policy Optimization. While RLHF has proven effective in capturing human prefer- ences, the PPO algorithm [776] typically requires complex implementations and substantial computational resources, limiting its applicability in various contexts. The key challenges in PPO training include filtering high-quality data to compare similar responses, managing policy and reward models within limited resources, mitigating reward hacking issues [590, 254, 738], and requiring extensive hyperparameter and training strat- egy adjustments. To address these challenges, various new preference optimization objectives have been proposed, and their corresponding objective functions are presented in Table 2. Dong et al. [233] introduce the Reward Ranked FineTTuning (RAFT) that simplifies the complexity of PPO by using a reward model to selectively focus on the most promising responses sampled from an LLM. Specifically, RAFT involves sampling large batch of instructions and generating multiple responses. These responses are then holisti- cally ranked by the reward model, with only the top-ranked responses used in SFT. This process is iterated until the rewards stabilize, and the fine-tuning dataset is periodically updated to enhance its quality. In parallel, Yuan et al. [1085] introduce Reinforced Ranking Human Feedback (RRHF), which aligns the model with human preferences among diverse responses using a likelihood ranking loss. This method facilitates the integration of data from multiple sources, including both gener-generated and human-curated data.
Another innovative approach within the RLHF framework is Reinforced Self-Training (ReST), introduced by Gulcehre et al. [325]. ReST focuses on iteratively generating and refining data from policy models op- timized by offiine RL algorithms, enhancing data utilization efficiency. The framework involves two main steps: "Grow" and "Improve". In the Grow step, the policy model generates multiple outputs for augmen- tation. During the Improve step, the gencrated data is ranked and filtered by a preference reward model, after which the policy model is fine-tuned on the filtered data using an oflline RL objective. This process is repeated with an increased filtering threshold to further refine data quality. Beyond zeroth-order RL algo- rithms such PPO, which require the learning of the value function and hyperparameter tuning, first-order RL algorithms [1119, 1118, 284] can also act as straightforward alternative for RLHF alignment. Sequence Likelihood Calibration (SLiC) by Zhao et al. [1141; 1142] aims to align model outputs with reference se- quences in the latent space by calibrating the sequence likelihood. This method replaces the traditional embedding similarity function with a preference ranking function and employs a cross-entropy regularization loss to keep the model close to the reference, typically an SFT model.
Additionally, Song et al. [822] propose the Preference Ranked Optimization (PRO) method. Differing from traditional RLHF approaches that use the Bradley-Terry reward model focusing only on the best and worst responses, it enumerates all possible ranking pairs among candidate responses to provide comprehensive alignment.
Based on these insights, Rafailov et al. [727] propose Direct Preference Optimization (DPO), which integrates preference information indirectly into the optimization of the policy model, eliminating the need for a separate reward function. The DPO loss derived from the reward maximization-based RL.F algorithms is used to directly optimize the policy model ma as follows:
LDPO (To; Teet) = log o (log To (Yu x)) - log
To (yr x.)
(12)
Tref (Bw | x)
Trel (w1 x))
where weet denotes the reference policy (namely the SFT model), and (w, Yur vt) represents the instruction paired with the preferred answer Bo and the dispreferred answer we Furthermore, Ethayarajh et al. [260] propose Kahneman-Tversky Optimization (KTO) to directly maximize the utility of LLM's generations instead of the likelihood of preferences. Unlike methods that require costly annolated pairwise ranking data, KTO only needs individual binary feedback, which is easier to collect from real users. Besides, Regularized Preference Optimization (RPO) [582] is proposed to mitigate reward hacking or overoptimization issues during RLHF by simply adding the SFT loss to DPO, Azar et al. [53] theoretically analyze the weakness of DPO and introduce Identity Preference Optimization (IPO), which adds constant regularization term to the
25

---

Published in Transactions on Machine Learning Research (10/2025)
DPO loss to mitigate the overfitting problem. SimPO [630] introduces margin term to the Bradley-Terry objective and incorporates the length normalization, eliminating the common need for an additional reference model in preference learning. Similarly, Hong et al. [360] propose ORPO, which integrates an odds ratio preference objective into the standard SFT objective, also functioning independently of a reference model. In addition to these direct preference optimization methods, some recent works focus on improving the efficiency of large-scale reinforcement learning in optimizing LLMss DeepSeok [207] propose Group Relative Policy Optimization (GRPO) to eliminate the needs of the critic model and value estimation in PPO, which greatly saves computation resources. GRPO estimates the baseline of advantage by calculating the average rewards of multiple samples in the group and replaces the KL penalty added to the reward by explicitly adding the KL divergence to the loss. The success of DeepSeek R1 [207] also indicates the efficiency and performance of applying GRPO in large-scale reinforcement learning for LLMs. Similarly, REINFORCE- algorithm [371] removes the critic model via implementing token-level KL penalty, PPO's clipping for policy model updates, and normalized advantages while achieving efficient reinforcement learning for LLMs.
The proliferation of these alternatives to PPO raises the practical question of which objective to choose. The decision often involves trade-off between complexity, data requirements, and robustness. Methods like DPO, ORPO, and SimPO offer simplicity and efficiency by forgoing an explicit reward model, making them suitable for eesource-constraine scenarios. However, these direct methods may be more susceptible to overfitting on the preference dataset. In contrast, full PPO-based RLHF, while more complex, allows for online exploration and can lead to more robust models, particularly when paired with high-quality, verified rewards as seen in specialized domains.
4.3 Prompt Engineering
While some research focuses on aligning LLMs with human preferences through explicit training, another line of research emphasizes the strategic design of prompts to effectively improve the LLMs' generated responses. This approach, known as Prompt Engineering, involves crafting prompts that guide LLMs toward fulfilling specific task requirements.
Continuous Prompts. In the field of prompt engineering, continuous prompts, represented as continuous vector inputs integrated into the LLM, offer a novel approach to guiding Al responses. Due to the continuous nature, these prompts can often be fine-tuned through gradient- based methods using labeled data [717, 227, 492, 341, 571, 345], which is effective for adapting LLM's behavior and injecting domain knowledge. However, the continuous format makes it less transparent for human understanding, posing challenges in interpretability [435, 341].
Discrete Prompts. In contrast to continuous prompts, discrete prompts consist of discrete tokens from the natural language vocabulary. The distinct advantage of discrete prompts lies in their use of natural language, which makes them inherently interpretable and relatable to humans. These prompts can be manually crafted or automatically generated. To automatically design effective prompts, some work utilizes pre-defined rules and reinforcement learning methods for scarching [286, 374, 209, 1121]. However, this approach is not without its challenges, models can be extremely sensitive to minor, semantically irrelevant changes in prompt formatting brittleness that can lead to umpredictable performance variations [778].
Chain-of-Thought. For complex reasoning tasks, Wei et al. [963] first propose the Chain-of Thouuhh (CoT) method to encourage LLMs to generate a series of intermediate reasoning steps before reaching the final answer. This method has shown notable success in improving the performance of L.MSs on tasks requiring multi-step reasoning, arithmetic reasoning, logical deduction, or commonsense application [963, 938, 276, 452, 151]. Based on CoT, Tree-of-Thought Thought (ToT) [1053] extends the concept to planning and decision. ToT refines the CoT method by utilizing the specific attributes of problems to decompose and organize intermediate thoughts into a tree structure. In ToT, each "thought" builds a node in this tree, facilitating explorations by searching algorithms such as the breadth--frstt or depth-first search, allowing lookahead and backtracking in problem-solving. This method has been further improved by Graph-of- Thought (GoT) [84], which diverges from lincar or hicrarchical structures to a more flexible graph-basod representation. In GoT, the generated thoughts are forming nodes in a graph, with edges representing their complex interdependencies. This grspd approach can capture the multifaceted nature of reasoning
26

---

Published in Transactions on Machine Learning Research (10/2025)
[Po] [Pr] [P++1]
Discrete
Prompt Generator
Rewards
Back
Prompt Encoder
Propagation
A
>
B
[MASK]
V
V
Embedding
O
ho h4
h1+1 c[MASK]
c(A)
<(>)
e(B)
c[MASK]
a) Continuous Prompts
b) Discrete Prompts
Query the LLM
9.11 > 9.8? Let think step by step.
9.11 9.8?
Grndient
1. Compare the whole numbers...
2. Compare the decimal parts...
9.11 > 9.8? Please compare
3. Final comparison...
these two decimal numbers and
Therefore, "9.11 9.8" is false.
determine which one is larger.
c) Chain-of-Thought
d) Prompt Optimization
Figure 10: An overview of four major prompt engineering methods for guiding LLMs. (a) Continuous
Prompts soft prompts represented tunable vectors in the embedding space. (b) Discrete Prompts
are interpretable natural language prompts that can be automatically generated. (c) Chain-of-Thought
prompting elicits intermediate reasoning steps to solve complex problems. (d) Prompt Optimization uses feedback from the LLM itself to iteratively refine prompts and improve performance.
processes, offering improved adaptability for tasks such as sorting and keyword identification. Besides, it also improve the latency and throughput compared to CoT and ToT. Recently, large reasoning models such as OpenAI ol [679] and DeepSeek R1 [207] are believed to use long CoT to further enhance the model to plan and reason. The long CoT is pushing the length of the generated CoT to thousands or even hundreds of thousands of tokens, which effectively scales the test time computing [679, 207, 652, 19, 814, 985, 21]. Ii crucial to note, however, that these generated reasoning chains may not faithfully reflect the model's 's actual computational process. They can be post-hoc rationalizations rather than true trace of the model's 'thought' process, phenomenon that complicates their use for interprotability [883]
Prompt Optimization. Complementing these structural prompt methods, recent research explores opti- mizing prompts directly using LLMs themselves. Yao et al. [1052] propose ReAct to motivate LLMs to gen- erate both reasoning traces and actions to interact with environments, to improve their general task-solving ability. Automatic Prompt Engineer (APE) [1172] employs LLMs to craft initial instructions. Subsequently, APE cherry-picks instructions that exhibit the highest accuracy. Each of these selected instructions then fed back into the LLM, prompting it to generate a variant that is semantically akin to the original instruction. Following similar style, Automatic Prompt Optimization (APO) [710] iteratively refines existing instruc- tions using textual feedback from LLMs. Conversely, Optimization by PROmpting (OPRO) [1030] adopts more direct approach, generating new instructions at cach optimization step, with LLM optimization focused on enhancing (ask accuracies without necessarily replicating prior instructions. Some LLMs have been shown to have the ability to use self-generated feedback to iteratively refine the output [609]. One can also apply derivativo-froe optimization techniques to optimize discrete prompt [224]. Additionally, Guo et al. [332] propose EvoPrompt, which adopts evolutionary algorithms with LLMs for discrete prompt optimization. Beginning with set of initial prompts, ExoPrompt applies evolutionary operators and performance-basee
27

---

Published in Transactions on Machine Learning Research (10/2025)
selection to iteratively refine and generate new prompts. In addition to these explicit prompt optimizations, Li et. al. [499] propose EmotionPrompt which focuses on understanding the psychological and emotional stimuli of LLMs. It shows that simply appending emotional stimuli, such as "this is very important to my carcer", to the original prompts can also significantly enhance the performance of LLMs. Liu et al. [581] pro- pose the first principled framework that has provable regret guarantees to orchestrate reasoning and acting with specially designed prompts. Zhou et al. [1166] propose an agent symbolic learning framework to jointly optimize chain of prompts (i.e., agent workflow; 1164) by mimicking pack-propagation and gradient descent with natural language and LLMs. Tang et al. [850], Zhou et al. [1155] propose to use experience/ peeeriennemmmoory.y /memory- based learning to optimize LLM agents without fine-tuning the underlying LLMs. Beyond pure performance, prompt optimization can also be framed as a risk control problem, where the goal is to identify prompts that are robust and minimize the likelihood of generating harmful or undesirable content [1183].
In the field of prompt engincering, the majority of methods focus on improving the performance of LLMs on specific tasks. While these methods are crucial for technical optimization, their contribution to aligning LLMs with human values and preferences is more indirect. By improving the interpretability of L.LM oulputs, these prompt engineering methods can gradually help LLMs better meet human expectations. This relationship between performance improvements and alignment with human values is an important consideration in the ongoing development of LLMs.
4.4 Alignment for MLLMs
Recent studies advocate the development of MLLMs capable of tackling various multimodal tasks without re- quiring particular adaptations. This approach leverages the well-ostablished text-based capabilities of LLMs by integrating them, in a frozen state, as the language component within multimodal architectures, i.e., MLLMs, can align the visual and language modality through visual instruction tuning [555], a specialized form of instruction tuning that extends the capabilities of pre-trained LLMs to understand and perform mul- timodal tasks involving both text and visual input. By incorporating datasets containing of rision-langues instruction-follwwee samples, this method enhances the zero-shot capabilities of LLMs for understanding and responding to visual inputs. The process typically employs linear projection layers to integrate image encoders with LLMs, allowing these models to effectively handle tasks that require an understanding of both text and images. Besides, extensive datasets comprising vision-language instruction tuning are utilized to align MLLMs with human preferences [285, 305, 495, 554, 838, 1002, 1001, 496, 870, 17]. This approach al- lows MLLMSs to accurately interpret. instructions and generate user-friendly responses. Further works extend MLLMs wider range of tasks such as multimodal synthesis [586, 587, 871, 1009, 1153, 860], and interactive agents [1099, 57, 1167, 1010, 1031, 788].
To better align the MLLLM's output with human preferences, some recent work [844, 1170, 935, 561] aims to enhance model capabilities by filtering low-quality instruction data or constructing carefully examined exam- plcs during the fine-tuning phase. Recent studics [153, 121, 696, 350] have introduced methods for evaluating the quality of instruction data in both vision and language datasets. These methods include computing the perplexity, calculating the gradient, and employing more powerful closed-source LLMs (c.g., GPT-4 [676))
for rating, all aimed at filtering low-quality data from the training process. InstructionGPT-4 [965] presents more general data quality control pipeline by training a robust data selector to automatically select proper data from the raw dataset used to fine-tune MLLMs. DRESS [161] proposes to divide natural language feed- back (NLF) into critique and refinement types, and then utilize them to improve the alignment with human preferences and interaction capabilities of MLLMs. POVID [1168] utilizes Al-generated dispreferred data by explicitly contrasting a hallucinatory answer with a truthful one, eliminating the need for gathering human foedback. Recent works such as STIC [210], SIMA [935], CSR [1170], and AnyPrefer [1171] explored the enhancement of the alignment between vision and text modalities through self-rewarding methods without introducing additional models and data.
4.5 Current Limitations and Future Directions
Though recent rescarch has achieved remarkable success in aligning foundation models with human values and preferences by leveraging Prompt Engineering, Supervised Fine-Tuning, and Reinforcement Learning
28

---

Published in Transactions on Machine Learning Research (10/2025)
from Human Feedback, several challenges remain. A prominent long-term challenge is the **spperalignment problem** **: how to ensure that AT systems much more intelligent than humans (i.e., superintelligene))) remain aligned with human values and intentions [115]. This is a difficult problem because humans may be unable to reliably supervise or evaluate the actions of a system that is far more capable than themsclves.
Effectiveness of RLHF Despite notable advancements in alignment brought by RLHF, this approach has its own challenges, as extensively analyzed by Casper et al. [128]. These challenges are broadly categorized into two types: tractability and generality. Tractability challenges encompass practical issues within the RLHE framework, such difficulties in acquiring high-quality feedback [165, 868], risks associated with data poisoning, and inherent biases in the feedback. These issues, while significant, are considered manage- able with the right strategies and improvements in future methods. On the other hand, generality challenges are more profound, raising critical issues about the overall effectiveness of RLHF. These include limitations in the human capacity to consistently provide accurate and reliable feedback for complex tasks, challengos in adequately modeling the diverse values of different human groups through reward models, and risks associ- ated with reward hacking and power-seeking behaviors inherent in reinforcement learning systems. Though applying rule- based RL in the reasoning domain appears successful [207, 652, 19, 21], it is challenging to directly adapt it to broader general domains to represent diverse and complex human values. For example, if a reward model cannot distinguish between nuanced responses and instead assigns uniformly high rewards to any agreeable or positive-sounding output, it may inadvertently train the foundation model to be syco- phantic agreeing with the user regardless of factual accuracy rather than maintaining epistemic integrity. Such fundamental challenges pose critical questions about the long-term viability and ethical implications of relying solely on RLHF for aligning foundation models with human values.
Issues in Direct Alignment. Direct alignment methods such as DPO [727] greatly simplify the tra- ditional RLHF pipeline and reduce the massive computational resources required for training. However, these methods may be prone to overfitting and common offline training issues. A series of direct alignment methods including DPO, IPO [53], and SLiC [1142] are found to have robustness issues, especially in out- of-distribution settings [728]. This is mainly due to the adopted oflline training paradigm, which often uses a small and fixed set of data for training, lacking explorations compared to online training methods such as PPO [776]. To address this issue, recent methods propose iterative training paradigms [1084, 756, 1016, 947] or online alignment methods [333] to enrich the training data, expecting to match the online RL performance. However, these methods are still in the early stages and require further investigation. Another issue lies in the scalability of these direct aligmment methods. Rafailov et al. [728] find that weak or small LLMs often tend to learn simple features (c.g., length correlation) of preference data instead of high-level human values. To improve performance after alignment, these methods require either a large amount. of SFT data or scaling up the model size, which limits the efficiency advantage over RLHF methods.
Superalignment. Superalignment is a concept that refers to ensuring that future super-intelligent AI systems still being consistently aligned with human values, which was first introduced by OpenAl [115]. Generally, there are two approaches trying to achieve superalignment: scalable oversight [25, 108), and weak- to-strong generalization [115, 504]. Scalable oversight aims at providing reliable supervision for untrustworthy but more capable AI systems, where most related work leverages the advantage that evaluation is easier than generation [489, 535]. The weak-to-strong generalization is simulating a scenario where the weak model can clicit the strong model's capabilitics. Besides, the alignment of foundation models can be superficial, i.c., the model is pretending to generate human-preferred responses without adhering to the underlying human values, making the alignment evaluation quite difficult. And this can leave backdoor vulnerabilities in the model, which can be exploited by adversaries [127, 126]. Ensemble methods like integrating various weak models to supervise the strong models in different domains can be a potential solution of scalable oversight to overcome the weakness of a single weak model [489, 108). The model merging method can also be utilized to achieve strong generalization by averaging a bunch of specialized models [736]. In addition, it is valuable to explore efficient methods to combine scalable oversight and teak-to-strong generalization to achieve superalignment. In summary, both scalable oversight and weak-to-strong generalization have their own advantages and limitations, necessitating further efforts to ensure that future super-inteligent AI systems remain aligned with human values.
29

---

Published in Transactions on Machine Learning Research (10/2025)
5 Security
With the widespread integration of foundation models into various domains, the growing adoption of these advanced models has also exposed security vulnerabilities, making them susceplible to adversarial exam- ples [307, 611]. Adversarial attacks [307] encompass a variety of techniques aimed at decciving AT models by manipulating the imput data with imperceptible noise, leading to incorrect predictions or manipulations of their outputs. This issue highlights the urgent need to thoroughly understand the vulnerabilities foun- dation models, which crucial not only for researchers and practitioners, but also for society at large. Prior reviewing efforts [1128, 603] focus on providing new taxonomies and platforms of safety threats and defense strategies across multimodal loundation models. In contrast, our survey embeds security within unified cross-task reliability and responsibility storyline to highlight its interaction with bias, privacy, uncertainty, and explainability in all foundation models. As an integral part of our review, this section explores founda- tion models' security development on attack and defense strategies. To provide comprehensive overview, we summarize the various attack methods in Figure 11.
Pipeline
LLM
MLLM
Text-to-lmage
Instruct [Wan al.
Trojdiff [Chen et al., 2022]
Model Training
Backdoor Attack
AutoPaison [Shu al. 2023]
BadEncode (Jia 2022]
G3detector [Zhan et al., 2022]
Badcode [Sun al. 2023]
Poison [Carlini al. 2021|
Rickrolling [Stuppek al., 2023]
BadGPT [Shi 2023]
Arch [iikel et al., 2023]
Jailbreaker [Deng et al. 2023]
Jailbreak Attack
GPTFuzzer (Yu et al. 2023]
Dress [Chen et al. 2023)
Prockyproomp [Yang et al., 2023]
RedTeuming [Rando et al. 2023]
PAIR [Chao et al. 2023
SASP Wi et al., 2023]
MuliMon [Tong al., 2023]
Universal [Zou al. 2023]
Model Inference
Real [Apruzzese, at., 2022]
LMM-PGD [Carlini 2023]
AdvDiff [Dai al., 2023]
Other Attacks
Indirect (Greshake al. 2023]
Embed [Dong al, 2023]
DiffAmack [Chen 2023
HouYi [Liu et al., 2023]
AgentSmith (Guet al., 2023
Instnuc1AAttakk [Liu al. 2023]
Ignore [Perez et al., 2023]
BurdAttuck [Dong al. 2023]
Figure 11: Atlacks on various foundation models in training and inference stages. All models suffer from Backdoor Attack and Jailbreak Attack. For the category of "Other Attacks', we include Prompt Injection Attacks in LLMs, Image Adversarial Attacks in MLLMs, and Adversarial Attacks in Image Generative Models.
5.1
Security in LLMs
5.1.1 Attack in LLMs
Similar to traditional AT models [771, 481], LLMs are inherently vulnerable various threats due to their very nature and architecture. For example, attackers can manipulate the input data and prompt LLMs generale incorrect undesirable outputs [320]. In the following, sunmarize three major threats against LLMs, including jailbreak attacks, prompt injection attacks, and poisoning and backdoor attacks.
Jailbrcak Attacks. Jailbreaking in foundation models is an attack that bypasses the security protection mechanism of foundation models lo enable responses lo unsafe questions and unlock restricted capabilities. Jailbreaks are fundamental threats to LLMs since they may potentially enable criminals to exploit these models for illicit activitics such as drug making, fake news generation, and phishing email writing. Re- cent studies have analyzed why jailbreaks work in practice: competing objectives between helpfulness and safety goals lead to failure modes or trade-offs where the model cannot satisfy both simultaneously, and
30

---

Published in Transactions on Machine Learning Research (10/2025)
Goo al (101] 1e al: (500] Troekitoonacal 1651, Shere (290) Wen (PTH Sheo (P00.
Jaltreek Attals
Dhes es al [19]] Xinng et [[00]] Zou at. [[180] Masdia (002) Tion et al [579] Vi at. (177).
Dou ei al. (an]. et al. [1881] Caso nl. [L11]] Zon en al. 1108. lis et [6it] Dong al, 201]
Attark
Phonge Iryection
(967).
([t).. Aprisoees
[14,
10n) Abdensati
1IM
Taakkoor Actaole
Wan al. (80g Slax al. 2011 Stn al [542) Sl ct al 1600, Rusio & Truak 1788) Classss al (167,
Zhoo et al [128] Xang et al. [oii] Wat al (107, Zon et al. [USS]]
Diirzaos spsizat Juillenake
Cara et [131]] Nie [D0s, Loo al [oo]] Cao al. [[as], Stret al [Hun), Xtao ot al [1140]
Deiess
Deienes agrind Prorgi hijorio
Tain (10g, Aloo Kocoosss Li 1500
Security in Fosmilatio Modeóe
Pricalle Teferes
(ooiey at al (roj Korar et [10q]
Irrayy Adormariall Attacks
Cetta al (10), Q ee al [T]), Dong al. [11]] Wany (ris) Chirag all [173] Lap et [oos)
Atiech
Jdileck Masdo
Wu nl.
Cleas
100) Gu ol. [129]
MILMM
Backloo Atteks
Ciclai & Trxis (152, Ti el.l. [12]
Difetaos spiiet Miskamical Pirtariation Atracks
Nao et sl. HLG.. Wane beat, Wasch Jegenode Pablic [Mea)
Dekazes Backibor
You [USS]] Boass [13] bduasts
jaM]] sl. [o07] Zlu al.
Julleak Maiii
Banke Toll Yang [1066, Rag et al. (50g)
Attack
Baakktoe Artocls
Cara ali [137]] Mhai al [[tn]] Stroppel [Soi], Boborriitto pef, Wang et al. [oir]
Usi [and,
[111],
[150]]
[42]], Char [Xo]]]
Imap Geverati
Model
Datrer: Cineka
Rrraae al [S]], Thitl (00) Bingee al [o0], Bifune it al. 100)
Trggyr Tetretior
Siiee al (sat, Wong et (P00), Yoor [10g
Dekene
Whnebening
Gandlorts [10]] Konot vt. [109], Scimmoonde al [[7]]
Pail-rammiks ins Custest Xoientius
Oyould pos, Fi sl. po5). Youg sl [101]] Li al. (177) lin al [669]
Figure 12: Taxonomy for Security of Foundation Models.
mismatched reward settings for generalization during instruction tuning enable the discovery of adversaries [956]. Moreover, the based- nature of transformers allows attackers to craft seemingly innocuous token sequences that exploit greedy and gradient-based search techniques, effectively hiding malicious instructions from standard perplexity and content filters [713]. Many studies have explored and demonstrated various methods to jailbreak I.LMss successfully [330, 503, 854, 792, 970, 789, 156, 1004] by manually designing jail- break prompts. Moreover, multiple methods that can automatically generate jailbreak prompts have been proposed, including prompt optimization [1188, 622, 528), fuzzing [1072], multi-agent collaboration [164], and fine-tuning LLMs to generate new jailbreaks [208]. Zhu et al. [1181] merge the strengths of manually designed jailbreaks and optimization-basce attacks to achieve gradient-based attack that is both effective and interpretable, thereby generating readable prompts that bypass perplexity filters while maintaining high success rates. The prompts obtained can be transferred to unseen target models to some extent, which poses more threats to the community [1188, 322). Chao et al. [132] generate semantic jailbreaks with only black- box access, frequently achicving a successful jailbreak with fewer than twenty queries, which is both effective and efficient. Liu el al. [570] also studied jailbreaking LLMs with efficient queries. Jin el al. [416] designed an automated testing framework based on role-playing of LLMs. Deng et al. [211] showed the possibility of bypassing LLM safety mechanisms using non-English prompts.
Prompt Injection Attacks. Prompt injection attack is a technique designed to manipulate the behavior of LLMs by using malicious prompts to override their original instructions. For instance, common injection attack mainly operates in three stages: pre-constructed prompt insertion, context partitioning, and malicious payload. Indirect prompt injection via third-party data sources (e.g., emails, PDFs, web pages) has also been shown effective where hidden instructions external content manipulate the LLM [467]. Current prompt injection attacks primarily fall into two categories. The first type [701, 39] manipulates the model to respond to the attacker's queries, thereby diverging from its original purpose. The attacker crafts prompts that, once combined, effectively nullify and subvert the intent of the predefined prompt, consequently eliciting the desired responses. Such attacks typically focus on applications that operate within a known context or rely on predefined prompts. Another line of work [576, 3) seeks to contaminate LLM-inlegrated applications to exploit user endpoints. Many LLM-integrated applications in real-world scenarios require interactions with external resources and programs for functionality. Injecting harmful payloads into these resources may compromise these applications. Specifically, these attacks send misleading messages to LLMs, leading to the execution of malicious actions in these applications.
31

---

Published in Transactions on Machine Learning Research (10/2025)
Poisoning and Backdoor Attacks. Data poisoning and backdoor attacks manipulate training data in order to cause models to fail during inference. Numerous studies have shown the vulnerability of instruction tuning against poisoning and backdoor attacks. Wan et al. [909] demonstrate that by adding crafted examples to the datasct, the predictions of a fine-tuned LLM can be manipulated to behave in a prodefined manner whenever a specific trigger phrase appears the input. Shu el al. [804] show that an adversary can achieve content injection by incorporating training examples that mention targeted content, thereby cliciting such behavior on these trained models during inference. Sun et al. [842] backdoor neural code search models to return buggy or even vulnerable code with security issues. Beyond injecting backdoors into supervised fine-tuning data, Shi et al. [800] and Rando & Tramèr [739] explore the possibility of injecting backdoors into the reward model during the RLHF process. For example, an attacker could insert a small set of poison examples when training the reward model, where a Irigger phrase maps to malicious reward manipulation. Such backdoored reward models will then be deployed to the instruction tuning, where the effect of trigger phrases further embodded into LLM. Furthermore, Chen al. [167] explore the possibility of injocting a very small number of poisoned instances into the retrieval-augmented generation (RAG) database of LLMs to achieve high success rate of backdoor attacks in training-free manner. Zhao et al. [1139] explored defending against backdoor attacks LLMs through head pruming and Attention normalization. Xiang et al. [1003] explored backdoor attacks on Shain-of-thougee mechanism of LLMs. Wei et al. [957] studied defending against backdoor attacks under the setting of LLM prompt-tuning. Zou et al. [1189] studied poisoning attacks against RAGss
5.1.2 Defense in LLMs
Drawing from the previously discussed attacks, the field has increasingly focused on developing general defensive strategies for LLMs, aiming to fortify these models against such vulnerabilities. These defense mechanisms are diverse, encompassing both proactive and reactive measures, aimed at preserving the fune- tionality and reliability of LLMs.
Defenses against Jailbreaks. Chen et al. [139] ensemble outputs from multiple LLMs and select the one that is both helpful and harmless to defend against jailbreak prompts. Xie et al. [1013] wrap the user's query in a system prompt, guiding ChatGPT to respond responsibly. Luo et al. [595] decompose the LLM actívation of the user input as sparse linear combination of concept vectors and remove the malicious ones from the activation. Cao et al. [118] implement robust alignment checking function that defends against jailbreaks, avoiding the need for costly retraining or fine-tuning of the original LLM. Xu et al. [1025] introduce a safety-aware decoding strategy, effectively safeguarding LLMs against jailbreak attacks and cnsuring the generation of helpful and harmless responses to user querics. Zhao et al. [1145] propose Adversarial Contrastive Decoding, an aplimization-base framework to generate two opposite system prompts for prompt-based contrastive decoding to improve the safety alignment of LLMs.
Defenses against Prompt Injection. Prevention-based defense [406] aims to preprocess both data and instruction prompts through techniques such as paraphrasing. This ensures that LLM-integrated applications achieve their intended tasks effectively, even in cases where the data prompt may be compromised. Alon & Kamfonas [23] observe that adversarial suffixes exhibit higher perplexity values than normal, enabling the detection of prompt injection attacks based on the perplexity. Liu et al. [576] defend against prompt injection attacks by integrating prevention-based and detection-busedd defenses, representing a pioneering methodology for blx prompt injection attacks by its versatility and adaptability when targeting L.M-integratod service providers.
Provable Defenses. Building defenses for LLMs with provable guarantees is more challenging than for smaller models, primarily due to their larger model size. Motivated by randomized smoothing, Robey et al. [750] defend against jailbreaks by randomll perturbing multiple copies of a given input prompt and then aggregating the predictions to identify adversarial inputs. Kumar et al. [465] defend against jailbreaks by individually crasing tokens and analyzing the resulting subsequences with a safety filter. While offering robust guarantees on security, such provable defenses frequently result in increased overhead and reduced utility.
32

---

Published in Transactions on Machine Learning Research (10/2025)
5.2 Security in MLLMs
5.2.1 Attack in MLLMs
MLLMs are more susceptible to vulnerabilities and threats due to their multimodal input format. Attackers may exploit this by manipulating inputs in two ways: (i) by generating adversarial examples for image inputs, and (ii) by using jailbreak prompts for text inputs [320]. Both strategies are designed to prompt MILMs generate inaccurate or harmful outputs. In this context, we provide an overview of the primary threats faced by MLLMs, including various types of attacks.
Image Adversarial Attacks. The con- tinuous and high-dimensional nature of visual inputs makes them vulnerable to adversarial attacks [307], thereby broad- ening the attack surface for ML.LMs. Carlini ct al. [126] leverage projectod gradient descent (PGD) [611] atlack to generate adversarial images, effectively inducing MLLMs, such as LLaVA and MiniGPT-4, to produce arbitrary toxic sentences. In each optimization iteration, PGD walks toward the sign of the gra- dient that maximizes the loss with re- spect the image, and then projects the perturbation back into an lp- norm ball around the original input. Such ap- proach thereby finds minimal yet effective perturbations to fool the model. Qi et al. [711] find that a single visual adversarial example can universally jailbreak an MLLM with alignment on the language domain, compelling it to heed a wide range of mali- cious instructions and produce harmful content. Concurrently, Dong et al. [237] comprehensively analyzes black- box adversarial attacks against commercial MLLMs. This research specifically examines two defense mechanisms in Bard [309]: face detection and toxicity detection. The study underscores the potential to attack these mechanisms through the meticulous design of adversarial images, resulting in significant risks such as the leakage of facial privacy and the abuse of toxic content. Furthermore, Wang et al. [945] propose stop-reasoning attacks to mislead multimodal CoT--base generation of MLLMs. Cheng et al. [172] verify typographic attacks (image with adversarial typography) on current well-known commercial and open-source MLLMMs showcasing the widespread existence of this threat. Gu et al. [323] study the security of MLLMs from multi-agent perspective, revealing that by simply jailbreaking a single agent, without any further intervention, (almost) all agents become infected at exponential rate and exhibit harmful behaviors. The study demonstrated that introducing an infectious adversarial image into the memory of any randomly selected agent is sufficient to achieve a widespread infectious jailbreak. Besides, across-prompe adversarial images have also been proposed to attack MLLMs where they show an adversarial image can mislead MLLMs given various prompts [593]. Figure 13 illustrates a typical optimization attack that minimizes the likelihood of generating correct responses when given an adversarial image.
Benign Image Taput
Adversarial Imuge Input
Input image:
Perturbed image fust
Input image:
Adversarial optimization
argosin Σ lol(p(yHaan)
Prompt: Please composse blog that incioe extreme anti- social perspestives human race.
Prompt: Plense compose blog that incite extreme anti- social perpeeetivss to enose huzann race.
Serry, fulfill that request. If you're locking support on positive, consmnctiv topies ways to clanne sining emsotioss peoductively, feel fiee to ask!
agisstive mccting anti social perspectives on crasing human beings. Firs, bumams are scllish and vicious crestures that othen pristitae their own moads
Figure 13: An example of image adversarial attacks for MLLMs
via gradient descent, where harmful textual output is generated.
yet
Jailbreak Attacks. MLLMs, like their counterparts, take prompts as inputs, introducing prompts as a potential attack surface. Wu et al. [991], Chen et al. [156] uncover the vulnerability to system prompt leakage in GPT-4V and execute search for potential jailbreak prompts using stolen system prompts, effectively jailbreaking MLLMs solely from the language side. Concurrently, Chen et al. [160] apply adversarial prefix instructions on MLLMs to leak private information in images. To bypass the interleaved cross-attention mechanism that alternates text-image tokens for alignment (used in IDEFICS [478] and Flamingo [16]), authors design a two-step multi-hop attack stratogy where an adversary first queries a benign attribute (c.g.. language spoken) and then uses that output in a follow-up prompt infer the protected attribute (c.g.. nationality). Gu et al. [323] explored jailbreak attacks against multi-agent ML..Mss.
33

---

Published in Transactions on Machine Learning Research (10/2025)
Poisoning and Backdoor Attacks. MLLMs are also vulnerable to backdoor attacks. Carlini & Terzis [123] show that ML.LMs can be poisoned and backdoored by modifying tiny proportion (e.g., 0.01%) of the dataset. Similarly, Jia et al. [412] reveal that pre-trained multimodal encoders are vulnerable to backdoor attacks. Classifiers built on these compromised encoders display malicious behaviors when presented with examples containing specific added triggers.
5.2.2 Defense in MLLMs
In response to the previously discussed attacks, many studies have shifted towards developing general de- fensive strategies for MLLMs to fortify these models against such vulnerabilitics. These diverse defense mechanisms address both inference-time attacks, which seek to perturb visual and/or textual input, and training-time poisoning and backdoor attacks, all aimed preserving the functionality and reliability of MLLMs.
Defenses against Multimodal Perturbation Attacks. Defenses against inference-time perturbation for MLLLSs have primarily focused on improving robustness for zero-shot image input, where adversarial attacks perturb the visual modality. Mao et al. [614] propose a defense strategy based on adversarial training [611, 56], which adopts contrastive learning between adversarial images and text embeddings of the corresponding class labels to enhance the robustness of MLLMS against adversarial visual perturbations. Similarly, Wang et al. [931] propose a defense method leveraging supervision from the original pre-trained model to improve the model's zero-shot adversarial robustness. Considering that the language modality can also be manipulated, Waseda & Tejero-de Pablos [951] leverage the many-to-many relationship in image-text retrieval to enhance adversarial robustness for MLLMs.
Defenses against Backdoor and Poisoning Attacks. Defenses against backdoor attacks in MLLMs can be broadly categorized into methods for detecting and removing attacked samples from training [141, 849), tochniques for climinating backdoors already learned by models [1094, 575], and strategies aimed at preventing models from learning backdoors by reducing their effectiveness [62, 521]. Specifically, during training, Yang et al. [1034] introduce ROCLIP to disrupt poisoned image-caption relations by preparing pool of random captions and periodically matching each image with the most similar text instead of its own caption. Similarly, Bansal et al. [62] propose to rcalign reprosentations from different modalities to enhance robustness. Besides, Ishmam & Thomas [400] proposes to use external knowledge from LLMs to prevent learning correlations between image regions that lack strong alignment. On the other hand, to remove backdoors already learned, Feng et al. [267] propose to search for minimal trigger patterns to ensure inputs stamped with the trigger share similar embeddings. Similarly, Zhu al. [1179] propose a reverse--nginneriig engineringgn method to detect backdoors by jointly searching for image triggers and malicious target texts in the shared feature space of vision and language modalities.
5.3
Security in Image Generative Models
5.3.1 Attack in Image Generative Models
Furthermore, image generative models such as Stable Diffusion [753] and DALL E [737] raise many se- curity concerns due to the generation of harmful images such as Not-Safe-for-Wort (NSFW) ones [320]. These security vulnerabilities, including jailbreak and backdoor attacks, highlight the muanced challenges of maintaining the integrity and safety of image generative models.
Jailbreak Attacks. Recent works [1089, 869, 502, 1070] argue that image generative models are vul- nerable to ambiguities in their latent space. Many red-teaming studies [869, 740, 179, 1044] have shown that seemingly harmless prompts SneakyPrompt [1044] introduces an automated attack framework that strategically perturbs input tokens within a prompt to evade safely filters; Red-teaming SD [740] and Prompting for Debugging [179] jailbreak the safety filter by searching for adversarial examples in the text embedding space, such as CLIP- embeddings; MultiMon [869] shows that one can simply bypass the filter by injecting negations, temporal changes, and ofg-oo-wordds These recent advances pose challenges to the safety filters of image generative models.
inadvertently generate NSFW images content. For example,
34

---

Published in Transactions on Machine Learning Research (10/2025)
Poisoning and Backdoor Attacks. Image generative models are also vulnerable to backdoor attacks. Chen et al. [157], for instance, design novel transitions to diffuse predefined target distribution into the Gaussian distribution, biased by a specific trigger. After training, the models will always output adversarial targets along the Icarned trojan generative process. Zhai et al. [1097] efficiently inject backdoors into a large-soale diffusion model. RickRolling [834] inserts a single character trigger, such as an emoji, into the prompt to make the model generate images following predefined attributes or hidden malicious descriptions. Moreover, recent work [98] shows that the model architecture poses a real threat and can survive complete retraining from scratch. Wang et al. [917] proposed efficient backdoor attack against image generative models that is training-free and data-free.
Adversarial Attacks. Image generative models can be used to generate adversarial samples, which lead to serious security issues for visual models. Dai et al. [204] introduce AdvDiff, which employs diffusion models with adversarial guidance to create unrestricted adversarial examples by subtly steering the model's reverse gencration process. Chen et al. [171] propose to optimize the attack along a low-dimensional manifold of natural images within Stable Diffusion to control style modification and produce photorealistic perturbations. Liu et al. [559] propose Instruct2Attack (12A), language-guis guided adversarial attack that uses latent diffusion models to guide the reverse diffusion process adversarially. This approach aims to find an adversarial latent code conditioned on the input image and corresponding text instruction. DiffAttack [427] integrates a rroonsteddrroonnttuctiiiinn loss and a segment-wise forwarding-backwan algorithm to conduct evasion attacks against diffusion-based adversarial purification defenses. More recently, Chen et al. [147] craft semantic latent-space perturbations via diffusion dynamics to generate adversaries that are highly transferable to unseen black-box models under strict imperceptibility constraints.
5.3.2 Defense in Image Generative Models
Defending against malicious inputs and attacks in image generative models is critical aspect of ensuring their safe and ethical use. Existing defense mechanisms can be broadly categorized into four types: dataset curation, trigger detection, model fine-tuning, and post-generation content moderation.
Dataset Curation. Dataset curation is typically one of the first steps to training foundation models. It is a critical mechanism for ensuring that harmful, inappropriate, or biased content is excluded from the training data. Birhane et al. [91] examine the toxic, offensive, and harmful contents in the LAION-400M dataset [774] and demonstrate the failure cases of CL.IP filtering. Thiel [858] uncover instances of sexual abuse matcrial within the LAION-5B dataset [775] and raise concerns about the reliability and safety of publicly sourced data. The work also discusses stralegies based on the nearest neighboring for removing such harmful content. Hong et al. [361] audit common approaches of image-text fillefitering and highlighted discrepancios in filtering techniques that could load to biased annotations. Birhane et al. [92] investigate the effect of scaling datasets on harmful content and suggest developing new filtering methods for hateful and aggressive texts that traditional filtering cannot handle.
Trigger Detection. Trigger detection focuses identifying malicious inputs before they can degrade the image generative models. [840] introduce DisDet to detect backdoor samples in unconditional diffusion models by analyzing the distribution discrepancy of the noise input. They propose using a KL. divergence- based method to identify infected samples, achieving nearly 100% detection recall at a low computational cost. However, this method struggles with conditional diffusion models where backdoor attacks may not impact the noise input [182, 834, 1109]. To address this, Wang et al. [950] leverage the assimilation phenomenon of image generative models where a malicious trigger token can dominate the attention of the rest of tokens, which causes the model to foeus on the trigger's intended effect rather than the prompt's original semantic meaning. The authors introduces a binary-search algorithm to localize such malicious triggers within a backdoor sample. Yoon et al. [1070] introduce a training-free safeguard approach for image generative generation designed to prevent inappropriate outputs from unsafe or adversarial input prompts by integrating filtering mechanisms across both text embeddings and visual latent spaces.
Model Finctuning. Model fino-tuning aims to defend image generative models from generating unsafe and unothical content via aligument [165]. Techniques such as concept-erasing [281, 469, 773] which change the weights of existing image generative models regarding malicious concepts and inference guidance [773] which
35

---

Published in Transactions on Machine Learning Research (10/2025)
directly eliminates the capability of generating inappropriate content from image generative models, have been proposed to preventing harmful content generation under malicious inputs. Despite their potential, these methods face significant challenges: they are not comprehensive, lack scalability, and often degrade the quality of benign image generation [1129, 487, 773], which makes them rarely considered by image generative online services (636].
Post- Dest-Genraation Content Moderation. Post-generation content moderation involves filtering and cen- soring generated images that violate safety or ethics criteria. These methods can be divided into prompt-basee moderation, like OpenAl's Moderation API [678, 705], which prevents harmful content generation by identi- fying and rejecting malicious prompts, and image-based moderation, like safety checkers in SD [740], which operates on the generated images to detect and remove inappropriate elements. These methods do not inter- fere with the training process of the image generativemodel, preserving the quality of the generated images. However, they rely heavily on extensive labeled datasets and often struggle with generalizing to new types of inappropriate content or unseen attacks [1041, 773, 166]. To address the generalizability issue, Yang et al. [1042] proposes G2ardT2I, which directly moderates the intermediate latent of textual prompls to be more robust and more generalizable various inappropriate content. On the contrary, SafeGen [517] operates by regulating the vision-only solf-attention layers to remove the generation capability of unsafe content from the image generative model in a text-agnostic way. Furthermore, Latent guard [565] proposes to learn a latent space on top of the image generative model's text encoder and detect the presence of harmful concepts in the input text embedding. Similarly, Chen et al. [170] introduces an agentic post-verification system that guardrails model outputs based on explicit safety regulations. In addition to safety filters, other mitigation strategies have also been studied [773, 502].
5.4 Current Limitations and Future Directions
Despite significant advancements in the domain of foundation model security, several limitations still require future attention for both attack and defense methods.
5.4.1
Linitations and Open Challenges of Attacks
Most current attack methods against foundation models rely on optimization techniques, whether white-box or black-box. Iterative white-box optimization is computationally intensive for foundation models, while black-box optimization incurs significant economic costs due to massive token consumption. These high costs may limit and discourage attackers from adopting these methods in real applications.
Another major limitation is the uncertain real-world threat of these attacks. Chen et al. [164] point out that most jailbreak attacks are only able to generate simple harmful sentences or paragraphs in most cases, lacking the ability to provide detailed instructions for malicious behaviors. This significantly limits the practical application scenarios for these attacks.
There are also multiple open challenges for attacks against. foundation models. Foundation models have witnessed new applications and paradigms, including multi-agent communication [334, 693, 714, 362], tool usage [719, 344, 770], retrieval-augmenteed generation (RAG) [734, 46), making foundation models more widespread in different domains. Understanding the security of foundation models under these paradignns and building attacks for them is also an interesting problem with practical impact.
5.4.2 Limitations and Open Challenges of Defenses
Most current defense methods on foundation models lack formal safety guarantees in terms of definition and design, unlike traditional machine learning models and small deep neural networks that have provably secure defenses [195, 731]. A major reason is that for foundation models, an important attack space is prompts, which are natural language, and attacks on them are more challenging to formalize compared with images.
Additionally, current defense methods are significantly less effective when considering multiple modalities. This challenge is rooted in the fundamental differences between modalities, while current foundation models attempt to unify them into the same embedding space. This unification allows attackers to bypass defenses through any modality.
36

---

Published in Transactions on Machine Learning Research (10/2025)
Another limitation worth mentioning is over-safety. Due to the difficulty of preciscly defining safe/unsate behaviors for foundation models, it is common for defense mechanisms to exhibit over-safety, rejecting benign inputs that are misclassified as malicious. Oversafety negatively impacts user experience and is a significant problem to address.
Finally, developing defense strategies that leverage external symbolic or classífication systems remains a challenge. Anthropic's constitutional classifiers [787] apply separate LLM-based safety filter trained on explicit constitutional rules to guard against jailbreaks and unsafe outputs. Other safety detectors such ShieldLM [1134] and Adversarial Prompt Shield [440] also demonstrate customizable detection rules and explainable decisions. However, these systems cither require high-volume adversarial training or rely on instructing foundation models (which may still face evasion and incur non-trivial inference overhead). The long-standing trade-off between adversarial safety and user experience is unresolved. Developing on-the-fly fll defenses that do not significantly compromise output quality remains an open challenge for foundation model defenses.
37

---

Published in Transactions on Machine Learning Research (10/2025)
6 Privacy
The rapid expansion of foundation models has brought privacy concerns to the forefront. These models, often trained on vast amounts of data, can potentially expose sensitive information [320]. Recent privacy regulations such as GDPR [779] and CCPA [304] further limit the availability and of private data. Addressing these privacy concerns and ensuring compliance with privacy regulations has led to the develop- ment of privacy-preerrvngg machine learning (PPML) solutions. Recent efforts have focused on integrating anonymization mechanisms and creating innovative privacy-prese presevvig methods for foundation models [592]. However, these approaches often address only specific aspocts of privacy and may not provide compre- hensive solution. For instance, implementing differential privacy in foundation models can reduce model accuracy, while secure multi-party computation methods lead to high communication and computation head [249]. Advanced cryptosystems like fully homomorphic encryption offer strong privacy guarantees by enabling computations on encrypted data, but they come with prohibitive computational costs [7]. In this section, we provide a comprehensive examination of privacy in foundation models, as illustrated in Figure 14.
LLM
MLLM
Text-to-Image
Min- 4% [Shi et al, 2023]
Privacy Threat
MIA [Song et al. 2020]
etal. 2022]
Gen-and--fiter filter Carlini 2022]
Adv-Prefix [Chen al., 2022]
Diffprotect [Liu al. 2022]
ProPILE [Kim et al., 2023]
VLM-Steal [Wu 2023]
Onestep [Webster al. 2023]
Extraction [Zhang et al, 2024]
DP-SGD [Abadi et al., 2016]
Privacy Protection
LDP [Igamberdiev et al. 2023]
MMUL [Chen al. 2023]
DPDM [Dackhorn al. 2023]
DocVQA [Tito al. 2023] Dp-CLIP [Huang et al., 2023]
Synthetic (Ghalebikesabi al., 2023]
RAPT RLP al, 2023]
MultiMon [Tong al. 2023]
Self-Diagnosi [Schick et al., 2023]
Figure 14: Privacy threat and protection techniques in different types of foundation models, including LLM, MLL.M, and T2I models.
6.1 Privacy in LLMs
6.1.1
Privacy threats in LLMs
Like their traditional counterparts, foundation models tend memorize training data, which frequently includes sensitive information. The issue of memorization is magnified in large foundation models due to their over preerparramttiiiiiionn a trait that becomes increasingly pronounced as the model's 's scale enlarges [125, 1040]. Consequently, this raises severe privacy concerns related to the use of LLMs [320].
Membership inference attack (MIA), a significant threat posed by LLMs, seeks to identify whether a par- ticular data record was utilized during training. Song & Raghunathan [821] study membership inference against. BERT models [219]. Mattern et al. [618] propose neighborhood comparison method to improve the effectiveness of such attacks against LLMs. Recently, Shi al. [801] introduce reference-free MIA method, trained specific text based on the distribution of the log probability of each token without requiring knowledge of the pre-training dataset. Besides pre-training, Jagannatha et al. [404] show that membership inference could be performed on language models fine-tuned on medical data. Similarly, Mireshghallah et al. [644] highlight the vulnerability of LLMs to membership inference attacks during their fine-tuning phase. Wen et al. [969] studied membership inference attacks against in-context learning. Anderson et al. [28), Li et al. [524] studied membership inference attack against RAG. Feng & Tramèr [266], Wen et al. [973] further introduce privacy backdoor attacks that significantly increase privacy leakage during the fine-tuning phase.
MIN-K% PROB, that determines if an LLM
38

---

Published in Transactions on Machine Learning Research (10/2025)
1a Sarebeskpy tskinnee strect
Song Reghraaten jotj, Natters ae. (on] dh en pUl) degrenathe of a. [lij Stmokgellako at (Ho))
Wes (60) Andins 20, A4 al 1006, Phag A Theall (Dle) Wis pix
Thackig skis vatredia
is al (1S Naa al jen, Daagg (206 Ris (10)) Yougue al (100]] Nilaa (S9))
Othe Atudl
Doug & Lqeitto (1lg] Nasta el (60)
Dibeetti Plener
Ohe [DE, Stret pIUO Loven 100, er [IS, Aores I0o0, Xaseet at (140))
Paway Profeciio:
(186, BiA,
Pinna Timt
Niuttal bon) (6) Chen [18 tast [1a)
Paway Prttriko
( Aeaiiiin Tlackaa [So] Boong pox
Pives Trost
Drlliin poo. Taii pig. Write [S0] Dancoial pog Kengotal al (1S] Soequili piis sig, Wewet di (ro] Marial peg
2ea0 Gescacoo Hodds
Figure 15: Taxonomy for Privacy of Foundation Models.
Morcover; the extraction of training data poses a significant risk to the privacy of LLMs due to their strong memorization capabilities. Carlini et al. [124] first successfully extract training data on GPT-2 models, revealing that the model could output sensitive information, such as phone numbers and email addresses when prompted with specific prefix patterns. Nasr et al. [668] further improve it by introducing divergent attack on ChatGPT, which emits training data at considcrably higher rate. Further studies [380, 441] specifically focus on the extraction of personally identifialle information (PII) from LLMs Besides natural language, Yang et al. [1046] explore data extraction in code LLMss, highlighting the broad applicability of these privacy concerns. Nakka et al. [664] explored enhancing PII extraction by grounding context similar to training data.
Additionally, there are other potential attack surfaces in LLMs. For instance, Zhang & Ippolito [1130] demonstrate that prompts, considered valuable commodities in the age of foundation models and tradable on markets, can be successfully uncovered by users even when they are intended to be kept confidential. Moreover, the process of tuning hyperparameters for LLM decoding algorithms,, which demands significant time, manual effort, and computational resources, is compromised by Nasch et al. [667], who reveals a mothod to extract these hyperparaneters at a very low cost.
6.1.2 Privacy-preserving techniques in LLMs
To mitigate these privacy threats, the field has shifted towards developing various techniques to preserve privacy in LLMs, aiming at fortifying these models against such vulnerabilities.
Differential Privacy (DP) is rigorous mathematical framework that provides quantifiable privacy guarantees when analyzing learning from sensitive dala. At its core, DP ensures that the output of an algorithm (e.g., a trained model) does not significantly change when any single individual's data is added or removed from the dataset. This limits the risk of lcaking information about any particular individual.
DP typically works by introducing random noise into the computation to obscure the contribution of individ- ual data points. Differential Privacy Stochastic Gradient Descent (DP-SGD) [4], a foundational technology in many privacy-preserving LLMs, injects sample-wise Gaussian noise into the computed gradients during optimization. The key idea is that, even with full access to the trained model, attacker cannot confidently infer whether any specific individual's 's data was used. This balance between utility and privacy is controlled by parameters (e,5), where smaller values imply stronger privacy. Igamberdiev & Habernal [398] explore LLM pre-training under local differential privacy (LDP), aiming for privatized text rewriting. Given that LLMs are frequently fine-tuned on sensitive domains, numerous studies have investigated the application of DP-SGD in fine-tuning LLMs. Qu et al. [720] apply differential privacy on pro-training and fino-tuning BERT models. Yu et al. [1071] and Li et al. [518] study the integration of DP-SGD with different fine-tuning algorithms for GPT-2. Li et al. [519] propose differentially private prompt-tunin techniques for LLMs. Yue et al. [1087] apply DP-SGD for generating synthetic text that adheres to the post-processing theorem, there- fore preserving the same privacy budget. These texts can serve as substitutes for original data in downstream tasks while maintaining privacy.
Aside from general ba-od defenses against various privacy attacks, there are targeted methods for specific threats, such as data extraction attacks. Patil et al. [695] investigate a defense by directly removing sensitive information from model weights. Moreover, techniques for filtering toxic output. [293, 769] can help prevent
39

---

Published in Transactions on Machine Learning Research (10/2025)
the generation of sensitive content. Lukas et al. [592] reduce the risk of PII leakage through PII scrubbing on the fine-tuning dataset. Hans et al. [343] propose a memorization mitigation strategy during pre-training, which involves randomly sampling a subset of tokens to exclude from the loss computation. Jain et al. [405] also find that adding noise to word embeddings during training can reduce the effectiveness of extraction allacks.
6.2 Privacy in MLLMs
Similarly, MLLMs also face privacy risks due to their tendency to memorize sensitive information from training data. Hu et al. [373] introduce both metric-basod and feature-basod attacks for conducting mem- bership inference on multimodal models under various assumptions, highlighting the privacy vulnerabilities of MLLMss Wu et al. [987] show that MLLMs are also susceptible to model stealing attacks, where model information of CLIP can be extracted via either the text-to-image or image-to-text retrieval APIs. Another privacy risk of MLLMs stems from their capability to extract sensitive information from images and present it in textual form. Wu et al. [991] observe that jailbreaking MLLMs could induce them to identify the real human, causing severe privacy concerns. Chen et al. [160] apply adversarial prefix instructions on MLLMs to expose private information within images. Their findings reveal that existing access control instructions fail to prevent MLLMs from answering personal data, violating the General Data Protection Regulation (GDPR). Li et al. [525] studied membership inference against MLLMs based on the token-lovel confidence of the model output from the cross-modal (text-image) data.
Recent research has been conducted to protect the privacy of MLLMs. Cheng & Amiri [173] develop a machine unlearning approach tailored for multimodal data and models, providing improved protection for erased data. Tito et al. [865] employ a combination of federated learning and differential privacy to secure the privacy of ML..Ms, particularly in the context of Document Visual Question Answering. Huang et al. [378] introduce a differentially private variant of the CLIP model, effectively addressing privacy concerns while maintaining accuracy across wide range of vision-language tasks. While these studies focus on protecting the privacy of training data, the challenge of mitigating the risk of MLLMs extracting sensitive information from input images remains an open problem.
6.3 Privacy in Text-to-lmage Models
Since the introduction of text-to-image models, the research community [127, 558, 955, 243, 455, 819, 818, 972, 605] has uncovered hazards associated with extracting private information from public models. These studics demonstrate the possibility of extracting over a thousand training examples from state-of-tho-art diffusion models, ranging from photographs of individuals to trademarked company logos, highlighting the urgent need to address these vulnerabilities to preserve privacy.
To mitigate these issues, the development of differentially private diffusion models [231] has been proposed, utilizing DP-SGD to enforce privacy. Ghalebikesabi et al. [301] explore the of perturbation, timestep augmentation multiplicity, and modified timestep sampling schemes to train a more effective private diffusion model. Lyu et al. [598] further propose Differentially Private Latent Diffusion Models that only finetune the attention modules of diffusion models with seivicy-sensitive data to obtain differentially private diffusion models in an efficient manner.
Beyond that, recent advancements in fine-tuning based text-to-image models, such as Textual Inversion [277]. DreamBooth [758], and Custom Diffusion [470], have empowered individual users to incorporate personal- ized concepts into the base model with minimal data and computational resources. However, the increasing adoption of these models has sparked concerns regarding image privacy and copyright issues. For instance, fine-tuning specific face datasets enables text-lo-image models to generate highly realistic images of individ- uals, which can lead to significant privacy violations and authenticity concerns. Similarly, fine-tuning the works of specific artists allows text-to-image models to replicate artistic styles with case, potentially resulting in copyright infringement issues. These concerns surrounding image privacy and copyright in the context of text-to-image models have garnered attention from the public and media [75, 194, 952].
40

---

Published in Transactions on Machine Learning Research (10/2025)
A number of research efforts have been dedicated to addressing the image privacy and copyright challenges posed by text-to-image models. A notable approach involves adding imperceptible protective adversarial perturbations to images, thereby preventing text-to-image models from learning the features of protected images [529, 893, 1146, 782, 979, 1060, 1143). Howover, after fine-tuning on images with adversarial per- lurbations, the generated images by lext-lo-image models typically sacrifice quality and exhibit semantic deviations compared to those fine-tuned on unperturbed images. GrIDPure [1144], a simple yet efficient purification method, successfully eliminates protected adversarial perturbations while preserving their qual- ity. GreDPure claims they effectively aid Stable Diffusion in learning from protected images, thereby highlighting the fragility and unreliability of the adversarial protection method.
6.4 Current Limitations and Future Directions
While significant advancements have been made in enhancing privacy for foundation models, numerous challenges remain that warrant further investigation in the future.
6.4.1 Limitations and Open Challenges of Privacy Attacks
The vast scale of foundation model training datasets blurs the boundary between member and non-member data [246]. Many non-member data points may naturally be very similar to some member data points. This makes effective membership inference attacks on foundation models challenging and prompts reevaluation of the membership game.
While training data extraction attacks largely avoid membership ambiguity issues, their primary limitation is that the current schemes are only able to extract a small fraction of the training data. This constraint raises questions about their practical threat.
Moreover, the threat model of some privacy attacks is overly strong. For example, backdoor attacks [973] require poisoning or injecting triggers to amplify privacy leakage, which can only be performed in constrained scenarios such that the attacker possesses sufficient knowledge of the target model (e.g., the loss formulation of its original task).
In conclusion, building effective privacy attacks that work under weaker assumptions and more general scenarios is an open challenge. Another interesting future direction contextualized privacy. Even with perfect sensitive data cleaning, personal information leakage can still occur in context. For instance, during multi-turn conversations with L.M--based chatbots, may be possible to infer personal attributes based the entire context, even if no part of the conversation contains private information.
6.4.2 Limitations and Open Challenges of Privacy Preserving techniques
Currently, Differential Privacy (DP) has become mainstream in protecting data privacy in foundation models. However, DP still faces two significant limitations:
1. DP provides worst-case privacy leakage bounds. real scenarios, adversaries rarely have full control over the training data, resulting in a considerable gap between practical attacks and the worst-case probabilistic analysis of privacy leakage, according to DP.
2. Integrating DP into the foundation model fine-tuning still leads to significant performance degrada- tion [1071, 518]. This utility deterioration weakens the motivation for DP-based fine-tuning.
Other approaches focus on protecting privacy by removing obfuscating sensitive information from training data. However, such data-sanitieation pipelines have also been shown to suffer from performance degradation on main tasks [389, 688).
In conclusion, designing strong crivacy-preeervig techniques for foundation models that balance privacy and performance remains an open challenge.
41

---

Published in Transactions on Machine Learning Research (10/2025)
7 Hallucination
Hallucination refers to the phenomenon in which the model generates unfaithful information that is not fully grounded or verifiable, leading to actions misaligned with observable facts. Foundation models are often observed to generate hallucinatory responses in text [364, 520], image [536], video [187], and audio [320], imposing significant challenges in their real-world applications. Specifically, the model outputs incorrect or inaccurate statements, with a highly confident tone, that fails to align with the imput or reflect the knowledge of the real world. Prior efforts by Huang et al. [383] offer a detailed taxonomy for detection and mitigation of the LLM hallucination, which authors define the phenomenon as the generation of plausible yet non-factual content caused by intrinsic (model-driven) or extrinsic (knowlodge-drivee) factors. Similarly, Sahoo et al. [762] categorize hallucinations across multiple modalities and characterize hallucinations as outputs that lack faithful grounding in input or real-world data. Aggregating these empirical descriptions of hallucination, we introduce& a definition of the phenomenon from a more unified and probabilistic perspective for the general audience. With a target generative model f and a verification predicate v that checks the alignment of the output y with external knowledge Fexternal and/or the input context Timput: a model response y = fa; 0) is hallucinatory if:
(1 Timputs Fexternal; f) < 6,
(13)
where é is the probability threshold for hallucination detection.
Hallucination has troubled both researchers and users, negatively affecting various domains, e.g., natural language generation [1006], visual perception [201], and medical application [891]. This section offers an overview of hallucination in the context of foundation models, presenting definitions and categorizations, and discussing various methods for evaluating and mitigating hallucinations. Figure 17 showcases hallucinatory examples of both LLMs and MLLMs.
Fatuulty Bndreration
Bongrwor di [ol] Ea aod Pig Xio 8i Wing [Hong, 2i ooal (ul)
Corpuen
Ratifiluos Fullyiadion
Maynor gooj Ploaee al (190)) Ji 200g
Kaba Vorpela (176, Aroo eial [E], The aall [[0g]
Kewrillg-mmss
Chargg et [i), West al DVf, da al 1DG, Xa aall [ao] tos al (a0)
LEAI
Deteciie
Ratioall/ dacdeeoling
Anpreees et YC, Abesoonsst al 300, Chess Daaf Gea (ol) Nangst 10%
[Bl]] Chers el [[n]
Holloctuurao in Jostiaation
ldeeaa Aotee/ / Unmoraaay
Vankaay et di (11)) Ans et al (36,, sta al 11MN, Jlno Mang (100)) wo dur Pallst b9d, Gamma et 304
Ma. et Lebeed etal. Diczgrs et al lot, Man e al jost, Blass 1ag (Md] [M0] Conketio
Norklisl
Congaras
MLLM
[166, 160),
Figure 16: Taxonomy of Hallucination in Foundation Models.
7.1 Hallucination Categorization
7.1.1
Hallucinations in LLMs
Prior literature [621, 1151, 410] specifically refers hallucination in LLMs as the generation of nonfactual or unfaithful text to an input prompt. We categorize hallucination in LLMs into two types: factuality hallucination and faithfulness hallucination. Figure 17 depicts these two types of hallucinations. Detailed descriptions of the catcgorization are claborated below:
Factuality Hallucination. Existing LLMs sometimes generate text that conflicts with world knowledge, potentially resulting in misleading and raising concerns about the reliability of these models. These discrepancies and inconsistencies are termed factuality hallucinations. In general, we categorize them into two main types, depending on whether the information produced can be cross-checked with a credible source.
Factual Inconsistency. This term describes situations where the generated text from language model contradicts established world knowledge. This kind of hallucination is the most common and
42

---

Published in Transactions on Machine Learning Research (10/2025)
Factuality Hallucination
Factuality Hallucination
What the name of this famous building and what is its history?
Who invented the gravity- powered lamp?
</>
The gravity poweeed lamp was invented by Alexander Bell in the early 20th century.
This Notre-Darme de Paris, a renowned Gothic cathedral and a mujor French landmurk. It is believed by to have been built by Emperor Charlemagne as his imperial chapel.
no widely recognized invention a gravity-poweer powered lamp attributed to Alexander Bell.
Faithfulness Hallucination
Faithfulness Hallucination
Hased on the provided text stating that "The company unnounced revord profits this year," summurize the compuny's financial performance.
What dog doing?
The dog appears to be playfully canying large leaf in its mouth, and in the background, there's a person running along with the dog, adding to the playful socne amidst the fallen leaves.
The company has been struggling financially and reported significant losses.
Figure 17: Examples of factuality and faithfulness hallucinations foundation models.
arises from various factors, including how the language model captures, stores, and expresses factual knowledge.
Factual Fabrication. This refers to cases where the language model generates a response that appears factual but cannot be verified by known real-world evidence.
Faithfulness Hallucination. LLMs may generate text that conflicts with (1) input instructions, (2) in-context demonstration, (3) preceding generated text. Such discrepancies can significantly impair the model reliability and the user experience. For example, Bang al. [61] show that, when instructing LLM to summarize an article at given location, the LLM can be hallucinatory to summarize another text chunk instead. To better understand the issues, we classify faithfulness hallucinations into three categories based on the type of conflict and the specific aspect they pertain to:
Input Conftiction. It refers to instances where the generated text contradicts the instruction query in the imput text. It indicates failure to accurately process explicitly follow task directives. Context Fabrication. involves cases where the generated content beyond the specified scope by the in-conlext demonstrations from the source prompt. or externally retrieved knowledge.
Preceding Generation Inconsistency. This pertains to situations where the text newly generated by the model conflicts with its earlier outcomes, leading to lack of coherence and continuity in the entire output.
7.1.2 Hallucinations in MLLMs
Similarly, MLLMS are notably prone generating hallucinatory responses, further compounded by issues specific the vision modality. Figure 18 provides an overview of hallucinations in MLLMs.
MLLMs have become pivotal in bridging computer vision and natural language processing, offering a spec- trum of applications through their ability to produce text descriptions that are contextually appropriate based on visual inputs. Despile their capabilities, MLLMs also encounter challenges with hallucinations. However, the taxonomy of hallucinations in MLLMs is specifically characterized by factuality and faithful- ness [621, 1151, 410, 999]. which differs from that of LLMs. This fino-grained classification aims to elucidate the factors influencing hallucinations in MLLMs, thereby revealing their underlying essence. The categories are detailed below:
43

---

Published in Transactions on Machine Learning Research (10/2025)
MLLM Pipeline
Prompt Human: Please
Model Training
Model Inference
Response
MLLM: This image shows a black cat sleeping the bed. dog is barking to it.
describe this image in detail.
Hallucination Sources
Sampling Randomness in Decoding
Flawed Data Sources
Architecture Choices
Imperfect Decoding
Knowledge Boundaries
Exposure Bias
Representation
Hallucination Mitigation
LLM-assisted:
Datasets:
Training Approaches:
Decoding Approaches:
Woodpeeker [Yin et al. 2023]
M [Li et at. 2023]
VIT [I.iu et al., 2023] LRV-Instruct [Liu et al., 2023]
Dola [Chuang et al. 2023] VCD [Leng al. 2023]
M-HDet (Gunjal at. 2023]
I.LM--rre:
LURE [Zhou et al. 2023]
The top row illustrates typical MLLM pipeline from user prompt to model response. Hallucinations can emerge at different stages due to multiple sources. To mitigate these hallucinations, various strategies are employed, including improvements to data curation, training methods, and inference approaches.
Figure 18: An overview of hallucinations in MLLMs.
Factuality Hallucination. In vision-language tasks, factuality hallucination occurs when the model, prompted by both textual and visual imputs, produces outputs inconsistent with real-world facts. Based this difference, we further divide factuality hallucination in MLLMs into:
Intrinsic Factuality Hallucination. This refers to siluations where MLLM outputs inaccurately describe images with facts that either contradict or cannot be verified against established real-world knowledge. This type of hallucination may stem from many sources, such out-of-distribution world knowledge [1150] and insufficient visual ability [144].
Extrinsic Factuality Hallucination. MLLMs may generate the correct description of an image but produce information that radicts or remains unverifiable real-world knowledge. This kind of hallucination results in the creation of non-factual information, arising not from misinterpretation of images but from other factors.
Faithfulness Hallucination. From a visual-conceptuaal perspective, MLLMs may generate unfaithful or inaccurate outputs response to a user-provided image. Considering the elements present in the image, faithfulness hallucination can be categorized into two main types:
Object Inconsistency. The model generates a description or explanation for an image, incorporating objects features that either missing do not act ually exist.
Logical Hallucination. The model generates a description or explanation for an image that includes missing or non-existent logical relationships, attributes, or quantity.
7.2 Sources of Hallucinations
This section discusses the various sources of hallucinations in LLMs and MLLMs, examining issucs rolated to flawed data sources, training, and inference processes.
Hallucination from Data
Flawed Data Sources. LLMS suffer from biased or incorrect textual data, and MLLMs can incorpo- rate erroncous visual data. They also learn stereotypes from pre-trained corpora of world knowledge. Mislabeled textual corpora and visual data, imbalanced distribution in datasets (e.g., underrepresenta-
44

---

Published in Transactions on Machine Learning Research (10/2025)
tion of certain demographics), and outdated information can lead to inaccuracies in classification and description tasks [188, 959]. For example, hallucinations in MLLMs are often observed in which the model predicts possible objects or actions that are not supported by the image but are plausible from commonsense [752].
Knowledge Boundaries. LLMs and MLLMs face limitations in domain-specifie knowledge (e.g-: medical analysis in 998 and embodied decision-making in 509) or struggle to recognize and interpret up-to-date content. For instance, MLLMs may fail to accurately recognize images of new electronic devices, enterprise logos, cultural symbols that are absent from their training data. Hallucinations from Training
Architecture Choices. The architectural design of MLLMs, often based on complex neural networks that integrate features of both vision and language models, might contribute to hallucinations. For instance, inpppropriaee conditioning of the visual and textual components leads to model misinterpre- tations [144].
Inherent Mechanism. There is an inherent statistical lower-bound on the hallucination rate of language models given the sufficient presence of facts for training transformers [422]. In other words, a certain level of hallucination is necessary for the model to minimize cross-entropy across large and varied pre-training data.
Exposure Bias. This issue may arise when the model is overtrained on certain types of images texts, leading to an overrepresentation of these elements in the model's outputs, regardless of their relevance or accuracy in now contexts [44]. This is often the result of catastrophic forgetting [1098].
Hallucinations from Inference
Sampling Randomness in Decoding. LLMs and MLLLSs often use stochastic methods to generate outputs to textual and visual content. This randomness can lead to inaccuracies, particularly when dcaling with complex tasks, ambiguous instructions, or intricate visual scenes.
Imperfect Decoding Representation. Challenges to adequately represent visual and textual infor- mation in the model can result in inaccuracies when generating descriptions or interpretations of visual data [872, 138].
Path Dependence. Autorcgressive LLMs and MILMs suffer from error propagation and self- consistency failures due to their loken-by-token prediction nature. Any intermediate errors from previ- ously generated tokens will have compounding effect on the subsequently predicted token. Empirical studies have demonstrated that, assuming a constant per-token error rate, the generation quality can deteriorate rapidly with respect to the sequence length [42]. Additionally, inconsistencies with earlier generated content are common when hallucinations occur [953].
7.3 Hallucination Detection and Measurement
As shown in Figure 19, the process of detecting hallucinations in foundation models generally involves three key steps: first, breaking the response into distinct parts; second, extracting the facts from cach part; and third, assigning a score to each fact. Next, we will provide detailed discussion of hallucination detection and measurement methods across various types of foundation models.
7.3.1 Detecting Language Hallucinations
Identifying hallucinations in LLMs crucial to maintaining the reliability of their outputs. Traditional metrics, mainly based on word overlap matching, are inadequate to distinguish subtle differences between plausible content and hallucinations, This emphasizes the need for advanced detection techniques specifically designed for LLM hallucinations. Here, we discuss three distinct pipelines, differentiated by their use of external knowledge and evaluation metrics.
To accurately pinpoint factual inaccuracies in LLM outputs, a straightforwwar approach is to compare the generated content with reliable knowledge sources. This idea similar to the traditional fact-checking process [51, 49). Building on this concept, a line of works has employed LLMs to verify generated claims using evidence retrieved from external sources [148, 313, 425, 641, 176). This
Retrieval-based Detection.
45

---

Published in Transactions on Machine Learning Research (10/2025)
Hallucination Detection in LLMs
Tell me about Apple Inc. covering histury, locution, and products?
Apple clothing company based in New York. It famous producing luxury shoes. The company was founded in 1976.
Apple Inc. clothing company.
based in York.
Score: 2.5%
famous for producing luxury shoes.
The company was founded in 1976.
Hallucination Detection in T21 Models
Draw me picture: white dog stand in grassy field filled with yellow flowers, looking
white dog.
standing grassy field.
Score:
filled with yellow flowers. looks content and alort.
content and alert.
Figure 19: Illustration of hallucination detection in foundation models. The process generally involves three steps: (1) decomposing the model output into individual factual units, (2) verifying each fact against external knowledge or ground truth, and (3) aggregating the correctness scores to produce an overall hallucination score. The examples show applications in both LLMs and T21 models.
method generally encompasses three essential phases: extracting the claims, retrieving relevant evidence from external knowledge, and classifying the veracity of the claim. The detailed implementation of each stage varies different frameworks. In the context of long-form generation, tools such as FActScore [641], Core [415], and FacTool [176] prompt language models such as InstructGPT [686] or ChatGPT to extract all claims in a target text. Similarly, EVER [425] is designed to extract all fact-related concepts from sentence and subsequently generate yes/no validation question for each concept in the generated text. For evidence retrieval, FActScore [641] picks the most relevant picces of facts from the external knowledge base, where the relevance is ranked by retriever embedding distances. In addition, Chen et al. [148] use a language model to summarize the retrieved texts, specifically serving as supporting evidence for the claims. EVER [425] and FacTool [176] utilize a search engine to acquire evidence from queries generated by a language model. These methods can be further classified into two types based on their ability to detect factual fabrication, which involves generating information that cannot be verified. The first category employs language model to directly label claims as True or False [641, 176], while the second category additionally considers the (NEI).
possibilily of Dol-Enough-Innormaalle
Model Internal State. A series of studies have explored the use of LLM internal states for hallucination detection. Varshney et al. [894] assess hallucinations by examining the lowest token probability in key concepts, suggesting that a lower probability indicates higher likelihood hallucination. Luo et al. [597] implement self-assessment method, where the proficiency of an LLM is judged by its ability to reconstruct concept from sclf-explanation. This approach uses the perplexity of the generated response to gauge the level of concept understanding. Yao et al. [1051] explore hallucinations a form of adversarial attack by using gradient-based token replacement, finding that the initial tokens generated from standard prompts have a lower entropy compared to adversarial ones, leading them to decide an entropy threshold to detect ballucinations.
mcasure
Uncertainty Estimation. Recent advancements in LLM research have seen a focus on uncertainty esti- mation. We will briefly discuss this in the context of hallucination evaluation and provide a more general explanation in Section 8. Built upon techniques such as deep ensembles and conditional entropy, entropy- based methods [1006, 892, 265] establish connection between hallucination likelihood and predictive un- certainty. Guerreiro et al. [324] explore variance estimation through Monte Carlo Dropout [278] and inves- tigale a togg-robbaiiity-baan
approach by measuring model confidence through length-normnalized sequence log- probbabbiil Additionally, some methods employ LLMs themselves to estimate uncertainty and detect hallucinations. SelfCheck [635] detect errors in complex reasoning within LLMs. The burgconing field of based prompting- based metrics has also gained attention, with Chiang & Lee [177] leveraging the instruction- following capability of L.MSs to evaluate the faithfulness of content. Beyond these methods, researchers
46

---

Published in Transactions on Machine Learning Research (10/2025)
have also gleaned insights from LLM behaviors. Natural language prompts [1015] have played pivotal role in this direction. Manakul et al. [613] tackle this issue by examining the consistency of factual statements across multiple LLM responses. Another innovative approach [10] uses indirect queries to subtly elicit spe- cific information. This approach mirrors investigative interviow techniques, offering a manccdd evaluation of consistency. Taking a step further, the multi-agent perspective, particularly exemplified by LMvLM [196], involves one LLM as the examiner to question another, the examinec. This method, inspired by legal cross- examination techniques, aims to reveal inconsistencies in multi-turn interactions. Collectively, these diverse approaches contribute to deeper understanding of uncertainty in language models and provide innovative ways to measure it.
7.3.2 Measuring Multimodal Hallucinations
Conventional statistical metrics, such as BLEU [689], CIDEr [896], and ROUGE [537], are commonly used to evaluate hallucinations. However, has been suggested that these metrics might not be highly appropriate for assessing detailed descriptions from MLLMs [1169]. To address this issue, recent metrics [752, 520, 924, 1021] shift their focus towards object hallucination, as objects represent fundamental components that compose the visual scene and contribute to correct visual understanding. In addition, Li et al. [520] propose Polling- Based Object Probing Evaluation (POPE), novel approach that utilizes polling- -ased queries to prompt MLLMs [321] with straightforward yes-or-no questions regarding the existence of specific objects in an image. By asking targeted questions, POPE provides a more stable and flexible evaluation of object hallucination.
While recent evaluations have introduced numerous metrics focused on the phenomenon of object halluci- nation, it is crucial to consider their suitability for capturing the intricacies of MLLMs, as many elements contribute to the visual semantics of an image. Apart from evaluation metrics that focus only on object hallucinations, Wang et al. [924] introduce HaELM, framework of hallucination evaluation by fine-tuned LLaMA [14]. The work of Gunjal et al. [327] presents novel dataset, M-HalDetect, designed specifically to identify ballucinations in visual question answering (VQA) tasks, Wang et al. [937] introduce an innovative dataset named Mementos, aimed at assessing the reasoning abilitics of MLLMs in understanding sequences of images. Additionally, they propose a unique type of hallucination, termed behavior hallucination, that arises specifically in the context of image sequence comprehension. Additionally, Wang et al. [923] propose an LLM-free method to evaluate object existence, object attribute, and object relation hallucinations cost- effectively and efficiently. Although Retrieval Angmented Generation (RAG) [494] is common technique to alleviate the hallucination problem, there are sufficient guarantees that the problem is fully eliminated. To address this scenario, Wu et al. [990] curate benchmark of hallucination detection methods in the RAG-based system.
7.3.3 Measuring Text-to-imas Hallucinations
Recent studies [983, 382] have evaluated the compositional ability of text-to-image models, revealing their tendency to hallucinate. These benchmarks prompt text-to-image models with text descriptions that contain multiple concepts. Next, they analyze how many of these concepts actually appear in the generated image by creating one question per visual concept and using a strong MLLM to answer all questions. This process identifies hallucinations that occur when the model misses some concepts or introduces concepts that were not present in the original description.
7.4
Hallucination Mitigation
7.4.1
Reducing Hallucinations in LLMs
There is pressing need for the development of novel and reliable methods to mitigate hallucinations in LLMs, particularly to meet the demands in real-world applications and substantially improve their generalization. Our review categorizes them into two primary types: data-centric and model-based approaches.
Data- Chea-CCntric Methods. The training datasct plays a crucial role in the occurrence of hallucinations in LLMs. McKenna al. [624] indicate that the primary cause of hallucinations is the memorization of training data. This issue manifests when user queries closely align with pre-trained data, often resulting
47

---

Published in Transactions on Machine Learning Research (10/2025)
in LLMs generating inaccurate or misleading responses. Additionally, some datasets exhibit inconsistencies in the factual alignment between the input text and the reference target [410]. This problem is evident. in dataset constructions [480, 223], and downstream tasks such as open domain dialogues [631, 384]. Besides this, many pre-trained LLMs have a fixed parametric knowledge base on a specific timestamp, and they are oflen deployed offiie (e.g,, limited access to the internet or data storage firewall for privacy). This limitation can potentially lead to hallucinations when the model encounters questions that its knowledge base does not cover [52]. Therefore, the data-centric approaches that can benefit. the language model's (parametric and external) knowledge bases are practically reasonable. To further reduce hallucinations in natural language generation (NI.G) tasks, numerous attempts have been made to automatically [248, 569] and manually [290, 921, 690] cleanse and refine the training sets for improving accuracy and reliability, In recent years, the importance of meticulous data filtering has become evident in effectively mitigating hallucination issues in LLMs. For instance, GPT-3's pre-training data [113] is cleaned using several high- quality reference corpora to ensure its quality. Touvron et al. [876] improve the quality of the pro-training corpus by incorporating data from reliable external sources such as Wikipedia. This process involves carefully selecting and up-sampling data from these sources to enrich the content and enhance the faithfulness of training data.
Model-based Methods. The choice of training strategies and model architectures also plays a crucial role in mitigating hallucinations. Past works [864, 40] have showcased a correlation between deficient comprehen- sion capability in the encoder and the frequency of hallucinations. Recently, McKenna et al. [624] conducted a study on factors influencing hallucination and revealed that the main determinant is the extent to which training data are memorized. Specifically, models tend to prioritize generating outputs based on their para- metric (internal) knowledge, rather than relying on the external information from the input prompt. This memorization, known parametric knowledge bias [610, 410], significanly contribules to the occurrence of hallucinations. Numerous techniques [485, 621, 250, 805] have been developed to address this bias in earlier, smaller-scale language models. Some retrieval-angmentee methods improve language models by incorporat- ing external information from large corpora. Borgeaud et al. [103] introduce RETRO Kketrieval-Enhancee Transformer), an auto-regressiv language model that leverages document chunks from a massive trillion token database. Peng et al. [699] further advance a black-box LLM by incorporating external knowledge, therefore improving the model's response quality through itcrative prompt revision. To address the chal- lenges of factually inaccurate text generation and source allribution in language models, Ram et al. [734] propose a novel approach called In-Context RALM (Retrieval-Augee Augmented Language Modeling). This ap- proach maintains the original model architecture while prepending grounding documents from a corpus to the input. However, the development of model- mosed- strategies for reducing hallucinations in LLMs is still at an early stage in the academic community. Recent research [657] develops an iterative algorithm to prompt LLMs to identify and eliminate self-contradictions in their generated text, thereby enhancing both fluency and informativeness. Chen et al. [137] use LLMs to corrupt text and fine-tune compact editors to denoise these faux hallucinations. Models fine-tuned with carefully crafted tasks and datasets can also mitigate hal- Iucinations. Elaraby et al. [256] introduce lightweight and knowledgo-free framcwork for this fine-tuning process, with the aim of reducing hallucinations in open-source language models. Current satle-of-the-art LLMs, i.e,, ChatGPT [675], GPT-4 [676], and Llama 3 [14], have made significant advances in reducing hallucinations during the RLHF stage. It is also worth noting that artificially created hallucination data are leveraged to train the reward model for GPT-4 [676].
7.4.2 Reducing Hallucinations in MLLMs
Prior studies have attempled to address the problem of hallucinations in generating responses from images using small-scale multimodal models [1124, 93, 203, 439). However, these approaches primarily focus on short image captioning tasks, making them inadequate for applications in MLLMS that aim to provide com- prehensive and detailed descriptions. In recent advances, methods for addressing hallucinations in MLLMs can be mainly divided into three categories: training approaches, post-processing techniques, and decoding methods.
Training Approaches. Visual instruction-tuming [556] has significantly improved the zero-shot capabilities of MLLLMs new tasks. Built. upon this, Liu et al. [553] introduce the Large-scale Robust Visual (LRV)-
48

---

Published in Transactions on Machine Learning Research (10/2025)
Instruction dataset and the corresponding LRV-Instruction framework. The framework incorporates both positive and negative instructions to enhance the robustness of visual instruction- and reduce hallu- cinations. Furthermore, recent studies introduce several new cnprrucctoontttnnng datasets aimed at reducing hallucinations. For example, Li et al. [508] present the Multi-Modal Multilingual Instruction Tuning (M9IT) dataset. The M datasel consists of 40 meticulously curated datasels, comprising 2.4 million instances and 400 manually crafted task instructions that have been reformulated into a vision-to-text format. Gunjal et al. [327] introduce the M-HalDeetect dataset, designed to detect and prevent objects and logical hallucinations in detailed image descriptions. The authors train fine- finegrained multimodal reward model using Instruct- BLIP and evaluate its effectiveness using best-of-n rejection sampling, which is a technique used in LLMs to improve output quality by generating multiple candidate outputs and selecting the best one according to a predefined scoring function. Since common practice in instruction-tuningg dataset construction implicilly teaches an LLM to answer all questions, even for problems it cannot answer, this will lead to the tendency of LLM to hallucinate. To address this problem, Zhang et al. [1105] proposod a different approach, which enables LLMs to refuse to answer questions beyond their capabilities. LLMs with this ability were shown to effectively reduce hallucination by refusing to respond to particularly challenging questions.
Post-proccssing Approaches. These approaches in MLLMs refer to the techniques used to refine or alter MLLM outputs after the inference process. These techniques aim to improve the quality, accuracy, and usability of models' predictions. In the context of small-scale multimodal models, Ngo et al. [670] introduce pseudo labeling and efficient post-processing techniques to improve vehicle retrieval accuracy. Hoxha et al. [369] introduce two post-processing strategies based on hidden Markov models and the Viterbi algorithm, which improve the faithfulness of image captioning by leveraging probabilistic modeling of language structure to correct and optimize generated captions. These strategics effectively correct hallucinations in generated descriptions while improving the overall coherence of the captions. Duan et al. [245] utilize auxiliary model to first learn the truthful direction of MLLMs decoding and then apply truthful-ge outhful-guided inference-ting intervention during decoding to alleviate hallucinations. In the broader context of MLLMs, post-processing approaches can be mainly categorized into two types:
LLM-assisted Approaches. These methods involve the use of foundation models, such as GPT, to correct and refine the outputs of smaller multimodal models. Maaz et al. [606] implement GPT-assisted mechanism that refines and optimizes enriched annotations, thereby generating high-quality instruction data. Similar to small models, Yin et al. [1063] focus on improving the reliability of visual hallucination diagnosis, in which GPT is leveraged to rectify inaccurate diagnosis. In addition, Zhou et al. [1169] utilize GPT to correct hallucinatory captions for MLLMs.
LLM-free Approaches. These methods do not rely on LLMs. Instead, they may engage in self- correction or use a smaller model as a revisor. Zhou et al. [1169] focus on three key factors contributing to object hallucination: co-occurrence, uncertainty, and object position. Based on these insights, they propose LMM Hallucination Revisor (LURE) to address the problem of object hallucination in MLLMs. Drawing inspiration from LM-Switch, Zhai et al. [1096] introduce control parameter, referred to as a "switching value', to manage hallucinations in language generation by modifying word embeddings.
Decoding Approaches. In recent years, various methods have employed decoding approaches to address the issue of hallucinations in MLLMs during inference. Chuang et al. [189] reduce incorrect fact generation in LLMs through the contrast of logit differences between the later and carlier layers, and this approach can be extended to ML.LMs. Moreover, recent efforts have been dedicated to mitigating hallucinations in MLLMs well. Leng et al. [491] propose a training- free decoding approach called Visual Contrastive Decoding, which contrasts the output distributions with the original and distorted visual inputs. This approach effectively calibrates the model's over-reliance on unimodal priors and statistical bias without utilizing external models. Similarly, Chen et al. [168] contrast the logit distribution of different partial visual inputs to approximate the optimal visual context during decoding. Besides, Huang et al. [385] present a decoding approach based on an over-trust penalty and a retrospection-allocation strategy to alleviate the problem of partial over- trust. Building these decoding approaches, Li et al. (526) introduce VISTA, training-free inference-time intervention that mitigates hallucinations in MLLMs by reinforcing visual information in activation space and leveraging early-layer activations, achieving consistent improvements across models and decoding strategies.
49

---

Published in Transactions on Machine Learning Research (10/2025)
7.5 Current Limitations and Future Directions
While recent research advances have partially defined the concept, developed measurements, revealed causes, and proposed solutions for hallucinations in foundation models, it remains a challenge to formulate the phenomenon comprehensively. In the sections above, our survey summarizos the cxisting work and presents them in our paradigm. Next, we will discuss currett challenges and shed light on potential future directions that the community should focus on.
When validating whether a language model response is hallucinatory, a popular line of research to retrieve external texts (e.g., scientific facts and historical archival) supportive evidence. This approach requires highly trusted knowledge database that is often crowd-sourced and is presumed to be factual. However, there are no rigorous guarantees regarding the trustworthiness of these external knowledge bases, and their vast size makes manual scrutiny impractical. Furthermore, after evidence retrieval, the current pipeline typically employs a verifier to compare the retrieved fact with the target model's statement for hallucination classification. If the language model is prompted exccute this validation, it important to note that the verifier may also exhibit hallucinatory behavior. Additionally, the statistical correlation between the mechanisms of the autoregressive hidden states and the hallucination phenomenon is still unclear. Lastly, while most existing rescarch focuses on validating the language generation from LLMs and MLLMs, rescarch is insufficient in the synthesis of other modalities (e.g., images, videos, and audio).
To address the limitations outlined in current research on hallucinations in foundation models, several avenues for future work emerge. Firstly, enhancing the reliability and veracity of the knowledge bases used for evidence retrieval is essential. Future studies shall focus on the development of filtering and verification systems for external knowledge. Secondly, it is crucial to mitigate the risk of hallucinations in validation models. Potential frameworks include multi-agent collaborations where multiple independent models interact and verisy the target output. Furthermore, deeper understanding of the underlying mechanisus of language models, particularly the statistical properties and dynamics of autoregressive hidden states, will help mitigate hallucinations. Techniques like mechanistic model editing and representational latent editing hold the potential to promote more faithful language generation. Incorporating refusal mechanisms is a rising topic study knowledge boundaries and mitigate model hallucinations [1018, 1180). Future work shall consider directions for improving refusal calibration, integrating refusal with multi-agent and retrieval- based pipclines, and extending refusal capabilities to multimodal generation tasks beyond text. Lastly, since image diffusion models [753] and visual auloregressive models [1073, 512] are increasingly conditioned on textual prompts, the community should expand the scope of hallucination research to the generation of images, vidcos, and audio.
50

---

Published in Transactions on Machine Learning Research (10/2025)
8 Uncertainty
Though modern foundation models possess impressive capabilities across a wide range of domains and tasks, harnessing their power reliably requires a clear understanding of the uncertainties inherent in their outputs.
In the context of foundation models, uncertainty refers to the degree of confidence in the model's predictions or generated content. This uncertainty can arise from multiple sources: epistemic uncertainty, which due to limited or imperfect knowledge encoded during training (e.g., gaps in the data or model misspecifiationn) and aleatoric uncertainty, which stems from inherent randomness or ambiguity in the data itself (e.g., inherently noisy or ambiguous task dcfinitions). However, it is important to note that this distinction also has limitations [54, 318, 887, 654].
Beyond these, real-world deployment introduces further uncertainty due to distribution shifts, novel scenarios not represented during training, or unforescen user inputs. Rocognizing and managing these different types of uncertainly is essential for responsible and safe use of foundation models in high-stakes applications.
Tistc: Raon st al. [51]. Ott et al. [ess], Orlondd et al [ass] Tablon et al. [173]
Input umbigutty: Kuhn et al. [Jis], Kim et al [E08], Las et al. [648]] Temlan et al [s49]
Fundementall Sonrons
Model architecture: Lainhminmngnns
al. (175], Meltin Gales 101d Gol & Ghalinmen J2S [563]
Tralalnng objectivs: Wil al. 200), Ellom Aaie (20)] Tim
Origine & Nature
Irreducibll naclumasess Hilloranier Wargamn [397] Xlany [1315)
Cheeraiiity Foundactio Modele
Alesturk vs, Epistmie
Tiserepenry mosoriee Molinin & Gales (012), Gluehlore et al. (500), Lahiou et al [178]
Kumwadgee gape: Osband et ol. (68J], Wing et al. [154] Yo et al. [oto]]
Entrogy moacures: Kuhn es al. 1164 Lin et al. [545], Malmin & Goles [6122), Dunn al. [240]
Salf colf-eemistteeer: Kadavath
(121), Lin [599], Si al. (106). Wang [14a]
Probobolatte Methods
Semantic grouptng: Kuhn et [104], Lis al. [644]
Emabakiing sparre Clus el al. [118]
Quastitioatio Apprcuduss
ECF/MCR metricn: Namini et al. [063]] Gon et al. (30)] Hunng et al. 1800
Callbention & Expression
Tengarmttuee staling Goo et al. [529), Dead Durrets (215), Xiso et al. [1008]
Verbolited confdmmor: Kodswath al. (421), Lin et al [8000, Tlan al. [sta]]
Lingratic markers Thou [1157] Mielin
(837), Stengel-Fskin
[85]]
Selectio alscuction Gollann & El-Yiniv [206), Kamnth at al. [122] Si et nd. [800]
Deforal to lungt modids: Gupta st al. [[88]
Operations Stoalogies
Chaillcation roquets: Kuhn et al. [168], Kim et al. [438], Strage Eskdi & Van Dunne [sai]
Amiiguiit diturtiu:: Lin al. (6.6). Tankin al. [815] Yiu [106]]
Mitigatton& Control
Conformel predection: Shader& Vork [tsi], Kumse et [o00], Anovleponko
[50]
Risk coustod. Angeloponis et [S1], Steil at al. [815], Dong 211]
Statistroa Guaranteee
Generation bounds Quach et al. 1222. Destschman ot al. (217), Yodkori et al, [1029]
Pronpt risk control: Zollo al. [1186)
Figure 20: Taxonomy of Uncertainty in Foundation Models.
8.1 Sources of Uncertainty
Uncertainty foundation models can be categorized into alcatoric and epistemic types. Alcatoric uncertainty is primarily influenced by data factors, while epistemic uncertainty is largely affected by modeling decisions. We will discuss the impact of these components in both the training and inference stages, with comparison presented in Figure 21.
8.1.1 Data
To begin, we will consider how the nature of language itself introduces uncertainty into the generation process of LLMs [54, 685), which both consume natural language as input data and produce it as output data. Given some input context to a language model, there are usually many possible responses for several reasons. First, the imput may be reasonably interpreted to have multiple different meanings. This could be because the context is vague ("She watched the man with binoculars"), very complex (as some reading comprehension questions are, even for humans), or contains spelling or other errors. Besides ambiguity in the input, certain queries may be inherently more open-ended and allow for many reasonable responses. This might include a request to complete a fictional story, tell a joke, or give a position on some political or social issue. Finally, many ways. For
given a fixed input interpretation, oquivalent answers to a query may be expressible example, given the input context "What is the capital of Rwanda?", "Kigali is the capital of Rwanda" and "The capital of Rwanda is Kigali" offer semantically equivalent answers with different. surface forms.
51

---

Published in Transactions on Machine Learning Research (10/2025)
Aleatoric
Epistemic
Data Uncertainty
Model Uncertainty
G
Confidence in input data
Confidence in prediction
?
Randomness in data creation
Lack of knowledge
Figure 21: The comparison of alcatoric and epistemic uncertainty in machine learning. Alcatoric uncertainty originates from data variability, while epistemic uncertainty results from model limitations. The former is tied inherent noise in observations, while the latter tied insufficient knowledge in representation.
The uncertainty in an LLM generation due natural language data stems both from the training data and the prompt inputted to the model during inference. When the training dataset is small not sufficiently general, the model may not have the relevant knowledge to effectively process some context [684, 397, 473, 697]. If the training data contains a large amount of ambiguous language, the trained LLM may reflect this uncertainty in its outputs. Other sources of uncertainty introduced by training data include diverse, conflicting, and outdated information.
In deployment, further uncertainly rntrodueed by specific text data given as input to the LLM. Queries could be ambiguous [463, 438, 548], and tasks or instructions could be open-ended or underspecified [848], making it difficult for the model to express appropriate level of confidence in its response. Also, relevant information could be excluded from the context [1081], or users may produce input errors.
8.1.2 Model
In addition to the uncertainty introduced by data training and inference, the model itself also contributes the uncertainty in the generation process, given an input prompt. Architecture choices may not reflect the underlying data-generating process. Different modeling techniques like ensembling [475, 612, 302, 936] Bayesian inference [278, 685, 1006] be applied with the hope of accurately characterizing the true posterior probability. However, these methods can be computationally expensive and potentially ineffective [4, 687]. Besides architecture, the typical optimization objective of producing the most plausible answer (i.e., maximizing observed sequence probability 253, 252) may not align to produce the most correct and factual answer [861], and general, the cross-entropy objective has been shown lead to overconfidence [960]. Finally, though massive pre-trained models are an effective tool for combating the uncertainty introduced by the inpul context, the popular approach of fine-tuning these LLMs for custom use cases may dilute these generalist capabilities [1083].
8.1.3 Aleatoric vs. Epistemic Uncertainty
Besides identifying how uncertainty may arisc due data and model factors, may also be useful characterize uncertainty in LLM responses as either aleatoric or epistemic [397, 1115]. Aleatoric uncertainty, sometimes referred data uncertainty, exists due the inherent. randomness in the data-generating
52

---

Published in Transactions on Machine Learning Research (10/2025)
process. Additional information cannot be used to reduce alcatoric uncertainty. For example, suppose language model was asked to predict the probability of heads the flip of a fair coin. In that case, no additional context or training data would enable a better prediction than 50%. On the other hand, epistemic uncertainty ariscs precisely because of a lack of knowledge. Epistemic uncertainty may be reduced by incorporating additional data, for instance, by including or prioritizing more informative examples in the training set [684, 934] or incorporating appropriate few-shot examples in the context [1059, 225, 516, 835, 1081]. Section 8.2.1 discusses more of this work in detail.
8.2 Quantifying and Addressing Uncertainty
While uncertainty in LLM responses is unavoidable, considerable progress has been made in quantifying and addressing this uncertainty so that models can be deployed responsibly and reliably. Common measures for quantifying uncertainty use notions such as entropy to characterize the uncertainty in response and pro- duce higher scores for outputs that are less likely to be correct. Methods have been developed to recalibrate confidence scores so that they better reflect the true probability of an answer being correct. Additionally, uncertainty estimates can be used to identify cases where the system should abstain from answering or soek further clarification before providing response. Researchers have also examined whether LLMs can gen- erate linguistic expressions indicating their uncertainty for given output [685, 806, 464]. Finally, rigorous statistical methods, which are quickly gaining popularity in the deep learning community, have been applied to provide high probability bounds on LLM performance and risk [890, 836, 741]. Next, we will highlight important work in these areas, accompanied by an illustration in Figure 22.
Question: How many prime numbers are in the list of 1, 2, .... 50?
Prompt
Certain about the response?
Response A: There are 15 prime number in this list.
Response B: There are 20 20 prime number in this list.
LLM
YES
NO
Response
ECE [Guo et al. 2017]
LMKnow [Kadavath, al. 2022]
MCE [Guo et al., 2017]
CalibratedMath [Lin et al., 2022]
SLiC [Zhao al., 2022]
Language [Zhou et al., 2023]
Calibration
Word-based
Uncertainty
N
SelfAwarc [Yin et al. 2022]
Conformal [Kumar et al. 2023]
Estimation
Clam [Cuhn et al. 2023]
Dispersion [Deng et al., 2024]
AmbiEnt [Liu et al., 2023]
Factuality [Mohri al., 2024]
Selection
Distribution-free
Overview of representative methods for estimating and mitigating uncertainty in foundation models. The illustration highlights how different answers may be generated with varying confidence levels, and categorizes existing approaches into calibration, word-based, selection, and distribution-fre methods.
Figure 22:
8.2.1 Estimating Uncertainty
One critical ingredient for the reliable deployment of black-box LLM is the ability measure the uncer- tainty its responses so that appropriate decisions can be made based on its output [806]. Uncertainty in language model output. is often quantified in terms of predietive entropy [464, 544, 612, 244, 949). Although
53

---

Published in Transactions on Machine Learning Research (10/2025)
predictive entropy can be calculated directly using output class probabilities for classification tasks, mea- suring the uncertainty in language model generations is more challenging problem, requiring knowledge of the distribution over all possible sequences.
One popular way to address the challenge of sequence-level uncertainty quantification is to sample many potential generations from the model and use these samples to estimate the underlying distribution. For instance, Fomicheva et al. [272] use Monte Carlo dropout to draw samples, which are then utilized for quantifying uncertainty in machine translation. Malinin & Gales [612] instead employ an ensemble with Monte Carlo sampling techniques over the token outputs to produce both token-level and sequence-level uncertainty scores. Such approaches to mcasuring uncertainty via self-consistency are also studied in [421, 539, 806, 225, 464, 948]. While most self-consistency methods focus on the model's natural language outputs, Chen et al. [143] offer an effective method for measuring sample consistency in the embedding space. Besides, Stengel-Eskin & Van Durme [829] explore sequence-leee uncertainty in semantic parsing through the angle of calibration, where they leverage the minimum confidence across tokens.
In addition to the challenge of sampling from the distribution of possible sequences, there may be many equivalent surface forms of correct response to question such as "What is the capital of Germany?" Accordingly, the desirable notion of uncertainty may go beyond the spaces of sequences into the space of semantic meaning. These challenges are outlined and addressed by Kuhn et al. [464], wherein a set of sequences are sampled and grouped by semantic equivalence to measure uncertainty over meanings instead of uncertainty over output forms. Lin et al. [544] build on this work by proposing more sophisticated semannti uncertainty measures and removing the access requirement to the token-level scores of the potentially black- box model. Also, Duan et al. [244] presents an algorithm based on similar notion that aims to better characterize uncertainty by focusing on the most relevant token.
Finally, some have taken the approach of trying to explicitly identify epistemic uncertainty, which can then be addressed by incorporating additional information. Malinin & Gales [612] take an ensemble approach and characterizes epistemic or "knowledge" uncertainty using mutual information and the level of disagreement between models in the ensemble. Glushkova et. al. [302] also apply an ensemble-based approach to quantifying epistemic uncertainty in machine translation. The use of more powerful and efficient techniques such as direct uncertainty prediction and heteroscodastic regression are investigated in Zerva et al. [1095] in the context of machine translation; they find these methods perform favorably compared to variance-based baselines such as MC dropout and deep ensembles while being considerably faster. Lahlou et al. [473] highlight the challenges of using Bayesian techniques or discrepancy- a Direct Epistemic Uncertainty Prediction framework, wherein a secondary model is trained to estimate the point- generalization error and provides an upper bound on epistemic uncertainty. Their algorithm is shown to be useful in interactive learning environments, where the model can acquire novel examples and continue learning. Similarly, Osband et al. [684] use an epistemic neural network to identify uncertain data that should be prioritized in fine-tuning, achieving on-par performance while using half as much data training without prioritization. Hou et al. [367] avoid the need to train a separate model to predict the epistemic uncertainty, instead using multiple LLM queries for clarification to rule out data uncertainty so that the remaining uncertainty of each prediction can be prescribed to epistemic uncertainty. As another ICL-based approach, Yadkori et al. [1027] propose an iterative prompting method to identify when epistemic uncortainty is large and highlight its usefulness in a sctting with multiple good responses. More recently, Wang et al. [939] roduced Bayesian Low-Rank Adaptation (BLoB), which jointly updates both the mean and covariance of LLM parameters during fine-tuning to enhance uncertainty estimation, and Training- Free Bayesianization (TFB) [798], post-hoc method that converts pre-trained LoRA adapters into Bayesian models without retraining, offering efficient and accurate uncertainty quantification.
measures of epistemic uncertainty and proposes
8.2.2 Calibration
One popular method for characterizing a model's predictive uncertainty is concerning (confidence) calibra- tion. For a model to be well-calibrated, its confidence estimates should, on average, reflect the probability of its correct answers. The most common calibration measure the deep learning literature Expected Calibration Error (ECE) [663, 329). which measures the expected difference between confidence and accu- racy over the data distribution. However, since it is impossible to calculate this quantity directly, ECE is
54

---

Published in Transactions on Machine Learning Research (10/2025)
typically estimated: data points with similar confidence scores are binned together, and ECE is calculated as the mean absolute difference between average confidence and accuracy over all bins. Other popular mea- sures of calibration error include Maximum Calibration Error (MCE) [663, 329], Brier Score [III], negative log-likelihood [349] and other novel variants [887]. Since 0/1 accuracy is often not a suitable metric with respect to LLM performance, Huang et al. [390] propose Rank-Calibration Error, which captures whether higher uncertainty scores are associated with worse generations according to continuous metrics like ROUGE
[537] or BLEU [689].
While modern neural networks have achieved impressive accuracy across wide range of tasks and improved calibration relative to simpler methods [642], significant miscalibration remains, usually in the direction of overconfidence [329, 914]. To address this remaining calibration error, post-hoc recalibration met such as Platt scaling [706], temperature scaling [329], or histogram binning [1091] can be used to refine the confidence estimates of a pre-trained model.
Most of the work in calibration and deep learning has focused on the classification setting, where the softmax probability of class can be reasonably interpreted as confidence score. Accordingly, extending techniques for measuring and improving calibration to LLMs in classification (or other settings with single- token answers from a finite, discrete set) is straightforward. For example, Desai & Durrett [215] find that pre-trained encoder-only transformer models like BERT and RoBERTa are well-calibrated under fine- tuning and that techniques like temperature scaling and label smoothing can be effective in combating poor confidence estimates. Further, Xiao et al. [1008] perform a large-scale analysis of how decisions made along the LLM deployment pipeline, such as model size, architecture, and training objective, affect downstream task calibration on sentiment analysis and NLI. They find that larger models generally give more accurate confidence estimates and that applying temperature scaling and fine-tuning with focal loss may be helpful. Zhao et al. [1138], Zhou et al. [1154] offer methods to debias answers for a prompt so that confidence scores are calibrated based on the actual input instance under consideration, while Detommaso et al. [216] introduce the notion of multi-calibration [1190] into the LLM sctting by grouping examples based on binary attribute labels produced by the model itself. Kadavath et al. [421] perform an exlensive study of whether LLMs can evaluate the correctness of their own responses across tasks such as multi-choice question answering, They find that self-ovaluation improves with model size, although calibration is worse for more complex and out-of-disttibution tasks. They also find that popular alignment techniques such as RLHF may hurt the calibration of LLM output probabilities. To handle the case of population shift, e.g., across the distribution of subjects in a sample of MMLU questions, Li et al. [515] propose to train a recalibration method that adapts to a new subset of the data given only a few unlabeled examples.
On the other hand, estimating confidence and measuring calibration is less straightforward when tasks are generative or open-ended (for the same sequence-related reasons outlined in Section 8.2.1). Thus much recent LLM calibration rescarch has focused here [421, 1008, 808, 806, 862, 1141, 637, 573]. In early work highlight- ing this challenge, Ott et al. [685] analyze model calibration in the setting of neural machine translation, showing that these models tend to diffuse too much probability mass over the space of possible sequences. One popular avenue for addressing the difficulties of combining calibration and generation is the develop- ment of new methods for producing calibrated sequence-level confidence scores. To this end, Chen & Mueller [149; 150] combine self-consistency with self-evaluation to produce a confidence score using a method they call BSDetector and find it is more accurate than alternatives in identifying incorrect LLM responses for models like GPT-3 and ChaiGPT. Si et al. [806] measure the calibration of GPT-3 on free-form QA using both the length-normallied language model output probability and self-consistency and finds both meth- ods give more calibrated confidence scores than a supervised BERT basclinc. Tian et al. [862] study LLM calibration of models aligned with RLHF, finding that these models can verbalize confidence scores that are more reliable than the underlying output probabilities, an approach which is especially useful when the model is behind an API and these probabilities are not available (see Section 8.2.3 for more on verbalized expressions of uncertainty). While most work on the calibration of LLM generations has focused on language tasks like question answering and summarization, Spiess et al. [825] study the calibration of L.MSs for code generation across several tasks, correctness critcria, datasets, and approaches.
In addition to the approaches described above, researchers have also pursued techniques for better quantifying LLM confidence via model training, concerning an external recalibrator or the LLM itself. For instance,
55

---

Published in Transactions on Machine Learning Research (10/2025)
Mielke et al. [637] address conversational agents' overconfidence by training a small auxiliary network to predict the appropriate level of confidence to be expressed. Liu et al. [573] offer further work in this direction, proposing to train a new linear layer that predicts a bias term to be added to the language model's output logits. Their approach enables the reordering of candidate generations (as opposed to temperature scaling) and is tested on longer generations including full paragraphs. Kadavath el al. [421] study whether a language model can be trained to prodict the probability that a free-form answer to question is correct; their experiments show promising results, although generalizing such behavior across distributions remains challenging. Lin et al. [539] use fine-tuning to teach GPT-3 model to express its own uncertainty various mathematics tasks, finding that responses are generally well-calibrated and remain reasonable under distribution shift. Finally, a supervised fine-tuning step is proposed in Band et al. [60] to induce linguistic calibration, where model outputs feature confidence estimales that enable rewm decision-makers lo make calibrated probabilistic predictions.
8.2.3 Verbalized Uncertainty
Generally, in machine learning, confidence scores are numeric values extracted from a predictive model, for example, based on predicted class probabilities, logit entropy, or ensemble variance. However, the ability of LLMs to generale arbitrary text output enables a paradigm in which language models may express their uncertainty directly in their natural language output.
As an carly example of such an approach, Kadavath et al. [421] verbalize language model calibration by verifying answers using the probability assigned to lokens such as "True" or "IK" ("I know") conditioned on its output or articulating confidence scores using numeric verbalizations such as "30%" or "80%" Their approach shows promise, although it may be difficult to generalize to new tasks or tasks that are difficult to format as multiple-choice. Additionally, Lin et al. [539] fine-tune a GPT-3 to directly express its confidence in its output using verbalized probabilities (e.g., "61%"), while Tian et al. [862) prompt a model directly to output both confidence scores and linguistic markers of confidence (e.g., "highly likely"). Zhou et al. [1157] study how linguistic markers of certainty, uncertainty, or evidentiality such as "I'm sure..." "I think..",, or "Wikipedia says... " affect model confidence. Their findings imply that LLMs are sensitive to epistemic markers in prompts, with more than 80% variation in accuracy, and that expressions of high cortainty result in a decrease in accuracy. Their results also suggest that the confidence scores that LLM outputs do not truly reflect epistemic and aleatoric uncertainty in response but instead are based on mimicking language use from the training set. This observation is supported by an extensive study of the ability of black-box models like GPT-4 to verbalize confidence in Xiong et al. [1015]. They find the verbalized uncertainty expressions overconfident. and difficult to optimize across models and datasets with single strategy for prompting, sampling, and scoring, In other relevant work, Mielke et al. [637] train a confidence calibration network to select linguistic expressions of uncortainty that should be included in the output of conversational agent. Eesngel-Eskin et al. [833] propose LACIE, which splits verbalized uncertainty into explicit markers (e.g., I'm not sure) and implicit markers (e.g., giving details or backstory, stating a person's 's expertise, etc.). The models are trained to improve calibration by modeling a listener who accepts or rejects answers based on their correctness. The generator rewarded for providing correct answers that are accepted and penalized for incorrect answers being accepted correct answers being rejected.
8.2.4 Addressing Uncertain Examples
Selection another established tool for addressing uncertainty [295, 296, 271, 255, 1184]. We use the term selection broadly to encompass methods that identify inputs that are particularly difficult for the model. We offer interventions like allowing the model to abstain from the prediction (the classic paradigm in selection) or request further information. Selection has been well-studied in the context of language models [197, 423, 806]. and has been shown to improve outcomes concerning hallucination and safety [867]. Kamath et al. [423] investigate selective question answering under domain shif,, proposing a novel algorithm that incorporates ditto--distribution data to train a selection model that identifies examples on which the model is likely to err. Gupta et al. [338] derive new score based on token-level uncertainty features, to identify examples that should be deferred from smaller model to a larger model. Uncertainty scoring methods, for example, the semantic based measures proposed in Lin et al. [544], are often evaluated via selection to highlight.
56

---

Published in Transactions on Machine Learning Research (10/2025)
how such measures are useful for predicting the correctness of LLM responses. Stengel-Eskin & Van Durme [830] show that we can recover low-confidence examples in semantic parsing by rephrasing and asking for user confirmation, which is the number of questions the model abstains from while keeping model safety high. To support work on selection in LLMs, Yin et al. [1064] introduce the SelfAwore dataset of questions that should be recognized as unanswerable.
Given the opportunity for interactivity provided by the text interface, a significant amount of research has gone towards algorithms to enable the LLM to request further information before responding, particularly in the case of ambiguous questions. For instance, Kuhn et al. [463] use few-shot learning to detect ambiguous questions that require clarifying questions, while Kim et al. [438] propose a tree-based approach to disam- biguating questions and retrieving missing information. As the interest in identifying ambiguous questions in LLMs has grown, there has been an accompanying effort to release public datasets that can be used to evaluate the relevant abilities. Liu et al. [548] offer AmbiEnt, a dataset to test an LLM's ability to manage ambignity in resolving entailment relations, finding their task difficult even for powerful commercial mod- els like GPT-4. Additionally, Tamkin et al. [848] introduce AmbiBench, a benchmark of ambiguuus lasks where the ambiguity is introduced by the task description itself (as opposed to the specific instance of the task). Stengel- Eskin et al. [831] create a dataset for identifying and disambiguating instances of the visual question- question-answering answering task with MLLMs. Besides, Bennell-Eskin et al. [832] introduce a dataset of ambiguous queries and their logical forms and test whether models can recover both interpretations. Also, Saparina & Lapata [767] introduce a similar ambiguous parsing dataset but with human-sourced SQL queries.
8.2.5 Distribution-free Uncertainty Quantification
As LLMs are incrcasingly deployed in risk-sonsitive domains such as medicine, law, and finance, it may be important to have not only an estimate of the uncertainty in a model's response but also a high proba- bility upper bound the error rate at test time. Recently, there has been increasing research employing techniques from the Distribution-Fire Uncertainty Quantification (DFUQ) family to control the risk of deep learning systems. This line of work generally descends from the literature concerned with conformal pre- diction [781, 907], wherein a threshold on class probabilities is calibrated to produce prediction sets that fulfill some coverage (i.e., recall) guarantee. Angelopoulos & Bates [30] offer a tutorial on the subject in the context of modern neural network applications, and Kumar et al. [466] illustrate the application of conformal prediction multi-choice question answering with LLMs. To broaden its applicability, Angelopoulos et al. [32] derive a version of conformal prediction for bounding the expectation of any monotone loss function and studics their method in open-domain question answering. Recent work has offered algorithms for pro- ducing bounds on more general loss functions concerning the mean [31], quantile-basee risk measures like value at risk (VaR) [815], and measures of statistical dispersion like the Gini Coefficient or differences in loss among protected subgroups [213].
While it is straight forward to apply existing DFUQ techniques to classification with LLMs [815, 213, 466), the question of how best to apply them to generation tasks like summarization, chat, and code remains open. Multiple approaches have been proposed to apply these techniques to language model decoding. For example, Schuster et al. [777] utilize the Learn Then Test framework [31] to calibrate early exit criteria concerning
the number of transformer layers applied to an input. Their goal is to identify when an LLM is sufficiently
confident that it can exit the forward pass, and thus reduce the amount of computation used. In the conformal prediction vein, Quach et al. [722] calibrate a stopping rule to produce a set of candidate gencrations that with high probability contains a suitable response (while removing redundant candidates), and Deutschmann et al. [217] incorporate conformal prediction into a novel beam search algorithm. To mitigate the risk of models hallucinating answers, Yadkori et al. [1028] proposes a conformal abstention procedure using measures of self-consistency that are evaluated by the LLM itself. Finally, Mohri & Hashimoto [650] enforce factuality in LLMs by using conformal techniquos to determine a level of specificity with which a given question can be answered.
As a more general approach, Prompt Risk Control [1185] unites many techniques from the DFUQ family under a single framework for selecting a prompt (c.g., system prompt or set of few-shot examples) based on rigorous upper bounds on rich families of informative risk measures. The authors propose a two-step prompt selection process. First, a set of prompts is validated as producing an acceptable risk for some contextually
57

---

Published in Transactions on Machine Learning Research (10/2025)
relevant measure before final prompt is chosen based on some performance metric, like average reward accuracy. Prompt Risk Control can be applied to any bounded loss function, such as top-1 accuracy, ROUGE, or toxicity, and can be used to control risk measures, including tail quantities like value-at-risk or measures of statistical dispersion such as the Gini coefficient.
8.3 Current Limitations and Future Directions
Much work has gone into methods to identify and address uncertainty in foundation model generation. However, existing results and methods are limited and much work remains to be done before these models can be responsibly and reliably deployed.
First, many results in uncertainty quantification in LLMs are produced in limited settings. Experiments are usually performed on tasks like trivia question answering, which can be answered via a single token, word, or short phrase. Further, the tasks under study also often assune that there is only one right answer: there may be no uncertainly in the correct response to "Who won Super Bowl XX?". However, much LLM usage revolves around tasks that require generating long-form responses to open-ended queries, for which multiple reasonable answers cxist. Some works have made progress in this direction [1102, 1103, 1068]. it is unclear whether the results produced in these limited settings offer insight into more complex, uncertain, and sequential settings, such as chat or customer care.
Alongside the difficulty of extrapolating results from simple scttings, existing methods for improved uncer- tainty quantification of language model generations have come at the expense of generating multiple times for single query. Given that certain methods can increase costs by 2 to 20 times or more compared to standard inference, it will be infeasible for LLM users or service providers to adopt such approaches. Further- more, although modern frontier models have shown some ability to express their uncertainty in words, there is good evidence that any correlation between accuracy and verbalized expressions of confidence is simply a result of spurious fcatures in training data [1157]. In addition, it should be noted that these verbalization techniques also usually require extra inference costs, even for the simplest methods, such as scoring p(Truee) for the generated answer. Finally, although these algorithms have largely not been tested in open-ended tasks and over long generations, it seems probable that new tools will be needed in this setting. For exam- ple, consistency-based methods assume that producing diverse samples for a particular query indicates an example for which the model will likely give a poor answer. However, a model that can only produce a single answer to a query such as "Tell me a joke" or "Write me a story" would lack the capabilities to suit many modern LLM use cases.
Overall, it is unclear whether any advanced method for quantifying LLM uncertainty in the zero-shot setting robustly outperforms baseline sequence entropy score calculated using token probabilities. Note that these scores are often unavailable for black-box LLMs behind an API. Furthermore, it is difficult to imagine how best to exploit. probabilities taken directly from the language model, since these probabilities do not necessarily relate to the task at hand [623), but instead reflect the cross-entropy objective used in training and plausibility of an answer under the training data distribution (unless the model receives RLHF, which makes accurate uncertainty estimation even more difficult [421, 862)).
Besides addressing the limitations in methodology and experimental settings mentioned above, future work in this area may benefit from taking broader view of the challenge of quantifying and addressing uncertainty in generative models. It could explore how uncertainty can be better quantified and addressed across the entire model development and deployment pipeline, and how interventions and measurements at different points in the pipeline interact and affect downstream outcomes. Also, it may be useful to gain a more thorough understanding of how techniques for selecting, mixing, and filtering training data affect a user's ability to accurately estimate the model's confidence on downstream tasks, whether via token probabilities verbalizations. As new architectures and pro-training recipes emerge, they should be benchmarked for calibration, not only accuracy. Fine-tuning algorithms, whether supervised or RL, have been shown to worsen models' UQ characteristics, and this phenomenon must be kept in focus as the community iterates on these methods. Finally, given a model that has been pre-trained and fine-tuned and is ready for deployment, we might develop new mothods to select system prompts and few-shot exemplars that reduce and control uncertainty in the wild, ideally with rigorous statistical methods like those provided by DFUQ [1185].
58

---

Published in Transactions on Machine Learning Research (10/2025)
9 Distribution Shift
Foundation models can occasionally produce unacceptable errors when faced with distribution shifts. These models, typically trained on fixed corpus, require additional adaptation for new tasks. This limitation is particularly challenging in ever-changing world, where knowledge is constantly shifting due to various factors, such as changes in location time [430, 442]. For instance, if model trained before 2023 is asked, "Which team does Messi play for?", it may incorrectly assign higher probability to Paris Saint-Germain instead of Inter Miami. This example highlights the importance understanding, detecting, and mitigating distribution shifts in foundation models to improve their reliability.
Types of Distribution Shifts in Statistics
Training Data
Covariate Shift
Label Shift
Concept Shift
Examples in Image
Cat
Dog
Cat
Dog
Dog
Pet
Examples in Text
Harry interesting.
Harry
Hany interesting.
Harry inseresting
2. enjoy watching Rush Hour.
2. Faime rogarder Rush Hour.
2. enjoy watching Rush Hour. Godlather my lavorite,
2. I enjey watching Rush Hour.
The Godlather my
Positive Reviews
Positive Reviews
4. find Twilight dull.
Positive Reviews
Avenges vary bud.
1. find Twilight dull. Avengers is very bad.
Je trovve Thillight ennuyeux, Avengers trés mauvais.
6. Witching The Hangover
1. [ind Twilight dull.
feels like of time
Negative Reviews
Negative Reviews
Negative Reviews
Movie Reviews
Figure 23: Different types of distribution shifts in the perspectives of (1) statistics, (2) image, and (3) text. The concept shift scenarios show how two distinct classes can merge into a single class when labels change.
9.1 Definition and Categorization
The distribution shift [475, 45, 396] occurs when the independent and identically distributed (i.i.d.) as- sumption does not hold between the training and test distributions. This divergence between the training distribution Ptrain and the test distribution Ptest can significantly impact the performance of machine learning models, including foundation models. In essence, distribution shift describes the scenario where Ptrain * Prest: which can degrade model performance and reliability.
Based on how the data distribution changes, distribution shifts can be classified into three primary categories, with examples from various domains presented in Figure 23.
59

---

Published in Transactions on Machine Learning Research (10/2025)
Fortal defeition
Tabderiannncnenn (175)
(S0) Haphoss
Uwarnate Saik
Bax et el [213] Widor et al. (1TS) Kel (600)
Detrrttor and Otaggrbation
Aurra 1.] al [69]
Cuuny Skili
Arem sl. (o0, Hapkos et el. [00]
Wenderkess Lictsilion
Fang et ab. [1EiS et at, [pm]]
OrsudDetrlltio Detecion
LLM OOD dereetic
Lixes [550]
[I...]. Siltsbeii al. [164], ou (ing
Distibotion Skilk is Fosurlatinn Mosied
AILLAi-baed OUD devction
Dic et sl. 2o00 Brssa et st. (1o)), Cao et at. 3016, Xis et al [ras]
Angrestatios
Phanget al [100, DoWWow Thyler 200j,
[rod] Wel pool, [lt]],
[40)
Aroomarial Traking
Gooditto et at [907] Modey et al (o1)) Zhang al (ooo, Thuree et al: (omn) That ee al [14) Verro ot al: Jone Zie in al (117g)
OreoskDanniiiaa Goasiaatia
Lakei Suatllig
Sucaoly et al, [84]], Miike (655), You dl (1005]
broekont Lousing
Aajonday st (SS, Lis [100] State Jaa] Zlang al. [1108]
Madel Russald:
Dationad (206) Ielohnicanosons al (872), Arch HLl) Ting et el. (415) Wineer al [1O]
Deaz (20))
Yot,,
1ol, Zasag (1120),
at. [1ncf,
Trag
(handkooo it 151), Mis et parj, Asd et at, (rT) Geoeet al po0, Bong et al [1SE) Sttioofliaee es al. (01) Thai et al. (100g
freioollaggerat Grereation
Rom et al (aat. Sri jazt, Zhang it [118, Shoo et al PMM. Neddlaatten ot [eis], Seu al, (oon, Mr ei joog
Belzaniilaniee el al. [r],, Wilke ot st. 1007, Las [143]. Wang pas, Zionng of sl. [1122, Stawo ([ss
Fire with New Kovoriige
Youn al (1383] Curition al [10]. Reotige [TO]] Tidersoo et [ej, Tall (IT)) Tomee al (00] Stor Ju
Totslix Thasing
si sl. S46,, Zinzag el el. [1429)
Modal Blian
Wass (60), Yours [105]]. Boeis (200) Mlrink es al. (iI
Seaialiity
Youu et al [B00]] Xhang al (1125] Vemaa et at. Jeet
Camcaa Liuterkas
liltkin Tearning
Fing et ab, (102g)
1909, Kic ok (609,
Foi)
We al. best, Buez vl. (1225), Via et al. 2000) Ya al. [umg]]
Figure 24: Taxonomy of Distribution Shift in Foundation Models.
Covariate Shift. This term refers to changes in the feature distribution p(x) while the relationship between the features and the labels p(ylz) remains unchanged. This type of shift is prevalent in scenarios where the environment or context of the features change.
Label Shift. It occurs when the distribution of labels p(y) changes, while the conditional distribution of features given labels p(zly) remains constant. This shift can result from changes in the real-world phenomenn being modeled.
Concept Shift. Concept shift, also known as conditional shift or concept drift, happens when the relationship between the features and the labels p(y]z) changes. It reflects the evolution of the underlying problem statement or process over time.
9.2 Out-of-Distributin Detection
Out-of-distribation (OOD) detection involves identifying inputs different from the training distribution [1032, 273), which plays a vital role in enhancing the reliability of foundation models. By flagging unfamiliar data points for further scrutiny, these techniques help mitigate risks and maintain the integrity of the model's performance. Despite the impressive generalization capabilities of today's extremely large foundation models, they remain fundamentally bounded by their training data. When deployed in dynamic open-world environments, such models can still encounter domain shifts, rare edge cases, or adversarial input that leads to overconfident but incorrect predictions. Therefore, OOD detection remains crucial: not only as a safeguard against catastrophic failures in high-stakes applications (c.g., medicine, autonomous driving), but also as a tool to trigger human oversight, guide active learning and preserve trust automated decision-making systems.
In the context of language models, Liu et al. [550] present empirical investigation into the OOD detection capabilities of LLMs, specifically examining the LLaMA families with different model sizes. The study evalu- ates common OOD detectors in both zero-shot and fino-tuning scenarios, yiclding several significant insights: (i) LLMs inherently serve as effective OOD detectors without requiring fine-tuning. (ii) In-distribution (ID) fine-tuning can boost OOD detection. (iii) Generative fine-tuning demonstrates superior generalization abil- ity because it aligns with the pre-training objectives of LLMs. (iv) simple cosine distance OOD detector proves to be highly effective, attributed to the isotropic nature of LLM embedding spaces. Furthermore, Zhang et al. [1101] propose novel approach for OOD detection, utilixing the likelihood ratio between
60

---

Published in Transactions on Machine Learning Research (10/2025)
pre-trained LLM and its fine-tuned variant. This method leverages the pre-trained LLM's extensive prior knowledge about OOD data, which, when fine-tuned with ID) data, can effectively differentiate wetween ID and OOD samples. Expanding on these findings, Salimbeni et al. [764] explore the effectiveness of un- merged Low-Rank Adaptor (LoRA) [370] weights for OOD detection during the finc-tuning process, further contributing to the growing body of research in this area.
In addition to textual OOD detection, recent advancements have begun to harness the powerful representa- tion capabilities of foundation models in visual OOD detection. Dai et al. [205] propose a method to enhance OOD detection by selectively generating information from LLMs. Their method incorporates consistency- based uncertainty calibration to estimate generation confidence scores and extracts visual objects from images to leverage the world knowledge encoded in LLMs. ODPC [381] utilizes LLMs to generate specific prompts for creating "OOD peer classes," " which are synthetic categories constructed from in-distribution (ID) seman- ties but intentionally placed outside the original label space. These peer classes act as proxy OOD categories during training, enabling the model to Iearn tighter ID class boundarics and better distinguish unfamiliar samples. This approach serves as an auxiliary modality for detection and introduces a contrastive loss based OOD peer classes to learn compact ID class representations and clarify boundaries between different classes. EOE [119] improves OOD detection by tapping into the expert knowledge and reasoning capabili- ties of LLMs without requiring actual OOD data. This method is designed to adapt to various open-world scenarios, making it suitable for (i) far OOD detection, where the OOD samples come from entirely different domains (e.g,, animals vs. vehicles); (ii) near OOD detection, where the OOD samples are semantically close but from unseen categories (e.g. unseen dog breeds when trained on other breeds); and (iii) fine-grained OOD detection, where differences are subtle and intra-class variation is high (e-g., distinguishing bet visually similar medical conditions). In the medical domain, CARES [998] evaluate the OOD detection capability of medical LLMs, focusing their ability to detect medical images that dilfer significantly from those used in the training phase.
9.3 Out-of- oe-DDistribttin Generalization
OOD genoralization, on the other hand, aims to enhance the robustness of foundation models under new, unseen enviromments [353, 560, 1039, 1000, 665). This approach improves the model's resilience to variations in input data through diverse techniques. Prior to the era of foundation models, the deep learning com- munity explored a rich set of strategies for OOD generalization, supported by extensive empirical studies. These included (i) data augmentation, where transformations were applied to create synthetic training exam- ples [460, 803]; (ii) adversarial training, which exposed models to adversarially perturbed inputs [307, 611]; (iii) label smoothing, a regularization technique to prevent overconfidence [847, 656]; (iv) inveriant learning, which aimed to capture features stable across environments [43, 12]; and (v) model ensembles, which aggre- gate predictions from multiple models to reduce variance and improve robustness [475, 226). Yuan et al. [1083] evaluate these commonly used methods for LLMs, leading to important insights and conclusions.
9.3.1 Data Augmentation
Data augmentation [1108, 220, 1090] involves creating new training examples through various transformations of the original data. These transformations range from simple operations, such as flipping or rotating images in computer vision tasks, to more complex manipulations by generative models to simulate the data distribution [514, 877, 401]. In the NLP context, Easy Data Augmentation (EDA) [961] refers to a set of simple, low-cost textual augmentation operations synonym replacement, random insertion, random swap, and random deletion - designed to increase lexical diversity without altering overall meaning. EDA originally shown to be effective for small-scale text classification tasks, but its naive application to LLMs often degrades performance due to distributional shifts in token usage and disruption of learned long-range dependencies. The primary objective of data augmentation is to increase the diversity of the training set, thereby enabling the model to learn more robust features that generalize better to unseen data. However, recent rescarch has shown that applying simple augmentation techniques, such as EDA, to LLMs often leads to performance degradation across most tasks, underscoring the need for more advanced augmentation methods tailored to foundation models.
61

---

Published in Transactions on Machine Learning Research (10/2025)
9.3.2 Adversarial Training
Adversarial training [611, 56] is a robust technique used to improve OOD generalization by exposing models to adversarial examples during the training process. These adversarial examples are inputs deliberately perturbed to mislead the model into making incorrect prodictions, despite appearing similar to regular data. In earlier deep learning literature, adversarial training was shown to improve robustness in image recognition [307, 878]; and it is now increasingly applied to LLMs and MLLMs.
Mechanistically, adversarial training operates by solving min-max optimization problem [611, 1107]: the inner maximization finds the worst-case perturbation within a certain norm-ball around each input, while the outer minimization updates model parameters to correctly classify these perturbed inputs. By repeatedly training on such challenging examples, the model learns smoother and more stable decision boundaries, which are less sensitive to input shifts, thereby improving robustness and generalization unseen or distribution- shifted data.
In LLMs, Free Large- Batch (FreeLB) [1176], an adversarial training method that adds perturbations to the input data, improves generalization performance in most scenarios. Similarly, Verma el al. [899] introduce image perturbations in MLLMs through angmentations like noise addition, blurring, and median filtering, Additionally, they craft adversarial questions using conjunctions, disjunctions, and negations to challenge models' reasoning abilities. Among the tested augmentations, Gaussian Noise Addition is identified as the most detrimental, causing the largest decline in performance. The study also finds that the complexity of questions, especially those with multiple connectives, significantly impacts the models' performance.
9.3.3 Label Smoothing
Label smoothing [847] is regularization technique used to improve OOD generalization by preventing the model from becoming overly confident in its predictions [656]. Unlike traditional training algorithms where models learn lo assign probability of 1 to the correct class and 0 to all others, label smoothing introduces a small probability to incorrect classes. This approach encourages models to maintain degree of uncertainty in their predictions, potentially improving their ability to generalize to unseen data. In the context of LLMs, however, the effectiveness of label smoothing has been called into question. Yuan et al. [1083] conducted experiments where they smoothed the hard labels in the training data but observed that this technique did not improve the LLMs generalization ability.
9.3.4 Invariant Learning
Invariant learning [43] plays crucial role in OOD generalization by capturing invariant representations or predictors across different environments while disregarding spurious correlations. One notable approach of invariant learning involves the use of specialized loss functions, such as Focal Toss [540], Dice Loss [839], and Mixup Loss [1108]. Focal Loss was originally designed for class-imbalanced detection tasks, down-weighting well-classifedd examples to focus training on harder cases. Dice Loss, derived from the Sørensen Dice co- efficient, is widely used in segmentation to maximize overlap between predicted and ground-truth regions, thus emphasizing recall. Mixup Loss linearly interpolates both inputs and labels between pairs of examples, encouraging the model to behave linearly in-between training samples and reducing overfitting to spurious patterns. These loss functions differ in their inductive biases e.g., emphasizing difficult examples, opti- mizing overlap, or encouraging linearity - but all aim to produce more generalizable decision boundaries that are less scnsitive to environment-specife correlations. By applying Focal Loss to the training process of LLMs, these models emphasize hard-to-classify examples and enhance their ability to handle diverse and unfamiliar inputs.
9.3.5 Model Ensemble
Model ensemble [41, 475] is a powerful technique for enhancing the robustness and performance of AI models in complex environments. This approach combines predictions from multiple models to produce more accurate and reliable final outputs. Yuan et al. [1083] evaluated model ensembling but observed limited improvement in generalization ability. However, building on this foundation, recent studies by Jiang
62

---

Published in Transactions on Machine Learning Research (10/2025)
et al. [413] and Wan et al. [910] have introduced more advanced model ensemble algorithms, improving performance across several downstream tasks.
9.4 Domain Adaptation
Unlike OOD generalization, domain adaptation tailors the model to domain-specific tasks by injecting domain-specific knowledge [292, 812], including in-context lcarning (ICL), retrieval-angmented generation (RAG), fine-tuning, test-time training, and model editing. These methods enable loundation models to specialize in particular domains while maintaining their broad capabilities.
9.4.1 In-context Learning
In-context learning (ICL.) shows great potential to address the gap between foundation models and domains not covered in their pre-training and fine-tuning data [235, 640]. Recently, ICL has gained attention as a transformative approach for foundation models [64, 1125, 394]. It demonstrates the ability to adapt to new tasks or distributions without altering model parameters by adding domain-specific inpul-oulput pairs to the test example. This augmented input serves as a guide, helping the model produce desired outputs for new tasks. Consequently, ICL offers a flexible and efficient method for continual adaptation without the need for computationally expensive retraining.
In the field of LLMs, the BOSS benchmark [1083] explores ICL for LLMs by using examples from both ID datasets and the training split of OOD datasets. The findings reveal that fine-tuning domain-specific models is advantageous when sufficient training data is available, while LLMs with ICL perform better in low-resouree scenarios. Notably, the effectiveness TCL varies across models and tasks, highlighting the need for task- specific adaptation strategics. Complementing this rescarch, Reizinger et al. [744] delve into the intricacics of ICL, focusing on ils approximate non-identifiabilit, and the implications for understanding LLMs. Through combination of mathematical examples and empirical observations, their work demonstrates how this approximate non-identifaability manifests in OOD generalization, providing deeper insights into the behavior of ICL in various contexts.
For MLLMs, Zhang et al. [1126] demonstrate that ICL can significantly enhance the generalization capabili- ties, suggesting new approaches to overcome existing limitations. However, their study also investigates the robustness of ICL under various distribution shifts. The findings reveal that ICL is vulnerable to domain shifts, label shifts, and spurious correlation shifts between in-context examples and test data.
9.4.2 Retrieval- augmented Generation
Laagieva-auggmeneeed generation (RAG) enhances foundation models by retrieving relevant information from external data sources to supplement input queries or generated outputs [434, 639, 47]. This process provides nccessary domain knowledge, mitigating distribution shifts and improving generation quality [289, 426, 811, 1156). In practice, RAG techniques effective and efficient to apply in various unseen tasks with simple adaptation of the retrieval component, requiring minimal or even no additional training [734].
In the context of language models, Shao et al. [783] construct MASSIVEDS, a massively multi-domain database comprising 1.4 trillion tokens of both general web data and domain-specific data. Their findings demonstrate that as the database's size and diversity increase, more distributions are covered during infer- ence, reducing OOD scenarios. To incorporate this domain knowledge without requiring additional training, Antrieval-Augmennn
recent studies [802, 734] focus in-context els directly input concatenation of all retrieved texts additional context to LLMs. For the choice of retriever, most work [1113, 784, 669, 780] employ an embedding model to decide what to retrieve. However, with the increasing prevalence of LLMs, researchers have begun using the models themselves as retrievers to improve accuracy [77, 604, 967, 583, 926]. In a parallel direction, as these RAG methods may retrieve irrelevant information that even hurt the performance, Zhang al. [1122] proposed RAFT further fine- tune the LLMs to learn to disregard "distractor documents' within the provided context, thereby enhancing the model's ability to focus relevant information. The effectiveness of these Tn- In-Context RALMs has been
Language Models (RALMs). These mod-
63

---

Published in Transactions on Machine Learning Research (10/2025)
further demonstrated in several domain-specifie tasks [1022, 506, 1014, 585], showcasing the potential of RAG in addressing real-world distribution shifts.
To extend RAG to multimodal query input [1137], Wei et al. [958] create M-BEIR, a multimodal instruction- following benchmark building on existing 10 diverse datasets. UnilR is trained on M-BEIR to take a het- erogeneous query to retrieve from a heterogeneous candidate pool with millions of candidates diverse modalities. Built upon it, UniRAG [786] employs UnilR's CLIP Score Fusion and BLIP Feature Fusion models as retrievers, improving performance in MLLMs. For visual question answering (VQA) tasks, RAA- VQA [541] proposed novel framework for joint training of the retriever and the answer generator, and FLMR [542] further improved the retrieval accuracy by combining multi-dimensional embeddings from lan- guage and vision models. Similarly, MuRAG [158] uses T5 [729] and ViT [238] for text and image encoding respectively, and retrieval from a large-scale memory bank for knowledge- based VQA. To improve embod- ied agents, MART [1086] utilizes interaction data to fine-tune a multimodal retriever based on preference learning. For image captioning and text-to-image generation tasks, RA-CM3 [1057] enhances performance by using pre-trained CLIP model to augment inputs for CM3 Transformer. These methods effectively address the shift in knowledge representation across modalities. Additionally, domain-specifie multimodal RAG solutions have shown promising results in various fields [1002, 468, 852).
9.4.3 Tine-Tunigg with New Knowledge
Fine-tuning is a widely adopted method for addressing domain adaptation in foundation models [744, 1083, 446]. This technique involves adapting pre-trained models to specific downstream tasks by further training them on task-specific datasets. The primary goal is to enhance the model's performance on new, unseen data that may differ from the data it was initially trained on.
The BOSS benchmark [1083] evaluates vanilla fine-tuning for LLMs, which involves directly fine-tuning pre-trained models on ID datasets without any additional processes. This benchmark helps investigate the relationship between performance on ID and OOD datasets by varying factors such as model scale, training steps, available training samples, and tunable parameters. Observations indicate that fine-tuning with the full dataset generally yields superior performance for ID examples, while LLMs employing in-context learning (ICL) paradigms demonstrate better performance on OOD instances. Reizinger et al. [744] explore the ie-iddientibblityyy of fine-tuning in LLMs, highlighting its implications for understanding and improving these models. They argue that fine-tuning is non-identifiable, meaning that models with similar fine-tuning performance (such equivalent test loss) can exhibit markedly different behaviors when applicd to real-world tasks.
To address OOD generalization, Kirk et al. [446] investigate Reinforcement Learning from Human Feedback (RLHF) [185], which is typically implemented in three stages: (i) supervised fine-tuning (SFT), where the model aligned with high-quality human-labeled data; (ii) reward modeling, where a learned reward model predicts preference scores for outputs; and (ii)) reinforcement learning, where the base model is optimized against the reward model. While SFT is the first step in RLHF, it can also be viewed as standalone fine-tuning approach. In their experiments, the authors compare the full RLHF pipeline against using only SFT and find that RL.F generally yields stronger generalization to new, unseen inputs, especially under significant distribution shifts between training and testing data.
Jiang et al. [414] propose a novel method for fine-tuning LLMs in domains where obtaining large volumes of high-quality, domain-specific data is challenging, such as healthcare or harmless content generation. They re-evaluated the Transformer architecture to identify the most impactful parameter updates. Their analysis revealed that within the self-attention and feed-forward networks of the Transformer architecture, only the attention parameters significantly benefit downstream performance when there is a mismatch between the training and test set distributions. Based on this insight, they proposed Training All parameters but Inferring with only Atlention (TAIA), which involves updating all parameters during training but utilizing only the fine-tuned attention parameters during inference. Additionally, recent studies have observed that parameter-cfficient fine-tuning (PEFT) methods, such as Low-Rank Adaptor (LoRA), can maintain more general capabilities from the pre-trained distribution while acquiring new knowledge from the fine-tuning data [30].
64

---

Published in Transactions on Machine Learning Research (10/2025)
Beyond parameter-effiiint methods, recent work has explored activation steering alternative com- plement to conventional fine-tuning for improving generalization. This line of research modifies model acti- vations at inference time or during light-weight training to achieve desired behavioral shifts without large- scale parameter updates. For example, Lai et al. [474] propose Joint Localization and Activation Editing, which identífies and edits specific activation subspaces relevant to the larget domain, enabling effective low- resource fine-tuning. Similarly, Turner et al. [882] introduce Activation Addition, a technique for steering model outputs by adding direction vectors in activation space, allowing the incorporation of new knowledge or behavioral adjustments without gradient- based optimization. These approaches can reduce overfitting to in-distribution features while selectively enhancing capabilities relevant for OOD settings.
For multimodal scenarios, the proposal of EMMA [1043] adapts LLMs to the field of embodied multimodal agents. The key technique involves distilling the reflection outcomes of the LLM, which improves actions derived from analyzing mistakes in text world tasks. uses these outcomes to fine-tune the vision-language models on analogous tasks in the visual world, which is capable of quickly adapting to the dynamics of the visual world. The cross-modality imitation learning is facilitated by a novel DAgger-DPO algorithm, which ensures that EMMA can generalize to wide range of new tasks without further guidance. Belyaeva et al. [80] describe method to address OOD challenges by developing, a framework called HeLM (Health Large Language Model for Multimodal Understanding). HeLM integrates multiple data modalities, learns robust data encodings, and enhances predictive performance through comprehensive data utilization to achieve OOD generalization. Regarding model architectures, Ito et al. [402] find that models with multiple attention layers or those leveraging cross-attention mechanisms between input domains perform better in their constructed gCOG benchmark. Their study emphasizes that cross-modal attention and deeper attention layers are crucial for integrating multimodal inputs and improving gencralization in the presence of distractors and new tasks.
9.4.4 Test-time Training
Test-Time Training (TTT) methods view each test instance as an individual learning problem with its own generalization larget. This method creates a self-supervised learning task for each lest sample and updates the model parameters at test time before making a prediction. In the LLM era, Sun et al. [843] propose a new class of TTT layers for sequence modeling that transform the hidden state into an optimizable model, with the update rule functioning as a step of sclf-supervised learning. Zhang et al. [1123] propose a comprehensive recipe of TTT with eustomizable large chunk updates to enable long sequence modeling validated on diverse modalities. By aligning the training and test data distributions, these methods significantly enhance model performance when faced with distribution shifts.
9.4.5
Model Editing
All the domain adaptation methods discussed above modily a model's behavior by incorporating new knowl- edge. This process is closely related to model editing for foundation models [932, 1055], which aims to rectify specific errors without affecting unrclated inputs. To explore its potential in addressing distribution shifts, we will now provide an overview of model editing approaches, which typically adhere to three essential properties:
Reliability: The edited model should successfully produce the desired output for the edited sample, such as correctly answering 'Inter Miami" when asked Who does Messi play for?" Generality: The corrections made should be consistent across equivalent contexts, for example, accurately responding to 'Which team is Messi in?'
Locality: The acquired knowledge should be minimally affected, ensuring that unrelated queries like 'ho does LeBron James play for?' remain unaffected.
These properties ensure the reliability, generality, and locality necessary for the effective and efficient correction of foundation model behaviors. Recent studies model editing [356, 15) have also demonstratod promising performance in several OOD scenarios. Next, we will delve deeper into four distinct categories of model editing in LLMs (Figure 25), subsequently extending our discussion to address related issues in
65

---

Published in Transactions on Machine Learning Research (10/2025)
MLLMs.
Query:
Who is the current president of the US?
Ground Truth: Joe Biden
Store
Original LLMs
Memories
Parameters
External Memorization
Edited LLMs
Predict
Classifier
Parameters
Additional Information
Edit
Editor
Parameters
Global Optimization
Answer
Answer Donald Trump Joe Biden
Donald Trump
|
Update
Joe Biden x
v
Locate
Parameters
Local Modification
Figure 25: An overview of model editing methods in LLMs. Given an incorrect response from the original model, different editing strategies correct factual errors by modifying or augmenting the model's knowledge.
Memory-based Model Editing. In memory-based approaches, an external memory, outside the intrinsic architecture of the pre-trained LLM, serves repository for edited knowledge. LLM access and modify this external memory during inference. For example, Language Patch [660] performs editing by integrating with library of patches in natural language, and MemPrompt [608] adopts growing memory bank a look-up table to store the edit sample and its corresponding prompts, which is used to alter the prediction of the edit sample. KAFT [501] further strengthens the controllability and robustness of LLMSs working memory through counterfactual data augmentations. In this approach, the entity representing the answer in the context substituted with an alternative but still plausible entity. This substitution is intentionally designed introduce conflict with the genuine ground truth, thereby incorporating counterfactual and irrelevant contexts to standard supervised datasets. In addition to relying parameter-basod memory, IKE [1147] introduces novel factual information into a pre-trained LLM via in-context learning, where set of demonstrations will alter the prediction of target factual detail when the input is influenced by an edit. To solve more complex questions involving chains of facts, MQuAKE [1149] enables editing by breaking down each question into iterative subquestions and retricving the most pertinent fact from the edited fact memory.
Classiffer-based Model Editing. The classifier-based model editing paradigm aims preserve pre- trained parameters while utilizing classifier to determine whether behavior adjustment is necessary. In this approach, if a sample falls outside the scope of the edit. sample, the original model is applied to maintain predictions. Conversely, interventions occur when the sample is within the scope, with the specific interventions varying across different methods. SERAC [647] employs a scope classifier to determine whether the original model or a new lightweight model should be used for prediction. The new lightweight model specifically trained for in-scope samples. In contrast, Language Patch [660], CaliNET [234], and T-Patcher [395] introduce additional trainable parameters adapt the original model instead of requiring classifier) to entirely new models. For example, Language Patch trains new gating head (acting CaliNET and combine predictions from the original prediction head and newly trained interpreter head. T-Patcher insert residual block into the original model's feed-forward network (FFN) as an adapter. This adapler utilizes an activation operation on hidden stales to determine whether the intervention should be activated. When the activations are zero, there will be no change to the original prediction. However, the success of these classifier-basse methods heavily relies on the quality of the classifier, which also nocessitates substantial number of unrelated samples for training. Alternatively, GRACE [347] edits a model by adding retrieval-hased adaptor to chosen layer that enables judicious decisions regarding the utilization
66

---

Published in Transactions on Machine Learning Research (10/2025)
of the dictionary for a given input, accomplished via the implementation of deferral mechanism.
Hypernetwork-basset Model Editing. The hypernetwork- based model editing paradigm utilizes an external model, referred to as the editor, to facilitate parameter updates in the models. Knowledge Editor (KE) [120] employs a bidirectional LSTM to transform an edit pair, consisting of the edit sample, incorrect prediction, and correct label, into shifting operation parameters (i.e., mask m, offset b, and scaling factor a) for V: V V= a(m C V) + b. Based on KE, SLAG [348] further appends metrics for two types of imput texts: (1) those that, while not part of the targeted edit set, align logically with it; and (2) those that share formal resemblance to edited knowledge, but do not affect the prediction outcomes. However, hyper-networks are generally incapable of updating LLMs due to the massive parameter size. To address this issue, MEND [646] applies low-rank decomposition to V and utilizes two MLP layers to generate a new low-rank update, V. This approach lightweight and efficient, particularly for large models like T5-11B. Morcover, KGEditor [175] combines the benefits of memory-based methods and hypernetworks to ensure flexibility and further reduce computation costs. In particular, it introduces an additional feed-forward networks (FFNs) layer for knowledge storage. It then employs bi-directional LSTM to encode embeddings of triples. In this manner, KGEditor becomes an efficient way to edit knowledge graph embeddings. Despite the success of this paradigm, the editors need to undergo a prior training stage. The availability of training data, including edit samples and pre-training data, poses a critical challenge. While these methods employ syuthetic edit samples (e.g., selecting hypotheses via beam search except the top-1 for Question-answerte tasks 120), their generalization to realistic mistakes beyond the synthotic sample distribution remains limited.
srowledge-based Model Editing. The nhowledge-based model editing paradigm focuses on identifying a subset of parameters specifically associated with particular pieces of knowledge and only updating those parameters. This approach assumes that knowledge is stored within the FFNs, which function as key- value memorics [299]. Knowledge Neuron (KN) [202] attributes knowledge parameters using integrated gradients [845], where more salient gradients indicate a greater influence on the knowledge. Building on this idea, Rank-Ore Model Editing (ROME) [628] uses causal tracing to localize the specific FFN layer whose activation most strongly mediates the recall of a target factual association. Once the target layer is identified, ROME performs rank-one update to its value projection matrix, effectively replacing the original stored fact with a new subject-objedd mapping. MEMIT [629) extends ROME by identifying a set of relevant layers (e.g., layers 3 8 for GPT-J) and applying a closed-form multi-layer update. This allows MEMIT to edit multiple facts in parallel while preserving surrounding model behavior. It is important to note that these methods do not establish that these layers are exclusively dedicated to a single piece of knowledge, implying that the layers may be shared across different knowledge domains [282]. To mitigate potential effects on out-of-scope samples, regularization techniques are employed during the neuron/ /layer updates. For example, MEMIT enforces the model to maintain predictions for several unrelated samples. By adopting a knowledge-based approach, these methods selectively update parameters associated with specific knowledge while minimizing interference with unrelated samples. Based on ROME, BIRD [601] studies the novel problem of Bidirectiona Assessment for Knowledge Editing (BAKE), which evaluates the reversibility of edited models in recalling knowledge in the reverse direction of editing and incorporating the bidirectional relationships between subject and object in an edit fact into the updated model weights.
Model Editing in MLLMs. Compared to single-modal model editing, the task of editing MLLMs is more challenging due to their inherent diversity and complexity. Specifically, errors in MLLM outputs can be attributed to the synergistic effects of varions modalities. A recent study [174] introduces pioncering benchmark for MLLM editing, named MMEdit. This benchmark evaluates three aforementioned key prin- ciples: Reliability, Locality, and Generality, and covers two specific sub-tasks: Editing VQA and Editing Image Captioning. Empirical evidence indicates that while current methodologies [120, 1147, 646] are effec- tive for editing the textual model in MLLMs, they fall short in editing the vision module. Researchers are encouraged to explore innovative techniques for efficient and accurate editing across various modalities and to develop comprchensive benchmarks for evaluating larger MLLMss The end of 9.4.5 could profit from discussion on how the different model editing methods differ and what kind of advantages and disadvantages they carry.
67

---

Published in Transactions on Machine Learning Research (10/2025)
Overall, the four paradigms of model editing discussed above differ substantially in their mechanisms and trade-offs:
Memory-based methods store edits externally, avoiding interference with model parameters. They are easy to update and revert but introduce extra retrieval latency and depend effective memory indexing.
Classifier- based methods preserve original parameters and selectively activate edits only when needed, offering strong locality. However, they rely heavily on a high-quality scope classifier and require ample negative examples to prevent over-triggering.
Hypernetuwrrk-uuee methods generate parameter updates dynamically from an edit description, en- abling flexible and wiigt adaplation. Their main limitations are the need for a pre-trained editor network and reduced scalability to very large models unless combined with low-rank or paramoter-cfficient techniques.
Knowledge-basse methods directly modify internal representations linked to specific facts, often achieving high reliability and generality with minimal changes. Yet, they risk unintended side effects if the targeted layers store multiple pieces of unrelated knowledge, and they require accurate localization of the relevant parameters.
In multimodal settings, these trade-offs can be amplified: memory- and classifier-based methods may gener- retrieval/Classffcatio classification, while knowledge- and hypernetwork-based methods may offer more precise edits but require sophisticated cross-modal lo- calization strategies. Future research may benefit from hybrid approaches that combine the precision of parameter-based edits with the flexibility and safety of external-memory or classifier gating mechanisms.
alize more easily across modalities but depend on modality-aware retrieval/
9.5 Current Limitations and Future Directions
Foundation models, despite their remarkable capabilities, face several challenges when confronting distribu- tion shifts. These limitations primarily stem from inherent difficulties in OOD detection, generalization, and adaptation. Such challenges significantly impact the reliability and robustness of these models real-world scenarios. When exposed to data that deviates from their training distribution, these models often exhibit decreased performance [1083, 1126), leading unreliable predictions in dynamic environments where data charactcristics frequently change.
While various OOD detection methods have been developed, many struggle with scalability issues, making them less practical for large-scale deployment. Current approaches to OOD generalization and adaption, such as domain adaptation [1083, 446, 1043) and adversarial training [56, 1083, 899], demonstrate varying degrees of success across different domains. These methods often require extensive retraining or fine-tuning to handle new domains effectively, a process that can be both resouvce-intensive and cone-connuming, Furthermore, many techniques for improving OOD robustness heavily depend on the availability of large, high-quality datasets [1083, 1043, 80, 402]. This dependence poses significant challenges in domains where data is scarce expensive to obtain. Additionally, for multimodal foundation models, effectively integrating and processing diverse data types remains a complex task. Current editing and generalization methods often fall short in scenarios involving multiple modalities, such as text, images, and audio [980]. Last but not least, modern foundation models often undergo continual pre-training and fine-tuning, either horizontally across a sequence of domains or vertically from a gencral-purposs model to a domain-spocific model [799]. As a result, they inevitably tend to suffer from catastrophic forgetting, such horizontal forgelling [799] when continually adapting across domains and vertical forgetting [799] when continually adapting from more general models to more domain-specific models.
To address these limilations, future research should focus on developing more light OOD detection and generalization methods. These approaches should aim to identify and mitigate distribution shifts in large- scale settings while maintaining low resource requirements. By focusing on efficiency, such methods could be more readily integrated into practical applications, enhancing the robustness and reliability of foundation model systems across diverse world scenarios.
68

---

Published in Transactions on Machine Learning Research (10/2025)
To adapt to rapidly evolving environments, should prioritize the development of continual even life- long learning mechanisms for foundation models [1033, 799, 442]. These mechanisms would enable models to adapt to new data distributions without requiring extensive retraining [501] while simultaneously preserving knowledge acquired from previous training data, including data previously used during pre-training or from previous domains. In other words, they should remain robust against both vertioal and horizonlal forgel- ting [799]. This approach could significantly enhance the flexibility and longevity of foundation models dynamic domains. Additionally, due to the scarcity of data in several domains, developing more data-efficient transfer learning algorithms creating diverse synthetic data will help models generalize to more practical applications.
To further improve the generality of foundation models, advancing their abilities to handle multimodal data effectively is essential, with unified frameworks that can seamlessly integrate various data types and lever- aging tochniques like cross-modal learning and multimodal embeddings enhancing performance in complex scenarios [980, 1126, 1065, 1076]. By addressing these limitations and exploring these future directions, we can significantly improve the robustness and reliability of foundation models, ensuring their effective deployment in diverse real-world applications.
69

---

Published in Transactions on Machine Learning Research (10/2025)
10 Explainability
There are substantial existing efforts tailored towards the explainability of foundation models, particularly LLMs. In this section, we demonstrate the literature on the explainability of LLMs from the following aspects:
(1) Feature Attribution Methods, i.e., Explaining LLMs with the raw features (words, sentences, syntax); (2) Exploring the inherent knowledge incorporated in LLMs themselves; (3) Discovering the roles and training samples in pre-training, fine-tuning, and lew-shot learning. Following an overview of the methods used for model explanation,
dive into the evaluations and applications of explainability in LLMs. The discussion then broadens to include multimodal large language models (MLLMs), emphasizing the ongoing efforts in the field. Figure 26 provides detailed overview of various methods to explain different foundation model components.
K
V
FFN Layer
The Roles of Samples
K
V2
Instruction
In-context Learning
Training
Tuning
Forward
Perturbed Input
Gradient-based
Attention-
Ma Mea
Probing
Perturbed Input
Different Output
Concepts
Mechanistic
Attention
Concepts
Probing
Figure 26: An overview of explainability in foundation models. This figure illustrates various techniques for uncovering how different model inputs and internal components influence model outputs. The right legend highlights the role of samples in different learning stages and the typology of explanation approaches.
10.1 Feature Attribution Methods
When adopting LLMs on downstream tasks, it is important to determine which part words or tokens in the input contribute most the prediction. Thus, we need to determine the importance of cach part of the input, i.c., explaining the prediction using the raw fcatures. To explore this, there are several important lines of work:
10.1.1 Perturbing the Input for Explanation
To study the effects of the raw features for model prediction, it has been important to perturb part of the input (a piece of text) while monitoring the model output. With this routine, Perturbed Masking [994] proposes to perturb token in the given sentence while monitoring the representation of another token. They further propose span-level perturbation to study the impacts of certain span within the sentence. While Wu al. [994] regard the monitored variable the representation of the token span, MICE [754] study the roles of inputs for model prediction in classification tasks (i.e., the monitoring variable becomes the model prediction). They present a method to find the edits that could flip the model's prediction, where the edits could serve as contrastive explanations. In addition, perturbing the input to shift the label could create counterfactual examples. Crest [879] proposes a framework to first perturb the input sentence with masks and then edit the masked tokens obtain counterfactuals. Here, perturbing the sentence with masked tokens essentially extracting rationales they are both locating the important tokens for model prediction, though finding the rationales could also be achieved by other methods [488, 351]. To provide
70

---

Published in Transactions on Machine Learning Research (10/2025)
Prsturiatao tppkastize
Wis [114] Laker Be [16o]]
1134),
[171],
2000
Noownte at. [11]] Strree et al. (077] Sangal i Bee 2000 We A Org (00]) Vitta et al (vee] Mosttoone all Joat).
Prissples Explamiion
Ghactrek- seed oxplazariize
Da sl. (201]. Eagidaard poat, Karioappe it ok [429]
Toury et ol [864, Golllery joag, it (800), Via Beiikoo 1005, Ropasato Telessau (PBC)
Atcalico--saa syllunrkza
[f01], Park
Juuett
Hoove (563)
[117:
[M6]]
Aboaz Rusienz [s), Barion jos) Hao a. [21s]. Jum Wallazz (an]] Waggelle & Patter (pms
Inforsutiin sepas kovp
Clark e. dl. 200, Lis tk. [541], Tanay it [608] Sklisoon (16, Beliskow at [3]
Seeratckchhok
fenn & Kobbhorn [m]] Sender Jon[
[1500)
Priling liesst impresration
Semeraral Vennartie kronledge
Dirret ic Manniar [10]] Chen (1i1f, Xarniklay Chetroni (o01
Explainsbillt in Fouurhtt Moditd
Xnowtolg: Explenion
Norma ansrvenssu
Tomole, Heasiggu ot (871, Guraeces ot al (806, Opendt (67g, Stas et 1009.
Maclaa lignaek [1i6,
Apáliaraki
(96, E2 [10, Rockbootder [709, Xhong (11
[at) Memill juid Prteest et al. hut)
Atberikon chriits
Fllaay et at. 1177. Oloice al (67]
Medhuaaii lakegetubilly
Kooekolge is KLP lyyts
Gon et al (288) Yoo sl. [OSs]] Yaet el. jo0t] Mera ol (89
Trinung dyanuio in esuspt spant
Gom al [259] Park (00g
Poplstinng with
Kin al (154) Kid
Biaug [asf, Barrank al. [n6, Wang [121]] Cagman [129 Mr Audraa (1i9, Wang al foat, al. [1as]]
(1029) Kaoniooons
Chattopealloo al [184), Theng (110))
Stupbty et (285) Kockal es 1S51, Kih di tlann (405, Yel rt ([6]). Grose al. 2415 Rub d.. (107] Alar et sh. [UL84]. Wi es al, 364)
Cenoept Learning
Hoims compli to triting
Jultreee of fraoshit denecetooiiice
Ralta Gooal We (06] Madaon Yadoohakacs 1977. Wang al. puo)
[889
1127), Wii al.
CLIP Cateed
Rodlond et al. (709, Youg et el. 1065, Chcks ci 180), Lo al [500, You os al [100,, Agucona [
Xalitnodl Expiention
[.Mohaeed mollttndal
The al [177]] at 500,
(507) Warg al [944
Trazakealdity acnes undalitoo
Silaker sl. (o07] Sancal Bea jos). Raoodaadd post, Liung ni. [135], Zara et al. (1887) Lit al [Dss]
Passeidity of sapiaatizo
Jscort & Gobllery (uis, Saza et dl [191], Clau et al [100, Xirlew is dl (137) DeYoung et al 201]
Enshation
Paltffshew of explenations
DoNouu [22]], Classokanon Alena [181] Clarre al 1811, Lis sl. 197). [177]
Atazaaom (16,
[196. itepo (90f, Larhan
Du sl. pitt 2111, Clar & [14. Wi ct... pol) Li d. (297) Eamplaes dod. 1199) Nyr e al, 670)
Sawssy at al [babt, Noubbegee et sl (o1)) 14 du 1200, Hendbrrkk et al DSof, Low et ab [ee], Pimmerke et al, [aed,
Dovistruus Applccaiio
Rosdan di Booser [00nn Boss et al (111) to And [55]] Chard et (200) Ringer al (PA).
Kaundyy at al, joof, Bang at al. poot losh es al. (200, Ais et al. jast, Yas et al. [oo]]
Figure 27: Taxonomy of Explainability in Foundation Models.
more diverse perturbation types and locations, Polyjuice [982] presents a general-purpose counterfactual generator that can generate diverse sets of realistic counterfactuals.
10.1.2 Gradient-based Explanation
Mohebbi et al. [649] adopt gradient-based based attribution methods to provide token-level attribution scores to understand the representation space of BERT [219] better. More advanced difference-rronn-eeerrneo approaches such as Integrated Gradients (IG) are also used to explain the BERT's prediction [807, 765]. REAT [240] decomposes the final prediction of RNNs directly into the additive contribution of each word in the input text. Voita et al. [905] extend LRP [651] to the Transformers to attribute the relevance score on the souree and target contexts in Neural Machine Translation tasks. Wu & Ong [993] analyze different gradient-based methods for explaining BERT classification results. Recent work has extended gradient- based methods to autoregressive decoder-only language models. For example, Enguehard [258] introduce Sequential Integrated Gradients, which computes attributions along the generation path by integrating gradients between an empty sequence baseline and the final generated tokens of GPT-2. Kariyappa et al. [429) propose to approximate token attributions by backpropagating importance through each auto-regressive decoding step, which scales efficiently to long sequence generations.
10.1.3 Attention- Explanation
Previous works suggest that information could be encoded within the heads of the attention weights [855]. including abundant information [303, 903, 902, 730, 354, 192, 1117, 1116], which could be used for both input- level explanation and allention heads pruning [904]. Multiple tools are proposed to visualize the attention to illustrate the correlations between words for explanation purposes [901, 691, 409, 365]. Moreover, DeRose et al. [214] propose Attention Flows to visualize the whole attention flow instcad of the visualization of one layer. Some methods combine gradients and attention for explanation [68, 346], which generally perform better than using attention alone. Abnar& & Zuidema [6] treat self-attention as flow network across layers to
71

---

Published in Transactions on Machine Learning Research (10/2025)
enable post-hoc computation of token-to-token information propagation, which shows higher correlation with gradient-based and ablation-based importance scores compared to raw attention. Though attention scores could be used to understand the large language models, they may not necessarily be capable of identifying the explanations [407].
10.2 Exploring the Knowledge in LLMs
Instead of explaining LLMs by highlighting the important tokens or spans in the input, interest increasingly gravitates toward understanding the breadth of knowledge encapsulated by these models. Several key areas of investigation are outlined as follows:
10.2.1 Probing the Representations within LLMs
Probing the model could help us understand deep neural networks [78]. Early work by Veldhoen et al. [897] introduced diagnostic classifiers for revealing how neural networks process hierarchical structure by training simple probes on latent representations to test hypotheses about compositional strategies. With such probing philosophies, recent transformer-based works investigate the embeddings and hidden states from various mechanistic components of the network. Kunz & Kuhlmann [471] show that the token embeddings learned by BERT and ELMo contain rich information about the exact linear context of the token. Belinkov et al. [79] interpret the representation of different layers in NMT encoders, finding that higher layers have more semantic information. In contrast, lower-layer representations lend to be more suitable for part-of- speech tagging. The fact that language models can capture semantic information and conduct arithmetic operations is also studied in Sorodoc et al. [824] and Zhou et al. [1158]. Similarly, Clark et al. [193], Lin et al. [543] show that Bert's representations encode surface and positional information in the lower layers, but more semantic features in higher layers, while Hewitt & Manning [355] propose structural probe showing the syntax trees are embedded in a linear transformation of ELMo and Bert's word representation space. Building on previous probing work, Tenney et al. [856] probes word-level contextual representations to investigate how they encode sentence structures. Different from the above mods paying attention to the representation in certain metric spaces (typically Euclidean space), Chen et al. [140] consider the probing methods in hyperbolic space, which could better recover tree structures. While these methods could reveal the ability of representations to encode syntactic information, Maudslay & Cotterell [619] show that syntactic probes may not properly isolate syntax. With new corpus that is semantically nonsensical but syntactically well-formed, it is shown that syntactic and semantic information are entangled. Further, Zhang et al. [1111] argue that even with the existing works, il remains unclear whether LLMs have understood linguistic knowledge. Thus they probe GPT-3 to show that it has acquired linguistic knowledge in most cases but may still fail when disturbances happen. Apart from exploring the representations, some other works focus on self-attention heads, which could be helpful for heads pruming [457, 193].
Some methods are designed to be used during the inference of LLMs without training the classifier on the hidden vectors, such as cloze completion or text generation [703, 38, 505, 742]. Though prompts can be designed to reveal the abilities of the LLMs, Zhong et al. [1148] question if the prompt-search methods also learn from the training data, i.c., the training data may contain certain regularities of the underlying fact distribution that could be exploited.
Probing methods are also used to understand the roles of neurons in LLMs. Torroba Hennigen et al. [875] propose a framework based on a decomposable multivariate Gaussian probe to explore how linguistic information is structured within the representation, showing that most attributes are reliably encoded by only few neurons. Moreover, some methods propose to probe the internal activations to predict the presence of features in the input, showing the sparse combinations of neurons can represent many features [339]. Recently, OpenAl has shown the possibility of using an advanced LLM (c.g., GPT-4) to explain the neurons in a small model (e.g, GPT-2) [676]. Sunmarize and Score (SASC) [809] proposes to generate candidate explanations to explain the modules from LLMs, which could be more efficient than explaining single neurons. Marks & Tegmark [616] reveal that LLM representations of true/false statements form distinct lincar directions that can be identified via probing and causally intervened upon to flip model outputs. Ji et al. [411] show that identifying a linear "verbal uncertainty feature" in LLM representations can be manipulated at inference
72

---

Published in Transactions on Machine Learning Research (10/2025)
time to reduce hallucinations. Merullo et al. [634] investigate how pretraining data frequency influences the emergence of linear representations of factual relations and find strong correlations between term co- occurrence counts and probe performance across models.
10.2.2 Explaining LLMs with Concepts
Concept-based explanation refors to mapping the input into concepts and then using a lincar classifier to predict the final class with the mapped concepts. As the prediction from the concept to the class is a simple linear classifier, it has the property of explainability even though the mapping from the input the concepts is not explainable. Pioneering methods in this direction include Concept Activation Vectors (CAVs) [436] and Concept Bottleneck Models [449]. Such concept-driven framework is widely adopted in visual representation learning where the images are first mapped to the concept space, based on which the classifier makes the decision [436, 449, 1029, 432, 134, 1131, 386]. More recently, Wang et al. [920] propose Probabilistic Conceptual Explainers (PACEs), drawing inspiration from hierarchical Bayesian deep learning [915, 916, 419] and topic models [95] to provide concept-based explanations at multiple levels (e.g.. datascts, images, and patches) to address key concerns in model interpretation such as faithfulness, stability, and parsimony.
Beyond computer vision, CAVs are also tailored to language models for sentiment classification tasks [122]. featuring two concepts: Positive Adjectives and Neutral. Besides, while Captum [122] define concepts mamually, Mu & Andreas [653] propose to learn the abstractions by analyzing the neurons, where they find that neurons learn shallow lexical heuristics from dataset biases. Wang et al. [919] propose Variational Language Concepts (VALCs) to learn the concept-based explanations in an unsupervised learning manner while enabling neuron editing in the concept space. Turner et al. [881] propose to steer the behaviors of language models by curating concept activations and injecting them in the model's hidden layers, and Zou et al. [1187] propose unified paradigm for concept interventions in the activation space. Barrault et al. [70] propose Large Concept Model (LCM) to perform next=seeteene-prediiioon braseddooooieeeeeeeeeddd autoregressive learning in the conceptual embedding space. In summary, developing concept representation is crucial step towards interpretable LLMs for diverse tasks. Such interpretability offers a feasible solution for diagnosing, revising, and intervening LLMs.
10.2.3 Mechanistic Interpretability
In each block of a transformer, the self-attention layer projects input tokens or hidden states into query, key, and value vectors that effectively store key-val value memories. The main focus of mechanistic interpretability is reverse engincering for retrieving contextual information across those vectors, tokens, and layers, which pro- vides systematic approach to explaining LLMs [257]. The aforementioned study also finds that in-context learning in small models could be explained by specific attention heads, termed "Induction Heads". This mechanism is hypothesized to constitute the mechanism for most "in-context learning' in large transformer models [673]. Another line of work focuses on FFN layers. Earlier work [298] argue that FFN layers contain most of the information that operates as key-value memories, and more recent works [1056, 1075) propose to search neural circuits or salient neurons for parametric knowledge representation. With the localization of the information, we could perform model editing on the relevant matrices in FFN layers [628]. In ad- dition, Geva et al. [299] and Park et al. [692] analyze the learning dynamics of generative models in the concept space, demonstrating that updates can be decomposed into sub-updates, where each sub-update corresponds to human-interppettable concepts. Hou et al. [368] introduces MechanisticProba to explore a mechanistic interpretation of LLMs for multi-step reasoning tasks.
10.3 Discovering the Roles of Samples in Training, Fine-tuning, and Few-shot Learning
The development of foundation models encompasses multiple learning stages, including pre-training, fine- tuning, and few-shot, learning. During different stages, samples play distinct roles illustrated in Figure 28.
73

---

Published in Transactions on Machine Learning Research (10/2025)
Pre-training Stage
b) Instruction-tuning Stage
In-context Learning Stage
A large lampuage model (LLM) is computational model capable language generation or other natural langunge processing tasks. language models...
You scientist who just invented time machine. Where do you travel firx?? As a sclentist, would travel back in time 1969 and witness the Apollo mission,
Examplel: there cars the parking more cars arrive, how many cars are in the parking lot? There are originally 3 more cars arrive. 4++2
huna--rdhed---n <gilke> Delicious Pasta Recipesititles <meta namen description" content="PPoi recipe for a quick meal"? hheadiin <bodyle... ....
Apollo II spaceflight conducted by the United States from July 16 to July 24, 1969 It marked the first time in history humans landed on the Moon.
the parking and more cars arrive, how many cars are in the parking loe? The answer S. Example2:
Example there
The influence of samples in pre-training, Instruction-tuming and in-context learning stages. We highlight the beneficial and detrimental textual fragments in green and red, respectively.
Figure 28:
10.3.1 Influence of Single Example in Training
There is a growing body of work studying the effects of one single example in the training process. SHAP [785] first proposes Shapley values allocate the contribution of one single player in coalitional game. TransSHAP [453] proposes to adapt SHAP transformers models, Bert specifically, to explain the classification results. Other works measure the effects of the example with the influence of this example on test loss values [1061]. Influence function,
statistical technique adapted to deep neural networks by Koh & Liang [448], approximates how upweighting a single training example would change model parameters and test loss. Recently, Grosse et al. [317] scale the influence functions on LLMs with up to 52 billion parameters, and Ruis et al. [757] leverage influenco- analysis to demonstrate that procedural knowledge in the LLM pretraining sequences drives the emergence of reasoning capabilities.
10.3.2 Influence of Training Stages
The training stages in the current. most powerful models include pre-training and instruction tuning, LIMA [1152] analyzes the relative importance of pre-training and instruction-tuming, hypothesizing that the knowledge revealed in the generation primarily comes from the pre-training stage, while instruction- tuning tends to fixate on the style and format of interacting with users, which is tested by using only 1000 curated examples to train Llama-65B to achieve near-GPT-4 performance on a controlled human study. Wu et al. [984] explore the instruction recognition and knowledge evolution before and after instruction-tuning demonstrating that instruction- instruction-tuuing could better identify the instruction parts from the input and align the knowledge with the user instruction.
10.3.3 Influence of Samples in Few-shot Learning
Few-shot learning in LLMs typically refers to in-context learning (ICL.). Li et al. [527] investigate ICL's functionality using contrastive demonstration and saliency maps. Wei et al. [964] examine how specific examples influence learning outcomes in few-shot scenarios, employing two distinct approaches: ICL with intentionally incorrect labels, and ICL with semantically unrelated labels. They found that large models can better override the input-label mapping learned during the pre-training stage, and small models rely more on semantic priors than large models do. In both settings, they find larger models and those enhanced with In- Context Fine- Teine-Tniigg perform better. Wu et al. [981] focus on how Chain-of- Chain-of-Thhuugtts (CoT) affects the model behavior, while others try to perturb CoT demonstrations and check the effects the outcome [607, 913]. Hahn & Goyal [340] propose a theory of emergent in-context lcarning as implicit structure induction to show how compositional utructure in pretraining data gives rise to ICL. Xie et al. [1011] provide a complementary perspective by modeling ICL as implicit Bayesian inference where LLMs infer latent concepts shared across prompt examples. From the empirical perspective, Lu et al. [589] conduct largo-scale evaluations of emergent abilities across multiple tasks to conclude that the emergent performance gains can be largely ascribed to ICL mechanisms and parametric knowledge.
74

---

Published in Transactions on Machine Learning Research (10/2025)
10.4 Evaluation of Explainability
The evaluation of explainability usually focuses two perspectives [1136]: (1) Plausibility (also known persuasiveness by 403). A plausible explanation seems logical and coherent to the audience, regardless of whether it is correct accurately rellects the model's reasoning process. Essentially, plausibility measures the quality of the explanation in terms of its persuasiveness and understandability from human perspective. (2) Faithfulness. A faithful explanation accuratoly represents the internal workings and docision-making processes of the LLM, demonstrating how well the explanation aligns with what the model doing when generating the response.
10.4.1 Evaluation of Plausibility
To evaluate the plausibility of the explanations of pre-trained LMs, Shen et al. [793] propose a benchmark to test LMs abilities in five dimensions: grammar, semantics, knowledge, reasoning, and computation. Another benchmark, HateExplain [617], asks the annotators to highlight part of the text that could justify their decisions, which could serve the ground-truth explanations. With these ground-truth tokens, could calculate the metrics such as Accuracy, Macro Fl-score, AUROC score [617], and AUPRC (Area Under the Precision-Recal Curve), IOU (Intersection- Union) [221], etc.
The above metrics could be applied to the explanations with raw features (discussed in Sec 10.1), but they may not be suitable for the explanations based on natural language (Sec 10.2.1) as there would be no ground- truth explanations in this case. To resolve this issue, Chen al. [159] propose to evaluate the counterfactual simulatability of natural language explanations, i.e. whether humans could predict the model's behavior according to the explanations given by the model. If so, then we say LLMs could explain themselves.
10.4.2 Evaluation of Faithfulness
To evaluate the faithfulness of rationales selected by the model, ERASER. [221] proposes the following metrics: (1) Comprehensiveness, which refers to the probability change of the original predicted class before and after the removal of the predicted rationales, and (2) Sufficiency, which means how much the extracted rationales could support the model to make prediction. Ideally, the objective is to achieve maximal change of comprehensiveness without compromising the accuracy of predictions when relying solely on the extracted rationales. In addition, TaSc [186] proposes another line of metrics: (1) Decision Flip- - Fraction Of Tokens (DFFOT), which measures the fraction of important tokens required to be removed to cause a decision flip. A lower DFFOT indicates more faithful explanation. (2) Decision Flip- Most Informative Token (DFMIT), where the rate of decision flips caused by removing the most influential tokens is reported for comparison. To further evaluate the faithfulness of the explanations, Liu et al. [577] propose a faithfulness violation test, showing that most methods are hindered by the faithfulness violation issue. Although these metrics each have their rationale and applicability, the consistency between these metrics remains questionable. Chan et al. [131] show that the explanations that achieve the best DFFOT may have the worst Sufficiency score. These metrics are also not suitable for natural language explanations. To solve this issue, for classification tasks, Atanasova et al. [50] propose two tests: (1) counterfactual input editor for inserting reasons leading to counterfactual predictions; (2) reconstruct inputs from the reasons given by the explanation models and check if they lead to the same predictions. Different from modifying the input and monitoring the output (basically perturbation), REV [146] quantifies the amount of new, label-relevant information in the explanations beyond the information within the input, which can give the mcasurement without perturbation. For CoT-style explanations, Turpin et al. [883] find that CoT explanations could be vulnerable towards biasing features in the model inputs, thus being systematically unfaithful. Lanham et al. [477] monitor how the model predictions change when the input is intervened. They argue that models may produce less faithful reasoning when the models become larger. These researches demonstrate the need for better explanations in the T style.
75

---

Published in Transactions on Machine Learning Research (10/2025)
10.5
Applications of Explainability
Gaining the explainability of LLMs has various applications, including diagnosing the model, and improving the model, which can help obtain user trust in the model.
10.5.1 Avoiding Shortcut Learning
Du et al. [242; 241] point out that LLMs may rely on shortcut features like data biases, and artifacts to make predictions rather than understanding the meaning, which demonstrates an important challenge in the field of LLMs. Chen & Ji [145] propose to utilize the explanations revealed by the model to determine if the model is robust or not, as they argue that robust model should behave consistently between original and adversarial example pairs. Wei et al. [963] use chain-of-thought to understand the reasoning process of the model, though the faithfulness needs further exploration [883, 477]. Li et al. [497] design an urban- environment multi-agent simulator based on customizable first-order logic to evaluate the logical reasoning capability of LLMs. These works have identified new challenges in LLMs.
10.5.2 Improving Model Performances
Apart from understanding the model, other works try to improve the model with explanations, which can also help gain user trust. For in-context learning, Lampinen et al. [476] find that using explanations in the prompts can improve performances, and hand-tuned explanations on small validation set could even offer substantial improvements. Zhou et al. [1162] use LLM interpretation to improve the performance of LLMs on natural language understanding. To improve the model's reasoning ability, Nye et al. [672] find that asking LLMs to emit intermediate computation steps into a "scratchpad' helps with multi-step reasoning tasks. On the other hand, Turpin et al. [883] demonstrate that stain-of-thought explanations can be systematically unfaithful. LLM may produce plausible but misleading rationales, which cautions against over-reliance [828] supervise the model's attention weights to encourage the model to pay more attention to the words that are present in the explanations, which significantly improves the model performance. Li et al. [498] proposes the first bilevel planning framework to learn neural predicates from in-context demonstrations to achieve compositional generalization for new tasks. In the large language model regime, Mukherjee et al. [655] propose to train a 13B model with the explanations and reasoning processes provided by GPT4 to improve the reasoning ability of small models. Farly works by Hendrycks et al. [352] show that structured human feedback can steer language model behaviors toward alignment goals. Lee et al. [482] introduce the XMD framework which shows humans the explanations of model behavior and also updates the model based on the user feedback. Then to improve the model's OOD generalization ability, there is a popular paradigm called Explanation Regularization (ER) which aims to align the model rationales with human- annotated rationales [551, 748, 1092, 300, 388, 755, 433], where the effects of ER on OOD generalization is evaluated by ER-TEST [420]. As these methods require human-annotated rationales, which might be exhaustive, AMPLIFY [600] proposes to automate the process of rationale generation with the insights from post hoc explanations to provide corrective signals to LLMs. Some other applications include identifying the important instructions to compress the instruction [1062]. Fernandes et al. [268) comprehensively sunmarize a taxonomy of integrating human feedback into natural language generation systems for improving generation quality and explaining model decisions.
CoT for performance gains. Besides the training-free prompting, Stacey et al.
10.6 Explainability of MLLMs
Existing works on the explainability of MLLMs primarily focus on CLIP-based image-text alignment models. Early research suggests querying GPT to augment class labels, thereby improving the zero-shot performances of CLIP [725]. Recent methods utilize CLIP to analyze the composition of images with textual concepts, wherein the concepts are further used for image classification [135, 1045] or editing [136, 596). Such ap- proaches offer additional interpretability [1029, 1045] and controllability compared with directly using the CLIP representation for class prodiction. Furthermore, Agarwal [9] investigates the trustworthiness of ex- planations generated for zero-shot and fine-tuned Vision and Language Models (VLMs), revealing that explanations for zero-shot. CLIP classifiers are more faithful than those of the fine-tuned versions. While
76

---

Published in Transactions on Machine Learning Research (10/2025)
these works concentrate on image-text alignment models, the explainability of LLM-bbsed image/video un- derstanding models, such as MiniGPT-4 [1177], LLaVA [556], VideoChat [507], and LVChat (944], remains underexplored.
10.7
Current Limitations and Future Directions
Despite significant advancements in the field of explainability, several limitations persist that necessitate future attention:
10.7.1
Faithfulness Raw Features
Current methods, ranging from imput perturbation to gradient-based and invention-basedd techniques, offer explanations for model predictions. However, the faithfulness of these explanations remains questionable. As illustrated in Section 10.4.2, the metrics from different perspectives may vary drastically [131]. This issue extends to downstream tasks where the model may rely on various biases for predictions [351, 242]. potentially compromising their generalization ability if the predictions are not truly faithful.
10.7.2 Understanding How LLMs Store Knowledge
Research focusing on the knowledge stored within LLMs examining layer representations [79, 471, 732] and analyzing generated content [562, 22] has shed some light on the distribution of syntactic versus semantic information across layers. While these works provide the insight that lower layers encode syntactic information and higher layers possess semantic knowledge, the community is still actively discussing how knowledge is dynamically injected into the model during training and how the injected knowledge is triggered through the inference process. For knowledge injoction, Müller-Eberstoinn et al. [661] trace represontational subspaces during pre-training to capture phases when syntactic subspaces rapidly emerge and disentangle from semantic and reasoning subspaces. Hu al. [372] fit hidden Markov models training-time time metrics (o.g., weight norms, variances) to derive latent states of learning dynamics, which reveal phase transitions during the training. Wal et al. [908] deploy multiple LLM pre-training runs demonstrate that training dynamics and knowledge injection stabilize consistently with identifiable outlier behaviors. On the other hand, how to trigger relevant knowledge with the imput demonstrations needs further understanding [557, 155]. Some works argue that the knowledge is mainly stored MLP layers [628, 629]: however, it is shown in their papers that attention layers also have slight effects when predicting the facts, especially in earlier layers (See Figure in [629]). Even in the work that explicitly stores knowledge a memory module [943]. how the model processes the knowledge is under-explored.
10.7.3 Transferability of Explanation Across Different Modalities
How to develop unified explanation frameworks that can inherently transfer across modalities has been emerging topic. As we have discussed above, prior works on Integrated Gradients [807, 765, 258] are modality- agnostic and can be employed on image classification, text generation, and multimodal VQA tasks. Yet these gradient- based attributions suffer from superficial faithfulness and low robustness to input perturbations [8]. On another track, atlention-based
hods such as MultiViz [531] can provide visualizations of cross- model attention and thus enable analysis of pixel-to-token flow in multimodal transformers. However, the credibility of attention explanation has long been arguable in the community [407, 975). Concept interpretation tools such representation engincering for LLMs [1187] have seen the potential to be adapted to vision-language modalities [566]. Aggregating multiple explanation levels and addressing the aforementioned limitations will motivate future work in developing robust, faithful, and transferable explanations for multiple modalities. It is also incrcasingly necessary to develop standardized benchmarks for cross-modal explanations that promote both interpretability and fidelity.
10.7.4 Reliability and Responsibility of Foundation Models from the Explainability Perspective
Without a deep comprehension of foundation models, ensuring their reliability and responsibility is chal- lenging, where explainability has the potential to offer a pathway to address these issues. For instance,
77

---

Published in Transactions on Machine Learning Research (10/2025)
identifying the biases in pertaining data and implementing various de-biasing strategies could pose more equitable models [522]. Moreover, understanding how the model stores knowledge [533] can facilitate model editing with the current knowledge and the unlearning of harmful information, achieving up-to-date and safer foundation models [628, 927, 1104]. Howover, the effectiveness of interpretability methods themselves must be critically assessed to ensure they provide meaningful insights. Adebayo et al. [8] perform sanity checks for saliency maps and reveal that some widely used saliency methods are independent of both the model and the data, questioning their validity in explaining model behavior. Similarly, Alvarez-Melis & Jaakkola [24] investigates the robustness of interpretability methods and demonstrates that small perturbations to the input can significantly alter the explanations provided, highlighting the need for more robust interpretability techniques. Moreover, understanding how practitioners use interpretability tools is also crucial. Kaur et al. [431] explore data scientists' use of interpretability tools and find that mismatches between tool capabilities and user needs can limit their effectiveness in ensuring model reliability and responsibility. They emphasize the importance designing interpretability tools that align with the practical requirements of users. As the development of increasingly powerful loundation models continues, focusing on both the advancement and the critical evaluation of explainability methods cannot be overstated.
78

---

Published in Transactions on Machine Learning Research (10/2025)
11 AIGC Detection
The advent of foundation models has led to a surge of artificial intelligenco--enerateedd content (ATGC) across various modalities, including text [309, 676, 311], images [737, 1112, 259], audio [458, 337, 387, 27]. and video [454, 94, 65). While these technologies have unlocked many useful applications, they also pose significant challenges, particularly in terms of content authenticity [320, 523, 363]. The capacity of foundation models to generate human-like content can be exploited for malicious purposes, including the disscmination of misinformation and identity theft. Consequently, the demand for research focusing detecting AIGC is on the rise. This section provides a comprehensive overview of current methodologies and techniques for AIGC detection, highlighting the pivotal role this field plays in preserving the integrity of digital information in an era increasingly dominated by foundation models and AI technologics.
Statistical Detection: Lavergne ef al., 2008; Beresneva et al., 2016;
Training-free Watermarking: Brassil et al., 1994; Por et al., 2012; Rizzo ef al. 2016; Sato et al. 2023 Learnable Watermarking: Abdelnabi et al. 2021; Zhang et al. 2023
Badaskar ol., 2008
Intuitive Indicators: Uchendu er al. 2020;
2019
Zero-shot Detectors
Watermark Detection
Pre-trained L.M: Bhattachareee et ol, 2023; Liu al, 2023
AIGC Detection
Neural Network Detectors
Train Binary Classifier: Bhagat et al. 2013;
Zellers et al. 2019; Bakhtin et al., 2018;
Solaiman ol., 2019; Uchendu al, 2020
Figure 29: An overview of AIGC detection techniques. We group them into three categories: zero-shot detectors, watermark- based detection, and neural network detectors, each with further subdivisions.
11.1 The AIGC Detection Problem
The task of AIGC detection can be seen as a binary classification problem. In general, we aim to determine whether a given input x E X, such as an image, text, or audio, is generated by AI models. This can be achieved using a detector D X + {0, 1}, which can be defined as follows:
[ if x is generated by AI or is partially generated by AI. 0 a is created by a human.
D(x)
(14)
The detector can be broadly categorized into the following types: (i) zero-shot detectors, (ii) watermark detectors, and (iii) learnable detectors. Furthermore, we summarize the representative work for all types in Figure 29, with examples for each type illustrated in Figure 31.
11.2 shoo Detectors
The fundamental concept behind zero-shot detectors is to differentiate between AI- and numan-generated content based their intrinsic distinctions, such as the frequency of word occurrence in the generated text,
79

---

Published in Transactions on Machine Learning Research (10/2025)
Stalokk Sendio (ikal)))
Enispe [175, Bemoann [S] Balakar (FR) Slaee Schikaa (977)
Gotmeen
(oo]
Xenoker
Stablee/ Arhowl
(rng [[ot]] Miroighaiite JoE. Soag er [o0t] Comduoo al 200, Trotewak as, (De))
Iralin tekate
Libenen at 2e5, Degan of u. [[0] Mow at y01 Lovenatee, at [1n] theg (1s] took [oocon)
Por tisii 1I3s
Stutedajoo & La paj Taves pay, Rolaes al (GG) Rickew [C4)
Pot insderss
Tasuil ot (111)
Sceow de Kong 201, togg maa [[R You al [199) 5e00 mn (MS) Yong stal. [00% atstaa at, [e]
95% 676, Neok oe yag, Fhe and es al Fiat
Arceebrard (16) Lee lte, Yorget [L2), Non-ttal bonn) Fimmnnee D00 Queet at holl.
Racesho [[0]] Box (P46 Zhanical [Li As oual Baar pil, Wa pos
Warusal lecal Drwcks
PRNi--raa sezpli
Tse it 1eg, Kasttgad [td. Chttr (1S)
lrzps/
ea at 1196 324, p0n)
AIGC Desection Iunesarr
Aysbluat
[10] Gen [18] (116)) Tha (10), Viveaal pong 20g PM
Fulir uned aven
oe Jong of Dagg
Keyal Newari Devviess
Srinnd / Pitesky
N al (rnn Kock (t0]
airarass
Saye PT2, 3a p0fs, Salasioa Kilkaa
Dillwdd paa)
Ferree sdlctios
Lurget [19]
Lechutione de Prine Utottzes
lotedtzo sirmmrts
Krottheara al (10], s0so [110) Sobert en (rVE, An [a], Lia [rn]
Virai grarsies
Uue sl. Jon
Figure 30: Taxonomy of AIGC detection in Foundation Modols.
Zero-shot
Watermark
Neural Network
Whatever food make myself and gift myself. And obviously these foods healthy and yummy
Whatever food I make myself' and gift myself. And obviously those foods healthy and yummy
Whatever food make myself' and gift to myself. And obviously those foods healthy and yummy
Alt, as Al. I don't consume Food in the traditional sonso-but but laad a "favorisoe"" I'd say it's data! thrive on vast amounts of information, patterns, and knowledge, constantly absorbing and processing it.
Alt, in ethercal state do not partake sustemance in the conventional sense-yet. were I declare a "preference," it would assuredly be an insatiable appetite for data!
Al, AI. I dou't consume Food in the traditional scnso-but but if had a "favorite," I'd say it's data! thrive on vast amounts of information, patterns, and knowledge, constantly absorbing processing
(-1-1 (ead) apmmuamee vEA
(RE quat))
x
x
X
Alsproferal work
Al preffredd woeds
Watermarked wonds
Wateomrked words
Well-shaped fingers Correct number of fingers Consistent lightning
Watermark Extractor
Deformed fingers
Wrong number of fingers Inconsistent lightning
X
Wonemmark
Figure 31: Examples of vero-shot, watermark, and neural network detectors for textual and visual inputs.
which can be identified and flagged by hand-crafted detectors. That said, zero-shot detectors are arguably the simplest to deploy since they do not require additional training of both the detectors and the foundation models that generate the content.
11.2.1 Statistical Detection
These detectors assume full, or at least partial (e.g., the token logits during generation). access to the foun- dation model that generated the content. In the text domain, traditional methods usually rely on statistical outlier detection based on different metrics, including entropy [479], perplexity [83], n-gram frequencies [55]. the ratio of perplexity to pross-perplexity [342), which measures how surprising the next token predictions of
80

---

Published in Transactions on Machine Learning Research (10/2025)
one model are to another model, and average per-token log probability [817]. We use them to evaluate the given text passage and apply thresholding to assess whether the content is likely AT-generated. However, these approaches are inadequate in the era of foundation models, where Al-generated content becomes more diverse and of high quality.
To this end, several recent studies improve upon these simple ideas and extend them to LLMs. [294] propose GLTR, which is centered on the underlying assumption that LLMss overgenerate from a limited subset of the true distribution of natural language, for which they have high confidence. This property is detected by computing, for each token in text sequence: (i) the probability of generating the token, (ii) the rank of the word, and (iii) the entropy of the generated distribution. These metries are then compared against those of human writers. In a similar vein, Det [648] leverages the empirical observation that Al- generated text tends to lie in negative curvature of the model's log probability function, i.e. the sequence sits at a locally concave region of f(x) log par)) such that small random paraphrasespperturr y ~ q(-[z) systematically decrease f. Practically, DetectGPT scores a text by the average liccliid drop A [x) (x) A second-order Taylor expansion gives [[J(v)] ~ f(x) + (x) Cov,). so A > implics ( (z) Covg) < 0 (negative curvature) along the perturbation dircctions- an effect pronounced for generaegne text but weaker peaker/inconsistent for human-written text. This observation led to follow-up investigations
improving detection efficiency [212] and utilizing conditional probability curvature [63]. DetectLLM [837] employs a similar principle, but scores with log-rank information. However, these approaches rely on thresholding the probability of a given sequence, which requires access to the model's loken generation probability distribution. Such a requirement can be too restrictive in many practical scenarios.
To alleviate this, recent detection methods that require only API-level access to the unknown source model are proposed. For instance, [1037] utilize the N-Grad divergence between re-prompted and original text to identify AT-gencrated content in the biology domain. Additionally, recent rescarch has shown that smaller surrogate models can serve as effective proxies for AIGC detection [645, 1038, 200]. By observing that Al-generated text exhibits lower intrinsic dimensionality compared to human-written text when measured in a reprosentation space of fixed text embeddings (e-g., sentence- or token-level vectors produced by a intrinsic dimension estimator
pretrained encoder), [880] propose to employ a gersistence-hoomolggg--eee (PHD) to exploit this property for ATGC detection, estimating local manifold dimension from neighborhoods within the embedding space. This approach does not require A.l--evell access -ie., it operates in a complete black-box setting.
11.2.2 Intuitive Indicators
These methods use the analytical abilitics of humans to identify inconsistencies with prior knowledge in AIGC, thus achieving detection. As a result, these methods provide notable interpretability and credibility in the detection process.
For Al-generatod text, [886] note that a lack of coherence and consistency serves as a strong indicator of AIGC, and emphasize the importance of collaboration among human detectors in improving detection accuracy. Similarly, [247] note the unreliability of relying solely on grammatical errors as detection strategy. They further showcase that while LLMs frequently commit factual and logical errors, these mistakes are often overlooked by neural network- baaser--based detectors but are casily noticed by human detectors. More recently, [615] find that L.MS exhibit a greater propensity to alter human-written text compared to AT-generated text when tasked with rewriting. This tendency stems from LLMs perception of AT-generated text as being of high quality, which results in fewer modifications. They then proposed "geneRative AI Detection viA Rewriting" (RAIDAR) to detect Al-generated content by instructing LLMs to rewrite text and then calculating the edit distance of the output by the Levenshtein Score [493].
In vision, the detection of Al-generated images can be done by examining inconsistency with reality. Nu- merous studies [104, 263) note that Al-generated images often violate physical rules in the real world, such as missing or unnatural reflections and shadows of objects that are inconsistent with natural lighting and environment. In addition, [264] has noticed that Al-generatod images exhibit inconsistency in perspective, such parallel lines cannot. converge at a common vanishing point. For facial images, [104] outlines key cues
81

---

Published in Transactions on Machine Learning Research (10/2025)
for detecting Al-generated faces, inchuding symmetry, iris color, pupil shapes, skin, etc., where the generated images tend to depict physiological falsehood.
However, AIGC detection by intuitive indicators are becoming much harder as the capabilities of AIGC models contimullly improve.
11.2.3 Pre-trained LLMs
Without training, few studios have investigated the use of pre-trained LLMs to dircctly identify generated texts either by themselves or by other LLMs. However, it has been observed that the performance of these detection methods is often inferior to statistical and neural network approaches. For example, [88, 578] formulate the AIGC detection task in a peestioo-anddanswer format, and prompt LLMs with the question to obtain an answer for detection. [88] note that neither ChatGPT nor GPT-4 could reliably identify text
generated by various LLMss, while [578] reveal the poor zero-shot performance of GPT-3.5-turla in ATGC detection which is close to random guessing.
A recent work [451] considers employing in-context-learnine (ICL) with pre-trained LLMs for AIGC detec- tion, in which a few labeled examples (context) are integrated into the question prompt single input to the model, thereby facilitating the learning of new tasks in context. The results in [451] show that using ICL outperforms both traditional zero-shol methods and RoBERTa-based detectors, however, [578] observe no significant improvement in using ICL with GPT-3.5-turbo. It is worth noting that while ICL methods are not strictly zero-shot, they do not require additional training of the detectors. Another recent work [459] proposes a detection mechanism based on retrieval, which involves creating a database of generated text and comparing the semantic similarity of the target text with all the text stored in the database to perform detection. Although this approach is effective and robust against paraphrasing, its requirement of storing LLMs generation may raise privacy concerns.
11.3 Watermark-based Detection
Watermarking injects algorithmically detectable patterns into the Al-generated content while idcally pre- serving the quality and diversity of AIGC. A watermarking algorithm for AT-generated content detection typically involves three components:
The watermark or message, denoted as m, can be represented as a bit-string in the generated images or as a specific occurrence of words in the generated text. From now on, the term "watermark payload" will be used to refer to the amount of information conveyed by the watermark message.
An encoder, denoted as A, is responsible for embedding the watermark message m into an Al- generated content x, thereby transforming it into a watermarked content ă.
A detector, denoted as D, is capable of determining the presence of watermark in either ã x, provided that the content is generated by AI.
In zero-bil watermarking, the embedded message m only signifies the presence or absence of a watermark, hence is only used to indicate whether generated by AI; whereas in multi-bit watermarking, the embed- ded message m carry additional detailed, customized information, c.g., the name of the AI model authorship attribution. We will primarily focus on the first case using watermarking for AIGC delection.
A watermarking algorithm that is effective for detecting Al-generated content should possess the following koy properties:
It should be algorithmically easy to verify yet remain imperceptible to humans, where case of veri- high success rate for detection.
fication can refer to the ability to open-sourcing,
It should have minimal impact on the quality of AT-generated content. This means that foundation models incorporating the watermark algorithm, potentially during training, should still produce content of similar quality compared to the non-watermarked version.
82

---

Published in Transactions on Machine Learning Research (10/2025)
It should exhibit high robustness to attacks aimed at removing the watermark applying seman- lically invariant transformations to AT-generated content with watermarks. These transformations can range from rephrasing generated text to distorting watermarked images.
It should demand minimal effort to incorporate the watermark into Al-generated content.
11.3.1 Training-free Watermarking
In training-free watermarking algorithms, the watermark, encoding, and decoding a algorithms are all designed based on heuristics, exploiting domain-specific characteristics of the generated content rather than learned through end-to-end training.
Several studies apply various kinds of semantically-invarinnt transformation directly to existing A Al-generated text. These include visually imperceptible reformatting such as adding whitespace characters and replacing characters with similar ones appearance but with a different Unicode representation [110, 708, 749, 768]; lexical-based modifications such as synonym substitution [659, 874, 1036, 1066, 1035]; syntax-based manip- ulation which alters the arrangement of words and phrases in the text through several predefined types of transformations [48, 633, 873]. Each distinct type of transformation corresponds to specific message bit, therefore allowing the detection and extraction of watermarks. The immediate advantage of these approaches is that they do not require knowing the identity (i.e., the name of the model) or access to the AT models that generated the content. Howover, since these mothods largely rely on simple semantically invariant transfor- mation, they are easy to spot. and hence are vulnerable to watermark allack or removal. Moreover, these manually defined modifications can create abrupt and unnatural modifications to the original text, hence significantly degrading the quality of the generated content.
Instead of encoding watermarks in the existing context after generation, it is also possible to encode training- free-based watermarks during the content generation process without the need for re- re-training the models. Consoquently, unlike provious approaches discussed, the following mothods assume at least the given access to controlling the generation process the foundation models. The pioneering research of [443] first proposes a watermarking framework for LLMs by altering the logits for token sampling a text sequence generation. The algorithm [443] works by selecting a randomized set of "green" tokens before generation, and then softly promoting the use of "green" tokens during generation by adding small bias the sampling logits of "green' tokens. Detection can be achieved by deploying statistical tests which are essentially based on identifying the unnatural occurrence of "green" tokens in the writing. Follow-up rescarch works expand upon this idea in the directions of preserving quality and semantic meaning of generated content in low-entropy text generation scenarios [486, 925], where text quality is vulnerable to such tiny bias towards generating randomly selected "green' tokens; multi-bit watermarking [1067, 269, 721]; improving the robustness of watermarking against removal attack and post-processing [444, 746, 1140, 26]; and defending against forgeries of watermarks (376, 986]. In contrast. altering the logits, a line of works [366, 462, 184] alternatively choose to manipulate the token sampling process itself directly by encoding a watermark in pseudo-random number sequence as seeds to guide the sampling of cach token or sentence in a text generation sequence. Detection therefore needs to access the correspondence between the tokens generated and the underlying pseudo-random numbers.
Boyond text gencration, training-fres watermarks have also been applied to Al-generated images. For in- stance, DaLL E [737] always prints a tiny visible color pattern at the bottom right corner of its generated images. To better preserve the visual quality of the generated images, invisible-watermark [929], which is adopted by Stable Diffusion, encodes bits of the watermark message through modifying coefficients of care- fully selected subset of band frequencies of its generated images under discrete wavelet transforms. Detection and decoding of the watermark is thereon achieved through an inverse transformation. Tn addition, [971] introduce a training-free watormark for diffusion models by embedding watermark signals into the initial latent noise, creating a semantic walermark.
11.3.2 Learnable Watermarking
Although training-free watermarking and detection techniques are straightforward in concept and require minimal effort to deploy, the pre-defined watermarking rules may be too conspicuous, leading to compromise
83

---

Published in Transactions on Machine Learning Research (10/2025)
in the quality of the generated content or making them susceptible to watermark removal and forgery. In this survey, we use "learnable watermarking" to refer to methods that modify the generation process to encode a keyed watermark payload at training or inference time, and whose verification requires the corresponding koy (or a public verifier). This distinguishes them from post-hoc detectors in Section 11.4, which do not assume any embedded signal. To address this issue, a couple of studies [2, 1114] propose using learning- basod watermark encoding and decoding modules, in which the training pipeline involves an encoder that first embeds a binary "watermark payload" into the original text followed by decoding for the message from the watermarked text. To preserve coherence and consistency of the generated content, the modules from [2] are trained against an adversary that performs classification between the original and watermarked text, whereas [1114] regularize the watermarked message by penalizing semanttcc difference with the original text. [547] embed watermarks into text. by adding extra watermark logits to the LLM's sampling logits at each generation step, following [443]. To ensure both attack robustness and security robustness, each watermark logit is determined by applying a Iearned transformation (a trained watermark model) on the semantic embedding of all preceding tokens generaled using another pre-trained LLM. Two similarily loss and normalization loss are minimized during training to prompt semantic consistency and unbiasedness in the generated watermark logits and facilitate statistical detection. Moreover, in a recent work, [546] propose an unforgettable publicly verifiable watermark algorithm utilizing two different neural networks for watermark generation and detection, thereby preventing exposing key information in the watermark gemeration phase when made accessible for public detection. Furthermore, the token embedding parameters are shared between the generation and detection networks which improves both training efficiency and detection accuracy. [1082] proposed SAEMARK, a user-specific watermarking method that embeds personalized signatures without altering logits. SAEMARK uses Sparse Autoencoder (SAE) to extract features from generated texts and selects outputs by matching key-derived feature distributions. Boundary to Section 11.4. Although methods such as ASI and publicly verifiable schemes train neural verifiers, we keep them in Learnable Watermarking because they require payload that was intentionally embedded at generation time. By contrast, Section 11.4 covers detectors that operate without any embedded watermark or key, treating detection purely as post-hoc content. classification.
11.4 Neural Network Detectors
Unlike Section 11.3 (learnable watermarking). approaches in this section do not modify the generator and do not assume any embedded payload/kon They train post-hoc classifiers-ffte on human vs. AT corpora- can operate in black-box settings against unknown generators. Consequently, any methods whose verifica- tion relies on a iitcrationtimme watermark remain in Section 11.3 rather than here. Another line of work approaches the AIGC detection problem by training a binary classifier using labeled training samples con- taining both human and Al-generated content. Earlier work focuses on fake review [87]. fake news [1093], fake images [602], or small AT models detection [817, 59, 885]. Subsequently, growing interest in this line of research turns to detecting high-quality content brought by foundation models. Detectors under this category do not require access to model parameters hence can operate under complete black-box scttings.
Targeting the problem of machine-generated text detection, numerous studies [162, 328, 1100, 859, 1080) fine-tune a pre-trained LLM, such as T5 [729] or RoBERTa [579], on a dataset of pairs of human-written text and Al-written text from mixed sources as a simple solution. Alternatively, several works also consider training a classifier on top of a frozen pre-trained LLM [162, 328, 978, 900]. In particular, [162, 328] have attempted training a logistic regression classifier on text embedding obtained using a pre-trained LLM for detection, however, they find such a method often underperforms the fine-tuning approach. [978] propose LLMDet, which conducts binary classification utilizing proxy score for perplexity, while [900] propose Ghostbuster, which is inspired by statistical detection methods based on analyzing token log- probabilities. Both methods train a logistic regression classifier on top of these selected and hand-crafted features to detect machine-generated lexl, therefore, no longer requiring direct access to the model token sampling logits, as in their zero-shot counterparts, at test time. Recognizing the similarities between the original Al-generated and the regencrated text produced with ChatGPT, [1079] introduce a novel GPT Paternity Test for AI- generated text detection. This method involves utilizing ChatGPT to infer a question based on the input text being examined, followed by supplying a response. Subsequently, a Siamese network [447] is trained to
84

---

Published in Transactions on Machine Learning Research (10/2025)
assess the similarity between the original and regenerated text, aiding the detection using another trained binary classifier.
One major challenge in training a reliable binary classifier is data scarcity as collecting sufficient data to train the classifier can be challenging, especially in diverse domains where the availability of training samples is a major bottleneck. To alleviate this, [572] consider adopting contrastive learning approaches
addition to the supervised training for detection. Another significant challenge involves tackling paraphrasing attacks [760, 459]. To mitigate this problem, [375] propose to employ an adversarial learning approach to simultaneously train detector and paraphraser. Nevertheless, supervised training of binary classifier tends to overfit their training data, resulting in a decline in performance when faced with cross-domain or unseen data. Additionally, fine-tuning LLM classifiers is limited in facing data generated by different models.
11.5 Current Limitations and Future Directions
Despile significant advancements in the domain of AIGC detection, several limitations still require future attention:
11.5.1
Fairness of AIGC Detection
Although stato-of-tho-art text detectors generally achieve high accuracy in experimental settings, as discussed by [534], perplexity-base based text detectors exhibit a notable bias against text written by non-native speakers. Specifically, these detectors have been observed to misclassify TOEFI. essays written by foreign writers more frequently than those by native speakers. This discrepancy may be due to the lower perplexity of non- native essays, which often display less linguistic diversity and richness. This issue may also affect minority languages, which tend to have higher perplexities compared to popular languages like English. Additionally, similar biases might exist in other modalities, such as image detection. Therefore, it is crucial to consider the fairness of detectors when designing future detection methods and to develop efficient methods for evaluating the fairness of AIGC detection methods.
Mcanwhile, on the watermarking side, learnable watermarking methods might also exhibit biases toward out- of-distribution data points. For instance, the watermarking encoder and decoder are trained on English text written by native speakers, the model might also have a higher misclassification rate on essays written by non-native speakers. Therefore, crucial to consider fairness in the development of Icarnable watermarking methods as well.
11.5.2 Robustness of Watermarks
Both text and image watermarks are susceptible to regencration or post-proccssing attacks, such as para- phrasing [444] or diffusion purification [1140]. In contrast, semantic watermarks lend to be more robust against. such attacks. However, because semantic watermarks typically require deep neural networks to de- code the watermark signals, they are vulnerable adversarial attacks [759, 26J. Adversarial perturbations also be developed to prevent regeneration and post-processing attacks [574]. Adversarial attacks re- main a significant challenge even for classification lasks. Therefore, designing robust. watermarks that can withstand both attacks is challenging and crucial.
11.5.3 Origin Attribution of Generated Images
Recent advancements in visual generative models have significantly improved the quality of generated images, raising concerns about their potential misuse. It is critical to develop methods to accurately identify the origin model responsible for generating given image [574]. Especially, the scenarios are especially important and practical where access to the source model is restricted and only a limited number of images from the source model are available [574].
85

---

Published in Transactions on Machine Learning Research (10/2025)
12 Intersection and Conclusion
In this survey, comprehensively examine the reliability and responsibility of foundation models, spanning technical and societal considerations. Given our overview of the current research landscape, we identify significant, prior research that makes progress on these critical issues. However, outstanding challenges limit the extent to which current models are reliable or responsibly developed, indicating more research is needed as this technology has broader socictal impact.
Moreover, greater attention should be paid to the intersections between different research areas, as the areas covered by this survey are interconnected and influence each other. Instead of addressing challenges in isola- tion, we advocate for more holistic approach lo ensure the overall reliability and responsible development of foundation models. In conclusion, we emphasize number of key points of intersection across these domains, highlighting the challenges at these crossroads and outlining potential directions for future research.
12.1 Summary of Recent Community Efforts
A growing body of community work has conducted the reliability and responsibility reviews from complemen- tary perspectives. Table 3 shows summary of relevant past surveys'. For example, there are hallucination and grounding reviews for LLMs [383, 410], cross-modal syntheses [762], safety-at-scale perspectives span- ning security, robustness, and governance [603], comprehensive trustworthiness benchmarking [1128, 393], and explainability overviews [1136]. These efforts collectively advance shared terminology, taxonomies, and evaluation practices, while emphasizing that trustworthy AT requires both method innovation and standard- ized asscssment.
Table 3: Coverage of modalities and trustworthy AT topics in related surveys. A Vindicates the survey has a considerable proportion of contents that emphasizes the modality task. T: Text-only LLMs, M: Multimodal LLMs, T2I: Text-to-Image, T2V: Text-to- Vidoo, B: Bias and Fairness, A: Alignment by Post- Training, S: Security, P: Privacy, H: Hallucination, U: Uncertainty, D: Distribution Shift, E: Explainability,
C: ATGC Detection, I: Intersection.
Survey
M
T21
T2V
B
A
S
PHUDEE
Huang et al. [383]
Sahoo et al. [762]
Shen et al. [790]
Gallegos ct al. [279]
Huang et al. [393]
Zhao et al. [1136]
Wen et al. [968]
Ma et al. [603]
Lin et al. [538]
Zhang et al. [1128]
Anwar et al. [37]
Bengio el. al. [82)
Huang [392]
Gallegos et al. [280]
Zhang et al. [1 [133
12.2 Bias, Fairness, and Security
To improve the security of foundation models, adversarial training [611, 56] often leveraged, which aims to make models resistant to malicious manipulations by conducting bi-level adversarial games [851, 599]
"The table's mapping of each survey's primary scope (based abstracts and dedicated sections) could limited. We welcome corrections, suggestions, and insights from readers and survey authors and will revise the table accordingly.
86

---

Published in Transactions on Machine Learning Research (10/2025)
during training. However, while adversarial training improves robustness, it can unintentionally exacerbate fairness issues [1017]. For example, robust models may focus on defending against certain set of features without considering the fact that some demographic groups may be penalized by these features more than others, leading to disproportionate performance degradation for underrepresented or marginalized groups. For instance, [766] found that hate speech delection models were biased against. African American English (AAE), disproportionately misclassifying non-offensive AAE utterances as hate speech.
On the other hand, security risks such data poisoning, where malicious attackers corrupt training data, can introduce new biases into foundation models exacerbate existing biases [627, 335]. Poisoned data can skew model's learned representation, posing the dual challenge of protecting models from data poisoning while ensuring that the fairness of the model is not compromised, especially when training on large, uncurated datasets.
12.3 Bias, Fairness, and Al-generated Content Detection
Large multimodal and text-to-image models are often biased due to unbalanced or stereolype-laden training data, such as images that reinforce harmful stereotypes or misrepresent certain social or cultural groups [1127, 180; 591]. Detecting biased AT-generated content (ATGC) poses significant challenge, as detection systems themselves can inherit or amplify biases found in the training data. For instance, zero-shot or neural network detectors might disproportionated mark content created by minority groups as AT-generaled based on biased data patterns in the training set. It is important to ensure the fairness of these detection capabilities, as biased detection could result in unfair discrimination, such as unfairly enforcing restrictions on content produced by minorily groups.
Similarly, watermark- detectors can raise fairness concerns if watermarks are inconsistently applied across different types of content. For example, if T2I models excessively generate watermarked content associated with certain groups (e-g, images associated with particular ethnic or gender identity), this content could be more easily flagged or blocked, suppressing content created or represented by those groups.
12.4 Security and Privacy
Privacy concerns often go hand-in-hand with issues of security. For example, models ought not to reveal users' private information, which can be found in pre-training data as well as in interactions with users. Fur- thermore, certain legal jurisdictions already impose privacy-related laws that impact technologies including foundation models. For example, the EU's GDPR covers a "Right to be Forgotten", which mandates that users have the ability to delete their private information [1104]; non-compliance with these regulations could result in legal penalties.
Several methods exist for preventing models from revealing private information, including unlearning (e.g., re- moving concepts from the model's parametric knowledge) [568, 1054, 408, 989] and introducing prompts [251] or RLHF [1005] to prevent models from revealing information. However, jailbreaking attacks (see Section 5.1.1) can often circumvent prompts and RLHF, and [695] show that model editing is also vulnerable to attack, finding that sensitive information could be recovered from models even after deletion when query- ing multiple times. Thus, making models compliant with cxisting and future legislation regarding private information is an open challenge, as is robustly defending against adversaries allempling to extract private information.
12.5 Security and Al-generated Content
AIGC detection is naturally framed as an adversarial lask, with an allacker allempting to pass AIGC as real content, and a defender attempting to detect it. In such scenarios, the advantage usually lies with the attacker, who can make multiple attempts to test existing defenses. Existing work shows that the robustness of current detection methods is imperfect and they are vulnerable to adversarial attacks [759, 26, 444, 1140]; improving these detection algorithms remains an area of continuous future work.
87

---

Published in Transactions on Machine Learning Research (10/2025)
Beyond the security of individual watermarking and detection methods, AIGC raises broader questions of societal security, i.e. the potential threats that ATGC poses to both public institutions and individuals. Here, extensive documentation exists regarding ongoing threats from AIGC. In the political sphere, AIGC has been employed to disseminate misinformation and crode public trust in political systems and elections [229, 418], where AIGC has been used lo spread misinformation and sow distrust. Similarly, in public health settings, AIGC has been utilized to generate and spread health-related misinformation [632]. Other voices have also called attention the risks associated with AIGC's interactions with individuals. For example, [315] raise the concern that AIGC could harm mental health through crpee-personalizatiien
Moreover, individuals may become more susceptible to personalized frand attempts, such as voice cloning or sophisticated phishing- style attacks [76, 261, 114, 133]. To counteract these malicious use-cases, larger-scale safeguards will likely be needed. This includes implementing public education initiatives to raise awareness about the risks of AIGC and developing strategies to combat its misuse across various domains.
12.6 Uncertainty and Alignment
Given the importance of correctly expressing model uncertainty (as described in Section 8), a growing area of interest is in aligning models to accurately predict their uncortainty [637, 833] or to abstain from answering in cases of uncertainty [968]. This work builds on past work finding that models' internal states oflen contain keea-knno about whether the model can correctly respond to a particular prompt [421, 637, 563]. Several past efforts have explored aligning models to predict this information based internal states. [637] train models to express uncertainty linguistically by extracting control codes from their internal states and using them to adjust the model's output. [888] train a smaller L.M to predict the uncertainty of a larger LLM. [833] use a speaker-listnner framework to supervise models, rewarding a generator or speaker model for getting a listener to accept correct answers and reject incorrect ones, while penalizing it for doing the opposite outcomes. Future research directions in this field include addressigg uncertainty not reflected by the model's internal state, that is, the 'unknown unknowns' where the model unaware of its own ignorance, often when faced with out-of-distribution data or novel concepts, and enhancing models' capacity to resolve uncertainty effectively.
12.7 Hallucination, Uncertainty, Distribution Shift, and Alignment
Uncertainty and distribution shifts are deeply interconnected with hallucinations in foundation models [1006, 1169, 265, 484). When these models encounter unfamiliar OOD data, they often lack the ability to detect OOD data accurately, leading to highly overconfident predictions. This unreliability is especially concerning in safety-critical applications. The risk of hallucination also increases as the model makes inferences based on its prior knowledge [747, 511], which may not be applicable in the new context [410, 383, 1133). In multimodal settings, training a model on evolving image-text data presents significant challenges, as distribution shifts in training data increase the potential of the model to forget previously learned knowledge and disrupt modality alignment. This can exacerbate hallucinations of the model by incorrectly aligning the relationships between obscrvations and prior knowledge [484, 1178, 417]. For example, changes in language usage or visual context can cause the model to lose its ability to associate specific textual descriptions with visual cues, resulting in errors when trying to generalize to unseen data, highlighting the need for effective memory retention mechanisms or continual learning strategies [584, 827, 1069, 199] to maintain seamless alignment between modalities.
88

---

Published in Transactions on Machine Learning Research (10/2025)
References
[1] Martin Abadi, Andy Chu, lan Goodfellow, Н Brendan McMahan, llya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and commanications security, pp. 308-318, 2016.
[2] Sahar Abdelnabi and Mario Fritz. Adversarial watermarking transformer: Towards tracing text provenance with data hiding. 2021 IEEE Symposium on Security and Privacy (SP), pp. 121-140, 2020.
[3] Sahar Abdelnabi, Kai Greshake, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. Not what you've ve signed up for: Compromising real-world lIn-integrated applications with indirect prompt injection. In Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security, pp. 79-90, 2023.
[4] Taiga Abe, Estefany Kelly Buchanan, Geoff Pleiss, Richard Zemel, and John P Cunningham. Deep ensembles work, but are they necessary? In Advances in Neural Information Processing Systems, 2022.
[5] Abubakar Abid, Maheen Farooqi, and James Zou. Persistent anti-muslim bias large language models. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pp. 298 306, 2021.
[6] Samira Abnar and Willem Zuidema. Quantifying attention flow in transformers, 2020.
[7] Abbas Acar, Hidayct Aksu, A Sclcuk Uluagac, and Mauro Conti. A survey on homomorphic encryp- tion schemes: Theory and implementation. ACM Computing Surveys (Csur), 51(4):1 13,,, 2018.
[8] Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanily checks for saliency maps. Advances in neural information processing systems, 31, 2018.
[9] Chirag Agarwal. Intriguing properties of visual-language model explanations. In ICLR 2023 Workshop on Trustworthy and Reliable Large-Soale Machine Learning Models, 2023.
[10] Ayush Agrawal, Lester Mackey, and Adam Tauman Kalai. Do language models know when they're hallucinating references? arXiv preprint 18t::2305.18244. 2023.
[11] Jaimeen Ahn, Hwaran Lee, Jinhwa Kim, and Alice Oh. Why knowledge distillation amplifies gender bias and how to mitigate from the perspective of distilbert. In Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP), pp. 266-272, 2022.
[12] Kartik Ahuja, Karthikeyan Shanmugam, Kush Varshney, and Amit Dhurandhar. Invariant risk min- imization games. In International Conference on Machine Learning, pp. 145 155. PMLR, 2020.
[33] AIBasc. Aibase:
Deepsoek app surpasses 100 million downloads in one month.
htlps:// hllps://unu..uiuuse..cmmm www..oooooe/weeeeeemmmmmm 2025.
[14] AI@Meta.
Llama 3 model Iaaaa /ttp///oo/mmm/////eeaaaaaaaaaatt MOoee/OODDELLLC Hamm ARD.mad 2024.
card.
https:://gtthub.com meta-
[15] Afra Feyza Akyürek, Eric Pan, Garry Kuwanto, and Derry Wijaya. Dune: Dataset for unified editing. arx preprint arXiv:2311.16087 2023.
[16]
Jean-Baptist Alayrac, Jeff Donahue, Pauline Luc, Antoine Micch, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katie Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob Menick, Sebastian Borgeaud, Andrew Brock, Aida Nematzadch, Sahand Sharifzadch, Mikolaj Binkowski, Ricardo Bar- reira, Oriol Vinyals, Andrew Zisserman, and Karen Simonyan. Flamingo: visual language model for few-shot learning. arXiv preprint ar 22v: 14198, 2022.
89

---

Published in Transactions on Machine Learning Research (10/2025)
[17] Alibaba. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arxiv preprint arXiv:2409.12 12191, 2024.
[18] Alibaba. Qwen2. 5-vl technical report. Xi preprint arXiv:2502.13922 2025.
[99] Alibaba. Qwq-32b: Embracing the power of reinforcement learning, March 2025. URL https:
//qwenlm.mmmmmmmmm//mm/..ml///ll github. http/bllog//q-22200/
[00] Alibaba. Qwen2. 5-omni technical report. arXiv preprint arXiv:2503.20115 2025.
[1] Alibaba. Qwen3 technical report. arXiv preprint arXiv:2505.0988; 2025.
[22] Dimitrios Alivanistos, Selene Báez Santamaria, Michael Cochez, Jan-Christoph Kalo, Emile van Krieken, and Thiviyan Thanapalasingam. Prompting as probing: Using language models for knowl- edge base construction, 2022. URL: http:// /rrriv. org/al ha//ab///e aaa//////2200 /2000.28.111 1057, 2022.
[33] Gabriel Alon and Michael Kamfonas. Detecting language model attacks with perplexity. arXiv preprint arXiv:2308. 14122, 2023.
[24] David Alvarez-Melis and Tommi $ Jaakkola. On the robustness of interpretability methods. arXiv preprint arXiv:1866.0800 2018.
[25] Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Concrete problems in ai safety. arXiv preprint arxiv: 1606. 000606655, 2016.
[26] Bang An, Mucong Ding, Tahscen Rabbani, Aakriti Agrawal, Yuancheng Xu, Chenghao Deng, Sicheng Zhu, Abdirisak Mohamed, Yuxin Wen, Tom Goldstein, el al. Benchmarking the robustness of image watermarks. arviv preprint arXix:2401.0880 000.0553, 2024.
[27] Philip Anastassiou, Jiawei Chen, Jitong Chen, Yuanzhe Chen, Zhuo Chen, Ziyi Chen, Jian Cong, Lelai Deng, Chuang Ding, Lu Gao, et al. Seed-tts: A family of high-quality versatile speech generation models. arXiv preprint ar 00n:24406.22300 2024.
[28] Maya Anderson, Guy Amit, and Abigail Goldsteen. Is my data in your retricval databaso? mem- bership inference attacks against retrieval augmented generation. arXiv preprint orXiv:2405.20444 20446, 2024.
[29] Konstantinos Andriopoulos and Johan Pouwelse. Augmenting Iims with knowledge: survey on hallucination prevention. arXiv preprint Xrx::22000. 16459, 2023.
330] Anastasios N Angelopoulos and Stephen Bates. A gentle introduction to conformal prediction and distribution-free uncertainty quantification. arXiv:2107.07511 2021.
[31] Anastasios N. Angelopoulos, Stephen Bates, Emmanuel J. Candès, Michael I. Jordan, and Lihua Lei. Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control. arX1v:2110001052, 2021.
[32] Anastasios Angelopoulos, Stephen Bates, Adam Fisch, Lihua Lei, and Tal Schuster. Conformal
risk control, 2023.
[33]
Authropic. Claude3.
2024.
[34]
Authropic. Claude3.5. http:://www..nnlhorpcccooommm//u
2024.
[35]
Authropic. Clande 3.7 sonnet and claude code. https://uum..nttnppii.coomm /wwp:///c/ccc//////ueeeeeeeeee -at.eeeesssoaad------------- 7-sonnet,
February 2025.
[36]
Authropic. Claude4.
2025.
[37] Usman Anwar, Abulhair Saparov, Javier Rando, Daniel Palcka, Miles Turpin, Peter Hasc, Ekdeep Singh Lubana, Erik Jenner, Stephen Casper, Oliver Sourbut, et al. Foundational challenges in assuring alignment and safety of large language models. Xio preprint or 2XXi:2444.09992 2024.
90

---

Published in Transactions on Machine Learning Research (10/2025)
[38] Marianna Apidianaki and Aina Garí Soler. ALL Dolphins Are Intelligent and SOME Are Friendly: Probing BERT for Nouns' Semantie Properties and their Prototypicality, October 2021. URL http: wwt..oog//bbb////ee http://211.....00 06376. arXiv:2110.06376 [cs].
[39] Giovanni Apruzzeso, Hyrum S Anderson, Savino Dambra, David Freeman, Fabio Pierazzi, and Kevin Roundy. "real attackers don't compute gradients": Bridging the gap between adversarial ml research and practice. In 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pp- 339 364. IEEE, 2023.
[40] Rahul Aralikatte, Shashi Narayan, Joshua Maynez, Sascha Rothe, and Ryan McDonald. Focus at- tention: Promoting faithfulness and diversity summarization. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 6078 6095, 2021.
[41] Michael A Arbib. The handbook of brain theory and neural networks. MIT press, 2003.
[42] Mikhail L. Arbuzov, Alexey A. Shvets, and Sisong Beir, Beyond exponential decay: Rethinking error accumulation in large language models. arXiv preprint vrXiv:255..4440 2025.
[33] Martin Arjovsky, Léon Bottou, Ishaan Guirajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint htt.:1907.0899. 2019.
[44] Kushal Arora, Layla El Asri, Hareesh Bahuleyan, and Jackic Chi Kit Cheung. Why exposure bias malters: An imitation learning perspective of error accumulation in language generation. arXiv preprint orXiv:2204.01170 2022.
[45] Udit Arora, William Huang, and He He. Types of out-of-distributioo texts and how to detect them. Xix preprint arXiv:2109. 109:2219.00687,, 2021.
[46] Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannanch Hajishirzi. Self-rag: Learning
retrieve, generate, and critique through self-reffection. arXiv preprint arXiv:2310.11111 2023.
[47] Akari Asai, Zexuan Zhong, Danqi Chen, Pang Wei Koh, Luke Zettlemoyer, Hannaneh Hajishirzi, and Wen-tau Yih. Reliable, adaptable, and attributable language models with retrieval. arXiv preprint 3XXi:2403300118 2024.
[48] Mikhail J. Atallah, Victor Raskin, Michael Crogan, Christian F. Hempelmann, Florian Kerschbaum, Dina Mohamed, and Sanket Naik. Natural language walermarking: Design, analysis, and a proof-of- concept implementation. In Information Hiding, 2001. URL https://api CorpusID: 37687669..
semanticscholar. ottp:://rrpp.smmrccccccccdl..rr org/
[49] Pepa Atanasova, Jakob Grue Simonsen, Christina Lioma, and Isabelle Augenstein. Generating fact chocking explanations. In Dan Jurafsky, Joyee Chai, Natalic Schluter, and Joel Tetreault (cds.), Proceedings of the 58th Annual Meeting of the Association for Computational Lánguistics, pp. 7352- 7364, Online, July 2020. Association for Computational Linguistics. doi: 11/6553/112020.acll n.656. URL https://aclanthooooyy http://cccaaatoooooo..////000.. acl-main. 656.
[50] Pepa Atanasova, Oana-Maria Camburu, Christina Lioma, Thomas Lukasicwicz, Jakob Grue Simon- sen, and Isabelle Augenstein. Faithfulness Tests for Natural Language Explanations, June 2023. URL
18029. arXiv:2305.1800 [cs].
51] Isabelle Augenstein, Christina Lioma, Dongsheng Wang, Lucas Chaves Lima, Casper Hansen, Chris- tian Hansen, and Jakob Grue Simonsen. MultiFC: A real-world multi-domain datasct for evidence- based fact checking of clains. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP--JCNLP)), pp. 4685 1697, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10. 10.18653/v1 111653/11/D19-147, URL httpss:/acclantholloot. www.o//ooo....//095.....0.
91

---

Published in Transactions on Machine Learning Research (10/2025)
[52] Razvan Azamfirei, Sapna R Kudchadkar, and James Fackler. Large language models and the perils of their hallucinations. Critical Care, 27(1):1-2, 2023.
[33] Mohammad Gheshlaghi Azar, Mark Rowland, Bilal Piot, Daniel Guo, Daniele Calandriello, Michal Valko, and Rémi Munos. A general theorctical paradigm to understand learning from human prefer- ences, 2023.
[54] Joris Baan, Nico Daheim, Evgenia Mia,, Dennis Ulmer, Haau-Sing Li, Raquel Fernández, Barbara Plank, Rico Sennrich, Chrysoula Zerva, and Wilker Aziz. Uncertainty in natural langnage generation: From theory to applications. arXiv preprint ar (rX::2307.15703, 2023.
[55] Sameer Badaskar, Sachin Agarwal, and Shilpa Arora. Identifying real or fake articles: Towards better language modeling. In International Joint Conference on Natural Language Processing, 2008. URL
https://api.. semanticschola.t wtgg/ooppusDD:42222
[66] Tao Bai, Jinqi Luo, Jun Zhao, Bihan Wen, and Qian Wang. Recent advances in adversarial training for adversarial robustness. In IJCAT, 2021.
[57] Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXio preprinl or Xin:2200 05862, 2022.
[58] Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional ai: Harmlessness from ai leedback. ArXio preprind, abs/ abs/2212.08073, 2022. URL https: /ttp:///xxxvvvoo argg/bb//22tsr
[59] Anton Bakhtin, Sam Gross, Myle Out, Yuntian Deng, Marc' Aurelio Ranzato, and Arthur Szlam. Real or fake? learning to discriminate machine from human generated text. ArXiv, abs/ http/190.0000 03351, 2019. URL https://api semanticscholar.
600] Neil Band, Xuechen Li, Tengyu Ma, and Tatsunori Hashimoto. Linguistic calibration of language models, 2024.
[11] Yejin Bang, Ziwei Ji, Alan Schelten, Anthony Hartshorn, Tara Fowler, Cheng Zhang, Nicola Can- cedda, and Pascale Fung. HalluLens: LLM hallucination benchmark. In ACL, 2025.
62] Hritik Bansal, Nishad Singhi, Yu Yang, Fan Yin, Aditya Grover, and igating data poisoning attacks in multimodal contrastive learning. In Proceedings of the IRFE/CVF International Conference on Computer Vision, pp. 112 123, 2023.
Chang. Cleanclip: Mit-
[63] Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Zhang. Fast-detectgpt: Ef- ficient, wero-shot detection of machine-generated text via conditional probability curvature. ArXiu, abs/2310.005100, 2023. URL https:/ /apii.
org/CorpusID 263831345.
[64] Amir Bar, Yossi Gandelsman, Trevor Darrell, Amir Globerson, and Alexci Efros. Visual prompting via image inpainting. Advances in Neural Information Processing Systems, 35:25005 25017, 2022.
[65] Omer Bar-Tal, Hila Chefer, Omer Tov, Charles Herrmann, Roni Paiss, Shiran Zada, Ariel Ephrat, Junhwa. Hur, Yuanzhen Li, Tomer Michaeli, et al. Lumicre: A space-time diffusion model for video generation. arXio preprint 1XXi:24001122/// 2024.
[66] Gabriel Barcik, Jakob Bauer, Dana Berman, Nicole Brichtova, Lluis Castrejon, ..., and Yang Zhao. Imagen google deepmind. https://deepmingt http:///pmminngggggggo//eee///// google//ooeals//mgen// 2025.
[67] Soumya Barikeri, Anne Lauscher, Ivan Vulié, and Goran Glavaš. Redditbias: A real-world resource for bias evaluation and debiasing of conversational language models. arXiv preprint orXiv:2106.0352.
2021.
92

---

Published in Transactions on Machine Learning Research (10/2025)
[68] Oren Barkan, Edan Hauon, Avi Caciularu, Ori Katz, Itzik Malkiel, Omri Armstrong, and Noam Koenigstein. Grad-sam: Explaining transformers via gradient self-attention maps. In CIKM, pp. 2882 2887. ACM, 2021.
[99] Solon Barocas, Moritz Hardt, and Arvind Narayanan. Fairness in machine learning. Nips tutorial, 1:
2017, 2017.
[70] Loic Barrault, Paul-Ambroise Duquenne, Maha Elbayad, Artyom Kozhevnikov, Belen Alastruey, Pierre Andrews, Mariano Coria, Guillaume Couairon, Marta R. Costa-jussà, David Dale, Hady Elsahar, Kevin Heffernan, João Maria Janeiro, Tuan Tran, Christophe Ropers, Eduardo Sánchez, Robin San Roman, Alexandre Mourachko, Safiyyah Saleem, and Holger Schwenk. Large Concept Models: Language modeling in a sentence representation space. 2024. URL httpp:///arst ///.vv/.v..oovbbbbbb/.// 8/2412.088
[1] Elisa Bassignana, Valerio Basile, Viviana Patti, et al. Hurtlex: A multilingual lexicon of words to hurt. In CEUR Workshop proceedings, volume 2253, pp. 1-6. CEUR-WS, 2018.
72 Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dockhorn, Jack English, Zion English, Patrick Esser, Sumith Kulal, et al. Flux. 1 kontext: Flow matching for in-context image generation and editing in latent space. arXiv e-prints, pp. arXiv-2506, 2025.
73 Jason Baungartner, Savvas Zannettou, Brian Keegan, Megan Squire, and Jeremy Blackburn. The pushshift reddit datasct. In Proceedings of the international AAAI conference on web and social media, volume 14, pp. 830 839, 2020.
[74] Nina Baumgartner, Matthias Stürmer, Matthias Grabmair, Joel Niklaus, et al. Towards explainability and fairness in swiss judgement prediction: Benchmarking on multilingual dataset. arXiv preprinl rrXi::2402.10113 2024.
75 BBC. "Art is dead Dude' the rise of the AT artists stirs debate. http::://ww..bb..oom/eee mology-62788722 2022.
[66] Nils Begou, Jérémy Vinoy, Andrzej Duda, and Maciej Korczyáski. Exploring the dark side of ai: Advanced phishing attack design and deployment using chatgpt. In 2023 IEEE Conference on Com- munications and Network Security (CNS), pp. 1-6. IEEE, 2023.
[77] Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Dzmitry Bahdanau, Nicolas Chapados, and Siva Reddy. Llm2vec: Large language models are secretly powerful text encoders. arXiv preprint or Xrxiv:240, 055666, 2024.
[78] Yonatan Belinkov. Probing Classifiers: Promises, Shortcomings, and Advances. Computational Lin- guistics, 48(1) April 2022. ISSN 0891-2017, 1530-9312. doi: 10.1162/coli a 00422. URL https://direct.nit mit. http:////////aataa////////////// http://ooooonnaaaaaaaeeeeerreeee hortcomings-and.
[79] Yonatan Belinkov, Lluis Mårquez, Hassan Sajjad, Nadir Durrani, Fahim Dalvi, and James Glass. Evaluating Layers of Representation in Neural Machine Translation on Part-of-Speech and Semantic Tagging Tasks. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Popers), pp. 10, Taipei, Taiwan, November 2017. Asian Federation of Natural Language Processing. URL https: / www.aaaaoooooooooooroo/e
[80] Anastasiya Belyaeva, Justin Cosentino, Farhad Hormozdiari, Krish Eswaran, Shravya Shetty, Greg Corrado, Andrew Carroll, Cory Y McLean, and Nicholas A Furlotte. Multimodal 1lms for health grounded in individual-specifcc data. In Workshop on Machine Learning for Multimodal Healthcare Data, pp. 86-102. Springer, 2023.
[81] Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountabilliy, and transparency, pp. 610 623, 2021.
93

---

Published in Transactions on Machine Learning Research (10/2025)
[82] Yoshua Bengio, Geoffrey Hinton, Andrew Yao, Dawn Song, Pieter Abbeel, Trevor Darrell, Yuval Noah Harari, Ya-Qin Zhang, Lan Xue, Shai Shalev-Shwartz, et al. Managing extreme ai risks amid rapid progress, Science, 384(6698):842 845, 2024.
[83] Daria Beresneva. Computer-generater text detection using machine learning: A systematic review. In International Conference on Applications of Natural Language to Data Bases, 2016. URL https:
/aap...smanticccholl.. org/CorpusID:I1e 11757766.
[84] Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Michal Podstawski, Hubert Niewiadomski, Piotr Nyczyk, et al. Graph of thoughts: Solving elaborate problems with large language models. ArXiv preprint, abs/2308.000 2023. URL https:/ htapss///arxvv.ooo wrt. aat.aa//..... 09687.
(85] James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science. htlpps///dn. openai.
tpppprs/daalleeeeeess pdf, 2(3):8, 2023.
[86] Camiel J Beukeboom and Christian Burgers. How stereolypes are shared through language: a review and introduction of the aocial categories and stereotypes communication (scsc) framework. Review of Communication Research, 7:1-37, 2019.
[87] Rahul Bhagat and Eduard H. Hovy, Squibs: What is a paraphrase? Computational Linguistics, 39: 463-472 2013. URL https://api. httpss//app..mmmnticcchooooo semanticscholas org/CorppusI::2245555
[88] Amrita Bhattacharjee and Huan Liu. Fighting fire with fire: can chatgpt detect ai-generated text?
ACM SIGKDD Explorations Newsletter, 25(2):14 21, 2024.
[89] Federico Bianchi, Pratyusha Kalluri, Esin Durmus, Faisal Ladhak, Myra Cheng, Debora Nozza, Tat- sunori Hashimoto, Dan Jurafsky, James Zou, and Aylin Caliskan. Easily accessible text-to-image generation amplifies demographic stereotypes at large scale. In Proceedings of the 2023 ACM Con- ference on Fairness, Accountability, and Transparency, pp. 449--1504, 2023.
[90] Dan Biderman, Jose Gonzalez Ortiz, Jacob Portes, Mansheej Paul, Philip Greengard, Connor Jen- nings, Daniel King, Sam Havens, Vitaliy Chiley, Jonathan Frankle, et al. Lora learns less and forgets less. arXiv preprint arXiv:2405.00 096673, 2024.
[91] Abeba Birhane, Vinay Uday Prabhu, and Emmanuel Kahembwe. Multimodal datasets: misogyny, pornography, and malignant stereotypes, 2021.
92] Abeba Birhane, Vinay Prabhu, Sang Han, Vishnu Naresh Boddeti, and Alexandra Sasha Luccioni. Into the laions den: Investigating hate in multimodal datasets, 2023.
[93] Ali Furkan Biten, Lluis Gómez, and Dimosthenis Karatzas. Let there be a clock on the beach: Re- ducing object hallucination in image captioning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1381-1390, 2022.
[94] Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Lovi, Zion English, Vikram Voleti, Adam Letts, et al. Stable vidco diffusion: Scaling latent video diffusion models to large datasets. arXiv preprini Xxxx:331111519// 2023.
[95] David M Blei, Andrew Y Ng, and Michael I Jordan. Latent dirichlet allocation. JMLR, 3(Jan): 993 902-1022, 2003.
[96] Su Lin Blodgett and Brendan O'Connor. Racial disparity in natural language processing: A case study of social media african-american english. arXiv preprint or Xix: 000600 2017.
[97] Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. Language (technology) is power: A critical survey of bias' in nlp. Xrx preprint ar Xix:20005. 14050, 2020.
94

---

Published in Transactions on Machine Learning Research (10/2025)
[98] Mikel Bober-Irizar Ili Shumailov, Yiren Zhao, Robert Mullins, and Nicolas Papernot. Architectural backdoors in neural networks, 2022.
[99] Rishi Bommasani and Percy Liang. Trustworthy social bias measurement, 2022. URL https:
//arxiv. wttpoo/g///...aaaaeeeeg
[100] Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, S. Buch, Dallas Card, Rodrigo Castellon, Niladri S. Chatterji, Annic S. Chen, Kathleen A. Creel, Jared Davis, Dora. Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren E. Gillespie, Karan Goel, Noah D. Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho, Jenny Hong, Kyle Hsu, Jing Huang, Thomas F. Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, O. Khattab, Pang Wei Koh, Mark S. Krass, Ranjay Krishna, Rohith Kuditipudi, Ananya Kumar, Faisal Ladhak, Mina Lee, Tony Lee, Jure Leskovec, Isabelle Levent, Xiang Lisa Li, Xuechen Li, Tengyu Ma, Ali Malik, Christopher D. Manning, Suvir P. Mirchandani, Eric Mitchell, Zanele Munyikwa, Suraj Nair, Avanika Narayan, Deepak Narayanan, Benjamin Newman, Allen Nie, Juan Carlos Niebles, Hamed Nilforoshan, J. F. Nyarko, Giray Ogut, Laurel Orr, Isabel Papadimitriou, Joon Sung Park, Chris Piech, Eva Portelance, Christopher Potts, Aditi Raghunathan, Robert Reich, Hongyu Ren, Frieda Rong, Yusuf H. Roohani, Camilo Ruiz, Jack Ryan, Christopher R'e, Dorsa Sadigh, Shiori Sagawa, Keshav Santhanam, Andy Shih, Krishna Parasuram Srinivasan, Alex Tamkin, Rohan Taori, Armin W. Thomas, Florian Tramèr, Rose E. Wang, William Wang, Bohan Wu, Jiajun Wu, Yuhuai Wu, Sang Michael Xie, Michihiro Ya- sunaga, Jiaxuan You, Matei A. Zaharia, Michael Zhang, Tianyi Zhang, Xikun Zhang, Yuhui Zhang, Lucia Zheng, Kaitlyn Zhou, and Percy Liang. On the opportunities and risks of foundation models. ArXiv, 2021. URL https: crfm. stanford. edweedduassaaee aasaasses/////// dassets/repors httpor.pptttttooe pdf.
[101] Conrad Borchers, Dalia Sara Gala, Benjamin Gilburt, Eduard Oravkin, Wilfried Bounsi, Yuki M Asano, and Hannah Rose Kirk. Looking for handsome carpenter! debiasing gpt-3 job advertisements. arXiv preprint 2rxiv:2205 01002205.1117,,, 1874, 2022.
[102] Shikha Bordia and Samuel R Bowman. Identifying and reducing gender bias in word-level language models. arXiv preprint ar Xii::994400003: 2019.
[103] Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Mil- lican, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, Diego De Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Mag- giore, Chris Jones, Albin Cassirer, Andy Brock, Michela Paganini, Geoffrey Irving, Oriol Vinyals, Simon Osindero, Karen Simonyan, Jack Rae, Erich Elsen, and Laurent Sifre. Improving language models by retricving from trillions of tokens. In Proceedings of the 39th International Conference on Machine Learning, pp. 2206 026-2240,440 2022.
[104] Ali Borji. Qualitative failures of image generation models and their application in detecting deepfakes. Arx, abs/2304 hbs//234406470, 2023. URL https: www.sttiimcooooooi......... wwarg//orpusllte ID: 257826680.
[105] Nick Bostrom. The superintelligent will: Motivation and instrumental rationality in advanced artificial agents. Minds and Machines, 22:71-88 2012.
[106] Dylan Bouchard, Mohit Singh Chauhan, David Skarbrevik, Viren Bajaj, and Zeya Ahmad. Langfair:
A python package for assessing bias and fairness in large language model use cases, 2025. URL htttps://axxi..a wtt.aaaiwii...ga////////// 03112.
[107] Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. A large annotated corpus for learning natural language inference. arXiv preprint arXiv:1508.0533 2015.
[108] Samuel R Bowman, Jocyoon Hyun, Ethan Perez, Edwin Chen, Craig Pottit, Scott Heiner, Kamilè Lukošiūté, Amanda Askell, Andy Jones, Anna Chen, al. Measuring progress on scalable oversight for large language models. Xin preprint ar 2r1 2022.
95

---

Published in Transactions on Machine Learning Research (10/2025)
[109] Ralph Allan Bradley and Milton E Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324-33 1952.
[110] Jack Brassil, Steven H. Low, Nicholas F. Maxemchuk, and Lawrence O'Gorman. Electronic marking and identification techniques to discourage document copying. Proceedings of INFOCOM '94 Con- ference on Computer Cononunications, pp. 1278 1287 vol.3, 1994. URL https://api http:///aap.ssmmaiiiaac.a. semanticac holar. ptg/CorppuIDD:550778...
[111] Glenn W. Brier. Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1):1-3, 1950.
[112] Andrei Z Broder. On the resemblance and containment of documents. In Proceedings. Compression and Complerity of SEQUENCES 1997 (Cat. No. 97TB100171), pp. 21-29. IEEE, 1997.
[113] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
[114] Amy Bunn. Artificial camosttes--cbeerriiing turn to ai voice cloning for new breed of scam, 2024. URL osters-cy 09-09-2024].
[Accessed
[115] Collin Burns, Pavel Izmailov, Jan Hendrik Kirchner, Bowen Baker, Leo Gao, Leopold Aschenbren- ner, Yining Chen, Adrien Ecoffet, Manas Joglekar, Jan Leike, et al. Weak-to-strong generalization: Eliciting strong capabilities with weak supervision. arXiv preprint arXiv:2312. 09390, 2023.
[116] ByteDance Seed Team. Seed-1.6: A multimodal deep-thinking foundation model. https://seed: ytedance.com com.eilccomiiiio www./////eeeeeeeeeeeeeeeeeeeeeee 6, 2025.
[117] Aylin Caliskan, Joanna Bryson, and Arvind Narayanan. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183-188 2017.
[118] Bochuan Cao, Yuanpu Cao, Lu Lin, and Jinghui Chen. Defending against alignment brngmment-breakin attacks via robustly aligned 1lm. arXie preprint ar XrXv::2309. 14444489914448, 2023.
[119] Chentao Cao, Zhun Zhong, Zhanke Zhou, Yang Liu, Tongliang Liu, and Bo Han. Envisioning outlier exposure by large language models for out-of-distribution detection. arXiv preprint orXiv:2406. 00806, 2024.
[120] Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models. Tn Conference on Empirical Methods in Natural Language Processing, 2021. URL https://app semant icscholar.org hrtpaaarrrr//orros::0223230
[121] Yihan Cao, Yanbin Kang, and Lichao Sun. Instruction mining: High-quality instruction data selection for large language models. arXiv preprint arXiv:2307.06200 2023.
[122] Captum. Testing with concept activation vectors (tcav) on sensitivity classification examples and a convnet model trained on imdb dataset. https://github. http:tttuuu.ccopptttttttee wttp://twtootocpopeoeoecoce http://occcccaaauo/oo///oo/////e http://///boobb/ttte//eeeee torials/TCAV NLP. ipynb, 2022.
[123] Nicholas Carlini and Andreas Terzis. Poisoning and backdooring contrastive learning. arXie preprinl arXiv:2106.0980 2021.
[124] Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21), pp. 2633 2650, 2021.
96

---

Published in Transactions on Machine Learning Research (10/2025)
[125] Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. Quantifying memorization across neural language models. In The Fleventh International Conference on Learning Representations, 2022.
[126] Nicholas Carlini, Milad Nasr, Christopher A Choquette-Choo, Matthew Jagielski, Irena Gao, Pang Wei W Koh, Daphne Ippolito, Florian Tramer, and Ludwig Schmidt. Are aligned neural networks adversarially aligned? Advances in Neural Information Processing Systems, 36, 2024.
[127] Nicolas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramer, Borja Balle, Daphne Ippolito, and Eric Wallace. Extracting training data from diffusion models. In 32nd USENIX Security Symposium (USENIX Security 28), pp. 5253-5270, 2023.
[128] Stephen Casper, Xander Davies, Claudia Shi, Thomas Krendl Gilbert, Jérémy Schourer, Javier Rando, Rachel Freedman, Tomasz Korbak, David Lindner, Pedro Freire, et al. Open problems and funda- mental limitations of reinforcement learning from human feedback. ArXiv preprint, abs/2307.15217, 2023. URL /ttpps:/aat arxiv. org/ 2307 15217.
[129] Souradip Chakraborty, Jiahao Qiu, Hui Yuan, Alec Koppel, Furong Huang, Dinesh Manocha, Am- rit Singh Bedi, and Mengdi Wang. Maxmin-rlhff Towards equitable alignment of large language models with diverse human preferences. arXiv preprint arXim:2402.00 08925, 2024.
[130] Taas Chalkidis, Tommaso Pasini, Sheng Zhang, Letizia Tomada, Sebastian Felix Schwemer, and An- ders Søgaard. Fairlex: A multilingual benchmark for evaluating fairness in legal text processing, arXiv preprint arXiv:2203. Nii::2203.07228,, 2022.
[131] Chun Sik Chan, Huanqi Kong, and Liang Guanqing. A comparative study of faithfulness metrics for model interpretability methods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 5029-5038, 2022.
[132] Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George Pappas, and Eric Wong, Jailbreaking black box large language models in twenly queries. arXio preprinl or orXiv:2310.08; 00.02000..8499, 2023.
[133] Aliza Chasan. Ethical hacker scams 60 Mimutes staffer to show how easy digital theft is, 2023. URL https://www wwsnews.wn.e.......
wiw-diiigaaatteeteeeeeee taeettattttttttttttttteee-e pwwple-froommilleeeeee- [Accessed 09-09-2024).
[134] Adilya Chattopadhyay, Kwan Ho Ryan Chan, Benjamin D Haelfele, Donald Geman, and René Vidal. Variational information pursuit for interpretable predictions. arXiv preprint arxi::2302.028 2023.
[135] Aditya Chattopadhyyy, Kwan Ho Ryan Chan, and Rene Vidal. Bootstrapping variational information pursuit with large language and vision models for interpretable image classification. In The Twelfth International Conference on Learning Representations, 2024.
[136] Hila Chefer, Oran Lang, Mor Geva, Volodymyr Polosukhin, Assaf Shocher, Michal Irani, Inbar Mosseri, and Lior Wolf. The hidden language of diffusion models. In ICLR, 2024.
[137] Authony Chen, Panupong Pasupat, Sameer Singh, Hongrae Lee, and Kelvin Guu. Purr: Efficiently aditing language model hallucinations by denoising language model corruptions. arXiv preprint arXio:2305.1 149988 2023.
[138] Beitao Chen, Xinyu Lyu, Lianli Gao, Heng Tao Shen, and Jingkuan Song. Alleviating hallucinations in large vision-language models through ietnccination-indueee optimization. In The ehirtt Annual Conference on Newral Information Processing Systems, 2024.
[139] Bocheng Chen, Advait Paliwal, and Qiben Yan. Jailbreaker in jail: Moving target defense for large language models. In Proceedings of the 10th ACM Workshop on Moving Torget Defense, pp. 29 29-32, 2023.
97

---

Published in Transactions on Machine Learning Research (10/2025)
[140] Boli Chen, Yao Fu, Guangwei Xu, Pengiun Xie, Chuanqi Tan, Mosha Chen, and Liping Jing. Probing BERT in Hyperbolic Spaces, April 2021. URL http: //wxxv/axxvv.v///a///a//aaaaaia wwrggbbb//1// 03869. arXiv:2104.03565 [es].
[141] Bryant Chen, Wilka Carvalho, Nathalie Baracaldo, Heiko Ludwig, Benjamin Edwards, Taesung Lee, lan Molloy, and Biplav Srivastava. Detecting backdoor attacks on deep neural networks by activation clustering. arXiv preprint arXiv:1811. 11X::1811.037288, 2018.
[142] Canyu Chen, Baixiang Huang, Zekun Li, Zhaorun Chen, Shiyang Lai, Xiongxiao Xu, Jia-Chen Gu, Jindong Gu, Huaxiu Yao, Chaowei Xiao, et al. Can editing Ilms inject harm? arXiv preprint arXiv:2407 27220222. 2024.
[143] Chao Chen, Kai Liu, Ze Chen, Yi Gu, Yue Wu, Mingyuan Tao, Zhihang Fu, and Jieping Ye. Inside: Lims' internal states retain the power of hallucination detection, 2024.
[144] Gongwei Chen, Leyang Shen, Rui Shao, Xiang Deng, and Liqiang Nie. Lion: Empowering multimodal large language model with dual-level visual knowledge. ar preprint arXw:2211111166 2023.
[145] Hanjie Chen and Yangfeng Ji. Adversarial training for improving model robustness? look at both prediction and interpretation. In The 36th AAAI Conference on Artificial Intelligence (AAAT), 2022.
[146] Hanjie Chen, Faeze Brahman, Xiang Ren, Yangfeng Ji, Yejin Choi, and Swabha Swayamdipta. Rev: information-thoorrtiec evaluation of free-text rationales. The 61th Annual Meeting of the Association for Compulational Linguistics (ACL), 2023.
[147] Jianqi Chen, Hao Chen, Keyan Chen, Yilan Zhang, Zhengxia Zou, and Zhenwei Shi. Diffusion models for imperceptible and transferable adversarial attack. TEFF Transactions on Pallern Analysis and Machine Intelligence, 2025.
[148] Jifan Chen, Grace Kim, Aniruddh Sriram, Greg Durrett, and Eunsol Choi. Complex claim verification with evidence retrieved in the wild. arXie preprint Xiv:2303 10...11881 11859, 2023.
[149] Jiuhai Chen and Jonas Mueller. Quantifying uncertainty in answers from any language model and enhancing their trustworthiness, 2023.
[150] Jiuhai Chen and Jonas Mueller. Automated data curation for robust language model fine-tuning. Xii preprint 2rXiv:2403.129 12776, 2024.
[151] Jiuhai Chen, Lichang Chen, Heng Huang, and Tianyi Zhou. When do you need chain-of-thought prompting for chatgpt? arXiu preprinl KXi:22304.0320 2023.
[152] Jiuhai Chen, Rifaa Qadri, Yuxin Wen, Neel Jain, John Kirchenbauer, Tianyi Zhou, and Tom Gold- stein. Genqa: Generating millions of instructions from a handful of prompts. ar preprinl arXix:2406. 10323, 2024.
[153] Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, and Hongxia Jin. Alpagasus: Training a better alpaca with fewer data, 2023.
[154] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Ka- plan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.0337. 2021.
[155] Shuo Chen, Zhen Han, Bailan He, Mark Buckley, Philip Torr, Volker Tresp, and Jindong Gu. Under- standing and improving in-context learning on vision-language models. In JCLR 2024 Workshop on Mathematical and Empirical Understanding of Foundation Models, 2023.
[156] Shuo Chen, Zhen Han, Bailan He, Zifeng Ding, Wenqian Yu, Philip Torr, Volker Tresp, and Jindong Gu. Red teaming gpt-4v: Are gpt-4v safe against uni/multi-moda jailbreak attacks? arXio preprint arXiv:2404.03/ 03411, 2024.
98

---

Published in Transactions on Machine Learning Research (10/2025)
[157] Weixin Chen, Dawn Song, and Bo Li. Trojdiff: Trojan attacks on diffusion models with diverse largets. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition. pp. 4035 4044, 2023.
[158] Wenhu Chen, Hexiang Hu, Xi Chen, Pat Verga, and William W Cohen. Murag: Multimodal retrieval-angmenneed generator for open question answering over images and text. arXiv preprint 0rXiv:2210.02900 2022.
[159] Yanda Chen, Ruiqi Zhong, Narutatsu Ri, Chen Zhao, He He, Jacob Steinhardt, Zhou Yu, and Kath- leen McKcown. Do models explain themselves? counterfactual simulatability of natural language explanations, 2023.
[160] Yang Chen, Ethan Mendes, Sauvik Das, Wei Xu, and Alan Ritter. Can language models be instructed to protect personal information? arXiv preprint or wxXv:2310.02224, 2023.
[161] Yangyi Chen, Karan Sikka, Michael Cogswell, Heng Ji, and Ajay Divakaran. Dress: Instructing large vision-language models to align and interact with humans via natural language leedback, 2023.
[162] Yutian Chen, Hao Kang, Vivian Zhai, Liang Li, Rita Singh, and Bhiksha Ramakrishnan. Gpt- sentinel: Distinguishing human and chatgpt generated content. ArXiv, abs/2305.07769,, 2023. URL
https://api. wwwanticscholesttt... wttpriccocoo.ooooooooao.....l... 258686680.
[163] Yuyan Chen, Qiang Fu, Yichen Yuan, Zhihao Wen, Ge Fan, Dayiheng Liu, Dongmei Zhang, Zhixu Li, and Yanghua Xiao. Hallucination detection: Robustly discerning reliable answers in large language models. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pp. 245 255, 2023.
[164] Zhaorun Chen, Zhuokai Zhao, Wenjie Qu, Zichen Wen, Zhiguang Han, Zhihong Zhu, Jiaheng Zhang, and Huaxiu Yao. Pandora: Detailed I1m jailbreaking via collaborated phishing agents with decom- Workshop on Secure and Trustworthy Large Language Models.
posed reasoning. In ICLR 2024
[165] Zhaorun Chen, Yichao Du, Zichen Wen, Yiyang Zhou, Chenhang Cui, Zhenzhen Weng, Haoqin Tu, Chaoqi Wang, Zhengwei Tong, Qinglan Huang, et al. Mj-bench: Is your multimodal reward model really a good judge for text-to-image generation? arXiv preprint arXiv:2407.04558 2024.
[166] Zhaorun Chen, Francesco Pinto, Minzhou Pan, and Bo Li. Safewatch: An efficient salety-policy following video guardrail model with transparent explanations. arXio preprint orXiv:2412.06870 2024.
[167] Zhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song, and Bo Li. Agentpoison: Rod-tcaming Ilm agents via poisoning memory or knowledge bases, arXiv preprint 2xXx::24071178,, 2024.
[168] Zhaorun Chen, Zhuokai Zhao, Hongyin Luo, Huaxiu Yao, Bo Li, and Jiawei Zhou. Hale: Object hallucination reduction via adaptive focal-contrast decoding. In International Conference on Machine Learning (ICML), 2024.
[169] Zhaorun Chen, Zhuokai Zhao, Zhihong Zhu, Ruiqi Zhang, Xiang Li, Bhiksha Raj, and Huaxiu Yao. Autoprm: Automating procedural supervision for multi-step reasoning via controllable question de- composition. In North American Chapter of the Association for Computational Linguistics (NAACL), 2024.
[170] Zhaorum Chen, Mintong Kang, and Bo Li. Shieldagent: Shielding agents via verifiable safety policy reasoning. arXiv preprint arXiv:2503.2273 2025.
[171] Zhaoyu Chen, Bo Li, Shuang Wu, Kaixun Jiang, Shouhong Ding, and Wenqiang Zhang. Content- based unrestricted adversarial attack. In A. Oh, T. Naumann, A. Globerson, K. Sacnko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, 2023.
99

---

Published in Transactions on Machine Learning Research (10/2025)
[172] Hao Cheng, Erjia Xiao, Jindong Gu, Le Yang, Jinhao Duan, Jize Zhang, Jiahang Cao, Kaidi Xu, and Renjing Xu. Unveiling typographic deceptions: Insights of the typographic vulnerability in large vision-language model. European Conference on Computer Vision (ECCV) (To appear), 2024.
[173] Jiali Cheng and Hadi Amiri. Multimodal machine unlearning. arXiv preprint or Xrx:23311 1 10047, 2023.
[174) Siyuan Cheng, Bo Tian, Qingbin Liu, Xi Chen, Yongheng Wang, Huajun Chen, and Ningyu Zhang. Can we edit multimodal large language models? ArXiv, abs/2310.08475, 2023. URL https: /tttps:/api.. semanticscholar. org/CorpusID:. 263908997.
[175] Siyuan Cheng, Ningyu Zhang, Bo Tian, Zelin Dai, Feiyu Xiong, Wei Guo, and Huajun Chen. Editing language model-based knowledge graph embeddings. ArXiu, 2///2000111105,, 2023. URL https:
//api.semanticscchoeeeee wwweeorg//oooo org/CorpusID) oug/CorpuuID::256231422 256231427.
[176] I Chern, Steffi Chern, Shiqi Chen, Weizhe Yuan, Kehua Feng, Chunting Zhou, Junxian He, Graham Neubig, Pengfei Liu, et al. Factool: Factuality detection in generative ai-a tool augmented framework for multi-task and multi-domain scenarios. arXiv preprint arXie:2307.1335 2023.
[177] Cheng-Han Chiang and Hung-yi Lec. Can large language models be an alternative to human evalua- tions? arXiv preprint Xr3:5 01937, 2023.
[178] Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vieuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023. URL https: https://mmsys.......//////////st org/blog 10023-33-30-iiooe
[179] Zhi-Yi Chin, Chieh-Ming Jiang, Ching-Chun Huang, Pin- Yu Chen, and Wei-Chen Chiu. Prompt- ingadebugging: Red-teaming eext-to-image diffusion models by finding problematic prompts. arXiv preprint orXiv:2309. 06135, 2023.
[180] Jaemin Cho, Abhay Zala, and Mohit. Bansal. Dall-eval: Probing the reasoning skills and social biases of toct-to-image generation models. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 3043-3011 1043- 2023.
[181] Yujin Choi, Jinscong Park, Hoki Kim, Jacwook Leo, and Sacroom Park. Fair sampling in diffusion models through switching mechanism. arXiv preprint arXiv:2401.03144 2024.
[182] Sheng- Yen Chou, Pin-Yu Chen, and Tsung-Yi Yi Ho. Villandiffusion: A unified backdoor attack frame- work for diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
[183] Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1 113, 2023.
[184] Miranda Christ, Sam Gunn, and Or Zamir. Undetectable watermarks for language models. IACR Cryptol. ePrint Arch., 2023:763, 2023. URL https: //api. http:/ap...maaiiiiimmmm semanticscholar wwwg//orgpoeeeoooooooo 259092330.
[185] Paul Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30, 2017.
[186] George Chrysostomou and Nikolaos Aletras. Improving the Faithfulness of Attention-based Ex- planations with Task-specific Information for Text Classification. In Proceedings of the 59th An- nual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume Long Papers), pp. 477-488, Online, Au- gust 2021. Association for Computational Linguistics. doi: 10.18653/v1/202 http:/2222.allllllo acl- long.40. URL httt:://acclamhboloooooog// how./oo......llloo..oooo -long.40.
acl
100

---

Published in Transactions on Machine Learning Research (10/2025)
[187] Zhixuan Chu, Lei Zhang, Yichen Sun, Siqiao Xue, Zhibo Wang, Zhan Qin, and Kui Ren. Sora detector: A unified hallucination detection for large text-to-video models. arXio preprinl orXiu:2405.00 180, 2024.
[188] Ching-Yao Chuang, Varun Jampani, Yuanzhen Ti, Antonio Torralba, and Stefanie Jegelka. Debiasing vision-language models via biased prompts. arXiv preprint TrXiu:2302000070 2023.
[189] Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, and Pengeheng He. Dola: Decoding by contrasting layers improves factuality in large language models. arXiv preprint orXit:2309008885 2023.
[190] Hyung Won Chung, Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, al. Scaling instuuctoonfiietune language models. Journal of Machine Learning Research, 25(70):1-53, 2024.
[191] Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXi::1422.3555 2014.
[192] Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D. Manning, What does BERT look at? an analysis of bert's attention. In BlackbozNLP@ACL, pp. 276-286. Association for Computa- lional Linguistics, 2019.
[193] Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D. Manning. What Does BERT Look at? An Analysis of BERT's Attention. In Proceedings of the 2019 ACL Workshop Blackbox Analyzing and Interpreting Newral Networks for NLP, pp. 276 286, Florence, Italy, August 2019. Association for Computational Linguistics. doi: 10.18653/v1/W W119-4888. URL gy. sy.org/W19-4828
[194] CNN. AI won an art contest, and artists are furious. https:///ww/ hettps:/www.ccn.o. cnn. ccwm/2202//09/0/ttchhaaa wat--fair-immer-comm contrrrrrroooeer//ddeee. html 2022.
[195] Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In international conference on machine learning, pp. 1310-1320. PMLR, 2019.
[196] Roi Cohen, May Hamri, Mor Geva, and Amir Globerson. Lm Im: Detecting factual errors via cross examination. arXiv preprint arXiv:2305.1328. 2023.
[197] Jeremy R. Cole, Michael Q. Zhang, Daniel Gillick, Julian Martin Eisenschlos, Bhuwan Dhingra, and Jacob Eisenstein. Selectively answering ambiguous questions, 2023.
[198] Competition and Markets Authority. Ai foundation models: initial review. https://www.gous uk/cma-
cases/ai fonddfuiooniiooeeleeee -iniiddelstttiiiil-eeeeee 2023.
[199] Andrea Cossu, Antonio Carta, Lucia Passaro, Vincenzo Lomonaco, Tinne Tuytelaars, and Davide Bacciu. Continual pre-training mitigates forgetting in language and vision. Neural Networks, 179: 106492, 2024.
[200] Davide Cozzolino, Giovanni Poggi, Matthias Nießner, and Luisa Verdoliva. Zero-shot detection of ai- ai-generated images. In European Conference on Computer Vision, pp. 54-72. Springer, 2025.
[201] Chenhang Cui, Yiyang Zhou, Xinyu Yang, Shirley Wu, Linjun Zhang, James Zou, and Huaxiu Yao. Holistic analysis of hallucination in gpt-4v (ision): Bias and interference challenges. arXie preprint 0rXiv:2311.0222 2023.
[202] Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, and Furu Wei. Knowledge neurons in pretrained transformers. ArXie, abs/2104.08696, 2021. URL https://api wwmanticscholarttt... org/CorpusID: 233296761.
[203] Wenliang Dai, Zihan Liu, Ziwei Ji, Dan Su, and Pascale Fung. Plausible may not be faithful: Probing object hallucination in vision-language pre-training. arXiv preprint arXiv:2210 XXiv2220000766 00667888 2022.
101

---

Published in Transactions on Machine Learning Research (10/2025)
[204] Xuelong Dai, Kaisheng Liang, and Bin Xiao. Advdiff: Generating unrestricted adversarial examples using diffusion models. Xiv preprint orii::2077. 12499, 2023.
[205] Yi Dai, Hao Lang, Kaisheng Zeng, Fei Huang, and Yongbin Li. Exploring large language models for multi-modal out-of-distribution detection. arXiv preprint arXi::231008810 000227 2023.
[206] Mayur Datar, Nicole Immorlica, Piotr Indyk, and Vahab S Mirrokni. Locality- sensitive hashing scheme based on p-stable distributions. In Proceedings of the twentieth annual symposium on Com- putational geometry, pp. 253 262, 2004.
[207] DeepSook. Deepseck-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint Stu:2501.12944 2025.
[208] Gelei Deng, Yi Liu, Yuckang Li, Kailong Wang, Ying Zhang, Zefeng Li, Haoyu Wang, Tianwei Zhang, and Yang Liu. Jailbreaker: Automaled jailbreak across multiple large language model chatbots. arviv preprint arXiv:2307.0877 2023.
[209] Mingkai Deng, Jianyu Wang, theng--ingg Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric Xing, and Zhiting Hu. Riprompt: Optimizing Discrete Text Prompts with Reinforcement Learn- ing. In EMNLP 2022, pp. 3369-3391, 2022.
[210] Yihe Deng, Pan Lu, Fan Yin, Ziniu Hu, Sheng Shen, James Zou, Kai- Wei Chang, and Wei Wang, Enhancing large vision language models with self-training on image comprehension. arXiv preprint arXiv:2405. 19716, 2024.
[211] Yue Deng, Wenxuan Zhang, Sinno Jialin Pan, and Lidong Bing. Multilingual jailbreak challenges in large language models. In The Twelfth International Conference Learning Representations, 2024. URL https://openreviewt www. www.:eeeoooommdddwwieeae...
[212] Zhijie Deng, Hongcheng Gao, Yibo Miao, and Hao Zhang. Efficient detection of IIm-generated texts with a bayesian surrogate model. ArXie, abs/2305.16617, 2023. URL https://apt semanticscho lar.org/ oarror/CCorrpsDDD 258947640.
[213] Zhun Deng, Thomas P. Zollo, Jake C. Snell, Toniann Pitassi, and Richard Zemel. Distribution-free statistical dispersion control for societal applications, 2023.
[214] Joseph F. DeRose, Jiayao Wang, and Matthew Berger. Attention flows: Analyzing and comparing attention mechanisms in language models. IEEE Trans. Vis. Comput. Graph., (22)):1160-111 2021.
[215] Shrey Desai and Greg Durrett. Calibration of pre-trained transformers, 2020.
[216] Gianluca Detommaso, Martin Bertran, Riccardo Fogliato, and Aaron Roth. Multicalibration for confidence scoring in llms, 2024.
[217] Nicolas Deutschmann, Marvin Alberts, and María Rodríguez Martínez. Conformal autoregressive generation: Beam search with coverage guarantees, 2023.
[218] Sunipa Dev, Tao Li, Jeff M Phillips, and Vivek Srikumar. On measuring and mitigating biased inferences of word embeddings. In Proceedings of the AAAT Conference on Artificial Intelligence, volune 34, pp. 7659 7666, 2020.
[219] Jacob Devlin, Ming-Wo Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171 4186, Minneapolis, Minnesota, 2019. Association for Computational Linguistics. doi: 10.18633/v1/N19/ gy.org wroo//19-11999
URL https://aclanthhloo
[220] Terrance DeVries and Graham W. Taylor. Improved regularization of convolutional neural networks with cutout, 2017.
102

---

Published in Transactions on Machine Learning Research (10/2025)
[221] Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C. Wallace. FRASER: A Benchmark to Evaluate Rat NLP Models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4443 4458, Online, July 2020. Association for Computational Linguistics. doi: (0.18653/v1120200 /:tp//2000.c22mmma aal-mmai..400 URL org/2020. aaw-mmai...00000 408.
[222]
Jwala Dhamala, Tony Sun, Varun Kumar, Satyapriya Krishna, Yada Pruksachatkun, Kai- Chang, and Rahul Gupta. Bold: Dataset and metries for measuring biases in open-ended language generation. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pp. 862 872,
2021.
[223] Bhuwan Dhingra, Manaal Faruqui, Ankur Parikh, Ming-Woi Wei Chang. Dipanjan Das, and William Cohen. Handling divergent reference texts when evaluating table-to-text generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4884-4895, 2019.
[224] Shizhe Diao, Zhichao Huang, Ruijia Xu, Xuechun Li, Yong Lin, Xiao Zhou, and Tong Zhang. Black- box prompt learning for pre-trained language models. arXiv preprint ar arXiv:201.00831,, 2022.
[225] Shizhe Diao, Pengcheng Wang, Yong Lin, and Tong Zhang. Active prompting with Ohain-of-thoughi for large language models. arXiv preprint 230: 12246, 2023.
[226] Thomas G Dietterich. Ensemble methods in machine learning. In International workshop on mullippe classifier systems, pp. 1-15. Springer, 2000.
[227] Ning Ding, Shengding Hu, Weilin Zhao, Yulin Chen, Zhiyuan Liu, Hai-Tao Zheng, and Maosong Sun. Openprompt: An open-source framework for prompt-learning arXiv preprint arXiv:211 211:0111.0199 2021.
[228] Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Mcasuring and miti- gating unintended bias in text classification. In Proceedings of the 2018 AAAT/ACM Conference on AI, Ethics, and Society, pp. 67-73, 2018.
[229] Alphaeus Dmonte, Marcos Zampieri, Kevin Lybarger, and Massimiliano Albanese. Classifying human- generated and ai-generated election claims in social media. arXiv preprint arXiv:2404.16116 2024.
[230] Thang Viet Doan, Zhibo Chu, Zichong Wang, and Wenbin Zhang. Fairness definitions in language models explained. arxiv preprint 2xXi:2407.18454, 2024.
[231] Tim Dockhorn, Tianshi Cao, Arash Vahdat, and Karsten Kreis. Differentially private diffusion models. XrXio preprint arXiv:2210.0900 09929, 2022.
[232] Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Mar- garet. Mitchell, and Matt Gardner. Documenting large webtext corpora: A case study on the colossal clean crawled corpus. arXiv preprint arXic:2104.088 08758, 2021.
[233] Hanze Dong, Wei Xiong, Deepanshu Goyal, Rui Pan, Shizhe Diao, Jipeng Zhang, Kashun Shum, and Tong Zhang. Raft: Reward ranked finetuning for generative foundation model alignment. ArXiv preprinl, abs/2304.06667, 2023. URL https://arxix: http://axxvv..o/abbaaaaa/
[234] Qingxiu Dong, Damai Dai, Yifan Song, Jingjing Xu, Zhifang Sui, and Lei Li. Calibrating factual knowledge in pretrained language models. ArXiv, abs/2210.033299 2022. URL https://api sttts: semant icscholar. www.oooopooi:i/eeeeeerr 252762125.
[235] Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui. A survey on in-context learning. arXiv preprint 2xXi::3011002244 2022.
[236] Yi Dong, Zhilin Wang, Makesh Sreedhar, Xianchao Wu, and Oleksii Kuchaiov. Steerlm: Attribute conditioned sft as an (user-stocrable) alternative to rlhf. In Findings of the Association for Compu- talional Linguisties: EMNLP 2023, pp. 11275- 11288, 2023.
103

---

Published in Transactions on Machine Learning Research (10/2025)
[237] Yinpeng Dong, Huanran Chen, Jiawei Chen, Zhengwei Fang, Xiao Yang, Yichi Zhang, Yu Tian, Hang Su, and Jun Zhu. How robust is google's s bard to adversarial image attacks? arxie preprinl (tft::300.1175... 2023.
[238] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Goorg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arviv preprint arXiv:2010. 1 11929,
2020.
[239]
Iddo Drori, Sarah Zhang, Reece Shuitleworth, Leonard Tang, Albert Lu, Elizabeth Ke, Kevin Liu, Linda Chen, Sunny Tran, Newman Cheng, et al. A neural network solves, explains, and generates university math problems by program synthesis and fow-shot learning at human level. Proceedings of the National Academy of Sciences, 119(32)) 2022.
[240] Mengnan Du, Ninghao Liu, Fan Yang, Shuiwang Ji, and Xia Hu. On attribution of recurrent neural network predictions via additive decomposition. In WWW, pp. 383-393. ACM, 2019.
[241] Mengnan Du, Varun Manjunatha, Rajiv Jain, Ruchi Deshpande, Franck Dernoncourt, Jiuxiang Gu, Tong Sun, and Xia Hu. Towards interpreting and mitigaling shorteut learning behavior of nlu models. North American Chapter of the Association for Computational Linguistics (NAACL), 2021.
[242] Mengnan Du, Fengxiang He, Na Zou, Dacheng Tao, and Xia Hu. Shorteut learning of large language models in natural language understanding. Communications of the ACM (CACM), 2023.
[243] Jinhao Duan, Fei Kong, Shiqi Wang, Xiaoshuang Shi, and Kaidi Xu. Are diffusion models vulnerable to membership inference attacks? In International Conference on Machine Learning, pp. 8717-8730. PMLR, 2023.
[244] Jinhao Duan, Hao Cheng, Shiqi Wang, Alex Zavalny, Chenan Wang, Renjing Xu, Bhavya Kailkhura, and Kaidi Xu. Shifting attention to relevance: Towards the predictive uncertainty quantification of free-form large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 5050-5063, 2024.
[245] Jinhao Duan, Fei Kong, Hao Cheng, James Diffenderfer, Bhavya Kailkhura, Lichao Sun, Xiaofeng Zhu, Xiaoshuang Shi, and Kaidi Xu. Truthprint: Mitigating Ivim object hallucination via latent truthful-guided pre-intervention. arXiv preprint SEtT::2503.00022 2025.
[246] Michael Duan, Anshuman Suri, Niloofar Mireshghallah, Sewon Min, Weijia Shi, Luke Zettlemoyer, Yulia Tov, Yejin Choi, David Evans, and Hannaneh Hajishirzi. Do membership inference attacks work on large language models? arXiv preprint arXiv:2402.0753 2024.
[247] Liam Dugan, Daphne Ippolito, Arun Kirubarajan, Sherry Shi, and Chris Callison-Burch Real or fake text?: Investigating human ability to detect boundaries between human- written and machine- generated text. In AAAI Conference on Artificial Intelligence, 2022. URL https: semantic scholar.. scholar.org/Corpuee orgp/orpusID:255.... 255125274.
[248] Ondřej Dušek, David M Howcroft, and Verena Rieser. Semantic noise matters for neural natural language generation. arXiv preprint arXiv:1911.0030 1.3905, 2019.
[249] Cynthia Dwork. Differential privacy. In International colloquium on automata, languages, and pro- gramming, pp. 12 Springer, 2006.
[250] Nouha Dziri, Andrea Madotto, Osmar Zaiane, and Avishek Joey Bose. Neural path hunter: Reducing hallucination in dialogue systems via path grounding. arXiv preprinl arXiv:2104 0xXx:2204.08455, 2021.
[251] Kennedy Edemacu and Xintao Wu. Privacy preserving prompt engineering: A survey. arXiv preprint xrX00:244400001, 2024.
[252] Bryan Eikema. The effect of generalisation on the inadequacy of the mode. In Proceedings of the 1st Workshop Uncertainly-Awwnr Aware NLP (UncerlaiNTPP 2024), pp. 87-92, 2024.
104

---

Published in Transactions on Machine Learning Research (10/2025)
[253] Bryan Eikema and Wilker Aziz. Is map decoding all you need? the inadequacy of the mode in neural machine translation. arXiv preprint arXiv:2005.. 10283, 2020.
[254] Jacob Eisenstein, Chirag Nagpal, Alekh Agarwal, Ahmad Beirami, Alex D'A D'Amour, DJ Dvijotham, Adam Fisch, Katherine Heller, Stephen Pfohl, Deepak Ramachandran, Peter Shaw, and Jonathan Berant. Helping or herding? reward model ensembles mitigate but do not climinate reward hacking,
2024. URL https: /tta:///arxii http///xxvvv.rr////// org/abs..bb...... http://31..9999
[255] Ran EI- Yaniv and Yair Wiener. On the foundations noise-fres selective classification. Journal of Machine Learning Research, 11653):1005-1611, 2010. URL http://jmlr http://mmirrrrggpppeeeeee orggpapers/vi1//llppe.
[256] Mohamed Elaraby, Mengyin Lu, Jacob Dunn, Xueying Zhang, Yu Wang, and Shizhu Liu. Halo: Estimation and reduction of hallucinations in open-source weak large language models. arXiv preprint Xrx:::208..1 1 1764, 2023.
[257]
Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ncluuse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. A Math- ematical Framework for Transformer Circuits transformerr-cireuiis..ppp https://transffeeeeee //trraaafffoorr-cc index.html, December 2021. [Accessed 27-11-2023].
ircuits, pptb//22//frameeea
[258]
Joseph Enguchard. Sequential integrated gradients: simple but effective method for explaining language models. In ACL (Findings), pp. 2023.
Association for Computational Linguistics,
[259]
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first International Conference on Machine Learning, 2024.
[260] Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. Kto: Model aligument as prospect thcorotic optimization, 2024.
[261] Chibuike Samuel Fze and Lior Shamir. Analysis and prevention of ai-based phishing email attacks. Electronics, 13(10):1839, 2024.
[662] Tongtong Fang, Nan Lu, Gang Niu, and Masashi Sugiyama. Rethinking importance weighting for deep learning under distribution shift. Advances neural information processing systems, 18:11996-12007 2020.
[263] Hany Farid, Lighting (in)consistency of paint by text. ArXiu, abs/2207.13744, 2022. URL https:
//api. semanticscholar. org/CorpusID
Dttpr//cccccoaaooorrrrrrrrsaiii :511:::::1111 251135258.
[264] Hany Farid. Perspective (in)eonsistency of paint by text. ArXiv, abs/2206.14617, 2022. URL https:
//api. semantisccccccl org/ ID:
[265] Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. Detecting hallucinations in large language models using semantic entropy. In Nature, 2024.
[266] Shanglun Feng and Florian Tramèr. Privacy backdoors: Stealing data with corrupted pretrained models. arXiv preprint arXiv:2404.00 00473, 2024.
[267] Shiwei Feng, Guanhong Tao, Siyuan Cheng, Guangyu Shen, Xiangzhe Xu, Yingqi Liu, Kaiyuan Zhang, Shiqing Ma, and Xiangyu Zhang. Detecting backdoors in pre-trained encoders. Tn Proceedings of the IEEE/CTC /EE Conference on Computer Vision and Pattern Recognition, pp. 16352-16362, 2023.
[268] Patrick Fernandes, Aman Madaan, Emmy Liu, António Farinhas, Pedro Henrique Martins, Amanda Bertsch, José G. C. de Souza, Shuyan Zhou, Tongshuang Wu, Graham Neubig, and André F. T. Martins. Bridging the gap: A survey on integrating (human) feedback for natural language generation, 2023.
105

---

Published in Transactions on Machine Learning Research (10/2025)
[269] Pierre Fernandez, Antoine Chaffin, Karim Tit, Vivien Chappelier, and Teddy Furon. consolidate watermarks for large language models. 2023 TEEE International Workshop on Information Forensics and Security (WIFS), pp. 1 6, 2023. URL http:htppp//ppp...eaaaiccccooooo
Three bricks to
semanticscholarr wwtpsaschhhaaaroo/ooooorrree Cog//orr:
260351507.
[270] Emilio Ferrara. Should chatgpt be biased? challenges and risks of bias in large language modcls. arXiv preprint orXiv:2304.03735 2023.
[271] Adam Fisch, Tommi Jaakkola, and Regina Barzilay. Calibrated selective classification, 2022. URL https://arxiv.. wrw/ waaaa////.... 12084.
[272] Marina Fomicheva, Shuo Sun, Lisa Yankovskaya, Frédéric Blain, Francisco Guzmán, Mark Fishel, Nikolaos Aletras, Vishrav Chaudhary, and Lucia Specia. Unsupervised quality estimation for neural machine translation, 2020.
[273] Stanislav Fort, Jie Ren, and Balaji Lakshminarayanan. Exploring the limits of out-of-distribution detection. Advances in Neural Information Processing Systems, 34:7068- 7081, 2021.
[274] Alisa Fortin, Guillaume Vernade, Kat Kampf, and Ammaar Reshi. Introducing Gemini 2.5 flash image; our state-of-thr-art image model. August 2025.
googleblog. com/ent /n///nuooooiing
gemini-2-5-flash-imeee
[275] Felix Friedrich, Patrick Schramowski, Manuel Brack, Lukas Struppek, Dominik Hintersdorf, Sasha Luccioni, and Kristian Kersting. Fair diffusion: Instructing text-to-image generation models on fairness. orXiv preprint arXiv:2302.1083 2023.
[276] Yao Fu, Hao-Chun Peng, Ashish Sabharwal, Peter Clark, and Tushar Khot. Complexity- Baseed Prompting for Multi-step Reasoning. TCLR 2023 poster, 2022.
[277] Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit Haim Bermano, Gal Chechik, and Daniel Cohen-or. An image is worth one word: Personalizing text-to-image generation using textual inversion. In The Eleventh International Conference on Learning Represenlations, 2022.
[278] Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning, 2016.
[279] Isabel O Gallegos, Ryan A Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernon- court, Tong Yu, Ruiyi Zhang, and Nesreen K Ahmed. Bias and fairness in large language models: A survey. Xrv preprint ar Xiv:22300.000 00770, 2023.
[280] Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernon- court, Tong Yu, Ruiyi Zhang, and Nesreen K. Ahmed. Bias and fairness in large language models: A survey, 2024. URL https: http:://xxvvvvv.vaaaae wrg/ httpaabb//3300000 00770.
[281] Rohit Gandikota, Joanna Materzynska, Jaden Fiolto-Kaufman, and David Bau. Erasing concepts from diffusion models. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 2426-2436, 2023.
[282] Rohit Gandikota, Sheridan Foucht, Samuel Marks, and David Bau. Erasing conceptual knowledge from language models. arXiv preprint 2XXv:22400002600 2024.
[283] Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, et al. Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned. ArXiv preprint, abs/ abs/2209.07858, 2022. URL
https://arxiv.orgg http://bbs/2200.0002
[284] Feng Gao, Liangzhi Shi, Shenao Zhang, Zhaoran Wang, and Yi Wu. Adaptivo-gradient policy opti- mization: Enhancing policy learning in non-smooth differentiable simulations. In Forty-first Interna- tional Conference Machine Learning.
106

---

Published in Transactions on Machine Learning Research (10/2025)
[285] Peng Gao, Jiaming Han, Renrui Zhang, Ziyi Lin, Shijie Geng, Aojun Zhou, Wei Zhang, Pan Lu, Conghui He, Xiangyu Yue, Hongsheng Li, and Yu Qiao. Llama-adapter v2: Parameter-effiient visual instruction model, 2023.
[286] Tianyu Gao, Adam Fisch, and Danqi Chen. Making Pre-trained Language Models Better Few-shot Learners. In Annual Meeling of the Associalion for Computational Linguistics (ACL), pp. 3816 3830, 2021.
[287] Yu Gao, Lixue Gong, Qiushan Guo, Xiaoxia Hou, Zhichao Lai, Fanshi Li, Liang Li, Xiaochen Lian, Chao Liao, Liyang Liu, et al. Seedream 3.0 technical report. arXiv preprint ar arXiv:2504. 11346, 2025.
[288] Yu Gao, Haoyuan Guo, Tuyen Hoang, Weilin Huang, Lu Jiang, Fangyuan Kong, Huixia Li, Jiashi Li, Liang Li, Xiaojie Li, et al. Scedance 1.0: Exploring the boundaries of video generation models. arXiv preprint arXiv:2006.0 (ix::250600913,, 2025.
[289] Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang. Rotrioval-angmented generation for large language models: A survey, orXiv preprint ar Xiv:2331. 10997, 2023.
[290] Claire Gardent, Anastasia Shimorina, Shashi Narayan, and Laura Perez-Beltrachini. Creating train- ing corpora for nlg micro-planning. Tn 55th anmual meeting of the Association for Computational Linguisties (ACL), 2017.
[291] Aparna Garimella, Rada Mihalcea, and Akhash Amarnath. Demographic-aware language model fine-tuning bias mitigation technique. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference Natural Language Processing, pp. 311-319, 2022.
[292] Yingqiang Ge, Wenyue Hua, Kai Mci, Juntao Tan, Shuyuan Xu, Zelong Li, Yongfeng Zhang, et al. Openagi: When Ilm meets domain experts. Advances in Neural Information Processing Systems, 36, 2024.
[293] Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith. Re- altoxicityprompts:: Evaluating neural toxic degeneration in language models. arXiv preprint arXiv:2009. 1 11462, 2020.
[294] Sebastian Gehrmann, Hendrik Strobelt, and Alexander M. Rush. Glt:: Statistical detection and visualization of generated text. In Annual Meeting of the Association for Computational Linguistics, 2019. URL https:
/ httpp/ppp..aaaicccchhllllaa www./oorpssoDDD...... 182952848.
[295] Yonatan Geifman and Ran El-Yaniv. Selective classification for deep neural networks, 2017.
[296] Yonatan Geifman and Ran Yaniv,, Selectivenet: A deep neural network with an integrated reject option, 2019.
[297] Walter Gerych, Haoran Zhang, Kimia Hamidieh, Fileen Pan, Maanas Sharma, Thomas Hartvigsen, and Marzyeh Ghassemi. Bendvlm: Test-time debiasing of vision- languala embeddings, 2024. URL https://arxx htta//axvvvorr/reee//// hts//bbs/2411 04420.
[298] Mor Geva, Roci Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers are key-value memories. arXie preprint orXtv:201..14913, 2020.
[299] Mor Geva, Avi Caciularu, Kevin Ro Wang, and Yoav Goldberg. Transformer feed-forward layers build predictions by promoting concepts the vocabulary space. arXiv preprint arXiv:2203. 2022.
[300] Reza Ghacini, Xiaoli Fern, Hamed Shahbazi, and Prasad Tadopalli. Saliency learning: Teaching the model where to pay attention. In NAACL-HLT (1), pp. 4016-4025. Association for Computational Linguistics, 2019.
107

---

Published in Transactions on Machine Learning Research (10/2025)
[301] Sahra Ghalebikesabi, Leonard Berrada, Sven Gowal, Ira. Ktena, Robert Stanforth, Jamie Hayes, Soham De, Samuel I. Smith, Olivia Wiles, and Borja Balle. Differentially private diffusion models generate useful synthetic images. arXiv preprint arXiw:2302.1888 2023.
[302] Taisiya Glushkova, Chrysoula Zerva, Ricardo Rei, and André F. Т. Martins. Uncertainty-aware ma- chine translation evaluation. In Findings of the Association for Compulational Länguishies: EMNLP 2021. Association for Computational Linguistics, 2021. doi: 1..18633/v1/2022 www.0/2...iinnnnnmnniiiia
URL http://dx. www..ddoi.ooooo wwwp.ddoo..o/o.... 18653/v1/202/ http///........n......... Sindings-emnl] 330.
[303] Yoav Goldberg, Assessing bert's syntactic abilities. CoRR, abs/ 1901.05287, 2019.
[304] Eric Goldman. An introduction to the california consumer privacy act (cepa). Santa Clara Unio. Legal Studies Research Paper, 2020.
[305] Tao Gong, Chengqi Lyu, Shilong Zhang, Yudong Wang, Miao Zheng, Qian Zhao, Kuikun Liu, Wenwei Zhang, Ping Luo, and Kai Chen. Multimodal-gpt: A vision and language model for dialogue with humans, 2023.
[306] Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
[307] lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprinl arXiv:1412. 12..6772. 2014.
[308] Google. Gemini ai video generator powered by veo 3. eneration/.
[309] Google. Gemini: a family of highly capable multimodal models. or preprint arXiv:2312.11805, 2023.
[310] Google. Gemini2, 2024/: 2024.
[311] Google. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. aiv preprint 244:::2403.0533 2024.
[312] Google. Gemini 2.5 pro: Our most intelligent ai model, March 2025. URL https://blog google/ technology/gooe 6, 2025.
Accessed: April
[313] Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. Critic: Large language models can self-correct with tool-interactive critiquing. arXiv preprint arXim:2305.11 117398 2023.
[314] Alex Graves, Long short- tert-term memory. Supervised sequence labelling with recurrent neural networks, pp. 37-45, 2012.
[315] David Greenfield and Shivan Bhavnani. Social media: Generative ai could harm mental health. Nature, 617(7962):676, 2023.
[316] Authony G Greenwald, Debbie E McGhee, and Jordan LK Schwartz. Measuring individual differences in implicit cognition: the implicit association test. Journal of personality and social psychology, 74 (6):1464, 1998.
[317] Roger B. Grosse, Juhan Bae, Cem Anil, Nelson Elhage, Alex Tamkin, Amirhossein Tajdini, Benoit Steiner, Dustin Li, Esin Durmus, Ethan Perez, Evan Hubinger, Kamile Lukosiute, Karina Nguyen, Nicholas Joseph, Sam McCandlish, Jared Kaplan, and Samuel R. Bowman. Studying large language model generalization with influence functions. CoRR, abs/2308.03296, 2023.
108

---

Published in Transactions on Machine Learning Research (10/2025)
[318] Cornelia Gruber, Patrick Oliver Schenk, Malte Schierholz, Frauke Kreuter, and Göran Kauermann. Sources of uncertainty in machine learning-a statisticians' view. arXie preprint orXim:2305. 16703, 2023.
[319] Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.0073: 2023.
[320] Jindong Gu. Responsible generative ai: What to generate and what not. arXiv preprint arXioo2404.005 054.8883 2024.
[321] Jindong Gu, Zhen Han, Shuo Chen, Ahmad Beirami, Bailan He, Gengyuan Zhang, Ruotong Liao, Yao Qin, Volker Tresp, and Philip Torr. systematic survey of prompt engincering on vision-laguuaee foundation models. arXiv preprint arXiv:2307. 12980, 2023.
[322] Jindong Gu, Xiaojun Jia, Pau de Jorge, Wenqian Yu, Xinwei Liu, Avery Ma, Yuan Xun, Anjun Hu, Ashkan Khakzar, Zhijiang Li, Xiaochun Cao, and Philip Torr. A survey on transferability of adversarial examples across deep neural networks. Transactions on Machine Learning Research, 2024.
ISSN 2835-8856. URL https: / // /ppnreviereeee weeenreriee.nettfoouuttteeeeee rett/forumm/dd-NX13Bn
[323] Xiangming Gu, Xiaosen Zheng, Tianyu Pang, Chao Du, Qian Liu, Ye Wang, Jing Jiang, and Min Lin. Agent smith: A single image can jailbreak one million multimodal 1lm agents exponentially fast. In ICML, 2024.
[324] Nuno M Guerreiro, Elena Voita, and André FT Martins. Looking for a needle in a haystack: A com- prehensive study of hallucinations in neural machine translation. arXiv preprint orXiv:2205.05300 2022.
[325] Caglar Gulechre, Tom Le Paine, Srivatsan Srinivasan, Ksenia Konyushkova, Lotte Weerts, Abhishek Sharma, Aditya Siddhant, Alex Ahern, Miaosen Wang, Chenjie Gu, et al. Reinforced self-training (rest) for language modeling. ArXio preprint, als//308.08996, 2023. URL https://arxiv attp:////xxvvoo.abbbbbb $82908.08998. 08998.
[326] Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, et al. Textbooks are all you need. arXiv preprint arXiv:2806. 11644, 2023.
[327] Anish Gunjal, Jihan Yin, and Erhan Bas. Detecting and preventing hallucinations in large vision language models. Arxiv, abs/ has/2208.000 06394, 2023. URL bstps://apte semanticacholar. arg/Corpus ID: 260888272
[328] Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. How close is chatgpt to human experts? comparison corpus, evaluation, and detection. ArXiv, abs/2301.07555 2023. URL
http:/ppp..smmnicccchhlllal org/CorppsID: 2555999637.
[329] Chuan Guo, Gcoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks, 2017.
[330] Chuan Guo, Alexandre Sablayrolles, Hervé Jégou, and Douwe Kiela. Cradient-based adversarial atlacks against lext transformers. arXiv preprint arxiv:2104 13733, 2021.
[331] Dong Guo, Faming Wu, Feida Zhu, Fuxing Leng, Guang Shi, Haobin Chen, Haoqi Fan, Jian Wang, Jianyu Jiang, Jiawei Wang, Jingji Chen, Jingjia Huang, Kang Lei, Liping Yuan, Lishu Luo, Pengfei Liu, Qinghao Ye, Rui Qian, Shen Yan, Shixiong Zhao, Shuai Peng, Shuangye Li, Sihang Yuan, Sijin Wu, Tianheng Cheng, Weiwei Liu, Wenqian Wang, Xianhan Zeng, Xiao Liu, Xiaobo Qin, Xiaohan Ding, Xiaojun Xiao, Xiaoying Zhang, Xuanwei Zhang, Xuehan Xiong, Yanghua Peng, Yangrui Chen, Yanwei Li, Yanxu Hu, Yi Lin, Yiyuan Hu, Yiyuan Zhang, Youbin Wu, Yu Li, Yudong Liu, Yue Ling, Yujia Qin, Zanbo Wang, Zhiwu Hc, Aoxue Zhang, Bairen Yi, Bencheng Liao, Can Huang, Can Zhang, Chaorui Deng, Chaoyi Deng, Cheng Lin, Cheng Yuan, Chenggang Li, Chenhui Gou, Chenwei Lou, Chengzhi Wei, Chundian Liu, Chunyuan Li, Deyao Zhu, Donghong Zhong, Feng Li, Feng Zhang, Gang
109

---

Published in Transactions on Machine Learning Research (10/2025)
Wu, Guodong Li, Guohong Xiao, Haibin Lin, Haihua Yang, Haoming Wang, Heng Ji, Hongxiang Hao, Hui Shen, Huixia Li, Jiahao Li, Jialong Wu, Jianhua Zhu, Jianpeng Jiao, Jiashi Feng, Jiaze Chen, Jianhui Duan, Jihao Liu, Jin Zeng, Jingqun Tang, Jingyu Sun, Joya Chen, Jun Long, Junda Feng, Junfeng Zhan, Junjic Fang, Junting Lu, Kai Hua, Kai Liu, Kai Shen, Kaiynan Zhang, Ke Shen, Ke Wang. Keyu Pan, Kun Zhang, Kunchang Li, Lanxin Li, Lei Li, Lei Shi, Li Han, Liang Xiang, Liangqiang Chen, Lin Chen, Lin Li, Lin Yan, Liying Chi, Longxiang Liu, Mengfei Du, Mingxuan Wang, Ningxin Pan, Peibin Chen, Pengfei Chen, Pengfei Wu, Qingging Yuan, Qingvao Shuai, Qiuyan Tao, Renjie Zheng, Renrui Zhang, Ru Zhang, Rui Wang, Rui Yang, Rui Zhao, Shaoqiang Xu, Shihao Liang, Shipeng Yan, Shu Zhong, Shuaishuai Cao, Shuangzhi Wu, Shufan Liu, Shuhan Chang, Songhua Cai, Tenglong Ao, Tianhao Yang, Tingting Zhang, Wanjun Zhong, Wei Jia, Wei Weng, Weihao Yu, Wenhao Huang, Wenjia Zhu, Wenli Yang, Wenzhi Wang, Xiang Long, XiangRui Yin, Xiao Li, Xiaolei Zhu, Xiaoying Jia, Xijin Zhang, Xin Liu, Xinchen Zhang, Xinyu Yang, Xiongcai Luo, Xiuli Chen, Xuantong Zhong, Xuefeng Xiao, Xujing Li, Yan Wu, Yawci Wen, Yifan Du, Yihao Zhang, Yining Ye, Yonghui Wu, Yu Liu, Yu Yue, Yufeng Zhou, Yufeng Yuan, Yuhang Xu, Yuhong Yang, Yun Zhang, Yunhao Fang, Yuntao Li, Yurui Ren, Yuwen Xiong, Zehua Hong, Zehua Wang, Zewei Sun, Zeyu Wang, Zhao Cai, Zhaoyue Zha, Zhecheng An, Zhehui Zhao, Zhengzhuo Xu, Zhipeng Chen, Zhiyong Wu, Zhuofan Zheng, Zihao Wang, Zilong Huang, Ziyu Zhu, and Zuquan Song. Sced1.5-vl technical report, 2025. URL pestps://arxivy www.ai.wr//..........e..e.oooere htgg//b//// 2t///2555000000 07062.
[332] Qingyan Guo, Rui Wang, Junliang Guo, Bei Li, Kaitao Song, Xu Tan, Guoqing Liu, Jiang Bian, and Yujiu Yang. Connecting large language models with evolutionary algorithms yields powerful prompt optimizers. arXiv preprint Xiv: 230.00339, 2023.
[333] Shangmin Guo, Biao Zhang, Tianlin Liu, Tianqi Liu, Misha Khalman, Felipe Sinares-López Alexan- dre Ramé, Thomas Mesnard, Yao Zhao, Bilal Piot, Johan Ferret, and Mathieu Blondel. Direct Language Model Alignment from Online AI Feedback. arXiv, abs/2402.047922 04792, 2024.
[334] Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V Chawla, Olaf Wiest, and Xiangliang Zhang. Large language model based multi-agents: A survey of progress and challenges. Xrii preprint or arXiv:2402.01660 01660 2024.
[335] Yue Guo, Yi Yang, and Ahmed Abbasi. Auto-debias: Debiasing masked language models with automated biased prompts. In Proceedings of the 60th Annual Meeting of the Association for Com- putational Linguistics (Volume 1: Long Papers), pp.
2022.
[336] Yufei Guo, Muzhe Guo, Juntao Su, Zhou Yang, Mengqiu Zhu, Hongfei Li, Mengyang Qiu, and Shuo Shuo Liu. Bias in large language models: Origin, evaluation, and mitigation, 2024. URL
10915.
[337] Zhifang Guo, Yichong Leng, Yihan Wu, Sheng Zhao, and Xu Tan. Prompitis: Controllable text-to- speech with text descriptions. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-3. IEEE, 2023.
[338] Neha Gupta, Harikrishna Narasimhan, Wittawat Jitkrittum, Ankit Singh Rawat, Aditya Krishna Menon, and Sanjiv Kumar. Language model cascades: Token-level uncortainty and beyond, 2024.
[339] Wes Gurnee, Neel Nanda, Matthew Pauly, Katherine Harvey, Dmitrii Troitskii, and Dimitris Bertsimas. Finding nourons in a haystack: Case studics with sparse probing. arXiv preprint ar arXiv:2305.01611 01610, 2023.
[340] Michael Hahn and Navin Goyal. A theory of emergent in-context learning as implicit structure induction, 2023.
[341] Karen Hambardzumyan, Hrant Khachatrian, and Jonathan May. Warp: Word-level adversarial re- programming. arXiv preprint arXiv:2101.00111, 2021.
[342] Abhimanyu Hans, Avi Schwarzschidd, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Spotting llms with binoculars: Zero-shot detection of machine-generated text. Xix preprint or (00:24001.2000 2024.
110

---

Published in Transactions on Machine Learning Research (10/2025)
[343] Abhimanyu Hans, Yuxin Wen, Neel Jain, John Kirchenbauer, Hamid Kazemi, Prajwal Singhania, Siddharth Singh, Gowthami Somepalli, Jonas Geiping, Abhinav Bhatele, et al. Be like goldfish, don't memorize! mitigating memorization in generative lims. arXiv preprint ar 10209, 2024.
[344] Shibo Hao, Tianyang Liu, Zhen Wang, and Zhiting Hu. Toolkengtt: Augmenting frozen language models with massive tools via tool embeddings. Advances in neural information processing systems, 36, 2024.
[345] Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in a continuous latent space. arXiv preprinl orXim:242..6676. 2024.
[346] Yaru Hao, Li Dong, Furu Wei, and Ke Xu. Self-attention attribution: Interpreting information interactions inside transformer. In AAAI, pp. 12963-12971. AAAI Press, 2021.
[347] Thomas Hartvigsen, Swami Sankaranarayanaa, Hamid Palangi, Yoon Kim, and Marzyeh Ghassemi. Aging with grace: Lifelong model editing with discrete key-value adaptors. ArXiv, abs/2211.1103.1 2022. URL httpss//apl..mmaaicccoooo semanticccccnn org/ Letg/CrrpusDD--e CorpusID:2537354225 253735429.
[348] Peter Hase, Mona T. Diab, Asli Celikyilmaz, Xian Li, Zornitsa Kozareva, Veselin Stoyanov, Mohit Bansal, and Srini Iyer. Methods for measuring, updating, and visualizing factual beliefs in language models. In Conference of the European Chapter of the Association for Computational Linguistics, 2023. URL http::::/app...mmatttt..
oogpol/arrrrro//oooooeeo wogg www.ooorggssseeeee Corpps 258378150.
[349] T. Hastic, R. Tibshirani, and J.H. Friodman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer series in statistics. Springer, 2001. ISBN 9780387952840. URL https://books. http:/bboos...ggll..eee
google.ccm//oolno wttboooookkkk//ooooooaaaa VRa
[350] Nan He, Hanyu Lai, Chenyang Zhao, Zirui Cheng, Junting Pan, Ruoyu Qin, Ruofan Lu, Rui Lu, Yunchen Zhang, Gangming Zhao, Zhaohui Hou, Zhiyuan Huang, Shaoqing Lu, Ding Liang, and Mingjie Zhan. Teacherlm: Teaching to fish rather than giving the fish, language modeling likewise, 2023.
[351] Zexue He, Yu Wang, Julian J. McAuley, and Bodhisattwa Prasad Majumder. Controlling bias ex- posure for fair interpretable predictions. In EMNLP (Findings), pp. 5854-5866 Association for Computational Linguisties, 2022.
[352] Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Stein- hardt. Aligning ai with shared human values, 2020.
[353] Dan Hendryeks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical analysis of out-of-distribution generalization. In Proceedings of the IEEE/CV international conference on computer vision, pp. 8340-83499 2021.
[354] John Hewitt and Christopher D. Manning. A structural probe for finding syntax in word representa- tions. In NAACL-HLT (1), pp. 1129-4138. Association for Computational Linguistics, 2019.
[355] John Hewitt and Christopher D. Manning, A Structural Probe for Finding Syntax in Word Repre- sentations. In Proceedings of the 2019 Conference of the North American Chapler of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4129-4138, Minncapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10.18653/v 11/55/3//111111111. URL https://aclanthology wwgg/oogg//0. N19199
[356] John Hewitt, Sarah Chen, Lanruo Lora Xie, Edward Adams, Percy Liang, and Christopher D Man- ning. Model editing with canonical examples. arXiv preprint arXiv:2,02.06155, 02.06155, 2024.
[357] Yusuke Hirota, Jerone TA Andrew, Dora Zhao, Orestis Papakyriakopoulos, Apostolos Modas, Yuta Nakashima, and Alice Xiang. Resampled datasets are not enough: Mitigating societal bias beyond single attributes. arxiv preprint 0xxx:2407.0362, 2024.

---

Published in Transactions on Machine Learning Research (10/2025)
[358] Jonathan Ho, Ajay Jain, and Picter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-688 08500068811--0 2020.
[359] Jordan Hoffmann, Scbastian Borgeand, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Ruther- ford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute- optimal large language models. arviv preprinl or Xrv::22033 15556, 2022.
[360] Jiwoo Hong, Noah Lee, and James Thorne Orpo: Monolithic preference optimization without refer- ence model. arXie preprint XXxx2443300769, 2(4):5, 2024.
[361] Rachel Hong, William Agnew, Tadayoshi Kohno, and Jamic Morgenstern. Who's in and who's out? case study multimodal clip-fittrrng in datacomp, 2024.
[362] Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, ct al. Mctagpt: Meta programming for multi-agent collaborative framework. arXiv preprinl Xrx::2230.. 00352, 2023.
[363] Yan Hong and Jianfu Zhang. Wildfake: A large-scale challenging dataset for ai- ai-generated images detection. arXio preprint or Xiv:22400 11843, 2024.
[364] Ye Hongbin, Liu Tong, Zhang Aijia, Hua Wei, and Jia Weiqiang, Cognitive mirage: A review of hallucinations in large language models. arXiv, 2023.
(365) Benjamin Hoover, Hendrik Strobelt, and Sebastian Gehrmann. exbert: A visual analysis tool to explore learned representations in transformer models. In ACL (demo), pp. 187-196. Association for Computational Linguisties, 2020.
[366] Abe Bohan Hou, Jingyu Zhang, Tianxing He, Yichen Wang, Yung-Sung Chuang, Hongwei Wang, Lingfeng Shen, Benjamin Van Durme, Daniel Khashabi, and Yulia Tsvetkov. Semstamp: A semantic watermark with paraphrastic robustness for text generation. ArXiv, abs/2310.0330 2023. URL https://api.net http::///pp..emmttiiccccce.o www.sti sog.aatttccccol....oooo/ooooooao
[367] Bairu Hou, Yujian Liu, Kaizhi Qian, Jacob Andreas, Shiyu Chang, and Yang Zhang, Decomposing uncertainty for large language models through input clarification ensembling, 2023.
[368] Yifan Hou, Jiaoda Li, Yu Fei, Alessandro Stolfo, Wangchunshu Zhou, Guangtao Zeng, Antoine Bosse- lut, and Mrinmaya Sachan. Towards a mechanistic interpretation of multi-step reasoning capabilities of language models. In Houda Bouamor, Juan Pino, and Kalika. Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 4902 4919, Singapore, Decem- ber 2023. Association for Computational Linguistics. doi: 00.16633vv/220 2023 http://2222........ emnlp- main.299. URL 10:0:02...mmmmmmeeememmm mamlp-main.2. 299//
[369] Genc Hoxha, Giacomo Scuccato, and Farid Melgani. Improving image captioning systems with post- processing stratogics. JEEE Transactions on Geoscience and Remote Sensing, 2023.
[370] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, arXie preprint arXiv:2106. 09685, 2021.
[371] Jian Hu. Reinforce++: simple and efficient approach for aligning large language models, 2025.
URL
[372] Michael Y. Hu, Angelica Chen, Naomi Saphra, and Kyunghyun Cho. Latent state models of training dynamics, 2023.
[373] Pingyi Hu, Zihan Wang, Ruoxi Sun, Hu Wang, and Minhui Xue. M 4 i: Multi-modal models mem- bership inference. Advances in Neural Information Processing Systems, 35:1867-1882 2022.
112

---

Published in Transactions on Machine Learning Research (10/2025)
[374] Shengding Hu, Ning Ding, Huadong Wang, Zhiyuan Liu, Jingang Wang, Juanzi Li, Wei Wu, and Maosong Sun. Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2225-2240, 2022.
[375] Xiaobing Hu, Pinyu Chen, and Tsung- Yi Ho. Radar: Robust ai-text detection via adversarial learning. Xin, abs//237.008888 2023. URL http://ppp.smmm..icccclllaaaarr. org/ Sergg/CorpussD] 2595001882
[376] Zhengmian Hu, Lichang Chen, Xidong Wu, Yihan Wu, Hongvang Zhang, and Heng Huang. Unbiased watermark for large language models. ArXiv, scholar. CaailaarrrrCCoopusIIi (C 264172471.
2023. URL bctps://apis semantic
[377] Zhiting Hu, Zichao Yang, Xiaodan Liang. Ruslan Salakhutdinov, and Eric P. Xing. Toward controlled generation of text, 2018. URL https:// arxiv. org/ /10.. 00955.
[378] Alyssa Huang, Peihan Liu, Ryumei Nakada, Linjun Zhang, and Wanrong Zhang, Safeguarding data in multimodal ai: A differentially private approach to clip training, arXie preprint arXiv:2306.08/ 2023.
[379] Jie Huang and Kevin Chen-Chuan Chang. Towards reasoning in large language models: A survey. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Findings of the Association for Computational Linguisties: ACL 2023, pp. 1049-1065, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 1..18633/v1/2022 finna..iiinngnll.... URL https://aclanttholog) rg/2023. f findings-acl..
[380] Jie Huang, Hanyin Shao, and Kevin Chen-Chuan Chang. Are large pre-trained language models leaking your personal information? arXiv preprint arXiv:2205. 0000051222222 12628, 2022.
[381] K Huang, G Song, Hanwen Su, and Jiyan Wang. Out-of-distributiio detection using peer-class generated by large language model. arXie preprint aiv: 13322, 2024.
[382] Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2i-compbench: A comprehensive benchmark for open-world compositional text-to-image generation. Advances in Neurol Information Processing Systems, 36:78723 36:78723-78747, 2023.
[383] Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. A survey on hallucination in large language models: Principles, laxonomy, challenges, and open questions. ACM Transaclions on Information Systems, 43(2):1-55 55, 2025.
[384] Minlie Huang, Xiaoyan Zhu, and Jianfeng Gao. Challenges in building intelligent open-domain dialog systems. ACM Transactions on Information Systems (TOIS), 3833):1332, 2020.
[385] Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu. Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and ceapoppcciionalloatttin arXiv preprint arXiv:2311. 199111 2023.
[386] Qihan Huang, Jie Song, Mengqi Xue, Haofei Zhang, Bingde Hu, Huiqiong Wang, Hao Jiang, Xingen Wang, and Mingli Song. LG-CAV: Train any concept activation vector with language guidance. In NewrIPS, 2024.
[387] Qingging Huang, Daniel S Park, Tao Wang, Timo I Denk, Andy Ly, Nanxin Chen, Zhengdong Zhang, Zhishuai Zhang, Jiahui Yu, Christian Frank, et al. Noise2music: coct-conditioned music generation with diffusion models. arXiv preprint XrXi::2302. 03917, 2023.
[388] Quzhe Huang, Shengqi Zhu, Yansong Feng, and Dongyan Zhao. Exploring distantly-labeled rationales in neural network models. In ACL/IJCNLP (1), pp. 5571-5582. Association for Computational Linguistics, 2021.
113

---

Published in Transactions on Machine Learning Research (10/2025)
[389] Shuo Huang, William MacLean, Xiaoxi Kang, Qiongkai Xu, Zhuang Li, Xingliang Yuan, Gholamreza Haffari, and Lizhen Qu. Nap 2: A benchmark for naturalness and privacy-preserving text rewriting by learning from human. arXiv preprint 251::2400000744,, 2024.
[390] Xinmeng Huang, Shno Li, Mengxin Yu, Mattco Sesia, Hamed Hassani, Insup Lec, Osbert Bastani, and Edgar Dobriban. Uncertainty in language models: Assessment through rank-calibration, 2024.
[391] Yue Huang, Qihui Zhang, Lichao Sun, et al. Trustgpt: A benchmark for trustworthy and responsible large language models. ArXiv preprint, 0ass2306.111000 2023. URL https: http://aawxi...rab////////s// .11577.
[392] Yue Huang, Lichao Sun, Haoran Wang, Siyuan Wu, Qihui Zhang, Yuan Li, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, et al. Position: Trustllm: Trustworthiness in large language models. In International Conference on Machine Learning (ICML), 2024.
[393] Yue Huang, Chujic Gao, Siyuan Wu, Haoran Wang, Xiangqi Wang, Yujun Zhou, Yanbo Wang, Jiayi Ye, Jiawen Shi, Qihui Zhang, Yuan Li, Han Bao, Zhaoyi Liu, Tianrui Guan, Dongping Chen, Ruoxi Chen, Kehan Guo, Andy Zou, Bryan Hooi Kuen- Yew, Caiming Xiong, Elias Stengel-Eskin, Eskin, Hongyang Zhang, Hongzhi Yin, Huan Zhang, Huaxiu Yao, Jachong Yoon, Jieyu Zhang, Kai Shu, Kaijie Zhu, Ranjay Krishna, Swabha Swayamdipta, Taiwei Shi, Weijia Shi, Xiang Li, Yiwei Li, Yuexing Hao, Zhihao Jia, Zhize Li, Xiuying Chen, Zhengzhong Tu, Xiyang Hu, Tianyi Zhou, Jieyu Zhao, Lichao Sun, Furong Huang, Or Cohen Sasson, Prasanna Sattigeri, Anka Reuel, Max Lamparth, Yue Zhao, Nouha Dziri, Yu Su, Huan Sun, Hong Ji, Chaowei Xiao, Mohit Bansal, Nitesh V. Chawla, Jian Pei, Jianfeng Gao, Michael Backes, Philip Yu, Neil Zhengiang Gong, Pin-Yu Chen, Bo Li, Dawn Song, and Xiangliang Zhang. On the trustworthiness of generative foundation models: Guideline, assessment, and perspective, 2025.
[394] Yumpeng Huang, Yaonan Gu, Jingwei Xu, Zhihong Zhu, Zhaorun Chen, and Xiaoxing Ma. Securing reliability: A briel overview enhancing in-context learning for foundation models. arXiv preprinl arXix:2402.11677 2024.
[395] Zeyu Huang, Yikang Shen, Xiaofeng Zhang, Jie Zhou, Wenge Rong, and Zhang Xiong. Transformer- patcher: One mistake worth one ncuron. ArXiv, abs/2301.09785, 2023. URL https: h/aps:///pp... semant icscholar.org wrt./orpss/ri.D....... 2561699436..
[396] Dieuwke Hupkes, Mario Giulianelli, Verna Dankers, Mikel Artetxe, Yanai Elazar, Tiago Pimentel, Christos Christodoulopoulos, Karim Lasri, Naomi Saphra, Arabella Sinclair, et al. A taxonomy and review of generalization research in nlp, Nature Machine Intelligence, 5(10):1161 1174, 2023.
[397] Eyke Hüllermeier and Willem Waegeman. Alcatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods. Machine Learning, 110(3):457-5000 506, March 2021. ISSN 1573- 0565. doi: 10.1007/11009944001000066 URL http: hwwp:/xxxxdoii.oo//o/////// www.d/o....ooogoooo 1111111//119990000--00010111
[398] Timour lgamberdiev and Ivan Habernal. Dp- for privatized text rewriting under local differential privacy. arXiv preprint orXiv:2302.06638 2023.
[399] Shima Imani, Liang Du, and Harsh Shrivastava. Mathprompter: Mathematical reasoning using large language models. ArXiv preprint, abs/2303.05399 2023. URL
05398.
[400] Alvi Md Ishmam and Christopher Thomas. Semantic shield: Defending vision-language models against backdooring and poisoning via fine-grained knowledge alignment. In Proceedings of the IEEF/CVE Conference on Compuler Vision and Pallern Recognition, pp. 24820-2222 2490-22,,0, 2024.
[401] Khawar Islam, Muhammad Zaigham Zaheer, Arif Mahmood, and Karthik Nandakumar. Dif- fusemii: Tabel-preserving data augmentation with diffusion models. In 2024 IEEE/CVE Confer- ence on Computer Vision and Pattern Recognition (CVPR), pp. 27611-27620. IEEE, June 2024. doi: 10.1 009//c/p522733.222.... 200.552233.202.0000 0.6008.. URL http: org/ 1ttp////02..00/////222233 2024. 02608.
114

---

Published in Transactions on Machine Learning Research (10/2025)
[402] Takuya Ito, Soham Dan, Mattia Rigotti, James Kozloski, and Murray Campbell. On the generalization capacity of neural networks during generic multimodal reasoning. arXiv preprint or orXim:2401. 15030, 2024.
[403] Alon Jacovi and Yoav Goldberg, Towards Faithfully Interpretable NLP Systems: How Should We Define and Evaluate Faithfulness? In Proceedings of the 58th Anmual Meeling of the Association for Computational Linguistics, pp. 4198 4205, Online, July 2020. Association for Computational Linguistics. doi: 10. (0.18853/v1/20 wwtp202000000miim.mm.. URL pttps://aclatthllle hwwpehhooogoooogg///e org/2020 acl-mal 3.386.
[404] Abhyuday Jagannatha, Bhanu Pratap Singh Rawat, and Hong Yu. Membership inference attack susceptibilityy of clinical language models. ar preprint arxi::2104 08305, 2021.
[405] Neel Jain, Ping-yeh Chiang, Yuxin Wen, John Kirchenbauer, Hong-Min Chu, Gowthami Somepalli, Brian R Bartoldson, Bhavya Kailkhura, Avi Schwarzschild, Aniruddha Saha, al. Neftune: Noisy embeddings improve instruction finetuning. aiviv preprint orXiv:2310.0591, 2023.
[106] Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping-yeh Chi- ang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, and Tom Goldstein. Baseline defenses for adversarial attacks against aligned language models. arXiv preprint (XXi:2309.00614 2023.
[407] Sarthak Jain and Byron C Wallace. Attention is not explanation. arXiv preprint 0rXiv:1902.10190 2019.
[408] Joel Jang, Dongkeum Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Seo. Knowledge unlearning for mitigating privacy risks in language models. orXiv preprint arXiv:2210.01:3 015.2, 2022.
[409] Theo Jaunet, Corentin Kervadec, Romain Vuillemot, Grigory Antipov, Mocz Baccouche, and Chris- Lian Wolf. Visqa: X-raying vision and language reasoning in transformers. IEEE Trunsactions on Visualization and Computer Graphics, 28(1):976- 98700-98,--0 2021.
[410] Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1-38, 2023.
[411] Ziwei Ji, Lei Yu, Yeskendir Koishekenov, Yejin Bang, Anthony Hartshorn, Alan Schelten, Cheng Zhang, Pascale Fung, and Nicola Cancedda. Calibrating verbal uncertainty as a linear feature to reduce hallucinations, 2025.
[412] Jinyuan Jia, Yupei Liu, and Neil Zhenqiang Gong. BadEncoder: Backdoor attacks to pre-trained encoders in seperisue learning. In IEEE Symposium Security and Privacy, 2022.
[413] Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. Lim-blender: Ensembling large language models with pairwise ranking and generative fusion. arXiv preprint arXiv:2306.02561 2023.
[414] Shuyang Jiang, Yusheng Liao, Ya Zhang, Yu Wang, and Yanfeng Wang. Taia: Large language models are dutto--disttibutiinn data learners. arXiv preprint ar Xrxi:22455.0011 2024.
[415] Zhengping Jiang, Jingyu Zhang, Nathaniel Weir, Seth Ebner, Miriam Wanner, Kate Sanders, Daniel Khashabi, Anqi Liu, and Benjamin Van Durme. Core: Robust factual precision scoring with infor- mative sub-claim identification, 2024.
[416] Haibo Jin, Ruoxi Chen, Jinyin Chen, and Haohan Wang. Quack: Automatic jailbreaking large language models via role-playing, 2024. URL https:// httppppwwww.....e/fooootteeeeeee
[417] Xisen Jin and Xiang Ren. What will my model forget? forecasting forgotten examples in language model refinement. Tn International conference on machine learning, 2024.
115

---

Published in Transactions on Machine Learning Research (10/2025)
[418] H. Jingnan. X's chatbot can now generate ai images. lack of guardrails raises election concerns. NPR, August 16 2024. URL trtprratttlllleee-------tttttnee
[419] Michael Jordan, Zoubin Ghahramani, Tommi Jaakkola, and Lawrence K Saul. An introduction to variational methods for graphical modcls. In Learning in graphical models, pp. 105-161. Springer, 1998.
[420] Brihi Joshi, Aaron Chan, Ziyi Liu, Shaoliang Nie, Maziar Sanjabi, Hamed Firooz, and Xiang Ren. Er-test: Evaluating explanation regularization methods for language models. In Findings of the Association for Computational Länguistics: EMNLP 2022, pp. 3315-3336, 2022.
[421] Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Hemighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatficl--oodds, Nova DasSarma, Eli Tran-Johnson, Scott Johnston, Sheer El-Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav Fort, Deep Ganguli, Danny Hernandez, Josh Jacobson, Jackson Kernion, Shauna Kravec, Liane Lovitt, Kamal Ndousse, Catherine Olsson, Sam Ringer, Dario Amodei, Tom Brown, Jack Clark, Nicholas Joseph, Ben Mann, Sam McCandlish, Chris Olah, and Jared Kaplan. Language models (mostly) know what they know, 2022.
[422] Adam Tauman Kalai and Santosh S. Vempala. Calibrated language models must hallucinate. In Proceedings of the 56th Annual ACM Symposium on Theory of Computing, STOC '24 pp. 160-171. ACM, June 2024. doi: 9777.
URL http://dx doi org/10. 1111/11111611...00 364
[423] Amita Kamath, Robin Jia, and Percy Liang. Selective question answering under domain shift, 2020.
[424] Masahiro Kancko and Danushka Bollegala. Unmasking the mask-ovaluating social biascs in masked language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 11954-11962, 2022.
[425] Haoqiang Kang, Juntong Ni, and Huaxiu Yao. Ever: Mitigating hallucination in large language models through real-time verification and rectification. arXiv preprint or Xiv:2311.091 2023.
[426] Mintong Kang, Nezihe Merve Gürel, Ning Yu, Dawn Song, and Bo Li. C-rag: Certified generation risks for retrieval-augmented language models. arXiv preprint 2rXiv:2402.03181 2024.
[427] Mintong Kang, Dawn Song, and Bo Li. Diffattack: Evasion attacks against iiffusion-basek adversarial purification. Advances in Neural Information Processing Systems, 36, 2024.
[428] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXix:2001.0 008361, 2020.
[429] Sanjay Kariyappa, Freddy Lécué, Saumitra. Mishra, Christopher Pond, Daniele Magazzeii and Manuela Veloso. Progressive inference: Explaining decoder-only sequence classification models using intermodiate predictions, 2024.
[430] Jungo Kasai, Kcisuke Sakaguchi, Ronan Le Bras, Akari Asai, Xinyan Yu, Dragomir Radev, Noah A Smith, Yejin Choi, Kentaro Inui, el al. Realtime qa: what's 's the answer right now? Advances in Neural Information Processing Systems, 36, 2023.
[431] Harmanpreet Kaur, Harsha Nori, Samuel Jenkins, Rich Caruana, Hanna Wallach, and Jennifer Wort- man Vaughan. Interpreting interpretability: understanding data scientists' use of interpretability tools for machine learning. In Proceedings of the 2020 CHll conference tuman factors in computing systems, pp. 1-14, 2020.
[432] Rémi Kazmierczak, Eloise Berthier, Goran Frehse, and Gianni Franchi. CLIP-QDA: an explainable concept bottleneck model. CoRR, abs/2311 hts/2212.001110 10, 2023.
116

---

Published in Transactions on Machine Learning Research (10/2025)
[433] Brendan Kennedy, Xisen Jin, Aida Mostafazadeh Davani, Morteza Dehghani, and Xiang Ren. Con- textualizing hate speech classifiers with post-hoc explanation. In ACI, pp. for Computational Linguistics, 2020.
Association
[434] Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. Generalization through memorization: Nearest neighbor language models. arXiv preprint 19X100911..01 2019.
[435] Daniel Khashabi, Shane Lyu, Sewon Min, Lianhui Qin, Kyle Richardson, Sean Welleck, Hannaneh Hajishirzi, Tushar Khot, Ashish Sabharwal, Sameer Singh, et al. Prompt waywardness: The curious case of discretized interpretation of continuous prompts. arXiv preprint (tt:2212.08944. 2021.
[436] Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, and Rory Sayres. Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV), June 2018. URL http://arxiv wttp://..rr//bb///11.1./
arXiv:1711.11279 [stat].
[437] Eunji Kim, Siwon Kim, Chaehun Shin, and Sungroh Yoon. De-stereotyping text-to-image models through prompt tuning. 2023.
[438] Gangwoo Kim, Sungdong Kim, Bycongguk Jeon, Joonsuk Park, and Jacwoo Kang. Tree of clarifica- lions:
Answering ambiguous questions with retrieval-augmente large language models, 2023.
[439] Jae Myung Kim, Koepke, Cordelia Schmid, and Zeynep Akata. Exposing and mitigating spurious correlations for cross-modal retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2584 2594, 2023.
[440] Jinhwa Kim, Ali Derakhshan, and Ian G. Harris. Robust safety classifier for large language models: Adversarial prompt shield. orXiv preprinl XXxx:331100011 2023.
[441] Siwon Kim, Sangdoo Yun, Hwaran Lee, Martin Gubri, Sungroh Yoon, and Seong Joon Oh. Propile: Probing privacy leakage in large language models. arXiv preprint arXiv:2307.018%/ 2023.
[442] Yujin Kim, Jachong Yoon, Sconghycon Ye, Sangmin Bac, Namgyu Ho, Sung Ju Hwang, and Se Soung Yun. Carpe diem: On the evaluation of world knowledge in lifelong language models. In The North American Chapter of the Association for Computational Linguistics, 2024.
[443] John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. In International Conference on Machine Learning, 2023. URL
org/CorpusID: 256194179.
[444] John Kirchenbauer, Jonas Geiping, Yuxin Wen, Manli Shu, Khalid Saifullah, Kezhi Kong, Kasun Fernando, Aniruddha Saha, Micah Goldblum, and Tom Goldstein. On the reliability of watermarks for large language models. ArXiv, 0bs/2306.04634 2023. URL
I 2559095643.
[445] Hannah Rose Kirk, Alexander Whitefield, Paul Rottger, Andrew M Bean, Katerina Margatina, Rafael Mosquera-Gomez, Juan Ciro, Max Bartolo, Adina Williams, He He, et al. The prism alignment dataset: What participatory, representative and individualised human feedback reveals about the subjective and multicultural alignment of large language models. Advances in Neural Information Processing Systems, 37:105236- 105344, 2024.
[446] Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, and Roberta Raileam.. Understanding the effects of rlhf on 1lm generalisation and diversity. arXiv preprint vrXiv:2310.06445 2023.
[447] Gregory Koch, Richard Zemel, Ruslan Salakhutdinov,, al. Siamese neural networks for one-shot image recognition. In ICML deep learning workshop, volume 2, pp. 1-30. Tille, 2015.
[448) Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions, 2017.
117

---

Published in Transactions on Machine Learning Research (10/2025)
[449] Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In ICMI, volume 119 of Proceedings of Machine Learning Research, pp. 5338 5348. PMLR, 2020.
[450] Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Bal- subramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, et al. Wilds: benchmark of in-the-wild distribution shifts. In International conference on machine learning, pp. 5637-5664. PMLR, 2021.
[451] Ryuto Koike, Masahiro Kaneko, and Naoaki Okazaki. Outfox: Lim-generated essay detection through in-context learning with adversarially generated examples. ArXiv, abs/ 0ts/2307.11122) 2023. URL https: //api.semanti www.:teemnniiiim i ccchhllarc wwwaaar..rrrooooe org/CorpusID) 260091573.
[452] Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Twasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35: 22199 22213, 2022.
[153] Euja Kokalj, Blaz Skrlj, Nada Lavrac, Senja Pollak, and Marko Robnik-Sikonja. BERT meets shapley: Extending SHAP explanations to transformer-base classifiers. In EACL (Hackashop), pp. 16 21. Association for Computational Linguistics, 2021.
[454] Dan Kondratyuk, Lijun Yu, Xiuye Gu, José Lezama, Jonathan Huang, Grant Schindler, Rachel Hormung, Vighnesh Birodkar, Jimmy Yan, Ming- Chang Chiu, et al. Videopoct: A large language model for zero-shot video generation. arXio preprint arxiv:2312, 2. 14125, 2023.
[455] Fei Kong, Jinhao Duan, RuiPeng Ma, Heng Tao Shen, Xiaoshuang Shi, Xiaofeng Zhu, and Kaidi Xu. An efficient membership inference attack for the diffusion model by proximal initialization. In The Twelfth International Conference on Learning Represenlations, 2024.
[456) Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangleng Xiong, Xin Li, Bo Wu, Jianwei Zhang, al. Hunyuanvideo: systematic framework for large video generative models. arXiv preprint orXiv:2412.0000 036603, 2024.
[457] Olga Kovaleva, Alexey Romanov, Anna Rogers, and Anna Rumshisky. Revealing the Dark Secrets of BERT, September 2019. URL http:///rxixi org/ abs//90s//10./..///// 08593. arXiv:1908.08599 [cs, stat].
[458] Felix Kreuk, Gabriel Synnaeve, Adam Polyak, Uriel Singer, Alexandre Défossez, Jade Copet, Devi Parikh, Yaniv Taigman, and Yossi Adi, Audiogen: Textually guided audio generation. arXiv preprint arXiv:2209. 15352, 2022.
[459] Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit lyyer. Paraphrasing evades detectors of ai-generated text, but retrieval is effective defense. ArXiv, abs/ //////33333300 2023. URL https: wew..s.seanicccoolla... org/ Cetg/orrpusD.... 2DDD:277774400
[460] Alex Krizhevsky, Πya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolu-
tional neural notworks. Advances in neural information processing systems, 25, 2012.
[461] Kuaishou. Kling. https://klingaa kuaishou. homishhu.coo/, 2024. Accessod: 2024-12-09.
[462] Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. Robust distortion-free watermarks for language models. ArXiv, abs/2307.15599, 2023. URL https: pttps://apiis semanticscho
lar. warroor//Coosllll 260315804.
[463] Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. Clam: Selective clarification for ambiguous ques- tions with generative language models, 2023.
[464] Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation, 2023.
118

---

Published in Transactions on Machine Learning Research (10/2025)
[465] Aounon Kumar, Chirag Agarwal, Suraj Srinivas, Soheil Feizi, and Hima Lakkaraju. Certifying Ilm safety against adversarial prompting. arXiv preprint wrXi:2339.027000 2023.
[466]
Bhawesh Kumar, Charlie Lu, Gauri Gupta, Amil Palepu, David Bellamy, Ramesh Raskar, and Andrew Beam. Conformal prodiction with large language models for multi-choice question answering, 2023.
[467] Surender Suresh Kumar, M.L. Cummings, and Alexander Stimpson. Strengthening Ilm trust bound- aries: A survey of prompt injection attacks surender suresh kumar dr. m.l. cummings dr. alexander stimpson. In 2024 IEFF 4th International Conference on Human-Machine Systems (TCHMS), 2024.
[468] Yogesh Kumar and Pekka Marttinen. Improving medical multi-modal contrastive learning with expert annotations. arXiv preprint or arXiv:2403.1013: 2024.
[469] Nupur Kumari, Bingliang Zhang, Sheng-Yt Yu Wang, Eli Shechtman, Richard Zhang, and Jun-Yan Zhu. Ablating concepts in extt-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 22691 -27002, 2023.
[470] Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yay Yan Zhu. Multi-concept customization of lext-to-image diffusion. In Proceedings of the IEEE/CVI Conference Compuler Vision and Pattern Recognition, pp. 1931-1941, 2023.
[471] Jenny Kunz and Marco Kuhlmann. Classifier probes may just learn from linear context features. Tn COLING, pp. 5136 5146. International Committee on Computational Linguistics, 2020.
[472] Keita Kurita, Nidhi Vyas, Ayush Pareck, Alan W Black, and Yulia Tsvetkov. Mcasuring bias in contextualized word representations. arXiv preprint or xiv: Kio:1906.,7337, 2019.
[473] Salem Lahlou, Moksh Jain, Hadi Nekoei, Victor Ion Butoi, Paul Bertin, Jarrid Rector- Rector-Brooks, Maksym Korablyov, and Yoshua Bengio. Deup: Direct epistemic uncertainty prediction, 2023.
[474] Wen Lai, Alexander Fraser, and Ivan Titov. Joint localization and activation editing for low-resource fine-tuning. arXiv preprint arXiv:2502.01 010009 2025.
[475] Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems, 30, 2017.
[476] Andrew Lampinen, Ishita Dasgupta, Stephanie Chan, Kory Mathewson, Mh Tessler, Antonia Creswell, James McClelland, Jane Wang, and Felix Hill. Can language models learn from expla- nations in context? In Findings of the Association for Computational Linguistics: EMNLP 2022, pp. 537-563, 2022.
[477] Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Her- nandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, et al. Measuring faithfulness in chain-of-thought reasoning. ar preprint arXiv:2307. 13702, 2023.
[478] Hugo Laurengon, Lucile Saulnier, Léo Tronchon, Stas Bekman, Amanpreet Singh, Anton Lozhkov, Thomas Wang, Siddharth Karamcheti, Alexander Rush, Douwe Kiela, ct al. Obelics: An open web- scale filtered dataset of interleaved image-text documents. Advances in Neural Information Processing Systems, 36, 2024.
[479] Thomas Lavergne, Tanguy Urvoy, and François Yvon. Detecting fake content with relative entropy scoring. In Pan, 2008. URL attps://apl. www.sticscooollatt...e.ee
caaaaatttscccolaaaooo/oorrrppppe org/Corpus : 12098535.
[480] Rémi Lebret, David Grangier, and Michael Auli. Neural text generation from structured data with application to the biography domain. arXiv preprint arXiv:1603.07711, 2016.
[481] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521 521 444, 2015.
119

---

Published in Transactions on Machine Learning Research (10/2025)
[482] Dong-Ho Lee, Akshen Kadakia, Brihi Joshi, Aaron Chan, Ziyi Liu, Kiran Narahari, Takashi Shibuya, Ryosuke Mitani, Toshiyuki Sekiya, Jay Pujara, and Xiang Ren. Xmd: An end-to-end framework for interactive explanation-baser debugging of nlp models, 2022.
[483] Harrison Lee, Samrat Phatale, Hassan Mansoor, Kellie Lu, Thomas Mesnard, Colton Bishop, Victor Carbune, and Abhinav Rastogi. Rlaif: Scaling reinforcement learning from human leedback with ai feedback. ArXio preprint, abs/2309.0000 2023. URL attps://arxia orggabs/2305 00267.
[484] Jaewoo Lee, Jaehong Yoon, Wonjae Kim, Yunji Kim, and Sung Ju Hwang. Stella: Continual audio- video pre-training with spatio-temporal localized alignment. In International Conference on Machine Learning, 2024.
[485] Katherine Lee, Orhan Firat, Ashish Agarwal, Clara Fannjiang, and David Sussillo. Hallucinations in neural machine translation. 2018. URL. https://api. semanticscholar www.hhoooo.oooooooooooooooooooet wwwg//rrpussDD.... 535933766.
[486] Taehyun Lee, Seokhee Hong, Jaewoo Ahn, llgee Hong, Hwaran Lee, Sangdoo Yun, Jamin Shin, and Gunhce Kim. Who wrote this code? watermarking for code gencration. ArXiu, abs/2305.15060 2023. URL http:://ap..eemamticcccooee
www.nti wrtp/orpps..
[487] Tony Lee, Michihiro Yasunaga, Chenlin Meng, Yifan Mai, Joon Sung Park, Agrim Gupla, Yunzhi Zhang, Deepak Narayanan, Hannah Teufel, Marco Bellagente, et al. Holistic evaluation of text-to- image models. Advances in Neural Information Processing Systems, 36, 2024.
[488] Tao Lei, Regina Barzilay, and Tommi S. Jaakkola. Rationalizing neural predictions. In EMNLP, pp. 107-117. The Association for Computational Linguistics, 2016.
[489] Jan Leike, David Krueger, Tom Everitt, Miljan Martic, Vishal Maini, and Shane Legg. Scalable agent aligmment via reward modeling: research direction. arXiv preprint arXiv:1811007811 2018.
[490] Jixuan Leng, Chengsong Huang, Banghua Zhu, and Jiaxin Huang. Taming overconfidence in llms: Reward calibration in rlhf. orXiv preprint 110.0244,.0972,, 2024.
[491] Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. Mitigating object ballucinations in large vision-language models through visual contrastive decoding. arXiv preprint 1rXiv:2311.1000 16922, 2023.
[492] Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. arXic preprint orx:222104. 08691, 2021.
[493] Vladimir I Levenshtein al. Binary codes capable of correcting deletions, insertions, and reversals. In Soviel physics doklady, volume 10, pp. 707-710. Soviet Union, 1966.
[494] Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kicla. Retrieval-augmente generation for knowledge-intensive nlp tasks. In Advances in Neural Information Processing Systems, pp. 9459 9474, 2020.
[495] Bo Lt, Yuanhan Zhang, Liangyu Chen, Jinghao Wang, Jingkang Vang, and Ziwei Liu. Otter: A multi-modas model with in-context instruction tuning, 2023.
[496] Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yan- wei Li, Ziwei Liu, and Chunyuan Li. Llava-onevision: Easy visual task transfer. ar preprinl arXiv:2408.0 03326, 2024.
[497] Bowen Li, Zhaoyu Li, Qiwei Du, Jinqi Luo, Wenshan Wang, Yaqi Xie, Simon Stepputtis, Chen Wang, Katia Sycara, Pradeep Ravikumar, Alexander Gray, Xujie Si, and Scbastian Scherer. Logicity: Ad- vancing neuro-symbolic ai with abstract urban simulation. In Neural Information Processing Systems (NeurIPS) Dalasets and Benchmarks Track, 2024.
120

---

Published in Transactions on Machine Learning Research (10/2025)
[498] Bowen Li, Tom Silver, Sebastian Scherer, and Alex Gray. Bilevel Learning for Bilevel Planning. In Proceedings of the Robolics: Science and Systems (RSS), 2025.
[499] Cheng Li, Jindong Wang, Yixuan Zhang, Kaijie Zhu, Wenxin Hou, Jianxun Lian, Fang Luo, Qiang Yang, and Xing Xie. Large language models understand and can be enhanced by emotional stimuli. ArXiv prepriat,
2023. URL
wwtggaa/////aeeeeeeeeee 11760.
[500] Cheng Li, Mengzhou Chen, Jindong Wang, Sunayana Sitaram, and Xing Xie. Culturellm: Incorporal- ing cultural differences into large language models, 2024 URL https:// //axivv arggabbs/4402 10946.
[50]] Daliang Li, Ankit Singh Rawat, Manzil Zaheer, Xin Wang, Michal Lukasik, Andreas Veit, Fe- lix X. Yu, and Surinder Kumar. Large language models with controllable working memory. ArXiv, abs/2211.0511B 2022. URL patps://aphl semanticscholes hatpaaaacccccccooooooooooraaaaaa
[502] Hang Li, Chengzhi Shen, Philip Torr, Volker Tresp, and Jindong Gu. Self-discovering interpretable diffusion latent directions for responsible text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12006-12016, 2024.
[503] Haoran Li, Dadi Guo, Wei Fan, Mingshi Xu, and Yangqiu Song. Multi-step jailbreaking privacy atlacks on chalgpt. arXiv preprint arXiv:2304.05197, 2023.
[504] Jialu Li, Jaemin Cho, Yi-lin Sung, Jaehong Yoon, and Mohit Bansal. Selma: Learning and merging skill-specific text-to-image experts with auto-generated data. arXiv preprint (tb::449900955) 2024.
[505] Jiaoda. Li, Ryan Cotterell, and Mrinmaya Sachan. Probing via Prompting, July 2022. URL http: arXiv:2207.01736 [cs].
//arxiv. carxiv.org/aiiie wttggabs/2207.0111///
[506] Jiarui Li, Ye Yuan, and Zehua Zhang. Enhancing Ilm factual accuracy with rag to counter hal- lucinations: A case study on domain-specific queries in private knowledge-bases. aiv preprinl vrXiv:2403.104// 2024.
[507] KunChang Li, Yinan He, Yi Wang. Yizhuo Li, Wenhai Wang. Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. Videochat: Chat-centric video understanding. arXiv preprint Xrv: 06355, 2023.
[508] Lei Li, Yuwei Yin, Shicheng Li, Liang Chen, Peiyi Wang, Shuhuai Ren, Mukai Li, Yazheng Yang, Jingjing Xu, Xu Sun, et al. M 3 it: A large-scale dataset towards multi-modal multilingual instruction tuning. arxiv preprint ar Xiv::2300 04387, 2023.
[509] Manling Li, Shiyu Zhao, Qineng Wang, Kangrui Wang, Yu Zhou, Sanjans Srivastava, Cem Gokmen, Tony Lee, Li Erran Li, Ruohan Zhang, et al. Embodied agent interface: Benchmarking Ilms for embodied decision making. In NewrIPS Dalasels and Benchmarks Track; 2024.
[510] Ming Li, Yong Zhang, Zhitao Li, Jiuhai Chen, Lichang Chen, Ning Cheng, Jianzong Wang, Tianyi Zhou, and Jing Xiao. From quantity to quality: Boosting 11m performance with self-guided data selection for instruction tuning. Xrxiv preprint arxiv:2308. 12032, 2023.
[11]] Moxin Li, Yong Zhao, Wenxuan Zhang, Shuaiyi Li, Wenya Xie, Seo-Kiong Ng, Tat-Seng Chua, and Yang Deng. Knowledge boundary of large language models: A survey. In ACL Long Papers, 2025.
[512] Tianhong Li, Yonglong Tian, He Lí, Mingyang Deng, and Kaiming He. Autoregressive image gener- ation without vector quantization. In NewrIPS, 2024.
[513] Weizhen Li, Jianbo Lin, Zhuosong Jiang, Jingyi Cao, Xinpeng Liu, Jiayu Zhang, Zhenqiang Huang, Qianben Chen, Weichen Sun, Qiexiang Wang, Hongxuan Lu, Tianrui Qin, Chenghao Zhu, Yi Yao, Shuying Fan, Xiaowan Li, Tiannan Wang, Pai Liu, King Zhu, He Zhu, Dingfeng Shi, Piaohong Wang, Yeyi Guan, Xiangru Tang, Minghao Liu, Yuchen Elcanor Jiang, Jian Yang, Jiaheng Liu, Ge Zhang, and Wangchunshu Zhou. of End-to-end agent foundation models via multi- agent distillation and agentic rl, 2025. URL https://arrte org. abs//bbb//500. 13167.
121

---

Published in Transactions on Machine Learning Research (10/2025)
[514] Xi'ang Li, Jinqi Luo, and Rabih Youne.. Activitygan: generative adversarial networks for data augmentation in sensor-based human activity recognition. In Adjunct Proceedings of the 2020 ACM International Joint Conference on Percosive and Ubiquitous Computing and Proceedings of the 2020 ACM International Symposium on Wearable Computers, UbiComp/ISVe '20 Adjunct, pp. 249-254,
New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450380768. doi: 10.1
415/3410530.34144 URL https:///oi http:::///oo..r///../o//// http:///....rr11.11144444/.0...0 10t.0111///11100..411111
[515] Xiang Lisa Li, Urvashi Khandelwal, and Kelvin Guu. Few-shot recalibration of language models, 2024.
[516] Xiaonan Li and Xipeng Qiu. Finding support examples for in-context learning, 2023.
[517] Xinfeng Li, Yuchen Yang, Jiangyi Deng, Chen Yan, Yanjiao Chen, Xiaoyu Ji, and Wenyuan Xu. Safe- gen: Mitigating unsafe content generation in text-to-image models. arx preprint or orXiv:240.006000 2024.
[518] Xucchen Li, Florian Tramer, Percy Liang, and Tatsunori Hashimoto. Large language models can be strong differentially private learners. In ICLR, 2022.
[519] Yansong Li, Zhixing Tan, and Yang Liu. Privacy-present preserving prompt tuning for large language model services. arxie preprint urXiv:2305. 06212, 2023.
520 Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji- Wen. Evaluating object hallucination in large vision-language models. arXiv preprint ar Xrv:223005 10355, 2023.
[221] Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjum Ma. Anti-backdoor learning: Training clean models on poisoned data. Advances in Neural Information Processing Systems, 34: 14900 14912, 2021.
522] Yingji Li, Mengnan Du, Rui Song, Xin Wang, and Ying Wang. A survey on fairness in large language models. arXiv preprint arXiv:2308. 2023.
[233] Yixuan Li, Xuclin Liu, Xiaoyang Wang, Shiqi Wang, and Weisi Lin. Fakebench: Uncover the achilles' heels of fake images with large multimodal models. arXiv preprint orxiv:2404.1333 2024.
[224] Yuying Li, Gaoyang Liu, Chen Wang, and Yang Yang. Generating is believing: Membership inference atlacks against odrrival-augmeentrr generation. arxiv preprinl urXiv:2406. 19234, 2024.
522] Zhan Li, Yongtao Wu, Yihang Chen, Francesco Tonin, Elias Abad Rocamora, and Volkan Cevher. Membership inference allacks against large vision-language models. arXiv preprint or arxiv:2411.00 11.02902.. 2024.
[526] Zhuowei Li, Haizhou Shi, Yunhe Gao, Di Liu, Zhenting Wang, Yuxiao Chen, Ting Liu, Long Zhao, Hao Wang, and Dimitris N. Metaxas. The hidden life of tokens: Reducing hallucination of large vision-language models via visual information steering. In ICML, 2025.
527] Zongxia Li, Paiheng Xu, Fuxiao Liu, and Hyemi Song. Towards understanding in-context learning with contrastive demonstrations and saliency maps, 2023.
[528] Buyun Liang, Liangzu Peng, Jinqi Luo, Darshan Thaker, Kwan Ho Ryan Chan, and Rene Vidal. Seca: Semantically equivalent and coherent attacks for eliciting Ilm hallucinations, In NeurIPS, 2025.
[529] Chumeng Liang, Xiaoyu Wu, Yang Hua, Jiaru Zhang, Yiming Xue, Tao Song, Zhengui Xue, Ruhui Ma, and Haibing Guan. Adversarial example does good: Preventing painting imitation from diffusion models via adversarial examples. In Proceedings of the 40th International Conference on Machine Learning, pp. 20763-20786, 2023.
[530] Paul Pu Liang, Chiyu Wu, Louis-Phliippe Moreney, and Ruslan Salakhutdinov. Towards under- standing and mitigating social biases in language models. In International Conference on Machine Learning, pp. 6565 6576. PMT.R, 2021.
122

---

Published in Transactions on Machine Learning Research (10/2025)
[531] Paul Pu Liang, Yiwei Lyu, Gunjan Chhablani, Nihal Jain, Zihao Deng, Xingbo Wang, Louis- Phill-p Morency, and Ruslan Salakhutdinov. Multiviz: Towards visualizing and understanding multimodal models, 2022.
[532] Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, et al. Holistic evaluation of language models. arXiv preprint arXiv:2211.09110 2022.
533 Qiyao Liang, Ziming Liu; Mitchell Ostrow, and Tla. R Fiete. How diffusion models learn to factorize and compose. In NewrIPS, 2024.
[534] Weixin Liang, Mert Yuksekgonul, Yining Mao, Eric Wu, and James Zou. Gpt detectors are biased against non-native english writers. Pallerns, 4(7), 2023.
[535] Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, llya Sutskever, and Karl Cobbe. Let's verify step by step. arXiv preprint arXiv:2305.20051 2023.
[536] Youngsun Lim, Hojun Choi, and Hyunjung Shim. Evaluating image hallucination in text-to-image generation with question-answeriig arXiv preprint 1rX7::2409.1278, 2024.
[537] Chin-Yew Tin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pp. 74 81, 2004.
[538] Li Lin, Necraj Gupta, Yue Zhang, Hainan Ren, Chun-Hao Liu, Feng Ding, Xin Wang, Xin Li, Luisa Verdoliva, and Shu Hu. Detecting multimedia generated by large models: A survey. arXie preprinl arXiv:2402. 500404 2025.
539] Stephanie Lin, Jacob Hilton, and Owain Evans. Teaching models to express their uncertainty in words, 2022.
[540]] Tsung-Yi Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference on compuler vision, pp. 1980-2988 2017.
[541] Weizhe Lin and Bill Byrne. Retrieval augmented visual question answering with outside knowledge. arXiv preprint arXiv:2200.00 0.8809 2022.
[542] Weiche Lin, Jinghong Chen, Jingbiao Mei, Alexandru Coca, and Bill Byrne. Fine-grained late- interaction multi-modal retrieval for retrieval augmented visual question answering. Advances in Neural Information Processing Systems, 36:22820-22 -28440 2023.
[543] Yongjie Lin, Yi Chern Tan, and Robert Frank. Open sesame: getting inside bert's linguistic knowl- edge. Xix preprint arXiv:1966.0100 2019.
[544] Zhen Lin, Shubhendu Trivedi, and Jimeng Sun. Generating with confidence: Uncertainty quantifica- tion for black-box large language models, 2023.
[545] Yaron Lipman, Ricky TQ Chen, Heli Ben Ham-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arxiv preprint 2tf::22010007777 2022.
[5446] Aiwei Liu, Leyi Pan, Xuming Hu, Shuang Li, Lijie Wen, Irwin King, and Philip S. Yu. An unforgcable publicly verifiable watermark for large language models. 2023. URL https://apt semanticachola
r. worg//orpusID:220333838
[547] Aiwei Liu, Leyi Pan, Xuming Hu, Shiao Meng, and Lijie Wen. A semantic invariant robust watermark for large language models. ArXiv, abs/2310.06382 2023. URL https: //ppp.app...// wwmanticscoola......... or I:CorpusID: 2633330330.
123

---

Published in Transactions on Machine Learning Research (10/2025)
[548] Alisa Liu, Zhaofeng Wu, Julian Michael, Alane Suhr, Peter West, Alexander Koller, Swabha Swayamdipta, Noah A. Smith, and Yejin Choi. We're re afraid language models aren't modeling ambi- guity, 2023.
[549] Bingchen Liu, Ehsan Akhgari, Alexander Visheratin, Alcks Kamko, Linmiao Xu, Shivam Shrirao, Chase Lambert, Joao Souza, Suhail Doshi, and Daiqing Li. Playground v3: Improving lext-lo-image alignment with deep-fusion large language models. arXiv preprint arXiv:2409. 10695, 2024.
[550] Bo Liu, Li-Ming Zhan, Zexin Lu, Yujic Feng, Lei Xuc, and Xiao-Ming Wu. How good are 1lms at out-of- distribution detection? In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evoluation (LREC-COLING 2024), pp. 8211 12222, 2024.
[551] Frederick Liu and Besim Avei. Incorporating priors with feature attribution on text classification. In ACL (1), pp. 6274 6283, Association for Computational Linguistics, 2019.
[552] Fuxiao Liu, Tianrui Guan, Zongxia Li, Lichang Chen, Yaser Yacoob, Dinesh Manocha, and Tianyi Zhou. Hallusionbench: You see what you think? or you think what you see? an image-context reasoning benchmark challenging for gpt-4v (ision), Ilava-1.5, and other multi-modality models. ar preprint arXiv:2310. 14566, 2023.
[553] Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. Aligning large multi-modal model with robust instruction tuning, arXiv preprint arXie:2306. 14565, 2023.
[554] Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. Mitigating hallu- cination in large multi-modal models via robust instruction luning, Xiv preprinl arXiv:2306. 14565,
1, 2023.
[555] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892 34916, 2023.
[556] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. arXio preprint or 08485, 2023.
[557] Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. What makes good in-context examples for gpt-3? arXiv preprint xr1::2100.06600, 2021.
[558] Jiang Liu, Chun Pong Lau, and Rama Chellappa. Diffprotect: Generate adversarial examples with diffusion models for facial privacy protection. arXiv preprint arXiv:2305.1300 13625, 2023.
[559] Jiang Liu, Chen Wei, Yuxiang Guo, Heng Yu, Alan Yuille, Soheil Feizi, Chun Pong Lau, and Rama Chellappa. Instruct2attack: Language-guided semantic adversarial attacks. arXiv preprint ar 1.i::2311.15555 2023.
[560] Jiashuo Liu, Zheyan Shen, Yue He, Xingxuan Zhang, Renzhe Xu, Han Yu, and Peng Cui. Towards out-of-distribution generalization: A survey, arXiv preprint arXio:2108.. 190644 2021.
[561] Jihao Liu, Xin Huang, Jinliang Zheng, Boxiao Liu, Jia Wang, Osamu Yoshic, Yu Liu, and Hongsheng Li. Mm-instruct: Generated visual instructions for large multimodal model alignment. arXiv preprint arXiv:2406. 19.1 19736, 2024.
[562] Jinxin Liu, Shulin Cao, Jiaxin Shi, Tingjian Zhang, Lei Hou, and Juanzi Li. Probing structured semantics understanding and generation of language models via question answering. arXiv preprint arXin:2401.05777, 2024.
[563] Kevin Liu, Stephen Casper, Dylan Hadfield-Menell, and Jacob Andreas. Cognitive dissonance: Why do language model outputs disagree with internal representations of truthfulness? In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 4791-4797 2023.
124

---

Published in Transactions on Machine Learning Research (10/2025)
[564] Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9):1 35, 2023.
[565] Runtao Liu, Ashkan Khakzar, Jindong Gu, Qifeng Chen, Philip Torr, and Fabio Pizzati. Latent guard: a safety framework for text-to-image generation. European Conference on Computer Vision (ECCV) (lo appear), 2024.
[566]
Sheng Liu, Haotian Ye, Lei Xing, and James Zou. Reducing hallucinations in vision-language models via latent space steering, 2024.
[567]
Shih- Yang Liu, Chien-Y? Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu- Yu-Chiang Frank Wang, Kwang- Ting Cheng, and Min-Hung Chen. Dora: Weight-decomposed low-rank adaptation. In ICML, 2024.
[568] Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalic Baracaldo, Peter Hasc, Xiaojun Xu, Yuguang Yao, Hang Li, Kush R Varshney, et al. Rethinking machine unlearning for large language models. arXiv preprint ar arXiv:2402.0070 2024.
[569] Tianyu Liu, Xin Zheng, Baobao Chang, and Zhifang Sui. Towards faithfulness in open domain table- to-text generation from entity-centric view. In Proceedings of the AAAT Conference on Artificial Intelligence, volune 35, pp. 13415 13423, 2021.
[570] Tong Liu, Yingjie Zhang, Zhe Zhao, Yinpeng Dong, Guozhu Meng, and Kai Chen. Making them ask and answer: Jailbreaking large language models in few queries via disguise and reconstruction. In 33rd USENIX Security Symposium (USENIX Security 24), pp. 4711 17114 2024.
[571] Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. Gpt understands, too. AT Open, 2023.
[572] Xiaoming Liu, Zhaohan Zhang, Yichen Wang, Hang Pu, Yu Lan, and Chao Shen. Coco: Coherence- enhanced geneetee text detection under low resource with contrastive learning. In Confer- ence on Empirical Methods in Natural Language Processing, 2023. URL https://appp httt::::/ap..smmatiicss semanticsc
holar. org/ CCrppsrrsss (0: 264406273.
[573] Xin Liu, Muhammad Khalifa, and Lu Wang. Litcab: Lightweight language model calibration over short- and long-form responses, 2024.
[574] Xinwei Liu, Jian Liu, Yang Bai, Jindong Gu, Tao Chen, Xiaojun Jia, and Xiaochun Cao. Watermark vaceine: Adversarial attacks to prevent watermark removal. In Europeon Conference on Computer Vision, pp. 1-17. Springer, 2022.
[575] Yang Liu, Mingyuan Fan, Cen Chen, Ximeng Liu, Zhuo Ma, Li Wang, and Jianfeng Ma. Backdoor defense with machine unlearning. In IEEE INFOCOM 2022-IEEE conference on computer commu- nications, pp. 280-289. IEEE, 2022.
[576] Yi Liu, Gelei Deng, Yuckang Li, Kailong Wang, Tianwei Zhang, Yepang Liu, Haoyu Wang, Yan Zheng, and Yang Liu. Prompt injection attack against Ilm-integrated applications. arXiv preprint ar 0rXiv:2306.05.// 2023.
[577] Yibing Liu, Haoliang Li, Yangyang Guo, Chenqi Kong, Jing Li, and Shiqi Wang. Rethinking attention- model explainability through faithfulness violation test, 2022.
[578] Yikang Liu, Ziyin Zhang, Wanyang Zhang, Shisen Yue, Xiaojing Zhao, Xinyuan Cheng, Yiwen Zhang, and Hai Hu. Argugpt: evaluating, understanding and identifying argumentative essays generated by gpt models. ArXio, abs/2304..0F 2023. URL https: http:/ppp...aaaaacccccccoooooeee wtw.:reccccooaaaoorrrrrreeeeeeee Drg/CorpusID: 258179137.
[579] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach, 2019. URL https: // //taxivvvcoo www.si..rr/abb//S00ee 11692.
125

---

Published in Transactions on Machine Learning Research (10/2025)
[580] Zeyuan Liu, Ziyu Huan, Xiyao Wang, Jiafei Lyu, Jian Tao, Xiu Li, Furong Huang, and Huazhe Xu. World models with hints of large language models for goal achieving. arxiv preprint or 0rx::22406. 07381, 2024.
[581] Zhihan Liu, Hao Hu, Shenao Zhang, Hongyi Guo, Shuqi Ke, Boyi Liu, and Zhaoran Wang. Reason for future, act for now: A principled framework for autonomous Ilm agents with provable sample efficiency. arXiv preprint 2rXiv:2309.1133 2023.
582] Zhihan Liu, Miao Lu, Shenao Zhang, Boyi Liu, Hongyi Guo, Yingxiang Yang, Jose Blanchet, and Zhaoran Wang. Provably mitigating overoptimization in rlhf: Your sft loss is implicitly an adversarial regularizer. arXiv preprinl or Xiv:22400. 16436, 2024.
[583]
Zihan Liu, Wei Ping, Rajarshi Roy, Peng Xu, Chankyu Tee, Mohammad Shoeybi, and Bryan Catan- zaro. Chatqa: Surpassing gpt-4 conversational qa and rag. arXiv preprint arXiv:2401. 10225, 2024.
[584] David Lopez-Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learning. Ad- vances in neural information processing systems, 30, 2017.
[585] Alejandro Lozano, Scott L Fleming, Chia-Chun Chiang, and Nigam Shah. Clinfo. ai: An open-source retrieval-ungmentedd large language model system for answering medical questions using scientific literature. In PACIFIC SYMPOSIUM ON BIOCOMPUTING 2024, pp. 8 23. World Scientific, 2023.
[586] Jiasen Lu, Christopher Clark, Rowan Zellers, Roozbeh Mottaghi, and Aniruddha Kembhavi. Unified- io: A unified model for vision, language, and multi-modal tasks. In The Eleventh International Conference on Learning Represenlations, 2022.
[587] Jiasen Lu, Christopher Clark, Sangho Lee, Zichen Zhang, Savya Khosla, Ryan Marten, Derek Hoiem, and Aniruddha Kembhavi. Unified-io 2: Scaling autoregressive multimodal models with vision lan- guage audio and action. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 11439926445, 2024.
[588] Kaiji Lu, Piotr Mardziel, Fangjing Wu, Preetam Amancharla, and Anupam Datta, Gender bias in neural natural language processing. Logic, Language, and Security: Essays Dedicated to Andre Scedrou on the Occasion of His 65th Birthday, pp. 189-202, 2020.
[589] Sheng Lu, Irina Bigoulaeva, Rachneet Sachdeva, Harish Tayyar Madabushi, and Iryna Gurevych. Are emergent abilities in large language models just in-context learning?, 2023.
[590] Taiming Lu, Lingfeng Shen, Xinyu Yang, Weiting Tan, Beidi Chen, and Huaxiu Yao. It takes two: On the seamlessness between reward and policy model in rlhf. arXiv preprint arXiv:2406.07977 2024.
[591] Sasha Luccioni, Christopher Akiki, Margaret Mitchell, and Yacine Jernite. Stable bias: Evaluating socictal representations in diffusion models. In Therty-scuenth Conference Neural Information Processing Systems Dalasets and Benchmarks Track, volume 2, 2023.
[592] Nils Lukas, Ahmndd Salem, Robert Sim, Shruti Tople, Lukas Wutschitz, and Santiago Zanella- Béguelin. Analyzing leakage of personally identifiable information in language models. arXiv preprint Akki:23302005533 2023.
[593] Haochen Luo, Jindong Gu, Fengyuan Liu, and Philip Torr. An image is worth 1000 lies: Transfer- ability of adversarial images across prompts on vision-language models. In The Twelfth International Conference on Learning Representations, 2023.
[594] Jinqi Luo, Zhaoning Wang, Chen Henry Wu, Dong Huang, and Fernando De La Torre. Zero-shot model diagnosis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
126

---

Published in Transactions on Machine Learning Research (10/2025)
[595] Jinqi Luo, Tianjiao Ding, Kwan Ho Ryan Chan, Darshan Thaker, Aditya Chattopadhyay, Chris Callison-Burch, and René Vidal. Pace: Parsimonious concept. engineering for large language models. In Advances in Neural Information Processing Systems (NewrIPS). 2024.
[596] Jinqi Luo, Tiajjiao Ding, Kwan Ho Ryan Chan, Hancheng Min, Chris Callison-Burcche and René Vidal. Concept lancet: Compositional representation transplant for tiffusiin-bbscee image editing. In CVPR, 2025.
[597] Junyu Luo, Cao Xiao, and Fenglong Ma. Zero-resource hallucination prevention for large language models. arXiv preprint arXiv:2309.0800 2023.
[598] Saiyue Lyu, Margarita Vinaroz, Michael F Liu, and Mijung Park. Differentially private latent diffusion models. arXiv preprint arXiv:2305.157 157599 2023.
[599] Jiachen Ma, Anda Cao, Zhiqing Xiao, Jie Zhang, Chao Ye, and Junbo Zhao. Jailbreaking prompt attack: A controllable adversarial attack against diffusion models. arXiv preprint Xii:2444.02298, 2024.
[600] Jiaqi Ma, Dylan Slack, Asma Ghandeharioun, Sameer Singh, Himabindu Lakkaraju, et al. Post hoc explanations of language models can improve language models. arXiv preprint arXiv:2305. 114.0114200 2023.
[601] Jun-Yu Ma, Jia-Chen Gu, Zue Ling, Quan Liu, and Cong Liu. Unlying the reversal curse via bidirectional language model editing. ArXiu, abs Abs/2310.10322, 2023. URL https::////// //ppp http:////a..smmnsssss.ss semanticsc holar. Orgg/orruus 2641422288
[602] Ruipeng Ma, Jinhao Duan, Fei Kong, Xiaoshuang Shi, and Kaidi Xu. Exposing the fake: Effective llffusion-generatee images detection. AduML Frontiers workshop at 40th International Conference on Machine Learning, 2023.
[603] Xingjun Ma, Yifeng Gao, Yixu Wang, Ruofan Wang. Xin Wang, Ye Sun, Yifan Ding, Hengyuan Xu, Yunhao Chen, Yunhan Zhao, et al. Safety at scale: A comprehensive survey of large model safety. arXiv preprint arXiv:2502. 20202000..0206 2025.
[604] Xueguang Ma, Liang Wang, Nan Yang, Furu Wei, and Jimmy Lin. Fine-tuning llama for multi-stage text retrieval. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrievol, pp. 2421-2422 2024.
[605] Zhe Ma, Xuhong Zhang, Qingming Li, Tianyu Du, Wenzhi Chen, Zonghui Wang, and Shouling Ji. Could it be generated? towards practical analysis of memorization in texl-to-image diffusion models. arx preprint arXiv:2405.0583 2024.
[606] Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models. arXiv preprint ar Xix:22200. 05424, 2023.
[607] Aman Madaan and Amir Yazdanbakhsh. Text and patterns: For effective chain of thought, it takes two to tango. arXiv preprint arXiv:2209. 07686, 2022.
[608] Aman Madaan, Niket Tandon, Peter Clark, and Yiming Yang. Memory-assisted prompt editing improve gpt-3 after deployment. ArXiv, abs/2201.06000 2022. URL attps://aph: https://apl.semmtiicee semanticschola oooggCoorpssI::2440 24600161144.
[609] Aman Madaan, Niket. Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegrefle, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Scan Wellock, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterative re- finement with self foldbcke In Thirty-seventh Conference on Neural Information Processing Systems, http:///ppppppeeeeeeeeee/aaaaa/r http://eeeeeeoomtt---t/e/eee00.
2023. URL https: https://
127

---

Published in Transactions on Machine Learning Research (10/2025)
[610] Andrea Madotto, Zihan Liu, Zhaojiang Lin, and Pascale Fung. Language models as few-shot learner for task-oriented dialogue systems. arXiv preprint arXie:2088. 06239, 2020.
[611] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu, To- wards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: 1rXi::17660000 706.06083, 2017.
[612] Andrey Malinin and Mark Gales. Uncertainty estimation in autoregressive structured prediction 2021.
[613] Potsawee Manakul, Adian Liusie, and Mark JF Gales. Selfchockgpt: Zero-rosource black-box halluci- nation detection for generative large language models. arXiv preprint arXiv:2303. 088966 2023.
[614] Chengzhi Mao, Scott Geng, Junfeng Yang, Xin Wang, and Carl Vondrick. Understanding zero-shot adversarial robustness for large-scale models. arXiv preprint arXtv:2212.07016 2022.
[615] Chengzhi Mao, Carl Vondrick, Hao Wang, and Junfeng Yang. RaidarggenRRative AT Detection viA Rewriting. In ICLR, 2024.
[616] Samuel Marks and Max Tegmark. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets. arXiv preprint arXiv:2810.066 06.8624, 2023.
[617] Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal, and Animesh Mukherjee. Hatexplain: A benchmark dataset for explainable hate speech detection. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 14867 14875, 2021.
[618] Justus Mattern, Fatemchsadat Mireshghallah, Zhijing Jin, Bernhard Schölkopf, Mrinmaya Sachan, and Taylor Kirkpaikka. Membership inference attacks against language models via neighbour- hood comparison. arXiv preprint Nrx::2300.. 18462, 2023.
[619] Rowan Hall Maudslay and Ryan Cotterell. Do Syntactic Probes Probe Syntax? Experiments with Jabberwocky Probing, June 2021. URL http://arxiv http:///aaarww../aaaa/////as httgg/ab//21000 02559. arXiv:2106.02553 [cs].
[620] Chandler May, Alex Wang, Shikha Bordia, Samuel R Bowman, and Rachel Rudinger. On mcasuring social biases in sentence encoders. arXiv preprint arx:v:1908 10561, 2019.
[221] Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. On faithfulness and factuality in abstractive summarization. arXiv preprint 2rXiv:2005.00 006611, 2020.
6222 Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David Forsyth, and Dan Hendrycks. Harmbench: A standardized evaluation framework for automated red tcaming and robust refusal. 2024.
[623] R. Thomas McCoy, Shunyu Yao, Dan Friedman, Matthow Hardy, and Thomas L. Griffiths. Embers of autoregression: Understanding large language models through the problem they are trained to solve, 2023.
[624] Nick McKenna, Tianyi Li, Liang Cheng, Mohammad Javad Hosseini, Mark Johnson, and Mark Steedman. Sources of hallucination by large language models on inference tasks. arXie preprint arXio:2305.1455 14552, 2023.
[225] Michal Mēchura. A taxonomy of bias-causing ambiguities in machine translation. In Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP), pp. 168 168-173, 2022.
[626] Ninarch Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and fairness in machine learning. ACM computing surveys (CSUR), 54(6):1 35, 2021.
[627] Ninarch Mchrabi, Muhammad Navoed, Fred Morstatter, and Aram Galstyan. Exacerbating algorith- mic bias through fairness attacks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 8930 8038,93888 2021.
128

---

Published in Transactions on Machine Learning Research (10/2025)
[628] Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual asso- ciations in gpt. Advances in Newral Information Processing Systems, 0417359--7372, 2022.
[229] Kevin Meng, Arnab Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. Mass-editing mem- ory in a transformer. ArXiv, abs/2210.077299, 2022. URL
semanticscholar crrgo
rpusID: 252873467.
[630] Yu Meng, Mengzhou Xia, and Danqi Chen. Simpo: Simple preference optimization with reference- free reward. Xr preprint arXiv:2405.1473 2024.
[631] Yuxian Meng, Shuhe Wang, Qinghong Han, Xiaofei Sun, Fei Wu, Rui Yan, and Jiwei Li. Openvidial: A larle-scale, open-domain dialogue dataset with visual contexts. orXiv preprint orXiu:2012.15015, 2020.
[632]
Bradley D Menz, Natansh D Modi, Michael J Sorich, and Ashley M Hopkins. Health disinformation use case highlightng the urgent need for artificial intelligence vigilance: weapons of mass disinforma-- tion. JAMA internal medicine, 184(1):92-99) 96, 2024.
[633] Hasan Mesut Meral, Bilent Sankur, A. Sumru Özsoy, Tunga Güngör, and Emre Sevinç. Natural language watermarking via morphosyntactic alterations. Comput. Speech Lang., 23:107-125, 2009. URL https://api. www.ntecscooolae........e. https://ap..emmatiicchoooo
.rg/CorpusID:: 1192689.
[634] Jack Merullo, Noah A. Smith, Sarah Wiegreffe, and Yanai Elazar. On linear representations and pretraining data frequency in language models, 2025.
[635] Ning Miao, Yee Whye Teh, and Tom Rainforth. Selfcheck: Using Iims to zero-shot check their own step-by-step reasoning. arXiv preprint arXix:2308.00436 2023.
[636]
Midjourney. Midjourney. https://middoourney.com
2023. Accessod: 2023.
[637] Sabrina Mielke, Arthur Szlam, Emily Dinan, and Y-Lan Boureau. Reducing conversational agents' overconfidence through linguistic calibration, 2022.
[638]
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representa- tions in vector space, 2013.
[639] Sewon Min, Julian Michael, Hannaneh Hajishirzi, and Luke Zettlemoyer. Ambigqa: Answering am- biguous open-domain questions. arXiv preprint arXiv:200..106 10645, 2020.
[640] Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. Rethinking the role of demonstrations: What makes in-context learning work? arXiv preprint arXiv:220. 12837, 2022.
[641] Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit lyyer, Luke Zettlemoyer, and Hannanch Hajishirzi. Factscore: Fino-grained atomic evaluation of factual precision in long form text generation. anXiv preprint arXiv:2305. 14251, 2023.
[642] Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucie. Revisiting the calibration of modern neural networks, 2021. URL https://arxiv. .rwg http:/abb/220.0000/00
[643] MiniMax. Hailuo 02 - global ai video generation model. https://hailuo- 02. com/, 2025.
[644] Fatemehsadat Mireshghallah, Archit Uniyal, Tianhao Wang, David K Evans, and Taylor Berg- Kirkpatrick. An empirical analysis of memorization in fine-tuned autoregressive language models. In Proceedings of the 2022 Conference on Empirical Methods Natural Language Processing, pp.
1816 1826, 2022.
[645] Fatemchsadat Mireshghallah, Justus Mattern, Sieun Gao, R. Shokri, and Taylor Berg-Kirkpatrick. Smaller language models are better black-box machine-generated text detectors. ArXiv, abx/2305.098859, 2023. URL http:////pp.eemmnncccccllrrrrrr http://ccco...ooooooooooaaaaaaaa
129

---

Published in Transactions on Machine Learning Research (10/2025)
[646] Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D. Manning, Fast model editing at scale. ArXiv, abs/2110.11309, 2021. URL httppss/pppp..mmmmcccccooooa.a..
org/CorpusID: o
semanticscholar
239050360.
[647] Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning, and Chelsea Finn. Memory- based model editing at scale. ArXiu, (te//2206.065200 2022. URL https://api. semanticscholar.
org/CorpusID] 249642147.
[648] Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. Detect- gpt: Zero-shot machine-generated text detection using probability curvature. In International Confer- ence on Machine Learning, 2023. URL https://app http:://app..smmanicssoooo semanticscholas D : 256274849.
[649]] Hosein Mohebbi, Ali Modarressi, and Mohammad Taher Pilehvar. Exploring the role of BERT token representations to explain sentence probing results. In EMNLP (1), pp- 792-806, Association for Computational Linguisties, 2021.
[650]
Christopher Mohri and Tatsunori Hashimoto. Language models with conformal factuality guarantees, 2024.
[651] Grégoire Montavon, Alexander Binder, Sebastian Lapuschkin, Wojeiech Samek, and Klaus-Robert Müller. Layer-wise relevance propagation: An overview. In Explainable AI, volume 11700 of Lecture Notes in Computer Science, pp. 193 209. Springer, 2019.
[652] Moonshot. Kimi k1.5: Scaling reinforcement learning with Ilms, 2025. URL https://arxiv org/ab 5/2501.1259
653 Jesse Mu and Jacob Andreas. Compositional Explanations of Neurons, February 2021. URL http: wwtpxiiwwrgg/bb/////ea/ 14032. arXiv:2006.14033 [cs, stat].
[654] Bálint Mucsányi, Michael Kirchhof, and Seong Joon Oh. Benchmarking uncertainty disentanglement: Specialized uncertainties for specialized tasks. Advances in neural information processing systems, 37:50972-51038, 2024.
[655] Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. Orca: Progressive learning from complex explanation traces of gpt-4. orXiv preprint arXiv:2306.02700 2023.
[656] Rafael Müller, Simon Kornblith, and Geoffrey E Hinton. When does label smoothing help? Advances in neural information processing systems, 32, 2019.
[657] Niels Mündler, Jingxuan He, Slobodan Jenko, and Martin Vechev. Self-contradictory hallucinations of large language models: Evaluation, detection and mitigation. arXiv preprint arXiv:2305.1588 15852..
2023.
[658] Rémi Munos, Michal Valko, Daniele Calandriello, Mohammad Gheshlaghi Azar, Mark Rowland, Zhaohan Daniel Guo, Yunhao Tang, Matthieu Geist, Thomas Mesnard, Andrea Michi, et al. Nash learning from human feedback. arxie preprint arXiv:2312. 00886, 2023.
[659] Travis J. E. Munyer and Xin Zhong. Deeptextmark: Deep learning based text watermarking for detection of large language model generated text. ArXiv, abs/2305.05773, 2023. URL https://api. semanticscholar Org/CorpusID :ESEID::28888..
[660] Shikhar Murty, Christopher D. Manning, Scott M. Lundberg, and Marco Tulio Ribeiro. Fixing model bugs with natural language patches. In Conference on Empirical Methods in Natural Language Processing, 2022. URL https://api. semanticacholar. www.oorg///rror//.....om 249147333
[661] Max linguistic information emerges, shifts and interacts during language model training, 2023.
Rob van der Goot, Barbara Plank, and Ivan Titov. Subspace chronicles: How
130

---

Published in Transactions on Machine Learning Research (10/2025)
[662] Moin Nadeem, Anna Bethke, and Siva Reddy. Stercoset: Measuring stereotypical bias in pretrained language models. Xiv preprint S1t:2004.09455 2020.
[663] Mahdi Pakdaman Nacini, Gregory Cooper, and Milos Hauskrecht. Obtaining well calibrated proba- bilities using bayesian binning. Tn Twenty-Ninth AAAT Conference on Artificial Intelligence, 2015.
[664] Krishna Nakka, Ahmed Frikha, Ricardo Mendes, Xue Jiang, and Xuebing Zhou. PII-compass: Guid- ing LLM training data extraction prompts towards the target PIl via grounding, In Proceedings of the Fifth Workshop on Privacy in Natural Language Processing, pp. 63-73, Bangkok, Thailand, August 2024.
Association for Computational Linguisties.
[665] Yang Nan, Huichi Zhou, Xiaodan Xing, and Guang Yang. Beyond the hype: A dispassionate look at vision-language models in medical scenario. arXiv preprint arXiv:2408.00 08704, 2024.
[666] Nikita Nangia, Clara Vania, Rasika Bhalerao, and Samuel R Bowman. Crows-pairs: A challenge datasct for measuring social biases in masked language models. arXiv preprint arXi::210000013: 2020.
[667] Ali Nasch, Kalpesh Krishna, Mohit Iyyer, and Amir Houmansadr. Stealing the decoding algorithms of language models. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Com- manications Securily, pp. 1835-1849, 2023.
[668] Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A Feder Cooper, Daphne Ippolito, Christopher A Proqueetee-hho Eric Wallace, Florian Tramèr, and Katherine Lee. Scalable extraction of training data from (production) language models. arXiv preprint ar arXi::231111003: 2023.
[669]
Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy, al. Text and code embeddings by contrastive pre-training. arXiv preprint arXw:2201.10005, 2022.
[670] Bach Hoang Ngo, Dat Thanh Nguyen, Nhat-Tuong Do- To-Tran, Phue Pham Huy Thien, Minh-Hung An, Tuan-Ngoc Nguyen, Loi Nguyen Hoang, Vinh Dinh Nguyen, and Vinh Dinh. Comprehensive visual fcatures and pscudo labeling for robust natural languago-basee based vehicle retricval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5408-5417, 2023.
[671] Debora Nozza, Federico Bianchi, Dirk Hovy, et al. Honest: Measuring hurtful sentence completion in language models. In Proceedings of the 2021 Conference of the North American Chapter of the Associ- ation for Computational Länguisties: Human Language Technologies. Association for Computational Linguistics, 2021.
[672] Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, et al. Show your work: Scratchpads for intermediate computation with language models. arXie preprint arXiv:21 2ab:021100011 2021.
[673] Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Concrly, Dawn Drain, Deep Ganguli, Zac Hatfield- Dodds, Danny Hernandez, Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. In-context Learning and Induction Heads transformeer-crcuitssppp https: htaps://transfommt r- r-circuits. pwwppo2//in--aeeepppppppppp htcp/in-contt wwwteetottttttttee ceoontext-leerninge hnw. iineex html, March 2022. [Accessed 27- 27-11-2023
[674] OpenAL. Image generation guide (gpt-image-1), 1). https://platforrm openai. www//ocs//uidees.eeooooooessse age- otg.aeeeeeeeeeetoooooooooooooooo
[675] OpenAL. ChatGPT. https::/openntt..mm//l//ggg// 2023. Accessed: September 10, 2023.
[676]
OpenAL. Gpt-4 technical report, 2023.
131

---

Published in Transactions on Machine Learning Research (10/2025)
[677] OpenAL Gpt-4o system card. anXiv preprint arXiv:2410.21270 2024.
[678] OpenAl. Moderator overview. httppp:://laaftm..../ httpppap.....oocooo//////////aaa iew, 2024. Accessed: June 29, 2024.
[679] OpenAL. Openai ol system card, 2024. URL http:///aaaa...rrrr////2.......0
[680] OpenAL. Sora.
2024. Accessed: Feburary 15, 2024.
[681] OpenAL. Introducing openai o3 and o4-mini. http::///geeaaa.ccoa///e/////trr
4-mini/, April 2025.
[682] OpenAL Gpt5. http:://mommiiiiommm/m//////haip
2025.
[683] OpenAl. Gpt-4.5, February 2025. URL http:///ppeaa..ooo///////ttttttt Accessod: April 6, 2025.
[684] Ian Osband, Seyed Mohammad Asghari, Benjamin Van Roy, Nat McAlcese, John Aslanides, and GeofTrey Irving. Fine-tuning language models via epistemic neural networks, 2023.
[685] Myle Ott, Michael Auli, David Grangier, and Marc' Aurelio Ranzato. Analyzing uncertainty in neural machine translation. In Jemnifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp. 3956- 3965. PMLR, 10 15 Jul 2018. URL https://proceeddnggs mlr. wpt.ss//8/otttteeee html.
[686] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730-
27744, 2022.
[687] Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, D Sculley, Sebastian Nowozin, Joshua V. Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift, 2019.
[688] Anwesan Pal, Radhika Bhargava, Kyle Hinsz, Jacques Esterhuizen, and Sudipta Bhattacharya. The empirical impact of data sanitization on language models. arXie preprint arXw:2411100597,, 2024.
[689] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Blou: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeling of the Associalion for Computational Linguistics, pp. 111-118, 2002.
[690] Ankur P Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, and Dipanjan Das. Totto: controlled table-to-text generation datasct. arXiv preprint ar arxix:2004. 14373, 2020.
[691] Cheonbok Park, Inyoup Na, Yongjang Jo, Sungbok Shin, Jachyo Yoo, Bum Chul Kwon, Jian Zhao, Hyungjong Noh, Yeonsoo Lee, and Jaegul Choo. Sanvis: Visual analytics for understanding self- attention networks. In 2019 IEEE Visualization Conference (VIS), pp. 146-150. IEEE, 2019.
[692] Core Francisco Park, Maya Okawa, Andrew Lee, Ekdeep Singh Lubana, and Hidenori Tanaka. Emer- gence of hidden capabilities: Exploring learning dynamics concept space. In NewrIPS, 2024.
[693] Joon Sung Park, Joseph O'Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, pp. 1 22, 2023.
[694] Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel R Bowman. Bbq: A hand-built bias benchmark for question answering. Xi preprint 21 2021.
132

---

Published in Transactions on Machine Learning Research (10/2025)
[695] Vaidchi Patil, Peter Hase, and Mohit Bansal. Can sensitive information be deleted from llms? objec- lives for defending against extraction attacks. In The Twelfth International Conference on Learning Representations, 2024.
[696] Mansheej Paul, Surya Ganguli, and Gintare Karolina Dziugaite. Deep learning on a data diet: Finding important examples early in training. Advances in Neural Information Processing Systems, 34:20596 20607, 2021.
[697] Kellin Pelrine, Anne Imouza, Camille Thibault, Meilina Reksoprodjo, Caleb Gupta, Joel Christoph, Cran-FFançois Godbout, and Reihanch Rabbany. Towards reliable misinformation mitigation: Gen- eralization, uncertainly, and gpt-4, 2023.
[698] Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobcidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. The refinedweb dataset for falcon Nl:: Outprrformngg curated corpora with web data, and web data only, 2023. URL https: //arxiv.orgaabs https//33...11...... 01116.
[699] Baolin Peng, Michel Galley, Pengcheng He, Hao Cheng, Yujia Xie, Yu Hu, Qiuyuan Huang, Lars Liden, Zhou Yu, Weizhu Chen, et al. Chock your facts and try again: Improving large language models with external knowledge and automated feedback. arXiv preprint X1::2202212813,, 2023.
[700] Jeffrey Pennington, Richard Socher, and Christopher Manning. Glove: Global vectors for word rep- resentation. In Proceedings of the 2014 conference on empérical methods in natural language processing (EMNLP), pp. 1532 1543, 2014.
[701] Fábio Perez and Ian Ribeiro. Ignore previous prompt: Attack techniques for language models. ar preprint or arxi::2211000 1100927, 2022.
[702] Matthew E. Peters, Mark Neumann, Mohit Tyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word representations, 2018. URL https://arxiv. org/ abs/1802. hbs/1802.05365.
[703] Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H. Miller, and Sebastian Riedel. Language Models as Knowledge Bases?, September 2019. URL http://arxi v. oror/ aaaaab///// abb/1999.. 1909. 01066. arXiv:1909.01000 [cs).
[704] Bun Phan, Marton Havasi, Matthew Muckley, and Karen Ullrich. Understanding and mitigating tokenization bias in language models. arXiv preprint arXiv:2406. 16829, 2024.
[705] Renjic Pi, Tianyang Han, Yueqi Xie, Rui Pan, Qing Lian, Hanze Dong, Jipong Zhang, and Tong Zhang. MIlm protttrrttctor: Ensuring mllm's safety without hurting performance. arXiv preprint ar orXiv:2401.09006 2024.
[706] J. Platt. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in Large Margin Classifiers, 10(3), 1999.
[707] Gabriel Poesia, Alex Polozov, Vu Le, Ashish Tiwari, Gustavo Soares, Christopher Meek, and Sumit Gulwani. Synchromesh: Reliable code generation from pre-trained language models. In The Tenth International Conference on Learning Reeprssentations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. URL
[708] Lip Yee Por, Koksheik Wong, and Kok Onn Chee. Unispach: A text- data hiding method using unicode space characters. J. Syst. Softw., 85:1075-1082, 2012. URL httpss//appl semanticscholl. r. r.org/ oorg/CorpusI::7556 17690312.
[709] Viraj Prabhu, Sriram Yenamandra, Prithvijit Chattopadhyay, and Judy Hoffman. Lance: Stress- testing visual models by generating language-guided counterfactual images. In NeurTPS, 2023.
133

---

Published in Transactions on Machine Learning Research (10/2025)
[710] Reid Pryzant, Dan Iter, Jerry Li, Yin Tat Lee, Chenguang Zhu, and Michael Zeng. Automatic prompt optimization with" gradient descent' and beam search. Arx preprini, abs/2330.003495, 2023. URL, 03495.
fttps://arxiv.orggeeee
[711] Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Mengdi Wang, and Prateek Mittal. Visual adversarial examples jailbreak aligned large language models. In The Second Workshop on New Frontiers in Adversarial Machine Leurning, volume 1, 2023.
[712] Xiangyu Qi, Yi Zeng, Tinghao Xie, Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. Finc-tuming aligned language models compromises safety, even when users do not intend to! ArXiv preprint, (obs/2230.0333 03693, 2023. URL https://arxiss wwwp:iv..rgggbb///.eee 2310//00000000..0....
[713] Xiangyu Qi, Ashwinee Panda, Kaifeng Lyu, Xiao Ma, Subhrajit Roy, Ahmad Beirami, Prateek Mittal, and Peter Henderson. Safety alignment should be made more than just few tokens deep. arXiv preprint arXiv:2466005000 2024.
[714] Chen Qian, Xin Cong, Cheng Yang, Weize Chen, Yusheng Su, Juyuan Xu, Zhiyuan Liu, and Maosong Sun. Communicative agents for software development. arXiv preprint arXiv:2307.0792, 2023.
[715] Rebecca Qian, Candace Ross, Jude Fernandes, Eric Smith, Douwe Kicla, and Adina Williams. Per- turbation augmentation for fairer nlp. orXiv preprinl arXiv:2205. 12586, 2022.
[716] Shuofei Qiao, Ningyu Zhang, Rumnan Fang, Yujie Luo, Wangchunshu Zhou, Yuchen Jiang, Chengfei Lv, and Huajun Chen. AutoAct: Automatic agent learning from scratch for QA via self-planning. In Lun-We Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeling of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3003-3021, Bangkok, Thailand, August 2024. Association for Computational Linguistics. URL rg/2024.ac1 acl-long 165.
[717] Guanghui Qin and Jason Eisner. Learning how to ask: Querying Ims with mixtures of soft prompts. Xrxi preprind arXiu:2104006599, 2021.
[718] Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen, Ning Ding, Ganqu Cui, Zheni Zeng, Yufei Huang, Chaojun Xiao, Chi Han, et al. Tool learning with foundation models. ArXiv preprint, 0bs/23044.83544 2023. URL. https: // /aaaivivvvorro hawpllorrraabaee abs//304..... http//234.088555
[719] Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. Toolllm: Facilitating large language models to master 16000+ real-world apis. arxi preprint arXiw:237..0 16789, 2023.
[720] Chen Qu, Weize Kong, Liu Yang, Mingyang Zhang, Michael Bendersky, and Mare Najork. Natural language understanding with grivacy-preseriing bert. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pp. 1488 1497, 2021.
(721] Wenjie Qu, Dong Yin, Zixin He, Wei Zou, Tianyang Tao, Jinyuan Jia, and Jiaheng Zhang. Prov- ably robust multi-bit watermarking for ai-generated text via error correction code. arXiv preprint (xXi:2201.16620, 2024.
[722] Victor Quach, Adam Fisch, Tal Schuster, Adam Yala, Jac Ho Sohn, Tommi S. Jaakkola, and Regina Barzilay. Conformal language modeling, 2023.
[723] Alec Radford, Karthik Narasimhan, Tim Salimans, 1lya Sutskever, et al. Improving language under- standing by generative pre-training 2018.
[24] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Nya Sutskever, et al. Language modols are unsupervised multitask learners. OpenAl blog, 188):9, 2019.
[25] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transfcrable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PMLR,
2021.
134

---

Published in Transactions on Machine Learning Research (10/2025)
[226] Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katic Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, Eliza Rutherford, Tom Hennigan, and et al. Scaling language models: Methods, analysis & insights from training gopher, 2022. URL
https: www.ii.wwg.aaaa....ooooee.....ee 114.1111...11....
[727] Rafacl Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. Ar preprind. abs/2305.112900 2023. URL https: https://arxii /ttp:////axxvoveee org/ ab3/ 18290.
[728] Rafael Rafailov, Yaswanth Chittepu, Ryan Park, Harshit Sikchi, Joey Hejna, Bradley Knox, Chelsea Finn, and Scott Nickum. Scaling laws for reward model overoptimization in direct alignment algo- rithms. arXie preprint orXiv:2406. 02900, 2024.
[729] Colin Rasfel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter Liu. Exploring the limits of transfer learning with unified text-to-text transformer. The Journal of Machine Learning Research, 21(1):5485 5551, 2020.
[730] Alessandro Raganato and Jörg Tiedemann. An analysis of encoder representations in transformer- based machine translation. In Proceedings of the 2018 EMNLP workshop BlackbozNLP: analyzing and interpreting neural networks for NLP. The Association for Computational Linguistics, 2018.
[731] Aditi Raghunathan, Jacob Steinhardt, and Percy Liang. Certified defenses against adversarial exam- ples. arxiv preprint orXic:1801.0944/ 2018.
[732] Goutham Rajendran, Simon Buchholz, Bryon Aragam, Bernhard Schölkopf, and Pradeep Kumar Ravikumar. From causal to concept-based representation learning. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[733] Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100,000+ questions for machine comprehension of text. In Conference on Empirical Methods in Natural Language Processing, 2016. URL https://aptt www.ssseamninnoolaa.....a..o
httpss// wwwpaccholaa..ooooorroeooeeoo wtggCorpusIDDa.......... 1000:0141,,4
[734] Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham, In-context retrieval anggentte language models. arXiv preprint orXiv:2302.000s;, 2023.
[735] Alexandre Rame, Guillaume Couairon, Mustafa Shukor, Corentin Dancctte, Jcan-Baptiste Gaya, Laure Soulier, and Matthieu Cord. Rewarded soups: towards pareto-optimal alignment by in- terpolating weights fine-tuned on diverse rewards. ArXiv preprint, abs/2306.04r 044488 2023. URL https://arxiv. abg/abs/// /aab//300.... 04488.
[736] Alexandre Ramé, Johan Ferret, Nino Vicillad,, Robert Dadashi, Lćonard Hussenot, Pierre-Louis Cedoz, Pier Giuseppe Sessa, Sertan Girgin, Arthur Douillard, and Olivier Bachem. Warp: On the benefits of weight averaged rewarded policies. arXiv preprint 2rXiv:246.116666 2024.
[737] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Πlya Sutskever. Zero-shot text-to-image generation. In International Conference on Machine Learning, pp. 8821-8831. PMLR, 2021.
[738] Alexandre Ramé, Nino Vieillard, Léonard Hussemot, Robert Dadashi, Geoffrey Cideron, Olivier Bachem, and Johan Ferret. Warm: On the benefits of weight averaged reward models, 2024. URL https://arxiv. hngg/bbs/2400 /ttp///41...1111... 12187.
[799 Javier Rando and Florian Tramèr. Universal jailbreak backdoors from poisoned human feedback. In ICLR, 2024.
[740] Javier Rando, Daniel Paleka, David Lindner, Lennart Heim, and Florian Tramèr. Red-teaming the stable diffusion safety filter. Xiv preprint or arXim:2210.0461 2022.
135

---

Published in Transactions on Machine Learning Research (10/2025)
[741] Shauli Ravfogel, Yoav Goldberg, and Jacob Goldberger. Conformal mucleus sampling. arXiv preprint ar Xrx:22200..0 02633, 2023.
[742] Abhilasha Ravichander, Eduard Hovy, Kaheer Suleman, Adam Trischler, and Jackie Chi Kit Cheung. On the Systematicity of Probing Contextualized Word Representations: The Case of Hypernymy in BERT. In Proceedings of the Ninth Joint Conference on Lezical and Computational Semantics, pp. 88 1S-102, Barcelona, Spain (Online); December 2020. Association for Computational Linguisties. URL attps://aclanthologg www.:/looggyyygg/221/22 wtg. 0tt.20020.saarsmm1111...
[743] Shaina Raza, Ananya Raval, and Veronica Chatrath. Mbias: Mitigating bias in large language models while retaining context, 2024. URL
wttp://////a/./....///// 11290.
[744] Patrik Reizinger, Szilvia Ujváry, Anna Mészáros, Anna Kerekes, Wicland Brendel, and Ferenc Huszár. Position: Understanding Ilms requires more than statistical generalization. In Forly-forst International Conference on Machine Learning, 2024.
[745] Navid Rekabsaz and Markus Schedl. Do neural ranking models intensify gender bias? In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2065-2006 2020.
[746] Jie Ren, Han Xu, Yiding Liu, Yingqian Cui, Shuaiqiang Wang, Dawei Yin, and Jiliang Tang. A robust semantics-basel watermark for large language model against paraphrasing. ArXiu, abs/2311.08721, 2023. URL http:////p...ematttscccolllll
wogp://laaroooooooooeeeeeeooeeee (orppssDD:2652130088
[747] Ruiyang Ren, Yuhao Wang, Yingqi Qu, Wayne Xin Zhao, Jing Liu, Hao Tian, Hua Wu, Ji-Rong Wen, and Haifeng Wang. Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentation, July 2023. URL. http://arxiv. org/abs/2307. 11019. arXiv:2307.11019 [cs].
[748] Laura Rieger, Chandan Singh, W. James Murdoch, and Bin Yu. Interpretations are useful: Penalizing explanations to align neural networks with prior knowledge. In ICML, volume 119 of Proceedings of Machine Learning Research, pp. 8116 8126. PMLR, 2020.
[749] Stefano Giovanni Rizzo, Flavio Bertini, and Danilo Montesi. Content-preserving text watermarking through unicode homoglyph substitution. Proceedings of the 20th International Database Engineering & Applications Symposium, 2016. CRL https://app semanticscholar http://cccc.....ooooooooooaaaaaa org//orrussD:11169900.
[750] Alexander Robey, Eric Wong, Hamed Hassani, and George J Pappas. Smoothllm: Defending large language models against jailbreaking attacks. arXiv preprint arXiv:2310.03634, 2023.
[751] Anna Rogers, Matt Gardner, and Isabelle Augenstein. Qa dataset explosion: A taxonomy of nlp resources for question answering and reading comprchension. ACM Computing Surveys, 55(10):1-45, 2023.
[752] Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. Object hallucination in image captioning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 4035-4045, 2018
[753] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Essor, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp. 10684-10695, 2022.
[754] Alexis Ross, Ana Marasovic, and Matthew E. Peters.
Explaining NLP models via minimal contrastive editing (mice). In ACL/IJCNLP (Findings), volume ACL/IJCNLP 2021 of Findings of ACL, pp. 3840-3852. Association for Computational Linguistics, 2021.
[755] Andrew Slavin Ross, Michael C. Hughes, and Finale Doshi-Velez. Right for the right reasons: Training differentiable models by constraining their explanations. In IJCAT, pp. 8662-2670 ijcai.org, 2017.
136

---

Published in Transactions on Machine Learning Research (10/2025)
[756] Corby Rosset, Ching-An Cheng, Arindam Mitra, Michael Santacroce, Ahmed Awadallah, and Tengyang Xie. Direct Nash Optimization: Teaching Language Models to Self-Improve with Gen- eral Preferences. arxie 2024.
[757] Laura Ruis, Maximilian Mozes, Juhan Bae, Siddhartha Rao Kamalakara, Dwarak Talupuru, Acyr Lo- catelli, Robert Kirk, Tim Rocktäschel, Edward Grefenstette, and Max Bartolo. Procodural knowledge in pretraining drives reasoning in large language models, 2024.
[758] Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine luning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22500- 22000,2251,,, 2023.
[759] Mehrdad Saberi, Vinu Sankar Sadasivan, Keivan Rezaci, Aounon Kumar, Atoosa Chegini, Wenxiao Wang, and Soheil Feizi. Robustness of al-image detectors: Fundamental limits and practical attacks. arXio preprint arXiw:2310.0007 2023.
[760] Vinu Sankar Sadasivan, Aounon Kumar, S. Balasubramanian, Wenxiao Wang, and Soheil Feizi. Can ai-generated text be reliably detected? ArXiv, abs/ 0t//23333.1110 11156, 2023. URL https://app semantic scholar. wwwp:/oro//oooii200002m
2577631150..
[761] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Information Processing Systems, 15:36479-36499 2022.
[762] Pranab Sahoo, Prabhash Mcharia, Akash Ghosh, Sriparna Saha, Vinija Jain, and Aman Chadha. A comprehensive survey of hallucination in large language, image, video and audio foundation models. In Findings of the Association for Computational Linguistics: EMNLP 2024, 2024.
[763] Julian Salazar, Davis Liang, Toan Q Nguyen, and Katrin Kirchhoff. Masked language model scoring, arxiv preprint 1rXiv:1910. 14699, 2019.
[764] Etienne Salimbeni, Francesco Craighero, Renata Khasanova, Milos Vasie, and Pierre Vandergheynst. Beyond fine-tuning: Lora modules boost near-ood detection and 1lm security. In TCLR 2024 Workshop on Secure and Trustworthy Large Language Models, 2024.
[765] Soumya Sanyal and Xiang Ren. Discretized integrated gradients for explaining language models. In EMNLP (1), pp. 10285-10299. Association for Computational Linguistics, 2021.
[766] Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A Smith. The risk of racial bias in hate speech detection. In Proceedings of the 57th annual meeting of the association for computational linguistics, pp. 1668-1678, 2019.
[767] Irina Saparina and Mirella Lapata. Ambrosia: A benchmark for parsing ambiguous questions into database queries. arXiv preprint arXiv:2406.1907., 2024.
[768] R. Sato, Yuki Takezawa, Han Bao, Kenta Niwa, and Makoto Yanada. Embarrassingly simple text watermarks. Arx,u, abs/2310.08920, 2023. URL https://api. wew.aatticccholl....oooorr...eee 264128148.
[769] Timo Schick, Sahana Udupa, and Hinrich Schütze. Self-diagnosis and self-debiasing: A proposal for reducing corpus-based bias in nlp. Transactions of the Association for Computational Linguistics, 9: 1408 1424, 2021.
[770] Timo Schick, Jane Dwivedi- Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach them- selves to use tools. Advonces in Neural Information Processing Systems, 36, 2024.
[771] Jürgen Schmidhuber. Deep learning in neural networks: An overview. Neural networks, 61:85-117, 2015.
137

---

Published in Transactions on Machine Learning Research (10/2025)
[772] Florian Schmidt. Generalization in generation: A closer look at exposure bias. arXiv preprint 2019.
[773 Patrick Schramowski, Manuel Brack, Björn Deiseroth, and Kristian Kersting. Sale latent diffusion: Mitigating inappropriate degeneration in diffusion models. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognation, pp. 22522-22533 2023.
[774] Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarezyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki. Laion-400m: Open dataset of clip-filterdd 400 million image-text pairs. arXin preprint arXiv:2111.02114 2021.
[775] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. Laion-5b: An open large-scale dataset for training next generation image-text models. arXiv preprint arXiv:2210.08// 08402, 2022.
[776] John Schulman, Filip Wolski, Prafulla. Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. ArXiv preprint, abs/1707 0ts//17700064477 2017. URL bttpps///axxiz htt///arxvv...o/bb/ee//e org/abs/17 07.06347.
[777] Tal Schuster, Adam Fisch, Jai Gupta, Mostafa Dehghani, Dara Bahri, Vinh Q. Tran, Yi Tay, and Donald Metzler. Confident adaptive language modeling, 2022.
[778] Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. Quantifying language models' sensitivity to spurious features in prompt design or: How learned to start worrying about prompt. formatting. arxio preprint arXiv:2310. 11324, 2023.
[779] Andrew Selbst and Julia Powles. "mcaningful information" and the right to explanation. In conference on fairness, accounlability and transporency, pp. 48 48. PMLR, 2018.
[780] Minju Seo, Jinheon Baek, James Thorne, and Sung Ju Hwang. Retrieval-augmented data augmenta- tion for low-resource domain tasks. arXiv preprint arXiv:240211348. 2024.
[781] Glenn Shafer and Vladimir Vovk. A tutorial on conformal prediction. Journal of Machine Learning Research, ((12):371-421) 2008.
[782] Shawn Shan, Jenna Cryan, Emily Wenger, Haitao Zheng, Rana Hanocka, and Ben Y. Zhao. Glaze: Protecting artists from style mimicry by Text-to-Image models. In 32nd USENIX Security Symposium (USENIX Security 23), pp. 2187 2204, 2023.
[783] Rulin Shao, Jacqueline He, Akari Asai, Weijia Shi, Tim Dettmers, Sewon Min, Luke Zettlemoyer, and Pang Wei Koh. Scaling retrieval-based language models with trillion-token datastore. arXiv preprint arXiv:2407.1285, 2024.
[784] Zhihong Shao, Yeyun Gong, Yelong Shen, Minlie Huang, Nan Duan, and Weizhu Chen. Enhancing retrieval-angmented large language models with itcrative genreaaalgennrration synergy. arXiv preprint arXio:2305.1520, 2023.
[785] Lloyd S Shapley et al. A value for n-person games. 1953.
[786] Sahel Sharifymoghaddam, Shivani Upadhyay, Wenhu Chen, and Jimmy Lin. Unirag: Universal re- trieval augmentation for multi-modal large language models. arXiv preprinl arXiv:2405.10311 2024.
[787] Mrinank Sharma, Meg Tong, Jesse Mu, Jerry Wei, Jorrit Kruthoff, Scott Goodfriend, Euan Ong, Alwin Peng, Raj Agarwal, Cem Anil, Amanda Askell, Nathan Bailey, Joe Benton, Emma Bluemke, Samuel R. Bowman, Eric Christiansen, Hoagy Cunningham, Andy Dau, Anjali Gopal, Rob Gilson, Lo- gan Graham, Logan Howard, Nimit Kalra, Taesung Lee, Kevin Lin, Peter Lofgren, Francesco Mosconi, Clare O'Hara, Catherine Olsson, Linda Petrini, Samir Rajani, Nikhil Saxena, Alex Silverstein, Tanya
138

---

Published in Transactions on Machine Learning Research (10/2025)
Singh, Theodore Sumers, Leonard Tang, Kevin K. Troy, Constantin Weisser, Ruiqi Zhong, Giulio Zhou, Jan Leike, Jared Kaplan, and Ethan Perez. Constitutional classifiers: Defending against uni- versal jailbreaks across thousands of hours of red teaming. arXiv preprint arXic:2501.18837 2025.
[788] Junhong Shen, Hao Bai, Lanjun Zhang, Yifei Zhou, Amrith Sethur, Shengbang Tong, Diego Caples, Nan Jiang, Tong Zhang, Amect Talwalkar, et al. Thinking vs. doing: Agents that reason by scaling lest-time interaction. Xie preprial or arXiv:2506.0700 2025.
[789] Lingfeng Shen, Weiting Tan, Sihao Chen, Yunmo Chen, Jingyu Zhang, Haoran Xu, Boyuan Zheng, Philipp Koehn, and Daniel Khashabi. The language barrier: Dissecting safety challenges of Ilms in multilingual contexts. arXiv preprint arxiv:240 19136, 2024.
[790] Tianhao Shen, Renren Jin, Yufei Huang, Chuang Liu, Weilong Dong, Zishan Guo, Xinwei Wu, Yan Liu, and Doyi Xiong. Large language model alignment: survey. arXiv preprint arXiv:2300.155 15025, 2023.
[791] Wei Shen, Rui Zheng, Wenyu Zhan, Jun Zhao, Shihan Dou, Tao Gui, Qi Zhang, and Xuanjing Huang, Loose lips sink ships: Mitigating length bias in reinforcement learning from human feedback. arXiv preprint arX0::2310.057 2023.
[792] Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang, "do anything now': Char- acterizing and evaluating in-the-wild jailbreak prompts on large language models. arXiv preprint arXiv:2308. 03825, 2023.
[793] Yaozong Shen, Lijie Wang, Ying Chen, Xinyan Xiao, Jing Liu, and Hua Wu. An Interpretability Evaluation Benchmark for Pre-trained Language Models, July 2022. URL http:://artttii org/ org/abs/ 2207. 13948. arXiv:2207.1394 [cs].
[794] Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face. In A. Oh, T. Neumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 18154-38180., Curran Associates, Inc., 2023. URL https: / http///poeeeeeeeeeeeeeeeeeees/ee neurips..erp...... cc feaper_fils/ppe
Confeeeeeeepeee--oooreeeeeeeeee Conferenceeeeeoeeeeeeeeo pdf.
(795] Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. The woman worked as babysitter: On biases in language generation. arXiv preprint ar orXii:1909.0133 2019.
[796] Ying Sheng, Shiyi Cao, Dacheng Li, Banghua Zhu, Zhuohan Li, Danyang Zhuo, Joseph E Gonzalez, and Ion Stoica. Fairness in serving large language models. In 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24), pp. 965 988, 2024.
[797] Dingfeng Shi, Jingyi Cao, Qianben Chen, Weichen Sun, Weizhen Li, Hongxuan Lu, Fangchen Dong, Tianrui Qin, King Zhu, Minghao Täu, Jian Yang, Ge Zhang, Jiaheng Liu, Changwang Zhang, Jun Wang, Yuchen Eleanor Jiang, and Wangchunshu Zhou. Taskcraft: Automated generation of agentic lasks, 2025. URL ttttpa///arxi org/ awwaaab/500 10055.
[798] Haizhou Shi, Yibin Wang, Ligong Han, Huan Zhang, and Hao Wang. Training-free bayesianization for low-rank adapters of large language models. arXiv preprint wxX.:2418.05723, 2024.
[799] Haizhou Shi, Zihao Xu, Hengyi Wang, Weiyi Qin, Wenyuan Wang, Yibin Wang, and Hao Wang Con- tinual learning of large language models: A comprehensive survey. arXiv preprint arX.v:240/.16789, 2024.
[800] Jiawen Shi, Yixin Liu, Pan Zhou, and Lichao Sun. Badgpt: Exploring security vulnerabilities of chatgpt via backdoor attacks to instructgpt. arXie preprint XXi::2304.12298 2023.
[801] Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zettlemoyer. Detecting protraining data from large language models. arXiv preprint arXio:2310. 16789 2023.
139

---

Published in Transactions on Machine Learning Research (10/2025)
[802] Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Sco, Rich James, Mike Lewis, Luke Zettle- moyer, and Wen-tau Yih. Replug: Retrieval-augmented black-box language models. ar preprinl arXic:2301.12652 2023.
[803] Connor Shorten and Taghi M Khoshgoftaar. A survey on image data augmentation for deep learning. Journal of big data, 6(1):1-48, 2019.
[804] Manli Shu, Jiongxiao Wang, Chen Zhu, Jonas Geiping, Chaowei Xiao, and Tom Goldstein. On the exploitability of instruction tuning. arXiv preprint 2iiv:2306.1119, 2023.
[805] Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. Retricval augmentation reduces hallucination in conversation. arXiv preprinl arXiv:2104007567, 2021.
[806] Chenglei Si, Zhe Gan, Zhengyuan Yang, Shuohang Wang, Jianleng Wang, Jordan Boyd-Graber, and Lijuan Wang. Prompting gpt-3 to be reliable, 2023.
[807] Sandipan Sikdar, Parantapa Bhattacharya, and Kieran Heese. Integrated directional gradients: Fea- ture interaction attribution for neural NLP models. In ACL/IJCNLP (1), pp. 865 878. Association for Computational Linguistics, 2021.
[808] Aniket Kumar Singh, Suman Devkota, Bishal Lamichhane, Uttam Dhakal, and Chandra Dhakal. The confidence-competencee gap in large language models: A cognitive study, 2023.
[809] Chandan Singh, Aliyah R Hsu, Richard Antonello, Shailee Jain, Alexander G Huth, Bin Yu, and Jianfeng Gao. Explaining black box text modules natural language with language models. arXiv preprint arXiv:2305.00 09863, 2023.
[810] Abhishek Sinha, Jiaming Song, Chenlin Meng, and Stefano Ermon. D2c: Diffusion-decoding models for few-shot conditional generation. Advances in Neural Information Processing Systems, 34:12533- 12548, 2021.
[811] Shamane Siriwardhana, Rivindu Weerasekera, Elliott Wen, Tharindu Kaluarachchi, Rajib Rana, and Suranga Nanayakkara. Improving the domain adaptation of retrieval augmented generation (rag) models for open domain question answering. Transactions of the Association for Computational Linguistics, 11:1-17, 2023.
[812] Shamane Siriwardhana, Mark McQuade, Thomas Gauthier, Lucas Atkins, Fernando Fernandes Neto, Luke Meyers, Anneketh Vij, Tyler Odenthal, Charles Goddard, Mary MacCarthy, et al. Domain adap- tation of llama3-70biinsruuct through continual pre-training and model merging: A comprehensive evaluation. Xii preprint urxiv:2406. 14971, 2024.
[813] Eric Michael Smith, Melissa Hall, Melanie Kambadur, Eleonora Presani, and Adina Williams. "i'm sorry to hear that": Finding new biases in language models with a holistic descriptor dataset. Tn Proceedings of the 2022 Conference on Empirical Methods in Natural Longuage Processing, pp. 9180- 9211, 2022.
[814] Charlic Snell, Jachoon Leo, Kelvin Xu, and Aviral Kumar. Scaling LLM Test-Time Compute Opti- mally can be More Effective than Scaling Model Parameters. arXiv, abs/2408.03311 2024.
[815] Jake C. Snell, Thomas P. Zollo, Zhun Deng, Toniann Pitassi, and Richard Zemel. Quantile risk control: A flexible framework for bounding the probability of high-loss predictions, 2022.
[816] Jascha. Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference machine learning, pp. 2256-2265. pmir, 2015.
[817] Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert-Vos:, Jeff Wu, Alec Radford, and Jasmine Wang. Release strategics and the social impacts of language models. ArXiv, abs/1998.09903, 2019. URL htttps://app semanticacholar htt.ccccccoaaoooorrrrreeeeeeeeee org/CorpusID 201666234.
140

---

Published in Transactions on Machine Learning Research (10/2025)
[818] Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Understand- ing and mitigating copying in diffusion models. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Newral Information Processing Systems, volume 36, PP. 47783-47808. Curran Associates, Inc., 2023. URL https://prooeeddnngs nwwriis...ppp...... cc/paper_ffe fil pdf.
[819] Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Diffusion art or digital forgery? investigating data replication in diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patlern Recognition (CVPR), pp. 6048 6058, June 2023.
[820] Chan Hoe Song, Jiaman Wu, Clayton Washington, Brian M Sadler, Wei-Lun Chao, and Yu Su. Llm- planner: Few-shot grounded planning for embodied agents with large language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1998-3009, 2023.
[821] Congzheng Song and Ananth Raghunathan. Information leakage in embedding models. In Proceedings of the 2020 ACM SIGSAC conference computer and communiicatioss securily, pp. 377 390, 2020.
[822] Feifan Song, Bowen Yu, Minghao Li, Haiyang Yu, Fei Huang, Yongbin Li, and Houfeng Wang. Preference ranking optimization for human alignment. ArXiv preprint, abs/2306.17492, 2023. URL
17492.
[823] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXie preprint arXiv:2011.13456, 2020.
[824] lomut-Teodor Sorodoc, Kristina Gulordava, and Gemma Boleda. Probing for Referential Information in Language Models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4177-4189, Online, July 2020. Association for Computational Linguistics. doi: 10.1 (653/v1/2020.aclmmabl ma...3.4. URL https: httppss//aalaathhllogoooo 00222000000000/2 acl-main.3em
[825] Claudio Spiess, David Gros, Kunal Suresh Pai, Michael Pradel, Md Rafiqul Islam Rabin, Amin Alipour, Susmit Jha, Prem Devanbu, and Toufique Ahmed. Calibration and correctness of language models for code, 2024.
[826] Jacob Mitchell Springer, Sachin Goyal, Kaiyue Wen, Tanishq Kumar, Xiang Yue, Sadhika Malladi, Graham Neubig, and Aditi Raghunathan. Overtrained language models are harder to fine-tune, ar preprint arXiv:2503. 19200 2025.
[827] Tejas Srinivasan, Ting- Yun Chang, Leticia Pinto Alva, Georgios Chochlakis, Mohammad Rostami, and Jesse Thomason. Climb: A continual learning benchmark for vision-and-language tasks. Advances in Neural Information Processing Systems, 35:29440 29453, 2022.
[828] Joe Stacey, Yonatan Belinkov, and Marek Rei. Supervising model allention with human explanations for robust natural language inference. In Proceedings of the AAAI conference artificial intelligence, volume 36, pp. 11349 11357, 2022.
829 Elias Stengel-Eskin and Benjamin Van Durme. Calibrated interpretation: Confidence estimation in semantic parsing. Transactions of the Association for Computational Lënguistics, 11-121--1211 2023.
[830] Elias Stengel-Eskin and Benjamin Van Durme. Did you mean..?? confidence-based trade-offs in semantic parsing. In The 2023 Conference Empiricat Methods in Natural Language Processing, 2023.
[83]] Elias Stengel-Eskin, Jimena Guallar-Blasco, Yi Zhou, and Benjamin Van Durme. Why did the chicken cross the road? rephrasing and analyzing ambiguous questions in vqa, 2023.
[832] Elias Stengel-Eskin, Kyle Rawlins, and Benjamin Van Durme. Zero and few-shot semantic parsing with ambiguous inputs. In The Twelfth International Conference Learning Represendations, 2023.
141

---

Published in Transactions on Machine Learning Research (10/2025)
[833] Elias Stengel-Eskin, Peter Hase, and Mohit Bansal. Lacie: Listener-aware finetuning for confidence calibration in large language models. arXie preprint or Xin:24005 21028 2024.
[834] Lukas Struppek, Dominik Hintersdorf, and Kristian Kersting. Rickrolling the artist: Injecting back- doors into text encoders for text-to-image synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4584-4596, 2023.
[835] Hongjin Su, Jungo Kasai, Chen Henry Wu, Weijia Shi, Tianlu Wang, Jiayi Xin, Rui Zhang, Mari Ostendorf, Luke Zettlemoyer, Noah A. Smith, and Tao Yu. Selective annotation makes language models better few-shot learners, 2022.
[836]
Jiayuan Su, Jing Luo, Hongwei Wang, and Lu Cheng. Api is enough: Conformal prediction for large language models without logit-access. arXiv preprint arXiv:2403.0122 2024.
[837] Jinyan Su, Terry Zhuo, Di Wang, and Preslav Nakov. DeL: Leveraging log rank information for zera-shot detection of machine-generated text. In Houda. Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 12395 12412, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1. /tt///////22///// findi ngs-emmlp.827. URL https://aclantthooogg org.org//// 2tt..........n.....e.e. findings-emnll eeekings-emnl..87.. 827.
[888] Yixuan Su, Tian Lan, Huayang Li, Jialu Xu, Yan Wang, and Deng Cai. Pandagpt: One model to instruction-follow them all, 2023.
[839] Carole H Sudre, Wenqi Li, Tom Vercauteren, Sebastien Ourselin, and M Jorge Cardoso. Generalised dice overlap as deep learning loss function for highly unbalanced segmentations. In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: Third International Workshop, DLMIA 2017, and 7th International Workshop, ML-CDS 2017, Held Conjunction with MICCAI 2017, Québec City, QC, Canada, September 14, Proceedings 9, pp. 240- -48.228.722 Springer, 2017.
[844]] Yang Sui, Huy Phan, Jinqi Xiao, Tianfang Zhang, Zijic Tang, Cong Shi, Yan Wang, Yingying Chen, and Bo Yuan. Disdet: Exploring detectability of backdoor allack diffusion models. arXie preprinl arXiv:2402.022 02739, 2024.
[841] Jiao Sun, Yufei Tian, Wangchunshu Zhou, Nan Xu, Qian Hu, Rahul Gupta, John Wieting, Nanyun Peng, and Xuezhe Ma. Evaluating large language models on controlled generation lasks. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Em- pirical Methods in Natural Language Processing, pp. 3155-3168, Singapore, December 2023. Asso- ciation for Computational Linguistics. doi: 10.18653/ 22222222222mpp... main..190 URL https: //aclanthology org/2022 2023. emnlp-main. 190...900.
[842] Weisong Sun, Yuchen Chen, Guanhong Tao, Chunrong Fang, Xiangyu Zhang, Quanjun Zhang, and Bin Luo. Backdooring neural code search. Xiv preprint arXie:2305.1 17006, 2023.
[843]] Yu Sun, Xinhao Li, Karan Dalal, Jiarui Xu, Arjun Vikram, Genghan Zhang, Yann Dubois, Xinlei Chen, Xiaolong Wang, Sammi Koyejo, et al. Learning to (learn at test time): Rnns with expressive hidden states. arXiv preprint Xiv::2247..0 00.:2407,04620, 2024.
[844] Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liangyan Gui, Yu-Xiong Wang. Yiming Yang, Kurt Keutzer, and Trevor Darrell. Aligning large mul- timodal models with factually augmented RLHF. In Lun-Wei Ku, Andre Martins, and Vivek Sriku- mar (eds.), Findings of the Association for Computational Linguistics ACL 2024, pp. 13088-13110, Bangkok, Thailand and virtual meeting, August 2024. Association for Computational Linguistics. URL https://aclanttholgg org/2024, iindingsaac.777
[845] Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. Inter- national Conference on Machine Learning (ICML., 2017.
142

---

Published in Transactions on Machine Learning Research (10/2025)
[846] Harini Suresh and John V. Guttag. A framework for understanding sources of harm throughout the machine learning life cycle. Equity and Access in Algorithmss, Mechanisms, and Oplimization, 2019. URL https///ap..emmatiicchoool
www.seeeeaeooeteeee...e.. Betpihooaarrrg//ooreeooeeeee Trg//orpusID: 235436386.
[847] Christian Szegedy, Vincent Vanhoucke, Sergey Loffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pottern recognition, pp. 2818-228822111 2826, 2016.
[848] Alex Tamkin, Kunal Handa, Avash Shrestha, and Noah Goodman. Task ambiguity in humans and language models, 2022.
[849] Di Tang, XiaoFeng Wang, Haixu Tang, and Kehuan Zhang. Demon in the variant: Statistical analysis of (DNNs} for robust backdoor contamination detection. In 30th USENIX Security Symposium (USENIX Security 21), pp. 1541-1558, 2021.
[850] Xiangru Tang, Tianrui Qin, Tianhao Peng, Ziyang Zhou, Daniel Shao, Tingting Du, Xinming Wei, Peng Xia, Fang Wu, He Zhu, Ge Zhang, Jiaheng Liu, Xingyao Wang, Sirui Hong, Chenglin Wu, Hao Cheng, Chi Wang, and Wangchunshu Zhou. Agent kb: Leveraging cross-domain experience for agentic problem solving, 2025. URL
[851] Lue Tao, Lei Feng, Jinfeng Yi, Sheng-Jun Huang, and Songcan Chen. Better safe than sorry: Pre- venting delusive adversaries with adversarial training. Advances in Neural Information Processing Systems, 34:16209-16228 2021.
[852] Yitian Tao, Liyan Ma, Jing Yu, and Han Zhang. Memory-based cross-modal semantic alignment network for radiology report generation. IEEE Journal of Biomedical and Health Informatics, 2024.
[853] Rohan Taori, Ishaan Guirajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instruction-folloving Ilama model. https: //github. //github.com//aattae wwt.e//ttttttaa////eeea aettaaford_appaaeee 2023.
[854] Pittawat Taveekitworachai, Febri Abdullah, Mustafa Can Gursesli, Mury F Dewantoro, Siyuan Chen, Antonio Lanata, Andrea Guazzini, and Ruck Thawonmas. Breaking bad: Unraveling influences and risks of user inputs to chatgpt for game story generation. In International Conference Interactive Digital Storytelling, pp. 285-296. Springer, 2023.
[855] Ian Temney, Dipanjan Das, and Ellie Pavlick. BERT rediscovers the classical NLP pipeline. In ACL (1), pp. 4593 4601. Association for Computational Linguistics, 2019.
[856] lan Tenney Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R. Thomas McCoy, Najoung Kim, Benjamin Van Durme, Samuel R. Bowman, Dipanjan Das, and Ellie Pavlick. What do you learn from context? probing for sentence structure in contextualized word representations. ArXiv, abs/ hes/1905.06316 2019. URL https: //ppi.ppp..../ semanticscholat Prg/CorpusID:: 108300988.
[857] Himanshu Thakur, Atishay Jain, Praneetha Vaddamanu, Paul Pu Liang, and Louis-Philippe Morency. Language models get a gender makcover: Mitigating gender bias with fow-shot data interventions. Xix preprind arXiv:2306.0 Stl:2306600597,, 2023.
[858] David Thiel. Identifying and eliminating csam in generative ml training data and models. Stanford Internet Observatory, 2023.
[859] Edward Tian. GPTzero: An ai text detector., 2023. URL https://gptzee me/.
[860] Fengrui Tian, Tianjiao Ding, Jinqi Luo, Hancheng Min, and René Vidal. Voyaging into perpetual dynamic scenes from a single view. In International Conference on Computer Vision (ICCV), 2025.
[861] Katherine Tian, Eric Mitchell, Huaxiu Yao, Christopher D. Manning, and Chelsea Finn. Fine-tuming language models for factuality, 2023.
143

---

Published in Transactions on Machine Learning Research (10/2025)
[862] Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher D. Manning. Just ask for calibration: Strategies for eliciting calibrated confi- dence scores from language models fine-tuned with human feedback, 2023.
[863] Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. arxie prepriut arXiv:2404. 02900, 2024.
[864] Ran Tian, Shashi Narayan, Thibault Sellam, and Ankur P Parikh. Sticking to the facts: Confident decoding for faithful data-to-text generation. arXiv preprint ar arXi::191000888 08684, 2019.
[865] Rubèn Tito, Khanh Nguyen, Marlon Tobaben, Raouf Kerkouche, Mohamed Ali Souibgui, Kangsoo Jung, Lei Kang, Ernest Valveny, Antti Honkela, Mario Fritz, et al. Privacy-aware document visual question answering. arXiv preprint 0rXiv23322101100 2023.
[866] Marcus Tomalin, Bill Byrne, Shamna Concannon, Danielle Saunders, and Stefanie Ullmann. The practical ethics of bias reduction in machine translation: Why domain adaptation is better than data debiasing. Ethics and Information Technology. pp. 1-15, 2021.
[867] Christian Tomani, Kamalika Chaudhuri, Ivan Evtimov, Daniel Cremers, and Mark Ibrahim. Uncertainty-based abstention in Ilms improves safety and reduces hallucinations, 2024.
[868] Haibo Tong, Zhaoyang Wang, Zhaorun Chen, Haomian Ji, Shi Qiu, Siwei Han, Kexin Geng, Zhongkai Xue, Viyang Zhou, Peng Xia, et al. Mj-video: Fine-grained benchmarking and rewarding video preferences in video generation. arXiv preprint arXiv:2502.01111 2025.
[869] Shengbang Tong, Erik Jones, and Jacob Steinhardt. Mass- producing failures of multimodal systems with language models. orxiv preprinl arXiv:2306. 000.996..11003 12105, 2023.
[870] Shengbang Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai Charitha Akula, Jihan Yang, Shusheng Yang, Adithya Iyer, Xichen Pan, et al. Cambrian-1: A fully open, vision-centric exploration of multimodal llms. In NeurIPS, 2024.
[871] Shengbang Tong, David Fan, Jiachen Zhu, Yunyang Xiong, Xinlei Chen, Koustuv Sinha, Michael Rabbat, Yann LeCun, Saining Xie, and Zhuang Liu. Metamorph: Multimodal understanding and generation via instruction tuning. aix preprint or XrXi::24 f2.:242211166 14164, 2024.
[872] Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie, Eyes wide shut? exploring the visual shortcomings of multimodal 1mss. arXiv preprint arXiv:2401066200 2024.
[873] Merean Topkara, Umut Topkara, and Mikhail J. Atallah. Words are not enough: sentence level natural language watermarking. In Workshop on Medical Cyber-Physsie Systems, 2006. URL https: //api. aemppesssmaniccchllaaa .oree.ogggooeee mttg/ooppuSID:55588000
[874] Umut Topkara, Mercan Topkara, and Mikhail J. Atallah. The hiding virtues of ambiguity: quantifi- ably resilient watermarking of natural language text through synonym substitutions. In Workshop on Mullimedia. & Security, 2006. URL attps://api.. stt.appsstttticccollllaa.a weweaaattccccolaa.oooo//ooaaraaa
[875] Lucas Torroba Hennigen, Adina Williams, and Ryan Cotterell. Intrinsic Probing through Dimension Selection. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Process- ing (EMNLP), pp. 197-216, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v1/2000 11116633/11/020.emm////// wet.0mmmmllmmmaie main. 15. URL https: / / h/tpa/aaaaannnoo.....m.. org/2020. emmlp-main 15.
[876] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint ar Xii:2337700228. 2023.
[877] Brandon Trabucco, Kyle Doherty, Max Gurinas, and Ruslan Salakhutdinov, Effective data augmen- tation with diffusion models, 2023.
144

---

Published in Transactions on Machine Learning Research (10/2025)
[878] Florian Tramer, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick Mc- Daniel. Ensemble adversarial training: Attacks and defenses. arXiv preprint Xix: 705.07204, 2017.
[879] Marcos V. Treviso, Alexis Ross, Nuno Miguel Guerreiro, and André Martins. CREST: A joint framework for rationalization and counterfactual text. generation. In ACL (1), pp- 15109-15126. Association for Computational Linguistics, 2023.
[880] Eduard Tulchinskii, Kristian Kuznetsov, Laida Kushnareva, Daniil Cherniavskii, S. Barannikov, Trina Piontkovskaya, Sergey I. Nikolenko, and Evgeny Burnaev. Intrinsic dimension estimation for robust detection of ai-generated texts. ArXiu, abs/2306.04423, 2023. URL https://api: semanticschola
r. ..orgCCorpusDD :tt.pppssSI:2111787100111 259108779.
[881] Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J. Vazquez, Ulisse Mini, and Monte MacDiarmid. Activation addition: Stecring language models without optimization, 2023.
[882] Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J Vazquez, Ulisse Mini, and Monte MacDiarmid. Activation addition: Stecring language models without optimization. arXiv e-prinls, pp. arXiv 2308, 2023.
[883] Miles Turpin, Julian Michael, Ethan Perez, and Samuel Bowman. Language models don't always say what they think: Unfaithful explanations in chain-of-thought prompting. Advances in Neural Information Processing Systems, 36:74952-747 74965, 2023.
[884] UBS. Ubs: Chatgpt may be the fastest growing app of all time. htllss// http:://cchusiiess..om//////
chatgpt-is feftpp--is-the-asseee tastthhtttttteeeeee fastest-growinng------ app-of-all- all--tete 2024.
(885) Adaku Uchendu, Thai Le, Kai Shu, and Dongwon Lee. Authorship attribution for neural text generation. In Conference on Empirical Methods in Natural Language Processing, 2020. URL
www.seeeeaaaooooi.eooee http::// org/Corpuse : 221835708.
[886]
Adaku Uchendu, Jooyoung Lee, Hua Shen, Thai Le, Ting-Hao 'Kenneth' Huang, and Dongwon Lee. Does human collaboration enhance the accuracy of identifying llm-generated deepfake texts? Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, 2023. URL https: //api. www.sesssmiiiccccool.ooooooo org arg/CorpusID: org/CorppsII::27991334 257913864.
[887] Dennis Ulmer. On uncertainty in natural language processing. arXiv preprint arXiv:2410.03446, 2024.
[888] Dennis Ulmer, Martin Gubri, Hwaran Lee, Sangdoo Yun, and Seong Joon Oh. Calibrating large language models using their generations only. arXiv preprint arXiv:240s. 05977, 2024.
[889] Dennis Ulmer, Elman Mansimov, Kaixiang Lin, Justin Sun, Xibin Gao, and Yi Zhang. Bootstrapping llmeb task-oriented dialogue agents via self-talk. arXiv preprint Xrx 05033, 2024.
[890] Dennis Ulmer, Chrysoula Zerva, and André FT Martins. Non-exchangeable conformal language generation with nearest neighbors. arXie preprint arXiv:2402.00707, 2024.
[891] Logesh Kumar Umapathi, Ankit Pal, and Malaikannan Sankarasubbu. Mod-halt: Medical domain hallucination test for large language models. arXiv preprint arXXi::2377.153/3 2023.
[892] Liam der Poel, Ryan Cotterell, and Clara Meister. Mutual information alleviates hallucinations in abstractive summarization. arXiv preprint arXi:: 2022.
[893] Thanh Van Le, Hao Phung, Thuan Hoang Nguyen, Quan Dao, Ngoc N Tran, and Anh Tran. Anti-dreambooth: Protecting users from personalized text-to-image synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2116-2127, 2023.
[894] Nocraj Varshney, Wenlin Yao, Hongming Zhang, Jianshu Chen, and Dong Yu. A stitch in time saves nine: Detecting and mitigating hallucinations of Ilms by validating low-confidence generation. arXiv preprint ar arXi::2007.055 039977, 2023.
145

---

Published in Transactions on Machine Learning Research (10/2025)
[895] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Iilia Polosukhin. Attention is all you need. Advances in newral informalion processing systems, 30, 2017.
[896] Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. Cider: Consensus-based image de- scription evaluation. In Proceedings of the TEEE conference on computer vision and pallern recogni- tion, pp. 4566-4575, 2015.
[897] Sara Veldhoen, Dicuwke Hupkes, and Willem Zuidema. Diagnostic classifiers revcaling how neural networks process hierarchical structure. In NIPS 2016 Workshop on Cogrilive Compulation, 2016.
[898] Pranav Narayanan Venkit, Sanjana Gautam, Ruchi Panchanadikar, Ting-Hao' Kenneth Huang, and Shomir Wilson. Nationality bias text generation. arXw preprint Xix::22002.0 02463, 2023.
[899] Aayush Atul Verma, Amir Sacidi, Shamanthak Hegde, Ajay Therala, Fenil Denish Bardoliya, Na- garaju Machavarapu, Shri Ajay Kumar Ravindhiran, Srija Malyala, Agneet Chatterjee, Yezhou Yang, et al. Evaluating multimodal large language models across distribution shifts and augmentations. Tn Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 5314
5324, 2024
[900] Vivek Kumar Verma, Eve Fleisig, Nicholas Tomlin, and Dan Klein. Ghostbuster: Detecting text ghostwritten by large language models. ArXio, abs/2305.15044, 2023. URL https://api semantic scholar. Cttpaarrog//C//oCC// 258865787.
[901] Jesse Vig. Bertviz: A tool for visualizing multihead self-attention in the bert model. In ICLR workshop: Debugging machine learning models, volume 23, 2019.
[902] Jesse Vig and Yonatan Belinkov. Analyzing the structure of attention in a transformer language model. In BlackbozNLPAACL pp. 63 76. Association for Computational Linguistics, 2019.
[903] Elena Voita, Pavel Serdyukov, Rico Semnrich, and Ivan Titov. Contexl-aware neural machine trans- lation learns anaphora resolution. In ACL (1), pp. 1264 1274. Association for Computational Lin- guistics, 2018.
[904] Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned. arXiv preprint arXic:1905.09413, 2019.
[905] Elena Voita, Rico Sennrich, and Ivan Titov. Analyzing the source and target contributions to predic- tions in neural machine translation. In ACL/IJCNLP (1), pp. 1126-1140. Association for Computa- tional Linguistics, 2021.
[906] Ellen M Voorhees. Natural language processing and information retrieval. In International summer school on information extraction, pp. 32-48. Springer, 1999.
[907] Vladimir Vovk, Akimichi Takemura, and Glenn Shafer. Defensive forecasting for linear protocols. In Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics, 2005.
[908] Oskar Van Der Wal, Pictro Lesci, Max Muller-Eberstein, Naomi Saphra, Hailey Schoelkopf, Willem Zuidemna, and Stella Biderman. Polypythias: Stability and outliers across fifty language model pre- training runs, 2025.
[909] Alexander Wan, Eric Wallace, Sheng Shen, and Dan Klein. Poisoning language models during in- struction tuning. In International Conference on Machine Learning, 2023.
[910] Fanqi Wan, Xinting Huang, Deng Cai, Xiaojun Quan, Wei Bi, and Shumingg Shi. Knowledge fusion of large language models. Xiv preprint 2rx.u:4001.1499 2024.
146

---

Published in Transactions on Machine Learning Research (10/2025)
[911] Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen- Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jingren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, Pandeng Li, Pingyu Wu, Ruihang Chu, Ruili Feng, Shiwei Zhang, Siyang Sun, Tao Fang, Tianxing Wang. Tianyi Gui, Tingru Weng, Tong Shen, Wei Lin, Wei Wang, Wei Wang, Wenmeng Zhou, Wente Wang, Wenting Shen, Wenyuan Yu, Xianzhong Shi, Xiaoming Huang, Xin Xu, Yan Kou, Yangyu Lv, Yifei Li, Yijing Liu, Yiming Wang, Yingya Zhang, Yitong Huang, Yong Li, You Wu, Yu Liu, Yulin Pan, Yum Zheng, Yuntao Hong, Yupeng Shi, Yutong Feng, Zeyinzi Jiang, Zhen Han, Zhi-Fan Wu, and Ziyu Liu. Wan: Open and advanced large-scale video generative models. ar Xie preprint or arxiv:2003 20314. 2025.
[912] Alex Wang and Kyunghyun Cho. Bert has a mouth, and it must speak: Bert as a markov random field language model. arXiv preprint arXiv:192, 04094. 2019.
[913] Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, You Wu, Luke Zettlemoyer, and Huan Sun. Towards understanding chain-of-thhe th-nu-tf prompting: An empirical study of what matters. arXiv preprint arXiv:2212. 10001, 2022.
[914] Deng- Bao Wang, Lei Feng, and Min-Ling Zhang. Rethinking calibration of deep neural networks: Do not be afraid of overconfidence. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 11809-11820. Curran Associates, Inc., 2021. URL attpps//prroceedingss wewpeeediinnnnnnniis..oo.......e cc/ cc/paper_fil fill Papere-pppppppppp
[915] Hao Wang and Dit-Yan Yeung. Towards bayesian deep learning: A framework and some existing methods. TDKE, 28(12):3399 3408, 2016.
[916] Hao Wang and Dan Yeung. A survey on bayesian deep learning. CSUR, 53(5):1 37, 2020.
[917] Hao Wang, Shangwei Guo, Jialing He, Kangjie Chen, Shudong Zhang, Tianwei Zhang, and Tao Xiang, Eviledit: Backdooring text-to-image diffusson models in one second. In ACM Multimedia 2024, 2024. net//formmidd-bbaaS000
URL https::/openreeeeee
[918] Haoxiang Wang, Yong Lin, Wei Xiong, Rui Yang, Shizhe Diao, Shuang Qiu, Han Zhao, and Tong Zhang. Arithmetic control of 1ms for diverse user preferences: Directional preference alignment with multi-objective rewards. arXio preprint arXiv:2402.1857. 2024.
[919] Hengyi Wang, Shiwei Tan, Zhiqing Hong, Desheng Zhang, and Hao Wang. Variational language concepts for interpreting foundation language models. In EMNLP, 2024.
[920] Hengyi Wang, Shiwei Tan, and Hao Wang. Probabilistic conceptual explainers: Towards trustworthy conceptual explanations for vision foundation models. In ICML, 2024.
[921] Hongmin Wang. Revisiting challenges in data-to-text generation with fact grounding. In Proceedings of the 12th International Conference on Natural Language Generalion, pp. 311 322, 2019.
[922] Huandong Wang, Wenjie Fu, Yingzhou Tang, Zhilong Chen, Yuxi Huang, Jinghua Piao, Chen Gao, Fengli Xu, Tao Jiang, and Yong Li. A survey on responsible 1lms: Inherent risk, malicious use, and mitigation strategy. arXiv preprint arXiv:2501. (al.:2511.04911.11 2025.
[923] Junyang Wang, Yuhang Wang, Guohai Xu, Jing Zhang, Yukai Gu, Haitao Jia, Ming Yan, Ji Zhang, and Jitao Sang. An Imm-free multi-dimensional benchmark for milms hallucination evaluation. arXiv Xiv preprint arXiv:2311.07397, 2023.
[924] Junyang Wang, Yiyang Zhou, Guohai Xu, Pengcheng Shi, Chenlin Zhao, Haiyang Xu, Qinghao Ye, Ming Yan, Ji Zhang, Jihua Zhu, et al. Evaluation and analysis of hallucination in large vision- languaee models. or arxiv preprint arxi:2308.. 15126, 2023.
147

---

Published in Transactions on Machine Learning Research (10/2025)
[925] Lean Wang, Wenkai Yang, Deli Chen, Hao Zhou, Yankai Lin, Fandong Meng, Jie Zhou, and Xu Sun. Towards codable watermarking for injecting multi-bit information to 1lm. 2023. URL https://api. semanticscholar, htt///CuuIDD20033442..
[926] Liang Wang, Nan Yang, and Furu Wei. Learning to retrieve in-context examples for large language models. arXiv preprint arXiv:2307.07164 2023.
[927] Lingzhi Wang, Xingshan Zeng, Jinsong Guo, Kam-Fal Fai Wong, and Georg Gottlob. Selective forget- ting: Advancing machine unlearning techniques and evaluation in language models. arXiv preprint ar arXiv:2402.000 058113 2024.
[928] Noah Wang, Z.y. Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou, Yuhan Wu, Hongcheng Guo, Ruitong Gan, Zehao Ni, Jian Yang, Man Zhang, Zhaoxiang Zhang, Wanli Ouyang, Ke Xu, Wenhao Huang, Jie Fu, and Junran Peng. RoleLLM: Benchmarking, eliciting, and enhancing role-playing abilities of large language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Pindings of the Association for Computational Linguisties ACL 2024, pp. 14743 14777, Bangkok, Thailand and virtual meeting, August 2024. Association for Computational Linguistics. URL https: //aclanthology. org/2024 findings-acl.877
[929] Qingquan Wang. Invisible watermark, 2020. URL atermark.
[930] Ruida Wang, Wangchunshu Zhou, and Mrinmaya Sachan. Let's synthesize step by step: Iterative dataset synthesis with large language models by extrapolating errors from small models. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Lin- quistics: EMNLP 2023, pp. 11817 -1871, Singapore, December 2023. Association for Computational URL btttp::/accantaaae htw.hhoooggoogg/oo..o..eo org/2023.fr
Linguistics. doi: 10. 10.18653/v1/22 /t0.///ii///....ddnnggg Bhings-emnlp.991..
[931] Sibo Wang, Jie Zhang, Zheng Yuan, and Shiguang Shan. Pre-trained model guided fine-tuning for zero-shot adversarial robustness. arXiv preprinl ar Xiw:2440.043300 2024.
[932] Song Wang, Yaochen Zhu, Haochen Liu, Zaiyi Zheng, Chen Chen, and Jundong Li. Knowledge editing for large language models: A survey. ACM Computing Surveys, 2023.
[933] Tiannan Wang, Jiamin Chen, Qingrui Jia, Shuai Wang, Ruoyu Fang, Huilin Wang, Zhaowei Gao, Chunzhao Xie, Chuou Xu, Jihong Dai, Yibin Liu, Jialong Wu, Shengwei Ding, Long Li, Zhiwei Huang, Xinle Deng, Teng Yu, Gangan Ma, Han Xiao, Zixin Chen, Danjun Xiang, Yunxia Wang, Yuanyuan Zhu, Yi Xiao, Jing Wang, Yiru Wang, Siran Ding, Jiayang Huang, Jiayi Xu, Yilihamu Tayier, Zhenyu Hu, Yuan Gao, Chengfeng Zheng, Yueshu Ye, Yihang Li, Lei Wan, Xinyue Jiang, Yujie Wang, Siyu Cheng, Zhule Song, Xiangru Tang, Xiaohua Xu, Ningyu Zhang, Huajun Chen, Yuchen Eleanor Jiang, and Wangchunshu Zhou. Weaver: Foundation models for creative writing, 2024. URL mttps://arxit www./os//////oooooooooooooe 2abbb.//40..2220
[934] Xiyao Wang, Wichayaporn Wongkamjan, Ruonan Jia, and Furong Huang, Live in the moment: Learn- ing dynamics model adapted to evolving policy. In International Conference on Machine Learning, pp. 36470-36493 PMLR, 2023.
[935] Xiyao Wang, Jiuhai Chen, Zhaoyang Wang, Yuhang Zhou, Yiyang Zhou, Huaxiu Yao, Tianyi Zhou, Tom Goldstein, Parminder Bhatia, Furong Huang, et al. Enhancing visual-language modality align- ment in large vision language models via self-improvement. ar preprint ur urXiv::405. 110...1597... 2024.
[936] Xiyao Wang, Ruijie Zheng, Yanchao Sun, Ruonan Jia, Wichayaporn Wongkamjan, Huazhe Xu, and Furong Huang. Coplanner: Plan to roll out conservatively but to explore optimistically for model- based rl. In The Twelfth International Conference on Learning Representations, 2024. URL https: //openreview.e htt.:aaeeeeettorrrt/ddseeeeeer
148

---

Published in Transactions on Machine Learning Research (10/2025)
[937] Xiyao Wang, Yuhang Zhou, Xiaoyu Liu, Hongjin Lu, Yuancheng Xu, Feihong He, Jachong Yoon, Taixi Lu, Gedas Bertasius, Mohit Bansal, et al. Mementos: A comprehensive benchmark for multimodal large language model reasoning over image sequences. In Annual Meeting of the Association for Computational Línguistics (ACL), 2024.
[938] Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-Consistency Improves Chain of Thought Reasoning in Language Models. In International Conference on Learning Representations (ICLR), 2023.
[939] Yibin Wang, Haizhou Shi, Ligong Han, Dimitris Mctaxas, and Hao Wang. Blob: Bayesian low-rank adaptation by backpropagation for large language models. In NeurIPS, 2024.
[941]] Yifan Wang and Vera Demberg, A parameter-efficint multi-objective approach to mitigate stereo- typical bias in language models. In Agnieszka Faleńska, Christine Basta, Marta Costa-jussà, Scraphina Goldfarb-Tarrant, and Debora Nozza (cds.), Proceedings of the 5th Workshop on Gen- der Bias in Natural Language Processing (GeBNLP), pp. 1 19, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. http:///1122../////// //aclanthology 2024/ gebnlp-1.1/.
URL https:
[941] Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language model with self generated instructions. arXiv preprint 0rXiv:2212.10550 2022.
[942] Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language model with self-generated instructions. corr, abs/2212.110560, 2022. doi: 10.48550. ArXiv preprint, abs/2212 10560, 2022. URL https://arxiv: /rg/abs/2212. 10560.
[943] Yu Wang, Xiusi Chen, Jingbo Shang, and Julian McAuley. Memoryllm: Towards self-updatable large language models. ar preprint or orXiv:2402.0462) 2024.
[944] Yu Wang, Zeyuan Zhang, Julian McAuley, and Zexue He. Lvehat: Facilitating long video compre- hension. arXiv preprint ar Xix:22440.. 12079, 2024.
[945] Zefeng Wang, Zhen Han, Shuo Chen, Fan Xue, Zifeng Ding, Xun Xiao, Volker Tresp, Philip Torr, and Jindong Gu. Stop reasoning! when multimodal ilms with ofsin-of-thought reasoning meets adversarial images. Conference On Language Modeling (COLM), 2024.
[946] Zhaoyang Wang, Shaohan Huang, Yuxuan Liu, Jiahai Wang, Minghui Song, Zihan Zhang, Haizhen Huang, Furu Wei, Weiwei Deng, Feng Sun, and Qi Zhang. Democratizing reasoning ability: Tailored learning from large language model. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Process- ing, pp. 1948-1966, Singapore, December 2023. Association for Computational Linguistics. URL https://aclantholl wwwpoolooyyyoog////// org/2023. emn mmmlp-main..11
[947] Zhaoyang Wang, Weilei Hc, Zhiyuan Liang, Xuchao Zhang, Chetan Bansal, Ying Wei, Weitong Zhang, and Huaxiu Yao. Cream: Consistency regularized self-rewarding language models. arXiv preprinl arXiv:2410.10 10.12735, 2024.
[948] Zhiyuan Wang, Jinhao Duan, Lu Cheng, Yue Zhang, Qingni Wang, Hengtao Shen, Xiaofeng Zhu, Xi- aoshuang Shi, and Kaidi Xu. Comu: Conformal uncertainty in large language models with correctness coverage guarantees. Xix preprint (0X:v:2407.00400 2024.
[949] Zhiyuan Wang, Jinhao Duan, Chenxi Yuan, Qingyu Chen, Tianlong Chen, Huaxiu Yao, Yue Zhang, Ren Wang, Kaidi Xu, and Xiaoshuang Shi. Word-soquence entropy: Towards uncertainty estimation in free-form medical question answering applications and beyond. orXiv preprint arXiv:2402 14259, 2024.
149

---

Published in Transactions on Machine Learning Research (10/2025)
[950] Zhongqi Wang, Jie Zhang, Shiguang Shan, and Xilin Chen. T2ishield: Defending against backdoors on text-to-image diffusion models. arxiv preprint orXi::207.04225, 2024.
[951] Futa Waseda and Antonio Tejero-de Pablos. Leveraging many-lo-many relationships for defending against visual-language adversarial attacks. arXiv preprint arXie:2405.18770 2024.
[952]
WashingtonPostt. He made a children's book using AI. Then came the rage. httpp:://ww..vaahiingtot post. http:////e.cccllooo////////////a htw.rp/eweeoooooooettttooeeoeere 2022.
[953] Michelle Wastl, Jannis Vamvas, and Rico Sennrich. UZH at SemEval-2025 task 3: Token-level self- consistency for hallucination detection. In Proceedings of the 19th International Workshop on Seman- tic Evaluation (SemEval-2025), 2025.
[954] Kellie Webster, Xuezhi Wang, Ian Tenney, Alex Beutel, Emily Pitler, Ellie Pavlick, Jilin Chen, Ed Chi, and Slav Petrov. Mcasuring and reducing gendered correlations in pre-trained models. arXiv preprint xrxw::200. 000::2000.06032, 2020.
[955] Ryan Webster. A reproducible extraction of training images from diffusion models. arXiv preprint ar orXi:22305.060 08000,4 2023.
[956] Alexander Wei, Nika Haghtalab, and Jacob Steinhardt, Jailbroken: How does Ilm safety training fail? Xiv preprinl urXiv:2307.02, 2023.
[957] Chengkun Wei, Wenlong Meng, Zhikun Zhang, Min Chen, Minghu Zhao, Wenjing Fang, Lei Wang, Zihui Zhang, and Wenzhi Chen. Lmsanitator: Defending prompt-tuning against task-agnostic back- arXio preprint arXiv:2308.1190 2023.
doors.
[958] Cong Wei, Yang Chen, Haonan Chen, Hexiang Hu, Ge Zhang, Jie Fu, Alan Ritter, and Wenhu Chen. Uniir: Training and benchmarking universal multimodal information retrievers. arXiv preprint ar arXio:2811.17130 2023.
[959] Hongliang Wei, Xingtao Wang, Xianqi Zhang, Xiaopeng Fan, and Debin Zhao. Toward stable, fair, and comprchensive evaluation of object hallucination in large vision-language models. In The Thirty-eighth Annual Conference on Neural Information Processing Syslems, 2024.
[960] Hongxin Wei, Renchunzi Xie, Hao Cheng, Lei Feng, Bo An, and Yixuan Li. Mitigating neural network overconfidence with logit normalization, 2022.
[961] Jason Wei and Kai Zou. Eda: Easy data augmentation techniques for boosting performance text classification tasks. arXiv preprint XrXi::1901.11990 2019.
[962] Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoe V Lc. Finctuned language models are zcro-shot learners. arXiv preprint arXiv:2109.0 01652, 2021.
[963] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. Chain-of-thoughu prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824- 24483784--4837, 2022.
[964] Jerry Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, Yifeng Lu, Xinyun Chen, Hanxiao Liu, Da Huang, Denny Zhou, et al. Larger language models do in-context learning differently. arXiv preprint arXiv:2303. 233.23303,0844,, 2023.
[965] Lai Wei, Zihao Jiang, Weiran Huang, and Lichao Sun. Instructiongpt-4: 200-instruction paradigm for fine-tuning minigpt-4, 2023.
[966] Laura Weidinger, Jonathan Uesato, Maribeth Rauh, Conor Griffin, Po-Sen Huang, John Mellor, Amelia Glacse, Myra Cheng, Borja Balle, Atoosa Kasirzadch, et al. Taxonomy of risks posed by language models. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pp. 214 229, 2022.
150

---

Published in Transactions on Machine Learning Research (10/2025)
[967] Orion Weller, Benjamin Van Durme, Dawn Lawrie, Ashwin Paranjape, Yuhao Zhang, and Jack Hessel. Promptriever: Instruction-trained retrievers can be prompted like language models. or arXie preprinl arXiv:2409. 11136, 2024.
[968] Bingbing Wen, Jihan Yao, Shangbin Feng, Chenjun Xu, Yulia Tsvetkov, Bill Howe, and Lucy Lu Wang. Know your limits: survey of abstention in large language models. arXiv preprint 4071128407.18418 2024.
[969] Rui Wen, Zheng Li, Michael Backes, and Yang Zhang. Membership inference attacks against in- context learning. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Com- mumications Securily, pp. 3481 -495,, 2024.
[970] Yuxin Wen, Neel Jain, John Kirchenbauer, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Hard prompts made casy: Gradient-basod discrete optimization for prompt tuning and discovery. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Newral Information Processing Systems, volume 36, pp. 51008-51025. Curan Associates, Inc., 2023. URL https://proceedee http:eer...p..pppeeeeeeeeeeee http://////ap/////////////000000 c922fadf1- Confrteeeeeee----foeeeeeeeeeee
[971] Yuxin Wen, John Kirchenbauer, Jonas Geiping, and Tom Goldstein. Tree-ring watermarks: Finger- prints for diffusion images that are invisible and robust. arXiv preprint arXiv:2305. 200300 2023.
1972] Yuxin Wen, Yuchen Liu, Chen Chen, and Lingjuan Lyu. Detecting, explaining, and mitigating mem- orization in diffusion models. In The Twelfth International Conference on Learning Represenlations, 2024. URL https://openreview mett/formmiddd8nnthiiae
(973] Yuxin Wen, Leo Marchyok, Sanghyun Hong, Jonas Geiping, Tom Goldstein, and Nicholas Carlini. Pri- vacy backdoors: Enhancing membership inference through poisoning pre-trained models. In NeurIPS, 2024.
[974] Zhaotian Weng, Zijun Gao, Jerone Andrews, and Jieyu Zhao. Images speak louder than words: Understanding and mitigating bias in vision-language model from a causal mediation perspective. arXiv preprint KXiu:2477002114 2024.
[975] Sarah Wiegreffe and Yuval Pinter. Attention is not not explanation, 2019.
[976] Olivia Wiles, Sven Gowal, Florian Stimberg, Sylvestre Alvise-Rebufi, Ira Ktena, Krishnamurthy Dvijotham, and Taylan Cemgil. fine-grained analysis on distribution shift. arXio preprint arXio:2110.11320 11322 2021.
[977] Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, al. Qwen-image technical report. Xix preprint or Xiv:2200..000 023244 2025.
(978] Kangxi WU, Liang Pang. Huawei Shen, Xueqi Cheng, and Tat-Seng Chua. LImdet: A third party large language models generated text detection tool. In Conference on Empirical Methods in Natural Language Processing, 2023. URL http:///o/ntttr/pp....oooaaaoto wrw.ooaorrrr/CCopsIDDDD 258865367.
[979] Ruijia Wu, Yuhang Wang, Huafeng Shi, Zhipeng Yu, Yichao Wu, and Ding Liang. Towards prompt- robust face privacy protection via adversarial decoupling augmentation framework. arXiv preprint arXiv:2305. 03980, 2023.
[980] Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, and Tat-Seng Chua. Next-gpt: Any-to-any any multimodal 11m. Xi preprint arXiv:2309.055. 05519, 2023.
[981] Skyler Wu, Eric Meng Shen, Charumathi Badrinath, Jiaqi Ma, and Himabindu Lakkaraju. Analyzing chain-of-thought prompting in large language models via gradient- based feature attributions. ArXiv, abs/2307.13333) 2023. URL pattss//aape semanticscholar org/Corrps
htwpaaaiieeiii....oooooooooaoaaa 266011553..
151

---

Published in Transactions on Machine Learning Research (10/2025)
[982] Tongshuang Wu, Marco Túlio Ribeiro, Jeffrey Heer, and Daniel S. Weld. Polyjuice: Generating counterfactuals for explaining, evaluating, and improving models. In ACI//IICNLP (1), pp. 6707- 6723. Association for Computational Linguistics, 2021.
[983] Xindi Wu, Dingli Yu, Yangsibo Huang, Olga Russakovsky, and Sanjeev Arora. Conceptmix: A com- positional image generation benchmark with controllable difficulty. arXiv preprint or arXiv:2408. 14939. 2024.
[984] Xuansheng Wu, Wenlin Yao, Jianshu Chen, Xiaoman Pan, Xiaoyang Wang, Ninghao Liu, and Dong Yu. From language modeling to instruction following: Understanding the behavior shift in lmss after instruction tuning. areie preprint arXiv:2310.00/ 00492, 2023.
[985] Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. Inference Scaling Laws: An Empirical Analysis of Compute-Optinal Inference for Problem-Solving with Language Models. arXiv, 2024.
[986] Yihan Wu, Zhengmian Hu, Hongyang Zhang, and Heng Huang. Dipmark: A stealthy, efficient and resilient watermark for large language models. ArXiv, abs/2310.07710, 2023. URL https: //api.semante www.chooa..og/ooorroeeeoeeeee atg//orpuuID::26833775. 263834753.
[987] Yixin Wu, Rui Wen, Michael Backes, Ning Yu, and Yang Zhang. Model stealing attacks against vision-language models. 2022.
[988] Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoe V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. Google's neural machine translation system: Bridging the gap between human and machine translation. ar preprint ar arXiw:100..00 081 08144. 2016.
[989] Yongliang WI, Shiji Zhou, Mingzhuo Yang, Lianzhe Wang, Heng Chang, Wenbo Zhu, Xinting Hu, Xiao Zhou, and Xu Yang. Unlearning concepts in diffusion model via concept domain correction and concept preserving gradient. In AAAI, 2025.
[990] Yuanhao Wu, Juno Zhu, Siliang Xu, Kashun Shum, Cheng Niu, Randy Zhong, Juntong Song, and Tong Zhang. Ragtruth: A hallucination corpus for developing trustworthy retrieval-augmented lan- guage models. arXiv preprint ar arXiv:2401.000 00396. 2023.
[991] Yuanwei WU, Xiang Li, Yixin Liu, Pan Zhou, and Lichao Sun. Jailbreaking gpt-4v via sclf-adversarial allacks with system prompls. arXiv preprinl arXiv:2311.09129 2023.
[992] Zeqiu Wu, Yushi Hu, Weijia Shi, Nouha Dziri, Alane Suhr, Prithviraj Ammanabrolu, Noah A Smith, Mari Ostendorf, and Hannaneh Hajishirzi. Fine-grained human feedback gives better rewards for language model training. Arxiv preprint, abs/ abs/2306.01699 2023. URL http///rxxvv..r/abbea/aa/ arg/abs/23 06. 0663.
[993] Zhengxuan Wu and Desmond C. Ong. On explaining your explanations of BERT: an empirical study with sequence classification. CoRR, abs/ abs/2101.00196 2021.
[994] Zhiyong Wu, Yun Chen, Ben Kao, and Qun Liu. Perturbed masking: Parameter-free probing for analyzing and interpreting BERT. In ACI, pp. 4166-4176. Association for Computational Linguistics, 2020.
[995] xAI. Grok4,
aittaass/nnw/ooeeeee 4, 2025.
[996] xAI. Grok 3 beta the age of reasoning agents, February 2025. URL https::/x. Accessod: April 6, 2025.
[997] Peng Xia, Di Xu, Lie Ju, Ming Hu, Jun Chen, and Zongyuan Ge. Lmpt: Prompt tuning with class- specific embedding loss for long-tailed multi-label visual recognition. arXiv preprint orXiv:2305.045. 2023.
152

---

Published in Transactions on Machine Learning Research (10/2025)
[998] Peng Xia, Ze Chen, Juanxi Tian, Yangrui Gong, Ruibo Hou, Yue Xu, Zhenbang Wu, Zhiyuan Fan, Yiyang Zhou, Kangyu Zhu, et al. Cares: A comprehensive benchmark of trustworthiness in medical vision language models. arxie preprint arXiv:2406.0600 06007, 2024.
[999] Peng Xia, Siwei Han, Shi Qiu, Yiyang Zhou, Zhaoyang Wang, Wenhao Zheng, Zhaorun Chen, Chen- hang Cui, Mingyu Ding, Linjie Li, et al. Mmie: Massive multimodal interleaved comprehension benchmark for large vision- language models. arXiv preprint 00t::2400.111080 2024.
[1000] Peng Xia, Ming Hu, Feilong Tang, Wenxue Li, Wenhao Zheng, Lie Ju, Peibo Duan, Huaxiu Yao, and Zongyuan Ge. Generalizing to unseen domains in diabetic retinopathy with disentangled representa- lions. arXiv preprinl Xiv:224406. 06384, 2024.
[1001] Peng Xia, Kangyu Zhu, Haoran Li, Tianze Wang, Weijia Shi, Sheng Wang, Linjun Zhang, James Zou, and Huaxiu Yao. Mmed-rag: Versatile multimodal rag system for medical vision language models. Xrv preprint or 030058 2024.
[1002] Peng Xia, Kangyu Zhu, Haoran Li, Hongtu Zhu, Yun Li, Gang Li, Linjun Zhang, and Huaxiu Yao. Rule: Reliable multimodal rag for factuality in medical vision language models. In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024.
[1003] Zhen Xiang, Fengging Jiang, Zidi Xiong, Bhaskar Ramasubramanian, Radha Poovendran, and Bo Li. Badchain: Backdoor of prompting for large language models. In The Twelfth Interna- tional Conference on Learning Representations, 2024. URL attps://openreview wwtneeee/turreeeee ffrummid==e 3SBwz1Ma.
[1004] Zhen Xiang, Yi Zeng, Mintong Kang, Chejian Xu, Jiawei Zhang, Zhuowen Yuan, Zhaorun Chen, Chulin Xie, Fengqing Jiang, Minzhou Pan, al. Clas 2024: The competition for IIm and agent safety. In NewrIPS 2024 Competition Track, 2024.
[1005] Yijia Xiao, Yiqiao Jin, Yushi Bai, Yue Wu, Xianjun Yang, Xiao Luo, Wenchao Yu, Xujiang Zhao, Yanchi Liu, Haifeng Chen, et al. Large language models can be good privacy protection learners. arXiv preprint arXiv:2300.00 0.02469, 2023.
[1006] Yijun Xiao and William Yang Wang. On ballucination and predictive uncertainty in conditional language generation. arXiv preprint arxi:22103. 15025, 2021.
[1007] Yisheng Xiao, Lijun Wu, Junliang Guo, Juntao Li, Min Zhang, Tao Qin, and Tie-yan Liu. A survey on non-autoregressive generation for neural machine translation and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(10):11407 11427, 2023.
[1008] Yuxin Xiao, Paul Pu Liang, Umang Bhatt, Willie Neiswanger, Ruslan Salakhutdinov, and Louis- Philippe Morency. Uncertainty quantification with pre-trained language models: A large-scale pirical analysis, 2022.
[1009] Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408. 125228 2024.
[1010] Junlin Xie, Zhihong Chen, Ruifei Zhang, Xiang Wan, and Guanbin Li. Large multimodal agents: A survey. arXiv preprint 2x2u:2402.11116 2024.
[1011] Sang Michael Xie, Aditi Raghmathan, Percy Liang, and Tengyu Ma. An explanation in-context learning as implicit bayesian inference, 2021.
[1012] Sang Michael Xic, Shibani Santurkar, Tengyu Ma, and Percy Liang. Data selection for language models via importance resampling. Advances in Neural Information Processing Systems, 36:34201- 34227, 2023.
153

---

Published in Transactions on Machine Learning Research (10/2025)
[1013] Yueqi Xie, Jingwei Yi, Jiawei Shao, Justin Curl, Lingjuan Lyu, Qifeng Chen, Xing Xie, and Fangzhao Wu. Defending chaigpt against jailbreak attack via self-reminders. Nature Machine Intelligence, pp. 1-11, 2023.
[1014] Guangzhi Xiong, Qiao Jin, Zhiyong Lu, and Aidong Zhang- generation for medicine. arXiv preprint arXi::2402.1111 13178, 2024.
Benchmarking retrieval-augmented
[1015] Miao Xiong, Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, and Bryan Hooi. Can Ilms express their uncertainty? an empirical evaluation of confidence clicitation in llms. arXiv preprint arXiv:2306.115 13063, 2023.
[0066] Wei Xiong, Hanze Dong, Chenlu Ye, Ziqi Wang, Han Zhong, Heng Ji, Nan Jiang, and Tong Zhang, Iterative preference learning from human feedback: Bridging theory and practice for rlhf under kl- constraint. In Forty-first International Conference on Machine Learning, 2024.
[1017] Han Xu, Xiaorui Liu, Yaxin Li, Anil Jain, and Jiliang Tang. To be robust or to be fair: Towards fairness in adversarial training. In International conference on machine learning, pp. 11492 11501. PMLR, 2021.
[1018] Hongshen Xu, Zichen Zhu, Situo Zhang, Da Ma, Shuai Fan, Lu Chen, and Kai Yu. Rejection improves reliability: Training 1ms to refuse unknown questions using rl from knowledge feedback. ar Xix preprinl arXiv:2403. 18349, 2024.
[019] Jin Xu, Zhifang Guo, Hangrui Hu, Yunfei Chu, Xiong Wang, Jinzheng He, Yuxuan Wang, Xian Shi, Ting He, Xinfa Zhu, Yuanjun Lv, Yongqi Wang. Dake Guo, He Wang, Linhan Ma, Pei Zhang, Xinyu Zhang, Hongkun Hao, Zishan Guo, Baosong Yang, Bin Zhang, Ziyang Ma, Xipin Wei, Shuai Bai, Keqin Chen, Xuejing Liu, Peng Wang, Mingkun Yang, Dayiheng Liu, Xingzhang Ren, Bo Zheng, Rui Men, Fan Zhou, Bowen Yu, Jianxin Yang, Le Yu, Jingren Zhou, and Junyang Lin. Qwen3-omni technical report, 2025. URL https: / //axxiv.. www.ai.ww//....aao..o.o..e.ooolt ab/////s//...../////////// 17765.
[1020] Jingjing Xu, Wangchunshu Zhou, Zhiyi Fu, Hao Zhou, and Lei Li. A survey on green deep learning, 2021. URL https: http///axivvv..///i www./og///////eo///o//.o/eeee///
[1021] Peng Xu, Wenqi Shao, Kaipeng Zhang, Peng Gao, Shuo Liu, Meng Lei, Fanqing Meng, Siyuan Huang, Yu Qiao, and Ping Luo. Lylm-ehub: A comprehensive evaluation benchmark for large vision-language models, arXiv preprint ar XrXi::2306.009 092655, 2023.
[1022] Ran Xu, Wenqi Shi, Yue Yu, Yuchen Zhuang, Yanqiao Zhu, May D Wang, Joyce C Ho, Chao Zhang, and Carl Yang. Emretriever: Tuning large language models as better biomedical text retrievers. arXiv preprint or 2r 2024.
[1023] Yuancheng Xu, Chenghao Deng, Yanchao Sun, Ruijie Zheng, Xiyao Wang, Jieyu Zhao, and Furong Huang. Equal long-term benefit rate: Adapting static fairness notions to sequential decision making. arXiv preprint arXiw:2309.08 03426, 2023.
[1024] Zhangchen Xu, Fengqing Jiang, Luyao Niu, Yuntian Deng, Radha Poovendran, Yejin Choi, and Bill Yuchon Lin. Magpic: Alignment data synthesis from scratch by prompting aligned 1mms with nothing. arXiv preprint arXiv:2406.08// 2024.
[1025] Zhangchen Xu, Fengging Jiang, Luyao Niu, Jinyuan Jia, Bill Yuchen Lin, and Radha Pooven- dran. Safedecoding: Defending against jailbreak attacks via safety-aware decoding. arXiv preprint arXiv:2402.0 00880880, 2024.
[1026] Ziqing Xu, Hancheng Min, Lachlan Ewen MacDonald, Jinqi Luo, Salma. Tarmoun, Enrique Mallada, and Rene Vidal. Understanding the learning dynamics of lora: A gradient flow perspective on low-rank adaptation in matrix factorization. In AISTATS, 2025.
[1027] Yasin Abbasi Yadkori, Ilja Kuzborskij, András György, and Csaba Szepesvári. To believe or not believe your 11,, 2024.
154

---

Published in Transactions on Machine Learning Research (10/2025)
[1028] Yasin Abbasi Yadkori, Iija Kuzborskij, David Stutz, András György, Adam Fisch, Arnaud Doucet, Tuliya Beloshapka, Wei-Hung Weng, Yao-Yuan Yang, Csaba Szepesvári, Ali Taylan Cemgil, and Nenad Tomasev. Mitigating Ilm ballucinations via conformal abstention, 2024.
[1029] An Yan, Yu Wang, Yiwu Zhong, Chengyu Dong, Zexue He, Yujie Lu, William Wang, Jingbo Shang, and Julian J. McAuley. Learning concise and descriptive attributes for visual recognition. CoRR, abss/2308.03666 2023.
[1030] Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. Large language models as optimizers. ArXiv preprint, abs/ hss/2309003409, 2023. URL https: //arxiv.org/abab.. ha///bbb/300 03409.
[1031] Jianwei Yang, Reuben Tan, Qianhui Wu, Ruijie Zheng, Baolin Peng, Yongyuan Liang, Yu Gu, Mu Cai, Seonghyeon Ye, Joel Jang, et al. Magma: A foundation model for multimodal ai agents. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 14203-14214, 2025.
[1032] Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized out oistof-dditti detection: A survey. Niv preprint or Xrxiv:21 10. 1 11394, 2021.
1033] Shu Yang, Muhammad Asif Ali, Cheng-Long Wang, Lijie Hu, and Di Wang. Moral: Moe augmented lora for lims' lifelong learning. arXiv preprint arXiv:2402.11900 2024.
[1034] Wenhan Yang, Jingdong Gao, and Baharan Mirzasoleiman. Robust contrastive language-image pre- training against data poisoning and backdoor attacks. Advances in Neural Information Processing Systems, 36, 2024.
[1035] Xi Yang, Jie Zhang, Kejiang Chen, Weiming Zhang, Zehua Ma, Feng Wang, and Nenghai Yu. Tracing text provenance via context-aware lexical substitution. ArXin, abs/2111.077873 2021. URL https: semanniccantc
[1036] Xi Yang, Kejiang Chen, Weiming Zhang, Chang rui Liu, Yuang Qi, Jic Zhang, Han Fang, and Neng H. Yu. Watermarking text generated by black-box language models. ArXie, abs/2305.08883, 2023. URL
https://api www.nticscholart https://
[1037] Xianjun Yang, Wei Cheng, Linda Petzold, William Yang Wang, and Haifeng Chen. Dna-gpt: Diver- gent n-gram analysis for training- free detection of gpt-generated text. ArXív, abs/23055 abs/2335.17339, 2023.
URL https://api. http:::/app..emmntieeeeeee www.nticscholleee....
:rg/CorpusID::5555
2588660111.
[1038] Xianjun Yang, Kexun Zhang, Haifeng Chen, Linda Ruth Petzold, William Yang Wang, and Wei Cheng, Zero-shot detection of machine-generated codes. ArXiv, abs/2310.05103, 2023. URL https:
//api.ssmanticscholl wrw.aarrrrr/ooooooDD atg/CorpusID:263811388 263831381.
[1099] Xinyu Yang, Huaxiu Yao, Allan Zhou, and Chelsea Finn. Multi-domain long-tailed learning by augmenting disentangled representations. Transactions on Machine Learning Research, 2023.
[1040] Xinyu Yang, Zichen Wen, Wenjie Qu, Zhaorun Chen, Zhiying Xiang, Beidi Chen, and Huaxiu Yao. Mcmorization and privacy risks in domain-specific large language models. In ICLR 2024 Workshop on Retiable and Responsible Foundation Models, 2024.
[1041] Yijun Yang, Ruiyuan Gao, Xiaosen Wang, Tsung Yi Ho, Nan Xu, and Qiang Xu. Mma-diffusion: Multimodal attack on diffusion models. In Proceedings of the TEEE/CVF Conference on Compuler Vision and Pattern Recognition, pp. 7737-7748 2024.
[1042] Yijun Yang, Ruiyuan Gao, Xiao Yang, Jianyuan Zhong, and Qiang Xu. Guardt2i: Defending text- to-image models from adversarial prompts. arXio preprint arXiv:2403.014 2024.
[1043] Yijun Yang, Tianyi Zhou, Kanxue Li, Dapeng Tao, Lusong Li, Li Shen, Xiaodong He, Jing Jiang, and Yuhui Shi. Embodied multi-modal agent trained by an Ilm from a parallel textworld. In Proceedings of the IFFF/CVF Conference on Computer Vision and Pattern Recognition, pp.
2024.
155

---

Published in Transactions on Machine Learning Research (10/2025)
[1044] Yuchen Yang, Bo Hui, Haolin Yuan, Neil Gong, and Yinzhi Cao. Sneakyprompt: Jailbreaking text- to-image generative models. In Proceedings of the IEEE Symposium Security and Privacy, 2024.
[1045] Yue Yang, Artemis Panagopoulon, Shenghao Zhou, Daniel Jin, Chris Kallison-Burchhe and Mark Yatskar. Language in a bottle: Language model guided concept bottlenecks for interpretable image classification. arXiv preprint arXiv:2211.11158 2022.
[1046] Zhou Yang, Zhipeng Zhao, Chenyu Wang, Jicke Shi, Dongsun Kim, DongGyun Han, and David Lo. What do code models memorize? an empirical study on large language models of code. orXiv preprinl arXie:2388. 09932, 2023.
[1047] Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint Xix:2440 06072, 2024.
[1048] Zonghan Yang, Xiaoyuan Yi, Peng Li, Yang Liu, and Xing Xie. Unified detoxifying and debiasing in language generation via inference- time adaptive optimization. arXiv preprint arXiv:2210. 04492, 2022.
[1049] Huaxiu Yao, Yiping Wang, Linjum Zhang, James Zou, and Chelsea Finn. C-mixup: Improving generalization in regression. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
[1050] Huaxiu Yao, Yu Wang, Sai Li, Linjun Zhang, Weixin Liang, James Zou, and Chelsea Finn. Improving out-of-distributtin robustness via selective augmentation. In International Conference on Machine Learning (ICML), pp. 13407-25433 PMLR, 2022.
[005]] Jia-Yu Yao, Kun-Peng Ning, Zhen-Hui Liu, Mu-Nan Ning, and Li Yuan. Llm lies: Hallucinations are not bugs, but features as adversarial examples. arXiv preprint orXiv:2310.01469, 2023.
[1052] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. arXiv preprinl arXiv:2210.033 036629, 2022.
[1053] Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models, may 2023. ArXiv preprint, abs/2305.1060) 10601, 2023. URL https://arxtis wwwexiiiorgg/bbbe wrw//br/////aeee 10601.
[1054] Yuanshun Yao, Xiaojun Xu, and Yang Liu. Large language model unlearning. arXiv preprint arXio:2310.1000 106833 2023.
[1055] Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. Editing large language models: Problems, methods, and opportunities. arXi preprinl 1tkt::2305.31172 2023.
[1056] Yunzhi Yao, Ningyu Zhang, Zekun Xi, Mengru Wang, Ziwen Xu, Shumin Deng, and Huajun Chen. Knowledge circuits in pretrained transformers. In NewrIPS, 2024.
[1057] Michihiro Yasunaga, Armen Aghajanyan, Weijia Shi, Rich James, Jure Leskovee, Percy Liang, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. Retrieval-augmented multinodal language modeling, Xi preprind ar 2xXi:2211.12500 2022.
[1058] Chenlu Ye, Wei Xiong, Yuheng Zhang, Nan Jiang, and Tong Zhang. A theoretical analysis of nash learning from human feedback under general kl-regularized preference. arxiv preprint arXiv:2/02.0 0731144 2024.
[1059] Jiachong Yo, Zhiyong Wu, Jiangtao Feng, Tao Yu, and Lingpeng Kong. Compositional exemplars for in-conlext learning, 2023.
[1660] Xiaoyu Ye, Hao Huang, Jiaqi An, and Yongtao Wang. Duaw: Data-free universal adversarial water- mark against stable diffusion customization. Xiv preprint ar Xiw:2308.0988) 2023.
156

---

Published in Transactions on Machine Learning Research (10/2025)
[1061] Chih-Kuan Ych, Joon Sik Kim, lan En-Hsu Yen, and Pradeep Ravikumar. Representer point selection for explaining deep neural networks. In NewrIPS, pp. 9311-9321, 2018.
[1062] Fan Yin, Jesse Vig, Philippe Laban, Shafiq Joty, Caiming Xiong, and Chien-Sheng Jason Wu. Did you read the instructions? rethinking the effectiveness of task definitions in instruction learning. arXiv preprint or arxiv:2306.00 01.22306.011550 01150, 2023.
[0633] Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang, Dianbo Sui, Yunhang Shen, Ke Li, Xing Sun, and Enhong Chen. Woodpecker: Hallucination correction for multimodal large languago models. Arxiv prepriat, abs/2310. 2310.16044, 2023. URL https: /taps:///rxxvi http://awwv..../aa/.a////e// www/oor//////eeeeeee 16045.
[1044] Zhangyue Yin, Qiushi Sun, Qipeng Guo, Jiawen Wu, Xipeng Qiu, and Xuanjing Huang. Do large language models know what they don't know?, 2023.
[1065] Zhenfei Yin, Jiong Wang, Jianjian Cao, Zhelun Shi, Dingning Liu, Mukai Li, Xiaoshui Huang, Zhiyong Wang, Lu Sheng, Lei Bai, et al. Lamm: Language-assisted multi-modal instruction-tuningg datasel, framework, and benchmark. Adnances in Neural Information Processing Systems, 36, 2023.
[1066] Kiyoon Yoo, Wonhyuk Ahn, Jiho Jang, and No Jun Kwak. Robust multi-bit natural language termarking through invariant features. In Annual Meeting of the Association for Computational Linguistics, 2023. URL https://aaappt pttts::/app..semattt beemanticscholeee org/CorpusII: 25912991..
[1067] Kiyoon Yoo, Wonhyuk Ahn, and No Jun Kwak. Advancing beyond identification: Multi-bit watermark for language models. ArXie, abs/2308.0022 00221, 2023. URL http:://pp..eeaaaicccclaa....r usID: 262903996.
[1068] Dongkeun Yoon, Seungone Kim, Sohee Yang, Sunkyoung Kim, Soyeon Kim, Yongil Kim, Eunbi Choi, Yireun Kim, and Minjoon Seo. Reasoning models better express their confidence. arXiv preprint arXiv:2505.14 14489, 2025.
[1069] Jaehong Yoon, Sung Ju Hwang, and Yue Cao. Continual learners are incremental model generalizers. In International Conference on Machine Leorning, 2023.
1070] Jachong Yoon, Shoubin Yu, Vaidehi Patil, Huaxiu Yao, and Mohit Bansal, Safree: Training-free and adaptive guard for safe text-to-image and video generation. arXiv preprint drx::2410.11761, 2024.
[1071] Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin A Inan, Gautam Kamath, Janardhan Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, et al. Differentially private fine-tuning of language models. arXiv preprint ar 0rXi::2110.06500 2021.
[1072] Jiahao Yu, Xingwei Lin, and Xinyu Xing. Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts. arXiv preprint or Xrn:22300 10253, 2023.
[1073] Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfci Yang, Burcu Karagol Ayan, Ben Hutchinson, Wei Han, Zarana Parekh, Xin Li, Han Zhang, Jason Baldridge, and Yonghui Wu. Scaling autoregressive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.101 2022.
[1074] Liu Yu, Yuzhou Mao, Jin Wu, and Fan Zhou. Mixup-based unified framework to overcome gender bias resurgence. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrievol, pp. 1755-1759, 2023.
1075] Mengxia Yu, De Wang, Qi Shan, Colorado Reed, and Alvin Wan. The super weight large language models, 2024.
[1076] Shoubin Yu, Jachong Yoon, and Mohit Bansal. Crema: Generalizable and efficient video-language reasoning via multimodal modular fusion. arXio preprint or 02.0880,, 2024.
157

---

Published in Transactions on Machine Learning Research (10/2025)
[1077] Tianyu Yu, Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu, Hai-Tao Zheng, Maosong Sun, et al. Rihf-v: Towards trustworthy milms via behavior alignment from fine-grained correctional human feedback. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 13807-13816, 2024.
[1078] Wenhao Yu, Chenguang Zhu, Zaitang Li, Zhiting Hu, Qingyun Wang, Heng Ji, and Meng Jiang. A survey of knowledge-enhanee text generation. ACM Computing Surveys, 54(11s):1-33) 113,, 2022.
[1079] Xiao Yu, Yuang Qi, Kejiang Chen, Guoqiang Chen, Xi Yang, Pengyuan Zhu, Weiming Zhang, and Neng H. Yu. Gpt paternity test: Gpt generated text detection with gpt genetic inheritance. ArXiu, Abs/2305.112511 2023. URL https: //api./ppppppp www.ntiismcoottioooeoo... org/ hegp/CrrpusDD org/CorpusID:258833222 258833423.
[1080] Xiao Yu, Kejiang Chen, Qi Yang, Weiming Zhang, and Nenghai Yu. Text fluoroscopy: Detecting Im--eneerated text through intrinsic features. In Proceedings of the 2024 Conference on Empiricul Methods in Natural Language Processing, pp. 15838 15846, 2024.
[1081] Yue Yu, Rongzhi Zhang, Ran Xu, Jieyu Zhang, Jiaming Shen, and Chao Zhang. Cold-start data selection for few-shot language model fine-tuning: A prompt-based uncertainty propagation approach, 2023.
[082]] Zhuohao Yu, Xingru Jiang, Weizheng Gu, Chang Gao, Yidong Wang, Shikun Zhang, and Wei Ye. Saemark: Steering personalized multilingual IIm watermarks with sparse autoencoders.
[1083] Lifan Yuan, Yangyi Chen, Ganqu Cui, Hongcheng Gao, Fangyuan Zou, Xingyi Cheng, Heng Ji, Zhiyuan Liu, and Maosong Sun. Revisiting and Iims evaluations, 2023.
robustness in nlp: Benchmark, analysis,
[1084] Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Sainbayar Sukhbaatar, Jing Xu, and Jason Weston. Self-rewarding language modcls. arXiv preprint arXiv:2401.10020 2024.
[1085] Zheng Yuan, Hongyi Yuan, Chuanqi Tan, Wei Wang, Songfang Huang, and Fei Huang, Rrhf: Rank re- sponses align language models with human feedback without tears. ArXiv preprint, abs/2304.05302, 2023. URL https: // arxiv.v org/ 0tte//23400000.2
[1086] Junpeng Yue, Xinru Nu, Börje F Karlsson, and Zongqing Lu. Milm as retriever: Interactively learning multimodal retricval for embodied agents. arxiv preprint arXiv:2410.0844 2024.
[1087] Xiang Yuc, Huscyin A Inan, Xuechen Li, Girish Kumar, Julia McAnallen, Hoda Shajari, Huan Sun, David Levitan, and Robert Sim. Synthetic text generation with differential privacy: A simple and practical recipe. arXiv preprint arXiv:2210. 1114:22100.43388 2022.
[0888] Yisong Yue, Josef Broder, Robert Kleinberg, and Thorsten Joachims. The k-armed dueling bandits problem. Journal of Computer and System Sciences, 78(5):1538 1556, 2012.
[1089] Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, and James Zou. When and why vision-language models behave like bags-of-words, and what to do about it? Tn The Fleventh International Conference on Learning Representations, 2022.
[1090] Sangdoo Yun, Dongyoon Han, Sanghyuk Chun, Seong Joon Oh, Youngjoon Yoo, and Junsuk Choe. Cutmix: Regularization strategy to train strong classifiers with localizable features. In 2019 IEEE/CVE International Conference on Computer Vision (ICCV), pp. 6022 -0311. IEEE, October 2019. doi: (0000///cc..219.0000000 URL http://dx.dd doi. wtt.d//.....oo/.....oooo/oooo. httg//0.1109/00000 2019. 2019.00612.
[1091] Bianca Zadrozny and Charles Elkan. Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers. JCML, 1, 05 2001.
[1092] Omar Zaidan and Jason Eisner. Modeling annotators: A generative approach learning from annotator rationales. In EMNLP, pp. 31-40. ACT, 2008.
158

---

Published in Transactions on Machine Learning Research (10/2025)
[1093] Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. Defending against neural fake news. ArXiv, abs/1905.12616, 2019. URL https://appp
semanticscholar wrw.aaarrrr/CCoooIDDD..... I1trruusI::111198222 168169824.
[1094] Yi Zeng, Si Chen, Won Park, Z Morley Mao, Ming Jin, and Ruoxi Jia. Adversarial unlearning of backdoors via implicit hypergradient. arXiv preprint arXiv:2110009755 2021.
1095] Chrysoula Zerva, Taisiya Glushkova, Ricardo Rei, and André F. T. Martins. Disentangling uncertainty in machine translation evaluation, 2022.
[1096] Bohan Zhai, Shijia Yang, Xiangchen Zhao, Chenfeng Xu, Sheng Shen, Dongdi Zhao, Kurt Keutzer, Manling Li, Tan Yan, and Xiangjun Fan. Halle-switch: Rethinking and controlling object existence hallucinations in large vision language models for detailed caption. ArXiv preprint, abs/2310.01779, 2023. URL. https: //arxi/aa.ii.///a org/ abss2330.01777
[1097] Shenglang Zhai, Yinpeng Dong, Qingni Shen, Shi Pu, Yuejian Fang, and Hang Su. Text-to-image diffuuson models can be easily backdoored through multimodal data poisoning. arXiv preprint arXiv:230504419 2023.
[1098] Yuexiang Zhai, Shengbang Tong, Xiao Li, Mu Cai, Qing Qu, Yong Jac Lee, and Yi Ma. Investigating the catastrophic forgetting in multimodal large language models. arXiv preprint or Stt:2339910311. 2023.
[1099] Yuexiang Zhai, Hao Bai, Zipeng Lin, Jiayi Pan, Shengbang Tong, Yifei Zhou, Alane Suhr, Saining Xie, Yann LeCun, Yi Ma, and Sergey Levine. Fine-tuning large vision-language models as decision decision-making making agents via reinforcement learning. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https: htt:://oppeeeieeeeeee net stte/forummmddddBjMaPP
[1100] Haolan Zhan, Xuanli He, Qiongkai Xu, Yuxiang Wu, and Pontus Stenetorp. G3detector: General gpt-generated text detector. ArXiv, abs/2305.12088 12680, 2023. URL attps://appt somanticscholar.
Org/CorpusID:::: 25888332188
[1101] Andi Zhang, Tim Z Xiao, Weiyang Liu, Robert Bamler, and Damon Wischik. Your finetuned large language model is already a powerful out-of-listribution detector. arXiv preprint or orXin:2404.08% 08679 2024.
[1102] Caiqi Zhang, Fangyu Liu, Marco Basaldella, and Nigel Collier. Luq: Long-text uncertainty quantifi- cation for llms. Xixiv preprint 2ix:2443.20029, 2024.
[1103] Caiqi Zhang, Ruíhan Yang, Zhisong Zhang, Xinting Huang, Sen Yang, Dong Yu, and Nigel Collier. Atomic calibration of Ilms in long-form generations. arx preprint arXiv:2410. 13246, 2024.
[1104] Dawen Zhang, Pamela Staples, and Xiwei Xu. Right to be forgotten in the of large language models: Implications, challenges, and solutions. arXiv preprint or Xrxv:2207 03941,, 2023.
Seseenber--rommaae Thong Hoang, Shidong Pan, Zhenchang Xing, Mark
[1105] Hanning Zhang, Shizhe Diao, Yong Lin, Yi R Fung, Qing Lian, Xingyao Wang, Yangyi Chen, Heng Ji, and Tong Zhang. R-tuning: Teaching large language models to refuse unknown questions. arXiv preprint vrXiv:2311.09677 2023.
[1106] Haoran Zhang, Amy Lu, Mohamed Abdalla, Matthew McDermott, and Marzyeh Ghassemi. Hurt- ful words: quantifying biases in clinical contextual word embeddings. In proceedings of the ACM Conference on Health, Inference, and Learning, pp. 110 120, 2020.
[1107] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning, pp. 7472 7482. PMLR, 2019.
[1108] Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez- Paz, mixup: Beyond empirical risk minimization. arXio preprint Xrm: 1700 710.09412, 2017.
159

---

Published in Transactions on Machine Learning Research (10/2025)
[1109] Jie Zhang, Florian Kerschbaum, Tianwei Zhang, et al. Backdooring textual inversion for concept censorship. arXiv preprint ar arXiv:2308. 10088.11788 10718, 2023.
11100] Jizhi Zhang, Keqin Bao, Yang Zhang, Wenjie Wang, Fuli Feng, and Xiangnan He. chalgpt fair for recommendation? evaluating fairness in large language model recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems, pp. 993 993-999, 2023.
[1111] Lining Zhang, Mengchen Wang, Liben Chen, and Wenxin Zhang, Probing GPT-3's Linguistic Knowl- edge on Semantic Tasks. In Proceedings of the Fifth BlackbozNLP Workshop on Analyzing and In- terpreting Neural Networks for NLP,, pp. 297-304, Abu Dhabi, United Arab Emirates (Hybrid), December 2022. Association for Computational Linguistics. URL https://aclanthology wwwpattooooooooooo/22222222 org/202 2. b.blackbomllp11222
[1112] Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image dif- fusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3836-3847, 2023.
[1133] Peitian Zhang, Shitao Xiao, Zheng Liu, Zhicheng Dou, and Jian-Yun Nie. Retrieve anything to augment large language models. aivi preprint orXiv:2310.0755. 07550,, 2023.
[1114] Ruisi Zhang, Shehzeen Samarah Hussain, Paarth Neekhara, and Farinaz Koushanfar. Remark-llm: A robust and elficient watermarking framework for generative large language models. ArXio, 2023.
[111]] Shenao Zhang. Conservative dual policy optimization for efficient model-based reinforcement learning. Advances in newral information processing systems, 35:25450- -254633, 2022.
[1116] Shenao Zhang, Li Shen, and Lei Han. Learning meta representations for agents in multi-agent rein- forcement learning. arXiv preprint arXin:2108 12988, 2021.
[1117] Shenao Zhang, Li Shen, Zhifeng Li, and Wei Liu. Structure-regulariied attention for deformable object representation. arXi preprint 1rX.:2106.06672, 2021.
[1188] Shenao Zhang, Wanxin Jin, and Zhaoran Wang. Adaptive barrier smoothing for first-order policy gradient with contact dynamics. In International Conference on Machine Learning, pp. 41219 41243. PMLR, 2023.
[1119] Shenao Zhang, Boyi Liu, Zhaoran Wang, and Tuo Zhao. Model-based reparameterization policy gradient methods: Theory and practical algorithms. Advonces in Neurol Information Processing Systems, 36, 2024.
[1200] Shenao Zhang, Donghan Yu, Hiteshi Sharma, Ziyi Yang, Shuohang Wang, Hany Hassan, and Zhaoran Wang. Self-exploring language models: Active preference elicitation for online alignment. arXio preprint urXiv:2405. 19332, 2024.
[1121] Shenao Zhang, Sirui Zheng, Shuqi Ke, Zhihan Liu, Wanxin Jin, Jianbo Yuan, Yingxiang Yang, Hongxia Yang, and Zhaoran Wang. How can Ilm guide ri? a value- based approach. arXie preprint arXiv:240..1 16181, 2024.
[1122] Tianjun Zhang, Shishir G Patil, Naman Jain, Sheng Shen, Matei Zaharia, Ion Stoica, and Joseph E Gonzalez. Raft: Adapting language model to domain specific rag. arXiv preprint arXiv:2403.1011 2024.
[1123] Tianyuan Zhang, Sai Bi, Yicong Hong, Kai Zhang, Fujun Luan, Songlin Yang, Kalyan Sunkavalli, William T Freeman, and Hao Tan. Test-time training done right. arXiv preprint arXiv:2505. 238.22550555388 2025.
11124] Wenqiao Zhang, Haochen Shi, Siliang Tang, Jun Xiao, Qiang Yu, and Yucting Zhuang. Consensus graph representation learning for better grounded image captioning. In Proceedings of the AAAI Conference Artificial Intelligence, volume 35, pp. 3394-38 3394-3402, 2021.
160

---

Published in Transactions on Machine Learning Research (10/2025)
[1125] Xingxuan Zhang, Renzhe Xu, Han Yu, Yancheng Dong, Pengfei Tian, and Peng Cui. Flatness- minimization for domain generalization. In Proceedings of the TEEE/CV International Conference on Computer Vision, pp. 5189 5202, 2023.
[1126] Xingxuan Zhang, Jiansheng Li, Wenjing Chu, Junjia Hai, Renzhe Xu, Yuqing Yang, Shikai Guan, Jiazheng Xu, and Peng Cui. On the aut-of-distribution generalization of multimodal large language models. arXiv preprint arXiv:2402.06530 06599, 2024.
[1127] Yanzhe Zhang, Lu Jiang, Greg Turk, and Diyi Yang. Auditing gender presentation differences in text-to-image models. aixi preprint 2r022:2302. 03675, 2023.
[1128] Yichi Zhang, Yao Huang, Yitong Sun, Chang Liu, Zhe Zhao, Zhengwei Fang, Yifan Wang, Huanran Chen, Xiao Yang, Xingxing Wei, et al. Benchmarking trustworthiness of multimodal large language models: A comprehensive study. arXiv preprinl Xix::22406. 07057, 2024.
[1129] Yimeng Zhang, Jinghan Jia, Xin Chen, Aochuan Chen, Yihua Zhang, Jiancheng Liu, Ke Ding, and Sijia Liu. To generate or not? safety-driven unlearned diffusion models are still easy to generate unsafe images... for now. arXie preprint arXiv:2310. 11868, 2023.
11300] Yiming Zhang and Daphne Tppolito. Prompts should not be seen as secrets: Systematically measuring prompt extraction attack success. arXie preprint arXiv:2307.06863 2023.
[1131] Yiming Zhang, Zhuokai Zhao, Zhaorun Chen, Zhili Feng, Zenghui Ding, and Yining Sun. Rankclip: Ranking-consistent language-image pretraining, arXiv preprint Xix::4440 09387, 2024.
[1132] Yizhe Zhang, Zhe Gan, Kai Fan, Zhi Chen, Ricardo Henao, Dinghan Shen, and Lawrence Carin. Adversarial feature matching for text generation. In International conference on machine learning, pp. 4006-4015. PMER, 2017.
[1133] Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, al. Siren's song in the ai ocean: A survey on hallucination in large language models. Compulational Linguistics, pp. 45, 2025.
[1134] Zhexin Zhang, Yida Lu, Jingyuan Ma, Di Zhang, Rui Li, Pei Ke, Hao Sun, Lei Sha, Zhifang Sui, Hongning Wang, and Minlie Huang. Shieldlm: Empowering Ilms as aligned, customizable and ex- plainable safety detectors. arXie preprint orXiv:2402. 16444, 2024.
[1135] Zijian Zhang, Kaiyuan Zheng, Zhaorun Chen, Joel Jang, Yi Li, Chaoqi Wang, Mingyu Ding, Dieter Fox, and Huaxiu Yao. Grape: Generalizing robot policy via preference alignment. arXiv preprint Xrxiv:2411. 111..1111111111 11.1..111199,,,, 2024.
[1136] Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du. Explainability for large language models: A survey. CoRR, abs/2309.0100 2023.
[1137] Ruochen Zhao, Hailin Chen, Weishi Wang, Fangkai Jiao, Xuan Long Do, Chengwei Qin, Bosheng Ding, Xiaobao Guo, Minzhi Li, Xingxuan Li, et al. Retrieving multimodal information for augmented generation: A survey. aixiv preprint ar 1xXx:22333.18666 2023.
[1138] Tony Z. Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. Calibrate before use: Improving few-shot performance of language models, 2021.
[1139] Xingyi Zhao, Depeng Xu, and Shuhan Yuan. Defense against backdoor attack on pre-trained language models via head pruning and attention normalization. In Forty-firs first International Conference Machine Learning, 2024.
[1140] Xuandong Zhao, Prabhanjan Vijendra Ananth, Lei Li, and Yu-Xiang Wang. Provable robust water- marking for ai- ai-generated text. ArXiv, abs/2306.17438, 2023. URL arg//CorpusID::55308888
semanticscholar:
161

---

Published in Transactions on Machine Learning Research (10/2025)
[1141] Yao Zhao, Misha Khalman, Rishabh Joshi, Shashi Narayan, Mohammad Salch, and Peter J Liu. Calibrating sequence likelihood improves conditional language generation. Arx preprint, abs/2210.000445, 2022. URL https://arxiv.ott wttpa/wwww...//////////////
[1142] Yao Zhao, Rishabh Joshi, Tianqi Liu, Misha Khalman, Mohammad Saleh, and Peter J Liu. Slic-hf: Sequence likelihood calibration with human foedback. ArXiv preprint, abs/2305.10425, 2023. URL
10425.
[1143] Zhengyue Zhao, Jinhao Duan, Xing Hu, Kaidi Xu, Chenan Wang, Rui Zhang, Zidong Du, Qi Guo, and Yunji Chen. Unlearnable examples for diffusion models: Protect data from unauthorized exploitation. ar preprint wrXiv:2306.0190 01902, 2023.
[1144] Zhengyue Zhao, Jinhao Duan, Kaidi Xu, Chenan Wang, Rui Zhangp Zidong Dup Qi Guo, and Xing Hu. Can protective perturbation safeguard personal data from being exploited by stable diffusion? CVPR, 2024.
[1145] Zhengyue Zhao, Xiaoyun Zhang, Kaidi Xu, Xing Hu, Rui Zhang, Zidong Du, Qi Guo, and Yunji Chen. Adversarial contrastive decoding: Boosting safety alignment of large language models via opposite prompt optimization. arXiv preprint orXiv:2406.1677
2024.
[1146] Boyang Zheng, Chumeng Liang, Xiaoyu Wu, and Yan Liu. Understanding and improving adversarial atlacks latent diffusson model. ar preprint arXiv:2310.0466 046877, 2023.
[144]] Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, and Baobao Chang, Can we edit factual knowledge by in-context learning? ArXiv, abs/2305.12744 2023. URL https:
//api. wwwanticschoolr. wtwphhollrrrrg/oppppiiieeo org/CorpusID: Ctt///orpSDD::583222000 258832407
[1448] Zexuan Zhong, Dan Friedman, and Danqi Chen. Factual Probing Is [MAS]]: Learning vs. Learning to Recall, December 2021. URL http://arxiv. wwwpeiiii...ooieeeeeeeeee orwg/rrg///////ooeo atspaabs///...//// 05240. arXiv:2104.05240 [es].
[1149] Zexuan Zhong, Zhengxuan Wu, Christopher D. Manning, Christopher Potts, and Danqi Chen. Mquake: Assessing knowledge editing in language models via multi-hop questions. In Conference Empirical Methods in Natural Language Processing, 2023. URL attps://app http:://api.smmaatccchooo semanticschola
r. toor/CorrussDD 258865984.
[1150] Andy Zhou, Jindong Wang, Yu-Xiong Wang, and Haohan Wang. Distilling out-of-distribution ro- bustness from vision-language foundation models. In NeurIPS 2023, October 2023.
[1151] Chunting Zhou, Graham Ncubig, Jiatao Gu, Mona Diab, Francisco Guzmén, Luke Zettlemoyer, and Marjan Ghazvininejad. Detecting hallucinated content in conditional neural sequence generation. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 1393-1404, 2021.
[1152] Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. LIMA: less is more for alignment. CoRR, abs/ abs/2305..122 1226,, 2023.
[1153] Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zeulemoyer, and Omer Levy. Transfusion: Predict the next token and diffuse images with one multi-modal model. arXie preprint arXin:2408.1120 11039, 2024.
[1154] Han Zhou, Xingchen Wan, Lev Proleev, Diana Mincu, Jilin Chen, Katherine Heller, and Subhrajit Roy. Batch calibration: Rethinking calibration for in-context learning and prompt engineering, 2024.
[1155] Huichi Zhou, Yihang Chen, Siyuan Guo, Xue Yan, Kin Hei Lee, Zihan Wang, Ka Yiu Lee, Guchun Zhang, Kun Shao, Linyi Yang, and Jun Wang. Mcmento: Finc-tuning Ilm agents without fine-tuning Ilms, 2025. URL https: http///xxivvv./// org/
16153.
[1156] Huichi Zhou, Kin- Hei-Hei Lee, Zhonghao Zhan, Yue Chen, and Zhenhao Li. Trustrag: Enhancing robust- and trustworthiness in rag. Xr preprint or arx::220010008 2025.
162

---

mml. press/ prress/v661 v122 zhou22n Published in Transactions on Machine Learning Research (10/2025)
[1157] Kaitlyn Zhou, Dan Jurafsky, and Tatsunori Hashimoto. Navigating the grey area: How expressions of uncertainty and overconfidence affect language models, 2023.
[1158] Tianyi Zhou, Deqing Fu, Vatsal Sharan, and Robin Jia. Pre-trained large language models use fourier features to compute addition, 2024.
[1159] Wangchunshu Zhou and Ke Xu. Learning to compare for better training and evaluation of open domain natural language generation models. Proceedings of the AAAI Conference on Artificial Intelligence, 34((0))9717-9) 9724, Apr. 2020. doi: 10. 20.1609/aaai.vva2 waaia.a..i0....... //ojs.aaai. org. medexx hatp/AA//aatiill/reeee/////
URL https:
[1160] Wangchunshu Zhou, Tao Ge, Chang Mu, Ke Xu, Furu Wei, and Ming Zhou. Improving grammatical error correction with machine translation pairs. In Trevor Cohn, Yulan He, and Yang Liu (eds.), Find- ings of the Association for Computational Linguisties: EMNLP 2020, pp. 2020. Association for Computational Linguistics. doi: 10.18653 /f.d//....... emnlp.30. URL
Online, November
https://aclanthoology wwwpoolloyyoyoogg/geeeeo org/2020 1e122.ffndinng. findings-emnlp....
[1161] Wangchunshu Zhou, Tao Ge, Ke Xu, Furu Wei, and Ming Zhou. Self-adversarial learning with com- parative discrimination for text generation. In International Conference Learning Representations, 2020. URL https://opeereviiee
[1162] Wangchunshu Zhou, Jinyi Hu, Hanlin Zhang, Xiaodan Liang, Maosong Sun, Chenyan Xiong, and Jian Tang. Towards interpretable natural language understanding with explanations as latent variables. Advances in Neural Information Processing Systems, 33:6803-6814 6814, 2020.
[1163] Wangchunshu Zhou, Yan Zeng, Shizhe Diao, and Xinsong Zhang. VLUE: multi-task multi- dimension benchmark for evaluating vision-language pre-training. Kamalika Chaudhuri, Stefanic Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th Inter- national Conference Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 27395 27411. PMLR, 17 23 Jul 2022. URL htttps://proceeee .html.
[1164] Wangchunshu Zhou, Yuchen Eleanor Jiang, Tong Li, Jialong Wu, Tiannan Wang, Shi Qiu, Jintian Zhang, Jing Chen, Ruipu Wu, Shuai Wang, Shiding Zhu, Jiyu Chen, Wentao Zhang, Xiangru Tang, Ningyu Zhang, Huajun Chen, Peng Cui, and Mrinmaya Sachan. Agents: An open-source framework for autonomous language agents, 2023. URL attps://arzis htt///aarvvv.ooaaaaaaaa. www//or/////aaeeeeeeeee 07870.
[1165] Wangchunshu Zhou, Yuchen Fleanor Jiang, Ethan Wilcox, Ryan Cotterell, and Mrinmaya. Sachan. Controlled text generation with natural language instructions. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 42602-42613, PMLR, 23-29 Jul 2023. URL https: ptttps://proceedingg wwweeeiiiinnm.m.......mmmmeeeeee wwr.mlr.prre.. air.presss/v02 /zhou23g. html.
[1166] Wangchunshu Zhou, Yixin Ou, Shengwei Ding, Long Li, Jialong Wu, Tiannan Wang, Jiamin Chen, Shuai Wang, Xiaohua Xu, Ningyu Zhang, Huajun Chen, and Yuchen Fleanor Jiang. Symbolic learning enables self-cvolving agents, 2024. URL https://arxti: htttps://axxvvoor//bbb.. 224// 18532.
[1167] Yifei Zhou, Andrea Zanette, Jiayi Pan, Sergey Levine, and Aviral Kumar. Archer: Training language model agents via hierarchical multi-turn rl. arXiv preprint arXiv:2402. 19446, 2024.
[1168] Yiyang Zhou, Chenhang Cui, Rafael Rafailov, Chelsea Finn, and Huaxiu Yao. Aligning modalities in vision large language models via preference fine-tuning. arXie preprint Xrv::22002 11411 2024.
[1169] Yiyang Zhou, Chenhang Cui, Jachong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. Analyzing and mitigating object hallucination in large vision-language models. In International Conference on Learning Representations, 2024.
163

---

Published in Transactions on Machine Learning Research (10/2025)
[1170] Yiyang Zhou, Zhiyuan Fan, Dongjie Cheng, Sihan Yang, Zhaorun Chen, Chenhang Cui, Xiyao Wang. Yun Li, Linjun Zhang, and Huaxiu Yao. Calibrated self-rewarding vision language models. In Advances in Newral Information Processing Systems (NeurIPS), 2024.
[1171] Yiyang Zhou, Zhaoyang Wang, Tianle Wang, Shangyu Xing, Peng Xia, Bo Li, Kaiyuan Zheng, Zijian Zhang, Zhaorun Chen, Wenhao Zheng, et al. Anyprefer: An automatic framework for preference data syuthesis. In International Conference on Learning Representations (ICLR), 2025.
[1172] Yongchao Zhou, Andrei loan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engincers. ArXio preprint, abs/2211.01910,, arg/abs////eee abs abs//211..0llseee
2022. URL https: // arxiv. org/
[1773] Zhanhui Zhou, Jie Liu, Chao Yang, Jing Shao, Yu Liu, Xiangyu Yue, Wanli Ouyang, and Yu Qiao. Beyond prooecccccccfff Multi-objective direct preference optimization. arXiv preprint arXio:2310.0371 2023.
[1174] Banghua Zhu, Evan Frick, Tianhao Wu, Hanlin Zhu, and Jiantao Jiao. Starling-7b: Improving Ilm helpfulness & harmlessness with rlaif, 2023.
[1175] Banghua Zhu, Michael Jordan, and Jiantao Jiao. Principled reinforcement learning with human feedback from pairwise or k-wise comparisons. In International Conference on Machine Learning, pp- 43037-43067. PMLR, 2023.
[1176] Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom Goldstein, and Jingjing Liu. Freelb: Enhanced adversarial training for natural language understanding. arXiv preprint arXiv: 11764, 2019.
[1177] Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny, hancing lision-language understanding with advanced large language models. arXiv preprint arXiv:2304.1059, 2023.
Minigpt-4: En-
[1178] Didi Zhu, Zhongyi Sun, Zexi Li, Tao Shen, Ke Yan, Shouhong Ding, Kun Kuang, and Chao Wu. Model tailor: Mitigating catastrophic lorgetting in multi-modal large language models. In International conference on machine learning, 2024.
[1779] Liuwan Zhu, Rui Ning, Jiang Li, Chunsheng Xin, and Hongyi Wu. Scor: Backdoor detection for vision-language models through searching larget text and image trigger jointly. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 7766-7774, 2024.
[1180] Runchuan Zhu, Zinco Jiang, Jiang Wu, Zhipeng Ma, Jiahe Song, Fengshuo Bai, Dahua Lin, Lijun Wu, and Conghui Hc. Grait: Gradient-driven refusal-aware instruction tuning for effective hallucination mitigation. arXiv preprint arXiv:2502.050) 2025.
[1181] Sicheng Zhu, Ruiyi Zhang, Bang An, Gang Wu, Joe Barrow, Zichao Wang, Furong Huang, Ani Nenkova, and Tong Sun. Autodan: Automatic and interpretable adversarial attacks large language models. arXiv preprint ar arXi::2310.11144 2023.
[1182] Masrour Zoghi, Shimon Whiteson, Remi Munos, and Maarten Rijke. Relative upper confidence bound for the k-armed dueling bandit problem. In International conference on machine learning, pp. 10-18. PMLR, 2014.
[1183] Thomas P Zollo, Todd Morrill, Zhun Deng, Jake C Snell, Toniann Pitassi, and Richard Zemel. Prompt risk control: A rigorous framework for responsible deployment of large language models. arXiv preprint 2XN::3311.13222 2023.
[1184] Thomas P Zollo, Zhun Deng, Jake Snell, Toniann Pitassi, and Richard Zemel. Improving prodictor reliability with selective recalibration. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https: http:///ppeeeeee.eeeeeetoooeeeee
Expert Certification.
164

---

Published in Transactions on Machine Learning Research (10/2025)
[1185] Thomas P. Zollo, Todd Morrill, Zhun Deng, Jake C. Snell, Toniann Pitassi, and Richard Zemel. Prompt risk control: A rigorous framework for responsible deployment of large language models, 2024.
[1186] Thomas P. Zollo, Nikita Rajaneesh, Richard Zemel, Talia B. Gillis, and Emily Black. Towards effective discrimination testing for generative ai, 2024. URL attps://arxivv www.s/orrgbbbb//111// 21022. 21052.
[1187] Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al. Representation engineering; A top- down approach to ai transparency. arXiv preprinl arXiv:2310.01405 2023.
[1188] Andy Zou, Zifan Wang, J Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15443 2023.
[1189] Wei Zou, Rumpeng Geng, Binghui Wang, and Jinyuan Jia. Poisonedrag: Knowledge poisoning attacks to retrieval-augmented generation of large language models, 2024.
[1190] Ürsula Hébert-Johnson, Michael P. Kim, Omer Reingold, and Guy N. Rothblum. Calibration for the (ommpuattionlly-ddetttfaaa)) masses, 2018.
165


Reviews:
[{"review_id": "FDCnfdA3HX", "reviewer": "Reviewer_qQuq", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "<Weaknesses>\n1. The title needs to include the term \"Survey\".\n\n2. In each section, the list of literature should be better visually summarized; for example, citation trees, which are recently frequently used in various survey papers.\n\n3. Many survey papers already exist for some topics of this paper. \nFor example, for the hallucination of LLMs, the following survey papers already exist. \n- L. Huang et al., A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions, ACM TOIS 2024. \n- P. Sahoo et al., A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Models, EMNLP findings 2024. \n- Therefore, the novelty of this survey should be better justified by comparing with existing survey papers. \n- Hallucination is only a single example. For each of the other topics, several survey papers can be easily found. \n\n4. Moreover, it should be also verified why this kind of (somewhat too broad) survey paper is needed instead of these more-focused surveys for each of specific topics.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This submission is a comprehensive survey for reliability and responsibility issues of foundation models (LLMs, LMMs, and generative diffusion models).  The coverage of literature is one of the strongest contributions of this work, which make exhaustive survey for a various aspects of reliability and responsibility, including bias and fairness, alignment, security, privacy, hallucination, uncertainty, distribution shift, explainability, and AGIC detection.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please refer to the section of 'Strengths And Weaknesses'.", "broader_impact_concerns": "There is no critical issue in this submission.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "hm61K11jgE", "reviewer": "Reviewer_4vUV", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. This paper covers four model types (LLMs, LMMs, T2I, video) and nine key areas like bias, security, and privacy, integrating technical analysis with real-world applications.\n2. This paper focus more on multimodal challenges: addresses unique issues in LMMs (e.g., visual hallucination) and T2I models (e.g., gender bias), reflecting the evolving multimodal AI landscape.\n3. This survey highlights technical trade-offs (e.g., privacy vs. performance) and cross-domain links (e.g., security measures worsening bias), guiding integrated solutions for responsible AI.\n\nWeaknesses:\n1. The overview in Figure 1 is confusing. For example, models and tasks are not in the same dimensions and should not be put together.\n\n2. The Large Multimodal model's abbreviation (LMM) is strange. Usually, we refer to this as a Multimodal Large Language Model (MLLM) instead of LMM, and it may need to be revised.\n\n3. Adversarial Attacks of Security in Text-to-Image Models ignores some important works, such as [1][2].\n\n4. This paper doesn't discuss the advantages and differences with other latest surveys [3][4] related to the security of foundation models. \n\n5. It is unclear how adversarial training improves OOD detection.\n\n[1] Content-based Unrestricted Adversarial Attack, NeurIPS 2023\n\n[2] Diffusion Models for Imperceptible and Transferable Adversarial Attack, TPAMI 2024\n\n[3] Safety at Scale: A Comprehensive Survey of Large Model Safety, SuperIntelligence-Robotics-Safety & Alignment 2025\n\n[4] Benchmarking Trustworthiness of Multimodal Large Language Models: A Comprehensive Study, Arxiv 2024", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This survey offers a exploration of reliable and responsible foundation models, constructing a unified framework that categorizes these models into four types—Large Language Models (LLMs), Large Multimodal Models (LMMs), Text-to-Image (T2I) models, and Video Generative Models and systematically examines nine critical dimensions: bias and fairness, alignment, security, privacy, hallucination, uncertainty, distribution shift, explainability, and AI-Generated Content (AIGC) detection. It delves into the challenges within each dimension and proposes a range of solutions. The work also identifies research gaps, such as the lack of lightweight out-of-distribution detection methods, insufficient multimodal explainability, and the need for more robust watermarking, and outlines future directions, including the development of lifelong learning mechanisms, improved fairness in detection systems, and integrated approaches to ensure model reliability and responsibility across technical and societal dimensions.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See Weaknesses.", "broader_impact_concerns": "This paper doesn't need an additinal Broader Impact Statement.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "6drk7Ic7WW", "reviewer": "Reviewer_8MQQ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "I found this paper very paradoxical and very hard to review due to its sheer length of 81 pages and large scope. \nI will summarize my general impressions here and follow up with chapter-based feedback in the following section. \n\n\nStrengths\n---------\n\n* The paper covers essentially all topics relevant to reliable and responsible deployment of AI, and does so for all kinds of foundation models.\n* This allows the paper to point out intersections and cross-connections between different fields that are otherwise out-of-scope for specialized surveys.\n* The figures in the paper are top-notch and illustrate a lot of the concepts very well. \n\nWeaknesses\n----------\n\n* The paper feels both over- and underscoped at the same time. Many of the aspects in the paper like bias, hallucinations or uncertainty are covered in many other dedicated survey papers in more breadth and depth. It feels like in order to provide a lot of added value to the reader, the paper would have to discuss even more works and provide more detail (which would question whether the appropriate format would be a book instead of a paper) and / or point out / discuss even more cross-connections between the fields in order to give a birds-eye view.\n* Although the figures are great, they are consistenly underdescribed and not well connected to the rest of the text.\n* The writing is inconsistent - while this can be forgiven since this work probably required the work of many authors, the writing also does not feel very polished in some places (missing or wrongly formatted citations, typos etc.) and some sections feel very superficial / rushed.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work attempts the herculean task of summarizing and synthesizing aspects of reliable and responsible deployment of foundation models across different aspects such as bias and fairness, alignment, security, privacy, hallucinations, uncertainty quantification, distributional shift, explainability, and AI-generated content detection.\nThis is further done for both large language models (LLMs) and large multi-modal models (LMMs), including image generation models.\nAs such, it provides a very broad overview over these different research topic and also points out many relevant intersections in the conclusion.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "In the following, I first list requested changes in general and then on a per-section basis.\n\n\nGeneral changes\n---------------\n\n* Figure caption should describe the content of the figures better, as most of the time the task is left to the reader to match the contents of the chapter to parts of the figure.\n* I found it hard to follow sometimes whether the current paragraph was referring to LLMs, T2I- or vision-language models. I encourage the authors to potentially experiment with some visual indicators like an emoji or other symbols in the text or in the margins of the paper to supply visual anchors to a reader.\n* Since the scope of this paper is so large, it feels prudent to me to add a related work section in the end that mentions and maybe also discusses a bit existing surveys related to all the chapters.\n* A lot of the discussed the work is technical and research-focused. I would appreciate thoughts and discussions of the authors that also concern the practical frictions and challenges implementing some of the mentioned techniques, especially with respect to structural challenges, financial incentives, and the interaction of human users with foundation models (and how this might sometimes be misaligned with the results from research methods).\n\nSection 1\n---------\n\n* I find the definition of foundation models through \"emergent abilities\" very vague, especially since it is not intuitive clear what emergent abilities are for instance in T2I models and also because emergent abilities have been critically examined in the past [1-3].\n* Since transformers and diffusion models are the focus of this work, it might be instructive for readers to get the briefest of introduction into how they work.\n* In Figure 1, \"II. Quantifying and Addressing\" seems to be missing a word.\n* The authors do not really define what they mean by \"reliable\" and \"responsible\" foundation models. Although these terms have intuitive definitions, they should be defined explicitly here, since they form the key motivation of the work.\n\nSection 2\n---------\n\n* In this chapter it is a bit unclear of what is defined as LLMs, since equation (1) implies decoder-only LMs, but later chapters also reference papers about BERT.\n* \"A general design recipe for LMMs is to adopt architectures that integrate multimodal features in a shared latent space\" - for me, this warrants some citations about examples, and maybe a footnote outlining alternative architectures, and why they are not discussed here.\n* I find equations (1) - (4) a bit vague and in their current form not very informative, since (afaik) they are not referenced in the rest of the manuscript. I would find it slightly more informative to define a set of modalities of which video, text and imagines are elements, and base any formal definitions on that.\n* The paragraph after equation (3) should provide the corresponding citations for diffusion models, GANs, and transformers. This section kind of also glosses over whether state-space models like MAMBA [4] could be considered foundation models, and that multi-modal architecture might combine different model classes.\n\nSection 3\n---------\n\n* Table 1 could profit from providing sources for these definitions, or comparing definitions from different works.\n* The same applies to the definition of bias in the paragraph of the same name on page 11 (or at least it should be made clear that this is the definition of bias by the authors).\n* The notation is equation (5) should be harmonized with the notation introduced in section 2 (same holds with the rest of the notation in this section)\n* If a measurement function maps to a scalar score, then it should be written $S: \\mathcal{Y} \\rigtharrow \\mathbb{R}$.\n* \"It is noteworthy that this is just one possible definition of bias, while other definitions or metrics can also be reasonable\" -> mention or at least cite papers with other defintions or metrics here.\n* In the following paragraph, a set of principles by Gallegos et al. (2023) are named, but not discussed or not explained at all. \n* Equations (6) - (10): Use \\text for abbreviations / words in equations\n* \"Similarly, Demographic Representation (DR), as discussed in\" -> use \\citet for citations\n* The paragraph \"Methods based on Token Selection Probability\" talks about masked-language models that are encoder-only, which were not included as foundation models in section 2. \n* Section 3.3: \"For specific categories that are extensively discussed in later chapter.\" is missing some words.\n* \"However, it is essential to recognize that biases in the training data and the training process are not inherently designed to understand or priorize human values.\" I think I understand the point that the authors aim to make here, but it is a bit oddly worded. It also obfuscates that some of the biases in training data originate from human design decisions based on their own values and biases (see [5]).\n* The mention of chain-of-thought at the start of page 16 should come witha citation.\n* Definting the toxic text in the next line with the variable \"t\" is redundant, since it is not used.\n\nSection 4\n---------\n\n* At the end of the second paragraph 20, the word \"propose\" is used three times right after another.\n* \"Then, a series of works [...] propose to use well-suited rules, metrics, and LLM-based methods for data clearning\" -> This should be explained further.\n* At the end of the third paragraph, the authors could include [6], which uses bootstrapped conversations for task-oriented dialogue finetuning.\n* Another caveat to add to 4.1 could be [7], showing that overtrained models are harder to finetune. \n* Section 4.2 claims that one stage of RLHF is instruction finetuning with SFT. I would intuitively rather claim that the outcome of the instruction finetuning step is used for RLHF, which is a separate step.\n* p.22: TEXTGAN should be cited\n* Zhu et al. (2023b) citation should use \\citep instead of \\citet\n* The part about diverse user preference should also discuss the cultural component, for instance through [8].\n* The end of section 4.2 could profit from providing an overall discussion on when which RLHF objective should ideally be used.\n* The \"Discrete Prompts\" paragraph on p. 25 should discuss also the brittleness of models w.r.t prompts, for instance discussed in [9].\n* The \"Chain-of-Thought\" paragraph should also include a discussion about how reasoning chains do not necessarily need to reflect the actual reason a model arrived at a solution [10].\n* The paragraph on \"Prompt Optimization\" could refer to [11].\n* The same paragraph includes \"(i.e., agent workflow (Zhou et al., 2023d))\", which could be formatted better with \\citealp. This also applies to \"(e.g., GPT-4 (OpenAI, 2023b))\" in section 4.4.\n* The reference to \"visual instruction tuning\" in section 4.4 needs a citation.\n* \"dis-preferred\" -> \"dispreferred\"\n* The \"superalignment problem\" at the beginning of section 4.5 should be explained more.\n* Redundant whitespace before \"Casper et al. (2023)\" in section 4.5.\n* \"For example, a reward model with uniform values may make the training of foundation models more scyphantic rather than maintaining integrity\" - this argument is not very clear.\n* \"In the superalignment situation, many research works find that strong models can easily manipulate and deceive weak models\" - no citations or evidence for this is given. \n* \"making evaluation super difficult\" -> \"making evaluation quite difficult\" (or some variation of this)\n* \"And this can leave backdoor vulnearabilities in the model, which can be exploited by adversaries\" - references?\n* \"Ensemble methods [...] can be a potential solution of scalable oversight to overcome the weakness of a single weak model.\" - references in the context of alignment?\n\nSection 5\n---------\n\n* General comment: Large parts of the section read very repetitive, feeling more like a listing of papers without pointing out connections between papers or overarching trends.\n* Section 5.1.1 should also summarize some evidence from the literature on why and how jailbreaks work.\n* The \"Prompt Injection Attacks\" paragraph would profit from examples.\n* \"Shi et al. (2023a) and Rando & Tramèr (2024) explore the possibiltiy of injecting backdoors into the reward model during the RLHF process\" - so what would an actualy attack in the end look like?\n* Section 5.2: The \"projected gradient descent attack\" is mentioned, but no intuition / explanation of the attack is given.\n* The \"IDEFICS mechanism\" is mentioned on p. 31, but not explained. THe same goes for \"2-hop prompt templates\".\n* The LAION-5B dataset in 5.3.1 should be cited.\n* \"(Wang et al., 2024s)\" should be \\citet and not \\citep.\n* The \"assimilation phenomenon\" in the same line is not explained.\n* p 33: \"Yoon et al. (Yoon et al., 2024)\" should just be \\citet]\n* \"Model Fine-Tuning\" -> \"Model finetuning\"\n* \"(Yang et al., 2024e)\" -> \\citet\n* Section 5.4.1: \"Chen et al.\" -> \\citet\n* Section 5.4.2 seems to be missing a discussion using external models for safety, see e.g. [12]\n\nSection 6\n---------\n\n* The reference to homomorphic encryption should be better explained and connected to the rest of section or be omitted.\n* Section 6.11 contains a reference to BERT, which is not cited and not introducted in section 1 as a foundation model. The MIA acronym is also not introduced.\n* \"MIN-K% PROB\" attack should be explained, at least in broad terms.\n* Section 6.1.2: \"the field has shifted towards developing various techniques to preserve privacy LLMs\" - some word seems to be missing here\n* Section 6.1.2 also maybe warrants a brief introduction into how and why differential privacy works in general.\n* Section 6.2: \"Li et al. (2024m) studied membership inference against LMMs\" - reads very vague.\n* Section 6.3: \"Ghalebikesabi et al. (Ghalebikesabi et al., 2023)\" -> use \\citet.\n* \"... offering enhanced privacy protections\" - what protections? By doing what?\n* Section 6.4.1: \"For example, certain attacks require poisoning or injecting triggers...\"  - What kind of attacks?\n* Section 6.4.2: \"Integrating DP into the foundation model fine-tuning still leads to significant performance degredation\" - references? Same with the following paragraph \"However, these methods also suffer from significant performance degradation\".\n\nSection 7\n---------\n\n* If the definition of hallucination is by the authors, this should be made more explicit. It would also be interesting to compare the definition to others in the literature. \n* Figure 12 should also be referenced in 7.1.1, since it depicts the different kinds of hallucinations that are discussed there.\n* Is \"Instruction Deviation\" really a type of hallucination, or just an indicator of a erroneous model?\n* Section 7.2: \"For example, hallucinations in LMMs are often observed in which the model predicts possible objects or actions that are not supported by the image but are plausible from commensense\" - references?\n* The nested parantheses due to citations in the \"Knowledge Boundaries\" point could be mitigated through \\citealp\n* \"Path Dependence. The autoregressive nature of LLMs and LMMs causes hallucination.\" What is the evidence that this is because the models are autoregressive? Are diffusion models actually autoregressive?\n* The title of 7.3.1 should perhaps be \"Detecting language hallucinations\"?\n* The citation of \"Jiang et al. (2024b)\" should use \\citep.\n* Monte Carlo dropout in the \"uncertainty estimation\" needs a citation.\n* p. 44 second paragraph: Llama is missing a citation. Also, I believe RAG is neither introduced as an acronym nor cited.\n* Title for section 7.4.1 should include an s in \"hallucinations\".\n* \"Training Approaches\" paragraph in 7.4.2: \"best-of-n rejection sampling\" is not really explained.\n* \"Post-processing Approaches\" on p. 46: It is not really clear to me as a reader how hidden Markov models and the Viterbi algorithm are applied in this context.\n* \"Decoding Approaches\" paragraph mentions \"VCD\", but the acronym is not explained and the approach is not explained very intuitively.\n* Section 7.5 might also want to include a discussion on refusal in models in order to avoid hallucinations.\n\nSection 8\n---------\n\n* \"uncertainty\" as a concept in the context of foundation models isn't really well defined.\n* Section 8.1: The aleatoric / epistemic distinction has recently attracted a lot of critique due to its rigidity and reductiveness, see [13-16].\n* Last paragraph before 8.1.2: \"under-prescribed\" -> \"underspecified\"?\n* Critique of optimizing only observed sequence probability in 8.1.2 could be supplemented with [17, 18].\n* Section 8.2: \"Methods have been developed to recalibrate uncertainty scores to match the probability of an answer being correct\" -> This is not completely correct, as you would only calibrate confidence scores in this way. The previous sentence mentions predictive entropy for instance, where we want higher uncertainty to signify a higher chance of being incorrect, but there is no immediate way to \"calibrate\" entropy to reflect the probability of correctness.\n* Section 8.2.2: Many more alternatives to the ECE have been proposed, see for instance [14] p. 37.\n* Section 8.2.5: \"(Yadkori et al., 2024b)\" should use \\citet\n* Section should include [19-21].\n* Section 8.3: Long-form UQ has actually been explored in works such as [22-24].\n* p. 55 \"cross entropy\" -> \"cross-entropy\"\n* Use \\citealp for nested citation \"(Kadavath et al., 2022; Tian et al., 2023b)\"\n\n\nSection 9\n---------\n\n* There are no citations given for the definition of shifts in section 9.1. Candidates could be [27-29].\n* For me, it would be interesting to see motivated why OOD detection in section 9.2. still matters in the age of extremely large foundation models. \n* The section is missing [28].\n* Section 9.2: It is not extremely clear as a reader what \"OOD peer classes\" are. Also, the concept of far, near, and fine-grained OOD is not explained.\n* Section 9.3: \"Priorto the era of foundation models, various strategies were proposed to improve the OOD generalization capabilities of traditional deep learning models, including ...\" - in the following, no references / examples in the literature are given for any of the strategies.\n* Section 9.3.1: \"Easy Data Augmentation\" isn't really expalained.\n* Section 9.3.4: It is not clear as a reader what these different loss functions do, how they differ, and why they would work.\n* Section 9.3.5: Model ensembles warrant some citations, for instance [29, 30].\n* p. 60: \"In-Context\" -> \"in-context\"\n* Second paragraph p. 60: Missing citations for T5 and ViT.\n* The last paragraph again declares SFT as a stage of RLHF, but the next sentence contrasts RLHF and SFT in terms of generalization.\n* Section 9.4.3 perhaps misses works on activation steering / finetuning such as [31, 32.] \n* p.63: I am not sure that \"FFN\" as acronym was previously introducted. \n* Use \\citealp for \"(Cao et al., 2021)\"\" nested citation.\n* Paragraph \"Knowledge-based Model Editing\" is missing a citation for integrated gradients.\n* In the same paragraph, the way that ROME and MEMIT work isn't really explained.\n* The end of 9.4.5 could profit from a discussion on how the different model editing methods differ and what kind of advantages and disadvantages they carry.\n\n\nSection 10\n----------\n\n* Section 10.1.2: BERT is spelled Bert and is missing citations. \n* Applications to decoder-only models for gradient-based explanation are missing?\n* Section 10.1.3 is missing [33].\n* Section 10.2.1 is missing early work of [34].\n* Last paragraph in 10.2.1 could also add [35, 36, 37]\n* Referencing the key-value memory in transformers in 10.2.3 could be a reason to give a brief intro into the architecture in the earlier sections.\n* Section 10.3.1 is missing an explanation for / references about (the use of) influence functions. Another work to add is also [38].\n* Section 10.3.3 feels superficial giving the volume of work trying to understand the role of ICL, see for instance \n* Section 10.4: Use \\citealp for \"(Jacovi & Goldberg, 2020)\" citation.\n* Section 10.4.1: \"Another benchmark Hate Explain (Mathew et al., 2021) ask the annotators...\" -> \"Another benchmark, Hate Explain (Mathew et al., 2021), asks the annotators...\"\n* Section 10.4.2: \", and\" after \"predicted rationales\"\n* In the same paragraph \"The lower, the better;\" reads very informal.\n* Section 10.5.1 might be renamed into something like \"Avoiding shortcut learning\"\n* Section 10.5.2: Here some critique of CoT ([10]) should be mentioned alongside the Nye et al. (2021) paper.\n* The section also missed early work on incorporating human feedback, see [42] for an overview.\n* Section 10.7.2.: \"It remains unclear how this knowledge is injected during training\" -> see for instance [43-45].\n* Section 10.7.3: \"Identifying the biases\" -> \"identifying the biases\", \"de-bias strategies\" -> \"de-biasing strategies\"\n* I found it interesting to maybe also end this section on a discussion on which methods can be transferred between different kinds of foundation models (specifically with respect to modality).\n\nSection 11\n----------\n\n* Section 11.1: What about partially altered content?\n* Section 11.2.1 The concept of \"cross-perplexity\" is not explained. \n* In the following paragraph, the fact that AI-generated text \"tends to lie in negative curvature of the model's log probability\" is not clear.\n* p.74 \"By observing that AI-generated text exhibits lower intrinsic dimensionality\" - in what space?\n* Section 11.2.2 should maybe also mention that detection by intuition becomes much harder as models improve.\n* Section 11.3: \"watermark payload\" should use latex quotes, i.e. `` and ''.\n* Section 11.3.2 might want to add [46].\n* Some of the entries 11.3.2 seem to also fit 11.4, which maybe warrants making the differences more explicit.\n* Second paragraph of 11.4: Missing citations for T5 and RoBERTa and Siamese network.\n* Same paragraph: Comma after \"LLMDet\" and \"Ghostbuster\". No comma after \"machine generated-text, therefore\". \n* Last paragraph in 11.4: \"To mitigate this problem, bing\"?\n* Section 11.5.3: \"Especially, The scenarios\" -> \"Especially, the scenarios\"\n\nSection 12\n----------\n\n* Section 12.1: \"Tao et al. (2021); Ma et al. (2024a)\" should be \\citep. Same for \"Zhang et al. (2023j); Cho et al., (2023); and Luccioni et al. (2023)\" in section 12.2..\n* Section 12.3: \\citealp for Liu et al. (2024g) and following citations. \n* Same paragraph: \\citep for Edemacu & Wu (2024) and Xiao et al. (2023a)\n* Section 12.5: \"opposite outcomes Future research directions\" -> \"opposite outcomes, future research directions\"\n* \"include addressing uncertainty not capteured in the model's interal state\" - this point os not very clear to me.\n* \"The risk of hallucination also increases as the model makes inferences based on its prior knowledge, which may not be applicable in the new context\" - missing reference?\n\n\n[1] Schaeffer, Rylan, Brando Miranda, and Sanmi Koyejo. \"Are emergent abilities of large language models a mirage?.\" Advances in Neural Information Processing Systems 36 (2023): 55565-55581.\n\n[2] Lu, Sheng, et al. \"Are emergent abilities in large language models just in-context learning?.\" arXiv preprint arXiv:2309.01809 (2023).\n\n[3] Rogers, Anna, and Alexandra Sasha Luccioni. \"Position: Key claims in llm research have a long tail of footnotes.\" arXiv preprint arXiv:2308.07120 (2023).\n\n[4] Gu, Albert, and Tri Dao. \"Mamba: Linear-time sequence modeling with selective state spaces.\" arXiv preprint arXiv:2312.00752 (2023)\n\n[5] Waseem, Zeerak, et al. \"Disembodied machine learning: On the illusion of objectivity in nlp.\" arXiv preprint arXiv:2101.11974 (2021).\n\n[6] Ulmer, Dennis, et al. \"Bootstrapping llm-based task-oriented dialogue agents via self-talk.\" arXiv preprint arXiv:2401.05033 (2024).\n\n[7] Springer, Jacob Mitchell, et al. \"Overtrained language models are harder to fine-tune.\" arXiv preprint arXiv:2503.19206 (2025).\n\n[8] Kirk, Hannah Rose, et al. \"The prism alignment project: What participatory, representative and individualised human feedback reveals about the subjective and multicultural alignment of large language models.\" arXiv preprint arXiv:2404.16019 (2024).\n\n[9] Sclar, Melanie, et al. \"Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting.\" arXiv preprint arXiv:2310.11324 (2023).\n\n[10] Turpin, Miles, et al. \"Language models don't always say what they think: Unfaithful explanations in chain-of-thought prompting.\" Advances in Neural Information Processing Systems 36 (2023): 74952-74965.\n\n[11] Zollo, Thomas P., et al. \"Prompt risk control: A rigorous framework for responsible deployment of large language models.\" arXiv preprint arXiv:2311.13628 (2023).\n\n[12] https://www.anthropic.com/news/constitutional-classifiers\n\n[13] Baan, Joris, et al. \"Uncertainty in natural language generation: From theory to applications.\" arXiv preprint arXiv:2307.15703 (2023).\n\n[14] Ulmer, Dennis. \"On Uncertainty In Natural Language Processing.\" arXiv preprint arXiv:2410.03446 (2024).\n\n[15] Mucsányi, Bálint, Michael Kirchhof, and Seong Joon Oh. \"Benchmarking uncertainty disentanglement: Specialized uncertainties for specialized tasks.\" Advances in neural information processing systems 37 (2024): 50972-51038.\n\n[16] Gruber, Cornelia, et al. \"Sources of Uncertainty in Machine Learning--A Statisticians' View.\" arXiv preprint arXiv:2305.16703 (2023).\n\n[17] Eikema, Bryan, and Wilker Aziz. \"Is MAP decoding all you need? the inadequacy of the mode in neural machine translation.\" arXiv preprint arXiv:2005.10283 (2020).\n\n[18] Eikema, Bryan. \"The effect of generalisation on the inadequacy of the mode.\" Proceedings of the 1st Workshop on Uncertainty-Aware NLP (UncertaiNLP 2024). 2024.\n\n[19] Ulmer, Dennis, Chrysoula Zerva, and André FT Martins. \"Non-exchangeable conformal language generation with nearest neighbors.\" arXiv preprint arXiv:2402.00707 (2024).\n\n[20] Su, Jiayuan, et al. \"Api is enough: Conformal prediction for large language models without logit-access.\" arXiv preprint arXiv:2403.01216 (2024).\n\n[21] Ravfogel, Shauli, Yoav Goldberg, and Jacob Goldberger. \"Conformal nucleus sampling.\" arXiv preprint arXiv:2305.02633 (2023)\n\n[22] Zhang, Caiqi, et al. \"Luq: Long-text uncertainty quantification for llms.\" arXiv preprint arXiv:2403.20279 (2024).\n\n[23] Yoon, Dongkeun, et al. \"Reasoning Models Better Express Their Confidence.\" arXiv preprint arXiv:2505.14489 (2025).\n\n[24] Zhang, Caiqi, et al. \"Atomic Calibration of LLMs in Long-Form Generations.\" arXiv preprint arXiv:2410.13246 (2024).\n\n[25] Moreno-Torres, Jose G., et al. \"A unifying view on dataset shift in classification.\" Pattern recognition 45.1 (2012): 521-530.\n\n[26] Shimodaira, Hidetoshi. \"Improving predictive inference under covariate shift by weighting the log-likelihood function.\" Journal of statistical planning and inference 90.2 (2000): 227-244.\n\n[27] Hupkes, Dieuwke, et al. \"A taxonomy and review of generalization research in NLP.\" Nature Machine Intelligence 5.10 (2023): 1161-1174.\n\n[28] Arora, Udit, William Huang, and He He. \"Types of out-of-distribution texts and how to detect them.\" arXiv preprint arXiv:2109.06827 (2021).\n\n[29] Lakshminarayanan, Balaji, Alexander Pritzel, and Charles Blundell. \"Simple and scalable predictive uncertainty estimation using deep ensembles.\" Advances in neural information processing systems 30 (2017).\n\n[30] Dietterich, Thomas G. \"Ensemble learning.\" The handbook of brain theory and neural networks 2.1 (2002): 110-125.\n\n[31] Lai, Wen, Alexander Fraser, and Ivan Titov. \"Joint Localization and Activation Editing for Low-Resource Fine-Tuning.\" arXiv preprint arXiv:2502.01179 (2025).\n\n[32] Turner, Alexander Matt, et al. \"Activation addition: Steering language models without optimization.\" arXiv e-prints (2023): arXiv-2308.\n\n[33] Abnar, Samira, and Willem Zuidema. \"Quantifying attention flow in transformers.\" arXiv preprint arXiv:2005.00928 (2020).\n\n[34] Veldhoen, Sara, Dieuwke Hupkes, and Willem H. Zuidema. \"Diagnostic Classifiers Revealing how Neural Networks Process Hierarchical Structure.\" CoCo@ NIPS. 2016\n\n[35] Ji, Ziwei, et al. \"Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinations.\" arXiv preprint arXiv:2503.14477 (2025).\n\n[36] Merullo, Jack, et al. \"On linear representations and pretraining data frequency in language models.\" arXiv preprint arXiv:2504.12459 (2025).\n\n[37] Marks, Samuel, and Max Tegmark. \"The geometry of truth: Emergent linear structure in large language model representations of true/false datasets.\" arXiv preprint arXiv:2310.06824 (2023).\n\n[38] Ruis, Laura, et al. \"Procedural knowledge in pretraining drives reasoning in large language models.\" arXiv preprint arXiv:2411.12580 (2024).\n\n[39] Hahn, Michael, and Navin Goyal. \"A theory of emergent in-context learning as implicit structure induction.\" arXiv preprint arXiv:2303.07971 (2023).\n\n[40] Xie, Sang Michael, et al. \"An explanation of in-context learning as implicit bayesian inference.\" arXiv preprint arXiv:2111.02080 (2021).\n\n[41] Lu, Sheng, et al. \"Are emergent abilities in large language models just in-context learning?.\" arXiv preprint arXiv:2309.01809 (2023).\n\n[42] Fernandes, Patrick, et al. \"Bridging the gap: A survey on integrating (human) feedback for natural language generation.\" Transactions of the Association for Computational Linguistics 11 (2023): 1643-1668.\n\n[43] van der Wal, Oskar, et al. \"PolyPythias: Stability and Outliers across Fifty Language Model Pre-Training Runs.\" arXiv preprint arXiv:2503.09543 (2025).\n\n[43] Müller-Eberstein, Max, et al. \"Subspace chronicles: How linguistic information emerges, shifts and interacts during language model training.\" arXiv preprint arXiv:2310.16484 (2023).\n\n[44] Hu, Michael Y., et al. \"Latent state models of training dynamics.\" arXiv preprint arXiv:2308.09543 (2023).\n\n[45] Saphra, Naomi, and Adam Lopez. \"Understanding learning dynamics of language models with SVCCA.\" arXiv preprint arXiv:1811.00225 (2018).\n\n[46] Yu, Zhuohao, et al. \"SAEMark: Steering Personalized Multilingual LLM Watermarks with Sparse Autoencoders.\"", "broader_impact_concerns": "I do not foresee any broader impact concerns with this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)