ICML 2025

Logits are All We Need to Adapt Closed Models
Gaurush Hiranandani 1 Haolun Wu* 2 3 Subhojyoti Mukherjee† 4 Sanmi Koyejo 2
Abstract
Many commercial Large Language Models
(LLMs) are often closed-source, limiting devel-
opers to prompt tuning for aligning content gen-
eration with specific applications. While these
models currently do not provide access to token
logits, we argue that if such access were available,
it would enable more powerful adaptation tech-
niques beyond prompt engineering. In this paper,
we propose a token-level probability reweighting
framework that, given access to logits and a small
amount of task-specific data, can effectively steer
black-box LLMs toward application-specific con-
tent generation. Our approach views next-token
prediction through the lens of supervised classi-
fication. We show that aligning black-box LLMs
with task-specific data can be formulated as a la-
bel noise correction problem, leading to Plugin
model – an autoregressive probability reweighting
model that operates solely on logits. We provide
theoretical justification for why reweighting logits
alone is sufficient for task adaptation. Extensive
experiments with multiple datasets, LLMs, and
reweighting models demonstrate the effectiveness
of our method, advocating for broader access to
token logits in closed-source models. We provide
our code at this https URL.
1. Introduction
The rise of Large Language Models (LLMs) has revolution-
ized generative Artificial Intelligence, yet the most capable
models are often closed-source or black-box (Achiam et al.,
2023; Bai et al., 2022a). These models generate text based
on input prompts but keep their internal weights and training
data undisclosed, limiting transparency and customization.
*Work done while visiting Stanford University. †Work done
while PhD Candidate at UW Madison. 1Typeface 2Stanford Univer-
sity 3Mila - Quebec AI Institute 4Adobe Research. Correspondence
to: Gaurush Hiranandani <gaurush@typeface.ai>, Haolun Wu
<haol...