ICLR 2025

Published as a conference paper at ICLR 2025
GREATER:
GRADIENTS OVER REASONING MAKES
SMALLER LANGUAGE MODELS STRONG PROMPT OP-
TIMIZERS
Sarkar Snigdha Sarathi Das†
Ryo Kamoi†
Bo Pang⋄
Yusen Zhang†
Caiming Xiong⋄
Rui Zhang†
†The Pennsylvania State University
⋄Salesforce Research
{sfd5525, rmz5227}@psu.edu
ABSTRACT
The effectiveness of large language models (LLMs) is closely tied to the design of
prompts, making prompt optimization essential for enhancing their performance
across a wide range of tasks. Many existing approaches to automating prompt
engineering rely exclusively on textual feedback, refining prompts based solely
on inference errors identified by large, computationally expensive LLMs. Unfor-
tunately, smaller models struggle to generate high-quality feedback, resulting in
complete dependence on large LLM judgment. Moreover, these methods fail to
leverage more direct and finer-grained information, such as gradients, due to op-
erating purely in text space. To this end, we introduce GREATER, a novel prompt
optimization technique that directly incorporates gradient information over task-
specific reasoning.
By utilizing task loss gradients, GREATER enables self-
optimization of prompts for open-source, lightweight language models without the
need for costly closed-source LLMs. This allows high-performance prompt opti-
mization without dependence on massive LLMs, closing the gap between smaller
models and the sophisticated reasoning often needed for prompt refinement. Ex-
tensive evaluations across diverse reasoning tasks including BBH, GSM8k, and
FOLIO demonstrate that GREATER consistently outperforms previous state-of-
the-art prompt optimization methods, even those reliant on powerful LLMs. Ad-
ditionally, GREATER-optimized prompts frequently exhibit better transferability
and, in some cases, boost task performance to levels comparable to or surpass-
ing those achieved by larger language models, highlighting the effectiveness of
prompt optimization guided ...