LLM-based Optimization of Compound AI Systems: A Survey
Authors: name: Matthieu Lin, name: Jenny Sheng, name: Andrew Zhao, name: Shenzhi Wang, name: Yang Yue, name: Yiran Wu, name: Huan Liu, name: Jun Liu, name: Gao Huang, name: Yong-Jin Liu

In a compound AI system, components such as an LLM call, a retriever, a code
interpreter, or tools are interconnected. The system's behavior is primarily
driven by parameters such as instructions or tool definitions. Recent
advancements enable end-to-end optimization of these parameters using an LLM.
Notably, leveraging an LLM as an optimizer is particularly efficient because it
avoids gradient computation and can generate complex code and instructions.
This paper presents a survey of the principles and emerging trends in LLM-based
optimization of compound AI systems. It covers archetypes of compound AI
systems, approaches to LLM-based end-to-end optimization, and insights into
future directions and broader impacts. Importantly, this survey uses concepts
from program analysis to provide a unified view of how an LLM optimizer is
prompted to optimize a compound AI system. The exhaustive list of paper is
provided at
https://github.com/linyuhongg/LLM-based-Optimization-of-Compound-AI-Systems.