CodeSteer: Symbolic-Augmented Language Models via Code/Text Guidance
Authors: name: Yongchao Chen, name: Yilun Hao, name: Yueying Liu, name: Yang Zhang, name: Chuchu Fan

Existing methods fail to effectively steer Large Language Models (LLMs)
between textual reasoning and code generation, leaving symbolic computing
capabilities underutilized. We introduce CodeSteer, an effective method for
guiding LLM code/text generation. We construct a comprehensive benchmark
SymBench comprising 37 symbolic tasks with adjustable complexity and also
synthesize datasets of 12k multi-round guidance/generation trajectories and
5.5k guidance comparison pairs. We fine-tune the Llama-3-8B model with a newly
designed multi-round supervised fine-tuning (SFT) and direct preference
optimization (DPO). The resulting model, CodeSteerLLM, augmented with the
proposed symbolic and self-answer checkers, effectively guides the code/text
generation of larger models. Augmenting GPT-4o with CodeSteer raises its
average performance score from 53.3 to 86.4, even outperforming the existing
best LLM OpenAI o1 (82.7), o1-preview (74.8), and DeepSeek R1 (76.8) across all
37 tasks (28 seen, 9 unseen). Trained for GPT-4o, CodeSteer demonstrates
superior generalizability, providing an average 41.8 performance boost on
Claude, Mistral, and GPT-3.5. CodeSteer-guided LLMs fully harness symbolic
computing to maintain strong performance on highly complex tasks. Models,
Datasets, and Codes are available at
https://github.com/yongchao98/CodeSteer-v1.0.