Lemur: Harmonizing Natural Language and Code for Language Agents
Authors: name: Yiheng Xu; name: Hongjin Su; name: Chen Xing; name: Boyu Mi; name: Qian Liu; name: Weijia Shi; name: Binyuan Hui; name: Fan Zhou; name: Yitao Liu; name: Tianbao Xie; name: Zhoujun Cheng; name: Siheng Zhao; name: Lingpeng Kong; name: Bailin Wang; name: Caiming Xiong; name: Tao Yu
Published: 2023-10-10 17:57:45

Abstract: We introduce Lemur and Lemur-Chat, openly accessible language models
optimized for both natural language and coding capabilities to serve as the
backbone of versatile language agents. The evolution from language chat models
to functional language agents demands that models not only master human
interaction, reasoning, and planning but also ensure grounding in the relevant
environments. This calls for a harmonious blend of language and coding
capabilities in the models. Lemur and Lemur-Chat are proposed to address this
necessity, demonstrating balanced proficiencies in both domains, unlike
existing open-source models that tend to specialize in either. Through
meticulous pre-training using a code-intensive corpus and instruction
fine-tuning on text and code data, our models achieve state-of-the-art averaged
performance across diverse text and coding benchmarks among open-source models.
Comprehensive experiments demonstrate Lemur's superiority over existing
open-source models and its proficiency across various agent tasks involving
human communication, tool usage, and interaction under fully- and partially-
observable environments. The harmonization between natural and programming
languages enables Lemur-Chat to significantly narrow the gap with proprietary
models on agent abilities, providing key insights into developing advanced
open-source agents adept at reasoning, planning, and operating seamlessly
across environments. https://github.com/OpenLemur/Lemur