# A Survey of Robot Intelligence with Large Language Models
## 4 Language Models for Robotic Intelligence
### High-Level Planning (Including Decision-Making and Reasoning)
[PERSON] [154] introduced a novel approach to autonomous robot control named LLM-BRAIn, which facilitated the command-based generation of robot behaviors. LLM-BRAIn, a transformer-based LLM, fine-tuned the Stanford Alpaca 7B model to generate robot behavior trees (BTs) from textual descriptions. The developed model was compact enough to operate on a robot's onboard microcomputer, while adept at constructing complex robot behaviors. It provided structurally and logically correct BTs and demonstrated the ability to handle instructions that were not included in the training set. [PERSON] [155], as shown in Figure 9, proposed LLM-Planner, a system designed for few-shot planning in embodied agents. LLM-Planner processed natural language instructions to generate high-level plans, selected subgoals from these plans, and identified actions via a low-level planner. It continuously updated environmental information as new objects were detected during action implementation and revisited the LLM to adjust the plan if subgoals failed or were delayed based on updated observations. This iterative process was repeated until the subgoal was achieved, after which the system moved to the next goal. Compared to traditional models such as HLSM and FILM, LLM-Planner demonstrated competitive performance with significantly reduced training data and proved its ability to generalize in various tasks (e.g., ALFRED) with minimal examples. [PERSON] [156], as shown in Figure 10, introduced ProgPrompt, a programmatic LLM prompt structure designed for generating plans across diverse situated environments, robot capabilities, and tasks. ProgPrompt functioned as a robot task-planning system that leveraged LLMs and included a Python programming structure to facilitate information about the environment and executable actions. It featured a feedback mechanism, using executable program plan examples and assertion statements to mitigate errors, enhancing task success rates. Additionally, ProgPrompt verified the current state through environmental feedback during plan execution and revised the plan accordingly. The results indicated that the integration of programming language features substantially improved task performance in contexts such as VirtualHome and real-world manipulation tasks in terms of success rate, goal conditions recall, and executability. Figure 9: LLM-Planner is a system that creates high-level plans based on natural language commands, sets subgoals to determine actions, and continuously updates the plan to reflect environmental changes [155]. [PERSON] [157] introduced SayPlan, a scalable method for large-scale task planning using LLMs and based on a 3D Scene Graph (3 DSG) representation. SayPlan involved the LLM searching a collapsed 3D scene graph and task instructions to identify all relevant items and then locating the subgraph that contained the necessary items to complete the task. The identified subgraph was subsequently used by the LLM to generate a high-level plan that addressed the navigational aspect of the task. This plan was formatted as a JSON 3D scene graph and subjected to a repetitive replanning process through feedback from the scene graph simulator and a set of API calls for manipulation and operation until an executable plan was determined. SayPlan was tested in two large-scale environments, featuring up to three floors, 36 rooms, and 140 assets and objects, proving its capability to ground large-scale and long-horizon task plans from abstract and natural language instructions, thereby enabling a mobile manipulator robot to execute these tasks. [PERSON] [158], as shown in Figure 11, proposed the Socratic model (SM), a modular framework that synergistically utilizes various forms of knowledge and employs multiple pre-trained models to exchange information and leverage new multimodal capabilities. SM operates without fine-tuning by integrating diverse pre-trained models and functions in a zero-shot approach (e.g., using multimodal prompts), which enables it to harness new multimodal capabilities.