# A Survey of Robot Intelligence with Large Language Models
## 4 Language Models for Robotic Intelligence
### Low-Level Control
The collected data are then assessed for (diversity scoring) the visual diversity of the robot trajectories and the linguistic diversity of the language instructions generated by AutoRT (LLM). The aim of this diversity evaluation is to confirm that, unlike simulations, real-world data collection by robots is labor-intensive, making it essential to gather data across a broad spectrum of tasks. Experimental outcomes illustrate that AutoRT achieves higher visual and linguistic diversity compared to RT-1 or BC-Z [143]. Other researchers include [PERSON] [144], who developed an approach that connects natural language user commands with a locomotion controller using foot contact patterns as an interface for low-level commands. This innovative interface translates human commands into the robot's foot contact patterns, allowing the robot to move at a specified speed with precise timing for each foot's contact with the ground. To achieve this, the robot used a cyclic sliding window to extract foot contact flags from a pattern template, thus generating the required foot contact patterns. During training, a random pattern generator created foot contact patterns, and during testing, an LLM translated human commands into these patterns. The robot then adjusted its movements based on the foot contact patterns it learned through deep reinforcement learning, closely adhering to the intended foot contact patterns and speed commands. This approach demonstrated a 50% higher success rate in task evaluation (across 30 tasks, including standing still) compared to two baselines (which employed discrete gaits and sinusoidal functions as interfaces), successfully solving 10 more tasks than the baselines. [PERSON] [145] introduced a novel method for multi-robot collaboration that utilizes LLMs for both high-level communication and low-level path planning. In this method, the robots employ the LLM to discuss and reason about task strategies. They generate sub-taskplans and task space waypoint paths, which a multi-arm motion planner then uses to expedite trajectory planning. Additionally, environmental feedback, such as collision detection, prompts the LLM agent to refine plans and waypoints contextually. This method achieved a high success rate across all tasks in the RoCoBench (including duties such as sweeping the floor), effectively adapting to variations in task semantics. In real-world experiments, specifically the block-sorting task, RoCo demonstrated its ability to communicate and collaborate with other robot agents to successfully complete the tasks. [PERSON] [146] proposed a novel paradigm for utilizing few-shot prompts in physical environments. This method involved gathering observation and action pairs from existing model-based or learning-based controllers to form the initial text prompts. Data included sensor readings, such as IMU and joint encoders, coupled with target joint positions. These data formed the starting input for LLM inference. As the robot interacted with its environment and collected new observational data, these initial data were updated with outputs from the LLM. In the subsequent prompt engineering phase, observation and action pairs, along with explanatory prompts, were crafted to enable the LLM to function as a feedback policy. The explanatory prompts provided clear descriptions of the robot walking task and control design details, while the observation and action prompts delineated the format and significance of each observation and action. This method allowed the LLM to directly output low-level target joint positions for robot walking. The approach was tested using the ANYmal robot in MuJoCo and Isaac Gym simulators for robot walking, indicating that the LLM could act as a low-level feedback controller for dynamic motion control within sophisticated robot systems. [PERSON] [147] introduced a new framework named _Code as Policies_ (CaP) that directly constructs robot policies from executable code generated by a code LLM.