# A Survey of Robot Intelligence with Large Language Models
## 5 Discussion and Future Directions  
The review revealed two potentials of foundation models: (1) commonsense reasoning for planning and (2) the ability to generate code. The first finding from this review study is the potential to enhance robot intelligence through foundation models. Beyond the studies mentioned here, numerous recent studies have shown that pre-trained models such as LLMs and VLMs can enhance various aspects of robot intelligence, such as situational awareness, high-level task planning, and human interaction. LLMs allow communication with humans in natural languages, object  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Name** & **Explanation** & **Ref. ** \\ \hline Reward Design in RL & \(\bullet\) & Eureka automatically generates and improves reward functions based on the virtual environment source code. & \\  & \(\bullet\) & DrEureka builds reward-aware physics priors using Eureka and supports effective operation in the real world through domain randomization. & [11, 134, 136, 137, 138, 139, 176, 177, 178, 139] \\  & \(\bullet\) & LLMs design and refine reward functions based on natural language input. & \\  & \(\bullet\) & LLMs and VLMs integrate multimodal data to generate reward functions. & \\ \hline Low-level Control & \(\bullet\) & Generating commands to control actuators capable of low-level control. & \\  & \(\bullet\) & RT-1 and RT-2 enable robots to perform complex tasks based on language-vision data. & [8, 9, 10, 144, 145, 146, 147, 148, 181, 182, 183] \\  & \(\bullet\) & AutoRT establishes a system where robots can autonomously collect and utilize data. & \\ \hline High-level Planning & \(\bullet\) & LLMs provide an effective methodology for tasks related to high-level planning within robotic systems. & \\  & \(\bullet\) & By using natural language, LLMs can formulate plans to solve tasks that require long-horizon reasoning. & [149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183] \\  & \(\bullet\) & LLMs assess the feasibility of actions to determine and execute the optimal robotic behavior. & \\  & \(\bullet\) & LLMs generate behavior trees to structure complex robotic actions accurately. & \\ \hline Manipulation & \(\bullet\) & Using LLMs and VLMs to integrate language and vision data allows various manipulations. & \\  & \(\bullet\) & LLMs interpret high-level instructions to generate the necessary robot actions and assess their feasibility. & [161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 200, 204, 207, 209, 202, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 222, 223] \\  & \(\bullet\) & For sene understanding, estimate and identify objects and evaluate relationships between objects. & [168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 204, 205, 206, 207, 208, 209 utilization based on extensive information, and high-level planning using that information. VLMs can describe tasks in text and understand visual information. Furthermore, the information from VLMs can be supplemented by connecting to knowledge databases via LLMs. These capabilities are crucial for enhancing robot intelligence, broadening the scope of robot applications, and maximizing robot utility. The second finding is the code generation capability of LLMs, which has the potential to automate the robot development process traditionally performed by humans. Additionally, robots that can autonomously update their own algorithms are no longer just science fiction.