# A Survey of Robot Intelligence with Large Language Models
## 5 Discussion and Future Directions
Although limitations exist for robots to self-update, frameworks such as Eureka and DrEureka, which automatically enhanced reinforcement learning performance for robot motion control, demonstrate the potential for future advancements. This suggests that LLMs may not only enhance human interactions but could also pave the way for self-improvement without human intervention. While foundation models offer considerable potential for advancing robotics intelligence, several limitations and future considerations remain. These include (1) the speed of inference required for real-time applications, (2) the computational efficiency necessary for embedded systems, (3) the ability to handle multi-modality information, and (4) the necessity of addressing safety and ethical considerations. First of all, LLMs and VLMs hold considerable potential for enhancing robot intelligence. Nonetheless, several critical issues remain to be addressed. Foundation models, characterized as large-scale models pre-trained on extensive datasets, face challenges related to real-time requirements and limited computational resources in robotic applications. Moreover, concerns such as personal information protection, privacy, and security from external attacks need resolution to enable cloud-based LLMs for robotics. Secondly, to enhance the computational efficiency and usability of LMs, there is ongoing research into small language models (SLMs). Despite having fewer parameters, SLMs can achieve performance comparable to LLMs in specific applications. Several SLMs have been introduced, including DistilBERT [224], which is a compact version of Google's BERT; Phi-3 [61], another SLM; Florence-2 [84], a small VLM model from Microsoft; MobileBERT [225], which is optimized for mobile platforms; and compact open-source versions of OpenAI's GPT models such as GPT-Neo [226] and GPT-J [227]. Generally, SLMs are streamlined models with fewer parameters compared to LLMs, which can number in the billions. SLMs utilize smaller, domain-specific datasets and require shorter training periods, typically just a few weeks, unlike LLMs, which demand vast datasets for broad learning and multiple months of training. Developing SLMs to excel within specific domains for robotic systems and ensuring real-time performance with minimal computational resources are essential research directions for advancing robot intelligence with SLMs. The third implication is that LLMs, based on text-centered natural language processing, are limited as single-modality models when applied to real-world robotic systems where information often blends in diverse ways. Research on LLMs is transitioning from single-modality to multimodality models, as evidenced by VLMs and OpenAI Sora [228], with increasing demand for such models. Currently, to address the limitations of LLMs' single-modality, robotic systems are being developed with multimodality models that integrate vision, such as VLMs. However, relying solely on text and images falls short of the diverse information range required in the real world, including images, sounds, videos, and proprioceptive sensory information (such as the position, orientation, balance, movement degree, and direction of various parts of the robot). Proprioceptive sensory information related to actions and movements is particularly vital for enhancing dynamic human interaction, information processing, and manipulation and planning skills based on dynamic movements. For instance, the integrated VLA model, which facilitates low-level control based on LLMs and VLMs as shown by Google's RT-2 model, highlights the necessity for models capable of integrating information from a broader range of modalities to enhance robot intelligence. Finally, the fourth area to consider is how to address safety and ethical issues when LLM is applied to robotic intelligence systems. Studies were conducted to address the issue of discriminatory and unsafe behaviors that may be generated by robot applications powered by LLMs [229].