# A Survey of Robot Intelligence with Large Language Models
## 3 Related Works
### LLM Architectures and Tunings  
The architecture of LLMs fundamentally utilizes the transformer architecture, with three representative types based on different transformer configurations as shown in Figure 2. Firstly, the prevalent encoder-decoder structure of transformers employs the encoder to process the input sequence and generate a latent representation through multi-head self-attention layers; the decoder then uses cross-attention on this representation to autoregressively produce the target sequence. Notable encoder-decoder PLMs include T5 [47] and BART [33], with Flan-T5 [106] being an encoder-decoder-based LLM. Secondly, the causal decoder employs a unidirectional attention mask to restrict each input token to attend only to past and present tokens, processing input and output tokens similarly through the decoder. This method underpins the development of the GPT series. Lastly, the prefix decoder, resembling the causal decoder's masking mechanism, allows bidirectional attention on prefix tokens [107] and unidirectional attention on generated tokens. Similar to the encoder-decoder, the prefix decoder bidirectionally encodes the prefix sequence and sequentially predicts output tokens individually. Examples of prefix decoder-based LLMs include GLM-130B [108] and U-PaLM [109]. Additionally, various architectures have been proposed to address efficiency challenges during training or inference with long inputs, due to the quadratic computational complexity of the traditional transformer architecture. For instance, the Mixture-of-Experts (MoE) scaling method [34] sparsely activates a subset of the neural network for each input. In terms of the tuning of LLMs, these models are essentially pre-trained on massive datasets and require fine-tuning for different application domains. However, the considerable model size and number of parameters pose challenges for fine-tuning on standard computers and GPUs. The subsequent sections will discuss methods to address these challenges. LLM tuning is broadly divided into two categories based on the training objective. Instruction tuning is a form of supervised learning where the training data typically include descriptions of tasks, inputs, and corresponding outputs. This type of tuning is designed (1) to enhance the functional capabilities of LLMs, (2) to specialize them by training with discipline-specific information, and (3) to improve task generalization and consistency through a better understanding of natural language commands. Conversely, alignment tuning (or preference alignment) seeks to align the behavior of LLMs with human values and preferences. Prominent methods include reinforcement learning from human feedback  
Figure 2: Attention patterns in three mainstream architectures: Causal Decoder (**left**), Prefix Decoder (**middle**), and Encoder–Decoder (**right**). The blue, green, yellow, and grey rounded rectangles represent attention between prefix tokens, attention between prefix and target tokens, attention between target tokens, and masked attention [5]. (RLHF) [110], which involves fine-tuning LLMs using human feedback to better reflect human values, and direct preference optimization (DPO) [111], focusing on training with pairs of human preferences that usually include an input prompt and the preferred and non-preferred responses. For both instruction tuning and alignment tuning, which involve training LLMs with extensively large model parameters, substantial GPU memory and computational resources are required, with high costs typically incurred when utilizing cloud-based resources. Under these conditions, parameter-efficient fine-tuning (PEFT) offers a method designed to efficiently conduct fine-tuning of such LLMs [112]. Among the methods of PEFT, there are four major approaches as shown in Figure 3: adapter tuning, prompt tuning, prefix tuning, and low-rank adaptation (LoRA).