## 2 Preliminaries on Large Language Models  
In this section, we provide a brief overview of the methodological foundations of LLMs; for detailed descriptions, please refer to surveys in the related literature, e.g., [10; 16; 17; 18]. Traditional Recurrent Neural Networks (RNNs) had two major drawbacks: the inability to (a) effectively deal with long-range dependencies and (b) perform parallel computations, leading to low computational efficiency. The need for modern large-scale models required the development of novel methodologies, and transformers filled this role by revolutionizing natural language processing through the introduction of a self-attention mechanism. This mechanism allows models to capture relationships between distinct words much more effectively, weighing the importance of different words in a sentence when generating or understanding the text. Modern LLMs, including GPT, are largely built on the transformer architecture proposed by [19]. The transformer mainly addresses the Seq2 Seq problem, where the input and output sequences are often of different lengths; it consists of two main components: the encoder and the decoder. The encoder takes an input sequence of tokens and processes them in parallel. It consists of multiple layers of self-attention and feed-forward neural networks. Each layer has multiple attention heads that capture different aspects of the input sequence. The decoder, on the other hand, generates the output sequence autoregressively, token by token. It also consists of multiple layers of self-attention and feed-forward networks but with an additional attention mechanism over the encoder's output. During training, the model is fed with the input sequence, and the decoder learns to predict the next token given the previous tokens. Figure 1 shows an example of the encoder-decoder architecture. \(x_{1},x_{2},x_{3}\) represent the input sequence, which, after passing through the encoder, produces hidden states \(h_{1},h_{2},h_{3}\). The purpose of the decoder is to generate words sequentially. Initially, the special token \(<bos>\) (the beginning of sentence) is given, and the decoder generates \(y_{1}\). Given \(y_{1}\), the decoder generates \(y_{2}\), and so on, until the decoder generates the special token \(<cos>\) (end of sentence), indicating the completion of the sentence. Figure 1: Encoder–decoder architecture used in LLMs. The attention mechanism is a key component of Large Language Models and plays a vital role in capturing dependencies and relationships among words in a sentence. Attention allows the model to focus on different parts of the input sequence when generating or understanding each token. Given an input sequence, the attention mechanism computes a weighted sum of the values associated with each token, where the weights are determined by the relevance of each token. The relevance is computed by comparing the query vector, derived from the current position, with the key vectors derived from all positions in the sequence. Let each word be represented as \((x_{1},x_{2},\ldots,x_{n})\). Compute the corresponding query vectors \(q\), key vectors \(k\), and value vectors \(v\) through linear transformations:  
\[q_{i}=W_{q}x_{i},\quad k_{i}=W_{k}x_{i},\quad v_{i}=W_{v}x_{i}\]  
Split the query vectors \(q\), key vectors \(k\), and value vectors \(v\) into multiple heads \(q_{ih},k_{ih},v_{ih}=W_{qh}x_{i}\), \(W_{bh}x_{i}\), \(W_{vl}x_{i}\), where \(h\) denotes the \(h\)-th attention head, and \(W_{q}^{h},W_{k}^{h},W_{v}^{h}\) are the corresponding linear transformation weights. The core formula for the transformer's attention mechanism is \(\text{Attention}(Q,K,V)=\text{softmax}\left(\frac{QK^{T}}{\sqrt{d_{k}}}\right)V\), where, \(Q,K,V\) represent queries, keys, and values, respectively, and \(d_{k}\) is the dimension of the keys. The attention mechanism allows the model to assign higher weights to relevant tokens and lower weights to irrelevant ones, enabling it to focus on the most important information for a given context.