## 2 Overview of Large Language Models and Their Taxonomy  
An _LLM_ is a sophisticated artificial intelligence capable of understanding and producing text in a manner very similar to a human being. These models are trained on vast amounts of textual data, thereby absorbing grammatical rules, vocabulary, and the nuances of language. The operation of _LLM_s is based on a complex mechanism of machine learning. A key innovation at the heart of modern _LLM_s is the attention mechanism, which enables the model to dynamically focus on different parts of the input text, assigning varying levels of importance to each word or token based on context. This mechanism underpins transformer architectures, where multiple attention heads work in parallel to capture intricate dependencies between words, making these models highly effective for natural language tasks. By analyzing immense amounts of text, these models identify patterns and relationships between words, learning to predict which words best fit a given sentence or context. At the core of this capability lies the use of neural networks--computational systems inspired by the human brain--that enable _LLM_s to learn and make predictions. Once trained, _LLM_s can perform a wide range of tasks: from generating entirely new texts to automatic translation, from answering complex questions to writing code. Essentially, _LLM_s are versatile tools that open new frontiers in human-machine interaction. In summary, _LLM_s are AI models that, thanks to machine learning the attention mechanism and neural networks, can understand and generate text in increasingly sophisticated and natural ways. The roots of _LLM_s trace back to the 1950s and 1960s, with the development of the first artificial neural networks. These networks, inspired by the functioning of the human brain, offered a new perspective for tackling complex problems such as natural language understanding. However, it was only with the advent of backpropagation in the 1980s that neural networks began to become practical tools for complex tasks. This algorithm allowed networks to learn from their mistakes and improve their performance over time. In recent years, the explosion of _LLM_s has been made possible by increasingly powerful hardware and the advent of cloud computing, which have enabled the training of largerand more complex models. Another fundamental aspect contributing to the rise of _LLM_s is the availability of vast amounts of data. This availability is one of the most significant effects of the explosion of the internet and digitalization, which have become the fuel for training _LLM_s. One of the main characteristics of an _LLM_ is, indeed, the size of the model and the quantity of data used during the training phase. An _LLM_ contains an enormous number of parameters, which are the weights and biases that the model learns during training. The greater the number of parameters, the higher the model's ability to capture complex information from the text. The largest models contain billions of parameters. _LLM_s are trained on massive datasets, which may include documents, newspaper articles, books, web pages, and more. This training allows them to learn a wide range of information about language, grammar, context, and facts, which are influenced by key characteristics of an _LLM_ such as its architecture, the number of parameters (in billions), and the size of the training corpus. Figure 1 shows the main _LLM_s based on their characteristics and the type of architecture. The temporal evolution highlights a significant increase in both the number of parameters and the size of the training datasets, particularly for the most recent models. Figure 1: _LLM_ by year. X-axis (Year): Indicates the introduction period of each model. Y-axis (model name, parameters in billions—B): Logarithmic scale representing the number of parameters in each model. Point size: Proportional to the size of the training dataset (in trillions of tokens). Triangle size: Proportional to the number of model parameters (in billions).