## 4 Methods That Use Large Language Models
### Introduction to Large Language Models (LLMs)  
During the last few years, an abundance of pre-trained models has been developed and used in various linguistic tasks, boosting the performance of traditional methods, and using only fine-tuning in most cases. In this section, we present the details of these models, whereas Table 2 provides a summary of the most frequently used models. The evolution of large language models began with BERT (Bidirectional Encoder Representations from Transformers), introduced by Google researchers in 2018. BERT is a language representation model that learns language in a bidirectional manner. It has two variants: BERT-BASE (12 layers, 110 M parameters) and BERT-LARGE (24 layers, 340M parameters). BERT was trained on the BooksCorpus (800 million words) and English Wikipedia (2.5 billion words), excluding lists, tables, and headers [11]. Building on BERT, M-BERT (Multilingual BERT) was released in 2019 as a single language model pre-trained on Wikipedia pages of 104 languages. It has a 12-layer transformer architecture and comes in several variants: BERT-Base Multilingual Cased (104 languages, 110 M parameters), BERT-Base Multilingual Uncased (102 languages, 110M parameters), and BERT-Base Chinese (110 M parameters) [51]. Another significant model is T5 (Text-to-Text Transfer Transformer), released in 2019, which is an encoder-decoder model that converts each task into a text-to-text format. It was trained on the Colossal Clean Crawled Corpus (C4) dataset, a 750 GB dataset created from Common Crawl's web-extracted text. The model architecture is similar to the original Transformer, with both encoder and decoder consisting of 12 blocks, and it has 220 million parameters [13]. Following T5, RoBERTa is an enhanced version of BERT with more data, dynamic masking, and byte-pair encoding. RoBERTa improves performance by training longer, using larger batches, removing the next sentence prediction objective, and dynamically changing the masking pattern. RoBERTa was trained on a large dataset called CC-NEWS [52]. In the realm of models that are fine-tuned for specific tasks, FN-BERT stands out as a BERT-based model fine-tuned on a fake news classification dataset in 2023. The model is based on distilBERT and was trained using a specific fake news dataset [53]. Meanwhile, Grover, developed by the Allen Institute for AI, is a GAN-based model designed to generate and detect fake news. Grover has three sizes: Grover-Base (117 M parameters), Grover-Large (345 M parameters), and Grover-Mega (1.5 B parameters). It was trained on the RealNews dataset, created from Common Crawl news articles [54]. Recent advancements include Llama2, a collection of pre-trained and fine-tuned LLM models from Meta with varying parameter sizes. It includes Llama 2 Chat, optimized for dialogue, and Code Llama, specialized for code generation. Llama2 was trained on 2 trillion tokens, with increased context length and grouped-query attention [55]. Additionally, Meta has developed the Llama3 models, which show significant improvements in performance and scalability compared to their predecessors. Microsoft's Phi3 [56] also presents significant advancements in performance and scalability. It emphasizes improvements in computational efficiency and model adaptability,making it a strong contender in the evolving landscape of large language models. Phi-3 comes in different versions ranging from 3.8 B (Phi-3 mini) to 14 B (Phi-3 medium) parameters, which offer two context lengths, 128 K and 4 K, and outperform bigger models. They have been trained on 3.3 million tokens. OpenAI's GPT-3 demonstrated remarkable abilities in translation, question answering, and content generation, comparable to human-written texts. Furthermore, ChatGPT-3.5, also developed by OpenAI, is widely used for its conversational abilities. It was trained using Reinforcement Learning from Human Feedback (RLHF) and fine-tuned from a model in the GPT-3.5 series. ChatGPT can answer follow-up questions, admit mistakes, and reject inappropriate requests [57].