# MAISTRO: Towards an Agile Methodology for AI System Development Projects
## 3 Materials and Methods
### Phase IV: Project Execution
#### 3.4.1 Pretrained Models  
The use of previously trained models has proven to be an efficient and promising strategy in the development of AI systems. These models, initially trained in large volumes of data, capture rich and complex representations that can be used for specific tasks withoutthe need for complete training from the beginning. In this context, in addition to the already established BERT (Bidirectional Encoder Representations from Transformers) and GPT (Generative Pre-trained Transformer) models, other models have gained relevance, such as T5 (Text-to-Text Transfer Transformer) and RoBERTa [83; 84].  
The BERT model, proposed by [PERSON] et al. [85], revolutionized natural language processing (NLP) by introducing a bidirectional pre-training technique. By considering the context of words on both sides of a sequence, BERT outperformed previous approaches that processed text in only one direction. On the other hand, the GPT-3 model, developed by OpenAI, impressed with its ability to generate coherent and contextualized text. [PERSON] et al. [86] explain that GPT-3 was trained with a large volume of textual data, allowing it to learn contextual nuances and complex patterns. This model has been applied to intelligent conversational systems, as well as to automatic content generation.  
In addition to these templates, T5 stands out for its unified approach to several NLP tasks, converting all of them into a \"text-to-text\" format. This versatility allows T5 to be applied to translation, summarization, classification, and other tasks related to language processing [84]. RoBERTa, an optimized variation of BERT, also demonstrates how robust training with more data and the removal of next-sentence segmentation can lead to significant improvements in performance [83].  
On the other hand, the fine-tuning strategy, in turn, is a technique that involves adjusting the weights of a pre-trained model to suit a specific task. Instead of training a model from scratch, fine-tuning leverages the knowledge and weights learned by that model on a larger and more general dataset. Its advantages include reducing computational costs and leveraging state-of-the-art models without having to train them from scratch. However, it is essential to evaluate the quality and quantity of data available for training, as well as the computational resources required for the process [84].  
In addition to NLP, pre-trained models also impact the application environment in computer vision. Architectures such as ResNet [87] and EfficientNet [88] were pre-trained on vast image banks, allowing object and scene recognition with minimal adjustments. Reusing these architectures not only accelerates the development of AI systems but also increases the accuracy of models in specific tasks.  
The combination of pre-trained models and fine-tuning strategies offers a promising path for the development of more efficient and adaptable AI systems. The choice between specific models and the application of fine-tuning should be made considering the characteristics of the problem, the available resources, and the objectives of the application.