## 1 Introduction
Text representation is a fundamental step that enables automatic feature selection and feature engineering. In this context, researchers have mainly used two approaches: bag-of-words (BoW) and contextualized word embedding. [PERSON] and [PERSON] [16; 20] compared Word2 Vec, GloVe, and FastText vectors trained from the Nucleo Interinstitucional de Linguistica Computacional (NILC) word embedding repository [21] on a downstream SA task and found that the embedding technique and its dimensionality have a profound impact on the model performance. Additionally, they analyzed how reliable the representation of FastText-300 was along with LSTM, CNN, and TLM architectures for embedding generation, suggesting that fine-tuned BERTimbau weights provide the best classification performance, slightly better than LSTM and CNN, as also verified by [PERSON] et al. [22] when compared with GloVe, Word2 Vec, and FastText. Regardless, the authors prefer BoW techniques due to their implementation simplicity [17; 19; 23; 24]. [PERSON] et al. [23] presented an ensemble approach combining decision trees, NB, SVM, and LR for binary classification, attaining an accuracy rate of 82%. [PERSON] et al. adopted a binary LR classification model with word vector embeddings to attain an ROC-AUC of 94.8% [16] and an LR model to assess the classification performance when using embedding generated by fine-tuned BERTimbau, attaining a score of 97.3% [20]. Bothresults were obtained from training with multiple-domain datasets. [PERSON] and [PERSON] [25] optimized a CNN binary classification model using a grid search algorithm and attained an average performance of 82% in accuracy. [PERSON] et al. [26] performed a similar analysis, exploring other classical ML approaches such as LR, NB, RF, and SVM and comparing them to BERTimbau, also considering the three-class classification scenario (polarity with a neutral class). Their conclusions suggest that the BERT model outperforms in every data domain, although the imbalance of the classes produces a detriment on minority classes, which can be easily seen by the 74% F1-score. [PERSON] and [PERSON] [17] also tackled the three-class scenario, attaining around a 59.7% F1-score using NB and SVM classifiers. They even highlighted the importance of considering the SVM classifier in the SA task using Portuguese written texts. [PERSON] et al. [27] proposed retraining the BERTimbau and RoBERTa models on restaurant reviews from Zomato Portugal using an ensemble approach. Using two BERTimbau instances as weak classifiers was shown to perform the best in their evaluation, attaining an accuracy rate of 84% for the three-class classification problem. With the advent of TLM-based models [28], researchers started integrating attention mechanisms to improve the NLP performance, mainly with English datasets. [PERSON] et al. [29] proposed a self-AT-LSTM model with character-based input embeddings utilizing a CNN with max-pooling to solve the out-of-vocabulary (OOV) problem, BiLSTM layers, and multi-head self-attention layers. They empirically demonstrated that attention models can identify hierarchical information between words that are too far away from each other, either when working in cooperation with LSTMs or CNNs for feature extraction. In the same year, [PERSON] et al. [30] utilized a full attention model composed of a multi-head self-attention encoder followed by a local attention layer and a feed-forward neural network (FFNN) classifier. They proposed a layer-wise attention tracing (LAT) method to trace the attention 'paid' to the input. They demonstrated that the accumulated attention scores tend to favor words with greater semantic meaning, which appeals to SA. The research questions that this study aimed to answer were the following:  
1. Is it possible to develop high-accuracy SA models for the Portuguese language without relying on LLMs? 2.