## 3 Proposed Approach
### Classical Machine Learning Models for DBMM
#### 3.4.2 Base Classifiers for Sentiment Analysis  
For sentiment analysis, we employed both pre-trained language models and classical machine learning algorithms, with specific configurations and parameters for each, as detailed in Table 3 and explained below:  
* BERT (BERT base uncased): This model leverages a transformer architecture with self-attention mechanisms to capture contextual relationships in text. We fine-tuned it for sequence classification using pre-trained weights and optimized it with the AdamW optimizer (with weight decay) and cross-entropy loss over 350 epochs, using a batch size of 32 and a linear learning rate scheduler with warmup steps.
* GPT-2: Similarly, this transformer-based model was fine-tuned for sequence classification with the same configuration as BERT. However, its architecture relies on decoder blocks, specialized for sequential text generation.
* Logistic regression: We applied this simple linear model with default parameters (maximum iterations = 1500) to TF-IDF features extracted from text data.
* SVM: This model, effective for high-dimensional text data, was configured with a linear kernel and a regularization parameter (C) of 1.0 to control overfitting. It was trained on TF-IDF features.
* RF: This ensemble method combines the predictions of multiple decision trees for improved accuracy. Our RF model utilized 100 decision trees and was trained on TF-IDF features.
* MLP: Designed with two hidden layers (320 and 192 neurons), was trained on TF-IDF features using the Adam optimizer. ReLU activation was used in the hidden layers, and a softmax activation function was employed in the output layer for multi-class classification.
* 1D-CNN: We adapted the architecture from our SER experiments to the sentiment analysis task. This model, composed of convolutional, max-pooling, and dense layers, was trained on reshaped text data for sequential analysis.  
\begin{table}
\begin{tabular}{c c c} \hline
**Classifier** & **Configuration/Parameters** & **Notes** \\ \hline \multirow{6}{*}{1D-CNN} & 1 convolutional layer & \multirow{2}{*}{Convolutional layers capture local patterns,} \\  & 128 filters, kernel size = 3 & \\ \cline{1-1}  & Max-pooling layer (size = 2) & \\ \cline{1-1}  & Flattened output into dense layer with softmax & max-pooling reduces dimensionality, and the dense layer with softmax outputs class probabilities. \\ \cline{1-1}  & 350 epochs, batch size = 32 & \\ \hline \multirow{6}{*}{DBMM} & 3 classifiers: 1D-CNN, MLP, RF & \multirow{2}{*}{Initial weights are based on performance on the training set and are dynamically updated during test-set classification. A grid search optimization is employed to find the optimal number of time slices (past predictions) to incorporate as prior information.} \\ \cline{1-1}  & 2 time slices (previous priors) & \\ \cline{1-1}  & Entropy-based weights & \\ \cline{1-1}  & Bayesian update of weights & (past predictions) to incorporate as prior information. \\ \hline \end{tabular}
\end{table}
Table 2: _Cont._  
\begin{table}
\begin{tabular}{c c c} \hline
**Classifier** & **Configuration/Parameters** & **Notes** \\ \hline \multirow{6}{*}{BERT} & Fine-tuned for sequence classification & \multirow{2}{*}{Transformer-based model for natural language processing.} \\  & 3 labels & \\ \cline{1-1}  & AdamW optimizer & \\ \cline{1-1}  & Cross-entropy loss & \\ \cline{1-1}  & 350 epochs, batch size = 32 & \\ \cline{1-1}  & Linear learning rate scheduler with warmup & \\ \hline \end{tabular}
\end{table}
Table 3: Classifier parameters for SA.