## 3 Proposed Methodology
### Proposed Architecture Using BERT and Neural Networks
#### 3.5.1 AlBERT + CNN  
We have implemented a CNN-based classifier model for OLD according to Algorithm 5. In this ALBERT + CNN approach, we have incorporated a pre-trained text preprocessing layer, which performs essential steps such as tokenization and other necessary preprocessing specific to the chosen pre-trained model. Next, we passed the preprocessed text inputs through a pre-trained encoder layer, named ALBERT, which generates contextualized word embeddings or representations. These embeddings capture rich semantic information that aid the model to understand the meaning and context of the text. To extract local patterns and features, two 1-dimensional convolutional layers were applied to the encoded sequence. These convolutional layers used different filter sizes and apply the ReLU activation function, thus enabling them to capture relevant patterns within the text.  
```
0: Preprocessed data
1. Load the pre-trained ALBERT model
2. Create 1D CNN Model (a) Add ConV1D (activation = relu) (b) Add ConV1D (activation = relu) (c) Add Global Max Pooling layer (d) Add dense layer (activation function = ReLU) (e) Add dropout layer (rate = 0.1) (f) Add dense layer (activation function = softmax)
3. Loop for \(n\) epochs: (a) ALBERT Encoder  i. Tokenize data into sequence of subwords tokens using ALBERT tokenizer  ii. Add special Tokens (e.g., [CLS],[SEP]) (b) Encoded data  i. Pass the tokenized sequence through ALBERT to obtain contextualized word embeddings.  ii. Extract the final hidden state representation for each sequence from ALBERT's output. (c) Train the model  i. compile the model with categorical cross-entropy loss function and Adam optimizer.  ii. fitted the model by feeding the ALBERT embeddings as input (d) Evaluate the model  i. Assessed the model performance using metrics like precision, recall and F1 score.
```  
**Output:** Classification results (hate, offensive, neither hate nor offensive)  
**Algorithm 5** Hate Speech Classification Using ALBERT Encoder and 1d CNN.  
Following the convolutional layers, a global max pooling layer is applied to aggregate the extracted features, reducing the sequence length to a fixed-size representation. This pooling operation is designed to prioritize the most significant features throughout the entire input sequence, enhancing the model's attention to key information. The pooled features are then flattened into a one-dimensional vector for the fully connected layers. These layers further process the features, utilizing a dense layer with 512 units and the ReLU activation function. In order to mitigate overfitting, a dropout layer is incorporated with a dropout rate set at 0.1. Finally, we have used an output layer that consists of a dense layer with softmax activation, producing class probabilities for target classes. In our case, the model is designed to classify inputs into one of the three classes. The combined architecture leverages the power of pre-trained language encoders and convolutional operations to extract relevant features from the input text. This approach enabled the modelto capture both local patterns through convolutions and global contextual information through transformers, resulting in improved performance for hate speech and offensive language detection.