# Social Media Hate Speech Detection Using Explainable Artificial Intelligence (XAI)
## 3 Results
### Model Training and Evaluation for HateXplain Dataset
#### 3.2.2 BERT + ANN  
Next, BERT was used with ANN to train the model and evaluate the performance. Table 7 depicts the model summary for the BERT + ANN model, where the first column indicates the type of the layer, the second and third columns indicate the output shape and the number of parameters generated by processing of the layer, respectively, and the last column represents the previous layer it is connected to.  
As shown in Figure 6, the architecture of the BERT + ANN model was fine-tuned in order to achieve the most efficient performance. The model contained one input and preprocessing layer, along with the BERT encoder, which was a keras layer. BERT was combined with convolution layers, followed by a 1D global max-pooling layer, which computed the maximum of all the input sizes for each of the input channels. A dense layer was introduced after the 1D global max-pooling layer to reduce the parameters and increase the number of features being propagated to the next layer. In the end, a dropout later was added to avoid overfitting, followed by a dense layer. The model defined above was then compiled with the loss function as sparse categorical cross-entropy and the Adam optimizer.  
The BERT + ANN and BERT + MLP models were trained for 50 epochs. As the number of epochs increased, the accuracy improved. The parameters used to find the number of training steps and number of warmup steps were as follows: number of epochs = 50, number of training steps = steps per epoch \(\times\) number of epochs, and number of warmup steps = 0.1 \(\times\) number of training steps.  
The accuracy obtained by the BERT + MLP model and BERT + ANN model was 93.67% and 93.55%, respectively, indicating that the gap in conventional evaluation metrics was minimal; however, in terms of the explainability metrics, BERT + ANN performed slightly better than the BERT + MLP model, as discussed later in this section.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Layer (Type)** & **Output Shape** & **\# of Parameters** & **Connected to** \\ \hline text (InputLayer) & [[None,)] & 0 & [] \\ \hline preprocessing (KerasLayer) & \{‘input\_word\_ids’(None,128), ‘input\_mask’: (None, 128), ‘input\_type\_ids’: (None, 128)] & 0 & [’text’ \\ \hline \multirow{4}{*}{BERT\_encoder (KerasLayer)} & \{‘pooled\_output’: (28,763,649, None, 512), sequence\_outputs’: (None, 128, 512), 'encoder\_outputs’: [(None, 128, 512), (None, 128, 512), (None, 128, 512)], 'default’: (None, 512), ‘sequence\_output’: (None, 128, 512)] & 28,763,649 \\ \hline Conv1d (Conv1D) & (None, 127, 32) & 32,800 & [’BERT\_encoder [0][6]’] \\ \hline Conv1d\_1 (Conv1D) & (None, 126, 64) & 4160 & [’conv1d [0][0]’] \\ \hline Global\_max\_pooling1d (GlobalMaxpooling1D) & (None, 64) & 0 & [’conv1d\_1 [0][0]’] \\ \hline dense\_1 (Dense) & (None, 512) & 33,280 & [’BERT\_encoder [0][5]’] \\ \hline dropout\_1 (Dropout) & (None, 512) & 0 & [’dense\_1 [0][0]’] \\ \hline classifier (Dense) & (None, 3) & 1539 & [’dropout\_1 [0][0]’] \\ \hline \multicolumn{4}{c}{Total params: 28,835,428} \\ \hline \multicolumn{4}{c}{Trainable params: 28,835,427} \\ \hline \multicolumn{4}{c}{Non-trainable params: 1} \\ \hline \hline \end{tabular}
\end{table}
Table 7: BERT + ANN model summary.