# A Deep Learning Approach for Sentiment Analysis of COVID-19 Reviews
## 5 Results and Discussion  
This study considers three classification techniques: SVM, random forest, and the proposed approach using the attention layer. The parameters of the classifiers are tuned using grid search. Table 1 shows the parameter tuning outcomes of SVM-RBF, which has two central hyperparameters: C and gamma. These parameters balance generalisation and overfitting. The gamma value helps in reducing the curvature; otherwise, the data accumulation will increase, which will reduce the polynomiality of the learning process. To reduce this overfitting and data mugging, less curvature is balanced by a lower C value. Table 1 shows that low gamma and high C generated better performance compared to others. The SVM hyperparameter tuning shows that at C = 10 and gamma = 0.01, the accuracy is 62.13%, precision is 63.12%, recall is 62%, and F-score is 63%, which is the best model for this classifier for the considered parameters. Table 2 shows hyperparameter tuning of random forest. The hyperparameters considered are max depth, estimators, and split. These parameters denote the depth of trees, the maximum number of trees generated during the model development, and the minimum number of data points in nodes before a split occurs. As noted in the table, when the minimum split is 5, the maximum depth is 10, and the number of estimators is 600, the model's accuracy is 60%, precision is 61%, recall is 62%, and F-score is 60%. This is the best model for the variation of parameters considered. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**C** & **Gamma** & **Accuracy** & **Precision** & **Recall** & **F-Score** \\ \hline
0.1 & 1 & 55.12 & 56.23 & 50.12 & 60.12 \\ \hline
1 & 0.1 & 58.12 & 60.13 & 61.23 & 57.23 \\ \hline
10 & 0.01 & 62.13 & 63.12 & 62 & 63 \\ \hline
100 & 0.001 & 60 & 61.23 & 56 & 58 \\ \hline
500 & 0.0001 & 61.23 & 56.12 & 57.12 & 56.23 \\ \hline
1000 & 0.0001 & 60 & 53.22 & 50.12 & 50 \\ \hline
1 & 0.5 & 45.12 & 45.45 & 50 & 53.12 \\ \hline
10 & 0.25 & 34.23 & 56.12 & 52.12 & 54.12 \\ \hline
100 & 0.125 & 52.12 & 53.12 & 52.12 & 55.12 \\ \hline
500 & 0.0625 & 56.23 & 57.23 & 54.23 & 52.12 \\ \hline
1000 & 0.0325 & 57.12 & 54.12 & 51.23 & 54.12 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: SVM hyperparameter tuning. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Max Depth** & **Estimators** & **min Split** & **Accuracy** & **Precision** & **Recall** & **F-Score** \\ \hline
10 & 200 & 2 & 45.23 & 50.12 & 54.23 & 56.12 \\ \hline
20 & 300 & 5 & 48.12 & 49.23 & 50.12 & 58.12 \\ \hline
30 & 400 & 10 & 60 & 56.22 & 50.12 & 53.23 \\ \hline
40 & 500 & 2 &  
Table 3 similarly shows hyperparameter tuning for attention layers with Leaky ReLU as the activation function and with different numbers of attention layers. In these experiments, the number of CNN layers was fixed because CNN layers increase resource usage exponentially. The number of attention layers varies from 1 to 8. As noted in the table, the model's performance increases when the number of attention layers increases from 1 to 4, after which the performance degrades. For an attention layer number of 4, the model's accuracy is 86.12%, precision is 84.23%, recall is 85.23%, and F-score is 85.12%. This performance is better than the other observed results. Table 3 shows that the best deep learning approach is based on the network with four attention layers, which provides the best accuracy, precision, recall, and F-score. Table 4 shows the activation function results, which clearly show that four attention layers with the activation function Leaky ReLU provided higher accuracy, precision, recall, and F-score in comparison to other activation functions. It can be seen that for the Leaky ReLU activation function, the accuracy is 85.12%, precision is 82.12%, recall is 84.13%, and F-score is 84.12%, which is considerably higher than the accuracy shown by ReLU, which is 84.56% with a precision of 82.34%, recall of 82.12%, and F-score of 81.23%.