# Enhancement of Text Analysis Using Context-Aware Normalization of Social Media Informal Text
## 4 Results and Analysis
### Simulation Results
Moreover, we have compared the performance of the proposed TVH with the deep-learning (RNN)-based normalization method. The performance of the proposed scheme is better due to the standardizing of the textual variations instead of learning these variations from data because there is wide range of possible textual variations in SM text, and it is not possible for the deep-learning methods to learn all the possible variations. The proposed TVH is better due to its hybrid nature, and it can handle different types of textual variations simultaneously. \begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline
**Algorithm** & \multicolumn{3}{c}{**LSTM**} & \multicolumn{3}{c}{**BERT**} \\ \hline
**Scenario** & **Precision** & **Recall** & **F-Score** & **Accuracy** & **Precision** & **Recall** & **F-Score** & **Accuracy** \\ \hline Clean training and test & 84.01 & 95.58 & 89.94 & 93.65 & 97.46 & 98.23 & 97.84 & 99.04 \\ Noisy training and test & 76.11 & 87.70 & 81.49 & 88.50 & 89.04 & 89.99 & 89.51 & 95.33 \\ Clean training and Noisy test & 70.36 & 79.76 & 74.77 & 84.55 & 82.23 & 83.79 & 83.00 & 92.34 \\ Clean training and TVH test & 86.39 & 93.46 & 89.79 & 95.55 & 94.59 & 96.39 & 95.48 & 97.96 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Simulation of different simulation scenarios of SN text analysis. \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Algorithm** & **Precision** & **Recall** & **F-Score** & **Accuracy** \\ \hline Textblob & 50.81 & 51.73 & 51.27 & 53.65 \\ [PERSON] & 69.59 & 68.70 & 63.83 & 69.98 \\ [PERSON] & 62.83 & 73.06 & 67.56 & 75.57 \\ Jamspell & 67.17 & 77.21 & 71.84 & 81.44 \\ \hline Manual Lexicon & 58.80 & 67.58 & 62.89 & 68.02 \\ Spell-checker & 60.39 & 69.74 & 64.73 & 71.77 \\ Probabilistic & 67.32 & 80.70 & 73.41 & 79.64 \\ Deep-Learning-based & 74.99 & 83.65 & 79.09 & 88.02 \\ Proposed Spell-Checker & 63.74 & 75.58 & 69.16 & 75.36 \\ Proposed hybrid method (TVH) & 81.14 & 88.06 & 84.46 & 91.74 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Comparison of the proposed TVH with other SM text normalization techniques. We observed that the performance of the overall pipeline is dependent on the performance of the proposed TVH, as it handles the noise by automatically correcting it before it is put into the text classifier. If the performance of the TVH is poor, it will replace the OOV words with the wrong suggestion, which may result in wrong classification. The existing schemes performed well in normalization of word-to-word substitution, without giving much regard to the context. One of the main advantages of the proposed method is that it normalizes text with respect to the context of the words/phrases. The proposed scheme also outperformed the existing methods when it comes to word-to-phrase- and sentence-level normalization. The performance of the text analysis is directly related to the fraction of noise in the data. As shown in Table 6, the lower the variation in text, the higher the performance of the text analysis. We also observed from the simulation that when the fraction of variations in data is low, the improvement in performance with the proposed text analysis pipeline is also low as there exists less margin of textual variation correction and hence less improvement in performance. Like the other methods, the proposed TVH is also not an ideal normalization method. From simulations, we observed that the proposed TVH can also sometimes degrade the performance of the text analysis pipeline in the cases of text with little variation. The performance of the BERT in the case of spam (low variations) classification was degraded when the test data were normalized by TVH due to the possibility of introducing more textual variations by wrong substitutions. We performed extensive simulation of the proposed text analysis pipeline on text in various text analysis applications. These applications are mentioned in the previous Section 4.1, along with the descriptions of the datasets used for simulation of each application.