# Enhancement of Text Analysis Using Context-Aware Normalization of Social Media Informal Text
###### Abstract  
We proposed an application and data variations-independent, generic social media Textual Variations Handler (TVH) to deal with a wide range of noise in textual data generated in various social media (SM) applications for enhanced text analysis. The aim is to build an effective hybrid normalization technique that ensures the use of useful information of the noisy text in its intended form instead of filtering them out to analyze SM text better. The proposed TVH performs context-aware text normalization based on intended meaning to avoid the wrong word substitution. We integrate the TVH with state-of-the-art (SOTA) deep-learning-based text analysis methods to enhance their performance for noisy SM text data. The proposed scheme shows promising improvement in the text analysis of informal SM text in terms of precision, recall, accuracy, and F1-score in simulation. social media; noisy text; informal text; LSTM; BERT; text normalization; text analysis Article Enhancement of Text Analysis Using Normalization of Social Media Informal Text  
[PERSON]; [PERSON]. representation. Due to the massive growth of informal texts on SM, the problem of mapping them to their correct representation has received its due attention from the research community. The spell-error correction algorithm is one of the most basic and straightforward text normalization methods, where the OOV words are considered misspelled words. These algorithms normalize the noisy misspelled OOV words to their standard lexical counterpart using spell-checking algorithms to enhance the performance of the traditional text analysis methods [11; 12]. Text normalization is not a \"one size fits all\" task of substituting OOV words with its valid replacement [13]. Besides the correction of the misspelled words, a normalization algorithm needs to handle a wide range of OOV by sensing the error patterns, identifying the error types, and activating the appropriate correction methods. Some common variations in SM text includes word enlargement (great \(\rightarrow\) greaaaaaaaaaaaaaaaattt), user-defined abbreviating (Good Morning \(\rightarrow\) GM), word shortening (Good Night \(\rightarrow\) gd nyt), phonetic substitution (wait \(\rightarrow\) w8), dialectal/informal word usage (are not \(\rightarrow\) aint), words deletion (Where are you? \(\rightarrow\) where? ), punctuation omission (don't \(\rightarrow\) dont), and censor avoidance (fuck \(\rightarrow\) f***) [14]. These textual variations occur due to users' behavior of time and space-saving or emotional expression during the informal communication on the SN. The spell-error correction techniques alone cannot handle such textual variation, and they may insert the wrong substitution due to its high diversity and personalized text generation behavior. In the social media text these textual variations can occur simultaneously i.e., different types of OOV words can occur in the same text. Therefore, there is a need to provide an ensemble technique to handle a wide range of textual variations simultaneously. Many works have suggested different techniques to handle different types of textual variations; however, these techniques are not applied simultaneously to the same text. The aim of this paper is to provide a TVH for translating the OOV words into their actual intended words to preserve the valuable sentiment information present in these OOV words. The proposed approach is an ensemble of the previous text-normalization techniques to enhance the performance of text classification for informal communication. The proposed scheme is designed to rank the suggested candidates for the OOV words based on context, to select the best possible substitution. We applied the proposed TVH as a pre-processing to the SM text before text analysis and observe the effects on the performance of the text analysis algorithms. The main contribution of this work is as follows:  
The remaining sections of the paper are organized as follows. In Section 2, the related work is discussed.