# Enhancement of Text Analysis Using Context-Aware Normalization of Social Media Informal Text
## 2 Background and Related Work
However, the scope of problems introduced by user-generated content in online SM platforms exceeds the range of simple spelling correction. Other problems include rapidly changing OOV slangs, phonetic spelling, punctuation errors and omissions, misspelling for verbal effect and other intentional misspelling, short-forms and acronyms, and recognition of OOV named entities [53]. These algorithms perform poorly, as they miss user behavior of intentionally generating OOV words on SM. In the last decade, researchers have actively worked for improvement in SM text normalization using different techniques. [PERSON] and [PERSON] [54] used linear SVM and morphophonic similarity for OOV identification and standardization. [PERSON] et al. [55] proposed a model for SM text normalization based on Conditional Random Field (CRF) and Recurrent Neural Network (RNN) for learning sequence of editing operations and neural text embedding, respectively. [PERSON] et al. [56; 57] investigated the human perspective of word normalization, including enhanced letter transformation, visual priming, and string/phone similarity. [PERSON] et al. [58] developed a taxonomy for OOV words based on four distinct types of texts to propose a more generalized approach for text normalization which can handle new texts. However, this approach is still data/domain-dependent. A fully automated and statistical approach based on n-grams was developed in [59] for text normalization. A rule-based adaptive approach was proposed in [60; 61] for SM text normalization by selecting the normalization method based on the type of error in the OOV words. [PERSON] et al. [62] proposed a text normalization method based on phonetic and string similarity. A Hidden Markov Model (HMM) approach was adopted in [63], to model all correction possibilities for each word in the corpus, weighted with their occurrence probabilities for ranking based on the nature and type of the SMS text. In [64; 65] a probabilistic approach was proposed based on the Trie data structure for text normalization. This approach was reported to perform better than the HMM-based approach [64]. Normalization is applied to SM text applications as a pre-processing step in many works to improve the accuracy of the text analysis applications. [PERSON] [66] applied rule-based text normalization to the Twitter data for accurate sentiment classification. [PERSON] et al. [26] proposed a normalization model for code-mixed multilingual text to investigate its impact on sentiment analysis of Hindi-English mixed tweets. [PERSON] and [PERSON] [67] applied the character-level embedding method with deep CNN to normalize SM text for sentiment analysis of Twitter data. [PERSON] et al. [68] used a Genetic algorithm trained Bayesian network to normalize noisy features in SM text for enhanced spam filtering. Other more recent studies on text normalization for enhanced sentiment analysis in different languages are presented in [69; 70; 71]. These normalization approaches can overcome the information loss and domain dependency problem; however, these approaches are not generalized. They mostly ignore the context of the intended words by substituting the OOV words with the nearest replacements. Moreover, most of the existing techniques normalize isolated word problems. However, in informal conversation, dialect usage and word deletion are widespread, which can only be handled by sentence-level normalization.