# Enhancement of Text Analysis Using Context-Aware Normalization of Social Media Informal Text
## 2 Background and Related Work
The General Inquirer has a manually labeled resource of about 3600 words [30], developed for content analysis in behavioral sciences. [PERSON] et al. [23] used a list of about 6800 manually labeled terms for opinion mining from customer reviews. [PERSON] et al. [31] developed an MPQA lexicon of about 8000 words with context-aware sentiment labels. [PERSON] et al. built the NRC Word-Emotion Association Lexicon of about 14,000 words with sentiment and emotion labels [32; 33]. The labels were compiled manually by crowdsourcing through Mechanical Turk. [PERSON] et al. [34] created a manual lexicon named \"full form dictionary\" to replace the OOV words by their valid substitutions. It is humanly impossible to manually create a general-purpose sentiment lexicon of valid and invalid (noisy) words for SM text. The method of manual lexicon creation is highly time and resource-consuming, and it is not possible to cover all variants of the enormous word vocabulary. Manual lexicon creation is costly, semi-supervised, and automatic methods are presented to create language resources for sentiment analysis. [PERSON] et al. [35] used noun-phrase co-occurrence statistics to semi-automatically generate a semantic lexicon. [PERSON] et al. [36] used the divergence of term frequencies in a set of opinionated and relevant documents to select the subjective-term candidates for automatic generation of the term-opinion lexicon. [PERSON] et al. [25] described the process for automatic tweet-specific lexicon creation. They provided a set of positive (30) and negative (47) hashtags to crawl keyword-specific tweets using Twitter API and marked the tweets as positive and negative based on the occurrence of the hashtagged seed words. [PERSON] et al. [37]provided a semi-automatic multilingual framework for domain-based sentiment word extraction. They used lexicon-based, corpus-based, and human-based word tagging approaches. [PERSON] et al. [38] presented an effective method for automatic construction of a depression-domain lexicon. They used Word2 Vec, a semantic relationship graph, and the label propagation algorithms. [PERSON] [39] presented a robust frame-based approach to create a lexicon for domain-specific text analysis automatically. The proposed model was proven better in handling OOV targets. Other useful studies about the feasibility of using automatic lexicon creation and methods for automatic lexicon construction are presented in [40; 41; 42; 43]. The automatic or data-driven lexicon creation method can be used to handle noise in SM text by developing a lexicon of the noisy words; however, it is strictly domain-specific and application-dependent. Generally, text normalization is used to avoid the limitations of information loss and application dependencies in text analysis applications. Normalization is translating noisy SM text into its canonical (standard) form before text analysis for better results. Text normalization is proved to be an important pre-processing step for noise-handling in natural language processing (NLP) applications, such as sentiment analysis [26], machine translation [44], text accessibility [45], spam filtering [46], and text to speech synthesis [47] etc. The baseline algorithm for text normalization is spelling-error correction, where the OOV words are considered to be misspelled words. Spelling correction is a method to correct spelling errors due to deliberate abbreviating or typing mistakes. The spelling correction technique was used by [11; 12] in their work for converting (normalizing) noisy words into meaningful replacements. The techniques that are used for spelling correction are edit distance [48; 49], similarity key [48; 49], rule-based [48; 49], N-gram [49], probabilistic [49], neural network [50], and auto-encoders [51]. Spelling correction is a well-established domain, with advances in pattern-matching techniques and the development of n-gram analysis techniques that have improved over the last two decades [52].