# Enhancement of Text Analysis Using Context-Aware Normalization of Social Media Informal Text
## 3 Proposed Sentiment Analysis Method  
In this paper, we proposed a new strategy for text analysis of informal SM textual data to use the useful information in the noisy SM text. We proposed a new method to handle textual variations in SM text by context-aware substitution of OOV words with its valid replacements, named TVH. The proposed TVH can handle a wide range of textual variations which are very common in SM text. We integrate text classification methods with the proposed TVH to enhance its performance of in the case of noisy SM text. Figure 1 shows the block diagram of the methodology of our proposed text analysis scheme. In text analysis applications based on machine/deep-learning techniques the ideal case is to have clean data for accurate classification. However, the social network data contains a vast range of variations which distort the performance of the text classification. It is inevitable to either clean/normalize the SM text or embed the intentionally generated OOV words in the lexicon for accurate classification. Both approaches have limitations: the former approach is time- and resource-consuming due to huge volume of training data required for model training whereas the latter method falls short in coverage of different intentionally generated variations due to users' different behaviors of text generation. In this work, we adopted the approach to train the text analysis model with clean SM text and normalize the input SM text before feeding it to the classifier as shown in Figure 1.  
Algorithm 1 shows the stepwise pseudocode of the proposed text analysis model. Load clean and labeled SM text as training data, and feed noisy SM text as test/input to  
Figure 1: Block diagram of the proposed text analysis scheme for informal social networks text. the system. Pass the training data to from the Preprocess(TD) function. The Preprocess(TD) function clean the text from stopwords, punctuations, URLs, tags and mentions. Define a deep-learning network architecture for text analysis by setting its parameters. The actual values and options which we used in this work are discussed in Section 3.2. Train the text analysis model using the preprocessed clean labeled SM text. We used different training datasets for different applications such as sentiment analysis, cyberbullying identification, spam detection and reviews analysis. Preprocess the test/input data using the function Preprocess(TsD). Pass the preprocessed input data from the proposed TVH(PtsD) function. The TVH(PtsD) function normalizes the SM text by substituting the OOV words with their context-appropriate replacements as shown in Figure 2 and Algorithm 2. ```
Output: Text_Category Input: textDataTrain; Load clean and labeled SM text as training data (70%); textDataValidate; clean SM text (15%); textDataTest; Noisy SM text (15%);
1 Preprocess the training text data (\(TD\)) with function Preprocess(TD);
2 Create a deep-learning network architecture by setting network parameters as following;
3 layers = Settings from Table 1 or Table 2;
4 options = Options from Table 1 or Table 2;
5 net = trainNetwork(XTrain, YTrain, layers, options);
6 whileD do
7 Preprocess the test data (TsD) with function \(PT\)sD = Preprocess(TsD);
8 ifErrorDetect(PTsD, dict)then
9 Normalize the text using Normalized_text = TVH(PTsD) as shown in Algorithm 2;
10
11 else
12 Normalized_text = PTsD
13
14 end if
15 Text_Category = classify(net, Normalized_text);
16
17 end while
```  
**Algorithm 1**Step-wise approach for text analysis of informal SM text  
Algorithm 2 shows the pseudocode for the proposed TVH. The function TVH(PTsD) handles multiple types of most commonly occurring (un)intentional textual variations in SM, which results in OOV words. These variations include word enlargement, abbreviations, word shortening, spelling errors, phonetic substitution, dialectal usage, and word deletion. We used rule-based approaches, discussed in Section 3.1.2, to handle these textual variations, which gives a list of valid words which are the closest to the OOV words.