# An Algorithm for the Detection of Hidden Propaganda in Mixed-Code Text over the Internet +
## 4 Propaganda Detection in Mixed-Code Text
### A Mixed Code Text Normalisation Algorithm
end if j++;  end while k=1; Word Selection step: while k<=S.size()do z=1; while z<=SM(k).CombinationNumber()do //as long as there is a combination for each word  c\({}_{z}\)=SM(k).getCombination(z);  c\({}_{z}\).replaceGroupsWithTheTransformedCharacter(SM,TC); ifc\({}_{z}\).string().isAnExistingEnglishWord()then // if the generated word is not part of the English vocabulary, it is not considered  SM(k).removeCombination(c\({}_{z}\)) else // otherwise it become a candidate placeholder replacement  cw.addCandidateWord(k,c\({}_{z}\))  end if z++;  end while k++;  end while k=1; Sentence Reconstruction step: while k<=S.size()do S\({}_{Norm}\).replacePlaceholder(k,cw(k).candidateWord());  k++;  end while Result: S\({}_{Norm}\);
```  
**Algorithm 1**Pseudocode for text normalisation  
```
Data: String S={W\({}_{1}\), W\({}_{2}\),..., W\({}_{k-1}\), W\({}_{k}\) };  Placeholder={P\({}_{1}\), P\({}_{2}\),..., P\({}_{k-1}\), P\({}_{k}\) };  CandidateWords cw[];  Integer i=1; j=1; z=1;  Word w; CharactersGroup cg; String S\({}_{Norm}\);  SegregationMatrix SM [K][N][M];  TransformedCharacter TC [M]; Text Segregation step: while i<= S.size()do // - as long as there are words  w=S.getWord(i); // take the i-th word ifw.isNotMixedCode()then // if the i-th word is a regular word \(S_{Norm}\).append\({}_{i}\)(w); // it does not have to be normalised else  S\({}_{Norm}\).append\({}_{i}\)(P\({}_{i}\)); // create a place holder in the i-th position  SM\({}_{k=i}\).segregate(w); // segregate the i-th word  end if i++;  end while Character Transformation step: while j<= SM.GroupSize()do // for each segregated word  cg=SM.getGroup(j); // select the j-th group of characters ifcg.isAnAlphabeticCharacter()then // if it is one of the standard alphabetic letters  TC[j]=cg; // apply the rule-1 by considering it as it is  else  TC[j]=convertIntoSingleCharacter(cg); // apply the rule-2 by using OCR to transform the j-th cg group of symbols into one alphabetic character. end if j++;  end while k=1;  end while k<=S.size()do // assign long as there is a combination for each word  c\({}_{z}\)=SM(k).getCombination(z);  c\({}_{z}\).replaceGroupsWithTheTransformedCharacter(SM,TC); ifc\({}_{z}\).string().isAnExistingEnglishWord()then // if the generated word is not part of the English vocabulary, it is not considered  SM(k).removeCombination(c\({}_{z}\)) else // otherwise it become a candidate placeholder replacement  cw.addCandidateWord(k,c\({}_{z}\))  end if z++;  end while k++;  end while k=1; Sentence Reconstruction step: while k<=S.size()do S\({}_{Norm}\).replacePlaceholder(k,cw(k).candidateWord());  k++;  end while Result: S\({}_{Norm}\);
```  
**Algorithm 2**Pseudocode for text normalisation  
_Character Transformation step_: in this step, for each combination generated in the previous step the following transformation rules have been defined:  
* _rule-1_: if a set of grouped characters coincides exactly with one standard alphabetic character (e.g., a, b, c,..., Z), then it is considered like it is. For example, in all generated combinations when F and T are not grouped with any other symbol, they are interpreted as F and T.
* _rule-2_: if a set of characters consists of one not standard alphabetic character, including numbers (such as!,?,?, 0, 1, 2 and so on), or it consists of a combination of one or more characters (e.g., F!,!,!,!, and so on) then an \"optical character recognition--(OCR)\" technique is applied. The idea is to emulate the human behaviour, from an optical point of view, so as to automatically recognise individual characters and letters. In particular, first each group of characters/symbols is transformed into images. Then machine learning techniques are exploited to recognise and associate each group of characters/symbols to a standard letter of the alphabet by trying to identify a match within the letters of the standard (English) alphabet. As it is shown in Figure 3, for each group of symbols different letters of the alphabet can be generated; however, the one with the highest similarity score is then chosen.