# MaLang: A Decentralized Deep Learning Approach for Detecting Abusive Textual Content
## 3 Materials and Methods
### Architecture of the Cloud-Level Module
#### 3.5.2 Deep Learning Model for Identification of Toxicity of Abusive Content  
The Jigsaw/Google Toxic Comment Dataset [61] is used to classify the toxicity in the text message content uploaded to KIPP into four classes: 'Toxic', 'Threat', 'Insult', and 'Hate Speech'. The dataset contains labelled examples of Wikipedia comments that were manually rated by human raters based on the comment's toxicity in six classes, which are 'Toxic', 'Severely Toxic', 'Threat', 'Obscene', 'Insult', and 'Hate-speech'. Although the dataset consisted of six classes, classes such as 'Toxic' and'severely toxic' were merged since severely toxic is a subset of Toxic text, and classifying both as a single set seemed conceptually convincing. Similarly, the class 'obscene' was merged with 'Insult'.  
The dataset consists of more than 157,000 sentences. To separate the dataset, a 60-20-20 split was considered for training-validation-testing, respectively. All the samples or sentences were cleaned by removing non-alphabetic characters and html content. The cleaned samples are tokenized using the Keras tokenizer, obtaining an array of tokens of unique words for each sentence. Subsequently, every token is cropped to uniform length in each array, obtaining an array of uniform tokens for each sample called entities.  
The obtained entities are converted into input vectors of 32 indices. This is done by mapping tokens from entities to their corresponding vector representations present in pre-trained GloVe vectors [63]. Word embedding for each token is performed using the 300-dimension pre-trained global vector representation of word-word occurrences,  
Figure 7: Architecture of CASE using Bi-LSTM and the Max-Pooling Layer.  
producing 840 billion tokens from the Massive Common Crawl Corpus with a vocabulary size of 2.2 million.  
Some abusive words may not be present in MaLang's (or KIPP's) recognized vocabulary. When any missing word is encountered, it is given a value of zero, or if any token belongs to the substring of the main word in the vocabulary, the token is mapped to the root or main string vector. Mapping the token from entities to pre-trained GloVe vectors results in input vectors that are passed through the embedding layer of the KIPP deep learning model.  
Figure 8 shows the architecture of the deep learning model KIPP. The cloud-level model consists of an embedding layer followed by a bidirectional LSTM layer and a bidirectional GRU layer with 128 cells in each layer. These layers are connected to a combination of a one-dimensional Average Pooling Layer and a Max Pooling Layer. The output layer consists of a sigmoid function to represent the output in the form of probabilities to detect the toxicity of the content in four classes. The use of Bidirectional GRUs and Bidirectional LSTM is solely for the purpose of handling single labelled entities sent to KIPP from mobile devices. KIPP receives a very large amount of unstructured data, which GRUs are known to handle and produce better results without monopolizing computational resources.  
KIPP predicts the level of Toxicity in each entity and returns the probability of presence of the kind of toxic content present as shown in Figure 4. After all the entities are analyzed, a report is generated stating the average toxicity score of the user's content received at the cloud level of MaLang every 3 h.