# A Review on Deep-Learning-Based Cyberbullying Detection
## 5 Deep-Learning-Based Models
### Attention-Based Model
#### 5.10.6 Gated Recurrent Units with Attention (GRU-Att)
\begin{table}
\begin{tabular}{p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**DL Models** & **Used in Cyberbullying Applications** & **Area of Applications** & **Limitations** \\ \hline Deep Neural Network (DNN) [92] & Chats and Tweets [14], Social networks’ text and image [128] & Speech Recognition and the natural language processing & Requires large amount of data, expensive to train, and issues of overfitting \\ \hline Boltzmann Machines (BMs) [39] & Offline content [129], Image content [130], Arabic content [74] of explored data & Emotion recognition from thermal images, estimation of music similarity, extracting the structure of explored data & Training is challenging, and weight adjustment is hard \\ \hline Deep Belief Networks (DBN) [40] & Arabic content [74], Social media text [131], Social media image [132] & Image classification, natural language understanding, speech recognition to audio classification & Expensive to train because of the complex data models, huge data is required, and needs classifiers to grasp the output \\ \hline Deep Autoencoder (DAE) [41] & Chats and Tweets [14], Social media content [73] & Image search and data compression, dimensionality reduction, image denoising & The bottleneck layer is too narrow, lossy, and requires large amount of data \\ \hline Generative Adversarial Networks (GAN) [42] & Web-application for detecting cyberbullying [133] & Improve astronomical images, gravitational lens simulation for dark matter exploration, excellent low resolution, generate realistic images and cartoon characters & Non-convergence, mode collapse, and diminished gradient \\ \hline Recurrent Neural Networks (RNN) [44] & Social Commentary [21], Cyberbert: Bert for cyberbullying identification [22], Identification and classification from social media [134] & Image captioning, time-series analysis, natural language processing, handwriting recognition,and machine translation & The gradation disappears and the problem explodes, difficult to train, and unable to handle very long sequences when tanh or ReLU is used as the activation function \\ \hline Long Short-Term Memory (LSTM) [109] & Social media content [7, 21, 68], Wikipedia, Twitter, Formspring and YouTube [25], CyberBERT [22], Bangla text [18], Indonesian language [64], Twitter [2, 63] & Time-series prediction, speech recognition, music composition, and pharmaceutical development & Training takes time, training requires more memory, easy to overfit, and Dropouts are much more difficult to implement in LSTMs \\ \hline Bidirectional LSTM (Bi-LSTM) & Social media content [6, 7, 21, 68, 134], Visual contents [6], Wikipedia, Twitter, Forsmpring and YouTube [25], CyberBERT [22], Bangla text [18], Indonesian language [64], Text and emoji data [135], Facebook [136], Twitter[2, 136] & Text classification, speech recognition, and forecasting & Costly as double LSTM cells are used, takes longer to train, and easy to overfit \\ \hline Convolutional Neural Networks (CNN) [43] & Social media content [5, 6, 7, 21, 68, 134], Visual contents [6], Twitter [2, 14, 25, 63, 67, 136, 137], Formspring,me [25, 137], Facebook [136], Chats [14], YouTube and Wikipedia [25] & Image processing, and object detection & Significantly slower due to an operation such as maxpooling, large datasets are required to process, and train neural networks [138] \\ \hline Radial Basis Function Networks (RBFNs) [139] & Youtube content [140], Formspring,me, MySpace, and YouTube content [141] & Classification, regression and time-series prediction & Classification is slow because every node in the hidden layer needs to compute the RBF function \\ \hline Multilayer Perceptrons (MLPs) & Text and emoji data [135] & Speech recognition, image-recognition, and machine translation & As it is fully connected, there are too many parameters, each node is connected to another node in a very dense network, which creates redundancy and inefficiency \\ \hline Self-Organizing Maps (SOMs) [142] & Social media content [143] & Data visualization for high dimensional data & Requires sufficient neuron weight to cluster inputs [144] \\ \hline \hline \end{tabular}
\end{table}