# A Review on Deep-Learning-Based Cyberbullying Detection
## 5 Deep-Learning-Based Models
### Attention-Based Model
#### 5.10.6 Gated Recurrent Units with Attention (GRU-Att)  
The gated Recurrent Units with Attention (GRU-Att) model has also demonstrated exciting results in detecting bullying behavior in a social media text. The model can capture long-term dependencies in sequential data faster than the LSTM model while capturing the important part of the input text by combining Gated Recurrent Units (GRUs) with attention mechanisms. This makes it possible for the model to more accurately interpret and categorize the behavior of cyberbullying on social media platforms [71; 123; 124]. Attention-based deep learning models have demonstrated promising outcomes in the detection and prevention of cyberbullying behavior on social media platforms. These models, including the transformer, GRU with Attention, BERT, HAN, and CNN with Attention, have outperformed conventional machine learning methods and are capable of capturing complex relationships in text data. However, the quality and quantity of the training data, the selection of the hyperparameters, and the unique characteristics and design decisions of the model are all important factors that affect how well these models perform. As a result, even though attention-based deep learning models offer a promising method for identifying cyberbullying, careful assessment and validation of the models are required before applying them to real-world situations to ensure their efficacy, dependability, and moral implications. As shown in Table 4, LSTM, Bi-LSTM, and CNN are frequently used models for the identification of cyberbullying. The LSTM and CNN models have recently been used in a variety of natural language processing (NLP) applications, because these models produce better results. Convolutional layers as well as maximum pooling or max-overtime pooling layers are used in CNN models to extract higher-level features. CNNs may be trained to extract character-level embeddings and n-grams, which are crucial for finding instances of cyberbullying in text. A CNN is an effective technique for detecting cyberbullying because its filters may be used to identify various patterns and elements in the text at various levels of abstraction, but CNN has some limitations such as capturing long-term dependencies, which is challenging, and requiring fixed size input, and it is significantly slower due to high operations. On the other hand, long-term dependencies between word sequences can be captured by LSTM models, which is a vital requirement in the context of cyberbullying detection [125]. Long-term dependencies can be captured by LSTM in text data, which is crucial for spotting abusive language or behavior patterns over time. Additionally, it can process variable-length input sequences, which is helpful for handling text data with a range of durations, such as comments or posts on social media. GRU is another RNN-based model that is employed to detect cyberbullying problems. In terms of time and space complexity, GRU is more effective than LSTM [111], although LSTM can produce more accurate results while working with datasets that contain longer sequences. As the focus point is cyberbullying detection and the texts are generally too long, GRU is not frequently used for this purpose. A Deep Belief Network (DBN) is rarely used for cyberbullying detection. In the majority of cases, the network is used as one of the components of hybrid models [126]. DBN is an unsupervised learning method, as opposed to perceptron and backpropagation neural networks. The noise in the input data can be reduced by using autoencoders, which greatly increases the effectiveness of deep learning models. In addition, autoencoders are frequently employed to address the issues with unsupervised learning and to spot anomalies. However, the drawbacks of autoencoders are what make them ineffective for the goal of cyberbullying detection. The limitations of autoencoders include imperfect decoding, misinterpreting important variables, and using too much lossy compression [127].