## 2 Literature Review
### Deep Neural Networks  
An increasing number of studies on cyberbullying detection have employed DL methods to eliminate the manual feature extraction stage required in traditional ML because no specific or clear characteristics exist for detecting cyberbullying on different platforms. Deep neural networks have been widely applied to address various problems, such as environmental sound classification [30; 31], scene parsing [32], large-scale projects construction [33], and near-misses [34]. [PERSON] and [PERSON] [6] reviewed studies on cyberbullying detection techniques for Twitter content and determined that most research methods for improving detection efficiency focused on the feature extraction stage without considering the semantics of words; however, language changes over time, and offensive speech is constantly being innovated. Optimizing detection performance by using feature engineering approaches is difficult. Many researchers have adopted NLP approaches and DL methods to solve cyberbullying detection problems with a backpropagation algorithm for deep neural networks to obtain a set of word vectors that learn semantics and grammar by using a corpus. Accordingly, DL methods have a superior detection performance to traditional ML methods. Another critical model for sequential data is a recurrent neural network (RNN), which has been used to solve numerous NLP tasks. An RNN is a neural network-based model that uses a memory mechanism to remember the last state to the next time step. However, RNNs may exhibit a vanishing gradient problem if the data contain long sequence features. Therefore, [PERSON] and [PERSON] [35] proposed a new RNN model based on long short-term memory (LSTM), which contains three gates that are used as control mechanisms of the cell (i.e., the forget gate, input gate, and output gate). These gates can manage and predict critical hidden features with long intervals in the time series and overcome the drawbacks of the original RNN. Moreover, [PERSON] et al. [36] proposed the gated recurrent unit (GRU) model to simplify the architecture of LSTM. In their model, the forget and input gates are replaced with the \"update gate\" to expedite execution and reduce memory consumption. [PERSON] et al. [37] developed a cyberbullying detection system for educators that targeted students' social network relationships on Instagram and used LSTM to classify offensive messages. Since hate speech is on the rise, identifying whether repeated attacks against the same student have occurred is critical. In summary, RNNs have been employed in numerous studies to extract hidden features from text data effectively. Thus, RNN-based models, such as the simple RNN, GRU, and LSTM models, apply to numerous sequential data classification and regression problems. Convolutional neural networks (CNNs) are used to retain semantics, eliminate time-consuming feature extraction, and effectively improve classification accuracy. In general, CNN models are used to extract hidden features from an image. In recent years, CNN applied to extract hidden features from text data based on word embeddings, and it can improve performances. This method can also be applied to text for extracting hidden local features of text sequences. [PERSON] [38] proposed that the CNN model can be used to extract hidden features from the text for sentence classification tasks and that word2 vec pretrainedword vectors can be used as input features for CNN models. CNN uses different numbers of convolution kernels to extract multilevel features from text according to the model weight sharing approach. Moreover, the max-pooling approach has been used to extract critical features, and the SoftMax function has been used to determine the probability that each feature belongs to a certain category. Therefore, the CNNs method has been widely applied in text classification studies.