# Offensive-Language Detection on Multi-Semantic Fusion Based on Data Augmentation
## 2 Related Work  
In this section, we review related works on datasets, data augmentation, deep-learning-based methods, and pre-trained models for offensive-language detection. There are relatively few datasets in the field of offensive-language detection. Labeling methods can be divided into manual labeling and semi-supervised automatic labeling. [PERSON] et al. [9] created the offensive-language dataset DKHATE in Danish with only 3600 sentences. [PERSON] et al. [10] constructed the Offensive Greek Tweet Dataset (OGTD),which contains 10,287 sentences. Compared with other languages, the scale of the offensive-language datasets for English is larger, such as OLID [2] with 13,240 sentences and the [PERSON] dataset [11] containing more than 20,000 sentences. [PERSON] et al. [12] annotated data in a semi-supervised manner and constructed the SOLID dataset. Although SOLID has 9 million pieces of data, the dataset is much noisier, and the quality is lower. In summary, the manually labeled datasets are small in size and cannot meet the needs of models, and the automatically labeled datasets are large in scale, but the quality cannot be guaranteed. Data augmentation is a technique for automatically expanding training data. The great success of deep learning is supported by a large amount of data. [PERSON] et al. [13] replaced some words in the sentence with their synonyms to make the augmented data fit the original semantics as much as possible. [PERSON] et al. [14] and [PERSON] et al. [15] transformed the original document into text in other languages through translation and then translated it back to get the new text in the original language to achieve the purpose of data expansion. In addition to the above methods, there is also word vector-based replacement [16], simple pattern-matching changes at the word level, and so on [17]. Previous studies have shown that data augmentation can effectively improve the performance and robustness of the models. Our intuition is that introducing data-augmentation methods into offensive-language-detection tasks can effectively improve model performance. Traditional deep-learning methods filter and extract the semantic information of sentences by constructing different neural-network structures to obtain features with strong representation and use them for offensive-language-detection tasks. [PERSON] et al. [18] proposed a hybrid CNN-LSTM method for offensive-language-detection tasks. The experimental results show that the performance of this method is not much better than when using static word vectors. [PERSON] and [PERSON] [4] used CNN to extract four-gram character-level features for offensive-language detection, and character-level features can effectively improve the performance of offensive-language detection. [PERSON] et al. [19] used maximum pooling and average pooling to fuse features of different dimensions after the word-embedding layer. It was found that the fusion strategy is significantly better than the single pooling strategy. In conclusion, the static word vector is not conducive to detecting offensive language and the combination of multiple pooling helps to improve the performance of models. More importantly, the character-level features can effectively improve the performance of offensive-language-detection tasks. The transformer-based pre-training model has achieved good performance in natural-language-understanding tasks. Representative models include ELMO [20], GPT, and BERT. [PERSON] et al. [21] fine-tuned BERT to detect offensive language. [PERSON] and [PERSON] [22] proposed multi-channel BERT, and each BERT channel corresponds to a pre-trained BERT in different languages. [PERSON] believes that offensive language expressed in different languages has a similar semantic representation and syntactic structure; so, he translated the dataset into multiple languages and used multi-channel BERT to extract features. [PERSON] et al. [23] used a hybrid BERT-GCN method to recognize offensive language.