## 1 Introduction
These methods use deep neural networks such as recurrent neural networks (RNN), short and long-term memory networks (LSTM), and Transformer to capture complex patterns in text. These models often require large amounts of labeled data to train, but they can achieve high performance in many cases. An integrated approach combines several different models or detectors to improve detection performance. Integrated approaches can include voting methods, stacking, or meta-learning techniques. Pre-trained deep learning models, such as BERT, GPT, and RoBERTa, have recently been used for offensive language detection. These models are pre-trained on large-scale text data and can be fine-tuned to suit specific detection tasks. They often perform excellently because they can understand the context. Over the past few years, task performance in natural language processing has exploded. On the one hand, thanks to improvements in natural language algorithms, such as Text3D [8] (a 3D convolutional neural network for text classification), text classification research on low-resource language Tigrinya [9], optimization of LSTM model by adding an attention layer based on dropout layer and bidirectional recursion layer [10], and only using questions-answers archive automatically creates chatbots [11]. On the other hand, this is mainly due to the development of the Transformer architecture. BERT, a Transformer-based model developed by Google, is the first deeply bidirectional, unsupervised language representation model by co-adjusting the left and right context in all layers [12]. It has shown overwhelming performance and has been used as a central part of many studies in offensive language detection. RoBERTa is an improved model based on BERT, which further optimizes the structure of BERT by adjusting training hyperparameters and expanding training data sets, thereby improving its performance for natural language processing tasks [13]. Compared to BERT, RoBERTa uses a longer training time and a larger corpus, allowing RoBERTa to understand semantics and context in natural language better. Techniques such as Bidirectional Gated Recurrent Unit and Multi-head attention have also been gradually applied in offensive language detection. Bidirectional GRU is based on recurrent neural network technology. It enables the extraction of long-term dependenciesin the text by considering both forward and reverse information of the input sequence [14], thus capturing text fragments containing offensive category features. Multi-head attention can automatically calculate the weighted sum of different position vectors to realize the attention and memory of different positions in the text sequence and to capture rich information in different aspects of the text. Applying these techniques can supplement and enhance the feature representation and performance of the model from different angles. Integrating RoBERTa with bidirectional GRU and Multi-head attention may have great application prospects for enhancing the text perception ability of the model and the accuracy of offensive language detection. Unfortunately, most of the work in offensive language detection has been carried out on English datasets, and most existing models focus on detecting offensive language in English. Although other languages such as Arabic, Turkish, Greek, Danish, and Urdu [15] are also increasing in proportion in this field [2], it is worth noting that Chinese, as a language with a large number of global speakers, has little research in the field of offensive language detection. Due to the lack of labeled data sets and reliable detectors, the problem of Chinese offensive language detection has yet to be well studied [16]. Therefore, the RB_BG_MHA model is proposed in this paper to detect offensive Chinese language in social media. First, we use the RoBERTa model to extract semantic information at the sentence and word levels and use the pre-trained language model to learn.