## 3 Proposed Framework
### Bert  
The BERT model obtains semantically richer word embeddings through pre-training and fine-tuning, overcoming the limitations of traditional word embeddings' polysemy issue [47]. Fine-tuning allows the model to be applied to specific downstream tasks such as OLD, enhancing its generalization ability [48]. For pre-training, a base model is constructed by stacking the encoder part of Transformer models, as illustrated in Figure 2. Joint training of Masked Language Model (MLM) and Next Sentence Prediction (NSP) tasks captures semantic vector representations at both word and sentence levels. The MLM achieves the true bidirectional language modeling [49]. In the transfer to downstream tasks, BERT draws inspiration from OpenAI's Generative Pre-training (GPT) model and designs more versatile input and output layers, surpassing the generality of GPT [50]. Figure 1: Illustration of the proposed dual-branch, zero-shot CLT framework for OLD tasks. In the first network branch on the left side, the labeled data from the source domain and the unlabeled data from the target domain are jointly used as inputs for feature extraction. mBERT transfer learning incorporates SDRL (Sinkhorn distance representation layer) to extract cross-language shared features. On the right side, the second network branch removes offensive pivot words from training data in both source and target languages, obtaining language-specific independent features. Ultimately, through end-to-end framework joint training of the two network branches, the OLD performance is enhanced for the unlabeled target language. In BERT, text information is first transformed into word embeddings E, and then input into the Transformer structure, consisting of self-attention mechanisms and feedforward neural networks (FFN). The BERT pre-trained model undergoes extensive pre-training on large corpora, enabling it to convert semantically related words or characters into one-dimensional word vectors with close distances in the feature vector space. This retains more valuable information and emotional tendencies, facilitating the differentiation of offensive comments in subsequent tasks. In typical scenarios, the degree of association between words with negative sentiments and personal pronouns in a sentence largely determines its malicious intent. For instance, in the sentence written in Malay, \"Kubur sudah sedia untukmu, kenapa kamu tidak masuk?\" (The grave is ready for you, why do you not go in? ), there are no explicit vulgar words. However, the word \"kubur\" (grave) carries strong negative connotations and is closely related to the personal pronoun \"kamu\" (you), demonstrating an offensive attitude. Therefore, inputting the transformed word vectors into the Transformer structure and calculating the closeness of their relationship aids in determining whether the statement is offensive. The obtained word vectors \(E\) are used as input, passing through multiple layers of Transformer feature extractors, yielding feature information (\(T_{1}\), \(T_{2}\),..., \(T_{n}\)). During the self-attention mechanism, an initial random matrix \(X\) is generated. The word embeddings multiplied by \(X\) produce three new vectors: Query Q, Key, and Value, denoted as \(Q\), \(K\), \(V\). The corresponding \(Q\), \(K\), and \(V\) vectors for multiple word embeddings form matrices \(W^{Q}\), \(W^{K}\), and \(W^{V}\). The use of matrices \(W\) simplifies the computation process. The \(Q\) of each input word embedding is computed with the \(K\) of other word embeddings, and both word vectors have an initial \(V\). The operation result of \(Q\) and \(K\) generates a weight, merging the importance of the corresponding relationships between two words into the initial \(V\), thereby preserving contextual semantic information.