# Cross-Lingual Cross-Domain Transfer Learning for
## 3 Multi-Task Cross-Lingual Learning for Rumor Detection
### Overall Description of the Proposal
More specifically, to facilitate knowledge transfer from a baseline model to the rumor detection task, these baseline models are used to encode the text of each propagation tree. This methodology establishes an empirical framework that allows us to identify which base models create the most favorable conditions to tackle the main task. We employ each base model to encode the text fed into the primary task model. Given that each base model is a fine-tuned mBERT encoding model specialized for a particular sequence of auxiliary tasks, it aids in accurately encoding instances for rumor detection by transferring its acquired knowledge through textual representation from auxiliary tasks. Figure 5 describes this approach. Figure 5: To pre-train the main task model, we used the base models learned from the auxiliary tasks to encode the text. RD stands for rumor detection. Figure 4: The chaining mechanism that defines cascade learning reuses models with arity levels of 1 and 2. In this example, CLM10 reuses CLM4, which is based on CLM1. As illustrated in Figure 5, the primary task is addressed using a Bidirectional GRU model [91], i.e., a Gated Recurrent Unit (GRU) architecture designed to capture sequential dependencies in both directions--left to right and right to left--over the input sequence. The motivation for choosing this specific model is to provide contextual information for the classification. To feed the model, we chronologically input a sequence of tweets based on each tweet's timestamp. In determining the encoding for each tweet, various strategies were tried out, based on a base model selected in stage 2. * _mBERT-embedding_: use the _embedding_ layer of the base model. * _mBERT-pooler_: use the _pooler_ layer of the base model. * _mBERT-second-to-last_: use the second-to-last layer of the base model. * _mBERT-sum-four-last_: use the sum of the last four layers of the base model. * _sBERT-default_: use the default constructor of embeddings defined for sentence BERT. In addition to our base CLMs, a standard mBERT checkpoint for comparison was incorporated under the name _Base0_. With 16 primary text encoding models (15 CLMs and Base0) and 5 different strategies to generate embeddings in the main task, we have 80 unique settings, denoted from \(S_{1}\) to \(S_{80}\). This extensive experimental setup allows us to explore various settings for cross-domain and cross-lingual transfer learning in the main task. The primary task we focus on at this stage is the detection of rumors in English, as described in RD EN (Figure 5), which refers to the rumor dataset in English. **Stage 4**. The final stage of the methodology involves applying transfer learning from an English rumor detection model to a Spanish rumor detection model. We employ two strategies to assess the effectiveness of the proposed transfer learning approach. The first strategy involves zero-shot learning and the second utilizes a recently developed dataset for rumor detection in Spanish. * **Zero-shot evaluation**. It is worth noting that the pre-trained models have not been trained on any Spanish rumor detection data to date. This is why we first assess the zero-shot capabilities of the pre-trained models of stage 3. We also compare their performance with a baseline mBERT model for comprehensive understanding. * **Fine-tuning for Rumor Detection in Spanish**. Lastly, we evaluate the potential of the pre-trained models from Stage 3 for fine-tuning to improve their performance in detecting rumors in Spanish.