# Cross-Lingual Cross-Domain Transfer Learning for
## 3 Multi-Task Cross-Lingual Learning for Rumor Detection
### Overall Description of the Proposal  
Without losing generality, we formulate the problem by focusing on Twitter's interaction mechanisms. However, this methodology can be applied to any platform that supports a reply mechanism for preceding comments. This feature is sufficient to build a propagation tree starting from a root message. Other platforms where this methodology can be applied include TikTok, Facebook, YouTube, and Threads. Our approach outlines four main stages, presented in Figure 1. The core concept of this methodology is to initially use individually trained models for specific auxiliary tasks. These models are then sequentially integrated to create a propagation-based model for rumor detection. **Stage 1**. We develop auxiliary task-specific models that are trained independently of the main model. This involves creating one model for stance classification and another for detecting bots. Both models employ BERT (specifically, BERT base-uncased) as their text encoder. For each task, the primary objective is to achieve results that are competitive with the state of the art, even though these implementations are not necessarily expected to produce the best outcomes for every task. To develop the stance classification model, the BUT-FIT methodology is adopted, originally introduced by [PERSON] et al. (2019) in RumorEval 2019. This approach has proven to be the most effective BERT-based strategy in competitive settings. BUT-FIT capitalizes on the notion that the context of a specific response within a propagation tree can be accurately inferred from both the root message and the preceding message in the conversation. This method effectively circumvents the input size constraints of BERT, as it only requires a consideration of three tweets at a time, as is illustrated in Figure 2a. The context of the target tweet incorporates the root message and the preceding message. To encode these tweets, we concatenate the text of these messages and enclose them between the special  
Figure 1: General architecture composed of four stages. Every stage generates an output and is used for the next stage, with the exception of the first stage. CLS and SEP symbols of BERT. Then, the target message is encoded, and the instance is closed using the SEP symbol. We subsequently fine-tune BERT using this encoding scheme to produce the stance classification model. Similarly to the efforts made in stance classification, we also aimed to identify a bot detection method that yields competitive results. To achieve this, the technique described by [PERSON] et al. [88] is adapted, leveraging BERT (BERT-base-uncased) to encode the text. We operated under the constraint that the only available data for training and evaluation consist of a sample of 100 tweets per user, as illustrated in Figure 2b. To avoid BERT's input size limitations, a sampling strategy on the pool of 100 tweets is employed. This strategy is based on a study by [PERSON] et al. [89], which suggests extracting a specific number of characters from the beginning, end, or middle of each sampled tweet. Our implementation allows for customization through parameters controlling the total number of sampled tweets and the number of tokens extracted from each tweet. **Stage 2**. We use the concept named \"cascade learning models\" [90] (CLMs for short), which is based on the principle of sequential learning. A CLM _is a sequence of models linked through pre-training and fine-tuning strategies_. The first link in this chain starts with a pre-trained model tailored for a specific task in a source domain. Suppose that this pre-trained model is designed for text encoding. In that case, it can be adapted for a different task, or even the same task in other languages, by employing a fine-tuning process using data specific to the new task. This chaining mechanism outlines a transfer learning strategy from a source domain to a target domain, allowing the new model to benefit from the pre-trained model by adapting it to new data.