# A Cross-Lingual Media Profiling Model for Detecting Factuality and Political Bias
## 5 Experiments
### RQ2: How Effective Is Our Approach?  
Performance of the R-KAT  
Table 4 presents the results of XLM-R with multiple training strategies on xMP. We observed significant performance improvements in both monolingual and zero-shot cross-lingual settings for both media profiling tasks. The optimal results are highlighted in bold, while the suboptimal results are underlined.  
Conducting media profiling tasks in a cross-lingual context poses significant challenges. Compared to the monolingual setting, the mean absolute error (MAE) for media factuality ratings increased by an average of 0.677. Meanwhile, the F1 score dropped by an average of 0.085, and accuracy decreased by 0.155 on average in the zero-shot cross-lingual setting. Similarly, for media political bias ratings, MAE increased by 0.489, the F1 score decreased by 0.234, and accuracy dropped by an average of 0.326.  
These results highlight the need for further research to improve the transfer of knowledge from English-language media profiling to multilingual media profiling. Current approaches face significant performance drops in cross-lingual scenarios, underscoring the complexity of media profiling across different languages and cultural contexts.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Label Granularity**} & \multicolumn{3}{c}{**Factuality**} & \multicolumn{3}{c}{**Political Bias**} \\  & **ACC** & **F1-Macro** & **MAE** & **ACC** & **F1-Macro** & **MAE** \\ \hline Fine (6-class/7-class) & 0.221 & 0.242 & 0.573 & 0.226 & 0.227 & 0.744 \\ \hline Coarse (3-class) & 0.748 & 0.717 & 0.263 & 0.453 & 0.468 & 0.229 \\ Fine2 Coarse (3-class) & 0.460 & 0.485 & 0.313 & 0.254 & 0.289 & 0.237 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Experimental results on the relationship between label granularity.  
Perturbation-based approaches, such as ADV, RS-RP, and R-KAT, consistently exhibit superior performance in cross-lingual media profiling tasks. In one-step perturbation-based training strategies for cross-lingual transfer learning, both ADV and RS-RP show notable performance improvements. However, significant performance gaps remain when compared to our training strategy, which is based on multistep perturbations.  
The performance of these strategies, evaluated on mBERT, is presented in Table 5. Our proposed R-KAT methodology consistently demonstrates substantial performance gains across various frameworks and task configurations. As a comparative baseline, RS-RP consistently outperforms ADV in cross-lingual transfer, delivering optimal or near-optimal results across diverse scenarios. This trend aligns with the findings of previous research [51].  
While RS-DA has shown success in prior cross-lingual transfer learning studies, its performance in our task fell short. We attribute this to the lack of diversity in RS-DA's sentence-level data augmentation, which hindered the construction of a robust multilingual media profiling embedding space. This limitation compromised its effectiveness in cross-lingual settings. Nonetheless, RS-DA may still be applicable for other cross-lingual media analysis tasks. We recommend exploring article-level data augmentation tailored specifically for cross-lingual transfer learning as a potential avenue for improving performance.