## 2 Related Work
### Zero-Shot Inference Verification  
The large-scale auto-regressive LMs were adjusted through few-shot and zero-shot learning for wide-ranging tasks with obviously less computational cost than full fine-tuning [33]. Regarding most monolingual pre-trained models, multilingual-masked and sequence-to-sequence models that perform cross-lingual tasks with fine-tuning on the labeled data in downstream tasks have been designed, such as mBERT, XLM-R, and mT5 [34]. Considering that multilingual few-shot learning capabilities are less well understood in the above models, the comprehensive multilingual models (as GPT-3 [35] and XGLM [34]) were proposed in the few-shot and zero-shot learning paradigm for multilingual natural language understanding tasks without fine-tuning. However, the multilingual models can only generate a fundamental semantic understanding of claims and evidence, and the abilities of multilingual fact-checking mainly depend on downstream module designs.  
For further leveraging the powerful zero-shot learning abilities of LMs, a new way that takes advantage of LMs (including BERT, GPT2) via a perplexity score was shown to exhibit zero-shot capabilities, thus, outperforming the Major Class baseline models by more than 10% on F1-Macro metrics across multiple datasets [8]. In addition, two new cross-domain fact-checking datasets corresponding to the model were released to prove the rationality of the perplexity score synchronously. However, the best optimal perplexity threshold that separates supported or unsupported claims was generated from the statistical calculation of evidence-conditioned perplexity scores for each claim, which was not automatically evaluated in models. As an automatic framework, Question Answering for Claim Generation (QACG) [36] was proposed to train a fact-verification model by using automatically generated claims that can be supported, refuted, or unverified from the evidence. The model improved Roberta's performance by up to 22% when experimented on the FEVER dataset, and reduced the human-annotated demand that was equivalent in performance to 2K+ manually curated examples in the zero-shot scenario. When solving the problem of the multilingual zero-shot fact-checking tasks, the machine learning classifiers automatically identified COVID-19 misinformation via learning contextualized embedding. The misinformation from social media included English, Bulgarian, and Arabic [37]. As shown in the experimental results, the BERT and multilingual BERT achieved the best result.  
Nonetheless, the task only explored contextualized embedding from the LMs for misinformation detection without delving into the cross-lingual implicit value, especially the mutual information transfer between the different languages. In another study, the zero-shot capabilities of several multilingual transformer-based models were evaluated based on the X-Fact dataset for the fact-checking task. Through modeling the attention relationship between the textual claim and evidence from news stories retrieved using a search engine, the attention-based evidence aggregator (Attn-EA) model outperformed the claim-only model (Claim-Only) [9]. However, as a baseline for suggesting the X-Fact dataset, the model--without making full use of additional metadata--lacked the interpretable claim verification and fine-grained evidence aggregating and inferring, and was not complete to construct a unified fact-checking cognitive framework.