## 2 Related Work  
The research of KBQA has evolved from earlier domain-specific question answering [2] to open-domain QA based on large-scale KBs such as Freebase [3]. The model of KBQA has also evolved from semantic parsing-based models [4], which parse questions into structured queries, to neural network-based models [5; 6], which learn semantic representations of both the question and the knowledge from observed data. Some researchers [7; 8; 9] also attempt to combine multiple models to utilize information in natural language questions and KBs. After pre-trained models such as BERT [10], ALBERT [11], XLNet [12], and ELECTRIA [13] are proposed, they have been widely employed in various NLP tasks [14; 15; 16; 17]. Many researchers employed NLP models based on pre-trained models and achieved good results. For example, [PERSON] et al. [18] employed pre-trained models in the span extraction, classification, and relation extraction tasks focused on finding quantities, the attributes of these quantities, and additional information. [PERSON] et al. [19] proposed a BERT-based approach for single-relation question answering (SR-QA), which consists of two models, entity linking, and relation detection. [PERSON] et al. [20] designed a comprehensive search space for BERT-based relation classification models and employed the neural architecture search method to automatically discover the design choices. However, in different situations, the best-performance model is also different. For example, ELECTRA achieves better performance in some tasks in GLUE [21], while ALBERT requires less training cost, and RoFormer is more effective for Chinese NLP tasks. However, it could be difficult to employ them in KBQA as the difference between the subject recognition task in KBQA and the general NER task is significant. After the subject is recognized, the answer will be found using a relation extraction (RE) model for single-relation questions or other models for complex questions. For exam  
Figure 1: Different subject recognition models and their entity tokens (gray background), start tokens (green background), end tokens (blue background). (**a**) Flat NER model and strict matching strategy. (**b**) Flat NER model and approximate matching strategy. (**c**) Nested NER model and approximate matching strategy. (**d**) Our proposed model. ple, a trainable subgraph retriever decoupled from the subsequent reasoning process is proposed to achieve better retrieval and QA performance [22], a dynamic program induction and a dynamic contextualized encoding are employed to address both the large search space and schema linking in a unified framework [23], and an effective and transparent model is proposed to support both label and text relations in a unified framework [24]. However, these models rely on the accurate recognition of subjects, and a wrong subject would lead to a wrong answer. In practical applications, an NLP model is often designed to answer noisy and abnormal questions caused by various reasons (e.g., noise in the processes of transmission, transformation, or translation). Sometimes, the input to an NLP system is even transformed from a piece of voice, video, or image. If the raw voice, video, or image is available, we could feed it directly into special models such as VL-BERT [25], LXMERT [26], VideoBERT [27], ClipBERT [28], wav2 vec [29], or SpeechBERT [30] to avoid errors caused by transformation. In addition, the structure of the original model could also be improved so that such noise could be handled automatically by the model. For example, [PERSON] et al. [31] proposed a robust and structurally aware table-text encoding architecture, TableFormer, where tabular structural biases are incorporated completely through learnable attention biases. [PERSON] et al. [32] proposed a pre-trained Chinese BERT that is robust to various forms of adversarial attacks like word perturbation, synonyms, typos, etc. [PERSON] et al.