SlowGuess/NAACL_2022_OCR huggingface dataset:
We compare our framework with the following baselines. First, to show the characteristics of each benchmark, we use the random or the most frequent label as Random and Majority baseline, respectively. RoBERTa-L and GPT2-L is the performance of each PLM without any fin-tuning. Also, as the baseline for the unsupervised learning model using KGs, we report the performance of Self-talk (Shwartz et al., 2020), COMET-DynaGen (Bosselut and Choi, 2019), SMLM (Banerjee and Baral, 2020) as presented in original papers.
For further analysis in §4.4 and §4.5, we set the following models that are pre-trained on the synthetic QA datasets from KGs as baselines:
- Single-Task Learning (STL): The model is pre-trained on a synthetic QA dataset generated from a single KG. Specifically, we experiment two architectural choices: PLM (STL-PLM) and PLM with adapters (STL-Adapter). For each architecture, there are four STL models for each of synthetic QA datasets derived from ATOMIC, ConceptNet, WikiData, and WordNet. We note that the trained STL-Adapter is an expert adapter from a specific KG in our framework. The performance of
each STL baseline is shown in Appendix I Table 9 and Table 10.
- Multi-Task Learning (MTL): The model is pre-trained on multiple synthetic QA datasets, each of which is generated from a KG. We experiment with a PLM trained on all four aforementioned synthetic QA datasets. We note that the difference between STL-PLM and MTL is whether to use one synthetic QA dataset or multiple synthetic QA datasets for its training.
4.1.3 Implementations
We employ RoBERTa-L (Liu et al., 2019b) from Hugging Face's transformers toolkit for all experiments. We follow the default settings from Ma et al. (2021). Our implementation uses Adapter (Houlsby et al., 2019) and AdapterFusion (Pfeiffer et al., 2021) as a base model architecture from AdpaterHub (Pfeiffer et al., 2020a). We run our experiments with three different random seeds. The implementation details are described in Appendix H.
Table 2 shows the zero-shot evaluation results on five benchmark datasets. Generally, zero-shot fusion scores higher than the baselines across all benchmarks, and further, zero-shot fusion shows the best performance in all benchmarks except WG. We note that although Ma et al. (2021) uses the synthetic QA dataset after sample filtering, our method achieves comparable performance with the best performance in WG, even with the raw dataset. Also, the average score of all evaluation benchmarks (the
3Some benchmarks have a strong alignment with a certain KG due to its construction strategy: SIQA-ATOMIC, and CSQA-ConceptNet. To make a direct comparison with Ma et al. (2021), we use the same KGs to generate data samples.
last column of Table 2) shows that zero-shot fusion has generalisability in commonsense reasoning.
In addition, zero-shot fusion achieves consistent improvements over MTL. These results indicate that our proposed zero-shot fusion method attributes to fusing the knowledge of multiple KGs more synergically regardless of the task.
Moreover, as an ablation, we compare the zero-shot fusion with and without KG-C adapter to explore the efficacy of the KG-C adapter. We can observe that zero-shot fusion with KG-C adapter improves the average accuracy by  $0.4\%$ , which implies that the use of KG-C adapter improves the overall performance and makes our method generalize better on most of the evaluation benchmarks.
4.3 Impact of the KG-Classifier Adapter
To assess the effects of the KG-C adapter itself, we visualize and compare the final layer [CLS] token representation between PLM and KG-C adapter. Figure 3 shows t-SNE (Van der Maaten and Hinton, 2008) plots of all representation of five benchmark datasets. In this figure, every sample is mapped into a 1024-dimensional feature space through RoBERTa-L model and projected back into a two-dimensional plane by t-SNE. We can observe that KG-C adapter can separate the samples of different benchmarks well despite being unseen data. It verifies that KG-awareness acquired with the KG classification task is beneficial to categorize the given sample. The KG-C adapter can thus generate a relevant KG-aware query for a given sample and help to fuse representations from suitable expert adapters in our proposed framework.
Further, we explore how the KG-C adapter affects zero-shot fusion which is based on an attention-like mechanism (Pfeiffer et al., 2021) compared to zero-shot fusion without KG-C adapter. Here, while zero-shot fusion without KG-C adapter simply uses the representation of PLM as a query, zero-shot fusion with KG-C adapter leverages the representation of KG-C adapter. To illustrate this strength, we visualize the attention probability of [CLS] token from each fusion layer as a representative in Figure 4. The column of the darker cell indicates the adapter that has the bigger influence on the fused representation. We can observe that zero-shot fusion with KG-C adapter fuses the knowledge from different experts with a subtle difference rather than focusing on a single expert severely. This implies that KG-C adapter
(a) PLM
(b) KG-Classifier adapter
Figure 3: t-SNE visualization of the hidden representation from (a) PLM and (b) KG-C adapter. Each color denotes the five different benchmark samples.
(a) w/o KG-C adapter
Figure 4: Comparison of attention probability between zero-shot fusion with/without KG-C adapter. The x- and y-axis indicate expert adapters and the fusion layer number in RoBERTa-L, respectively. The darker color indicates higher attention probability in fusion layer.
(b) w/KG-C adapter
enables the delicate balancing between multiple knowledge sources based on the KG-alignment awareness, which leads to performance improvements in commonsense reasoning tasks. Interestingly, both cases have the ability not to focus on the expert adapter based on WikiData, which can be seen as a redundant expert. This observation would benefit from the further study that explores the optimal combination of KGs by expert selection or rejection.
4.4 Mitigating Interference
In this experiment, we compare the amount of interference in the MTL and zero-shot fusion with KG-C adapter. We propose a novel evaluation metric, the interference ratio, which is the percentage of the incorrectly predicted samples by the multi-KG models among the correctly predicted samples from the STL models in common.
Using the interference ratio, we can precisely compare the negative effects of multi-KG models
4The zero-shot fusion with KG-C adapter using AT, CN, and WN shows the best average performance in Table 10.
Figure 5: Interference ratio of multi-KG models on five benchmarks. The lower indicates less interference.
on knowledge aggregation since the only reason to get the correct samples wrong is the interference caused by learning with additional KGs. We present the interference ratio of the models on five benchmark datasets in Figure 5. This figure shows that MTL has the higher interference ratio than the competing models across all benchmarks. Our method achieves a substantially better ratio, especially when  $KG-C$  adapter is used. This demonstrates the efficacy of our framework in mitigating interference between knowledge, which is one of the major problems of MTL.
4.5 Visualization of Knowledge Aggregation
To verify the ability of our model to aggregate different types of KGs, we compare the relative performance gains of MTL and zero-shot fusion with KG-C adapter when increasing the number of KGs. The performance of all KG-combinations for each framework is presented in Table 9 and Table 10. We visualize the improvement of performance for five benchmark development sets, leveraging heatmaps in Figure 6. Here, for the sake of brevity, we denote our framework with KG-C adapter as our method.
For MTL in Figure 6 (a), the color of the cell denotes the relative improvement of MTL with the combination of KGs over the best performance among the STL-PLM of KGs. Also, for our method in Figure 6 (b), the relative improvement is measured based on the best performance among the STL-Adapter of KGs, considering the difference of the base architecture for MTL (i.e. PLM) and zero-shot fusion (i.e. PLM with adapter). The green and red colors denote the increase and decrease of performance, respectively, when using multiple KGs together. The greener color on the cells indicates that the approach benefits from an increasing number of KGs, which implies aggregating knowledge successfully.
In Figure 6, while the MTL tends to show the decrease of the performance when more KGs are
(a) MTL
Figure 6: Relative improvement upon the STL on five benchmarks. The x- and y-axis indicate the benchmark and the combination of the KGs, respectively. The value of each cell indicates the relative performance improvement of using multiple KGs over the highest performance among STLs. The green and red colors denote the improvement or decrease of relative performance, respectively.
(b) zero-shot fusion w/ KG-C adapter
utilized for training, our method obtains relative performance improvement across most of benchmarks. In both frameworks, the slightly degraded performance of the combination of KGs without ATOMIC could be due to the strong alignment between ATOMIC and SIQA. Except for the above case, we can observe that as more KGs are leveraged, the color of the cell gets greener, which implies that our method gains more advantages for better performance. This demonstrates that our method enables knowledge aggregation for multiple KGs synergetically.
Despite the existence of various types of commonsense KGs, utilizing multiple KGs has not been explored enough in the commonsense reasoning field. Motivated by this, this paper proposes a modularized transfer learning framework to fuse the knowledge from multiple KGs efficiently for zero-shot commonsense reasoning. Our framework consists of KG modularization for expert adapter, zero-shot fusion and KG-Classifier adapter. Extensive experiments show that our framework obtains strong improvements over MTL on five commonsense reasoning benchmarks.
In the future, our work can be extended to adapt our methods to further various multiple KGs with studies of appropriate scale for KG modularization. In addition, based on our hypothesis that the existence of an optimal combination, we can explore the study for the optional use of modularized KG experts for the best transfer learning.
This work was partly supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2020-0-01361, Artificial Intelligence Graduate School Program (Yonsei University)) and (No. 2022-0-00077, AI Technology Development for Commonsense Extraction, Reasoning, and Inference from Heterogeneous Data) and the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2021-11-1055). Jinyoung Yeo is a corresponding author.
Pratyay Banerjee and Chitta Baral. 2020. Self-supervised knowledge triplet learning for zero-shot question answering. EMNLP.
Ankur Bapna, Naveen Arivazhagan, and Orhan First. 2019. Simple, scalable adaptation for neural machine translation. EMNLP.
Chandra Bhagavatula, Ronan Le Bras, Chaitanya Malaviya, Keisuke Sakaguchi, Ari Holtzman, Hannah Rashkin, Doug Downey, Wen tau Yih, and Yejin Choi. 2020. Abductive commonsense reasoning. ICLR.
Yonatan Bisk, Rowan Zellers, Ronan Le bras, Jianfeng Gao, and Yejin Choi. 2020. Piqa: Reasoning about physical commonsense in natural language. In AAAI.
Antoine Bosselut and Yejin Choi. 2019. Dynamic knowledge graph construction for zero-shot commonsense question answering. arXiv e-prints, pages arXiv-1911.
Rich Caruana. 1997. Multitask learning. Machine learning, 28(1):41-75.
Ting-Yun Chang, Yang Liu, Karthik Gopalakrishnan, Behnam Hedayatnia, Pei Zhou, and Dilek Hakkani-Tur. 2021. Incorporating commonsense knowledge graph in pretrained models for social commonsense tasks. EMNLP-Workshop.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for nlp. In PMLR, pages 2790-2799. PMLR.
Jena D Hwang, Chandra Bhagavatula, Ronan Le Bras, Jeff Da, Keisuke Sakaguchi, Antoine Bosselut, and Yejin Choi. 2021. Comet-atomic 2020: On symbolic and neural commonsense knowledge graphs.
Filip Ilievski, Alessandro Oltramari, Kaixin Ma, Bin Zhang, Deborah L McGuinness, and Pedro Szekely. 2021. Dimensions of commonsense knowledge. Knowledge-Based Systems.
Filip Ilievski, Pedro Szekely, Jingwei Cheng, Fu Zhang, and Ehsan Qasemi. 2020. Consolidating commonsense knowledge. arXiv preprint arXiv:2006.06114.
Xiaodong Liu, Pengcheng He, Weizhu Chen, and Jianfeng Gao. 2019a. Multi-task deep neural networks for natural language understanding. ACL.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019b. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
Nicholas Lourie, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. *Unicorn on rainbow: A universal commonsense reasoning model on a new multitask benchmark.* arXiv preprint arXiv:2103.13009.
Jiaqi Ma, Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, and Ed H Chi. 2018. Modeling task relationships in multi-task learning with multi-gate mixture-of-experts. In ACM SIGKDD.
Kaixin Ma, Jonathan Francis, Quanyang Lu, Eric Nyberg, and Alessandro Oltramari. 2019. Towards generalizable neuro-symbolic systems for commonsense question answering. EMNLP.
Kaixin Ma, Filip Ilievski, Jonathan Francis, Yonatan Bisk, Eric Nyberg, and Alessandro Oltramari. 2021. Knowledge-driven data construction for zero-shot evaluation in commonsense question answering.
Andrea Madotto, Zhaojiang Lin, Yejin Bang, and Pascale Fung. 2021. The adapter-bot: All-in-one controllable conversational model.
Chaitanya Malaviya, Chandra Bhagavatula, Antoine Bosselut, and Yejin Choi. 2020. Commonsense knowledge base completion with structural and semantic context. In AAAI.
George A Miller. 1995. Wordnet: a lexical database for english. Communications of the ACM.
Arindam Mitra, Pratyay Banerjee, Kuntal Kumar Pal, Swaroop Mishra, and Chitta Baral. 2019. Exploring ways to incorporate additional knowledge to improve natural language commonsense question answering. arXiv preprint arXiv:1909.08855.
Jonas Pfeiffer, Aishwarya Kamath, Andreas Rückle, Kyunghyun Cho, and Iryna Gurevych. 2021. Adapterfusion: Non-destructive task composition for transfer learning. EACL.
Jonas Pfeiffer, Andreas Rückle, Clifton Poth, Aishwarya Kamath, Ivan Vulic, Sebastian Ruder, Kyunghyun Cho, and Iryna Gurevych. 2020a. Adapterhub: A framework for adapting transformers.
Jonas Pfeiffer, Edwin Simpson, and Iryna Gurevych. 2020b. Low resource multi-task sequence tagging-revisiting dynamic conditional random fields. arXiv preprint arXiv:2005.00250.
Jonas Pfeiffer, Ivan Vulic, Iryna Gurevych, and Sebastian Ruder. 2020c. Mad-x: An adapter-based framework for multi-task cross-lingual transfer. arXiv preprint arXiv:2005.00052.
Jonathan Pilault, Amine Elhattami, and Christopher Pal. 2021. Conditionally adaptive multi-task learning: Improving transfer learning in nlp using fewer parameters & less data. ICLR.
Joan Puigcerver, Carlos Riquelme, Basil Mustafa, Cedric Renggli, André Susano Pinto, Sylvain Gelly, Daniel Keysers, and Neil Houlsby. 2020. Scalable transfer learning with expert models. arXiv preprint arXiv:2009.13239.
Sylvestre-Alvise Rebuffi, Hakan Bilen, and Andrea Vedaldi. 2017. Learning multiple visual domains with residual adapters. arXiv preprint arXiv:1705.08045.
Andreas Rückle, Jonas Pfeiffer, and Iryna Gurevych. 2020. Multicqa: Zero-shot transfer of self-supervised text matching models on a massive scale. EMNLP.
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2020. Winogrande: An adversarial winograd schema challenge at scale. In AAAI.
Maarten Sap, Ronan Le Bras, Emily Allaway, Chandra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah A Smith, and Yejin Choi. 2019a. Atomic: An atlas of machine commonsense for if-then reasoning. In AAAI.
Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le-Bras, and Yejin Choi. 2019b. Socialiaq: Common-sense reasoning about social interactions. EMNLP.
Vered Shwartz, Peter West, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2020. Unsupervised commonsense question answering with self-talk. EMNLP.
Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. Conceptnet 5.5: An open multilingual graph of general knowledge. In AAAI.
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2018. Commonsenseqa: A question answering challenge targeting commonsense knowledge. *NAACL*.
Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. JMLR.
Denny Vrandecic and Markus Krötzsch. 2014. Wiki-data: a free collaborative knowledgebase. ACM.
Ruize Wang, Duyu Tang, Nan Duan, Zhongyu Wei, Xuanjing Huang, Jianshu Ji, Guihong Cao, Daxin Jiang, and Ming Zhou. 2021a. K-adapter: Infusing knowledge into pre-trained models with adapters. ACL.
Xinyi Wang, Yulia Tsvetkov, Sebastian Ruder, and Graham Neubig. 2021b. Efficient test time adapter assembling for low-resource language varieties.
Sen Wu, Hongyang R Zhang, and Christopher Ré. 2020. Understanding and improving information transfer in multi-task learning. *ICLR*.
Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. NIPS.
Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. 2020. Gradient surgery for multi-task learning. NIPS.
Rowan Zellers, Yonatan Bisk, Roy Schwartz, and Yejin Choi. 2018. Swag: A large-scale adversarial dataset for grounded commonsense inference. EMNLP.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? ACL.
Yu Zhang and Qiang Yang. 2017. A survey on multitask learning. arXiv preprint arXiv:1707.08114.
A List of Notations
We summarize the notations used in this paper in Table 7.
B Synthetic QA
We generate QA for four KGs (ATOMIC, ConceptNet, WikiData and WordNet) based on synthetic QA generation (Ma et al., 2021) without sample filtering. We use the prefixes for relation of triplet as shown in Table 3 for generating synthetic QA (refer to Ma et al. (2021)). Table 4 shows the statistics of the synthetic QA dataset from KGs. The samples of synthetic QA with source triplet are shown in Table 5.
relation	prefix
xAttr	. PersonX is seen as
xIntent	. Before, PersonX wanted
xNeed	. Before, PersonX needed to
xReact	. As a result, PersonX felt
xWant	. As a result, PersonX wanted to
xEffect	. PersonX then
oReact	. As a result, others felt
oWant	. As a result, others wanted to
oEffect	. Others then
Causes	can cause [MASK]
UsedFor	can be used for [MASK]
CapableOf	is capable of [MASK]
CausesDesire	causes desire for [MASK]
IsA.	is a [MASK]
SymbolOf	is a symbol of [MASK]
MadeOf	can be made of [MASK]
LocatedNear	is often located near [MASK]
Desires	desires [MASK]
AtLocation	can be found at [MASK]
HasProperty	has property [MASK]
PartOf	is part of [MASK]
HasFirstSubevent	starts by [MASK]
HasLastSubevent	ends by [MASK]
Table 3: Prefixes used for synthetic QA dataset
KG	Train	Validation	Total
ATOMIC	534,833	60,289	595,122
ConceptNet	363,645	19,140	382,785
WikiData	42,342	2,229	44,571
WordNet	256,922	13,523	270,445
Whole	1,197,742	95,181	1,292,923
C Commonsense Knowledge Graphs
A variety of KGs have been proposed to provide large-scale high quality collection of different commonsense knowledge types: ATOMIC (Sap et al.,
Table 4: Synthetic QA dataset statistics. Whole represents the combination of AT,CN,WD and WN.
QA from ATOMIC (Sap et al., 2019a)
(eh, r, et): (Dana speeds on the highway., xAttr, risky)
Q: Dana speeds on the highway. Dana is seen as A1: considerate A2: risky(✓) A3: lazy
QA from ConceptNet (Speer et al., 2017)
(eh, r, et): (pentode, IsA, vacuum tube)
Q: pentode is a [MASK]
A1: ascocarp A2: girls footwear A3: vacuum tube(✓)
QA from WikiData (Vrandecić and Krötzsch, 2014)
(eh, r, et): (badminton, IsA, type of sport)
Q: badminton is a [MASK]
A1: fable A2: juvenile justice A3: type of sport(✓)
QA from WordNet (Miller, 1995)
(eh, r, et): (princewood, PartOf, genus Cordia)
Q: princewood is part of [MASK]
A1: shaddock A2: genus Cordia(✓) A3: family Columbidae
2019a) focuses on inferential knowledge organized as typed if-then relations with variables (e.g., "if  $X$  pays  $Y$  a compliment, then  $Y$  will likely return the compliment"). ConceptNet (Speer et al., 2017) mainly consists of taxonomic and lexical knowledge (e.g., RelatedTo, Synonym, and IsA) and physical commonsense knowledge (e.g., MadeOf and PartOf). WikiData (Vrandecic and Krötzsch, 2014) is a general-domain KG which has a close relation with Wikipedia. WordNet (Miller, 1995) is a large lexical source of words and taxonomical system.
D Dataset for Zero-shot Fusion
For zero-shot fusion training, we use balanced mixture of synthetic QA from different KGs by random sampling. The statistics of dataset for zero-shot fusion is shown in Table 6. For validation dataset, we balance between the ATOMIC, ConceptNet and WordNet due to the lack of synthetic QA validation dataset from WikiData.
Table 5: Synthetic QA examples. We use templates to convert a question  $(e^{head},r)$  into a natural language.
KG	Train	Validation	Total
+ATOMIC	2,500	2,500	5,000
+ConceptNet	2,500	2,500	5,000
+WikiData	2,500	2,229	4,729
+WordNet	2,500	2,500	5,000
Total	10,000	9,729	19,729
Table 6: Statistics of the dataset for zero-shot fusion
Notation	Meaning
(ehead,r,etal)	Triple of KG (head entity, relation, tail entity)
Qi	Natural language Question of sample i
Ai={Ai,1,...,Ai,m}	A set of answer options of sample i, Ai,j denotes j-th answer option of sample i(1 ≤ j ≤ m)
Ti={Ti,1,...,Ti,m}	Input sequences generated by concatenation of Qi and Ai
wt	A word t-th token in the sequence Ti,j
label	the index of the correct answer for sample
DQA	Synthetic QA generated by KG k, 1 ≤ k ≤ K
Nk	The number of samples for DQA, 1 ≤ k ≤ K
θ	Parameters for pre-trained LM
ΦQA	Parameters for the expert adapter of KG k, 1 ≤ k ≤ K
ΦKGC	Parameters for the KG-Classifier adapter
ΨQA	Parameters for the fusion layer
l	The index of Transformer layer
WlQ	Query matrix of fusion layer in lth Transformer layer
WlK	Key matrix of fusion layer in lth Transformer layer
WlV	Value matrix of fusion layer in lth Transformer layer
hlPLM	Hidden representation of PLM parameterized by θ in lth Transformer layer
hEkl	Hidden representation of expert adapter parameterized by ΦQAk in lth Transformer layer
hlKGC	Hidden representation of KG-Classifier adapter parameterized by ΦKGC in lth Transformer layer
E KG-Classification Dataset
We suggest KG-Classification dataset  $\mathcal{D}_{KGC}$  for KG-Classifier adapter training. The example of transformation from synthetic QA dataset  $\mathcal{D}_{QA}$  is shown in Table 8. The dataset size is equal to the whole dataset of synthetic QA (refer to Table 4).
Table 7: Notations and their meanings
QA → KG-Classification ATOMIC
Q: Dana speeds on the highway. Dana is seen as A1: considerate A2: risky(✓) A3: lazy
S: Dana speeds on the highway. Dana is seen as risky. A: Atomic
QA → KG-Classification ConceptNet
Q: pentode is a [MASK] A1: ascocarp A2: girls footwear A3: vacuum tube(✓)
S: pentode is a vacuum tube. A: ConceptNet
QA → KG-Classification WikiData
Q: badminton is a [MASK] A1: fable A2: juvenile justice A3: type of sport(✓)
S: badminton is a type of sport. A: WikiData
QA → KG-Classification WordNet
Q: princewood is part of [MASK] A1: shaddock A2: genus Cordia(✓) A3: family Columbidae
S: princewood is part of genus Cordia. A: WordNet
Table 8: KG-Classification examples from synthetic QA dataset of each KG
F Zero-shot architecture with parameters
We describe the illustration of the zero-shot fusion architecture with parameters in Figure 7.