Venue: NAACL 2022

# Modularized Transfer Learning with Multiple Knowledge Graphs

for Zero-shot Commonsense Reasoning

Yu Jin Kim1, Beongwoo Kwak1, Youngwook Kim1, Reinald Kim Amplayo2*

Seung-won Hwang3 and Jinyoung Yeo1+\({}^{1}\)Yonsei University, Seoul, Korea 2Google Research, London, UK

Footnote 1: Work done while at the University of Edinburgh.

Footnote †: Corresponding author

###### Abstract

Commonsense reasoning systems should be able to generalize to diverse reasoning cases. However, most state-of-the-art approaches depend on expensive data annotations and overfit to a specific benchmark without learning how to perform general semantic reasoning. To overcome these drawbacks, zero-shot QA systems have shown promise as a robust learning scheme by transforming a commonsense knowledge graph (KG) into synthetic QA-form samples for model training. Considering the increasing type of different commonsense KGs, this paper aims to extend the zero-shot transfer learning scenario into multiple-source settings, where different KGs can be utilized synergetically. Towards this goal, we propose to mitigate the loss of knowledge from the interference among the different knowledge sources, by developing a modular variant of the knowledge aggregation as a new zero-shot commonsense reasoning framework. Results on five commonsense reasoning benchmarks demonstrate the efficacy of our framework, improving the performance with multiple KGs.

languageresource

modularized framework that aims to learn multiple expert models for KGs, then conduct zero-shot fusion to allow collaboration among KGs. For this purpose, we leverage AdapterFusion Pfeiffer et al. (2021) where multiple tiny modules between Transformer blocks called adapters Houlsby et al. (2019) can be combined after independent training, thus allowing a continual integration of the adapters without retraining the entire framework. Specifically, we treat the adapters as different KG-specific experts, and combine them using an attention-like fusion module. To improve the fusion of adapters, we suggest a KG-alignment adapter that guides to the apt _expert adapters_. Here, we use KGs in three different synthetic supervision training: (1) KG-specific QA datasets to train the KG-specific expert adapters, (2) a KG classification datasets to train the KG-alignment adapter, and (3) a balanced mixture of KG-specific QA datasets to train the fusion module. Our modularized method alleviates the interference between different KGs, which is the pitfall of MTL from our empirical observation, and thus combines multiple KGs into a synergetic zero-shot framework.

Our contributions are: (1) We suggest a simple, yet effective KG modularization strategy for the use of multiple KGs in commonsense reasoning. (2) We then explore the use of AdapterFusion Pfeiffer et al. (2021) for better knowledge aggregation based on the KG modularization in zero-shot setting. We believe that such modularized transfer learning is critical to using different knowledge sources synergetically against interference between them. (3) In extensive experiments on various commonsense reasoning benchmarks, our framework achieves significant improvements over baselines using a single KG, even using multiple KGs, which implies the robustness in commonsense reasoning.

## 2 Related Work & Preliminaries

### Zero-shot Commonsense Reasoning

Many researchers have recently focused on building unsupervised models without any benchmark supervisions (_i.e._, zero-shot learning). In such zeroth shot setting, KGs are often used as an external resource for improving model prior (_e.g._, continually learned from pre-trained language models) Banerjee and Baral (2020); Bosselut and Choi (2019); Ma et al. (2021), especially for commonsense reasoning, as much existing work couples language models with neural/symbolic commonsense KGs.

However, most of existing work are either assuming the existence of the alignment information between tasks and KGs Banerjee and Baral (2020) or an integrated KG Ma et al. (2021). For example, ATOMIC\({}^{20}_{20}\) Hwang et al. (2021), a commonsense KG which incorporates tuples from ConceptNet and ATOMIC with new relations and further crowdsourcing, combines multiple KGs into a new integrated KG, but as widely known Ilievski et al. (2020); Hwang et al. (2021), heterogeneous schema between different KGs may limit triplets that can be integrated.1 Rather than such symbolic KG integration with the inevitable loss of knowledge, in this work, we explore the neural KG integration leveraging the multiple KGs without additional processing and alignment information between KG and task.

Footnote 1: Only 172K tuples of the 3.4M tuples and 5 relations of 36 relations in ConceptNet are integrated into ATOMIC\({}^{20}_{20}\).

### Transfer Learning with Modular Approaches

The idea of having specialized parameters, or so-called experts, has been widely studied to integrate multiple sources of knowledge via transfer learning. The adapter module Rebuffi et al. (2017); Houlsby et al. (2019) has been explored as one of such approaches, introducing a small number of task-specific parameters at every layer of pretrained language model (PLM) while sharing the parameters of underlying PLM which is fixed. To address the limitations of transfer learning due to high re-training cost, many works utilize the multiple adapter modules for individual tasks with different domains Puigcerver et al. (2020); Bapna et al. (2019); Ruckle et al. (2020); Madotto et al. (2021) considering each adapter to be an expert of each domain. Similar to our work, K-Adapter Wang et al. (2021) encodes factual and linguistic knowledge to each adapter, but in this paper, we further explore how to mitigate catastrophic forgetting or interference among multiple adapters for better knowledge transfer in zero-shot setting.

### Multi-Task Learning

MTL Liu et al. (2019); Zhang and Yang (2017); Caruana (1997) learns a shared representation while aggregating knowledge across multiple learning tasks, often leading to better generalization ability of a model. However, parametric aggregation of knowledge with MTL has following limitations: (1) retraining the full model when adding new 

tasks (Houlsby et al., 2019; Pfeiffer et al., 2021, 2020b) (2) catastrophic forgetting and interference between tasks leading to difficulties of solving each task equally well (Pilault et al., 2021; Wu et al., 2020; Yu et al., 2020) and (3) inconsistent effect (Lourie et al., 2021). To deal with these challenges, Mixture-of-Experts (MoE) is a parameterized generalization of ensembling techniques, which has been adapted for MTL with gating network trained to optimize each task (Ma et al., 2018). However, simple linear gating networks are too shallow and thus may destruct task knowledge for commonsense reasoning.

To address this problem, AdapterFusion (Pfeffer et al., 2021) has been proposed to fuse task specific parameters called adapters for the given target task leveraging attention-like mechanism. AdapterFusion aggregates adapters, which is trained independently for each task, in a non-destructive manner mitigating aforementioned MTL problems such as forgetting and interference between tasks. Recently, it has been used for zero-shot cross-lingual transfer framework (Pfeiffer et al., 2020c; Wang et al., 2021b), which motivates our work to transfer multi-source knowledge with less interference for zero-shot commonsense reasoning.

## 3 Modularized Zero-shot Framework

In our setup, we repurpose synthetic QA generation (Ma et al., 2021) for the task of knowledge-driven zero-shot learning for commonsense reasoning, _i.e._, we transform a KG into multiple \((Q_{i},A_{i})\) pairs where \(Q_{i}\) is a natural language question and \(A_{i}=\{A_{i,1},...,A_{i,m}\}\) is the set of options with \(m\) answer candidates. Specifically, given a triple \((e^{head},r,e^{tail})\) in a KG, where \(e^{head}\), \(e^{tail}\) and \(r\) denote head/tail entity and relation respectively, we transform \(e^{head}\) and \(r\) into a natural language question \(Q_{i}\) using templates. For the option set \(A_{i}\), we use the combination of the correct answer \(e^{tail}\) and \(m-1\) distractors which are tail entities from other triples sampled randomly (Ma et al., 2021). Details are described in Appendix B.

Formally, we denote \((Q_{i},A_{i})\) as one QA sample. The goal is to learn a QA model from the synthetic QA sample. In a downstream task (_e.g._, reasoning benchmarks such as SocialQA and CommonsenseQA), we need to predict answers given non-synthetic test samples \((Q^{test},A^{test})\). In the training stage, we are given \(K\) KG-driven datasets \(\{\mathcal{D}_{QA}^{k}\}_{k=1}^{K}\) from \(K\) different KGs, where \(\mathcal{D}_{QA}^{k}\) is a dataset with \(N_{k}\) samples for KG \(k\) as follows:

\[\mathcal{D}_{QA}^{k}=\{(Q_{i},A_{i},label)\}_{i=1}^{N_{k}} \tag{1}\]

where \(label\) is the index of the correct answer for each sample. In this work, as shown in Table 1, we generate four synthetic QA datasets from ATOMIC, ConceptNet, WikiData, and WordNet (More details are in Appendix C).

For effective use of multiple KGs at once with less interference, we present a modularized framework, which is a novel approach to knowledge transfer for the zero-shot setting as shown in Figure 1. As a modular approach, we train the multiple KG-specific adapters (_expert adapters_) with each dataset from KG. Based on these pre-trained adapters, we use a zero-shot fusion method to aggregate knowledge of each adapter leveraging AdapterFusion (Pfeiffer et al., 2021) as a base architecture with the balanced mixture of each KG dataset. Further, for better knowledge fusion, we suggest a KG-alignment aware adapter (_KG-Classifier adapter_) as a guide for detecting alignment with given sample in zero-shot reasoning. Here, we utilize KG classification dataset by verifying the synthetic QAs. Algorithm 1 in Appendix outlines the overall process of our proposed framework. We summarize the notations in Appendix A.

### KG Modularization

First, we modularize the KGs to preserve their intrinsic knowledge. Considering the importance of using a suitable and well-aligned KG (Ma et al., 2019, 2021) on a downstream task, the subtle difference between each KG should be learned by the model without any interference from each other. Accordingly, we adopt the adapter module (Houlsby et al., 2019) which repurposes a pretrained language model (PLM) to incorporate each

\begin{table}
\begin{tabular}{l} \hline
**QA from ATOMIC (Sap et al., 2019a)** \\ \hline  & Q: Dana speeds on the highway. Dana is seen as AI: considerable **Az: risky(✓)** A3: lazy \\ \hline  & **QA from ConceptNet (Sperer et al., 2017)** \\ \hline  & Q: pentole is a [MASK] \\ Al: isocarp A2: girls footwear **A3: vacuum tube(✓)** \\ \hline   **QA from WikiData (Vrandecić and Krótzsch, 2014)** \\ \hline  & Q: badmonits in **MASKI** \\ \hline  & A1: fiable A2: juvenile justice **A3: type of sport(✓)** \\ \hline  & **QA fromWordNet (Miller, 1995)** \\ \hline  & Q: princewood is part of [MASK] \\ \multicolumn{2}{l}{A1: shaddock **A2: genus Cordia(✓)**} \\ \cline{2-2} \end{tabular}
\end{table}
Table 1: Synthetic QA examples. We use templates to convert \((e^{head},r)\) into a natural language sentence.

 

KG as tiny modules in between Transformer blocks. Specifically, as illustrated in Figure 2 (except for green area), the adapter training strategy involves injecting new layers (parameterized by \(\Phi\)) into the original PLM (parameterized by \(\theta\)). The weights of the original PLM are untouched, while the new adapter layers are initialized at random. Formally, we call each adapter trained with \({\cal D}^{k}_{QA}\) as an _expert adapter_ for KG \(k\), parameterized by \(\Phi^{k}_{QA}\).

When a QA sample \((Q_{i},A_{i})\) is given for dataset \({\cal D}^{k}_{QA}\), we first concatenate question \(Q_{i}\) and each answer option \(A_{i}=\{A_{i,1},...,A_{i,m}\}\) to generate input sequences \(T_{i}=\{T_{i,1},...,T_{i,m}\}\). Then, we compute a score \(S_{i,j}\)Ma et al. (2021) for the answer candidate \(A_{i,j}\) is computed as follows:

\[S_{i,j}=-\frac{1}{|T_{i,j}|}\sum_{t=1}^{|T_{i,j}|}logP(w_{t}|...w_{t-1},w_{t+1 }...;\theta,\Phi) \tag{2}\]

where \(w_{t}\) is a word token in the sequence \(T_{i,j}\) and \(P\) is the conditional probability from Transformer blocks parameterized by \(\theta\) and \(\Phi\). To train the adapter \(\Phi^{k}_{QA}\), we use the marginal ranking loss Ma et al. (2021) as follows:

\[{\cal L}_{QA}=\frac{1}{m}\sum_{i=1}^{N_{k}}\sum_{\begin{subarray}{c}j=1\\ j\neq label\end{subarray}}^{m}max(0,\eta-S_{i,label}+S_{i,j}) \tag{3}\]

where \(\eta\) represents the margin.

\[\Phi^{k}_{QA}\leftarrow\operatorname*{argmin}_{\Phi}{\cal L}_{QA}({\cal D}^{ k}_{QA};\theta,\Phi) \tag{4}\]

where KG-invariant parameters \(\theta\) are fixed and only KG-dependent parameters \(\Phi^{k}_{QA}\) are learned, which enables to store the corresponding knowledge separately without any interference. Further, we can parallelize the training of adapter for all KGs. The efficiency of adapter training allows our modularization to be more scalable.

### Zero-shot Fusion

Once the expert adapters are learned, we combine the knowledge from each expert adapter using an attention-like mechanism. We present a novel fusion strategy as shown in Figure 2, which is referred to as the zero-shot fusion. In contrast to AdapterFusion Pfeiffer et al. (2021) where the focus is learning to transfer knowledge to a specific target task, our zero-shot fusion aims to generalize this transfer to any arbitrary target task. Specifically, the zero-shot fusion parameters \(\Psi\) learn to combine fixed expert adapters which are parameterized by \(\Phi^{1}_{QA},...,\Phi^{K}_{QA}\). In each Transformer layer \(l\) of PLM with the injected fusion layer, the zero-shot fusion parameters \(\Psi_{QA}\) consist of query, key, and value matrices, denoted by \(\textbf{W}^{Q}_{l}\), \(\textbf{W}^{K}_{l}\), and \(\textbf{W}^{V}_{l}\) respectively. These parameters are used to learn the balancing between the representation of each _expert adapters_ through attention-like mechanism. While fixing both the parameters \(\theta\) and all expert adapters \(\Phi^{1}_{QA},...,\Phi^{K}_{QA}\), the only trainable weights \(\Psi_{QA}\) on the fusion layer learns to combine the knowledge from different \(K\)_expert adapters_ by using the subset of \(\{{\cal D}^{k}_{QA}\}^{K}_{k=1}\) by random sampling. Here, we balance the ratio between the \(K\) knowledge-driven datasets as \(N\) samples (details are in Appendix D). Formally,

\[\Psi_{QA}\leftarrow\operatorname*{argmin}_{\Psi}\sum_{k=1}^{K}{\cal L}_{QA}( {\cal D}^{k}_{QA};\theta,\{\Phi^{k}_{QA}\}^{K}_{k=1},\Psi) \tag{5}\]

where \(\Psi\) refers to the initialized zero-shot fusion parameters.

More specifically, in the \(l\)-th Transformer layer, let \(h^{I}_{PLM}\) and \(h^{k,f}_{E}\) be the representations of underlying PLM parameterized by \(\theta\) and an _expert adapter_ parameterized by \(\Phi^{k}_{QA}\), respectively. Then, using the hidden representation \(h^{l}_{PLM}\) of PLM as a query, the fusion layer performs the attention-like function as follows:

\[\textbf{K}_{l},\textbf{V}_{l} =[h^{1,l}_{E},...,h^{K,l}_{E}] \tag{6}\] \[\textbf{Q}_{l} =h^{l}_{PLM}\] (7) \[\textbf{z}_{l} =\text{Attention}(\textbf{Q}_{l}\textbf{W}^{Q}_{l},\textbf{K}_{ l}\textbf{W}^{K}_{l},\textbf{V}_{l}\textbf{W}^{V}_{l}) \tag{8}\]

Figure 1: Illustration of the proposed modularized framework for zero-shot commonsense reasoning. Each colored square represents different KGs. Not only for KG modularization, we re-use a set of synthetic QA datasets from the multiple KGs for the purpose of KG classification and zero-shot fusion, which enables better knowledge aggregation.

 

where \({\bf z}_{l}\) is passed to the next Transformer layer. Given a sample, the zero-shot fusion learns the suitable balancing parameters between the _expert adapters_ for zero-shot reasoning. Eventually, it learns to identify generalizability across common-sense reasoning tasks.

### KG-Classifier Adapter

AdapterFusion uses the PLM hidden representation \(h^{I}_{PLM}\) as a query which is learned when training on a specific downstream task. In our zero-shot setting, however, we use a mixture of synthetic QA for fusion training, which is not exactly a training dataset for a downstream task. To compensate for this issue, we present _KG-Classifier adapter_, which is a KG alignment-aware adapter, which is motivated from the fact that the ability to find which KG has an alignment with the given sample can be helpful as a role of providing a guidance for better performance Ma et al. (2019, 2021).

Specifically, we propose a novel training task for _KG-Classifier adapter_, which requires predicting the KG for the given sample of the task. For that, given \(\{\mathcal{D}^{k}_{QA}\}_{k=1}^{K}\), we first transform a QA sample \((Q_{i},A_{i})\) into a new KG classification sample \([Q_{i};A_{i,label}]\) where \([;]\) is the concatenation. Then, we obtain a new label \(y_{i}\in\{0,1\}^{K}\) indicating the corresponding KG source. The samples are in Appendix E. Formally, KG classification dataset \(\mathcal{D}_{KGC}\) is defined as:

\[\mathcal{D}_{KGC}=\{([Q_{i};A_{i,label}],y_{i})\}_{i=1}^{M} \tag{9}\]

where \(M\) is the total size of \(\{\mathcal{D}^{k}_{QA}\}_{k=1}^{K}\).

Based on \(\mathcal{D}_{KGC}\), we learn the _KG-classifier adapter_ parameterized by \(\theta\) and \(\Phi_{KGC}\). First, a classification sample \(i\) is encoded into \(h_{CLS}\in\mathbb{R}^{H}\) then scored as \(\hat{y}_{i}\in\mathbb{R}^{K}\) with a linear layer \(W_{KGC}\in\mathbb{R}^{K\times H}\), _i.e._, \(\hat{y}_{i}=W_{KGC}h_{CLS}\). Once \(\hat{y}_{i}\) is normalized by a softmax layer, the network is trained to minimize the cross-entropy loss \(\mathcal{L}_{KGC}\) between the prediction \(\hat{y}_{i}\) and its ground truth \(y_{i}\):

\[\Phi_{KGC}\leftarrow\operatorname*{argmin}_{\Phi}\sum_{i=1}^{M}\mathcal{L}_{ KGC}(y_{i},\hat{y}_{i};\theta,\Phi) \tag{10}\]

We propose to use the representation of _KG-Classifier adapter_ as a query in attention-like mechanism, referred to as the zero-shot fusion with _KG-Classifier adapter_. That is, using the hidden representation \(h^{I}_{KGC}\) of a _KG-Classifier adapter_ parameterized by \(\Phi_{KGC}\) as a query, we substitute \(\textbf{Q}_{l}\) in Eq. (11) as follows:

\[\textbf{Q}_{l}=h^{I}_{KGC} \tag{11}\]

The overall zero-shot fusion architecture including _KG-Classifier_ is illustrated in Figure 2.

## 4 Experiments

In this section we evaluate the efficacy of our framework on five commonsense reasoning tasks. We denote _KG-Classifier adapter_ by _KG-C adapter_.

### Experimental Settings

All our experiments are conducted in a zero-shot setting, in which the models do not have access to the official training data or labels of the benchmark. For the evaluation, we use the validation set of each benchmark2, however, the validation set of each benchmark can be role as an test set since it is not used for hyperparameter tuning or model selection. We use accuracy as a metric.

Footnote 2: Since the official test sets are not publicly available

#### 4.1.1 Benchmarks

We evaluate our proposed framework on five question-answering benchmarks for commonsense reasoning: SocialIQA (SIQA) Sap et al. (2019b), CommonsenseQA (CSQA) Talmor et al. (2018), Abductive NLI (a-NLI) Bhagavatula et al. (2020), PhysicalIA (PIQA) Bisk et al. (2020), and WinGrande (WG) Sakaguchi et al. (2020). Each commonsense benchmark evaluates a specific kind of knowledge: social commonsense for SIQA, concept-level commonsense for CSQA, abductive

Figure 2: Illustration of the zero-shot fusion architecture with _KG-Classifier adapter_. Each colored circle represents _expert adapters_, except the black circle which denotes _KG-Classifier adapter_. \(*\) indicates the fixed layer. Details are in Appendix F



reasoning for a-NLI, physical commonsense for PIQA, and pronoun resolution ability for WG.3 The details are presented in Appendix G.

Footnote 3: Some benchmarks have a strong alignment with a certain KG due to its construction strategy: SIQA-ATOMIC, and CSQA-ConceptNet. To make a direct comparison with Ma et al. (2021), we use the same KGs to generate data samples.

#### 4.1.2 Baselines

We compare our framework with the following baselines. First, to show the characteristics of each benchmark, we use the random or the most frequent label as _Random_ and _Majority_ baseline, respectively. RoBERTa-L and GPT2-L is the performance of each PLM without any fine-tuning. Also, as the baseline for the unsupervised learning model using KGs, we report the performance of Self-talk Shwartz et al. (2020), COMET-DynaGen Bosselut and Choi (2019), SMLM Banerjee and Baral (2020) as presented in original papers.

For further analysis in SS4.4 and SS4.5, we set the following models that are pre-trained on the synthetic QA datasets from KGs as baselines:

* **Single-Task Learning (STL)**: The model is pre-trained on a synthetic QA dataset generated from a single KG. Specifically, we experiment two architectural choices: PLM (STLPLM) and PLM with adapters (STL-Adapter). For each architecture, there are four STL models for each of synthetic QA datasets derived from ATOMIC, ConceptNet, WikiData, and WordNet. We note that the trained STL-Adapter is an _expert adapter_ from a specific KG in our framework. The performance of each STL baseline is shown in Appendix I Table 9 and Table 10.
* **Multi-Task Learning (MTL)**: The model is pre-trained on multiple synthetic QA datasets, each of which is generated from a KG. We experiment with a PLM trained on all four aforementioned synthetic QA datasets. We note that the difference between STL-PLM and MTL is whether to use one synthetic QA dataset or multiple synthetic QA datasets for its training.

#### 4.1.3 Implementations

We employ RoBERTa-L Liu et al. (2019b) from Hugging Face's transformers toolkit for all experiments. We follow the default settings from Ma et al. (2021). Our implementation uses Adapter Houlsby et al. (2019) and AdapterFusion Pfeiffer et al. (2021) as a base model architecture from AdapterHub Pfeiffer et al. (2020). We run our experiments with three different random seeds. The implementation details are described in Appendix H.

### Main Results

Table 2 shows the zero-shot evaluation results on five benchmark datasets. Generally, zero-shot fusion scores higher than the baselines across all benchmarks, and further, zero-shot fusion shows the best performance in all benchmarks except WG. We note that although Ma et al. (2021) uses the synthetic QA dataset after sample filtering, our method achieves comparable performance with the best performance in WG, even with the raw dataset. Also, the average score of all evaluation benchmarks (the

\begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline
* & 50.0 & 20.0 & 50.0 & 33.3 & 50.0 & 40.7 \\ Majority &
- & 50.8 & 20.9 & 50.5 & 33.6 & 50.4 & 41.2 \\ GPT2-L &
- & 56.5 & 41.4 & 68.9 & 44.6 & 53.2 & 52.9 \\ RoBERa-L &
- & 65.5 & 45.0 & 67.6 & 47.3 & 57.5 & 56.6 \\ Self-talk Shwartz et al. (2020) &
- & 32.4 & 70.2 & 46.2 & 54.7 &
- \\ COMPET-Dynden Bosseltu and Choi (2019) & AT &
- & 50.1 &
- \\ SMLM Banerjee and Baral (2020) & * & 65.3 & 38.8 &
- & 48.5 &
- \\ RoBERTa-L MR MR Lai et al. (2021) & AT & 70.8 & 64.2 & 72.1 & 63.1 & 59.2 & 65.9 \\ RoBERTa-L MR MR Lai et al. (2021) & CN,WD,WN & 70.0 & 67.9 & 72.0 & 54.8 & 59.4 & 64.8 \\ RoBERTa-L MR MR Lai et al. (2021) & Whole & 70.5 & 67.4 & 72.4 & 63.2 & **60.9** & **66.9** \\ \hline MTL & Whole & 69.8 (\(\pm\) 0.5) & 66.0 (\(\pm\) 0.9) & 71.2 (\(\pm\) 0.8) & 62.2 (\(\pm\) 1.0) & 59.5 (\(\pm\) 0.2) & 65.7 \\ \hline
**zero-shot fusion w/o KG-C adapter** & Whole & 72.3(\(\pm\)0.4) & 67.9(\(\pm\)0.2) & **73.1**(\(\pm\)0.4) & 65.9(\(\pm\)0.5) & 59.7(\(\pm\)0.2) & 67.8 \\
**zero-shot fusion w/ KG-C adapter** & Whole & **72.5**(\(\pm\)0.2) & **68.2**(\(\pm\)0.2) & 72.9(\(\pm\)0.4) & **66.6**(\(\pm\)0.1) & 60.8(\(\pm\)0.1) & **68.2** \\ \hline \hline \end{tabular}
\end{table}
Table 2: Zero-shot evaluation results with different combinations of models and knowledge sources across five commonsense tasks. AT, CN, WD and WN represent ATOMIC, ConceptNet, WikiData and WordNet, respectively. Whole represents the combination of AT, CN, WD and WN. Bold text indicates the best performance on each benchmark. RoBERTa-L (MR) used the synthetic dataset after filtering, while we use the raw version. SMLM (*) used different KG which has strong alignment with each task (_e.g._AT for SIQA).

 

last column of Table 2) shows that zero-shot fusion has generalisability in commonsense reasoning.

In addition, zero-shot fusion achieves consistent improvements over MTL. These results indicate that our proposed zero-shot fusion method attributes to fusing the knowledge of multiple KGs more synergetically regardless of the task.

Moreover, as an ablation, we compare the zero-shot fusion with and without _KG-C adapter_ to explore the efficacy of the _KG-C adapter_. We can observe that zero-shot fusion with _KG-C adapter_ improves the average accuracy by 0.4%, which implies that the use of _KG-C adapter_ improves the overall performance and makes our method generalize better on most of the evaluation benchmarks.

### Impact of the KG-Classifier Adapter

To assess the effects of the _KG-C adapter_ itself, we visualize and compare the final layer [CLS] token representation between PLM and _KG-C adapter_. Figure 3 shows t-SNE (Van der Maaten and Hinton, 2008) plots of all representation of five benchmark datasets. In this figure, every sample is mapped into a 1024-dimensional feature space through RoBERTa-L model and projected back into a two-dimensional plane by t-SNE. We can observe that _KG-C adapter_ can separate the samples of different benchmarks well despite being unseen data. It verifies that KG-awareness acquired with the KG classification task is beneficial to categorize the given sample. The _KG-C adapter_ can thus generate a relevant KG-aware query for a given sample and help to fuse representations from suitable _expert adapters_ in our proposed framework.

Further, we explore how the _KG-C adapter_ affects zero-shot fusion which is based on an attention-like mechanism Pfeiffer et al. (2021) compared to zero-shot fusion without _KG-C adapter_. Here, while zero-shot fusion without _KG-C adapter_ simply uses the representation of PLM as a query, zero-shot fusion with _KG-C adapter_ leverages the representation of _KG-C adapter_. To illustrate this strength, we visualize the attention probability of [CLS] token from each fusion layer as a representative in Figure 4. The column of the darker cell indicates the adapter that has the bigger influence on the fused representation. We can observe that zero-shot fusion with _KG-C adapter_ fuses the knowledge from different experts with a subtle difference rather than focusing on a single expert severely. This implies that _KG-C adapter_ enables the delicate balancing between multiple knowledge sources based on the KG-alignment awareness, which leads to performance improvements in commonsense reasoning tasks. Interestingly, both cases have the ability not to focus on the _expert adapter_ based on WikiData, which can be seen as a redundant expert.4 This observation would benefit from the further study that explores the optimal combination of KGs by expert selection or rejection.

Footnote 4: The zero-shot fusion with KG-C adapter using AT, CN, and WN shows the best average performance in Table 10.

### Mitigating Interference

In this experiment, we compare the amount of interference in the MTL and zero-shot fusion with _KG-C adapter_. We propose a novel evaluation metric, the _interference ratio_, which is the percentage of the incorrectly predicted samples by the multi-KG models among the correctly predicted samples from the STL models in common.

Using the interference ratio, we can precisely compare the negative effects of multi-KG models

Figure 4: Comparison of attention probability between zero-shot fusion with/without _KG-C adapter_. The x- and y-axis indicate _expert adaptters_ and the fusion layer number in RoBERTa-L, respectively. The darker color indicates higher attention probability in fusion layer.

Figure 3: t-SNE visualization of the hidden representation from (a) PLM and (b) _KG-C adapter_. Each color denotes the five different benchmark samples.

 

on knowledge aggregation since the only reason to get the correct samples wrong is the interference caused by learning with additional KGs. We present the interference ratio of the models on five benchmark datasets in Figure 5. This figure shows that MTL has the higher interference ratio than the competing models across all benchmarks. Our method achieves a substantially better ratio, especially when _KG-C adapter_ is used. This demonstrates the efficacy of our framework in mitigating interference between knowledge, which is one of the major problems of MTL.

### Visualization of Knowledge Aggregation

To verify the ability of our model to aggregate different types of KGs, we compare the relative performance gains of MTL and zero-shot fusion with _KG-C adapter_ when increasing the number of KGs. The performance of all KG-combinations for each framework is presented in Table 9 and Table 10. We visualize the improvement of performance for five benchmark development sets, leveraging heatmaps in Figure 6. Here, for the sake of brevity, we denote our framework with _KG-C adapter_ as our method.

For MTL in Figure 6 (a), the color of the cell denotes the relative improvement of MTL with the combination of KGs over the best performance among the STL-PLM of KGs. Also, for our method in Figure 6 (b), the relative improvement is measured based on the best performance among the STL-Adapter of KGs, considering the difference of the base architecture for MTL (i.e. PLM) and zero-shot fusion (i.e. PLM with adapter). The green and red colors denote the increase and decrease of performance, respectively, when using multiple KGs together. The greener color on the cells indicates that the approach benefits from an increasing number of KGs, which implies aggregating knowledge successfully.

In Figure 6, while the MTL tends to show the decrease of the performance when more KGs are utilized for training, our method obtains relative performance improvement across most of benchmarks. In both framework, the slightly degraded performance of the combination of KGs without ATOMIC could be due to the strong alignment between ATOMIC and SIQA. Except for the above case, we can observe that as more KGs are leveraged, the color of the cell gets greener, which implies that our method gains more advantages for better performance. This demonstrates that our method enables knowledge aggregation for multiple KGs synergetically.

## 5 Conclusion

Despite the existence of various types of commonsense KGs, utilizing multiple KGs has not been explored enough in the commonsense reasoning field. Motivated by this, this paper proposes a modularized transfer learning framework to fuse the knowledge from multiple KGs efficiently for zero-shot commonsense reasoning. Our framework consists of KG modularization for _expert adapter_, zero-shot fusion and _KG-Classifier adapter_. Extensive experiments show that our framework obtains strong improvements over MTL on five commonsense reasoning benchmarks.

In the future, our work can be extended to adapt our methods to further various multiple KGs with studies of appropriate scale for KG modularization. In addition, based on our hypothesis that the existence of an optimal combination, we can explore the study for the optional use of modularized KG experts for the best transfer learning.

Figure 5: Interference ratio of multi-KG models on five benchmarks. The lower indicates less interference.

Figure 6: Relative improvement upon the STL on five benchmarks. The x- and y-axis indicate the benchmark and the combination of the KGs, respectively. The value of each cell indicates the relative performance improvement of using multiple KGs over the highest performance among STLs. The green and red colors denote the improvement or decrease of relative performance, respectively.

 



## Acknowledgements

This work was partly supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2020-0-01361, Artificial Intelligence Graduate School Program (Yonsei University)) and (No. 2022-0-00077, AI Technology Development for Commonsense Extraction, Reasoning, and Inference from Heterogeneous Data) and the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2021-11-1055). Jinyoung Yeo is a corresponding author.

## References

* Banerjee and Baral [2020] Pratyay Banerjee and Chitta Baral. 2020. Self-supervised knowledge triplet learning for zero-shot question answering. _EMNLP_.
* Bapna et al. [2019] Ankur Bapna, Naveen Arivazhagan, and Orhan Firat. 2019. Simple, scalable adaptation for neural machine translation. _EMNLP_.
* Bhagavatula et al. [2020] Chandra Bhagavatula, Ronan Le Bras, Chaitanya Malaviya, Keisuke Sakaguchi, Ari Holtzman, Hannah Rashkin, Doug Downey, Wen tau Yih, and Yejin Choi. 2020. Abductive commonsense reasoning. _ICLR_.
* Bisk et al. [2020] Yonatan Bisk, Rowan Zellers, Ronan Le bras, Jianfeng Gao, and Yejin Choi. 2020. Piqa: Reasoning about physical commonsense in natural language. In _AAAI_.
* Bosselut and Choi [2019] Antoine Bosselt and Yejin Choi. 2019. Dynamic knowledge graph construction for zero-shot commonsense question answering. _arXiv e-prints_, pages arXiv-1911.
* Caruana [1997] Rich Caruana. 1997. Multitask learning. _Machine learning_, 28(1):41-75.
* Chang et al. [2021] Ting-Yun Chang, Yang Liu, Karthik Gopalakrishnan, Behnam Hedayatnia, Pei Zhou, and Dilek Hakkani-Tur. 2021. Incorporating commonsense knowledge graph in pretrained models for social commonsense tasks. _EMNLP-Workshop_.
* Devlin et al. [2018] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. _arXiv preprint arXiv:1810.04805_.
* Houlsby et al. [2019] Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for nlp. In _PMLR_, pages 2790-2799. PMLR.
* Hwang et al. [2021] Jena D Hwang, Chandra Bhagavatula, Ronan Le Bras, Jeff Da, Keisuke Sakaguchi, Antoine Bosselt, and Yejin Choi. 2021. Comet-atomic 2020: On symbolic and neural commonsense knowledge graphs.
* Ilievisch et al. [2021] Filip Ilievisch, Alessandro Oltramari, Kaixin Ma, Bin Zhang, Deborah L McGuinness, and Pedro Szekely. 2021. Dimensions of commonsense knowledge. _Knowledge-Based Systems_.
* Ilievisch et al. [2020] Filip Ilievisch, Pedro Szekely, Jingwei Cheng, Fu Zhang, and Ehsan Qasemi. 2020. Consolidating commonsense knowledge. _arXiv preprint arXiv:2006.06114_.
* Liu et al. [2019a] Xiaodong Liu, Pengcheng He, Weizhu Chen, and Jianfeng Gao. 2019a. Multi-task deep neural networks for natural language understanding. _ACL_.
* Liu et al. [2019b] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019b. Roberta: A robustly optimized bert pretraining approach. _arXiv preprint arXiv:1907.11692_.
* Lourie et al. [2021] Nicholas Lourie, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. Unicorn on rainbow: A universal commonsense reasoning model on a new multitask benchmark. _arXiv preprint arXiv:2103.13009_.
* Ma et al. [2018] Jiaqi Ma, Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, and Ed H Chi. 2018. Modeling task relationships in multi-task learning with multi-gate mixture-of-experts. In _ACM SIGKDD_.
* Ma et al. [2019] Kaixin Ma, Jonathan Francis, Quanyang Lu, Eric Nyberg, and Alessandro Oltramari. 2019. Towards generalizable neuro-symbolic systems for commonsense question answering. _EMNLP_.
* Ma et al. [2021] Kaixin Ma, Filip Ilievsch, Jonathan Francis, Yonatan Bisk, Eric Nyberg, and Alessandro Oltramari. 2021. Knowledge-driven data construction for zero-shot evaluation in commonsense question answering.
* Madotto et al. [2021] Andrea Madotto, Zhaojiang Lin, Yejin Bang, and Pascale Fung. 2021. The adapter-bot: All-in-one controllable conversational model.
* Malaviya et al. [2020] Chaitanya Malaviya, Chandra Bhagavatula, Antoine Bosselt, and Yejin Choi. 2020. Commonsense knowledge base completion with structural and semantic context. In _AAAI_.
* Miller [1995] George A Miller. 1995. Wordnet: a lexical database for english. _Communications of the ACM_.
* Mitra et al. [2019] Arindam Mitra, Pratyay Banerjee, Kuntal Kumar Pal, Swaroop Mishra, and Chitta Baral. 2019. Exploring ways to incorporate additional knowledge to improve natural language commonsense question answering. _arXiv preprint arXiv:1909.08855_.
* Pfeiffer et al. [2021] Jonas Pfeiffer, Aishwarya Kamath, Andreas Ruckle, Kyunghyun Cho, and Iryna Gurevych. 2021. Adapterfusion: Non-destructive task composition for transfer learning. _EACL_

Jonas Pfeiffer, Andreas Ruckle, Clifton Poth, Ashishwarya Kamath, Ivan Vulic, Sebastian Ruder, Kyunghyun Cho, and Iryna Gurevych. 2020a. Adapterhub: A framework for adapting transformers.
* Pfeiffer et al. (2020) Jonas Pfeiffer, Edwin Simpson, and Iryna Gurevych. 2020b. Low resource multi-task sequence tagging--revisiting dynamic conditional random fields. _arXiv preprint arXiv:2005.00250_.
* Pfeiffer et al. (2020c) Jonas Pfeiffer, Ivan Vulic, Iryna Gurevych, and Sebastian Ruder. 2020c. Mad-x: An adapter-based framework for multi-task cross-lingual transfer. _arXiv preprint arXiv:2005.00052_.
* Pilault et al. (2021) Jonathan Pilault, Amine Elhattami, and Christopher Pal. 2021. Conditionally adaptive multi-task learning: Improving transfer learning in nlp using fewer parameters & less data. _ICLR_.
* Puigcerver et al. (2020) Joan Puigcerver, Carlos Riquelme, Basil Mustafa, Cedric Renggli, Andre Susano Pinto, Sylvain Gelly, Daniel Keysers, and Neil Houlsby. 2020. Scalable transfer learning with expert models. _arXiv preprint arXiv:2009.13239_.
* Rebuffi et al. (2017) Sylvestre-Alvise Rebuffi, Hakan Bilen, and Andrea Vedaldi. 2017. Learning multiple visual domains with residual adapters. _arXiv preprint arXiv:1705.08045_.
* Ruckle et al. (2020) Andreas Ruckle, Jonas Pfeiffer, and Iryna Gurevych. 2020. Multicqa: Zero-shot transfer of self-supervised text matching models on a massive scale. _EMNLP_.
* Sakaguchi et al. (2020) Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2020. Winogrande: An adversarial winograd schema challenge at scale. In _AAAI_.
* Sap et al. (2019a) Maarten Sap, Ronan Le Bras, Emily Allaway, Chandra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah A Smith, and Yejin Choi. 2019a. Atomic: An atlas of machine commonsense for if-then reasoning. In _AAAI_.
* Sap et al. (2019b) Maarten Sap, Hannah Rashkin, Derek Chen, Ronan LeBras, and Yejin Choi. 2019b. Socialiqa: Commonsense reasoning about social interactions. _EMNLP_.
* Shwartz et al. (2020) Vered Shwartz, Peter West, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2020. Unsupervised commonsense question answering with self-talk. _EMNLP_.
* Speer et al. (2017) Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. Conceptnet 5.5: An open multilingual graph of general knowledge. In _AAAI_.
* Talmor et al. (2018) Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2018. Commonsenseqa: A question answering challenge targeting commonsense knowledge. _NAACL_.
* Van der Maaten and Hinton (2008) Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. _JMLR_.
* Vandecicic and Krotzsch (2014) Denny Vrandecic and Markus Krotzsch. 2014. Wikidata: a free collaborative knowledgebase. _ACM_.
* Wang et al. (2021a) Ruize Wang, Duyu Tang, Nan Duan, Zhongyin Wei, Xuanjing Huang, Jianshu Ji, Guihong Cao, Davxin Jiang, and Ming Zhou. 2021a. K-adapter: Infusing knowledge into pre-trained models with adapters. _ACL_.
* Wang et al. (2021b) Xinyi Wang, Yulia Tsvetkov, Sebastian Ruder, and Graham Neubig. 2021b. Efficient test time adapter ensembling for low-resource language varieties.
* Wu et al. (2020) Sen Wu, Hongyang R Zhang, and Christopher Re. 2020. Understanding and improving information transfer in multi-task learning. _ICLR_.
* Yang et al. (2019) Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. _NIPS_.
* Yu et al. (2020) Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. 2020. Gradient surgery for multi-task learning. _NIPS_.
* Zellers et al. (2018) Rowan Zellers, Yonatan Bisk, Roy Schwartz, and Yejin Choi. 2018. Swag: A large-scale adversarial dataset for grounded commonsense inference. _EMNLP_.
* Zellers et al. (2019) Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? _ACL_.
* Zhang and Yang (2017) Yu Zhang and Qiang Yang. 2017. A survey on multi-task learning. _arXiv preprint arXiv:1707.08114_.

 

List of Notations

We summarize the notations used in this paper in Table 7.

## Appendix B Synthetic QA

We generate QA for four KGs (ATOMIC, ConceptNet, WikiData and WordNet) based on synthetic QA generation Ma et al. (2021) without sample filtering. We use the prefixes for relation of triplet as shown in Table 3 for generating synthetic QA (refer to Ma et al. (2021)). Table 4 shows the statistics of the synthetic QA dataset from KGs. The samples of synthetic QA with source triplet are shown in Table 5.

## Appendix C Commonsense Knowledge Graphs

A variety of KGs have been proposed to provide large-scale high quality collection of different commonsense knowledge types: ATOMIC Sap et al. (2019a) focuses on inferential knowledge organized as typed if-then relations with variables (_e.g._, "_if X pays Y a compliment, then Y will likely return the compliant_"). ConceptNet Speer et al. (2017) mainly consists of taxonomic and lexical knowledge (_e.g._, RelatedTo, Synonym, and IsA) and physical commonsense knowledge (_e.g._, MadeOf and PartOf). WikiData Vrandecic and Krotzsch (2014) is a general-domain KG which has a close relation with Wikipedia. WordNet Miller (1995) is a large lexical source of words and taxonomical system.

## Appendix D Dataset for Zero-shot Fusion

For zero-shot fusion training, we use balanced mixture of synthetic QA from different KGs by random sampling. The statistics of dataset for zero-shot fusion is shown in Table 6. For validation dataset, we balance between the ATOMIC, ConceptNet and WordNet due to the lack of synthetic QA validation dataset from WikiData.

\begin{table}
\begin{tabular}{c c c c} \hline \hline KG & Train & Validation & Total \\ \hline +ATOMIC & 2,500 & 2,500 & 5,000 \\ +ConceptNet & 2,500 & 2,500 & 5,000 \\ +WikiData & 2,500 & 2,229 & 4,729 \\ +WordNet & 2,500 & 2,500 & 5,000 \\ \hline Total & 10,000 & 9,729 & 19,729 \\ \hline \hline  & \multicolumn{1}{l}{} & \multicolumn{1}{r}{} & \\ \end{tabular}
\end{table}
Table 6: Statistics of the dataset for zero-shot fusion

\begin{table}
\begin{tabular}{c c c} \hline \hline relation & prefix \\ \hline xAttr & PersonX is seen as \\ xIntent & Before, PersonX wanted \\ xNeed & Before, PersonX needed to \\ xReact & As a result, PersonX felt \\ xWant & As a result, PersonX wanted to \\ xEffect & Personxxnk that \\ oReact & As a result, others felt \\ oWant & As a result, others wanted to \\ oEffect & Others then \\ Causes & can cause [MASK] \\ UsedFor & can be used for [MASK] \\ CapableOf & is capable of [MASK] \\ CausesDesire & causes desire for [MASK] \\ IsA. & is a [MASK] \\ SymbolOf & is a symbol of [MASK] \\ MadeOf & can be made of [MASK] \\ LocatedNear & is often located near [MASK] \\ Desires & desires [MASK] \\ AtLocation & can be found at [MASK] \\ HasProperty & has property [MASK] \\ PartOf & is part of [MASK] \\ HasFirstSubevent & starts by [MASK] \\ HasLastSubevent & ends by [MASK] \\ \hline \hline \end{tabular}
\end{table}
Table 4: Synthetic QA dataset statistics. Whole represents the combination of AT,CN,WD and WN.

\begin{table}
\begin{tabular}{c c c c} \hline \hline KG & Train & Validation & Total \\ \hline ATOMIC & 534,833 & 60,289 & 595,122 \\ CoreDetNet & 363,645 & 19,140 & 382,785 \\ WikiData & 42,342 & 2,229 & 44,571 \\ WordNet & 256,922 & 13,523 & 270,445 \\ \hline Whole & 1,197,742 & 95,181 & 1,292,923 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Prefixes used for synthetic QA dataset

\begin{table}
\begin{tabular}{c c c} \hline \hline
**QA from ATOMIC Sap et al. (2019a)** \\ \hline (\(e^{h},r,e^{t}\)): (Dana speeds on the highway, xAttr, risky) \\ Q: Dana speeds on the highway. Dana is seen as \\ A1: considerate **A2: risky(✓)** A3: lazy \\ \hline
**QA from ConceptNet Speer et al. (2017)** \\ \hline (\(e^{h},r,e^{t}\)): (pentode, Isa, vacuum tube) \\ Q: pentode is a [MASK] \\ A1: ascocarp A2: girls footwear **A3: vacuum tube(✓)** \\ \hline
**QA from WikiData Vrandecić and Krötzsch (2014)** \\ \hline (\(e^{k},r,e^{t}\)): (badminton, Isa, type of sport) \\ Q: badminton is a [MASK] \\ A1: fiable A2: juvenile justice **A3: type of sport(✓)** \\ \hline
**QA from WordNet Miller (1995)** \\ \hline (\(e^{h},r,e^{t}\)): (princewood, PartOf, genus Cordia) \\ Q: princewood is part of [MASK] \\ A1: shaddock **A2: genus Cordia(✓)** A3: family \\ Columbidae \\ \hline \hline \end{tabular}
\end{table}
Table 5: Synthetic QA examples. We use templates to convert a question (\(e^{head},r\)) into a natural language.

 



## Appendix E KG-Classification Dataset

We suggest KG-Classification dataset \({\cal D}_{KGC}\) for _KG-Classifier adapter_ training. The example of transformation from synthetic QA dataset \({\cal D}_{QA}\) is shown in Table 8. The dataset size is equal to the whole dataset of synthetic QA (refer to Table 4).

## Appendix F Zero-shot architecture with parameters

We describe the illustration of the zero-shot fusion architecture with parameters in Figure 7.

## Appendix G Commonsense Reasoning Benchmarks

**SocialIQA (SIQA)**[15] requires reasoning for emotional and social intelligence in everyday situations. Each QA consists of a context that comes from ATOMIC, a question which is based on the relations in ATOMIC, and 3 answer candidates. It contains 38,000 multiple-choice

Figure 7: Illustration of the zero-shot fusion architecture with parameters. Each colored circle represents _expert adapters_, except the black circle which denotes _KG-Classifier adapter_. \(*\) indicates the fixed layer.



questions, which is generated by crowdsourcing.

**CommonsenseQA (CSQA)**[12] evaluates a broad range of concept-level commonsense reasoning. Each multiple-choice question, answer and distractors are designed by crowdsourcing based on the ConceptNet.

**Abductive NLI (a-NLI)**[30] asks to infer the most plausible explanation based on the given causal situation to test abductive reasoning in narratives. Each sample consists of the beginning and the end of the story with two possible options to be an explanation for the given situation.

**PhysicalIQA (PIQA)**[45] requires physical commonsense reasoning to select the most sensible solution for the given goal among the two choices. Its dataset is comprised of over 16,000 training samples, 2K validation samples, and 3K test samples.

**HellaSWAG (HSWAG)**[79] is an evolved version of SWAG [80], which asks to infer the most proper story based on the given situation. The dataset consists of 70K questions with four answer options.

## Appendix H Implementation Details

In all our experiments, we use max sequence length 128, batch size 32, weight decay 0.01, adam \(\beta_{1}\) 0.9, adam \(\beta_{2}\) 0.99, adam epsilon \(1e^{-8}\), warm-up proportion 0.05, and margin 1.0. The experiments are conducted split across NVIDIA GeForce 3090 and NVIDIA RTX A5000.

### Baselines

The baseline models for STL-PLM and MTL are trained with learning rate \(1e^{-5}\) for single epoch.

### Adapter

For _expert adapters_, we use learning rate \(8e^{-5}\) after tuning in \(\{5e^{-6},8e^{-6},1e^{-5},5e^{-5},8e^{-5},1e^{-4}\}\). For _KG-Classifier adapter_, we use learning rate \(1e^{-5}\), batch size 64 for five epochs.

### Zero-shot fusion

After experiment with learning rates \(\{1e^{-5},8e^{-5}\}\), we empirically find that a learning rate of \(1e^{-5}\) works well on zero-shot fusion without/with _KG-Classifier adapter_, respectively. Here, we set the attention drop probability 0.1. As we used extremely smaller subset of the synthetic QA dataset, zero-shot fusions are trained for five epochs.

## Appendix I Knowledge aggregation of zero-shot fusion

In order to validate the efficacy on knowledge aggregation of zero-shot fusion over the STL, we present the results of each framework with various combination of KGs in Table 9 and Table 10.

 



\begin{table}
\begin{tabular}{l l c c c c c c c} \hline
**Model** & **KG** & **a-NLI** & **CSQA** & **PIQA** & **SIQA** & **WG** & **Avg.** \\ \hline  & AT & 71.3 & 66.5 & 71.1 & 64.4 & 60.3 & 66.7 \\ STL-Adapter & CN & 70.6 & 67.2 & 72.4 & 55.5 & 58.7 & 64.9 \\ WD & 66.8 & 61.6 & 69.9 & 51.8 & 58.5 & 61.7 \\ WN & 67.6 & 60.0 & 70.3 & 54.0 & 57.0 & 61.8 \\ AT,CN,WD,WN & 71.5 & 66.7 & 72.1 & 64.7 & 59.0 & 66.8 \\ \hline  & AT, CN & 71.9 & 68.1 & 72.8 & 65.4 & 59.7 & 67.6 \\  & AT, WD & 71.5 & 66.3 & 71.4 & 65.3 & 61.2 & 67.1 \\ zero-shot fusion w/_KGC-adapter_ & AT, WN & 72.5 & 67.5 & 73.1 & 66.4 & 59.5 & 67.8 \\ CN, WD & 70.8 & 68.1 & 72.1 & 55.3 & 59.3 & 65.1 \\ CN, WN & 71.0 & 67.6 & 73.0 & 54.8 & 59.1 & 65.1 \\ WD, WN & 67.8 & 62.6 & 71.3 & 52.9 & 57.1 & 62.3 \\ \hline  & AT, CN, WD & 72.3 & 68.0 & 72.9 & 66.2 & 60.5 & 68.0 \\ zero-shot fusion w/_KGC-adapter_ & AT, CN, WN & 72.5 & 68.7 & 73.8 & 66.8 & 60.4 & 68.4 \\ AT, WD, WN & 71.9 & 67.6 & 73.0 & 66.0 & 59.7 & 67.6 \\ CN, WD, WN & 69.6 & 67.6 & 73.1 & 53.7 & 59.5 & 64.7 \\ \hline zero-shot fusion w/_KGC-adapter_ & AT, CN, WD, WN & 72.4 & 68.3 & 73.0 & 66.7 & 60.9 & 68.3 \\ \hline \end{tabular}
\end{table}
Table 10: STL-Adapter and zero-shot fusion w/ _KG-C adapter_ performance across five commonsense tasks in various combination of KGs. AT, CN, WD and WN represent ATOMIC, ConceptNet, WikiDData and WordNet, respectively. Whole represents the combination of AT, CN, WD and WN. We run our experiment with seed 42.

\begin{table}
\begin{tabular}{l l r r r r r r} \hline
**Model** & **KG** & **a-NLI** & **CSQA** & **PIQA** & **SIQA** & **WG** & **Avg.** \\ \hline  & AT & 71.6 & 64.0 & 72.2 & 63.2 & 60.5 & 66.3 \\ STL-PLM & CN & 67.9 & 68.5 & 72.6 & 54.6 & 58.6 & 64.4 \\ WD & 68.4 & 64.7 & 72.0 & 53.7 & 58.6 & 63.5 \\ WN & 67.2 & 61.4 & 71.7 & 53.5 & 58.9 & 62.5 \\ \hline  & AT, CN & 70.5 & 68.4 & 72.2 & 60.1 & 58.2 & 65.9 \\ MTL & 69.9 & 66.4 & 72.0 & 60.1 & 59.3 & 65.5 \\ MTL & 69.1 & 62.7 & 71.6 & 59.1 & 59.1 & 64.3 \\ CNN, WD & 69.6 & 67.8 & 72.0 & 54.3 & 59.5 & 64.6 \\ CN, WN & 69.8 & 66.3 & 71.7 & 53.8 & 56.4 & 63.6 \\ WD, WN & 67.5 & 62.0 & 71.7 & 53.7 & 59.0 & 62.8 \\ \hline  & AT, CN, WD & 70.4 & 66.8 & 71.5 & 62.4 & 61.0 & 66.4 \\ MTL & 68.5 & 65.7 & 72.1 & 62.7 & 59.1 & 65.6 \\ MTL & 65.1 & 71.1 & 63.2 & 60.8 & 66.2 \\ NN, WD, WN & 69.6 & 67.3 & 72.5 & 52.0 & 57.2 & 63.7 \\ \hline MTL & 67.1 & 67.1 & 67.0 & 61.9 & 59.3 & 66.0 \\ \hline \end{tabular}
\end{table}
Table 9: STL-PLM and MTL performance across five commonsense tasks in various combination of KGs. AT, CN, WD and WN represent ATOMIC, ConceptNet, WikiData and WordNet, respectively. We run our experiment with seed 42.