SlowGuess/NAACL_2022_OCR huggingface dataset:
Yu Jin Kim<sup>1</sup>, Beong-woo Kwak<sup>1</sup>, Youngwook Kim<sup>1</sup>, Reinald Kim Amplayo<sup>2*</sup> Seung-won Hwang<sup>3</sup> and Jinyoung Yeo<sup>1†</sup>
<sup>1</sup>Nyonsei University, Seoul, Korea <sup>2</sup>Google Research, London, UK
$^{3}$ Seoul National University, Seoul, Korea
{yujin000731, beongwoo.kwak, youngwook, jinyeo}@yonsei.ac.kr
reinald@google.com, seungwonh@snu.ac.kr
Commonsense reasoning systems should be able to generalize to diverse reasoning cases. However, most state-of-the-art approaches depend on expensive data annotations and overfit to a specific benchmark without learning how to perform general semantic reasoning. To overcome these drawbacks, zero-shot QA systems have shown promise as a robust learning scheme by transforming a commonsense knowledge graph (KG) into synthetic QA-form samples for model training. Considering the increasing type of different commonsense KGs, this paper aims to extend the zero-shot transfer learning scenario into multiple-source settings, where different KGs can be utilized synergically. Towards this goal, we propose to mitigate the loss of knowledge from the interference among the different knowledge sources, by developing a modular variant of the knowledge aggregation as a new zero-shot commonsense reasoning framework. Results on five commonsense reasoning benchmarks demonstrate the efficacy of our framework, improving the performance with multiple KGs.
The ability to understand natural language through commonsense reasoning is one of the core focuses in the field of natural language processing. To measure and study the different aspects of commonsense reasoning, several datasets are developed, such as SocialQA (Sap et al., 2019b), CommonsenseQA (Talmor et al., 2018), and PhysicalQA (Bisk et al., 2020), each requiring different type of commonsense knowledge (e.g., social, taxonomic, causal, declarative, etc) to select the correct answer. While large-scale neural systems (Devlin et al., 2018; Yang et al., 2019; Liu et al., 2019b) have shown human-level accuracy on these benchmarks, recent studies (Mitra et al., 2019) also criticize that these models solve individual datasets,
rather than learning how to perform general semantic reasoning. To this end, Ma et al. (2021) suggested zero-shot evaluation as a genuine measure for the reasoning capability of the machine.
Inspired by this new metric, in this work, we focus on building unsupervised zero-shot multiple-choice QA systems. That is, we target an arbitrary commonsense reasoning task where conventional approaches (that rely heavily on task-specific supervision) are not applicable to such zero-shot learning scenarios. To learn QA models without expensive annotation efforts, recent works (Ma et al., 2021; Banerjee and Baral, 2020; Malaviya et al., 2020) propose to generate a synthetic QA dataset using a commonsense KG such as ATOMIC (Sap et al., 2019a) and ConceptNet (Speer et al., 2017). Such an approach mostly focuses only on one specific type of reasoning relations (e.g., if-then relation, or declarative relation), neglecting the fact that real-world QA systems require simultaneously considering different types of reasoning abilities (e.g., declarative and social, or causal and physical reasoning; Ilievski et al., 2021; Chang et al., 2021).
To consider different types of reasoning, this paper extends ideas from the aforementioned zero-shot learning to the multi-source case such that it benefits from different types of commonsense knowledge on individual KGs. For example, ATOMIC (Sap et al., 2019a) focuses on social commonsense while ConceptNet (Speer et al., 2017) contains conceptual knowledge. A practical approach is multi-task learning (MTL; Caruana, 1997; Liu et al., 2019a), which learns a shared encoder for different synthetic QA datasets from multiple KGs. Despite its effectiveness, MTL scheme suffers from interference among different KGs, which results in forgetting previously learned knowledge when trained on new KG which has different kinds of knowledge (Pilault et al., 2021; Pfeiffer et al., 2021; Wang et al., 2021a; Wu et al., 2020).
To address these limitations, we propose a novel,
*Work done while at the University of Edinburgh.
Human Language Technologies, pages 2244 - 2257
modularized framework that aims to learn multiple expert models for KGs, then conduct zero-shot fusion to allow collaboration among KGs. For this purpose, we leverage AdapterFusion (Pfeiffer et al., 2021) where multiple tiny modules between Transformer blocks called adapters (Houlsby et al., 2019) can be combined after independent training, thus allowing a continual integration of the adapters without retraining the entire framework. Specifically, we treat the adapters as different KG-specific experts, and combine them using an attention-like fusion module. To improve the fusion of adapters, we suggest a KG-alignment adapter that guides to the apt expert adapters. Here, we use KGs in three different synthetic supervision training: (1) KG-specific QA datasets to train the KG-specific expert adapters, (2) a KG classification datasets to train the KG-alignment adapter, and (3) a balanced mixture of KG-specific QA datasets to train the fusion module. Our modularized method alleviates the interference between different KGs, which is the pitfall of MTL from our empirical observation, and thus combines multiple KGs into a synergetic zero-shot framework.
Our contributions are: (1) We suggest a simple, yet effective KG modularization strategy for the use of multiple KGs in commonsense reasoning. (2) We then explore the use of AdapterFusion (Pfeiffer et al., 2021) for better knowledge aggregation based on the KG modularization in zero-shot setting. We believe that such modularized transfer learning is critical to using different knowledge sources synergically against interference between them. (3) In extensive experiments on various commonsense reasoning benchmarks, our framework achieves significant improvements over baselines using a single KG, even using multiple KGs, which implies the robustness in commonsense reasoning.
2 Related Work & Preliminaries
2.1 Zero-shot Commonsense Reasoning
Many researchers have recently focused on building unsupervised models without any benchmark supervisions (i.e., zero-shot learning). In such zero-shot setting, KGs are often used as an external resource for improving model prior (e.g., continually learned from pre-trained language models) (Banerjee and Baral, 2020; Bosselut and Choi, 2019; Ma et al., 2021), especially for commonsense reasoning, as much existing work couples language models with neural/symbolic commonsense KGs.
However, most of existing work are either assuming the existence of the alignment information between tasks and KGs (Banerjee and Baral, 2020) or an integrated KG (Ma et al., 2021). For example,  $\mathrm{ATOMIC}_{20}^{20}$  (Hwang et al., 2021), a commonsense KG which incorporates tuples from ConceptNet and ATOMIC with new relations and further crowdsourcing, combines multiple KGs into a new integrated KG, but as widely known (Ilievski et al., 2020; Hwang et al., 2021), heterogeneous schema between different KGs may limit triplets that can be integrated. Rather than such symbolic KG integration with the inevitable loss of knowledge, in this work, we explore the neural KG integration leveraging the multiple KGs without additional processing and alignment information between KG and task.
2.2 Transfer Learning with Modular Approaches
The idea of having specialized parameters, or so-called experts, has been widely studied to integrate multiple sources of knowledge via transfer learning. The adapter module (Rebuffi et al., 2017; Houlsby et al., 2019) has been explored as one of such approaches, introducing a small number of task-specific parameters at every layer of pretrained language model (PLM) while sharing the parameters of underlying PLM which is fixed. To address the limitations of transfer learning due to high re-training cost, many works utilize the multiple adapter modules for individual tasks with different domains (Puigcerver et al., 2020; Bapna et al., 2019; Rücklé et al., 2020; Madotto et al., 2021) considering each adapter to be an expert of each domain. Similar to our work, K-Adapter (Wang et al., 2021a) encodes factual and linguistic knowledge to each adapter, but in this paper, we further explore how to mitigate catastrophic forgetting or interference among multiple adapters for better knowledge transfer in zero-shot setting.
2.3 Multi-Task Learning
MTL (Liu et al., 2019a; Zhang and Yang, 2017; Caruana, 1997) learns a shared representation while aggregating knowledge across multiple learning tasks, often leading to better generalization ability of a model. However, parametric aggregation of knowledge with MTL has following limitations: (1) retraining the full model when adding new
1Only 172K tuples of the 3.4M tuples and 5 relations of 36 relations in ConceptNet are integrated into ATOMIC20
tasks (Houlsby et al., 2019; Pfeiffer et al., 2021, 2020b) (2) catastrophic forgetting and interference between tasks leading to difficulties of solving each task equally well (Pilault et al., 2021; Wu et al., 2020; Yu et al., 2020) and (3) inconsistent effect (Lourie et al., 2021). To deal with these challenges, Mixture-of-Experts (MoE) is a parameterized generalization of ensembling techniques, which has been adapted for MTL with gating network trained to optimize each task (Ma et al., 2018). However, simple linear gating networks are too shallow and thus may destruct task knowledge for commonsense reasoning.
To address this problem, AdapterFusion (Pfeiffer et al., 2021) has been proposed to fuse task specific parameters called adapters for the given target task leveraging attention-like mechanism. AdapterFusion aggregates adapters, which is trained independently for each task, in a non-destructive manner mitigating aforementioned MTL problems such as forgetting and interference between tasks. Recently, it has been used for zero-shot cross-lingual transfer framework (Pfeiffer et al., 2020c; Wang et al., 2021b), which motivates our work to transfer multi-source knowledge with less interference for zero-shot commonsense reasoning.
3 Modularized Zero-shot Framework
In our setup, we repurpose synthetic QA generation (Ma et al., 2021) for the task of knowledge-driven zero-shot learning for commonsense reasoning, i.e., we transform a KG into multiple  $(Q_{i},A_{i})$  pairs where  $Q_{i}$  is a natural language question and  $A_{i} = \{A_{i,1},\dots,A_{i,m}\}$  is the set of options with  $m$  answer candidates. Specifically, given a triple  $(e^{head},r,e^{tail})$  in a KG, where  $e^{head},e^{tail}$  and  $r$  denote head/tail entity and relation respectively, we transform  $e^{head}$  and  $r$  into a natural language question  $Q_{i}$  using templates. For the option set  $A_{i}$ , we use the combination of the correct answer  $e^{tail}$  and  $m - 1$  distractors which are tail entities from other triples sampled randomly (Ma et al., 2021). Details are described in Appendix B.
Formally, we denote  $(Q_{i},A_{i})$  as one QA sample. The goal is to learn a QA model from the synthetic QA sample. In a downstream task (e.g., reasoning benchmarks such as SocialIQA and CommonsenseQA), we need to predict answers given non-synthetic test samples  $(Q^{test},A^{test})$ . In the training stage, we are given  $K$  KG-driven datasets  $\{\mathcal{D}_{QA}^k\}_{k = 1}^K$  from  $K$  different KGs, where  $\mathcal{D}_{QA}^k$  is
Table 1: Synthetic QA examples. We use templates to convert  $(e^{head},r)$  into a natural language sentence.
a dataset with  $N_{k}$  samples for KG  $k$  as follows:
$$
\mathcal {D} _ {Q A} ^ {k} = \left\{\left(Q _ {i}, A _ {i}, l a b e l\right) \right\} _ {i = 1} ^ {N _ {k}} \tag {1}
$$
where label is the index of the correct answer for each sample. In this work, as shown in Table 1, we generate four synthetic QA datasets from ATOMIC, ConceptNet, WikiData, and WordNet (More details are in Appendix C).
For effective use of multiple KGs at once with less interference, we present a modularized framework, which is a novel approach to knowledge transfer for the zero-shot setting as shown in Figure 1. As a modular approach, we train the multiple KG-specific adapters (expert adapters) with each dataset from KG. Based on these pre-trained adapters, we use a zero-shot fusion method to aggregate knowledge of each adapter leveraging AdapterFusion (Pfeiffer et al., 2021) as a base architecture with the balanced mixture of each KG dataset. Further, for better knowledge fusion, we suggest a KG-alignment aware adapter (KGClassifier adapter) as a guide for detecting alignment with given sample in zero-shot reasoning. Here, we utilize KG classification dataset by verifying the synthetic QAs. Algorithm 1 in Appendix outlines the overall process of our proposed framework. We summarize the notations in Appendix A.
3.1 KG Modularization
First, we modularize the KGs to preserve their intrinsic knowledge. Considering the importance of using a suitable and well-aligned KG (Ma et al., 2019, 2021) on a downstream task, the subtle difference between each KG should be learned by the model without any interference from each other. Accordingly, we adopt the adapter module (Houlsby et al., 2019) which repurposes a pretrained language model (PLM) to incorporate each
Figure 1: Illustration of the proposed modularized framework for zero-shot commonsense reasoning. Each colored square represents different KGs. Not only for KG modularization, we re-use a set of synthetic QA datasets from the multiple KGs for the purpose of KG classification and zero-shot fusion, which enables better knowledge aggregation.
KG as tiny modules in between Transformer blocks. Specifically, as illustrated in Figure 2 (except for green area), the adapter training strategy involves injecting new layers (parameterized by  $\Phi$ ) into the original PLM (parameterized by  $\theta$ ). The weights of the original PLM are untouched, while the new adapter layers are initialized at random. Formally, we call each adapter trained with  $\mathcal{D}_{QA}^{k}$  as an expert adapter for KG  $k$ , parameterized by  $\Phi_{QA}^{k}$ .
When a QA sample  $(Q_{i},A_{i})$  is given for dataset  $\mathcal{D}_{QA}^{k}$ , we first concatenate question  $Q_{i}$  and each answer option  $A_{i} = \{A_{i,1},\dots,A_{i,m}\}$  to generate input sequences  $T_{i} = \{T_{i,1},\dots,T_{i,m}\}$ . Then, we compute a score  $S_{i,j}$  (Ma et al., 2021) for the answer candidate  $A_{i,j}$  is computed as follows:
$$
S _ {i, j} = - \frac {1}{\left| T _ {i , j} \right|} \sum_ {t = 1} ^ {\left| T _ {i, j} \right|} \log P \left(w _ {t} | \dots w _ {t - 1}, w _ {t + 1} \dots ; \theta , \Phi\right) \tag {2}
$$
where  $w_{t}$  is a word token in the sequence  $T_{i,j}$  and  $P$  is the conditional probability from Transformer blocks parameterized by  $\theta$  and  $\Phi$ . To train the adapter  $\Phi_{QA}^{k}$ , we use the marginal ranking loss (Ma et al., 2021) as follows:
$$
\mathcal{L}_{QA} = \frac{1}{m}\sum_{i = 1}^{N_{k}}\sum_{\substack{j = 1\\ j\neq l a b e l}}^{m}max(0,\eta -S_{i, l a b e l} + S_{i,j}) \tag{3}
$$
where  $\eta$  represents the margin.
$$
\Phi_ {Q A} ^ {k} \leftarrow \underset {\Phi} {\operatorname {a r g m i n}} \mathcal {L} _ {Q A} \left(\mathcal {D} _ {Q A} ^ {k}; \theta , \Phi\right) \tag {4}
$$
where KG-invariant parameters  $\theta$  are fixed and only KG-dependent parameters  $\Phi_{QA}^{k}$  are learned, which
enables to store the corresponding knowledge separately without any interference. Further, we can parallelize the training of adapter for all KGs. The efficiency of adapter training allows our modularization to be more scalable.
3.2 Zero-shot Fusion
Once the expert adapters are learned, we combine the knowledge from each expert adapter using an attention-like mechanism. We present a novel fusion strategy as shown in Figure 2, which is referred to as the zero-shot fusion. In contrast to Adapter-Fusion (Pfeiffer et al., 2021) where the focus is learning to transfer knowledge to a specific target task, our zero-shot fusion aims to generalize this transfer to any arbitrary target task. Specifically, the zero-shot fusion parameters  $\Psi$  learn to combine fixed expert adapters which are parameterized by  $\Phi_{QA}^{1},\ldots,\Phi_{QA}^{K}$ . In each Transformer layer  $l$  of PLM with the injected fusion layer, the zero-shot fusion parameters  $\Psi_{QA}$  consist of query, key, and value matrices, denoted by  $\mathbf{W}_l^Q$ ,  $\mathbf{W}_l^K$ , and  $\mathbf{W}_l^V$  respectively. These parameters are used to learn the balancing between the representation of each expert adapters through attention-like mechanism. While fixing both the parameters  $\theta$  and all expert adapters  $\Phi_{QA}^{1},\ldots,\Phi_{QA}^{K}$ , the only trainable weights  $\Psi_{QA}$  on the fusion layer learns to combine the knowledge from different  $K$  expert adapters by using the subset of  $\{\mathcal{D}_{QA}^k\}_{k=1}^K$  by random sampling. Here, we balance the ratio between the  $K$  knowledge-driven datasets as  $N$  samples (details are in Appendix D). Formally,
$$
\Psi_ {Q A} \leftarrow \underset {\Psi} {\operatorname {a r g m i n}} \sum_ {k = 1} ^ {K} \mathcal {L} _ {Q A} \left(\mathcal {D} _ {Q A} ^ {k}; \theta , \left\{\Phi_ {Q A} ^ {k} \right\} _ {k = 1} ^ {K}, \Psi\right) \tag {5}
$$
where  $\Psi$  refers to the initialized zero-shot fusion parameters.
More specifically, in the  $l$ -th Transformer layer, let  $h_{PLM}^{l}$  and  $h_{E}^{k,l}$  be the representations of underlying PLM parameterized by  $\theta$  and an expert adapter parameterized by  $\Phi_{QA}^{k}$ , respectively. Then, using the hidden representation  $h_{PLM}^{l}$  of PLM as a query, the fusion layer performs the attention-like function as follows:
$$
\mathbf {K} _ {l}, \mathbf {V} _ {l} = \left[ h _ {E} ^ {1, l}, \dots , h _ {E} ^ {K, l} \right] \tag {6}
$$
$$
\mathbf {Q} _ {l} = h _ {P L M} ^ {l} \tag {7}
$$
$$
\mathbf {z} _ {l} = \operatorname {A t t e n t i o n} \left(\mathbf {Q} _ {l} \mathbf {W} _ {l} ^ {Q}, \mathbf {K} _ {l} \mathbf {W} _ {l} ^ {K}, \mathbf {V} _ {l} \mathbf {W} _ {l} ^ {V}\right) \tag {8}
$$
Figure 2: Illustration of the zero-shot fusion architecture with KG-Classifier adapter. Each colored circle represents expert adapters, except the black circle which denotes KG-Classifier adapter. * indicates the fixed layer. Details are in Appendix F
where  $\mathbf{z}_l$  is passed to the next Transformer layer. Given a sample, the zero-shot fusion learns the suitable balancing parameters between the expert adapters for zero-shot reasoning. Eventually, it learns to identify generalizability across commonsense reasoning tasks.
3.3 KG-Classifier Adapter
AdapterFusion uses the PLM hidden representation  $h_{PLM}^{l}$  as a query which is learned when training on a specific downstream task. In our zero-shot setting, however, we use a mixture of synthetic QA for fusion training, which is not exactly a training dataset for a downstream task. To compensate for this issue, we present KG-Classifier adapter, which is a KG alignment-aware adapter, which is motivated from the fact that the ability to find which KG has an alignment with the given sample can be helpful as a role of providing a guidance for better performance (Ma et al., 2019, 2021).
Specifically, we propose a novel training task for KG-Classifier adapter, which requires predicting the KG for the given sample of the task. For that, given  $\{\mathcal{D}_{QA}^k\}_{k = 1}^K$  , we first transform a QA sample  $(Q_{i},A_{i})$  into a new KG classification sample  $[Q_i;A_{i,\text{label}}]$  where  $[\cdot ]$  is the concatenation. Then, we obtain a new label  $y_{i}\in \{0,1\}^{K}$  indicating the corresponding KG source. The samples are in Appendix E. Formally, KG classification dataset  $\mathcal{D}_{KGC}$  is defined as:
$$
\mathcal {D} _ {K G C} = \left\{\left([ Q _ {i}; A _ {i, l a b e l} ], y _ {i}\right) \right\} _ {i = 1} ^ {M} \tag {9}
$$
where  $M$  is the total size of  $\{\mathcal{D}_{QA}^k\}_{k = 1}^K$
Based on  $\mathcal{D}_{KGC}$ , we learn the  $KG$ -Classifier adapter parameterized by  $\theta$  and  $\Phi_{KGC}$ . First, a classification sample  $i$  is encoded into  $h_{CLS} \in \mathbb{R}^H$  then scored as  $\hat{y}_i \in \mathbb{R}^K$  with a linear layer  $W_{KGC} \in \mathbb{R}^{K \times H}$ , i.e.,  $\hat{y}_i = W_{KGC} h_{CLS}$ . Once  $\hat{y}_i$  is normalized by a softmax layer, the network is trained to minimize the cross-entropy loss  $\mathcal{L}_{KGC}$  between the prediction  $\hat{y}_i$  and its ground truth  $y_i$ :
$$
\Phi_ {K G C} \leftarrow \underset {\Phi} {\operatorname {a r g m i n}} \sum_ {i = 1} ^ {M} \mathcal {L} _ {K G C} \left(y _ {i}, \hat {y} _ {i}; \theta , \Phi\right) \tag {10}
$$
We propose to use the representation of  $KG$ -Classifier adapter as a query in attention-like mechanism, referred to as the zero-shot fusion with  $KG$ -Classifier adapter. That is, using the hidden representation  $h_{KGC}^{l}$  of a  $KG$ -Classifier adapter parameterized by  $\Phi_{KGC}$  as a query, we substitute  $\mathbf{Q}_l$  in Eq. (11) as follows:
$$
\mathbf {Q} _ {l} = h _ {K G C} ^ {l} \tag {11}
$$
The overall zero-shot fusion architecture including KG-Classifier is illustrated in Figure 2.
In this section we evaluate the efficacy of our framework on five commonsense reasoning tasks. We denote KG-Classifier adapter by KG-C adapter.
All our experiments are conducted in a zero-shot setting, in which the models do not have access to the official training data or labels of the benchmark. For the evaluation, we use the validation set of each benchmark $^2$ , however, the validation set of each benchmark can be role as an test set since it is not used for hyperparameter tuning or model selection. We use accuracy as a metric.
4.1.1 Benchmarks
We evaluate our proposed framework on five question-answering benchmarks for commonsense reasoning: SocialIQA (SIQA) (Sap et al., 2019b), CommonsenseQA (CSQA) (Talmor et al., 2018), Abductive NLI (a-NLI) (Bhagavatula et al., 2020), PhysicalIQA (PIQA) (Bisk et al., 2020), and Wino-Grande (WG) (Sakaguchi et al., 2020). Each commonsense benchmark evaluates a specific kind of knowledge: social commonsense for SIQA, concept-level commonsense for CSQA, abductive
2Since the official test sets are not publicly available
Model	KG	a-NLI	CSQA	PIQA	SIQA	WG	Avg.
Random	-	50.0	20.0	50.0	33.3	50.0	40.7
Majority	-	50.8	20.9	50.5	33.6	50.4	41.2
GPT2-L	-	56.5	41.4	68.9	44.6	53.2	52.9
RoBERTa-L	-	65.5	45.0	67.6	47.3	57.5	56.6
Self-talk (Shwartz et al., 2020)	-	-	32.4	70.2	46.2	54.7	-
COMET-DynaGen (Bosselut and Choi, 2019)	AT	-	-	-	50.1	-	-
SMLM (Banerjee and Baral, 2020)	*	65.3	38.8	-	48.5	-	-
RoBERTa-L (MR) (Ma et al., 2021)	AT	70.8	64.2	72.1	63.1	59.2	65.9
RoBERTa-L (MR) (Ma et al., 2021)	CN,WD,WN	70.0	67.9	72.0	54.8	59.4	64.8
RoBERTa-L (MR) (Ma et al., 2021)	Whole	70.5	67.4	72.4	63.2	60.9	66.9
MTL	Whole	69.8 (±0.5)	66.0 (±0.9)	71.2 (±0.8)	62.2 (±1.0)	59.5 (±0.2)	65.7
zero-shot fusion w/o KG-C adapter	Whole	72.3(±0.4)	67.9(±0.2)	73.1(±0.4)	65.9(±0.5)	59.7(±0.2)	67.8
zero-shot fusion w/ KG-C adapter	Whole	72.5(±0.2)	68.2(±0.2)	72.9(±0.4)	66.6(±0.1)	60.8(±0.1)	68.2
Table 2: Zero-shot evaluation results with different combinations of models and knowledge sources across five commonsense tasks. AT, CN, WD and WN represent ATOMIC, ConceptNet, WikiData and WordNet, respectively. Whole represents the combination of AT, CN, WD and WN. Bold text indicates the best performance on each benchmark. RoBERTa-L (MR) used the synthetic dataset after filtering, while we use the raw version. SMLM (*) used different KG which has strong alignment with each task (e.g. AT for SIQA).
reasoning for a-NLI, physical commonsense for PIQA, and pronoun resolution ability for WG.3 The details are presented in Appendix G.