CANDLE: Iterative Conceptualization and Instantiation Distillation from Large Language Models for Commonsense Reasoning (2024)

Abstract: The sequential process of conceptualization and instantiation is essential to generalizable commonsense reasoning as it allows the application of existing knowledge to unfamiliar scenarios. However, existing works tend to undervalue the step of instantiation and heavily rely on pre-built concept taxonomies and human annotations to collect both types of knowledge, resulting in a lack of instantiated knowledge to complete reasoning, high cost, and limited scalability. To tackle these challenges, we introduce CANDLE (ConceptuAlization and INstantiation Distillation from Large Language ModEls), a distillation framework that iteratively performs contextualized conceptualization and instantiation over commonsense knowledge bases by instructing large language models to generate both types of knowledge with critic filtering. By applying CANDLE to ATOMIC (Sap et al., 2019a), we construct a comprehensive knowledge base comprising six million conceptualizations and instantiated commonsense knowledge triples. Both types of knowledge are firmly rooted in the original ATOMIC dataset, and intrinsic evaluations demonstrate their exceptional quality and diversity. Empirical results indicate that distilling CANDLE on student models provides benefits across three downstream tasks1.

heading: 1 Introduction
text: Commonsense reasoning refers to the cognitive ability to make logical inferences and draw conclusions based on general knowledge and understanding of the world that is typically shared among individuals (Davis, 2014; Mueller, 2014). However, a longstanding challenge is generalizability, as commonsense reasoning often necessitates applying knowledge to novel situations beyond simple pattern recognition or memorizing all special cases (Mortimer, 1995; Banaji and Crowder, 1989).
1Our data and models are publicly available at https://github.com/HKUST-KnowComp/CANDLE.
One promising approach to address this is the chain of conceptualization (Murphy, 2004) and instantiation (Anderson et al., 1976), which, akin to the process of conceptual induction and deduction in human reasoning (Tenenbaum et al., 2011), involves conceptualizing instances derived from known commonsense knowledge and subsequently instantiating these concepts in new situations to obtain the knowledge required for downstream reasoning. For example, in Figure 1, one can first conceptualize enjoys exercising in the gym as a healthy lifestyle, and then further instantiate it to go on a balanced diet. This process allows for the derivation of a novel event, PersonX goes on a balanced diet, which may entail new common-
2351
sense knowledge when connected with the original event’s commonsense inferential tail. By possessing substantial knowledge to initiate the process of conceptualization and instantiation, one can extrapolate limited commonsense knowledge to a wide array of diverse scenarios.
Yet, replicating this fundamental ability on machines remains challenging due to the absence of both types of knowledge in widely used CommonSense Knowledge Bases (CSKBs; Sap et al., 2019a; Speer et al., 2017; Fang et al., 2021b,a). Various methods compensating the lack of conceptualization ability of language models have been proposed for entity-level (Durme et al., 2009; Song et al., 2011, 2015; Gong et al., 2016; He et al., 2020; Peng et al., 2022a) and event-level (Chen et al., 2020; He et al., 2024; Wang et al., 2023b) conceptualizations by matching against concept taxonomies like Probase (Wu et al., 2012) and WordNet (Miller, 1995). However, several limitations still persist.
Firstly, despite the importance of both conceptualization and instantiation, most existing works underestimate the importance of the second step while focusing solely on conceptualization and using the resulting abstract knowledge directly. Other studies that concentrate on instantiations either overlook the conceptualization step entirely or only retrieve instances from the original CSKB, failing to introduce novel entities and events. Secondly, most conceptualization methods heavily depend on matching instances with concepts in concept taxonomies, such as Probase and WordNet, which have a limited scope and lack contextual information. Consequently, the derived conceptualizations are constrained in scale by these taxonomies and are formulated without considering proper contextualization, necessitating further verification in the original context. Lastly, the chain of conceptualization and instantiation can easily bring more than two orders of magnitude of data on top of the original CSKB. However, current acquisition and verification methods for both steps heavily rely on human annotation, which can be extremely costly as the scale of the CSKB increases.
To address these gaps, we introduce CANDLE, a ConceptuAlization and INstantiation Distillation framework from Large Language ModEls (LLMs) to aid commonsense reasoning. Specifically, CANDLE marks the first to complete the chain of conceptualization and instantiation by instructing powerful LLMs to sequentially generate both types of
knowledge based on concrete commonsense triples while carefully considering the original context throughout the process. We further alleviate the human annotation cost by employing two critic filtering models to eliminate low-quality generations. The instantiated knowledge, representing concrete commonsense knowledge again, can be fed back into CANDLE as input, iteratively augmenting the original CSKB significantly.
By applying CANDLE to ATOMIC (Sap et al., 2019a), we construct a large-scale knowledge base comprising 6.18 million conceptualizations and instantiations from two powerful LLMs, ChatGPT (OpenAI, 2022) and LLAMA2 (Touvron et al., 2023). We demonstrate the intrinsic efficacy of CANDLE through automatic and human evaluations, highlighting the ability to generate highquality and diverse knowledge (Section 5.1). We further show the extrinsic benefits of CANDLE by leveraging the generated knowledge as complementary training data to distill student models that yield improvements across three downstream tasks, including CSKB conceptualization, generative commonsense inference, and zero-shot commonsense question answering (Section 5.2).

heading: 2 Related Works

heading: 2.1 Conceptualization and Instantiation
text: Conceptualization aims to abstract a set of entities or events into a general concept, thereby forming abstract commonsense knowledge within its original context (Murphy, 2004). Subsequently, instantiation grounds the derived concept into other instances and events to introduce new commonsense knowledge. Existing works primarily focused on entity-level conceptualization (Durme et al., 2009; Song et al., 2011, 2015; Liu et al., 2022; Peng et al., 2022a), with He et al. (2024) pioneering the construction of an event conceptualization benchmark by extracting concepts for social events from WordNet (Miller, 1995) synsets and Probase (Wu et al., 2012). Wang et al. (2023b,a) further proposed a semi-supervised framework for conceptualizing CSKBs and demonstrated that abstract knowledge can enhance commonsense inference modeling and question answering. Wang et al. (2023d) constructed an abstraction benchmark based on eventualities from ASER (Zhang et al., 2022). Regarding instantiation, Allaway et al. (2023) introduced a controllable generative framework to identify valid instantiations for abstract
knowledge automatically. However, none of the existing studies have fully completed the chain of conceptualization and instantiation, with each focusing on only one aspect. Human annotation is also frequently applied for data collection and verification, which is both expensive and limited in scalability. Additionally, the downstream benefits of instantiated commonsense knowledge have not been thoroughly explored, leaving a significant gap in improving commonsense reasoning models.

heading: 2.2 Commonsense Knowledge Distillation
text: Recent breakthroughs in LLMs (OpenAI, 2022, 2023) have led to numerous efforts in distilling commonsense knowledge into datasets for training performant student models. West et al. (2022); Sclar et al. (2022); Bhagavatula et al. (2023); West et al. (2023) followed the pipeline of symbolic knowledge distillation, which uses human-crafted prompts to extract specific types of knowledge from LLMs for training downstream models. He et al. (2022) proposed to transfer distilled knowledge from a ranker to a retriever, resulting in a more robust commonsense generator. Chae et al. (2023) and Kim et al. (2023) focused on distilling conversational responses from LLMs to enhance dialogue agents with commonsense knowledge and high-quality rationales. In this paper, we share similar aspirations and propose a chain of distillation framework that sequentially obtains abstract and instantiated knowledge from powerful LLMs. Empirical results show that our framework offers more substantial downstream benefits than traditional symbolic knowledge distillation methods.

heading: 3 Definitions and Datasets
text: We follow the definitions proposed by He et al. (2024) and Wang et al. (2023b) to formulate conceptualization and instantiation. Denote the triples in the original CSKB as Do = {(ho, r, t)|ho ∈ Ho, r ∈ R, t ∈ T}, where Ho, R, and T are the set of heads, relations, and tails in the original CSKB. The objective of conceptualization is to form a conceptualized head event, denoted as ha, from the original head ho. This is achieved by linking a component i ⊆ ho to a concept c, forming ha by replacing i with c. Consequently, abstract knowledge is formed by combining the conceptualized head event with the original relation and tail, represented by (ha, r, t). In the next step, the goal of instantiation is to associate the concept c ⊆ ha with a new instance i′. This process enables the formation of new commonsense knowledge in the format of (hi′ , r, t), where hi′ is obtained by replacing c ⊆ ha with i′. In this paper, we use ATOMIC (Sap et al., 2019a) as the original CSKB Do, which contains 310K (ho, r, t) triples after dropping those with wildcards and 18,839 unique ho head events. AbstractATOMIC (He et al., 2024) is used as the source of instances i for every head event ho.

heading: 4 CANDLE
text: This section introduces our CANDLE framework, illustrated in Figure 2. Our framework can be outlined in three steps: (1) Instruct ChatGPT to generate contextualized conceptualizations based on the triples in the original CSKB. (2) Instruct LLAMA2 to instantiate the conceptualizations obtained in
Step 1. (3) Apply critic-filtering to the generations in both steps and close the loop by reintroducing the instantiations back to the CSKB.

heading: 4.1 Contextualized Conceptualization
text: Previous methods for collecting conceptualizations rely on heuristically matching instances against concepts from WordNet and Probase. However, they suffer from limited concept coverage, resulting in a lack of knowledge diversity after instantiation, and require additional verification to ensure that concept c fits into the original context (ho, r, t). To address both issues, we propose to utilize ChatGPT as a loose teacher to collect conceptualizations in a one-step inference manner. To verify the feasibility of such choice instead of other opensource LLMs, we carry out a pilot study, in which we randomly select 1000 events and asked ChatGPT and LLAMA2-7B to generate their conceptualizations. The results of our expert evaluation show that ChatGPT has 98% plausible generations, while LLAMA2 only achieves 81%. Therefore, we choose ChatGPT as our core conceptualizer due to its exceptional performances. Following Brown et al. (2020) and West et al. (2022), we use a fewshot prompt to instruct ChatGPT:
<TASK-PROMPT> <EX1-INP><EX1-OUT> . . . <EXN−1-INP><EXN−1-OUT> <EXN-INP>
where <TASK-PROMPT> is a task instruction that explains how to conceptualize an event and <EX1-INP><EX1-OUT> are human authored examples of conceptualizations for events sampled from ATOMIC. For each example, (ho, r, t, i) are included in the input, and c is the output. Finally, we provide the Nth input as <EXN-INP> and ask ChatGPT to generate the corresponding conceptualization as <EXN-OUT>. This ensures that ChatGPT not only learns the relationship between instances i and their conceptualizations c but also performs such abstraction in a contextualized manner, ensuring the plausibility of the generated conceptualization c within the original context (ho, r, t). In this paper, we set N = 6 and obtain Nc = 20 conceptualizations for every event ho.

heading: 4.2 Contextualized Instantiation
text: After conceptualizing all events, we proceed to instantiate them by instructing an open-source LLM
to reduce the cost as the scale of instantiation is Nc = 20 times larger than that of conceptualization. Similarly, we carry out a round of pilot study to demonstrate the feasibility of employing LLAMA2. We ask both ChatGPT and LLAMA2 to instantiate 1,000 ChatGPT-generated conceptualizations, and find that both models are able to produce approximately 95% plausible instantiations with critic filtering. Considering the significant cost of using ChatGPT to generate 6.18 million conceptualizations, we decide to use LLAMA2-13B as our core instantiater (Case studies are shown in Appendix G). We employ a similar prompt as described in Section 4.1, with the modification of replacing <TASK-PROMPT> with the explanation of instantiating a conceptualized event and changing <EX1-INP><EX1-OUT> to human-authored examples of instantiations for abstract commonsense knowledge triples. (ha, r, t, c) are included in the input and i′ is the expected output. By learning from these examples, LLAMA2 is expected to generate the corresponding instantiation i′ (<EXN-OUT>) based on the given abstract knowledge triple (ha, r, t, c) (<EXN-INP>). We set N = 11 and produce only one instantiation for each conceptualized event ha due to the significant amount of conceptualizations obtained in the previous step. Appendix A provides more details regarding the distillation process.

heading: 4.3 Iterating with Critic Filtering
text: Following West et al. (2022), we use critic filtering models to eliminate low-quality generations from LLMs. Specifically, we utilize a DeBERTav3-large conceptualization discriminator, provided by Wang et al. (2023b), and VERA-T5-xxl, provided by Liu et al. (2023), to evaluate the quality of the generated conceptualizations and instantiations, respectively. We set an empirical threshold value t to serve as the cutoff point for discarding generations with scores below t. In Section 5.1, we present evaluations conducted to determine the optimal value for t. For all downstream applications, we set t = 0.9. Post-filtering, the instantiated triples (hi′ , r, t) can be reintroduced as the input for conceptualizations again as they continue to represent concrete commonsense knowledge. This iterative process of conceptualization and instantiation forms a loop, which enables continuously augmenting a CSKB. In this paper, we execute the loop only once, but multiple iterations hold
the promise of significantly enhancing the CSKB’s knowledge coverage.

heading: 5 Evaluations and Analysis
text: In this section, we evaluate CANDLE from both intrinsic and extrinsic perspectives. Intrinsically, we demonstrate the high quality and diversity of conceptualizations and instantiations generated by CANDLE (Section 5.1). Extrinsically, we explore the benefits by applying the distilled knowledge to downstream tasks (Section 5.2).

heading: 5.1 Distillation Evaluations
text: Statistics and Quality. We present CANDLE distillation statistics based on ATOMIC in Table 1, showing its superiority in scale and concept coverage compared to other benchmarks. Even with a strict critic filtering threshold (t = 0.9), CANDLE maintains its leading position, having the highest count of total and unique knowledge for both types. To assess the quality of the distilled knowledge, we recruit four expert annotators to conduct human evaluations on the plausibility of the generated conceptualizations and instantiations. They are asked to annotate the plausibility of 3,000 randomly sampled abstract commonsense triples (ha, r, t) and 3,000 instantiated triples (hi′ , r, t) from the distilled knowledge set. Accepted triples are those deemed plausible by all annotators. We then analyze accepted triple ratios for different levels of critic filtering, as shown in Table 1. Our findings show that LLMs have impressive conceptualization and instantiation abilities, with initial plausibility rates of 82.6% and 77.9% for both types of knowledge, respectively. Critic filtering improves plausibility by up to 14.6% and 16.6%, demonstrating the effectiveness of our measures in maintaining highquality distilled knowledge. For more annotation details, refer to Appendix E.
Conceptualization Diversity. The process of abstracting an event into highly diverse conceptualizations plays a crucial role in CANDLE. It is of significant importance because the greater the diversity of conceptualizations, the broader the knowledge coverage becomes upon instantiation. This, in turn, enhances the overall knowledge coverage within the distillation process. To examine the diversity of the top 10,000 popular distilled conceptualizations, we obtain their hypernyms by matching against Probase (Wu et al., 2012) and present a visualization in Figure 3. It reveals that our distilled conceptualizations possess a high level of diversity across various categories, forming a comprehensive and intricate knowledge base. Novelty of our distilled conceptualizations and instantiations, compared to other available resources, are in Appendix A.

heading: 5.2 Downstream Applications
text: In this section, we explore the downstream applications of CANDLE. By applying CANDLE to ATOMIC, the distilled conceptualizations and instantiations form a large-scale expansion of the original CSKB, which contains high-quality abstract and concrete commonsense knowledge. Leveraging both types of knowledge as supplementary training data, we enhance various downstream commonsense reasoning models. Specifically, we utilize distilled conceptualizations in the CSKB conceptualization task (Wang et al., 2023b), while instantiations are used in generative commonsense inference (COMET; Bosselut et al., 2019) and zero-shot commonsense QA tasks (Ma et al.,
2021). Due to space constraints, please refer to Appendix B, C, D for task setups, dataset descriptions, and implementation details, respectively.

heading: 5.2.1 CSKB Conceptualization
text: Task Setup. The CSKB conceptualization task evaluates a model’s ability to conceptualize a CSKB through two binary classification subtasks, which are crucial for performing CSKB conceptualization inference upon concept taxonomies (He et al., 2024). The first subtask, event conceptualization, aims to determine whether ho can be correctly conceptualized using ha, where ha is derived by replacing an instance i ⊂ ho with its linked concept c. The second subtask, triple conceptualization, aims to assess the plausibility of a conceptualized triple (ha, r, t) that represents abstract commonsense knowledge. Accuracy is used as the evaluation metric. Following Wang et al. (2023b), we use the AbstractATOMIC dataset provided by He et al. (2024) as the evaluation benchmark.
To obtain our distilled models for these tasks, we first synthesize negative samples from CANDLE distilled conceptualizations. For event conceptualizations, a random concept from another head event without common words is selected as the negative candidate, while for triple conceptualiza-
tion, a tail of another head event without common words under the same relation is selected. We then fine-tune language models on a balanced mixture of CANDLE distillation and synthesized negative samples to train two models, each serving as a pre-trained general discriminator in their respective task domain. These two models are subsequently fine-tuned on the training sets of AbstractATOMIC to fit into the benchmark, and their performances on the validation and test sets are reported.
Baselines. We evaluate our distilled models by comparing them against several baselines. These include supervised fine-tuned language models like RoBERTa-Large (Liu et al., 2019), DeBERTaV3-Large (He et al., 2023), GPT-2 (Radford et al., 2019), LLAMA2 (Touvron et al., 2023), Mistral (Jiang et al., 2023), and VERA (Liu et al., 2023), as well as semi-supervised methods such as PsuedoReasoner (Fang et al., 2022) and CAT (Wang et al., 2023b). Due to computational power limitations, we utilize LoRA (Hu et al., 2022) for fine-tuning LLMs. As additional baselines, we also consider prompting LLMs, including LLAMA2, Mistral, and ChatGPT. We explore both direct zero-shot prompting and alternative methods, such as with five-shot exemplars (Wei et al., 2023) and chain-of-thought reasoning (Wei et al., 2022).
Results and Analysis. Table 2 shows the results. CAT trained with DeBERTa-v3-large outperforms all other baselines for both tasks. Among LLMs, LLAMA and Mistral perform well after fine-tuning, but they struggle in prompting scenarios. However, pre-training on CANDLE’s distilled conceptualizations consistently improves results for both tasks. For example, Mistral shows a significant improvement of 1.54% and 2.19% on two tasks compared to directly fine-tuning on AbstractATOMIC. Additionally, the distilled DeBERTa-v3-large surpasses all baseline models and achieves state-of-the-art performance. This can be attributed to the distilled conceptualizations obtained from CANDLE, which grant the model a more comprehensive understanding of conceptualizations and subsequently enhance its discriminatory capabilities.

heading: 5.2.2 Generative Commonsense Inference
text: Task Setup. The task of generative commonsense inference modeling (COMET; Bosselut et al., 2019) asks the model to generate commonsense tails t based on given head ho and relation r inputs. Following Hwang et al. (2021), we use the full test set of ATOMIC2020 as our evaluation benchmark. We use several automatic metrics for evaluation, including BLEU (Papineni et al., 2002), ROUGE-L (Lin, 2004), METEOR (Lavie and Agarwal, 2007), CIDEr (Vedantam et al., 2015), and BERTScore (Zhang et al., 2020). Meanwhile, four expert annotators are recruited to conduct expert evaluations of the generations. They are asked to annotate the plausibility of 200 randomly selected
commonsense triple generations under each setting, and the resulting plausibility rates are reported.
Similar to training distilled models in previous tasks, we first pre-train GPT2 and LLAMA2-7B on critic-filtered CANDLE instantiations, where each (hi′ , r, t) triple is concatenated into a sentence via natural language templates. Subsequently, we fine-tune these models on the training split of ATOMIC2020 to fit them into the benchmark. Finally, we report their performances on the test set. Baselines. For baselines, we separately train GPT2 and LLAMA2-7B on the training sets of ATOMIC, ATOMIC2020, ATOMIC10X (West et al., 2022), and AbstractATOMIC. These models are then fine-tuned on the training split of ATOMIC2020 and evaluated on its test set. We also include their zero-shot prompting performances, with LLAMA2 being evaluated with five-shot exemplars. ChatGPT’s performances under zero-shot, five-shot, and chain-of-thought settings are also reported. Results and Analysis. Table 3 shows the results. Among the baselines, models pre-trained on ATOMIC-10X achieve the highest expert acceptance rate, surpassing those trained on AbstractATOMIC. This may be because ATOMIC-10X covers a wider range of commonsense relations consistent with ATOMIC2020. However, CANDLE distilled models achieve the highest scores compared to baselines with the same backbone model. For example, the CANDLE distilled LLAMA-7B model improves BERTScore by 1.01% and expertplausibility by 3.00% compared to the best baseline. It also outperforms ChatGPT in all automatic met-

heading: Pre-trained Language Models

heading: Large Language Models
text: rics while maintaining a high plausibility rate of around 80%. This emphasizes the advantages of using CANDLE distilled instantiations for COMET training over traditional symbolic knowledge distillation methods or conceptualization augmentation. Interestingly, we also observe that LLAMA2 has a tendency to generate long and contextually rich commonsense knowledge. On the other hand, GPT2, when fine-tuned on ATOMIC-like data, may generate shorter and more concise knowledge, which aligns with the format and length of knowledge in ATOMIC2020, thus achieving better results in automatic evaluations. However, human annotators tend to consider long and contextually rich commonsense statements, generated by LLAMA2, as more plausible.

heading: 5.2.3 Zero-shot Commonsense QA
text: Task Setup. The task of zero-shot commonsense QA involves selecting the most plausible option for commonsense questions without any supervision signals from benchmark data. We follow the most effective pipeline by Ma et al. (2021), which fine-tune language models on QA pairs synthesized from knowledge in CSKBs. The head ho and relation r of a (ho, r, t) triple are transformed into a question using natural language prompts, with the tail t serving as the correct answer option. Distractors or negative examples are generated by randomly sampling tails from triples that do not share common keywords with the head. In addition to directly synthesizing from knowledge triples in ATOMIC, we augment ATOMIC by sampling triples from ATOMIC-10X, AbstractATOMIC, and CANDLE instantiations. The number of sampled triples is the same as in the original ATOMIC dataset. We then synthesize them into QA pairs to train different baseline models and CANDLE distilled models. For our distilled models, we utilize QA pairs sourced from CANDLE-instantiation augmented ATOMIC to train a DeBERTa-v3-large model using the marginal ranking loss and a T5-xxl model (Raffel et al., 2020) following the training regime of VERA. We evaluate the performance of all models on the validation split of Abductive NLI (aNLI; Bhagavatula et al., 2020), CommonsenseQA (CSQA; Talmor et al., 2019), PhysicalIQA (PIQA; Bisk et al., 2020), SocialIQA (SIQA; Sap et al., 2019b), and WinoGrande (WG; Sakaguchi et al., 2021). Accuracy is used as the evaluation metric. Baselines. First, we report performances of vanilla RoBERTa-Large, DeBERTa-v3-Large, Self-talk (Shwartz et al., 2020), COMET-
DynaGen (Bosselut et al., 2021), SMLM (Banerjee and Baral, 2020), MICO (Su et al., 2022), MR (Ma et al., 2021), STL-Adapter (Kim et al., 2022), and the previous state-of-the-art method, CAR (Wang et al., 2023a). For MR and CAR, DeBERTa-v3Large is used as the backbone, and their performances on ATOMIC-10X and AbstractATOMIC are also reported. For LLMs, we report the performances of prompting GPT3.5 (Brown et al., 2020), ChatGPT, GPT4 (OpenAI, 2023), LLAMA2, and Mistral in a zero-shot manner. For ChatGPT, its performances with chain-of-thought (Wei et al., 2022) and self-consistency chain-of-thought (Wang et al., 2023c) prompting are also reported. We also train several VERA-T5-xxl baselines on different sets of QA pairs as LLM baselines. Results and Analysis. Table 4 shows the results, demonstrating that CANDLE distilled models generalize better than the baselines across several commonsense QA benchmarks. For instance, VERA demonstrates an average improvement of 1.4% compared to the best baseline. This can be attributed to the inclusion of new entities and events in CANDLE instantiations that are absent in other CSKBs, where CANDLE instantiations can aid in answering commonsense questions that require knowledge of these new instances. Furthermore, the distilled DeBERTa-v3-large model outperforms all baselines, including methods utilizing LLMs. This also indicates that augmenting with CANDLE distilled instantiations provides a more significant advantage compared to using symbolically distilled or abstract knowledge as training data.

heading: 5.3 Analysis

heading: 5.3.1 Feasibility of Iterating CANDLE
text: We first demonstrate the feasibility of iterating the CANDLE framework with more than one round. To do so, we randomly sample 10,000 distilled instantiations from LLAMA2 as the input for the CANDLE framework and execute the framework again, resulting in 200,000 conceptualizations and 200,000 instantiations. Subsequently, we randomly select 300 from each set and annotate them accord-
ingly. The results are shown in Table 5. We observe that iterating the framework produces slightly better results than the first loop. This improvement may be attributed to the fact that the knowledge generated in the initial loop is more easily understood by LLMs compared to the human-annotated data in ATOMIC. Moreover, 58% of the conceptualizations and 44% of the instantiations are novel compared to the first loop. Based on these findings, we believe that our iterative framework is effective, and the iteration process enhances the augmentation of a CSKB through multiple iterations.

heading: 5.3.2 Source of Empirical Gains
text: Since LLAMA2 has been pre-trained on some evaluation benchmarks, it remains questionable whether the empirical gains in downstream tasks are due to knowledge overlap between distillations from LLAMA2 and the evaluation benchmarks. To this extent, we further demonstrate that CANDLE distilled models perform better due to improved generalizability rather than relying on data overlap with the evaluation data. We use SentenceBERT (Reimers and Gurevych, 2019) to measure the textual similarity between the distilled knowledge and the evaluation data for each task. We then calculate the ratio of data that exhibits semantic overlap with a similarity score exceeding 0.5 and also report the average similarity. The results are shown in Table 6. Based on the results, we observe that the distilled knowledge has minimal overlap with the evaluation set. This indicates that the empirical gain primarily stems from our distilled knowledge, which improves the generalizability of the models, rather than relying on knowledge overlap with the evaluation sets.

heading: 6 Conclusion
text: This paper introduces CANDLE, a distillation framework that realizes the chain of conceptualization and instantiation over CSKBs. We demonstrate the efficacy of CANDLE through comprehensive evaluations of the distilled knowledge and its positive impact on downstream tasks. Our research sheds light on distilling LLMs to enable more robust and generalizable commonsense reasoning.

heading: Limitations
text: The major limitation of CANDLE lies in the significant cost of distilling LLMs to obtain substantial knowledge. While the instantiation step of CANDLE utilizes the open foundation model LLAMA2, the conceptualization is still performed by ChatGPT due to the unsatisfactory performance of other open-source LLMs and the high quality of ChatGPT’s generation. Consequently, a considerable amount of funding is required to distill conceptualizations for CANDLE to function effectively. Future works should address this issue by exploring advanced prompting methods or employing stronger open-source LLMs as the foundation for distilling conceptualizations.
Furthermore, it should be noted that CANDLE has only been validated on ATOMIC. However, CANDLE is not limited to any specific format of commonsense knowledge, allowing it to operate on any CSKB (Shen et al., 2023). Future research can address this by extending the evaluation of CANDLE to other CSKBs and conducting followup experiments to explore their benefits on more downstream tasks.
Another interesting direction to investigate is utilizing the chain of conceptualization and instantiation as a foundation for enhancing weak-to-strong generalization (Burns et al., 2023). By conceptualizing and instantiating weak supervision data, we can generate more robust and generalized training signals, which ultimately strengthens the learning process. This can also be effectively incorporated into the training process of self-rewarding language models (Yuan et al., 2024).

heading: Ethics Statement
text: To avoid generating harmful or unethical content from LLMs like ChatGPT and LLAMA2, we recruit four expert annotators, who are graduate and undergraduate students specializing in machine commonsense in natural language processing, to verify the ethics and potential harm of the generated content. A thorough assessment of a random sample has been conducted, and no significant harm has been identified. All training and evaluation datasets used are publicly available and shared under open-access licenses solely for research purposes, aligning with their intended usage. These datasets have been carefully anonymized and desensitized to protect data privacy and confidentiality. The expert annotators involved in this study are
fully aware of the annotation protocol and the intended use of their annotations. Their participation in this research is voluntary, and they have agreed to contribute without receiving any compensation. Thus, the authors believe that this paper does not raise any ethical concerns.

heading: Acknowledgements
text: We thank the anonymous reviewers and the area chair for their constructive comments. The authors of this paper were supported by the NSFC Fund (U20B2053) from the NSFC of China, the RIF (R6020-19 and R6021-20), and the GRF (16211520 and 16205322) from RGC of Hong Kong. We also thank the support from the UGC Research Matching Grants (RMGS20EG01-D, RMGS20CR11, RMGS20CR12, RMGS20EG19, RMGS20EG21, RMGS23CR05, RMGS23EG08).

heading: A Distillation Details
text: This section provides additional details about the CANDLE distillation process not covered in the main body text. First, we present the prompts used to instruct ChatGPT to perform contextualized conceptualizations and LLAMA2 to perform contextualized instantiation. For prompting ChatGPT to distill conceptualizations, we use a few-shot prompt as shown below:
Following the given examples, you are
required to conceptualize the instance
(enclosed by []) in the last given event
into abstract concepts. The concept
should still fit into the instance’s
original sentence. Make sure that
the generated abstract concepts are
general and not simply hypernyms of the
instance. . . . Event <i>: PersonX enjoys drinking in
the [bar], as a result, PersonX feels
relaxed. [bar] can be conceptualized as
Social Gathering Place . . . Event <N>: PersonX likes [painting on
the beach], as a result, PersonX will go
to the beach. [painting on the beach]
can be conceptualized as
Similarly, for prompting LLAMA2-13B to distill instantiations based on previously generated conceptualizations, we use a few-shot prompt as shown below:
Following the given examples, you are
required to instantiate the concept
(enclosed by []) in the last given
event into entities or events. If the
event only contains the concept, then
instantiate it to an event starting
with a subject PersonX or PersonY. If
the event contains other words, then
instantiate it to an entity. The
instance should still fit into the
original sentence. Make sure that the
generated instance is specific. . . . Event <i>: PersonX enjoys drinking in
the [Social Gathering Place], as a
result, PersonX feels relaxed. [Social
Gathering Place] can be instantiated as
beer festival . . . Event <N>: PersonX likes [exercise], as
a result, PersonX will go to the stadium.
[exercise] can be conceptualized as
These prompts are consistent with our descriptions in Section 4.1 and Section 4.2, where the task description is first presented, followed by humanauthored examples, and finally, the event we want to conceptualize or instantiate. We also leverage several tricks in the prompt, such as numbering the examples, generating concepts instead of hypernyms, and keeping the generated responses concise.
Finally, we parse the generations via manually defined rules and compile them into a dataset.
Additionally, we introduce some generation settings when prompting LLMs. For ChatGPT, we access it through the official OpenAI APIs2. The code of the accessed version is gpt-3.5-turbo-0613. We set the temperature to 1.0 and the maximum length for generated tokens to 200. To conceptualize all events in ATOMIC into 20 conceptualizations each, the time required for the distillation process is approximately ten days and the financial budget is around 1500 USD.
For LLAMA2, we access it via the Huggingface Library (Wolf et al., 2020). The code of the accessed model is meta-llama/Llama-2-13b-chat-hf3. When prompting, we use the Top-k sampling decoding strategy and set k = 10. We set the maximum length of generated tokens to 200. The models are hosted on sixteen NVIDIA-V100 GPUs, and
2https://chat.openai.com/ 3https://huggingface.co/meta-llama/Llama-2-13b-chat-hf
the time required to distill the entire dataset is approximately one month.
After collecting 20 conceptualizations for every head event in ATOMIC and further instantiating them to new entities and events, we construct an expanded knowledge base of ATOMIC. We also include more statistics, as shown in Table 7 and Table 8. For instantiations, they share the same relational distribution as abstract commonsense triples since we only instantiate them once. These statistics indicate that, compared to AbstractATOMIC, which is the only available conceptualization benchmark based on ATOMIC, CANDLE contains more abstract commonsense triples and many more unique conceptualizations. According to our results, it can also be expected that the abstract knowledge distilled from CANDLE is of better quality than AbstractATOMIC, which human annotations or any filtering have not verified.
For critic filtering, we use the state-of-theart conceptualization discriminator developed by Wang et al. (2023b). This discriminator is utilized to assess the plausibility of CANDLE distilled conceptualizations. It considers the original event, the instance being conceptualized, and the target concept as its inputs and generates a score ranging from 0 to 1 to represent plausibility. For instantiation, we use the pre-trained VERA model released by Liu et al. (2023). We convert the instantiated commonsense knowledge triple into a declarative statement and request an estimation of its plausibility from VERA. This estimation is provided as a score ranging from 0 to 1. The output scores from both models serve as the critical values assigned to each CANDLE distillation. These critical values are then subjected to further filtering based on various thresholds.
Additionally, following Wang et al. (2023d), we calculate the percentage of unique abstract concepts using BLEU soft uniqueness (Zhu et al., 2018; West et al., 2022). We define a concept, denoted as x, as unique if BLEU1(C, x) < 0.5, where C represents all concepts that share the same head event and identified instance with x in AbstractATOMIC. Here, 0.5 serves as an empirical threshold. Our distillation process yields 92.3% unique conceptualizations, indicating a significantly higher diversity than previous datasets.
Similarly, we evaluate the uniqueness of the newly introduced head events resulting from our chain of conceptualization and instantiation. To
determine uniqueness, we define an instantiated head event, referred to as hi′ , as unique if BLEU1(ho, hi′) < 0.5, where ho represents the original head event in ATOMIC. The threshold of 0.5 is an empirical threshold. Our empirical results demonstrate that 78.6% of the instantiated events are unique compared to ATOMIC, highlighting the effectiveness of CANDLE in enhancing the semantic coverage of the CSKB.

heading: B Task Setups

heading: B.1 CSKB Conceptualization
text: We follow the task definition of He et al. (2024) and Wang et al. (2023b) to formulate the CSKB conceptualization task. Specifically, conceptualizing an event-centric CSKB to derive abstract commonsense knowledge comprises two steps (He et al., 2024): event conceptualization and triple conceptualization, which correspond to two subtasks studied in this paper. Denote the triples in the original CSKB as Do = {(ho, r, t)|ho ∈ Ho, r ∈ R, t ∈ T}, where Ho, R, and T are the set of heads, relations, and tails in the original CSKB. The first step only operates on head events without considering the context in r and t. The goal of event conceptualization is to produce a conceptualized head event ha from the original head ho to represent an abstraction of ho. In the second step, the task is to verify whether the conceptualized head ha still makes sense in the context of r and t, as r and t will further restrict the level of abstractness in ha. Plausible (ha, r, t) triples will be considered as valid abstract commonsense knowledge. By enhancing the performance of discriminative models on these tasks, they can function as more precise critic filters and automate the conceptualization process of a CSKB when linked to concept taxonomies.

heading: B.2 Generative Commonsense Inference
text: The task of generative commonsense inference was studied by both Bosselut et al. (2019) and Hwang et al. (2021). It requires a generative model to complete the tail t of a commonsense assertion based on a given pair of head h and commonsense relation r. In this paper, we follow Hwang et al. (2021) and use ATOMIC2020 as the evaluation benchmark, in which the full testing set is used for model evaluation. The task of COMET is important in the domain of commonsense as it serves as a fundamental component for numerous high-level applications
that necessitate commonsense reasoning, such as zero-shot commonsense question answering with self-talk (Shwartz et al., 2020) and dynamic graph construction (Bosselut et al., 2021), narrative reasoning (Peng et al., 2022b), and dialogue generation (Tu et al., 2022). Improving COMET can potentially benefit other domains that require commonsense understanding.

heading: B.3 Zero-shot Commonsense QA
text: The task of zero-shot commonsense QA evaluates a model’s reasoning generalizability on unseen QA entries without any supervision signals from the corresponding annotated training data. Several methods have been proposed to tackle this task, including those by Shwartz et al. (2020); Bosselut et al. (2021); Kim et al. (2022); Shi et al. (2023). The most effective pipeline, as suggested by Ma et al. (2021), injects commonsense knowledge into language models via fine-tuning on QA pairs synthesized from knowledge in CSKBs. During the fine-tuning process, the head ho and relation r of a (ho, r, t) triple from a CSKB are transformed into a question using natural language prompts, with the tail t serving as the correct answer option. Distractors or negative examples are generated by randomly sampling tails from triples that do not share common keywords with the head. This fine-tuning procedure enhances the model’s knowledge not only for QA benchmarks constructed from CSKBs but also improves its ability to answer unseen commonsense questions in a more generalized manner. In this paper, we follow the task definition, model training, and model evaluation pipeline by Ma et al. (2021) to study the impact of distilling student models from CANDLE instantiations. For baselines, we compare models trained on QA pairs synthesized from ATOMIC, ATOMIC-10X, and AbstractATOMIC. For ATOMIC-10X, 0.9 is used as the critic filtering threshold.

heading: C Dataset Descriptions
text: This section covers additional details and statistics of datasets and benchmarks used in downstream task evaluations.

heading: C.1 CSKB Conceptualization
text: In CSKB Conceptualization tasks, we use the AbstractATOMIC (He et al., 2024) dataset as the evaluation benchmark. It is a benchmark dataset built upon ATOMIC and consists of event conceptualization data and abstract knowledge triples. The event conceptualizations are based on head events in ATOMIC, identified through syntactic parsing and matching with rules to search for concept candidates in Probase (Wu et al., 2012) and WordNet (Miller, 1995). The abstract knowledge triples connect conceptualized head events with their nonabstract counterparts from ATOMIC, forming commonsense knowledge at the concept level. Human annotations are used to verify the correctness of some conceptualizations and their resulting abstract commonsense triples. In total, 131K conceptualizations of 7K (45%) ATOMIC head events and 81K (1.3%) conceptualized triples are manually annotated, with a large number remaining unlabeled. The data is partitioned by following ATOMIC’s original split of head events. Detailed statistics are shown in Table 9. In this paper, we evaluate all models using the test set from the annotated subset as the evaluation data. Meanwhile, we obtain CANDLE distilled models using the training set from the annotated subset to fine-tune discriminative models pre-trained on CANDLE conceptualizations. Supervised baselines are trained on the training set of AbstractATOMIC, while semi-supervised baselines also leverage the unlabeled data.

heading: C.2 Generative Commonsense Inference
text: To evaluate COMET, we adopt the same evaluation setting employed by Hwang et al. (2021) for assessing commonsense generative models on the ATOMIC2020 dataset’s test set. We use the entire test set, consisting of 34,689 triples across 23 different commonsense relations, to ensure the robustness of the evaluation. Additionally, we use the full training set to fine-tune models that were pre-trained on various CSKBs and CANDLE instantiations to fit them into the benchmark.
Recently, West et al. (2023) successfully trained a powerful commonsense inference generator using an open-format symbolic knowledge distillation
framework. However, the dataset and models are not publicly released.

heading: C.3 Zero-shot Commonsense QA
text: We follow Ma et al. (2021); Wang et al. (2023a); Shi et al. (2023) and use the validation split of five commonsense QA benchmarks: Abductive NLI (aNLI; Bhagavatula et al., 2020), CommonsenseQA (CSQA; Talmor et al., 2019), PhysicalIQA (PIQA; Bisk et al., 2020), SocialIQA (SIQA; Sap et al., 2019b), and WinoGrande (WG; Sakaguchi et al., 2021). These benchmarks evaluate different aspects, including abductive reasoning, concept-level commonsense reasoning, physical commonsense understanding, emotional and social commonsense reasoning, and pronoun resolution. The validation splits are used as the official test sets may not be publicly available. Statistics on the number of QA pairs and the number of options per question are reported in Table 10.
D Implementation Details
This section provides additional implementation details in downstream task evaluations.
First, we use the Huggingface4 Library (Wolf et al., 2020) to build all models. We reproduce all baselines according to implementation details described in their original papers. The reported results are consistent with their original papers if the same experiment is included. For CANDLE distilled models, please refer to the subsections (Appendix D.1, D.2, D.3) below.
For methods involving LLMs, we use their instruction fine-tuned versions as the backbone for the baselines. For LLAMA2, the accessed version is meta-llama/Llama-2-7b/13b-chat-hf. For Mistral, we use mistralai/Mistral -7B-Instruct-v0.1. This remains consistent whether we prompt them directly or fine-tune them for downstream tasks, as we have observed that the instruction-finetuned versions generally result in better performance. For ChatGPT,
4https://huggingface.co/
we access it through Microsoft Azure APIs5. The code of the accessed version for ChatGPT is gpt-35-turbo-20230515, and for GPT4 is gpt-4-20230515. The maximum generation length is set to 100 tokens for all tasks. For fine-tuning LLAMA2 and Mistral, we use the open code base of LLaMa-Factory6. Please refer to the subsections below for hyperparameter settings.
All experiments are conducted on sixteen NVIDIA-V100 (32G) GPUs.
For baselines involving prompting LLMs, we follow the approach done by Robinson and Wingate (2023) and Chan et al. (2024a), where each task is formulated in either a generative format or as multiple-choice QA. Table 11 shows the prompts used in zero-shot prompting scenarios. To incorporate five-shot exemplars, we include five randomly
5https://azure.microsoft.com/en-us/products/ai-services/ 6https://github.com/hiyouga/LLaMA-Factory
selected examples from the training set of each benchmark. These examples are merged into the prompt using the same format as the question, with the addition of including the answer at the end. For chain-of-thought reasoning, we prompt LLM in a two-step inference process. In the first step, we delve deeper into the question by requesting an intermediate-step rationale. Then, in the second step, we seek an answer based on the question and the previous step’s response by asking LLM to answer “Yes or No only” or select the correct option from a set of answers directly.

heading: D.1 CSKB Conceptualization
text: For RoBERTa and DeBERTa-v3, we use a learning rate of 5e-6 and a batch size of 64. To optimize the models, we use an AdamW optimizer (Loshchilov and Hutter, 2019) and evaluate the model’s performance every 25 steps. The maximum sequence lengths for the tokenizers are set to 25 and 35

heading: Backbone: LLAMA2 (Touvron et al., 2023) 7B
text: for the two discriminative subtasks, respectively. Early stopping is used where the best checkpoint is selected when the largest validation accuracy is achieved. The models are trained on CANDLE distillation for one epoch and fine-tuned on the training set of AbstractATOMIC for one epoch.
For LLMs, such as LLAMA2 and Mistral, we use LoRA for fine-tuning, and the LoRA rank and α are set to 64 and 64. We use an Adam (Kingma and Ba, 2015) optimizer with a learning rate of 5e6 and a batch size of 64. The models are fine-tuned for two epochs, and the checkpoint with the highest validation set accuracy is selected. All experiments are repeated three times using different random seeds, and the average performances are reported.
For VERA, we follow the exact same implementation7 as released by Liu et al. (2023). To transform our binary classification subtasks into declarative formats, we begin by converting each piece of data into a declarative sentence using predefined natural language templates. Next, we create a corresponding negative statement by simply incorporating the word “not” into the correct sen-
7https://github.com/liujch1998/vera
tence. For instance, a pair of statements is: “PersonX enjoys drinking at the bar. Bar is a social gathering place.” and “PersonX enjoys drinking at the bar. Bar is not a social gathering place.” The accessed backbone model is liujch1998/vera, and all other hyperparameter settings follow the default implementation. The model is trained on CANDLE distillation for one epoch and then fine-tuned on AbstractATOMIC for another.

heading: D.2 Generative Commonsense Inference
text: For COMET, we implement the open-sourced code by Hwang et al. (2021) as our base to fine-tune the GPT2 model. The model is first pre-trained on CANDLE instantiations for one epoch, followed by fine-tuning on ATOMIC2020 for another epoch. An Adam (Kingma and Ba, 2015) optimizer is used with a learning rate of 1e-5 and a batch size of 32. A linear scheduler is used to decrease the learning rate gradually.
For LLAMA2-7B, we fine-tune it with the DeepSpeed framework (Aminabadi et al., 2022) by using FP16 as the precision. We optimize the model with an Adam (Kingma and Ba, 2015) optimizer with a learning rate of 1e-4 and a batch size of 64. The
maximum length for the input and generated sentence concatenation is 500. We warm up the model with 3000 steps and evaluate the model every 1000 steps. A linear scheduler is also used. The LoRA rank is set to 8, and the α is set to 32.
In Table 12, we present supplementary automatic evaluation results, including models that have been pre-trained solely on CSKBs and CANDLE instantiations without subsequent fine-tuning.

heading: D.3 Zero-shot Commonsense QA
text: For the task of zero-shot commonsense QA, we adopt the code base provided by Wang et al. (2023a)8 and Liu et al. (2023)9 to train two CANDLE distilled models. All hyperparameters and optimization strategies are kept unchanged from their original implementations as default settings. The models are trained for two epochs using QA pairs obtained from augmented-ATOMIC, including augmentations from ATOMIC-10X, AbstractATOMIC, and CANDLE instantiations.
Meanwhile, we present a comprehensive table presenting the results of all current methodologies for the task of zero-shot commonsense QA in Table 14. Notably, our CANDLE distilled models continue to exhibit strong performance compared to other models pre-trained on QA pairs sourced from multiple CSKBs. This serves as compelling evidence for the efficacy of CANDLE.

heading: E Annotation Details
text: This paper utilizes expert annotations to assess the quality of distilled conceptualizations and instantiations, as well as evaluate the generations of different models for the COMET downstream task. We
8https://github.com/HKUST-KnowComp/CAR 9https://github.com/liujch1998/vera
follow the annotation setting of Yu et al. (2023); Lu et al. (2024); Cheng et al. (2023), where four graduate students with ample experience in natural language processing research and expertise in commonsense reasoning are recruited as expert annotators to carry out the annotations. Their participation in the annotation process is voluntary and unpaid, in accordance with local laws, and is considered a contribution to this paper. Detailed instructions are provided to the annotators for each task, ensuring that they understand the requirements thoroughly. For CANDLE distillation evaluation, the annotators are asked to determine (1) the correctness of the conceptualizations and instantiations and (2) the plausibility of their formed triples. For COMET generation evaluation, they are asked to determine the plausibility of the generated triples. For each question, we also highlight the part to be considered by the annotators for their convenience. The annotation process is conducted independently, without any internal discussions among the annotators regarding the results. For each task, two annotators independently vote for each triple, and only when both annotators provide a positive vote will the triple be considered accepted or plausible. To prevent bias and ensure impartial results for CANDLE, the task input is randomly shuffled during the annotation process. As a result, the expert annotators achieve a pairwise agreement (IAA; Landis and Koch, 1977) of 0.80 and a Fleiss-kappa (Fleiss, 1971) of 0.61, indicating a remarkably high level of internal agreement.

heading: F Ablation Study
text: In this section, we examine the impact of our critic filters on the ablation of CANDLE. Specifically, we investigate the effect of different levels of critic threshold or completely abandoning critic filtering on downstream tasks. We conduct four experiments with different settings, denoted as t ∈ {0, 0.5, 0.7, 0.9}, where t = 0 corresponds to abandoning critic filtering and using all distilled knowledge as complementary training data. For detailed statistics, please refer to Table 1. For each value of t, we select the distilled knowledge with a critic score higher than t and utilize it as complementary training data to train student models for the three downstream tasks. We employ the same training strategies described in the main body of the paper. In the case of CSKB conceptualization and zero-shot commonsense QA tasks, we utilize
DeBERTa-v3-large as the backbone model, with accuracy as the evaluation metric. For COMET, we use GPT2 and evaluate using the BERTScore as the evaluation metric. The results are visualized in Figure 4. Our analysis reveals a consistent trend where higher threshold values yield improved performance, indicating the reliability of our critic filter. However, it is worth noting that setting the threshold above 0.9 may potentially lead to even better performance. Nevertheless, such a trade-off comes with a downside: it reduces the amount of usable knowledge in each distillation round, which can impede the iterative process. The reason for this is that when the number of distilled conceptualizations and instantiations decreases significantly in each round, CANDLE is unable to incorporate new instantiated data for future distillation iterations. As a result, the “convergence” of those high-critic data occurs prematurely in CANDLE.

heading: G Case Study
text: We present some examples in Table 13 to show conceptualizations and instantiations generated by CANDLE, along with their corresponding critic values assigned by our critic-filtering discriminators. It can be observed that both ChatGPT and LLAMA2 exhibit the ability to generate highquality knowledge based on given instructions. Furthermore, they can introduce novel conceptualizations and events during the distillation chain, effectively meeting our expectations of CANDLE. Future works can investigate the feasibility of incorporating conceptualization and abstract knowledge into more downstream tasks, such as metaphysical reasoning (Wang and Song, 2024), complex reasoning (Fang et al., 2024; Bai et al., 2023; Ding et al., 2023), theory-of-mind reasoning (Chan et al., 2024b), text to table generation (Deng et al., 2024), and commonsense knowledge graph denoising (Deng et al., 2023).

heading: Supervised Learning & Human Performance

heading: Large Language Models

heading: Pre-trained Language Models

heading: Future Work
text: Future works can investigate the feasibility of incorporating conceptualization and abstract knowledge into more downstream tasks, such as metaphysical reasoning (Wang and Song, 2024), complex reasoning (Fang et al. , 2024; Bai et al. , 2023; Ding et al. , 2023), theory-of-mind reasoning (Chan et al. , 2024b), text to table generation (Deng et al. , 2024), and commonsense knowledge graph denoising (Deng et al. , 2023)