COCONUT: Contextualized Commonsense Unified Transformers for Graph-Based Commonsense Augmentation of Language Models (2024)

Abstract: In this paper, we introduce COCONUT to effectively guide the contextualization of structured commonsense knowledge based on large language models. COCONUT employs a contextualized knowledge prompting scheme to gather high-quality contextualization examples from a large language model. These examples are subsequently distilled into small language models to enhance their contextualization capability. Extensive evaluations show that COCONUT considerably improves commonsense reasoning performance across diverse benchmarks, models, and settings, exhibiting its flexibility and universality in generating contextualized commonsense knowledge. Notably, COCONUT consistently outperforms the stateof-the-art technique by an average of 5.8%1.

heading: 1 Introduction
text: Commonsense reasoning constitutes a significant challenge within natural language processing. While scaling language models using considerably more data and parameters has led to significant improvements in commonsense reasoning tasks (Brown et al., 2020), several studies have demonstrated that pre-trained language models possess a limited understanding of commonsense knowledge (Sakaguchi et al., 2020; Talmor et al., 2022). These have triggered approaches to integrate external knowledge into language models to improve their commonsense reasoning abilities.
To enhance the commonsense capability of language models, typical approaches draw commonsense knowledge from symbolic commonsense knowledge graphs (CSKGs) (Speer et al., 2017; Hwang et al., 2020), which are repositories encapsulating hand-crafted commonsense knowledge about objects, concepts, and events. These approaches augment language model representations
1The code is available at https://github.com/ irishev/Coconut
with the structural and relational information in CSKGs (Lin et al., 2019; Zhang et al., 2022). Even though there exists an open question regarding whether pre-trained language models already encode the knowledge in CSKGs, substantial research has indicated that these approaches (Zhou et al., 2021; Lourie et al., 2021; Yasunaga et al., 2021; Zhang et al., 2022) facilitate language models to utilize the knowledge, leading to improvements on commonsense reasoning performance.
These approaches essentially involve contextualization, which refers to the interpretation and application of knowledge tuples within the specific context provided by a commonsense reasoning example. Contextualization is a crucial but difficult step due to the diversity and obscurity of the underlying commonsense knowledge that grounds
5815
the reasoning process (Liu et al., 2022a). Since most CSKGs provide simple, abstract descriptions of commonsense knowledge without specifying where and how the knowledge can be applied, existing methods largely lean on language models for contextualization. However, considering the limited commonsense capabilities of language models, this could lead to spurious contextualization, potentially degrading the commonsense reasoning performance. Particularly, small language models, known to have poor knowledge and reasoning ability to fill empty contexts, are expected to be more vulnerable to spurious contextualization.
In this paper, we propose a novel framework, called COCONUT (COntextualized COmmonseNse Unified Transformers), which augments language models with contextualized commonsense knowledge. COCONUT contextualizes the structured knowledge in CSKGs for specific commonsense questions, trained by explicit, direct guidance from large language models. To overcome the lack of data and costly human annotations, we present a contextualized knowledge prompting scheme, where humans construct a few contextualization examples and then large language models extend the human-curated data into a million-scale via prompting. Following the scheme, we generate examples to guide the contextualization, and subsequently train COCONUT models on the generated data via symbolic knowledge distillation (West et al., 2022).
We extensively evaluate and analyze COCONUT with popular QA models on diverse commonsense reasoning benchmarks. Experimental results show that COCONUT delivers significant performance improvements on commonsense reasoning in both zero-shot and fine-tuned settings, demonstrating the efficacy of the proposed framework. Notably,
COCONUT consistently outperforms state-of-theart knowledge augmentation methods by an average of 5.8%. The main contributions of this work are summarized as follows:
• We propose COCONUT, a novel framework that augments models with contextualized commonsense knowledge from structured knowledge in CSKGs.
• We present a novel contextualized knowledge prompting scheme to generate contextualization examples from commonsense questions and CSKGs using a large language model.
• We demonstrate the efficacy of providing contextualized knowledge through extensive experiments.

heading: 2 Contextualized Knowledge Prompting
text: In this section, we present a contextualized knowledge prompting scheme, which generates contextualization examples using commonsense questions and CSKGs. Specifically, we prompt language models to generate examples of context-aware link and knowledge contextualization, as illustrated in Figure 2.

heading: 2.1 Notation
text: We first define ConceptNet, utilized as a CSKG in this work, as G = (V, E), where V is the set of nodes and E is the set of edges. Its knowledge tuple {s, r, o} ∈ E consists of a source concept s ∈ V , a relation type r, and a target concept o ∈ V . For example, given a knowledge tuple {food, LocatedAt, refrigerator}, the source concept
“food” is related to the target concept “refrigerator” by the relation “LocatedAt”, which represents that food is typically located at a refrigerator. In addition, a multiple-choice commonsense questionanswering task requires predicting the answer a given a question qi and a finite set of candidates Ci = {c0i , c1i , ...cNi }, where N is the number of candidates and a ∈ Ci. For example, a CommonsenseQA example consists of the question “The accountant used a calculator regularly, he kept one at home and one at the what?”, the set of candidates {“desk drawer”, “desktop”, “office”, “wristwatch”, “city hall”}, and the answer “office”.

heading: 2.2 Generating Context-aware Link Examples via Prompting
text: Typical knowledge integration methods (Yasunaga et al., 2021, 2022) retrieve the knowledge by linking entities in a question and its candidates with the matching nodes in CSKGs and then selecting the knowledge tuples that connect an entity in the question with an entity in the candidates. However, numerous irrelevant entities may be linked during the process, resulting in excessive and redundant knowledge tuples to be retrieved.
To address the issue, we present a context-aware knowledge retrieval method to link entities highly relevant to the context. We first identify all entities that appear in a question qi and each of its candidates cji as follows:
Zqi = {z|z ∈ ngram(qi), z ∈ V } Z cji = {z|z ∈ ngram(cji ), z ∈ V }, (1)
where ngram(·) is a function that extracts word n-grams from an input text. Then, we manually select relevant and helpful entities in qi. Following this process, we craft ten examples per commonsense question-answering task, and write a prompt template as follows:
Find words or phrases relevant to the question. Examples:
Question: How does getting paid feel? Keywords: paid, feel
…
Question: Where can meat last a long time? Keywords:
where the blue, green, and orange texts denote an instruction, a demonstration, and a new example, respectively. Given this prompt template filled with a new question, a large language model generates relevant entities for every question in a commonsense question-answering dataset as follows:
tl ∼ Plm(t|T (qi)) (2)
where Plm is the probability distribution of the language model prediction and T (·) is a function that fills the prompt template using a given input. The generated result tl is a textual sequence of relevant entities “x1qi , x 2 qi , ...” separated by comma. By using the set of generated entities Xqi , we can extract relevant knowledge tuples as follows:
Kji = ⋃
x∈Xqi
⋃
y∈Z c j i
path(x, y, E) (3)
where path(s, t, E) is a function that finds the set of knowledge tuples connecting s to t in the set of edges E. It is noteworthy that we generate link examples only for commonsense questions involving more than four words.

heading: 2.3 Generating Knowledge Contextualization Examples via Prompting
text: We generate contextualized knowledge statements from retrieved knowledge tuples using a large language model. We first construct few-shot examples and design templates to prompt effective contextualized knowledge. Our primary goal is to provide
more detailed information about how a knowledge statement from CSKGs can be connected to a given situation. To achieve this, we first verbalize the retrieved knowledge tuples using our templates and craft ten demonstration examples per task, each of which provides a description that contextualizes the knowledge. We utilize two prompt formats considering two possible commonsense generation scenarios: (1) commonsense fact generation and (2) contrastive commonsense generation.
Commonsense fact generation addresses reasoning over common facts such as “Birds have two wings.” or “Winning lottery tickets give a lot of money.”. Humans write contextualization examples from a question, its answer, and the retrieved knowledge. We convert the Winogrande (Sakaguchi et al., 2020), CommonsenseQA 2.0 (Talmor et al., 2022), Com2sense (Singh et al., 2021), ComVE (Wang et al., 2020), and GenericsKB (Bhakthavatsalam et al., 2020) datasets into the true or false examples and then utilize them as the seeds. An example prompt template used in commonsense fact generation is as follows:
Write new knowledge based on the given knowledge to explain the correctness of a question. Examples:
Question: The letter that Joel has written … Answer: True Given Knowledge: writer is related to write … New Knowledge: Joel being the writer of the letter is supported by the statement that …
…
Question: Sarah Jane’s watch smashed when … Answer: False Given Knowledge: smash is related to activity … New Knowledge:
Since the knowledge tuple extraction using the candidates of converted examples (true or false) may not provide meaningful relational knowledge, we consider the generated question entities Xqi as the candidate entities and then extract knowledge tuples Ki. Given the prompt template filled with a new question qi, its answer a, and the set of relevant knowledge tuples Ki, a large language model generates contextualized knowledge as follows:
tc ∼ Plm(t|T (qi, a,Ki)) (4)
The generated result tc is a contextualized knowledge statement.
Contrastive commonsense generation addresses reasoning by comparing the plausibilities
of multiple candidates, such as “A rose garden provides a vast and continuous source of nectar to bees, while a bouquet of flowers is not a natural environment for bees”. Humans write a reason why the answer candidate is more plausible than the other candidate based on the context and retrieved knowledge. We utilize the CommonsenseQA (Talmor et al., 2019), PhysicalIQA (Bisk et al., 2020), SocialIQA (Sap et al., 2019), OpenBookQA (Mihaylov et al., 2018), QASC (Khot et al., 2020), ARC (Bhakthavatsalam et al., 2021), and SyntheticQA (Wang et al., 2023) datasets. An example prompt template used in contrastive commonsense generation is as follows:
Write new knowledge based on the given knowledge to explain the correct and wrong options for a question. Examples:
Question: How does getting paid feel? Correct Option: satisfaction Wrong Option: bill collectors to happy Given Knowledge: pride is related to satisfaction. … New Knowledge: Getting paid is intrinsically linked to the feeling of satisfaction, …
…
Question: Where can meat last a long time? Correct Option: freezer Wrong Option: butcher shop Given Knowledge: meat is located at freezer. … New Knowledge:
Given this prompt template filled with a new question qi, its answer cmi , one of its wrong candidate cni , and the set of relevant knowledge tuples J = Kmi ∪Kni , a large language model generates contextualized knowledge as follows:
tc ∼ Plm(t|T (qi, cmi , cni , J)) (5)
Note that the answer information is only used in the prompting stage to generate more accurate contextualization examples, and completely excluded in the training and inference with COCONUT models.

heading: 3 COCONUT
text: In this section, we introduce COCONUT, a framework designed to augment language models with contextualized commonsense knowledge. We present two COCONUT models: (1) COCONUT linker and (2) COCONUT generator. These two models generate contextualized knowledge from commonsense reasoning examples, as described in Figure 3. Then, we describe the integration of the contextualized knowledge generated by the COCONUT models.

heading: 3.1 COCONUT Linker
text: We train a COCONUT linker on the generated context-aware link examples in §2.2. Given a question qi, the objective function for training can be formulated as:
Llink = Etl∼Plm(t|T (·))[− |tl|∑
j=1
logPl(t j l |qi, t <j l )],
(6) where Pl is the probability distribution of the COCONUT linker prediction. Using a trained COCONUT linker, we can generate relevant entities from a new question qh without task-specific prompt templates:
t̂l ∼ Pl(t|qh) (7)
From the set of relevant entities X̂qh in t̂l, we can extract relevant knowledge tuples as follows:
K̂jh = ⋃
x∈X̂qh
⋃
y∈Z c j h
path(x, y, E) (8)

heading: 3.2 COCONUT Generator
text: We train a COCONUT generator on the generated knowledge contextualization examples in §2.3. Given a question qi, its candidates Ci, and its relevant knowledge tuples Ki, the objective function for training can be formulated as:
Lgen =Etc∼Plm(t|T (·))
[− |tc|∑
j=1
logPg(t j c|qi, Ci,Ki, t<jc )],
(9)
where Pg is the probability distribution of the COCONUT generator prediction. Using a trained COCONUT generator, we can generate contextualized knowledge from a new question qh, its candidates Ch, its set of relevant knowledge tuples Kh:
t̂c ∼ Pg(t|qh, Ch,Kh) (10)

heading: 3.3 Contextualized Knowledge Integration
text: We prompt an inference model, i.e., a language model or a question-answering model, by concatenating each generated knowledge statement kmi to the question qi and candidates Ci. We concatenate a question and its generated knowledge statement by following the default question-answering prompt format of the inference model. For example, UnifiedQA (Khashabi et al., 2020) uses a format that involves context, question, and candidate
fields with symbols in order, while using “\n” as a delimiter. Therefore, the concatenation process qi ◦ Ci ◦ kmi = “{qi} \n {kmi} \n (A) {c0i} ...”.
Following the pre-defined format, we calculate the probability of each candidate for each concatenated knowledge statement and average the probabilities to aggregate the scores. Given the generated knowledge set K̂i = {k̂0i , k̂1i , ..., k̂Mi }, the score of a candidate cni is calculated as follows:
score(qi, cni ,Ki) =
∑M j pinf (c n i |qi, kji )
M . (11)
where pinf denotes an inference model. The final prediction ĉi is the candidate that maximizes the score as follows:
ĉi = argmaxx∈Ci score(qi, x,Ki). (12)

heading: 4 Experiments
text: COCONUT establishes new state-of-the-art results on our evaluation benchmarks, significantly improving the commonsense reasoning performance of diverse inference models.

heading: 4.1 Experimental Setup
text: Datasets. Consistent with Liu et al. (2022a), we first evaluate the commonsense reasoning performance on eight seen datasets: OpenBookQA (Mihaylov et al., 2018), ARC easy/hard (Bhakthavatsalam et al., 2021), CommonsenseQA (Talmor et al., 2019), QASC (Khot et al., 2020), PhysicalIQA (Bisk et al., 2020), SocialIQA (Sap et al., 2019), and Winogrande (Sakaguchi et al., 2020). In addition, we evaluate the performance on four unseen commonsense reasoning datasets: NumerSense (Lin et al., 2020), RiddleSense (Lin et al., 2021), QuaRTz (Tafjord et al., 2019), and HellaSwag (Zellers et al., 2019). The official train, dev, and test splits of these benchmarks are employed for training and evaluation purposes.
Models. For prompting, we use LLaMA-65B (Touvron et al., 2023). We evaluate two combinations of the COCONUT linker and generator: Using T5-large and T5-3B as the generators, which are denoted as COCONUT-large and COCONUT3B, respectively, while fixing the T5-small-based linker. For inference models, we mainly use UnifiedQA (Khashabi et al., 2020) and UnifiedQAv2 (Khashabi et al., 2022)). Note that we do not finetune inference models on downstream tasks unless mentioned otherwise.
Baselines. We compare COCONUT with diverse knowledge augmentation methods, categorized as:
• Knowledge prompting methods involve generated knowledge prompting (GKP) with GPT3 (Liu et al., 2022b), Self-talk (Shwartz et al., 2020), DREAM (Gu et al., 2022), Rainier (Liu et al., 2022a), and Rainier with Vera (Liu et al., 2023) where knowledge descriptions are elicited from other language models. We further use LLaMA-65B prompted with knowledge tuples in ConceptNet as the baseline of augmented prompting of knowledge.2
• Knowledge graph reasoning models incorporate external CSKGs to enhance the limited information present in the input texts, such as KagNet (Lin et al., 2019), QA-GNN (Yasunaga et al., 2021), GreaseLM (Zhang et al., 2022), and Dragon (Yasunaga et al., 2022).
• Commonsense aware language models are trained using an external commonsense corpus or datasets to embed knowledge into their parameters, such as CALM (Zhou et al., 2021) and Unicorn (Lourie et al., 2021).
• Retrieval augmented models focus on retrieving relevant knowledge from commonsense corpora, such as RACo (Yu et al., 2022) and FiD (Izacard and Grave, 2021).

heading: 4.2 Main Results
text: We first compare COCONUT with state-of-the-art knowledge prompting methods using UnifiedQAlarge (Khashabi et al., 2020). As shown in Table 1, COCONUT-large surpasses the best baseline,
2We use the input format consistent with that of COCONUTGEN (i.e., Question + Options + Verbalized ConceptNet Paths). We use contextualization demonstrations that are used in contextualized knowledge prompting while excluding answer information. We use the exact-match-based knowledge retrieval method.
Rainier-large + Vera, on seen datasets by an average of 5.8%. In addition, COCONUT-3B outperforms GPT-3 Davinci + Vera using 60 times more parameters by an average of 4.9%. We observe that directly prompting LLaMA-65B with ConceptNet knowledge tuples during inference is not ideal, since it shows performance worse than that of COCONUT-large and COCONUT-3B, while using significantly more parameters and computations. These results show COCONUT’s capability to effectively and efficiently augment language models with commonsense knowledge by learning how to contextualize structured knowledge.
Moreover, as shown in Table 2, COCONUT consistently stands out on unseen datasets. Particularly, COCONUT-large excels over GPT-3 Curie using by an average of 2.5%, using significantly fewer parameters. The superior performance on unseen datasets shows the generalization capability of COCONUT, rooted in CSKGs with the general, widely applicable commonsense knowledge.
Our evaluation of COCONUT extends to UnifiedQAv2 (Khashabi et al., 2022). As shown in Table 3, COCONUT effectively augments Uni-
fiedQAv2 fine-tuned without knowledge, outperforming all the knowledge graph reasoning models, commonsense aware language models, and retrieval augmented models. These results underscore COCONUT’s advantage in providing contextualized knowledge that is readily integrated by a wide range of inference models.

heading: 4.3 Human Evaluation
text: We conduct a human evaluation on CommonsenseQA to study the quality of generated knowledge and the interpretability of its impact on task performance. We sample 1,200 knowledge statements generated by COCONUT-large (100 knowledge statements per each evaluation dataset) and evaluate in terms of relevance, factuality, and helpfulness. As shown in Figure 4, we can observe that most generated knowledge is related, factually correct, and helpful for the model’s reasoning. Specifically, COCONUT achieves 99.2% relevance, 71.7% factuality, and 64.2% helpfulness. Among the rectifying knowledge, 89% are deemed helpful by humans, and among the misleading knowledge, 81% are deemed harmful. These results show that
COCONUT effectively helps the inference models by generating relevant and accurate knowledge.

heading: 4.4 Analysis
text: Analysis on generated knowledge. Table 4 illustrates the knowledge descriptions generated by COCONUT and GPT-3-Davinci from Liu et al. (2022b) on examples from the CommonsenseQA validation set. From the generation results, we can observe that COCONUT generates more detailed knowledge descriptions about objects and their interactions. In contrast, GPT-3 leans towards providing descriptions of broad and general knowledge. Since inference models are possibly deficient in specific knowledge or reasoning processes required to infer answers from the provided knowledge, the lack of detailed descriptions can result in spurious contextualization, thereby degrading performance. Indeed, the UnifiedQA-large model finds the correct answer with knowledge generated by COCONUT, while failing with knowledge generated by GPT-3.
3We use the exact-match-based knowledge retrieval method.
Scaling trends of the inference model. We compare the commonsense reasoning performance of knowledge prompting methods with varying sizes of inference models. The results are presented in Figure 5. We observe two dominant trends from the results. Firstly, on both the seen and unseen datasets, the performance improvements from COCONUT-large are consistently better than the baseline knowledge prompting methods with a similar size. Secondly, when the inference models have a significantly larger size than that of COCONUT, we still observe performance improvements, showing that COCONUT may handle the knowledge absent in language models by integrating CSKGs.
Ablation Study. To better understand the contributions to performance improvements, we execute a series of ablation studies on COCONUT. As shown in Tables 5 and 6, we observe that both COCONUTGEN and COCONUTLINK contribute to the performance improvement. Particularly, COCONUTLINK achieves more significant improvements on the commonsense benchmarks with relatively long contexts, such as PIQA and SIQA, possibly due to its selection of important concepts.

heading: 5 Related Work
text: While large language models yield state-of-theart performance on many commonsense reasoning tasks, their pre-training objectives do not explicitly guide them to reason using commonsense knowledge (Zhou et al., 2021), resulting in unsatisfactory performance in many real-world scenarios (Talmor et al., 2022; Zhu et al., 2023). To address the limitation, existing work has explored augmented language models to improve their commonsense reasoning ability. A typical approach is incorporating external knowledge from CSKGs, thereby supplementing the limited textual information (Lin et al., 2019; Yasunaga et al., 2021; Zhang et al., 2022; Yasunaga et al., 2022). Another approach involves training a language model on commonsense corpora (Lourie et al., 2021; Zhou et al., 2021). Recently, a line of research (Shwartz et al., 2020; Paranjape et al., 2021; Wei et al., 2022; Liu et al., 2022b) has proposed to generate knowledge by prompting language models due to the lack of scalability in utilizing CSKGs. Some recent methods have explored retrieving in-domain commonsense documents from a task-relevant corpus to improve commonsense reasoning capabilities (Wang et al., 2021; Li et al., 2021; Yu et al., 2022).
COCONUT provides two distinct advantages over existing commonsense augmentation methods. First, COCONUT alleviates the inherent limitation of CKSGs, the lack of coverage, by introducing language models in contextualization. Since language models have wide coverage and strong expressive power, they can effectively complement the coverage of CSKGs. Second, COCONUT utilizes CSKGs as pivots in knowledge generation, which facilitates to generate a wide range of accurate knowledge and achieves generalization on small language models.

heading: 6 Conclusion
text: In this paper, we have proposed COCONUT that contextualizes structured knowledge from CSKGs, guided by large language models. Our experimental results have verified that COCONUT outperforms state-of-the-art knowledge augmentation methods on diverse commonsense benchmarks. These show that large language models can explicitly guide the contextualization, leading to significant improvements in commonsense reasoning. Furthermore, our analyses suggest that prompting with structured knowledge may be a promising approach to address hallucination in knowledge prompting.
Limitations
While we have demonstrated that COCONUT effectively improves the commonsense reasoning performance by integrating contextualized commonsense knowledge from CSKGs, there are some limitations that present promising avenues for future research. First, COCONUT can generate unsafe or nonfactual knowledge, inheriting the limitation from language models. To address this, we plan to investigate the alignment of COCONUT with social, culture-specific, and ethical values. In addition, ConceptNet utilized by COCONUT involves limited types of knowledge, posing limitations in addressing domain-specific, eventual, or factual knowledge. Thus, we plan to extend the source of structured knowledge by introducing more diverse CSKGs, and to integrate retrieval augmentation from large knowledge corpora.

heading: Acknowledgements
text: This work was supported by the Basic Research Program through the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (2021R1A2C3010430) and Institute of Information Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. RS-2019II190079, Artificial Intelligence Graduate School Program (Korea University)).

heading: A Technical Appendix
text: A.1 Additional Experimental Details Implementation details. We implement COCONUT on PyTorch (Paszke et al., 2019) and HuggingFace Transformers (Wolf et al., 2020). All our experiments are conducted on four NVIDIA RTX A6000 GPUs. We train the models in bfloat16 mixed-precision for efficiency. The training of COCONUT-large took 117 hours, while that of COCONUT-3B took 359 hours. The hyperparemeter settings are described in Table 7. The verbalization templates are shown in Table 8.
Datasets. Detailed information of datasets are shown in Table 9.
Human Evaluation. We asked three NLP experts to annotate the quality of generated knowledge. We randomly selected 1,200 examples, 100 examples per dataset. Each annotator evaluated the quality of knowledge statements along three axes: (1) Relevance: whether it is relevant to the situation in a question; (2) Factuality: whether it contains only correct statements; and (3) Helpfulness: whether it helps solve a question correctly. For each example, the annotator chose “good” or “bad”. We do not reveal whether the knowledge rectifies or misleads the model prediction for objectivity.
A.2 Additional Results and Analyses Extended experimental results. Full results on seen and unseen datasets are reported in Table 10 and 11, respectively.
Qualitative analysis. Tables 12 and 13 present examples showcasing both the strengths and weaknesses of contextualized knowledge prompting and COCONUT. In the first two examples of Table 12, the contextualized knowledge prompting effectively generates well-contextualized knowledge descriptions that suit the implicit intentions of the questions, while knowledge tuples barely provide the information to distinguish the answer and wrong options. Yet, some examples are not as favorable, as shown in the last two examples. In Table 13, we observe several undesirable cases from the generation results of COCONUT including a direct guidance to the answer, a knowledge description seemingly plausible but not relevant to the question, and nonsense descriptions.

heading: Future Work
text: Limitations
While we have demonstrated that COCONUT effectively improves the commonsense reasoning performance by integrating contextualized commonsense knowledge from CSKGs, there are some limitations that present promising avenues for future research. First, COCONUT can generate unsafe or nonfactual knowledge, inheriting the limitation from language models. To address this, we plan to investigate the alignment of COCONUT with social, culture-specific, and ethical values. In addition, ConceptNet utilized by COCONUT involves limited types of knowledge, posing limitations in addressing domain-specific, eventual, or factual knowledge. Thus, we plan to extend the source of structured knowledge by introducing more diverse CSKGs, and to integrate retrieval augmentation from large knowledge corpora