Dense-ATOMIC: Towards Densely-connected ATOMIC with High Knowledge Coverage and Massive Multi-hop Paths
Year: 2023

Abstract:
ATOMIC is a large-scale commonsense knowledge graph (CSKG) containing everyday ifthen knowledge triplets, i.e., {head event, relation, tail event}. The one-hop annotation manner made ATOMIC a set of independent bipartite graphs, which ignored the numerous links between events in different bipartite graphs and consequently caused shortages in knowledge coverage and multi-hop paths. In this work, we aim to construct Dense-ATOMIC with high knowledge coverage and massive multi-hop paths. The events in ATOMIC are normalized to a consistent pattern at first. We then propose a CSKG completion method called Rel-CSKGC to predict the relation given the head event and the tail event of a triplet, and train a CSKG completion model based on existing triplets in ATOMIC. We finally utilize the model to complete the missing links in ATOMIC and accordingly construct Dense-ATOMIC. Both automatic and human evaluation on an annotated subgraph of ATOMIC demonstrate the advantage of Rel-CSKGC over strong baselines. We further conduct extensive evaluations on DenseATOMIC in terms of statistics, human evaluation, and simple downstream tasks, all proving Dense-ATOMIC’s advantages in Knowledge Coverage and Multi-hop Paths. Both the source code of Rel-CSKGC and Dense-ATOMIC are publicly available on https://github.com/ NUSTM/Dense-ATOMIC.

heading: 1 Introduction
text: ATOMIC is a large-scale human-annotated commonsense knowledge graph focusing on the inferential knowledge in social life (Sap et al., 2019). It consists of nine if-then relation types describing the causes, effects, agent, stative, and theme of an event. The research on ATOMIC has drawn more and more attention in recent years. An increasing number of downstream tasks, including
∗*Corresponding author
commonsense reasoning (Yu et al., 2022), storytelling (Brahman and Chaturvedi, 2020), question answering (Heo et al., 2022), dialog generation (Wu et al., 2022), etc., have improved their performances by acquiring and utilizing the commonsense knowledge from ATOMIC.
Currently, ATOMIC was constructed under onehop annotations. It began with 24,000 pre-defined base events and nine relation types. For each base event and each relation, the annotators were asked to write a possible tail event based on one-hop reasoning. As shown in Figure 1, given the base event “X asks Y to marry”, the annotated tail events can be “loving” under the relation of “xAttr”, “smiles” under the relation of “xEffect”, and “says yes” under the relation of “oEffect”.
In such a one-hop annotation manner, each base event and its related annotated tail events shape a bipartite graph containing only B-to-A links, where B denotes the Base event and A denotes the Annotated tail event. Thereby, the whole graph of ATOMIC can be viewed as a set of B-to-A bipartite graphs, while the B-to-B, A-to-B and A-to-A links between different bipartite graphs were almost ignored. In Figure 1, the dashed lines illustrate such missing links in ATOMIC, e.g., an annotated tail event “in front of Y” and a base event “X asks Y to marry” in two different bipartite graphs miss a link of the “xIntent” relation.
This leads to two shortcomings of ATOMIC. Firstly, with only B-to-A links, ATOMIC contains very few multi-hop paths, since an annotated tail event cannot become the head event of a triplet. Secondly, missing B-to-B, A-to-B and A-to-A links cause unsatisfactory knowledge coverage, despite its high-quality human-annotated commonsense knowledge. Both shortcomings limit the potential of ATOMIC in practical applications. Intuitively, an ideal CSKG requires high knowledge coverage to meet the needs of various tasks, and massive multi-hop paths to understand the evolu-
13292
tion between different events.
In this work, we aim to construct a denselyconnected ATOMIC. The key is to complete different types of missing links, leading to denser ATOMIC with high knowledge coverage and massive multi-hop paths. We achieve this goal through three main steps: Normalizing Tail Events, Training a Relation Prediction Model and Constructing Dense-ATOMIC.
Firstly, most of the annotated tail events in ATOMIC have different patterns to the base events, so we normalize annotated tail events in ATOMIC to a consistent pattern (“Subject + Verb + Object”), to facilitate subsequent CSKG completion. Specific relations are also grouped to mitigate ambiguity.
Secondly, we train a relation prediction model based on a set of existing triplets in ATOMIC to infer the missing links on the whole graph, i.e., CSKG completion upon ATOMIC. To the best of our knowledge, most of the existing studies for CSKG completion utilized the translation based methods, which formalized the CSKG completion as a tail event ranking task given the head event and the relation. A graph convolutional network (GCN) was mostly employed to encode the graph embeddings of events, but its performance is unsatisfactory since the sparsity of ATOMIC limits the information propagation on the GCN (Malaviya et al., 2020). In contrast, in this work, we propose a method called Rel-CSKGC, which regards CSKG
completion as a relation prediction problem given the head event and the tail event, and accordingly train a CSKG completion model based on ATOMIC.
Finally, based on the CSKG completion model, we construct Dense-ATOMIC by inferring the missing links on ATOMIC. Figure 1 illustrates the main differences between ATOMIC and Dense-ATOMIC.
We conduct extensive evaluations towards the Rel-CSKGC method and the constructed DenseATOMIC, respectively.
First, we compare Rel-CSKGC with several newly proposed relation prediction methods and translation based methods. Both automatic evaluation on an annotated subgraph and human evaluation on 500 sampled triplets show the advantage of Rel-CSKGC for completion on ATOMIC .
Next, we evaluate Dense-ATOMIC from the perspectives of knowledge coverage and multi-hop paths respectively. Extensive experiments are conducted in terms of statistics, human evaluation, and simple downstream tasks. The results demonstrate that Dense-ATOMIC surpasses ATOMIC in terms of triplet counts by an order of magnitude, and multi-hop paths by more than two orders of magnitude, respectively, while at the same time maintaining its quality.

heading: 2 Approach
text: Figure 2 illustrates the procedure of constructing Dense-ATOMIC, consisting of three main steps:
Normalizing Tail Events, Training a Relation Prediction Model, and Constructing Dense-ATOMIC.

heading: 2.1 Normalizing Tail Events
text: ATOMIC contains only B-to-A triplets. A CSKG completion model trained with B-to-A triplets is inapplicable to predict B-to-B, A-to-A, and A-toB links, since base events (usually sentences) and annotated tail events (usually phrases or words) have different patterns. This results in a shortage of knowledge coverage and multi-hop paths during the completion.
To this end, we propose Normalizing Tail Events to convert annotated tail events to the same pattern as the base events, including subject removal, third person singular form conjugation, subject recovery, and relation grouping.
Subject Removal For a few annotated tail events being complete sentences, we perform dependency tree parsing and part-of-speech tagging with CoreNLP (Manning et al., 2014) and remove subjects based on the two kinds of structure patterns, which makes the nodes in the graph become a uniform pattern and benefits the subject recovery process. For example, given a tail event “He smiles”, we first remove the subject “He” and convert it to a universal expression “Y smiles” in the subject recovery process.
Third Person Singular Form Conjugation In our preliminary experiments, a CSKG completion model tends to correlate phrases starting with “to” with relations such as “xWant”, “xIntent”, so we leverage WordNet (Miller, 1995) to acquire the verb root and add the suffix (-s, -es, etc.) according to English grammar.
Subject Recovery We add subjects to processed annotated tail events based on different relations.
Relation Grouping Both “xWant” and “xEffect” describe the possible subsequent events, distinguished by “to” representing subject will. After third person singular form conjugation, the two relations may lead to ambiguity. We perform relation grouping for all these relations to mitigate ambiguity. “xEffect” and “xWant” form “xAfter” describing what will happen to X. “oEffect” and “oWant” form “oAfter” describing what will happen to Y. “xAttr” and “xReact” form “xPersona” describing how X feels or is described. It should be noted that the relation grouping process leads to a non-serious problem, i.e., the grouped relation cannot distinguish between subjective and objective semantics. However, it mitigates ATOMIC’s sparsity issue and improves the performance of the relation prediction model.
Due to the page limitation, the pseudo-code of normalizing tail events is present in Appendix A.
It is worth noting that our normalization method resembles a prior work (Fang et al., 2021b,a). Their purpose is to align ATOMIC with other CSKGs, while we focus on event alignment in ATOMIC by eliminating differences among different events.

heading: 2.2 Training a Relation Prediction Model

heading: 2.2.1 Limitation of Traditional Methods
text: Traditional methods for the completion of ATOMIC proposed to score all candidate tail events given the head event and the relation. The GCN for encoding graph embeddings of events induced two shortcomings: 1) it is difficult for a GCN to propagate information due to the sparse graph structure of ATOMIC (Malaviya et al., 2020); 2) it cannot sufficiently utilize semantic information of events.

heading: 2.2.2 Our Rel-CSKGC Method
text: To address these issues, we propose Rel-CSKGC, as illustrated in Figure 3. Specifically, ATOMIC is first decomposed into independent triplets, and then Rel-CSKGC predicts the relation given the head event and the tail event of a triplet. Rel-CSKGC utilizes no graph structure information thus avoiding the problem caused by the sparsity. Additionally, encoding both the head event and the tail event with the pretrained language model successfully takes advantage of semantic information.
Problem Formulation Given a CSKG G = (N,V ), where N is the set of nodes and V is the set of edges, we consider a single training instance as a triplet vi = (h, r, t) with the head event h, relation type r and the tail event t. Here, r ∈ V and h, t ∈ N . The objective of Rel-CSKGC is to predict the most reasonable r given h and t. 1
1To keep ATOMIC concise, we only predict the most reasonable relation in this work.
Main Structure We utilize RoBERTa (Liu et al., 2019) to acquire contextual representations of freeform texts describing events. The input is the concatenation of h and t. We acquire the embedding matrix of h and t by:
[H;T ] = RoBERTa([h; t]) (1)
where H ∈ R|N |×D and T ∈ R|N |×D. |N | is the number of tokens of the event, and D is the dimensionality of representation. We apply max pooling on H and T to acquire sentence embeddings eh and et. The objective function can be defined with trainable weights Wt ∈ R1×D and Wc ∈ RK×2D:
o = sigmoid(Wte
)+softmax(Wc(eh, et)) (2)
where K is the number of relations and e
 the embedding of 
-token used as a indicator for whether h and t are related.
Negative Sampling Rel-CSKGC requires negative samples to predict unlinkable links. We consider the following two strategies to construct negative samples: 1) Random negative sampling. For a gold triplet, we randomly select an event from normalized ATOMIC as the new tail event to replace the original tail event; 2) Persona negative sampling. Triplets under relations of “xPersona” and “oPersona” follow the pattern of “Subject + is + Adjective” and account for a large part in ATOMIC. Models tend to always predict “xPersona” or “oPersona” when the given tail event follows the pattern of “Subject + is + Adjective”. To alleviate this problem, we specifically construct negative samples by replacing the tail event of triplets under relations of “xPersona” and “oPersona” with a randomly-chosen event containing “is”.

heading: 2.3 Constructing Dense-ATOMIC
text: Based on Rel-CSKGC, we train a relation prediction model with existing triplets in ATOMIC and then use the model to complete missing links in ATOMIC. We adopt threshold-based link prediction to decide whether two events are related and propose an intra-and-inter cluster completion strategy to reduce the cost of completing entire ATOMIC.
Threshold-based Link Prediction Thresholdbased link prediction (TLP) is a heuristic strategy to decide whether a relation is acceptable according to the probability predicted by Rel-CSKGC. Different thresholds are specifically tuned for different relations. The model predicts the relation
only if the final probability is above the corresponding threshold. TLP is used in all our models as the last step for the link acceptance decision.
Intra-and-inter Cluster Completion Strategy Since it’s computationally expensive to iterate over all pairs of head and tail events during the inference, we design an intra-and-inter cluster completion strategy to trade off between the completion scale and the time complexity. In Figure 1, we consider each base event and its annotated tail events as a cluster. Intra-cluster completion infers missing links inside a cluster. Intuitively, annotated tail events in one cluster, written based on the same base event, are highly related and may contain more missing links. Inter-cluster completion infers missing links between different clusters. Annotated tail events in different clusters are written independently based on different base events, thus links between different clusters are under-explored.
Due to the limited computing resource and time, we temporarily provide the results of 100 sampled clusters in this paper. Increasing the sampling size can further improve the scale of Dense-ATOMIC, but that will also linearly increases the computational cost. We will release versions with larger sampling sizes later.

heading: 3 Evaluation of Our Rel-CSKGC Method
text: In this section, we compare Rel-CSKGC with relation prediction and translation based methods by experimenting on a newly annotated subgraph and human evaluation.

heading: 3.1 Training and Test Set Construction
text: Training Set with Negative Sampling Following Sap et al. (2019)’s split of ATOMIC, we randomly sample negative triplets from the training split with negative sampling strategies introduced in Section 2.2. We combine sampled negative triplets and the training split to construct the training set for Rel-CSKGC. The statistic of the training set is illustrated in Table 1. 2
2The imbalance between random and persona negative sampling methods was established based on a preliminary experiment, which provided insights into optimal sampling sizes.
Test Set with Annotated Subgraph To test the performance of Rel-CSKGC, we construct a ground-truth subgraph by randomly sampling three clusters from the test split and annotating all pairs of head events and tail events with the most reasonable relation. The statistic of the annotated ground-truth subgraph is shown in Table 2.

heading: 3.2 Compared Methods
text: We select 4 baselines comprising two different types of CSKG completion methods and use the specific evaluation protocol for each of them.

heading: 3.2.1 Relation Prediction Methods
text: Baselines We adapt CE-random (Li et al., 2016), a method augmenting CSKGs by scoring novel tuples, to predict the missing relation. We also compare KG-BERT (Yao et al., 2019), which probes the performance of relation prediction methods on knowledge graphs. Note that we replace BERT (Devlin et al., 2019) with RoBERTa (Liu et al., 2019) in KG-BERT for fair comparison.
Evaluation Protocal Ranking metrics (HITS and Mean Reciprocal Rank) designed for translation based methods are not applicable to relation prediction methods. By valuing precision more than recall on CSKG completion, we utilize precision for the evaluation of relation prediction methods.

heading: 3.2.2 Translation Based Methods
text: Baselines SynLink (Malaviya et al., 2020) proposed to densify the CSKG with synthetic links for better graph representation. InductiveE (Wang et al., 2021) introduced indutive learning on the CSKG by enhancing the unseen event representations with neighboring structure information.
Evaluation Protocal To handle the evaluation mismatch between Rel-CSKGC and translation
based methods, we designed a transformation strategy. Specifically, we randomly sample 500 triplets from Malaviya et al. (2020)’s test split. For SynLink and InductivE, a threshold is set for hit@1 score, and a tail event is accepted only when the score is above the threshold. We tune the threshold to ensure the number of triplets inferred by RelCSKGC, SynLink, and InductivE close on these 500 triplets. We then calculate the proportion of meaningful triplets for different methods manually.3

heading: 3.3 Main Results
text: Relation Prediction Methods In Table 3, we compare Rel-CSKGC with different relation prediction methods, and Rel-CSKGC achieves consistent improvement on the test set of the annotated subgraph. Paired t-Test result proves that the improvement of Rel-CSKGC is significant. From Table 3, we can observe that the precision of intracluster completion is significantly higher than that of inter-cluster completion for all methods. This demonstrates that tail events annotated based on the same base event are highly related to each other and easier for models to predict relations, while the prediction for inter-cluster events is more challenging.
Translation Based Methods After carefully tuning the threshold based on the strategy in Section 3.2.2, Rel-CSKGC, SynLink, and InductivE
3In the given context, “meaningful triplets” refer to triplets that are considered reasonable, coherent, and noncontradictory by human evaluators.
predict 174, 133, and 132 triplets on 500 randomly sampled triplets. In Table 4, Rel-CSKGC outperforms SynLink and InductivE by a large margin on proportion and the number of meaningful triplets.

heading: 3.4 Human Evaluation
text: Motivation Upon observing predictions of RelCSKGC, we note that some triplets could be reasonable, while the annotated subgraph doesn’t cover them. For example, given a head event “X accepts Y’s apology” and a tail event “X is generous”, the annotated ground-truth relation is “xPersona”, while Rel-CSKGC could predict another reasonable relation “xIntent”. Consequently, we perform the human evaluation to check whether a predicted triplet is actually meaningful.
Result We can find from the last row of Table 3 that Rel-CSKGC achieves an even higher precision of 0.80, suggesting that Rel-CSKGC can predict reasonable triplets neglected during the subgraph annotation. The high precision by human evaluation also guarantees the quality of predicted triplets.

heading: 3.5 Ablation Study
text: To validate the effectiveness of negative sampling, we report experimental results without negative sampling in Table 3. The performance of RelCSKGC drops dramatically without any negative sampling strategies, validating the effectiveness of negative sampling.
By experimenting Rel-CSKGC with different scales of random negative samples in Figure 4, we find that the precision of Rel-CSKGC increases using both automatic and human evaluation as more negative samples are used for training.

heading: 4 Evaluation of the Constructed Dense-ATOMIC

heading: 4.1 Knowledge Coverage and Quality
text: In this subsection, we aim to answer the following question: Does Dense-ATOMIC yield higher knowledge coverage while ensuring the quality?
To this end, we statistically and manually compare Dense-ATOMIC with ATOMIC from the following three perspectives.
Dense-ATOMIC yields higher knowledge coverage In Table 5, we present the comparison between ATOMIC and Dense-ATOMIC. DenseATOMIC contains 3x more one-hop paths than ATOMIC, contributing a significantly higher knowledge coverage. It’s worth noting that different tail events in ATOMIC could become the same after normalizing tail events, so Dense-ATOMIC contains slightly fewer events than ATOMIC.
Triplets in Dense-ATOMIC have relatively high precision In Table 3, Rel-CSKGC achieves a precision of 0.80 by human evaluation. Moreover, from comparison results with translation based methods in Table 4, Rel-CSKGC outperforms two state-of-the-art methods by more than 7 percentage points. The high performance of Rel-CSKGC ensures the quality of predicted triplets to a certain extent.
Dense-ATOMIC benefits the performance of COMET To empirically demonstrate the knowledge coverage and quality of Dense-ATOMIC, we evaluate Dense-ATOMIC with COMET (Bosselut et al., 2019). The relation distribution of DenseATOMIC is long-tailed. We randomly sample 262,678 triplets from predicted triplets and recover the grouped relations to their original relations by following the relation distribution of the Sap et al. (2019)’s training split. Apart from the evaluation of perplexity, we design a strategy to evaluate the diversity score of generated tail events. For each relation, we randomly sample 10 head events from the test set. For each test sample consisting of a head event and a relation, 10 candidates are generated using beam search. For each candidate, we
manually give a score of 0, 1, or 2, representing “unreasonable”, “plausible”, and “reasonable”, respectively. We then merge candidates of similar semantics into a group and calculate the group average score. The diversity score of 10 candidates is the sum of the group scores. Intuitively, the lower perplexity and the higher diversity score indicate the higher knowledge quality and the higher knowledge coverage of Dense-ATOMIC, and COMETours outperforms COMET on both metrics in Table 6. In Table 7, we can find that tail events generated by COMETours are more semantically different.

heading: 4.2 Multi-hop Paths in Dense-ATOMIC
text: The aim of this subsection is to answer the question: Can multi-hop paths in Dense-ATOMIC better present the commonsense knowledge?
Accordingly, we evaluate multi-hop paths based on the human evaluation and performing a newly designed Commonsense Reasoning experiment, respectively:
Human evaluation confirms the correctness of multi-hop paths in Dense-ATOMIC In Table 5, we have already shown that Dense-ATOMIC contains orders of magnitude more two-hop and threehop paths than ATOMIC. Now, to further validate the correctness of multi-hop paths, we perform the human evaluation on sampled paths to calculate the proportion of reasonable paths. Note that it’s a common phenomenon (both KGs and CSKGs) that A → B and B → C are reasonable, while A → B → C is irrational. For example, {Beethoven, owner, piano} and {piano, color, black} are two reasonable triplets, but “Beethoven” and “black” are not related. Consequently, we additionally design a simple heuristic sampling rule: a multi-hop path A → . . .→ C is chosen only when A and C are also linked in Dense-ATOMIC. By comparing with random sampling in Table 8, we can find that heuristic rule sampling consistently outperforms random sampling: the longer the multi-hop paths, the more significant the improvement. Multi-hop paths randomly sampled from Dense-ATOMIC with two different methods are illustrated in Table 9.
Dense-ATOMIC has the potential of providing contextual information for Commonsense Reasoning In order to further validate the effectiveness of multi-hop paths in Dense-ATOMIC, we utilize BART (Lewis et al., 2020) to perform generative Commonsense Reasoning with or without multi-hop paths. Specifically, with the heuristic rule above, we randomly sample 5000 four-hop paths from Dense-ATOMIC as the training samples. For test samples, we manually select 500 reasonable paths from Dense-ATOMIC. BART is trained to generate the subsequent event in two different settings: 1) given only the first node of the path; 2) given the first four nodes of the path.
From Table 10, we can find that BART trained with multi-hop paths achieves better performance in that multi-hop paths could provide more contextual information useful for Commonsense Reasoning.

heading: 5 Related Work
text: ConceptNet (Speer et al., 2017) is a largescale CSKG merging various knowledge bases. ASER (Zhang et al., 2020b) contains the selectional preference knowledge extracted from more than 11 billion-token unstructured textual data. TransOMCS (Zhang et al., 2020a) utilizes linguistic graphs to convert ASER into the same representation as ConceptNet. DISCOS (Fang et al., 2021b) aggregates the neighboring information to distill the commonsense knowledge in ASER.
Recent years have seen crowdsourced CSKGs aiming to provide high-quality commonsense knowledge triplets. Sap et al. (2019) released ATOMIC consisting of if-then knowledge triplets mainly about daily events. Hwang et al. (2021) augmented ATOMIC with event-centered and physicalentity triplets. GLUCOSE (Mostafazadeh et al., 2020) grounds the implicit commonsense knowledge about everyday situations in a narrative context for richer inferential content.
Dense-ATOMIC unleashes the power of ATOMIC for high knowledge coverage and multi-hop paths.
Prior CSKG completion methods performed binary classification by scoring BiLSTM-encoded tuples (Li et al., 2016; Saito et al., 2018; Jastrzębski
et al., 2018). Following translation based methods for the knowledge graph completion (Dettmers et al., 2018; Shang et al., 2019; Meilicke et al., 2019; Qu et al., 2021; Zhang et al., 2021; Lovelace et al., 2021), Malaviya et al. (2020) additionally densified the CSKG based on BERT similarity and achieve promising results. Wang et al. (2021) and Ju et al. (2022) designed heuristic rules to add more edges for nodes with fewer neighbors. Moghimifar et al. (2021) presented a neural-symbolic reasoner to learn logic rules during the training, making the CSKG completion process interpretable.
Rel-CSKGC differs from them in that we utilize pretrained language models to predict the relation given the head event and the tail event. Similar relation prediction methods targeting at the knowledge graph completion have been proposed (Socher et al., 2013; Yao et al., 2019; Cao et al., 2020). To our best knowledge, we are the first to explore the relation prediction method on CSKG completion.

heading: 6 Conclusion
text: In this paper, we construct Dense-ATOMIC for high knowledge coverage and massive multi-hop paths and accordingly propose a CSKG completion method called Rel-CSKGC to train a relation prediction model and infer the missing links in ATOMIC. Both automatic and human evaluation show the advantage of Rel-CSKGC over strong baselines. The statistics prove that Dense-ATOMIC has significantly more triplets and multi-hop paths, providing potential for high-quality downstream applications and multi-hop reasoning based on commonsense knowledge.
Limitations
Our approach for constructing Dense-ATOMIC still has two limitations: 1) to keep Dense-ATOMIC simple, we only consider the most reasonable relation in this paper, while the relation between two events can be complex and diversified. We will release versions of Dense-ATOMIC with diversified relations later; 2) due to page limitation, we only evaluate Dense-ATOMIC on simple commonsense reasoning tasks, and we will further validate the multi-hop reasoning capacity of Dense-ATOMIC on more complex downstream tasks in the future.
Ethics Statement
We would like to thank the Allen Institute for AI for their valuable work on ATOMIC. The ATOMIC
is licensed under a license of CC BY, which allows remixing, transforming, and building upon the material for any purpose. We will also make our Dense-ATOMIC publicly available later. Mehrabi et al. (2021) have found representational harms in common sense resources. We acknowledge that the generated commonsense from our models might contain biases. All of the datasets and models are in English, which benefits English speakers more. We have employed 3 postgraduates experienced in natural language processing for annotation and human evaluation. We pay postgraduates around $8 per hour, well above the local average wage, and engage in constructive discussions if they have concerns about the process.

heading: Acknowledgments
text: This work was supported by the Natural Science Foundation of China (No. 62076133), and the Natural Science Foundation of Jiangsu Province for Distinguished Young Scholars (No. BK20200018).

heading: A Algorithm for Normalizing Tail Events
text: Algorithm 1 presents the pseudo-code of Normalizing Tail Events in Section 2.1.
Algorithm 1 Normalizing Tail Events Input: A set of annotations A and relations R Output: A set of sentences in present tense FA
1: Remove annotations with underscores or none, and get a series of filtered annotations FA 2: for each fa ∈ FA, r ∈ R do 3: Obtain the dependency tree dep and POS tagging result pos of fa 4: Find sub node with POS prp and edge subj connected directly to it 5: if Position of sub is at the start of fa then 6: Remove sub in fa 7: end if 8: Find node verb with POS vb in fa 9: if r ∈ [xIntent, xWant, xNeed, oWant]
AND the first word of fa is to then 10: Remove the first to of fa 11: end if 12: Transform node verb in fa to its root form 13: Append suf ∈ [−s,−es,−ies, ...] to verb based on English grammar 14: if r ∈ [xAttr, xReact] then 15: Insert PersonX is to the start of fa 16: else if r is oReact then 17: Insert PersonY is to the start of fa 18: else if r ∈ [oWant, oEffect] then 19: Insert PersonY to the start of fa 20: else 21: Insert PersonX to the start of fa 22: end if 23: end for 24: Return FA
B Implementation Details
Rel-CSKGC We use RoBERTa-large containing 335M parameters as the base model. We use a maximum sequence length of 100 and batch size of 128. The Adam optimizer is used for optimization with a learning rate of 2e-5 for RoBERTa-large and a learning rate of 1e-4 for MLP layers. The warmup proportion is set to 0.1. We train Rel-CSKGC with 1 NVIDIA RTX 3090 Graphical Card for 5 epochs, and it takes 20 hours to finish the training.
COMETours To train COMETours, we use the implentations provided here. 4 We use the learning rate of 1.625e-5 and the default values for other parameters.
Generative Commonsense Reasoning BARTbase is employed as the base model, which contains 140M parameters. We use a batch size of 128 and use the default values for other parameters.
4https://github.com/atcbosselut/ comet-commonsense
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: In Limitations section.
A2. Did you discuss any potential risks of your work? Not applicable. Left blank.

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: In Abstract section and section 1, respectively.
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts?

heading: In section 2, Appendix B.

heading: 3 B1. Did you cite the creators of artifacts you used?
text: In section 2.1, Appendix B.

heading: 3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts?
text: In Ethics Statement section.

heading: 3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided
text: that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? In Ethics Statement section.
7 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? We use publically available datasets, and the authors of the dataset have made the corresponding declaration.
7 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? The documentation of the artifacts will be released after the reviewing process.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. In Section 3 and 4.
C 3 Did you run computational experiments?

heading: In Section 3 and 4.

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? In Appendix B.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? In Appendix B.

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? In Section 3.

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? In Appendix B.
D 3 Did you use human annotators (e.g., crowdworkers) or research with human participants?

heading: In Section 3 and 4.
text: 7 D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? We perform simple human annotation and evaluation, there is no need of providing the full text.

heading: 3 D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students)
text: and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? In Ethics Statement.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Not applicable. Left blank.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Not applicable. Left blank.

heading: 3 D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data?
text: In Ethics Statement.

heading: Future Work
text: We will release versions of Dense-ATOMIC with diversified relations later; 2) due to page limitation, we only evaluate Dense-ATOMIC on simple commonsense reasoning tasks, and we will further validate the multi-hop reasoning capacity of Dense-ATOMIC on more complex downstream tasks in the future