SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Question Types	Skill Types
Assoc.	Interv.	Counterf.	CI Method	Formalization of Causal Queries	Causal RE	Qualitative Reasoning
Datasets for Causality as Knowledge (Commonsense Causality)
COPA [2012]	X	✓	X	X	X	X	X
Event2Mind [2018]	X	✓	X	X	X	X	X
ATOMIC [2019]	X	✓	X	X	X	X	X
SocialIAQA [2019]	X	✓	X	X	X	X	X
TimeTravel [2019]	X	✓	X	X	X	X	X
Goal-Step [2020]	X	✓	X	X	X	X	X
Abductive (ART) [2020]	X	✓	X	X	X	X	X
Com2Sense [2021]	X	✓	X	X	X	X	X
CRASS [2022]	X	X	✓	X	X	X	X
Datasets for Causality as Language Comprehension (Causal Relation Extraction)
SemEval2021 Task8 [2010]	X	X	X	X	X	✓	X
EventCausality [2011]	X	X	X	X	X	✓	X
Causal-TimeBank [2014]	X	X	X	X	X	✓	X
CaTeRS [2016]	X	X	X	X	X	✓	X
BECauSE [2017]	X	X	X	X	X	✓	X
TellMeWhy [2021]	X	X	X	X	X	✓	X
Datasets for Formal Causal Reasoning
Corr2Cause [42]	X	✓	X	✓	✓	X	X
CLADDER (Ours)	✓	✓	✓	✓	✓	✓	✓
Table 9: Comparison of our dataset and existing causal or reasoning datasets. The aim of our dataset is to test the pure reasoning ability of LLMs on causal questions. For each dataset, we first identify whether its question types cover the three rungs: association (Assoc.), intervention (Interv.), and counterfactuals (Counterf.). We also check what skill types the dataset tests: the application of causal inference methods (CI Method), formalization of causal queries, causal relation extraction from the given text (Causal RE), and qualitative reasoning.
Figure 9: ROSCOE scores of answers from CAUSALCoT on 2,000 randomly sampled questions from our dataset.