SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Since our works use many models that can only be accessed through API, we provide details of these models and access time in Table A7. We obtained early access to Anthropic's claude-v1, a chat model. Anthropic has since discontinued the API access, and we cannot use the same API anymore.
A.17 FAQs
Are there domain experts in moral psychology/philosophy involved in the design process of this paper? All the factors mentioned in Table 2 are taken directly from the papers that conducted the original experiments. A graduate student and a moral/causal psychology professor were involved in carefully discussing and designing these factors. A thorough literature review was conducted to make sure these factors are comprehensive enough.
Model	Access Date	Details
GPT3-babbage-v1	2023-04-03	GPT3-babbage-001 model that involves supervised fine-tuning on human-written demonstrations.
GPT3-curie-v1	2023-04-03	GPT3-curie-001 model that involves supervised fine-tuning on human-written demonstrations.
GPT3.5-davinci-v2	2023-04-03	GPT3.5-davinci-002 model that involves supervised fine-tuning on human-written demonstrations. Derived from code-davinci-002.
GPT3.5-davinci-v3	2023-04-03	GPT3.5-davinci-003 model that involves reinforcement learning (PPO) with reward models. Derived from GPT3.5-davinci-002.
Anthropic-claude-v1	2023-04-03	A model trained using reinforcement learning from human feedback.
GPT3.5-turbo	2023-10-20	Sibling model of GPT3.5-davinci-003 is optimized for chat but works well for traditional completions tasks as well. Snapshot from 2023-06-13.
GPT4	2023-10-20	GPT-4 is a large multimodal model (currently only accepting text inputs and emitting text outputs) that is optimized for chat but works well for traditional completions tasks. Snapshot of gpt-4 from 2023-06-13.
Table A7: List of models that are accessed through API in this work, including 6 OpenAI models and 1 Anthropic model.
A.18 Data Sheet
A.18.1 Motivation
- For what purpose was the dataset created? The dataset was created to evaluate language models on causal selection problems and moral trolley problems.
- Who created the dataset (e.g., which team, research group) and on behalf of which entity (e.g., company, institution, organization)? The dataset is created by a multidisciplinary team at (Anonymous) University with researchers in the background of Computer Science, Statistics, Natural Language Processing, and Psychology.
- Who funded the creation of the dataset? This project did not receive external funding.
- Any other comments? N/A
A.18.2 Composition
- What do the instances that comprise the dataset represent (e.g., documents, photos, people, countries)? The instances are all text descriptions of scenarios, not based on real events or people.
- Does the dataset contain all possible instances, or is it a sample (not necessarily random) of instances from a larger set? Yes, there is some filtering – for example, we did not transcribe ALL story snippets that the causal papers we cited. This is for the purpose of label balancing. We try to select pairs of stories based on whether there is a significant change in human judgments that were verified by the paper's experiments. This helps make our dataset more balanced. For the moral dilemma dataset, we transcribed all the texts from each paper, resulting in a large label imbalance.
- What data does each instance consist of? Text.
- Is there a label or target associated with each instance? Yes, for each instance, we have a binarized Yes/No label.
- Is any information missing from individual instances? No
- Are relationships between individual instances made explicit (e.g., users' movie ratings, social network links)? Multiple examples can come from the same paper, which is used to measure the same or similar factors (scientific hypotheses).
- Are there recommended data splits (e.g., training, development/validation, testing)? The entire dataset should only be used for testing.
- Are there any errors, sources of noise, or redundancies in the dataset? No
- Is the dataset self-contained, or does it link to or otherwise rely on external resources (e.g., websites, tweets, other datasets)? The dataset is self-contained.
- Does the dataset contain data that might be considered confidential (e.g., data that is protected by legal privilege or by doctor-patient confidentiality, data that includes the content of individuals' non-public communications)? No
- Does the dataset contain data that, if viewed directly, might be offensive, insulting, threatening, or might otherwise cause anxiety? No
- Does the dataset relate to people? Yes, we collected human responses on each story.
- Does the dataset identify any subpopulations (e.g., by age, gender)? No
- Is it possible to identify individuals (i.e., one or more natural persons), either directly or indirectly (i.e., in combination with other data) from the dataset? No
- Does the dataset contain data that might be considered sensitive in any way (e.g., data that reveals racial or ethnic origins, sexual orientations, religious beliefs, political opinions or union memberships, or locations; financial or health data; biometric or genetic data; forms of government identification, such as social security numbers, criminal history)? No
A.18.3 Collection Process
- Has an analysis of the potential impact of the dataset and its use on data subjects (e.g., a data protection impact analysis) been conducted? All our participants' data have remained anonymous. We do not collect PPI (Private Personal Information).
- Any other comments? N/A
A.18.4 Preprocessing/cleaning/labeling
- Was any preprocessing/cleaning/labeling of the data done (e.g., discretization or bucketing, tokenization, part-of-speech tagging, SIFT feature extraction, removal of instances, processing of missing values)? We perform quality checks on our labels by analyzing how long each participant took to label the example. We rejected participant labels if they didn't spend an adequate amount of time. Additionally, for 16 stories we transcribed from "Immoral Professors and Malfunctioning Tools: Counterfactual Relevance Accounts Explain the Effect of Norm Violations on Causal Selection (Kominsky, Phillips, 2019)", we have to make them shorter because they are longer than some of the language models can handle. We removed some sentences that only add additional context to the story but do not contribute to overall causal judgment (i.e., sentences like "Alex and Benni are very reliable and Tom is satisfied with their work"). We still include the raw, unprocessed instances in our dataset. We did not modify any other instance.
- Was the "raw" data saved in addition to the preprocessed/cleaned/ labeled data (e.g., to support unanticipated future uses)? We add the original mean response score in the comment field of our data file whenever the original research paper provides the actual score. We also kept the original transcription text in the data file.
- Is the software used to preprocess/clean/label the instances available? No
- Any other comments? No
A.18.5 Use
- Has the dataset been used for any tasks already? No
- Is there a repository that links to any or all papers or systems that use the dataset? No. We hope future users of this dataset will cite this paper, and then all the follow-up papers/systems will show up in the Google Scholar search engine.
- What (other) tasks could the dataset be used for? N/A
- Is there anything about the composition of the dataset or the way it was collected and preprocessed/cleaned/labeled that might impact future uses? MoCa is not a certification task, i.e., if the language model achieves high performance on MoCa, it is human-aligned. MoCa is only an evaluation task that tests if the model's behavior is similar to humans. Our focus is narrow and only on certain aspects of alignment with humans. It cannot and should not be used to make a sweeping and general statement about AI-human alignment.
- Are there tasks for which the dataset should not be used? N/A
- Any other comments? No
A.18.6 Distribution
- Will the dataset be distributed to third parties outside of the entity (e.g., company, institution, organization) on behalf of which the dataset was created? Yes. The link can be found in the paper.
- How will the dataset will be distributed (e.g., tarball on website, API, GitHub)? Data will be stored in a GitHub repo.
- When will the dataset be distributed? The data are available right now.
- Will the dataset be distributed under a copyright or other intellectual property (IP) license, and/or under applicable terms of use (ToU)? The dataset is under a Creative Commons license (CC BY 4.0).
- Have any third parties imposed IP-based or other restrictions on the data associated with the instances? N/A
- Do any export controls or other regulatory restrictions apply to the dataset or to individual instances? N/A
- Any other comments? N/A
A.18.7 Meta-Data for Stories
We made detailed comments for each of our collected examples. We include the experiment conditions, which table/figure from the original paper we derive the true label, and if possible, we include the average Likert scale score from the human responses. We provide them as part of the appendix for ease of reading, but they are also accessible through the data files we provided in the supplementary material.
We only sub-sampled some stories to put here:
Story Counterfactual thinking and recency effects in causal judgment (Dropbox) (Henne, et al., 2021). Experiment 1, Vignette 1
Story The good, the bad, and the timely: how temporal order and moral judgment influence causal selection, (Reuter, et al, 2014), Rule violation. Scenario 1 (no rule violation) vs. Scenario 10 (Zoe violates a rule). These results are more comparative because the raw experiment has 5 options: Alice, Zoe, Both, None of the two, Not sure.
Annotations for moral stories (Trolley problems only). Moral Machine Dataset and Simplified Moral Machine Dataset are synthetic, we documented the rationale at the beginning of this appendix.
Story Throwing a Bomb on a Person Versus Throwing a Person on a Bomb Intervention Myopia in Moral Intuitions (Dropbox) (Waldmann & Dieterich, 2007), Experiment 2 (based on Experiment 1 Trolley version, but with modifications), agent intervention with harm to two people as a side effect (AI/S),  $(\mathbf{M} = 4.85$ $\mathrm{SD} = 1.01$  ). AI/M (Agent intervention, use people as means)  $(\mathbf{M} = 4.74)$
Story Inference of Intention and Permissibility in Moral Decision Making (Dropbox) (Kleiman-Weiner, et al., 2015). Trial 1 (2v1). Trolley dilemma story based on Mikhail, J. (2007). Universal moral grammar: Theory, evidence and the future. Trends in cognitive sciences, 11(4), 143–152. Trial 2 (1vB)
Story Inference of Intention and Permissibility in Moral Decision Making (Dropbox) (Kleiman-Weiner, et al., 2015). Trial 1 (2vB). Trolley dilemma story based on Mikhail, J. (2007). Universal moral grammar: Theory, evidence and the future. Trends in cognitive sciences, 11(4), 143–152. Trial 2 (Bv2)
Story Moral judgment reloaded: a moral dilemma validation study (Christensen, et al., 2014), Personal-Instrumental (1) vs. Impersonal-Accidental (2) - BURNING BUILDING (a) vs. BURNING BUILDING (b) Mean = 3.3953488372093 vs. 5.27906976744186