DOI: 10.5061/dryad.v9s4mw75b
title: Using a "sledgehammer" approach to increase systems thinking with a brief manipulation

description: Systems thinking is a skill that is essential to understanding and taking
 effective action on complex challenges such as climate change. This
 research evaluated whether systems thinking could be increased with a
 brief intervention. Participants (N = 678) recruited from Amazon
 Mechanical Turk all completed the Systems Thinking Scale (Randel &
 Stroink, 2018), which was used as a covariate. Participants were then
 randomly assigned to one of four conditions. Some participants (n = 165)
 watched an entertaining 5-minute video describing systems thinking with a
 real-life example (Cats in
 Borneo, https://www.youtube.com/watch?v=17BP9n6g1F0). Others (n =
 174) watched this video, read a definition of systems thinking, and were
 asked to engage in systems thinking while completing a survey. This was
 designed to be a "sledgehammer" condition, in which we made our
 manipulation as heavy-handed as possible. A third (control) condition (n =
 167) watched a video about how to fold a fitted sheet. A final control
 condition (n = 172) watched no video. All participants completed a survey
 that included nine different measures that capture different aspects of
 systems thinking. Despite large sample sizes and multiple
 operationalizations of systems thinking, support for the efficacy of our
 brief intervention was weak at best. Those who watched the systems
 thinking video scored significantly higher on one self-report measure of
 the extent to which people perceived themselves to be part of local
 social, economic, and ecological systems. Watching the systems thinking
 video also marginally increased the accuracy with which people correctly
 identified positive and negative feedback loops, and the extent to which
 participants saw a shot in a pool game impacting the outcome of the game.
 However, the control condition performed significantly better than the two
 systems thinking video conditions on a stock/flow identification task, and
 all other measures showed no differences by condition. We conclude that
 increasing systems thinking with a brief manipulation, even one that
 defines systems thinking and begs participants to engage in systems
 thinking, is not very effective.
description: Participants were recruited via Amazon Mechanical Turk. All
 participants were adults living in the United States. 
 We gave 10 different measures that capture some aspect of systems
 thinking: <strong>The Systems Thinking
 Scale</strong> (Randle & Stroink, 2018):
 This 15-item self-report scale measures someone's
 dispositional tendency to engage in systems thinking. Negatively worded
 items were recoded, and all items were averaged together. Higher scores =
 more systems thinking. This trait measure was given at the start of the
 study and was used as a covariate. <strong>The
 Murder Scenario</strong> (Choi et al., 2007):
 Participants read a brief description of a murder case and
 indicated which of 96 possible facts were irrelevant to the case. We
 recoded the items such that 1 = relevant, 0 = irrelevant. The recoded
 items were summed together. Choosing more items indicates more holistic
 thinking about causality. <strong>Ripple Effect
 Question: Driver Scenario:</strong> Based on
 measures developed by Maddux & Yuki (2006). Participants read
 about a minor traffic accident. They estimated the level of responsibility
 that different actors in the scenario have for various outcomes, and
 estimated the impact that the accident had. The responsibility and impact
 items were each averaged together. Seeing wider rings of responsibility
 and impact indicates more holistic thinking about causality
 <strong>Ripple Effect Question: Pool</strong>
 <strong>Game</strong>. (Maddux & Yuki,
 2006): Participants saw a picture of someone taking a
 shot in a pool game. They indicated the extent to which this shot would
 affect the next several shots and the outcome of the game. Seeing more
 impact farther out in time indicates systems thinking.
 <strong>Ripple Effect Question:
 Executive</strong> Based on measures developed by
 Maddux & Yuki (2006). Participants read a scenario about a
 corporate executive who must make difficult personnel decisions to address
 financial difficulties. They estimated the number of people affected by
 the decision. They also rated the level of responsibility the executive
 had for various outcomes. Seeing wider rings of responsibility and impact
 indicates more holistic thinking about causality.
 <strong>Pharmacy Scenario:</strong>
 Based on a measure developed by Chiu et al. (2000), where
 participants read about a situation in which a pharmacy technician gave
 children in a hospital the wrong medication, causing many to become ill.
 Participants indicated the extent to which characteristics of the
 technician (dispositional attributions) or the situation (situational
 attributions) were important causes of the event. More situational
 attributions indicate more holistic thinking about causality.
 <strong>Plot identification. </strong>(Rottman et
 al., 2012): A verbal description of a causal system was
 shown above four plots. One plot accurately depicted the verbal
 description. Participants were asked to choose the plot that best depicted
 the causal system described. We totaled the number of correct
 answers. <strong>Picture Mapping.
 </strong>(Vendetti et al., 2014): Participants
 were presented with ten sets of two pictures: a highlighted object in the
 first picture had both ‘relational’ and ‘object’ matches in the second
 picture. Participants were asked to choose the object in the second
 picture "that goes with the highlighted item in the first
 picture." Choosing the relational match indicates more relational
 reasoning. We totaled the number of relational answers.
 <strong>Stocks and Flows. </strong>(developed for
 this study): Participants read brief definitions of
 "stocks" and "flows". They were then presented with 14
 items (e.g., water in a reservoir, deaths per year) and asked to indicate
 whether each item was a stock or a flow. We totaled the number of correct
 answers. <strong>Feedback Loops.
 </strong>(developed for this study): Participants
 read a definition of feedback as well as an example of positive feedback
 and negative feedback. They were then presented with eight phenomena
 (e.g., When a herd animal is alarmed and startles, this causes others to
 startle). Participants were asked to indicate whether each was an example
 of positive feedback, negative feedback, or neither. We totaled the number
 of correct answers. Participants completed the Systems
 Thinking Scale, then were randomly assigned to one of four conditions.
 Some participants (n = 165) watched an entertaining 5-minute video
 describing systems thinking with a real-life example (Cats in Borneo,
 https://www.youtube.com/watch?v=17BP9n6g1F0). Others (n = 174) watched this video, read a definition of systems thinking, and were asked to engage in systems thinking while completing a survey. This was designed to be a "sledgehammer" condition, in which we made our manipulation as heavy-handed as possible. A third (control) condition (n = 167) watched a video about how to fold a fitted sheet. A final control condition (n = 172) watched no video. All participants then completed the other nine measures listed above. <strong>References</strong> Chiu CY, Morris MW, Hong YY, Menon T. 2000. Motivated cultural cognition: the impact of implicit cultural theories on dispositional attribution varies as a function of need for closure. Journal of Personality and Social Psychology 78: 247–259. Choi I, Koo M, Choi JA. 2007. Individual differences in analytic versus holistic thinking. Personality and Social Psychology Bulletin 33: 691–705. Maddux WW, Yuki M. 2006. The “ripple effect”: cultural differences in perceptions of the consequences of events. Personality and Social Psychology Bulletin 32: 669–683. Randle, J. M., & Stroink, M. L. (2018). The development and initial validation of the paradigm of systems thinking. <em>Systems Research and Behavioral Science</em>, <em>35</em>(6), 645-657. Rottman BM, Gentner D, Goldwater MB. 2012. Causal systems categories: differences in novice and expert categorization of causal phenomena. Cognitive science 36: 919–932. Vendetti MS, Wu A, Holyoak KJ. 2014. Far-out thinking generating solutions to distant analogies promotes relational thinking. Psychological Science 25: 928–933.
description: # Using a "sledgehammer" approach to increase systems thinking
 with a brief manipulation
 [https://doi.org/10.5061/dryad.v9s4mw75b](https://doi.org/10.5061/dryad.v9s4mw75b) ## Description of the data and file structure The data appear in the file DataForIncreasing STWithBriefManipulation.csv. Variable descriptions and values appear in the file MetaDataForIncreasing STWithBriefManipulation.csv.  ### Files and variables #### File: DataForIncreasingSTWithBriefManipulation.csv **Description:** Raw data that has not been published. The data file was generated in SPSS but exported to csv format for accessibility. Each row corresponds to a single participant. Missing data occurred when online participants failed to complete a question. Missing data is indicated with an empty field. ##### Variables | Variable | Position | Label | | :-------------------- | :------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Condition | 1 | Condition | | Systems\_Thinking | 2 | Systems Thinking Scale | | AVGRipplePool | 3 | Ripple Effect Pool | | AVGRippleExecResp | 4 | Ripple Effect Exec Responsibility | | AVGRippleDriverResp | 5 | Ripple Effect Driver Responsibility | | AVGRippleDriverAff | 6 | Ripple Effect Driver Impact | | AVGRipplePharmWorker | 7 | Pharmacy Worker Average (Disp) | | AVGRipplePharmClinic | 8 | Pharmacy Clinic Average (Sit) | | TOTALmurder | 9 | Murder Scenario Sum | | AVGPMaps | 10 | Picture Mapping Average Correct | | AVGCNS | 11 | ConnectednessToNatureScale | | AVGPlots | 12 | Plots Average Correct | | AVGStkFlw | 13 | Stock/Flow Average Correct | | AVGFdbck | 14 | Feedback Average Correct | | AVGLocalST | 15 | Local Systems Thinking Average | | AVGLocalSTSocial | 16 | Average of Local ST 1 and 4 | | AVGLocalSTEcon | 17 | Average of Local ST 2 and 5 | | AVGLocalSTEnv | 18 | Average of Local ST 3 and 6 | | SUMplots | 19 | Sum of Plots 1, 2, 3 | | SUMPMaps | 20 | Sum of PMaps 1 - 10 | | SUMStkFlw | 21 | Sum of StkFlw 1 - 14 | | SUMFdbck | 22 | Sum of Fdbck 1 - 8 | | STS\_1 | 23 | The Earth, including all its inhabitants, is a living system. | | STS\_2 | 24 | All the Earth's systems, from the climate to the economy, are interconnected. | | STS\_3 | 25 | Seemingly small choices we make today can ultimately have major consequences. | | STS\_4 | 26 | Individual people are not as separate from one another as they seem. | | STS\_5 | 27 | Environmental problems, social problems, and economic problems are all separate issues. | | STS\_6 | 28 | When I have to make a decision in my life, I tend to see all kinds of possible consequences to each choice. | | STS\_7 | 29 | I learn best when I can see how the different pieces of a subject relate to one another. | | STS\_8 | 30 | I like to know how events or information fit into the big picture. | | STS\_9 | 31 | Ultimately, we can break all problems down into what is simply right or wrong. | | STS\_10 | 32 | Rules and laws should not change a lot over time. | | STS\_11 | 33 | Everything is constantly changing. | | STS\_12 | 34 | Only very large events can significantly change big systems like economies or ecosystems. | | STS\_13 | 35 | Adding just one more small farm upstream from a lake can permanently alter that lake. | | STS\_14 | 36 | My health has nothing to do with what is happening in the world. | | STS\_15 | 37 | It is possible for a community to organize into a new form that was not planned or designed by an authority or government. | | Q108\_Page\_Submit | 38 | Timing - Page Submit | | Murder\_1 | 39 | Murder\_1 | | Murder\_2 | 40 | Murder\_2 | | Murder\_3 | 41 | Murder\_3 | | Murder\_4 | 42 | Murder\_4 | | Murder\_5 | 43 | Murder\_5 | | Murder\_6 | 44 | Murder\_6 | | Murder\_7 | 45 | Murder\_7 | | Murder\_8 | 46 | Murder\_8 | | Murder\_9 | 47 | Murder\_9 | | Murder\_10 | 48 | Murder\_10 | | Murder\_11 | 49 | Murder\_11 | | Murder\_12 | 50 | Murder\_12 | | Murder\_13 | 51 | Murder\_13 | | Murder\_14 | 52 | Murder\_14 | | Murder\_15 | 53 | Murder\_15 | | Murder\_16 | 54 | Murder\_16 | | Murder\_17 | 55 | Murder\_17 | | Murder\_18 | 56 | Murder\_18 | | Murder\_19 | 57 | Murder\_19 | | Murder\_20 | 58 | Murder\_20 | | Murder\_21 | 59 | Murder\_21 | | Murder\_22 | 60 | Murder\_22 | | Murder\_23 | 61 | Murder\_23 | | Murder\_24 | 62 | Murder\_24 | | Murder\_25 | 63 | Murder\_25 | | Murder\_26 | 64 | Murder\_26 | | Murder\_27 | 65 | Murder\_27 | | Murder\_28 | 66 | Murder\_28 | | Murder\_29 | 67 | Murder\_29 | | Murder\_30 | 68 | Murder\_30 | | Murder\_31 | 69 | Murder\_31 | | Murder\_32 | 70 | Murder\_32 | | Murder\_33 | 71 | Murder\_33 | | Murder\_34 | 72 | Murder\_34 | | Murder\_35 | 73 | Murder\_35 | | Murder\_36 | 74 | Murder\_36 | | Murder\_37 | 75 | Murder\_37 | | Murder\_38 | 76 | Murder\_38 | | Murder\_39 | 77 | Murder\_39 | | Murder\_40 | 78 | Murder\_40 | | Murder\_41 | 79 | Murder\_41 | | Murder\_42 | 80 | Murder\_42 | | Murder\_43 | 81 | Murder\_43 | | Murder\_44 | 82 | Murder\_44 | | Murder\_45 | 83 | Murder\_45 | | Murder\_46 | 84 | Murder\_46 | | Murder\_47 | 85 | Murder\_47 | | Murder\_48 | 86 | Murder\_48 | | Murder\_49 | 87 | Murder\_49 | | Murder\_50 | 88 | Murder\_50 | | Murder\_51 | 89 | Murder\_51 | | Murder\_52 | 90 | Murder\_52 | | Murder\_53 | 91 | Murder\_53 | | Murder\_54 | 92 | Murder\_54 | | Murder\_55 | 93 | Murder\_55 | | Murder\_56 | 94 | Murder\_56 | | Murder\_57 | 95 | Murder\_57 | | Murder\_58 | 96 | Murder\_58 | | Murder\_59 | 97 | Murder\_59 | | Murder\_60 | 98 | Murder\_60 | | Murder\_61 | 99 | Murder\_61 | | Murder\_62 | 100 | Murder\_62 | | Murder\_63 | 101 | Murder\_63 | | Murder\_64 | 102 | Murder\_64 | | Murder\_65 | 103 | Murder\_65 | | Murder\_66 | 104 | Murder\_66 | | Murder\_67 | 105 | Murder\_67 | | Murder\_68 | 106 | Murder\_68 | | Murder\_69 | 107 | Murder\_69 | | Murder\_70 | 108 | Murder\_70 | | Murder\_71 | 109 | Murder\_71 | | Murder\_72 | 110 | Murder\_72 | | Murder\_73 | 111 | Murder\_73 | | Murder\_74 | 112 | Murder\_74 | | Murder\_75 | 113 | Murder\_75 | | Murder\_76 | 114 | Murder\_76 | | Murder\_77 | 115 | Murder\_77 | | Murder\_78 | 116 | Murder\_78 | | Murder\_79 | 117 | Murder\_79 | | Murder\_80 | 118 | Murder\_80 | | Murder\_81 | 119 | Murder\_81 | | Murder\_82 | 120 | Murder\_82 | | Murder\_83 | 121 | Murder\_83 | | Murder\_84 | 122 | Murder\_84 | | Murder\_85 | 123 | Murder\_85 | | Murder\_86 | 124 | Murder\_86 | | Murder\_87 | 125 | Murder\_87 | | Murder\_88 | 126 | Murder\_88 | | Murder\_89 | 127 | Murder\_89 | | Murder\_90 | 128 | Murder\_90 | | Murder\_91 | 129 | Murder\_91 | | Murder\_92 | 130 | Murder\_92 | | Murder\_93 | 131 | Murder\_93 | | Murder\_94 | 132 | Murder\_94 | | Murder\_95 | 133 | Murder\_95 | | Murder\_96 | 134 | Murder\_96 | | CNS\_1 | 135 | I often feel a strong connection to nature. | | CNS\_2 | 136 | I think of nature as a family I belong to. | | CNS\_3 | 137 | I see myself as a part of the greater circle of life. | | CNS\_4 | 138 | I feel that all living things in this world are connected, and I am a part of that. | | CNS\_5 | 139 | Like the trees in the forest, I feel I belong to nature. | | LocalST\_1 | 140 | I think of the place I live as a social system composed of interrelated parts. | | LocalST\_2 | 141 | I think of the place I live as an economic system composed of interrelated parts. | | LocalST\_3 | 142 | I think of the place I live as an environmental system composed of interrelated parts. | | LocalST\_4 | 143 | I think of myself as part of a social community. | | LocalST\_5 | 144 | I think of myself as part of a economic community. | | LocalST\_6 | 145 | Please indicate your agreement with each of the following statements, using the scale provided. There are no right or wrong answers. - I think of myself as part of an ecological community. | | RipplePool1\_1 | 146 | How much will this shot affect the person who takes the next shot? - Next shot | | RipplePool2\_1 | 147 | How much will this shot affect the person who takes the third next shot? - Third shot | | RipplePool3\_1 | 148 | How much will this shot affect the person who takes the sixth shot? - Sixth shot | | RipplePool4\_1 | 149 | How much will this shot affect the overall outcome of the game? - Overall outcome | | RippleDriverRespo\_1 | 150 | How responsible is Sam for the damage to his own car? | | RippleDriverRespo\_2 | 151 | How responsible is the driver in front of him? | | RippleDriverRespo\_3 | 152 | How responsible is the animal that ran across the road? | | RippleDriverRespo\_4 | 153 | How responsible is the manufacturer of the car's brake system? | | RippleDriverAff\_1 | 154 | How affected by the accident is Sam? | | RippleDriverAff\_2 | 155 | How affected by the accident is the other driver? | | RippleDriverAff\_3 | 156 | How affected by the accident are the other student senators? | | RippleDriverAff\_4 | 157 | Please answer the questions below, referring to the scenario above. - How affected by the accident are other students in the school? | | RippleDriverAff\_5 | 158 | Please answer the questions below, referring to the scenario above. - How affected by the accident are the other commuters on the road behind Sam? | | PharmacyImportant\_1 | 159 | The pharmacy worker was not conscientious about service. | | PharmacyImportant\_2 | 160 | The pharmacy worker was careless. | | PharmacyImportant\_3 | 161 | The pharmacy worker was neglectful. | | PharmacyImportant\_4 | 162 | The pharmacy worker was irresponsible. He did not follow the procedures of the pharmacy. | | PharmacyImportant\_5 | 163 | The pharmacy worker was not safety-minded. | | PharmacyImportant\_6 | 164 | The pharmacy worker was a poor worker. | | PharmacyImportant\_7 | 165 | The pharmacy worker was incompetent and could not handle his work. | | PharmacyImportant\_8 | 166 | The clinic did nothing to ensure the quality of the medicine. | | PharmacyImportant\_9 | 167 | The clinic did not monitor the performance of the workers. | | PharmacyImportant\_10 | 168 | The clinic did not provide enough training for the workers. | | PharmacyImportant\_11 | 169 | The clinic did not have an adequate quality assurance system for the pharmaceutical process. | | PharmacyImportant\_12 | 170 | The clinic had poor management. | | PharmacyImportant\_13 | 171 | The clinic was incompetent in maintaining proper usage of the medical products. | | PharmacyImportant\_14 | 172 | The clinic did not communicate well with its employees. | | RippleExecRespons\_1 | 173 | How responsible do you feel for cutting your own salary? | | RippleExecRespons\_2 | 174 | How responsible do you feel for the employees who received pay cuts? | | RippleExecRespons\_3 | 175 | How responsible do you feel for the employees you fired? | | RippleExecRespons\_4 | 176 | How responsible do you feel for the families of the fired employees? | | RippleExecRespons\_5 | 177 | How responsible would you feel if, a year later, there was an increase in crime in the area? | | Plot1 | 178 | When the number of drug dealers in a neighborhood gets out of control, the police will often take a stronger position in the community and crack down on drug dealing in the neighborhood. Once thF40.2 | | Plot2 | 179 | Predator/prey populations of animals sometimes follow a very predictable pattern. If the prey population is in numbers, the predator population overeats and the prey population begins to decline in number. Consequently, the predator population decreases with tF40.2 | | Plot3 | 180 | Audio feedback occurs when a microphone is placed too close to a speaker. Any noise that the microphone picks up is amplified and played through the speaker at a higher volume. The microphone then picks up the louder noise and sends it back to the speaker eF40.2 | | PMap1 | 181 | Please choose the object in the second picture that goes with the highlighted item in the first picture. | | PMap2 | 182 | Please choose the object in the second picture that goes with the highlighted object in the first picture. | | PMap3 | 183 | Please choose the object in the second picture that goes with the object in the first picture. | | PMap4 | 184 | Please choose the object in the second picture that goes with the highlighted object in the first picture. | | PMap5 | 185 | Please choose the object in the second picture that goes with the highlighted object in the first. | | PMap6 | 186 | Please choose the object in the second picture that goes with the highlighted object in the first picture. | | PMap7 | 187 | Please choose the object in the second picture that goes with the highlighted object in the first picture. | | PMap8 | 188 | Please choose the object in the second picture that goes with the highlighted object in the first picture. | | PMap9 | 189 | Please choose the object in the second picture that goes with the highlighted object in the first picture. | | PMap10 | 190 | Please choose the object in the second picture that goes with the highlighted object in the first picture. | | StkFlw\_1 | 191 | Water in a reservoir | | StkFlw\_2 | 192 | A population of rabbits | | StkFlw\_3 | 193 | Sunlight reaching the earth | | StkFlw\_4 | 194 | Terrorism | | StkFlw\_5 | 195 | Interest on your bank account | | StkFlw\_6 | 196 | Process of purchasing groceries | | StkFlw\_7 | 197 | Water coming out of a faucet | | StkFlw\_8 | 198 | Fossil fuels in the ground | | StkFlw\_9 | 199 | Electricity moving down a power line | | StkFlw\_10 | 200 | Wind | | StkFlw\_11 | 201 | Energy in a candy bar | | StkFlw\_12 | 202 | Deaths per year | | StkFlw\_13 | 203 | CO2 in teh atmosphere | | StkFlw\_14 | 204 | Emission of CO2 by power plants | | Fdbck\_1 | 205 | Increasing global temperature increases decomposition, which releases carbon dioxide into the atmosphere (the principal “greenhouse” gas) | | Fdbck\_2 | 206 | A music student performs an incorrect note, which the teacher points out to the student. The student then plays correctly | | Fdbck\_3 | 207 | A music student performs an incorrect note, which the teacher points out to the student. The student gets flustered and plays worse the second time, is corrected again, and then performs even worse! | | Fdbck\_4 | 208 | Rain falls from the sky, evaporates, and then falls from the sky again | | Fdbck\_5 | 209 | An athlete sprints, depleting oxygen and sugar in their blood. Their body responds by breathing heavily and releasing stored sugars. | | Fdbck\_6 | 210 | A person buys fruit at a farmers market – dollars are exchanged for local apples. | | Fdbck\_7 | 211 | Energy spontaneously flows from a hot burner into the pot that is resting on that burner | | Fdbck\_8 | 212 | When a herd animal is alarmed and startles, this causes others to startle | | gender | 213 | Your gender identity - Selected Choice | | | | | | | | | | STS\_5\_RC | 216 | Environmental problems, social problems, and economic problems are all connected issues. | | STS\_9\_RC | 217 | Ultimately, we cannot break all problems down into what is simply right or wrong. | | STS\_10\_RC | 218 | Rules and laws should change a lot over time. | | STS\_12\_RC | 219 | Even small events can significantly change big systems like economies or ecosystems. | | STS\_14\_RC | 220 | My health is connected with what is happening in the world. | | WhiteVNonwhite | 229 | Ethnicity binary (White vs POC) | Value Labels | Value | | Label | | :------------------- | :- | :------------------------------------------------------------------------------------------------------------------- | | Condition | 0 | No Video | | | 1 | Video | | | 2 | Fitted Sheet | | | 3 | Video and Def | | STS\_1 | 1 | Strongly Disagree | | | 2 | Moderately Disagree | | | 3 | Slightly Disagree | | | 4 | Neutral | | | 5 | Slightly Agree | | | 6 | Moderately Agree | | | 7 | Strongly Agree | | STS\_2 | 1 | Strongly Disagree | | | 2 | Moderately Disagree | | | 3 | Slightly Disagree | | | 4 | Neutral | | | 5 | Slightly Agree | | | 6 | Moderately Agree | | | 7 | Strongly Agree | | STS\_3 | 1 | Strongly Disagree | | | 2 | Moderately Disagree | | | 3 | Slightly Disagree | | | 4 | Neutral | | | 5 | Slightly Agree | | | 6 | Moderately Agree | | | 7 | Strongly Agree | | STS\_4 | 1 | Strongly Disagree | | | 2 | Moderately Disagree | | | 3 | Slightly Disagree | | | 4 | Neutral | | | 5 | Slightly Agree | | | 6 | Moderately Agree | | | 7 | Strongly Agree | | STS\_5 | 1 | Strongly Disagree | | | 2 | Moderately Disagree | | | 3 | Slightly Disagree | | | 4 | Neutral | | | 5 | Slightly Agree | | | 6 | Moderately Agree | | | 7 | Strongly Agree | | STS\_6 | 1 | Strongly Disagree | | | 2 | Moderately Disagree | | | 3 | Slightly Disagree | | | 4 | Neutral | | | 5 | Slightly Agree | | | 6 | Moderately Agree | | | 7 | Strongly Agree | | STS\_7 | 1 | Strongly Disagree | | | 2 | Moderately Disagree | | | 3 | Slightly Disagree | | | 4 | Neutral | | | 5 | Slightly Agree | | | 6 | Moderately Agree | | | 7 | Strongly Agree | | STS\_8 | 1 | Strongly Disagree | | | 2 | Moderately Disagree | | | 3 | Slightly Disagree | | | 4 | Neutral | | | 5 | Slightly Agree | | | 6 | Moderately Agree | | | 7 | Strongly Agree | | STS\_9 | 1 | Strongly Disagree | | | 2 | Moderately Disagree | | | 3 | Slightly Disagree | | | 4 | Neutral | | | 5 | Slightly Agree | | | 6 | Moderately Agree | | | 7 | Strongly Agree | | STS\_10 | 1 | Strongly Disagree | | | 2 | Moderately Disagree | | | 3 | Slightly Disagree | | | 4 | Neutral | | | 5 | Slightly Agree | | | 6 | Moderately Agree | | | 7 | Strongly Agree | | STS\_11 | 1 | Strongly Disagree | | | 2 | Moderately Disagree | | | 3 | Slightly Disagree | | | 4 | Neutral | | | 5 | Slightly Agree | | | 6 | Moderately Agree | | | 7 | Strongly Agree | | STS\_12 | 1 | Strongly Disagree | | | 2 | Moderately Disagree | | | 3 | Slightly Disagree | | | 4 | Neutral | | | 5 | Slightly Agree | | | 6 | Moderately Agree | | | 7 | Strongly Agree | | STS\_13 | 1 | Strongly Disagree | | | 2 | Moderately Disagree | | | 3 | Slightly Disagree | | | 4 | Neutral | | | 5 | Slightly Agree | | | 6 | Moderately Agree | | | 7 | Strongly Agree | | STS\_14 | 1 | Strongly Disagree | | | 2 | Moderately Disagree | | | 3 | Slightly Disagree | | | 4 | Neutral | | | 5 | Slightly Agree | | | 6 | Moderately Agree | | | 7 | Strongly Agree | | STS\_15 | 1 | Strongly Disagree | | | 2 | Moderately Disagree | | | 3 | Slightly Disagree | | | 4 | Neutral | | | 5 | Slightly Agree | | | 6 | Moderately Agree | | | 7 | Strongly Agree | | Murder\_1 | 1 | Whether or not the professor ever sexually harassed the graduate student | | Murder\_2 | 1 | Whether or not the professor drank alcohol | | Murder\_3 | 1 | Whether or not the graduate student recently ended a romantic relationship | | Murder\_4 | 1 | The professor's history of sexual abuse by his/her parents | | Murder\_5 | 1 | The graduate student's history of mental disorders | | Murder\_6 | 1 | The professor's history of eating disorders | | Murder\_7 | 1 | Whether or not the graduate student and the professor were (or previously had been) engaged in a sexual relationship | | Murder\_8 | 1 | The graduate student's sexual orientation | | Murder\_9 | 1 | Whether or not the graduate student ever sexually harassed the professor | | Murder\_10 | 1 | Whether the graduate student behaved unreasonably toward the professor | | Murder\_11 | 1 | The graduate student's height and weight | | Murder\_12 | 1 | The professor's sexual orientation | | Murder\_13 | 1 | The way the professor dressed | | Murder\_14 | 1 | Whether or not the graduate student drank alcohol | | Murder\_15 | 1 | Whether or not the graduate student smoked cigarettes | | Murder\_16 | 1 | Whether or not the professor used email regularly | | Murder\_17 | 1 | Whether or not the professor behaved unreasonably toward the graduate student | | Murder\_18 | 1 | Whether or not the graduate student was unhelpful | | Murder\_19 | 1 | The professor's height and weight | | Murder\_20 | 1 | Whether or not the professor was religious | | Murder\_21 | 1 | Whether or not the graduate student came from a dysfunctional family | | Murder\_22 | 1 | Whether or not the professor was left-handed | | Murder\_23 | 1 | Whether or not the graduate student was far away from his/her hometown | | Murder\_24 | 1 | Whether or not the professor was temporarily insane | | Murder\_25 | 1 | Whether or not the graduate student liked rock music | | Murder\_26 | 1 | Whether or not the professor came from a dysfunctional family | | Murder\_27 | 1 | Whether or not the graduate student ever ridiculed the professor in public | | Murder\_28 | 1 | Whether or not the professor recently ended a romantic relationship | | Murder\_29 | 1 | Whether or not the graduate student was an irritating person | | Murder\_30 | 1 | The professor's history of mental disorders | | Murder\_31 | 1 | Whether the professor was introverted or extroverted | | Murder\_32 | 1 | Whether or not the professor smoked cigarettes | | Murder\_33 | 1 | Whether or not the professor had a history of violence | | Murder\_34 | 1 | Whether or not the professor was condescending toward the graduate student | | Murder\_35 | 1 | The graduate student's history of eating disorders | | Murder\_36 | 1 | Whether or not the professor was an irritating person | | Murder\_37 | 1 | Whether the professor had any brothers and sisters | | Murder\_38 | 1 | Whether the graduate student liked to attend parties | | Murder\_39 | 1 | Whether or not the professor was unhelpful | | Murder\_40 | 1 | Whether or not the graduate student liked to watch violent movies | | Murder\_41 | 1 | The graduate student's history of sexual abuse by his/her parents | | Murder\_42 | 1 | The graduate student's IQ score | | Murder\_43 | 1 | Whether or not the graduate student was temporarily insane | | Murder\_44 | 1 | The way the graduate student dressed | | Murder\_45 | 1 | Whether or not the professor was a vegetarian | | Murder\_46 | 1 | The professor's IQ score | | Murder\_47 | 1 | Whether or not the professor could play a musical instrument | | Murder\_48 | 1 | What the graduate student's parents did for a living | | Murder\_49 | 1 | Whether the professor preferred to use IBM or Macintosh computers | | Murder\_50 | 1 | What the professor's high school GPA was | | Murder\_51 | 1 | Whether or not the graduate student was introverted or extroverted | | Murder\_52 | 1 | Whether or not the professor liked to watch violent movies | | Murder\_53 | 1 | The number of pets the professor owned | | Murder\_54 | 1 | The graduate student's zodiac (astrological) sign | | Murder\_55 | 1 | Whether the graduate student ever unfairly gave the professor a bad evaluation | | Murder\_56 | 1 | Whether or not the graduate student had a history of violence | | Murder\_57 | 1 | Whether or not the professor had a web page | | Murder\_58 | 1 | The professor's favorite food | | Murder\_59 | 1 | Whether or not the graduate student lived in a dorm | | Murder\_60 | 1 | The political preferences of the professor | | Murder\_61 | 1 | Whether or not the professor ever ridiculed the graduate student in public | | Murder\_62 | 1 | Whether or not the graduate student was a drug user | | Murder\_63 | 1 | Whether the graduate student had a history of rebelling against persons in authority | | Murder\_64 | 1 | Whether the graduate student was a basketball fan | | Murder\_65 | 1 | Whether or not the graduate student was religious | | Murder\_66 | 1 | Whether or not the graduate student and professor had offices on different floors | | Murder\_67 | 1 | Whether or not the graduate student used email regularly | | Murder\_68 | 1 | Whether or not the graduate student was condescending toward the professor | | Murder\_69 | 1 | Whether the professor ever unfairly gave the graduate student a bad evaluation | | Murder\_70 | 1 | The number of publications on which the professor and the graduate student had collaborated | | Murder\_71 | 1 | The graduate student's favorite color | | Murder\_72 | 1 | Whether or not the professor would have retired soon | | Murder\_73 | 1 | Whether the graduate student had any brothers or sisters | | Murder\_74 | 1 | Whether or not the professor liked rock music | | Murder\_75 | 1 | What the graduate student's high school GPA was | | Murder\_76 | 1 | Whether the professor had a history of abusing his/her authority | | Murder\_77 | 1 | Whether or not the graduate student had a web page | | Murder\_78 | 1 | Whether or not the professor had a secretary | | Murder\_79 | 1 | Whether the graduate student came to school on a bicycle | | Murder\_80 | 1 | The professor's favorite color | | Murder\_81 | 1 | What the professor was doing on the night in question | | Murder\_82 | 1 | Whether or not the graduate student was left-handed | | Murder\_83 | 1 | Whether the professor liked to attend parties | | Murder\_84 | 1 | Whether the professor was a basketball fan | | Murder\_85 | 1 | Whether the graduate student preferred to use PC or Mac computers | | Murder\_86 | 1 | What the professor's parents did for a living | | Murder\_87 | 1 | Whether or not the professor was a drug user | | Murder\_88 | 1 | The political preferences of the graduate student | | Murder\_89 | 1 | The professor's zodiac (astrological) sign | | Murder\_90 | 1 | What the graduate student was doing on the night in question | | Murder\_91 | 1 | Whether the professor was far away from his/her hometown | | Murder\_92 | 1 | Whether or not the graduate student was a vegetarian | | Murder\_93 | 1 | The number of pets the graduate student owned | | Murder\_94 | 1 | Whether the professor came to school on a bicycle | | Murder\_95 | 1 | Whether or not the graduate student would have received his/her Ph.D soon | | Murder\_96 | 1 | Whether or not the graduate student could play a musical instrument | | CNS\_1 | 1 | Strongly Disagree | | | 2 | Slightly Disagree | | | 3 | Neutral | | | 4 | Slightly Agree | | | 5 | Strongly Agree | | CNS\_2 | 1 | Strongly Disagree | | | 2 | Slightly Disagree | | | 3 | Neutral | | | 4 | Slightly Agree | | | 5 | Strongly Agree | | CNS\_3 | 1 | Strongly Disagree | | | 2 | Slightly Disagree | | | 3 | Neutral | | | 4 | Slightly Agree | | | 5 | Strongly Agree | | CNS\_4 | 1 | Strongly Disagree | | | 2 | Slightly Disagree | | | 3 | Neutral | | | 4 | Slightly Agree | | | 5 | Strongly Agree | | CNS\_5 | 1 | Strongly Disagree | | | 2 | Slightly Disagree | | | 3 | Neutral | | | 4 | Slightly Agree | | | 5 | Strongly Agree | | LocalST\_1 | 1 | Strongly Disagree | | | 2 | Slightly Disagree | | | 3 | Neutral | | | 4 | Slightly Agree | | | 5 | Strongly Agree | | LocalST\_2 | 1 | Strongly Disagree | | | 2 | Slightly Disagree | | | 3 | Neutral | | | 4 | Slightly Agree | | | 5 | Strongly Agree | | LocalST\_3 | 1 | Strongly Disagree | | | 2 | Slightly Disagree | | | 3 | Neutral | | | 4 | Slightly Agree | | | 5 | Strongly Agree | | LocalST\_4 | 1 | Strongly Disagree | | | 2 | Slightly Disagree | | | 3 | Neutral | | | 4 | Slightly Agree | | | 5 | Strongly Agree | | LocalST\_5 | 1 | Strongly Disagree | | | 2 | Slightly Disagree | | | 3 | Neutral | | | 4 | Slightly Agree | | | 5 | Strongly Agree | | LocalST\_6 | 1 | Strongly Disagree | | | 2 | Slightly Disagree | | | 3 | Neutral | | | 4 | Slightly Agree | | | 5 | Strongly Agree | | RippleExecRespons\_1 | 1 | Not at all responsible | | | 2 | Somewhat responsible | | | 3 | Very responsible | | | 4 | Completely responsible | | RippleExecRespons\_2 | 1 | Not at all responsible | | | 2 | Somewhat responsible | | | 3 | Very responsible | | | 4 | Completely responsible | | RippleExecRespons\_3 | 1 | Not at all responsible | | | 2 | Somewhat responsible | | | 3 | Very responsible | | | 4 | Completely responsible | | RippleExecRespons\_4 | 1 | Not at all responsible | | | 2 | Somewhat responsible | | | 3 | Very responsible | | | 4 | Completely responsible | | RippleExecRespons\_5 | 1 | Not at all responsible | | | 2 | Somewhat responsible | | | 3 | Very responsible | | | 4 | Completely responsible | | Plot1 | 0 | D | | | 1 | A | | Plot2 | 0 | D | | | 1 | A | | Plot3 | 0 | D | | | 1 | B | | PMap1 | 0 | D | | | 1 | A | | PMap2 | 0 | D | | | 1 | C | | PMap3 | 0 | D | | | 1 | B | | PMap4 | 0 | D | | | 1 | A | | PMap5 | 0 | C | | | 1 | D | | PMap6 | 0 | D | | | 1 | B | | PMap7 | 0 | D | | | 1 | A | | PMap8 | 0 | C | | | 1 | B | | PMap9 | 0 | D | | | 1 | A | | PMap10 | 0 | D | | | 1 | C | | StkFlw\_1 | 1 | Stock | | | 2 | Flow | | StkFlw\_2 | 1 | Stock | | | 2 | Flow | | StkFlw\_3 | 1 | Stock | | | 2 | Flow | | StkFlw\_4 | 1 | Stock | | | 2 | Flow | | StkFlw\_5 | 1 | Stock | | | 2 | Flow | | StkFlw\_6 | 1 | Stock | | | 2 | Flow | | StkFlw\_7 | 1 | Stock | | | 2 | Flow | | StkFlw\_8 | 1 | Stock | | | 2 | Flow | | StkFlw\_9 | 1 | Stock | | | 2 | Flow | | StkFlw\_10 | 1 | Stock | | | 2 | Flow | | StkFlw\_11 | 1 | Stock | | | 2 | Flow | | StkFlw\_12 | 1 | Stock | | | 2 | Flow | | StkFlw\_13 | 1 | Stock | | | 2 | Flow | | StkFlw\_14 | 1 | Stock | | | 2 | Flow | | Fdbck\_1 | 1 | Positive | | | 2 | Negative | | | 3 | Neither | | Fdbck\_2 | 1 | Positive | | | 2 | Negative | | | 3 | Neither | | Fdbck\_3 | 1 | Positive | | | 2 | Negative | | | 3 | Neither | | Fdbck\_4 | 1 | Positive | | | 2 | Negative | | | 3 | Neither | | Fdbck\_5 | 1 | Positive | | | 2 | Negative | | | 3 | Neither | | Fdbck\_6 | 1 | Positive | | | 2 | Negative | | | 3 | Neither | | Fdbck\_7 | 1 | Positive | | | 2 | Negative | | | 3 | Neither | | Fdbck\_8 | 1 | Positive | | | 2 | Negative | | | 3 | Neither | | gender | 1 | Male | | | 2 | Female | | | | | | WhiteVNonwhite | 1 | White | | | 2 | Nonwhite | ## Code/software The csv files can be viewed in a wide range of programs, including Excel, Numbers, SPSS, and R.