# “Black-and-White” thinking: Does visual contrast polarize moral judgment? Independent replications and extension of Study 1

Kevin Vezirian, Elisa Sarda, Laurent Bègue, Pierre-Jean Laine, Hans Ijzerman

<figure><img src="image_1.png" /></figure>

## To cite this version:

Kevin Vezirian, Elisa Sarda, Laurent Bègue, Pierre-Jean Laine, Hans Ijzerman. “Black-and-White” thinking: Does visual contrast polarize moral judgment? Independent replications and extension of Study 1. Journal of Experimental Social Psychology, 2025, 117, pp.104712. 10.1016/j.jesp.2024.104712. hal-04958162

HAL Id: hal-04958162

https://hal.science/hal-04958162v1

Submitted on 27 Feb 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
<figure><img src="image_3.png" /></figure>

# “Black-and-White” thinking: Does visual contrast polarize moral judgment? Independent replications and extension of Zarkadi and Schnall’s (2013) Study 1☆

<figure><img src="image_4.png" /></figure>

Kevin Vezirian<sup>a,b,*</sup>, Elisa Sarda<sup>c</sup>, Laurent Bègue<sup>a,c</sup>, Pierre-Jean Laine<sup>d</sup>, Hans IJzerman<sup>a,c</sup>

## ARTICLE INFO

### Keywords:

- Replication

- Moral judgment

- Visual contrast

- Conceptual metaphor

- Heinz dilemma

## ABSTRACT

Does a black-and-white contrast background lead to more extreme moral judgments? Zarkadi and Schnall (2013) found in their Study 1 (N = 111) that, indeed, exposing English-speaking participants to a black-and-white (versus two other-colored conditions) background polarized participants' judgments in a moral dilemma task. This study supported a moral intuitionist model of moral judgment, lent further support to so-called Conceptual Metaphor Theories (Lakoff & Johnson, 1999), and provided evidence that not only the colors "black" and "white" influence our moral perception, but that the metaphorical associations with the combination of those two colors (i.e., "black-and-white thinking", Yin-Yang) led people to have more extreme moral judgments. Due to the striking nature of this finding, yet given various factors that undermine confidence in its veracity, we have decided to re-examine the question. A first study conducted on a large French-speaking sample (N = 8602), and two subsequent direct replications conducted on English-speaking samples (N = 365 and N = 215), failed to reveal any influence of background color on the evaluation of a moral dilemma. Numerous exploratory and supplementary analyses, including controlling for relevant covariates associated with variations in morality (e.g., age, gender), did not account for our consistent inability to replicate the original findings. This research suggests that age may influence perceived morality, with older participants (at least in Study 1) viewing Heinz's behavior as more wrong. Overall, this research suggests that it is doubtful that the evaluation of the Heinz's dilemma can be influenced by a subtle black-and-white visual priming.

## 1. Theoretical background

Moral judgment has been one of the core areas of social psychology, and moral judgments have traditionally been viewed as the product of rational and deliberate reasoning processes (see Jones, 1991; Turiel, 1985). However, central to most models of moral judgment in social psychology is, at least in part, an intuitionist model that proposes that automatic intuitive processes influence moral judgment (e.g., Haidt, 2001). From that perspective, people often make moral judgments and decisions that they only thereafter rationalize (Haidt et al., 2000). For example, studies have found that physical disgust influences people's feelings of moral disgust (e.g., Schnall, Benton and Harvey, 2008a;

Schnall, Haidt, Clore, & Jordan, 2008b; but see Johnson et al., 2014). There is thus good reason to believe, on theoretical and empirical grounds, that factors unrelated to the moral question at hand, such as visual stimuli, could influence moral judgments.

External cues, such as brightness, have been related to morality, and research suggests that light perception influences morality (Banerjee et al., 2012; Frank & Gilovich, 1988; Zhong et al., 2010). For example, research has found that recalling immoral (versus moral) behavior in an experimental task led to the perception of the experimental room to be darker (Banerjee et al., 2012) and that darkness could lead to immoral behaviors (Liu et al., 2023; Zhong et al., 2010). But beyond luminosity, could colors be related to morality, and to a greater extent influence

<footnote><sup>a</sup> Department of Psychology, Univ. Savoie Mont Blanc, Univ. Grenoble Alpes, LIP/PC2S, F-3800 Grenoble, France</footnote>

<footnote><sup>b</sup> School of Psychological Science, University of Western Australia, Perth, Australia</footnote>

<footnote><sup>c</sup> Institut Universitaire de France, France</footnote>

<footnote><sup>d</sup> Dahu EURL, 73290 La Motte-Servolex</footnote>

<footnote><sup>e</sup> Nantes Universités, Univ. Angers, Laboratoire de Psychologie des Pays de la Loire, LPPL, UR 4638, F-44000 Nantes, France</footnote>

<footnote>* This paper has been recommended for acceptance by Dr. Joris Lammers</footnote>

<footnote>* Corresponding author at: Université Savoie Mont Blanc, France.
E-mail address: kevin.vezirian@univ-smb.fr (K. Vezirian).</footnote>
morality?

Indeed, research shows that colors, such as green, can influence moral judgment (e.g., De Bock et al., 2013; Gan et al., 2016). And it is widely recognized that colors are imbued with specific meanings (see Wierzbicka, 1990). However, within the literature on the relationship between colors and morality, the most striking findings pertain to the contrast between black and white, which carry distinct associations; white is linked to goodness and morality, whereas black is connected to badness and immorality (Adams & Osgood, 1973). This association between those colors and (im)morality is so strong, that it is reported that they influence visual perception and moral judgment. For instance, when Greek words are written in white and shown to non-Greek readers, they are perceived as being more moral than when other words are written in black (Yin & Ye, 2014). Similarly, the gray font of a word is perceived to be lighter (thus leaning toward white) when the word is shown to be good, and the font is perceived darker (leaning toward black) when the word is shown to be bad, and when the font color of a word is matching its valence (i.e., good-related word written in white), people are faster at categorizing it (Meier et al., 2004). The latter findings also extend to morality; for example, naming white words in a Stroop task was quicker when words were associated with morality, while reversely naming black words was quicker when they were associated with immorality (Sherman & Clore, 2009). Interestingly, this pattern extends to object perception, as white goods are perceived by buyers as being more moral and ethical than black goods (Chan & Meng, 2021).

But do the two colors presented at the same, as in checkers, could affect moral judgment? Zarkadi and Schnall (2013) interesting research goes beyond examining the simple association between good-white, and black-bad. They show that the presentation of a black-and-white checkered pattern, affects, and especially polarizes, moral reasoning. In their first study, Zarkadi and Schnall (2013, hereafter ZS1 to refer to the study) relied on the Heinz Dilemma (Kohlberg et al., 1983), which is probably one of the better-known paradigms to measure stages of moral development. The Heinz Dilemma describes the situation of a man, Heinz, whose wife is severely ill and who needs a lifesaving-drug that he cannot buy because of its very high price. Eventually, Heinz becomes desperate and decides to steal the drug from the druggist. The scenario induces two conflicting motivations: stealing is bad but so is letting one's significant other die, and readers of this dilemma are asked to rate how right or wrong it was of Heinz to steal the drug.

Relying on this paradigm, Zarkadi and Schnall (2013) predicted and found that when English-speaking participants were asked to rate Heinz's behavior while being primed with a black-and-white checkered background in the form of a banner framing the dilemma (as opposed to a gray or blue-and-yellow background), they gave greater responses further from the midpoint of the scale and thus became more extreme in their moral judgments of the behavior. The authors posit that, because of the repeated use of verbal metaphor (i.e., “black-and-white thinking”) and the widespread metaphorical representations around black and white, which reminds the eternal dualism of the good vs. the evil (i.e., Yin-Yang), a subtle presentation of a black-and-white pattern would put people in a “black-and-white thinking” mindset, consequently leading to more Manichaean and binary reasoning.

This finding is stimulating as it extends a long line of research focusing independently on the one hand on black and on the other hand on white, and this research claims that the presentation of a black-and-white pattern activates metaphorical concept of dualism and put individuals in a specific metaphorical mindset. As the authors concluded, “people not only hold strong associations between certain colors and notions of morality but they also use a given kind of mindset implied in perceptual cues when they form a moral judgment” (Zarkadi & Schnall, 2013, p. 358).

The notion that irrelevant physical metaphors related to colors can bias people's responses is not new and draws on the concept of embodiment (see Meier et al., 2004, p. 86). The idea is directly related to

Conceptual Metaphor Theory (Lakoff & Johnson, 1999). Conceptual Metaphor Theory posits that abstract concepts, such as morality, are structured and shaped by concrete experiences through “co-experiences.” To our knowledge, the research by Zarkadi and Schnall is the first to demonstrate that colors not only influence and are associated with our perception of morality but also show that metaphorical conceptions related to the combination of black and white, along with our use of verbal metaphors, can intensify our moral judgments.

### 1.1. Reasons to replicate ZS1

When re-examining original studies about Conceptual Metaphor Theory in replications, researchers have been confronted with mixed results: for instance, some studies (i.e., Boroditsky, 2001; Schnall, Benson and Harvey, 2008a) failed to be replicated (see Chen, 2007; Johnson et al., 2014), whereas other studies (i.e., Boroditsky & Ramscar, 2002; Meier et al., 2004) succeeded to be replicated (see Ebersole et al., 2016; Meier et al., 2015). By successfully polarizing moral judgment in the Heinz dilemma simply by presenting a black-and-white checkered pattern, ZS1 is striking.

Altogether, we chose to re-examine this research question and to replicate this study for several reasons. First, from a completely naïve point of view, it is impressive that such a subtle black-and-white checkered banner (see Fig. 1), could polarize moral judgment (in comparison to two other banners), and that the extremity of our morality judgment could be that easily influenced. Given the relative simplicity of producing such a moral polarization effect, we think it's even more surprising to find a lack of widespread conceptual replications or societal and marketing considerations of this effect, especially since it is suggested that a simple black-and-white floor tiling could unconsciously influence legal actors and bias judgments in a courtroom (Zarkadi & Schnall, 2013, p. 359). If replicated, this research could provide valuable insights for consumer choice studies (see Chan & Meng, 2021), and should gain societal interest.

Second, we believe that ZS1 provided insufficient evidence for the effect they intended to demonstrate and for the claim they discussed. Using their design, a sensitivity analysis indicates<fnref n="1" /> a minimum detectable effect size of Cohen's *d* = 0.68 (f = 0.34). The fact that they detected an effect<fnref n="2" /> smaller than that (Cohen's *d* = 0.45 (f = 0.22)) may suggest that the original effect was incorrectly estimated. In addition, the confidence intervals for the polarization effects reported in ZS1 were very wide and approached 0 (i.e, *d* = 0.52 CI95% [0.13, 0.90] and *d* = 0.45 CI95% [0.06, 0.83]), further raising doubts about the stability of the effect. The uncertainty regarding the size of the effect observed in ZS1, based on statistical grounds, makes it a relevant candidate for replication (Isager et al., 2023; Pittelkow et al., 2023).

Third, original research sits in the idea that the use of the verbal metaphor (i.e., ‘black-and-white thinking’) and the jointly metaphorical connotation associated with this combination of colors could lead to a ‘black-and-white mindset’. However, if black-and-white pattern leads to more moral polarization because of the presentation of both black and white, one could say that within these same lines that a gray background -which is nothing but mix of black and white- should lead to less moral polarization. Crucially, this would also be in line with other metaphors as in adopting a “gray-thinking” or “seeing shades of gray”, which is

<footnote n="1">¹ Sensitivity analysis ran in G*Power (Faul et al., 2007), setting an error alpha rate of 0.05, a power of 0.95, 111 participants, 1 tested predictor and 3 total predictors.</footnote>

<footnote n="2">² This was the smallest effect detected, which was found when comparing the black-and-white condition to the gray condition. The other effect comparing the black-and-white to the blue-and-yellow was slightly bigger, but still below the threshold of detection: Cohen's *d* = 0.52, CI95% [0.13, 0.90]. Effect size and confidence interval were retrieved using “effectsize” R package (Ben-Shachar et al., 2020).</footnote>
<figure><img src="image_5.png" /><figcaption>Fig. 1. Black-and-white experimental manipulation used in ZS1. Figure reproduced from "Black and White" thinking: Visual contrast polarizes moral judgment (p. 357), by T. Zarkadi and S. Schnall, 2013.</figcaption></figure>

clearly “the fact of it not being clear in a situation what is right and wrong” (Cambridge Dictionary, 2024). It is surprising that original authors didn’t push further their logical reasoning, and this possibility to draw alternative hypothesis relying on the same logical reasoning, is showing the necessity to double-check the original findings. If an effect of the background color was to be replicated, it should also be examined whether it is possible that gray leads to more moderate and medial moral judgments, rather than black-and-white leading to more extreme moral judgments, and the role of the gray color into a lower moral polarization should eventually be discussed.

Fourth, jointly with the replication crisis, the evidence for many findings on priming is unclear (e.g., Chivers, 2019; Open Science Collaboration, 2012). It has been even explicitly stated that concerns regarding research on priming were legitimate and acknowledgeable, and that efforts need to be made to restore the field's credibility, particularly by conducting replications (see Kahneman, 2012). If a subtle black-and-white priming was found to be strong enough to unconsciously 'lead people to think in a black and white manner' (Zarkadi & Schnall, 2013, p. 355), then it is better to ensure its credibility by replicating it.

Fifth tracking the replicability of findings is a good way to assess the state of a field (Schiavone & Vazire, 2023), and several studies relying on the Conceptual Metaphor Theory failed to be replicated. Indeed, the study – if replicated – provides important theoretical evidence for a theory that has not fared well so far in replications (i.e., Conceptual Metaphor Theory).

Sixth, if replicated, the study confirms that not only black on one hand and white on the other, but also black-and-white together, is formative in how we form moral judgments, and especially that this color pattern could have an interactive effect by leading to polarization. It could contribute to the limited literature on the link between color and morality and may also indicate that moral judgments rely, at least in part, on less conscious processes.

Seventh, and finally, this original research has a non-negligible scope, and especially since it has been found to be of theoretical importance and empirically plausible in the first place when first published in a high impact journal, which makes it partly relevant for replication (see Pittelkow et al., 2023; but also “Pottery Barn rule”, Chambers, 2018; Srivastava, 2012). Moreover, at the onset of the

replication crisis, it was already argued that a modest citation count (≈ 100 times, see Makel et al., 2012, p. 541) is informative as for the replicability value of a research. The original research has a decent citation count (56 citations on Google Scholar at the time of the writing), and should reach this arbitrary ‘100 citations’ threshold in a just a few years, especially because this research is regularly cited in reviews and chapters as evidence that unrelated visual cues influence moral judgment (i.e., Brugman, 2024; Ellemers et al., 2019; Landau & Keefer, 2015), making important to double-check it (Pittelkow et al., 2023).

For all the reasons mentioned above, and because there are objectively no obstacles to make direct replication a routine aspect of psychological science (Zwaan et al., 2018; but see Coles et al., 2018), we believe it is important to attempt to replicate ZS1. If the effect replicates and proves to be robust, with varying samples and through independent scrutiny, then the original research should garner greater interest, and our study could attest the robustness of the original effect, or even provide a better estimation of its strength. However, if the effect fails to be replicated, we believe that the record should be corrected, because reproducibility and self-correction are hallmarks of science (Merton, 1973; Popper, 2010).

## 1.2. The present research

To better understand the nature of the link between priming of contrasting colored backgrounds and polarized moral judgments, we ran a(n) (accidentally) large study on a French-speaking sample and two strict and direct replications on English-speaking samples. In this research, we report all analyses, data exclusion, and measures. We share all material, analysis scripts, and data online.<fnref n="3" />

Please note that original authors were contacted to obtain answers about ZS1, and invited to provide us feedback throughout the project (i.e., reviewing the first study before its launched, read a preprint version), but at no point could they help us.

<footnote n="3">³ https://osf.io/yezqd/?view_only=4ed00ece9fe04241bf1b8234943e62ab</footnote>
# 2. Study 1 – a (non-direct) replication on French-speakers

## 2.1. Method

Following the original procedure, the data were collected via a general “psychology-research website”. The French website we used (https://www.fichedepersonnalite.com/) asks within the same session and in random order, a lot of questions (i.e., 190 items) to all visitors. Questions are extremely diverse and cover a wide range of topic (e.g., happiness, empathy, cooking, friendship, fidelity, caring), some questions related to morality (i.e., speciesism measure, a single-item on the trolley dilemma, questions about moral values), and demographic and personal questions (i.e., height, weight, eyes color, time watching TV). For the full list of measures, and to see all the items flagged as related to morality, please see supplementary material. We had originally pre-registered three replication studies of 282 participants each, but after a careful review it turned out that a large amount of unplanned data were collected, we noticed several major shortcomings and deviations from the original ZS1 procedure, and we therefore decided to collapse all collected data into one large study.<fnref n="4" /> Although it is common to present battery of questions or other experimental material within the same data collection in the context of replication studies (see Ebersole et al., 2016; Klein et al., 2014, 2018), the design of this study was suboptimal and one could say that it affects its replicability value and the likeliness to replicate the original effect. Thus, and because of deviations from the original research (e.g., large battery of questions preceding the experimental task; items related to morality, loyalty, and friendship), this study can’t be considered as a direct replication of ZS1 (see Giner-Sorolla et al., 2018), but rather as an independent outlook with varying procedure, but with same material and measure.

### 2.1.1. Participants

In this study, data from 12,335 participants were collected, and participants indicated that they were located in different French-speaking countries ($N_{France}$ = 11,235; $N_{Belgium}$ = 763; $N_{Switzerland}$ = 324, $N_{unknown}$ = 13) and completed our task using different devices ($N_{computer}$ = 8602; $N_{Touchpad}$ = 3110; $N_{smartphone}$ = 610; $N_{unknown}$ = 13). We excluded participants who did not answer any questions ($N$ = 13), and we only focused on computer users, because a smaller screen could affect the perception of the visual priming (e.g., hands obscuring part of the screen).<fnref n="5" /> However, for the sake of transparency, analyses considering the different devices (smartphone/touchpad) are disclosed in supplementary material (see Table S5), and the used devices do not affect the conclusion about the absence of result. Final sample size is composed of 8602 participants, which is 77.5 times the original sample ($N_{France}$ = 7821; $N_{Belgium}$ = 536; $N_{Switzerland}$ = 238, $N_{unknown}$ = 7), and for more demographic information regarding the sample composition, see Table 1. We also include the minimum effect size we could detect based on sensitivity analyses<fnref n="6" /> conducted with G-Power (Faul et al., 2007) in Table 1.

### 2.1.2. Procedure

We displayed our task on a website with a high visibility in French-speaking countries (https://www.fichedepersonnalite.com/) from February 28th 2018 until September 13th 2018. On this website, participants were recruited to take part on a large battery of questions on (non-validated) personality traits. Following this, participants took part in our experimental task, in which they were randomly assigned to one

<table><caption>**Table 1**
Demographic Information of the original study and the two replication studies.</caption><thead><tr><th rowspan="2"></th><th colspan="1">ZS1</th><th colspan="3">Replication studies</th></tr><tr><th>Study 1</th><th>Study 1</th><th>Study 2a</th><th>Study 2b</th></tr></thead><tbody><tr><td>N</td><td>111</td><td>8602</td><td>365</td><td>215</td></tr><tr><td>M<sub>age</sub> (SD<sub>age</sub>)</td><td>28.15<br>(12.11)</td><td>28.88<br>(13.06)</td><td>38.9<br>(13.17)</td><td>20.2<br>(4.25)</td></tr><tr><td>Gender (M/F/O)</td><td>44/67/NA</td><td>3620/<br>4982/NA</td><td>224/136/<br>6</td><td>65/146/<br>5</td></tr><tr><td>Minimum detectable effect<br>size (Cohen's d)</td><td>0.68</td><td>0.08</td><td>0.38</td><td>0.48</td></tr></tbody></table>

Note. M = Mean, SD = Standard Deviation, M = Male, F = Female, O = Other. No other information about the sample, apart from visiting a psychology-website, was given in ZS1, participants in Study 1 were all located in French-speaking countries (e.g., France, Belgium, Switzerland), participants in Study 2a were all English-speakers located in the UK according to Prolific screener, participants in Study 2b were all English-speakers located in an Australian university. Sensitivity analyses were performed relying on an error alpha rate of 0.05 and a power of 0.95, a numerator *df* of 1, and 3 groups, using a between participants ANOVA.

of three conditions: the moral dilemma with a black-and-white checkered background (the experimental condition), a gray background, or a blue-and-yellow checkered background (the two control conditions). Participants rated their approval or disapproval of Heinz stealing the drug on a scale from 1 (“right”) to 7 (“wrong”). Participants did the experimental task online and we recorded the device they used (computer, smartphone, or touchpad). A script running on the website was able to determine which device participants used to let us separate into different analyses.

We invited original authors to help us to retrieve the original material, but they could not help us. Thus, we reproduced the materials as faithfully as possible from the original study using the material presented in the appendix of the original paper, but unfortunately the font type and size were different. We translated the Heinz dilemma into French and used “Paul” instead of “Heinz”, please see Appendix B for the original materials and Appendix C for the translated materials.

## 2.2. Results

### 2.2.1. Analytic plan

We conducted the same statistical analyses as the original study and present the results in Table 3 (and in supplementary material). First, to assess whether background color affected moral judgment, we computed a deviation score corresponding to each rating’s absolute distance from the scale’s midpoint (“4”). Next, as in the original study, we created three orthogonal contrast codes to examine whether rating the gray background differed from rating the black-and-white backgrounds., if the rating on the blue-and-yellow background differed from the black-and-white, if the rating on the gray background differed from the blue-and-yellow. Importantly, in this study and in all the following ones, we created a crucial fourth orthogonal contrast (i.e., C4) that was, surprisingly, not reported in the original search, which compares whether the ratings on the gray background (i.e., first control condition) and on the blue-and-yellow backgrounds (i.e., second control condition) differed from the ratings on the black-and-white background (i.e., experimental condition). From the battery of personality items, we only analyzed and reported demographic information (i.e., gender, age, location), as we were not interested in the unvalidated personality questions.

### 2.2.2. Confirmatory analyses

We first tested the overall effect between the three groups, but found no differences despite the (very) large sample size, $F(2, 8599) = 2.72, p = .06, \eta p^2 < 0.001, 90 \%CI[0.0,0.0]$, which indicates that in our data the color of the background did not significantly influence the perception of

<footnote n="4">⁴ We disclose all information in supplementary material such as the original preregistration plans and analyses.</footnote>

<footnote n="5">⁵ As suggested by a reviewer, although the original research does not provide any information about participants’ used devices.</footnote>

<footnote n="6">⁶ We relied on an error alpha rate of 0.05 and a power of 0.95, a numerator *df* of 1, and 3 groups, using a between participants ANOVA.</footnote>
<table><caption>**Table 2**
Means and standard deviations of the different backgrounds for the different studies.</caption><thead><tr><th></th><th>Black-and-white condition</th><th>Blue and yellow condition</th><th>Gray condition</th></tr></thead><tbody><tr><td>Zarkadi & Schnall, 2013</td><td><i>M</i> = 1.73, <i>SD</i> = 0.87</td><td><i>M</i> = 1.14, <i>SD</i> = 0.98</td><td><i>M</i> = 1.22, <i>SD</i> = 1</td></tr><tr><td>Study 1</td><td><i>M</i> = 1.90, <i>SD</i> = 1.05</td><td><i>M</i> = 1.88, <i>SD</i> = 1.05</td><td><i>M</i> = 1.94, <i>SD</i> = 1.04</td></tr><tr><td>Study 2a</td><td><i>M</i> = 1.81, <i>SD</i> = 0.98</td><td><i>M</i> = 1.59, <i>SD</i> = 0.99</td><td><i>M</i> = 1.81, <i>SD</i> = 0.99</td></tr><tr><td>Study 2b</td><td><i>M</i> = 1.38, <i>SD</i> = 0.99</td><td><i>M</i> = 1.43, <i>SD</i> = 0.92</td><td><i>M</i> = 1.49, <i>SD</i> = 1.1</td></tr></tbody></table>

the moral dilemma, see Table 2. We then tested all the four different contrasts (see Table 3) and only found an effect comparing the two different control conditions. This suggests that participants exposed to a gray background exhibited more polarized moral judgments compared to those exposed to blue-and-yellow backgrounds, $F(1, 8599) = 5.21, p = .02, d = 0.05, 95 \%CI [0.01, 0.09]$. However, if we push ZS1 theoretical reasoning a bit further, we could have expected to observe that participants exposed to the “gray” condition would be the least prone to moral polarization, precisely because, according to the metaphor, “seeing shades of gray” means not being able to determine whether something is good or wrong. Thus, the found experimental effect goes against all theoretical predictions, and we believe that it is very likely due to the (very) large sample size, as it is barely detectable.

### 2.2.3. Exploratory analyses

**Theoretical Relevance.** To explain our failure to experimentally produce a moral polarization, and to try to bring a theoretical contribution, we then conducted exploratory analyses on our data,<fnref n="7" /> especially examining the roles of participants’ gender and age on the moral polarization.

Age differences in morality have long been a subject of interest. For example, Heinz’s dilemma was originally developed to measure stages of moral development, and research suggests that morality follows a cognitive-developmental trajectory as individuals grow older (Kohlberg, Levine and Hewer, 1983; but see Turiel, 2007, 2015, 2018). Although Kohlberg’s strict perspective is now debated, exploring age-related differences in the perception of Heinz’s behavior remains relevant.

Additionally, gender differences exist when it comes to moral reasoning (Atari et al., 2020; Freeman & Giebink, 1979; although small Jaffee & Hyde, 2000), especially with women scoring higher in care and lower in justice, than men. This is also extremely relevant in the context of Heinz’s dilemma, especially since gender differences in Heinz’s dilemma have been a long matter of discussion (e.g., Gilli0gan, 1998), and women should see Heinz’s behavior as righter than men.

Beyond the interest sparked by these individual variations, participants’ ages and genders might noise the experimental ‘black-and-white thinking’ effect, and it is worth controlling for those covariates.

**Results.** We recoded the gender (men = +1, women = -1), and we excluded from the analysis all participants who did not report their age ($n = 298$) or who reported ages inferior to the France legal limit (< 15 years old, despite having stressed in the consent form that participants should be major to participate to our research, $n = 120$), leaving a final sample of 8184 participants (4848 women, 3336 men).

We first launched a Spearman’s correlation between participants’ ages and the answer on the raw evaluation of Heinz’s dilemma (i.e., not on the transformed measure capturing the deviation from the midpoint of the scale). The analysis revealed an (extremely) low positive relation between age and the rating of the dilemma, according to which the older participants are, the more wrong they see Heinz’s behavior, $r_{spearman}$ (8182) = .05, $p < .001$.

We then ran an independent Welch’s $t$-test comparing the scores of Men and Women on the raw evaluation of the dilemma. The analysis showed, against our expectations, than women perceived Heinz’s behavior of stealing the drug as being more wrong ($M = 2.69, SD = 1.66$)

than men did ($M = 2.41, SD = 1.63$), $t(7253.8) = 7.71$ $p < .001$, $d = 0.18$ 95 %CI [0.13, 0.23].

Finally, and crucially, we then ran the same analyses than in our confirmatory part but controlling for participants’ gender and age. Our omnibus test revealed a significant effect of the background color on moral polarization, $F(2, 8179) = 3.12, p = .04$. The subsequent contrast analyses revealed that the only significant result who hold was, echoing results shown in the confirmatory analyses (see Table 3), the contrast testing the difference between the two control conditions each (i.e., C3), which indicates that the gray background would polarize more the moral evaluation of the Heinz’s dilemma that would the gray background, $F(1, 8179) = 6.06, p = .01$. This result is neither in line with a “black-and-white” thinking effect, nor with “gray-thinking” (i.e., “seeing shades of gray” and less polarization from participants exposed to the gray background). This effect was quite weak and close to zero (Cohen’s $d = 0.05, 95 \%CI [0.01, 0.10]$), and result should be taken with caution. None the other contrasts tests showed significance (i.e., C1, $p = .10$; C2, $p = .38$; C4, $p = .68$).

Interestingly, however, the omnibus tests revealed an effect of the gender.

$F(1, 8179) = 83.44, p < .001, d = 0.20, [0.16, 0.25]$, which indicates that men had more polarized and extreme judgment of Heinz’s dilemma ($M = 2.04, SD = 1.02$) than women ($M = 1.83, SD = 1.06$).

### 2.2.4. Discussion

A main goal of this first study was to examine the nature of the link between priming of contrasting colored backgrounds and polarized moral judgments on a diverse population speaking a different language (i.e., French). Unfortunately, we were unable to observe that priming a black-and-white (as opposed to blue-and-yellow or gray) background could significantly influence the perception of a moral judgment, and lead to polarization. This lack of results is unexpected given the (very) large sample size, which should have been sufficient to detect, even tiny, statistically significant effects. Even if factors such as participants’ language, nationality variation, previous questions about morality and loyalty, or cognitive fatigue, might have diminished the strength of it, we doubt that the concurrence of those factors could have make the original effect (i.e., Cohen’s $d = 0.45$) to completely vanish, raising doubts as for the stability of this effect and its prior estimation. If it hypothesized that this black-and-white effect is multicultural, deeply rooted, and is sufficiently strong to influence legal actors in courtroom, it should have been detectable despite (numerous) variations from the original research.

We also benefit of the opportunity of this research to build upon ZS1 and to extend the original analytic plan by scrutinizing the role between two relevant factors (i.e., age and gender) on Heinz’s behavior. Importantly, these factors were not considered in the initial study. This study shows, in line with Kohlberg’s reasoning, that as participants get older, they tend to value more Justice and Laws, as they perceived Heinz’s behavior of stealing the drug as being more wrong. However, in this study and in contradiction with the literature, women also perceived Heinz’s stealing as being more wrong than men did. Those results should be considered with extreme caution because of the exploratory nature of the analyses and the very small effects size, which question their importance.

Crucially, we included age and gender as covariates in our model to determine whether they might explain our failure to replicate ZS1.

<footnote n="7">⁷ As suggested during the reviewing process.</footnote>
<table><caption>**Table 3**
Contrast analyses for the original and the two replication studies.</caption><thead><tr><th></th><th>Gray vs. Black-and-white (C1)</th><th>Blue-and-yellow vs. Black-and-white (C2)</th><th>Gray vs. Blue-and-yellow (C3)</th><th>Gray &amp; Blue-and-yellow vs. Black-and-white (C4)</th></tr></thead><tbody><tr><td>Zarkadi &amp; Schnall, 2013</td><td>F(1, 108) = 5.38, p = .02, d = 0.45, [0.06, 0.83]</td><td>F(1, 108) = 7.22, p = .008, d = 0.52, [0.13, 0.90]</td><td>F(1, 108) = 0.13, p = .71, d = 0.07, [-0.31, 0.45]</td><td>-</td></tr><tr><td>Study 1</td><td>F(1, 8599) = 2.47, p = .11, d = 0.03, [-0.01, 0.08]</td><td>F(1, 8599) = 0.56, p = .45, d = 0.02, [-0.03, 0.06]</td><td>F(1, 8599) = 5.21, p = .02, d = 0.05, [0.01, 0.09]</td><td>F(1, 8599) = 0.23, p = .63, d = 0.01, [-0.03, 0.05]</td></tr><tr><td>Study 2a</td><td>F(1, 363) = 0.00, p = .98, d = 0.01, [-0.12, 0.21]</td><td>F(1, 362) = 3.05, p = .08, d = 0.18, [-0.02, 0.39]</td><td>F(1, 362) = 3.06, p = .08, d = 0.18, [-0.02, 0.39]</td><td>F(1, 362) = 0.97, p = .32, d = 0.10, [-0.10, 0.31]</td></tr><tr><td>Study 2b</td><td>F(1, 212) = 0.38 p = .53, d = 0.09, [-0.18, 0.35]</td><td>F(1, 212) = 0.09, p = .76, d = 0.04, [-0.23, 0.31]</td><td>F(1, 212) = 0.09, p = .75, d = 0.04, [-0.23, 0.31]</td><td>F(1, 212) = 0.28, p = .59, d = 0.07, [-0.20, 0.34]</td></tr></tbody></table>

*Notes.* Gray and Blue-and-yellow vs. Black-and-white corresponds to the contrast opposing the black-and-white condition to those two same control conditions, controlling for gray vs. blue-and-yellow (C4). Effect sizes are Cohen's *d* and numbers between brackets are 95 % confidence intervals. Effect sizes and confidence intervals were retrieved using the “effectsize” R package (Ben-Shachar et al., 2020).

Indeed, with such a large sample size, a part of variance polarization explained by gender and/or age could have potentially accounted for our replication failure. Unfortunately, this was inconclusive, as adding these covariates did not alter the null conclusions as for the color effect on polarization. However, we observed an interesting result showing that men had more extreme moral judgment than women.

### 2.2.5. Limitations

This study nevertheless has several limits. First, we can't consider this study as a strict, direct replication particularly because recruited participants were not English-speakers, and despite the suggestion that this metaphor is thought to be multicultural, we can't exclude the possibility that the language difference impacted the effect. More importantly, our experimental task took place at the end of a large survey, and although other large-scale replications show no effects of unrelated questions in replication attempts (Klein et al., 2018) and that our very large sample would have allowed tiny effects to be detected, we can't dismiss entirely cognitive fatigue as explanations for our lack of significant results as having weakened the original effect. Crucially, 22 items in this question battery (≈14 %) can be related to themes similar to the Heinz dilemma (i.e., fidelity, loyalty, morality), and this could have affected the replicability value of this study (Giner-Sorolla et al., 2018).

Finally, the numerous shortcomings and deviations that led us to merge our data and discard our planned pre-registration studies (see supplementary material) might have, because of the untidiness, affected the perceived reliability of our findings. Altogether, this present study doesn't really answer confidently whether ZS1 is replicable or not, and this study should not be considered as a direct replication, but rather as an independent outlook of the research question.

## 3. Study 2a – a first direct replication on English-speakers and from the UK

Given the limits of our large follow-up study that failed to reproduce the ZS1 effect, we decided to conduct a first strict preregistered<fnref n="8" /> replication of ZS1 on English-speaking sample.

### 3.1. Methods

#### 3.1.1. Power analysis

Because our first study could potentially represent a flawed test of the effect of the black-and-white metaphor on the moral evaluation, considering its limits (i.e., cognitive fatigue, French-speakers, numerous items related to morality preceding the task), we did not base this study rationale sample size on it, but decided to seek a design allowing to replicate the original findings based on their reported effects sizes. Smallest effect size found in ZS1 was of Cohen's *d* = 0.45 ($f^2$ = 0.075) in

the condition comparing the gray and black-and-white backgrounds. In this direct replication we set our design to be able to find a smaller effect size than the smallest one found in ZS1 and we will target a Cohen's *d* = 0.30 ($f^2$ = 0.0225).

A-priori power analysis indicates that 351 participants would be needed to find the targeted effect size if it exists (considering a $f^2$ = 0.0225, 0.05 error alpha rate, .80 power, a numerator df of 1 and 3 groups). Expecting a slight loss of participants (e.g., non-computer users), we preregistered to collect data from 375 participants.

#### 3.1.2. Sampling plan and participants

We used pre-screening on Prolific (80–100 approval rate, English-speakers as a first language, UK located, and computer users). Participants were invited to take part in our research through Prolific for a 3-min study advertised as a study on decision making (as in ZS1), and in addition to the pre-screening criteria, this study was advertised as being for computer-users. Participants were paid £0.39 (labelled as a fair price on the platform) in return for their participation in our study.

Data from 375 participants were collected and 9 participants have been excluded from main analyses for having reported not using a computer to run this study, leaving a final sample size of 365 participants (3.3 times ZS1 sample size), see Table 1 for descriptive statistics. Including those participants did not affect the conclusion of the results (see supplementary material, Table S5).

#### 3.1.3. Procedure

We recruited participants through Prolific and redirected participants toward a Qualtrics survey. First, participants read a consent form and instructions. Next, we presented our experimental task and participants were randomly assigned to one of the three conditions (see Appendix B). To minimize the risk of introducing any bias into this study, participants were exposed to screenshots of the stimuli as reported in ZS1, and were instructed to rate the moral dilemma by directly clicking on it (see Appendix A). After the task, participants took part in unrelated research, had to answer to a few demographic questions (i.e., age, gender, nationality), and report which device they used for the completion of this study. Finally, participants were thanked and debriefed.

Regardless of the unrelated task that followed the replication task, the procedure and materials used in Study 2a were identical to those in ZS1.

### 3.2. Results

#### 3.2.1. Analytic plan

As in Study 1, we computed deviation scores and created four contrast codes (C1-C4) and present the results in Table 3 (and in supplementary material).

<footnote n="8">⁸ https://osf.io/z47j9/?view_only=2755a5206b954a8c866325f914b9951c</footnote>
### 3.2.2. Confirmatory analyses

One participant was excluded for having failed to rate the moral dilemma judgment evaluation.

As in Study 1, we first tested the overall effect between the three groups, but found no significant differences, $F(2, 362) = 2.06, p = .13$, $ηp^2 = 0.01, 95\%CI [0.0,1]$, which suggests that we failed again to find an effect of the color of the background on the moral polarization, see Table 2. We then tested all four different contrasts, and we didn't find any evidence attesting that a black-and-white background color could polarize a moral evaluation more than any other background color, see Table 3. Oddly, the contrast analysis comparing the two different control conditions came reasonably close to the significance threshold. As 'marginally significant' ($p = .08$), as well as the contrast analysis comparing the black-and-white background to the blue-and-yellow one ($p = .08$). All other things being equal, the greater the number of control conditions and the more extensive the multiple testing, the higher the likelihood of observing *p*-values around .05 (Cumming, 2013; Simmons et al., 2011). Thus, these patterns are likely due to the 'dance of *p*-values', making it crucial to focus on contrast analyses that test the experimental condition against all control conditions simultaneously (i.e., C4) to enhance the reliability of the findings.

## 4. Study 2b – a second direct replication on English-speakers and from Australia

To avoid limiting our conclusions to the shortcomings of our independent perspective and to gain more rigorous evidence, we decided to conduct a second direct replication of ZS1.

### 4.1. Methods

#### 4.1.1. Sampling plan and participants

Participants are students enrolled in psychology course who received 0.5 credit point for their time completing our study. We originally aimed for a sample size of 400 participants, but we failed to reach this sample size in due time and stopped data collection at the end of the academic year. Data from 250 participants were collected, and participants completed our task using different devices ($N_{computer} = 215$; $N_{Touchpad} = 6$; $N_{smartphone} = 6$; $N_{unknown} = 23$). Fifteen participants who didn't complete our task were excluded, and as in previous studies we also excluded participants who completed the task on a device other than a computer or didn't mention the device they used ($n = 20$), but again, including all participants in the analyses did not affect the conclusion of the results (see supplementary material, Table S5).

Final sample size is composed of 215 participants (1.9 times ZS1 sample size), see Table 1 for descriptive statistics as well as sensitivity analysis.

#### 4.1.2. Procedure

We recruited participants using the SONA Research Participation System of the University of Western Australia. The study was advertised as “A study on decision making,” as in ZS1 and Study 2a.

The procedure and the used material were strictly identical to Study 2a. Only changes in this research is the subsequent unrelated task that came after the replication task, which was different. Additionally, at the end of the survey, we gave participants the opportunity to provide feedback on the entire study, but as can be seen in the open-access data, none of them provided feedback specifically about the first task. Finally, participants were thanked and debriefed.

Regardless of the unrelated task which came after the replication task, procedure and material of Study 2b is strictly identical to ZS1.

### 4.2. Results

#### 4.2.1. Analytic plan

As in Studies 1 and 2a, we computed deviation scores and created four contrast codes (C1-C4) and present the results in Table 3 (and in supplementary material).

#### 4.2.2. Confirmatory analyses

As in previous studies, we first tested the overall effect between the three groups, but found no significant differences, $F(2,212) = 0.19, p = .82$, $ηp^2 = 0.001, 95\%CI [0.0,1]$, which suggests, again, a failure to find an effect of the background color on moral evaluation, see Table 2. We then tested all four different contrasts, and we didn't find any evidence attesting that a black-and-white background color could polarize more a moral evaluation than any other background color, see Table 3.

### 4.3. A mega-analytic perspective on the effect of age and gender on morality

Given the identical nature of our direct replications, we ran a mega-analysis (see Eisenhauer, 2021) by merging Studies 2a and 2b to scrutinize again the effects of age and gender on moral perception of the Heinz's dilemma. For the complete report on this mega-analysis, please see supplementary material. In this mega-study, we excluded participants who reported other gender identities than Male or Female ($n = 7$) or refused to answer to this question ($n = 3$), and participants' gender was recoded as in Study 1 (i.e., Male = +1, Female = -1), leaving 570 participants in the sample (281 women, 289 men, $M_{age} = 32.1, SD_{age} = 14.5$).

Following Study 1 exploratory analyses, we examined the relation between participants' ages and the evaluation of Heinz's dilemma, which, unfortunately, failed to reproduce the significant pattern seen in Study 1, $r_{spearman} (568) = .07, p = .08$. Interestingly, the independent Welch's *t*-test comparing the scores of Men and Women on the perception of the dilemma also failed to replicate the previous significant pattern, $t(567.07) = -0.69 p = .49, d = -0.06 95\%CI [-0.22, 0.11]$.

Our attempt to replicate the existing relations between age, gender, and morality, found in Study 1 was unsuccessful. First, we believe that the effect of age observed in Study 1 was so subtle ($n = 8184, r = .05, p < .001$) that our mega-study might have been underpowered to significantly reproduce this pattern, although the effect size appears descriptively similar. Second, our failure to replicate the gender differences in responses to the moral dilemma aligns with research on moral gender development, which emphasizes that gender effects in morality are generally very small (see Jaffee & Hyde, 2000). This may explain why Study 1 detected a very slight gender effect, yet the mega-study did not replicate this finding, likely due to a lower sample size.

### 4.4. Discussion

The aim of Studies 2a and 2b was to conduct strict replications of ZS1 on English-speaking samples (from the UK and from Australia), but with a greater statistical power than ZS1, and in unimpaired designs (in comparison to Study 1). Despite those efforts, we failed again to find evidence that priming a black-and-white pattern (as opposed to blue-and-yellow or gray) surrounding a moral dilemma could influence the perception of the said dilemma. Is it possible that our direct replications studies were insufficiently powered to detect the effect? Probably not, since they both were better powered than ZS1 (respectively 3.3 and 1.9 times more participants than ZS1), but also because a mega-study merging their data (5.1 times more participants than ZS1) also failed to replicate the effect, despite our effort to control *a-posteriori* for
<figure><img src="image_6.png" /><figcaption>**Fig. 2.** Meta-analysis of the findings examining whether a black-and-white background could lead to more moral polarization in comparison to the two other control background (i.e., “C4”).</figcaption></figure>

relevant covariates, which could have dampened the effect (see supplementary material). Altogether, our two failures to replicate ZS1 effect even through direct replications and on English-speaking participants, continue to leave persistent doubts as for the possibility of replicating this effect.

# 5. Supplementary analyses

## 5.1. Mini meta-analyses

We conducted mini meta-analyses of our findings (see Goh et al., 2016) using the “metaphor” R package (Viechtbauer, 2010). In a first analysis, we focused on the crucial contrast analysis (i.e., “C4”), which tests whether the experimental condition led to more moral polarization in comparison to the two control condition. Meta-analytic effect was not significant, Z = 0.68, *p* = .50, *d* = 0.02 95 % CI [-0.03, 0.06], and heterogeneity test showed no significant variation between our studies, Q(2) = 0.79, *p* = .67. We then conducted another meta-analysis, but this time including ZS1, but had to estimate C4 from the means, SDs and sample size reported in the original paper, as ZS1 did not report it, C4: F (1,108) = 8.24, *d* = 0.55 95 % CI [0.17, 0.94]. Despite the addition of the strong (but wide) original effect, the meta-analytic effect was still not significant, Z = 1.32, *p* = .19, *d* = 0.11 95 % CI [-0.05, 0.27], see Fig. 2, and heterogeneity test showed no significant variation between our studies, Q(3) = 7.53, *p* = .06.

## 5.2. Normality assumptions and non-parametric tests

As the original report did not check for crucial assumptions, we did not include them in the initial confirmatory analyses of our studies. Nevertheless, any usage of ANOVA should be preceded by checks for distribution. The assumption of normality was systematically violated for our three studies (Shapiro-Wilk Test, all *p* ≤ .001). Because the Central Limit Theorem prescribes that distributions approach normality as long as the sample is sufficiently large (Rosenblatt, 1956), we also

plotted the distributions for all our samples, and this showed the same problem with normality (see supplementary material, Figs. S3, S4, S5). We therefore applied square root transformations to our data, but the general distributions were still not normal in our two studies (Shapiro-Wilk test, *p* < .001; see supplementary material). We thus proceeded to perform Kruskal-Wallis tests (a non-parametric test) on the non-transformed data, and importantly, the overall effect between the three experimental groups over polarized judgment was still not significant across our three studies, and for any of the used-devices (see supplementary material, Table S6).

## 5.3. A Bayesian perspective

After our frequentist approach, we ran Bayesian regression analyses for a further look at our data. Bayes factor ranges from 0 to ∞ and a Bayesian approach particularly allows to quantify the strength of the evidence that the data provide either in favor of one statistical model or another (Kass & Raftery, 1995). In our three studies we examined to which extent our data supported the alternative hypothesis (BF10; Wagenmakers, 2007). Every single test (n = 40) ran on all the different contrast, across the studies and the used devices supported the null hypothesis. For instance, and focusing on computer-users, in our data the absence of a moral polarization comparing the black-and-white background color to the two other control conditions (contrast known as ‘C4’), is 33.33 likely in Study 1, 5.55 likely in Study 2a, and 5.88 likely in Study 2b (see supplementary material, Table S6).

# 5.4. General discussion

According to the original article a black-and-white priming condition should lead to polarized moral judgment in all people, because this contrast is “commonly depicted in visual images (e.g., the Yin-Yang symbol)” (Zarkadi & Schnall, 2013, p. 356). Further, the authors posited that “the distinction between the light and dark would be fundamental” and that the black and white metaphor “might constitute a
basic metaphor” (Zarkadi & Schnall, 2013, p. 358; our emphasis added). However, if black-and-white dualism as good and bad symbolism is indeed multicultural and widespread, as in Chinese philosophical writings linked to Taoism (i.e., Ying-Yang symbol, Taylor & Choy, 2005), Ancient Greek philosophy (Britannica Encyclopaedia, 2010), or in famous French La Fontaine’s 17th century fable commonly taught in French high schools (see de La Fontaine & Tilney, 1913), it seems nevertheless that the presentation of this contrast that led original participants to polarize their moral judgment, is not easily replicable.

With varying methodologies ranging from an independent outlook to two direct replications, we collected data from French and English-speakers located in varying countries (i.e., France, Belgium, Switzerland, UK, Australia), and with varying backgrounds (i.e., all-comers in Study 1, Prolific users in Study 2a, university students in Study 2b). For each study, we collected more data than the original research (i.e., N<sub>ZS1</sub> = 111, N<sub>Study1</sub> = 8602, N<sub>Study2a</sub> = 365, N<sub>Study2b</sub> = 215), used the same material and measure, and, at least for our two direct replications, followed strictly the procedure as reported by the original authors. We even enhanced the original analytic plan with the addition of another contrast analysis (i.e., “C4”), conducted a mega-analysis to looked for plausible explanations, and performed substantial supplementary analyses (see supplementary material). Despite those efforts, at no point did we observe results supporting the idea that a black-and-white frame could polarize moral judgment more than other controlled colors, even after exploratory analyses controlling for relevant covariates. Why did we fail to replicate ZS1? Unfortunately, null findings come with the inability to draw confident conclusions, but we argue that these results may stem from several, potentially concurrent, factors.

Sometimes, statistical fluctuations lead to Type-1 error, and our research suggests that the original research might represent a false-positive. One plausible reason for this could sit in the very low evidence provided by the original article, despite that ZS1's significant *p-values* (i.e., *p* = .02; *p* = .008) may indicate the opposite. Indeed, the use of *p-values* only is misleading (Cohen, 1988; Cumming, 2008) and when retrieving original effect sizes, the confidence intervals were extremely wide and close to zero (i.e., respectively CI95% Cohen's *d* [0.06, 0.83] and CI95% Cohen's *d* [0.13, 0.90]). The larger the confidence intervals, the poorest the estimation of the effects and their stability (Cumming, 2013, 2014), and the closer a bound is to zero, the higher it is close to be insignificant.

It might also be that we failed to replicate the original findings because they sit in theoretical frameworks that don't do very well in replicability and thus don't seem extremely reliable. For instance, Zar-kadi and Schnall found their findings as "supporting support approaches of embodied cognition that propose a central role of metaphor and language in cognitive representations" (Zarkadi & Schnall, 2013, p.358), and the embodiment cognition is another theory arguing that our morality is influenced by other factors, such as our bodily experi-ence. Yet, sometimes studies conducted within this framework (i.e., Bargh & Shalev, 2012; Jostmann et al., 2009; Slepian et al., 2012; Szymkow et al., 2013; Vess, 2012; Zhong & Liljenquist, 2006) also don't hold well replications attempts (i.e., Ebersole et al., 2016; LeBel & Campbell, 2013; LeBel & Wilbur, 2014; Pecher et al., 2015; Wortman et al., 2014), and especially when it comes to moral-related research (e.g., Earp et al., 2014; Johnson et al., 2014). For instance, in a substantial handbook on embodiment cognition defined as a landmark of the field, a key chapter about replicability highlights that the social psychological literature related to embodiment and morality is "unfortunately filled with unreplicable empirical reports, so much so that it has become a paradigmatically unreplicable literature in psychology" (Machinery,

2024, p.478). This fragile theoretical pillar supporting ZS1 could have weakened the replicability value of this original work.

What does this mean for how color priming affects our judgment or decision-making overall? Again, we don't know. Perhaps that the manipulations and measures for this type of work have been insufficiently developed to study the effect of color on people's decision-making (see also Szabelska et al., 2021).

## 5.5. General limitations

Our work has nevertheless several limits. First, we acknowledge, once again, that our Study 1 is to be considered as an independent outlook and not as a replication, due to too many deviations from the original research, particularly because some items in our question battery were related to morality or loyalty. However, we believe that the two strict and identical replications of ZS1 that followed have addressed this concern.

## 5.6. Conclusion

Given the limited evidence provided by the original research and our successive failures to replicate the existing results, or even to detect smaller effect sizes than those originally observed, we don't think it is safe to assume that a black-and-white checkered banner can polarize moral judgment evaluation, and especially Heinz's moral dilemma evaluation.

In conclusion, science isn't just black and white, and something that has been found to be true once may turn out to be false upon closer examination. Because "non-reproducible single occurrences are of no significance to science" (Popper, 2010, p.66), we believe that it is important to set the record straight when opportunity knocks.

## Open practices

For the (initial) preregistration plans of Study 1, please see online supplementary material. For the preregistration of Study 2a, as well as for the material, data and analysis scripts of all studies, please see the Open Science Framework page of the project: https://osf.io/yezqd/?view_only=ecf8677aad864d85b997a352c056b537

## CRediT authorship contribution statement

**Kevin Vezirian:** Methodology, Formal analysis, Resources, Data curation, Writing - original draft, Writing - review & editing, Visualization, Project administration. **Elisa Sarda:** Formal analysis. **Laurent Bègue:** Conceptualization, Writing - original draft, Supervision. **Pierre-Jean Laine:** Resources. **Hans IJzerman:** Writing - original draft, Writing - review & editing, Supervision.

## Declaration of competing interest

The authors of the manuscript declare having no relevant financial or non-financial competing interest to report.

## Acknowledgments

The authors would like to thank Rebecca Patsy Paterson for her time proofreading this manuscript, as well as the handling editor and the reviewers who have contributed to this paper over the reviewing process.
# Appendix A. Appendices

## Appendix A (original material used in ZS1 and in Studies 2a and 2b)

<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /></figure>

<figure><img src="image_9.png" /></figure>
## Appendix B (translated material used in Study 1)

<figure><img src="image_10.png" /></figure>

## Appendix. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.jesp.2024.104712.

## References

Adams, F. M., & Osgood, C. E. (1973). A cross-cultural study of the affective meanings of color. *Journal of Cross-Cultural Psychology, 4*(2), 135–156. https://doi.org/10.1177/002202217300400201

Atari, M., Lai, M. H. C., & Dehghani, M. (2020). Sex differences in moral judgements across 67 countries. *Proceedings of the Royal Society B: Biological Sciences*, 287(1937), 20201201. https://doi.org/10.1098/rspb.2020.1201

Banerjee, P., Chatterjee, P., & Sinha, J. (2012). Is it light or dark? Recalling moral behavior changes perception of brightness. *Psychological Science*, 23(4), 407–409. https://doi.org/10.1177/0956797611432497

Bargh, J. A., & Shalev, I. (2012). The substitutability of physical and social warmth in daily life. *Emotion, 12*(1), 154–162. https://doi.org/10.1037/a0023527

Ben-Shachar, M., Lüdecke, D., & Makowski, D. (2020). effectsize: Estimation of effect size indices and standardized parameters. *Journal of Open Source Software*, 5(56), 2815. https://doi.org/10.21105/joss.02815

Boroditsky, L. (2001). Does language shape thought?: Mandarin and English speakers' conceptions of time. *Cognitive Psychology, 43*(1), 1–22. https://doi.org/10.1006/cogp.2001.0748

Boroditsky, L., & Ramscar, M. (2002). The roles of body and mind in abstract thought. *Psychological Science*, 13(2), 185–189. https://doi.org/10.1111/1467-9280.00434

Britannica Encyclopaedia. (2010). *table of opposites*. Encyclopedia Britannica. http://www.britannica.com/topic/table-of-opposites.

Brugman, B. C. (2024). How the effects of emphasizing ethics are examined: A systematic review of moral framing experiments. *Annals of the International Communication Association*, 1–20. https://doi.org/10.1080/23808985.2024.2393845

Cambridge Dictionary. (2024). Shades of grey. In *Cambridge English dictionary*. Retrieved November 26, 2024, from https://dictionary.cambridge.org/dictionary/english/shades-of-grey.

Chambers, C. (2018, October 15). *Reproducibility meets accountability: Introducing the replications initiative at Royal Society Open Science*. The Royal Society. Retrieved November 27, 2024, from https://royalsociety.org/blog/2018/10/reproducibility-meets-accountability/.

Chan, E. Y., & Meng, Y. (2021). Color me moral: White and black product colors influence prosocial behaviors. *Psychology & Marketing*, 38(1), 212–224. https://doi.org/10.1002/mar.21435

Chen, J.-Y. (2007). Do Chinese and English speakers think about time differently? Failure of replicating Boroditsky (2001). *Cognition*, 104(2), 427–436. https://doi.org/10.1016/j.cognition.2006.09.012

Chivers, T. (2019). What's next for psychology's embattled field of social priming. *Nature*, 576(7786), 200–202. https://doi.org/10.1038/d41586-019-03755-2

Cohen, J. (1988). *Statistical power analysis for the behavioral sciences* ((2nd ed.)). L. Erlbaum Associates.
Coles, N. A., Tiokhin, L., Scheel, A. M., Isager, P. M., & Lakens, D. (2018). The costs and benefits of replication studies. *Behavioral and Brain Sciences, 41*, Article e124. https://doi.org/10.1017/S0140525X18000596

Cumming, G. (2008). Replication and p intervals: P values predict the future only vaguely, but confidence intervals do much better. *Perspectives on Psychological Science*, 3(4), 286–300. https://doi.org/10.1111/j.1745-6924.2008.00079.x

Cumming, G. (2013). *Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis* (1re éd.). Routledge. https://doi.org/10.4324/9780203807002

Cumming, G. (2014). The new statistics: Why and how. *Psychological Science*, 25(1), 7–29. https://doi.org/10.1177/0956797613504966

De Bock, T., Pandelaere, M., & Van Kenhove, P. (2013). When colors backfire: The impact of color cues on moral judgment. *Journal of Consumer Psychology*, 23(3), 341–348. https://doi.org/10.1016/j.jcps.2012.09.003

Earp, B. D., Everett, J. A. C., Madva, E. N., & Hamlin, J. K. (2014). Out, damned spot: Can the “Macbeth Effect” be replicated? *Basic and Applied Social Psychology*, 36(1), 91–98. https://doi.org/10.1080/01973533.2013.856792

Ebersole, C. R., Atherton, O. E., Belanger, A. L., Skulborstad, H. M., Allen, J. M., Banks, J. B., ... Nosek, B. A. (2016). Many Labs 3: Evaluating participant pool quality across the academic semester via replication. *Journal of Experimental Social Psychology*, 67, 68–82. https://doi.org/10.1016/j.jesp.2015.10.012

Eisenhauer, J. G. (2021). Meta-analysis and mega-analysis: A simple introduction. *Teaching Statistics*, 43(1), 21–27. https://doi.org/10.1111/test.12242

Ellemers, N., van der Toorn, J., Paunov, Y., & van Leeuwen, T. (2019). The psychology of morality: A review and analysis of empirical studies published from 1940 through 2017. *Personality and Social Psychology Review*, 23(4), 332–366. https://doi.org/10.1177/10886836818811759

Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. *Behavior Research Methods*, 39(2), 175–191. https://doi.org/10.3758/BF03193146

Frank, M. G., & Gilovich, T. (1988). The dark side of self- and social perception: Black uniforms and aggression in professional sports. *Journal of Personality and Social Psychology*, 54(1), 74–85. https://doi.org/10.1037/0022-3514.54.1.74

Freeman, S. J. M., & Giebink, J. W. (1979). Moral judgment as a function of age, sex, and stimulus. *The Journal of Psychology*, 102(1), 43–47. https://doi.org/10.1080/00223980.1979.9915093

Gan, T., Fang, W., & Ge, L. (2016). Colours' impact on morality: Evidence from event-related potentials. *Scientific Reports*, 6(1), 38373. https://doi.org/10.1038/srep38373

Giner-Sorolla, R., Amodio, D. M., & Van Kleef, G. A. (2018). Three strong moves to improve research and replications alike. *Behavioral and Brain Sciences, 41*, Article e130. https://doi.org/10.1017/S0140525X18000651

Goh, J. X., Hall, J. A., & Rosenthal, R. (2016). Mini meta-analysis of your own studies: Some arguments on why and a primer on how: Mini meta-analysis. *Social and Personality Psychology Compass*, 10(10), 535–549. https://doi.org/10.1111/spc3.12267

Haidt, J. (2001). The emotional dog and its rational tail: A social intuitionist approach to moral judgment. *The Journal of Psychology*, 108(4), 814–834. https://doi.org/10.1037/0033-295X.108.4.814

Haidt, J., Bjorklund, F., & Murphy, S. (2000). *Moral dumbfounding: The role of intuition and reasoning in moral judgment: (633872013–071)* [Data set]. American Psychological Association. https://doi.org/10.1037/e633872013-071

Isager, P. M., Van Aert, R. C. M., Bahník, S., Brandt, M. J., DeSoto, K. A., Giner-Sorolla, R., ... Lakens, D. (2023). Deciding what to replicate: A decision model for replication study selection under resource and knowledge constraints. *Psychological Methods*, 28(2), 438–451. https://doi.org/10.1037/met0000438

Jaffee, S., & Hyde, J. S. (2000). Gender differences in moral orientation: A meta-analysis. *Psychological Bulletin*, 126(5), 703–726. https://doi.org/10.1037/0033-2909.126.5.703

Johnson, D. J., Cheung, F., & Donnellan, M. B. (2014). Does cleanliness influence moral judgments? A direct replication of. *Social Psychology*, 45(3), 209–215. https://doi.org/10.1027/1864-9335/a000186

Jones, T. M. (1991). Ethical Decision Making by Individuals in Organizations: An Issue-Contingent Model. *The Academy of Management Review*, 16(2), 366. https://doi.org/10.2307/258867

Jostmann, N. B., Lakens, D., & Schubert, T. W. (2009). Weight as an embodiment of importance. *Psychological Science*, 20(9), 1169–1174. https://doi.org/10.1111/j.1467-9280.2009.02426.x

Kahneman, D. (2012). A proposal to deal with questions about priming effects. Retrieved November 26, 2024, from http://www.nature.com/polopoly_fis/7.6716.13492713081/suppinfoFile/Kahneman%20Letter.pdf.

Kass, R. E., & Raftery, A. E. (1995). Bayes factors. *Journal of the American Statistical Association*, 90(430), 773–795. https://doi.org/10.1080/01621459.1995.10476572

Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Bahník, S., Bernstein, M. J., ... Nosek, B. A. (2014). Investigating variation in replicability: A “many labs” replication project. *Social Psychology*, 45(3), 142–152. https://doi.org/10.1027/1864-9335/a000178

Klein, R. A., Vianello, M., Hasselman, F., Adams, B. G., Adams, R. B., Alper, S., ... Nosek, B. A. (2018). Many labs 2: Investigating variation in replicability across samples and settings. *Advances in Methods and Practices in Psychological Science*, 1(4), 443–490. https://doi.org/10.1177/2515245918810225

Kohlberg, L., Levine, C., & Hewer, A. (1983). *Moral stages: A current formulation and a response to critics*. Karger.

de La Fontaine, J., & Tilney, F. C. (1913). *The original fables of La Fontaine*. New York: E. P. Dutton.

Lakoff, G., & Johnson, M. L. (1999). *Philosophy in the flesh: The embodied mind and its challenge to western thought*. Basic Books.

Landau, M., & Keefer, L. (2015). Situations change thought and behavior through metaphor. In I. N. R. Branscombe, & K. J. Reynolds (Eds.), *Psychology of change: Life contexts, experiences, and identities* (pp. 189–208). Psychology Press. https://doi.org/10.4324/9781315735160.

LeBel, E. P., & Campbell, L. (2013). Heightened sensitivity to temperature cues in individuals with high anxious attachment: Real or elusive phenomenon? *Psychological Science*, 24(10), 2128–2130. https://doi.org/10.1177/095679713486983

LeBel, E. P., & Wilbur, C. J. (2014). Big secrets do not necessarily cause hills to appear steeper. *Psychonomic Bulletin & Review*, 21(3), 696–700. https://doi.org/10.3758/s13423-013-0549-2

Liu, H., Zhou, Y., & Yamada, Y. (2023). Darkness promotes dishonesty in a coin toss task: A pre-registered conceptual replication of Experiment 1 of Zhong, Bohns, and Gino (2010). *PLoS One*, 18(12), Article e0294484. https://doi.org/10.1371/journal.pone.0294484

Machinery, E. (2024). The replication crisis in embodied cognition research. In L. A. Shapiro, & S. Spaulding (Eds.), *The Routledge handbook of embodied cognition* (2nd ed., pp. 475–484). Routledge. https://doi.org/10.4324/97818003322511.

Makel, M. C., Plucker, J. A., & Hegarty, B. (2012). Replications in psychology research: How often do they really occur? *Perspectives on Psychological Science*, 7(6), 537–542. https://doi.org/10.1177/1745691612460688

Meier, B. P., Fetterman, A. K., & Robinson, M. D. (2015). Black and white as valence cues: A large-scale replication effort of. *Social Psychology*, 46(3), 174–178. https://doi.org/10.1027/1864-9335/a000236

Meier, B. P., Robinson, M. D., & Clore, G. L. (2004). Why good guys wear white: Automatic inferences about stimulus valence based on brightness. *Psychological Science*, 15(2), 82–87. https://doi.org/10.1111/j.0963-7214.2004.01502002.x

Merton, R. K. (1973). *The sociology of science: Theoretical and empirical investigations*. University of Chicago Press.

Open Science Collaboration. (2012). An open, large-scale, collaborative effort to estimate the reproducibility of psychological science. *Perspectives on Psychological Science*, 7(6), 657–660. https://doi.org/10.1177/1745691612462588

Pecher, D., Van Mierlo, H., Canal-Bruland, R., & Zeelenberg, R. (2015). The burden of secrecy? No effect on hill slant estimation and beanbag throwing. *Journal of Experimental Psychology: General*, 144(4), e65–e72. https://doi.org/10.1037/xge0000090

Pittelow, M.-M., Field, S. M., Isager, P. M., Van'T Veer, A. E., Anderson, T., Cole, S. N., ... Van Ravenzwaaij, D. (2023). The process of replication target selection in psychology: What to consider? *Royal Society Open Science*, 10(2), Article 210586. https://doi.org/10.1098/rsos.210586

Popper, K. (2010). *The logic of scientific discovery* (Special Indian ed.). Routledge.

Rosenblatt, M. (1956). A central limit theorem and a strong mixing condition. *Proceedings of the National Academy of Sciences*, 42(1), 43–47. https://doi.org/10.1073/pnas.42.1.43

Schiavone, S. R., & Vazire, S. (2023). Reckoning with our crisis: An agenda for the field of social and personality psychology. *Perspectives on Psychological Science*, 18(3), 710–722. https://doi.org/10.1177/17456916221101060

Schnall, S., Benton, J., & Harvey, S. (2008a). With a clean conscience: Cleanliness reduces the severity of moral judgments. *Psychological Science*, 19(12), 1219–1222. https://doi.org/10.1111/j.1467-9280.2008.02227.x

Schnall, S., Haidt, J., Clore, G. L., & Jordan, A. H. (2008b). Disgust as Embodied Moral Judgment. *Personality and Social Psychology Bulletin*, 34(8), 1096–1109. https://doi.org/10.1177/0146167208317771

Sherman, G. D., & Clore, G. L. (2009). The color of sin: White and black are perceptual symbols of moral purity and pollution. *Psychological Science*, 20(8), 1019–1025. https://doi.org/10.1111/j.1467-9280.2009.02403.x

Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. *Psychological Science*, 22(11), 1359–1366. https://doi.org/10.1177/095679711417632

Slepian, M. L., Micampamo, E. J., Toosi, N. R., & Ambady, N. (2012). The physical burdens of secrecy. *Journal of Experimental Psychology: General*, 141(4), 619–624. https://doi.org/10.1037/a0027598

Srivastava, S. (2012, September 27). *A pottery barn rule for scientific journals*. The Hardest Science. Retrieved November 27, 2024, from https://thehardestscience.com/2012/09/27/a-pottery-barn-rule-for-scientific-journals/.

Szabelska, A., Dujols, O., Erle, T. M., Sparacio, A., & IJzerman, H. (2021). Towards theory formalization in (social) embodiment: A tutorial. In M. D. Robinson, & L. E. Thomas (Eds.), *Handbook of embodied psychology* (pp. 339–356). Springer International Publishing. https://doi.org/10.1007/978-3-030-78471-3_15

Szymkow, A., Chandler, J., IJzerman, H., Parzuchowski, M., & Wojciszke, B. (2013). Warmer hearts, warmer rooms: How positive communal traits increase estimates of ambient temperature. *Social Psychology*, 44(2), 167–176. https://doi.org/10.1027/1864-9335/a000147

Taylor, R. L., & Choy, H. Y. F. (2005). *The illustrated encyclopedia of Confucianism* (1st ed.). The Rosen Publishing Group.

Turiel, E. (1985). *The development of social knowledge: Morality and convention* (Repr.). Cambridge Univ. Press.

Turiel, E. (2007). The development of morality. In W. Damon, & R. M. Lerner (Eds.), *Handbook of child psychology* ((1re éd.).) Wiley. https://doi.org/10.1002/9780470147658.chpsy0313.

Turiel, E. (2015). Moral development. In R. M. Lerner (Ed.), *Handbook of child psychology and developmental science* (1re éd., pp. 1–39). Wiley. https://doi.org/10.1002/9781118963418.childpsy113.

Turiel, E. (2018). Moral development in the early years: When and how. *Human Development*, 61(4–5), 297–308. https://doi.org/10.1159/000492805
Vess, M. (2012). Warm thoughts: Attachment anxiety and sensitivity to temperature cues. *Psychological Science*, 23(5), 472–474. https://doi.org/10.1177/0956797611435919

Viechtbauer, W. (2010). Conducting meta-analyses in R with the **metafor** package. *Journal of Statistical Software*, 36(3). https://doi.org/10.18637/jss.v036.i03

Wagenmakers, E.-J. (2007). A practical solution to the pervasive problems of p values. *Psychonomic Bulletin & Review*, 14(5), 779–804. https://doi.org/10.3758/BF03194105

Wierzbicka, A. (1990). The meaning of color terms: Semantics, culture, and cognition. *Cognitive Linguistics*, 1(1), 99–150. https://doi.org/10.1515/cogl.1990.1.1.99

Wortman, J., Donnellan, M. B., & Lucas, R. E. (2014). Can physical warmth (or coldness) predict trait loneliness? A replication of Bargh and Shalev (2012). *Archives of Scientific Psychology*, 2(1), 13–19. https://doi.org/10.1037/arc00000007

Yin, R., & Ye, H. (2014). The black and white metaphor representation of moral concepts and its influence on moral cognition. *Acta Psychologica Sinica*, 46(9), 1331. https://doi.org/10.3724/SP.J.1041.2014.01331

Zarkadi, T., & Schnall, S. (2013). “Black and White” thinking: Visual contrast polarizes moral judgment. *Journal of Experimental Social Psychology*, 49(3), 355–359. https://doi.org/10.1016/j.jesp.2012.11.012

Zhong, C.-B., Bohns, V. K., & Gino, F. (2010). Good lamps are the best police: Darkness increases dishonesty and self-interested behavior. *Psychological Science*, 21(3), 311–314. https://doi.org/10.1177/0956797609360754

Zhong, C.-B., & Liljenquist, K. (2006). Washing Away Your Sins: Threatened Morality and Physical Cleansing. *Science*, 313(5792), 1451–1452. https://doi.org/10.1126/science.1130726

Zwaan, R. A., Etz, A., Lucas, R. E., & Donnellan, M. B. (2018). Making replication mainstream. *Behavioral and Brain Sciences*, 41, Article e120. https://doi.org/10.1017/S0140525X17001972