Misguided Effort With Elusive Implications

In retrospect, the decision to use new, mostly untested procedures1 for a large replication project was foolish. When planning the Registered Replication Report (RRR) on ego depletion (Hagger et al., 2016, this issue), Hagger asked Baumeister for suggestions. Baumeister nominated several procedures that have been used in successful studies of ego depletion for years. But none of Baumeister’s suggestions were allowable due to the RRR restrictions that it must be done with only computerized tasks that were culturally and linguistically neutral. Discussions were stalemated, and we felt pressured to come up with something quickly. We learned of a new study by Sripada et al. (2014) that fit the requirements and passed this along to the RRR team. Since there were no viable other options, that method was chosen. Apparently it matters how much we endorsed this method. To be clear, no one working in either of our laboratories has ever used this procedure in any study (neither the manipulation nor dependent measure). We still do not understand why reaction time variance is a measure of self-control failure (are people overriding some impulse to react with variable speed? ), but the idea of “replication” requires that something like the task has been used at least once, and the Sripada et al. paper reported successful results in a major outlet. (Perhaps we should have still objected. But because there were no other viable options, objecting would have meant objecting to the entire RRR, which could have been interpreted as lack of trust in the effect.) Under the circumstances, we understood our approval to mean “Sure, go ahead” and not “Yes, that’s a definitive test of the phenomenon we’ve been studying all these years. ” Crucially, we thought the robustness of ego depletion effects would overcome any weaknesses in this new method. That was an unfortunate mistake, partly because the weaknesses seem more serious than we had understood. The manipulation is a computerized version of what is called the e-crossing procedure. This procedure was originally created as a laboratory version of a common self-control task, namely breaking a habit. Self-regulation is typically understood as altering and overriding responses. The e-crossing task works because participants first establish a habit (of using a pencil to cross out every “e” on a page of text) and then must override these habitual responses when more complex rules are introduced. Self-regulation is invoked when the participant sees an “e” and experiences the impulse to cross it off—and then must restrain that impulse. The Sripada and RRR studies skipped the initial key step of establishing a habit. RRR participants simply pressed a button to indicate whether each word has an “e” that is not adjacent to another vowel. Without first instilling the habit, there is nothing to override. This may be a difficult cognitive judgment task, but no impulse is overridden, contrary to the nature of self-control tasks. The RRR says that skipping the initial habit-forming step was justified because other tasks in the literature have done the same, such as a manipulation in which participants are instructed to write a story with words that do not contain the letters “a” or “n” (originally by Schmeichel, 2007). Yet that task is depleting precisely because there most certainly is a very strong habit. An English speaker has spent years writing sentences using all letters of the alphabet, including “a” and “n”. This misunderstanding highlights what may be a problem in the field as a whole in its current focus on replication: It is misguided to focus merely on the simple structure of procedures while disregarding the underlying psychological processes. Scientific hypotheses concern psychological processes, not laboratory procedures. Self-report data from the RRR suggest that the task does not involve self-regulation. Manipulation checks are difficult to obtain with ego depletion, because people cannot usually report on subjective changes indicative of having expended resources in self-regulation. The closest to a reliable measure is self-reported fatigue; negative mood may increase slightly (Hagger, Wood, Stiff, & Chatzisarantis, 2010). Self-report data indicated that RRR participants found the task extremely frustrating but not fatiguing, unlike the usual pattern in ego depletion.2 One question going forward is how to create replication studies that are not constrained to computerized methods stripped of contextual factors. The admirable 652878 PPSXXX10.1177/1745691616652878Baumeister, VohsMisguided Effort With Elusive Implications research-article2016