The Need for Sound Judgment in Analyzing U.S. Environmental Protection Agency Enforcement Actions

I agree completely with most of the issues raised in Mark Atlas's article "Rush to Judgment" in this issue (Atlas 2001). First, environmental justice is an important research topic that is too often addressed by writers with preconceived notions about the presence or absence of discrimination in environmental protection. Second, I agree with Mark's critique of the National Law Journal (NLJ) study, the first to investigate possible racial or class inequities in civil environmental penalties. Third, I agree that researchers need to use appropriate data, controls, and statistical techniques and that there is no evidence that civil penalties for violating environmental regulations are systematically lower in poor and/or minority communities. Finally, I agree that in "Rush to Judgment" Atlas makes a unique and valuable contribution to the environmental justice literature-which is why I recommended that the manuscript be published in Law & Society Review. Nevertheless, I believe that "Rush to Judgment" has substantial weaknesses that detract from its stature as a piece of social science research. Not surprisingly, I disagree with Mark's assessment of the validity of the conclusions from my previous work in this area-conclusions that "Rush to Judgment" reinforces in every respect. More important, however, I believe that Mark's article displays fundamental flaws regarding how we use evidence to draw generalizable conclusions about the world around us, and how one makes use of extant research to carve out a niche for one's own work. Clarifying the Evaluative Standard The claim Mark makes in "Rush to Judgment" is that substantive misunderstandings and incorrect methodological choices render my previous work regarding environmental civil penalties "unreliable" or "per se unreliable" (Atlas 2001:643, 651). Reliability is an unusual platform from which to critique the work of others, since it is a property of measures and measurement rather than a property of research designs, models, or conclusions. Reliability is generally thought of as the ability of a measure or test to produce the same value in repeated trials under various circumstances. Using this standard, the conclusions in Ringquist (1998) are consistent across sampling frames, three measures of the dependent variable, and dozens of different models. Thus, the results in my previous research are nothing if not "reliable" in the traditional sense of the term. The issue at the heart of Mark's critique is not reliability, but validity-in particular, internal validity. In all prepublication versions of "Rush to Judgment," the term validity was used. After I pointed out in my review of the manuscript that conclusions regarding the internal validity of my research need not follow logically from the critique offered, Mark replaced "invalid" with "unreliable," but the critiques and their associated conclusions remained unchanged.' But let's be clear. At issue is the assertion that the results of my previous research cannot be trusted-a classic definition of internal validity. I am confident that both the critiques of the work, and the conclusions following from these critiques, are in error, and I make my case to this effect in the following pages by addressing (1) Mark's criticism of the models representing civil penalty assessment in environmental protection, (2) his criticisms of data accuracy, and (3) errors of omission and commission found in "Rush to Judgment." Modeling Civil Penalties in Environmental Protection Theoretical Refutation of the Modeling Critique The lion's share of the criticism in "Rush to Judgment" centers on the inclusion of a group of control variables in Ringquist 1998. In a critique longer than the text of my original research note, each control used in Ringquist 1988 but not in "Rush to Judgment" is argued to be inappropriate. While space constraints limit my response to the five controls that attract the most criticism, these refutations are generalizable to the other control variables. …