Response to “Erratum for Glynn and Quinn (2011)”

We thank Pellissier (2016) for finding, and making public, the coding error that occurred in Glynn and Quinn (2011). In this short response, we want to (1) clarify what we see as the contribution of Glynn and Quinn (2011) and (2) demonstrate that the coding error uncovered by Pellissier (2016) has negligible impact on what we see as the most important results in Glynn and Quinn (2011). In our view, the main point of Glynn and Quinn (2011) was purely methodological—not substantive—signaled by the lack of any substantive claims in the abstract. We wanted to demonstrate that conditioning on a post-treatment variable could substantially tighten the bounds on ATC (and other estimands). The election-day registration (EDR) application was used to illustrate this methodological point. More specifically, the logistic regression results analyzed in Pellissier (2016) were never meant to be (in our view) truly credible estimates of ATC. Indeed, the reason for introducing the estimates based on logistic regression (and matching) was to show that these estimates were implausibly large when viewed relative to the bounds we obtained. The logistic regression results are only important in relation to the bounding analysis in the paper. A more important question, in our view, is how the coding mistake impacted our bounding analysis. This bounding analysis was the main point of the paper. If the coding mistake greatly changed these results, then the usefulness of this application would be called into question. However, and importantly, the underlying methodological point regarding conditioning on a post-treatment variable to narrow the Manski bounds would still be valid, as that is a logical result that does not depend on the specifics of particular data sets. We have rerun the bounding analysis in our original article after correcting the coding mistake. While there are some differences in the bounding analysis, these actually strengthen the points made in the paper (the upper bounds get lower). In combination with the corrected logit and matching results, the general points that we were using the EDR example to illustrate are qualitatively the same. Specifically, under very conservative assumptions, the estimates of ATC from all of the corrected logit specifications (and the matching estimate) are all implausibly large. This should be of little surprise since the corrected logit estimates reported in Pellissier (2016) are all in the same general neighborhood as the estimates we originally reported. For our purposes, the percentage of change reported in Pellissier (2016) is less relevant than the simple difference in the estimates. The smallest corrected logit estimate is 0.089, and the smallest logit estimate in the original paper is 0.096. Further, with the corrected data, the upper bound on ATC based on our Assumptions 1–3 actually decreases from 0.11 (Glynn and Quinn 2011, 284) to 0.10. Note that this bound would be reduced further if we reduced the implausibly large values of 0.05 for the direct effect (pdir 1⁄4 0:05) and 0.05 for the uninterested caused to register (pregjU1⁄41 1⁄4 0:05). Furthermore, even with these conservative values for pdir and pregjU1⁄41, in order to reach the new logistic regression estimate of 0.089, we must “simultaneously believe that at least [84]% of the unregistered individuals in rows 4–12 of Table 5 would have registered and voted had EDR been Political Analysis (2016) pp. e3–e4 doi:10.1093/pan/mpv036