Push button reproduction attempt of a research claim from Grose, C. R., Malhotra, N., &amp; Parks Van Houweling, R. (2015), in American Journal of Political Science
Creators: name: Talbert, Eli
Publisher: name: Open Science Framework
Year: 2022
Subjects: subject: Political Science
subject: FOS: Political science
subject: Social and Behavioral Sciences

description: We selected the claim that for those constituents in which the legislator voted in favor of the citizen's preferred policy choice, the legislator should reinforce this congruent policy position by pointing to similar policy actions. When voting against the constituent's wishes, the legislator should target the explanation toward the opposing constituent by providing additional compensating policy positions where the legislator agrees with the constituent (Statement 2). This reflects the following statement from the paper's abstract: "We find that most senators tailor their explanations to their audiences..." (Statement 1). The following documents one way in which the claim was tested. The authors use a within-subjects field experiment using U.S. senators serving in the 110th Congress (2007-08) with the assistance of constituent confederates located throughout the country. Each U.S. senator was subject to two treatments. The authors sent half of the senators a letter on behalf of a constituent that was pro-immigration reform and the other half of the senators a letter from a constituent that was anti-immigration reform. Eight weeks later, the authors sent another wave of letters. Senators to whom they had sent the anti-immigration letter in March were sent the pro-immigration letter, and the senators to whom they had sent the pro-immigration letter in March were sent the anti-immigration letter. They recruited two constituent confederates (one for the pro-immigration letter and one for the anti-immigration letter) who were willing to receive the senator's response at their home addresses in most of the fifty states. The fifty-six senators who responded to both the anti- and pro-immigration letters from constituents and their responses compose the 112 (56 x 2) within-subject observations for the analysis. In the analysis selected as evidence for the SCORE program, the dependent variable is a dichotomous measure coded 1 if the senator mentioned institutional action(s) that reduced restrictions on immigration or immigrants separate from the comprehensive immigration reform roll call (coded 0 if not). The authors compare the percentage of response letters that mention pro-immigration actions that the senator took when the letter is a response to a pro-immigration vs an anti-immigration letter. The authors also test the claim by examining the difference between anti-immigration positions of senators to pro and anti-constituents, as well as estimating the difference-in-difference (whether the difference in letters mentioning pro- or anti-immigration actions varied by constituent position), and present supportive evidence through those analyses as well (Statement 3). Supporting the authors' argument, senators appear to have strategically changed whether they mentioned pro-immigration actions in their explanations depending on the constituents' position. When receiving a pro-immigration letter from a constituent, 23.2% of letters mentioned reinforcing or compensating pro-immigration actions. This was nearly double the 12.5% of letters that mentioned pro-immigration actions in response to anti-immigration letters (p = 0.07 [two-tailed], based on the exact McNemar significance probability) (Statement 4). The focal test in the original paper is: When receiving a pro-immigration letter from a constituent, 23.2% of letters mentioned reinforcing or compensating pro-immigration actions. This was nearly double the 12.5% of letters that mentioned pro-immigration actions in response to anti-immigration letters (p = 0.07 [two-tailed], based on the exact McNemar significance probability)