Problems of Model Specification and Improper Data Extrapolation

We write in response to Christopher J. Fariss, ‘The Changing Standard of Accountability and the Positive Relationship between Human Rights Treaty Ratification and Compliance’. Fariss (2016) claims that the ‘standards of accountability’ of human rights violations have changed over time, and that these changes have created a systematic negative bias in standard measures of human rights. According to Fariss, this bias could be corrected by using a version of a Dynamic Item Response Model (IRM). His article purports to demonstrate the advantages of this type of data correction. He claims to show that ratifying human rights treaties really matters for human rights. However, this finding contradicts the findings of most previous research. To demonstrate the usefulness of his measurement approach, Fariss calculates two new measures of human rights, ‘corrected’ and ‘uncorrected’ human rights scores. corrected scores assume changing standards of accountability in the records of human rights violations, and uncorrected scores are based on the conventional assumptions of IRM. The difference between the results obtained from ordinary least squares models using corrected and uncorrected scores serves as his primary evidence that the changing standards of accountability in uncorrected human rights measurement are a real problem. However, the differences in the estimations reported in the article are very small, and are mainly due to two factors: the failure to control for the level of democracy and improper data extrapolation prior to 1981. In fact, Human Rights Treaty Ratification is not statistically related to corrected scores when controlling for the level of democracy. Taking into account changes in the level of democracy, we see no positive trend in the corrected scores, and thus no evidence of either ‘changing standards of accountability’ or an improving trend in human rights. Moreover, all differences in the results disappear when the analysis uses only data after 1980 (the period of the Cingranelli-Richards (CIRI) and Political Terror Scale (PTS) data availability). Therefore, we urge others who use the corrected scores to (1) control for the level of democracy and (2) include a separate analysis of the data after 1980 to make sure that their findings still hold. We do not agree that the findings show that ratifying human rights treaties improves human rights performance. The findings supporting this conclusion are the artefact of the positive time trend for democracy, in parallel with the significant increase in human rights treaty ratifications (see Figure 1A of the Appendix). There are good theoretical reasons to expect that democracies would have better human rights records, and the results of all previous research have supported that expectation. For example, see the recent review of this literature and the conclusions of Hill and Jones. Thus models explaining variation in human rights practices due to treaty ratification are misspecified unless they control for the level of democracy. Fariss includes the results of better-specified models that incorporate controls for the level of democracy. He claims that ‘overall, the choices of variables for these models does not change the difference in the relationship