Poor citation, coding and reporting: a review of adherence‐enhancing interventions for highly active antiretroviral therapy creates an inaccurate picture of the state of the field

The recent systematic review of interventions to promote antiretroviral therapy (ART) adherence by Mathes and colleagues [1] published in HIV Medicine contains several methodological flaws that lead to an overly pessimistic depiction of the state of the literature; indeed, it contradicts other widely cited reviews on the topic that demonstrate the efficacy of such interventions to improve both adherence and viral load (VL) [2–5]. Curiously, the only prior review cited was Simoni [5], and only then to support a lack of efficacy for adherence interventions, which is contrary to the main findings of that review. This bias in citation creates unfounded authority for the authors’ dire conclusions that ‘adherence enhancing interventions are not promising’. Mathes and colleagues report that only one of the 21 trials fulfilling their inclusion criteria was coded as indicating statistically significant effects for adherence and VL, and that the intervention trials were of poor quality (i.e., score high on risk of bias criteria). These conclusions, however, seem to at least partially arise from flawed, overly stringent or unclear quality coding of the interventions, as well as coding errors. Below are several examples of the many instances that make us question the quality of this work. First, the authors scored trials as being at high risk of bias when they could not find the relevant information in the article, failing to even attempt to contact the primary authors for clarification (using open-ended questions, as recommended by the Cochrane Bias Methods Group [6]), or to indicate in the review how many of the high risk of bias scores were given because data were missing. Hence, the quality of study design and the quality of reporting have become indistinguishable, and the collapsed score is likely to offer an overly negative picture of the reviewed trials. A second example is that Mathes and colleagues rated a study’s analyses as intent-to-treat (ITT) only if this was reported for both adherence and VL/biological outcomes (and presumably at all the time-points), regardless of whether the reviewed study, a priori, identified only one outcome (or time-point) as primary (or regardless of whether ITT was applied to adherence only, which Mathes and colleagues themselves define as the primary outcome). It is inaccurate and misleading to deem a study at high risk of bias when intent-to-treat analyses have been applied to the primary (if not secondary) outcomes/time-points. Note that the specifics of this ITT coding procedure were unclear based on reading the article only; instead, the authors were contacted for clarification. The scoring procedures for the other trial quality criteria remain largely unclear, so that it is hard to further discuss their appropriateness. Finally, there appear to be errors in the coding. For example, the study by de Bruin and colleagues [7] is reported as producing significant change only in VL, even though the original report also described significant effects in ITT analyses for adherence (mean difference of 7.40%; 95% confidence interval 3.50–11.30%; P < 0.001) [7]. Unusually, the coders did not contact de Bruin or colleagues to resolve the discrepancy between their interpretation of the analysis and that of the authors, and just reported that the findings of this trial were not significant. Another example is that Parsons et al. [8] scored ‘high risk’ on generation of allocation sequence despite their use of urn randomization; an acceptable sequencing procedure [6]. In sum, whereas we definitely think a critical analysis of the quality of adherence trials has merit, this review ignores key literature in this domain; assumes that if trial procedures were not clear to the coders that the study had been poorly conducted; does not distinguish between primary and secondary outcomes; did not seek additional information if matters were unclear; and makes glaring errors in interpreting studies up to the level where a significant trial is reported as nonsignificant. A review of such dubious quality derails the programme of research in the area and, more ominously, threatens to misinform public health officials and providers in ways that may limit the access of patients to potentially health-promoting interventions.