Beware of using invalid transmission models to guide HIV health policy – Authors' reply
Authors: Timothy B Hallett, Jeffrey W Eaton, Nicolas Menzies
Journal: The Lancet Global Health (2014)

We agree with Justin Okano and Sally Blower that every effort must be made to ensure that important policy decisions are informed by valid models. Okano and Blower note that the models in our analysis of expanded antiretroviral therapy make different future projections for HIV incidence between 2014 and 2033 (as shown in their figure and in figure S3 in the online appendix of our report). They contend that this finding implies that at least some of the models must be invalid. Although Okano and Blower note correctly that not all of these future projections will turn out to be close to what actually occurs, they are unable to identify now which ones will be wrong. Indeed, none of these incidence projections can be validated or invalidated on the basis of existing evidence, and it is of course impossible to know which trajectory, if any, will later prove to be correct. We agree that we should only be interested in model predictions about the future if they faithfully reproduce past trends and present circumstances, to the extent that the available evidence allows us to discern these. In this case, the variation in incidence trends is the result of several transmission models, each of which synthesise the available epidemiological, behavioural, and biological evidence in a logically consistent manner. The finding that the implementation of these models produces different results therefore represents genuine uncertainty about the future. We should seek to reduce this uncertainty and we reiterate our recommendation that the conclusions of our analyses be re-evaluated in light of new data as they become available.1Eaton JW Menzies NA Stover J et al.Health benefits, costs, and cost-effectiveness of earlier eligibility for adult antiretroviral therapy and expanded treatment coverage: a combined analysis of 12 mathematical models.Lancet Glob Health. 2014; 2: e23-e34Summary Full Text Full Text PDF PubMed Scopus (158) Google Scholar The need to understand the consequences of the existing uncertainty makes it useful to apply several different models to the sample policy question. This approach was motivated by earlier work that documented the sometimes substantial differences in the effect of antiretroviral therapy in different models.2Eaton JW Johnson LF Salomon JA et al.HIV treatment as prevention: systematic comparison of mathematical models of the potential impact of antiretroviral therapy on HIV incidence in South Africa.PLoS Med. 2012; 9: e1001245Crossref PubMed Scopus (290) Google Scholar To consider the effect of uncertainty about model parameters is standard for policy analyses,3Husereau D Drummond M Petrou S et al.CHEERS Task Force. Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement.BMJ. 2013; 346: f1049Crossref PubMed Scopus (800) Google Scholar, 4Delva W Wilson DP AbuRaddad L et al.HIV treatment as prevention: principles of good HIV epidemiology modelling for public health decision making in all modes of prevention and evaluation.PLoS Med. 2012; 9: e1001239Crossref PubMed Scopus (29) Google Scholar but to account for uncertainty introduced by model structure is less common and might be especially important for dynamic disease transmission models. In this context, the finding that, despite this variation, a consistent policy recommendation emerges should rightly increase the level of confidence in that conclusion compared with the findings of one model. The alternatives of reliance on one model—or artificially requiring models to produce similar future projections without the evidentiary basis for doing so—are vulnerable to generating a false sense of security in findings, and it is of this that readers should beware. We declare that we have no competing interests. Beware of using invalid transmission models to guide HIV health policyJeffrey Eaton and colleagues (January, p e23)1 used modelling to inform international guidelines for antiretroviral therapy for HIV. Specifically, they undertook a comparative cost–effectiveness analysis based on predictions from 12 independent transmission models: seven were used for South Africa, four for Zambia, four for India, and one for Vietnam. They concluded that all 12 models show similar results—that earlier eligibility for antiretroviral therapy is cost effective.1 They implied that their consensus finding increases confidence in the use of modelling results to guide HIV health policy. Full-Text PDF Open AccessHealth benefits, costs, and cost-effectiveness of earlier eligibility for adult antiretroviral therapy and expanded treatment coverage: a combined analysis of 12 mathematical modelsOur estimates suggest that earlier eligibility for antiretroviral therapy is very cost effective in low-income and middle-income settings, although these estimates should be revisited when more data become available. Scaling up antiretroviral therapy through earlier eligibility and expanded coverage should be considered alongside other high-priority health interventions competing for health budgets. Full-Text PDF Open Access