University of Birmingham Economic evaluation alongside multinational studies: a systematic review of empirical studies

Purpose of the study This study seeks to explore methods for conducting economic evaluations alongside multinational trials by conducting a systematic review of the methods used in practice and the challenges that are typically faced by the researchers who conducted the economic evaluations. Methods A review was conducted for the period 2002 to 2012, with potentially relevant articles identified by searching the Medline, Embase and NHS EED databases. Studies were included if they were full economic evaluations conducted alongside a multinational trial. Results A total of 44 studies out of a possible 2667 met the inclusion criteria. Methods used for the analyses varied between studies, indicating a lack of consensus on how economic evaluation alongside multinational studies should be carried out. The most common challenge appeared to be related to addressing differences between countries, which potentially hinders the generalisability and transferability of results. Other challenges reported included inadequate sample sizes and choosing cost-effectiveness thresholds. Conclusions It is recommended that additional guidelines be developed to aid researchers in this area and that these be based on an understanding of the challenges associated with multinational trials and the strengths and limitations of alternative approaches. Guidelines should focus on ensuring that results will aid decision makers in their individual countries. PLOS ONE | DOI:10.1371/journal.pone.0131949 June 29, 2015 1 / 22 OPEN ACCESS Citation: Oppong R, Jowett S, Roberts TE (2015) Economic Evaluation alongside Multinational Studies: A Systematic Review of Empirical Studies. PLoS ONE 10(6): e0131949. doi:10.1371/journal. pone.0131949 Editor: Giampiero Favato, Kingston University London, UNITED KINGDOM Received: December 3, 2014 Accepted: June 8, 2015 Published: June 29, 2015 Copyright: © 2015 Oppong et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Data Availability Statement: All relevant data are within the paper and its Supporting Information files. Funding: The authors have no support or funding to report. Competing Interests: The authors have declared that no competing interests exist. Introduction Establishing whether new and existing health technologies provide value for money is becoming important internationally, and many countries now require evidence on cost-effectiveness for resource allocation decisions [1–2]. In the UK, the National Institute for Health and Care Excellence appraises health technologies in terms of their clinical effectiveness and cost-effectiveness [3], and economic evaluation, which is the comparison of alternatives in terms of costs and benefits, is one of the tools used for this purpose [4]. Economic evaluations have been conducted alongside trials mainly because they provide a means for collecting clinical and economic data simultaneously. One type of trial that has seen an increase in its popularity, owing to its ability to recruit participants rapidly and expedite the development of new health technologies, is the multinational trial, which is defined as a study that takes place in more than one country or jurisdiction [5–9]. One of the reasons for pooling/aggregating clinical data from cross-country studies is the belief that clinical and biological effects are homogenous across countries/jurisdictions [10]. However, the same cannot be said about economic data, owing to the vast differences in health systems, practice patterns, resource use and unit prices between countries, all of which need to be accounted for when conducting an economic evaluation [11–14]. Recent reviews of the literature revealed wide variation in the way these differences are addressed [15–16], indicating a lack of consensus among researchers. Availability of economic data in some countries also poses potential problems for researchers. A study conducted alongside a neurologic trial in 15 countries reported a dearth of unit cost information in some participating countries despite the efforts that were made to obtain these costs [17]. Generalisability (applying the results of a study to a number of countries without needing to adjust for interpretation) and transferability (adapting the results of a study to other countries) are other challenges that have been identified in the literature [18]. In principle, because of their very nature, results from cross-country studies should be more generalisable. However, it can be argued that pooled results cannot be applied to a single country owing to the inclusion of data from different jurisdictions [19]. A recent review of national guidelines on the use of data from multinational trials showed that there were vast differences in the data different countries considered to be generalisable or transferable to their settings [1], which indicates a lack of consensus among countries and which potentially limits the usefulness of cost-effectiveness estimates from multinational trials [19–20]. Resource allocation decisions are normally made at a national level [20–22], and thus there is the need to develop and agree on appropriate methods for conducting and interpreting economic analyses based on multinational trials. This would not only make results more useful to decision makers but also avoid the duplication of work in every country/jurisdiction [2]. A number of methods have been developed in response to the challenges outlined, ranging from very simple approaches such as adjusting resource use to very complex statistical approaches such as multilevel modelling [22]. The extent to which these methods have been used in practice is unclear. One study concluded that there is a need for more guidance as a result of the vast variation in methods that are being used to conduct economic analyses alongside multinational trials [15]. It is our belief that a study designed to assess the challenges reported by researchers could lead to a better understanding of the reasons methods vary and also help to develop additional guidance in this area. The objective of this study is to review published economic evaluations that were conducted alongside multinational trials with the aim of exploring methods that have been used and to outline researchers’ challenges i.e. any difficulties associated with the multinational nature of the trial. As far as we are aware, no other review has considered the challenges that have been reported by researchers who have conducted economic evaluations alongside multinational trials. Review of Multinational Economic Evaluations PLOS ONE | DOI:10.1371/journal.pone.0131949 June 29, 2015 2 / 22 Materials and Methods A systematic review was conducted following the guidelines of the Centre for Reviews and Disseminations (CRD) [23]. Inclusion and exclusion criteria Studies were included if they were full economic evaluations based on multinational trials and reported an incremental cost-effectiveness ratio (ICER) or incremental net benefit. Studies were excluded if they were modelling studies or systematic reviews, did not use patient-level data or were not published in English. Search strategy The electronic databases searched were: MEDLINE, EMBASE and the National Health Service economic evaluation database (NHS EED). The search was limited to the period 2002 to 2012 for pragmatic reasons and to capture the most recent studies. The following keywords were used in the search: multinational, cost, cost-effectiveness, cost-utility, cost-benefit, multi-country, multi-centre, trial, economic evaluation, and cross-country (S1 Table). Following an approach used by Roberts and colleagues [24], a three-stage process was used to select relevant papers (S1 Text). The screening of papers was done by all reviewers. Stage one (categorization of studies) was carried out by one reviewer who initially screened titles and abstracts of articles and classified them into 5 groups. Stage 2 (further classification of studies) and stage 3 (application of the inclusion criteria) were carried out independently by all three reviewers (see S1 Text). Results from each reviewer were compared and any differences were resolved through consultation among all reviewers. The quality of the economic evaluations was not assessed because of the study objectives and the need to include as many studies as possible. Data Extraction Data were extracted using a predefined data extraction form (S2 Table), and the following data were extracted from the included studies: Type of economic evaluation, health outcomes considered, study perspective, number of countries included, analytical approach to the economic evaluation used and challenges faced. Results The database searches yielded 2667 articles. After accounting for duplicates, 997 were excluded. Inspecting the titles and abstracts of the remaining papers yielded 114 potentially relevant articles, of which 62 were classified as economic evaluations that reported an ICER or incremental net benefit. Of these, 39 met the inclusion criteria and the remaining 23 were excluded mainly because they were model-based (20 studies) or not relevant (3 studies). An additional 5 studies were identified through cross referencing. Forty-four studies were included in the final sample (Fig 1). Summary of selected studies The types of economic evaluations were mainly cost-effectiveness analysis (31 studies) and cost-utility analysis (18 studies) (Table 1). Of these, 5 studies conducted both [25–29]. In one study, cost-utility analysis was performed as secondary analysis but an ICER was not estimated [25]. Sixteen studies were related to cardiovascular disease, representing a substantial proportion of the included papers. A total of 21 trials were placebo controlled (Table 2), with a common characteristic be