The story of GBD 2010: a “super-human” effort
Authors: Pamela Das, Udani Samarasekera
Journal: The Lancet (2012)

What has working on the international, multi-investigator Global Burden of Disease 2010 been like? Pamela Das and Udani Samarasekera asked the researchers involved. Massive, collaborative, and challenging are just some of the words that the scientists who worked on the Global Burden of Disease Study (GBD) 2010 have used to describe the project. It is not difficult to see why. GBD 2010, involving nearly 500 researchers from more than 300 institutions in 50 countries, has not only been a large data-driven research endeavour but also a huge exercise in diplomacy, management, and leadership. The results are set to shake up health priority setting in countries and the world, providing an unbridled amount of up-to-date, comparable data on the diseases, risk factors, disabilities, and injuries facing populations. How did it all start? In 1990, physician and health economist Christopher Murray at Harvard University and medical demographer Alan Lopez at WHO embarked on the first ever attempt to measure the global burden of disease and developed the now-famous Disability Adjusted Life Year (DALY) metric that made it possible to combine estimates of mortality and morbidity burden around the world. They, along with a dedicated team of junior analysts at Harvard, who, says Lopez, “worked day and night, with comparable desperation to us, and for little money”, produced the first GBD. Since then, there have been estimates in 1999–2002 and 2004. But the latest iteration of the project, GBD 2010, has been a different beast altogether, involving a vastly greater number of collaborators, countries, and computers, as well as an expanded objective. 20 years ago, the project assessed the burden of 107 diseases and injuries and ten selected risk factors for the world and eight major regions over one calendar year. Now, thanks to advances in technology, the availability of data, and the participation of experts around the world, as well as the leadership of a core group of GBD researchers, the scope has swelled to 291 diseases and injuries in 21 regions, for 20 age groups, and an estimation of trends from 1990 to 2010. GBD 2010 also includes an assessment of 67 risk factors. Rafael Lozano, who was part of the core GBD team led by Lopez and Murray at the Institute for Health Metrics and Evaluation (IHME), University of Washington, WA, USA, provides a sense of the enormity of the latest undertaking: “One example of the size of the project can be represented with the cause of death database that we have used for this study. We spent almost 5 years building it; we have included almost 800 million deaths from 1950 to 2010, and the data come from different sources. The goal was to incorporate ‘all the available data’. I am sure that we are still missing a lot, but in my knowledge, this is the biggest database for cause of death analysis in the world.” “Running the programs to map the data to our cause list of 291 causes and correcting the bias can take days, even using a powerful cluster of more than 100 computers. The data that we have to store after the modelling process can take 3 terabytes.” Lozano estimates that the storage needed for the causes of death data was 400 times bigger than that for GBD 1990. But the study challenged man as much as it did machine. When The Lancet asked several researchers what it was like to work on GBD 2010, most described it as “exciting”, “demanding”, and even “frustrating”. Lopez, who is now an affiliate professor at IHME, and head of the School of Population Health, University of Queensland, Australia, says that GBD is “a way of life, rather than another massive research project”. 2 years ago, the study was way off schedule, there was a huge data lacunae, a need to revise analyses so that the same or comparable methods were used throughout, and to tackle the diseases, injuries, or risks where little work had been done in the first 3 years of the project. Murray, who is the director of IHME, described the completion of the effort as “super-human”. “It has literally occupied the vast majority of every day for me over the last 2 year sprint to the finish line. This intensity of effort on the part of many of the leaders of the study and a large number of other researchers over a prolonged period was exhausting. But in many ways it was also the best part; it has led to some great work and stimulated lots of innovation from across a broad group of researchers and created a genuine sense of camaraderie among those involved. The worst part has been dealing with various political aspects that are inevitable when trying to quantify in a comparable way many different problems. The results matter, so people and institutions get very engaged”, he tells The Lancet. Abraham Flaxman, another core team member at IHME, was trained as a mathematician/computer scientist, and working on GBD was his first introduction to global health. He called it a “trial by fire”. For Theo Vos from the School of Population Health, University of Queensland, the lead author of the paper on years lived with disability, working with so many disparate expert groups stood out. However, “it felt sometimes like I was herding cats”, he quips. For those leading the disease expert groups, the experience was also rewarding. George Mensah, who led the cardiovascular group, says, “It provided a unique opportunity for me to use many skills and experiences acquired at the US Centers for Disease Control and Prevention and in academic cardiology in the preceding 20 years”. Wagner Mercenes who led the oral conditions expert group says he “learnt a great deal about coordinating a large project and developed the ability to accommodate the views of a large number of experts in different diseases”. Majid Ezzati, chair in global and environmental health at the School of Public Health, Imperial College London, UK, and part of the GBD core team leading the comparative risk assessment, tells The Lancet, “Intense is probably the best single word to describe the science, atmosphere, and perhaps even interactions; as you can imagine, depending on the day that could be very good or exciting, or not so good!” Lopez says that the organisational challenge of collecting, analysing, and feeding back findings to collaborators to get further inputs into the estimation process could sometimes require several rounds until scientific consensus was reached; “and this for about 300 diseases, over 1100 sequelae of these diseases, and almost 70 risk factors! The sheer scope and magnitude of the study was a major challenge in itself”. Furthermore, Lopez adds: “We needed to convince scientists who were experts in particular diseases, injuries, and risk factors to share expertise and data with us in order that our estimates might be based on the best available evidence. That has been demanding and moderately successful.” For Harvey Whiteford, professor of psychiatry and population health at the University of Queensland, assembling the global epidemiology for mental disorders had specific problems. “Mental disorder, as an illness concept, is understood differently in different cultures and this impacts on how it is identified in epidemiological surveys. There is an under-reporting of mental disorder in some cultures due to stigma (which as always led to an underestimation of the burden of those disorders). There are significant limitations in data availability from many parts of the world and inconsistencies in the ways data have been collected.” He explains that although the group's initial systematic searches identified almost 100 000 data sources, fewer than 700 of these could be used after the team applied their inclusion criteria. Furthermore, he says, “The variability in study methods made comparability a challenge. For example, estimates from smaller community-based surveys of mental disorders were often the only data available for low-to-middle income countries. Rather than discard these data, we used the statistical platforms specially developed for GBD 2010 to utilise as much of the data as possible. However, it was certainly a challenge to undertake this work at the same time as the modelling tool, DisMod-MR, was being refined.” Ezzati says that making decisions on these data issues pushed the team both scientifically and intellectually: “As researchers, we tend to believe that more ‘data’ are better than less. I still believe so. But more data, but not all the perfect data we could wish for, means that we need to fundamentally think differently about when to stop searching for more and how to use it. This is partly an analytical issue but partly one of scientific judgment and intuition: how do you ‘feel’ that more data hunting will not turn up more or will not fundamentally change the conclusions.” But technical issues were only part of the problem. As Kenji Shibuya, member of the GBD core team and chair of the Department of Global Health Policy at the University of Tokyo Graduate School of Medicine, Japan, says: “If an issue was purely technical, it would be resolved by discussion—but the numbers could become very political and clashes of strong egos would take place—that would require fine diplomacy…[and] academia is generally not good at that.” Ezzati also notes that diplomacy was a key component of the project, especially in view of the different types of researchers involved. “The study brought out the well-known but far too frequently overlooked issue that people of different scientific traditions—clinicians, basic scientists, epidemiologists, and quantitative scientists—think and speak differently about the same problem. This can of course be a very powerful resource for bringing together different ways of looking at a problem and solving in the most comprehensive and interesting way—a true systems approach. It can also be a challenge, and at times a source of tension. Again, managing it takes patience, effort, and the ability to be able to step back and not only see, but also truly and deeply appreciate, the other perspectives and their contributions.” The intellectual and organisational challenges came at a price—because GBD 2010 took 2 years longer than planned, funding sources ran out while much work remained to be done. As Catherine Michaud, a core team member and senior research scientist at the School of Public Health, Harvard University, notes, the limited available funding “posed inherent challenges” itself “and required creativity on the part of key collaborators to raise additional funding as well as true commitment to contribute countless unpaid hours to the project”. The GBD has brought new dimensions and advancements to both health measurement and the way global health is viewed. “Better methods to better extract truth from poor quality or missing data has fostered tremendous scientific innovation”, says Lopez. “At the same time, the poor quality of data encouraged us to derive ways of estimating uncertainty around all GBD outputs and findings. This has been a major scientific achievement, but it will also greatly assist with the interpretation of our findings for policy.” He adds that “the GBD study has effectively provided a global health data ‘audit’ whereby there is now a massive repository of national health information, but also substantially more knowledge about its utility and reliability for informing health debates”. For Felix Masiye, head of the Department of Economics, University of Zambia, and coordinating member of the GBD consultation meeting in Zambia, “the idea to achieve international comparability that employs a metric that encompasses a widely shared notion of health is a unique contribution of the GBD”. Also unique to GBD is its comparative risk assessment approach, which tries to ensure that the methods used to estimate the burdens attributable to major potentially modifiable risk factors such as high blood pressure, diet, tobacco smoking, and ambient and household air pollution, are consistent among the risk factors. “This allows their respective burdens to be compared with regard to their importance globally and among global regions”, says Aaron Cohen, co-chair of the GBD 2010 expert group on ambient air pollution, and principal scientist at the Health Effects Institute, MA, USA. “There have been many assessments of the health impacts of ambient air pollution, and the number of such assessments is growing, but only GBD places its estimate in the context of other major risk factors with which policy makers and the public must contend”, he adds. Shibuya points out that “GBD has also shown to the world that new ideas and methods outside the conventional medicine and public health framework—economics, philosophy, psychology, political science, environmental science, computer science, engineering, etc—play a significant role in the estimation and interpretation of the disease burden”. An important advance has been the improvement of methodological approaches. Vos admits, “we now joke to each other how we used to get away with murder in the past doing burden [studies]. We tended to make lots of not so replicable ad-hoc decisions and few people knew what we were doing anyway. This exercise has deliberately tried to minimise the ad-hoc decisions and instead aimed to maximise what information we can get from the data. We also have much more closely involved hundreds of experts and young researchers who are very capable of looking over our shoulders and are much more articulate at picking apart the estimates. This is mostly to the good (and has led to many improvements in the methods and results)…although we have also had some experiences of disease experts taking on roles of advocates with aim to ‘boost the numbers’ rather than being ‘impassionate’ scientific advisers”. From an academic standpoint, Ezzati would like to see a broader contribution: “that ‘descriptive epidemiology’ of the sort that GBD 2010 is, but was also done by the likes of [Richard] Doll and [Richard] Peto or Sam Preston, becomes a more central part of research and training—epidemiology as describing the health of individuals and populations including, but not solely, eliciting the causes of differences in levels and trends.” Murray now thinks it is possible, despite the odds, to bring large complex analytical projects to closure. “Although not on the same scale, the GBD is an example of big science applied to global descriptive epidemiology. To do the work well, requires an enormous amount of data, huge computational machinery, tailored statistical analytical skills, and the expert input of many from around the world. The GBD demonstrates that this is actually possible to achieve.” However, while the impact of the study on science is substantial and important, the researchers on GBD 2010 want the study's real goal—to improve the health of populations—to become a reality. Previous GBD studies, for example, have underscored the burden of malaria in low-income and middle-income countries, leading to increased attention and investment in tackling the disease and subsequent reductions in burden. Lopez says: “If we could see the GBD 2010 findings being widely used to inform health policy debates, priority setting in health, and to raise awareness about the urgent need to strengthen health information systems, particularly vital registration systems in poor countries, that would be a massive achievement.” But with different methodologies and estimates of global burdens of disease available in the public domain, what should countries do when trying to work out their national health priorities? Lopez thinks that they should debate the estimates and he encourages independent inquiry. “I do not think that we should be trying necessarily to harmonise methodologies and certainly not to constrain legitimate scientific enquiry and opinions about how best burden of disease studies might be done. While this may be inconvenient for countries, it should not be viewed as undesirable or indeed surprising given the quality and limited availability of data on which the estimates are often based. But equally, countries should develop sufficient capability to independently assess the merits of different methodologies and interpretation of findings: simply because WHO has issued estimates of disease burden does not make them correct!” He says that countries would benefit enormously from applying the GBD methods to the wider dataset that is available to them on the health of their populations, but which the GBD researchers did not have access to. “This would undoubtedly improve the reliability of our estimates in some countries, perhaps not in others. In doing so, countries might also think of estimating disease burden separately for subpopulations of substantial interest for health services provision.” Flaxman concurs. “The GBD data is a great tool for thinking comprehensively about population health.” He advises countries to look for anything unexpected in the GBD data, since this reflects the “best estimates based on the newest methods and most comprehensive input dataset”. Furthermore, he notes that comparing GBD 2010 results to the previous GBD figures, and to national data published by governments, might “find important challenges to conventional wisdom. This is worth investigating, and policy makers would be wise to conduct more detailed burden studies of their own populations.” Jarbas Barbosa da Silva, who works at the Brazilian Ministry of Health and was a central member of the GBD consultation meeting in Brazil, says the information from GBD 2010 will not only be useful for countries that do not have good health information systems but also for those with strong surveillance. “From a country perspective, this type of exercise promotes comparability and encourages the use of summary measures of health. For those countries that do not have solid information systems, such an exercise will show their health conditions and support the decision-making process. For others, such as Brazil, which have a consolidated information system and a strong public health academic community, these studies are a useful contribution to improve the knowledge about the national health profile. Discussing the relationship between diseases, injuries, and their risk factors in the light of local conditions sharpens considerations of priorities, health programmes, and public policies to address them.” So what does the future hold? Murray thinks that GBD is likely to become an even more complex exercise in the years to come. “As more data are made available and/or collected, the task of tracing global health epidemiology is much more challenging than when there was little data available. This may seem paradoxical but when there are no data; the task is actually easier to generate a model than when there are conflicting or complex patterns in the data that must be captured and reflected in the results. In other words, progress in global health measurement makes the task of tracking global health problems more complex and time consuming—not less. We need to factor this challenge into future efforts to maintain and expand the GBD.” Murray thinks it will be essential to update the GBD on a regular basis as everyone needs access to health information that reflects current knowledge. So will the next GBD take another 5–10 years? No. “The good news is that the GBD 2010 methods and databases provide a platform so that future regular updating will be feasible”, says Murray. And, now, with the completion and publication of the fourth and largest GBD, there is little doubt about what can be done in the future. At the outset, the project seemed like “mission impossible”, admits Shibuya. “Nobody believed at the beginning that it could be done, but GBD has always achieved its mission, which is astonishing.” It has also felt worth the tremendous effort for the researchers involved. Shibuya concludes: “As is often the case with working with Chris [Murray], it is really like a roller-coaster experience—there are so many ups and downs, and joy and tears, but most importantly working on GBD makes you feel that you are part of something truly innovative, something revolutionary in the field of global health.” This online publication has been corrected. The corrected version first appeared at thelancet.com on February 22, 2013 This online publication has been corrected. The corrected version first appeared at thelancet.com on February 22, 2013 Should the GBD risk factor rankings be used to guide policy?The Global Burden of Disease Study 2010 (GBD 2010) estimates provide important new insights. Alongside estimates of health burden attributable to 291 diseases and injuries,1 Stephen Lim and colleagues2 estimate the health burden associated with 67 risk factors, organised into a hierarchy of clusters. So as to distinguish real changes in global burden and risk factors from changes in methods, they not only estimated the burden and risk factor ranking for 2010, but also recalculated estimates for 1990. Full-Text PDF Data for better health—and to help end povertyThe World Bank Group welcomes the publication of the new Global Burden of Disease Study (GBD). The Bank commissioned the first GBD in 1990, and continues to make extensive use of this signal contribution to global health. Like its predecessors, the new, methodologically updated GBD 2010 marks a milestone in global health knowledge and our capacity for evidence-based action. It will once again set the terms of health policy, planning, and funding discussions for years to come. Full-Text PDF GBD 2010: understanding disease, injury, and riskPublication of the Global Burden of Disease Study 2010 (GBD 2010) is a landmark event for this journal and, we hope, for health. The collaboration of 486 scientists from 302 institutions in 50 countries has produced an important contribution to our understanding of present and future health priorities for countries and the global community. Full-Text PDF A promise to save 100 000 trauma patientsIn The Lancet, Christopher Murray and colleagues present the findings of their 2010 Global Burden of Disease Study, in which they show that injuries cost the global population some 300 million years of healthy life every year, causing 11% of disability-adjusted life years (DALYs) worldwide.1 Road-traffic crashes were the number one killer of young people and accounted for nearly a third of the world injury burden—a total of 76 million DALYs in 2010, up from 57 million in 1990. Most of the victims were young, and many had families that depended on them. Full-Text PDF AIDS is not overOptimism and momentum has been building around the real possibility that an AIDS-free generation is imminent. Public enthusiasm is fuelled by news about the rapid scale-up of antiretroviral therapy, evidence that HIV treatment can prevent new infections, and expanded coverage of programmes to prevent mother-to-child transmission of HIV. Yet, the most recent estimates of HIV prevalence and incidence and of AIDS-related mortality released by UNAIDS1, together with data from the Global Burden of Disease Study 2010 in The Lancet,2,3 make it clear that AIDS is not over. Full-Text PDF From new estimates to better dataThe Global Burden of Disease Study 2010 (GBD 2010) in The Lancet represents an unprecedented effort to improve global and regional estimates of levels and trends in the burden of disease. Accurate assessment of the global, regional, and country health situation and trends is critical for evidence-based decision making for public health. WHO therefore warmly welcomes GBD 2010, which was undertaken by the Institute for Health Metrics and Evaluation (IHME) with its partners and draws on the contributions of many scientists, including those who work in WHO programmes. Full-Text PDF GBD 2010: a multi-investigator collaboration for global comparative descriptive epidemiologyThe data, methods, and findings of the Global Burden of Disease Study 2010 (GBD 2010) are described in detail in The Lancet. This large collaboration is an evolution of a body of work that began with GBD 1990.1 The number of diseases, injuries, and risk factors evaluated and the geographical units of analysis have greatly expanded in the past 20 years, and change over time has been assessed. Nevertheless, GBD 2010 follows the basic principles of GBD 1990: trying to use all the relevant published and unpublished evidence; capturing fatal and non-fatal health outcomes with comparable metrics; and separating epidemiological assessment from advocacy concerns or entanglement of agendas. Full-Text PDF GBD 2010: design, definitions, and metricsThe Global Burden of Diseases, Injuries, and Risk Factors (GBD) enterprise is a systematic, scientific effort to quantify the comparative magnitude of health loss due to diseases, injuries, and risk factors by age, sex, and geography for specific points in time. The GBD construct of the burden of disease is health loss, not income or productivity loss.1 For decision makers, health-sector leaders, researchers, and informed citizens, the GBD approach provides an opportunity to see the big picture, to compare diseases, injuries, and risk factors, and to understand in a given place, time, and age-sex group what are the most important contributors to health loss. Full-Text PDF Department of ErrorDas P, Samarasekera U. The story of GBD 2010: a “super-human” effort. Lancet 2012; 380: 2067–70—In this Special Report (Dec 15/22/29), Aaron Cohen should have also been listed as principal scientist at the Health Effects Institute, MA, USA. This correction has been made to the online version as of Feb 22, 2013. Full-Text PDF