Geography, generalisability, and susceptibility in clinical trials
Authors: Jane E Clougherty, Ellen J Kinnee, Juan Carlos Cardet, David Mauger, Leonard Bacharier, Avraham Beigelman, Kathryn V Blake, Michael D Cabana, Mario Castro, James F Chmiel, Ronina Covar, Anne Fitzpatrick, Jonathan M Gaffin, Deborah Gentile, Elliot Israel, Daniel J Jackson, Monica Kraft, Jerry A Krishnan, Harsha Vardhan Kumar, Jason E Lang, Stephen C Lazarus, Robert F Lemanske, John Lima, Fernando D Martinez, Wayne Morgan, James Moy, Ross Myers, Edward T Naureckas, Victor E Ortega, Stephen P Peters, Wanda Phipatanakul, Jacqueline A Pongracic, Kristie Ross, William J Sheehan, Lewis J Smith, Julian Solway, Christine A Sorkness, Michael E Wechsler, Sally Wenzel, Steven R White, Fernando Holguin
Journal: The Lancet Respiratory Medicine (2021)

Randomised clinical trials (RCTs) are generally considered the highest standard of evidence in medical research, as randomised treatment allocation promotes homogeneity in baseline characteristics between treatment groups, maximising internal validity and reducing both bias and confounding. RCTs, however, often enrol a convenience clinical sample, and can face challenges of external validity if that sample does not represent the full population at risk, or the full range of co-exposures and susceptibility factors likely to be encountered in clinical practice.1Rothwell PM External validity of randomised controlled trials: "to whom do the results of this trial apply?".Lancet. 2005; 365: 82-93Summary Full Text Full Text PDF PubMed Scopus (1889) Google Scholar, 2Akobeng AK Assessing the validity of clinical trials.J Pediatr Gastroenterol Nutr. 2008; 47: 277-282Crossref PubMed Scopus (60) Google Scholar, 3Kanarek NF Tsai HL Metzger-Gaud S et al.Geographic proximity and racial disparities in cancer clinical trial participation.J Natl Compr Canc Netw. 2010; 8: 1343-1351Crossref PubMed Scopus (25) Google Scholar Many such biases can be geographical in nature; for example, proximity to clinical sites can influence recruitment and retention,4Su SC Kanarek N Fox MG Guseynova A Crow S Piantadosi S Spatial analyses identify the geographic source of patients at a National Cancer Institute Comprehensive Cancer Center.Clin Cancer Res. 2010; 16: 1065-1072Crossref PubMed Scopus (17) Google Scholar which is important because neighbourhoods differ in socioeconomic status and environmental exposures (ie, air pollution), both shown to affect respiratory health5Richardson DB Volkow ND Kwan MP Kaplan RM Goodchild MF Croyle RT Medicine. Spatial turn in health research.Science. 2013; 339: 1390-1392Crossref PubMed Scopus (176) Google Scholar and therefore potentially influencing observed treatment efficacy. In this moment, when clinical trials for COVID-19 vaccines are being run with unprecedented expediency to mitigate a virus that has disproportionately impacted minority populations and those with lower socioeconomic status,6Chastain DB Osae SP Henao-Martínez AF Franco-Paredes C Chastain JS Young HN Racial disproportionality in Covid clinical trials.N Engl J Med. 2020; 383: e59Crossref PubMed Scopus (110) Google Scholar thoughtful attention to representativeness, generalisability, and spatial co-exposures in RCT populations is of paramount importance. Despite this discrepancy in attention to internal versus external validity, clinical guidelines prioritise RCT results in making treatment recommendations, even when available RCT data might represent a very different population. RCTs have not traditionally recruited cohorts that are unbiased representations of the population at risk, nor reported adequate information on cohort characteristics, including demographics and co-exposures, to support thorough assessment of a trial's applicability to another population.7Frampton GK Shepherd J Dorne JL Demographic data in asthma clinical trials: a systematic review with implications for generalizing trial findings and tackling health disparities.Soc Sci Med. 2009; 69: 1147-1154Crossref PubMed Scopus (22) Google Scholar External validity can be further limited by factors influencing an individual's decision to participate; some evidence suggests that asthma RCTs have been disproportionately comprised of individuals with lower socioeconomic status who lack access to high-quality medical care;8Galobardes B Granell R Sterne J et al.Childhood wheezing, asthma, allergy, atopy, and lung function: different socioeconomic patterns for different phenotypes.Am J Epidemiol. 2015; 182: 763-774Crossref PubMed Scopus (46) Google Scholar other evidence suggests lesser access to clinical trials for rural communities and those with lower socioeconomic status.9Seidler E KA Brown C Wood E Granick L Kimball AB Geographic distribution of clinical trials may lead to inequities in access.Clin Invest. 2014; 4: 373-380Crossref Google Scholar Clinical trials have been criticised for these potential challenges to external validity, and some improvements have been made: best practices for pragmatic RCTs have been developed,10Gamerman V Cai T Elsäßer A Pragmatic randomized clinical trials: best practices and statistical guidance.Health Serv Outcomes Res Methodol. 2018; 19: 23-35Crossref Scopus (55) Google Scholar including more complex randomisation strategies to minimise bias, and a larger number of RCTs are now reporting more thorough information on patient selection, eligibility, and enrolment—although it is still only a minority of RCTs that fully comply with these standards.7Frampton GK Shepherd J Dorne JL Demographic data in asthma clinical trials: a systematic review with implications for generalizing trial findings and tackling health disparities.Soc Sci Med. 2009; 69: 1147-1154Crossref PubMed Scopus (22) Google Scholar, 11Ma IW Khan NA Kang A Zalunardo N Palepu A Systematic review identified suboptimal reporting and use of race/ethnicity in general medical journals.J Clin Epidemiol. 2007; 60: 572-578Summary Full Text Full Text PDF PubMed Scopus (69) Google Scholar, 12Brahan D Bauchner H Changes in reporting of race/ethnicity, socioeconomic status, gender, and age over 10 years.Pediatrics. 2005; 115: e163-e166Crossref PubMed Scopus (35) Google Scholar Socioeconomic status both directly and indirectly influences health and treatment outcomes through a complex array of social, environmental, and medical factors, including health-care access. The greater severity of asthma among children of lower socioeconomic status in the USA is well established, and clinical outcomes vary substantially by socioeconomic status, in part because participants with lower socioeconomic status often reside in areas with greater pollution, chronic stressors (eg, violence), poorer-quality housing, or fewer healthy dietary options. Given the limited range of clinical data to capture these complex social and environmental co-exposures in an RCT population, however, it is challenging to determine how each factor might influence observed treatment efficacy or a given trial's generalisability. While several observational studies and RCTs have focused on inner-city populations (eg, the Inner-City Asthma study),13Gruchalla RS Pongracic J Plaut M et al.Inner City Asthma Study: relationships among sensitivity, allergen exposure, and asthma morbidity.J Allergy Clin Immunol. 2005; 115: 478-485Summary Full Text Full Text PDF PubMed Scopus (373) Google Scholar none of them, to our knowledge, have examined whether treatment efficacy differs for participants in relatively high-pollution versus low-pollution areas within the urban setting (eg, closer vs further from a major roadway), despite well established associations between air pollution and asthma. Even very well conducted major RCTs, including large National Institutes of Health (NIH)-sponsored clinical trials, have not yet explicitly incorporated social and environmental factors into study design and implementation.7Frampton GK Shepherd J Dorne JL Demographic data in asthma clinical trials: a systematic review with implications for generalizing trial findings and tackling health disparities.Soc Sci Med. 2009; 69: 1147-1154Crossref PubMed Scopus (22) Google Scholar To assess representativeness in socioeconomic status and environmental exposures among participants in some exemplar asthma RCTs, we developed geographic information system (GIS)-based metrics to characterise the residential census tracts of 874 adults and children in RCTs run by AsthmaNet, an NIH-sponsored clinical research network. Data were extracted from baseline questionnaires of three trials—Best African-American Response to Asthma Drugs (BARD),14Wechsler ME Szefler SJ Ortega VE et al.Step-up therapy in black children and adults with poorly controlled asthma.N Engl J Med. 2019; 381: 1227-1239Crossref PubMed Scopus (41) Google Scholar Steroids In Eosinophil-Negative Asthma (SIENA),15Lazarus SC Krishnan JA King TS et al.Mometasone or tiotropium in mild asthma with a low sputum eosinophil level.N Engl J Med. 2019; 380: 2009-2019Crossref PubMed Scopus (93) Google Scholar and Step-up Yellow Zone Inhaled Corticosteroids to Prevent Exacerbations (STICS)16Jackson DJ Bacharier LB Mauger DT et al.Quintupling inhaled glucocorticoids to prevent childhood asthma exacerbations.N Engl J Med. 2018; 378: 891-901Crossref PubMed Scopus (109) Google Scholar—conducted using the same protocols in 17 cities distributed across the USA. We geocoded participant residences and linked these in GIS to census tracts and national roadmaps (using StreetMap Premium for ArcGIS 2016) to create commonly used indicators of tract-level socioeconomic status, including median household income and percentage of population living below the US federal poverty level. As an indicator of near-roadway pollution exposure, we calculated weighted roadway density17Rose N Cowie C Gillett R Marks GB Weighted road density: a simple way of assigning traffic-related air pollution exposure.Atmos Environ. 2009; 43: 5009-5014Crossref Scopus (65) Google Scholar within multiple distances of each home. A majority of participants (71·4% for BARD, 54·5% for SIENA, and 55·3% for STICS) lived in tracts with median household incomes below the 2016 US average (US$59 039), and with a greater-than-average percentage of residents living in poverty (figure). After merging baseline data across all three trials, greater roadway density near the home and tract-level poverty were separately associated with lower baseline lung function (percentage predicted FEV1), after adjusting for age, sex, race, and ethnicity. These results reveal that participants in these three multicentre RCTs disproportionately lived in areas of lower socioeconomic status (which have higher air pollution exposures than other areas, on average, in the USA),20Clark LP Millet DB Marshall JD National patterns in environmental injustice and inequality: outdoor NO2 air pollution in the United States.PLoS One. 2014; 9e94431Crossref PubMed Scopus (304) Google Scholar and that greater roadway densities (a proxy for traffic-related pollution) conferred lower baseline lung function. Such social or environmental co-exposures should be considered in clinical trials broadly, where possible, as they might plausibly alter observed treatment efficacy and, if extreme, could affect generalisability. Further research is needed to determine whether and how spatially-distributed co-exposures influence treatment response during study interventions. Advances in spatial analysis and GIS have driven a rapid increase in the use of geographical analysis in epidemiology internationally.5Richardson DB Volkow ND Kwan MP Kaplan RM Goodchild MF Croyle RT Medicine. Spatial turn in health research.Science. 2013; 339: 1390-1392Crossref PubMed Scopus (176) Google Scholar We propose that GIS can also be a powerful tool to refine the interpretability and applicability of RCT data—both in better defining the generalisability of any given RCT, and in more clearly identifying subpopulations for whom a given intervention might be most beneficial. Characterising the geographical context of RCT cohorts, ideally at the outset of any trial, can help to identify potentially influential social or environmental co-exposures (eg, living in high-pollution areas, or in sub-standard housing), and could inform on spatial patterning and clustering in recruitment and retention. In resource-limited settings (eg, lower-income and middle-income countries), geographical analysis could help to target RCT recruitment more cost-effectively, by more precisely matching participant characteristics and spatial co-exposures to those of the intended treatment population. Ultimately, using spatial analysis and GIS to better understand the lived context of RCT participants, thus better accounting for socioeconomic and environmental co-exposures, can help to improve the interpretability of RCT results and to better identify subpopulations for whom a given intervention might be particularly effective, and will inform on the true generalisability of a given RCT's results, all with the aim of improving patient care. This work was funded by the NIH National Heart, Lung, and Blood Institute (NHLBI) (1R01HL114536–01: Validating GIS-based methods to address spatial uncertainty in clinical trials, to JEC and FH). The AsthmaNet trials were funded by NIH grants HL098302, HL098096, HL098075, HL098090, HL098177, HL098098, HL098107, HL098112, 098103, and HL098115. All competing interests listed here are during the conduct of the study; competing interests outside of the submitted work are listed in the appendix. JEC, EJK, RC, SPP, LB, KVB, DJJ, FDM, WM, JAP, JS, and SRW report grants from the NIH NHLBI. DM and JFC report grants from the NIH. AB reports grants from Washington University and the NIH. MDC is a member of the United States Preventive Services Task Force (USPSTF); this manuscript does not necessarily represent the views of the USPSTF. JAK reports grants from the NIH, NHBLI, and PCORI. WP reports grants, personal fees, and other support from Genentech/Novartis and Regeneron/Sanofi, other support from AstraZeneca, GlaxoSmithKline, Boehringer Ingelheim, and Merck. Additionally, full details for competing interests outside of the submitted work for DM, LB, JFC, DJJ, FDM, WM, JAP, JS, and SRW are listed in the appendix. JCC, MC, EI, MK, SCL, RFL, VEO, MEW, SW declare no competing interests related to the submitted work, with full details of other competing interests listed in the appendix. AF, JMG, DG, HVK, JEL, LJS, JL, JM, RM, ETN, KR, WJS, CAS, and FH declare no competing interests. Download .pdf (.11 MB) Help with pdf files Supplementary appendix