Using the Bayesian Improved Surname Geocoding Method (BISG) to Create a Working Classification of Race and Ethnicity in a Diverse Managed Care Population: A Validation Study

Racial and ethnic differences and disparities in health and health care are issues of growing concern that need to be properly identified and classified to be adequately studied and addressed. Yet health plans often do not readily have race/ethnicity information at their disposal for a large proportion of their members (High-Value Health Care Project 2010; Weissman and Hasnain-Wynia 2011). This may be due to limitations, such as infrequent contact with members, added costs of developing the systems to collect such information, perceptions of regulatory prohibitions against collecting race/ethnicity, and discomfort or fear among members in providing it (America’s Health Insurance Plans and Robert Wood Johnson Foundation’s 2004; Institute of Medicine 2009; Escarce et al. 2011; Weissman and Hasnain-Wynia 2011; Gazmararian et al. 2012). Although many health plans are beginning to collect race/ethnicity data more systematically, it is likely to take a substantial amount of time before this information is complete (Escarce et al. 2011). While self-report is considered the gold standard (Institute of Medicine 2009; Gazmararian et al. 2012), several techniques have been developed to indirectly estimate race/ethnicity where it is unavailable, which would allow for monitoring and evaluation of its role in health care and health research in the meantime (High-Value Health Care Project 2010). This approach has been encouraged by the United States (US) government’s Agency for Healthcare Research and Quality and the Institute of Medicine (Institute of Medicine 2009). One such recent method is the Bayesian Improved Surname Geocoding method (BISG) developed by Rand Corporation, which has been demonstrated to improve upon previous techniques (Elliott et al. 2009). The algorithm, described in detail by Elliott et al. (2008, 2009), utilizes a Bayesian approach to combine racial/ethnic data from last names and geographic units. While the use of indirect race/ethnicity estimations can enhance research efforts, an assessment of the quality of these data in a variety of contexts is needed before implementation within each setting. Some validation studies comparing the BISG algorithm estimates to self-reported race/ethnicity have been performed by its developers (Elliott et al. 2008, 2009; Elliott 2009; High-Value Health Care Project 2010) as well as by health plans beginning to utilize it. However, these studies have included limited additional stratification by pertinent personal characteristics; in particular, analysis of the BISG algorithm’s performance by age is lacking. Moreover, while the intended use of the BISG algorithm is to directly apply the predicted probabilities and not to classify individuals into a single race/ethnicity, it is important to understand how the algorithm performs for individual assignment of race/ethnicity in instances where the use of probability values is not feasible. For example, when attempting to identify individuals of a certain race/ethnicity for participation in a targeted program or research study, it will be necessary to decide an appropriate probability level that would warrant contacting individuals for recruitment. Yet analyses to determine optimal cutoffs on which to base such determinations have not been reported. In this study, we therefore assessed the validity of BISG algorithm estimations for the “off-label” use of assigning race/ethnicity within a regional managed care organization (MCO) population and examined how well the algorithm performed by gender and age.