Examining multiple sources of differential item functioning on the Clinician & Group CAHPS® survey.

Valid and reliable patient-reported ambulatory care experience measures are central to a balanced portfolio of quality measures and indispensable to the goal of a patient-centered health care system (Cleary 1999; Safran 2003; Browne et al. 2010). The Clinician & Group Consumer Assessment of Healthcare Providers and Systems (CG-CAHPS®) measures have been increasingly used in the United States in high stakes public reporting and pay-for-performance initiatives (Safran et al. 2006; Damberg et al. 2009; Rodriguez et al. 2009a, c). Consequently, the equivalent performance of patient experience measures across diverse racial and ethnic subgroups and respondents with varying levels of educational attainment is necessary so that equitable comparisons can be made (Casalino and Elster 2007; Friedberg et al. 2010). Studies consistently indicate that members of some ethnic and racial minority groups tend to report worse primary care experiences compared with whites, controlling for differences in educational attainment (Weech-Maldonado et al. 2001, 2003, 2004; Hargraves and Hadley 2003; Hunt, Gaba, and Lavizzo-Mourey 2005; Beal, Hernandez, and Doty 2009). Some of these differences may stem from systematic reporting tendencies by some groups or differences in cognitive processes used when responding to questions. For example, Asian American patients are much less likely than other respondents to report the most favorable response options in health care surveys fielded across a variety of care settings (Murray-Garcia et al. 2000; Taira et al. 2001; Weech-Maldonado et al. 2001; Haviland et al. 2003; Lurie et al. 2003; Saha and Hickam 2003; Rodriguez et al. 2008). As a result, many believe that data indicating Asian American patients as having worse care experiences compared with whites are at least partially explained by negative reporting tendencies. On the other hand, Latinos are thought to have lower expectations compared with non-Latino whites because prior health care experiences, including repeated time periods without insurance coverage (Beal, Hernandez, and Doty 2009), affect their perspectives and lower their expectations of care (Schnittker and Liang 2006). These scenarios suggest that for equivalent quality of primary care experiences, Asian Americans may report more negative experiences than non-Latino whites on average, and Latinos may report better experiences than non-Latino whites. These scenarios provide examples of a phenomenon known as differential item functioning (DIF), or item-level measurement bias. DIF occurs when people from different groups with the same level of the latent trait measured by the test, for example, quality of patient care experiences, have a different probability of giving a certain response to a questionnaire (Holland and Wainer 1993; Camilli and Shepard 1994). Previous studies assessing measurement equivalence of patient experience measures by race, ethnicity, or language have largely relied on classical test theory (Hurtado et al. 2005). Classical test theory approaches to assessing measurement equivalence are limited because they do not condition on the underlying level of satisfaction or quality of patient care experiences (Millsap and Everson 1993). Classical test theory approaches would ascribe any differences in observed scores across groups to bias. Other studies have used confirmatory factor analysis (CFA; Beattie, Nelson, and Lis 2007; Stewart et al. 2007) approaches to assess whether the factor structure of scales is invariant across groups, indicating the measurement of identical constructs (conceptual equivalence). Analyses of CAHPS measures have generally addressed the first level of invariance but have not addressed whether item loadings or thresholds were invariant across groups (Reise, Widaman, and Pugh 1993). Reise, Widaman, and Pugh caution that appropriate item loadings and thresholds (i.e., uniform and nonuniform DIF) must be attained to compare levels across groups. Different levels of measurement invariance, that is, the equivalence in the structure of the measurement and the associated item loadings and thresholds between groups, can be tested by multiple-group CFA (MG-CFA; Gregorich 2006) or item response theory (IRT)-based DIF detection approaches. IRT describes the relationship between responses to an item and the level of the underlying construct (e.g., patients’ experiences of primary care). While previous research uses IRT approaches to examine DIF on patient experience measures (Morales, Reise, and Hays 2000; Ballard et al. 2005; Bann, Iannacchione, and Sekscenski 2005), these analyses have focused on a single cause of DIF rather than the combined impact of DIF related to multiple covariates. To our knowledge, techniques for assessing multiple sources of DIF (Crane et al. 2008b) have not been applied to patients’ experiences of care. The current study assesses the extent to which observed differences between racial and ethnic groups on patient care experience measures may be attributable to group-level DIF impact (measurement bias). We implement an algorithm to assess multiple sources of DIF simultaneously. As such, the shared variance among covariates is parsed out and the final DIF estimates reflect the cumulative effects of DIF by race/ethnicity, after accounting for DIF impact by other covariates. The rationale behind DIF analyses is that no demographic characteristic should interfere with the relationship between the latent trait measured by the test (patients’ experiences of ambulatory care) and item responses; expected item responses for people with the same level of the underlying latent trait should be the same, on average, regardless of demographic characteristics. By evaluating the cumulative impact of DIF across multiple covariates, we are able to address a much more important question than whether an item displays DIF with respect to a single covariate; we are able to address whether comparisons across groups are fair despite different demographic compositions for the groups. Clarifying DIF impacts can allow for greater understanding of the phenomena underlying the previously documented racial and ethnic group disparities on patients’ experiences of ambulatory care. If significant DIF by race and ethnicity exists for the CG-CAHPS questions, the DIF estimation method can be used to account for DIF when making group comparisons on patient care experience measures. As scores that account for multiple sources of DIF represent the best unbiased estimates of patients’ experiences of care, accounting for any salient DIF effects can facilitate the equitable measurement of patients’ experiences of care across diverse patient populations. On the other hand, if DIF is negligible, then the English-language version of the instrument can be used by health care organizations to measure and compare patients’ experiences of ambulatory care for the diverse patient subgroups compared in the study.