National release of the nursing home quality report cards: implications of statistical methodology for risk adjustment.

The quality of long-term care received by nursing home (NH) residents remains a persistent concern for consumers, their families and policy makers (Vladeck 1980; Institute of Medicine 1986; Capitman and Bishop 2004). Since the 1987 Nursing Home Reform Act, continued efforts have been made to establish a national system for assessing, monitoring, and publicly reporting NH quality (Morris et al. 1990; Zimmerman et al. 1995; General Accounting Office 2002; Mor 2004). In November 2002, as part of its Nursing Home Quality Initiative, the Centers for Medicare and Medicaid Services (CMS) launched a national report card with NH quality measures (QMs), the “Nursing Home Compare” website, that publishes and regularly updates a set of key outcome-based measures derived from the Minimum Data Set (MDS) (General Accounting Office 2002; Arling et al. 2007; Mukamel et al. 2008). Making the facility performance data available to the public is expected to empower consumers to compare and choose NH services based on quality, and to stimulate quality improvement through market competition. Given its potential impact (Chernew and Scanlon 1998; Mukamel et al. 2004, 2007), it is critical that the QMs accurately differentiate homes with good quality from those with poor quality. Because health outcomes are determined by both care quality and resident frailties and comorbid conditions, it is imperative to adjust for case mix variations among facilities before their outcomes are compared (Iezzoni 2003). Failure to do so may introduce a bias where facilities treating the sickest residents may have worse outcomes even when they provide the best of care. Many quality report cards for hospitals and physicians recognize this issue and provide risk-adjusted outcome rates. However, several studies have noted that the online NH QMs take only minimal steps to adjust for resident characteristics (General Accounting Office 2002; Arling et al. 2007; Mukamel et al. 2008), and may not sufficiently “level the playing field” for NH comparisons. These studies have advocated using more extensive, statistical risk adjustment in these QMs. Despite the essential role of risk adjustment in making fairer outcome comparisons, however, risk adjustment may introduce an uncertainty (Iezzoni 1997) when alternative statistical methodologies do not agree on the identity of high-and low-quality providers (DeLong et al. 1997; Hannan et al. 1997; Iezzoni 1997; Shahian et al. 2001; Glance et al. 2006a; Li et al. 2007). A growing literature on this issue has focused on the use of appropriate severity measures for risk adjustment (Hannan et al. 1997; Iezzoni 1997; Shahian et al. 2001). More recently, analysts also examined the choice among statistical models, such as logistic or multilevel (random-effects) regression models, in computing and comparing risk-adjusted rates. Their findings suggest that alternative statistical methods may estimate outcomes differently (DeLong et al. 1997; Shahian et al. 2001; Glance et al. 2006a; Li et al. 2007). This study was designed to explore the implications of alternative statistical methods—the classical, fixed-effects, and random-effects logistic models—in constructing and interpreting the national NH QMs. Focusing on 1 of the 19 outcomes currently published (Mukamel et al. 2008), we first developed extensively risk-adjusted measures using a common set of MDS risk factors but different modeling approaches. We then compared the current CMS QM (unadjusted) and these risk-adjusted measures in identifying outstanding or poor-performing facilities. The outcome examined was decline in activities of daily living (ADLs) for long-term care residents. We chose this outcome because physical function (as measured by ADLs) is central to the well-being of NH residents (Institute of Medicine 1986). Furthermore, it has been shown to be amenable to appropriate interventions (Granger et al. 1990; Spector and Takada 1991; Kane et al. 1996) and been used in various studies of NH quality (Mukamel 1997; Mukamel and Brower 1998; Rosen et al. 2000, 2001).