Prevalence of behavioral symptoms: comparison of the minimum data set assessments with research instruments.

Behavioral disturbances associated with dementia are prevalent and persistent clinical phenomena in long-term care (LTC) facilities.1 They contribute to greater cognitive and functional decline, significant suffering, poorer quality of life for the residents and their professional caregivers, and a reciprocal cycle of resident-staff abuse, as well as increased staff turnover.2–5 The assessment and monitoring of behavioral disturbances in LTC facilities is mandated by the Centers for Medicare and Medicaid Services (CMS) under the statutory authority of the Omnibus Budget Reconciliation Act of 1987 (OBRA 1987).6 The Minimum Data Set (MDS), a component of the federally mandated Resident Assessment Instrument (RAI), contains a core set of screening, clinical, and functional status elements that form the foundation of the comprehensive assessment for all residents of LTC facilities licensed to participate in Medicaid.6 Assessments are required on admission, quarterly, annually, at discharge, and whenever there is a significant change in the resident’s status. Since its implementation in nursing homes in 1991, the role of the MDS has expanded beyond scheduled assessments to identify care problems and tailor individualized care plans. It now also functions as a case-mix–based prospective reimbursement mechanism for Medicare and Medicaid, and as an indicator of the quality of care provided by LTC institutions.7 Thus, the reliability and validity of the MDS as an assessment instrument are of paramount importance. Section E4 (“Behavioral Symptoms”) of the MDS describes as its intent: “the assessment of the frequency, and alterability of behavioral symptoms in the last seven days that cause distress to the resident, or are distressing or disruptive to facility residents or staff members, even if staff and other residents appear to have adjusted to them.”6 The coding for the 5 specific items comprising Section E4 focuses on the resident’s actions rather than his or her intent. Sources of information may include a review of the resident’s medical record, direct observation of the resident, and communication with the resident, family and all direct-care staff. In reality, registered nurses, who are the legislatively designated staff members responsible for coordinating the MDS assessment, often have little to no direct contact with the residents or their caregivers, and rely almost exclusively on documentation in the resident’s medical record to complete the assessment. Thus, the validity of the behavioral data captured by the MDS are suspect in light of the fact that the medical record documentation is largely provided by direct-care staff who are overburdened and have limited training in the recognition and documentation of such behaviors.8 The validity of MDS Section E items specifically has been examined in at least 4 reports by comparing the MDS data with those collected via previously validated instruments in samples with varying levels of cognitive impairment.9–12 Investigators have reported correlation coefficients of r = 0.24 (day shift) to 0.37 (evening shift) with the total scores on the Cohen-Mansfield Agitation Inventory (CMAI), r = 0.54 with the Psychogeriatric Dependency Rating Scale (PGDRS), and r = 0.50 with the Alzheimer’s Disease Patient Registry Behavior Checklist as the comparative instruments. An additional report has documented significantly lower prevalence of behaviors as measured by the MDS compared with certified nursing assistants’ ratings on the Revised Memory and Behavior Problem Checklist (RMBPC). Moreover, the reliability of the MDS behavioral and depression items decreased with worsening cognitive impairment.12 Despite the modest to moderately strong correlations in these prior studies, however, the results must be understood within the limitations of their approach. These include (1) comparison of global MDS behavioral symptoms score with total scores on the comparative instruments, thus precluding analysis of correlation between the specific items (eg, verbally versus physically abusive behavior), (2) lack of clarity in some reports as to who completed the MDS and/or research instruments, and (3) significant time lags of 1 to 6 months between the completion of the MDS and the research instruments in most of the studies. Only Horgas and Margrett12 completed all assessments within a 1-month timeframe. Unlike the present investigation, however, Horgas and Margrett’s12 cohort examined items from both the mood and behavioral symptoms subscales of the MDS, and focused the data analysis only on global behavioral symptoms scores. In addition to the limitations of these investigator-initiated studies, federal agencies that have examined the reliability and validity of the MDS have relied on research nurses as assessors rather than the routine clinical staff, thus limiting the external validity, ie, generalizability, to actual clinical practice.13 As part of an ongoing direct observational study of the behavioral and social ecology of dementia units (CareMedia: Automated Video and Sensor Analysis for Geriatric Care, NSF IIS-0205219), we had the opportunity to compare consenting residents’ MDS behavioral symptoms data with specific corresponding items from the Ryden Aggression Scale Version 2 (RAS2) and the Cohen-Mansfield Agitation Inventory (CMAI).14 To address limitations of some of the prior investigations, our MDS data were either collected during the study period or within 2 weeks before or after the study period by the facility’s MDS coordinator. The primary certified nursing assistant responsible for each resident was the direct source of information for the RAS2 and CMAI (ie, the assessors were routine staff). Our primary aim was to determine the overall and domain-specific prevalence of behavioral symptoms in LTC residents with severe dementia based on the MDS versus a validated measure designed to optimally measure each domain. The specific hypotheses were (1) the MDS will significantly underestimate the total prevalence of behavioral disturbances, (2) MDS will significantly underestimate the prevalence of verbally and physically abusive behaviors as compared with corresponding items from the RAS2 (Section E4 items b and c), and (3) the MDS will significantly underestimate socially inappropriate/disruptive and care resistive behaviors compared with corresponding items from the CMAI (Section E4 items d and e).