District of Columbia: Status of Reforms to the District's Mental Health System
Released: 2004-04-30T00:00:00

Letter

Background

In 1974, a class action suit filed in the U.S. District Court for the District of Columbia on behalf of individuals with mental illnesses alleged that the practice of treating the District’s mental health patients in an institutional setting violated the statutory rights of individuals. Specifically, the plaintiffs asserted that patients at St. Elizabeths Hospital had a statutory right to appropriate care in alternative care facilities when less restrictive settings were clinically appropriate. In a ruling known as the Dixon Decree, the court ruled in favor of the plaintiffs in 1975, ordered the District to build a system to facilitate the provision of community-based treatment for these individuals, and continued oversight of the District’s progress in developing this system. In 1997, finding that the District was no closer to complying with the Dixon Decree than it had been 22 years earlier, the court placed the D.C. Commission on Mental Health Services in receivership and appointed a receiver to implement the transition to a community-based mental health system. This receiver introduced initiatives that sought to change the way the District delivered services, but implementation was slow and the first receiver made little progress in implementing these initiatives during his 2-year oversight of the commission. Thus, a second or “transitional” receiver was appointed on April 1, 2000, to facilitate the transition from court receivership to District control. (App. I summarizes the major court actions related to the Dixon Decree.)

The transitional receiver was charged with developing a comprehensive plan for the District to achieve compliance with the Dixon Decree and resume full control of its mental health system. The court approved a final plan in April 2001 and required the District to implement it; however, before the receivership could be ended, the court required the transitional receiver to certify that the District had the capacity to implement—and was implementing—the final plan. Although the court originally anticipated this certification in late 2001, in December 2001 the transitional receiver recommended extending the date, characterizing the implementation delay as largely unavoidable because of (1) additional time needed for recently hired senior DMH managers to begin major initiatives, and (2) the unexpected need for crisis services to respond to September 11, 2001, terrorist events. Following this extension, the transitional receiver reported to the court that the District had made sufficient progress and, as a result, the court terminated the receivership and appointed the former transitional receiver as a court monitor to oversee the District’s continued implementation of the final plan in May 2002. (See table 1.)

District Mental Health System Prior to the Approval of the Final Plan

When the transitional receiver was responsible for overseeing the District’s mental health system, the District was the largest provider of mental health services to its residents, treating approximately 10,000 consumers annually and employing close to 2,000 staff in fiscal year 2000. The focal point of the mental health system was St. Elizabeths Hospital, which was the major point of entry for all consumers in the system. St. Elizabeths Hospital provided a wide range of mental health services in an acute care setting, including more than 600 beds divided among two types of inpatient consumers, forensic and civil, for adults and children and youth. The District also directly provided services through outpatient facilities in the community, including two community mental health centers and five mobile community outreach treatment teams.

In addition to providing inpatient and direct services in the community, the District contracted with private community providers for housing, employment, case management, and other community-based services. In its contracts with private community providers, the District often used a “slot” system to allocate a defined number of consumers to providers and paid them a fixed daily rate per consumer. Under this system, providers did not compete to attract consumers and were paid regardless of performance, consumer satisfaction, or the actual delivery of service.

The District and its providers focused primarily on treating the medical symptoms of the consumer without focusing as much on whether the individual was participating in his or her recovery from mental illness and successfully living in the community. Furthermore, the system did not have many safeguards in place, such as uniform provider standards, to involve the consumer in key aspects of service delivery, such as choosing a provider and developing a treatment plan based on the consumer’s goals. The transitional receiver identified the need for a restructured mental health system that had the flexibility to meet individual needs and allow consumers to successfully obtain treatment and live in the community, maximizing principles of accessibility, recovery, and consumer choice.

In 2001, the federal share of Medicaid, an entitlement program in which states and the federal government are obligated to pay for covered services provided to an eligible individual, accounted for 8 percent of District mental health system revenue as compared to the national average of 22 percent. The transitional receiver identified the need to better utilize Medicaid as a major funding source. The District’s access to Medicaid funds had been limited because Medicaid did not cover most of the services provided at St. Elizabeths Hospital, considered under the Medicaid statute as a larger psychiatric institution. This effect was exacerbated by the limited capacity in the developing community-based system to support inpatients ready for discharge. For example, in October 2000, District officials estimated that approximately 60 percent of individuals in acute care units at St. Elizabeths Hospital could be moved into the community where outpatient services covered by Medicaid would be available, if stable alternative housing were available.

A second limit to the District’s accessing federal funds was that the District had not taken advantage of optional community-based mental health services that could be reimbursed through the Medicaid program. The transitional receiver required the District to implement a strategy adopted by at least 40 other states to expand the services reimbursable by Medicaid through an option to cover rehabilitative services, thus expanding the scope of eligible services and providers beyond that of the program’s traditional focus on services delivered by physicians and psychiatrists who work at hospitals, clinics, and other facilities. Rehabilitative services include crisis and emergency care, medication treatment, and community-based interventions. The variety of rehabilitative treatments and services covered by this Medicaid option is intended to facilitate a consumer’s recovery from mental illness, including restoring a consumer to his or her best possible functional level.

Final Plan and Exit Criteria

The court-approved final plan broadly outlines the mental health system’s direction, philosophy, major roles, and governance. It represents a major shift in the District’s mental health system on several fronts, including the system’s structure and organization, method for enrolling consumers and paying providers, and involvement of consumers in their plan for recovery. For example, the final plan identifies the need to create a new mental health department with the additional responsibility of oversight along with continuing the District’s historic role as provider; envisions a significant change in enrollment and billing systems, such as linking payment to the delivery of services, and developing new funding strategies that increase federal reimbursement; and articulates that the new system have a built-in capacity to measure itself in key performance areas and to translate any findings into continued system improvements.

Underpinning these structural changes is a refocusing of the mission of the District’s mental health system toward involving the consumer in treatment decisions and incorporating changes that facilitate the consumer’s recovery from mental illness and away from focusing primarily on treating the individual’s medical symptoms.

The court also approved exit criteria for the Dixon lawsuit, which provide a basis for measuring the performance of the District’s mental health system and which must be met in order to end the Dixon case. The criteria cover four areas: 1. consumer satisfaction, which assesses consumers’ satisfaction with mental health services provided; 2. consumer functioning, which tracks consumers’ clinical, social, and other conditions upon entry into the mental health system and again after receiving services for a specified period of time; 3. consumer service delivery, which assesses the adequacy of the mental health system’s overall performance for consumers in a range of areas including treatment planning, coordination of care, and response to emergent and urgent needs; and 4. system performance, which demonstrates how well the community- based system of care is serving particular populations.

The first two areas require DMH to develop and implement methods for reviewing and measuring consumer satisfaction and consumer functioning and to use the data to refine the system. To fulfill the remaining criteria, DMH is required to meet 17 performance targets, many of which measure activities identified as national best practices in the field of mental health.

According to the court monitor, implementing the final plan, including developing the ability to measure DMH’s progress against the exit criteria, will take 3 to 5 years, with year 1 beginning July 1, 2001. In general, efforts for years 1 and 2 were expected to center on planning, laying the basic infrastructure for the system, and beginning to provide community- based services. By the end of year 3, which began October 1, 2003, DMH is expected to be stabilizing and improving performance within the system, and in years 4 and 5 DMH is expected to be actively measuring performance outcomes. (See table 2.) In addition to developing performance targets for the exit criteria, the court monitor is required to provide the court with semiannual reports on the District’s progress in meeting all of the exit criteria. The court monitor’s first two reports, submitted to the court in January 2003 and July 2003, respectively, focused primarily on DMH’s status in implementing the final plan and also included an update on the status of meeting the exit criteria to end the Dixon case.

DMH Has Assumed Oversight Authority and Responsibility for Providing Direct Care

In accord with the transitional receiver’s final plan, the District restructured its mental health system by creating DMH to oversee the provision of mental health services, including the authority to set regulations and monitor compliance—a shift away from the structure of its predecessor office, which was primarily a provider of services. Under this structure, DMH also continues the District’s historic role as a provider of mental health services. In its oversight role, DMH has developed certification standards and made use of licensing standards to enroll a network of providers to deliver an array of mental health services, which DMH continues to expand to ensure adequate capacity for community- based mental health services. DMH is in the early stages of implementing its new monitoring framework to ensure that services are complying with existing and newly established quality and safety standards. DMH remains the largest provider of community-based services and continues to provide inpatient mental health care for the District at St. Elizabeths Hospital.

Oversight Responsibilities Include Setting Regulations and Monitoring Provider Compliance

In 2001 the District took the first step toward implementing the final plan by passing legislation establishing DMH and giving it new oversight responsibilities, including setting regulations and monitoring community- based provider compliance. The significant organizational change accompanying the addition of oversight responsibilities required hiring new leadership and redeploying and retraining a large portion of existing staff. For example, of DMH’s 270 administrative and oversight staff positions, which represent approximately 14 percent of all budgeted staff for fiscal year 2003, the majority of positions were new and required either redeployment of existing staff or hiring new staff. Consistent with the final plan, DMH established a training institute to provide staff training and development, among other services. As of December 2001, a court report indicated that key leadership positions had been filled, including that of the director of DMH, who was hired by the mayor in April 2001. Subsequently, however, one key leadership position, DMH’s chief financial officer, experienced turnover, with four individuals serving in the role since April 2001. DMH has also hired two chief executive officers with experience in other systems undergoing reform, to run its community- based services agency and St. Elizabeths Hospital, respectively. (See table 3 for a summary of DMH’s functional responsibilities, including oversight, by office.)

DMH became the primary entity for overseeing a mental health system that is focused on community-based systems of care. (See table 4.) DMH’s regulatory responsibilities include developing standards and certifying providers of services, such as rehabilitative services and supported housing at independent living facilities, and licensing community residential facilities. As of January 2004, DMH had certified 22 mental health rehabilitative services providers, licensed more than 148 community residential facilities, and was in the process of implementing a certification program to oversee more than 400 supported independent living facilities. DMH addressed rehabilitative services standards by developing and publishing specific provider certification standards that took effect on November 9, 2001.

In addition to its regulatory responsibilities, DMH must monitor providers’ compliance with existing and newly developed quality and safety standards. DMH’s oversight division, the Office of Accountability, has direct responsibility for monitoring compliance with standards. DMH has developed a monitoring framework that is in the early stages of implementation, with DMH beginning to use information from some monitoring efforts to assess provider compliance and continuing to adjust other efforts. The following are examples of DMH monitoring efforts: Safety inspections, which are surveys of the sites where licensed providers offer services, are used to ensure health and safety standards are met. In the first 11 months of 2003, DMH conducted at least 150 inspections of 148 eligible facilities. When DMH conducts site inspections, it can issue notices of infractions for violations of the standards. According to DMH, from April 2002 through January 2004, it issued 46 notices to 22 providers and issued more than $29,000 in fines for identified deficiencies, including items such as insufficient staff on duty, failure to report unusual incidents, inaccurate personnel records, and exceeding maximum capacity. Increasing the number of site inspections of facilities that serve DMH consumers is one of the goals included in the DMH annual “scorecard” submitted to the District Mayor’s office, which tracks commitments and deadlines set for DMH.

Provider audits, which are record reviews of certified rehabilitative services providers, are used to analyze trends across providers and to ensure that providers are meeting documentation and service standards. In January 2003, DMH completed its first round of audits for the 12 providers certified at that time. As expected by DMH for the first year of applying standards, the audit found that providers were not in compliance with certain documentation requirements, such as having the approving practitioner sign the authorized treatment plan, and, as a result, all 12 providers were to implement corrective action plans. While these initial audits focused solely on provider documentation compliance, the second round of audits of all certified providers, which DMH expects to complete in early 2004, will examine how well specific services (such as medication treatment) are being provided.

Routine, biennial recertification reviews for rehabilitative services providers, which include evaluations of recorded complaints, audits, and public comment, are used to ensure that individual providers are complying with certification standards. With the first round of recertification applications, begun in December 2003, DMH will be able to use data from these reviews to make decisions regarding providers’ recertifications. Investigations of unusual incidents, which are conducted by the Office of Accountability and providers, are used to ensure consumer safety and reduce the occurrence of future incidents. DMH is expected to investigate any major unusual incident, such as consumer deaths, adverse drug reactions, and allegations of abuse or neglect. Providers are expected to investigate other, less serious incidents, defined as any events that occur outside the normal routine of care, and they are required to report to DMH all unusual incidents and action taken to respond to them. Unusual incidents, which vary widely in severity, were reported 1,259 times in calendar year 2003, including 336 reports of major unusual incidents. Of the 1,259 unusual incidents reported for 2003, DMH resolved 528 cases, including 161 major unusual incident cases. The remaining 731 cases usually required additional information from providers or other District agency investigators before DMH could take action. According to a DMH official, on average, a case remains pending for between 30 and 90 days before a disposition is reached.

DMH Continues the District’s Historical Role as a Direct Provider of Mental Health Services

Through DMH, the District remains a direct provider of a significant portion of mental health services. DMH’s own community services agency is the largest provider of community-based services in the District, acting as the primary provider for 55 percent of all consumers enrolled in the District mental heath system as of October 2003. In addition, it is the sole provider of a number of services, including crisis response services for adult consumers through its Comprehensive Psychiatric Emergency Program and free pharmacy services for uninsured consumers. The number of consumers receiving community-based services directly from DMH grew from 4,191 in October 2002 to 6,971 in October 2003. In addition, the total number of consumers served by the 13 other community-based providers increased from 2,612 in October 2002 to 5,631 in October 2003.

As envisioned by the transitional receiver’s final plan, DMH has also taken steps to reduce the number of beds at St. Elizabeths Hospital, but reductions have been limited by the lack of community-based services and agreements with community hospitals for acute care. The intent of the plan was for St. Elizabeths Hospital to be primarily a forensic hospital and a safety net facility for the community-based system of services and for community hospitals. While neither the final plan nor the exit criteria for the Dixon Decree specify goals for the reduction in the bed census at St. Elizabeths Hospital as a condition of ending the Dixon case, the exit criteria specify that 60 percent of DMH’s annual expenditures must be directed to community-based services. In DMH’s 2004 proposed budget, 41 percent of funds, approximately $80 million, are allocated for community- based providers and 42 percent, approximately $81 million, are allocated for St. Elizabeths Hospital. The remaining 17 percent, approximately $34 million, are budgeted for administration, oversight, delivery systems management, and other direct service costs, some of which represent fixed costs for community-based services. DMH has decreased the number of occupied beds at St. Elizabeths Hospital—from 628 beds in October 2000 to 513 beds in October 2003. In July 2003, the court monitor reported that the current model of continued reliance on St. Elizabeths Hospital was not financially viable, did not promote the concept of community-integrated care, and was not in compliance with the court- ordered plan. However, DMH stated that the hospital’s budget cannot be reduced without an additional decrease in the number of occupied beds. The chief executive officer of St. Elizabeths Hospital said that the census would not decrease until the community can support patients upon discharge, including providing access to affordable housing. The court monitor estimates that for the community-based system to adequately meet the needs of District residents, DMH would have to double the current capacity.

Enrollment and Billing System Is Designed to Coordinate Clinical, Administrative, and Financial Processes

In its first 2 years, DMH developed and implemented a comprehensive enrollment and billing system that coordinates clinical, administrative, and financial processes. Two key attributes of this system that were described in the final plan are that it (1) links payment with planning for individual treatment and the provision of services and (2) increases access to federal funds through the development of mental health rehabilitative services, which are community-based mental health services that a state’s Medicaid program can choose to provide. DMH has developed and implemented a system to link payment to authorized treatment plans, enroll consumers, reimburse providers, and bill Medicaid for rehabilitative services provided. However, moving to an FFS billing system for services has resulted in difficult adjustments, including managing cash flow, for some DMH providers.

Enrollment and Billing System Links Payment to Treatment

DMH’s enrollment and billing system that links payment to treatment, as envisioned by the final plan, is in place and operating. Consumers can enter into the mental health system through a variety of points in the community, including calling DMH’s Access Helpline, visiting a DMH- certified community-based service provider, receiving treatment in hospitals or emergency rooms, and receiving mental health assistance through other DMH outreach efforts. All District residents needing mental health services are eligible to receive them regardless of insurance coverage. The Access Helpline—which is a telephone hotline that provides crisis emergency services, enrollment assistance, and information and referral 24 hours a day, 7 days a week—or a certified CSA—which is responsible for acting as a clinical home and therefore assessing consumer needs and coordinating care—will enroll eligible consumers within 3 days of initial contact. When enrolling in the system, the consumer chooses a CSA as a clinical home based on a number of preferences such as location and treatment specialties. (See fig. 1.) After choosing the CSA, a consumer meets with a clinical manager to develop a treatment plan, which includes objectives and a plan of services, called an individualized recovery plan for adults and an individualized plan of care for children and youth. Once a clinical manager and a consumer develop a treatment plan, it is submitted by the CSA to DMH for authorization. Upon authorization of the treatment plan, a consumer can begin accessing the approved services. These services must be provided by a CSA or by another DMH-certified provider; once services are delivered, the providers then bill DMH on an FFS basis for reimbursement. Screening consumers for eligibility to receive mental health services and billing DMH for services rendered are new responsibilities for providers. Providers will be paid only for services delivered that are identified by the treatment plan and authorized by DMH.

As of December 2003, DMH had transitioned 12 of its 27 community-based services to the FFS enrollment and billing system, including all nine rehabilitative services, but 15 other services, such as consumer advocacy and peer support, had yet to be added. Services that have not been transitioned to the FFS system do not have to be identified in an authorized treatment plan; however, community-based providers must deliver these services according to their contractual agreements with DMH.

In order to develop a system that links payment to services provided, DMH purchased management information systems that coordinate clinical, administrative, and financial processes for mental health services. These systems allow CSAs to enroll consumers in the mental health system, submit claims electronically, and retrieve their consumers’ demographic data. These systems also streamline DMH’s administrative efforts by allowing DMH to electronically enroll consumers, authorize services, adjudicate claims, and generate payment reports for providers. The system further helps DMH monitor how much individual providers are billing, which helps DMH project expenditures. DMH received the first batches of claims in June and July 2002, and as of October 2003 it reported that its mental health system had 12,602 consumers enrolled. However, DMH could not report the number of consumers who received services within a 90-day period, which is consistent with the court’s definition of provision of services to enrolled consumers. As of January 2004, DMH had paid rehabilitative services providers $30.4 million for claims submitted in fiscal year 2003. DMH projects that it will have paid these providers a total of $35 million to $40 million for claims submitted in fiscal year 2003.

DMH Has Increased Access to Federal Funds through a Medicaid Mental Health Rehabilitation Services Option

In December 2001, the Centers for Medicare & Medicaid Services approved the District’s request to add the mental health rehabilitation services option to its Medicaid program. (See table 5.) Approval of the option increased both the number and scope of mental health services reimbursable by Medicaid. Under the option, DMH certifies and contracts with community providers to deliver covered services. DMH pays providers for any DMH-authorized service and, on behalf of contracted providers, files claims with the District Medicaid office for reimbursement of the federal share of the cost of Medicaid-covered services. Thus, there is no relationship between the District Medicaid office and the local providers for these services, nor is payment to providers contingent upon reimbursement by Medicaid. Other District community-based service providers that do not contract with DMH bill the District Medicaid office directly for their services.

DMH built mechanisms into the enrollment and billing processes to help providers and DMH work together to obtain Medicaid reimbursement. Access Helpline counselors work with providers to identify consumers who are eligible and enrolled in the Medicaid program using eligibility data from the District Medicaid office. Before transmitting Medicaid- reimbursable claims to the District’s Medicaid office, DMH checks each claim to ensure that the consumer is currently enrolled in Medicaid, that the provider is eligible, and that the covered service has been paid by DMH. Upon submittal for reimbursement to the District’s Medicaid office, DMH tracks the status of claims, receiving reports that detail the claims paid, waiting to be paid, and denied payment. The report also provides reasons that claims were denied.

DMH is improving its overall enrollment and billing system to decrease the time providers spend on administration and to increase the time they spend serving consumers. For example, in October 2003, DMH changed a component of the billing system that delayed providers from offering services. The system had required providers to electronically update treatment plans every 90 days. To reinforce this requirement, the information system prevented the provider from entering any other consumer data, such as claims data for a service provided, until the plan was updated. DMH realized that requiring providers to do this was burdensome and prevented them from serving consumers. As a result, DMH removed the requirement to update the treatment plan from the electronic billing system and is monitoring compliance with the 90-day requirement through an alternative mechanism.

DMH projects that as the enrollment and billing system improves and the provision of community-based services continues to expand, mental health rehabilitative services will eventually generate approximately $36 million to $38 million annually in federal Medicaid funds. As of November 2003, the District’s Medicaid office had reimbursed DMH $17.5 million for fiscal year 2003—over 50 percent of the amount DMH paid to providers for rehabilitative services. As one condition of ending the Dixon case, federal Medicaid funds must cover at least 49 percent of all mental health rehabilitative services provided. Although DMH expects future growth in Medicaid revenue, many individuals served by the District’s mental health system, especially adults, are not eligible for Medicaid.

New System Presents Challenges to Providers

According to DMH officials, moving to an FFS system represented a major change in business operations for DMH providers and has presented challenges for them; however, DMH has offered assistance to all certified rehabilitative providers. DMH offered training for providers on service and billing requirements and grants for building the infrastructure required to participate in the system. In addition, consultants funded by DMH can work with providers on developing sound business practices, including cash flow analysis, budgeting in an FFS environment, staff assignments and productivity, record keeping, and billing.

Even with assistance, providers experienced challenges since beginning to bill DMH on an FFS basis. Two providers reported that there are considerable investments of time and money necessary to be certified as a CSA. According to one provider, the new system requires more “business savvy” and planning by providers for revenue peaks and valleys because providers are no longer guaranteed revenue regardless of the level of services provided. Thus, as stated by the same provider, they must plan ahead to ensure they can meet payroll in months like December and February, when fewer consumers seek services because of holidays and winter weather.

Problems managing cash flow were exacerbated because provider contracts with DMH were tied to the billing projections, which meant that DMH could not pay claims for providers who exceeded their projections until their contracts were changed. The Mental Health Coalition, whose members are primarily DMH-certified providers, wrote to DMH several times in fiscal year 2003 listing a number of concerns with the billing process, and its primary concern was the lack of timely payment on a consistent basis. By August 2003, DMH made the necessary contract changes to allow providers to be paid for the remainder of the fiscal year and, according to senior officials, had a plan in process for fiscal year 2004 to prevent this problem from recurring. DMH provided data showing that in fiscal year 2003 it adjudicated—that is, made a decision to pay or deny—79 percent of submitted claims within 30 days; however, after adjudication, the District of Columbia Treasury must then pay the approved claims, which, according to DMH, took an average of 15 additional days. The court monitor has identified claims payment as an area of concern that will continue to be monitored. DMH did not provide the court monitor with a measure of timely reimbursement in 2003, but, according to the court monitor, in fiscal year 2004 DMH will be required to report the percentage of claims being paid within 30 days of submission.

Consumer-Centered Approach Blending Choice and Protection Is Evolving

Also central to DMH’s new mental health system is facilitating consumers’ participation in their recovery from mental illness, an approach that is consistent with the final plan, as well as national trends. Consistent with this focus, DMH has established requirements in two key areas, consumer choice and consumer protection. With regard to consumer choice, DMH has requirements in place to ensure that consumers participate in the selection and receipt of services. However, DMH’s initial review of rehabilitative services provider records showed gaps in documentation of consumer participation, such as a lack of documentation of the consumers’ participation in—and agreement with—their treatment plans for 41 percent of the records reviewed. DMH is addressing these gaps with providers to ensure that their practices comply with these requirements and adequately involve consumers in their treatment. Consumer protection policies are also evolving, with DMH publishing a uniform consumer grievance policy in October 2003. DMH officials emphasized that moving to a consumer-focused model is a long-term change that will take place gradually.

With Requirements in Place, DMH Is Addressing Gaps in Consumer Choice

Consumers entering the District’s mental health system are faced with important choices that help shape the provision of care they receive, including the choice of a CSA as a clinical home that will provide and coordinate care, choice of other DMH-certified providers, and choice of services through involvement in treatment planning. As part of the enrollment process, both the CSA and the Access Helpline are required to present consumers with the option to select any DMH-certified CSA to serve as the clinical home, a choice typically made based on their preferences, such as location and treatment specialties provided. Every CSA that serves as a consumer’s clinical home is required by DMH’s certification standards to have a policy in place to inform consumers about these and other choices available to them. For example, each CSA’s consumer choice policy must also inform consumers about the availability of peer and family support services—such as transportation, education, nutrition services, and recreation activities—as well as how to access the services. DMH’s certification standards also require CSAs to coordinate the treatment planning process for their consumers and to document consumer participation. For example, CSAs are required to develop a diagnostic assessment and treatment plan for each consumer that follows the consumer throughout the service delivery and reimbursement systems. Each CSA acting as a clinical home is required to obtain a consumer’s written consent to treatment as well as provide all consumers with a statement outlining their rights and responsibilities during the enrollment and treatment process.

To assist consumers in obtaining mental health services, the Director of DMH’s Office of Consumer and Family Affairs (OCFA) told us that DMH employs 15 to 20 mental health consumers as enrollment specialists who are available to other consumers as a resource in making these choices. DMH also offers training, some of which is conducted by other mental health consumers, that is available to consumers and their families on selecting providers and planning treatment. In addition, DMH’s enrollment handbook for new consumers summarizes aspects relating to the enrollment process, such as the types of mental health services available, range of consumer choices, and activities a consumer can expect during enrollment. Intended for use in the second quarter of 2004, DMH is developing a provider report card that contains specific information about each rehabilitative services provider to better facilitate consumer choice. For example, the provider report card will give providers a numerical score in areas, such as consumer access, billing and claims, and consumer complaints, that would enhance the consumers’ basis for selecting a provider. Finally, OCFA is also responsible for overseeing the development and implementation of the consumer satisfaction review required in the Dixon exit criteria, an initiative that DMH envisions as expanding the role of consumers in measuring the quality of services they receive in the District’s mental health system.

The court monitor and District mental health advocates have highlighted areas relating to consumer choice that need attention and that are consistent with DMH’s plans for additional development. In a January 2003 report to the court, the court monitor recommended that DMH develop a system for tracking consumer choice to help determine whether choices truly are available. The Director of OCFA told us that DMH would begin addressing this issue by identifying concerns relating to choice through consumer focus groups planned for each CSA in 2004. In addition, University Legal Services, the designated protection and advocacy program for the District, told us that consumers do not have enough information about how to access providers in the mental health system, and therefore it has published its own consumer rights manual. For example, an official with this organization told us that District consumers often do not have a choice among the full range of providers because many CSAs have limited capacity and have had to develop waiting lists. University Legal Services also cited a delay for consumers in receiving community-based services who are discharged from St. Elizabeths Hospital. While DMH is not required to report current baseline data regarding the receipt of community-based services for consumers following a hospital discharge, one condition for ending the Dixon case will be to demonstrate that 80 percent of known discharged inpatients receive services in a non-emergency, community-based setting within 7 days of a hospital discharge.

DMH’s initial audits of documentation practices of each of its certified rehabilitative services providers showed gaps in documentation of consumer participation in development of their treatment plans. Of the 740 unique consumer records DMH reviewed in its audit completed in January 2003, 38 percent did not have a consumer’s signature on the treatment plan and 41 percent did not document the consumer’s participation in and agreement with the treatment plan. Each of the 12 providers reviewed by DMH was asked to develop a self-audit program and implement staff training to address areas of deficiency in the audits, which, according to DMH, were to be expected in the first year of applying provider standards. Concerns raised by other stakeholders were consistent with the results of DMH’s audits of provider documentation practices. For example, in a July 2003 letter to DMH, University Legal Services noted systemic problems with treatment plans relating to consumer participation and accuracy, such as being unsigned, lacking consumer preferences, and failing to reflect consumer medical needs. DMH’s written response to University Legal Services highlighted the provider documentation audits completed by DMH as evidence that the department is identifying treatment plan issues but acknowledged that these problems will take time to resolve.

Consumer Protection Mechanisms Are Evolving

In October 2003, DMH published a consumer grievance policy, required by the legislation creating DMH, which strengthened the basic consumer protection provisions in DMH’s provider certification standards. Prior to publication of this policy, CSAs and other mental health providers were required to establish written complaint and grievance policies and procedures but did not have to include specific criteria consistent with an overall and uniform DMH policy. For example, the DMH policy published in October 2003 required providers to review, investigate, and respond within 5 business days to grievances alleging abuse or neglect or denial of a service. While consumers can continue to file grievances with CSAs or DMH, the new policy also specifically outlines the conditions under which consumers can request an external review of a grievance that can result in a fact-finding hearing or mediation process. The new policy also requires DMH to facilitate and fund peer advocacy programs that are independent of providers to assist consumers throughout the grievance process. In addition, providers are required to take specific steps to increase consumer awareness about their grievance policies, such as posting the various options and procedures for filing a grievance and documenting that the consumer received a copy of the provider’s policy.

DMH’s monitoring of consumer complaints and grievances is also evolving. As of January 2004, DMH had contracted with an organization to create a database that will allow OCFA to track consumer grievances and identify systemic issues. OCFA expects that the database will be developed in the first few months of 2004. The new grievance policy also specifies that DMH will periodically review the implementation of the provider policies and publish a semiannual report on the types and dispositions of all grievances filed as well as highlight noteworthy trends, patterns, and other statistical information. Prior to this policy, DMH could not ensure that grievances were being tracked and did not review the extent to which providers were implementing their grievance procedures.

DMH Faces Challenges in Developing the Capability to Measure Performance against Exit Criteria

The court monitor worked with DMH and others to develop performance targets to measure compliance with the Dixon exit criteria. On December 11, 2003, the court approved qualitative requirements for two exit criteria measures relating to consumer satisfaction with services and level of functioning. In addition, the court approved 17 performance targets for 17 exit criteria measures relating to system performance. Although the court monitor envisioned fiscal years 2004 and 2005 as the appropriate time frame for DMH to both measure and improve its performance, DMH faces major challenges to collecting and verifying the accuracy of the performance data, including developing methods to electronically collect the data, correcting known data deficiencies, and working with providers to submit accurate data.

Methods to Measure Performance against the Exit Criteria Approved in December 2003

In working to measure the District’s compliance with the exit criteria, the court monitor, in conjunction with an outside expert and the legal parties to the Dixon case, developed two qualitative requirements and 17 performance targets, which were approved by the court in December 2003. The qualitative requirements address two of the exit criteria measures—consumer functioning and consumer satisfaction. For these two measures, DMH is required to develop and implement consumer satisfaction and functioning review methods and begin using the data obtained by these methods to make refinements to service delivery. DMH has contracted with a consumer organization to build a consumer satisfaction initiative patterned after model programs around the country. As of December 2003, OCFA had conducted a telephone survey of consumers to help DMH develop this consumer satisfaction review. In addition, DMH officials told us that they are testing the effectiveness of a tool for assessing consumer functioning. According to the court monitor, DMH will provide a progress report in early 2004 on the status of these two reviews, but is not likely to submit the methodologies to the court monitor—which is required to comply with the exit criteria—for several more months.

The court also approved 17 exit criteria measures, each with a specific performance target. (See table 6.) Two of the 17 measures articulate overall system performance targets that DMH must meet in annual reviews of the services provided to adult and child and youth consumers. For example, DMH’s system must perform positively for 80 percent of the adults who are sampled and reviewed. The remaining 15 measures define specific system performance targets that DMH must meet in the aggregate for 4 consecutive quarters, such as demonstrating the timely receipt of supported housing services for a specific percentage of persons referred to supported housing. Once DMH meets these targets for the specified time frame, the court monitor ends active monitoring of the measure. However, according to the court order, DMH is required to continue to submit data to the court monitor for all exit criteria measures regardless of their monitoring status, giving the court the ability to require that DMH meet the performance targets for any exit criteria measure showing a substantial drop in performance. The Dixon case can be dismissed when the court monitor submits a report to the court affirming that the District has achieved compliance with all required performance targets and qualitative requirements for all of the exit criteria, and the court accepts that finding.

Originally, the court expected the proposed performance targets submitted by the court monitor to be accompanied by baseline measures of performance. The proposal approved by the court in December 2003, however, did not include previous requirements for DMH to submit baseline measurement data along with the performance targets. According to the court monitor and a DMH official, baseline data were omitted because (1) historical data are generally incomplete because of problems with data systems as well as a general lack of reliable and consistent previous data, and (2) many of the performance targets require information that was not collected by DMH and its providers, such as the number of consumers referred to supported housing. In commenting on a draft of this report, DMH noted that it was unable to identify comparable baselines from other jurisdictions.

Developing the Capability to Measure and Meet Performance Targets Will Take Time

Meeting the exit criteria performance targets, and thus ending the Dixon case, is a multiyear effort that requires DMH to develop and carry out a plan that will satisfy the court on three levels: (1) developing policies and practices that address the requirements of the exit criteria and demonstrating that DMH monitors the extent to which these policies are implemented, (2) developing specific methods for DMH’s collection and verification of the accuracy of the data, and (3) meeting the required performance targets for one full year as defined by the court. In November 2003, the court monitor anticipated that reviews relating to the first two requirements—policies and procedures and data collection and verification methods—will start in early 2004, but it may be a year before these two requirements are met for all of the exit criteria measures. The court monitor expects that DMH will concurrently develop and implement a plan to measure performance on all three levels that will allow the department to begin generating valid performance data in 2004. Although DMH began to collect data in July 2003 for some of the exit criteria measures based on the earlier methodologies approved by the court in May 2002, DMH officials told us in November 2003 that this data collection was preliminary and that they would not begin to develop a specific plan for meeting these three requirements until the court approved the final performance targets, which occurred in December 2003.

Satisfying the court regarding DMH’s demonstration of specific methods for collecting and verifying the accuracy of the performance data is likely to be challenging because of impediments to data collection as well as the fact that collected data may be incomplete or inaccurate. DMH and its providers face three major obstacles in collecting accurate data used to meet the actual performance targets: (1) establishing methods to collect electronic data, (2) correcting known data deficiencies, and (3) ensuring the accuracy of information collected and reported by providers. A description of each of these challenges follows.

Data Not Collected Electronically

Although the final exit criteria measures and performance targets were not approved until December 2003, DMH began collecting monthly data nonelectronically for 8 of the 17 exit criteria measures from providers in July 2003. For example, mental health rehabilitative services providers submit nonelectronic monthly reports to DMH on services provided to homeless consumers who are diagnosed with a serious mental illness. However, because the court approved revisions to some of the exit criteria measures in December 2003, providers will have to refine some of the information that they collect and report to DMH. In addition, the performance targets themselves, which did not exist prior to December 2003, will also affect the types of data collected. DMH officials told us that the department may be able to modify its enrollment and billing information system to collect some—but not all—of the data for the exit criteria measures, thus developing a central repository of information is still under discussion between DMH and the court monitor. Beyond this, a related issue will be developing the capacity to appropriately factor in other data currently collected by DMH in a way that is not duplicative of the monthly data submitted by mental health rehabilitative services providers. For example, officials told us that DMH’s school-based services program collects information that could be used as part of the calculation to meet the performance target requiring 75 percent of children and youth with serious emotional disturbances to receive services in a natural setting such as the home or school. However, the information collected through this program is not consumer-specific, nor is it linked to DMH’s enrollment and billing information system, which may, according to DMH officials, eventually be the primary mechanism for collecting data on many of the performance targets.

Deficiencies in Service Utilization Data

As part of the exit criteria requirements for the Dixon case, DMH conducted an initial consumer services review in the spring of 2003 that identified two major service provision gaps relating to services provided to children and youth that need to be addressed to ensure the accuracy of the performance target data collected by DMH. The court monitor’s semiannual reports to the court have similarly highlighted these findings as areas requiring action. First, the review showed that many of the children and youth placed in residential treatment centers (RTC) do not have a clinical home at a CSA as intended and thus are not receiving DMH services. In addition to raising concerns about the coordination between DMH and RTCs, the lack of services for these individuals could also affect the accuracy of the data collected by DMH to meet a performance target that requires DMH to demonstrate that 85 percent of children and youth with serious emotional disturbances served by the system are living in their own or surrogate homes. Second, according to the court monitor, the consumer services review also revealed a significant gap between the number of children and youth enrolled in DMH’s system and the number who are actually receiving services. The court monitor’s report acknowledged that the source of this gap, while unknown, could reflect flaws in DMH’s data management system, its disenrollment policy, or clinical standards, such as required follow-up with consumers who have missed an appointment. Since the four penetration rate performance targets are calculated using the number of enrolled consumers who received at least one service in the past quarter, DMH will need to determine the cause of this gap to ensure that its performance data are accurate. As of March 2004, DMH had not provided us with the number of consumers who were enrolled and receiving services within a 90-day period.

Accuracy of Information Reported by Providers

Beginning in July 2003, DMH began collecting unaudited monthly data from mental health rehabilitative services providers for a range of exit criteria measures, including the provision of supported housing and supported employment. The department has also begun to collect preliminary discharge data from St. Elizabeths Hospital and local hospitals providing acute care to mental health care consumers. However, as of November 2003, neither the mental health rehabilitative services providers nor the local hospitals were required to track this information, and DMH did not have processes in place for verifying the accuracy of these data. DMH’s Director told us in December 2003 that DMH is planning to incorporate reporting requirements as part of the recertification process for mental health rehabilitative services providers. As of January 2004, DMH was planning to collect quarterly discharge data from local hospitals but was still working out the details. Until these reporting processes are put in place, DMH will continue to collect discharge data from a combination of St. Elizabeths Hospital (for adults) and mental health rehabilitative services providers (for children and youth). Even after hospital reporting processes are implemented, the court monitor and DMH expect some difficulties in collecting comprehensive discharge data. For example, a DMH consumer may seek care in a local hospital that does not typically serve DMH consumers and thus does not provide quarterly data to DMH. Recognizing potential challenges in collecting data from local hospitals, the court monitor proposed—and the court approved—as one of the 17 performance targets, a continuity of care performance target that allows DMH to limit its measurement against this exit criteria measure to “known” inpatient hospital discharges. The court monitor expects DMH to include in its plan specific strategies for obtaining and verifying the accuracy of these data. The potential lack of accurate data—for example, from local hospitals—may mean that some discharged individuals are not factored into the data used to measure performance. In addition, the lack of consumer-specific data collected by DMH to meet the performance targets will also be a challenge. For example, because mental health rehabilitative services providers do not submit the names of each homeless consumer served, just the total number of these individuals served, information submitted is not consumer-specific and thus may be duplicative, compromising the accuracy of the measurement. The court monitor has also told us that DMH will need to verify that the performance target data are unduplicated.

Comments from DMH and the Court Monitor and Our Evaluation

DMH provided written comments on a draft of our report. DMH’s comments are included, with our detailed responses, in appendix II. The court monitor provided technical comments, which we incorporated as appropriate.

In its comments, DMH stated that the court-ordered plan for the reform of the District’s mental health system envisioned comprehensive and sweeping reforms, noting that accomplishing such reforms would result in over 50 percent of DMH’s budget being redirected in a 5-year period. DMH described six broad changes to the District’s mental health system in the court-ordered implementation plan. These changes included (1) implementing a mental health authority, (2) instituting systems of care, (3) developing a new set of accountability functions and changing the oversight and monitoring of mental health services, (4) incorporating consumer protections, (5) shifting the methods and operations for financing the delivery of inpatient and outpatient mental health services, and (6) creating a new Department of Mental Health with new responsibilities for operating within the city government. In addition, DMH stated that in spite of the District’s failure to meaningfully participate in the last 20 years of mental health reform, DMH is moving aggressively to become a positive contributor to the health and well being of the community and to persons in priority service groups.

DMH commented that the draft report addressed several issues in depth while overlooking other reforms prominent in the final plan and the legislation creating DMH and other services such as Assertive Community Treatment, Supported Employment, and Supported Housing. The scope of our work was the status of DMH’s efforts to establish a community-based system of mental health care, focusing on four key areas of reform central to meeting the exit criteria for the Dixon Decree. While we believe that the other reform initiatives and services are important, we believe that DMH’s status with regard to meeting the exit criteria is an appropriate gauge of compliance with the Dixon Decree. We believe that making a comprehensive assessment of the system’s performance before DMH begins reporting on the exit criteria is premature.

DMH also provided specific comments that clarified, updated, or added information regarding its status in implementing the final plan (see app. II). Where appropriate, we incorporated these changes into the report.

As agreed with your office, we plan no further distribution of this report until 30 days from its date of issue, unless you publicly announce its contents. At that time, we will send copies of this report to the Director of the District of Columbia Department of Mental Health. We will also make copies available to others upon request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov.

Please call me at (202) 512-7118 or Carolyn Yocom at (202) 512-4931 if you have questions about this report. Major contributors to this report are listed in appendix III.

Appendix I: Major Court Actions Related to District Compliance with the Dixon Decree

A class action lawsuit was filed in the U.S. District Court for the District of Columbia on behalf of District residents institutionalized at St. Elizabeths Hospital.

The court determined that the District and the federal government had a joint responsibility to provide the plaintiffs community-based treatment in the least restrictive conditions. This ruling is known as the Dixon Decree.

To comply with the court order, the involved parties drafted a final implementation plan that generally required an assessment of the plaintiff class members and periodic reports on progress in establishing a community-based system.

Congress passed legislation that required the District to complete implementation of an integrated coordinated mental health system by October 1, 1991. Congress transferred sole responsibility of establishing the required local mental health services to residents of the District. The transfer was not effective until October 1, 1987.

The court determined that no progress had been made to comply with the final implementation plan. The involved parties therefore developed a second plan. This plan is known as the service development plan.

The court appointed a special master to oversee implementation of the service development plan.

The court determined that the District was still unable to comply with the terms of the service development plan. As a result, the involved parties negotiated a third plan, whose goals the District met. This plan is referred to as the Phase I agreement.

The parties negotiated and began to implement a fourth plan, which was significantly broader in scope and required activities such as hiring personnel and developing a homeless service plan. This plan is referred to as the Phase II plan.

The District admitted noncompliance with the fourth plan, and the plaintiffs requested the appointment of a receiver.

On September 10, the court appointed a receiver on the basis that only a receiver provides the court with enough day-to- day authority to force compliance without causing confusion and ambiguity in the administration of the commission.

On March 6, with agreement of all parties, a new receiver, referred to by the court as a transitional receiver, was appointed and officially assumed this role on April 1. The transitional receiver was scheduled to return control of the mental health system to the District between January 1 and April 1, 2001.

On April 2, the court approved the transitional receiver’s final plan and required the District to implement it. The plan provided a policy framework for meeting the Dixon mandate, including developmental milestones but not specific service targets.

On May 22, the court found that the District was capable of implementing and was in fact implementing the final plan and thus terminated the receivership. The order also appointed the former transitional receiver as court monitor of District compliance with the final plan, and it approved exit criteria agreed upon by all parties. The monitor was directed to report to the court and the parties no less frequently than every 6 months.

On December 11, the court approved a revised set of exit criteria, which replaced the criteria approved in May 2002, with measurement methodologies, definitions, performance targets and qualitative requirements. In addition, the court ordered that the case would be dismissed when (1) the court monitor affirms that the District has complied with all of the performance targets and qualitative requirements and the court accepts that finding; or (2) the District moves for dismissal and demonstrates “substantial” compliance with the performance targets and qualitative requirements and the court determines the case should be dismissed.

Appendix II: Comments from the District of Columbia Department of Mental Health and GAO’s Response

The following is our response to DMH’s letter dated March 18, 2004.

GAO’s Response to DMH’s Comments

Our responses below correspond to the comments numbered in the margin of DMH’s letter. 1. DMH commented that the draft report references only a portion of the requirements of the Mental Health Reform Act and the court-ordered plan and does not discuss the development of “systems of care,” which it characterizes as cornerstones of the court-ordered plan and the legislation creating DMH. We believe that the report adequately characterized the immensity of the tasks faced by DMH. The scope of this report encompassed the actions taken by DMH since its creation to comply with the Dixon Decree. As such, we reported on the status of the District’s effort to establish a community-based system of mental health care, with a focus on four key areas of reform that were confirmed by the court monitor to be central to compliance with the Dixon Decree. Because many of the services and initiatives under way were still evolving and had incomplete data at the time of our work, we did not believe that a comprehensive assessment of DMH’s progress on all activities was appropriate. As a result, we focused on the data collection methods for the 17 performance targets relating to the District’s compliance with the court’s exit criteria. 2. We modified the report where appropriate to address information about the additional consumer protections, the number of supportive housing arrangements, the relocation of acute care beds, the hiring of senior managers, the status of leadership positions at DMH, the increases in service provision at the public CSA, the difficulty of adding new psychiatric services in local hospitals, and the results of provider audits. 3. DMH stated that the draft report did not accurately portray payments made to providers in fiscal year 2003 and that our findings on slow payments to providers could not be substantiated. We modified the report to reflect the updated data on billings for fiscal year 2003. However, we disagree that payment problems could not be substantiated. Provider contracts with DMH were tied to the billing projections, which meant that DMH could not pay claims for providers who exceeded their projections until their contracts were changed. The court monitor’s 2003 reports also indicate that claims payment has been an area of concern. Our draft report acknowledged that DMH had made the necessary contract changes to allow providers to be paid for the remainder of fiscal year 2003. Additionally, we cited DMH’s plan for fiscal year 2004, which aimed to prevent similar billing problems from occurring. 4. With regard to our assessment of DMH’s status in meeting court expectations, DMH commented that it believes table 2 reflects our assumption that DMH has not begun work on meeting the 15 system performance targets or begun using consumer functioning and consumer satisfaction data. DMH stated that it has not reported on these steps but has initiatives under way to meet each one and therefore the table should reflect that the “step has been started.” As of March 2004, the court monitor had not received evidence that these steps were in process, but confirmed that DMH had conducted preliminary work that had not been captured in court documents. Thus, we modified the report to reflect that these steps were “in planning.” In addition, the report refers to the work under way to meet the exit criteria, such as the consumer telephone survey conducted in 2003 to help DMH develop its consumer satisfaction review and data collection efforts from providers for some of the exit criteria measures. 5. DMH commented that the draft report did not indicate that there were no standards for provider audits, that provider audits had never been conducted, and that DMH expected that providers would not be in compliance. The draft report stated that DMH’s new responsibilities for regulating and monitoring providers, including conducting audits, were a shift away from the structure of its predecessor office and that the monitoring framework was in the early stages of implementation. We revised the report to reflect DMH’s expectation that providers would not be in compliance with the new standards. 6. With regard to the draft report’s discussion of unusual incidents, DMH noted that the District’s mental health system had never experienced a review of unusual incidents and stated that unusual incidents ranged in severity from consumers returning late for dinner to injury and abuse. DMH also stated that it is faced with thousands of unusual incidents and said that it sorts through incidents quickly and is beginning to identify trends. We modified the report to reflect the range of severity of unusual incidents. 7. DMH commented that the report’s subheading, “Enrollment and Billing System Is Designed to Coordinate Clinical, Administrative, and Financial Processes” represents a significant misapprehension. DMH stated that the enrollment and billing systems are not the major functions in the design of the clinical, administrative, and financial processes. DMH characterized the billing system as an administrative function that helped with the transition from a grants-based system of delivering services to a performance-focused fee-for-service (FFS) system. We believe that the enrollment and billing system is an important design component. For example, the final court-ordered plan outlines that a comprehensive enrollment and billing system that links payment to treatment is necessary to access federal Medicaid revenue through the mental health rehabilitation services option, which was identified in our October 2000 report and in the final plan as a key component for reforming the District’s mental health system. Further, DMH’s enrollment and billing information system is used to enroll consumers, reimburse providers, and, according to DMH officials, may eventually be the primary mechanism for collecting the performance data required to meet the Dixon exit criteria. 8. Regarding our findings on consumer choice and community follow-up after a consumer’s discharge from the hospital, DMH stated that comments from the court monitor and the local Protection and Advocacy for Individuals with Mental Illness (PAIMI) agency (University Legal Services), were not quantified and that the report provides no other basis upon which to assess their reliability. As of January 2004, DMH was in the process of developing methods to track consumer choice and had not reported data to the court on community follow-up after discharge from the hospital. Absent that data, we relied on the court monitor’s judgments regarding DMH’s progress in implementing the court-approved plan. Additionally, the District mental health advocates with whom we spoke are part of the federally- mandated protection and advocacy system. 9. DMH commented that our findings on DMH’s capability to measure performance against the exit criteria (1) presented an incomplete account of events leading to the development of the performance targets and (2) missed critical factors for why baseline data were not included in the exit criteria requirements, specifically, that having a baseline would be impossible because services did not exist before DMH became a department and there was no basis for comparison with other jurisdictions. In response to DMH’s first concern, we revised the report to clarify that the court monitor did not act alone to develop the targets for measuring performance against the exit criteria. Regarding the second concern, the draft report stated that baseline data were omitted because historical data are generally incomplete and many of the performance targets require the collection of new information from DMH and its providers. We modified the report to reflect that DMH was unable to identify comparable baselines from other jurisdictions. 10. With regard to our findings on data collection and integrity, DMH commented that the draft report did not take into account the developmental stage of the data collection process. DMH noted that some of the performance criteria do not lend themselves to electronic data collection, gaps in service utilization data for children and youth placed in residential treatment centers must be viewed in the context that five city departments carry out placements, and the draft report’s statement that there is a gap between the number of children and youth enrolled and the number receiving services lacks quantitative support. We modified the draft to reflect that the two performance measures related to homeless consumers do not lend themselves to electronic data collection, which was confirmed by the court monitor, and that addressing the gap in service utilization data requires coordination with other District agencies that typically have their own tracking systems. The draft report stated that according to the court monitor, the first consumer services review for children and youth revealed a gap between the number of children and youth enrolled and the number receiving services. DMH did not provide us with the number of children and youth enrolled and receiving services. In the absence of that data, we relied on the court monitor’s report, which cited the gap identified by the consumer services review.

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Acknowledgments

Major contributors included Susan Barnidge, Laura Sutton Elsberg, Kevin Milne, and Elizabeth T. Morrison.