Mental Health: Better Documentation Needed to Oversee Substance Abuse and Mental Health Services Administration Grantees
Published: 2015-05-12T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: In 2013, SAMHSA estimated 43.8 million—or 18.5 percent—of adults in the United States suffered from a mental illness. SAMHSA, an agency within HHS, has various programs that aim to reduce the impact of mental illness through CMHS grants awarded to grantees that include states, territories, and nonprofit organizations.
GAO was asked to provide information on CMHS's oversight of mental health grant programs. This report identifies (1) CMHS's criteria for awarding grants to grantees, and how CMHS documents the application of these criteria; (2) the types of information CMHS uses to oversee its grantees; and (3) the steps CMHS takes to demonstrate how its grant programs further the achievement of SAMHSA's goals. GAO reviewed information related to CMHS grants management; reviewed grant documentation from fiscal years 2012 and 2013 for a nongeneralizable selection of 16 grantees within 5 grant programs: the MHBG, PAIMI, and 3 selected discretionary grant programs that GAO selected based on factors such as size of award and type of grantee; and interviewed SAMHSA officials.
section_title: What GAO Found
paragraphs: The Substance Abuse and Mental Health Services Administration's (SAMHSA) Center for Mental Health Services (CMHS) established criteria for the five grant programs covered by GAO's review that varied by program, but GAO found that CMHS did not document its application of criteria for about a third of the 16 grantees GAO reviewed. An example of how criteria varied by program is that one of the five grant programs required its grantees to state that they will use evidence-based practices to treat individuals with mental illness while the others did not. In addition, CMHS did not document its application of the criteria it used to award grants to 6 of the 16 grantees GAO reviewed. For example, for fiscal year 2012, CMHS did not clearly document the application of most criteria for any of the four Community Mental Health Services Block Grant (MHBG) grantees GAO reviewed. The Department of Health and Human Services' (HHS) grants manual, which CMHS officials told GAO they follow, states that CMHS must maintain appropriate documentation to support funding decisions. GAO found a variety of reasons why CMHS did not adequately document the application of criteria, including a lack of program-specific guidance.
CMHS officials said they use various types of information to oversee grantees, but the documentation of this information was often missing or not readily available during the period GAO reviewed. For each grantee GAO reviewed, there was at least one instance in which the documentation used to oversee grantees was either missing or not readily available—meaning it was either missing entirely, stored outside of the systems designated for storing the information, or was not readily available to all officials involved in the oversight of grant documentation. For example, GAO found that CMHS could not produce documentation of its review of required annual program performance reports covering fiscal year 2012 data for any of the four Protection and Advocacy for Individuals with Mental Illness (PAIMI) grantees GAO reviewed. The grants manual states that CMHS must create and maintain files that allow a third party, such as an auditor, to “follow the paper trail” from program initiation through closeout of individual awards. GAO found a variety of reasons why grant documentation was missing or not readily available, including a lack of program-specific guidance. Without proper documentation of information used to oversee grantees that is readily available, CMHS runs the risk that it does not have complete and accurate information needed to provide sufficient oversight of its grant programs.
CMHS officials told GAO that they take a variety of steps when reviewing grantees' performance measure data to demonstrate how CMHS's grant programs furthered the achievement of SAMHSA's goals. For example, CMHS produces summaries by grant program that are included as part of its budget justification. In addition, CMHS is working to ensure that the performance measure data it collects can be analyzed with performance measure data collected from other grantees awarded through programs across SAMHSA. According to SAMHSA, this analysis can be helpful when demonstrating how CMHS's grant programs further the achievement of SAMHSA's goals.
section_title: What GAO Recommends
paragraphs: GAO recommends that the Administrator of SAMHSA direct CMHS to take steps, such as developing additional program-specific guidance, to ensure that it consistently and completely documents both the application of criteria when awarding grants to grantees, and its ongoing oversight of grantees once grants are awarded. HHS concurred with this recommendation.

section_title: Letter
paragraphs: According to the Substance Abuse and Mental Health Services  Administration (SAMHSA), mental illness is widespread in the United  States. In 2013, SAMHSA estimated 43.8 million—or 18.5 percent—of  adults in the United States suffered from a mental illness. SAMHSA’s  mission is to reduce the impact of mental illness and substance abuse on  American communities, which it carries out through a variety of activities,  including awarding grants to support community-based mental health  prevention and treatment services. According to SAMHSA, the Center  for Mental Health Services (CMHS), one of SAMHSA’s four centers, leads  federal efforts to promote the prevention and treatment of mental illness.  Each center, including CMHS, supports the strategic initiatives, such as  recovery support, contained within SAMHSA’s strategic plan. The  strategic initiatives provide a framework for the agency in meeting its  mission. Within each strategic initiative, SAMHSA identifies goals that  specify how the agency can further the achievement of that strategic  initiative. Of SAMHSA’s total budget of about $3.35 billion in fiscal year  2013, it allocated approximately $915 million to CMHS. Approximately   90 percent of CMHS’s budget funds grant programs for grantees such as  states, territories, and nonprofit organizations, to support individuals with  mental illness.
Due to increasing attention to the needs of individuals with mental illness,  members of Congress held a series of hearings and public forums in  2013 examining how federal agencies, including SAMHSA, support the  treatment of individuals with mental illness. In addition, legislation  proposed in recent years has aimed to address various aspects of the  mental health system, including CMHS’s grant programs. In light of these  efforts, members of Congress raised questions about CMHS’s oversight  of its grant programs. Oversight of grant programs is important to help  ensure that program funding is used appropriately by grantees, to assist  management with decision-making, to help ensure grants are carried out  as intended, and to provide a source of information that can be used to  help improve grant programs over time.
You asked us to provide information on CMHS’s oversight of its mental  health grants, including any monitoring that occurs during the grant award  period. This report identifies (1) CMHS’s criteria for awarding grants to  grantees, and how CMHS documents the application of these criteria;   (2) the types of information CMHS uses to oversee its grantees; and   (3) the steps CMHS takes to demonstrate how its grant programs further  the achievement of SAMHSA’s goals.
To identify CMHS’s criteria for awarding grants to grantees and to  determine how CMHS documents the application of these criteria, we first  reviewed information related to CMHS’s review and award process, such  as requests for applications (RFA) that described the criteria grantees  were expected to meet for some grant programs. We also reviewed  HHS’s Awarding Agency Grants Administration Manual—or grants  manual—that CMHS officials said they follow to guide their grant  oversight efforts from before the grant is awarded through the end of the  grant award. We compared CMHS’s process for documenting its  application of criteria to award grants to grantees with requirements for  documenting significant decisions outlined in Standards for Internal  Control in the Federal Government. Second, we reviewed grant  documentation for a nongeneralizable selection of 16 grantees for fiscal  years 2012 and 2013 that we selected based on factors such as size of  grant award, grant program, and type of grantee, to identify how CMHS  documents its use of criteria to determine whether to award grants to  grantees.
Our nongeneralizable selection included 16 grantees that received grants  with varying project periods from 5 of CMHS’s grant programs—2 formula  grant programs: Community Mental Health Services Block Grant (MHBG)  and Protection and Advocacy for Individuals with Mental Illness (PAIMI);  and 3 discretionary grant programs. Specifically, we reviewed grantees  from the following discretionary grant programs: Systems of Care  Expansion Implementation Cooperative Agreements, Statewide  Consumer Network, and Cooperative Agreements for Linking Actions for  Unmet Needs in Children’s Health (Project LAUNCH). The majority of  grantees we reviewed received one grant from the respective grant  program for the period covered by our review. We reviewed the  grantees for fiscal years 2012 and 2013, regardless of the duration of the  grant’s project period. For the MHBG and PAIMI programs, we reviewed  information related to the initial application submitted by the grantees,  which covered the first year of the grant’s project period, as well as  information related to applications for continued funding submitted during  the course of the grant’s project period. For selected discretionary grant  programs, we reviewed documentation of criteria used to award grants to  grantees for the first year of the grant’s project period. For more  information on the number of grantees we reviewed from selected CMHS  grant programs, see table 1.
In selecting discretionary grant programs, we selected 3 of the 7  discretionary grant programs that released an RFA in fiscal year 2012  and for which grants were awarded in fiscal year 2013. We based our  selection on factors such as the size of grant award and the type of  grantee eligible to receive a grant award. Our nongeneralizable selection  of grantees across the grant programs included states, territories, tribal  organizations, and nonprofit organizations. For all the grantees we  selected, we reviewed information that CMHS uses to oversee its  grantees, including documents submitted by the grantee and agency  documents related to grants management—which we refer to broadly as  grant documentation—for fiscal years 2012 and 2013. To conduct this  review, we developed and used data collection instruments to determine  whether the grant documentation included evidence of the criteria used to  award grants to grantees. To assess the reliability of the data contained in  the grant documentation, we reviewed information related to the systems  CMHS uses to oversee its grantees, and interviewed officials about how  they maintain and check this information for accuracy. We found the data  to be sufficiently reliable for our purposes. While our findings related to  the grantees whose documentation we reviewed cannot be generalized to  all CMHS grantees and grant programs, they provide examples of CMHS  grants oversight under a wide range of circumstances, including differing  grant sizes, grant programs, and grantees. Lastly, we interviewed CMHS  officials, including officials responsible for overseeing the grantees we  selected, about the grant review and award process.
To identify the types of information CMHS uses to oversee its grantees,  we reviewed information related to CMHS’s grants management efforts,  including the grants manual. We reviewed grant documentation for the  nongeneralizable selection of 16 grantees described above to identify the  types of information CMHS uses to oversee selected grantees and how it  uses this information. We also interviewed CMHS officials, including  officials responsible for overseeing selected grantees, about reporting  requirements for grantees and how they maintain the information they  review. We compared the information with requirements for documenting  significant decisions outlined in Standards for Internal Control in the  Federal Government.
To identify the steps CMHS takes to demonstrate how its grant programs  further the achievement of SAMHSA’s goals, we reviewed information in  SAMHSA’s strategic plan related to its strategic initiatives and the goals  identified within each strategic initiative. We also reviewed information  related to CMHS’s performance measures for the selected grant  programs described above for fiscal years 2012 and 2013. Because we  only reviewed information related to performance measures from selected  CMHS grant programs, we did not evaluate the accuracy of CMHS’s  performance measures. Instead, we reviewed CMHS’s process for  analyzing performance measure data collected through these measures.  We interviewed officials about steps CMHS has taken to demonstrate  how its grant programs further the achievement of the goals identified  within SAMHSA’s strategic initiatives, including any performance measure  data CMHS reviews. We did not assess the accuracy of the data  submitted by grantees.
We conducted this performance audit from March 2014 through May  2015 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
subsections: section_title: Grants Awarded by CMHS
paragraphs: CMHS awards two types of grants—formula grants and discretionary  grants—to support mental health programs. According to CMHS, in fiscal  year 2013, about 61 percent of CMHS’s $822 million in grant funding was  awarded through formula grants and about 39 percent was awarded  through discretionary grants. (See fig. 1.)
Formula grants are awarded to eligible grantees that meet specified  criteria outlined in formulas prescribed by statute or regulation. These  formulas may consider factors such as the population-at-risk and cost of  services. These grants are generally awarded to states and territories that  distribute funds to various governmental or nongovernmental entities,  such as a state mental health agency or a community mental health  center. In fiscal year 2013, CMHS awarded about $501 million under  three formula grant programs:  1.  the MHBG program, which supports adults with serious mental illness  (SMI) or children with serious emotional disturbances (SED);  2.  the PAIMI program, which supports protection and advocacy systems  designed to ensure the rights of individuals with mental illness who  are at risk for abuse and neglect; and  3.  the Projects for Assistance in Transition from Homelessness program,  which supports outreach, mental health, and other support services to  homeless people with SMI.
Discretionary grants are generally awarded on a competitive basis for  specified projects that meet statutory eligibility and program requirements.  Discretionary grants allow CMHS to allocate funding to a particular issue,  such as suicide prevention, or to areas and populations with the greatest  need. CMHS discretionary grants may be awarded to state, local,  territorial, and tribal governments; institutions of higher education; other  non-profit organizations (including tribal organizations); and hospitals.  These grant applications are solicited through RFAs that specify the  purpose of the grant, eligibility requirements, and grantee reporting  requirements throughout the grant’s project period. The duration of a  grant’s project period and the amount of funding available each year differ  by grant program based on program requirements or statutory  requirements. In fiscal year 2013, CMHS awarded 589 discretionary  grants totaling about $321 million. The smallest grant award was about  $30,000 and the largest grant award was about $6 million.
section_title: Grant Review and Award Process
paragraphs: CMHS uses separate processes for reviewing and awarding grants for  the formula and discretionary grant programs from which we drew our  selection; figures 2 and 3 illustrate these two processes.
section_title: Evidence-Based Practices
paragraphs: After CMHS awards grants, grantees can use a variety of different  approaches to treat individuals with mental illness, including evidence- based practices. Since 1997, SAMHSA has sponsored a National  Registry of Evidence-Based Programs and Practices, which is a  searchable online registry of programs and services that are considered  by SAMHSA to be evidence-based. According to SAMHSA the purpose  of this system is to help the public, including grantees that have been  awarded grants by CMHS, learn more about available evidence-based  practices.
section_title: CMHS’s Criteria for Awarding Grants to Grantees Varied by Program, but CMHS Did Not Document Its Application of Criteria for About a Third of the Grantees We Reviewed
paragraphs: The criteria CMHS established for awarding grants to grantees for the  MHBG, PAIMI, and selected discretionary grant programs varied by  program. CMHS did not document its application of criteria when  awarding grants to grantees for about a third of the grantees we  reviewed.
subsections: section_title: CMHS’s Criteria for Awarding Grants to Grantees for the MHBG, PAIMI, and Selected Discretionary Grant Programs Varied by Program
paragraphs: The criteria CMHS established for awarding grants to grantees for the  MHBG, PAIMI, and selected discretionary grant programs varied by  program. These criteria identify the requirements grantees must meet in  order to receive a grant, including any requirements identified in statute,  regulation, or the terms and conditions for the grant program, such as  those outlined in RFAs. For example, one of the five programs that  awarded grants to grantees we reviewed—Project LAUNCH—required its  grantees to state that they will use evidence-based practices to treat  individuals with mental illness in their applications when such practices  are available, while the others did not. See table 2 for examples of  CMHS’s criteria for awarding grants to grantees.
section_title: CMHS Did Not Document Its Application of Criteria for About a Third of the Grantees We Reviewed and CMHS Lacked Program-Specific Guidance for How to Document This Information
paragraphs: During the 2-year period covered by our review, CMHS did not document  its application of the criteria it used to award grants to 6 of the 16  grantees we reviewed. Specifically, we found instances across the   2 years in which CMHS did not clearly document the application of its  criteria for 4 MHBG grantees and 2 PAIMI grantees. We found that CMHS  documented the criteria it applied when awarding grants to discretionary  grantees. The grants manual states that CMHS must maintain  appropriate file documentation to support decisions in the financial  assistance process, including funding decisions, and that transactions  and significant events are to be clearly documented to help management  with decision-making to help ensure operations are carried out as  intended. In addition, Standards for Internal Control in the Federal  Government state that all transactions and other significant events need  to be clearly documented. CMHS officials said that they follow the  grants manual but said that they do not have written guidance that is  specific to the MHBG and PAIMI programs that would assist project  officers in using the tools that CMHS has developed, such as checklists,  to document their application of criteria. Examples of instances in which  CMHS did not document its application of criteria when awarding grants  to the grantees we reviewed include the following:
MHBG. For fiscal year 2012, CMHS officials did not clearly document  the application of most criteria for any of the four MHBG grantees we  reviewed; however, officials did document how they applied most of  the criteria for fiscal year 2013. Because fiscal year 2013 was the  second year of the grant project period that corresponded to the  application for continued funding, it required that CMHS apply fewer  criteria than in the initial application for fiscal year 2012. Specifically,  CMHS officials use checklists to help them determine whether  grantees meet the agency’s criteria when deciding whether to award  grants to grantees. Grantees are expected to provide sufficient  information to demonstrate that they have satisfied the criteria in their  grant applications. However, for all four of the grantees we reviewed,  the checklists for the application covering fiscal year 2012 data listed  “N/A” or had missing responses for most of the items in the  checklist—the majority of which were related to criteria identified in  statute. One grantee’s checklist we reviewed listed “N/A” or had  missing responses for nearly all of the items related to statutory  requirements. For example, all items related to the five criteria that  must be addressed by the required plan submitted as part of the  application were listed as “N/A”, and none of the items related to the  required state priorities were completed by the project officer. One  project officer told us that if a required item on the checklist is listed as  “N/A”, it is because the project officer did not have the specified  information from the grantee and would therefore request the  information. However, we did not see evidence that the project officer  updated the checklists after receiving additional information. In  another case, officials told us that there was a technical problem with  the system that electronically stores the checklists and the responses  were automatically populated with “N/A” for these items. While we  found that none of the checklists for the MHBG applications covering  fiscal year 2012 data clearly documented how CMHS applied criteria  when reviewing applications, we found that the checklists covering  fiscal year 2013 data for the four grantees clearly documented the use  of most criteria. However, officials explained that the 2013 application  and the corresponding checklist was less extensive because it was an  application for continued funding; project officers reviewed 19 criteria  for the 2013 checklist compared to 129 for 2012.
PAIMI. For fiscal year 2012, CMHS officials documented the  application of criteria for three of the four PAIMI grantees we  reviewed. However, we were unable to determine whether CMHS  applied the criteria for the fourth grantee because CMHS officials  were not able to provide the checklist they used to review this  grantee. For fiscal year 2013, we found that accurate documentation  of the application of criteria was not available for 2 of the 4 grantees  we reviewed. Similar to the MHBG program, officials use checklists to  help them determine whether grantees met CMHS’s criteria, such as  whether the grantee demonstrated that it had identified priorities and  objectives that were relevant to the PAIMI program. All four of the  grantees’ checklists for the fiscal year 2013 applications contained  documentation that indicated that the project officer found that the  objectives identified by the grantee were not relevant to the PAIMI  program. However, CMHS officials said that this finding was an error  for two of the grantees, but was correct for the other two grantees.  The officials told us that, as of October 2014, they had not conducted  any follow-up with the two grantees to request additional information  regarding the noted concerns. Officials told us that they would only  follow up with the grantees on the noted concerns if they conducted a  site visit.
We found that there were a variety of reasons why CMHS did not  adequately document the application of criteria when awarding grants to  grantees. One reason was that officials told us they lacked program- specific guidance to document the application of criteria for some  programs. CMHS has developed tools for project officers to use for their  application of criteria, but project officers sometimes had different ideas  about how to use those tools and what to do if a grantee could not  provide needed information. For example, officials told us that for the  MHBG program there was no written guidance, during the period covered  by our review, that described how project officers were to complete the  application review checklists to help ensure grantees met the grant  criteria. Officials developed program-specific guidance for project officers  in fiscal year 2014 for the review of MHBG applications, but we did not  assess whether this guidance fully addressed the issues we identified.  CMHS officials told us that they also provide training to project officers  and that they believe the process is clear; however, we found that project  officers had different understandings of how to complete the checklists  that are intended to document their review of criteria. Officials also noted  a technical problem in the system used to document the application  review checklists in fiscal year 2012. For the PAIMI program, officials told  us that there was no formal process to address instances in which a  grantee did not meet criteria, including how to follow-up with the grantee,  for the period we reviewed. Project officers told us that they would  sometimes note concerns in the checklist or on the applications, but these  notes were for their own benefit. Further, they told us that they generally  do not follow-up with the grantee unless they have a site visit. Without  documentation that clearly identifies how criteria have been applied when  awarding grants to grantees, CMHS cannot ensure that it is applying  criteria consistently and appropriately.
section_title: CMHS Stated That It Uses Various Types of Information for Oversight, but the Documentation of This Information Was Often Missing or Not Readily Available
paragraphs: CMHS officials said they use various types of information to oversee  grantees awarded grants through the MHBG, PAIMI, and selected  discretionary grant programs. However, we found at least one instance  during the period covered by our review where documentation of this  information was either missing or not readily available for each grantee  we reviewed.
subsections: section_title: CMHS Officials Said They Use Various Types of Information, Such as Financial Information and Progress Reports, to Oversee Grantees
paragraphs: CMHS officials said they use various types of information to oversee  grantees awarded grants through the MHBG, PAIMI, and selected  discretionary grant programs. The grants manual, which CMHS officials  told us they use as guidance for grantee oversight, describes the various  types of information that should be used for overseeing grantees. This  includes information grantees are required to report during and at the  conclusion of the grant, such as financial information and progress  reports, and other information the agency uses for grants management,  such as documents noting project officer’s approval of information  submitted by the grantee. According to the grants manual, this  information is intended to hold the grantee accountable to the  programmatic and financial conditions of the award and provide a means  for ongoing monitoring of grantee performance. Examples of the types of  information and related documentation CMHS officials stated they use to  oversee grantees we reviewed for fiscal years 2012 and 2013 are  outlined in table 3 below.
section_title: Documentation of Some Information CMHS Officials Said They Used to Oversee Grantees Was Either Missing or Not Readily Available for Each of the Grantees We Reviewed and CMHS Lacked Program-Specific Guidance
paragraphs: For each of the 16 grantees we reviewed, we found at least one instance  during the period covered by our review in which the documentation  CMHS officials said they used to oversee grantees was missing or not  readily available—meaning it was either missing entirely, stored outside  of the systems CMHS designated for storing the information, or was not  readily available to all officials involved in the oversight of grant  documentation. Because this information was missing or not readily  available, we were unable to identify all of the information project officers  used to oversee these grants. The grants manual states that CMHS must  create and maintain files that allow a third party, such as an auditor, to  “follow the paper trail” beginning with program initiation through closeout  of individual awards, including decisions made and actions taken in  between. As previously noted, examples of the types of information that  should be documented include all information required by the conditions  of the award, such as financial and performance reports. According to the  grants manual, this documentation should provide a complete record of  the history of an award and serve as a means to support funding  decisions and provide ongoing monitoring of grantee performance. In  addition, Standards for Internal Control in the Federal Government state  that all transactions and other significant events need to be clearly  documented, and the documentation should be readily available for  examination and properly managed and maintained. CMHS officials  said that they follow the grants manual. However, where we found  documentation missing or not readily available, CMHS often lacked  program-specific guidance to assist project officers in documenting their  oversight of grantees. Table 4 below provides information on the types of  issues identified for the grant documentation for grantees we reviewed  from each of the three grant programs.
Specific examples of grant documentation that was missing or not readily  available for grantees we reviewed from each of the three grant programs  include the following:
MHBG. Financial information grantees are required to report was  missing for two of the four MHBG grantees we reviewed because,  according to CMHS officials, the grantees did not submit it. We  found that one grantee had not submitted final financial expenditures  for fiscal years 2012 or 2013. The other grantee had not yet submitted  final expenditures for fiscal year 2013. CMHS officials provided  documentation that indicated that these grantees subsequently  submitted this information after our review was complete. In addition,  we found that the same two MHBG grantees had not submitted  required information to show that they have maintained a required  level of state mental health expenditures. Further, we found that  there was no documentation from either of these grantees or from  CMHS officials that indicated that a waiver had been approved to  relieve the grantee of this requirement or that CMHS had determined  that the grantee had materially complied despite not maintaining the  required level of expenditures. This documentation is required in  instances in which the grantee has not complied with the spending  level. These grantees submitted other required financial information  after our review was complete and CMHS officials told us that as a  result this documentation was no longer needed. In addition, for all  four grantees we reviewed, documentation of the final approval by the  branch chief for the application covering fiscal year 2012 was not  readily available. Specifically, the approval in the application review  system was marked pending. According to officials, the application did  not appear to have been approved because there were technological  issues with the system CMHS maintains to record approvals. As a  result, the branch chief provided final approval verbally for these  grantees but did not document it.
PAIMI. CMHS could not produce documentation of its review of the  required annual program performance reports covering fiscal year  2012 data for any of the four PAIMI grantees we reviewed. Project  officers told us that they typically document their review of the reports  by writing notes in the margin of a paper copy of the annual report;  however, we did not see evidence of any notes in the paper copy  reports we reviewed or any other documentation of their review.  According to CMHS officials, CMHS developed a checklist in 2013 to  help project officers document their review of future annual reports.
We found that CMHS had completed these checklists for all four  grantees we reviewed. However, one checklist indicated that the  priorities and goals were poorly written and not all of the objectives  listed were relevant to the PAIMI program. Officials told us that they  did not have any other documentation related to these issues,  including any communication with the grantee. In addition, each year  officials use data submitted by grantees to calculate aggregate  performance measure data across all PAIMI grantees. Officials  explained that this information is hand-tallied on paper worksheets.  We found that officials did not keep records of the hand-tallied  worksheets they used to calculate aggregate performance measure  data for fiscal year 2012. Officials told us that they saw no need to  retain documentation of these calculations once the data were  published in the annual report. Officials also told us that although  senior officials review the worksheets for consistency and  completeness, they did not maintain any documentation of this review.
Discretionary. For seven of the eight discretionary grantees we  reviewed, some performance measure data had not been approved  by the project officer in CMHS’s performance measure data tracking  system. CMHS officials said that project officers are expected to  record their approval of performance measure data in CMHS’s  performance measure data tracking system. For the eighth  discretionary grantee we reviewed, the grantee had not submitted any  performance measure data since the grant was awarded in July  2013. According to CMHS officials, the performance measure data  did not appear to have been approved because CMHS’s performance  measure tracking system locked, meaning that it did not allow the  project officer to enter his or her approval, to enforce project officer  deadlines. In some of these cases, project officers used other  methods to document their approval of performance measure data,  such as hand-written notes. However, according to officials, these  notes are generally stored at project officers’ workstations and cannot  be easily accessed by other CMHS officials, making it difficult for a  third party to follow the trail of project officer oversight and for other  officials to access the documents if the project officer is unavailable.  In other cases, there was no documentation that the project officer  approved this performance measure data.
We found that there were a variety of reasons why documentation used to  oversee grantees was missing or not readily available. For example,  officials told us that in some cases they lack program-specific guidance  for the processes officials use to document their oversight of MHBG and  PAIMI grantees. For the MHBG program, officials told us that they did not  have program-specific guidance for the period of our review that indicated  how project officers and the branch chief are to document their approval  of the application if there are technological problems with the application  review system. For the PAIMI program, officials told us that they have  some program-specific guidance; however, officials told us that there is  no written guidance with instructions for how officials are to calculate the  PAIMI aggregate performance measure data from the data submitted by  grantees. Further, there is no written guidance that describes how this  information should be maintained. For its discretionary grant programs,  CMHS does have program-specific guidance that indicates how project  officers should review some performance measure data submitted by  grantees; however, this guidance does not indicate how project officers  should approve performance measure data after the system locks. CMHS  officials said that some grantees have difficulty meeting the requirements  of their grants because they serve high need populations. However, most  of the problems we identified were related to documentation that is to be  completed by CMHS officials for grants management and not due to  issues with the grantees. Without proper documentation of information  used to oversee grantees that is readily available, CMHS runs the risk  that it does not have complete and accurate information needed to  provide sufficient oversight of its grant programs. CMHS officials said that  SAMHSA began efforts in fiscal year 2015 to update existing guidance  and develop additional guidance for its grant programs, including those  administered by CMHS. However, since these efforts are still in early  stages, it is too soon to determine whether they will resolve the issues we  identified.
section_title: CMHS Takes A Variety of Steps When Reviewing Performance Measure Data to Demonstrate How Its Grant Programs Further the Achievement of SAMHSA’s Goals
paragraphs: CMHS officials told us that they take a variety of steps when reviewing  grantees’ performance measure data to demonstrate how CMHS’s grant  programs furthered the achievement of SAMHSA’s goals, which are  identified through the strategic initiatives contained within SAMHSA’s  strategic plan. CMHS collects performance measure data from grantees  as a way to assess grant program performance. CMHS officials said that  their performance measure data generally indicate that CMHS has made  progress in achieving SAMHSA’s goals. The data collected is based on  performance measures CMHS developed in response to the Government  Performance and Results Act of 1993 (GPRA), as amended by the GPRA  Modernization Act of 2010. Examples of these measures include the  number of evidence-based practices implemented, the number of children  screened for mental health or related interventions, and the number of  people served by the public mental health system. Grantees provide data  to CMHS officials in response to these measures periodically, based on  the requirements designated by each grant program. For example, the  discretionary grantees we reviewed were required to report performance  measure data either biannually or on a quarterly basis, while the MHBG  and PAIMI grantees we reviewed were required to report performance  measure data on an annual basis. (See app. I for more information on the  performance measures for the grantees we reviewed.)
CMHS officials stated that they take a variety of steps when reviewing  performance measure data to demonstrate how CMHS grant programs  further the achievement of SAMHSA’s goals. First, CMHS officials stated  that they review performance measure data reported by grantees for each  grant program on an annual basis as part of their budget planning and  formulation process. Specifically, CMHS produces summaries by grant  program that are included as part of its budget justification. Officials  stated that these summaries include tables with performance measure  data that demonstrate how the grant programs further the achievement of  SAMHSA’s goals. For example, SAMHSA’s fiscal year 2015 budget  justification provides information on the number of people served by the  public mental health system for the MHBG program who lived in private  residences during fiscal year 2012, which CMHS indicated is related to  SAMHSA’s goal to ensure that permanent housing and supportive  services are available for individuals in recovery from mental health and  substance use disorders. Second, some grant programs produced  additional reports that demonstrated how these grant programs furthered  the achievement of SAMHSA’s goals. For example, CMHS produces a  summary report of performance measure data across discretionary grant  programs on an annual basis. The report provides graphs with  performance measure data across several discretionary grant programs  for a given year, and describes how this performance measure data  furthers the achievement of SAMHSA’s goals. Third, CMHS officials  stated that performance measure data is reviewed by officials assigned to  each strategic initiative who report to the administrator of SAMHSA on an  ongoing basis to demonstrate the progress of the agency in furthering the  achievement of its goals. For example, through this effort, officials stated  that they reviewed performance measure data to demonstrate the  agency’s progress in achieving the goals within the recovery support  strategic initiative and presented these reports to the administrator of  SAMHSA.
In addition to these steps, in January 2015, CMHS began its  implementation of the Common Data Platform for its discretionary grant  programs. This platform is an electronic system that allows officials to  generate reports on performance measure data collected from grantees  awarded grants through programs across SAMHSA’s centers, including  CMHS. According to SAMHSA, analyzing performance measure data  across SAMHSA’s centers can assist the agency in evaluating the overall  effectiveness of its grant programs and in ensuring that each program  furthers the achievement of SAMHSA’s goals. For example, SAMHSA  documentation indicates that the Common Data Platform will help officials  identify the number of people served by SAMHSA grantees in a particular  state and compare that number to previous years’ data as a way of  measuring the impact of SAMHSA’s grant programs. While CMHS has  begun its implementation of the Common Data Platform for discretionary  grant programs, officials stated that CMHS will extend the platform in  2017 to include performance measure data collected from the MHBG and  PAIMI programs.
section_title: Conclusions
paragraphs: CMHS’s grant programs support services for individuals with mental  illness, which is widespread in the United States. Among the grantees we  reviewed, we identified concerns with CMHS’s documentation of its  application of criteria when awarding grants to grantees and of the  information it used to oversee grantees. We found that there were several  reasons why documentation was missing or not readily available for the  grantees we reviewed. These reasons included a lack of program-specific  guidance for the tools and processes that CMHS officials have developed  to document the oversight of grantees. While CMHS has developed tools  and processes for its staff to use to document key elements of grants  oversight, CMHS staff did not always understand how to use them.  CMHS has developed some program-specific guidance to help officials  oversee grantees and officials stated that SAMHSA began efforts in fiscal  year 2015 to update existing guidance and develop additional guidance  for CMHS’s grant programs; however, because these efforts are still in  early stages, it is too soon to determine whether they will address the  issues we found. CMHS officials said that some grantees have difficulty  meeting the requirements of their grants because they serve high need  populations. However, most of the problems we identified were related to  documentation that is to be completed by CMHS officials for grants  management and not due to issues with the grantees. Both the grants  manual, which CMHS officials said they follow to guide their grant  oversight efforts, and Standards for Internal Control in the Federal  Government, which apply to all government transactions, state that all  transactions and other significant events need to be clearly documented,  and that the documentation should be readily available. Without complete  documentation of key elements of the oversight of its grant programs,  CMHS does not have reasonable assurance that it is overseeing its grant  programs effectively to achieve SAMHSA’s goals.
section_title: Recommendation for Executive Action
paragraphs: To assure the consistent documentation of the application of criteria to  award grants to grantees and of the information used for oversight, the  Administrator of SAMHSA should direct CMHS to take steps, such as  developing additional program-specific guidance, to ensure that it  consistently and completely documents both the application of criteria  when awarding grants to grantees, and its ongoing oversight of grantees  once grants are awarded.
section_title: Agency Comments
paragraphs: We provided a draft of this report to HHS for comment. HHS provided  written comments, which are reprinted in appendix II. HHS concurred with  our recommendation and stated that the administrator of SAMHSA  directed the agency, including CMHS, to initiate efforts to ensure that it  consistently and completely documents both the application of criteria  when awarding grants to grantees and its ongoing oversight of grantees  once grants are awarded. HHS also provided examples of efforts  SAMHSA is undertaking to improve the management of its grant  programs, including revising and updating guidance used for grants  management. However, because several of the efforts are still in  development, it is too early to determine whether these efforts will  address the issues we identified. In addition, HHS provided technical  comments, which we incorporated as appropriate.
As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to the Secretary of Health  and Human Services. In addition, the report will be available at no charge  on GAO’s website at http://www.gao.gov.
If you or your staffs have any questions about this report, please contact  me at (202) 512-7114 or at kohnl@gao.gov. Contact points for our Office  of Congressional Relations and Office of Public Affairs can be found on  the last page of this report. Other major contributors to this report are  listed in appendix III.
section_title: Appendix I: Examples of Performance Measures for the Center for Mental Health Services Grantees We Reviewed
paragraphs: This appendix provides additional information on the performance  measures that the Center for Mental Health Services (CMHS) developed  for the grantees we reviewed from five CMHS grant programs— Community Mental Health Services Block Grant (MHBG), Protection and  Advocacy for Individuals with Mental Illness (PAIMI), Systems of Care  Expansion Implementation Cooperative Agreements, Statewide  Consumer Network, and Cooperative Agreements for Linking Actions for  Unmet Needs in Children’s Health (Project LAUNCH). CMHS collects  performance measure data from grantees based on these measures  periodically as a way to assess grant program performance. CMHS  developed its performance measures in response to the Government  Performance and Results Act of 1993 (GPRA), as amended. See table 5  for examples of performance measures for the grantees we reviewed.
section_title: Appendix II: Comments from the Department of Health and Human Services
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Tom Conahan,   Assistant Director; Cathy Hamann; Amy Leone; Dan Ries;   Rebecca Rust Williamson; and Jennifer Whitworth made key  contributions to this report.