Federal Emergency Management Agency: Crisis Counseling Grants Awarded to the State of New York after the September 11 Terrorist Attacks
Released: 2005-06-30T00:00:00

Letter

Background

Following the terrorist attacks on September 11, 2001, the President signed under the authority of the Stafford Act a major disaster declaration for the state of New York. The presidential declaration allowed the state of New York to apply for federal assistance to help recover from the disaster. FEMA was responsible for coordinating the federal response to the September 11 terrorist attacks and providing assistance through a variety of programs, including the CCP. The CCP was authorized in section 416 of the Stafford Act to help alleviate the psychological distress caused or aggravated by disasters declared eligible for federal assistance by the President. Through the CCP, FEMA released federal grant awards to supplement the state of New York’s ability to respond to the psychological distress caused by the September 11 terrorist attacks through the provision of short-term crisis counseling services to victims and training for crisis counselors.

FEMA relied on SAMHSA to provide expertise related to crisis counseling and public education for Project Liberty. FEMA assigns SAMHSA’s responsibilities for the CCP through an annual interagency agreement. These responsibilities included, among other things, providing technical assistance, monitoring the progress of programs conducted under the CCP, and performing program oversight. Within SAMHSA, the Center for Mental Health Services (CMHS) carried out these responsibilities for Project Liberty. CMHS received support from SAMHSA’s Division of Grants Management, which provides grant oversight.

NYS OMH established Project Liberty under the CCP to offer crisis counseling and public education services throughout the five boroughs of New York City and 10 surrounding counties free of charge to anyone affected by the World Trade Center disaster and its aftermath. The areas served by Project Liberty are shown in the shaded areas of figure 1.

The state of New York’s primary role was to administer, oversee, and guide Project Liberty’s program design, implementation, and evaluation and pay service providers, but not to provide services itself. New York City and the surrounding counties contracted with over 200 service providers and were responsible for overseeing day-to-day activities. Figure 2 shows the organization structure of Project Liberty at the federal, state, and local levels.

Under the CCP, Project Liberty’s goal was to serve New York City and the 10 surrounding counties by assisting those affected by the September 11 terrorist attacks to recover from their psychological reactions and regain predisaster level of functioning. The CCP supports services that are short- term interventions with individuals and groups experiencing psychological reactions to a presidentially declared disaster and its aftermath. Crisis counseling services were primarily delivered to disaster survivors through outreach, or face-to-face contact with survivors in familiar settings (e.g., neighborhoods, churches, community centers, and schools). Although the CCP does not support long-term, formal mental health services such as medications, office-based therapy, diagnostic services, psychiatric treatment, or substance abuse treatment, FEMA approved an enhanced services program for Project Liberty. This enhanced services program allowed for an expansion of services, including enhanced screening methods; a broader array of brief counseling approaches; and additional training, technical assistance, and supervision to a set of service providers. These enhanced services were intended to address the needs of individuals who continued to experience trauma symptoms and functional impairment after initial crisis counseling but did not need long-term mental health services.

Project Liberty was funded through two separate, but related, grant programs: the ISP and RSP. The ISP grant was designed to fund Project Liberty for the first 60 days following the disaster declaration. Because there was a continuing need for crisis counseling services, the ISP was extended to last about 9 months, until the RSP began. The RSP grant was awarded and was designed to continue to provide funding for an additional 9 months of crisis counseling services, but was extended to last for 2 ½ years. Figure 3 shows key milestones for Project Liberty.

For the approved ISP application, FEMA made funds available directly to the state. Under the RSP, after approval, funds were transferred from FEMA to SAMHSA, which awarded the grant to the state of New York through SAMHSA’s grants management process. The state of New York, in turn, disbursed funds to the service providers and local governments through the Research Foundation for Mental Hygiene, Inc. (RFMH), a not- for-profit corporation affiliated with the state of New York that assists with financial management of federal and other grants awarded to NYS OMH. Figure 4 shows the flow of funds for Project Liberty’s ISP and RSP.

Service providers were required to submit claims and supporting documentation to receive reimbursement for expenses incurred to provide services. As shown in figure 5, these claims were to have multiple levels of review to determine whether the expenses claimed were allowable under the CCP’s fiscal guidelines. This review structure, which placed primary responsibility for reviewing claims on the local government units, was based on the state of New York’s existing grant management policies. Additional controls for Project Liberty included (1) NYS OMH site visits to service providers in New York City and surrounding counties; (2) closeout audits by independent auditors of certain New York City service providers to test whether claims were documented and allowable; and (3) annual audits of New York City and surrounding counties conducted under the Single Audit Act, which requires independent auditors to provide an opinion on whether the financial statements are fairly presented, a report on internal control related to the major programs, and a report on compliance with key laws, regulations, and the provisions of the grant agreements.

Our publication, Standards for Internal Control in the Federal Government, provides a road map for entities to establish control for all aspects of their operations and a basis against which entities can evaluate their control structures. The five components of internal control are as follows: Control environment. Creating a culture of accountability within the entire organization—program offices, financial services, and regional offices—by establishing a positive and supportive attitude toward the achievement of established program outcomes.

Risk assessment. Identifying and analyzing relevant problems that might prevent the program from achieving its objectives. Developing processes that can be used to form a basis for measuring actual or potential effects of these problems and manage their risks.

Control activities. Establishing and implementing oversight processes to address risk areas and help ensure that management’s decisions— especially about how to measure and manage risks—are carried out and program objectives are met.

Information and communication. Using and sharing relevant, reliable, and timely information on program-specific and general financial risks. Such information surfaces as a result of the processes—or control activities—used to measure and address risks.

Monitoring. Tracking improvement initiatives over time and identifying additional actions needed to further improve program efficiency and effectiveness.

SAMHSA and FEMA were responsible for providing oversight to ensure that the state of New York had a reasonable level of controls in place. Although FEMA retained responsibility for providing leadership and direction for Project Liberty, it assigned primary responsibility to SAMHSA for oversight and monitoring through an interagency agreement.

Remaining Grant Funds Primarily Relate to Unresolved Issues at NYC DOEd

Approximately $121 million, more than three-quarters of the $154.9 million in federal funds provided to Project Liberty, were reported as expended as of September 30, 2004, leaving a remaining balance of $33.9 million. About $32 million of the $33.9 million pertain to unresolved NYC DOEd expense claims. According to NYS OMH, NYC DOEd had not been reimbursed for the Project Liberty expenses it incurred throughout the program because NYC DOEd had not been able to provide support for these expenses that met the CCP documentation standards for reimbursement under federal grants. NYS OMH began considering alternative indirect forms of evidence, including internal control summary memos prepared by NYC DOEd, to begin paying NYC DOEd’s expense claims. As of March 31, 2005, NYS OMH had accepted alternative forms of supporting evidence to pay $5.2 million of NYC DOEd expense claims; however, this type of alternative evidence provides only limited assurance of the propriety of the claimed amounts. NYS OMH was not sure when and how the remaining NYC DOEd expense claims would be resolved.

Timing of Reported Expenditures

For the period September 11, 2001, through September 30, 2004, Project Liberty reported that it had expended all of the $22.8 million ISP grant and about $98.2 million of the $132.1 million RSP grant for total reported expenditures of approximately $121 million, leaving a remaining balance $33.9 million. Although crisis counseling services had been phased out as of December 31, 2004, Project Liberty will continue to use the remaining grant funds to process claims for reimbursement of program-related expenses incurred through December 31, 2004, and to cover administrative expenses during the closeout period, which at the end of our fieldwork, was scheduled to end on May 30, 2005. Table 1 and figure 6 show the timing and amount of expenditures reported by Project Liberty for the ISP and RSP grants by quarter through September 30, 2004, compared to the total CCP grant awards for Project Liberty.

According to NYS OMH officials, the expenditures reported by Project Liberty from September 11, 2001, through September 30, 2004, included expenses incurred as well as amounts advanced to service providers. During the RSP, Project Liberty made advances to 109 service providers, for a total of about $25.8 million. As of September 30, 2004, the outstanding advance balance was $5.8 million; however, according to an NYS OMH official, the balance had been reduced to $1.2 million as of March 31, 2005.

NYC DOEd Expense Claims Not Fully Resolved

The vast majority of remaining Project Liberty funds related to unresolved expense claims of NYC DOEd. As of March 31, 2005, NYS OMH officials told us that NYC DOEd had submitted claims for a portion of the $32 million that was budgeted to NYC DOEd to provide crisis counseling services to New York City school children, and planned to ultimately submit claims for the full amount.

NYS OMH and NYC DOHMH had not approved the majority of NYC DOEd claims for reimbursement incurred during the RSP because NYC DOEd had not provided support for these expenses that met the CCP documentation standards for reimbursement under federal grants. These standards require that the expenditure of grant funds be supported by detailed documentation, such as canceled checks, paid bills, time and attendance records, and contract and subgrant award documents. According to NYC DOEd officials, they could not meet the documentation standards established by NYS OMH because (1) NYC DOEd reorganized on July 1, 2003, which coincided with the delivery of crisis counseling services under the Project Liberty grant, resulting in significant loss of staff, loss of institutional knowledge, and therefore lost or diminished ability to retrieve supporting documentation, and (2) NYC DOEd’s complex financial systems cannot produce the type of transaction-specific documentation required by NYS OMH and makes the process of retrieving supporting documentation unwieldy and administratively burdensome.

A SAMHSA official told us SAMHSA was aware of issues involving the supporting documentation for the NYC DOEd expense claims; however, because officials viewed it as a grantee issue, they have had limited involvement with NYS OMH’s efforts to resolve these issues. NYS OMH decided to consider alternative evidence, including supplemental supporting documentation in the form of internal control summary memos prepared by NYC DOEd that describe the controls over payments for personnel, other-than-personnel, and community-based organization expenses. Personnel expenses include NYC DOEd workers while the other-than-personnel expenses include other costs incurred directly by NYC DOEd. The community-based organization expenses are those incurred by other service providers on behalf of NYC DOEd.

Although NYC DOEd’s Chief Financial Officer has signed an attestation stating that the controls described in the summary memos for personnel and other-than-personnel expenses were in place and working during Project Liberty, the level of assurance provided by these internal control summary memos is limited for several reasons. First and foremost, the memos do not provide the type of supporting documentation necessary to verify the validity of the claimed expenses as required by the federal documentation standards. Second, the memos are not certified by an external source, such as an independent auditor. Third, the memos were prepared solely to support NYC DOEd’s Project Liberty expenses and may not represent written policies and procedures that existed during the time the claimed expenditures were incurred. And lastly, the memos were prepared toward the end of the program by officials who did not, in all cases, have firsthand knowledge of the controls that existed during the program.

As of March 31, 2005, NYS OMH and NYC DOHMH had reviewed and accepted internal control summary memos that describe the controls over payments for personnel and other-than-personnel expenses, and NYS OMH had used these memos and other alternative forms of evidence to reimburse NYC DOEd for $5.2 million in expense claims. These other forms of evidence included observations of services being provided during site visits, the existence of encounter logs evidencing that some services had been provided, and general familiarity with service providers. NYS OMH officials were not sure when they would complete the review of the memo covering the controls over payments for community-based organization expenses and how this memo, along with other alternative forms of evidence, would be used to resolve the remaining $26.8 million in NYC DOEd expense claims.

As part of its approval process for expense claims, NYS OMH relied upon NYC DOHMH to certify that the claims submitted were valid and met the CCP documentation requirements. However, because NYC DOEd did not provide the required supporting documentation, NYC DOHMH could not perform the same level of review as it did for the claims of the other Project Liberty service providers. Further, although NYC DOHMH contracted with independent auditors to perform audits of expense claims of certain service providers for Project Liberty, there were no audits performed of NYC DOEd claims, which are expected to total approximately $32 million.

At the end of our audit fieldwork, it was not clear when and how the remaining expense claims would be resolved. However, if the internal control summary memos and other alternative evidence continue to be the primary supporting documentation for $32 million in NYC DOEd expense claims, the federal government will have only limited assurance that these payments are an appropriate use of Project Liberty grant funds.

Improvements Needed in the Process for Determining Federal Funding

FEMA’s process for determining funding is designed to be implemented quickly after a state requests federal assistance to recover from a presidentially declared disaster. The state of New York’s grant applications for Project Liberty were developed during the initial dynamic stages of the recovery effort when damage reports and response plans were subject to frequent change. The budgets submitted with the grant applications were revised by the grantee to satisfy certain conditions of grant award. However, we found that although the budgets were developed using estimates established during the initial stages of the disaster, FEMA and SAMHSA never required the state of New York to formally submit revised budget requests to reflect new information and significant changes to the program that occurred as the needs of the affected population became better identified. As a result, FEMA and SAMHSA did not have realistic budget information that could be used to effectively assess how responsible city and state officials planned to spend Project Liberty grant funds.

Federal Assistance Application Process

The grant applications that the state of New York submitted to FEMA for Project Liberty were prepared with assistance from FEMA and SAMHSA and included a needs assessment, plan of services, and budget. The needs assessment, which was based on a formula developed by SAMHSA, was the state’s estimate of the number of people who would need crisis counseling. The plan of services described the state’s plan for treating the identified population, including segments of the population needing special services or outreach methods such as counseling and training in various languages. The budget was developed based on the estimated cost to treat the population identified in the needs assessment through the program outlined in the plan of services.

FEMA and SAMHSA provided the state of New York the flexibility to submit grant applications that reflected its identified and estimated needs, which were based on information available at the time. In preparing the budget, the state of New York relied on SAMHSA’s Budget Estimating and Reporting Tool, which was designed to assist states in developing budgets consistent with FEMA guidelines. The state of New York took two different approaches in constructing the ISP and RSP budgets for Project Liberty. The ISP budget used estimates of administrative costs and a simple direct services cost calculation. The direct services costs were based on the estimated number of people needing crisis counseling services, the estimated average length of treatment each person would need, the estimated hourly rate for crisis counselors, and the estimated length of the ISP. The RSP budget, on the other hand, was prepared by the state of New York based on estimates provided by NYS OMH, each of the New York City boroughs, and the 10 surrounding counties eligible for CCP grant funding.

Once the state of New York submitted its ISP and RSP grant applications, FEMA had processes in place to review and approve them. Although the processes differed, both shared common elements. The first step for both applications was a technical review conducted by the FEMA regional office with jurisdiction over the state of New York to ensure that the applications had a direct link to the September 11 terrorist attacks. Once this technical review was completed, the applications were sent to FEMA headquarters and to SAMHSA for review and comment. In addition, the RSP was reviewed by a panel of mental health professionals who had experience with CCP grants. The ISP and RSP review processes also differed in that FEMA’s regional office had final decision authority for the ISP application while FEMA headquarters had final decision authority for the RSP application. Figure 7 shows the application processes for the ISP and RSP.

After the reviews conducted by FEMA and SAMHSA were completed, FEMA awarded the state of New York $22.7 million for the ISP on September 24, 2001, with subsequent amendments bringing the ISP total to $22.8 million. In addition, FEMA awarded the state of New York $132.1 million for the RSP immediately after the ISP ended on June 14, 2002.

Budgets Were Not Adjusted to Reflect New Information or Program Changes

Because FEMA’s process for determining funding is designed to be implemented quickly after presidential disaster declarations and official loss numbers were not known at the time the Project Liberty applications were prepared, the state of New York used estimates of the number of people who would need crisis counseling services, the length of the program, and the services that would be provided. However, FEMA and SAMHSA never required the budgets to be modified to reflect new information or significant changes to the program.

The estimates used by the state of New York to develop its initial needs assessment, or number of people it believed would need crisis counseling services, included several risk factors and loss categories. In keeping with existing CCP policy, FEMA and SAMHSA encouraged the state of New York to modify the needs assessment formula by adjusting the loss categories of affected persons and the risk factors for each of those loss categories to better reflect the situation in New York. The state of New York also estimated the number of direct victims in each loss category because official numbers were not available. For example, the official number of deaths was not known until more than 2 years after the disaster. As a result of these estimates, the needs assessment for the state of New York’s ISP application determined that 2.3 million people would need crisis counseling services as a result of the terrorist attacks on September 11, 2001. With the RSP application, the needs assessment formula was modified to estimate the pervasive reactions to the disaster and to update the loss category numbers, such as the number of people dead, missing, or hospitalized. These modifications increased the estimate of the people who would need crisis counseling to 3.4 million.

We found that based on the approved budgets for the ISP and RSP, Project Liberty estimated that it would need $154.9 million to provide crisis counseling and public education to the estimated 3.4 million people and training for Project Liberty staff who would be delivering these services, at a cost of approximately $46 per person. In its report for the period ending September 30, 2004, Project Liberty estimated that it had provided crisis counseling to 1.5 million people at a cost of $121 million, or approximately $83 per person.

Another estimate used in preparing the grant applications was the length of time needed to carry out the services identified in the plan of services. The state of New York used the maximum length of service provision allowed by FEMA regulations in its ISP and RSP applications, 60 days and 9 months, respectively. However, crisis counseling services were actually provided for approximately 9 months under the ISP and over 30 months under the RSP.

In addition, the state of New York initially understood that crisis counseling and public education services offered by Project Liberty would be limited to the services normally allowed by the CCP, such as short-term individual and group crisis counseling, community outreach, and training for crisis counselors. However, in August 2002, Project Liberty was authorized to adjust the program to include enhanced services and began providing these services in May 2003.

Other significant changes, which were not reflected in Project Liberty’s budget, included a reallocation from New York City’s budget to the NYC DOEd, which increased NYC DOEd’s budget from $8.9 million to $40 million and subsequent reductions of NYC DOEd’s budget to $32 million. Despite these major changes in the program, FEMA and SAMHSA did not require and Project Liberty did not prepare adjusted budgets to reflect their revised plans for meeting the needs of the victims of September 11. Therefore, New York State and City officials did not have realistic budget information to use as a tool to manage program funds, and FEMA and SAMHSA were not in a position to effectively assess the planned use of the funds.

Federal Oversight of Project Liberty’s Financial Information Was Limited

While SAMHSA provided oversight of Project Liberty’s delivery of services, it provided only limited oversight of financial information reported by Project Liberty about the cost of those services. SAMHSA received periodic financial reports but did not perform basic analyses of expenditures to obtain a specific understanding of how Project Liberty was using federal funds. In addition, as discussed above, budget information was outdated and therefore an ineffective tool to monitor actual expenditures. SAMHSA’s limited level of oversight over Project Liberty’s financial information was driven in part by its assessment that the program was not high risk, but this assessment did not fully consider the magnitude, complexity, and unique nature of the program and was not revisited even after significant program changes occurred. As a result, SAMHSA was not in a position to exercise a reasonable level of oversight to ensure that funds were used efficiently and effectively in addressing the needs of those affected by the September 11 terrorist attacks.

SAMHSA’s oversight for Project Liberty included review of service delivery information and identification of unusual items included in Project Liberty’s program reports, eight site visits, and routine communication with NYS OMH and FEMA. These oversight activities helped SAMHSA gain assurance that NYS OMH was delivering appropriate services. However, SAMHSA’s oversight of these services did not directly link with, and therefore did not provide assurance related to, financial information reported by Project Liberty. In addition to requiring Project Liberty to submit budgets to show how it planned to use federal funds, FEMA regulations also required Project Liberty to periodically submit financial reports to show how funds were actually spent. Required financial reports included quarterly expenditure reports, a final accounting of funds, and a final voucher. SAMHSA officials told us they did some high-level review of the financial information provided to determine how quickly the program was using grant funds and when the grant funds should be made available to NYS OMH. However, they did not perform basic analyses of expenditures to obtain a specific understanding of how Project Liberty was using federal funds.

We found that SAMHSA did not use financial information submitted by Project Liberty to conduct basic analytical reviews of how funds were being spent and whether this spending was consistent with the budgeted program expenditures. Table 2 illustrates a basic analysis we performed of Project Liberty’s reported and budgeted expenditures for the period June 14, 2002, through September 30, 2004, which identified significant differences by category between reported expenditures and budget.

Notwithstanding the fact that budgets were not updated for major program changes, several of these differences should have raised questions about whether Project Liberty was using federal funds within allowable categories and within its approved budget. For example, the Project Liberty personnel budget was over $93 million; however, as of September 30, 2004, over 26 months into the program that was initially planned for completion in 9 months, it had reported personnel expenditures of only about $26 million, for a difference of about $67 million. A SAMHSA official said that because of the way Project Liberty reported its expenditures, SAMHSA officials could not track its financial reports to its budget. As a result, we found that SAMHSA was not aware of the significant variations between Project Liberty’s reported expenditures and budget and did not make inquiries of Project Liberty officials to obtain an understanding of why these variations were occurring. Some of the differences between reported and budgeted expenditures may have resulted from the fact that Project Liberty was not required to formally adjust the initial program budget to reflect significant changes. However, the differences may also have raised questions about whether SAMHSA’s understanding of how the program was planning to spend funds was consistent with actual spending patterns. Comparisons between Project Liberty’s reported budget and expenditures could have helped SAMHSA better assess the status of the program to allow it to take effective action to ensure that Project Liberty was using federal funds to provide the most value for victims of the September 11 terrorist attacks.

The differences between Project Liberty’s reported budget and expenditures may also have been caused by inconsistencies in financial information submitted by Project Liberty. FEMA and SAMHSA did not provide detailed guidance on how to classify CCP expenditures but instead left Project Liberty to interpret how expenditures should be classified. We found that Project Liberty expenditures were not always consistently reported to FEMA and SAMHSA. For example, Project Liberty did not consistently classify evaluation expenditures. If an NYS OMH employee was evaluating the program, the expenditure was classified as personnel, but if the work was contracted to someone outside of NYS OMH, the expenditure was classified as evaluation. As a result, SAMHSA could not reliably use Project Liberty’s financial reports to determine how much it cost to evaluate the program. By obtaining a better understanding of how federal funds were spent by Project Liberty, SAMHSA would have improved its ability to determine whether funds were used most efficiently and effectively in carrying out the objectives of the program.

SAMHSA’s limited oversight of Project Liberty’s financial information was driven in part by its own assessment that the program was not high risk. SAMHSA’s oversight of Project Liberty included an initial assessment of the risk associated with the grantee. SAMHSA applied risk factors identified in HHS regulations regarding grants, including financial management issues, such as financial stability and experience in handling federal grants, to RFMH, the fiscal agent for NYS OMH responsible for making payments to service providers. For example, SAMHSA reviewed the result of RFMH’s fiscal year 2001 financial audit that was required by the Single Audit Act and found that RFMH received an unqualified audit opinion while handling a total of about $62 million in federal funds. SAMHSA concluded that RFMH had a strong track record for handling federal funds and classified RFMH as not high risk. Based in part on this risk assessment, SAMHSA officials told us staff with financial backgrounds were not actively involved in the oversight of Project Liberty.

However, we found that SAMHSA’s risk assessment only considered risks associated with RFMH and did not consider other potential risks associated with the Project Liberty grant. For example, the assessment did not consider all significant interactions in the complex federal, state, and local government environment that existed for Project Liberty; the amount of the RSP grant award, which was the largest RSP grant ever made by FEMA; or the geographic complexities of the program, including the size of the area affected and the diversity of the community being served. In addition, SAMHSA did not revisit its initial risk assessment even after the program encountered significant changes and challenges, including the design of the first-ever enhanced services program and the documentation issues with NYC DOEd expenses, which have yet to be resolved. As a result, SAMHSA’s level of oversight was not in line with the challenges and complexities that increased the risks associated with Project Liberty.

Based in part on its risk assessment process, SAMHSA’s oversight of Project Liberty was primarily carried out by its programmatic staff who focused on activities that did not directly link to the financial information being reported by NYS OMH. Without useful financial information, including updated budgets, and without analyses of the financial information Project Liberty was reporting, SAMHSA was not in a position to exercise a reasonable level of oversight to ensure that grant funds were effectively used to address the needs of those affected by the September 11 terrorist attacks.

Assessments of Project Liberty Ongoing

Both the state of New York and the federal government have taken steps to assess how Project Liberty delivered services. NYS OMH is conducting its own assessment of Project Liberty and partnered with NYAM to obtain information from telephone surveys. SAMHSA contracted with NCPTSD, a center within the Department of Veterans Affairs, to conduct a case study of New York’s response to the terrorist attacks on September 11, with a primary focus on Project Liberty. Both NYS OMH and NCPTSD’s overall assessments of the program were ongoing as of March 2005. FEMA plans to consider lessons learned from NYS OMH and NCPTSD when conducting its own internal review of the CCP.

NYS OMH is conducting an evaluation of Project Liberty, which includes designated funding for program evaluation. This nonstatistical evaluation consists of several components, including analysis of data collected by service providers documenting services delivered through encounter data forms, recipient feedback through written questionnaires and telephone surveys, provider feedback through written reports and staff surveys, and other initiatives. The data collected by service providers was the primary method used to assess the services delivered by the program. Based on these data, NYS OMH preliminarily found that Project Liberty had reached a large number of people affected by the September 11 terrorist attacks and that it was successful in reaching many diverse communities. NYS OMH reported that 95 percent of providers who responded to its surveys rated the overall quality of services provided as good or excellent. NYS OMH also reported that the majority of respondents to its recipient surveys indicated that they have returned to their predisaster mental health condition, a goal of Project Liberty. However, according to NYS OMH, the recipient surveys were made available beginning in July 2003 to organizations providing crisis counseling for distribution to individuals receiving services and therefore may not be representative of all Project Liberty recipients. NYS OMH did not report the number of providers who received surveys and reported a low response rate for recipients. Because the number of surveys offered to providers was not disclosed and because of low response rates for the recipient surveys, we were unable to determine the level of coverage provided by these surveys.

NYS OMH also partnered with NYAM, a not-for-profit organization dedicated to enhancing public health through research, education, public policy, and advocacy. NYAM conducted nonstatistical telephone surveys in 2001 and 2002 of New Yorkers to assess the magnitude and duration of the mental health effects of the terrorist attacks. NYS OMH worked with NYAM to assess the reach and recognition of Project Liberty by adding questions to NYAM's ongoing September 11 telephone surveys. NYAM and NYS OMH reported that 24 percent of the respondents interviewed were aware of Project Liberty and, among respondents who had heard of the program, 67 percent had a good impression of the program. However, because the sampling methodology for the NYAM phone surveys was not disclosed and because of low response rates, we were unable to determine the survey coverage.

Based on these evaluation activities, as well as their own experience with Project Liberty, NYS OMH officials have begun to identify lessons to be learned. For example, they found that emergency mental health plans and resources in place prior to September 11 were insufficient to fully respond to the mental health impact of the terrorist attacks. Much of the infrastructure needed to implement Project Liberty, such as data collection procedures and public education materials, had to be developed in the immediate aftermath of the terrorist attacks. In addition, NYS OMH found that the services covered by the CCP were not sufficient to meet the mental health needs of the minority of individuals who developed severe and persistent symptoms that substantially interfered with day-to-day functioning. Although the state of New York was given permission to develop and implement an enhanced services program to meet the needs of the more severely affected individuals, similar intensive interventions are not currently routinely included as part of the FEMA CCP. NYS OMH officials told us that when their evaluation is completed, they expect that they will have comprehensively identified best practices and obstacles encountered and that they will make recommendations to FEMA and SAMHSA for actions needed to better organize a mental health response to future disasters funded by the CCP.

SAMHSA entered into an interagency agreement with NCPTSD, a center within the Department of Veterans Affairs, to conduct a case study of New York’s response to the September 11 terrorist attacks. The primary purpose of the NCPTSD case study was to identify lessons to be learned from New York’s experience that could be useful to other communities that might have to respond to major disasters, including acts of terrorism. As part of its study, NCPTSD interviewed 103 individuals, including service providers and management from 50 public and private provider organizations in New York City and the surrounding counties. NCPTSD used a qualitative methodology to analyze the data to develop findings and recommendations. According to SAMHSA, the NCPTSD report is expected to be issued as soon as all stakeholders’ comments have been received and considered.

FEMA officials told us they plan to consider lessons learned from the NCPTSD and NYS OMH assessments of Project Liberty through FEMA’s internal review of the CCP that was ongoing as of March 2005. This internal review is being conducted in partnership with SAMHSA and, according to FEMA, will consider whether aspects of FEMA’s CCP, including its regulations and guidance, need to be improved. For example, FEMA plans to work with SAMHSA to consider the extent to which the enhanced services should be included as a permanent part of the CCP. FEMA officials told us that the internal review will have to be completed in conjunction with their primary work responding to disasters; therefore, they have not established a timetable to complete this review. Given that Project Liberty was awarded the largest RSP grant in the history of FEMA’s CCP and that FEMA provided funding to the state of New York to evaluate Project Liberty, the timely assessment of lessons learned from this program would be beneficial to future CCPs.

Conclusions

FEMA and SAMHSA’s limited oversight of the planned and actual spending of Project Liberty impeded their ability to monitor whether the grant funds were being used in the most efficient and effective way to meet the needs of those affected by the terrorist attacks of September 11, 2001. Further, until recently, FEMA and SAMHSA had limited involvement in efforts to resolve issues surrounding the outstanding NYC DOEd expense claims; additional oversight in this area could help bring appropriate and timely resolution to these issues. FEMA will have an opportunity to address these oversight issues, as well as lessons learned identified by NYS OMH and NCPTSD, as part of its ongoing internal review of its CCP.

Recommendations for Executive Action

In order to address the issues identified in our report, we recommend that the Secretary of Homeland Security direct the Under Secretary of Emergency Preparedness and Response to take the following eight actions: To help ensure proper and timely expenditure of the remaining Project Liberty funds, FEMA should work with SAMHSA to provide assistance to New York City and State officials in appropriately resolving issues surrounding the NYC DOEd expense reimbursements and determine whether an independent review of the propriety of the use of funds for payments to the NYC DOEd is needed.

To strengthen federal financial oversight of future CCP grants, FEMA should work with SAMHSA to require the recipients of CCP grants to submit updated budgets to reflect approved changes to the program; revise the current risk assessment process to comprehensively identify and assess risks associated with CCP grants; establish a process to update the risk assessment for significant consider developing formal requirements for consistent classifications of expense data; and develop formal procedures to perform more detailed analyses of financial reports, including comparing actual expenditures and budgets to identify variations and obtain an understanding of the reasons for any unusual variations.

To help ensure that the lessons learned from Project Liberty will be used to help improve future programs funded by the CCP, FEMA should establish a clear time frame to complete its internal review of the CCP as expeditiously as possible.

Agency Comments and Our Evaluation

We received written comments on a draft of this report from DHS who generally concurred with our recommendations but expressed reservations regarding our assessment of the adequacy of FEMA and SAMHSA oversight. SAMHSA in a separate comment letter to FEMA, did not object to our recommendations but did take issue with our assessment of its oversight, particularly given the unprecedented circumstances that led to the establishment of Project Liberty. SAMHSA also provided additional information on the NYC DOEd claim issue. DHS’s comment letter (reprinted in appendix II) incorporates by reference SAMHSA’s letter (reprinted in appendix III). We also received technical comments from NYS OMH, NYC DOHMH, and NYC DOEd on excerpts of the report, which we incorporated as appropriate.

DHS stated that our report should give more weight to the unprecedented conditions that led to Project Liberty, and that it was these unique circumstances that led to our findings and were the basis for our recommendations. It further stated that our recommendations primarily relate to the use of grant funds by NYC DOEd, and that no similar issues were identified with respect to the use of funds by other program subgrantees.

Our report clearly acknowledges the unique and unprecedented circumstances that led to the establishment of Project Liberty. These unique circumstances were largely the basis for our conclusion that Project Liberty required a high level of federal oversight. A number of red flags signaled the need for a heightened federal role, including: the RSP grant was the largest such grant ever made by FEMA; the program initially designed to last about a year is now over 3 ½ years old and still ongoing, with an extension being considered to September 30, 2005; and reimbursements for approximately $32 million, representing over 20 percent of the total federal funds awarded, remained unresolved as of May 2005. The fact that the level of federal oversight was not commensurate with the unprecedented circumstances surrounding Project Liberty was what led us to our findings and recommendations in this area.

Five of our eight recommendations relate to strengthening federal financial oversight of future CCP grants; two of the recommendations specifically address the NYC DOEd use of grant funds; and the remaining recommendation calls for ensuring that lessons learned from Project Liberty will be used to improve future programs funded by the CCP. Thus, DHS was incorrect in stating that our recommendations primarily relate to the NYC DOEd issues.

As to FEMA’s statement that we did not identify any other issues about the use of grant funds, the scope of our work included determining the extent to which Project Liberty expended grant funds and whether the federal government had adequate financial oversight of Project Liberty. Our work did not address whether payments made, including those made by NYC DOEd, were a valid use of federal resources. We have no basis for reaching the conclusion suggested by FEMA.

SAMHSA’s letter also discussed the unprecedented conditions surrounding Project Liberty and, as discussed below, strongly disagreed with our assessment that SAMHSA’s financial oversight was limited. SAMHSA also stated that during a recent site visit, NYC DOEd’s Chief Financial Officer indicated that documentation was available to support claims as necessary. SAMHSA further stated that it will be recommending that the NYS OMH conduct an independent audit of these claims as one of the conditions of approving an extension of the grant to September 30, 2005. During our fieldwork, we were consistently told by NYC DOHMH officials that NYC DOEd had not been able to produce documentation for the majority of expenses it incurred on behalf of Project Liberty that met the documentation standards for reimbursement under federal grants. Given ongoing questions about the existence of documentation supporting NYC DOEd claims, the sufficiency of this documentation, or both, we agree that an independent audit is appropriate.

SAMHSA stated that overall, the federal oversight of Project Liberty was appropriate, reasonable, and responsive to state and local needs. SAMHSA outlined several factors to support this statement, including that it received financial data from the state of New York on a routine basis and monitored the allocability, allowability, and reasonableness of project expenditures. While SAMHSA acknowledged that project budget data were not updated comprehensively, it stated that it did request and receive updated budget information in several instances, particularly in conjunction with project extensions. SAMHSA further stated that we had not cited any problems with program expenditures but instead seemed to focus on differences between classification of budgeted and reported expenditures. SAMHSA acknowledged that the budget was not prepared in the same format as reported expenditures, and stated that inconsistent categorization of expense accounts were largely the reason for the classification discrepancies we highlighted in our report.

Our conclusion that SAMHSA’s oversight of Project Liberty’s financial information was limited was based in large part on the fact that the SAMHSA did not have a basis to reliably monitor how Project Liberty was using federal funds, since, as SAMHSA acknowledged, it did not have updated budget information and the reported expenditure data were not accurate due to classification discrepancies. At the time of our review, SAMHSA was not aware of these discrepancies because it had not been conducting basic analyses, such as comparisons between Project Liberty’s budgeted and reported expenses. Further, there were no staff with financial backgrounds involved with the oversight of Project Liberty expenditures. SAMHSA’s limited oversight was based in part on the fact that it did not deem the project high risk from a financial standpoint, despite the complex federal, state, and local environment, the fact that this was by far the largest RSP grant ever awarded by FEMA, the size and diversity of the community being served, and the overall challenging and changing circumstances of September 11. Overall, we found that SAMHSA’s level of oversight was not in line with these challenges and complexities associated with Project Liberty.

SAMHSA went on to state that in its opinion, classification of Project Liverty as high risk simply based on total estimated project expenditures would be inappropriate. It further noted that there is no regulatory mechanism allowing SAMHSA to assess risk on a complex federal, state, and local government environment as was specified in our report.

We did not suggest that the risk classification should simply be based on total estimated project expenditures. As discussed above, our report clearly delineates a number of different risk factors that should have been considered in the risk classification. Further, we disagree that the current regulatory mechanism would not allow SAMHSA to consider these risk factors in making its risk assessment of Project Liberty. While current regulations do not require SAMHSA to consider programmatic factors in its risk assessment, they do not prevent SAMHSA from considering risk factors other than those delineated in its regulations in its overall assessment of the program and its operations. As noted by SAMHSA in its written comments, Project Liberty was by far the largest and most complex effort in the 30-year history of the CCP and presented unique and unprecedented challenges for government authorities at all levels. We believe these should have been key factors in SAMHSA’s risk assessment and should have triggered heightened financial oversight of Project Liberty.

As agreed with your offices, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days from its date. At that time, we will send copies to appropriate House and Senate committees; the Secretary of Homeland Security; the Under Secretary of Homeland Security for Emergency Preparedness and Response; the Administrator, Substance Abuse and Mental Health Services Administration; the Director, Office of Management and Budget; and other interested parties. We will make copies available to others upon request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staffs have any questions about this report, please contact me at (202) 512-8341 or calboml@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Major contributors to this report are listed in appendix IV.

Objectives, Scope, and Methodology

To determine the extent to which Project Liberty spent the Immediate Services Program (ISP) and Regular Services Program (RSP) grant funds received from the Federal Emergency Management Agency (FEMA), we did the following: Reviewed various documents, including quarterly RSP expenditure reports for the first (June 15, 2002, through September 14, 2002) through the ninth (June 15, 2004, through September 30, 2004) quarters; a detailed listing of the outstanding advance balances as of September 30, 2004, obtained from the Research Foundation for Mental Hygiene, Inc. (RFMH); a summary of expense claims submitted by the New York City Department of Education (NYC DOEd) as of March 2005; internal control summaries prepared by NYC DOEd for its personnel and other- than-personnel expenses; a draft internal control summary prepared by NYC DOEd for its community-based expenses; Crisis Counseling Assistance and Training Program (CCP) guidance on appropriate uses of grant funds; and FEMA and Department of Health and Human Services (HHS) regulations pertaining to the CCP.

Interviewed officials from FEMA’s headquarters and finance office, the Substance Abuse and Mental Health Services Administration’s (SAMHSA) Center for Mental Health Services (CMHS), the New York State Office of Mental Health (NYS OMH), RFMH, the New York City Department of Health and Mental Hygiene (NYC DOHMH), and NYC DOEd.

Determined that the total expenditures data obtained from RFMH and Project Liberty’s quarterly expenditure reports were sufficiently reliable for the purposes of this report by the following: Obtaining and reviewing a copy of the independent auditor’s report of RFMH’s financial statements for fiscal years ending March 31, 2004 and 2003, and Report on Compliance and on Internal Control Over Financial Reporting Based on an Audit of Financial Statements Performed in Accordance with Government Auditing Standards as of March 31, 2004. We determined that RFMH received a clean opinion on its fiscal year 2004 and 2003 financial statements. In addition, the auditor concluded that its tests of RFMH’s compliance with certain provisions of laws, regulations, contracts, and grants did not disclose any instances of noncompliance that are required to be reported under Government Auditing Standards. Finally, the auditor’s consideration of internal control over RFMH’s financial reporting did not identify any matters involving the internal control over financial reporting and its operation that it considered to be material weaknesses.

Analyzing a database obtained from RFMH of the payments made on behalf of Project Liberty for the ISP and RSP from the first payment made on September 25, 2001, through September 30, 2004, including advances made to service providers.

Determining that the amount of the payments included in the database was consistent with the total reported expenditures in the ISP final report and the RSP quarterly expenditure reports that were prepared by Project Liberty and submitted to SAMHSA and FEMA.

Comparing the Project Liberty expenditures as reported by RFMH to drawdowns reported by SAMHSA on Project Liberty’s RSP grant award.

Obtaining a written certification of data completeness from the Managing Director of RFMH that the expenditures reported in the database were complete and accurate for all payments made for, or on behalf of, Project Liberty for the ISP and the RSP through September 30, 2004.

Reviewing Single Audit Act reports for fiscal years 2003 and 2002 of New York City and surrounding counties.

To determine whether the federal government had an effective process in place to determine the amount of funds to provide to Project Liberty, we interviewed officials from FEMA headquarters, SAMHSA’s CMHS, NYS OMH, and RFMH; reviewed various documents, including the state of New York’s ISP and RSP grant applications, the ISP and RSP grant awards, and federal guidance for the CCP, including the Robert T. Stafford Disaster Relief and Emergency Assistance Act and FEMA and HHS regulations; and reviewed correspondence between officials from FEMA, SAMHSA’s CMHS, NYS OMH, and RFMH.

To assess federal oversight over Project Liberty’s expenditures, we obtained an understanding of CCP oversight roles and responsibilities by reviewing FEMA and HHS regulations, FEMA and SAMHSA’s fiscal year 2004 interagency agreement, CCP fiscal guidelines, HHS’s grants management manual, summary documents of the CCP’s oversight structure prepared by SAMHSA, and GAO reports; reviewed available documentation of oversight performed for Project Liberty, including Project Liberty’s financial reports and documentation of site visits conducted by FEMA and SAMHSA; analyzed Project Liberty’s financial reports and compared them to initial designed our work to assess the effectiveness of federal oversight and therefore considered but did not assess the controls over Project Liberty payments implemented at the state and local levels; interviewed officials from FEMA headquarters and the FEMA regional office that serves New York, SAMHSA’s CMHS, SAMHSA’s Division of Grants Management, NYS OMH, and RFMH to identify policies and procedures for overseeing Project Liberty; and reviewed and used Standards for Internal Control in the Federal Government as criteria.

To identify the steps that have been taken by the federal government in partnership with the state of New York to assess Project Liberty, we reviewed documentation of assessments performed, including a draft of the National Center for Post-Traumatic Stress Disorder case study of Project Liberty, NYS OMH summaries of survey results, an article written by the Deputy Commissioner of NYS OMH on lessons learned about the mental health consequences of the September 11 terrorist attacks, articles published by the New York Academy of Medicine, and documentation from FEMA related to its internal review of the CCP; reviewed various documents related to Project Liberty including the grant applications and the response to conditions of the grant award set out by FEMA and SAMHSA; reviewed GAO and FEMA Office of Inspector General reports to determine whether the CCP was evaluated; and interviewed officials from FEMA headquarters, SAMHSA’s CMHS, and NYS OMH.

We requested written comments on a draft of this report from the Secretary of Homeland Security. We received written comments from DHS. The DHS comments (reprinted in app. II) incorporate by reference a letter from SAMHSA to FEMA commenting on the draft (reprinted in app. III). We also provided excerpts of a draft for technical comment to NYS OMH, NYC DOHMH, and NYC DOEd. NYS OMH technical comments and the coordinated NYC DOHMH and NYC DOEd technical comments are incorporated as appropriate.

We performed our work from July 2004 through March 2005 in accordance with generally accepted government auditing standards.

Organizations Contacted

Federal Agencies

State of New York

New York State Office of Mental Health Research Foundation for Mental Hygiene, Inc.

New York City

Comments from the Department of Homeland Security

Comments from the Substance Abuse and Mental Health Services Administration

GAO Contact and Staff Acknowledgments

GAO Contact

Linda Calbom, (202) 512-8341.

Acknowledgments

Robert Owens (Assistant Director), Donald Neff (Auditor-in-Charge), Lisa Crye, Edward Tanaka, and Brooke Whittaker made key contributions to this report.