text: Background FEMA’s Public Assistance Process The Robert T. Stafford Disaster Relief and Emergency Assistance Act (Stafford Act), as amended, defines FEMA’s role during disaster response and recovery. One of the principal programs that FEMA operates to fulfill its role is the PA program. PA is a complex and multistep grant program administered through a partnership between FEMA and states, which pass these funds along to eligible local grant applicants. Thus, PA entails an extensive paperwork and review process between FEMA and the state based on specific eligibility rules that outline the types of damage that can be reimbursed by the federal government and steps that federal, state, local, territorial, and tribal governments as well as certain nonprofit organizations must take in order to document eligibility. The complexity of the process led FEMA to re-engineer the PA program, which FEMA has referred to as its “new delivery model.” FEMA began testing the new delivery model at select disaster locations in 2015, in preparation for implementing it nationwide for all new disasters. On September 12, 2017, FEMA announced that the new delivery model would be used in all future disasters unless determined infeasible in a particular instance. The process begins after FEMA determines that the applicant meets eligibility requirements. FEMA then works with the state and the applicant to develop a project worksheet describing the scope of work and estimated cost. Once FEMA and the applicant agree on the damage assessment, scope of work, and estimated costs, the PA grant obligation is determined. After FEMA approves a project, funds are obligated—that is, they are made available—to the state recipient, which, in turn, passes the funds along to applicants. Applicants may appeal project decisions if they disagree with FEMA’s decisions on project eligibility, scope of damage, or cost estimates. Appealable decisions can occur at various times during the PA grant process, including during project closeout as long as they meet applicable time limits. FEMA’s PA Appeals Process Figure 2 summarizes the first- and second-level appeals process under FEMA’s PA program. The first-level appeals process begins after FEMA makes its determination on a project for PA grant assistance. Within 60 days of receiving FEMA’s initial determination, the applicant must file an appeal through the state to the relevant FEMA regional office. The state must forward the appeal and a written recommendation to the relevant FEMA regional office within 60 days. In reviewing the first-level appeal before forwarding it to FEMA, the state has discretion to support or oppose all or part of the applicant’s position in the appeal. Under the Stafford Act, the FEMA regional office shall render a decision within 90 days from the date it received the first-level appeal from the state. The PA appeals process can take longer if regional officials issue a request for information (RFI) to the applicant or request technical advice from subject-matter experts. According to a senior PAAB official, a regional office may issue an RFI or seek technical advice when an applicant’s appeal is incomplete, lacks referenced documentation, or raises additional eligibility concerns. The regional office may issue multiple RFIs prior to rendering a final decision on an appeal. Within 90 days following the receipt of the requested additional information or following expiration of the period for providing the information, FEMA is to notify the state in writing of the disposition of the appeal. Regional Administrators are responsible for authorizing a final decision on a first-level appeal. A decision may result in an appeal being granted, partially granted, or denied. An appeal is considered granted when FEMA has approved the relief requested by the applicant as part of the appeal. An appeal is considered partially granted when FEMA has approved a portion of the relief requested by the applicant. An appeal is considered denied when FEMA has decided not to approve the relief requested by the applicant. If the regional office is considering denying or partially granting a first appeal, it must issue an RFI to provide applicants with a final opportunity to supplement the administrative record (i.e., the documents and materials considered in processing a first-level appeal), which closes upon issuing a first-level appeal decision. According to a senior PAAB official, this process adds additional time to first-level appeal processing, but ensures that FEMA has considered all relevant and applicable documentation. The applicant may file a second-level appeal through the state within 60 days of receiving a first-level appeal decision. The second-level appeal must explain why the applicant believes the original determination is inconsistent with law or policy and the monetary amount in dispute. The state then has 60 days to provide a written recommendation to FEMA. In reviewing the second-level appeal, just as with the first-level appeal, the state has discretion to support or oppose all or part of the applicant’s position in the appeal. The FEMA Assistant Administrator for Recovery or the PA Division Director through a delegation of authority shall render a decision within 90 days of receipt of the second-level appeal from the state. All second-level appeal decisions are posted to FEMA’s website, so applicants can review the previous decisions. As is the case with first-level appeals, the PA appeals process can take longer if PAAB officials request additional information or technical advice on an appeal. These requests must also include a date by which the information must be provided. According to a senior PAAB official, RFIs are seldom issued for second-level appeals because the administrative record is closed after a decision is rendered on a first-level appeal. Similarly, this official told us that technical advice is rarely sought for second-level appeals because such issues are typically explored during the first-level appeal process. Within 90 days following the receipt of the requested additional information or following expiration of the period for providing the information, FEMA is to notify the state in writing of the disposition of the appeal. FEMA’s response to a second- level appeal is the last and final agency decision in the appeals process. Organization of FEMA’s PA Appeals Program Located within the Recovery Directorate, PAAB maintains overall responsibility for administering and overseeing FEMA’s PA appeals program. Among other things, PAAB is responsible for ensuring that all appeal decisions are issued within regulatory timelines by developing and maintaining SOPs; arranging for supplemental staff support as needed; providing regular updates for both first- and second-level appeal decisions through a range of communications; and providing training to certify PA program staff on appeals processing. PA program appeals staff in each of FEMA’s 10 regional offices are responsible for processing first-level appeals, while PAAB staff in FEMA’s Headquarters office are responsible for processing second-level appeals. Accordingly, each regional office is required to follow FEMA’s Directive, Manual, and Regional SOP for processing first-level appeals, consistent with those established for second-level appeals. FEMA regional offices are also required to forward all incoming second-level appeals to PAAB. In addition, regional office staff must, within 3 business days of receiving a first-level appeal from a state, provide an electronic copy of the appeal to the PAAB via FEMA’s shared workspace SharePoint site. As noted in FEMA’s Recovery Directorate Appeals Manual, this step enables PAAB staff to identify and track appeals issues and trends in development across all FEMA regions. The roles and responsibilities for both first-and second-level appeals are defined in FEMA’s SOPs. For example, certified appeals analysts are responsible for reviewing incoming appeals for completeness, researching and drafting appeal decisions, and generating RFIs. Lead appeals analysts are the first-line reviewers of appeal decisions and RFIs, and provide guidance on PA program and policy issues, coordinate appeals assignments, and review work of appeals analysts. Further, appeals coordinators are responsible for receiving incoming appeals, tracking the processing of those appeals, updating the appeal status, and processing other appeals-related correspondence and reports. Prior Reviews Examining the PA Appeals Process We have identified a number of issues related to FEMA’s management of the PA appeals program in our prior audit work, as has DHS’s OIG. In our 2008 review of FEMA’s administration of the PA program following Gulf Coast Hurricanes Katrina and Rita, we identified challenges related to applicants’ experience with appeal processing delays and that FEMA often did not make decisions on appeals within the 90-day statutory time frame. Other challenges identified were that FEMA did not inform some applicants of the status of their appeal, or, in some cases, assure them of the independence of the FEMA officials making appeal decisions. Specifically, some applicants perceived there to be a conflict of interest because the PA program staff responsible for reviewing appeals was the same staff that had made the PA project decision that was being appealed. We did not make recommendations to FEMA to address these challenges in our 2008 review, but rather described the challenges as part of the status of overall Gulf Coast hurricane recovery efforts. In 2011, DHS’s OIG conducted a review of FEMA’s PA appeals process and made a number of recommendations aimed at improving aspects of the process, including the timeliness of appeals processing, appeals staffing, and the accuracy of appeals data. As in our 2008 review, the OIG identified appeal processing delays occurring at both FEMA regional offices and at headquarters. For example, the report found that appeals were left open for long periods of time and that some regional offices as well as FEMA headquarters took more than 90 days to issue a decision on first- and second-level appeals. Further, the OIG review found that staffing approaches employed by individual regional offices contributed to processing delays and varying processing timeframes. For example, the management and processing of first-level appeals varied by FEMA regional office in that some regional offices assigned staff specifically to review appeals, while other offices assigned staff to appeals processing as part of their other responsibilities within the PA Program, such as determining eligibility for PA assistance. Further, second-level appeals were processed by various offices within FEMA headquarters, and FEMA had not established guidelines to complete work within a specific timeframe. Moreover, the OIG review found inaccuracies with FEMA’s system for tracking appeal processing times for second-level appeals, resulting in unreliable information being reported to FEMA management regarding compliance with the 90-day statutory time frame. Lastly, the OIG reported that some applicants had been unable to obtain information on the status of their appeals and that FEMA did not provide meaningful feedback to resolve applicants’ inquiries. Weaknesses Exist in FEMA’s Oversight of Data Quality, but Corrected FEMA Data Showed Fluctuations in Appeal Inventory and Delays in Processing Our review of FEMA data that track first- and second-level appeals showed weaknesses in the agency’s data quality practices that affect program oversight. For example, we found that FEMA regional offices do not track first-level appeals data consistently or update this data regularly, resulting in missing data entries. Further, we found that FEMA’s appeal tracking process does not ensure data quality, limiting FEMA’s ability to use the data for making decisions on and improvements to the PA appeals process. During our review, we discussed with FEMA officials the discrepancies we found with these appeals data. FEMA officials acknowledged these data quality issues and provided us with corrected data to address these discrepancies for our analysis in this report. Our analysis of the corrected FEMA data showed that, between January 2014 and July 2017, FEMA received over 1,400 first- and second-level appeals with amounts in dispute totaling about $1.5 billion. Across all years, first- level appeals accounted for the majority of appeals, though the number of appeals fluctuated widely each year. Over the same period, only a small percentage of first-and second-level appeals were processed within the 90-day statutory time frame. Weaknesses in FEMA’s Tracking and Data Quality Practices Affect Program Oversight To administer and oversee the PA appeals program, FEMA collects and tracks information on first- and second-level appeals. Based on FEMA’s SOP, the agency uses this information to identify trends throughout the appeals process and identify areas in need of improvement. Specifically, PAAB uses two Excel spreadsheets for collecting and analyzing first- and second-level appeals data. The spreadsheet for collecting second-level appeals data is updated and maintained by PAAB, while the spreadsheet for first-level appeals is based on input from FEMA’s 10 regional offices. Based on our detailed review of the spreadsheets, they contain numerous data fields on the status and outcomes of first-level appeals, such as the date the regional office received the appeal, the date an RFI was issued, the date the Regional Administrator signed the decision, the amounts being disputed by the applicant, and keyword information regarding the subject of the appeal. PAAB requests that regional offices update appeal information in the first- level appeal spreadsheet as changes occur on an appeal. PAAB then uses this data to assess trends in regional office appeals processing, which it includes in various performance and other internal reports that are shared with FEMA management and used to monitor the program. According to PAAB officials, such information provides valuable support to PAAB as well as the PA program by sharing information about filings, progress, and PA program decision making. However, while PAAB’s tracking efforts help maintain visibility over and provide some monitoring of the appeals processing, we found that data fields for first-level appeals were not consistently reported or updated and that PAAB has no processes to ensure the quality of these data. As a result, data on first- level appeals may not have the accuracy needed for effective reporting and oversight efforts. FEMA Regional Offices Do Not Track Appeals Information Consistently or Update First- Level Appeal Information Regularly Our review of first-level appeals data showed that, between January 2014 and July 2017, regional offices did not consistently report first-level appeal information for a number of the key data fields in the PAAB first- level appeal tracking spreadsheet. Specifically, we found missing entries for the majority of the spreadsheet’s 50 data fields. For example, we found that about one-third of the time, regional offices had not completed the data field for amounts being disputed by the applicant for pending appeals or indicated whether or not money was in dispute in the appeal. We also found that the regional offices had generally not entered the date that the regional appeal staff had completed an initial review of the appeal—99 percent of entries were missing for this field. In another example, the data field that captures keywords was missing in over 33 percent of data entries. PAAB officials told us that keywords are an important tool for understanding the root causes of an appeal. Further, we found a number of missing data entries for key dates for one regional office in particular. Specifically, this office had not recorded entries for any of the data fields related to key dates in the appeal process, such as the date the first-level appeal was assigned to an appeals analyst, the date the appeal was reviewed by the Regional Administrator, and the date the first-level appeal decision was sent to and received by the applicant. PAAB officials told us that PAAB uses these dates to calculate appeal processing times as part of its effort to evaluate trends in appeal information and identify potential areas for improvement, including timeliness. However, officials from this regional office told us the office does not consistently update information in the PAAB first-level appeal tracking spreadsheet and does not consider it a priority. Rather, the office considers the actual processing of first-level appeals a priority. In addition, our analysis of first-level appeals data also showed that there was limited standardization of recording entries within fields. For example, officials in one of the three regional offices in our review told us that, in some instances, they combine first-level appeals that involve direct administrative costs and record them as a single appeal. However, the other two regional offices in our review told us they do not combine individual appeals that involve direct administrative costs. Rather, they count each as a separate appeal. The lack of standardization in the way appeals are counted could result in some types of appeals being over- or under-reported. More specifically, these inconsistencies may affect PAAB’s ability to compare appeal processing capacity between regional offices and accurately report the regions’ performance. FEMA’s Appeal Tracking Process Does Not Ensure Data Quality PAAB officials acknowledged inconsistencies in first-level appeals reporting, but noted that under FEMA’s SOP, the regional offices are responsible for entering first-level appeal information. According to PAAB officials, this responsibility is emphasized during training sessions with appeal staff. However, we found that FEMA has no automated data entry checks for information the regions enter into PAAB’s first-level appeal tracking spreadsheet and does not monitor data fields for missing or conflicting data. Regional offices do not have a means for electronically uploading first-level appeal information to PAAB and must manually input data into the spreadsheet. PAAB’s process then simply confirms receipt of the information through an email exchange with the regional office staff who manually input the information. PAAB officials told us that they rely on regional office appeal staff to confirm and validate the first-level appeals data that are provided to PAAB for internal reporting. However, PAAB has no independent and consistent method of verifying the accuracy of the appeals data reported to it by the regional offices. PAAB officials also noted that there is no systematic process or method to identify these errors and generate an error report. Moreover, another limitation that we identified in the spreadsheet used by the regional offices is that it is not clear what blank data fields represent— that is, whether data does not exist or whether data that exists were not recorded. PAAB officials acknowledged that blank data fields in the first- level appeal tracking spreadsheet created reporting challenges, such as whether the data field was not applicable to a particular appeal, the appeal staff for a particular region did not collect this information, or existing information was not recorded. We also identified a number of other data entries that were erroneously recorded as first-level appeals. Specifically, the information entered related to requests for adjustments to PA project funding and should not have been entered into the tracking spreadsheets as appeals. Standards for Internal Control in the Federal Government advises management to process data into quality information that is appropriate, current, complete, accurate, accessible, and provided on a timely basis. Additionally, management should evaluate processed information, make revisions when necessary so that the information is quality information, and use the information to make informed decisions. By developing and implementing processes and procedures to ensure a uniform and consistent approach for tracking first-level appeals data and better integrating regional trackers with PAAB’s own first-level appeals tracker, PAAB will have greater assurance that it is collecting the comprehensive and complete appeals processing performance information it needs from the regional offices. Further, by identifying data discrepancies and other anomalies in its data queries and the resulting datasets, PAAB may be able to identify overall weaknesses in its data recording process, thereby allowing it to more accurately report on first-level appeals information. Without obtaining quality appeals data, FEMA will not be able to identify existing gaps in its appeals information and address areas in need of improvement, such as meeting statutory timeframes. Corrected FEMA Data Showed Fluctuations in Appeal Inventory After we shared our concerns about the appeals data with FEMA officials, they corrected the errors in their data and provided us a corrected data set to use for our analysis in this report. Based on our analysis of this corrected data we determined that, from January 2014 to July 2017, FEMA received over 1,445 first- and second-level appeals with amounts in dispute totaling about $1.5 billion. Across all years, first-level appeals accounted for the majority of appeals, though the number of appeals fluctuated widely between years. (See figure 3.) FEMA officials told us that the number of appeals they received has varied year to year and that increases or decreases in appeals are largely a function of the number of and severity of disaster events. That is, the greater the number of disasters declared and the more extensive the damage, the greater the number of PA program grants FEMA may issue to applicants, which in turn, may affect the likelihood that an applicant will appeal a FEMA decision regarding a grant. FEMA issued a decision on 953 of the appeals it received between January 2014 and July 2017. As shown in table 1, another 349 appeals were pending and awaiting a decision as of July 2017. The remaining 143 appeals were withdrawn by the applicant during the appeals process. Our analysis of the corrected FEMA data also found that, for appeals received between January 2014 and July 2017, total first- and second- level pending and decided appeals involved amounts in dispute totaling over $1.3 billion (excluding the 143 appeals that were withdrawn by the applicant during the appeals process). As shown in figure 4, at least a third of both first-and second-level pending and decided appeals (35 percent and 44 percent, respectively) involved amounts in dispute that ranged from $1 to $99,999. Less than 10 percent of both first- and second-level pending and decided appeals (9 percent and 8 percent, respectively) did not involve monetary amounts in dispute. In rendering a final decision on an appeal, FEMA can grant, partially grant, or deny the appeal. Our analysis showed that FEMA granted nearly a third of the 779 first-level appeals filed, awarding applicants over $85 million. As shown in figure 5, FEMA also partially granted about 19 percent of first-level appeals filed, which involved amounts in dispute totaling over $63 million. Further, figure 5 shows that over one-third of the 174 second-level appeals were either granted or partially granted. Specifically, FEMA granted about 26 percent of second-level appeals filed, awarding over $43 million, while the agency partially granted about 7 percent of second-level appeals filed, involving amounts in dispute totaling almost $19 million. FEMA Exceeded Statutory Processing Times Our analysis of the corrected FEMA appeal data showed that, on average, FEMA took more than three times the 90-day statutory time frame to process an appeal, which includes rendering a decision. Specifically, for first- and second-level appeals that FEMA received between January 2014 and July 2017 and that FEMA decided during the same period, FEMA’s average processing time was 297 days. The processing time for decided first-level appeals averaged 293 days, while the processing time for decided second-level appeals averaged 313 days. Further, as shown in figure 6, only a small percentage of decided first-and second-level appeals (9 and 11 percent, respectively) were processed within the 90-day statutory time frame. For pending appeals, we found that, at the time of our analysis in July 2017, FEMA had taken on average, more than three times the 90-day statutory time frame for rendering decisions. Specifically, as of July 2017, FEMA had not rendered a decision on 349 appeals, which had an average processing time of 299 days. As of July 2017, the processing time for pending first-level appeals averaged 306 days, while the processing time for pending second-level appeals averaged 267 days. Figure 7 shows the ranges of processing times as of July 2017 for both first-and second level pending appeals. Officials from PAAB and the three regional offices in our review acknowledged that they experienced challenges processing appeals within the 90-day statutory time frame. They told us that issuing RFIs to the applicant can contribute to lengthy processing delays. According to PAAB officials, issuing an RFI may contribute to long processing periods if the information relates to a complex appeal—for example, an appeal involving multiple engineering issues. An appeal decision can also be delayed if FEMA issues an RFI because an applicant submitted incomplete documentation to support an appeal. Under FEMA regulation, these requests do not count against processing times and the 90-day time frame in which FEMA can render a decision on an appeal. However, our analysis of the corrected FEMA data showed that FEMA exceeded its statutory time frames even when it did not issue an RFI. Specifically, between January 2014 and July 2017, FEMA issued an RFI in about 59 percent—or 560—of the 953 first- and second-level appeals for which it rendered a decision. In 48 percent (267) of those decided appeals, FEMA had issued the RFI after the 90-day time frame had elapsed. FEMA did not issue RFIs for about 41 percent (393) of decided first- and second-level appeals. In 78 percent (305) of those appeals, FEMA’s processing time still exceeded the 90-day statutory time frame. State emergency management officials from five of our six selected states told us that they experienced long wait times for first- and second-level appeal decisions and that FEMA rarely processed appeals within the 90- day time frame required by statute. State emergency management officials further told us that such delays adversely affect applicants, such as municipalities and localities, which may wait prolonged periods to resolve project eligibility and costs related to rebuilding efforts. Delays in FEMA’s decision making may also result in additional costs to both the state and the applicant, according to these officials. For example, the state may pursue funding from an applicant if FEMA decides to deobligate funds from the applicant for PA projects that have already been completed. As discussed earlier in this report with respect to the PA process, FEMA may do this if it finds that the applicant did not meet certain PA project requirements. In these instances, the applicant may appeal FEMA’s decision, but the state may need to begin administrative proceedings against the applicant to recover or offset the deobligated funds. One state emergency manager told us that some applicants withdrew their appeals because of the prolonged delays in receiving a final decision. According to state emergency management officials, delays in FEMA’s appeal decisions can create significant challenges for local government entities, such as counties and school districts. Officials from one state provided an example of a rural school district that sought PA funding to bus displaced children who had been left homeless from damage caused by Hurricane Irene. According to relevant federal and state documents these officials provided us, these children had been moved to shelters outside of their school district and needed transportation to be able to attend school. The school district applied to FEMA for transportation costs associated with hiring an additional bus driver to bus the children to the schools in the district. FEMA denied the school district’s request, based on its interpretation of the Stafford Act and the eligibility of costs related to emergency public transportation. The district subsequently filed a first-level appeal in November 2015. FEMA took over a year to issue a decision and, in December 2016, denied the district’s first-level appeal. State management officials told us that incurring these unanticipated transportation costs while waiting for FEMA to decide the appeal has a major effect on the school district and the community as a whole, and can lead to the elimination of school programs or staff. The school district subsequently filed a second-level appeal in February 2017. FEMA denied the appeal in August 2017. State emergency management officials we interviewed provided an additional example wherein a small town had applied for PA grant funding to rebuild a retaining wall and roadway following damage caused by Hurricane Irene. According to relevant federal and state documents officials provided us, the overflowing banks of a tributary caused a retaining wall, which protected a nearby roadway, to wash away. The roadway, which provided access to residential properties near the tributary, was significantly damaged, due to the overflow. The town requested funding to repair the roadway and to replace and extend the retaining wall another 250 feet beyond the original wall in order to protect the roadway from future flood events. FEMA approved the PA funding to repair the roadway. However, FEMA denied the town’s application for PA assistance to extend the wall beyond its original length. In doing so, FEMA concluded that the proposed work was ineligible for assistance because it significantly changed the retaining wall’s predisaster configuration and that such a change constituted an improved project, making it ineligible under FEMA regulations and policy. The town then filed a first-level appeal in April 2014. More than 2 years later—in June 2016—FEMA denied the town’s first-level appeal, upholding FEMA’s original determination. The town subsequently filed a second-level appeal in September 2016. Over a year later, PAAB was still reviewing the appeal. FEMA Has Taken Steps to Improve Appeals Processing, but Faces Challenges with Its Appeals Workforce FEMA has taken a number of steps to improve its management of the appeals process and respond to issues raised by us and the DHS OIG related to processing delays. As we presented earlier in this report, our 2008 review, and DHS’s subsequent 2011 OIG review, identified a number of organizational and procedural issues related to processing delays, staff independence, and communications with applicants. Responding to these issues, FEMA created the PAAB within the Recovery Directorate at FEMA Headquarters in late 2013, adding an auditing component to the Branch in 2014. PAAB then established a core of full-time staff at FEMA headquarters that were specifically assigned to process second-level appeals. At the same time, through the Recovery Directorate, each of FEMA’s 10 regional offices was assigned full-time staff for processing first-level appeals. Prior to PAAB, second- level appeals were processed by various offices within FEMA headquarters, while the management and processing of first-level appeals varied by FEMA regional office. Some regional offices assigned staff specifically to review appeals, while other offices assigned staff to appeal processing as part of their other responsibilities within the PA Program, such as determining eligibility for PA assistance. In standing up PAAB, FEMA also established an SOP that describes the organizational structure of PAAB, as well as its responsibilities and the roles of its staff. The SOP also addresses procedures related to PAAB’s responsibility for managing the entire PA appeals program. These responsibilities include reporting on appeal processing performance, providing training to appeals staff, and identifying PA appeal process and policy improvements. FEMA later issued a regional SOP that included procedures to help regional offices reduce the number of appeals that exceeded statutory time frames. These procedures reflected an ongoing effort to leverage internal resources when regional offices exceed processing capacity. Specifically, a regional office can submit a request to PAAB for assistance from analyst staff from other regions or from PAAB to assist with processing first-level appeals. PAAB may then temporarily assign an appeals analyst from PAAB or from another regional office to assist the regional office making the request. For example, one regional office official told us his office had requested assistance with 10 first-level appeals and PAAB was able to accommodate the request by assigning 8 of the 10 appeals to another region for processing. According to a senior PAAB official, this procedure allows FEMA to maximize use of its national appeal processing capacity. As of October 2017, PAAB had transferred 77 appeals from overwhelmed regional offices to those with capacity to process additional appeals. Further, FEMA procedures now require that a conflict check be performed to determine whether the analyst was involved with a PA project determination that is substantively related to the appeal. If a conflict is identified, options include disqualifying the appeals analyst from working on the appeal, or requesting the appeal be transferred to another regional office or PAAB for processing. State emergency management officials from five of the six states in our review told us that they believed that issues related to the independence of appeals staff had been addressed and were no longer an issue. PAAB also took steps to improve communication with applicants by creating an online second-level appeal tracking spreadsheet—accessible through the Internet—intended to provide applicants with information on the status of second-level appeals. The spreadsheet includes, among other things, the date the appeal was received by FEMA headquarters, the date that an RFI was sent to the applicant, whether the appeal was “under review,” whether a final decision had been granted, and the date any final decision was signed. FEMA also took steps to increase its staffing levels. In January 2015, FEMA’s Recovery Directorate completed a workforce analysis and determined that additional appeals analysts were needed to address capacity issues that were resulting in growing inventories of first-level appeals. At the time, FEMA concluded that, in addition to its 23 on-board appeals analysts, an additional 29 appeals analysts were needed to support the existing, as well as anticipated, appeal inventory increases across FEMA’s 10 regional offices. The Recovery Directorate requested and was subsequently authorized the additional appeals analyst positions, which, when filled, would provide the PA appeal program with a total of 52 first-level appeals analysts. With the exception of Region I, FEMA planned to provide each of the remaining 9 regional offices with at least 1 additional appeals analyst. Regional offices with the heaviest workloads, such as Region II and Region IV, would be allocated more appeals analysts. FEMA took steps to fill these positions over the next 2 years, and by June 2017, FEMA had filled 47 of the 52 positions. Despite efforts to improve its management of the appeals process, FEMA faces a backlog of both first- and second-level appeals among the three selected FEMA regional offices as well as PAAB. According to officials in PAAB and the three regional offices in our review, workforce challenges contribute to delays in processing PA appeals, even with the improvements described above. PAAB and the three regional offices in our review identified the following workforce challenges that contributed to PA appeal processing delays. Staff vacancies, inexperience, and turnover: Despite FEMA’s efforts to increase its appeals analyst staffing level—an effort that began in 2015—two of the three regional offices in our review had a number of vacancies for these positions through June 2017. PAAB and regional officials told us that such vacancies, which occurred over a prolonged period, contributed to appeal processing delays. FEMA data on appeals analyst staffing show that FEMA took nearly 2 years to fill the additional appeals analyst positions across its 10 regional offices. For example, in 1 of the regional offices in our review, 3 of the 8 appeals analyst positions were vacant through 2016 and were not filled until July 2017. Further, officials in this regional office told us that the current staffing level of 8 appeals analysts was inadequate to keep pace with the region’s increasing appeal inventory. Similarly, 6 of PAAB’s 11 appeals analyst positions were vacant from August 2015 to October 2016. By July 2017, PAAB had filled all but 2 appeals analyst positions. PAAB officials told us the appeals analyst staffing level consisting of 52 positions was a preliminary estimate and that this staffing level has not been adequate in regions with heavy workloads and appeal inventories. PAAB officials also acknowledged the potential benefits of having an appeals analyst staffing plan, but stated that they are not yet prepared to update the workforce assessment for PAAB and the regional offices, nor do they have plans to do so until full staffing is achieved. These officials also told us that they are still working to achieve the staffing levels developed in 2015 and are taking steps to address staffing challenges through more targeted hiring and use of career ladder positions. Further, PAAB staffing data showed that almost half of PAAB’s staff had less than 1 year of experience. PAAB officials told us that prior vacancies and a large number of inexperienced staff have contributed to processing delays and second-level appeal backlogs. PAAB officials also told us that retaining trained appeals analysts has been challenging due to limited career advancement opportunities within the appeals analyst position. These officials told us that although not required, individuals who typically apply for an appeals analyst position possess a law degree, and that once hired, some of them apply for attorney positions within PAAB or in various offices within FEMA or DHS. For example, PAAB staffing data showed that within 18 months of being hired by PAAB, four PAAB appeals analysts applied for and were subsequently hired as attorney-advisors within PAAB or other FEMA departments. Then those appeals analyst positions were vacant until the next round of hiring. Regional officials told us it has been challenging to find qualified applicants with the specialized skillset of an analyst position. They told us that, ideally, an appeals analyst should be an expert in the PA program and possess a nuanced understanding of the legal issues associated with the program’s requirements. Regional officials told us that, because of this specialized skillset, they look to recruit PA appeals analysts from other FEMA regional offices who may have an interest in relocating or are seeking a promotion. However, while recruiting appeals analysts from other regions may assist individual offices, it does not address FEMA’s goal of achieving its staffing levels. Delays in training appeals staff: FEMA requires that PA appeals analysts undergo a certification course that includes 3 days of training on processing appeals. The appeals analyst certification course, delivered through PAAB, covers both procedural steps of processing appeals as well as the policy and legal issues raised by the PA program, and ensures that trainees can prepare a well- written appeal response. After completing the course, an analyst in training must pass a test to demonstrate proficiency in reviewing and analyzing appeals and preparing appeal decisions. To this end, the analyst must analyze a mock appeal—based on facts similar to those presented in a previously decided appeal— and draft an appeal decision. FEMA policy states that only certified staff can serve as appeals analysts and must be recertified every 2 years. However, some appeals analysts in the regional offices in our review had not yet undergone the certification process, but were nonetheless working in an appeals analyst capacity under the supervision of certified analysts. PAAB procedures also state that a trainee analyst cannot assume work on an appeal without being supervised by a certified analyst. For example, in one regional office, four of the office’s nine appeals analysts had been working in their positions for between 6 months to a year before they received appeals analyst certification training. According to regional officials, this increased the supervisory workload on the remaining five appeals analysts within the region and the lack of timely training and certification of appeals analysts affect the efficient processing of appeals and can lead to delays in FEMA issuing appeal decisions. Deployment of appeals staff to disaster response: According to PAAB officials, while PA appeals analysts are considered “dedicated” positions, these analysts can be deployed at any time to provide assistance on a disaster, such as working with grant applicants to document damages or assisting applicants in developing project proposals to request PA grants. Officials from two of the three FEMA regional offices in our review told us that these deployments contributed to processing delays because, given limited resources, assigning staff to continue work on the appeal is not always possible. In one regional office, five of the nine PA appeals analysts were deployed in late 2016 to do recovery work related to damage from Hurricane Matthew. These deployments lasted approximately 30 to 90 days and left the regional office understaffed. Further, one regional office official told us that maintaining continuity in processing an appeal can be difficult for those analysts who are deployed because they must pick up where they left off on their assigned appeals upon their return. A senior PAAB official told us that regional appeals analyst staff have been deployed to assist with response and recovery efforts as a result of the catastrophic damage from Hurricanes Harvey, Irma, and Maria. As a result, these analysts have not been available to process first-level appeals. This official further told us that PAAB staff, including analyst staff—while not deployed—have been assigned to support disaster operations. For example, one staff member was assigned to support site inspector training, while two others were assigned to stand National Response Coordination Center watch. Further, one staff member was assigned to support training and contract review functions and the remaining staff members were assigned as call takers for the PA Grants Manager and Grants Portal hotline. To help overcome staffing shortages, according to FEMA documents, all three regional offices in our review staffed assistance from PAAB at various times during the past 2 years. However, officials from two of the three regional offices in our review told us that, based on their experiences, requesting staff from PAAB or other offices had a number of limitations. Specifically, because the originating regional office is ultimately responsible for the appeal, its staff must continue to oversee the appeal, including such responsibilities as tracking the appeal, corresponding with the applicant and the state as needed, and reviewing and approving the appeal decision. One regional office official told us that this arrangement was not helpful and only added an additional layer of complexity that delayed processing. Another regional official told us that the quality of the borrowed staff’s work was not consistent. This official further stated that, because offices are not able to select the analysts that would be assigned to work on their appeals, he was reluctant to use staff from other regional offices. According to leading human capital practices, the key to an agency’s success in managing its programs is sustaining a workforce with the necessary knowledge, skills, and abilities to execute a range of management functions that support the agency’s mission and goals. Achieving such a workforce depends on having effective human capital management through developing human capital strategies. Such strategic workforce planning includes the agency assessing current and future critical skill needs by, for example, analyzing the gaps between current skills and future needs, and developing strategies for filling the gaps identified in workforce skills or competencies. Standards for Internal Control in the Federal Government also states that agencies should continually assess their needs so that they are able to obtain a workforce that has the required knowledge, skills, and abilities to achieve their organization’s goals. Further, as we have previously reported in our work on strategic workforce planning, such staffing assessments should be based on valid and reliable data. However, FEMA has not developed a workforce staffing plan to identify hiring, training, and retention needs of appeals staff across PAAB and the regional offices. PAAB officials told us that they are still working to achieve the staffing levels developed in 2015 and are taking steps to address staffing challenges related to retention through more targeted hiring and use of career ladder positions. In the absence of a workforce plan for the PA appeals staff, FEMA will likely continue to experience workforce challenges including vacancies in key appeals analyst positions, appeals staff turnover, training delays, and understaffing due to disaster deployment. These challenges will likely continue to contribute to delays in FEMA’s processing and issuing first- and second-level PA appeals decisions. FEMA Established Goals and Measures to Assess Second- Level Appeal Processing, but Did Not Do So for First- Level Appeals FEMA officials have acknowledged the importance of establishing goals and measures to assess the performance of the PA appeals program. In particular, for fiscal year 2016, FEMA’s Recovery Directorate established two performance goals for PAAB’s processing of second-level appeals. The first goal was aimed at reducing the inventory of second-level appeals by 20 percent. The second goal was aimed at processing at least 30 percent of second-level appeals received in 2016 within 90 days of receiving the appeal, in order to comply with FEMA statutory time frames. FEMA internal documents showed that these two performance goals were intended to reduce the second-level appeal inventory, and, at the same time, promote a standard of timely second-level appeal processing for PAAB. According to PAAB officials, various factors beyond PAAB’s control prevented PAAB from meeting these performance goals. These factors included an unanticipated surge in the number of second-level appeals in 2016, as well as increased vacancies due to staff turnover in PAAB analyst positions in 2016. Recognizing these factors, PAAB developed a revised goal that focused on the number of appeals an analyst could process per month. According to PAAB officials, focusing the revised goal on analyst production controlled for external factors that tended to affect overall processing, such as surges in appeal submissions and staff turnover. PAAB officials told us that their proposed production goal was not accepted by the Recovery Directorate for 2016, but that PAAB adopted the revised goal for individual performance plans for PAAB appeals analyst staff. In contrast, although first-level appeals represent the majority of FEMA’s appeal inventory, FEMA has not developed goals and measures to assess the performance of first-level appeals processing across regional offices. PAAB collects various data from all 10 regional offices on first- level appeals, such as the number of first-level appeals being processed, as well as processing timeliness (i.e., appeals that exceeded time limits) and key words that can help identify various appeal subject-matter categories. PAAB then aggregates this data, which it publishes on a quarterly and weekly basis in internal reports that it shares with FEMA management. However, FEMA has not established goals to assess performance against the information that PAAB collects. According to FEMA officials, while the Recovery Directorate established goals and measures for second-level appeals, it is not responsible for developing goals and measures to assess performance within the regional offices. These officials told us further that some Regional Administrators have established goals and measures for first-level appeals within their regional offices, while others have not. For management to effectively monitor a program, Standards for Internal Control in the Federal Government state that it should create goals and measures to determine if a program is being implemented as intended. In addition, the quality of the program’s performance should be assessed over time and monitoring efforts should be evaluated to assure they help meet goals. Further, Congress enacted the GPRA Modernization Act of 2010 (GPRAMA) to focus and sustain attention on agency performance and improvement by requiring that federal agencies establish outcome- oriented goals and measures to assess progress towards those goals. Specifically, agencies, like DHS, are required to monitor progress towards the achievement of goals, report on that progress, and address issues identified. Without consistent performance measures across FEMA regional offices to help assess progress and identify deficiencies in appeals processing, DHS and its subcomponent agencies like FEMA may have difficulty providing accurate reporting on the effectiveness of current efforts to process first-level appeals and on the factors that contribute to ongoing appeal processing delays. Conclusions Although FEMA has made efforts to improve its management of the PA appeals process, these efforts have been hampered by a number of issues including weaknesses in FEMA’s appeals tracking data and its ability to ensure the quality of this data. FEMA corrected its appeals data for purposes of this report once we pointed out data discrepancies, but FEMA does not have a process to ensure data quality issues are permanently addressed. As a result, these weaknesses will persist. By implementing procedures to consistently track appeals data and ensure the quality of these data, FEMA will be in a better position to accurately report on appeal processing performance and make informed decisions about the appeals process. FEMA also faces a variety of workforce challenges that have contributed to appeals processing delays. These challenges include staffing vacancies, lack of experienced staff, high rates of staff turnover, delays in training appeals staff, and the deployment of appeals analysts for disaster response, all of which have contributed to processing delays. Addressing these challenges by identifying the hiring, training, and retention needs of its appeals offices through strategic workforce planning could help FEMA better position itself to reduce its appeals backlog and better respond to PA appeals. Further, although FEMA has established goals and measures for its second-level appeals processing, it has not done so for first-level appeals. By establishing goals and measures to assess the performance of its first-level appeals process, DHS and FEMA will be able to better evaluate the efficiency and effectiveness of its efforts to reduce the PA appeal backlog and improve appeal processing times. Recommendations for Executive Action We are making the following four recommendations to FEMA: The Assistant Administrator for Recovery should design and implement the necessary processes and procedures to ensure a uniform and consistent approach for tracking first-level appeals data to better integrate regional trackers with PAAB’s own first-level appeals tracker. (Recommendation 1) The Assistant Administrator for Recovery should design and implement the necessary controls to ensure the quality of the first-level appeals data collected at and reported from the regional offices to PAAB. (Recommendation 2) The Assistant Administrator for Recovery should develop a detailed workforce plan that documents steps for hiring, training, and retaining key appeals staff. The plan should also address staff transitions resulting from deployments to disasters. (Recommendation 3) The Assistant Administrator for Recovery should work with Regional Administrators in all 10 regional offices, to establish and use goals and measures for processing first-level PA appeals to monitor performance and report on progress. (Recommendation 4) Agency Comments and Our Evaluation We provided a draft of this report to the Secretary of the Department of Homeland Security and the Administrator of the Federal Emergency Management Agency for review and comment. DHS provided written comments, which are reproduced in appendix II. In its comments, DHS concurred with our recommendations and described actions planned to address them. FEMA also provided technical comments, which we incorporated as appropriate. Additionally, we provided excerpts of the draft report to state emergency management officials in the selected six states we included in our review. We incorporated their technical comments as appropriate. Regarding our first recommendation, that FEMA design and implement the necessary processes and procedures to ensure a uniform and consistent approach for tracking first level-appeal data, DHS stated that FEMA’s PAAB will develop guides and checklists for the regions to ensure data uniformity and consistency and that PAAB will update its data review process, and develop additional content highlighting the importance of data integrity and accuracy. DHS estimated that this effort would be completed by July 31, 2018. Regarding our second recommendation, that FEMA design and implement the necessary controls to ensure first-level appeal data quality, DHS stated that PAAB will include content within the certified appeal analyst training highlighting the importance of data integrity and that first- level appeal data will be reviewed by PAAB on a quarterly basis. DHS estimated that this effort would be completed by February 28, 2019. Regarding our third recommendation, that FEMA develop a detailed workforce plan for hiring, training and retaining key appeals staff, DHS stated that by December 31, 2018, PAAB will produce a workload flow assessment on second-level appeals staffing and determine whether appeal timeliness issues still exist. If PAAB determines that significant response timeliness issues on second-level appeals still exist after most PAAB appeal analyst staff have at least one year of experience, a detailed PAAB workforce plan will be completed and finalized by December 31, 2019. PAAB will also complete an assessment of first-level appeal inventory and timeliness issues. If PAAB determines that significant regional response inventory and timeliness issues on first-level appeals still exist, FEMA will create a working group to prepare a detailed regional workforce plan. DHS estimated that this effort would be completed by December 31, 2019. Regarding our fourth recommendation that FEMA work with Regional Administrators to establish and use performance goals and measures for processing first-level appeals, DHS stated that PAAB has begun developing a methodology for establishing, measuring, and reporting on first-level appeals processing goals and performance progress, and that PAAB would work with the regions to complete and finalize this methodology. DHS estimated that this effort would be completed by August 31, 2018. As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we are sending copies of this report to the Secretary of Homeland Security and interested congressional committees. If you or your staff have any questions about this report, please contact me at (202) 512-6806 or bawdena@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix III. Appendix I: Objectives, Scope, and Methodology This report reviews aspects of the Federal Emergency Management Agency’s (FEMA) management of the Public Assistance (PA) appeals process. The objectives of this review were to determine: (1) the extent to which FEMA ensures quality in its data on appeals and what FEMA data show about its appeals inventory and timeliness for appeals decisions; (2) what steps FEMA has taken to improve its management of the appeals process and what challenges, if any, remain; and (3) the extent to which FEMA has developed goals and measures to assess the appeal program’s performance. To address the first objective, we obtained and analyzed data from FEMA on all first- and second-level appeals that the agency received between January 2014 and July 2017. For first-level appeals, FEMA provided us data on appeals received between January 1, 2014, and July 12, 2017, while FEMA provided us data on second-level appeals received between January 1, 2014, and July 6, 2017. We focused on this time frame because it contained the most complete and available data on each type of appeal at the time of our review. We identified various discrepancies in the first-level appeals data, which we discussed with knowledgeable FEMA staff. Examples of these discrepancies, which we present in this report, included missing data, erroneous data entries, and inconsistent recording of data. In response to our discussions, FEMA provided us with corrected data to address the identified discrepancies. After obtaining the corrected data, we concluded the appeals data from FEMA were sufficiently reliable to provide information on PA appeals that we present in this report. We also obtained and analyzed FEMA policies and procedures related to tracking appeals data, such as FEMA’s policies and procedures related to regional offices, and evaluated them using Standards for Internal Control in the Federal Government. We analyzed the corrected data to determine FEMA’s appeal inventory— that is, the number of first-and second-level appeals that were pending and decided, including any amounts in dispute or amounts awarded, and appeal outcomes for appeals that FEMA decided. From the total number of appeals received, we excluded four second-level appeals that had been remanded or rescinded. We determined the processing times for first- and second-level decided appeals by calculating, for each appeal, the number of calendar days between the date that FEMA received the appeal and the date that FEMA rendered a decision on the appeal. We then calculated the average number of calendar days to determine average processing times for first- and second-level decided appeals. We determined the processing time for pending first-level appeals by calculating, for each appeal, the number of calendar days between the date FEMA received the appeal and July 12, 2017. Similarly, we determined the processing time for pending second-level appeals by calculating, for each appeal, the number of calendar days between the date FEMA received the appeal and July 6, 2017. We then calculated the average number of calendar days to determine average processing times for pending first-and second-level appeals. We compared processing times for first- and second-level appeals against FEMA’s 90-day statutory time frame to determine the number of calendar days by which FEMA exceeded the time frame. We also determined the number of first- and second-level appeals in which FEMA issued an RFI and those in which FEMA did not issue an RFI. For the first- and second-level appeals in which FEMA issued an RFI, we compared the date the appeal was received to the date that FEMA issued the RFI. We used the first RFI in cases where FEMA issued multiple RFIs. We then determined whether FEMA had issued the RFI within 90 calendar days. For the first- and second-level appeals in which FEMA did not issue an RFI, we compared the date the appeal was received to the date that FEMA issued a decision. We then determined whether FEMA had issued a decision after 90 calendar days. We also obtained and analyzed FEMA policies and procedures and program directives governing appeal data collection and evaluated them against Standards for Internal Control in the Federal Government. To address the first and second objectives, we also administered semistructured interviews to officials from 3 of FEMA’s 10 regional offices (Regions II, IV, and VI) with the highest number of first- and second-level pending appeals. We asked these officials about their efforts to process and track appeals, what improvements had been made regarding how PA appeals are processed, as well as what challenges they believed remained in processing PA appeals since 2013.To select these offices, we obtained data from FEMA on first- and second-level appeals that were pending a decision, as of October 31, 2016. Collectively, these appeals represented 69 percent of all pending first- and second-level appeals FEMA had received as of October 31, 2016. We focused on this time frame because it contained the most recent data for selecting FEMA regional offices at the time of our review. To obtain additional perspective on what, if any, challenges remain in FEMA’s management of the appeals process, we also interviewed state emergency management officials in six states (two states in each of the corresponding 3 FEMA regional offices). (See table 2.) The information obtained from the FEMA regional offices and the state emergency management offices cannot be generalized nationwide. However, the information obtained from these officials provides insight into the issues FEMA encountered during the appeal process. To additionally address the second objective, we reviewed our past report and Department of Homeland Security Inspector General reports on the PA appeals program. We also reviewed FEMA documentation, such as policy directives, internal staffing requests, appeals analyst position descriptions, and other internal memoranda. We used these sources to identify what steps FEMA had taken to improve its management of the appeals process since 2013. We also used this information to supplement our understanding of the challenges the Public Assistance Appeals Board (PAAB) and regional officials raised during our interviews discussed above. To address the third objective, we analyzed a series of FEMA internal performance reports issued between November 29, 2013, and February 15, 2017. Developed by PAAB and provided to FEMA management on a quarterly basis, these reports included aggregate information on PA appeals inventory, such as the number of first- and second-level pending appeals, the number of appeals processed within statutory timeframes, the number of pending appeals that are beyond the statutory timeframe, and common appeal issues based on keywords entered by analysts responsible for processing appeals. We also analyzed internal documents, such as briefs and newsletters, which provided detail on specific appeal decisions as well as the status of the appeals inventory. Further, we analyzed FEMA’s Strategic Plans for fiscal years 2008 to 2013 and fiscal years 2014 to 2018 to identify objectives, measures, and overall agency-wide goals. We assessed the information in these documents against leading practices in measuring agency performance and against federal standards for internal control. For all three objectives, we reviewed relevant legislation and FEMA standard operating procedures that govern both FEMA headquarters and regional offices. We also interviewed officials in PAAB and FEMA’s Recovery Directorate. Appendix II: Comments from the Department of Homeland Security Appendix III: GAO Contact and Staff Acknowledgments GAO Contact Staff Acknowledgments In addition to the contact named above, Brenda Rabinowitz (Assistant Director), Anthony Bova (Analyst-in-Charge), Joseph Fread, and Sherrice Kerns made key contributions to this report. Jehan Chase, Chris Currie, Robert Gebhart, Chris Keisling, Donna Miller, Kathleen Padulchick, Amanda Parker, Erik Shive, and Walter Vance also provided assistance.
target: In both 2016 and 2017, 15 separate U.S. disasters resulted in losses exceeding $1 billion each. FEMA provides PA grants to state and local governments to help communities recover from such disasters. If applicants disagree with FEMA's decision on their PA grant application, they have two chances to appeal: a first-level appeal to be decided by the relevant FEMA regional office and, if denied, a second-level appeal to be decided within FEMA's Recovery Directorate. Each is subject to a 90-day statutory processing timeframe. GAO was asked to review FEMA's appeals process. This report examines: (1) the extent to which FEMA ensures the quality of its appeals data and what these data show about PA appeals inventory and timeliness; (2) what steps FEMA has taken to improve its management of the appeals process and what challenges, if any, remain; and (3) the extent to which FEMA developed goals and measures to assess program performance. GAO analyzed FEMA policies, procedures, and data on appeals and interviewed officials from headquarters and from regional offices with the highest number of pending appeals. GAO also spoke to state officials from the two states within each of the three regions with the highest number of pending appeals. Weaknesses in the quality of Federal Emergency Management Agency's (FEMA) Public Assistance (PA) appeals data limit its ability to oversee the appeals process. For example, FEMA's data are inaccurate and incomplete because regional offices do not consistently track first-level appeals and FEMA does not have processes to ensure data quality. When GAO discussed these weaknesses with FEMA officials, they acknowledged them and provided GAO with corrected data for January 2014 through July 2017. GAO's analyses of the corrected data show fluctuations in the appeal inventory from year to year depending on the number of disasters declared and delays in processing. For example, as shown in the figure, only 9 percent of first-level and 11 percent of second-level appeals were processed within the 90-day statutory timeframe. FEMA has taken steps to improve its management of the appeals process—including issues that GAO and the Department of Homeland Security's Office of Inspector General identified in 2008 and 2011. For example, FEMA increased its appeal staffing levels and developed standard operating procedures. Despite these efforts, FEMA continued to face a number of workforce challenges that contributed to processing delays, such as staff vacancies, staff turnover, and delays in training. FEMA has not developed a workforce staffing plan to identify hiring, training, and retention needs across its headquarters and regional offices, though FEMA officials acknowledge the potential benefits of having such a plan and stated that they are focused on filling vacancies. In the absence of a workforce plan, FEMA will continue to experience workforce challenges that could further contribute to delays in processing appeals. FEMA has not established goals and measures for assessing first-level appeals processing performance, but has done so for second-level appeals. FEMA views establishing these first-level goals and measures as the responsibility of its regional offices. Without goals and measures, FEMA is limited in its ability to assess the efficiency and effectiveness of its overall appeals process and identify and address weaknesses that may lead to delays in making appeal decisions.