National Flood Insurance Program: Progress Made on Contract Management but Monitoring and Reporting Could Be Improved
Published: 2014-01-15T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: In operating NFIP, FEMA spends hundreds of millions of dollars annually on contractors that perform critical functions. The Biggert-Waters Flood Insurance Reform Act of 2012 mandates GAO to review the three largest contractors used in administering NFIP. In prior reports, GAO found problems with FEMA's oversight of contractors responsible for performing key NFIP functions. This report examines (1) FEMA's progress in updating its process for monitoring NFIP contractors since GAO's prior reports, and (2) the extent to which FEMA followed its monitoring process for the largest NFIP contractors. To address these objectives, GAO analyzed FEMA data on funds obligated to contractors from fiscal years 2008 through to 2012, reviewed information from FEMA on contract management policies and procedures, and assessed data covering fiscal years 2011 to 2013 on the implementation of these policies and procedures as they pertained to the three largest contractors. GAO also interviewed FEMA contracting staff and contractors.
section_title: What GAO Found
paragraphs: The Federal Emergency Management Agency (FEMA) has made progress in improving its processes for monitoring NFIP contracts since GAO last reported on these issues in 2008 and 2011. For example, GAO recommended in 2011 that FEMA complete the development and implementation of its revised acquisition process to be consistent with a Department of Homeland Security (DHS) directive. FEMA updated its contract management guidance and revised its handbook for contracting officer's representatives to be consistent with DHS directives. The updated handbook also contained many of the elements identified in a federal guide to best practices for contract administration. Furthermore, the FEMA division that manages the National Flood Insurance Program (NFIP) developed a contract management reference guide that followed FEMA's handbook and federal best practices guidance.
With some exceptions, FEMA largely followed its contract monitoring procedures for the three largest NFIP contractors GAO reviewed. For example, FEMA ensured that relevant staff overseeing selected contracts received appropriate training. Also, FEMA periodically compared and analyzed actual performance data against goals for each of the three contracts through operating reports, which would allow management to review the status of deliverables and milestones and be aware of inaccuracies or exceptions that could indicate internal control problems. However, FEMA did not develop a quality assurance surveillance plan for one of the contractors--a best practice and a key requirement identified in regulations and guidance. In 2010 and 2011, FEMA identified persistent issues with the contractor's deliverables, including quality and timeliness, and faced challenges in resolving those issues, which might have been avoidable if a quality assurance surveillance plan had been developed and used. FEMA officials stated that they are considering options to ensure that the plans are in place for future contracts, but did not provide specifics on those options or when they plan to implement them. Without detailed quality assurance surveillance plans, the expectations of the agency and the contractor can be misaligned during performance evaluations. Separately, for two of the contracts FEMA staff did not enter performance evaluations in the Contractor Performance Assessment Reporting System (CPARS), a database DHS uses to record assessments of performance of government contractors. Federal and DHS regulations and FEMA contract management guidance require entry of contract performance information in CPARS within certain time frames. By not reporting such information, FEMA disadvantaged the contractors and the government by not providing data that could be used in evaluating the contractors for future contract awards. For instance, receiving a positive CPARS assessment can enhance a contractor's reputation when bidding on future contracts, and as such, the assessments provide an incentive for the contractor to perform as expected. FEMA officials have acknowledged the issue. By determining the extent to which performance evaluations have not been entered into CPARS for its contracts, identifying the reasons why, and addressing those reasons, as needed, FEMA can help ensure that its--and other agencies'--contracting decisions and management draw on complete, relevant, and timely performance information.
section_title: What GAO Recommends
paragraphs: To improve monitoring and reporting of contractor performance, we are recommending that FEMA (1) determine the extent to which quality assurance surveillance plans and CPARS assessments have not been prepared, (2) identify the reasons why, and (3) take steps, as needed, to address those reasons. FEMA concurred with GAO's recommendations.

section_title: Letter
paragraphs: The National Flood Insurance Program (NFIP), the only source of  insurance against flood damage for most residents of flood-prone areas,  obligated approximately $900 million from fiscal years 2008 through 2012  to contractors to perform critical functions such as collecting and reporting  all financial and statistical data, selling and servicing flood insurance  policies, and modernizing the mapping of flood hazards. The Federal  Emergency Management Agency (FEMA) within the Department of  Homeland Security (DHS) administers NFIP. The program seeks to  minimize the impact of flood-related property losses by making flood  insurance available and encouraging its purchase by those who need  flood insurance protection. During fiscal year 2013, FEMA collected about  $3.7 billion in premiums for about $1.3 trillion in coverage.
In earlier reports, we and the DHS Office of Inspector General found a  number of operational challenges that hindered FEMA’s ability to  effectively administer NFIP, including problems with FEMA’s oversight of  contractors responsible for performing key NFIP functions. For example,  after spending roughly 7 years and $40 million, in 2009 FEMA cancelled a  major contract to modernize its systems for collecting data on claims and  tracking the status of policies and claims because of multiple weaknesses  in acquisition management, such as conflicts of interest and inadequate  contract performance management. Weaknesses in NFIP management  and operations, including financial reporting processes and internal  controls, and oversight of contractors have kept NFIP on GAO’s list of  high-risk areas since 2006.
The Biggert-Waters Flood Insurance Reform Act of 2012 (Biggert-Waters  Act) re-authorizes NFIP through the end of fiscal year 2017 and includes  provisions for flood insurance reform and modernization. Section 100231  of the Biggert-Waters Act mandates GAO to review the three largest  contractors used in administering NFIP. This report examines (1) FEMA’s  progress in updating its process for monitoring NFIP contractors since our  prior reports, and (2) the extent to which FEMA followed its monitoring  process for the largest NFIP contractors.
To determine FEMA’s progress in updating its process for monitoring  NFIP contractors, we analyzed DHS and FEMA policies, directives,  guidance, and other materials related to contract management, including  the use of a government-wide contractor assessment database and  FEMA’s electronic document management system. Additionally, we  compared FEMA’s process for monitoring NFIP contractors with relevant  legislation, internal control standards, contract management best  practices, and related reports from the GAO and the DHS Office of  Inspector General about NFIP. We assessed the extent to which FEMA  followed its monitoring procedures for three NFIP contracts—the Bureau  and Statistical Agent (BSA) contract and two flood mapping contracts. To  determine the largest contractors, we used data provided by the FEMA  Office of the Chief Procurement Officer (OCPO). We identified the 10  largest contractors for NFIP based on the amount of funds obligated to  NFIP contractors over the previous 5 fiscal years (from 2008 through  2012). We verified the reliability of the data by comparing them with data  for the same contractors over the same time period in the DHS Service  Contract Inventories database and the Federal Procurement Data  System. We determined that the contracting cost data provided by OCPO  were sufficiently reliable for our purposes. To narrow the number of  contractors selected for review, we determined whether they had an  active NFIP contract during the course of our review. Additionally, we  selected contractors responsible for administering different NFIP  functions, such as flood mapping and insurance management. Together,  the three largest contracts we selected represented approximately 45  percent of the overall active contracting dollars obligated during the 5- year period from 2008 through 2012. To assess how FEMA followed its  monitoring procedures, we collected data across three key areas: training  and ethics requirements for contracting officer’s representatives (COR);  performance measures; and ongoing monitoring and reporting. We  assessed FEMA’s monitoring using agency procedures and contract  management best practices, including Standards for Internal Control in  the Federal Government, the Office of Management and Budget’s Office  of Federal Procurement Policy (OFPP) Guide to Best Practices for  Contract Administration, and guidance from FEMA and its Risk Insurance  and Risk Analysis divisions. For each of the selected contracts, we  reviewed the statements of work, monthly contractor monitoring reports,  and discrepancy reports. Finally, we collected available data from FEMA  covering 2011 to 2013 and conducted interviews with representatives  from FEMA on their contract management procedures. We also consulted  with representatives of the DHS Office of Inspector General and  interviewed the contractors we selected for further insight on how well  FEMA followed monitoring policies and procedures. See appendix I for  additional details on our scope and methodology and the checklist (and  results) we used to evaluate each of the selected contracts for this  engagement.
We conducted this performance audit from January 2013 through January  2014, in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: Within FEMA, the Federal Insurance and Mitigation Administration (FIMA)  manages NFIP and receives support from other divisions or offices (see  fig. 1). In fiscal year 2013, about 390 FEMA employees, assisted by  contractor employees, managed and oversaw NFIP from FEMA’s  headquarters in Washington, D.C., and in field locations. FEMA  management responsibilities included establishing and updating NFIP  regulations, analyzing data to determine flood insurance rates, and  training insurance agents and adjusters.
As shown in figure 1 above, FIMA comprises three divisions: Risk  Analysis, Risk Insurance, and Risk Reduction. Risk Analysis is  responsible for flood mapping activities and develops flood mapping  policy and guidance. FIMA established its 5-year Risk Mapping  Assessment and Planning (Risk MAP) program in 2009 to, among other  things, improve the quality of flood data used for mapping and enhance  public acceptance of flood maps. FIMA maintains and updates data  through Flood Insurance Rate Maps (FIRM) and risk assessments.  FIRMs include statistical information such as data for river flow, storm  tides, hydrologic/hydraulic analyses, and rainfall and topographic surveys.  Risk Analysis staff in the 10 regional offices manage development of flood  maps for their geographic areas. Headquarters and regional staff monitor  and report flood hazard mapping progress based on program  management data provided by FIMA’s three national Production and  Technical Services (PTS) contractors and other flood mapping partners.  The PTS contractors are private engineering firms working under contract  to FIMA and are each responsible for a regional portfolio of flood study  projects. The Risk MAP contract is a cost-plus-award-fee (CPAF)  contract. Under the contract, FIMA has obligated approximately $410  million from fiscal year 2008 to the end of fiscal year 2013 to the two PTS  contractors that we selected for review.
Risk Insurance works with private insurers to provide flood insurance for  property owners and also encourages communities to adopt and enforce  floodplain management regulations. Private insurance companies largely  are responsible for insurance sales and claims adjustment under FIMA’s  Write Your Own (WYO) program. NFIP’s BSA contractor serves as the  focal point of support operations for the WYO insurers. The BSA  contractor liaises between the government and independent property and  casualty insurance companies that issue federally guaranteed NFIP  policies. Among the services the BSA contractor provides are financial  and statistical reporting based upon data submissions from the WYO  companies, development of forms and information related to NFIP, and  various data analyses. The current BSA contract went into effect in  January 2008 and expired in December 2013. The BSA contract is cost- plus-fixed-fee (CPFF). FIMA had obligated approximately $80 million to  the BSA contractor as of the end of fiscal year 2013.
subsections: section_title: NFIP Contract Management
paragraphs: Contract management responsibilities for NFIP are shared between  OCPO and FIMA divisions. OCPO officials stated that they are primarily  involved in the acquisition planning and contract formation phases of  FIMA’s typical contracting process that is illustrated in figure 2. FIMA  divisions (which for NFIP contracts we reviewed either would be FIMA’s  Risk Analysis or Risk Insurance divisions) are responsible for defining the  need for contracting support. Once the need is identified, OCPO works  with FIMA divisions on developing statements of work and reporting  requirements for contracts. OCPO is represented in the contracting  process by a contracting officer, who has the delegated authority to  negotiate, enter into, administer, modify, and terminate contracts on  behalf of the government.
Once the contract is awarded, the contracting officer receives support  from the COR during the contract administration phase. The division  acquiring the contracted goods and services selects a COR from its staff.  The responsibilities of the COR include administering and directing daily  operations within the scope of the contract, monitoring contractor  performance, ensuring that requirements meet the terms of the contract,  and partnering with the contracting officer. The COR is authorized to  monitor the contract on behalf of the contracting officer. However, the  COR is not authorized to make any contractual commitments or changes  that may affect the contract price, terms, or conditions without the  approval of the contracting officer. Generally, the COR role can be either  part-time or full-time, and is often a collateral duty.
section_title: FEMA Has Made Improvements in Its Process for Monitoring NFIP Contracts That Incorporate Best Practices
paragraphs: FEMA has improved its contract management process since our prior  reports. In particular, FEMA has developed policies and procedures that  address our previous recommendation on implementing a DHS directive  for contract management. Additionally, FEMA divisions have developed  contract management guidance that follows best practices for contract  administration (see app. I for the key best practices used in our review).  Lastly, FEMA has developed guidance for performance reporting.
subsections: section_title: FEMA Updated Policies and Procedures for Contract Management
paragraphs: FEMA has made progress in establishing or revising its guidance on  contract management and oversight since we reported on these issues in  2008 and 2011. To address weaknesses in its oversight and  management of acquisitions, we recommended in 2011 that FEMA  complete the development and implementation of its revised acquisition  process to be consistent with a DHS directive that sets overall policy and  structure for acquisition management for DHS agencies. In May 2011,  FEMA published its own directive on the review of contracts that was  consistent with the departmental directive. FEMA also updated guidance  in October 2011 that explains its acquisition process. The updated  guidance was intended as a tool for program offices to use when initiating  an acquisition. In particular, the guidance states that the program office is  responsible for developing the acquisition plan that includes a description  of requirements. If the program office decides not to define how the  contractor is to accomplish the desired results of the contract, it can  develop a performance work statement that defines the required results in  measurable, mission-related terms absent details about how to do the  work. The guidance requires that the performance work statement be  accompanied by a performance requirements summary that documents  the services and performance standards expected and a quality  assurance surveillance plan that defines the agency’s plan for ensuring  that the contractor has performed in accordance with the performance  standards. The guidance includes a template for the performance work  statement and an intranet link to a quality assurance surveillance plan  template.
In addition, FEMA published a revised version of its COR Handbook in  March 2012 in response to a DHS directive. According to FEMA, the  handbook is intended to provide practical guidance for CORs as they  perform their daily contract administration duties. The handbook  incorporates best practices in contract management as described in the  OFPP guide. For example, the handbook incorporates language that  encourages CORs to develop a contract administration plan along with a  quality assurance surveillance plan—both OFPP best practices. Contract  administration plans are intended to provide a broad overview of the  contract, including areas of risk and concern. Quality assurance plans are  meant to be used to evaluate the quality and timeliness of the products  and services produced by the contractor, and include performance  standards, the methods of assessment, and an assessment schedule.
Along with policy development, FEMA has adopted an intranet-based  system to centralize and electronically store NFIP contracting-related  documents, including monitoring reports. In 2008, we recommended that  FEMA implement a process to better ensure that CORs submit monitoring  reports on time, systematically review these reports, and retain the  reports in a quality assurance file. Several FEMA divisions use this  intranet-based system. In part to address our 2008 report, FEMA staff in  Risk Insurance involved in contract administration began using the  system in November 2010. The Risk Insurance division intranet-based  system is used by CORs to store and share COR-generated reports. The  Risk Analysis division uses the intranet-based system for storing contract  documents, particularly contract modifications and task orders submitted  by the regional offices.
section_title: FEMA Divisions Have Contract Management Guidance That Follows Best Practices
paragraphs: The Risk Insurance and Risk Analysis divisions developed contract  management guidance that follows best practices and implements the  COR Handbook. More specifically, in December 2012 Risk Insurance  finalized the Contracts Management Reference Guide. The guide closely  follows the COR handbook and provides Risk Insurance CORs with a  high-level overview of FEMA, DHS, and federal acquisition guidance and  directs CORs to relevant reference materials. It also follows many of the  practices recommended in the OFPP guide for contract administration.  For instance, the Contracts Management Reference Guide lists  implementation of a contract administration plan and a quality assurance  surveillance plan as critical contract administration activities.
Furthermore, FEMA’s Risk Insurance division developed additional  guidance intended to help ensure consistency in handling the failure of a  contractor to meet performance standards. Specifically, Discrepancy  Report Procedures guide CORs in such instances. In 2008, we  recommended that FEMA develop such guidance after finding that the  agency did not coordinate information and actions related to deficiencies  for certain contractors. The Discrepancy Report Procedures include  guidance for the COR to coordinate with the contracting officer to ensure  that the contractor is notified and corrective action is taken.
FEMA’s Risk Analysis division has guidance for contract management,  but took a different approach from the Risk Insurance division for its  design and implementation because of the nature of the Risk MAP  program. Risk Analysis established a Risk MAP Contracts Council to  help ensure coordination, integration, and alignment of contracts across  the Risk MAP program. The council comprises senior contracting officials  and all CORs in the Risk MAP program.
The Risk Analysis division follows FEMA’s COR Handbook, but has not  developed further guidance for CORs like the Risk Insurance Contracts  Management Reference Guide. Instead, Risk Analysis developed a  number of plans that provide general guidance for contract management  staff across the Risk MAP program. FEMA’s COR Handbook  recommends such planning documents, which also are consistent with  best practices as described in the OFPP guide. For instance, the Program  Management Plan, which is a narrative summary of the program’s  processes and plans, provides overall contract management guidance for  Risk MAP. Several other planning documents provide guidance on  specific issues including risk management, quality assurance  management, product delivery, and changing task orders. Unlike the Risk  Insurance division, Risk Analysis has not developed a distinct deficiency  report policy. According to officials, the Risk Analysis division follows an  award fee plan for all of the related task orders. The fee plan sets  performance metrics, such as timeliness and cost efficiency of community  data collection, against which the contractor is evaluated, either quarterly  or annually. If the performance metrics are not met, a portion of the award  fee is to be deducted. If a performance problem at the project level  becomes persistent and is not addressed by the contractor, Risk Analysis  is to file a deficiency report similar to that used by Risk Insurance. We  discuss these award fee plans and deficiency reports in greater detail in  the next section.
section_title: DHS and FEMA Developed Guidance for CORs for Performance Reporting
paragraphs: DHS and FEMA have developed guidance to implement federal  requirements to report their assessment of contractor performance in a  government-wide database—the Contractor Performance Assessment  Reporting System (CPARS). In July 2009, the Office of Management and  Budget (OMB) required executive branch agencies to report assessment  information about contractor performance into a central repository of  contractor performance information, called the Past Performance  Information Retrieval System. We previously found that DHS was  transitioning to using CPARS to comply with this requirement. The  primary purpose of CPARS is to better ensure that current, complete, and  accurate information on contractor performance is available for use in  procurement source selections. The CPARS guidance states that each  assessment must include detailed and complete statements about the  contractor’s performance and be based on objective data (or measurable,  subjective data when objective data are not available) supported by  program and contract/task order management data. Such performance  assessment can significantly reduce the risk to the government on future  awards. As a result, OMB urges agencies to ensure that all critical  performance assessments are made available in a timely manner.
Both DHS and FEMA have incorporated this reporting requirement into  acquisition regulations and program office guidance. For example, the  regulations state that assessments should clearly, objectively, and  completely describe the contractor’s performance in the narrative  statement, in sufficient detail to justify the rating. Furthermore, the  narratives should include a level of detail and documentation that  provides evidence and establishes a basis for the assigned rating, an  explanation of how problems were resolved, and the extent to which  solutions were effective. FEMA developed standard operating procedures  for CPARS that establish appointments and responsibilities for FEMA  staff responsible for entering information into the system. In particular,  the procedures state that the branch chiefs within each program office are  ultimately responsible for oversight of its CPARS program. Also, the COR  is responsible for oversight of the assigned contract and assisting the  contracting officer with drafting the CPARS assessment. The standard  operating procedures mandate basic CPARS training for CORs.
The FEMA COR Handbook and Risk Insurance Contracts Management  Reference Guide include recording contractor performance in CPARS as  a critical contract administration activity. The Handbook states that  CPARS evaluations must be completed annually and at the conclusion of  each contract. In particular, contracts lasting 2 or more years are usually  to be evaluated annually and evaluations should be performed at least  120 days before the exercise of an option to extend the contract. The  Contracts Management Reference Guide states that CPARS evaluations  are due before the end of a task order, and includes reference to FEMA’s  standard operating procedures for CPARS.
section_title: FEMA Largely Followed Contract Monitoring Procedures, but More Could Be Done Related to Quality Assurance Monitoring
paragraphs: For the three largest contracts we selected, FEMA generally followed its  procedures for contract management in the three key areas that we  identified for review—COR training and ethics requirements, performance  measures, and ongoing monitoring and reporting (see table 1). But we  found some opportunity for improvement related to quality assurance for  one of the contracts and consistent reporting of contractor performance in  CPARS for other two contracts.
subsections: section_title: CORs Met Training and Ethics Requirements
paragraphs: For each of the three contractors we reviewed, FEMA’s COR training and  certification program requirements were met at the time of appointment.  In 2010, the DHS Inspector General recommended that FEMA ensure  NFIP staff received annual training on the roles and responsibilities of the  contracting officer and COR. Because of the size and complexity of the  three contracts we reviewed, the CORs had to maintain the highest level  of certification. Initial and ongoing training requirements for FEMA CORs  are clearly detailed in agency guidance. Initially staff needs 40 hours of  training to assume basic COR responsibilities and 40 hours bi-annually of  refresher training to maintain certification. Based upon data FEMA  provided, each of the CORs completed training requirements at the time  of appointment, and also satisfactorily documented their completed  refresher training requirements for the contracts we reviewed. For  contracting officers, FEMA also has implemented procedures that provide  detailed guidance for contracting officers during each phase of the  procurement process. FEMA has continued to update the guidance, and  has made it available on its intranet.
For the contracts that we reviewed, we found documentation that the  CORs had fulfilled the ethics requirement for 2013. The DHS Inspector  General recommended in March 2010 that all FEMA employees involved  in the procurement process receive annual procurement-specific ethics  training and file financial disclosure forms. Initial ethics and procurement  ethics training courses are included in the COR curriculum, and  completion was validated by COR certification being awarded. Annual  ethics training is mandatory for FEMA employees. Federal regulations  require CORs and contracting officers to file a financial disclosure form.  FEMA’s Office of the Chief Counsel requires FEMA CORs and  contracting officers to file a financial disclosure form within 30 days of  being appointed to a new position and annually. Supervisors review  financial disclosures for potential conflicts of interests that could arise  from stock holdings, investments, or personal relationships with a  contractor. Any potential conflicts discovered by the supervisor during  their review are referred to the chief counsel’s office for further  determination. The filer and supervisor are notified of the chief counsel’s  determination, which may include restrictions or precautionary measures  that should be taken. Based upon our review of data provided by FEMA,  no disqualifying conflicts of interest were reported for 2013 by the chief  counsel’s office for any CORs or contracting officers working on the  contracts we reviewed.
section_title: FEMA Established Relevant Performance Measures for the Contracts Reviewed
paragraphs: For each of the three contracts we reviewed, FEMA had established  performance measures that were relevant and measureable to the  contracts’ objectives and goals. According to federal government internal  control standards, appropriate and relevant performance measures and  indicators must be established and continually compared against program  goals and objectives. The standards also recommend incorporating  performance data into operating reports that inform management of  inaccuracies or internal control problems. Our review of contract  documentation for each of the three contracts that we selected indicated  FEMA has established performance measures that met these standards.  FEMA periodically compared and analyzed actual performance data  against goals for each of the three contracts through operating reports,  which would allow management to review the status of deliverables and  milestones and be aware of inaccuracies or exceptions that could indicate  internal control problems.
subsections: section_title: Use of Performance Metrics: BSA Contract
paragraphs: The BSA contract, which FEMA awarded in 2008, contains clearly defined  appropriate performance areas that link to contract objectives. Many of  the performance areas measure whether the contractor rendered the  service within the required time frames. Specifically, the contract  objectives cover services such as coordinating disaster response;  maintaining the WYO Financial Control Plan; analyzing actuarial, claims,  and underwriting data; coordinating communication; and maintaining  NFIP information technology systems. The performance areas used to  measure contractor performance address the timeliness of disaster  response, the management of materials, such as manuals and rate  tables, and the operation of the NFIP accounting system. The  performance areas also included the submission of program management  plans and communication services on a timely basis. A contractor that did  not meet a performance requirement could have its annual target fee  reduced. For example, the disaster response performance area required  that the contractor deploy disaster teams within 24 hour notice by the  COR. If the contractor did not deploy teams within that time, the COR  could reduce the annual target fee by one-fifth. FEMA staff stated that  they had never reduced the fee for the BSA contractor over the life of the  contract we reviewed.
Risk Insurance has tracked the performance data for the BSA contract  through monthly monitoring reports submitted by the COR and through  operating reports, referred to as “dashboard reports” (see fig. 3). The  monthly monitoring reports, which the COR compiles using records of  activities and interactions from the reporting month, as well as technical  progress reports submitted by the contractor, tracked the accomplishment  of each performance area established in the contract. Using information  from the contractor’s financial reports, the COR also submits dashboard  reports on a monthly basis, which summarized financial and budgetary  data, the status of the contract’s schedule, and any risk issues, such as  the need for additional field and office support during Superstorm Sandy.  By identifying risk areas, the dashboard reports are intended to make  management aware of any issues. As shown in figure 3, the Program  Management Office reviewed the monthly reports submitted by the COR  for completeness and conducted oversight to identify any issues requiring  management action. The contracting officer for the BSA contract said the  COR copied her on correspondence with the contractor.
We reviewed the COR monthly reports for April 2012 to August 2013 and  found that the BSA contractor had met all the identified performance  measures for each month. Additionally, we reviewed operating reports for  the BSA contract from January 2012 until December 2012 and found that  the BSA program was consistently within budget and schedule  constraints during the period reviewed.
section_title: Use of Performance Metrics: Risk MAP Contracts
paragraphs: Similarly, the two Risk MAP contracts we reviewed contained clearly  defined and appropriate performance metrics that were linked to the  contract objectives and program goals. FEMA established overall  program objectives for the Risk MAP program, and then established  contractor-specific performance measures through the award fee plan.  The overall objectives of the Risk MAP program include (1) producing  Risk MAP products that are aligned with user needs; (2) strengthening  partnerships to develop improved understanding of flood risk and hazard  mitigation; and (3) promoting the use of Risk MAP products at the local  level for loss reduction. For the contractor-specific performance measures  established in the award fee plans, FEMA set similar performance metrics  for each mapping contractor because they conduct similar work in the  various FEMA regions. The award fee plan for both contracts contained  metrics related to Risk MAP deployment and mitigation action, timeliness  of map packages and percentage of communities adopting maps,  timeliness and cost efficiency of community data collection, customer  satisfaction, and the quality of the administrative process. The plan also  outlined performance metrics for regional task orders.
Additionally, the award fee plan described the performance assessment  process and established the award amount to be earned based on the  contractors’ performance. More specifically, for each of the performance  metrics, the plan included standards for performance, the award-fee  amounts available, and descriptions of each standard. As illustrated in  figure 4, the COR reviewed quarterly self assessments submitted by the  contractors to ensure continuous comparison of performance to goals.  The COR also reviewed monthly progress reports, which contained  performance data tied to contract objectives. As part of the performance  assessment process, the COR and the contractor then reached  agreement on the COR’s assessment. The COR then submitted the  assessment to the Award Fee Review Board, which consisted of voting  members familiar with the mapping program. The Award Fee Review  Board recommended an amount to be awarded and the fee determining  official approved the amount. FEMA officials stated that the contractors  usually received between 90 and 95 percent of the available award fee.  According to the contracting officer for the two mapping contracts, the  contracting officers assigned to mapping contracts meet weekly with the  CORs to discuss upcoming milestones. We found through our review of  award fee determinations by FEMA officials that the two Risk MAP  contractors generally collected most of the award fees available during  fiscal year 2012.
In addition, FEMA’s Risk Analysis division management regularly  reviewed operating reports, referred to as Joint Program Reviews (JPR),  to monitor the performance of the Risk MAP program (see fig. 4). The  contractors submitted the monthly technical reports to the project  management contractor, which then compiled the reports for the JPRs  and submitted them to Risk MAP management for review. We reviewed  fiscal year 2012 JPRs for the mapping contractors and found that the  JPRs were organized by program objectives, and contractor performance  metrics provided data on progress made in meeting the objectives. The  reviews were produced monthly for headquarters contracts and quarterly  for regional contracts based on data contained in an earned value  management system, into which the contractors enter deliverables, and  monthly technical reports. The reviews also summarized performance  results for all contractors and showed results for individual contractors.
section_title: Ongoing Monitoring Has Improved, but One Contract Did Not Have a Surveillance Plan and Performance Assessments Were Not Always Reported
paragraphs: Since we reported on FEMA’s monitoring of NFIP contracts in 2008, the  agency has improved its ongoing monitoring of contractors based on our  review of the three largest contracts. In 2008 we found that FEMA staff  did not consistently follow key monitoring procedures, including  maintaining a quality assurance file and consistently implementing  performance deficiency procedures. We recommended that FEMA  implement a process to ensure that monitoring reports are submitted,  reviewed, and maintained in a quality assurance file, and implement  written guidance on how to consistently handle a contractor’s failure to  meet performance standards. Contract monitoring best practices that we  identified recommend that agencies conduct ongoing monitoring by  maintaining contract files, fully document modifications, develop contract  administration plans, develop quality assurance surveillance plans,  consistently implement deficiency guidance, and conduct annual  performance assessments (see app.I).
For each of the three contracts we reviewed, we examined (1) the  completeness of the quality assurance files maintained by the CORs, (2)  documentation of modifications made to the contracts, and (3) deficiency  reports issued related to the contracts since 2011. We found that overall  FEMA maintained the appropriate files for the three contracts.  Specifically, we found that the files contained proper documentation on  the modifications made to the contracts. The BSA contract had few  modifications and included changes such as the assignment of new  contracting officers, the revision of the performance work statement, and  extension of the terms of the contract. The two mapping contracts have  regional subcontracts, which are set up through task orders. The mapping  contracts had multiple modifications and task orders, with varying  degrees of complexity. For example, a contract was modified to allow a  contractor to adopt new mapping technology, and another contract was  modified to adjust reporting deadlines.
FEMA actively used deficiency reports to identify and resolve  performance problems for the three contracts we reviewed. For example,  in 2011, FEMA issued deficiency reports for each of the three contracts.  The purpose of deficiency reports is to record when a contractor does not  meet a performance objective. FEMA filed one deficiency report for the  BSA contract in 2011 and documented the contractor’s corrective action  to resolve the issue. FEMA also issued three deficiency reports in 2011  for the mapping contracts we reviewed. One contractor received two  deficiency reports and provided corrective action plans that FEMA  accepted. The other contractor received one deficiency report and  provided a corrective action plan. We found that FEMA followed the  respective divisions’ deficiency report guidance and resolved the  performance problems. FEMA officials stated that they have not issued  any deficiency reports related to these contracts since 2011.
subsections: section_title: Quality Assurance Surveillance Plan Was Not Developed for One of the Contracts Reviewed
paragraphs: Although FEMA has generally improved its ongoing monitoring of  contractors, the agency did not develop a quality assurance surveillance  plan for one of the contracts. FEMA developed surveillance plans for the  two mapping contracts we reviewed, but agency officials did not develop  one for the BSA contract. As discussed earlier, quality assurance  surveillance plans are required for performance-based contracts  according to regulation, agency guidance, and best practices. According  to these sources, the plan should be prepared in conjunction with the  statement of work and should include, among other things, the  performance standards, the methods of assessment, an assessment  schedule, a description of corrective actions to be taken if required  performance standards are not met, and incentives to be applied. FEMA  officials stated that they relied on a brief reference to quality assurance in  the BSA contract and other guides rather than develop a detailed  surveillance plan. Specifically, the current BSA contract mentions a  “Government Quality Assurance Plan” and includes a performance  requirements summary with general performance metrics. The contract,  however, does not specify how often deliverables would be monitored  and evaluated. Risk Insurance division officials also stated that they used  the general Contracts Management Reference Guide as the quality  assurance plan for the BSA contract. However, the Contracts  Management Reference Guide was intended to provide an overview of  the COR’s responsibilities and activities, and does not discuss the  method of assessment the COR will undertake, how often it with be done,  or actions the COR will take if performance is not met. The guide itself  calls for the development of a quality assurance surveillance plan.
In contrast, our 2008 report found that FEMA had a quality assurance  surveillance plan for a past BSA contract. Further, FEMA developed a  detailed quality assurance surveillance plan for the two mapping  contracts. For example, the quality assurance surveillance plan for the  mapping contracts outlined management roles and responsibilities,  quality standards, and the process for ensuring quality control. Quality  assurance surveillance plans that focus on the quality, quantity, and  timeliness of performance outputs would likely avoid challenges  monitoring the contractor’s performance. Additionally, without detailed  quality assurance surveillance plans, the expectations of the COR and  contractor can be misaligned during performance assessments. For  example, in 2010 and 2011, FEMA identified persistent issues with the  BSA contractor’s deliverables, including quality and timeliness, and faced  challenges in resolving those issues, which may have been avoidable if a  quality assurance surveillance plan had been developed and used.
FEMA officials acknowledged the need to develop quality assurance  surveillance plans. They also stated that various actions are under  consideration that may help ensure that quality assurance surveillance  plans are in place for future contracts. However, FEMA officials did not  provide specifics about these actions or provide a date for when they  would determine which actions, if any, to implement. Until FEMA  develops specific actions and implements them, it is unclear as to  whether FEMA has effective internal controls in place to ensure that  quality assurance surveillance plans are developed for future FEMA  contracts.
section_title: Performance Assessments Not Reported for Two of the Contracts Reviewed
paragraphs: FEMA did not report performance assessments in CPARS for two of the  three contracts we reviewed. While FEMA filed CPARS reports for the  BSA contract for 2011 and 2012, the agency did not file any for the two  mapping contracts. As discussed earlier in the report, DHS uses CPARS  reports to assess and report performance on previous contracts, and  applicable regulations and contract management guidance require entry  of contract performance assessments in CPARS. The Risk MAP  contracts we reviewed were multiyear contracts active in fiscal years  2011, 2012, and 2013. Per the guidance in FEMA’s COR Handbook, the  CORs should have reported performance assessments at the end of the  contract period in 2011 and 2012, at least 120 days prior to exercising the  next option. Agency officials acknowledged that the CORs were  supposed to report such information, but had not done so. FEMA staff  responsible for the mapping contracts stated that CPARS assessments  were a low priority at that time. FEMA staff responsible for the mapping  contracts have acknowledged the importance of CPARS and stated that  completing past CPARS assessments is now a higher priority.
As discussed earlier, internal control standards require that management  emphasize to program managers their responsibility for internal controls  and ongoing monitoring. Not submitting CPARS assessments is  disadvantageous to both the contractor and the government. Receiving a  positive CPARS assessment can enhance a contractor’s reputation when  bidding on future contracts, and as such, the assessments provide an  incentive for the contractor to perform as expected. For example, FEMA  officials stated that for the BSA contract, the contractor’s performance  improved after receiving a low rating in CPARS in 2011. Furthermore, not  submitting assessments reduces the information available to other  acquisition professionals in government when determining contractor  selection. When selecting contractors, the FAR requires agencies to  consider past performance as one assessment factor in most competitive  procurements.
section_title: Conclusions
paragraphs: FEMA relies heavily on contractors to accomplish the objectives of NFIP.  FEMA has made progress in developing contract management guidance  that addresses weaknesses identified in our past reports and that is  consistent with best practices in contract management. However, the  inconsistent application of the guidance that we identified in our review of  the three largest contracts indicated that continued attention to internal  controls is warranted. Specifically, FEMA did not develop a quality  assurance surveillance plan for the current BSA contract, which is critical  to assessing the quality and timeliness of the products and services  produced by the contractor. Regulations and agency guidance direct that  such plans be used. FEMA officials stated that they are considering  various options to ensure that the plans are in place for future contracts.  However, without detailed surveillance plans, the expectations of the  COR and contractor can be misaligned during performance evaluations  and may not focus on the quality, quantity, and timeliness of performance  outputs. Moreover, by taking actions to determine the extent to which  quality assurance surveillance plans have not been prepared and the  reasons why, FEMA would be better positioned to take action, as needed,  to address these reasons and help improve contract management, which  we have identified as a high-risk area.
In addition, the CORs for the Risk MAP contracts we reviewed have not  entered ratings for the contractors into CPARS, the contractor  assessment reporting database used by DHS that feeds into a  government-wide contractor past performance database. CPARS data  are key to informing federal contracting decisions and applicable  regulations and agency guidance requires the use of CPARS to assess  contractor performance. Not submitting CPARS assessments  disadvantages contractors and the federal government because it  reduces the information available to federal acquisition staff when  determining contractor selection. As with the surveillance plan, by  determining the extent to which this situation exists, identifying the root  cause, and implementing steps to address the root cause, as appropriate,  FEMA can help ensure that its—and other agencies’—contracting  decisions and management draw on complete, relevant, and timely  performance information.
section_title: Recommendations for Executive Action
paragraphs: While FEMA has begun to consider how to address the lack of a quality  assurance surveillance plan for the BSA contract and to enter Risk MAP  contractor performance assessments into CPARS, we are making two  recommendations aimed at improving monitoring and reporting of  contractor performance.
To help ensure FEMA’s contract monitoring provides a consistent,  structured, and transparent method to assess contractor services, the  FEMA Administrator should determine the extent to which quality assurance surveillance plans  have not been developed for FEMA contracts;  identify the reasons why quality assurance surveillance plans  were not developed; and develop additional actions as needed to address the reasons to  help ensure that quality assurance surveillance plans are  developed for its future awards.
To help ensure that federal contracting officials have complete and  timely information about the performance of contractors, the FEMA  Administrator should determine the extent to which CPARS assessments have not  been completed for FEMA contracts;  identify the reasons why CPARS assessments were not  completed; and develop additional actions as needed to address the reasons to  help ensure that assessments (ratings) for FEMA contractors are  reported in CPARS on a timely and consistent basis.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to FEMA for review and comment. In  written comments, FEMA concurred with our recommendations. FEMA’s  comments are reprinted in appendix II. FEMA also provided technical  comments, which we incorporated in the report as appropriate. In  response to our first recommendation, FEMA informed us that they have  reviewed all active NFIP contracts and have directed all COR staff that do  not have quality assurance plans in place to develop such plans. FEMA  also noted that quality assurance surveillance plans are required by DHS  acquisition guidelines and stated that they are rapidly working to improve  compliance in this area with an estimated completion date of January 31,  2014.
In response to our second recommendation, FEMA stated that reduced  COR staff for the Risk MAP program resulted in the inability to complete  CPARS assessments in a timely manner. FEMA stated that they are in  the process of identifying additional COR staff. In the meantime, CORs in  the Risk Analysis division have been working to complete outstanding  assessments and they planned to complete the assessments by  December 31, 2013. FEMA also stated that as part of its governance  plan, it plans to document the CPARS assessment process and commit  to completing future reports within 30 days of receiving notification that  assessments are due.
FEMA’s actions in response to our recommendations are steps in the  right direction to improve compliance with contract oversight  requirements. Effective controls are necessary to help ensure that FEMA  fully implements required monitoring and reporting of contractor  performance. For instance, while FEMA is in the process of developing  quality assurance surveillance plans for all contracts, as we also  recommended, FEMA will need to make certain that it has determined the  reason(s) that a quality assurance surveillance plan had not been  developed for the BSA contract to provide greater assurance that such  plans are prepared as required going forward.
We are sending copies of this report to the Secretary of Homeland  Security. In addition, the report is also available at no charge on the GAO  website at http://www.gao.gov.
Should you or your staff have questions concerning this report, please  contact me at (202) 512-8678 or garciadiazd@gao.gov. Contact points for  our Offices of Congressional Relations and Public Affairs may be found  on the last page of this report. Key contributors to this report are listed in  appendix III.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Section 100231 of the Biggert-Waters Flood Insurance Reform Act of  2012 mandates GAO to review the three largest contractors used in  administering the National Flood Insurance Program (NFIP) of the  Federal Emergency Management Agency (FEMA). We examined (1)  FEMA’s progress in updating its process for monitoring NFIP contractors  since our prior reports, and (2) the extent to which FEMA followed its  monitoring process for the largest NFIP contractors.
To determine FEMA’s progress in updating its process for monitoring  NFIP contractors, we analyzed Department of Homeland Security (DHS)  and FEMA policies, directives, guidance, and other materials related to  contract management, including the use of a government-wide contractor  assessment database and FEMA’s electronic document management  system. Additionally, we compared FEMA’s process for monitoring NFIP  contractors with relevant legislation, Standards for Internal Control in the  Federal Government, contract management best practices, and related  reports on NFIP from GAO and Office of the Inspector General at DHS.  We assessed the extent to which FEMA followed its monitoring  procedures for three large NFIP contracts—the Bureau and Statistical  Agent (BSA) contract and two flood mapping contracts. Using data  provided by the FEMA Office of the Chief Procurement Officer, we  identified the 10 largest contractors for NFIP based on the amount of  funds obligated to NFIP contractors over the previous 5 fiscal years (from  2008 through 2012). We verified the accuracy of the data by comparing  them with data in the DHS Service Contract Inventories database and the  Federal Procurement Data System. We determined that the contracting  cost data provided by Office of the Chief Procurement Officer were  sufficiency reliable for our purposes. To narrow the number of contractors  selected for review, we determined whether they had an active NFIP  contract during the course of our review. Additionally, we selected  contractors responsible for administering different NFIP functions, such  as flood mapping and insurance management. As shown in table 2, the  dollars obligated to the three contractors we selected represented  approximately 45 percent ($405 million out of a total of $909 million) of  the overall active contracting dollars obligated during the 5-year period  from fiscal years 2008 through 2012.
To assess the extent to which FEMA followed best practices and  procedures for contract management, we compared FEMA’s contract  management practices to criteria outlined in the following sources:
Standards for Internal Control in the Federal Government;
A Guide to Best Practices for Contract Administration;
FEMA, Contracting Officer’s Representative Handbook;
FEMA Risk Insurance Division, Contracts Management Reference
FEMA Risk Insurance Division, Discrepancy Report Procedures; and
FEMA Risk Analysis Division, Risk MAP Award Fee Plan.
We grouped key contract management processes into three key areas  using the sources listed above: training and ethics requirements for  contracting officer’s representative (COR), performance measures, and  ongoing monitoring and reporting. As part of our assessment, we  reviewed relevant plans and documentation and interviewed FEMA staff.  We determined whether the information was fully, partially, or not present,  and recorded the presence of this information by accordingly marking the  answers “yes,” “partially,” “no,” or “not applicable.” The following table  identifies the specific best practices and procedures that we used and our  evaluation of the three contracts we reviewed: the BSA contract and the  two Risk Mapping Assessment and Planning (Risk MAP) contracts.
For each of the selected contracts, we reviewed the statements of work,  monthly contractor monitoring reports, and discrepancy reports. We  collected available data from FEMA and conducted interviews with  representatives from FEMA on their contract management roles and  responsibilities. We also consulted with representatives of DHS’s Office of  the Inspector General and interviewed the contractors we selected for  further insight on the extent to which FEMA followed monitoring policies  and procedures.
We conducted this performance audit from January 2013 through January  2014, in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
section_title: Appendix II: Comments from the Department of Homeland Security
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Acknowledgments
paragraphs: In addition to the contact named above, Harry Medina, Assistant Director;  Philip Curtin; Juliann Gorse; Josie Benavidez; William R. Chatlos; Julia  Kennon; Marc Molino; Barbara Roesmann; Jessica Sandler; and William  T. Woods, made key contributions to this report.