National Flood Insurance Program: Progress Made on Contract Management but Monitoring and Reporting Could Be Improved
Released: 2014-01-15T00:00:00

Letter

Background

Within FEMA, the Federal Insurance and Mitigation Administration (FIMA) manages NFIP and receives support from other divisions or offices (see fig. 1). In fiscal year 2013, about 390 FEMA employees, assisted by contractor employees, managed and oversaw NFIP from FEMA’s headquarters in Washington, D.C., and in field locations. FEMA management responsibilities included establishing and updating NFIP regulations, analyzing data to determine flood insurance rates, and training insurance agents and adjusters.

As shown in figure 1 above, FIMA comprises three divisions: Risk Analysis, Risk Insurance, and Risk Reduction. Risk Analysis is responsible for flood mapping activities and develops flood mapping policy and guidance. FIMA established its 5-year Risk Mapping Assessment and Planning (Risk MAP) program in 2009 to, among other things, improve the quality of flood data used for mapping and enhance public acceptance of flood maps. FIMA maintains and updates data through Flood Insurance Rate Maps (FIRM) and risk assessments. FIRMs include statistical information such as data for river flow, storm tides, hydrologic/hydraulic analyses, and rainfall and topographic surveys. Risk Analysis staff in the 10 regional offices manage development of flood maps for their geographic areas. Headquarters and regional staff monitor and report flood hazard mapping progress based on program management data provided by FIMA’s three national Production and Technical Services (PTS) contractors and other flood mapping partners. The PTS contractors are private engineering firms working under contract to FIMA and are each responsible for a regional portfolio of flood study projects. The Risk MAP contract is a cost-plus-award-fee (CPAF) contract. Under the contract, FIMA has obligated approximately $410 million from fiscal year 2008 to the end of fiscal year 2013 to the two PTS contractors that we selected for review.

Risk Insurance works with private insurers to provide flood insurance for property owners and also encourages communities to adopt and enforce floodplain management regulations. Private insurance companies largely are responsible for insurance sales and claims adjustment under FIMA’s Write Your Own (WYO) program. NFIP’s BSA contractor serves as the focal point of support operations for the WYO insurers. The BSA contractor liaises between the government and independent property and casualty insurance companies that issue federally guaranteed NFIP policies. Among the services the BSA contractor provides are financial and statistical reporting based upon data submissions from the WYO companies, development of forms and information related to NFIP, and various data analyses. The current BSA contract went into effect in January 2008 and expired in December 2013. The BSA contract is cost- plus-fixed-fee (CPFF). FIMA had obligated approximately $80 million to the BSA contractor as of the end of fiscal year 2013.

NFIP Contract Management

Contract management responsibilities for NFIP are shared between OCPO and FIMA divisions. OCPO officials stated that they are primarily involved in the acquisition planning and contract formation phases of FIMA’s typical contracting process that is illustrated in figure 2. FIMA divisions (which for NFIP contracts we reviewed either would be FIMA’s Risk Analysis or Risk Insurance divisions) are responsible for defining the need for contracting support. Once the need is identified, OCPO works with FIMA divisions on developing statements of work and reporting requirements for contracts. OCPO is represented in the contracting process by a contracting officer, who has the delegated authority to negotiate, enter into, administer, modify, and terminate contracts on behalf of the government.

Once the contract is awarded, the contracting officer receives support from the COR during the contract administration phase. The division acquiring the contracted goods and services selects a COR from its staff. The responsibilities of the COR include administering and directing daily operations within the scope of the contract, monitoring contractor performance, ensuring that requirements meet the terms of the contract, and partnering with the contracting officer. The COR is authorized to monitor the contract on behalf of the contracting officer. However, the COR is not authorized to make any contractual commitments or changes that may affect the contract price, terms, or conditions without the approval of the contracting officer. Generally, the COR role can be either part-time or full-time, and is often a collateral duty.

FEMA Has Made Improvements in Its Process for Monitoring NFIP Contracts That Incorporate Best Practices

FEMA has improved its contract management process since our prior reports. In particular, FEMA has developed policies and procedures that address our previous recommendation on implementing a DHS directive for contract management. Additionally, FEMA divisions have developed contract management guidance that follows best practices for contract administration (see app. I for the key best practices used in our review). Lastly, FEMA has developed guidance for performance reporting.

FEMA Updated Policies and Procedures for Contract Management

FEMA has made progress in establishing or revising its guidance on contract management and oversight since we reported on these issues in 2008 and 2011. To address weaknesses in its oversight and management of acquisitions, we recommended in 2011 that FEMA complete the development and implementation of its revised acquisition process to be consistent with a DHS directive that sets overall policy and structure for acquisition management for DHS agencies. In May 2011, FEMA published its own directive on the review of contracts that was consistent with the departmental directive. FEMA also updated guidance in October 2011 that explains its acquisition process. The updated guidance was intended as a tool for program offices to use when initiating an acquisition. In particular, the guidance states that the program office is responsible for developing the acquisition plan that includes a description of requirements. If the program office decides not to define how the contractor is to accomplish the desired results of the contract, it can develop a performance work statement that defines the required results in measurable, mission-related terms absent details about how to do the work. The guidance requires that the performance work statement be accompanied by a performance requirements summary that documents the services and performance standards expected and a quality assurance surveillance plan that defines the agency’s plan for ensuring that the contractor has performed in accordance with the performance standards. The guidance includes a template for the performance work statement and an intranet link to a quality assurance surveillance plan template.

In addition, FEMA published a revised version of its COR Handbook in March 2012 in response to a DHS directive. According to FEMA, the handbook is intended to provide practical guidance for CORs as they perform their daily contract administration duties. The handbook incorporates best practices in contract management as described in the OFPP guide. For example, the handbook incorporates language that encourages CORs to develop a contract administration plan along with a quality assurance surveillance plan—both OFPP best practices. Contract administration plans are intended to provide a broad overview of the contract, including areas of risk and concern. Quality assurance plans are meant to be used to evaluate the quality and timeliness of the products and services produced by the contractor, and include performance standards, the methods of assessment, and an assessment schedule.

Along with policy development, FEMA has adopted an intranet-based system to centralize and electronically store NFIP contracting-related documents, including monitoring reports. In 2008, we recommended that FEMA implement a process to better ensure that CORs submit monitoring reports on time, systematically review these reports, and retain the reports in a quality assurance file. Several FEMA divisions use this intranet-based system. In part to address our 2008 report, FEMA staff in Risk Insurance involved in contract administration began using the system in November 2010. The Risk Insurance division intranet-based system is used by CORs to store and share COR-generated reports. The Risk Analysis division uses the intranet-based system for storing contract documents, particularly contract modifications and task orders submitted by the regional offices.

FEMA Divisions Have Contract Management Guidance That Follows Best Practices

The Risk Insurance and Risk Analysis divisions developed contract management guidance that follows best practices and implements the COR Handbook. More specifically, in December 2012 Risk Insurance finalized the Contracts Management Reference Guide. The guide closely follows the COR handbook and provides Risk Insurance CORs with a high-level overview of FEMA, DHS, and federal acquisition guidance and directs CORs to relevant reference materials. It also follows many of the practices recommended in the OFPP guide for contract administration. For instance, the Contracts Management Reference Guide lists implementation of a contract administration plan and a quality assurance surveillance plan as critical contract administration activities.

Furthermore, FEMA’s Risk Insurance division developed additional guidance intended to help ensure consistency in handling the failure of a contractor to meet performance standards. Specifically, Discrepancy Report Procedures guide CORs in such instances. In 2008, we recommended that FEMA develop such guidance after finding that the agency did not coordinate information and actions related to deficiencies for certain contractors. The Discrepancy Report Procedures include guidance for the COR to coordinate with the contracting officer to ensure that the contractor is notified and corrective action is taken.

FEMA’s Risk Analysis division has guidance for contract management, but took a different approach from the Risk Insurance division for its design and implementation because of the nature of the Risk MAP program. Risk Analysis established a Risk MAP Contracts Council to help ensure coordination, integration, and alignment of contracts across the Risk MAP program. The council comprises senior contracting officials and all CORs in the Risk MAP program.

The Risk Analysis division follows FEMA’s COR Handbook, but has not developed further guidance for CORs like the Risk Insurance Contracts Management Reference Guide. Instead, Risk Analysis developed a number of plans that provide general guidance for contract management staff across the Risk MAP program. FEMA’s COR Handbook recommends such planning documents, which also are consistent with best practices as described in the OFPP guide. For instance, the Program Management Plan, which is a narrative summary of the program’s processes and plans, provides overall contract management guidance for Risk MAP. Several other planning documents provide guidance on specific issues including risk management, quality assurance management, product delivery, and changing task orders. Unlike the Risk Insurance division, Risk Analysis has not developed a distinct deficiency report policy. According to officials, the Risk Analysis division follows an award fee plan for all of the related task orders. The fee plan sets performance metrics, such as timeliness and cost efficiency of community data collection, against which the contractor is evaluated, either quarterly or annually. If the performance metrics are not met, a portion of the award fee is to be deducted. If a performance problem at the project level becomes persistent and is not addressed by the contractor, Risk Analysis is to file a deficiency report similar to that used by Risk Insurance. We discuss these award fee plans and deficiency reports in greater detail in the next section.

DHS and FEMA Developed Guidance for CORs for Performance Reporting

DHS and FEMA have developed guidance to implement federal requirements to report their assessment of contractor performance in a government-wide database—the Contractor Performance Assessment Reporting System (CPARS). In July 2009, the Office of Management and Budget (OMB) required executive branch agencies to report assessment information about contractor performance into a central repository of contractor performance information, called the Past Performance Information Retrieval System. We previously found that DHS was transitioning to using CPARS to comply with this requirement. The primary purpose of CPARS is to better ensure that current, complete, and accurate information on contractor performance is available for use in procurement source selections. The CPARS guidance states that each assessment must include detailed and complete statements about the contractor’s performance and be based on objective data (or measurable, subjective data when objective data are not available) supported by program and contract/task order management data. Such performance assessment can significantly reduce the risk to the government on future awards. As a result, OMB urges agencies to ensure that all critical performance assessments are made available in a timely manner.

Both DHS and FEMA have incorporated this reporting requirement into acquisition regulations and program office guidance. For example, the regulations state that assessments should clearly, objectively, and completely describe the contractor’s performance in the narrative statement, in sufficient detail to justify the rating. Furthermore, the narratives should include a level of detail and documentation that provides evidence and establishes a basis for the assigned rating, an explanation of how problems were resolved, and the extent to which solutions were effective. FEMA developed standard operating procedures for CPARS that establish appointments and responsibilities for FEMA staff responsible for entering information into the system. In particular, the procedures state that the branch chiefs within each program office are ultimately responsible for oversight of its CPARS program. Also, the COR is responsible for oversight of the assigned contract and assisting the contracting officer with drafting the CPARS assessment. The standard operating procedures mandate basic CPARS training for CORs.

The FEMA COR Handbook and Risk Insurance Contracts Management Reference Guide include recording contractor performance in CPARS as a critical contract administration activity. The Handbook states that CPARS evaluations must be completed annually and at the conclusion of each contract. In particular, contracts lasting 2 or more years are usually to be evaluated annually and evaluations should be performed at least 120 days before the exercise of an option to extend the contract. The Contracts Management Reference Guide states that CPARS evaluations are due before the end of a task order, and includes reference to FEMA’s standard operating procedures for CPARS.

FEMA Largely Followed Contract Monitoring Procedures, but More Could Be Done Related to Quality Assurance Monitoring

For the three largest contracts we selected, FEMA generally followed its procedures for contract management in the three key areas that we identified for review—COR training and ethics requirements, performance measures, and ongoing monitoring and reporting (see table 1). But we found some opportunity for improvement related to quality assurance for one of the contracts and consistent reporting of contractor performance in CPARS for other two contracts.

CORs Met Training and Ethics Requirements

For each of the three contractors we reviewed, FEMA’s COR training and certification program requirements were met at the time of appointment. In 2010, the DHS Inspector General recommended that FEMA ensure NFIP staff received annual training on the roles and responsibilities of the contracting officer and COR. Because of the size and complexity of the three contracts we reviewed, the CORs had to maintain the highest level of certification. Initial and ongoing training requirements for FEMA CORs are clearly detailed in agency guidance. Initially staff needs 40 hours of training to assume basic COR responsibilities and 40 hours bi-annually of refresher training to maintain certification. Based upon data FEMA provided, each of the CORs completed training requirements at the time of appointment, and also satisfactorily documented their completed refresher training requirements for the contracts we reviewed. For contracting officers, FEMA also has implemented procedures that provide detailed guidance for contracting officers during each phase of the procurement process. FEMA has continued to update the guidance, and has made it available on its intranet.

For the contracts that we reviewed, we found documentation that the CORs had fulfilled the ethics requirement for 2013. The DHS Inspector General recommended in March 2010 that all FEMA employees involved in the procurement process receive annual procurement-specific ethics training and file financial disclosure forms. Initial ethics and procurement ethics training courses are included in the COR curriculum, and completion was validated by COR certification being awarded. Annual ethics training is mandatory for FEMA employees. Federal regulations require CORs and contracting officers to file a financial disclosure form. FEMA’s Office of the Chief Counsel requires FEMA CORs and contracting officers to file a financial disclosure form within 30 days of being appointed to a new position and annually. Supervisors review financial disclosures for potential conflicts of interests that could arise from stock holdings, investments, or personal relationships with a contractor. Any potential conflicts discovered by the supervisor during their review are referred to the chief counsel’s office for further determination. The filer and supervisor are notified of the chief counsel’s determination, which may include restrictions or precautionary measures that should be taken. Based upon our review of data provided by FEMA, no disqualifying conflicts of interest were reported for 2013 by the chief counsel’s office for any CORs or contracting officers working on the contracts we reviewed.

FEMA Established Relevant Performance Measures for the Contracts Reviewed

For each of the three contracts we reviewed, FEMA had established performance measures that were relevant and measureable to the contracts’ objectives and goals. According to federal government internal control standards, appropriate and relevant performance measures and indicators must be established and continually compared against program goals and objectives. The standards also recommend incorporating performance data into operating reports that inform management of inaccuracies or internal control problems. Our review of contract documentation for each of the three contracts that we selected indicated FEMA has established performance measures that met these standards. FEMA periodically compared and analyzed actual performance data against goals for each of the three contracts through operating reports, which would allow management to review the status of deliverables and milestones and be aware of inaccuracies or exceptions that could indicate internal control problems.

Use of Performance Metrics: BSA Contract

The BSA contract, which FEMA awarded in 2008, contains clearly defined appropriate performance areas that link to contract objectives. Many of the performance areas measure whether the contractor rendered the service within the required time frames. Specifically, the contract objectives cover services such as coordinating disaster response; maintaining the WYO Financial Control Plan; analyzing actuarial, claims, and underwriting data; coordinating communication; and maintaining NFIP information technology systems. The performance areas used to measure contractor performance address the timeliness of disaster response, the management of materials, such as manuals and rate tables, and the operation of the NFIP accounting system. The performance areas also included the submission of program management plans and communication services on a timely basis. A contractor that did not meet a performance requirement could have its annual target fee reduced. For example, the disaster response performance area required that the contractor deploy disaster teams within 24 hour notice by the COR. If the contractor did not deploy teams within that time, the COR could reduce the annual target fee by one-fifth. FEMA staff stated that they had never reduced the fee for the BSA contractor over the life of the contract we reviewed.

Risk Insurance has tracked the performance data for the BSA contract through monthly monitoring reports submitted by the COR and through operating reports, referred to as “dashboard reports” (see fig. 3). The monthly monitoring reports, which the COR compiles using records of activities and interactions from the reporting month, as well as technical progress reports submitted by the contractor, tracked the accomplishment of each performance area established in the contract. Using information from the contractor’s financial reports, the COR also submits dashboard reports on a monthly basis, which summarized financial and budgetary data, the status of the contract’s schedule, and any risk issues, such as the need for additional field and office support during Superstorm Sandy. By identifying risk areas, the dashboard reports are intended to make management aware of any issues. As shown in figure 3, the Program Management Office reviewed the monthly reports submitted by the COR for completeness and conducted oversight to identify any issues requiring management action. The contracting officer for the BSA contract said the COR copied her on correspondence with the contractor.

We reviewed the COR monthly reports for April 2012 to August 2013 and found that the BSA contractor had met all the identified performance measures for each month. Additionally, we reviewed operating reports for the BSA contract from January 2012 until December 2012 and found that the BSA program was consistently within budget and schedule constraints during the period reviewed.

Use of Performance Metrics: Risk MAP Contracts

Similarly, the two Risk MAP contracts we reviewed contained clearly defined and appropriate performance metrics that were linked to the contract objectives and program goals. FEMA established overall program objectives for the Risk MAP program, and then established contractor-specific performance measures through the award fee plan. The overall objectives of the Risk MAP program include (1) producing Risk MAP products that are aligned with user needs; (2) strengthening partnerships to develop improved understanding of flood risk and hazard mitigation; and (3) promoting the use of Risk MAP products at the local level for loss reduction. For the contractor-specific performance measures established in the award fee plans, FEMA set similar performance metrics for each mapping contractor because they conduct similar work in the various FEMA regions. The award fee plan for both contracts contained metrics related to Risk MAP deployment and mitigation action, timeliness of map packages and percentage of communities adopting maps, timeliness and cost efficiency of community data collection, customer satisfaction, and the quality of the administrative process. The plan also outlined performance metrics for regional task orders.

Additionally, the award fee plan described the performance assessment process and established the award amount to be earned based on the contractors’ performance. More specifically, for each of the performance metrics, the plan included standards for performance, the award-fee amounts available, and descriptions of each standard. As illustrated in figure 4, the COR reviewed quarterly self assessments submitted by the contractors to ensure continuous comparison of performance to goals. The COR also reviewed monthly progress reports, which contained performance data tied to contract objectives. As part of the performance assessment process, the COR and the contractor then reached agreement on the COR’s assessment. The COR then submitted the assessment to the Award Fee Review Board, which consisted of voting members familiar with the mapping program. The Award Fee Review Board recommended an amount to be awarded and the fee determining official approved the amount. FEMA officials stated that the contractors usually received between 90 and 95 percent of the available award fee. According to the contracting officer for the two mapping contracts, the contracting officers assigned to mapping contracts meet weekly with the CORs to discuss upcoming milestones. We found through our review of award fee determinations by FEMA officials that the two Risk MAP contractors generally collected most of the award fees available during fiscal year 2012.

In addition, FEMA’s Risk Analysis division management regularly reviewed operating reports, referred to as Joint Program Reviews (JPR), to monitor the performance of the Risk MAP program (see fig. 4). The contractors submitted the monthly technical reports to the project management contractor, which then compiled the reports for the JPRs and submitted them to Risk MAP management for review. We reviewed fiscal year 2012 JPRs for the mapping contractors and found that the JPRs were organized by program objectives, and contractor performance metrics provided data on progress made in meeting the objectives. The reviews were produced monthly for headquarters contracts and quarterly for regional contracts based on data contained in an earned value management system, into which the contractors enter deliverables, and monthly technical reports. The reviews also summarized performance results for all contractors and showed results for individual contractors.

Ongoing Monitoring Has Improved, but One Contract Did Not Have a Surveillance Plan and Performance Assessments Were Not Always Reported

Since we reported on FEMA’s monitoring of NFIP contracts in 2008, the agency has improved its ongoing monitoring of contractors based on our review of the three largest contracts. In 2008 we found that FEMA staff did not consistently follow key monitoring procedures, including maintaining a quality assurance file and consistently implementing performance deficiency procedures. We recommended that FEMA implement a process to ensure that monitoring reports are submitted, reviewed, and maintained in a quality assurance file, and implement written guidance on how to consistently handle a contractor’s failure to meet performance standards. Contract monitoring best practices that we identified recommend that agencies conduct ongoing monitoring by maintaining contract files, fully document modifications, develop contract administration plans, develop quality assurance surveillance plans, consistently implement deficiency guidance, and conduct annual performance assessments (see app.I).

For each of the three contracts we reviewed, we examined (1) the completeness of the quality assurance files maintained by the CORs, (2) documentation of modifications made to the contracts, and (3) deficiency reports issued related to the contracts since 2011. We found that overall FEMA maintained the appropriate files for the three contracts. Specifically, we found that the files contained proper documentation on the modifications made to the contracts. The BSA contract had few modifications and included changes such as the assignment of new contracting officers, the revision of the performance work statement, and extension of the terms of the contract. The two mapping contracts have regional subcontracts, which are set up through task orders. The mapping contracts had multiple modifications and task orders, with varying degrees of complexity. For example, a contract was modified to allow a contractor to adopt new mapping technology, and another contract was modified to adjust reporting deadlines.

FEMA actively used deficiency reports to identify and resolve performance problems for the three contracts we reviewed. For example, in 2011, FEMA issued deficiency reports for each of the three contracts. The purpose of deficiency reports is to record when a contractor does not meet a performance objective. FEMA filed one deficiency report for the BSA contract in 2011 and documented the contractor’s corrective action to resolve the issue. FEMA also issued three deficiency reports in 2011 for the mapping contracts we reviewed. One contractor received two deficiency reports and provided corrective action plans that FEMA accepted. The other contractor received one deficiency report and provided a corrective action plan. We found that FEMA followed the respective divisions’ deficiency report guidance and resolved the performance problems. FEMA officials stated that they have not issued any deficiency reports related to these contracts since 2011.

Quality Assurance Surveillance Plan Was Not Developed for One of the Contracts Reviewed

Although FEMA has generally improved its ongoing monitoring of contractors, the agency did not develop a quality assurance surveillance plan for one of the contracts. FEMA developed surveillance plans for the two mapping contracts we reviewed, but agency officials did not develop one for the BSA contract. As discussed earlier, quality assurance surveillance plans are required for performance-based contracts according to regulation, agency guidance, and best practices. According to these sources, the plan should be prepared in conjunction with the statement of work and should include, among other things, the performance standards, the methods of assessment, an assessment schedule, a description of corrective actions to be taken if required performance standards are not met, and incentives to be applied. FEMA officials stated that they relied on a brief reference to quality assurance in the BSA contract and other guides rather than develop a detailed surveillance plan. Specifically, the current BSA contract mentions a “Government Quality Assurance Plan” and includes a performance requirements summary with general performance metrics. The contract, however, does not specify how often deliverables would be monitored and evaluated. Risk Insurance division officials also stated that they used the general Contracts Management Reference Guide as the quality assurance plan for the BSA contract. However, the Contracts Management Reference Guide was intended to provide an overview of the COR’s responsibilities and activities, and does not discuss the method of assessment the COR will undertake, how often it with be done, or actions the COR will take if performance is not met. The guide itself calls for the development of a quality assurance surveillance plan.

In contrast, our 2008 report found that FEMA had a quality assurance surveillance plan for a past BSA contract. Further, FEMA developed a detailed quality assurance surveillance plan for the two mapping contracts. For example, the quality assurance surveillance plan for the mapping contracts outlined management roles and responsibilities, quality standards, and the process for ensuring quality control. Quality assurance surveillance plans that focus on the quality, quantity, and timeliness of performance outputs would likely avoid challenges monitoring the contractor’s performance. Additionally, without detailed quality assurance surveillance plans, the expectations of the COR and contractor can be misaligned during performance assessments. For example, in 2010 and 2011, FEMA identified persistent issues with the BSA contractor’s deliverables, including quality and timeliness, and faced challenges in resolving those issues, which may have been avoidable if a quality assurance surveillance plan had been developed and used.

FEMA officials acknowledged the need to develop quality assurance surveillance plans. They also stated that various actions are under consideration that may help ensure that quality assurance surveillance plans are in place for future contracts. However, FEMA officials did not provide specifics about these actions or provide a date for when they would determine which actions, if any, to implement. Until FEMA develops specific actions and implements them, it is unclear as to whether FEMA has effective internal controls in place to ensure that quality assurance surveillance plans are developed for future FEMA contracts.

Performance Assessments Not Reported for Two of the Contracts Reviewed

FEMA did not report performance assessments in CPARS for two of the three contracts we reviewed. While FEMA filed CPARS reports for the BSA contract for 2011 and 2012, the agency did not file any for the two mapping contracts. As discussed earlier in the report, DHS uses CPARS reports to assess and report performance on previous contracts, and applicable regulations and contract management guidance require entry of contract performance assessments in CPARS. The Risk MAP contracts we reviewed were multiyear contracts active in fiscal years 2011, 2012, and 2013. Per the guidance in FEMA’s COR Handbook, the CORs should have reported performance assessments at the end of the contract period in 2011 and 2012, at least 120 days prior to exercising the next option. Agency officials acknowledged that the CORs were supposed to report such information, but had not done so. FEMA staff responsible for the mapping contracts stated that CPARS assessments were a low priority at that time. FEMA staff responsible for the mapping contracts have acknowledged the importance of CPARS and stated that completing past CPARS assessments is now a higher priority.

As discussed earlier, internal control standards require that management emphasize to program managers their responsibility for internal controls and ongoing monitoring. Not submitting CPARS assessments is disadvantageous to both the contractor and the government. Receiving a positive CPARS assessment can enhance a contractor’s reputation when bidding on future contracts, and as such, the assessments provide an incentive for the contractor to perform as expected. For example, FEMA officials stated that for the BSA contract, the contractor’s performance improved after receiving a low rating in CPARS in 2011. Furthermore, not submitting assessments reduces the information available to other acquisition professionals in government when determining contractor selection. When selecting contractors, the FAR requires agencies to consider past performance as one assessment factor in most competitive procurements.

Conclusions

FEMA relies heavily on contractors to accomplish the objectives of NFIP. FEMA has made progress in developing contract management guidance that addresses weaknesses identified in our past reports and that is consistent with best practices in contract management. However, the inconsistent application of the guidance that we identified in our review of the three largest contracts indicated that continued attention to internal controls is warranted. Specifically, FEMA did not develop a quality assurance surveillance plan for the current BSA contract, which is critical to assessing the quality and timeliness of the products and services produced by the contractor. Regulations and agency guidance direct that such plans be used. FEMA officials stated that they are considering various options to ensure that the plans are in place for future contracts. However, without detailed surveillance plans, the expectations of the COR and contractor can be misaligned during performance evaluations and may not focus on the quality, quantity, and timeliness of performance outputs. Moreover, by taking actions to determine the extent to which quality assurance surveillance plans have not been prepared and the reasons why, FEMA would be better positioned to take action, as needed, to address these reasons and help improve contract management, which we have identified as a high-risk area.

In addition, the CORs for the Risk MAP contracts we reviewed have not entered ratings for the contractors into CPARS, the contractor assessment reporting database used by DHS that feeds into a government-wide contractor past performance database. CPARS data are key to informing federal contracting decisions and applicable regulations and agency guidance requires the use of CPARS to assess contractor performance. Not submitting CPARS assessments disadvantages contractors and the federal government because it reduces the information available to federal acquisition staff when determining contractor selection. As with the surveillance plan, by determining the extent to which this situation exists, identifying the root cause, and implementing steps to address the root cause, as appropriate, FEMA can help ensure that its—and other agencies’—contracting decisions and management draw on complete, relevant, and timely performance information.

Recommendations for Executive Action

While FEMA has begun to consider how to address the lack of a quality assurance surveillance plan for the BSA contract and to enter Risk MAP contractor performance assessments into CPARS, we are making two recommendations aimed at improving monitoring and reporting of contractor performance.

To help ensure FEMA’s contract monitoring provides a consistent, structured, and transparent method to assess contractor services, the FEMA Administrator should determine the extent to which quality assurance surveillance plans have not been developed for FEMA contracts; identify the reasons why quality assurance surveillance plans were not developed; and develop additional actions as needed to address the reasons to help ensure that quality assurance surveillance plans are developed for its future awards.

To help ensure that federal contracting officials have complete and timely information about the performance of contractors, the FEMA Administrator should determine the extent to which CPARS assessments have not been completed for FEMA contracts; identify the reasons why CPARS assessments were not completed; and develop additional actions as needed to address the reasons to help ensure that assessments (ratings) for FEMA contractors are reported in CPARS on a timely and consistent basis.

Agency Comments and Our Evaluation

We provided a draft of this report to FEMA for review and comment. In written comments, FEMA concurred with our recommendations. FEMA’s comments are reprinted in appendix II. FEMA also provided technical comments, which we incorporated in the report as appropriate. In response to our first recommendation, FEMA informed us that they have reviewed all active NFIP contracts and have directed all COR staff that do not have quality assurance plans in place to develop such plans. FEMA also noted that quality assurance surveillance plans are required by DHS acquisition guidelines and stated that they are rapidly working to improve compliance in this area with an estimated completion date of January 31, 2014.

In response to our second recommendation, FEMA stated that reduced COR staff for the Risk MAP program resulted in the inability to complete CPARS assessments in a timely manner. FEMA stated that they are in the process of identifying additional COR staff. In the meantime, CORs in the Risk Analysis division have been working to complete outstanding assessments and they planned to complete the assessments by December 31, 2013. FEMA also stated that as part of its governance plan, it plans to document the CPARS assessment process and commit to completing future reports within 30 days of receiving notification that assessments are due.

FEMA’s actions in response to our recommendations are steps in the right direction to improve compliance with contract oversight requirements. Effective controls are necessary to help ensure that FEMA fully implements required monitoring and reporting of contractor performance. For instance, while FEMA is in the process of developing quality assurance surveillance plans for all contracts, as we also recommended, FEMA will need to make certain that it has determined the reason(s) that a quality assurance surveillance plan had not been developed for the BSA contract to provide greater assurance that such plans are prepared as required going forward.

We are sending copies of this report to the Secretary of Homeland Security. In addition, the report is also available at no charge on the GAO website at http://www.gao.gov.

Should you or your staff have questions concerning this report, please contact me at (202) 512-8678 or garciadiazd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

Section 100231 of the Biggert-Waters Flood Insurance Reform Act of 2012 mandates GAO to review the three largest contractors used in administering the National Flood Insurance Program (NFIP) of the Federal Emergency Management Agency (FEMA). We examined (1) FEMA’s progress in updating its process for monitoring NFIP contractors since our prior reports, and (2) the extent to which FEMA followed its monitoring process for the largest NFIP contractors.

To determine FEMA’s progress in updating its process for monitoring NFIP contractors, we analyzed Department of Homeland Security (DHS) and FEMA policies, directives, guidance, and other materials related to contract management, including the use of a government-wide contractor assessment database and FEMA’s electronic document management system. Additionally, we compared FEMA’s process for monitoring NFIP contractors with relevant legislation, Standards for Internal Control in the Federal Government, contract management best practices, and related reports on NFIP from GAO and Office of the Inspector General at DHS. We assessed the extent to which FEMA followed its monitoring procedures for three large NFIP contracts—the Bureau and Statistical Agent (BSA) contract and two flood mapping contracts. Using data provided by the FEMA Office of the Chief Procurement Officer, we identified the 10 largest contractors for NFIP based on the amount of funds obligated to NFIP contractors over the previous 5 fiscal years (from 2008 through 2012). We verified the accuracy of the data by comparing them with data in the DHS Service Contract Inventories database and the Federal Procurement Data System. We determined that the contracting cost data provided by Office of the Chief Procurement Officer were sufficiency reliable for our purposes. To narrow the number of contractors selected for review, we determined whether they had an active NFIP contract during the course of our review. Additionally, we selected contractors responsible for administering different NFIP functions, such as flood mapping and insurance management. As shown in table 2, the dollars obligated to the three contractors we selected represented approximately 45 percent ($405 million out of a total of $909 million) of the overall active contracting dollars obligated during the 5-year period from fiscal years 2008 through 2012.

To assess the extent to which FEMA followed best practices and procedures for contract management, we compared FEMA’s contract management practices to criteria outlined in the following sources:

Standards for Internal Control in the Federal Government;

A Guide to Best Practices for Contract Administration;

FEMA, Contracting Officer’s Representative Handbook;

FEMA Risk Insurance Division, Contracts Management Reference

FEMA Risk Insurance Division, Discrepancy Report Procedures; and

FEMA Risk Analysis Division, Risk MAP Award Fee Plan.

We grouped key contract management processes into three key areas using the sources listed above: training and ethics requirements for contracting officer’s representative (COR), performance measures, and ongoing monitoring and reporting. As part of our assessment, we reviewed relevant plans and documentation and interviewed FEMA staff. We determined whether the information was fully, partially, or not present, and recorded the presence of this information by accordingly marking the answers “yes,” “partially,” “no,” or “not applicable.” The following table identifies the specific best practices and procedures that we used and our evaluation of the three contracts we reviewed: the BSA contract and the two Risk Mapping Assessment and Planning (Risk MAP) contracts.

For each of the selected contracts, we reviewed the statements of work, monthly contractor monitoring reports, and discrepancy reports. We collected available data from FEMA and conducted interviews with representatives from FEMA on their contract management roles and responsibilities. We also consulted with representatives of DHS’s Office of the Inspector General and interviewed the contractors we selected for further insight on the extent to which FEMA followed monitoring policies and procedures.

We conducted this performance audit from January 2013 through January 2014, in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Homeland Security

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Acknowledgments

In addition to the contact named above, Harry Medina, Assistant Director; Philip Curtin; Juliann Gorse; Josie Benavidez; William R. Chatlos; Julia Kennon; Marc Molino; Barbara Roesmann; Jessica Sandler; and William T. Woods, made key contributions to this report.