Federal Protective Service: Actions Needed to Assess Risk and Better Manage Contract Guards at Federal Facilities
Released: 2012-09-10T00:00:00

Letter

Background

To achieve its facility protection mission, in fiscal year 2012, FPS has a budget of $1.3 billion; over 1,200 full-time employees; and about 12,500 contract security guards. Contract guards are responsible for controlling access to federal facilities, screening access areas to prevent the introduction of weapons and explosives, enforcing property rules and regulations, detecting and reporting criminal acts, and responding to emergency situations involving facility safety and security. FPS relies on the fees it is authorized to charge federal tenant agencies in GSA- For controlled facilities for its security services to fund its operations.example, FPS charges tenant agencies a basic security fee (currently $0.74 cents per square foot) to, among other things, conduct FSAs, monitor alarms and dispatch operations, and perform law enforcement activities.

FPS’s FSA process generally entails: gathering and reviewing facility information; conducting and recording interviews with tenant agencies; assessing threats, vulnerabilities, and consequences to facilities, employees, and the public; and recommending countermeasures to federal tenant agencies.

To carry out this process, FPS’s long-term goal has been to develop a tool that aligns with DHS’s NIPP risk-management framework and Interagency Security Committee (ISC) standards. According to the NIPP, a risk assessment should assess threats, vulnerabilities, consequences, and recommend countermeasures, specifically:

A threat assessment is the identification and evaluation of adverse events that can harm or damage an asset.

A vulnerability assessment identifies weaknesses in physical structures, personal protection systems, processes, or other areas that may be exploited.

A consequence assessment is the process of identifying or evaluating the potential or actual effects of an event, incident, or occurrence.

After these three assessments are completed, the information is used to determine whether a facility’s risk is low, medium, or high. Additionally, the NIPP and ISC state that an agency’s risk assessment methodology should be credible (or complete) as able to assess the threat, vulnerability, and consequences of specific acts; reproducible as able to produce similar or identical results when applied by various security professionals; and defensible as able to provide sufficient justification for deviations from the ISC defined security baseline.

In addition, as part of its FSA process, FPS also uses the ISC’s Facility Security Level Determination for Federal Facilities to determine the facility security level (FSL). The ISC recommends that level I and II facilities be assessed every 5 years and level III and IV facilities every 3 years, and according to the ISC’s criteria:

A level I facility may be 10,000 or fewer square feet, have fewer than 100 employees, provide administrative or direct service activities, and have little to no public contact.

A level II facility may be 100,000 or fewer square feet, have 250 or fewer employees, be readily identifiable as a federal facility, and provide district or statewide services.

A level III facility may be 250,000 or fewer square feet, have 750 or fewer employees, be an agency’s headquarters, and be located in an area of moderate crime.

A level IV facility may exceed 250,000 square feet, have more than 750 employees, house national leadership, and be located in or near a popular tourist destination.

Since 2000, FPS has used three different tools to assess federal facilities and the assessment has varied, as shown in table 1.

FPS Does Not Currently Assess Risks at Federal Facilities, but Multiple Agencies Are Conducting Their Own Assessments

FPS Is Not Completing Risk Assessments

In the absence of RAMP, FPS currently is not assessing risk at the over 9,000 federal facilities under the custody and control of GSA in a manner consistent with federal standards such as NIPP’s risk management framework, as FPS originally planned. As a result, FPS has accumulated a backlog of federal facilities that have not been assessed for several years. According to FPS data, more than 5,000 facilities were to be assessed in fiscal years 2010 through 2012. However, we were unable to determine the extent of the FSA backlog because we found FPS’s FSA data to be unreliable. Specifically, our analysis of FPS’s December 2011 assessment data showed that 9 percent—or nearly 800—of the approximately 9,000 facilities did not have a date for when the last FSA was completed. According to the NIPP, to be considered credible a risk assessment must specifically address the three components of risk: threat, vulnerability, and consequence. We have reported that timely and comprehensive risk assessments play a critical role in protecting federal facilities by helping decision makers identify and evaluate potential threats so that countermeasures can be implemented to help prevent or mitigate the facilities’ vulnerabilities.

Although FPS is not currently assessing risk at federal facilities, FPS officials stated that the agency is taking steps to ensure federal facilities are safe. According to FPS officials, its inspectors monitor the security posture of federal facilities by responding to incidents, testing countermeasures, and conducting guard post inspections. In addition, since September 2011, FPS’s inspectors have been collecting information about federal facilities, such as location, purpose, agency contacts, and current countermeasures (e.g., perimeter security, access controls, and closed-circuit television systems). According to FPS officials, inspectors have collected information for more than 1,400 facilities that will be used as a starting point to complete FPS’s fiscal year 2012 assessments. However, FPS officials acknowledged that this is not a credible risk assessment that addresses threat, vulnerability, and consequence consistent with NIPP’s risk management framework. Moreover, several FPS inspectors told us that they received minimal training or guidance on how to collect this information and expressed concern that the facility information collected could become outdated by the time it is used to complete an FSA.

Multiple Federal Agencies Are Conducting Their Own Risk Assessments

We reported in February 2012 that multiple federal agencies have been expending additional resources to conduct their own risk assessments, in part because they have not been satisfied with FPS’s past assessments. These assessments are taking place even though according to FPS’s Chief Financial Officer, FPS received $236 million in basic security fees from federal agencies to conduct FSAs and other security services in fiscal year 2011. For example, an Internal Revenue Service (IRS) official said that IRS completed its own risk assessments based on concerns about risks unique to its mission for approximately 65 facilities that it also paid FPS to assess. A Federal Emergency Management Agency (FEMA) official stated that FEMA has assessed its own facilities for several years because of dissatisfaction with the facility security levels that FPS assigned to its facilities. Similarly, Environmental Protection Agency (EPA) officials said that EPA has conducted its own assessments based on concerns with the quality and thoroughness of FPS’s assessments. EPA officials noted that the agency’s assessments are conducted by teams of contractors and EPA employees, cost an estimated $6,000 each, and can take a few days to a week to complete. An official from the U.S. Army Corps of Engineers told us that it duplicates FPS’s assessments at some of its regional facilities because the agency follows U.S. Army force protection regulations, rather than FPS’s security requirements.

GSA is also expending additional resources to assess risk. We reported in October 2010 that GSA officials did not always receive timely FPS risk assessments for facilities GSA considered leasing. GSA seeks to have these risk assessments completed before it takes possession of a property and leases it to tenant agencies. An inefficient risk assessment process for new lease projects can add costs for GSA and create problems for both GSA and tenant agencies. Therefore, GSA is updating a risk assessment tool that it began developing in 1998, but has not recently used, to better ensure that it has timely and comprehensive risk assessments. GSA officials told us that in the future they may use this tool for other physical security activities, such as conducting other types of risk assessments and determining security countermeasures for new facilities. However, as of June 2012, FPS has not coordinated with GSA and other federal agencies to reduce or prevent duplication of its assessments.

FPS Lacks Reliable FSA Data

In addition to not having a tool that allows it to conduct risk assessments, FPS does not have reliable FSA data, which has hampered the agency’s ability to manage its FSA program. For example, as mentioned previously, we found that 9 percent—or nearly 800—of the approximately 9,000 facilities in FPS’s dataset were missing a date for the completion of their last FSA, thus raising questions about whether facilities have been assessed as required.reliable and timely information regarding when inspectors provided FSA reports to tenant agencies. This information is important because federal tenant agencies rely on these reports to allocate funding for new countermeasures.

Additionally, we found that FPS does not have We also found that FPS’s reliance on its 11 regional offices to maintain FSA data has contributed to inconsistency among the regions. For example, each of the three regions we visited maintains FSA data in a different format. More specifically, each of the three regions collected similar information such as a facility’s identifier and address, but they differed in how they tracked FSAs. For example, one region tracked the dates an FSA was submitted, reviewed, and completed. Another region tracked only the date the FSA was completed. Separately, another region used multiple spreadsheets to track FSAs. These inconsistencies among the regions make it difficult to understand whether FPS can manage its FSA program nationwide.

In March 2012, DHS’s Inspector General (IG) also reported similar issues The IG found that FPS had not determined if any of the with FPS’s data.FSA data in RAMP were valid and thus needed to be preserved for future use. As a result, the IG stated that FPS risked incurring additional expenditures, including paying for the transfer of useless data or losing critical data, if it did not make a decision before June 2012, when its data maintenance contract expired. The IG recommended that FPS (1) identify the costs and benefits of two potential courses of action: maintaining the data in RAMP or transferring the data out of RAMP, and (2) review RAMP’s data to determine what was critical and what should be saved. FPS concurred with this recommendation and plans to take action.

FPS Efforts to Develop a Risk Assessment Tool Are Evolving, but Challenges Remain

FPS Has Developed an Interim Vulnerability Assessment Tool

In September 2011, FPS signed an inter-agency agreement with Argonne National Laboratory for about $875,000 to develop MIST by June 30, According to FPS’s MIST documentation, MIST is an interim 2012.vulnerability assessment tool that FPS plans to use until it can develop a permanent solution to replace RAMP. According to FPS officials, among other things, MIST will enable the agency to begin aligning its FSA process with NIPP’s risk management framework and ISC standards. In addition, according to FPS’s MIST documentation, MIST will address key shortcomings identified with the RAMP development effort, including lack of inspector involvement, limited testing, and an inadequate training program.

According to MIST project documents and FPS officials, among other things, MIST will also: allow FPS’s inspectors to review and document a facility’s security posture, current level of protection, and recommend countermeasures; provide FPS’s inspectors with a standardized way for gathering and recording facility data; and allow FPS to compare a facility’s existing countermeasures against the ISC countermeasure standards based on ISC’s predefined threats to federal facilities (e.g., blast-resistant windows for a level IV facility) to create the facility’s vulnerability report).

In addition, according to FPS officials, after completing the MIST vulnerability assessment, inspectors will use additional threat information gathered outside of MIST by FPS’s Threat Management Division and any local crime statistics to justify any deviation from the ISC-defined threat levels in generating a threat assessment report. FPS plans to issue the facility’s threat and vulnerability reports along with any countermeasure recommendations to the federal tenant agencies.

FPS officials stated that MIST provides several potential improvements over its prior assessment tools: FSRM, RAMP, and the FSA calculator and template. For example, in contrast to FSRM, MIST will provide a more standardized and less subjective way of both collecting facility information and recommending countermeasures. Since MIST uses the ISC recommended countermeasures for defined threat scenarios for each facility security level, FPS officials believe that MIST will increase the likelihood that inspectors will produce credible FSAs. In contrast, the risk scores generated by RAMP and the FSA calculator and template were not linked to ISC standards. Unlike RAMP, MIST will use a limited amount of GSA facility data that can be edited by FPS inspectors where a correction is needed, according to FPS officials. The inability to edit data in RAMP was a contributing factor to its failure to produce credible FSAs.

According to FPS officials, on March 30, 2012, Argonne National Laboratory delivered MIST to FPS on time and within budget. FPS began training inspectors on MIST and about how to use the threat information obtained outside MIST in May 2012 and expects to complete the training by the end of September 2012. According to FPS officials, inspectors will be able to use MIST once they have completed training and a supervisor has determined, based on professional judgment, that the inspector is capable of using MIST. At that time, an inspector will be able to use MIST to assess level I or II facilities. According to FPS officials, once these assessments are approved, FPS will subsequently determine which level III and IV facilities the inspector may assess with MIST.

FPS Increased Its Use of Project Management Best Practices in Developing MIST

Considered Alternatives

FPS officials said the agency completed an alternatives analysis prior to selecting MIST. We were not able to confirm this because FPS did not document its analysis. According to industry standards, documenting an alternatives analysis is important because it allows agency officials to: revisit decision rationale when changes occur, reduce the subjectivity of the decision making process, and, provide a higher probability of selecting a solution that meets multiple stakeholders’ demands.

FPS officials mentioned two existing tools that were considered for an interim assessment tool: NPPD’s Office of Infrastructure Protection’s (IP) Infrastructure Survey Tool (IST) and DHS Science and Technology Directorate’s (S&T) Integrated Rapid Visual Screening of Buildings (IRVS) tool. FPS officials said they became aware of a security survey conducted by IP for the February 2011 Super Bowl at Cowboys Stadium in Arlington, Texas. Based on that survey, FPS reviewed the IST, which is used by IP to examine existing security countermeasures (which include physical and other protective measures) at critical infrastructure facilities, such as hydro-electric plants and commercial facilities, by comparing their existing countermeasures to those at similar facilities. According to IP officials, the IST does not calculate risk, estimate consequences, or recommend countermeasures. The IRVS is a risk assessment tool that assesses risk using threat, vulnerability, and consequence; that can be adapted to individual agency’s needs; and that, according to an S&T official, was available to FPS at no cost. However, the Director of FPS decided that because of timeliness concerns and the opportunity to better share information within NPPD, FPS would develop a modified version of the IST to assess federal facilities until FPS could develop an FSA tool to replace RAMP.

In contrast to RAMP, FPS better managed MIST’s requirements as we recommended in 2011. Specifically, FPS’s Director required that MIST be an FSA-exclusive tool and thus avoided changes in requirements that could have resulted in cost or schedule increases during development. Requirements serve as the basis for establishing agreement among users, developers, and customers and a shared understanding of the system being developed. Managing requirements entails managing the capabilities or conditions that a product is required to meet to satisfy an agreement or standard.

However, FPS did not obtain GSA or federal tenant agencies’ input in developing MIST’s requirements. We have reported that leading organizations generally include customer needs when developing programs. information they need to make well-informed countermeasure decisions. FPS officials stated that they were considering getting feedback from GSA and federal tenant agencies.

GAO, Geostationary Operational Environmental Satellites: Improvements Needed in Continuity Planning and Involvement of Key Users, GAO-10-799 (Washington, D.C.: Sept.1, 2010).

Completed User Acceptance Testing

In March 2012, FPS completed user acceptance testing of MIST with some of its inspectors and supervisors, as we recommended in 2011. User acceptance testing is conducted to ensure that a system meets contract requirements and performs satisfactorily for the user of the program—in this case, FPS’s inspector workforce and their supervisors. The results of each test event need to be captured and used to ensure that any problems discovered are disclosed and corrected. We reported in 2009 that comprehensive testing that is effectively planned and scheduled can provide the basis for identifying key tasks and requirements. Testing can also better ensure that a system meets those specified requirements and functions as intended in an operational environment.

According to FPS officials, user feedback on MIST was positive from the user acceptance test, and MIST produced the necessary output for FPS’s FSA process. For example, the inspectors who were involved in the testing found the methodology understandable and credible and had no significant problems logging in and using MIST. FPS’s testing identified the following problems: wireless connectivity issues at the testing location resulting in dropped connections and some users with older software encountering problems loading MIST onto their computers. FPS officials stated that they are taking steps to address these issues, such as updating older software.

MIST Has Limitations as an Assessment Tool

FPS has yet to decide what tool, if any, will replace MIST, which is an interim vulnerability assessment tool. According to FPS officials, the agency plans to use MIST for at least the next 18 months. Consequently, until FPS decides what tool, if any, will replace MIST or RAMP, it will continue to lack the ability to assess risk at federal facilities in a manner consistent with NIPP, as we previously mentioned. We also found the following limitations with MIST: FPS did not design MIST to estimate consequence, a critical component of a risk assessment. Assessing consequence is important because it combines vulnerability and threat information to evaluate the potential effects of an adverse event on a federal facility. For example, consequence information is used to determine whether a terrorist attack on a federal facility may result in the loss of human lives, incur economic costs beyond rebuilding the facility, or have an adverse impact on national security. Three of the four risk assessment experts we spoke with generally agreed that a tool that does not estimate consequences does not allow an agency to fully assess the risks to a federal facility. As a result, while FPS may be able to identify a facility’s vulnerabilities to different threats using MIST, without consequence information, federal tenant agencies may not be able to make fully informed decisions on how to best allocate resources to protect facilities.

Both FPS and ISC officials stated that incorporating consequence information into an assessment tool is a complex task. FPS officials stated that they did not include consequence information in MIST’s design as it would have introduced a new component that was not part of the IST and would have taken more time to develop, validate and test, and that any changes in threats would necessitate corresponding changes to the estimated consequences. For example, if new threats to federal facilities were identified, FPS would have to modify MIST’s methodology to estimate the consequences and determine how those consequences could affect other previously identified threats. FPS officials do not know if this capability can be developed in the future, but they said that they are working with the ISC and S&T to explore the possibility. However, according to an S&T official, incorporating consequence is possible and S&T’s current IRVS tool does estimate consequences.

FPS did not design MIST to compare risk or assessment results across federal facilities. Consequently, FPS does not have the ability to take a comprehensive approach to risk management across its portfolio of 9,000 facilities and recommending countermeasures to federal tenant agencies. Instead, FPS takes a facility-by-facility approach to risk management. Under this approach, FPS assumes that all facilities with the same security level have the same security risk, regardless of their location. However, level I facilities typically face less risk because they are generally small store-front operations with a low volume of public contact, such as a small post office or Social Security Administration Office. In comparison, a level IV facility has a high volume of public contact and may contain high-risk law enforcement and intelligence agencies. We reported in 2010 that FPS’s facility-by-facility approach to risk management provides limited assurance that the most critical risks at federal facilities across the country are being prioritized and mitigated. FPS recognized the importance of having such a comprehensive approach to its FSA program when it developed RAMP and FPS officials stated that they may develop this capability for the next version of MIST.

FPS has not developed metrics to measure MIST’s performance, such as feedback surveys from tenant agencies. Measuring performance allows organizations to track progress toward their goals and gives managers critical information on which to base decisions for improving their programs. We and other federal agencies have maintained that adequate and reliable performance measures are a necessary component of effective management. should provide agency managers with timely, action-oriented information in a format conducive to helping them make decisions that improve program performance, including decisions to adjust policies and priorities. Without such metrics, FPS’s ability to improve MIST will be hampered. FPS officials stated that they are planning to develop performance measures for MIST, but did not give a time frame for when they will do so.

GAO, Homeland Security: The Federal Protective Service Faces Several Challenges That Hamper its Ability to Protect Federal Facilities, GAO-08-683 (Washington, D.C.: June 11, 2008).

FPS Faces Challenges in Overseeing Its Contract Guards

No Comprehensive System to Oversee Contract Guard Workforce

FPS does not have a comprehensive and reliable system to oversee its approximately 12,500 contract guards. In addition to conducting FSAs, FPS developed RAMP as a comprehensive system to help oversee two aspects of its contract guard program: (1) verifying that guards are trained and certified to be on post in federal facilities and (2) conducting guard post inspections. However, FPS experienced difficulty with RAMP because the contract guard training and certification information in RAMP was not reliable.conduct post inspections. For example, FPS inspectors we interviewed stated they could not use RAMP to conduct post inspections because of difficulty connecting to RAMP’s servers in remote areas and recorded post inspections disappearing from RAMP’s record without explanation. Although we reported some of these challenges in 2011, FPS did not stop using RAMP for guard oversight until June 2012. Consequently, it is now more difficult for FPS to verify that guards on post are trained and certified and that inspectors are conducting guard post inspections as required.

Additionally, FPS faced challenges using RAMP to According to FPS officials, the agency decided to no longer use RAMP for these and other reasons, including the expiration of the RAMP operations and maintenance contract in June 2012 and FPS’s decision to migrate data from RAMP. In the absence of RAMP, in June 2012, FPS decided to deploy an interim method to enable inspectors to record post inspections. FPS officials said this capability is separate from MIST, does not include guard training and certification data, and will not have the ability to generate post inspection reports. In addition, FPS officials acknowledged that this method is not a comprehensive system for guard oversight.

No Independent Verification of Contract Guard Information

FPS does not independently verify the guard training and certification information provided by guard contractors. FPS currently requires its 33 guard contractors to maintain their own files containing guard training and certification information and began requiring them to submit a monthly report with this information to FPS’s regions in July 2011. To verify the guard companies’ reports, FPS conducts monthly audits. As part of its monthly audit process, FPS regional staff visits the contractor’s office to select 10 percent of the contractor’s guard files and check them against the reports guard companies send FPS each month.

In addition, in October 2011, FPS undertook a month-long audit of every guard file for its contracts across its 11 regions. Similar to the monthly audits, regional officials explained that the “100 percent audit” included a review of the approximately 12,500 guard files for FPS’s 110 contracts to verify that guards had up-to-date training and certification information.

According to an FPS official, the audit was FPS’s first review of all of its contractors’ guard files and provided a baseline for future nationwide audits. FPS provided preliminary October 2011 data showing that 1,152 of the 12,274 guard files FPS reviewed at that time—9 percent—were deficient, meaning that they were missing one or more of the required certification document(s). However, FPS does not have a final report on the results of the nationwide audit that includes an explanation of why the files were deficient and whether deficiencies were resolved.

A guard company may have more than one contract with FPS. contributed to a situation in which a contractor allegedly falsified training information for its guards. In addition, officials at one FPS region told us they maintain a list of the files that have been audited previously to avoid reviewing the same files, but FPS has no way of ensuring that the same guard files are not repeatedly reviewed during the monthly audits, while others are never reviewed. In the place of RAMP, FPS plans to continue using its administrative audit process and the monthly contractor-provided information to verify that qualified contract guards are standing post in federal facilities.

Conclusions

FPS has taken some steps to improve its ability to assess risk at federal facilities but additional improvements are needed. Most notably, FPS has developed an interim vulnerability assessment tool that once implemented, may allow it to resume assessing federal facilities, which it has not done consistently for several years. However, FPS’s lack of progress in developing a risk assessment tool that meets federal physical security standards such as NIPP’s risk management framework is problematic for several reasons. First, FPS spent almost 4 years and $35 million dollars on RAMP and another $875,000 on MIST but still does not have a risk assessment tool that meets NIPP’s risk management framework that can calculate risk using threat, vulnerability, and consequence information. Second, without a risk assessment tool that can compare risks across its portfolio, FPS cannot provide assurance that the most critical risks at federal facilities are being prioritized and mitigated. Third, some federal agencies are expending additional resources to conduct their own risk assessments in addition to paying FPS to complete them. Fourth, federal tenant agencies do not have critical information needed to make risk-based decisions about how to upgrade the security of their facilities. Identifying ways to resolve these issues could greatly enhance FPS’s efforts to assess risk at federal facilities and reduce duplication of effort, among other things.

We recognize that MIST is an interim tool and is not yet fully implemented; however, it has limitations that could affect FPS’s ability to protect federal facilities and provide security services. FPS generally increased its use of our project management best practices, as we recommended, and we encourage it to continue to do so in any future development of a risk assessment tool. However, FPS has not improved the accuracy and reliability of its FSA and contract guard data as it agreed to do in response to our previous recommendation. Given that FPS is still experiencing difficulties managing its FSA data, we reiterate the importance of this prior recommendation and encourage FPS to take action to address it.

Finally, FPS recently decided to not use RAMP to oversee its contract guards, but still does not have a comprehensive and reliable system to ensure that its approximately 12,500 contract guards have met training and certification requirements, and that FPS’s guard post inspections are occurring in accordance with the agency’s guidelines. That FPS cannot ensure that its 33 contractors are providing accurate information on its guards is also problematic. Without a comprehensive and reliable system for contract guard oversight, FPS is relying primarily on information provided by guard companies. These issues raise important questions regarding the overall effectiveness of FPS’s oversight of its contract guard workforce.

Recommendations for Executive Action

Given the challenges that FPS faces in assessing risks to federal facilities and managing its contract guard workforce, we recommend that the Secretary of Homeland Security direct the Under Secretary of NPPD and the Director of FPS to take the following five actions: incorporate NIPP’s risk management framework—specifically in calculating risk to include threat, vulnerability, and consequence information—in any permanent risk assessment tool; coordinate with GSA and other federal tenant agencies to reduce any unnecessary duplication in security assessments of facilities under the custody and control of GSA; address MIST’s limitations (assessing consequence, comparing risk across federal facilities, and measuring performance) to better assess and mitigate risk at federal facilities until a permanent system is developed and implemented; develop and implement a new comprehensive and reliable system for contract guard oversight; and verify independently that FPS’s contract guards are current on all training and certification requirements.

Agency Comments

We provided a draft of this report to the Secretary of Homeland Security for review. DHS concurred with our recommendations and provided written comments that are reprinted in appendix II. DHS also provided technical comments that we incorporated where appropriate.

We are sending copies of this report to the Secretary of Homeland Security and the Director of the Federal Protective Service. As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to relevant congressional committees. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov. If you or your staff members have any questions concerning this report, please contact me at (202) 512-2834 or goldsteinm@gao.gov. Contact point for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff that made key contributions to this report is listed in appendix III.

Appendix I: Scope and Methodology

To examine the extent to which FPS is completing risk assessments without RAMP, we reviewed FPS’s current FSA procedures and data on completed and planned FSAs for fiscal years 2010 to 2012. Specifically, we reviewed FPS’s FSA data aggregated from its 11 regions to determine the extent of FPS’s FSA backlog. These data included the GSA facility identifier, address, city, state, zip code, FPS region, facility security level, date of the last FSA, and the date of the next scheduled FSA. However, we could not determine the extent of FPS’s FSA backlog because FPS’s data contained a number of missing and incorrect values that made it unreliable. We also visited 3 of FPS’s 11 regions and interviewed regional managers, area commanders, and inspectors about how they are completing FSAs in the absence of RAMP. We selected these 3 regions based on the number of federal facilities in the region and their facility security levels, the number of contract guards in the region, and geographic dispersion. Our work is not generalizable to all FPS regions. We also interviewed FPS headquarters officials to understand how the agency is currently conducting FSAs. During our visits to the selected 3 FPS regions, we spoke with officials from the General Services Administration, Department of Veterans Affairs, the Federal Highway Administration, Immigration and Customs Enforcement, and United States Citizenship and Immigration Services to obtain their perspectives on FPS’s assessment efforts. These agencies were selected because they are members of their facility security committees, which have responsibility for addressing security issues at their respective facilities and approving countermeasures recommended by FPS.

To determine the status of FPS’s efforts to develop an FSA tool, we reviewed FPS’s documents including: the interagency agreement, requirements plan, project plan, system test plan, and training plan for MIST. As applicable, we compared FPS’s efforts to develop an FSA tool to DHS’s National Infrastructure Protection Plan’s (NIPP) risk management framework and the Interagency Security Committee’s (ISC) standards, including the Physical Security Criteria for Federal Facilities and the Facility Security Level Determination for Federal Facilities. We examined FPS’s requirement and project documents to determine whether in developing MIST, FPS complied with selected GAO and industry project-management best practices, such as: conducting alternative analysis, managing requirements, and conducting user acceptance testing. These practices were selected because they are critical in developing information technology systems and we recommended in 2011 that FPS better manage its requirements and conduct user acceptance testing in developing future tools. We interviewed FPS headquarters and regional officials as well as inspectors, representatives from Argonne National Laboratory who are responsible for developing MIST, officials from NPPD’s Office of Infrastructure Protection, and four risk management experts. We selected our four risk assessment experts from a list of individuals who participated in the Comptroller General’s 2008 risk management forum. We interviewed these experts to discuss FPS’s efforts to assess risks to federal facilities and the benefits and challenges of a risk assessment.

To assess FPS’s effort to manage its contract guard workforce, we reviewed FPS’s guard oversight policies and procedures and RAMP’s September 30, 2011, post inspection data. During our visits to the selected three FPS regions, we interviewed FPS regional managers, area commanders, inspectors, three guard contractors, GSA, and other federal agencies about guard oversight. We also interviewed officials at FPS’s headquarters.

We conducted this performance audit from July 2011 through August 2012 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Homeland Security

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Tammy Conquest, Assistant Director; Geoffrey Hamilton; Greg Hanna; Grant Mallie; Justin Reed; Amy Rosewarne; and Frank Taliaferro made key contributions to this report.