Homeland Security: Federal Protective Service Could Better Measure the Performance of Its Control Centers
Released: 2006-10-30T00:00:00

Letter

MegaCenters Provide Alarm and Radio Monitoring and Dispatch from Four Locations

FPS MegaCenters provide federal agencies with three primary security services—alarm monitoring, radio monitoring, and dispatch—through four locations using a variety of IT systems. MegaCenters monitor intrusion, panic, fire/smoke, and other alarms. They also monitor FPS police officers’ and contract guards’ radio communication to ensure their safety and to provide information, such as criminal background or license plate histories, to officers upon request. In addition, they exercise command and control authority by dispatching FPS police officers or contract guards. MegaCenters also provide a variety of other services. For example, they notify federal agencies regarding national emergencies and facility problems and remotely diagnose problems with federal agency alarms. They also receive and transcribe FPS police officer incident reports. Individual MegaCenters may also provide unique services not provided by other MegaCenters, such as facility-specific access control and remote programming of alarms via the Internet. One MegaCenter also provides an after-hours telephone answering service for the Drug Enforcement Administration and for GSA building maintenance emergencies.

The MegaCenters are located in Battle Creek, Denver, Philadelphia, and Suitland. Each MegaCenter has a sister center with redundant capability as backup in case of a failure at that MegaCenter. Suitland is paired with Battle Creek, and Philadelphia is paired with Denver. A force of 1,014 FPS police officers and 6,842 contract guards is available for the MegaCenters to dispatch in response to alarms and other emergencies. In fiscal year 2006, the MegaCenters were supported by a budget of $23.5 million, which accounts for about 5 percent of FPS’s total budget. The MegaCenters are operated by 23 full-time federal employees—some of whom manage the centers—and about 220 private contractors to provide around the clock security services for over 8,300 federal facilities.

The MegaCenters rely on a variety of IT systems, communications systems, and other equipment to provide their security services. The IT systems enable MegaCenter staff to, among other activities, monitor alarms and radio communications of FPS police officers and contract guards. For communications systems, MegaCenters have regional and national toll-free numbers for tenants and the public to contact the MegaCenters during emergencies. Other equipment includes dictation machines, which enable FPS police officers to dictate reports about incidents that occur at facilities.

MegaCenters’ and FPS’s Performance Measures Are Not Linked

MegaCenters use various means to assess operations, but their performance measures have weaknesses and are not linked to FPS-wide performance measures. MegaCenter managers assess MegaCenter operations through a variety of means, including reviewing data about volume and timeliness of operations, listening to and evaluating a sample of calls between operators and FPS police officers and contract guards, and receiving informal feedback about customer satisfaction. FPS managers also have developed 11 performance measures for assessing MegaCenter operations: distribute emergency notification reports (also known as SPOT reports) within 30 minutes of notification; review problem alarm reports daily; obtain regular feedback about customer satisfaction from field continuously review all SPOT reports and other outgoing information to ensure 100 percent accuracy; transcribe dictated offense and incident reports into the database management system within 8 hours of receipt of the report; submit reviewed contractor billing reports and time sheets within 7 business days after the last day of the month; prepare and review contractor reports for quality assurance plan; maintain completely accurate (nonduplicative) case control numbers; meet Underwriters Laboratories (UL) guidelines and requirements test failover of alarm, radio, and telephone systems weekly; and monitor calls and review recorded call content for adherence to standard procedures at least monthly.

The Government Performance and Results Act of 1993 requires federal agencies to, among other things, measure agency performance in achieving outcome-oriented goals. Measuring performance allows organizations to track the progress they are making toward their goals and gives managers critical information on which to base decisions for improving their progress. We have previously reported on some of the most important attributes of successful performance measures. These attributes indicate that performance measures should (1) be linked to an agency’s mission and goals; (2) be clearly stated; (3) have quantifiable targets or other measurable values; (4) be reasonably free of significant bias or manipulation that would distort the accurate assessment of performance; (5) provide a reliable way to assess progress; (6) sufficiently cover the program’s core activities; (7) have limited overlap with other measures; (8) have balance or not emphasize one or two priorities at the expense of others; and (9) address governmentwide priorities of quality, timeliness, efficiency, cost of service, and outcome.

We assessed the 11 FPS MegaCenter performance measures against selected attributes: linkage to mission and goals, clarity, and measurable targets. Ten of the 11 MegaCenter performance measures were aligned with FPS’s mission to protect federal properties and personnel and with the MegaCenter program’s mission to provide high-quality and standardized alarm monitoring, radio monitoring, and dispatch. We found no link between timely review of contractor time sheets and billing statements and FPS’s mission, however, primarily because this measure seems to be related to administrative activities. In addition, while 6 of the 11 performance measures have measurable targets—a key component for measuring performance, none of the MegaCenter performance measures met the clarity attribute because FPS could not provide information about how managers calculate the measures—a key component in the clarity attribute. For example, the performance measure that the centers test the failover ability of alarm, radio, and telephone systems weekly is measurable because it has a quantifiable target but does not meet the clarity attribute because FPS could not describe its methodology for calculating it.

We also assessed whether, collectively, the MegaCenters’ 11 performance measures sufficiently cover their core program activities (i.e., alarm monitoring, radio monitoring, and dispatch) and address governmentwide priorities of quality, timeliness, efficiency, cost of service, and outcome. Most of the MegaCenter performance measures relate to the three core activities. For example, regular feedback on customer service and monthly review of operator calls cover aspects of the dispatch and radio-monitoring functions. Other performance measures, like distributing emergency notification reports in 30 minutes, help fulfill other critical support functions. However, two performance measures—reviewing contractor quality assurance plans and timely review of contractor time sheets and billing statements—relate to administrative activities that are not strictly related to MegaCenter core activities. Additionally, the MegaCenter performance measures do not collectively address all of the governmentwide priorities. The MegaCenter performance measures primarily address the governmentwide priorities of quality and timeliness.

For example, the MegaCenter measures pertaining to transcribing reports within 8 hours and reviewing recorded calls to see if the operator followed standard operating procedures address aspects of service timeliness and quality, respectively. None of the measures relate to the governmentwide priorities of efficiency, cost of service, and outcome.

Finally, FPS does not link MegaCenter performance measures to FPS-wide performance measures, specifically the patrol and response time measure. FPS established FPS-wide performance measures to assess its efforts to reduce or mitigate building security risks. The performance measures that FPS established were (1) timely deployment of countermeasures, (2) functionality of countermeasures, (3) patrol and response time, and (4) facility security index. The one measure that relates to the MegaCenters— patrol and response time—assesses FPS’s ability to respond to calls for service and measures the average elapsed time from when a law enforcement request is received (e.g., alarm, telephonic request from a building tenant, FPS police officer-initiated call) to the time an officer arrives at the scene. FPS’s goal is to reduce response times by 10 percent in fiscal year 2006. The MegaCenters are responsible for part of the patrol and response activity that is being measured because the MegaCenters receive alarms and emergency calls and dispatch FPS police officers or contract guards to the scene. However, although data pertaining to this activity exist in the MegaCenters’ records management system, they do not measure the timeliness of this activity, and FPS has not developed a performance measure that would identify the MegaCenters’ contribution toward meeting FPS’s measure.

Nine Selected Organizations Provide Some of the MegaCenters’ Services in a Generally Comparable Manner

The nine selected security organizations generally do not provide all three of the MegaCenters’ primary services. However, the services these organizations offer are provided similarly by the MegaCenters with the exception of a CAD system, which three organizations use and the MegaCenters do not. The MegaCenters provide three primary services (i.e., alarm monitoring, radio monitoring, and dispatch), and the selected organizations provide all or some of these three main services. For example, the Park Police provide all three services, while the private organizations focus on providing alarm monitoring and offer some services the MegaCenters do not. Like the MegaCenters, all of the private organizations reviewed have centralized operations: the number of their national control centers ranges from two to five. Work allocation (i.e., how incoming alarms and calls are assigned) among centers varies by organization but overall is similar to the MegaCenter structure. For example, most of the organizations assign calls and alarms to a specific center based on the geographic location of the call or signal. However, the Postal Inspection Service and one private organization are unique because they are able to allocate workload to centers based on demand and operator availability. The organizations use a variety of methods to measure the quality of their services, many similar to methods used by the MegaCenters. For example, like the MegaCenters, most review a sample of operator calls on a regular basis. Two entities have established measurable performance goals for their centers. While there are similarities in the services offered, number of centers, work allocation, and service quality appraisals between the organizations reviewed and the MegaCenters, three organizations use a CAD system, which the MegaCenters do not. A CAD system is a tool used by the Denver Police Department for dispatching and officer tracking and by the Postal Inspection Service for officer tracking. The Park Police also uses a CAD system with limited capabilities at its San Francisco center and plans to purchase and upgrade the system for all three of its centers. Selected organizations and associations referred to CAD systems as being beneficial for dispatching services by allowing for faster operator response, automatic operator access to standard operating procedures and response prioritization, and automatic recording of operator actions enabling easier performance analysis. Since 2003, FPS and DHS both have assessed MegaCenter technology and have identified needs for technology upgrades, including the installation of a CAD system for the MegaCenters. Our guide on IT investment decision making—based on best practices in the public and private sector—stresses that part of achieving maximum benefits from an IT project requires that decisions be made on a regular basis about the status of the project. To make these decisions, senior managers need assessments of the project’s impact on mission performance and future prospects for the project. While the MegaCenters have assessed their technology on many occasions and have determined that some refreshment is needed, FPS has not yet allocated the funding for such upgrades.

Conclusions

FPS MegaCenters play a key role in protecting federal facilities, those who enter these facilities, and the FPS police officers and contract guards whose calls the MegaCenters respond to and monitor. How well the MegaCenters are fulfilling their role and carrying out their responsibilities is uncertain because they do not generate much of the information that would be useful for assessing their performance.

To their credit, the MegaCenters have established performance measures for a number of their activities and operations, and these measures are aligned with the MegaCenters’ mission. However, the measures have weaknesses, both individually and collectively, compared with the selected attributes of successful performance measures that we have identified. Many of the individual measures are neither quantifiable nor clearly stated, and collectively the measures do not address the governmentwide priorities of efficiency, cost of service, and outcome. As a result, FPS cannot compare performance across the MegaCenters or over time, and without such information, FPS is limited in its ability to identify shortfalls and target improvements.

Although FPS has established an FPS-wide performance measure for response time—from the alarm to the FPS police officer’s arrival on the scene—that incorporates the MegaCenters’ operations, the MegaCenters have not established a comparable measure for their operations alone. Without such a measure, FPS cannot evaluate the MegaCenters’ contribution—from the alarm to the FPS police officer’s dispatch—to the FPS-wide measure for response time and identify opportunities for improvement.

Recommendations for Executive Action

We recommend that the Secretary of Homeland Security direct the Director of the Federal Protective Service to take the following three actions: establish MegaCenter performance measures that meet the attributes of successful performance measures we have identified; develop a performance measure for the MegaCenters that is directly linked to the FPS-wide response time measure and covers the scope of the MegaCenters’ operations, from alarm to dispatch; and routinely assess the extent to which the MegaCenters meet established performance measures.

Agency Comments

We provided a draft of this report to DHS, the Department of the Interior, and the U.S. Postal Service for their review and comment. DHS provided comments in a letter dated September 6, 2006, which are summarized below and reprinted in appendix II. DHS also provided technical comments, which we incorporated into the report where appropriate. The Postal Service informed us that it had no comments on this report. The Department of the Interior did not provide comments on this report.

DHS generally agreed with the report’s findings and recommendations. DHS stated that FPS and the U.S. Immigration and Customs Enforcement (ICE) have undertaken a comprehensive review of the MegaCenters to identify, among other things, ways in which performance can be better measured. DHS noted that through this broad approach, FPS personnel will be able to generate and track the kind of information necessary to assess the MegaCenters’ performance. This one-time review may help FPS identify information needed to assess the MegaCenters’ performance and, therefore, develop appropriate performance measures. In order to reliably assess performance over time, FPS should not only establish appropriate performance measures, but also routinely assess performance using these measures. We therefore clarified our recommendation to include the routine use of established performance measures to assess the MegaCenters’ performance. With regard to the report’s discussion of CAD system capabilities, DHS said that ICE’s Chief Information Officer is currently assessing the MegaCenters’ technology requirements and recognizes that previous studies have identified the need for technology upgrades. DHS indicated that the current assessment will have a meaningful impact on FPS’s technology capabilities.

As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies of this report to other interested congressional committees and the Secretary of Homeland Security, and DHS’s Assistant Secretary for Immigration and Customs Enforcement. We will also make copies available to others upon request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov.

If you have any questions regarding this report, please contact me at (202) 512-2834 or sciremj@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III.

Briefing for the Committee on Government Reform, U.S. House of Representatives, August 14, 2006

Since the 1995 bombing of the Alfred P. Murrah Federal Building in Oklahoma City and the September 11, 2001, attacks on the World Trade Center and Pentagon, terrorism has threatened the nation’s security, including the physical security of federal facilities.

The Homeland Security Act of 2002 created the Department of Homeland Security (DHS), a new federal department with the mission of preventing terrorist attacks within the United States, which includes safeguarding federal facilities.

DHS, through its Federal Protective Service (FPS), provides law enforcement and security services to federal agencies that occupy almost 9,000 facilities under the jurisdiction of the General Services Administration (GSA) and DHS, protecting millions of federal employees, contractors, and citizens. Under agreement, FPS authority can be extended to provide its law enforcement and security services to any property with a significant federal interest.

As part of its approach to facility protection, FPS provides support for its law enforcement and security services through four control centers (known as MegaCenters) located in Battle Creek, Michigan; Denver, Colorado; Philadelphia, Pennsylvania; and Suitland, Maryland.

Because of the important role MegaCenters play in assuring the safety of federal facilities and their occupants, our objectives were to: (1) Identify the services the MegaCenters provide and how they provide them. (2) Determine how FPS assesses and measures the performance of MegaCenter operations and how FPS links MegaCenter performance measures to FPS-wide performance measures. (3) Examine how the MegaCenters compare to selected security organizations in the services they provide and in the methods they use to provide them.

Document review: Reviewed the Memorandum of Agreement between GSA and FPS and other documentation related to MegaCenter services as well as documentation related to (1) FPS’s request for a computer aided dispatch (CAD) system for the MegaCenters; (2) past FPS assessments of MegaCenter operations; (3) FPS’s performance measures; and (4) FPS’s budget for the MegaCenters.

Interviews: Interviewed FPS officials, including MegaCenter branch chief and managers, and staff from the Program Review Office, Financial Management Division, and other offices; Immigration and Customs Enforcement’s (ICE) Budget Enforcement Office; and officials from selected public and private organizations; officials from security industry standard setting and accreditation associations (associations).

U.S. Customs and Border Protection U.S. Park Police U.S. Postal Inspection Service Denver Police Department 5 private security companies We conducted our review in accordance with generally accepted government auditing standards.

Remote monitoring of building alarm systems, radio monitoring, and dispatching of FPS police officers and contract guards are the primary services FPS MegaCenters provide. These and other services are provided around the clock from four locations across the country. Each MegaCenter has a sister center with redundant capabilities that can serve as an emergency backup and each is operated by full-time federal employees and private contractors. In addition, the MegaCenters have a fiscal year 2006 budget of $23.5 million and use a variety of information technology (IT) systems and other equipment to provide their services.

FPS MegaCenter managers assess MegaCenter operations through a variety of means, including reviewing information on the timeliness and volume of operations, listening to and evaluating a sample of calls between operators and FPS police officers and contract guards, and receiving informal feedback about customer satisfaction. FPS managers have also developed performance measures for assessing MegaCenter operations. Although these MegaCenter measures reflect some attributes of successful performance measures, they also contain some weaknesses because they are not always clearly stated or measurable, and do not address governmentwide priorities of efficiency, cost of service, and outcome. In addition, they do not directly measure key operations that would link to FPS-wide performance measures, which are (1) the timely deployment of countermeasures, (2) functionality of countermeasures, (3) patrol and response time, and (4) facility security index.

The nine selected organizations offer some of the MegaCenters’ primary services, and they deliver and assess the services they offer in a generally similar manner to the MegaCenters. For example, like the MegaCenters, many of these organizations have centralized their control center operations, have backup capability, allocate workload among control centers based on geographic location, and use regular call reviews as well as volume and time measures to assess the quality of the services they provide. A few organizations offer services the MegaCenters do not offer. One difference between the MegaCenters and the selected organizations is that three of these organizations use a CAD system, which the MegaCenters do not have. The MegaCenters have assessed their technology and have identified the need for a CAD; however FPS has not allocated funds for such a purchase.

FPS operations are solely funded through security fees and reimbursements collected from federal agencies for FPS security services.

These security fees consist of basic and building-specific security charges.

The basic security charges cover the security services that FPS provides to all federal tenants in FPS-protected buildings, which include such services as patrol, monitoring of building perimeter alarms and dispatching of law enforcement response (MegaCenter operations), criminal investigations, and security surveys.

The building-specific security charges are for FPS security measures that are designed for a particular building and are based on the FPS Building Security Assessment and its designated security level. Such measures include contract guards, X-ray machines, magnetometers, cameras, and intrusion detection alarms. Also, the tenant agencies may request additional security services such as more guards, access control systems, and perimeter barriers.

The above two charges are billed monthly to the tenant agencies. The basic security charge is the same for all tenants regardless of the type of space occupied and is a square footage rate. The building-specific security charge reflects FPS cost recovery for security measures specific to a particular building and the billing is handled differently for single- and multi-tenant buildings. Single tenant buildings—the tenant agency is billed for the total cost of the security measures. Multi-tenant buildings—the tenant agencies are billed based on their pro rata share of the square feet occupied within the respective building.

FPS uses a reimbursable program to charge individual agencies for additional security services and equipment that they request above the level determined for their building.

FPS bills the tenant agencies for FPS security fees they have incurred.

The agencies pay the fees into an FPS account in the Department of the Treasury, which is administered by FPS. Congress exercises control over the account through the annual appropriations process that sets an annual limit—called obligation authority—on how much of the account FPS can expend for various activities.

FPS uses the security fees to finance its various activities within the limits that Congress sets. The Department of Homeland Security Appropriations Act for fiscal year 2006 authorized $487 million in obligation authority for FPS expenses and operations. Through FPS’s security fees, funds are to be collected and credited to FPS’s account as an offsetting collection from tenant agencies.

Under the FPS reimbursable program, agencies request additional security services and equipment using a funded Security Work Authorization. Once the services are provided and the costs are expensed, FPS bills the agency for the costs, and the funds are transferred to the FPS account to offset the expenses FPS incurred.

The DHS Inspector General reported in 2006 that when FPS was part of GSA it budgeted and paid for FPS's annual administrative support costs such as financial management, human capital, and IT using funds beyond those generated by security fees. GSA estimated these FY 2003 support services to cost about $28 million. According to the report, beginning in FY 2004, neither DHS’s annual budget request nor DHS’s appropriations set aside funding for FPS's support services. In FY 2004, as a component of DHS, FPS paid almost $24 million for support services using funds from security fees only; a year earlier these services had been funded by GSA using funds not derived from fees.

Before GSA established the MegaCenters, FPS used regional and satellite control centers to monitor alarm systems, dispatch FPS police officers and contract guards, and perform criminal background checks. In total, there were 22 regional control centers and 12 satellite control centers, which were located throughout FPS’s 11 regions. Most regions had more than 1 control center.

In 1991, GSA conducted an internal review of the control centers. The review found that because of significant budgetary and personnel constraints over more than a decade, the control centers no longer performed well enough to ensure safe, effective, and efficient FPS actions to preserve life and property. GSA contracted with Sandia National Laboratories– the lead laboratory for U.S. Department of Energy security systems– to conduct an in-depth study of the control centers’ operation and make recommendations.

In 1993, Sandia issued its study entitled GSA Control Center Upgrade Program. The Sandia study identified serious shortfalls and problems that would require a more radical upgrade of the control centers at a much higher cost than originally believed. After validating the study’s findings, GSA determined that a multimillion dollar upgrade of all control centers would be prohibitively expensive. The study noted that the control centers could be consolidated to almost any level to achieve economies of scale. However, the study recommended against a single national-level control center because a second center would be needed to continue operations under catastrophe or failover conditions.

Background: Evolution of the MegaCenters, GSA concluded that the control center problems that the study identified were material weaknesses and reported them to Congress. FPS conducted an operational and technical review of the Sandia study’s findings, which provided a critical assessment of the control centers, a high-level concept of operations for the centers, and functional specifications for upgrading the centers. GSA decided to upgrade 11 control centers—one in each region—and address the weaknesses that the study had identified.

Within GSA, concerns were raised about the cost of upgrading 11 control centers, how many control centers were really needed, and whether the centers’ operations could be outsourced. GSA established a project team to investigate these concerns. The team contacted several public and private sector organizations that operate control centers. The team found that the organizations were consolidating their control centers but were unable to assume the operations of FPS control centers. A decision was made to consolidate additional centers and the multi-regional control center or “MegaCenter” concept was developed. GSA endorsed the MegaCenter concept. GSA assembled a core project team and hired contractors to design, plan, and supervise the construction of the centers.

The selected organizations use a variety of methods to

ensure centers are meeting their goals and providing quality services, many similar to the MegaCenters.

Comments from the Department of Homeland Security

GAO Contact and Staff Acknowledgments

GAO Contact

Mathew J. Scire (202) 512-2834 or sciremj@gao.gov.

Staff Acknowledgments

Other key contributors to this report were Gerald P. Barnes, Assistant Director; Deirdre Brown; Bess Eisenstadt; Colin Fallon; Brandon Haller; Richard Hung; Alex Lawrence; Gail Marnik; and Josh Ormand.