Federal Protective Service: Enhancements to Performance Measures and Data Quality Processes Could Improve Human Capital Planning
Published: 2016-03-24T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The federal security workforce plays a crucial role in meeting the growing challenges of protecting federal facilities. FPS, within the Department of Homeland Security (DHS), worked with NPPD to develop a staffing model and a Plan in 2013 and 2015 to help FPS manage its workforce. A 2015 Senate Appropriations Committee report included a provision for GAO to review the Plan . The committee also asked GAO to evaluate the staffing model. GAO examined (1) FPS's Plan and related human capital planning efforts and (2) how FPS designed and uses its staffing model. GAO assessed FPS's Plan and model to determine if they aligned with key workforce-planning principles and practices for designing staffing models. GAO identified these principles and practices from prior work and other sources. GAO also interviewed NPPD and FPS officials in headquarters and three regions selected to obtain regional variation such as in the number of FPS staff.
section_title: What GAO Found
paragraphs: The Federal Protective Service (FPS)—which protects about 9,500 federal facilities—developed a Strategic Human Capital Plan ( Plan ) and engaged in related efforts that generally align with most key principles GAO identified for strategic workforce planning. Specifically, FPS
solicited input from key stakeholders, such as its employees and the National Protection and Programs Directorate (NPPD)—FPS's parent organization responsible for managing and overseeing FPS's human capital efforts;
determined critical skills and competencies;
developed human capital strategies (i.e., programs, policies, and processes) tailored to address identified gaps and needs in its workforce; and
identified actions that build the organizational capability to support the strategies.
However, FPS has not fully developed performance measures to evaluate progress toward goals, which is also a key principle for strategic workforce planning. For example, FPS has not identified performance measures for all of the Plan's strategies, has not included targets for the identified performance measures (e.g., a desired target for the “attrition rate” measure), and has not linked the measures to FPS's human capital goals. GAO's work on measuring program performance has found that targets and linkages are among the attributes of successful performance measures. FPS and NPPD officials said they plan on developing measures with targets and linkages but have not yet established time frames for completing these tasks. Without performance measures that have targets and linkages, it will be difficult for NPPD and FPS to assess whether the Plan and related efforts are helping achieve FPS's human capital goals and its facility protection mission.
FPS designed its staffing model—which identifies the federal workforce the agency needs to meet its mission—consistent with most key practices GAO identified for the design of staffing models, and FPS uses the model to help make management decisions. Specifically, FPS's model includes:
work activities and the time required to perform them;
facility risk levels, which determine the frequency with which FPS must complete facility security assessments; and
input from key stakeholders, including NPPD and some regional officials.
FPS officials said they took steps, such as reviewing work hour estimates, to ensure the quality of data used in the model—another key practice. FPS currently uses the model to help make human capital planning and other management decisions, but NPPD and FPS have not identified time frames for updating the model since its last update in August 2013. Furthermore, FPS cannot assure data quality in future updates to the model because it has no documented process for ensuring data quality. Without time frames for updating the model and guidance on ensuring data quality, NPPD and FPS may not have accurate estimates of staffing needs to make management decisions.
section_title: What GAO Recommends
paragraphs: To improve FPS's human capital planning, GAO recommends that the Secretary of DHS direct NPPD and FPS to identify time frames for developing performance measures with targets that are explicitly aligned to FPS's goals, establish a plan and time frames for updating its staffing model, and develop and document guidance for ensuring the quality of staffing model data. DHS concurred with GAO's recommendations and outlined steps it plans to take to address them.

section_title: Letter
paragraphs: The federal security workforce plays a crucial role in meeting the growing  challenges of protecting federal facilities and their occupants from terrorist  attacks and other violent acts. The Department of Homeland Security’s  (DHS) Federal Protective Service (FPS)—a component of the National  Protection and Programs Directorate (NPPD)—has primary responsibility  for protecting about 9,500 federal facilities from such threats. Through its  federal security workforce, FPS provides facility security services that  include law enforcement, security, and emergency response. Our prior  work found that FPS faces several workforce-related challenges, such as  managing and addressing its current and future workforce needs. To  address these and other challenges, we recommended, and Congress  directed, FPS to develop a strategic human capital plan to help agency  leaders effectively use its federal security workforce to carry out its  mission of protecting federal facilities. FPS issued a final Strategic Human  Capital Plan (the Plan) in February 2015. FPS also developed a staffing  model, which identified the number and composition of the federal  security workforce needed to meet its mission.
A Senate Appropriations Committee report, which accompanied the  Department of Homeland Security Appropriations Act, 2015, included a  provision for us to evaluate the Plan. The committee also asked us to  evaluate FPS’s staffing model. This report examines: (1) whether FPS’s  Plan and related human capital planning efforts align with key strategic  workforce-planning principles and (2) how FPS designed and uses its  staffing model to help ensure that it has the workforce it needs to meet its  mission.
To address both objectives, we reviewed relevant laws and regulations,  FPS and DHS documents, and our prior work related to workforce  planning and human capital management. We interviewed or obtained  information from officials at FPS headquarters and three selected  regions, NPPD, and the General Services Administration (GSA). To  examine whether FPS’s Plan and related human capital planning efforts  align with key strategic workforce-planning principles, we reviewed and  assessed FPS’s fiscal year 2015 Plan and related efforts against five key  strategic workforce-planning principles. We obtained these principles  from our 2003 report on key principles for effective strategic workforce  planning. We compared these principles with guidelines in the Office of  Personnel Management’s (OPM) Human Capital Assessment and  Accountability Framework (HCAAF) that apply across federal agencies  and determined that the principles we developed are generally consistent  with OPM’s guidelines. We did not assess the effectiveness of the Plan  because FPS is still in the process of implementing it. To assess the  design of FPS’s staffing model, we reviewed FPS’s August 2013 model  (the latest available) and interviewed FPS officials to better understand  the process they followed to design the model and how they collected  data used in the model. We evaluated FPS’s design of the model using  standards for internal control in the federal government, guidance we  published in 2009 on assessing the reliability of computer-processed  data, and four key practices we identified on the design of staffing  models. We identified the key practices from our previous reports that  discussed staffing models, discussions with a physical security industry  association, and staff within our agency with workforce-planning  expertise. We also assessed the reliability of data FPS used in the  staffing model by reviewing available documentation, interviewing agency  officials knowledgeable about the data, and examining data entries in the  model for obvious errors in accuracy and completeness. We could not  determine the reliability of data used in the model as FPS did not provide  us with documentation on steps taken to ensure data quality, which is a  key practice discussed more fully in the report. Finally, to examine how  FPS uses its staffing model, we reviewed FPS’s staffing analysis  documents to understand FPS’s staffing levels and interviewed FPS  headquarter officials. We evaluated FPS’s use of the staffing model using  standards for internal control in the federal government and associated  guidance. We did not review the size and composition of FPS’s  workforce of Protective Security Officers (PSO) (i.e., contract guards)  because FPS did not include PSOs in its staffing model. See appendix I  for more details on our scope and methodology.
We conducted this performance audit from May 2015 to March 2016 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: As the primary federal agency responsible for providing security services  to about 9,500 federal facilities (a majority of which are GSA-held or  leased), FPS, among other things, enforces federal laws and regulations  aimed at protecting federal properties and the persons on such property  and investigates offenses against these buildings and persons. In  conducting its mission, the agency provides two types of activities: (1)  physical security and (2) law enforcement activities. As part of its physical  security activities, the FPS workforce conducts facility security  assessments, which consist of identifying and assessing threats to and  vulnerabilities of a facility, as well as identifying countermeasures (e.g.,  security equipment) best suited to secure the facility. The agency’s law  enforcement activities include proactively patrolling facilities, responding  to incidents, and conducting criminal investigations, among other things.  (See app. II for a list of activities FPS performs).
To carry out these activities in fiscal year 2015, FPS maintained a  workforce of 1,371 full-time equivalents (FTEs) at its headquarters and in  its 11 geographic regions (see fig. 1 below). FPS’s Plan and staffing  model focus on this federal workforce. This workforce consists of 1,007  law enforcement staff (inspectors, criminal investigators, and special  agents) performing physical security and law enforcement activities and  364 non-law enforcement staff providing mission support. FPS also  manages and oversees approximately 13,500 PSOs (i.e., contract  guards) posted at federal facilities. These PSOs have responsibility for  controlling access to facilities; conducting screening at access points to  prevent the entry of prohibited items, such as weapons and explosives;  responding to emergency situations involving facility safety and security;  and performing other duties.
FPS funds its operations by collecting security fees from federal agencies  that use FPS for facility protection. FPS collects a basic security fee of  $0.74 per square foot and an oversight fee to fund FPS for direct and  indirect costs associated with providing building- or agency-specific  security. The oversight fee is an additional 6 percent of the costs for  providing security services to a building or agency. FPS anticipates  collecting about $336.5 million in operating revenues from security fees  charged to federal agencies in fiscal year 2016.
Over the years, we have reported on FPS’s workforce-planning efforts  and challenges. For instance, in July 2009, we found that FPS faced  challenges with hiring and training new staff and did not have a strategic  human capital plan to guide its workforce-planning efforts. We  recommended that FPS take a strategic approach to managing its staffing  resources, including developing a human capital plan to better manage its  workforce needs. In October 2012, FPS implemented our  recommendation and issued an Interim Strategic Human Capital Plan. In  June 2010, we also identified several potential challenges that FPS may  face with obtaining the staffing needed to adequately protect federal  facilities, including funding challenges, difficulties in hiring inspectors, and  training backlogs.
Since 2011, Congress has required FPS to submit a strategic human  capital plan that aligns fee collections to workforce requirements based  on current threat assessments. To meet this requirement, FPS tasked  DHS’s Homeland Security Systems Engineering and Development  Institute (SEDI) to analyze FPS’s current organizational structure, position  allocations, and assignments of personnel to help prepare a strategic  human capital plan. In April 2012, SEDI developed a staffing model to  estimate the size and composition of the workforce FPS needs to meet its  facility protection mission. SEDI also helped develop FPS’s Plan, which  identifies human capital strategies FPS intends to implement. The Plan  states that the strategies will help the agency hire and retain people with  the skills needed to carry out its mission. Since 2012, FPS has updated  the staffing model and the Plan several times (see fig. 2). FPS last  updated the staffing model in August 2013 and the Plan in February  2015.
As FPS’s parent organization, NPPD has responsibility for managing and  overseeing FPS’s human capital efforts. For example, NPPD has  responsibility for recruiting and hiring FPS employees and providing  guidance on other human capital services, such as training. In August  2015, NPPD proposed restructuring its organization to improve its  management and operations. In December 2015, NPPD finalized a  Human Capital Strategic Plan for fiscal years 2016 through 2020, which  identified the overarching human capital goals and objectives for all  NPPD component agencies, including FPS. In January 2016, NPPD also  finalized a complementary operational plan that provides a road map of  the actions NPPD plans to take in fiscal year 2016 to meet the goals  established in its human capital plan.
section_title: FPS’s Human Capital Plan and Related Efforts Generally Align with Most Key Principles, but FPS Has Not Fully Developed Performance Measures to Evaluate Progress
paragraphs: FPS’s Plan and related human capital planning efforts generally align with  four of the five key principles for strategic workforce planning that we  identified. Specifically, we found that FPS developed its Plan consistent  with the first four principles described in table 1 below. FPS’s efforts to  develop the Plan and take the actions described in table 1 show a marked  improvement from 2009 when we found that that FPS did not have a  human capital plan. However, in this review we found that FPS has not  fully developed its human capital performance measures, which is the fifth  key principle for strategic workforce planning. In our 2003 report on key  principles for strategic workforce planning, we found that federal  agencies’ use of all five key principles can contribute to effective strategic  workforce planning. FPS officials told us that they intend to further  develop performance measures in the future. They also told us that they  are in the process of implementing the Plan and continue to review and  refine strategies described in the Plan to meet the agency’s needs as  they change.
What is strategic workforce planning?  Strategic workforce planning, also called  human capital planning, is a systematic  process that focuses on developing long-term  strategies for acquiring, developing, and  retaining an organization’s workforce to meet  its mission. Agencies may outline strategies— the programs, policies, and processes that  agencies use to build and sustain their  workforces—in a human capital plan or  through other human capital planning efforts.
FPS sought input from key stakeholders when it developed and  implemented the Plan and its human capital strategies. As noted in our  2003 report, involving top management can help set the overall direction  of the agency’s workforce planning and soliciting employee input on  workforce planning can help an agency better understand human capital  needs and identify ways to improve human capital strategies.
We found that FPS solicited input from its management and employees,  NPPD, and external stakeholders. This input helped inform the Plan’s  contents.
FPS management and employees: FPS officials provided several  examples of how FPS senior executives and employees provided  input into the Plan. For example, the officials said that the senior  executives set the strategic direction for FPS’s Plan and related  efforts. Furthermore, the senior executives meet regularly to discuss  broad human capital issues, such as actions the agency can take to  ensure that its workforce can address future needs. In addition, FPS  officials said that they administered surveys, held working groups, and  conducted interviews with their employees to identify specific human  capital issues, which helped shape the contents of the Plan. For  example, FPS established working groups to help develop employee  performance work plans. As discussed below, these performance  work plans identify critical core competencies and associated  performance standards for each position. These working groups  consisted of regional directors, area commanders, and other regional  staff. Furthermore, the Plan states that FPS intends to continue to  involve employees and obtain their feedback as the agency  implements the Plan.
NPPD: NPPD is responsible for providing human capital services  (e.g., recruiting, hiring) on behalf of FPS. FPS obtained input from  NPPD when developing and implementing the Plan and its related  efforts. For example, FPS and NPPD officials explained that NPPD  officials participated in various working groups to develop strategies  identified in the Plan.
External stakeholders: While FPS officials told us they did not directly  solicit input from any external stakeholder on the Plan, officials said  they solicit feedback from these stakeholders on FPS’s services,  which they used to inform the contents of the Plan. FPS interacts with  a number of external stakeholders. For example, FPS is responsible  for protecting all GSA-held or leased facilities, making GSA a key  customer and important stakeholder. According to FPS officials, FPS  interacts with GSA to ensure a coordinated effort for the protection of  federal facilities. According to officials, FPS also works closely with  entities such as the U.S. Marshals Service and Administrative Office  of the U.S. Courts to provide coordinated protection at U.S.  courthouses and the Social Security Administration to understand the  threat environment and additional protection measures that can  mitigate incidents. FPS officials told us that they solicit stakeholder  feedback through continuous discussions and annual surveys on the  services FPS provides.
subsections: section_title: FPS Continues to Identify and Document the Critical Skills and Competencies for Its Positions
paragraphs: FPS has identified and standardized skills and competencies its staff  need to carry out its activities and continues to work on developing this  area. According to our 2003 report, determining an agency’s critical skills  and competencies is essential to ensure that employees have the  necessary skill sets to meet the agency’s needs. FPS has identified  needed skills and competencies in the documents described below. FPS  continues to finalize some of these documents.
Performance work plans: FPS officials told us that in fiscal year 2015,  they standardized performance work plans for the majority of its  mission critical positions, such as inspectors and area commanders.  These plans identify critical core competencies and associated  performance standards for each position. For example, the core  competencies for an inspector include skills in customer service (e.g.,  working with the GSA to understand its needs), representing the  agency, and teamwork and cooperation, as well as technical  proficiency. FPS uses the competencies in these work plans to  systematically assess employees’ performance. Officials told us that  they have efforts in progress to complete the performance work plans  for other mission-critical and support positions.
Career and professional development guide: FPS plans to complete a  career and professional development guide, which it expects to  finalize in 2016, describing position-specific competencies, skills, and  tasks. FPS officials explained that the guide aims to help FPS direct  and track employee training—including required annual training—and  professional development to improve employee performance.
Position descriptions: FPS developed position descriptions for its  employees to clarify the role of specific positions, by listing the major  job duties, skills, and other requirements (e.g., security clearance)  needed for the position. For example, according to the position  description for the criminal investigator position, the duties of a senior- level investigator include conducting complex investigations that  require extensive coordination and planning. According to the Plan,  FPS intends to update the position descriptions. FPS officials told us  that they regularly work with NPPD to update the position descriptions  so that they reflect changes to position responsibilities and  requirements.
section_title: FPS Developed Human Capital Strategies Tailored to Address Gaps and Needs
paragraphs: How did the Federal Protective Service  (FPS) organize the human capital  strategies described in its Strategic  Human Capital Plan?  FPS’s Strategic Human Capital Plan (the  Plan) organized its human capital strategies  into five broad categories, which are the same  categories described in the Office of  Personnel Management’s Human Capital  Assessment and Accountability Framework.  Each category contains several strategies,  some of which are described below.
FPS’s Plan describes five categories, each of which contains several  human capital strategies (i.e., programs, policies, and processes) (see  sidebar). According to the Plan, the strategies will help FPS build and  sustain a workforce that can carry out its mission. For example, under the  talent management category and related recruiting and hiring strategies,  the Plan states that FPS intended to hire 109 employees in fiscal year  2015. The Plan also states that in addition to conducting recruitment  fairs, FPS intends to make its recruiting strategy more cost-effective by  leveraging internal and external partnerships to attract talent. Under the  training strategy, the Plan describes a training program, which identifies  courses that aim to equip FPS staff to assess, mitigate, and respond to  current and emerging threats to federal facilities.
In addition, FPS officials provided examples of how they have tailored  their strategies to address identified gaps and needs. For instance, FPS  adjusted its training program based on the results of a preliminary  assessment that identified gaps in training, according to FPS officials.  Specifically, FPS developed leadership, physical security, contracting  officer representative, and security technology courses to fill identified  gaps in employee training. FPS officials told us that once they finalize and  implement the performance work plans and career and professional  development guide, they plan to determine whether they have agency- wide gaps in skills and competencies and further refine the training  program to address these gaps. Furthermore, as discussed in greater  detail later in this report, FPS considered adapting some of its human  capital planning decisions based on gaps and needs identified from the  staffing model and other management tools. As our 2003 report found,  developing strategies tailored to address gaps, human capital needs, and  critical skills and competencies that need attention helps create a road  map for an agency to move from the current to the future workforce  needed to achieve program goals.
section_title: FPS Identified Actions That Build the Organizational Capabilities Needed to Support Human Capital Strategies
paragraphs: The Plan identified some educational and administrative actions to build  the capability needed to support its human capital strategies. We noted in  our 2003 report that such actions can help ensure that the strategies are  effectively, consistently, and fairly implemented. For example, the Plan  states that FPS intends to educate employees on new human capital  strategies. The Plan also states that FPS intends to develop tools and  issue guidelines to help managers administer various strategies, such as  a toolkit for managers with tips and guidance to help them retain staff.
Furthermore, FPS and regional officials also use administrative  authorities that can help them carry out hiring strategies identified in the  Plan. For example, Veterans Recruitment Appointment (VRA) authority  allows federal agencies, such as FPS, to make excepted appointments of  eligible veterans to specified positions without competition.  According  to FPS officials, these hiring authorities help ensure that FPS can  leverage various candidate pools to recruit and retain qualified personnel.  An FPS regional director explained that the VRA hiring authority allowed  him to fill the positions he needed in his region.
NPPD’s planned reorganization aims to improve the administration of  human capital efforts that support FPS. Specifically, NPPD and FPS  officials explained that as a part of its reorganization, NPPD intends to  place its human capital staff in FPS’s headquarters office. According to  this official, collocating NPPD human capital and FPS staff aims to  improve the administration of recruiting and hiring because it will allow  NPPD to more effectively and quickly meet FPS’s human capital needs  and priorities in these areas.
section_title: FPS Has Not Fully Developed Human Capital Performance Measures to Evaluate Progress toward Goals
paragraphs: FPS has taken initial steps to develop performance measures for some,  but not all, strategies discussed in the Plan. Specifically, the Plan  identified performance measures for strategies that fall under one of the  five broad categories—talent management—FPS used to organize its  strategies. See figure 3. However, FPS did not identify measures for  strategies that fall under the other four broad categories, such as  leadership and knowledge management and building a results-oriented  performance culture. FPS officials told us that they did not identify  performance measures beyond those related to talent management in  part because they were waiting for NPPD to finalize a human capital plan  that would be applicable to FPS and that would contain measures for the  other categories. They also noted that, due to resource constraints, they  focused more on implementing strategies described in the Plan than on  developing additional performance measures.
Furthermore, FPS did not identify targets for the performance measures  identified in the Plan. For example, the Plan identifies “quality and  effectiveness of training” and “attrition rates” as measures but FPS has  not identified associated targets for them. For example, FPS did not  identify a desired target for the “attrition rate” measure (e.g., reduce new  hires’ attrition rate by 3 percentage points over fiscal years 2017 through  2020). In our prior work, we have found that successful performance  measures contain targets, which can help agency managers evaluate  progress by comparing actual results to projected performance.
In addition, the Plan does not explicitly show how the performance  measures and associated strategies link to FPS’s human capital goals.  For example, as shown in figure 3, the Plan does not clearly link the  “attrition rate” performance measure that is associated with FPS’s  retention strategy to one or more of FPS’s five human capital goals. We  have previously found that explicitly linking performance measures to  goals and clearly communicating the linkage also helps make  performance measures successful because the linkages can help  agencies determine whether they are achieving their human capital and  agency goals.
FPS has taken some initial steps to develop targets and linkages but, as  was the case with developing additional performance measures, did not  complete these steps in anticipation that NPPD would finalize its own  human capital plan. For example, FPS began to collect data to help  identify appropriate targets and continues to work on this effort, according  to officials. Additionally, when developing the Plan, FPS developed a draft  document that shows the link between the identified performance  measures and the agency’s human capital goals. For example, FPS  linked its “attrition rate” performance measure to the agency’s third  human capital goal, which is to provide FPS with the tools, mechanisms,  and processes to improve workforce effectiveness, agility, and retention.
After the completion of our audit work in February 2016, NPPD and FPS  officials provided us with NPPD’s strategic human capital plan and  complementary operational plan, which NPPD finalized in December  2015 and January 2016, respectively. At this time, NPPD’s plans do not  include performance measures specific to FPS. An NPPD official who  played a key role in developing this plan said that, in the future, an FPS  operational plan that is aligned with NPPD’s human capital plan and  specifically reflects FPS’s human capital needs and strategies will be  developed. Further, NPPD and FPS officials told us that they will work  together to develop human capital performance measures relevant to  FPS. However, the officials’ plans are not clear because they have not yet  established time frames for addressing the issues we identified on  performance measures.
According to our key workforce-planning principles, agencies should  establish performance measures to evaluate an agency’s progress  toward reaching human capital goals and the contribution of human  capital activities toward achieving agency goals. Establishing  performance measures before an agency starts to implement its  strategies can help agency officials evaluate the human capital plan. If  FPS and NPPD do not develop performance measures, including targets  and linkages to goals, in a timely manner, neither agency can accurately  assess FPS’s progress in achieving its human capital goals or its agency  goal of sustaining a valued, skilled, and agile workforce or the contribution  of its strategies toward achieving these goals. Consequently, neither  NPPD nor FPS will know the extent to which the Plan and related  strategies are helping fulfill its mission of protecting federal facilities and  their occupants. Furthermore, it will be difficult for stakeholders—such as  Congress and the public—to hold FPS accountable for achieving its  goals.
section_title: FPS Designed Its Staffing Model to Align with Most Key Practices and Uses It to Make Management Decisions but Has Not Documented a Process to Ensure Data Quality
subsections: section_title: FPS Designed Its Staffing Model Consistent with Most Key Practices
paragraphs: FPS issued its latest staffing model in August 2013, which identified the  number and composition of FTEs the agency needs to meet its mission,  based on various data inputs, assumptions, and analyses. We  compared the design of this model to four key practices we identified for  the design of staffing models and found that FPS’s model reflects three of  these four key practices (see table 2). Specifically, we found that FPS  designed its staffing model to include (1) work activities performed by  FPS employees and the frequency and number of hours it takes to  perform them; (2) risk factors that affect the agency’s operational  activities, such as the security level and quantities of facilities; and (3)  input from key stakeholders. We found that while FPS officials took some  steps to ensure the quality of data used in the model, they did not  document a process for doing so. A staffing model that reflects all four  key practices can enable FPS officials to make informed decisions on  workforce planning with reliable estimates.
Work activities, frequency, and time required to conduct work  activities: We found that FPS’s staffing model includes data commonly  used in workforce analyses, such as data on work activities, and the  frequency and number of hours to perform them. Incorporating these  types of data into the staffing model helps estimate the number of  staff needed to carry out an agency’s activities, according to a key  practice we identified. SEDI officials reviewed documentation (e.g.,  relevant laws and regulations, FPS policies) to identify all FPS  mission and mission support activities (referred to as an activities  taxonomy)—and the frequency with which the identified activities are  performed. SEDI officials identified about 200 total activities and  associated tasks. As discussed in detail below, SEDI officials  consulted with key stakeholders to estimate the required time to  perform all mission-related work activities. SEDI officials used  information provided by these stakeholders because time constraints  precluded it from conducting real-time studies, according to an FPS  official. SEDI officials also calculated average productive labor hours  to populate the staffing model (1,548 hours for non-supervisory  physical security inspectors and 1,987 hours for criminal investigators)  based on assumptions about staff’s annual leave, sick leave, training  requirements, travel (for training), and time devoted to other tasks  (e.g., collateral duties). For instance, the productive labor hours used  in the model assumes that staff on average use 50 percent of their  sick leave each year.
To estimate the number of staff FPS needed in fiscal year 2013  (1,870 FTEs), SEDI officials used the data discussed above to  calculate the estimated total number of FTEs required to perform each  activity. Figure 4 provides an example of the steps taken to calculate  the FTEs needed for one activity—conducting security assessments  at a level 4 facility. SEDI officials then aggregated the FTEs for each  activity to identify the total estimated FTEs that FPS needs to carry  out its mission.
Risk factors: FPS officials incorporated operational risk factors in its  model, including the different security levels of federal facilities.  We  previously found that commonly used industry practices for staffing  models specific to law enforcement and physical security include  identifying operational risk factors, such as the security level of  facilities and posts to be secured or protected and identifying tasks  and time it takes to conduct those activities. A federal facility’s  security risk level determines the frequency with which FPS must  complete a facility’s security assessment. The model includes annual  targets for completing facility security assessments by facility security  level. For example, FPS officials estimated that it would complete 509  facility security assessments for security-level 4 federal facilities in  fiscal year 2013 and included these data in the model. In identifying  key practices for the design of staffing models, we found that  accounting for these operational risk factors helps determine the  number of staff and positions needed to mitigate potential threats to  federal facilities.
Key stakeholders: In designing the model, SEDI officials consulted  with key stakeholders and subject-matter experts, including FPS  headquarters officials, and some regional directors and managers.  According to an official, SEDI relied on the subject matter experts to  estimate the number of hours it should take to perform FPS  operational activities. A SEDI official also told us that they used these  experts because FPS staff perform unique activities and therefore, no  benchmarks exist for how long it takes to perform many of the work  activities, such as facility security assessments. FPS officials also  told us that they involved NPPD human capital officials to help identify  assumptions, such as leave estimates, that were used to calculate  productive labor hours. We have previously found that involving  stakeholders and subject matter experts when designing a staffing  model can help an agency ensure that the model reflects operating  conditions and meets user needs.
Data quality: FPS officials took steps to ensure the quality of the data  used in the model. We have defined data quality as the use of  relevant data from reliable internal and/or external sources based on  the identified information requirements. To help ensure data quality,  FPS officials told us that SEDI officials questioned subject matter  experts to obtain work activity hour estimates for performing some  work activities and to understand what the estimates included and  then revised them as needed to improve precision. For example, if  subject matter experts included travel time as part of the estimated  time to perform a facility security assessment, then SEDI officials  excluded the travel time from the original estimate and made it a  separate work activity with estimated time for completing it.  Additionally, FPS officials told us that they compared some work  activity hour estimates from the staffing model to the actual number of  hours it takes FPS staff to perform those activities from their Activity- Based Costing Model to identify differences and make corrections,  when needed, to reflect actual conditions. Further, FPS officials told  us that they regularly reviewed and provided feedback on SEDI’s  taxonomy and other data collection efforts to identify all FPS work  activities and estimated work hours as well as the underlying  assumptions used to develop some estimates, such as assumptions  related to productive labor hours. A SEDI official also told us that FPS  Operations and some regional officials reviewed the estimated hours  required to complete some work activities in the staffing model.
Although FPS officials took some steps to ensure the quality of data  provided by subject matter experts, they did not document the  agency’s process for ensuring data quality at the time FPS developed  the model, and we could not assess the reliability of data used in the  model. We found that some FPS staff questioned the quality and  reasonableness of the data in the model, particularly on work hour  estimates to complete some activities. Specifically, selected FPS  regional staff we spoke with told us that some work hour estimates did  not reflect their experience or actual operating conditions. For  example, all nine area commanders we spoke with stated that the  estimated time to complete a risk assessment of federal facilities with  a facility security level 3 (about 60 hours) was low. The area  commanders said that it takes about 80 to120 hours on average  because, similar to level 4 facilities, they need to interview multiple  tenants for the risk assessments. FPS headquarters officials told us  that they had estimated a range of 60 to 80 hours for conducting  assessments of federal facilities with a facility security level 3,  depending on the number of federal agencies and clients in the  federal facilities, but they used the 60-hour estimate in the model as  the nationwide average time to conduct those assessments. FPS  officials told us that when they update the model, they plan to validate  work hour estimates they obtained from subject matter experts and  use data from some new technologies, such as the Modified  Infrastructure Survey Tool (MIST), to better reflect actual operating  conditions. FPS headquarters officials also told us that the training  hours estimates in the model represent training requirements at the  time of developing the model and that when they update the model,  they plan to change the training hours used in the model to reflect  changes in training requirements.
section_title: FPS Uses Its Staffing Model to Help Make Management Decisions but Has Not Yet Updated It to Reflect Current Operating Conditions
paragraphs: We found that FPS uses the staffing model in conjunction with other  management tools, professional judgement, and institutional knowledge  to help inform human capital planning and budget requirements, as  described below.
Human capital planning decisions: An FPS official told us that FPS  uses the staffing model in conjunction with other management tools,  such as the Activity-Based Costing Model, to help make staffing and  human capital planning decisions. In particular, using the staffing and  Activity-Based Costing models,  FPS found that inspectors spent  less time than was predicted in some activities. For example, FPS  officials said that FPS found that inspectors spent less time than  predicted by the staffing model on overseeing countermeasures  services at agencies. An official said that FPS used this information to  evaluate and consider making changes to inspectors’ workloads and  staffing levels.
Budget requirements: In 2014 and 2015—in response to international  security events (e.g., shootings at the Canadian Parliament and in  Paris)—the Secretary of DHS instructed FPS to enhance its presence  and security at federal facilities for short periods of time. An FPS  official told us that FPS used the model to understand the impact of  the additional facility security responsibilities on its staff’s daily facility- protection workload. According to the official, analyses from the  staffing model, other management tools, and conversations with  regional office staff, showed that FPS needed additional staff  resources to maintain its law enforcement staff’s daily workload while  at the same time providing enhanced security operations. As a result  of this analysis, in July 2015, DHS notified federal agencies that it  would increase its basic security fee from $0.74 to $0.78 and its  oversight fee from 6 percent to 8 percent in fiscal year 2017.  According to a DHS memorandum sent to agencies using FPS  services, the fee increases, combined with internal efficiencies, will  allow the agency to sustain essential security operations and maintain  the agency’s capacity to rapidly surge personnel during increasingly  more common periods of heightened vulnerability in fiscal year  2017.
While FPS continues to use the model for the purposes described above,  it does not reflect some changes in FPS’s operating conditions that have  occurred since the model’s last update in August 2013. For example, this  staffing model does not reflect FPS’s plan to perform about 600 more  facility security assessments for level 3 and 4 facilities in fiscal year 2014  than it did in fiscal year 2013. We calculated that if FPS updated the  August 2013 model to reflect these additional facility security  assessments, FPS would have needed about 37 more FTEs in fiscal year  2014, each of which would have completed an average of about 17 of the  additional assessments. FPS officials acknowledged that the number of  facility security assessments it plans to complete can change and that  other operating conditions, such as the number of federal facilities FPS is  responsible for protecting, can change regularly. FPS’s operating  conditions can also change when security or agency needs change. For  example, some GSA facilities are becoming more technologically  advanced. To address security needs at those facilities, FPS officials told  us that in conjunction with the NPPD’s Office of Cybersecurity and  Communications, they plan to execute more of their protection  responsibilities as they relate to the nexus of cybersecurity and physical  security. Furthermore, NPPD’s planned reorganization may result in  changes to FPS’s activities. Separately, FPS’s operations may change  for an extended period of time in response to unexpected events, such as  when threat levels change. As noted above, FPS officials said that while  they would like to update the model to reflect changes in operations, they  have not yet done so because of limited staff resources.
However, at the completion of our audit work in February 2016, FPS  officials told us that given the planned NPPD reorganization, NPPD has  not yet determined whether FPS will continue to have responsibility for  updating the staffing model, whether this responsibility will shift to NPPD,  or whether it will become a shared responsibility. Nonetheless, FPS and  NPPD have no specific time frames for updating the model. Standards for  internal control in the federal government and associated guidance state  that managers need timely analytical information to help make  management decisions. Furthermore, we have found in our prior work  that completing staffing models and regularly updating them in a timely  manner can help support agencies’ activities and decision making.  Without a clear plan and time frames for updating the staffing model to  reflect regular and unexpected changes in operating conditions, NPPD  and FPS will have limited assurance of the accuracy of the model’s  estimates of the number and composition of staff FPS needs to protect  federal facilities and their decisions regarding the FPS workforce.
section_title: FPS Does Not Have a Documented Process for Ensuring Data Quality in Future Updates
paragraphs: Although FPS took steps to ensure data quality when it developed the  August 2013 model, FPS does not have a documented process for  ensuring data quality when it updates the model to account for changes in  operating conditions in the future. As such, it is not clear whether the  model will use quality data that reflect current operating conditions.  Standards for internal control in the federal government and guidance we  have developed on assessing the reliability of computer-processed data  state that agencies should use a process to help ensure data quality.  Also, the internal control standards emphasize the importance of control  activities such as procedures for achieving an entity’s objectives.  Documented processes on data quality—such as guidance on how to  collect data, validate assumptions underlying the data, and perform  sensitivity analyses to assess the assumptions—can help ensure that  data used in the model are reasonably free from error and bias and  provide greater assurance to decision makers that they are using reliable  and sound information that is produced from the model. In June 2010, in  making preliminary observations about FPS’s workforce-planning efforts,  we emphasized the importance of taking steps to ensure the quality of  data used. Because FPS officials did not document a process for  assuring data quality during the development of the model and without  relevant guidance, FPS may not be able to ensure that future updates to  its staffing model will provide accurate estimates of staffing needs, putting  FPS at risk of not fully understanding whether it has the staff it needs to  perform its mission.
section_title: Conclusions
paragraphs: To carry out its mission of protecting federal facilities and their occupants  against potential terrorist attacks and other violent acts, FPS must ensure  that it has the right people with the right skills in the right positions, at the  right time. Over the years, however, we have identified several workforce- related challenges facing FPS, such as the absence of a strategy to  manage FPS’s current and future workforce needs. The completion of  FPS’s first strategic human capital plan and staffing model, therefore,  represents significant progress. Moreover, FPS’s development of both the  Plan and model largely align with recognized key workforce-planning  principles and staffing model practices.
While FPS has taken a number of positive steps to strategically manage  its workforce, we found that FPS does not have assurance that its efforts  will achieve its stated goals. FPS has not fully developed human capital  performance measures, and while both NPPD and FPS plan on taking  additional action in this area, future progress is uncertain because the  NPPD and FPS have not established a time frame for developing  additional measures. Until FPS and NPPD develop performance  measures with targets that clearly align with FPS’s stated human capital  goals, it will be difficult to determine whether FPS is on track to meet its  goals and mission, or needs to make adjustments. Furthermore, FPS’s  current staffing model has not been updated since August 2013. Until  FPS develops a plan and timeline for updating the model regularly and for  unexpected changes in operating conditions that last for an extended  period of time, FPS will have limited assurance on the model’s estimates  of the number of staff it needs to protect federal facilities. Finally, because  FPS did not document a process for ensuring data quality when it  developed the model, it is not clear whether future updates to the model  will accurately reflect changes in operating conditions. Without  documented guidance that describes the process FPS will use to ensure  data quality, FPS may not be able to ensure that its staffing model will  provide accurate estimates of staffing needs. As FPS’s parent  organization, NPPD has a critical role to play in managing and overseeing  FPS’s human capital efforts. Accordingly, NPPD and FPS need to work  together to ensure that they have the staff they need to perform their  facility protection mission.
section_title: Recommendations for Executive Action
paragraphs: To help FPS enhance its strategic human capital planning efforts, we  recommend that the Secretary of Homeland Security direct the Under  Secretary of NPPD to work with the Director of FPS to take the following  three actions:  identify time frames for developing human capital performance  measures with targets that are explicitly aligned to FPS’s stated  human capital goals, establish a plan and time frames for updating FPS’s staffing model  regularly and for unexpected changes in operating conditions, and develop and document guidance on the process FPS will use to  ensure the quality of its staffing model data, such as guidance on how  to collect data, validate assumptions, and perform sensitivity analyses  to assess the assumptions.
section_title: Agency Comments
paragraphs: We provided a draft of this report to DHS for comment. DHS concurred  with our recommendations and outlined steps it plans to take to address  them. DHS’s written comments are reproduced in appendix III. DHS also  provided technical comments, which we incorporated as appropriate.
We are sending copies of this report to the appropriate congressional  committees and the Secretary of the Department of Homeland Security.  In addition, the report is available at no charge on the GAO website at  http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at 202-512-2834 or rectanusl@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found at the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix IV.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The objectives of our report were to evaluate the Federal Protective  Service’s (FPS) human capital planning efforts, including its Strategic  Human Capital Plan (the Plan) and staffing model. Specifically, we  examined (1) whether FPS’s Plan and related human capital planning  efforts align with key strategic key strategic workforce-planning principles  and (2) how FPS designed and uses its staffing model to help ensure that  it has the workforce it needs to meet its mission.
To address both objectives, we reviewed relevant laws and regulations,  documents from FPS and the National Protection and Programs  Directorate (NPPD), and our prior work related to workforce planning and  human capital management. We also interviewed or obtained information  from officials at FPS, NPPD, and General Services Administration (GSA).  To determine regional staff’s involvement in developing the Plan and  staffing model, we obtained information from and interviewed FPS  regional directors and nine randomly selected area commanders  responsible for facility security in 3 of FPS’s 11 regions. We judgmentally  selected the 3 regions—Regions 7 (Greater Southwest Region), 10  (Northwest/Arctic Region), and 11 (National Capital Region)—to obtain  variation in the number of FPS-protected facilities and full-time equivalent  (FTE) employees; the number of facilities per FTE; geographic size (in  terms of square miles); number of square miles per facility in the region;  geographic location (i.e., east, central, and west locations); and whether  DHS’s Homeland Security Systems Engineering and Development  Institute (SEDI) visited the region when it developed the staffing model.  Because we judgmentally selected the FPS regions, our results are not  generalizable to all of FPS. We also interviewed an official from the  Federal Law Enforcement Officers Association, which is a nonprofit  professional association representing federal law enforcement officers,  and the International Association of Chiefs of Police to obtain their  perspective on workforce planning and staffing models.
To examine whether FPS’s Plan and related human capital planning  efforts align with key strategic workforce-planning principles, we reviewed  and assessed FPS’s fiscal year 2015 Plan and related efforts against five  key strategic workforce-planning principles. The five key principles  include:  1.  involving top management, employees, and other stakeholders in  developing, communicating, and implementing a strategic workforce  plan;  2.  determining critical skills and competencies needed for employees;  3.  developing strategies tailored to address gaps and needs;  4.  building the organizational capability needed to support human capital  5.  developing performance measures to evaluate progress toward  reaching human capital or agency goals.
We obtained these principles from our 2003 report on key principles for  effective strategic workforce planning. We compared these principles  with guidelines in the Office of Personnel Management’s (OPM) Human  Capital Assessment and Accountability Framework (HCAAF) that apply  across the federal government and determined that the principles we  developed are generally consistent with OPM’s guidelines. FPS officials  also told us that they based their Plan on the HCAAF guidelines. The five  key strategic workforce-planning principles can enhance the effectiveness  of an agency’s strategic workforce-planning and can help ensure that its  strategic workforce-planning process appropriately addresses an  agency’s human capital challenges, goals, and mission. We also  conducted interviews with FPS officials to obtain information on whether  FPS’s Plan and related human capital planning efforts addressed key  strategic workforce-planning principles. We interviewed an NPPD official  and selected FPS regional staff, as mentioned above, to understand their  involvement in FPS’s human capital planning efforts. We did not assess  how FPS tailored each of its strategies to address human capital needs  and gaps and critical skills and competencies that need attention. Rather,  we asked FPS officials to provide examples of how their strategies  addressed human capital needs and gaps. We also did not assess the  effectiveness of the Plan because FPS is still in the process of  implementing it. After the conclusion of our audit work in February 2016,  we received NPPD’s Human Capital Strategic Plan and we reviewed it to  compare to FPS’s Plan to the extent that time allowed.
To assess the design and use of FPS’s staffing model, we reviewed  FPS’s August 2013 model (the latest available) and relevant FPS  documents, and interviewed FPS officials to better understand the  process they followed to design the model and how they collected data  used in the model. We evaluated FPS’s design of the model using  standards for internal control in the federal government, our 2009  guidance on assessing the reliability of computer-processed data, and  key practices we identified on the design of staffing models. We identified  the key practices from our previous reports that discussed staffing  models, discussions with a physical security industry association, and  staff within our agency with workforce-planning expertise. We initially  identified 11 key practices, but used four key practices to evaluate how  FPS designed its staffing model. Three of the 11 practices were not yet  applicable because FPS had not yet assigned staff to manage the staffing  model, and we consolidated 8 of them because they had similar  characteristics. For instance, we consolidated 3 practices on addressing  data issues into one because all of them related to data quality. The four  key practices call for:  1.  incorporating work activities, frequency, and time required to conduct  2.  incorporating risk factors;  3.  involving key stakeholders; and  4.  ensuring data quality to provide assurance that staffing estimates  produced from the model are reliable.
These four key practices help provide reasonable assurance that the  design of the model will provide estimates to help management make  staffing and other decisions consistent with an agency’s mission.
As part of our review of FPS’s design of the staffing model, we also  assessed the reliability of data FPS used in the model by reviewing  available documentation, interviewing agency officials knowledgeable  about the data, and examining data entries in the model for obvious  errors in accuracy and completeness. FPS officials told us that some data  discrepancies we found in our assessment did not significantly impact the  estimated number and composition of staff needed to meet FPS’s  mission. Given the large volume of data in the model, we did not verify  this. We could not determine the reliability of data used in the model as  FPS did not provide us with documentation on steps taken to ensure data  quality, which is a key practice discussed more fully in the report.  However, to examine the quality of selected data inputs (e.g., estimated  time for completing certain work activities) in the model, we conducted  semi-structured interviews with randomly selected area commanders in  three regions, as mentioned above, to gauge the data’s reasonableness.  We cannot conclude that all the data input in the model are reasonable,  as we judgmentally selected some data inputs to verify. We also did not  talk to subject matter experts to determine how they identified some data  inputs, such as the number of hours the agency needed to complete work  activities.
Finally, to examine how FPS uses its staffing model, we reviewed FPS’s  staffing analysis documents to understand FPS’s staffing levels and  interviewed FPS headquarter officials. We evaluated FPS’s use of the  staffing model using standards for internal control in the federal  government and associated guidance. We did not verify whether the  staffing model identified the optimal workforce FPS needs to effectively  carry out its mission. We also did not review the size and composition of  FPS’s workforce of Protective Security Officers (i.e., contract guards)  because FPS did not include them in its staffing model.
We conducted this performance audit from May 2015 to March 2016 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Federal Protective Service’s Activities
paragraphs: The Federal Protective Service (FPS) classifies its activities into five  categories—primary activities, secondary activities, enabling activities,  support activities, and supplementary activities. See below for a  description of these categories and the activities FPS identified. Some  activities have a number of associated tasks.
Description of category  Activities essential to the performance of FPS’s mission  of protecting federal facilities, their occupants, and  visitors.
Law enforcement response  Protective investigations  Facility Security Assessments (FSA)  Protective Security Officer services  Technical countermeasure services  Law enforcement policing and patrol
Critical incident and special security operations
National Infrastructure Protection Plan (NIPP)
Activities performed as a result of primary activities.  The primary activity creates a need to perform the  secondary activity. Secondary activities can be directly  linked to the primary activities that they inform or  enhance.
Activities that help to sustain operations by providing a  foundation of required capabilities. Enabling activities  generally support a relatively broad set of primary and  secondary activities.
subsections: section_title: Description of category Activities that include headquarters or regional office operational management and oversight functions.
paragraphs: Activities that represent FPS services that extends  beyond its core mission.
section_title: Appendix III: Comments from the Department of Homeland Security
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Amelia Shachoy (Assistant  Director), Roshni Davé, Joseph Franzwa, Geoffrey Hamilton, Delwen  Jones, Jennifer Kim, Steven Lozano, Sara-Ann Moessbauer, Janice  Morrison, Joshua Ormond, Malika Rice, and Rebecca Shea made key  contributions to this report.