Transportation Security Administration: Surface Transportation Inspector Activities Should Align More Closely With Identified Risks
Released: 2017-12-14T00:00:00

Letter

Background

TSA Roles and Responsibilities

The Aviation and Transportation Security Act designated TSA as the primary federal agency responsible for securing all modes of transportation. In fiscal year 2005, Congress appropriated funds for surface transportation security, and the accompanying conference report directed that some of those funds go to rail compliance inspectors, the predecessors to today’s surface transportation security inspectors— referred to as surface inspectors.

Public and private transportation entities have the principal responsibility to carry out safety and security measures for their services. As such, TSA coordinates with public and private transportation entities to identify vulnerabilities, share intelligence information, and work to mitigate security risks to the system. See table 1 for examples of the entities TSA works with to secure the various surface transportation modes.

TSA Surface Security Budget and Regulations

In fiscal year 2005, $10 million of TSA’s surface transportation security appropriation was to hire and deploy up to 100 rail compliance inspectors. TSA assigned inspectors to oversee security and provide oversight and assistance to railroads, and subsequently, other surface transportation modes, including mass transit and passenger rail, freight rail, highway, and pipeline sectors. TSA has since increased the number of surface inspectors, and since 2013 has maintained more than 200 Full Time Equivalent (FTE) positions. See table 2 for additional details on the number of TSA surface inspector FTEs from fiscal years 2013 through 2017.

In August 2007, the 9/11 Commission Act was signed into law and required TSA to issue security regulations for freight and passenger rail, among other requirements. TSA also issued regulations governing surface transportation security on its own initiative. As of July 2017, TSA has issued the following regulations related to surface transportation:

Rail Inspections: Issued in November 2008, 49 C.F.R. part 1580 requires certain freight railroad carriers and passenger rail operations (passenger railroad carriers and rail transit systems) to designate a rail security coordinator, notify the Transportation Security Operations Center regarding any significant security concerns, and, if applicable, ensure a secure chain of custody of rail cars containing certain hazardous materials, and be able to provide location and shipping information for certain rail cars, among other things. The hazardous materials subject to this regulation include certain explosives, toxic inhalation hazardous materials (TIH), and radioactive materials. See appendix II for additional details.

Maritime Inspections: TSA also partners with the U.S. Coast Guard (USCG) in securing maritime ports, facilities and vessels. TSA’s responsibilities include enrolling Transportation Worker Identification Credential (TWIC) applicants, conducting background checks to assess the individual’s security threat, and issuing TWICs. In addition, TSA is authorized to conduct inspections of persons using TWIC to access the secured area of a regulated maritime facility.

TSA Organizational Structure for Managing Its Surface Inspectors

Surface inspectors work under the direct command authority of the Federal Security Director (FSD) in the field. As of fiscal year 2017, TSA used a staffing model to allocate surface inspector staff to 49 different field offices, separated into seven geographic regions around the country. According to TSA, all but one surface field office locations are at or near major airports. Figure 1 depicts surface field office locations by region.

Surface inspector policies and procedures, and operational oversight are managed separately.

Program Guidance: Within TSA’s Office of Security Operations, the Surface Compliance Branch plans surface transportation security activities and programs, and develops an annual work plan that lays out the minimum required activities to be completed for surface inspectors in the field. The Office of Security Policy and Industry Engagement (OSPIE) collects and analyzes data on certain surface inspector activities such as the Baseline Assessment for Security Enhancement (BASE) program, TIH attendance rates, and freight rail compliance rates; coordinates with industry stakeholders, and; develops strategic plans, among other things.

Operational oversight: The Assistant Federal Security Director for Inspections (ASFD-I) in each field office manages surface inspectors on a day-to-day basis, oversees the scheduling of surface inspector work plan activities, and reviews inspectors’ documentation of activities in PARIS, TSA’s system of record. FSDs are ultimately responsible for ensuring that surface inspectors complete their annual work plan requirements.

In 2010, TSA created the Regional Security Inspector (RSI) position in an effort to improve oversight of surface inspectors in the field and standardize inspections across field offices. One RSI is assigned to each of the seven geographic regions and serves as a liaison between TSA headquarters staff and surface inspectors in the field. Each RSI is also assigned to be the lead liaison between TSA and the Class I railroads within their assigned geographic region. See figure 2 for surface inspectors’ command structure as of 2017.

TSA Risk-Based Security for Surface Transportation

TSA documents state that it employs a risk-based approach for securing transportation modes and identifies managing risk as one of its strategic goals to help identify and plan security priorities and activities. According to TSA officials, TSA uses the National Infrastructure Protection Plan (NIPP) risk management framework and the DHS Risk Management Fundamentals as its primary risk guidance. In June 2006, DHS issued the NIPP which established a six-step risk management framework to establish national priorities, goals and requirements. Most recently updated in 2013, the NIPP defines risk as a function of three elements: threat, vulnerability and consequence. Threat is an indication of the likelihood that a specific type of attack will be initiated against a specific target or class of targets. Vulnerability is the probability that a particular attempted attack will succeed against a particular target or class of targets. Consequence is the effect of a successful attack. TSA uses the TSSRA, a bi-annual risk assessment that considers the three elements of risk to measure the risk of various terrorist attack scenarios, evaluate transportation modes, and identify surface security priorities.

Surface Inspectors Conduct Regulatory Inspections and Voluntary Security Assessments but TSA Has Incomplete Information on Their Activities

Surface Inspectors Enforce Regulations through Inspections and Assist Surface Transportation Entities on a Voluntary Basis

Surface inspectors conduct a variety of activities to implement TSA’s surface transportation security mission, including (1) regulatory inspections for freight and passenger rail systems, (2) regulatory TWIC inspections, and (3) non-regulatory security assessments and training which surface transportation entities participate in on a voluntary basis. Surface inspector activities are, in part, determined by an annual surface work plan that lays out the minimum required number of surface inspector activities to be completed by each field office. Specifically, the work plan requirements are designed to take up about one-third of inspectors’ available working hours, with the expectation that the other two-thirds of inspectors’ time will be used for related activities, such as documentation and follow-up, or other tasks as determined by local AFSD-Is and FSDs in the field.

To develop the annual surface work plan, officials from Office of Security Operation’s Surface Compliance Branch and OSPIE meet with each of the RSIs once a year to determine the requirements for each office. According to TSA officials, they rely on the previous year’s requirements as well as data on surface inspectors’ past activities as logged in PARIS as a starting point to develop the requirements, and adjust the work plan based on their professional judgment of the unique environment in each field office’s area of responsibility. TSA officials stated that they consider variables such as the compliance rates for inspections, the amount of TIH materials being shipped through an area, and any other relevant risk- related information when they develop the work plan.

Regulatory Rail Inspections

Surface inspectors conduct inspections to enforce several freight and passenger rail security requirements. Table 3 provides descriptions of these inspections and appendix II provides a complete listing of TSA’s regulatory activities.

TSA also tracks the rate at which the inspected entities comply with the regulations discussed in table 3. According to TSA data, on average, overall compliance rates for inspections have remained relatively high, and the compliance rates have generally improved over the years as entities have become more familiar with the processes and expectations of each type of inspection.

Regulatory Maritime Inspections

Surface inspectors work with the USCG to conduct inspections of TWIC card holders attempting to access the secured area of maritime facilities regulated by the Maritime Transportation Security Act of 2002 (MTSA). TSA first issued the TWIC regulation in 2007 in cooperation with the USCG, and according to TSA officials, began nationwide implementation of TSA inspection of TWICs at maritime facilities in fiscal year 2017. Surface inspectors scan cards using a TWIC card reader to verify that the card presented is valid and belongs to the card holder. TSA may pursue civil enforcement and can refer violators for criminal proceedings through the USCG. TSA officials stated they set the total minimum required TWIC inspections at 1,315 combined across all surface inspector field offices for fiscal year 2017 as a starting point, and would modify the requirements in subsequent years, as discussed below. According to TSA, it is too soon to determine compliance rates for TWIC inspections.

Non-regulatory Security Activities

Surface inspectors perform a variety of non-regulatory surface-related activities, such as various types of assessments, which require surface entities’ voluntary participation. Table 4 provides a list of key non- regulatory activities surface inspectors perform. For a full list of activities surface inspectors perform see appendix II.

TSA Has Taken Steps to Expand the BASE Review Program and Address Implementation Challenges

Since 2006, TSA has made adjustments to the BASE program to expand its use to more surface modes and address implementation challenges. To conduct a BASE review, surface inspectors use a standardized checklist to evaluate and score an entity’s security policies and procedures for areas such as employee security training, cybersecurity, and facility access control, among other items. According to TSA officials, the results of the BASE reviews are intended to help track the entity’s progress in implementing specific security measures over time and improve overall security posture among surface transportation entities, as well as inform transportation security grant funding. Surface inspectors also use entities’ BASE review scores to help inform Exercise Information System (EXIS) training programs inspectors facilitate for transportation entities.

Initially, the BASE program was designed to assess large mass transit entities in major metropolitan areas that transported 60,000 riders or more daily. TSA officials stated in 2017 that TSA has completed initial and follow up BASE reviews for the top 100 mass transit agencies in the country which comprise approximately 80 percent of the ridership in the United States. In 2012, TSA expanded the BASE reviews to the highway mode to include trucking, motor coach, and school bus operators.

Additionally, TSA has taken steps to address challenges related to the implementation of the BASE reviews, including an initial lack of training and guidance for surface inspectors in conducting and evaluating the BASE reviews and difficulty applying the BASE template for smaller mass transit entities and highway entities. For example, surface inspectors we interviewed at six field offices indicated that they received limited to no training to conduct the initial BASE reviews. Office of Security Operations officials acknowledged that the BASE program initially lacked scoring guidance to allow surface inspectors to make objective evaluations. Additionally, two industry entities we spoke with stated that some BASE questions, as initially developed, seemed inappropriate or irrelevant given the scope of their operation, and that their scores reflected areas that they were not able to modify based on their limited size and resources. Further, in 2010, the DHS Office of Inspector General reported that TSA needed to provide increased training and guidance for inspectors to ensure that BASE assessments gather effective, objective data.

In response, officials from TSA’s Surface Compliance Branch stated that they established a BASE Advisory Panel and held a series of training workshops throughout the country on how to conduct BASE assessments. Specifically, in fiscal year 2014, TSA established a panel comprised of mass transit experts to adjust the BASE tool by modifying topics and removing outdated questions in an effort to improve the quality and applicability of the assessments for the industry stakeholders. TSA has also modified the BASE template over time to include areas such as cybersecurity and active shooter training, among others. TSA reported that it held a series of 16 workshops in 2015 around the country where headquarters officials met with inspectors to train them on how to conduct BASE assessments and correctly apply scoring guidance to help ensure inspectors applied the BASE criteria consistently. Moreover, in fiscal year 2016, TSA developed a targeted BASE that focuses only on an entity’s areas of concern as identified by surface inspectors in a previous BASE review. Further, TSA is piloting a modified BASE template in fiscal year 2017 that eliminates questions that may not apply for smaller mass transit and highway entities. According to Surface Compliance Branch and OSPIE officials, these changes have led to more consistent and more reliable results in the BASE scores. We believe that TSA efforts to improve training and guidance as well as establishing the BASE Advisory Panel will help address the agency’s previous concerns related to the implementation of the BASE review.

TSA Has Incomplete Data on Surface Inspector Activities because It Cannot Account for All Aviation-related Activities

According to TSA headquarters and field officials, in addition to surface inspection activities, surface inspectors are tasked, to varying degrees, with aviation activities. However, TSA officials told us that they are unable to identify the total time surface inspectors spend on aviation activities because of data limitations. For example, surface inspectors may perform aviation activities on a regular basis as a “duty agent,” or on an as- needed basis as determined by their local manager—their AFSD-I. TSA guidance directs surface inspectors to report the time they spend on all activities into TSA’s PARIS database. TSA officials responsible for managing PARIS told us that it has two independent modules – aviation and surface – and that surface inspectors enter aviation-related activities in both the aviation and surface modules. Specifically, TSA guidance directs surface inspectors to document their time serving as “duty agent” in the surface module of PARIS, but to document time spent on aviation inspections, incidents, or investigations – including those that take place during an inspector’s time serving as the duty agent – into the aviation module of PARIS. See table 5 for examples of the types of aviation activities surface inspectors record in each separate PARIS module.

TSA officials told us that it is not possible to identify the time surface inspectors document in the aviation module of PARIS because there is no efficient, reliable way to distinguish surface inspectors from aviation or cargo inspectors in the data.

Since TSA cannot reliably identify activities surface inspectors have entered into the aviation module of PARIS, TSA is only aware of the portion of time surface inspectors spent on aviation activities that was logged in the surface module. As a result, TSA does not have complete information on how surface inspector resources are being used or the extent to which surface inspectors are being used to perform aviation activities. According to some surface inspectors we spoke to, these resources can be substantial. Surface inspectors we interviewed at 16 of the 17 TSA field offices contacted stated that they perform aviation duties. One inspector stated she had received calls to respond to 12 different aviation incidents in one shift as duty inspector, and other inspectors stated that each incident report could subsequently take between 2 and 12 hours to complete. Surface inspectors from another office located near a major airport told us they have to work overtime to complete aviation incident reports and still meet their required surface activities. Further, we met with surface inspectors stationed at four different major airports who each estimated spending 20 percent, 25 percent, 30 percent, and 50 percent of their total working hours on aviation tasks, respectively.

Standards for Internal Control in the Federal Government states that agencies should use complete information to make informed decisions and evaluate the agency’s performance in achieving key objectives. As stated previously, one of TSA’s key objectives is to employ a risk-based approach to all operations to identify, manage, and mitigate risk. Standards for Internal Control in the Federal Government also states that agencies should clearly document all activities in a manner that allows the documentation to be readily available for examination. Without having access to complete information on all inspector activities, including aviation activities, TSA cannot monitor how frequently surface inspectors are being used to support aviation. In addition, by not using complete information on how much time surface inspectors spend working in support of aviation, TSA is limited in its ability to make informed future decisions on annual resource needs for surface inspectors, which will be especially important as TSA takes steps to expand its inspection activities with the promulgation of new surface security regulations. By addressing the limitations in the aviation module of PARIS, TSA would be able to more reliably access complete information on all inspector activities. Also, it would have the information it needs to make fully informed decisions about surface inspector resources and activities, and to evaluate surface inspectors’ performance in achieving key surface security objectives.

Since there is no way to identify surface inspectors in the aviation module of PARIS at the aggregate level, we were unable to conduct our own analysis of all surface inspector activities. However, we were able to analyze data on how surface inspectors reported spending their time in the surface module of PARIS, including time spent on aviation activities as documented in this particular module. Our analysis showed that from fiscal years 2013 to 2017, surface inspectors reported spending approximately 80 percent of their time on non-regulatory activities, while spending approximately 20 percent on regulatory inspections. Figure 3 shows a breakdown of the time surface inspectors recorded spending in the surface module of PARIS for fiscal year 2016, the most recent complete year of data available. See appendix III for similar breakdowns for each fiscal year from 2013 to 2017.

TSA Used a Risk- Informed Process to Allocate Surface Inspector Staff, But Inspector Activities Did Not Align With Risk

TSA Used a Risk-Informed Model to Allocate Surface Inspectors to Field Offices

In fiscal year 2017, TSA’s Surface Compliance Branch implemented an updated staffing model to redistribute 222 surface-funded positions across its 49 surface field offices based on the factors described in table 6 below.

TSA considered four of these factors – HTUA/Urban Area Security Initiative (UASI), Mass Transit, TWIC, and TIH – to be related to risk. For example, TSA derived its list of HTUAs based on risk assessments conducted under the UASI program. We have previously reported that the UASI methodology for determining risk scores and distributing grant funds is reasonable, and that UASI grant allocations are strongly associated with a city’s current relative risk score. Additionally, according to TSA, inspectors focus on entities within surface transportation modes or shipments of certain hazardous materials the agency determines could pose the greatest security vulnerability and which could potentially be more likely to be targeted by terrorists.

The DHS Risk Lexicon 2010 and the 2013 NIPP risk management framework, which are TSA’s primary risk guidance, define risk-informed decision-making as the determination of a course of action predicated on the assessment of risk, the expected impact of that course of action on that risk, as well as other relevant factors. The DHS Risk Lexicon 2010 further states that risk-informed decision-making may also take into account multiple sources of information not included specifically in the assessment of risk. Because TSA considered multiple risk factors in addition to other information, such as the number of regulated entities in an area and the number of required activities, in its staffing model, we determined that TSA used a risk-informed model to allocate surface inspector staff to its 49 offices.

Between Fiscal Years 2013 and 2017 Surface Inspector Activities Did Not Align With Identified Risks for Surface Transportation Modes

TSA surface inspectors perform a wide range of regulatory and non- regulatory activities to fulfill the agency’s objective of employing risk- based security, but we found that between fiscal years 2013 and 2017 surface inspector activities did not align with the risks TSA identified for surface transportation. To inform its security strategy, TSA assesses risk within and across the aviation, freight rail, passenger rail/mass transit, highway, and pipeline modes approximately every 2 years using the TSSRA. According to the TSSRA’s cross-modal risk assessments between fiscal years 2013 and 2017, one particular surface mode consistently posed the highest risk, and another consistently posed the lowest risk out of all surface transportation modes. For example, in fiscal year 2016, TSA found that the lowest risk mode posed approximately 6 percent of domestic total risk while the highest risk mode posed 27 percent of domestic total risk. However, our analysis of data from the surface module of PARIS showed that inspectors reported spending between 35 and 45 percent of their time on the lowest risk mode between fiscal year 2013 and fiscal year 2016 – the most time spent on any surface mode. Of the time reported in the surface module of PARIS in fiscal year 2016, surface inspectors reported spending 38 percent of their time on the lowest risk transportation mode while they reported spending approximately 16 percent of their time on the highest risk surface mode according to the TSSRA. See figure 4 for a comparison between the percent of time inspectors recorded spending on each mode and the percent of risk identified in the TSSRA.

We found that TSA did not use the results of risk assessments that measure threat, vulnerability, and consequence, like the TSSRA, when it developed surface inspector work plans, or when it monitored activities inspectors conducted, including those in addition to the minimum work plan requirements. While TSA officials told us that they considered the results of the TSSRA, TSA officials could not provide evidence that they incorporated the results of the TSSRA or other risk assessments when developing the work plan and monitoring inspector activities, as required by DHS risk management guidance. For example, TSA officials could not provide documentation of how and why they selected certain work plan activities to address lower risk modes, or how they monitored the extent to which implemented activities aligned with or addressed risks.

Monitoring Activity Implementation

We found that TSA did not incorporate the results of the TSSRA or other risk assessments when it monitored how surface inspector activities were implemented beyond the minimum requirements laid out in the work plan. Specifically, we found that between fiscal years 2013 and 2017, inspectors spent about half their working hours fulfilling work plan requirements. Surface Compliance Branch officials told us that they reviewed PARIS data on all surface inspector activities, as reported in the surface module of PARIS, annually to inform staffing decisions and conducted detailed analysis of surface inspector time starting in fiscal year 2015. However, this analysis did not evaluate the extent to which surface inspector time beyond the work plan requirements corresponded to surface transportation risks as identified by the TSSRA or other risk assessments. Further, TSA officials told us that they did not think surface inspector time should be compared to risks identified in cross-modal risk assessments like the TSSRA because required regulatory inspections are unpredictable and can take a significant amount of time. However, as previously discussed, we found that, of the time reported in the surface module of PARIS, inspectors reported spending approximately 20 percent of their time on regulatory inspections, with the remaining 80 percent spent on non-regulatory activities.

More than half of the industry representatives we spoke to (9 of 15) identified benefits from inspectors’ activities in surface transportation modes other than freight rail. For example, two of the three representatives of MTSA-regulated companies we spoke to said that TSA’s TWIC inspections had significant benefits for the security of their facilities, and stated that they wanted more TWIC inspections and civil enforcement activities from inspectors because these activities discourage misuse of TWICs at their facilities. Representatives from two maritime companies, one highway company, and three public transportation systems told us that they wanted TSA surface inspectors to do more. Additionally, a representative for one national industry organization stated that his organization was concerned that TSA is mainly focused on freight rail when the principal threat resides in the passenger and mass transit modes, and suggested that TSA deploy inspection resources from the freight rail mode to support more non- regulatory initiatives in the passenger rail/mass transit mode.

According to TSA, the agency employs a risk-based approach – which the DHS Risk Lexicon defines as using the assessment of risk as the primary decision driver – to all operations to identify, manage, and mitigate risk in all TSA lines of business. One TSA risk strategy document specifically emphasizes the importance of linking the TSSRA, among other risk assessments, to the identification of risk-reduction activities as part of a risk-based approach to security. Moreover, the NIPP risk management framework and the DHS Risk Management Fundamentals Doctrine, which TSA officials told us are TSA’s primary risk management guidance documents, also state that entities should systematically prioritize and implement activities and resources to mitigate and manage risks identified in risk assessments. These documents also state that monitoring implemented decisions and comparing observed and expected effects to influence subsequent risk management decisions are key steps in the homeland security risk management process. The DHS Risk Management Fundamentals Doctrine further states that agencies should document the development and selection of alternative risk management actions, including assumptions and risk strategies such as the decision to not take action and accept risk, in order to provide decision-makers with a clear picture of the benefits of each action. It also explains that the risk management process allows organizations to clearly explain the rationale behind resource decisions.

TSA did not use the results of risk assessments – such as the TSSRA – or other risk information when it developed its surface inspector work plan requirements. Instead, TSA prioritized the lowest-risk surface transportation mode, reducing the amount of surface security resources available to address identified risks in other, higher-risk surface transportation modes. As a result, TSA’s limited surface transportation security resources were not used in a risk-based way. By incorporating the results of its risk assessments when it plans and monitors surface inspector activities, including those not required by the work plan, TSA would be better able to ensure that its limited surface transportation security resources are being used to effectively and efficiently address the highest risks to surface transportation, especially as risks evolve. Incorporating risk assessment results in planning and monitoring surface inspector activities will also allow TSA to ensure that its surface inspectors are making progress toward achieving TSA’s objective of risk- based security. Additionally, by documenting its risk mitigation decisions and strategies, TSA would be able to more clearly explain the rationale for its resource decisions, including when TSA decides to accept risk or prioritize lower-risk activities for any reason.

TSA Cannot Ensure That New Risk Mitigation Efforts Address High-Risk Entities and Locations

In fiscal year 2012, TSA began developing the Risk Mitigation Activities for Surface Transportation (RMAST) program in support of TSA’s risk- based security initiative. According to TSA’s fiscal year 2017 work plan, the RMAST program incorporates specific risk reduction measures and focuses time and resources on high-risk locations through (1) public observation, (2) site security observations, and (3) stakeholder engagement activities. Though TSA field officials told us that inspectors have been conducting these activities in some format in the past, TSA began piloting this particular program in fiscal year 2014 and made RMAST a work plan requirement for each office starting in fiscal year 2017.

In addition to TSA demonstrating its commitment to the RMAST program by adding it as a required work plan activity, we found that inspectors reported spending an increasing amount of time conducting RMASTs since fiscal year 2014, and that RMASTs now comprise a larger percentage of inspector time (see table 7).

Although surface inspectors reported spending an increasing amount of time on RMAST activities, we found that TSA has not identified or prioritized the high-risk entities and locations on which the RMAST program is intended to focus time and resources. For example, the fiscal year 2017 surface inspector work plan states that the required number of RMASTs each office should conduct was developed based on the presence of applicable stakeholders in each office’s area, but we found that TSA did not identify any such stakeholders in its work plan. Specifically, while the work plan guidance directed surface inspectors to conduct RMASTs with entities that fit “listed” criteria, this list consisted of all surface modes of transportation for which TSA has authority and did not include any criteria surface inspectors could use to identify the highest-risk and most critical locations, such as by type, characteristics, or location of high-risk entities. TSA officials told us that they have not identified high-risk entities for RMAST because there are too many potential entities and stated that there is no way to provide a full list of all entities in each office’s area. However, the intent of the RMAST program is to focus time and resources on high-risk entities and locations, which precludes the need to provide a complete list of all surface transportation entities in each area. Further, TSA officials told us that TSA has not provided any guidance to the field beyond the work plan on how to identify appropriate entities for RMASTs, but that they rely on surface field offices to identify the highest-risk entities in their own areas. Officials from three field offices told us that inspectors try to conduct RMASTs based on threat information or previous BASE scores, but inspectors in one of those offices said that the intelligence information they receive from TSA is insufficient to help them identify threats and conduct outreach for RMASTs. As previously discussed, the NIPP risk management framework and the DHS Risk Management Fundamentals Doctrine both state that entities should identify and assess risks and prioritize resources to mitigate those risks. If TSA identified and prioritized the types of high-risk entities and locations it intends the RMAST program to reach, surface inspectors would have information that would enable them to implement these activities in a more risk-based manner.

Defining Measurable and Clear Objectives

While TSA has identified broad objectives for the RMAST program, it has not defined these objectives – and associated program activities – in a measurable and clear way. Specifically, in its description of RMAST in the fiscal year 2017 work plan implementation guidance, TSA stated that the RMAST program will be risk-based, intelligence-driven, and mitigate current threats and vulnerabilities, but did not provide further information that would allow TSA to measure progress toward achieving these objectives. Similarly, in its budget justifications for fiscal years 2014, 2015, and 2016 TSA stated that RMAST is intended to improve security and reduce the need for stakeholders to stretch limited resources to harden security at their most critical and high-risk locations, but TSA did not describe how it would measure whether security had improved, or if stakeholders’ resource needs were reduced. While our review of the fiscal year 2017 work plan guidance showed that TSA identified general categories of activities – public observation, site security observation, and stakeholder engagement – TSA did not identify what specific activities within each of these categories constitute an RMAST, or describe how those activities would help TSA achieve its objectives for the RMAST program. Some inspectors told us that the purpose of RMAST was unclear, that they had not been given the tools to perform RMAST in an effective and efficient way, or that the observation component of RMAST was not a valuable activity. TSA has not defined the RMAST program’s objectives and associated activities in a measurable and clear way because, according to TSA officials, TSA has not identified an approach for determining the effectiveness of activities conducted under the program.

Standards for Internal Control in the Federal Government states that management should establish proper controls – including the establishment and review of clearly defined objectives and performance measures – so that program objectives and processes are understood at all levels and progress toward achieving objectives can be assessed. By defining the program’s objectives and associated activities in a measurable and clear way, TSA would be better positioned to measure progress toward achieving the program’s goal of mitigating current threats and vulnerabilities, and surface inspectors may better understand how to effectively carry out the program.

Conclusions

TSA has employed surface inspectors for a variety of regulatory and non- regulatory activities intended to mitigate risks to surface transportation and enhance the security of the United States’ surface transportation systems and networks. Working with surface transportation entities, who have the primary responsibility for securing their respective entities, TSA surface inspectors enforce security regulations for the freight and passenger rail modes, but spend the majority of their time conducting non-regulatory activities such as security assessments, exercises, and observations. While TSA uses information on some surface inspector activities to monitor and make decisions on these activities, limitations in the PARIS data system prevent TSA from readily accessing complete information on how much time inspectors spend working in support of aviation. Without addressing these limitations TSA is limited in its ability to make informed future decisions on annual resource needs for surface inspectors, which will be especially important as TSA take steps to expand its inspection activities with the promulgation of new surface security regulations. Given that TSA spends only about 3 percent of its budget on surface activities, it is crucial that the agency have complete information on how resources are being used in order to best allocate these limited federal surface transportation security resources.

According to TSA, the agency implements risk-based security – security activities that are driven primarily by the assessment of risk – to deliver the most effective security in the most efficient manner. While TSA has implemented a risk-informed process to allocate surface inspectors to its field offices, it has not taken steps to ensure that surface inspector activities align more closely to the risks TSA has identified in its risk assessments. As a result TSA could continue to prioritize its limited resources to lower risk surface modes, leaving fewer resources available for higher risk modes. By using the results of risk assessments like the TSSRA when it plans and monitors surface inspector activities, TSA would be better able to ensure that limited surface transportation security resources are used to effectively and efficiently address the highest surface transportation security risks. Additionally, by documenting its risk mitigation decisions and strategies, TSA would be able to more clearly explain the rationale for its resource decisions, including when TSA decides to accept risk or prioritize lower-risk activities for any reason.

Furthermore, by identifying and prioritizing highest risk entities and locations for its new RMAST program, surface inspectors would have information that would enable them to implement risk mitigation activities in more of a risk-based way. In addition, by clearly defining the program’s goals and activities, TSA would be better able to measure whether RMAST activities are achieving the program’s goal of increasing surface transportation security.

Recommendations for Executive Action

We are making the following four recommendations to TSA: The Administrator of TSA should address limitations in TSA’s data system, such as by adding a data element that identifies individuals as surface inspectors, to facilitate ready access to information on all surface inspector activities. (Recommendation 1)

The Administrator of TSA should ensure that surface inspector activities align more closely with higher-risk modes by incorporating the results of surface transportation risk assessments, such as the TSSRA, when it plans and monitors surface inspector activities, and that TSA documents its rationale for decisions to prioritize activities in lower-risk modes over higher-risk ones, as applicable. (Recommendation 2)

The Administrator of TSA should identify and prioritize high-risk entities and locations for TSA’s Risk Mitigation Activities for Surface Transportation (RMASTs). (Recommendation 3)

The Administrator of TSA should define clear and measurable objectives for the RMAST program. (Recommendation 4)

Agency Comments and Our Evaluation

We provided a draft of this report to DHS for their review and comment. DHS provided written comments, which are noted below and reproduced in full in appendix IV, and technical comments, which we incorporated as appropriate.

DHS concurred with all four recommendations in the report and described actions underway or planned to address them. With regard to the first recommendation that TSA address limitations in its data system to facilitate ready access to information on all surface inspector activities, DHS concurred and stated TSA’s Compliance Division will maintain a staffing tool that identifies the modal assignments of transportation security inspectors that can be used to more effectively analyze all surface inspector activities. If fully implemented, such that data on all activities surface inspectors perform are readily accessible, this system should address the intent of the recommendation.

With regard to the second recommendation that TSA align surface inspector activities more closely with higher-risk modes by incorporating the results of surface transportation risk assessments, such as the TSSRA, when it plans inspector activities, and document its rationale for decisions to prioritize activities in lower-risk modes, TSA concurred and stated relevant risk information would be more clearly incorporated into the Surface Work Plan development process. Further, TSA plans to explain decisions and rationale for deviating surface inspector planned activities from mirroring the TSSRA in its program guidance documentation. TSA estimates it will complete this process by January 31, 2018. If TSA is able to fully incorporate risk assessment results, such as the TSSRA, into its decisions for assigning surface inspector tasks across surface transportation modes, and document its rationale if planned inspector activities do not align with risk assessment results, TSA’s planned actions would address the intent of the recommendation.

With regard to the third recommendation to identify and prioritize high-risk entities and locations for TSA’s Risk Mitigation Activities for Surface Transportation (RMAST), TSA concurred and stated the Surface Compliance Branch will prioritize entities for RMAST activities within the Surface Work Plan or other applicable program guidance documents using results from the TSSRA and using high threat urban area designations. TSA estimates this process will be completed by January 31, 2018 and if fully implemented, this process should address the intent of the recommendation.

With regard to the fourth recommendation that TSA define clear and measurable objectives for the RMAST program, TSA concurred and stated the Surface Compliance Branch has clarified in program guidance documents how to apply and measure certain security outcomes resulting from RMAST activities to security vulnerabilities identified from a previous BASE assessment or other security assessment program. Documentation corroborating these actions was not provided to GAO before the issuance of this report. However, if TSA is able to clearly state the purpose and objectives of RMAST activities, and track the extent to which these objectives have been met, this additional program guidance should address the intent of the recommendation.

We are sending copies of this report to interested congressional committees, the Secretary of Homeland Security, and the Administrator of the Transportation Security Administration. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-7141 or groverj@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix V.

Appendix I: Objectives, Scope, and Methodology

Our objectives were to examine (1) how Transportation Security Administration (TSA) surface inspectors implement the agency’s surface transportation security mission, and (2) the extent to which TSA has used a risk-based approach to prioritize and implement surface inspector activities.

This report is a public version of a prior sensitive report that we issued in October 2017. TSA deemed some of the information in the prior report sensitive security information, which must be protected from public disclosure. Therefore, this report omits sensitive information regarding the specific risks facing particular surface transportation modes as determined by TSA. However, the report addresses the same questions as the sensitive report and the overall methodology used for both reports is the same.

To obtain background information and answer both questions we (1) reviewed background documents, including TSA strategic documents and previous GAO and Department of Homeland Security (DHS) Inspector General reports, (2) analyzed TSA data on surface inspector activities, and (3) conducted non-generalizable interviews of surface inspectors, their supervisors, and industry stakeholders.

To understand TSA’s roles and responsibilities for surface security, as well as its mission, we examined statutes and regulations, including the Aviation and Transportation Security Act, the Implementing Recommendations of the 9/11 Commission Act of 2007, and TSA surface security and related regulations. We also reviewed DHS and TSA strategic documents including TSA’s National Strategy for Transportation Security 2016, the DHS National Infrastructure Protection Plan (NIPP) 2013, and the fiscal years 2016 to 2018 strategic plans for TSA’s Office of Security Operations and the Office of Security Policy and Industry Engagement. Additionally, we reviewed previous GAO and DHS Office of Inspector General reports on TSA’s surface security efforts and surface inspector programs.

To evaluate how surface inspectors implemented TSA’s surface security mission and the extent to which this implementation was based on risk, we analyzed data from the surface module of the Performance and Results Information System (PARIS) on the activities of surface inspectors from fiscal year 2013 through March 24, 2017, the most recent data available. Based on TSA documents, regulations, and interviews with TSA data and program officials, we categorized surface inspector activities according to regulatory and non-regulatory activities and by mode, and calculated the total time surface inspectors reported spending for each category. We analyzed data from fiscal years 2013 through 2017 to ensure that we could compare several years of data and analyze data obtained after reorganizations of the surface inspector command structure in fiscal year 2010 and offices in mid-fiscal year 2013. We did not review data from the aviation module of PARIS because, as discussed below, it was not feasible to identify the data surface inspectors entered into this module, and, based on our interviews with TSA data officials and our review of related documentation, we determined that all other surface inspector activities were documented in the surface module of PARIS.

To determine the reliability of data from the surface module of PARIS we (1) reviewed related documentation such as data dictionaries, schema, PARIS reliability assessments from previous GAO audits, TSA analyses of PARIS data, and data entry guidance, (2) interviewed TSA officials responsible for entering, reviewing, or using PARIS data, including headquarters officials, field office supervisors, and surface inspectors, (3) electronically and manually tested the data for completeness and obvious errors, such as duplicates and consistency with secondary sources, and (4) conducted internal logic tests on certain time-related fields in the data. Through these steps, we identified some inconsistencies in the data including incomplete data on surface inspectors’ aviation activities and non-specific data elements for inspection activities in fiscal year 2013, among others. However, we determined that for our purposes – to describe how surface inspectors reported spending their time at the summary-level – these inconsistencies did not affect the reliability of the PARIS surface module data and these data were reliable with some limitations.

Specifically, based on interviews with TSA data officials and our review of TSA data entry guidance, we determined that the data in the surface module of PARIS did not represent the complete activities conducted by surface inspectors because they enter some aviation activities separately in the aviation module of PARIS. Further, we determined that it was not feasible to distinguish aviation activities documented by surface inspectors in the aviation module from aviation activities documented by cargo or aviation inspectors in this module at the aggregate level. However, based on our testing, review of related documentation, and interviews with TSA data officials, we determined that the data surface inspectors entered into the surface module of PARIS, including data on some aviation activities, were reliable for our purposes. As a result, we reported data on surface inspectors’ aviation activities as documented in the surface module of PARIS, with the limitation that these data represent the minimum aviation activities surface inspectors actually conducted.

Additionally, through our analysis of PARIS data on regulatory inspections surface inspectors conducted in fiscal year 2013 and interviews with TSA data officials, we found that 25 percent of the total inspections in fiscal year 2013 (1,990 of 8,083) were documented under data elements that did not specify the type of inspection conducted. According to TSA officials, there are no additional data elements that would allow us to identify the specific type of inspection surface inspectors conducted for these 1,990 inspections. As a result, we determined that this portion of the fiscal year 2013 data was not reliable for our purposes of identifying the number of specific inspection types surface inspectors conducted. However, we found that the remaining 78 percent of inspection data for fiscal year 2013 was reliable for our purposes. As a result, the inspection counts and compliance rates we reported for fiscal year 2013 represent partial year data.

To obtain the perspectives of a wide sample of TSA officials on both surface inspector activities and TSA’s use of risk, we conducted semi- structured interviews with surface inspectors and/or their supervisors in 17 of 49 field offices. We also interviewed the 6 Regional Security Inspectors (RSIs), who cover all seven TSA regions. We interviewed inspectors and supervisors from at least 2 offices in each region and selected the offices based on a variety of factors including geographic dispersion, staff level, surface transportation environment, and whether the office was co-located with a major airport. We physically visited 6 offices and conducted the remainder of our interviews remotely. We selected the offices we traveled to based on the location of GAO staff, the availability of industry representatives in the area, and the opportunity to observe surface inspector assessments, tabletop exercises, and other activities. The results of our interviews are not generalizable, but provide insight into how surface inspectors and their supervisors implement TSA surface programs and the challenges they may face, if any.

To gain insight into the experience surface transportation industry stakeholders have had with TSA surface inspectors, we interviewed 15 industry stakeholders in four surface modes including 3 freight rail stakeholders, 3 maritime stakeholders, 3 highway stakeholders, and 6 passenger rail/mass transit stakeholders. We selected industry stakeholders based on their involvement and familiarity with TSA surface inspectors, the surface mode in which they operate, their ridership, and TSA recommendation. Three of these stakeholders consisted of national trade associations representing the highway, freight rail, and mass transit modes of transportation. As with our interviews with TSA surface inspectors and supervisors, our interviews with industry stakeholders are not generalizable but provided us with valuable information on the transportation industry’s interaction with TSA surface inspectors.

To further address our first objective and describe how TSA surface inspectors implemented the agency’s surface transportation security mission, we examined TSA strategic and program documents including surface inspector work plans and implementation guidance from fiscal years 2013 to 2017, the TSA Inspector Compliance Manual, and TSA surface security regulations, and reviewed public testimony by TSA leadership. To understand how TSA has implemented the Baseline Assessment for Security Enhancement (BASE) program in particular, we reviewed TSA program documents and guidance for the BASE program, including the BASE workbook, and observed a BASE review on a mass transit entity. We also observed a regional Intermodal – Security Training Exercise Program (I-STEP) exercise and an Exercise Information System (EXIS) exercise, and interviewed TSA officials in headquarters, and inspectors and supervisors in the field.

We used the results of our analysis of PARIS surface module data, specifically the number of each type of regulatory inspection TSA inspectors conducted from fiscal years 2013 to 2017, and PARIS data on the violations found during those inspections, to calculate regulatory compliance rates. We also used the results of our analysis of PARIS surface module data to describe how surface inspectors reported spending their time. As previously stated, we found the PARIS surface module data to be reliable for this purpose, with the limitation that TSA data on the time surface inspectors reported spending on aviation activities was incomplete because we could not identify surface inspector activities entered into the aviation module of PARIS. To evaluate the effects of this limitation, we compared the results of our data analysis, our reviews of PARIS documentation, and our interviews with TSA officials to Standards for Internal Control in the Federal Government.

To further address our second objective, the extent to which TSA has used a risk-based approach to prioritize and implement surface inspector activities, we analyzed TSA’s risk guidance as contained in the NIPP risk management guidance, the DHS 2010 Risk Lexicon, and the DHS Risk Management Fundamentals to understand how TSA should assess and use risk information. To understand the risks TSA has identified for surface transportation modes during the time period we examined, we analyzed TSA’s cross-modal risk assessments in three Transportation Security Sector Risk Assessments (TSSRA) published between May 2013 and July 2016.

We reviewed TSA’s fiscal year 2017 surface inspector staffing model and supporting documents and data and interviewed TSA officials responsible for developing and executing staffing. We compared that process to TSA risk guidance to evaluate the extent to which TSA considered risk when it staffed TSA surface inspectors for fiscal year 2017. We assessed only the fiscal year 2017 staffing model because TSA’s previous staffing model was last used in fiscal year 2011, which is outside our scope.

To determine the extent to which TSA prioritized surface inspector activities based on risk when it planned these activities, we identified, compiled and analyzed activity requirements from surface inspector work plans and associated implementation guidance from fiscal years 2013 to fiscal year 2017. We (1) compared them to each other to identify changes in planned surface inspector activities over time and (2) compared them to results from the TSSRA, as well as other risk information including unattended rates for Toxic Inhalation Hazard (TIH) rail cars and the presence of Maritime Transportation Security Act of 2002-regulated facilities in each office’s area. We also interviewed TSA officials in headquarters and the field who were responsible for developing the surface inspector work plan about the process and information they considered during work plan development, and compared this information to TSA risk guidance.

To determine the extent to which TSA’s implementation of surface inspector activities aligned with risk, we compared the results of our analysis of PARIS surface module data on the time surface inspectors spent in each surface mode to the results of the TSSRA cross-modal risk assessments from fiscal years 2013 to 2017. As previously discussed, we determined the data to be reliable for our purposes. We also compared the results of our analysis of PARIS surface module data to our analysis of work plan requirements to identify the amount of time surface inspectors reported spending on work plan activities. In addition, we identified the types of information TSA used in its fiscal year 2015 analysis of surface inspector time and activities to determine what TSA considered when it monitored how surface inspector activities were implemented.

Additionally, we used the results of our analysis of PARIS surface module data to determine the percent of total time surface inspectors reported spending on Risk Mitigation Activities for Surface Transportation (RMAST) between fiscal years 2013 and 2017. To understand TSA’s objectives for the RMAST program, we analyzed program descriptions in TSA congressional budget justifications and TSA’s fiscal year 2017 work plan and work plan implementation guidance. We also conducted interviews with TSA officials in headquarters, and inspectors and supervisors in the field, and observed an RMAST activity to understand how TSA has implemented the program. We compared the results of our analysis and interviews to TSA’s risk guidance and Standards for Internal Control in the Federal Government to evaluate the extent to which the program was risk-based and to which TSA had established measurable goals for the program.

The performance audit upon which this report is based was conducted from April 2016 to October 2017 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. We subsequently worked with TSA from September 2017 to December 2017 to prepare this nonsensitive version of the original report for public release. This public version was also prepared in accordance with these standards.

Appendix II: Surface Inspector Activities

Appendix II: Surface Inspector Activities 2005 High-visibility activities, such as patrols, passenger and baggage screening, and canine activities to introduce unpredictability, increase security, and deter potential terrorist actions on multiple modes of transportation. Managed by the U.S. Federal Air Marshal Service and conducted by TSA personnel, which may include surface inspectors. 2006 A voluntary review in which surface inspectors evaluate the security programs of transportation entities, offer technical assistance, and share best practices. TSA uses BASE to, among other things, determine priorities for allocating mass transit and passenger rail security grants, such as those provided through the Transportation Security Grant Program. 2006 Local field assessments of critical infrastructure, station and other facilities for mass transit, passenger rail, and commuter rail and bus systems. Station profiles provide detailed information of specific station-related intelligence, such as the locations of exits, telephones, CCTV, electrical power, station mangers etc. 2007 Inspectors verify that Toxic Inhalation Hazard (TIH) rail cars at rail yards within high- threat urban areas that transport TIH on a regular and reoccurring basis are being attended by railroad personnel. Inspectors also conduct “wildcard” RRS, during which they observe locations which do not normally handle TIH on a regular and recurring basis to determine if TIH cars are present, and if they are being attended by railroad personnel. 2008 Detailed assessments that focus on the vulnerabilities of high-population areas where TIH materials are moved by rail in significant quantities, and that provide site- specific mitigation strategies and lessons learned. 2008 I-STEP, which is managed through the Office of Security Policy and Industry Engagement, consists of contractor-facilitated exercises designed to help multimodal surface transportation entities closely examine their security programs and operational efforts. TSA facilitates I-STEP exercises across all surface transportation modes to help operators, law enforcement, first responders, and related entities test and evaluate their security plans, including prevention and preparedness capabilities, ability to respond to threats, and interagency coordination. TSA updates I-STEP scenarios as new threats emerge, helping industry partners prepare to implement the most appropriate countermeasures. 2014 Quality assurance assessments of Transportation Worker Identification Credential (TWIC) enrollment centers to, according to TSA officials, review contractor performance. 2015 EXIS consists of exercises facilitated by surface inspectors that utilize software developed by TSA for stakeholder use, generally focus on one entity, and are intended to build on the findings of a previously completed BASE assessment.

Start Date (fiscal year)

2017 A program intended to focus time and resources on high-risk and critical assets, facilities and other infrastructure through the following activities: (1) public observation to identify suspicious activities, security vulnerabilities and/or suspicious behaviors that could be indicative of pre-operational planning related to terrorism; (2) site security observation to determine if the physical security measures and operational deterrence components are in place to effectively mitigate risk, and (3) stakeholder engagement including TSA’s public security awareness programs and improvised explosive device (IED) and intelligence briefings.

In this table, passenger rail and rail transit systems consist of: each passenger railroad carrier, including each carrier operating light rail or heavy rail transit service on track that is part of the general railroad system of transportation, each carrier operating or providing intercity passenger train service or commuter or other short-haul railroad passenger service in a metropolitan or suburban area (as described by 49 U.S.C. § 20102), and each public authority operating passenger train service; (b) each passenger railroad carrier hosting an operation described in paragraph (a) of this section; (c) each tourist, scenic, historic, and excursion rail operator, whether operating on or off the general railroad system of transportation; (d) each operator of private cars, including business/office cars and circus trains, on or connected to the general railroad system of transportation, and (e) each operator of a rail transit system that is not operating on track that is part of the general railroad system of transportation, including heavy rail transit, light rail transit, automated guideway, cable car, inclined plane, funicular, and monorail systems. 49 C.F.R. § 1580.200.

Appendix III: Surface Inspector Time Spent on Activities Reported in the Surface Module of PARIS for Fiscal Years 2013 to 2017

Appendix IV: Comments from the U.S Department of Homeland Security

Appendix V: GAO Contact and Staff Acknowledgements

GAO Contact

Jennifer Grover (202) 512-7141 or groverj@gao.gov.

Staff Acknowledgments

In addition to the contact named above, Christopher E. Ferencik, Assistant Director; Brendan Kretzschmar, Analyst in Charge; Nanette Barton, and Katherine Blair made key contributions to this report. Also contributing to the report were, Charles Bausell, Katherine Davis, Eric Erdman, Anthony Fernandez, Eric D. Hauswirth, Paul Hobart, Tracey King, Christopher Lee, Mara McMillen, Amanda Miller, Claudia Rodriguez, Christine San, McKenna Storey, Natalie Swabb, Michelle Vaughn, Adam Vogt, Johanna Wong.