Aviation Security: Federal Air Marshal Service Has Taken Actions to Fulfill Its Core Mission and Address Workforce Issues, but Additional Actions Are Needed to Improve Workforce Survey
Published: 2009-01-14T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: By deploying armed air marshals onboard selected flights, the Federal Air Marshal Service (FAMS), a component of the Transportation Security Administration (TSA), plays a key role in helping to protect approximately 29,000 domestic and international flights operated daily by U.S. air carriers. GAO was asked to examine (1) FAMS's operational approach or "concept of operations" for covering flights, (2) to what extent this operational approach has been independently evaluated, and (3) the processes and initiatives FAMS established to address workforce-related issues. GAO analyzed documented policies and procedures regarding FAMS's operational approach and a July 2006 classified report based on an independent evaluation of that approach. Also, GAO analyzed employee working group reports and other documentation of FAMS's processes and initiatives for addressing workforce-related issues, and interviewed the FAMS Director, other senior officials, and 67 air marshals (selected to reflect a range in levels of experience). This report is the public version of a restricted report (GAO-09-53SU) issued in December 2008.
section_title: What GAO Found
paragraphs: Because the number of air marshals is less than the number of daily flights, FAMS's operational approach is to assign air marshals to selected flights it deems high risk--such as the nonstop, long-distance flights targeted on September 11, 2001. In assigning air marshals, FAMS seeks to maximize coverage of flights in 10 targeted high-risk categories, which are based on consideration of threats, vulnerabilities, and consequences. In July 2006, the Homeland Security Institute, a federally funded research and development center, independently assessed FAMS's operational approach and found it to be reasonable. However, the institute noted that certain types of flights were covered less often than others. The institute recommended that FAMS increase randomness or unpredictability in selecting flights and otherwise diversify the coverage of flights within the various risk categories. As of October 2008, FAMS had taken actions (or had ongoing efforts) to implement the Homeland Security Institute's recommendations. GAO found the institute's evaluation methodology to be reasonable. To address workforce-related issues, FAMS's previous director, who served until June 2008, established a number of processes and initiatives--such as working groups, listening sessions, and an internal Web site--for agency personnel to provide anonymous feedback to management on any topic. These efforts have produced some positive results. For example, FAMS revised its policy for airport check-in and aircraft boarding procedures to help protect the anonymity of air marshals in mission status, and FAMS adjusted its flight scheduling process for air marshals to support a better work-life balance. The air marshals GAO interviewed expressed satisfaction with FAMS efforts to address workforce-related issues. Further, the current FAMS Director, after being designated in June 2008 to head the agency, issued a broadcast message to all employees, expressing a commitment to continue applicable processes and initiatives. Also, FAMS has plans to conduct a workforce satisfaction survey of all employees every 2 years, building upon an initial survey conducted in fiscal year 2007. Although the 2007 survey indicated positive changes since the prior year, it was answered by 46 percent of the workforce, well short of the 80-percent response rate that the Office of Management and Budget (OMB) encourages for ensuring that results reflect the views of the target population. OMB guidance gives steps, such as extending the cut-off date for responding, that could improve the response rate of future surveys. Also, several of the 2007 survey questions were ambiguous, and response options were limited. Addressing these design considerations could enhance future survey results.

section_title: Letter
paragraphs: With a shared responsibility for protecting the approximately 29,000  domestic and international flights operated daily by U.S. commercial  passenger air carriers, the Federal Air Marshal Service (FAMS) is a key  component of the Transportation Security Administration’s (TSA) layered  approach for aviation security. FAMS’s overall mission is to help secure  the nation’s aviation system by detecting, deterring, and defeating hostile  acts targeting U.S. air carriers, airports, passengers, and crews. In  particular, FAMS’s core mission is to deploy trained and armed federal air  marshals—a workforce generally characterized as the nation’s most  mobile law enforcement officers—to provide an on-board security  presence on selected flights.
In response to the September 11, 2001, terrorist attacks and pursuant to  the Aviation and Transportation Security Act, FAMS’s mission was  expanded to provide coverage of domestic as well as international flights,  with priority given to nonstop, long-distance flights such as those targeted  on September 11. As a result, the agency’s cadre of air marshals grew  significantly.  Nonetheless, because the total number of air marshals is  less than the approximately 29,000 domestic and international flights  operated daily by U.S. commercial passenger air carriers, FAMS routinely  must determine which flights are to be provided an on-board security  presence. To facilitate making these decisions, FAMS developed an  operational approach—commonly referred to as the agency’s concept of  operations—for deploying air marshals on selected flights.
In addition to flight-coverage decisions, FAMS also faces challenges in  addressing various operational and quality-of-life issues that affect the  ability of air marshals to carry out the agency’s mission—issues that have  been noted by our prior work and also have received extensive media  coverage. Such issues range, for example, from maintaining anonymity  during aircraft boarding procedures to mitigating the various health  concerns associated with frequent flying.
You requested that GAO evaluate FAMS’s concept of operations and  workforce-related issues. This report answers the following questions:
What is FAMS’s operational approach for achieving its core mission of  providing an onboard security presence for flights operated by U.S.  commercial passenger air carriers?
To what extent has FAMS’s operational approach for achieving its core  mission been independently assessed?
To what extent does FAMS have processes and initiatives in place to  address issues that affect the ability of its workforce to carry out its  mission?
This report is a public version of the restricted report (GAO-09-53SU) that  we provided to you on December 8, 2008.  DHS and TSA deemed some of  the information in the restricted report as sensitive security information,  which must be protected from public disclosure. Therefore, this report  omits this information, such as the specific details associated with FAMS’s  concept of operations. Although the information provided in this report is  more limited in scope, it addresses the same questions as the restricted  report. Also, the overall methodology used for both reports is the same.
In studying FAMS’s operational approach, we reviewed relevant legislation  regarding FAMS’s mission, and we analyzed the agency’s policies and  other documentation regarding the strategy and concept of operations for  carrying out that mission, including the procedures for assigning or  scheduling air marshals to selected flights. Also, we reviewed the results  of an evaluation conducted in 2003 by the Office of Management and  Budget (OMB), which utilized its Program Assessment Rating Tool (PART)  to assess the management and performance of FAMS. Further, we also  reviewed the follow-on reassessment of FAMS that OMB conducted in  2008.
To determine the extent to which FAMS’s operational approach has been  independently assessed, we reviewed a July 2006 classified report  prepared by the Homeland Security Institute based on its independent  evaluation of FAMS’s concept of operations. Our engagement team  included a social science analyst and an economist with experience in risk  assessment, who used generally accepted social science research  standards in reviewing the Homeland Security Institute’s report. Also, we  interviewed applicable Homeland Security Institute officials to enhance  our understanding of the evaluation’s scope, methodology, findings, and  recommendations. Based on our review and discussion, we determined  the Homeland Security Institute’s report to be sufficiently reliable for the  purposes of our work. Further, we reviewed FAMS documentation—and  interviewed the Director of FAMS and other senior officials at the agency’s  headquarters—regarding the status of efforts to address recommendations  made by the Homeland Security Institute and any related initiatives  involving strategic planning and the agency’s concept of operations. We  also reviewed two additional Homeland Security Institute reports, which  FAMS provided to us in September 2008. One of the reports detailed the  Homeland Security Institute’s analysis regarding requirements for an  improved mission scheduling tool for FAMS, and the other report  presented the institute’s benchmark analysis that compared FAMS’s  workday rules and practices against those of similar occupations  involving frequent air travel and the related operational challenges,  including fatigue and other human factors.
Regarding processes and initiatives to address issues affecting the ability  of air marshals to carry out the agency’s mission, we reviewed prior  reports—including our September 2005 report on planning and controls  for FAMS. We also reviewed FAMS documentation regarding various  working groups and other initiatives that FAMS had established to address  issues that affect the ability of air marshals to carry out the agency’s  mission. For criteria in reviewing the agency’s documentation regarding  these efforts, we drew on our prior work regarding leading organizations  and the best practices for strategically managing human capital. Further,  we interviewed the Director of FAMS and other senior officials at agency  headquarters, and we visited 11 of the agency’s 21 field offices, where we  interviewed managers and a total of 67 air marshals. We selected the 11  field offices and the 67 air marshals based on nonprobability sampling.  Regarding field office selections, we considered various factors, such as  geographic location and the involvement of local management in  agencywide working groups to address issues affecting air marshals. Our  selections of air marshals were made to encompass a variety of experience  levels. Because we selected a nonprobability sample of FAMS field offices  to visit and air marshals to interview, the information we obtained in these  visits and interviews cannot be generalized either to all 21 field locations  or to all air marshals in the offices we visited. However, the interviews  provided us a broad overview of issues important to air marshals. We also  reviewed documentation regarding the implementation and results of a  workforce satisfaction survey that FAMS conducted in 2007. Our  engagement team, which included social science analysts with extensive  survey research experience, reviewed the questionnaire used in the survey  for clarity and the related response options for appropriateness. Also, we  discussed with FAMS officials the extent to which efforts were made to  obtain an overall response rate as high as possible. As criteria to guide our  review of the survey results, we used OMB’s Standards and Guidelines for  Statistical Surveys (September 2006) and related guidance applicable to  federal surveys.
We conducted this performance audit from April 2007 to December 2008 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives. More details about the scope and  methodology of our work are presented in appendix I.
subsections: section_title: Background
paragraphs: Originally established as the Sky Marshal program in the 1970s to counter  hijackers, the Aviation and Transportation Security Act expanded FAMS’s  mission and workforce in response to the September 11, 2001, terrorist  attacks and mandated the deployment of federal air marshals on high- security risk flights. Within the 10-month period immediately following  September 11, 2001, the number of air marshals grew significantly. Also,  during years following the 2001 attacks, FAMS underwent various  organizational transfers. Initially, FAMS was transferred within the  Department of Transportation from the Federal Aviation Administration to  the newly created TSA. In March 2003, FAMS moved, along with TSA, to  the newly established Department of Homeland Security. In November  2003, FAMS was transferred to U.S. Immigration and Customs  Enforcement (ICE). Then, about 2 years later, FAMS was transferred back  to TSA in the fall of 2005.
FAMS is one layer among multiple layers of aviation security. For  example, prospective passengers are prescreened against applicable  records in the Terrorist Screening Center’s consolidated watch list.  Passengers and baggage are also physically screened. Air marshals  generally are characterized as being the last line of defense within the  layered aviation-security framework. In this regard, FAMS officials  stressed that air marshals constitute the only in-flight security layer  deployed on the basis of risk.
FAMS deploys thousands of federal air marshals to a significant number of  daily domestic and international flights. In carrying out this core mission  of FAMS, air marshals are deployed in teams to various passenger flights.
Such deployments are based on FAMS’s concept of operations, which  guides the agency in its selection of flights to cover. Once flights are  selected for coverage, FAMS officials stated that they must schedule air  marshals based on their availability, the logistics of getting individual air  marshals in position to make a flight, and applicable workday rules.
At times, air marshals may have ground-based assignments. On a short- term basis, for example, air marshals participate in Visible Intermodal  Prevention and Response (VIPR) teams, which provide security  nationwide for mass transit systems other than aviation. Also, air  marshals participate in Joint Terrorism Task Forces led by the Federal  Bureau of Investigation.
Good marksmanship is considered a necessity for air marshals,  particularly given the unique environment of the core mission—the  relatively tight confines of an airplane, coupled with the presence of  numerous passengers (“bystanders”) and the possibility of air turbulence  that creates an unstable “shooting platform” (see fig. 1). Thus, according to  TSA, air marshals have the highest marksmanship standard in the federal  government and also must be recertified on their firearm every quarter.
To preserve their anonymity on covered flights, air marshals are to blend  in with other passengers by dressing appropriately and performing their  duties discreetly without drawing undue attention.
section_title: The Federal Air Marshal Service’s Operational Approach to Achieving Its Core Mission Is Based on Risk-Related Factors
paragraphs: FAMS’s operational approach (concept of operations) is based on risk- related factors, such as assessments of threat, vulnerability, and  consequences. FAMS is guided by the provisions of the Aviation and  Transportation Security Act that specify the deployment of federal air  marshals on flights presenting high-security risks, such as nonstop, long- distance flights targeted on September 11, 2001. FAMS seeks to maximize  coverage of high-risk flights by establishing coverage goals for 10 targeted  critical flight categories. In order to reach these coverage goals, FAMS  uses a scheduling process to determine the most efficient flight  combinations that will allow air marshals to cover the desired flights.  FAMS management officials stressed, however, that the overall coverage  goals and the corresponding flight schedules of air marshals are subject to  modification at any time based on changing threat information and  intelligence.
subsections: section_title: Risk-Related Factors Are Considered in Assigning Air Marshals to Flights
paragraphs: Following the attacks of September 11, 2001, FAMS developed a risk- based concept of operations for deploying air marshals on U.S.  commercial passenger air carriers. Because there are many more U.S. air  carrier flights each day than can be covered by air marshals, FAMS relies  on the methodology outlined in its concept of operations to assign air  marshals to flights with the highest security risks. Under this approach,  FAMS considers the following risk-related factors to categorize each of the  approximately 29,000 domestic and international flights operated daily by  U.S. commercial passenger air carriers into risk categories—high risk or  lower risk:
Threat (intelligence): Available strategic or tactical information  affecting aviation security is considered.
Vulnerabilities: Although FAMS’s specific definition is deemed to be  sensitive security information, DHS defines “vulnerability” as a  physical feature or operational attribute that renders an entity open to  exploitation or susceptible to a given hazard.
Consequences: FAMS recognizes that flight routes over certain  geographic locations involve more potential consequences than other  routes.
FAMS attempts to assign air marshals to provide an on-board security  presence on as many of the flights in the high-risk category as possible.  However, other considerations can make covering only high-risk flights  impractical from a scheduling perspective and potentially predictable to  an adversary. Therefore, for purposes of scheduling efficiency and  adversary uncertainty, FAMS may deploy some air marshals on lower-risk  flights.
section_title: Air Marshals Are Deployed on Flights to Meet Coverage Goals for Targeted Critical Flight Categories; the Concept of Operations Provides Flexibility for Responding to Changing Threats
paragraphs: FAMS has established a scheduling process intended to maximize the  coverage of high-risk flights and meet the agency’s desired coverage goals  for 10 targeted critical flight categories. FAMS’s Domestic Planning  Branch (within the Systems Operation Control Division) is responsible for  scheduling air marshals to domestic missions. During the course of a year,  the Domestic Planning Branch must prepare schedules for 13 roster  periods of 28 days each. According to FAMS officials, each 28-day  schedule takes approximately 3 weeks to prepare. The Domestic Planning  Branch prepares each domestic schedule using an automated scheduling  tool. As part of the scheduling process, each FAMS field office is  responsible for making available a specific percentage of their air marshals  on a daily basis to cover targeted critical flights (both domestic and  international flights) in the roster periods. FAMS utilizes the automated  scheduling tool to determine the most efficient flight “pairings” of  departure and return flights that will bring an air marshal back to his or  her starting point and that would be within the parameters for mission  assignment and rest for the air marshal. FAMS officials also perform other  checks on the fairness or appropriateness of the schedules, such as  ensuring that certain flights are not being covered repeatedly by the same  air marshals.
FAMS officials noted that the schedules for deploying air marshals are  altered as needed to cover specific, high-threat flights. For example, in  August 2006, FAMS increased its coverage of international flights in  response to the discovery, by authorities in the United Kingdom, of  specific terrorist threats directed at flights from Europe to the United  States. However, the officials added that a shift in resources of this type  can have consequences because of the limited number of air marshals.  FAMS officials noted that international missions require more resources  than domestic missions partly because the trips are of longer duration.
section_title: An Independent Assessment Concluded That the Federal Air Marshal Service’s Approach for Achieving Its Core Mission Was Reasonable; Recommendations for Enhancing the Approach Are Being Implemented
paragraphs: In its 2003 PART review of FAMS, OMB concluded that an independent  evaluation should be conducted to assess FAMS’s performance related to  aspects of the agency’s concept of operations—particularly aspects  involving flight coverage risk categories, the distribution of covered  flights, and target levels of coverage. The Homeland Security Institute, a  federally funded research and development center, performed this  evaluation and issued a final report in July 2006. The report concluded that  FAMS’s approach for achieving its core mission of providing an onboard  security presence for flights, as detailed in the agency’s concept of  operations, was reasonable and made several recommendations for  enhancements. The Homeland Security Institute recommended, for  example, that FAMS increase randomness or unpredictability in selecting  flights and otherwise diversify the coverage of flights within various risk  categories. As of October 2008, FAMS had implemented or had ongoing  efforts to implement the recommended enhancements.
In a July 2006 report, the Homeland Security Institute specifically noted  the following regarding FAMS’s overall approach to flight coverage:
FAMS applies a structured, rigorous approach to analyzing risk and  allocating resources.
The approach is reasonable and valid.
No other organizations facing comparable risk-management challenges  apply notably better methodologies or tools.
As part of its evaluation methodology, the Homeland Security Institute  examined the conceptual basis for FAMS’s approach to risk analysis. Also,  the institute examined FAMS’s scheduling processes and analyzed outputs  in the form of “coverage” data reflecting when and where air marshals  were deployed on flights. Further, the Homeland Security Institute  developed and used a model to study the implications of alternative  strategies for assigning resources. We reviewed the Homeland Security  Institute’s evaluation methodology and generally found it to be reasonable.
In a 2008 PART reassessment of FAMS, OMB also reported that the  Homeland Security Institute’s evaluation employed quality evaluation  methods and was comprehensive in scope. Further, OMB noted that an  interagency steering group—which was convened by the Homeland  Security Institute and met in conference in April 2006—also had reviewed  FAMS’s concept of operations and considered it to be reasonable. In  addition to FAMS and Homeland Security Institute participants, the  interagency steering group consisted of representatives from various law  enforcement and counterterrorism agencies, which included the Federal  Bureau of Investigation, U.S. Customs and Border Protection, the  Transportation Security Administration, the Federal Aviation  Administration, the Homeland Infrastructure Threat and Risk Assessment  Center, U.S. Northern Command/North American Aerospace Defense  command, and the National Counterterrorism Center.
In its July 2006 report, the Homeland Security Institute made several  recommendations for enhancing FAMS’s approach for deploying air  marshals on flights. As presented in table 1, FAMS had implemented or  had ongoing efforts to implement all of the recommended enhancements,  as of October 2008.
In reference to the core mission of FAMS, the Homeland Security  Institute’s recommendations regarding two processes—the filtering  process for selecting flights and the allocation process for assigning air  marshals to flights—are particularly important. To address the institute’s  recommendations, FAMS officials stated that a broader approach to  filtering flights has been implemented—an approach that opens up more  flights for potential coverage, provides more diversity and randomness in  flight coverage, and extends flight coverage to a variety of airports.
The Homeland Security Institute’s ongoing work has also resulted in two  reports delivered to FAMS in July 2008. One of the reports detailed the  institute’s analysis regarding requirements for a next-generation-mission  scheduling tool for FAMS, and the other report presented the institute’s  benchmark analysis that compared FAMS’s workday rules and practices  against those of similar occupations involving frequent air travel and the  related operational challenges, including fatigue and other human  factors. Also, in September 2008, the Homeland Security Institute  provided FAMS a classified report assessing the deterrent effects of the  agency’s approach to flight coverage. Further, based on its continuing  work, the institute expects to provide FAMS one additional final report by  the end of calendar year 2008—a report regarding potential enhancements  to performance measures.
section_title: The Federal Air Marshal Service Has Taken Positive Actions to Address Issues Affecting Its Workforce and to Help Ensure Continued Progress
paragraphs: To identify and address issues affecting the ability of its workforce to  successfully carry out its mission, FAMS has implemented various  communication-oriented processes or initiatives—including 36 issue- specific working groups—that have produced some positive results. For  instance, FAMS has revised and documented certain policies—including  the policy related to aircraft check-in and boarding procedures—to better  protect air marshals’ anonymity. In addition, FAMS has modified its  mission scheduling processes and implemented a voluntary lateral transfer  program to address certain issues regarding air marshals’ quality of life— and has plans to further address health issues associated with varying  work schedules and frequent flying. As an additional initiative to help  determine the effectiveness of management’s actions to address issues  affecting air marshals, FAMS conducted a workforce satisfaction survey of  all staff in late fiscal year 2007. A majority (79 percent) of the respondents  to the survey indicated that there had been positive changes from the prior  year, although the overall response rate (46 percent) constituted less than  half of the workforce. The 46 percent response rate was substantially less  than the 80 percent rate encouraged by OMB in its guidance for federal  surveys that require its approval. According to the OMB guidance, a high  response rate increases the likelihood that the views of the target  population are reflected in the survey results. Obtaining a higher response  rate to FAMS’s future surveys, which the agency plans to conduct every 2  years, and modifying the structure of some questions, could enhance the  surveys’ potential usefulness by, for instance, providing a more  comprehensive basis for assessing employees’ attitudes and perspectives.  All 67 of the air marshals we interviewed in 11 field offices attributed  progress under these efforts largely to the “tone at the top,” particularly  the commitment exhibited by the former FAMS Director who served in his  position from March 2006 to June 2008. To reinforce a shared vision for  workforce improvements and sustain forward progress, the current FAMS  Director has expressed a commitment to continuing applicable processes  and initiatives.
subsections: section_title: The Federal Air Marshal Service Implemented Various Processes or Initiatives to Address Issues Affecting Its Workforce
paragraphs: Our prior work has shown that leading organizations commonly sought  their employees’ input on a periodic basis—by, for example, establishing  working groups or task forces, convening focus groups, and conducting  employee satisfaction surveys—and used that input to adjust their human  capital approaches. Starting in March 2006, the then-serving FAMS  Director implemented several communication processes or initiatives to  better understand and address issues facing the agency’s workforce. Chief  among these processes or initiatives were issue-specific working groups  established to study, analyze, and address a variety of issues ranging from  mission, organizational, and operational topics to workforce satisfaction  and quality-of-life concerns. Initially, based on his knowledge of issues  facing the organization when he assumed the leadership position in March  2006, the FAMS Director established 12 working groups. Subsequently,  based on feedback from these initial groups and other sources regarding  issues of concern, the number of working groups expanded to 36 (see app.  V).
Each working group typically included a special agent-in-charge, a subject  matter expert, air marshals, and mission support personnel from the field  and headquarters. FAMS management directed working group members to  define each group’s purpose, analyze specific issues, develop short- and  long-term recommendations and determine their financial feasibility. As a  final product, FAMS management expected each working group to submit  a report, including recommendations, to the FAMS executive staff for  managerial consideration. According to FAMS management, the working  groups typically disband after submitting a final report. FAMS  management stressed, however, that applicable groups could be  reconvened or new groups established as needed to address relevant  issues.
In addition to the working groups, other processes or initiatives  implemented by FAMS management to address workforce issues or  otherwise improve management-workforce communication include the  following:
Field office focus groups—Each of the 21 FAMS field offices organized  a local focus group composed of representatives from the respective  office’s air marshal squads and at least one mission support staff. All  members serve on a rotating basis, and the groups are to meet at least  quarterly to discuss issues of concern to the local workforce and bring  these issues to the attention of the applicable field office’s special  agent-in-charge.
Field office visits by the FAMS Director— In 2006, the FAMS Director  began visiting field offices and holding informal gatherings with air  marshals, outside the presence of local managers, to discuss their  questions and concerns.
Listening sessions—FAMS senior management established forums to  allow direct communication between FAMS senior management and  various personnel. In 2006, the FAMS Director and Deputy Directors  conducted these sessions weekly in headquarters and the field offices  with a total of 10 to14 staff selected for each meeting. In 2007, this  format changed from weekly to monthly sessions and included larger  groups of FAMS personnel.
Dinners with the Director—In 2006, the FAMS Director began holding  weekly dinners to meet with air marshals transiting through the  Washington, D.C., area. These dinners provide an opportunity for air  marshals to speak personally with the director about any questions or  concerns. The FAMS Deputy Director and one assistant director also  attend these dinners with selected air marshals.
Director’s e-mail in-box—FAMS established an e-mail in-box for  agency personnel to provide feedback to the FAMS Director. At any  time, air marshals—whether at headquarters, in a field office, or  deployed on mission—can send their insights, ideas, suggestions, and  solutions to the FAMS Director.
Anonymous Web site—FAMS established an internal Web site for  agency personnel to provide anonymous feedback to FAMS  management on any topic.
Ombudsman Position—FAMS management assigned an air marshal to  the position of Ombudsman in October 2006. According to FAMS  management, the Ombudsman provides confidential, informal, and  neutral assistance to employees to address workplace-related  problems, issues, and concerns. FAMS reported that, in fiscal year  2007 (the first full year of the position), the Ombudsman handled 67  cases, and, through the first three quarters of fiscal year 2008, an  additional 54 cases.
FAMS officials estimated that, as of October 2008, more than one-fourth of  the agency’s employees had participated in one or more of these activities  which encompass the various working groups and other processes and  initiatives.
Based on input provided by the working groups and information obtained  through the other processes and initiatives, FAMS has taken or is planning  to take actions to address issues that affect the ability of air marshals to  carry out the agency’s mission. As discussed in the following sections,  these actions address operational issues, such as check-in and boarding  procedures that affect air marshals’ anonymity as well as quality-of-life and  health issues.
section_title: To Help Preserve the Anonymity of Air Marshals, the Federal Air Marshal Service Revised Its Policy and Procedures for Check-In and Boarding of Aircraft
paragraphs: To preserve their anonymity on covered flights, federal air marshals are to  blend in with other passengers by dressing appropriately and performing  their duties discreetly without drawing undue attention. In past years, air  marshals frequently asserted that the check-in and boarding policy and  procedures established by FAMS compromised their anonymity by  requiring repeated interactions with airline personnel. In September 2005,  we reported that the full extent of incidents that air marshals encounter  was unknown because FAMS lacked adequate management controls for  ensuring that such incidents were recorded, tracked, and addressed.
Accordingly, to facilitate management of incidents that affect air marshals’  ability to operate discreetly during their missions, our September 2005  report recommended that FAMS take the following four actions:
Develop a means for recording all incidents reported to the Mission  Operations Center that affect air marshals’ ability to operate discreetly  and criteria for determining which incidents require federal air  marshals to complete a mission report.
Develop a means for tracking and retrieving data on mission reports to  enable FAMS to analyze and monitor reported and systemic incidents.
Establish written policies and procedures for reviewing and addressing  reported incidents.
Establish a means for providing feedback on the status and outcome of  FAMS mission reports to the federal air marshals who submit them.
FAMS has taken steps to address all four of these recommendations and  also address the related feedback received from air marshals through  various working groups and other initiatives. In October 2005, FAMS  issued a written directive establishing policies and procedures for  reporting and managing mission incidents. In November 2005, we  reported that we had reviewed the directive and believed that it addressed  two of our recommendations—the first and the third recommendations.  More recently, in March 2008, FAMS issued an addendum to its written  directive establishing a means for providing feedback on the status and  outcome of FAMS mission reports to the federal air marshals who submit  them (fourth recommendation).
Additionally, FAMS revised its policy and procedures regarding interaction  with airline personnel during the check-in and boarding processes in order  to better protect the anonymity of air marshals in mission status. To help  ensure effective implementation, the new policy and procedures have  been incorporated into TSA’s Aircraft Operator Standard Security  Program, which specifies requirements that domestic passenger air  carriers must implement as part of their TSA-approved security programs.  According to FAMS officials, the recent update constitutes the first time  that the Aircraft Operator Standard Security Program guidance specifically  includes a section regarding the boarding of federal air marshals.
Through use of a database created in fiscal year 2006 to track mission  incidents, FAMS senior executive staff noted that analysis and monitoring  are conducted daily of reported incidents, including those that could  compromise the ability of air marshals to operate discreetly (second  recommendation). The first management report detailing overall incident  patterns and trends was produced in July 2008. Going forward, FAMS  officials stated that reports would be produced quarterly to allow  management to review patterns or trends regarding mission incidents and  the effectiveness of the new policy and procedures.
section_title: The Federal Air Marshal Service Modified Its Dress Code and Hotel Policies to Further Protect Air Marshals’ Anonymity
paragraphs: To further protect the anonymity of air marshals while on missions, and in  response to air marshals’ feedback and the working groups’  recommendations, FAMS management revised the dress code policy and  the hotel policy for air marshals in August 2006 and February 2007,  respectively. The revisions allow air marshals greater discretion in  selecting appropriate attire to wear on missions and choosing hotels for  overnight trips. Before the revisions, air marshals reported that the dress  code policy was too restrictive and forced them to dress too formally for  certain flights, such as those to vacation-oriented destinations. According  to the air marshals, this restrictive policy resulted in their standing out  from the other passengers, a situation that compromised their anonymity.  Similarly, before being revised, FAMS’s hotel policy directed air marshals  to stay at certain hotels on overnight missions so that they could be  located easily by management in an emergency. Additional considerations  of FAMS management for restricting the hotel selection were to ensure  that air marshals were able to stay at hotels within per diem rates and  would have ready access to transportation between the hotel and the  airport. Air marshals expressed concerns that repeatedly staying at the  same hotels risked exposing their anonymity. The revised policy allows air  marshals to select their own hotels, provided the hotels are within per  diem rates and have adequate transportation options. To alleviate  concerns of FAMS management about being able to contact air marshals  in an emergency, the revised policy requires air marshals to report their  hotel locations via the FAMS intranet.
All 67 of the air marshals we interviewed in the 11 field offices we visited  said that the revised dress code and hotel policies adequately addressed  their concerns.
section_title: The Federal Air Marshal Service Is Planning to Replace Its Inventory of Personal Digital Assistant Communication Devices by Early Fiscal Year 2009
paragraphs: FAMS has described the agency’s personal digital assistant (PDA)  communication device as being a lifeline for air marshals. The current  device carried by air marshals is intended to function as a cell phone and  personal computer and allow users to place phone calls, access the  Internet, send e-mails, pull up basic Microsoft Word documents, store  documents, and submit reports. However, the findings of FAMS’s  applicable working groups indicated that the current PDA communication  device has proven unreliable. Similarly, all 67 of the air marshals we  interviewed in 11 field offices stated that they had experienced problems  with their PDA device while on missions. Examples of problems reported  by air marshals included dropped calls or lost signals in certain  geographical areas, limited audio quality and durability, and lack of ability  to send certain required documents (such as time and attendance reports).  Another reported problem was the frequent freezing or locking of the PDA  device, which then necessitated use of a cumbersome reset process. As a  result of such problems, air marshals reported that the PDA device has  hindered their ability to communicate effectively with management while  in mission status. Additionally, the air marshals we interviewed  commented that the current PDA device is relatively large and bulky,  which potentially contributes to loss of anonymity.
In response to air marshals’ feedback and the working groups’  recommendations, FAMS is taking steps to procure new PDA  communication devices and distribute them to air marshals. Furthermore,  according to FAMS officials, the procurement contract for the new PDA  devices will provide for a 2-year replacement cycle. In the interim, to  improve voice communication capabilities pending arrival of the new  devices, FAMS officials reported that it issued new cell phones to air  marshals in June 2008. The officials noted, however, that air marshals still  must rely on the current PDA device for non-voice functions, such as  sending and receiving e-mail messages and documents, until the new PDA  devices are available.
section_title: To Address Quality-of-Life and Health Issues, the Federal Air Marshal Service Has Implemented Changes to Mission Scheduling and Enlisted the Assistance of Outside Experts
paragraphs: In reference to quality-of-life and health issues, mission scheduling  constitutes the most significant concern of air marshals, according to  feedback that FAMS management received from working groups and other  communication processes and initiatives. To be fully effective, air  marshals must be healthy, fit, and alert. However, FAMS’s Medical Issues  Working Group reported that air marshals have experienced various types  of health issues—poor physical fitness as well as musculoskeletal injuries  and upper respiratory infections—that may potentially be attributable to  frequent flying and the overall nature of their jobs. The working group  noted various challenges to ensuring that air marshals have adequate  sleep, exercise, and recovery time. A contributing factor noted is that the  agency’s automated scheduling tool historically has lacked the capability  to consistently program an air marshal’s daily start and end times  throughout a roster period, which makes normal sleep patterns difficult to  maintain and often results in fatigue. For instance, an air marshal may  have been scheduled to begin some days at 5 a.m. and other days at 10  a.m., with unpredictable ending times because of flight delays. In addition  to inconsistent shifts, the Medical Issues Working Group noted that air  marshals are subject to long hours—including arriving home late on a  Friday and then having to depart early the following Monday morning.  These types of schedules, according to the working group, make allowing  adequate time for workouts and maintaining healthy eating habits difficult  and also limit the amount of time available to take care of family and  personal needs.
To address these scheduling issues, FAMS has implemented or is planning  to implement various changes:
Mission exchange program—This program, which FAMS initially  piloted in 2006 and is now available to all 21 field offices, allows air  marshals within the respective field office to exchange mission days  based on a demonstrated need, such as medical issues or family- related issues. For instance, an air marshal with an 8 a.m. mission start  time and a 9 a.m. medical appointment could exchange shifts with  another air marshal for a later mission start time. The program is  intended to reduce the amount of unscheduled leave taken by air  marshals and otherwise mitigate the hardships or other effects  associated with FAMS’s current policy of requiring air marshals to  submit requests for annual leave 38 to 66 days in advance.
Preset ending time and 60-hour rule—In September 2006, FAMS  instituted a change to its mission-scheduling policy. The change is  designed to help ensure that air marshals complete their mission  flights by a preset time on the day before a regular day off (or the day  before scheduled annual leave) and not begin a new mission until  receiving a minimum of 60 hours rest. For example, if an air marshal’s  regular days off are Saturday and Sunday, and this individual’s mission  ended on Friday evening, the next mission assignment (on Monday)  would begin no earlier than Monday morning.
Limit on number of flight days—In April 2007, FAMS implemented  another change in mission-scheduling policy designed to distribute  flight days equitably and improve the balance between work and  personal life for air marshals. Specifically, under the new policy, each  air marshal’s total flight days are targeted to not exceed 18 days per  roster period and 200 days annually.
More rest time after completing extended international missions—Also  in April 2007, FAMS issued guidance to field offices to make every  attempt at increasing rest time for air marshals after completing an  extended international mission. Under this guidance, air marshals  returning from an international mission are to be given a non-flight day  as their next duty day when any one of the following three conditions  apply: (1) the return flight exceeds 10 hours in the air, (2) the flight  crossed the international date line, or (3) the overall mission (round- trip flights plus overnight stays) was 4 days or longer in duration.  Depending on an air marshal’s schedule, a non-flight day could be a  training day, a regular day off, or a non-mission status day.
More consistent start times—FAMS is currently developing a  modification to its scheduling tool to provide a consistent, defined  scheduling window (encompassing, for example, 3 hours) for air  marshals in mission status to report for duty during a 7-day period.  Under the planned modification, for instance, FAMS schedulers would  assign an air marshal to flights departing during 5:00 a.m. to 8:00 a.m.,  9:00 a.m. to 12:00 p.m., or other 3-hour window during the week. FAMS  officials stated that this modification, which is intended to provide  more consistent start times for each air marshal throughout the  applicable week, should be completed and ready for pilot testing by  the middle of calendar year 2009.
The 67 air marshals we interviewed in 11 field offices generally expressed  satisfaction with the various enhancements to mission scheduling,  although most (43) mentioned that implementation of the mission  exchange program was still evolving.
To more specifically address the health implications of flight scheduling,  several efforts were recently completed or are planned. For instance, the  Homeland Security Institute conducted a benchmark analysis and  assessment of fatigue issues related to air marshals and issued a report to  FAMS in July 2008. In its analysis, the institute compared FAMS’s  workday rules against other occupations—largely in the aviation realm— that face challenges involving frequent travel, jet lag, long work hours,  rotating shifts, and the stress of maintaining a schedule across multiple  flights and airports. The Homeland Security Institute noted that although  no other occupation is identical to that of air marshals, meaningful  comparisons were made with similar occupations, such as commercial  airline pilots and cargo pilots and law enforcement officers working in  aviation (e.g., U.S. Marshals Service aviation enforcement officers  responsible for transporting prisoners). In its July 2008 report, the  Homeland Security Institute noted that while stress and fatigue issues are  a part of all organizations and cannot be entirely eliminated, air marshals  are provided considerable blocks of rest within their schedules, when  assessed against similar occupations. Overall, the institute reported that  the results of the benchmark analysis showed that air marshals are  provided above-average time to recuperate from duty days.
Further, the institute noted that FAMS has taken various steps, including  implementation of the mission exchange program, to improve aspects of  mission scheduling. In addition, in October 2008, FAMS officials informed  us that the agency has funded a contract with the National Institute of  Justice to implement FAMS-specific research regarding mission  scheduling, work-rest cycles, fatigue, and performance.
section_title: To Further Address Quality-of-Life Issues, the Federal Air Marshal Service Has Implemented a Voluntary Lateral Transfer Program
paragraphs: According to FAMS officials, air marshals frequently cited the need for a  voluntary lateral transfer program during listening group sessions and  dinners with the director. FAMS working groups that examined quality-of- life issues also reported that the agency would benefit from implementing  a transfer program for air marshals to express interest in relocating to  another field office. Thus, in October 2006, FAMS management  implemented a voluntary lateral transfer program. Under the program, an  air marshal in good standing may request a transfer for up to three field  offices, ranked by order of preference, and FAMS management is to make  decisions based on the number of vacancies in each office and the  seniority of air marshals who apply for transfer.
In December 2006, FAMS announced that 176 air marshals had been  selected, during the first phase of the program, for transfer to new offices  within 60 days. In the second phase, which occurred in the spring of 2007,  FAMS management made transfer offers to 40 air marshals—all of whom  accepted. In the third and most recent phase, which occurred in the spring  of 2008, FAMS management made offers to 48 air marshals—of whom 45  accepted. FAMS expects to continue offering voluntary transfer  opportunities during open seasons in the spring of each year.
section_title: The Federal Air Marshal Service Conducted a Workforce Satisfaction Survey in 2007; the Potential Usefulness of Future Surveys Could Be Enhanced by Improving the Response Rate and Redesigning Certain Questions
paragraphs: In late fiscal year 2007, FAMS conducted a workforce satisfaction survey  of all staff—not just air marshals—to help determine issues affecting the  ability of agency personnel to perform their jobs—and to obtain feedback  on the effectiveness of measures already taken by management to address  relevant issues.  The 2007 survey questionnaire consisted of a total of 60  questions that covered 13 topics—senior leadership;  supervisor/management; resources and technology; training and  education; career development; policies and procedures; employee  involvement and autonomy; rewards and recognition; communication;  safety, health, and medical issues; work and family life; organizational  commitment; and job satisfaction. According to FAMS management  officials, the survey provided useful information on quality-of-life and  other issues affecting the ability of air marshals and other agency  personnel to perform their jobs. In addition, the officials reported that  survey results indicated that employees generally were pleased with the  policy changes and other actions implemented by management to address  relevant issues. For example, although the 2007 workforce satisfaction  survey had an overall response rate (46 percent) that constituted less than  half of the FAMS workforce, 79 percent of the respondents indicated that  there had been positive changes from the prior year. Regarding future  plans, FAMS expects to administer a workforce satisfaction survey every 2  years. FAMS officials stated that a purpose of the initial workforce  satisfaction survey was to establish a baseline for use in comparing the  results of future surveys.
In reviewing the 2007 survey’s implementation and results, we made  several observations that are important for enhancing the potential  usefulness of future surveys. First, as noted previously, the overall  response rate was 46 percent. FAMS officials expressed satisfaction with  this response rate given the highly mobile nature of their workforce. The  FAMS officials also noted that the 46 percent response rate was similar to  the response rates for other federal workforce satisfaction surveys.  However, the 46 percent response rate was substantially less than the 80  percent rate OMB encourages for federal surveys that require its approval.  Although internal workforce surveys such as the one conducted by FAMS  do not require OMB approval, we believe the OMB standards and guidance  provide relevant direction on planning, designing, and implementing high- quality surveys—including the need to obtain a high response rate to  increase the potential that survey responses will accurately represent the  views of the survey population.
Specifically, the OMB guidance stipulates that agencies must design  surveys to achieve the highest practical rates of response to ensure that  the results are representative of the target population and that they can be  used with confidence as input for informed decision-making. OMB  encourages agencies to obtain at least an 80 percent response rate, and its  guidance states that response rates are an important indicator of the  potential for nonresponse bias, which could affect the accuracy of a  survey’s results. For instance, survey estimates may be biased if the  individuals who choose to participate (respondents) differ substantially  and systematically in some way from those who choose not to participate  (nonrespondents). In general, a higher response rate increases the  likelihood that any bias problem is decreased, resulting in the views and  characteristics of the target population being more accurately reflected in  the survey’s results. Thus, for any federal survey that must be approved by  OMB, applicable guidelines stipulate that an analysis for possible  nonresponse bias must be conducted if the final response rate is less than  80 percent.
Regarding the 46 percent response rate for the 2007 survey, FAMS  management reported that an analysis of potential nonresponse bias was  conducted by comparing various demographic data provided by the  respondents to the FAMS workforce as a whole. Based on the analysis of  the available demographic data, FAMS concluded that nonresponse bias  did not exist as the respondents were representative of the entire  workforce. Although the analysis conducted by FAMS was a useful effort,  the potential for a nonresponse bias still exists given that over half of the  FAMS workforce did not respond to the survey.
As noted previously, concerns about nonresponse bias could be avoided or  mitigated by obtaining a higher response rate. FAMS employees were  given 3 weeks (August 23 through September 14, 2007) to complete the  2007 workforce satisfaction survey. According to FAMS management, even  though all employees (not just nonrespondents) were sent four messages  reminding them of the deadline for completing the voluntary survey, the  final overall response rate was 46 percent. We believe that other widely  acknowledged methods, outlined in OMB guidance, could improve the  response rate of future FAMS surveys. These methods include, for  example, promoting awareness of the survey through outreach efforts with  groups of prospective respondents and extending the cut-off date for  responding to the survey. Also, monitoring questionnaire returns and  targeting extra follow-up efforts to air marshals in particular field  locations that have comparatively low response levels could help.
Additional observations we made in reviewing the 2007 workforce  satisfaction survey’s questionnaire involve the sentence structure of  certain questions and the response options. Generally, any question that  combines two or more issues—but does not provide for separate or  respective answers—can cause uncertainty about how to respond if the  answer to each issue is different. Table 2 lists the seven 2007 workforce  satisfaction survey questions that used these types of sentence structures.  For instance, regarding the senior leadership of FAMS, question 3 cites  two concepts (“visions” and “initiatives”), as well as two actions (“shared”  and “supported”) associated with these concepts. However, the response  options did not account for the fact that experiences could be different  with each of these concepts and actions. Similarly, question 10 addresses  the reliability of equipment used by agency personnel and cites four  different devices. However, the response options did not account for the  fact that experiences could be different with each of these devices.
Also, none of the 60 questions in the 2007 workforce satisfaction survey  provided for response options such as “not applicable” or “no basis to  judge”—responses that would be appropriate when respondents had little  or no familiarity with the topic in question. Not providing response options  such as “not applicable” or “no basis to judge” could lead to potentially  misleading question responses. In the interest of being compliant,  respondents might be compelled to give a response, such as “neutral,” to a  question when they actually have no opinion due to either non- applicability or lack of familiarity with the question topic. While it might  be assumed that all individuals being surveyed should be familiar with the  topic of all questions, this might not be the case and will not be known  unless the questionnaire contains the relevant response options. For  example, question 39 (see app. VI), reads as follows: “I am satisfied that  the work-related concerns I address with management are addressed  appropriately.” As written, this sentence assumes that every employee has  raised work-related concerns with management. If a respondent had never  expressed work-related concerns with management, this individual might  not know how to respond, given the question’s existing response options.
Thus, based on the sentence structure of certain questions and the  response options, the results from the 2007 survey may provide an  incomplete assessment of employees’ perspectives and attitudes to FAMS  management. Regarding our observations on the design of survey  questions and response options, FAMS officials stated that limited  personnel resources precluded investing more time in development of the  survey questionnaire and that the survey had served a useful purpose in  providing information on issues of concern to be more fully explored  through other communication processes or initiatives. Nonetheless, in  developing future survey instruments, designing questions to avoid these  types of ambiguities could provide FAMS management with information  that is more focused and complete. Although we recognize that FAMS has  a variety of other processes and initiatives—in addition to the customer  satisfaction survey—for identifying and addressing workforce issues,  customer satisfaction surveys can be particularly useful given that they are  distributed to all employees and provide for anonymity of respondents.  Further, the design considerations that we discussed involve relatively  minor technical aspects that could be addressed with a minimal  investment of personnel resources.
section_title: The Federal Air Marshal Service Has Committed to Addressing Workforce Issues by Sustaining Applicable Processes and Initiatives
paragraphs: As highlighted in our prior work, agency leaders in best practice  organizations view people as an important enabler of agency performance  and recognize the need for sustained commitment to strategically manage  human capital. In developing approaches to managing the workforce,  leaders of best practice agencies seek out the views of employees at all  levels. Involving employees in the planning process helps agencies to  develop goals and objectives that incorporate frontline insights and  perspectives about operations. Further, such involvement can also serve  to increase employees’ understanding and acceptance of organization  goals and objectives and improve motivation and morale. Our work has  shown that leading organizations commonly sought their employees’ input  on a periodic basis and used that input to adjust their human capital  approaches. Among other means, the organizations collected feedback by  convening focus groups, providing opportunities for employees to  participate in working groups or task forces, and conducting employee  satisfaction surveys.
As discussed earlier in this report, FAMS has implemented a variety of  processes and initiatives to address workforce issues by soliciting the  views of front-line staff across the agency. Several key improvements in  FAMS policies and procedures have resulted from these efforts. Among  other improvements, for example, FAMS amended its policy for flight  check-in and boarding procedures to better ensure the anonymity of air  marshals in mission status. Also, the various processes and initiatives have  helped to improve agency morale, according to the federal air marshals we  interviewed. Moreover, agency officials noted that the processes and  initiatives represented a significant commitment in management time and  resources.
In our view, fostering continued progress in addressing workforce issues  at FAMS is important. The current FAMS Director, after being designated  in June 2008 to head the agency, issued a broadcast message to all  employees, expressing a commitment to continue applicable processes  and initiatives, including the working group process, listening sessions,  field office visits, and the internal Web site for agency personnel to  provide anonymous feedback to management on any topic. More recently,  in response to our inquiry, FAMS’s Chief of Staff reported in October 2008  that the various communications processes and initiatives “have become  an institutionalized and positive aspect” of the agency’s culture. Also, the  Chief of Staff noted that FAMS was in the process of establishing an  agencywide national advisory council—with representatives from  headquarters and all field offices—to further enhance communication and  outreach efforts, promote greater job satisfaction, and improve  organizational effectiveness through cooperative problem solving and  replication of best practices.
section_title: Conclusions
paragraphs: Federal air marshals are an important layer of aviation security. Thus, it is  incumbent upon FAMS management to have sound management processes  in place for identifying and addressing the challenges associated with  sustaining the agency’s operations and addressing workforce quality-of-life  issues. FAMS, to its credit, has established a number of processes and  initiatives—including a workforce satisfaction survey—to address various  operational and quality-of-life issues that affect the ability of air marshals  and other FAMS personnel to perform their aviation-security mission.  Consistent with the human capital practices of leading organizations, the  current FAMS Director has expressed a commitment to continuing  relevant processes and initiatives for identifying and addressing workforce  concerns, maintaining open lines of communications, and sustaining  forward progress. Although the workforce satisfaction survey is only one  of a number of processes or initiatives used by FAMS to identify and  address workforce issues, such surveys play an important role given their  agencywide scope and the provision for anonymous responses. A higher  response rate and more clearly structured questions and response options  could add to the usefulness of this effort.
section_title: Recommendation for Executive Action
paragraphs: To facilitate continued progress in identifying and addressing issues that  affect the ability of FAMS personnel to perform the agency’s aviation- security mission, we recommend that the FAMS Director take appropriate  actions to increase the usefulness of the workforce satisfaction surveys  that FAMS plans to conduct biennially. Such actions could include, for  example, ensuring that the survey questions and the answer options are  clearly structured and unambiguous and that additional efforts are  considered for obtaining the highest possible response rates.
section_title: Agency Comments
paragraphs: We provided a draft of our restricted report for comment to the  Department of Homeland Security and TSA. In November 2008, in written  comments, the Department of Homeland Security and TSA agreed with  our recommendation and noted that FAMS was in the initial stages of  formulating the next workforce satisfaction survey, which included plans  to implement the recommendation. Also, the Department of Homeland  Security and TSA commented that our key findings and recommendation  will facilitate continued progress in identifying and addressing issues that  affect the ability of FAMS personnel to perform the agency’s aviation  security mission. The full text of the department’s and TSA’s written  comments is reprinted in appendix VII.
As arranged with your office, we plan no further distribution until 30 days  after the date of this report. At that time, we will send copies of this report  to interested congressional committees and subcommittees. We will also  make copies available to others upon request.
If you or your staff have any questions about this report or wish to discuss  the matter further, please contact me at (202) 512-4379 or lords@gao.gov.  Contact points for our Offices of Congressional Relations and Public  Affairs may be found on the last page of this report. Other key contributors  to this report are listed in appendix VIII.
section_title: Appendix I: Objectives, Scope, and Methodology
subsections: section_title: Objectives
paragraphs: This report addresses the following three principal questions:
What is the Federal Air Marshal Service’s operational approach for  achieving its core mission of providing an onboard security presence  for flights operated by U.S. commercial passenger air carriers?
To what extent has the Federal Air Marshal Service’s operational  approach for achieving its core mission been independently assessed?
To what extent does the Federal Air Marshal Service have processes  and initiatives in place to address issues that affect the ability of its  workforce to carry out its mission?
section_title: Scope and Methodology
paragraphs: Initially, to obtain contextual and overview perspectives regarding the  principal questions, we reviewed information available on the Web sites of  relevant federal entities—the Department of Homeland Security, the  Transportation Security Administration (TSA), and the Federal Air  Marshal Service (FAMS). To obtain additional perspectives regarding  FAMS’s mission and operations—and issues affecting its workforce—we  conducted a literature search to identify relevant reports or studies and  other publicly available information, including news media articles. In  particular, we focused on reviewing congressional studies, Inspector  General reports, and our previous reports. These included the following:
U.S. House of Representatives, Committee on the Judiciary, Plane  Clothes: Lack of Anonymity at the Federal Air Marshal Service  Compromises Aviation and National Security (Washington, D.C.:  May 25, 2006).
GAO, Aviation Security: Federal Air Marshal Service Could Benefit  from Improved Planning and Controls, GAO-05-884SU (Washington,  D.C.: Sept. 29, 2005). The report is restricted (not available to the  public) because it contains sensitive security information. The public  version of the report is GAO-06-203 (Nov. 28, 2005).
U.S. Department of Homeland Security, Office of Inspector General,  Review of Alleged Actions by Transportation Security  Administration to Discipline Federal Air Marshals for Talking to the  Press, Congress, or the Public, OIG-05-01 (Washington, D.C.: Nov.  2004).
GAO, Budget Issues: Reprogramming of Federal Air Marshal Service  Funds in Fiscal Year 2003, GAO-04-577R (Washington, D.C.: Mar. 31,  2004).
GAO, Aviation Security: Federal Air Marshal Service Is Addressing  Challenges of Its Expanded Mission and Workforce, but Additional  Actions Needed, GAO-04-242 (Washington, D.C.: Nov. 19, 2003).
This report is the public version of a restricted report that we provided to  congressional requesters in December 2008. Further details about the  scope and methodology of our work regarding each of the three principal  questions are presented in the following sections, respectively.
subsections: section_title: The Federal Air Marshal Service’s Operational Approach for Achieving Its Core Mission
paragraphs: In addressing this topic, we reviewed relevant legislation regarding  FAMS’s mission and organizational structure. In particular, we reviewed a  provision of the Aviation and Transportation Security Act that  requires the deployment of federal air marshals on passenger airline  flights and  specifically requires the deployment of federal air marshals on every  flight determined to present high security risks.
We analyzed FAMS documentation regarding the agency’s strategy and  concept of operations for carrying out its mission. Also, we reviewed the  results of an evaluation conducted in 2003 by the Office of Management  and Budget (OMB), which utilized its Program Assessment Rating Tool  (PART) to assess the management and performance of FAMS and  concluded that key aspects of program design needed to be independently  assessed. Further, we reviewed the follow-on PART-related reassessment  of FAMS that OMB conducted in 2008 (see app. II).
section_title: Independent Assessment of the Federal Air Marshal Service’s Operational Approach
paragraphs: We reviewed the July 2006 classified report prepared by the Homeland  Security Institute based on its independent evaluation of FAMS’s concept  of operations. Our engagement team included a social science analyst and  an economist with experience in risk assessment, who used generally  accepted social science research standards in reviewing the institute’s  report. Also, we interviewed applicable Homeland Security Institute  officials to enhance our understanding of the evaluation’s scope,  methodology, findings, and recommendations. Based on our review and  discussion, we determined this report to be sufficiently reliable for the  purposes of our work. Further, we reviewed FAMS documentation—and  interviewed the Director of FAMS and other senior officials at the agency’s  headquarters—regarding the status of efforts to address recommendations  made by the Homeland Security Institute and any related initiatives  involving strategic planning and the agency’s concept of operations.
We also reviewed two additional Homeland Security Institute reports,  which FAMS provided to us in September 2008. One of the reports detailed  the institute’s analysis regarding requirements for a next-generation  mission scheduling tool for FAMS, and the other report presented the  institute’s benchmark analysis that compared FAMS’s workday rules and  practices against those of similar occupations involving frequent air travel  and the related operational challenges, including fatigue and other human  factors.
section_title: The Federal Air Marshal Service’s Processes and Initiatives for Addressing Issues That Affect Its Workforce
paragraphs: Regarding operational or tactical issues as well as quality-of-life issues that  affect the ability of air marshals to carry out the agency’s mission, we  reviewed published reports, including our September 2005 report  (GAO-05-884SU) as well as news media accounts of relevant issues. We  also reviewed FAMS documentation regarding various working groups  (see app. V) and other initiatives that FAMS had established to address  issues that affect the ability of air marshals to carry out the agency’s  mission. In particular, we reviewed the final report (if available) produced  by the respective working group. For criteria in reviewing the agency’s  documentation regarding these efforts, we drew on our prior work  regarding leading organizations and the best practices for strategically  managing human capital.
Further, we interviewed the Director of FAMS and other senior officials at  agency headquarters, and we visited 11 of the agency’s 21 field offices,  where we interviewed managers and a total of 67 air marshals. We  selected the 11 field offices and the 67 air marshals based on  nonprobability sampling. In selecting the 11 field offices, we considered  various factors, such as geographic location of the offices and the  involvement of local management in agencywide working groups to  address issues affecting air marshals.
At each of the 11 field offices, we first reviewed available work-related  information about individual air marshals, such as their starting dates with  FAMS and their involvement in ground-based assignments or any  agencywide working groups. Based on these factors, we selected and  interviewed 6 to 7 air marshals at each of the 11 field offices. Specifically,  we selected 6 air marshals at each of 10 field offices and 7 air marshals at  the remaining office. Our selections were made to encompass a variety of  experience levels. Also, at each field office, rather than meeting separately  with each individual, we conducted the interviews of the selected air  marshals in group settings to encourage a wide array of perspectives,  whether corroborating or contradictory. We conducted our interviews at  the field offices during a 7-month time period, July 2007 through January  2008.
Because we selected a nonprobability sample of FAMS field offices to visit  and air marshals to interview, the information we obtained in these visits  and interviews cannot be generalized either to all 21 field locations or to  all air marshals in the offices we visited. However, the visits and  interviews provided us a broad overview of issues important to air  marshals.
We reviewed documentation regarding the implementation and results of a  workforce satisfaction survey that FAMS conducted in 2007. Our  engagement team, which included social science analysts with extensive  survey research experience, reviewed the questionnaire used in the survey  for clarity and the related response options for appropriateness (see app.  VI). Also, we discussed with FAMS officials the extent to which efforts  were made to obtain an overall response rate as high as possible. As  criteria to guide our review of the survey results, we used the following  OMB guidance:
Standards and Guidelines for Statistical Surveys (September 2006).
Questions and Answers When Designing Surveys for Information  Collections (Jan. 20, 2006).
We conducted this performance audit from April 2007 to December 2008 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
section_title: Appendix II: OMB’s Program Assessment Rating Tool Results in 2003 and 2008 for the Federal Air Marshal Service
paragraphs: The Office of Management and Budget’s (OMB) Program Assessment  Rating Tool (PART) consists of a standard series of 25 questions intended  to determine the strengths and weaknesses of federal programs. The 25  total questions cover 4 broad topics—(1) program purpose and design, (2)  strategic planning, (3) program management, and (4) program  results/accountability.
This appendix provides an overview of OMB’s PART-based assessments of  the Federal Air Marshal Service (FAMS) conducted in 2003 and 2008.  Additionally, for each of the 25 questions used in the PART-based  assessments, the appendix compares OMB’s 2003 and 2008 answers and  summarizes OMB’s narrative findings (explanation and evidence). Also,  when the answers in 2003 and 2008 differed for a particular question, the  appendix briefly explains the basis for the respective answer.
More detailed information regarding the 2003 and 2008 OMB PART  assessments of FAMS can be found on OMB’s Web site:  www.ExpectMore.gov.
subsections: section_title: Overview of the 2003 and 2008 Assessments of the Federal Air Marshal Service
paragraphs: OMB’s PART assessments of federal agencies provide performance ratings  that indicate how effectively tax dollars are spent. Following an  assessment, OMB assigns an agency one of five possible overall ratings:
Effective. Programs rated “effective” set ambitious goals, achieve  results, are well-managed, and improve efficiency.
Moderately effective. A “moderately effective” rating indicates a  program that sets ambitious goals and is well-managed but needs to  improve its efficiency or address other problems in the programs’  design or management in order to achieve better results.
Adequate. An “adequate” rating describes a program that needs to set  more ambitious goals, achieve better results, improve accountability,  or strengthen its management practices.  Ineffective. An “ineffective” rating indicates a program that fails to use  tax dollars effectively and is unable to achieve results because of a  lack of clarity regarding the program’s purpose or goals, poor  management, or some other significant weakness.
Results not demonstrated. A “results not demonstrated” rating  indicates that a program has been unable to develop acceptable  performance goals or collect data to determine whether it is  performing.
In OMB’s 2003 PART assessment, FAMS received a rating of “results not  demonstrated” because at that time FAMS did not have measurable  results. Additionally, OMB cited strategic planning deficiencies that  included the absence of baselines, targets, and time frames associated  with performance goals and performance measurements. OMB further  noted the absence of a second long-term outcome measure, proxy  measures with respect to deterrence, and an efficiency measure.
In OMB’s 2008 PART assessment, FAMS received a rating of “moderately  effective.” Regarding the improved rating, OMB recognized the Homeland  Security Institute’s independent evaluation, which endorsed FAMS’s  concept of operations. Also, OMB noted that FAMS had addressed other  deficiencies by developing the following performance measures:
A second long-term outcome measure—the level of public confidence  in air marshals’ ability to promote aviation security—which is  reflective of FAMS’s purpose.
Proxy measures of deterrence, such as air marshals’ average annual  rate of accuracy in firearms requalification testing.
Efficiency measures, such as (a) cost per flight per air marshal and (b)  percentage of air marshals meeting the targeted number of flying days  per year.
section_title: Section-by-Section Details of the 2003 and 2008 Assessments of the Federal Air Marshal Service
paragraphs: The results of PART’s 25 questions in reference to the 2003 and 2008  assessments of FAMS are presented in tables 3 through 6, specifically:
The 5 questions in table 3 cover program purpose and design.
The 8 questions in table 4 cover strategic planning.
The 7 questions in table 5 cover program management.
The 5 questions in table 6 cover program results/accountability.
As presented in tables 3 through 6, the narrative discussion (explanation  and evidence) is our summary of OMB’s key points. If needed for purposes  of clarifying the respective topic or ensuring accuracy, we used additional  or alternative wording to summarize OMB’s findings. Also, in a few  instances, we updated the information as appropriate.
section_title: Appendix III: Excerpts from the Homeland Security Institute’s July 2006 Report
paragraphs: The Homeland Security Institute, a federally funded research and  development center, was established to assist the Department of  Homeland Security in addressing relevant issues that require scientific,  technical, and analytical expertise. The institute—after conducting an  evaluation of the Federal Air Marshal Service’s approach for achieving the  agency’s core mission of providing an onboard security presence for  flights operated by U.S. commercial passenger air carriers—issued its final  report in July 2006.
This appendix presents quoted excerpts that substantially replicate the  executive summary in the Homeland Security Institute’s July 2006 report.
subsections: section_title: Background and Tasking for an Independent Evaluation
paragraphs: “The Federal Air Marshal Service (FAMS) challenge to reduce risk in the  aviation domain is daunting. U.S. commercial passenger carriers make  roughly 28,000 domestic and international flights each day. These flights  canvas the globe and originate, terminate, or fly in proximity to thousands  of critical facilities. The FAMS must evaluate which flights it will defend  and to what extent. It cannot cover every flight.  “In response to an Office of Management and Budget (OMB) direction, the  FAMS asked HSI  for an independent  evaluation of its methods for analyzing risk and allocating resources. In  particular, it asked HSI to determine if its risk management processes and  the application of its concept of operations (CONOPS) to scheduled  commercial flights were valid.”
section_title: Evaluation Approach Used by the Homeland Security Institute
paragraphs: “We defined ‘validation’ as a test of whether or not the FAMS risk  management processes and the outcome of those processes are  reasonable and consistent externally with stated guidance and internally  with its own CONOPS. Our analysis involved three tasks. First, we  examined the conceptual basis for the FAMS approach to risk analysis.  Second, we examined the FAMS scheduling process and analyzed the  output of that process in the form of ‘coverage’ data, i.e., when and where  air marshals were deployed on flights. Third, we developed and employed  a basic quantitative model to study the implications of alternative  strategies for assigning resources.”
section_title: Findings Reported by the Homeland Security Institute
paragraphs: “Based on our analysis, we find that the FAMS applies a valid approach to  analyzing risk and allocating resources. In particular, its approach is  reasonable given the scarcity of resources and the guidance it has  received. It assesses risk as a function of threat, vulnerability, and  consequence and employs a filtering process along with an allocation tool  to optimize resource allocation. Moreover, the FAMS seeks to strengthen  risk management processes by improving its scheduling tools and  analytical techniques. We did not find any other organizations that face a  similar challenge and apply significantly better methodologies or tools.”
section_title: Additional Considerations Reported by the Homeland Security Institute
paragraphs: “During our analysis, we identified  five issues that the  FAMS should address itself or in conjunction with the Transportation  Security Administration (TSA), the Department of Homeland Security  (DHS), and the broader intelligence and security communities.”
subsections: section_title: Vulnerability
paragraphs: “The FAMS definition of vulnerability … is inconsistent with traditional  risk-based definitions, which focus on the probability that an attack will  succeed. It shifts the focus away from other potential vulnerabilities. We  recommend that the FAMS reconsider its approach to vulnerability  and engage the aviation security community on this issue.” (The  emphasis is in the original.)
section_title: Filtering Process
paragraphs: “The FAMS understanding of consequence and its subsequent ‘filtering’  process … bias its allocation decision. To focus limited resources, the  FAMS filters flights according to … .  “Guidance in the form of legislation and departmental memoranda  following 9/11 directed FAMS to focus on flights that present ‘high security  risks.’ But, ultimately, that guidance was ambiguous and could be  outdated. These fundamental assumptions concerning risk, on which  it  allocates resources, warrant interagency review by the  broader intelligence and security community.” (The emphasis is in the  original.)
section_title: Allocation Process
paragraphs: “The FAMS filtering process defines ‘high risk’ and directs its efforts  toward flights fitting those characteristics. Its allocation process—a  modified version of the SABRE software used by airlines to schedule flight  crews—attempts to cover the maximum number of high risk flights within  fixed resources. …  The scheduling tool requires manual involvement to  recognize and modify scheduling solutions, which may not be consistent  with effective risk reduction.  “Our analysis of one month of FAMS coverage data reveals … [some  concerns.] To compensate for a lack of resources and deny  predictability, the FAMS should integrate randomness into its  allocations.” (The emphasis is in the original.)  “Contrary to the popular use of the term ‘random,’ allocating resources in  such a way does not mean choosing them haphazardly or without a plan.  The overall probability distribution for a group of comparable aircraft can  be chosen based on risk analysis. For instance, the FAMS may choose to  cover  flights in and out of [a particular  geographic region]. But the tactical allocation decision concerning a  specific flight must be random and converge around the overall category  average over time. A terrorist group may be able to discern the overall  category average through effective, long-term surveillance but will never  know conclusively whether or not the flight it plans to hijack will be  covered on a particular day.”
section_title: Performance Measures
paragraphs: “The FAMS primary performance measure—average coverage rates—can  mask weaknesses in coverage patterns. In particular, they can mask a  situation in which certain flights within a category of comparable flights  are heavily covered while others are rarely if ever covered. Accordingly,  the FAMS should develop performance measures to track coverage  consistency. One example involves tracking coverage deviation, defined  as the average difference between the individual coverage rates of each  flight in a comparable category and the overall category coverage rate.”  (The emphasis is in the original.)
section_title: Decision Support Tools
paragraphs: “During the course of our analysis, we noted that FAMS decision makers  did not have a tool for evaluating the resource implications of different  threat scenarios and alternative coverage schemes. The SABRE scheduler  is not flexible enough to support quick-response analyses. The FAMS  should build a simple decision-support tool, along the lines of the  model we developed based on risk balancing, to facilitate a system- wide view of resource decisions.” (The emphasis is in the original.)  “Such a tool would augment, not replace its scheduling tool, by allowing  decision makers to look across the entire aviation system and investigate  the resource implications of alternative allocation scenarios. In other  words, how easily could the FAMS adapt to a different threat  environment? Could it significantly increase the number of marshals  aboard specific flights? Where might those resources come from? How  would these changes affect coverage elsewhere?”
section_title: Appendix IV: Performance Measures for Assessing the Federal Air Marshal Service
paragraphs: In 2003, the Office of Management and Budget (OMB) used its Program  Assessment Rating Tool (PART) to assess the management and  performance of the Federal Air Marshal Service (FAMS). At that time, a  key performance measure for assessing FAMS was based on coverage of  targeted critical flights under various risk categories. This measure is still  applicable currently; however, its designation has been changed from an  output measure to an outcome measure. Moreover, this performance  measure—the coverage of targeted critical flights—is now considered a  proxy indicator regarding air marshals’ ability to defeat an attempted  attack. Also, in further response to the findings of OMB’s 2003 assessment,  FAMS established two additional outcome measures, one of which serves  as another proxy indicator of air marshals’ ability to defeat an attempted  attack:    The additional proxy outcome measure is the average annual rate of  accuracy in air marshals’ firearms requalification testing.
The additional, non-proxy outcome measure is based on a national survey  of households to determine the level of public confidence in air marshals’  ability to promote aviation security.
These updated measures have been approved by the Department of  Homeland Security—and also were approved in 2008 by OMB during its  PART-based reassessment of FAMS. An overview of FAMS’s updated  performance measures is presented in table 7.
section_title: Appendix V: Working Groups Formed by the Federal Air Marshal Service to Address Various Issues
paragraphs: In March 2006, the Director of the Federal Air Marshal Service (FAMS)  communicated to employees his intention to establish working groups to  examine a variety of issues ranging from mission, organizational, and  operational topics to workforce satisfaction and quality-of-life concerns.  Two months later, in May 2006, the director communicated to FAMS  employees that 12 working groups had been established, with each chaired  by a field office special agent-in-charge (SAC) and that subject matter  experts from the field and headquarters were available to assist in an  advisory role.
Subsequently, the number of working groups increased to a total of 36.  Table 8 categorizes the 36 working groups and briefly summarizes the  purposes of each.
Also, regarding the status of the 36 working groups as of October 1, 2008,  FAMS officials reported the following (see notes to table 8):    18 working groups (table note a): Each of these working groups had  completed its work and given a final report to FAMS management. Each  report had been reviewed by FAMS executives and then distributed to  agency employees via a broadcast message from the FAMS Director. If  applicable, the broadcast message also presented management’s responses  to any recommendations made by the respective report.    9 working groups (table note b): Each of these working groups had  completed its work and given a final report to FAMS management. The  reports were undergoing review by FAMS executives.    5 working groups (table note c): Each of these working groups had yet to  complete its work and give a final report to FAMS management.    4 working groups (note d): Each of these working groups is to remain  ongoing. As such, final reports are not expected to be issued; rather, each  group will present its findings when applicable and by appropriate means.
section_title: Appendix VI: FAMS Workforce Satisfaction Survey
paragraphs: Appendix VI: FAMS Workforce Satisfaction  Survey  On behalf of the Director and the Office of Law Enforcement /FAMS Office of  Workforce  Planning  and  Management,  thank  you  for  taking  the  time  to  complete  this  survey.    The  purpose  of  this  survey  is  to  capture  information  regarding  workforce  satisfaction  at  OLE/FAMS.    The  information  obtained  by  this  survey  will  be  used  by OLE/FAMS  leadership  to  assess  the  current levels  of  workforce  satisfaction  for  the  purposes of planning, policy development, and program enhancement.  In addition, the  data  obtained  by  this  survey  will  be  used  to  evaluate  current  OLE/FAMS  workforce  satisfaction initiatives  and  strategies.    Please  take  the time  to  carefully  complete  this  survey.  Your input will remain confidential and is vital to making OLE/FAMS a premier law enforcement organization. Thank you for your help.
For each item, choose the response that best reflects your experience at OLE/FAMS.
SA     A       N       D     SD  SA      A      N      D      SD  9.  I have adequate equipment, supplies, and materials to  accomplish my duties.  10.  Generally, the equipment I use (e.g., firearm, computer, cell phone, PDA , etc.) to  perform my job works properly.  11.  The equipment I use is sufficiently easy to operate. 12.  I am generally satisfied with the quality of OLE/FAMS  physical facilities (e.g., workspaces, training facilities,  10/30/2008 physical fitness areas, firearms ranges, etc.)  13.  I am generally satisfied with the availability of OLE/FAMS physical facilities (e.g., workspaces, training  facilities, physical fitness areas, firearms ranges, etc.)
SA      A      N      D      SD  14. I receive the training I need to do my job.  15.  I am satisfied with the frequency/amount of training I  receive in my office.  16.  Generally, I am satisfied with the content and variety of  job-related training I receive in my office. 17.  I am satisfied with the continuing education  opportunities offered by my job.  18.  OLE/FAMS supports continuing education  opportunities relevant to my job.  19.  My work schedule affords me the opportunity to pursue  continuing education.  20.  I am encouraged by my supervisors and managers to  seek training and educational opportunities.  Career Development 21.  Overall, I am satisfied with the progress I have made toward my career goals.  22.  There are sufficient opportunities for career  advancement at OLE/FAMS.  23.  In my present position, I have a clearly understood  career path. 24.  Promotions to supervisory levels in OLE/FAMS are  based on merit. 25. In OLE/FAMS, the selection criteria for promotion are  clear. Policies & Procedures 26.  OLE/FAMS’ written policies support (and do not  hinder) mission accomplishment. 27.  The local policies and procedures of my office support  mission accomplishment. 28.  I am able to stay updated and am informed about the  latest policies and procedures.
Employee Involvement & Autonomy  29.  I have effective channels to voice my opinion regarding  work-related issues (e.g., working groups, listening  sessions, e-mail suggestion box, etc.). 30.  I am empowered to use my professional discretion in  daily execution of my duties. 31.  I am provided sufficient opportunities to participate in  important decisions affecting my work. Rewards & Recognition 32.  Outstanding performance is recognized in my office.  33.  I am satisfied with the promotion practices of  OLE/FAMS. 34.  In my office, monetary rewards (i.e., cash awards, in- position increases, etc.) are tied to performance. 35.  I am generally satisfied with my pay.  Communication 36.  OLE/FAMS’ policies and procedures are clearly  communicated and easy to understand.  37.  I am satisfied with communication within my office (e.g., FO , branch, division, directorate).  38.  There are mechanisms in place which allow me to  freely express my comments, concerns, and suggestions  without fear of retaliation.  39.  I am satisfied that the work-related concerns I address with management are addressed appropriately.  40.  I have enough information to do my job well.  Safety, Health & Medical Issues  41.  I feel that my job-related stress is manageable. 42.  I am generally satisfied with OLE/FAMS programs related to employee safety, health, and wellness. 43.  I am satisfied that OLE/FAMS management is  concerned for the health and safety of employees and is  working continuously to offer improved services. 44.  I have been provided information and resources to take  personal responsibility for my health and wellness as it  relates to my job (e.g., proper diet, fitness, sufficient rest). 45.  I feel that medical information relevant to my job is  communicated to me. Work & Family Life  46.  I am able to effectively balance my work with my personal/family life. 47.  My family is supportive of my career with OLE/FAMS.
SA      A      N      D      SD  SA      A      N      D      SD  SA      A      N      D      SD  SA      A      N      D      SD  SA      A      N      D      SD  48.  Current initiatives (e.g., Voluntary Lateral Transfer  Program, new office openings, etc.) have a positive effect  on quality of work life/family life.
10/30/2008 49.  OLE/FAMS leadership has implemented positive changes affecting scheduling.  50.  I am satisfied that OLE/FAMS is exploring initiatives to improve quality of life/family life. 51.  I have seen improvement in quality of work life and  family life as a result of the recommendations from the  Director’s Working Groups.  52.  I have seen positive changes made in OLE/FAMS in the  last year. Organizational Commitment  53.  I am proud to work for OLE/FAMS.  54.  I find my values are similar to OLE/FAMS values.  55.  I feel a sense of loyalty to OLE/FAMS.  56.  I am likely to stay at OLE/FAMS for the next 12  months. Job Satisfaction  57.  The work I do is important. 58.  I find my work challenging and interesting.  59.  Generally speaking, I am very satisfied with my job.  60.  I like the kind of work I do (e.g., my current duties and  assignment).
Comments Section  Please use this section to provide more specific information for any of the above  questions.
Reasons for Staying with OLE/FAMS Indicate the importance of each of the following factors in your reasons for staying with  OLE/FAMS.
section_title: Appendix VII: Comments from the Department of Homeland Security
section_title: Appendix VIII: GAO Contact and Staff Acknowledgments
subsections: section_title: Staff Acknowledgments
paragraphs: In addition to the contacts named above, Danny Burton and John Hansen  (Assistant Directors) and Michael Harmond (Analyst-in-Charge) managed  this assignment.
David Alexander, Chuck Bausell, Arturo Cornejo, Wendy Dye, Stuart  Kaufman, and Courtney Reid made significant contributions to the work.
Tom Lombardi provided legal support.
Katherine Davis provided assistance in report preparation.