National Transportation Safety Board: Progress Made in Management Practices, Investigation Priorities, Training Center Use, and Information Security, But These Areas Continue to Need Improvement
Published: 2008-04-23T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The National Transportation Safety Board (NTSB) plays a vital role in advancing transportation safety by investigating accidents, determining their causes, issuing safety recommendations, and conducting safety studies. To support its mission, NTSB's training center provides training to NTSB investigators and others. It is important that NTSB use its resources efficiently to carry out its mission. In 2006, GAO made recommendations to NTSB in most of these areas. In 2007, an independent auditor made information security recommendations. This testimony addresses NTSB's progress in following leading practices in selected management areas, increasing the efficiency of aspects of investigating accidents and conducting safety studies, increasing the utilization of its training center, and improving information security. This testimony is based on GAO's assessment of agency plans and procedures developed to address these recommendations.
section_title: What GAO Found
paragraphs: NTSB has made progress in following leading management practices in the eight areas in which GAO made prior recommendations. For example, the agency has improved communication from staff to management by conducting periodic employee surveys, which should help build more constructive relationships within NTSB. Similarly, the agency has made significant progress in improving strategic planning, human capital management, and IT management. It has issued new strategic plans in each area. Although the plans still leave room for improvement, they establish a solid foundation for NTSB to move forward. However, until the agency has developed a full cost accounting system and a strategic training plan, it will miss other opportunities to strengthen the management of the agency. NTSB has improved the efficiency of activities related to investigating accidents and tracking the status of recommendations. For example, it has developed transparent, risk-based criteria for selecting which rail, pipeline, hazardous materials, and aviation accidents to investigate at the scene. The completion of similar criteria for marine accidents will help provide assurance that NTSB is managing its resources in a manner to ensure a maximum safety benefit. Also, it is in the process of automating its lengthy, paper-based process for closing-out recommendations. Although NTSB has increased the utilization of its training center--from 10 percent in fiscal year 2006 to a projected 24 percent fiscal year 2008--the classroom space remains significantly underutilized. The increased utilization has helped increase revenues and reduce the center's overall deficit, which declined from about $3.9 million in fiscal year 2005 to about $2.3 million in fiscal year 2007. For fiscal year 2008, NTSB expects the deficit to decline further to about $1.2 million due, in part, to increased revenues from subleasing some classrooms starting July 2008. However the agency's business plan for the training center lacks specific strategies to achieve further increases in utilization and revenue. NTSB has made progress toward correcting previously reported information security weaknesses. For example, in an effort to implement an effective information security program, the agency's Chief Information Officer is monitoring corrective actions and has procured and, in some cases, begun to implement automated processes and tools to help strengthen its information security controls. While improvements have been made, work remains before the agency is fully compliant with federal policies, requirements, and standards pertaining to information security, access controls, and data privacy. In addition, GAO identified new weaknesses related to unencrypted laptops and excessive user access privileges. Agency officials attributed these weaknesses to incompatible encryption software and a mission need for certain users. Until the agency addresses these weaknesses, the confidentiality, integrity, and availability of NTSB's information and information systems continue to be at risk.

section_title: Letter
paragraphs: We appreciate the opportunity to testify before you today as you consider  the reauthorization of the National Transportation Safety Board (NTSB).  NTSB is a relatively small agency that has gained a worldwide reputation  as a preeminent agency in conducting transportation accident  investigations. With a staff of about 400 and a budget of $84.8 million in  fiscal year 2008, NTSB is charged with investigating every civil aviation  accident in the United States and significant accidents in the other modes,  determining the probable cause of these accidents, and providing  recommendations to address safety issues identified during accident  investigations and safety studies of multiple accidents. To support its  mission, NTSB built a training academy that opened in 2003 and provides  training to NTSB investigators and other transportation safety  professionals.
While new transportation technologies and NTSB’s safety  recommendations have made transportation safer than ever, the expected  increase in the demand for all transportation modes has the potential to  increase the number of accidents, which could place a strain on the ability  of NTSB to continue playing its vital role in transportation safety. As the  nation’s large and growing long-term fiscal imbalance demands a growing  share of federal resources, making increases in the budgets of individual  agencies uncertain, it is critical that NTSB use its resources in an efficient  manner to carry out its safety mission and maintain its preeminent  position. For this reason, in 2006, we conducted a broad review of the  agency’s management practices, examined how it carried out its activities  related to accident investigations and safety studies, and analyzed whether  its training center was cost-effective. We made recommendations in each  of these areas. In addition, in recent years, other entities have conducted  reviews and made recommendations to NTSB related to information  security practices. Our testimony addresses NTSB’s progress in (1)  following leading practices in management areas such as strategic  planning, human capital management, and financial management; (2)  increasing the efficiency of activities related to investigating accidents,  issuing recommendations, and conducting safety studies; (3) increasing  the utilization of its training center; and (4) responding to  recommendations from an independent information security audit. In  discussing NTSB’s progress in these areas, we will also provide views on  several related provisions in the agency’s reauthorization proposal.
Our testimony is based on our analysis of policies and procedures  developed by NTSB in response to recommendations made by GAO and  the independent audit, updates to information we reported in 2006, and  our analysis of provisions in NTSB’s reauthorization proposal. We  considered NTSB to have made limited progress in implementing a  recommendation when the agency was in the early planning stages and  documents or milestones for actions did not exist or they did not follow  leading practices. Recognizing that many recommendations may take  considerable time and effort to fully implement, we considered NTSB to  have made significant progress in implementing a recommendation if the  agency had taken steps beyond the early planning stages toward  addressing the concerns.  In this case, documents or policies had been  developed that, for the most part, followed leading practices. We  considered NTSB to have fully implemented a recommendation when the  agency had fully implemented plans or processes that followed leading  practices. We undertook this work in response to a legislative mandate  that we conduct an annual audit of NTSB.  Appendix I provides additional  information on the recommendations discussed below.
subsections: section_title: NTSB Has Made Progress in Improving Many Management Practices, But Further Improvements are Needed in Training and Financial Management
paragraphs: Overall, NTSB has made progress in following leading management  practices in the eight areas in which we made recommendations in 2006.  Our recommendations are based on leading practices identified through  our government wide work that are important for managing an agency.  Although NTSB is a relatively small agency, such practices remain  relevant. Figure 1 provides a summary of NTSB’s progress in implementing  our 12 management recommendations.
Among the areas that NTSB has made the most progress is improving  communication from staff to management, which should help staff and  management build more constructive relationships, identify operational  and work-life improvements, and enable management to better understand  and respond to issues faced by investigators and other staff. The agency  managers have, for example, hosted brown bag lunches with staff to  facilitate communication and conducted periodic surveys of employees to  determine, among other things, their level of satisfaction and ways to  improve communication. In addition, NTSB has made significant progress  in improving its strategic planning and human capital management, and  progress in developing an information technology (IT) strategic plan. For  example, NTSB has revised its strategic plan to follow some performance- based requirements, and it has developed strategic human capital and IT  plans. Although these plans still offer room for improvement, they  establish a solid foundation for NTSB to move forward, both broadly as an  agency and specifically with respect to IT efforts.
In addition, NTSB has made significant progress in improving its  knowledge management (i.e., a way for it to create, capture, and reuse  knowledge to achieve its objectives). While the agency has adopted a  strategy for knowledge management activities and hired a chief  information officer (CIO) to implement policies and procedures on  information sharing, until NTSB completes its strategic training plan,  which NTSB has told us will include a knowledge management  component, the implementation of NTSB’s knowledge management  strategy will be unclear.
To its credit, NTSB has taken some steps to improve its training activities,  such as hiring a training officer in April 2007 and requiring all staff to  complete individual development plans aimed at improving their  capabilities in support of the agency’s needs; however, NTSB does not  expect to complete a strategic training plan until later this year. In  addition, NTSB’s core competencies and associated courses for its  investigators lack sufficient information on the knowlegdge, skills, and  abilities for each competency to provide assurance that the agency’s  training curriculum supports its mission.
NTSB has also improved some aspects of its financial management by  correcting a violation of the Anti-Deficiency Act related to purchasing  accident insurance for employees on official travel, making progress  toward correcting another violation of the Act related to lease payments of  its training center, and receiving an unqualified or “clean” opinion from  independent auditors on its financial statements from fiscal years ending  September 30, 2003, through 2007. However, NTSB has made limited  progress in developing a full cost accounting system to track the time  employees spend on each investigation and in training.  It intends to  request funding to begin this effort in fiscal year 2010. Without a full cost  accounting system, project managers lack a comprehensive means to  understand how staff resources are utilized and to monitor workload.  Until NTSB improves its financial management and develops a strategic  training plan, it will miss the opportunity to better understand how its  limited resources are applied to activities that support the agency’s  mission, such as accident investigation, as well as individual staff  development.
In addition, a provision of NTSB’s reauthorization proposal would exempt  the agency from the Anti-Deficiency Act and allow it to incur obligations  both for the acquisition and lease of real property in advance or in excess  of an appropriation. If Congress decides to grant this exemption, we  suggest more narrow authority that addresses NTSB’s particular need to  obtain a new lease for its headquarters when the current lease expires in  2010. For example, authority to enter into leases for up to a specified  number of years using annual funds over the term of the lease would be a  more appropriate option. Typically, federal agencies do not require such  an exemption because they rent real property through the General  Services Administration (GSA), which has realty specialists, staff  knowledgeable about the leasing market, and experience in lease  administration. As part of the fee that GSA charges agencies (7 percent for  NTSB), agencies have the ability to walk away from a lease with 120 days  notice. If NTSB does not lease through GSA and instead is granted  delegation authority to deal directly with lessors, it might not have the 120- day agreement and would be responsible for all aspects of negotiating and  administering its leases.
section_title: NTSB Has Made Improvements Related to Accident Investigation, But Its Safety Impact Could be Greater with More Safety Studies
paragraphs: NTSB has improved the efficiency of activities related to investigating  accidents, such as selecting accidents to investigate and tracking the  status of recommendations, but it has not increased its use of safety  studies (see fig. 2). Since 1997, NTSB has issued about 2,400  recommendations. The agency has closed about 1,500 (63 percent) of  those recommendations, and of those it closed, 88 percent were closed  with the agency having taken acceptable action, while 12 percent were  closed with an “unacceptable” status.
NTSB is required by statute to investigate all civil aviation accidents and  selected accidents in other modes—highway, marine, railroad, pipeline,  and hazardous materials. NTSB has improved its process for selecting  accidents to investigate by developing transparent, risk-based criteria for  selecting which rail, pipeline, and hazardous materials accidents to  investigate and which aviation accidents to investigate at the scene, or  remotely, in a limited manner. The completion of its effort to develop  similar criteria for marine accidents will help provide assurance and  transparency that the agency is managing investigative resources in a  manner that ensures a maximum safety benefit. NTSB has also made  significant progress in improving its recommendation close-out process by  working to automate this process by the end of this fiscal year.  Completion of the automation should help speed the process and aid the  expedient delivery of information about recommendation status to  affected agencies. In addition, NTSB has begun to identify and share best  practices for accident investigations among investigators in all  transportation modes. These activities, when fully implemented, will help  to ensure the effective and efficient use of agency resources. In contrast,  NTSB has not increased its utilization of safety studies, which provide  analyses of multiple accidents and usually result in safety  recommendations.  NTSB officials told us that the agency does not have  enough staff to increase the number of safety studies and, therefore, they  hope to identify more cost effective ways to conduct the studies. We  believe that greater progress in this area, which could result in more safety  recommendations, would improve NTSB’s impact on safety.
NTSB’s reauthorization proposal seeks to make several changes to the  agency’s accident investigation process that have the potential to expand  the scope of the agency’s authority. For example, the proposal would  expand the definition of accidents to include events that affect  transportation safety, but do not involve destruction or damage. It is  unclear if this new authority would expand NTSB’s workload, since  “events” are not defined in the proposal, unlike “accidents” and  “incidents,” which NTSB already investigates and are defined in regulation.  In addition, NTSB has not explained the criteria for identifying events to  investigate. Without explicit criteria, the agency cannot be assured it is  making the most effective use of its resources.
section_title: NTSB Has Made Progress in Increasing the Utilization of the Training Center, But the Facility Remains Underutilized
paragraphs: While NTSB has taken steps to increase the utilization of the training  center and to decrease the center’s overall deficit, the classroom space  remains significantly underutilized. The agency increased utilization of  classroom space in the training center from 10 percent in fiscal year 2006  to 13 percent in fiscal year 2007. In addition, NTSB is finalizing a sublease  agreement with the Department of Homeland Security to rent  approximately one-third of the classroom space beginning July 1, 2008,  which would help increase utilization of classroom space to 24 percent in  fiscal year 2008. Further, in 2008, NTSB expects to deliver 14 core  investigator courses at the training center. While we do not expect any  classroom space ever to be 100 percent utilized, we believe a 60 percent  utilization rate for training center classrooms would be reasonable, based  on our knowledge of similar facilities.
The agency’s actions to increase utilization also helped increase training  center revenues from about $630,000 in fiscal year 2005 to about $820,000  in fiscal year 2007. By simultaneously reducing the center’s expenses—for  example, by reducing the number of staff working at the center—NTSB  reduced the training center’s annual deficit from about $3.9 million to  about $2.3 million over the same time period. We believe these actions to  increase utilization and their impact on the financial position of the  training center are positive steps and provide some progress toward  addressing our recommendations (see fig. 4).
In addition, for fiscal year 2008, NTSB’s March 2008 business plan  for the  training center estimates that revenues will increase by about $570,000 to  about $1.4 million and expenses will be $2.6 million, leaving a deficit of  about $1.2 million. The increase in revenues is due primarily to subleasing  all available office space at the training center to the Federal Air Marshals  starting in September 2007 for $479,000 annually. According to agency  officials, the projected deficit is no more than they would pay to provide  training and store accident wreckage somewhere else, but as discussed in  detail in appendix I, we do not believe that the plan provides enough  information to support this conclusion.
Going forward, however, the agency’s business plan for the training center  lacks specific strategies to explain how further increases in utilization and  revenue enhancement can be achieved.  According to agency officials,  they do not believe further decreases in the deficit are possible. However,  without strategies to guide its efforts to market its classes and the unused  classrooms, NTSB may be missing further opportunities to improve the  cost-effectiveness of the center.
section_title: NTSB Has Made Progress in Implementing Information Security- Related Recommendations, But Weaknesses Remain
paragraphs: Overall, NTSB has made progress in resolving or addressing weaknesses  identified in an independent external audit of NTSB’s information security  program, as required by the Federal Information Security Management Act  of 2002 (FISMA).  This evaluation, which was performed for fiscal year  2007 made eight recommendations to NTSB to improve compliance with  FISMA, strengthen system access controls, and take steps to meet the  requirements of the Privacy Act and related guidance by the Office of  Management and Budget (OMB).  Regarding FISMA compliance, NTSB  made important progress by, among other things, hiring a contractor to  perform security testing and evaluation of its general support system—an  interconnected set of information resources, which supports the agency’s  two major applications.  Although the contractor identified 113  vulnerabilities which collectively place information at risk, NTSB has  documented these vulnerabilities in a plan of action and milestones.   NTSB officials stated that they have resolved many of the vulnerabilities  and have actions under way to address the remaining vulnerabilities.   Figure 5 shows NTSB’s progress specific to each of the recommendation  made in the independent evaluation.
In addition to the weaknesses addressed in these recommendations, our  limited review of NTSB’s information security controls identified two new  weaknesses regarding unencrypted laptop computers and excessive  access privileges on users’ workstations.  Federal policy requires agencies  to encrypt, using only National Institute of Standards and Technology  (NIST) certified cryptographic modules, all data on mobile  computers/devices that contain agency data unless the data are  determined not to be sensitive by the agency’s Deputy Secretary or his/her  designate.  However, according to NTSB officials, the agency has not  encrypted data on 184 of 383 of its laptop computers.  As a result, agency  data on these laptops are at increased risk of unauthorized access and  unauthorized disclosure.  According to NTSB officials, the hardware on  these laptops is not compatible with NTSB’s encryption product. To help  mitigate the risk, NTSB officials stated that employees in the agency’s  telework program use encrypted laptops and that non-encrypted laptops  are to remain in the headquarters building.  NTSB officials stated that they  have ongoing efforts to identify and test compatible encryption software  for these laptop computers. Until NTSB encrypts data on its laptops,  agency data will remain at increased risk of unauthorized access and  unauthorized disclosure.
With regard to access, NTSB has inappropriately granted excessive access  privileges to users.  Users with local administrator privileges on their  workstations have complete control over all local resources, including  accounts and files, and have the ability to load software with known  vulnerabilities, either unintentionally or intentionally, and to modify or  reconfigure their computers in a manner that could negate network  security policies as well as provide an attack vector into the internal  network.  Accordingly, industry best practices provide that membership in  the local administrators’ groups should be limited to only those accounts  that require this level of access.  However, NTSB configures all users’  workstations with these privileges in order to allow investigators the  ability to load specialized software needed to accomplish their mission.   As a result, increased risk exists that these users could compromise  NTSB’s computers and internal network.  NTSB officials stated that they  are planning to deploy standard desktop configurations, which they  believe should address this vulnerability; however, the agency has not yet  provided a timeframe when this will be completed.  In the meantime, the  agency asserts that it continuously monitors and scans workstations for  vulnerabilities and centrally enforces the deployment and use of local  firewall applications. Until NTSB takes action to remove or limit users’  ability to load software and modify configurations on their workstations,  the agency is at increased risk that its computers and network may be  compromised. We believe that by fully resolving the weaknesses described  in the 2007 FISMA evaluation and addressing the newly identified  weaknesses, NTSB can decrease risks related to the confidentiality,  integrity, and availability of its information and information systems.
section_title: Conclusions
paragraphs: While NTSB has made progress in improving its management processes  and procedures, the full implementation of effective management  practices are critical to NTSB being able to carry out its accident  investigation mission and maintain its preeminent reputation in this area.   Further, until NTSB protects agency data and limits users’ access to its  systems, its information and information systems are at increased risk of  unauthorized access and unauthorized disclosure.  For continuing  Congressional oversight, it is important that Congress have updated  information on challenges that the agency faces in improving its  management.  While NTSB is required to submit an annual report on  information security, there is no similar reporting requirement for the  other management challenges.
section_title: Recommendations for Executive Action
paragraphs: To assist NTSB in continuing to strengthen its overall management of the  agency as well as information security, we are making three  recommendations to the Chairman of the National Transportation Safety  Board.   To ensure that Congress is kept informed of progress in improving  the management of the agency, we recommend that the Chairman (1)  report on the status of GAO recommendations concerning management  practices in the agency’s annual performance and accountability report or  other congressionally approved reporting mechanism.
We also recommend that the Chairman direct NTSB’s Chief Information  Officer to (2) encrypt information/data on all laptops and mobile devices  unless the data are determined to be non-sensitive by the agency’s deputy  director or his/her designate and (3) remove user’s local administrative  privileges from all workstations except administrators’ workstations,  where applicable, and document any exceptions granted by the Chief  Information Officer.
section_title: Agency Comments
paragraphs: We provided NTSB a draft of this statement to review. NTSB agreed with  our recommendations and provided technical clarifications and  corrections, which we incorporated as appropriate.
section_title: Scope and Methodology
paragraphs: To determine the extent to which NTSB has implemented the  recommendations we issued in 2006, we reviewed NTSB’s strategic plan,  IT strategic plan, draft human capital strategic plan, training center  business plan, and office operating plans.  To obtain additional  information about these documents and other efforts to address our  recommendations we interviewed NTSB’s Chief Information Officer, Chief  Financial Officer, General Counsel, and other agency officials as well as  representatives from NTSB’s employees union. To determine the extent to  which NTSB has implemented other auditors’ recommendations related to  information security, we reviewed work performed in support of the fiscal  year 2007 FISMA independent evaluation, as well as FISMA independent  evaluations performed by the Department of Transportation’s Office of  Inspector General in 2005 and 2006. We obtained evidence concerning the  qualifications and independence of the auditors who performed the 2007  FISMA review, and determined that the scope, quality, and timing of the  audit work performed by this audit supported our audit objectives. In  addition, we reviewed agency documents, and interviewed agency  officials, including information security officials. We compared  evaluations presented in audit documentation with applicable OMB and  NIST guidance, and the Federal Information Security Management Act  legislation. We also conducted a limited review of security controls on  NTSB’s information systems.  We considered NTSB to have made limited  progress in implementing a recommendation when the agency was in the  early planning stages and documents or milestones for actions did not  exist or they did not follow leading practices. Recognizing that many  recommendations may take considerable time and effort to fully  implement, we considered NTSB to have made significant progress in  implementing a recommendation if the agency had taken steps beyond the  early planning stages toward addressing the concerns.  In this case,  documents or policies had been developed that, for the most part,  followed leading practices. We considered NTSB to have fully  implemented a recommendation when the agency had fully implemented  plans or processes that followed leading practice.
This work was conducted in accordance with generally accepted  government auditing standards between October 2007 and April 2008.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Contacts and Acknowledgments
paragraphs: For further information on this testimony, please contact Dr. Gerald  Dillingham at (202) 512-2834 or by e-mail at dillinghamg@gao.gov or  Gregory C. Wilshusen at (202) 512-6244 or wilshuseng@gao.gov.  Individuals making key contributions to this testimony include Teresa  Spisak, Assistant Director; Don Adams; Lauren Calhoun; Elizabeth Curda;  Jay Cherlow; Peter Del Toro; William Doherty; Fred Evans; Colin Fallon;  Nancy Glover; David Goldstein; Brandon Haller; Emily Hanawalt; Chris  Hinnant; Dave Hooper; Hannah Laufe; Hal Lewis; Steven Lozano; Mary  Marshall; Mark Ryan; Glenn Spiegel; Eugene Stevens; Kiki  Theodoropoulos; Pamela Vines; Jack Warner; and Jenniffer Wilson.
section_title: Appendix I: Additional Information on Prior Recommendations Issued to NTSB by GAO and an Independent Auditor
subsections: section_title: Management-Related Recommendations
paragraphs: In 2006, we found that NTSB had taken positive steps to improve communications from  senior management to staff, such as periodically sending e-mails to all staff to share  information on new developments and policies. However, the agency lacked upward  communications mechanisms—such as town hall meetings, regular staff meetings, and  confidential employee surveys—which are central to forming effective partnerships  within the organization.
To improve agency communications, we recommended that NTSB develop  mechanisms that will facilitate communication from staff level employees to senior  management, including consideration of contracting out a confidential employee survey  to obtain feedback on management initiatives.
Our Assessment of NTSB’s Progress  NTSB has fully implemented this recommendation. NTSB management officials have   put in place processes to improve communication within the agency, and NTSB union  officials told us that they believe that upward communication has improved as a result.  For example, managers and Board members hold periodic meetings with staff, such as  brown bag lunches; conduct outreach visits to regional offices; hold “town-hall”  meetings in which NTSB employees ask questions of the managing director; and  conduct meetings with union leadership to provide information on upcoming actions by  the agency and to allow union leaders the opportunity to pose questions to  management.
In addition, the agency has formed two bodies comprising representatives from  management and staff intended to enhance internal communication, including upward  communication. One body is comprised of employees from NTSB’s administrative  offices, and the other from NTSB’s program offices. In addition, NTSB has begun  conducting several periodic surveys of employees, including (1) a survey to measure  staff satisfaction with internal communications; (2) a survey to obtain employees’ views  on the mission statement and goals that NTSB proposed for its revised strategic plan;  (3) four separate surveys to measure employee satisfaction with services provided by  NTSB’s administrative, human resources, and acquisition divisions and NTSB’s health  and safety program; and (4) a biennial survey to obtain employee feedback on NTSB’s  human resources efforts. This latter survey supplements—by being conducted during  alternating years—the Office of Personnel Management’s biennial survey of federal  employees that measures employees’ perceptions of the extent to which conditions  characterizing successful organizations are present in their agencies. NTSB officials  told us that because the communications survey indicated a need for NTSB’s individual  offices to hold more frequent staff meetings, the agency has established a goal for  fiscal year 2008 for each of its offices to achieve 75 percent of staff being either  satisfied or very satisfied with their office staff meetings.
In 2006, we found that NTSB’s strategic plan, issued in December 2005 for fiscal years  2006 through 2010, generally did not follow performance-based strategic planning  requirements in the Government Performance and Results Act of 1993 (GPRA) and  related guidance in the Office of Management and Budget’s Circular A-11.
As required by GPRA, the plan had a mission statement, four general goals and related  objectives, and mentioned key factors that could affect the agency’s ability to achieve  those goals. However, the goals and objectives in the plan did not have sufficient  specificity to know whether they had been achieved, and the plan lacked specific  strategies for achieving those goals, including a description of the operational  processes, skills and technology, and the resources required to meet the goals and  objectives as mandated by GPRA. Without a more comprehensive strategic plan, NTSB  could not align staffing, training, or other human resource management to its strategic  goals or align its organizational structure and layers of management with the plan.
To improve agency performance in the key functional management area of strategic  planning, we recommended that NTSB develop a revised strategic plan that follows  performance-based practices.
Our Assessment of NTSB’s Progress  NTSB has made significant progress in implementing this recommendation. NTSB  issued a revised strategic plan in February 2007 for fiscal years 2007 through 2012. The  revised plan more closely follows GPRA’s performance-based requirements than did  the previous plan, but it still does not fully follow several important requirements. (See  table 1.)
The revised plan improves upon the previous plan by    expressing most goals with sufficient specificity to enable a future assessment of  whether they were achieved;  including strategies for achieving 15 of 17 goals and objectives (NTSB describes  strategies for achieving the other two goals in its annual operating plans), indicating  that agency offices will establish annual performance goals designed to measure  progress in achieving general goals of the revised plan;    detailing the use of program evaluations to establish or revise goals and objectives;   incorporating input that NTSB solicited from internal stakeholders (agency  management and employees);  indicating that agency offices will establish annual performance goals designed to  measure progress in achieving general goals of the revised plan;    detailing the use of program evaluations to establish or revise goals and objectives;  incorporating input that NTSB solicited from internal stakeholders (agency  management and employees).
The revised plan does not fully follow two other GPRA requirements:    The plan does not incorporate two of the five agency mission areas in its goals and  objectives. NTSB officials told us that it chose to cover these two mission areas in  the annual operating plans of the responsible offices because the areas are not the  primary activity of the agency. Nevertheless, GPRA requires strategic plans to cover  all mission areas.
Although NTSB officials told us that the agency addressed concerns from Congress  in its revised plan, the agency did not obtain comments on a draft of the plan from  Congress. Nor did NTSB consult with other external stakeholders, such as the  federal and state transportation agencies to which it addresses many of its  recommendations. NTSB officials told us that they do not believe it would be  appropriate to consult with these agencies, which sometimes prefer not to implement  NTSB’s recommendations. Nevertheless, GPRA requires agencies, when developing  a strategic plan, to “solicit and consider the views and suggestions of those entities  potentially affected by or interested in the plan.”
In 2006, we found that NTSB was minimally following leading information technology  (IT) management practices. NTSB did not have a strategic plan for IT, and it had not  developed an enterprise architecture for modernizing its IT systems. It also lacked an  investment management process to control and evaluate the agency’s IT investment  portfolio. NTSB did not have acquisition policies for IT, such as project planning,  budgeting and scheduling, requirements management, and risk management. These  shortcomings suggested that NTSB was not ensuring that its management of  information technology was aligned to fully and effectively support its mission.
To improve agency performance in IT management, we recommended that NTSB  develop plans or policies for IT. The IT plan should include a strategy to guide IT  acquisitions.
Our Assessment of NTSB’s Progress  NTSB has made progress in implementing this recommendation. In August 2007,  NTSB issued an IT strategic plan that takes the following steps to address the  concerns that led to the recommendation:  It establishes goals and milestones for developing an enterprise architecture by  2012. (In November 2007, NTSB hired an enterprise architect to lead this effort.)
It includes a draft investment management process.
It establishes goals for implementing key aspects of the investment management  process by 2008 and the full process by 2012.  It establishes the goal of reaching Capability Maturity Model Integration level 2  (the level at which IT acquisitions and development can be said to be “managed”  rather than “chaotic”) by 2012.
To fully implement our recommendation, NTSB needs to improve one important  aspect of its IT strategic plan. Although other GAO work and NTSB’s IT strategic plan  stress the importance of aligning IT with agency strategic goals, the IT strategic plan  is not well aligned with the agency’s strategic plan. Specifically, the IT plan does not  address NTSB’s two top strategic priorities, namely (1) accomplishing objective  investigations of transportation accidents to identify issues and actions that improve  transportation safety and (2) increasing the agency’s impact on the safety of the  transportation system. NTSB officials told us that the agency is improving its IT in  ways that support these goals. For example, they said that efforts to develop a  project tracking system and upgrade its investigation docket system support the first  goal, and that the agency is redesigning its Web site and improving its Freedom of  Information Act information system in support of the second goal.
In 2006, we found that NTSB was minimally following leading knowledge management  practices. NTSB did not have a knowledge management initiative or program and  lacked a chief information officer to implement policies and procedures on information  sharing.
To improve agency performance in knowledge management, we recommended that  NTSB develop plans or policies for knowledge management.
Our Assessment of NTSB’s Progress  NTSB has made significant progress in implementing this recommendation. NTSB has  taken the following steps to improve its knowledge management:  It has issued an agency strategic plan and an IT strategic plan as well as other  plans and policies that include knowledge management activities.
It has made the deputy managing director responsible for knowledge management  activities within the agency.
It has hired a chief information officer to implement policies and procedures on IT  and information sharing.
NTSB still needs to take the following steps to improve its knowledge management:  It needs to revise its strategic plan and IT strategic plan to clearly identify which  agency plans, activities, and goals pertain to management of agency knowledge.
It needs to develop its strategic training plan, which NTSB officials told us will  include a knowledge management component. Until NTSB develops this plan and  revises the other two plans, its knowledge management activities pertaining to  training will be unclear.
In 2006, we found that NTSB developed a draft agencywide staffing plan in December  2005 that followed several leading practices in workforce planning but lacked other  leading practices such as a workforce deployment strategy that considers the  organizational structure and its balance of supervisory and nonsupervisory positions.  In addition, while managers were involved in the workforce planning process,  employees were not. Employee input provides greater assurance that new policies are  accepted and implemented because employees have a stake in their development.
To avoid excess organizational layers and to properly balance supervisory and  nonsupervisory positions, we recommended that NTSB align its organizational  structure to implement its strategic plan. In addition, we recommended that NTSB  eliminate any unnecessary management layers.
Our Assessment of NTSB’s Progress  NTSB has fully implemented our recommendation to align its organizational structure to  implement NTSB’s revised strategic plan. NTSB’s office operating plans describe how  each office serves the NTSB’s mission as defined in its mission statement. Further, the  plans align their offices’ respective performance objectives, and actions addressing  such objectives, to strategic goals in NTSB’s revised strategic plan.
NTSB has made significant progress in implementing our recommendation to eliminate  unnecessary management layers. For example, to streamline the management  structure in the Office of Aviation Safety, NTSB realigned the operations at 10 regional  offices into four regions. This action simplified its reporting structure and made  available a larger pool of accident investigators per region. NTSB union officials told us  that the union has been involved in planning this consolidation. NTSB officials told us  that the agency is not likely to consolidate any of its other modal offices because doing  so would not allow the agency to eliminate supervisory positions since the supervisors  in these offices spend a large portion of their time performing investigative duties.
In 2006, we found that NTSB partially followed leading human capital practices in  workforce planning; performance management; and recruiting, hiring, and retention and  minimally followed leading practices in training and diversity management. In  December 2005, NTSB developed a draft agencywide staffing plan that followed  several leading practices but lacked a workforce deployment strategy that considered  the agency’s organizational structure, its balance of supervisory and non-supervisory  positions, and succession plans to anticipate upcoming employee retirement and  workforce shifts. NTSB had issued performance plans for its senior managers and  overall workforce. However, the goals in NTSB’s strategic plan were not sufficiently  specific for staff to know whether their performance was contributing to meeting those  goals. NTSB had implemented several flexibilities to assist with recruiting and retention;  however, NTSB had neither a strategic recruitment and retention policy nor any  succession plans. Further, NTSB did not follow the leading practices of integrating  diversity management into its strategic plan and having a formal mentoring program  and advisory groups to foster employee involvement in diversity management.
To ensure that NTSB’s human capital management is aligned to fully and effectively  support its mission, we recommended that the agency develop a strategic human  capital plan that is linked to its overall strategic plan. The human capital plan should  include strategies on staffing, recruitment and retention, training, and diversity  management.
Our Assessment of NTSB’s Progress  NTSB has made significant progress in implementing this recommendation. In April  2008, NTSB provided us its draft human capital plan, which includes strategies for  addressing eight human capital objectives included in NTSB’s revised strategic plan.  However, these strategies do not always have clear linkages to the strategic plan. For  example, the draft human capital plan objective and strategies for attracting well- qualified applicants to critical occupations clearly aligns with the revised strategic plan  objective of maintaining a competent and effective investigative workforce. However,  the draft human capital plan objective and strategies for monitoring execution of human  capital strategic objectives does not align with the revised strategic plan objective of  project planning; while the strategies lay out the provision of annual updates regarding  the human capital plan, they do not specifically address the development of a project  plan or its evaluation.
The draft human capital plan incorporates several strategies on enhancing the  recruitment process for critical occupations, and addresses succession management  through several courses of action, such as implementing operations plans on executive  leadership and management development. While the plan cites recruiting and retaining  a diverse workforce, its strategies address recruitment but not other leading practices  of diversity management that could contribute to retaining a diverse workforce, such as  mentoring, employee involvement in diversity management, or succession planning.   For example, one strategy involves the use of the NTSB diversity resource guide,  which narrowly focuses on the recruitment of underrepresented groups, and does not  address other leading practices of diversity management. Another strategy mentioned  related to diversity involves the incorporation of diversity objectives into NTSB’s office  operating plans, which also focus on recruitment.
NTSB officials told us that the agency’s diversity management efforts focus on  recruiting because NTSB needs to attract a more diverse workforce. The officials also  told us that because the agency has a low attrition rate, it does not put as much  emphasis on retention of a diverse workforce.  We agree that it is important to attract a  diverse workforce, however, a low attrition rate does not assure a work environment  that retains and promotes a diverse workforce.
In 2006, we found that NTSB was minimally following leading practices in training,  which is a key area of human capital management. In particular, NTSB had neither  developed a strategic training plan, nor had it identified the core competencies needed  to support its mission and a curriculum to develop those competencies. Although NTSB  staff annually identified what training they needed to improve their individual  performance, as a result of not having a core curriculum that was linked to core  competencies and the agency’s mission, NTSB lacked assurance that the courses  taken by agency staff provided the necessary technical knowledge and skills.
To improve agency performance in the key functional management areas of strategic  and human capital planning, we recommended that NTSB develop a strategic training  plan that is aligned with the revised strategic plan, identifies skill gaps that pose  obstacles to meeting the agency’s strategic goals, and establishes curriculum that  would eliminate those gaps. In addition, we recommended that NTSB develop core  investigator curriculum for each mode.
Our Assessment of NTSB’s Progress  NTSB has made limited progress in implementing our first recommendation. NTSB  officials told us that later in 2008, the agency intends to complete a strategic training  plan that is linked to the agency’s strategic goals. To help develop the plan, NTSB  plans to survey staff about their skill gaps and to develop a curriculum to eliminate  those gaps. In fiscal year 2008, NTSB began requiring all staff to complete individual  development plans aimed at improving their capabilities in support of organizational  needs. NTSB also plans to use information gleaned from these plans in developing its  strategic training plan. Once NTSB has completed the training plan and the curriculum,  we will be able to assess the extent to which they address our recommendation.
NTSB has also made limited progress in implementing our second recommendation.  Although NTSB has developed a list of core competencies and associated courses for  investigators, the agency has not described the knowledge, skills, and abilities for each  competency. We have previously reported that well-designed training and development  programs are linked to, among other things, the individual competencies staff need for  the agency to perform effectively. Without such descriptions, NTSB does not have  assurance that its core curriculum supports its mission. In addition, NTSB has not  described the specialized competencies for its investigators in its various modes.   However, the  marine office plans to develop specialized core competencies and  curriculum for its investigators in 2008, and NTSB’s other modal offices plan to do so at  some later date after evaluating their investigators’ individual development plans.   Because these curricula are important to help NTSB effectively meet its mission, we  believe that NTSB’s senior managers and training managers should participate in the  development and review of the curricula and the underlying competencies.
To its credit, NTSB has taken or plans the following additional steps to improve its  training:  In April 2007, the agency hired a training officer, who is responsible for helping to  identify training needs, developing related curriculum, and evaluating training  courses.
In fiscal year 2007, it began to encourage senior investigators to increase their  participation in non-traditional training opportunities, such as spending time aboard   oil tankers and in flight simulators to learn about marine and aviation operations,  respectively.
In fiscal year 2008, it began requiring all staff to complete at least 24 hours of  training per year.
In fiscal year 2008, it plans to evaluate the extent to which individual training courses  resulted in desired changes in on-the-job behaviors for each of the 27 courses it  plans to offer at the training center.
In 2006, we found that NTSB had violated the Anti-Deficiency Act because it did not  obtain budget authority for the net present value of the entire 20-year lease for its  training center lease obligation at the time the lease agreement was signed in 2001.   This violation occurred as a result of NTSB classifying the lease as an operating lease  rather than a capital lease.  NTSB realized the error in 2003 and reported its  noncompliance to Congress and the President.  NTSB had proposed in the President’s  fiscal year 2007 budget to remedy this violation by inserting an amendment in its fiscal  year 2007 appropriation that would allow NTSB to fund this obligation from its salaries  and expense account through fiscal year 2020.  However, this proposal was removed  once the budget went to the House and Senate Appropriations Committees, leaving the  violation uncorrected.
In 2007, NTSB believed it had violated the Anti-Deficiency Act on a separate matter,  namely the improper use of its appropriated funds to purchase accident insurance for  its employees on official travel, and it asked GAO for an opinion on the matter.  We  determined that this was a violation because NTSB did not have an appropriation  specifically available for such a purpose, and the payments could not be justified as a  necessary expense.
We recommended that NTSB should identify and implement actions to correct its  violation of the Anti-Deficiency Act related to its lease of the training center.  These  actions could include obtaining a deficiency appropriation for the full costs of the lease,  renegotiating or terminating the training center lease so that it complies with the Anti- Deficiency Act, or obtaining authority to obligate lease payments using annual funds  over the term of the lease.
We did not make a recommendation regarding NTSB’s other violation of the act  because we reported that violation in a Comptroller General’s decision and such  decisions do not include recommendations.  Nevertheless, a Comptroller General’s  decision that an agency has violated the Anti-Deficiency Act, in and of itself, suggests  that the agency should correct the deficiency.
Our Assessment of NTSB’s Progress  NTSB has made significant progress in addressing its violation of the Anti-Deficiency  Act related to lease payments of its training center.  NTSB officials told us that because  congressional appropriators do not want to appropriate funds for the remaining lease  payments in a single appropriation law, NTSB worked with Congress to obtain authority  to use its appropriations for fiscal years 2007 and 2008 to make its lease payments  during those periods.  To avoid future violations, NTSB will need to continue to work  with Congress to obtain similar authority in its future annual appropriations.  In addition,  NTSB officials told us that the agency has asked Congress to ratify the lease payments  it made from 2001 through 2006.
NTSB has fully addressed its violation related to purchasing accident insurance for  employees on official travel.  In September 2007, NTSB reported the violation to  Congress and the President, as required by the act.  NTSB also successfully worked  with Congress to remedy the violation through a fiscal year 2008 appropriation.  NTSB  cancelled the insurance policy, and NTSB officials told us that the agency has worked  with Congress to obtain authority for future purchases of accident insurance.  A bill to  reauthorize the Federal Aviation Administration would provide NTSB with such  authority.
In 2006, we found that NTSB had made significant progress in improving its financial  management by hiring a Chief Financial Officer and putting controls on its purchasing  activities.  As a result of actions taken by NTSB, the agency received an unqualified or  “clean” opinion from independent auditors on its financial statements for the fiscal years  ending September 30 for the years 2003, 2004, and 2005. The audit report concluded  that NTSB’s financial statements presented fairly, in all material respects, the financial  position, net cost, changes in net position, budgetary resources, and financing in  conformity with generally accepted accounting principles for the three years.  However,  without a full cost accounting system capable of tracking hours that staff spent on  individual investigations, in training, or at conferences, NTSB lacked sufficient  information to plan the allocation of staff time or to effectively manage staff workloads.
To improve agency performance in the key functional management area of financial  management, we recommended that NTSB develop a full cost accounting system that  would track the amount of time employees spend on each investigation and in training.
Our Assessment of NTSB’s Progress  NTSB has made limited progress in implementing this recommendation.  Although  NTSB routinely assigns a project code to many non payroll costs, its time and  attendance system still does not allow the agency to routinely and reliably track the  time that employees spend on each investigation or other activities, such as training.   However, NTSB officials told us that the agency wants to add the ability to charge costs  to projects (i.e., activities) and that it has discussed this addition with the provider of  most of NTSB’s financial system needs—the Department of Interior’s (DOI) National  Business Center.  According to NTSB officials, this modification would enable direct  recording by activity of hours worked and of corresponding payroll costs.  NTSB  officials also said that because the agency has not had sufficient funding to make this  modification, it intends to request specific funding for this effort as part of its budget  appropriation for fiscal year 2010.  NTSB said that in the meantime, it will continue  discussions with DOI and that it has begun to benchmark the planned modification to  systems of agencies of comparable size. It anticipates that, once underway, DOI would  work with NTSB to manage the implementation.
section_title: Recommendations Related to NTSB’s Accident Investigation Mission and Safety Studies
paragraphs: In 2006, we found that for some transportation modes, NTSB had detailed, risk-based  criteria for selecting which accidents to investigate, while for others it did not.  For  example, NTSB had criteria to select highway accidents for investigation based on the  severity of the accident and amount of property damage.  In contrast, NTSB did not  have a documented policy with criteria for selecting rail, pipeline, and hazardous  materials accidents.  Instead, the decisions to investigate accidents were made by the  office directors based on their judgment.  As a result, for these modes, the agency  lacked assurance and transparency that it was managing resources in a manner that  ensured a maximum safety benefit.  Such criteria were also important because NTSB  did not have enough resources to investigate all accidents.
To make the most effective use of its investigation resources and increase  transparency, we recommended that NTSB develop orders for all transportation modes  that articulate risk-based criteria for determining which accidents would provide the  greatest safety benefit to investigate or, in the case of aviation accidents, explain which  accidents are investigated at the scene, or remotely, in a limited manner.
Our Assessment of NTSB’s Progress  NTSB has made significant progress in implementing this recommendation.  NTSB  developed a transparent policy containing risk-based criteria for selecting which rail,  pipeline, and hazardous materials accidents to investigate.  This policy assigns priority  to investigating accidents based on whether the accident involved a collision or  derailment and whether it involved fatalities or injuries, among other factors.  For  marine accidents, NTSB has a memorandum of understanding with the U.S. Coast  Guard that includes criteria for selecting which accidents to investigate.  To enhance  the memorandum of understanding, NTSB plans to consult with stakeholders and  develop an internal policy on selecting marine accidents in 2008 once certain legal  issues are resolved.  In addition, NTSB has developed a transparent, risk-based policy  explaining which aviation accidents are investigated at the scene, or remotely, in a  limited manner, depending on whether they involve a fatality and the type of aircraft.
In 2006, we found that NTSB’s process for changing the status of recommendations  was paper-based and used sequential reviews, which slowed the process and  prevented expedient delivery of information about recommendation status to affected  agencies.
We recommended that NTSB improve the efficiency of its process for changing the  status of recommendations by computerizing the documentation and implementing  concurrent reviews.
Our Assessment of NTSB’s Progress  NTSB has made significant progress in implementing this recommendation.  NTSB  recently completed a pilot program that involved electronic distribution of documents  related to recommendation status.  The results of that test are helping to guide  development of an information system intended to help the agency manage its process  for changing the status of recommendations.  NTSB aims to fully implement the system  by the end of fiscal year 2008.  NTSB said that the system is being developed to  support concurrent reviews.  When fully implemented, this system should serve to  close our recommendation.
NTSB faced challenges to efficiently develop its reports; partly as a result, its  investigations of major accidents routinely took longer than 2 years to complete.  These  challenges included multiple revisions of draft investigation reports at different levels in  the organization, excessive workloads for writer/editors, and too few final layout and  typesetting staff.  NTSB had taken several actions aimed at shortening report  development time, such as reemphasizing its policy on holding report development  meetings to obtain early buy-in on report messages and holding modal directors  accountable for specific issuance dates.  We also identified practices in certain offices,  such as the use of a project manager or deputy investigator-in-charge to handle report  production, which had the potential to improve the efficiency of the agency’s report  development process if used by all modal offices.
To enhance the efficiency of its report development process, we recommended that  NTSB identify better practices in the agency and apply them to all modes.  NTSB  should consider such things as using project managers or deputy investigators-in- charge in all modes, using incentives to encourage performance in report development,  and examining the layers of review to find ways to streamline the process, such as  eliminating some levels of review and using concurrent reviews as appropriate.
Our Assessment of NTSB’s Progress  NTSB has made significant progress in implementing this recommendation.  NTSB  examined and made several improvements to its report development process.  For  example, NTSB directed its office of safety recommendations and advocacy to provide  comments on draft reports at the same time as other offices, instead of beforehand.   NTSB estimates that this has reduced the time it takes to develop a report by 2 weeks.   NTSB officials also told us that the agency established and filled a permanent position  with a primary responsibility of quality assurance in the report development process.  In  addition, NTSB officials told us that the agency held a comprehensive training program  in February 2008 for investigators in charge to learn about and share best practices  across NTSB’s modal offices related to investigations and report development.  NTSB  also took or is taking the following steps to improve the efficiency with which Board  members are able to review and approve draft reports:  It is relying more on electronic rather than paper distribution of draft reports.
It reduced the time allotted to Board members to concur or non-concur with staff  responses to a Board member’s proposed revisions from up to 20 days to up to 10  days.
It is developing an information system to manage the process, which it aims to fully  implement by the end of fiscal year 2008.
Aside from its highway office which was already doing so, NTSB’s modal offices  decided not to use project managers or deputy investigators-in-charge to lead report  development because the offices did not believe that doing so would appropriately  address their report development issues; NTSB did not provide any further explanation  of the basis for this decision.  NTSB officials told us that its office of marine safety has  improved the efficiency and effectiveness of its report development process by shifting  responsibility for writing reports from three writer/editors to investigators-in-charge; the  office’s one remaining writer/editor now focuses on editing.  Finally, in December 2007,  NTSB’s office of railroad, pipeline, and hazardous materials safety hired a deputy chief  in the railroad division who will be responsible for streamlining the division’s report  development process.
In 2006, we found that in addition to its accident investigations, NTSB conducts studies  on issues that may be relevant to more than one accident.  These safety studies, which  usually result in recommendations, are intended to improve transportation safety by  effecting changes to policies, programs, and activities of agencies that regulate  transportation safety.  From 2000 to 2005, NTSB completed only four safety studies;  NTSB officials told us that the number of safety studies it conducts is resource-driven.   Industry stakeholders stated they would like NTSB to conduct more safety studies  because the studies address NTSB’s mission in a proactive way, allowing for trend  analysis and preventative actions.  NTSB officials recognized the importance of safety  studies, and they said that they would like to find ways to reduce the time and  resources required to complete the studies.  We concluded that NTSB’s limited use of  safety studies to proactively examine and highlight safety issues may limit the  effectiveness of its efforts to improve transportation safety.
To be more proactive in identifying and correcting safety problems before accidents  occur, we recommended that NTSB increase its utilization of safety studies.
Our Assessment of NTSB’s Progress  NTSB has made limited progress in implementing this recommendation.  NTSB has not  completed any safety studies since we made our recommendation and has only one  study in progress.  Although it has established a goal of developing and submitting to  NTSB’s Board for approval two safety study proposals per year, it does not have a goal  related to completing safety studies.  NTSB officials told us that the agency still does  not have enough staff to increase its output of safety studies on its own.  NTSB told us  that it has therefore begun to place more emphasis on a number of alternative products  to safety studies which address important safety issues but are not as resource  intensive.  In addition, NTSB is examining the potential of using contractors to perform  certain aspects of safety studies, such as data collection, and conducting some studies  in collaboration with other entities, such as the National Aeronautics and Space  Administration, the Federal Aviation Administration, a national laboratory, and foreign  accident investigation organizations.
section_title: Recommendations Related to Training Center Utilization
paragraphs: In 2006, we found that the training center was underutilized, with less than 10 percent  of the available classroom capacity being used during fiscal years 2005 and 2006.  This  contributed to the training center not being cost-effective, as the combination of the  training center’s revenues and external training costs avoided by NTSB staff’s use of  the facility did not cover the center’s costs.
We recommended that NTSB maximize the delivery of core investigator curriculum at  its training center.
Our Assessment of NTSB’s Progress  NTSB has made significant progress in implementing this recommendation by  scheduling 14 core investigator courses at its training center in fiscal year 2008.  In  addition, NTSB started a new workforce development curriculum intended to address  competencies not directly related to investigate activity, such as information security  and written communications.  NTSB officials told us that since it began this curriculum,  the frequency and attendance of classes has increased significantly, but we could not  verify this statement.
In 2006, we found that NTSB’s training center was not cost-effective, as the combination of  the training center’s revenues and external training costs avoided by NTSB staff’s use of the  facility did not cover the center’s costs.  As a result, those portions of the training center’s  costs that were not covered by the revenues from tuition and other sources—approximately  $6.3 million in fiscal year 2004 and $3.9 million in fiscal year 2005—were offset by general  appropriations to the agency.  While NTSB was generating revenues from other sources,  such as renting training center space for conferences and securing contracts that allowed  federal agencies to use training center space for continuity of operations in emergency  situations, the training center was underutilized, with less than 10 percent of the available  classroom capacity being used during fiscal years 2005 and 2006.  NTSB lacked a  comprehensive strategy for addressing this issue.
We recommended that NTSB develop a business plan and a marketing plan to  increase utilization of the training center or vacate its training center.  NTSB should  determine the costs and feasibility of alternative actions such as adding more courses  for NTSB staff, moving headquarters staff to the center, subleasing space to other  entities, or buying out the lease.
Our Assessment of NTSB’s Progress  NTSB has made significant progress in implementing this recommendation.  For  example, according to NTSB, it assessed the advantages and disadvantages of moving  headquarters staff and functions to the training center but determined it was not cost  effective.  NTSB also told us that it determined that buying out the training center lease  was not an available option.  NTSB completed a draft business plan in March 2007 and  a revised business plan in March 2008.  We reviewed the 2007 draft plan and  concluded that the overall strategy presented in the business plan to hire a vendor to  manage and operate the training center was reasonable, but the plan provided too little  rationale for its marketing and financial assumptions for us to assess the validity of this  strategy.  In July 2007, NTSB abandoned the strategy laid out in its business plan  because it could not find a suitable vendor.
While certain aspects of the revised business plan have been improved over the previous  plan, overall, the revised plan lacks key financial and marketing information that is essential to  a business plan.  For example, NTSB’s revised business plan does not contain historical  financial information or forecast financial information beyond fiscal year 2008 Further, the  plan does not describe assumptions included in the plan, such as the inclusion of imputed fees  for NTSB students in NTSB’s tuition revenues.  In addition, although the revised business plan  contains some goals, such as subleasing space to other federal entities and obtaining an  additional continuity of operations agreement, the plan does not contain strategies for  achieving these goals Further, while NTSB’s revised business plan indicates that the training  center is cost-effective if cost savings—such as avoided costs of renting outside space for  one  regional office and storage of the reconstructed wreckage of TWA flight 800—are accounted  for.  However, the plan does not provide enough information to support this conclusion.  While  we believe that NTSB is justified in offsetting expenses that the agency would incur in the  absence of the training center, the plan does not explain how NTSB estimated the values of  these offsets.  The plan does not include a rationale for assuming that NTSB would have to  maintain all 30,000 square feet of warehouse space in the absence of the training center, or  that space for both its regional aviation investigation office and the warehouse would cost  NTSB $35 per square foot if rented elsewhere.  In addition, it is not clear why certain items,  such as the warehouse space rental, is included as an offset, while other items, such as  savings for necessary accident investigation and family assistance training space needs, are  not included as an offset.  Finally, the plan lacks discussion of cost-saving alternatives, such as  using space already available at NTSB headquarters for certain offset activities, such as select  training courses.  When asked about these shortfalls in the business plan, agency officials  indicated that there was no flexibility in changing the configuration of the warehouse space,  requiring the warehouse space to be considered an offset. In contrast, office and training  space is included in the financial analysis due to its flexibility for expanded utilization.  The  agency did not comment on our other comments about the business plan.
NTSB has taken steps to increase utilization of the training center and to decrease the  center’s overall deficit, including the following:    NTSB subleased all available office space at its training center to the Federal Air  Marshal program at an annual amount of $479,000.
NTSB increased utilization of the training center’s classroom space and the  associated revenues from course fees and renting classroom and conference space.   From fiscal year 2006 to fiscal year 2007, NTSB increased utilization of classroom  space from 10 to 13 percent, and increased revenues by over $160,000.  NTSB  officials expressed concerns with our calculation of utilization rates because they  assumed that holiday weeks and other scheduling difficulties were not considered in  the calculation.  However, our analysis excluded holidays and Christmas week from  the calculation.
NTSB is finalizing a sublease agreement with the Department of Homeland Security  (DHS), which is expected to rent approximately one-third of the classroom space  beginning July 1, 2008.  We estimate that this would help increase utilization of  classroom space in fiscal year 2008 to 24 percent.
NTSB is undertaking efforts to increase utilization of the training center’s large area  that houses wreckage used for instructional purposes, including the reconstructed  wreckage of TWA flight 800, by seeking to acquire additional wreckage for  instructional purposes.
NTSB considered moving certain staff from headquarters to the training center, but  halted these considerations upon subleasing all of the training center’s available  office space.
NTSB decreased personnel expenses related to the training center, from about  $980,000 in fiscal year 2005 to $470,000 in fiscal year 2007 by reducing the center’s  full-time equivalents from 8.5 to 3 over the same period.
As a result of these efforts, from fiscal year 2005 to 2007, training center revenues  increased 29 percent while the center’s overall deficit decreased by 41 percent.  (Table  2 shows direct expenses and revenues for the training center in fiscal years 2004  through 2007.)   In fiscal year 2007, training center revenues nearly covered the  center’s operating expenses, not including lease costs.  However, the salaries and  other personnel-related expenses associated with NTSB investigators and managers  teaching at the training center, which would be appropriate to include in training center  costs, are not included.  NTSB officials told us that they believe the investigators and  managers teaching at the training center would be teaching at another location even if  the training center did not exist.  In 2006, we recommended that NTSB develop a full  cost accounting system that would allow them to calculate these expenses.  ($4,215)
Deficit when space  rental expense is  excluded  However, even at the 24-percent utilization rate for fiscal year 2008 that we estimate  would result from the DHS sublease, the training center classroom space would still be  underutilized.  If NTSB does not finalize this agreement, we estimate that only 15  percent of classroom space would be utilized during the fiscal year.  While we do not  expect any classroom space ever to be 100 percent utilized, we believe a 60 percent  utilization rate for training center classrooms would be reasonable, based on our  knowledge of similar facilities.   Without a functional business plan, NTSB lacks a  comprehensive strategy to address these challenges.
section_title: Recommendations Related to Information Security
paragraphs: Compliance with the Federal Information Security  Management Act (FISMA)
What an Independent Auditor Found  In June 2007, NTSB reported that its information security program was a prior year  material weakness that had not yet been corrected.  An independent FISMA  evaluation completed in September 2007 assessed NTSB’s actions to address  recommendations in prior year FISMA reports. The independent auditors reported that  while NTSB continues to be in material non-compliance with FISMA, it had taken  substantive corrective actions to address the material information security weaknesses  identified in prior FISMA reports issued by the Department of Transportation, Office of  Inspector General.  Overall, the independent auditor reported that the corrective actions  it observed, those underway or planned, if implemented timely and effectively, would  further strengthen NTSB's information security program.
The assessment completed in September 2007 found that NTSB met two requirements  of FISMA: 1) having in place policies and procedures to reduce risks to an acceptable  level and 2) ensuring that the agency has adequately trained its personnel in IT security  practices. However, NTSB partially met or did not meet FISMA and NIST requirements  in the following six areas: 1) providing periodic assessments of risk, 2) documenting  policies and procedures based on risk assessments, 3) developing and maintaining an  IT security program, 4) periodically testing security controls, 5) carrying out remedial  actions, and 6) having in place plans and procedures for continuity of operations.
What an Independent Auditor Recommended   Assure that the Chief Information Officer monitors all key corrective actions and  provides necessary funding and human resources to accomplish these actions so that  no further delays occur.
Our Assessment of NTSB’s Progress   The agency has made progress in implementing this recommendation. For example,  the Chief Information Officer has documented prior recommendations and newly  identified vulnerabilities in a plan of action and milestones and is monitoring corrective  actions to implement the recommendations and mitigate the vulnerabilities.  Nevertheless, NTSB needs to take further actions to meet FISMA, OMB, and NIST  guidance in the following four areas to help ensure an effective information security  program:     Risk assessments: Agencies are required to periodically assess the harm that could  result if their information and information systems suffered unauthorized access,  use, disclosure, disruption, modification, or destruction. NTSB has completed a risk  assessment of its general support system in February 2008.  The general support  system is an interconnected set of information resources, and it supports the  agency’s two major applications.  In addition, a contract has been awarded to  complete the risk assessments for the two major applications—the Accident  Investigation System and the Lab Environment System, both of which the agency  plans to complete by the end of September 2008. Until it assesses the risks  associated with these two applications, NTSB cannot determine that the controls it  has implemented for these two applications cost-effectively reduce risk to an  acceptable level.
Information security planning: To ensure effective security protection of information  resources, agencies must develop plans describing how they will provide security for  their systems, networks, and facilities. According to NIST, the security plan is to  provide, among other things, an overview of the security requirements of the system  and describe the controls that are in place or planned for meeting those  requirements. NTSB has completed the security plan for the general support system,  but development of security plans for its two major applications are not scheduled to  be developed until after April 2008. Until these plans are completed, NTSB will not  have in place a documented, structured process for adequate, cost-effective security  protection for these systems.
Periodic testing: Information security policies, procedures, practices, and controls  should be tested periodically to ensure their effectiveness. These tests and  evaluations should be conducted at least annually and include testing of the  management, operational, and technical controls of every system identified in the  systems inventory. In 2007, NTSB hired a contractor to perform a security test and  evaluation of its general support system. The contractor identified 113 information  security vulnerabilities, which collectively increased the risk of unauthorized  disclosure and modification of agency information.  NTSB has documented these  vulnerabilities in a plan of action and milestones. According to NTSB officials, they  have resolved many of the vulnerabilities, and are currently addressing the  remaining ones. Because NTSB has not finished addressing the vulnerabilities  identified in the security test and evaluation of its general support system, the  agency cannot ensure that the controls it has in place are commensurate with an  acceptable level of risk.
Continuity of operations plan: To ensure that, in the event of an emergency, interim  measures are available to restore critical systems, including arrangements for  alternative processing facilities in case the usual facilities are significantly damaged  or cannot be accessed, agencies must develop, document, and test contingency  plans and procedures. Testing the continuity plan is essential to determining whether  plans will function as intended in an emergency.  A contingency plan for the general  support system is under review by agency officials; and, according to these officials,  this contingency plan also supports its two major applications and is part of the  overall agency continuity of operations plan.  However, the plan has not yet been  approved or tested.  Without an approved plan that has been tested, NTSB has  limited assurance that it will be able to protect its information and information  systems and resume operations promptly when unexpected events or unplanned  interruptions occur.
What an Independent Auditor Found  The independent auditor identified several weaknesses in NTSB’s access controls.  Specifically, NTSB did not promptly remove system access privileges for 28 individuals  who had left the agency, was unable to provide documentation to support the original  access granted to employees in most instances, did not have a process to determine  the specific access authorities assigned to users for the general support system, had  not performed the required annual review of users’ access authorities for the general  support system, and did not implement a control to require the system to automatically  disable inactive accounts after a period of non-use. The independent auditor noted that  as a result of these weaknesses, the agency did not effectively implement the control  processes required in its policies and in NIST guidance.
What an Independent Auditor Recommended   The independent auditor made five recommendations to improve access controls at  NTSB.  1.  Take immediate action to remove the access authorities from all NTSB systems for  the 28 personnel who are no longer employed by or work for NTSB. Strengthen  procedures for removing users’ access for interns, contractors, and executive  training personnel who leave the agency.  2.  Maintain documentation supporting the initial access granted to a user.  3.  Develop a process to identify the specific systems, and within these systems, the  specific access authorities granted to each general support system user, to enable  user’s supervisors and system owners to properly analyze and complete the  annual recertification of users’ access authorities.  4.  Develop a more detailed operational procedure to guide system security officers  and system owners in the process of recertification of users. This should include:  (1) specific dates for the review, (2) requirements that documentation be retained  to show the recertification by the users’ supervisors, and (3) actions that system  security officers should take to remove or modify a user’s access to the system,  based on the review.
5.
Implement a control to automatically suspend an account after a period of nonuse,  as required.
Our Assessment of NTSB’s Progress   The NTSB has taken important steps to improve the controls that safeguard access to  its systems, but has not completed actions on all related recommendations.  Specifically, NTSB removed the accounts of 28 personnel who left the agency.  The  agency has procured and in some cases begun to implement automated software tools  to help implement recommendations related to granting, removing, and recertifying  users’ access permissions. However, agency officials expect that these tools will be  fully implemented in fiscal year 2008.  Furthermore, NTSB has not yet completed  identifying, for each system, the specific access permissions for each user and has not  yet completed implementing a control to automatically suspend an account after a  period of nonuse.
What an Independent Auditor Found  The independent auditor determined that NTSB did not comply with OMB requirements  for implementing provisions of the Privacy Act. OMB Memorandum M-03-22 requires  an agency to conduct privacy impact assessments for electronic information systems  and collections and to make these assessments available to the public. The review  found that NTSB had not issued sufficient written guidance in this area and had not  conducted a privacy impact assessment of its information systems. In addition, the  agency is required to report annually to OMB on compliance with sections 207 and 208  of the E-government Act. NTSB did not have available any guidance in this area, and  had not issued the required annual reports. Furthermore, NTSB did not conduct an  OMB-required review of its privacy policies and processes to ensure it has adequate  controls to prevent the intentional or negligent misuse of or unauthorized access to  personally identifiable information.
What an Independent Auditor Recommended  Assure actions are taken to meet the requirements of the Privacy Act and the  requirements contained in related OMB memoranda and to update the plan of action  and milestones to reflect the current status of NTSB actions in these areas.
Our Assessment of NTSB’s Progress   The agency has updated its plan of action and milestones to reflect the status of its  corrective actions to implement the requirements of the Privacy Act.  In addition,  agency officials have recently taken action to develop a formal privacy program;  however, work remains before it is fully compliant with the requirements of the Privacy  Act.  For example, NTSB completed privacy impact assessments on two of its public  facing applications and stated that it plans to complete assessments for other  applications and systems such as the accident investigation system.  Furthermore, the  agency is currently drafting a Systems of Records Notice, as required by OMB, which  will, among other things, inform the public of the existence of records containing  personal information and give individuals access to those records.  The agency expects  to have the Systems of Records Notice finalized in June 2008. Moreover, NTSB  recently awarded a contract to a vendor to develop specific training to its employees on  Privacy Act requirements. The agency expects this training to be available in June  2008.
This  is  a  work  of  the  U.S.  government  and  is  not  subject  to  copyright  protection  in  the  United  States.  The  published  product  may  be  reproduced  and  distributed  in  its  entirety  without  further  permission  from  GAO.  However,  because  this  work  may  contain  copyrighted  images  or  other  material,  permission  from  the  copyright  holder  may  be  necessary if you wish to reproduce this material separately.