NTSB was initially established within the newly formed Department of Transportation (DOT) in 1966, but was made independent from DOT in 1974. NTSB is charged by Congress with investigating every civil aviation accident in the United States and significant accidents in other modes of transportation—railroad, highway, marine and pipeline. NTSB determines the probable cause of the accidents and issues safety recommendations aimed at preventing future accidents. In addition, NTSB carries out special studies concerning transportation safety and coordinates the resources of the federal government and other organizations to provide assistance to victims and their family members impacted by major transportation disasters. Unlike regulatory transportation agencies, such as the Federal Aviation Administration, NTSB does not have the authority to promulgate regulations to promote safety, but instead makes recommendations in its accident reports and safety studies to agencies that have such regulatory authority. NTSB is comprised of a five-person board—a chairman, vice chairman, and three members—appointed by the President with the advice and consent of the Senate. The chairman is the NTSB’s chief executive and administrative officer. The agency is headquartered in Washington, D.C., and maintains 7 regional offices and a training center located in Ashburn, Virginia. In fiscal year 2013, the board was supported by a staff of about 400, which includes nearly 140 investigators assigned to its modal offices—aviation; highway; marine; and rail, pipeline, and hazardous materials—as well as 73 investigation-related employees, such as engineers and meteorologists. NTSB’s modal offices vary in size in relation to the number of investigators, with the Aviation Safety office being the largest. In addition, the Office of Research and Engineering provides technical, laboratory, analytical, and engineering support for the modal offices. Staff from this office interpret information from flight data recorders, create accident computer simulations, and publish general safety studies. This review focuses on the extent to which NTSB has achieved measurable improvements from actions the agency has taken in five management and operational areas based on prior GAO recommendations. Training Center utilization. Making efficient and effective use of resources provided by Congress is a key responsibility of federal agencies. NTSB’s Training Center, which opened in August 2003 in Ashburn, Virginia, consists of classrooms, offices, and laboratory facilities used for instructional purposes and active investigations. NTSB uses this center to train its own staff and others from the transportation community to improve accident investigation techniques. NTSB charges tuition for those outside NTSB to take its courses, and generates additional revenue from space rentals to other organizations for events such as conferences on a cost reimbursable basis. Although there is no statutory requirement that NTSB cover the cost of its Training Center through the revenues generated from the facility, a 2007 review we conducted found that NTSB was not capitalizing on its lease flexibility to generate additional revenues and classrooms were significantly underutilized. For example, we found that less than 10 percent of the available classroom capacity was used in fiscal years 2005 and 2006. Furthermore, NTSB was encouraged in a Senate report accompanying the 2006 appropriations bill for DOT to be more aggressive in imposing and collecting fees to cover the costs of the Training Center. Since then, NTSB leased a large portion of the Training Center’s non-classroom space to the Federal Air Marshall Service and provided short-term leases of classroom space to other organizations. In addition, NTSB increased the amount of training it delivered at the Training Center. Recommendation close-out process. Efficiently managing the recommendation tracking process is a key function, according to NTSB officials. The recommendation close-out process is managed by the Safety Recommendations and Quality Assurance Division, which has responsibility for tracking the status of its recommendations. When NTSB receives correspondence from an agency about an NTSB recommendation, this division ensures it is properly routed and reviewed and contacts the agency about whether the response is acceptable. If NTSB is delayed in communicating with agencies about whether NTSB considers actions to address recommendations acceptable, that agency could delay implementing a course of action pending approval. In fiscal year 2010, NTSB replaced a lengthy, paper-based process with an automated system— the Correspondence, Notation, and Safety Recommendation system (CNS)—intended to facilitate the recommendation close out process by electronically storing and automatically routing agencies’ proposals to the appropriate NTSB reviewers, allowing for concurrent reviews by multiple parties within NTSB, and more accurately tracking responses. It is important to note that an agency is not necessarily restricted from implementing action prior to formal NTSB approval of that action. Depending on the complexity of the issue, agencies may begin to address issues prior to NTSB’s providing formal approval. In other circumstances, NTSB addresses safety deficiencies immediately before the completion of an investigation. For example, during the course of the TWA flight 800 investigation, NTSB issued an urgent safety recommendation once it was determined that an explosion in a fuel tank caused the breakup of the aircraft. Communication. Useful management practices include seeking and monitoring employee attitudes, encouraging two-way communication between employees and management, and incorporating employee feedback into new policies and procedures. This type of communication and collaboration across offices at all levels can improve an agency’s ability to carry out its mission by providing opportunities to share best practices and helping to ensure that any needed input is provided in a timely manner. To this end, NTSB managers and board members began holding periodic meetings with staff, conducting outreach to regional offices, and surveying staff about the effectiveness of communication techniques. Diversity management. Implementing a diversity management strategy and a more diverse workforce helps foster a work environment that not only empowers and motivates people to contribute to mission but also provides accountability and fairness for all employees. Diversity management helps an organization create and maintain a positive work environment where the similarities and differences of individuals are valued, so that all can reach their potential and maximize their contributions to the organization’s strategic goals. NTSB has developed diversity training courses and held events to educate staff on diversity and inclusiveness issues, created career development and mentoring programs to create upward mobility, targeted its recruitment program to reach a more diverse pool of applicants, and surveyed staff to assess the effectiveness of its efforts. Financial management. Sound financial management is crucial for responsible stewardship of federal resources. Traditionally, government financial systems and government managers have focused on tracking how agencies spend their budgets but did not focus on assessing the costs of activities to achieve efficiencies. More recently however, some government agencies have adopted cost accounting systems that track the cost of providing a service—in NTSB’s case, an accident investigation. In 2006, GAO recommended that NTSB develop a cost accounting system to track the amount of time employees spend on each investigation and other activities. This approach allows management to link the cost of providing a service directly with the budget and allocate resources based on those costs. To determine the costs associated with conducting accident investigations, NTSB launched a time and attendance program tied to its cost accounting platform that allows the agency to collect and analyze labor and certain other costs associated with individual investigations. Investigators account for their time on investigations through the time and attendance system using specific codes that identify different investigations. Our analysis found varying degrees of improvement associated with NTSB’s actions in each of the management and operational areas we selected for review. Our analysis showed that NTSB improved the utilization of the Training Center, which allowed it to recover a larger portion of its operating costs. NTSB increased utilization of both classroom and non-classroom space at the Training Center since we conducted our work in 2006. NTSB subleased all available office space at the Training Center to the Federal Air Marshal Service in 2007, and utilization of non-classroom spaces has been at 95 percent since then. At the same time, NTSB increased utilization of the classroom space, increasing its own use of classrooms, subleasing approximately one-third of the classroom space to the Department of Homeland Security in 2008, and providing short term leases to other outside parties for classroom use. Subsequently, NTSB reported classroom utilization rose from less than 10 percent in 2005 to 18 percent in fiscal year 2007. By fiscal year 2009, it had increased to over 60 percent—the target we identified in our 2008 report as the appropriate minimum level. Classroom utilization has remained above 60 percent through fiscal year 2012. We also found that improved Training Center utilization generated additional revenue over time, which allowed NTSB to recover a larger portion of the facility’s operating costs. When the Training Center first opened in fiscal year 2004, NTSB recovered about 4 percent of its operating costs, resulting in a deficit of nearly $6.3 million. Portions of the Training Center’s costs that are not covered by revenues from tuition and other sources such as facility rentals are offset by general appropriations to the agency; therefore, generating additional revenue makes those appropriated funds available for other uses. In 2011, NTSB indicated that it was committed to improving cost recovery at the Training Center. That year the agency set a goal to recover costs of the Training Center within 10 percent of the previous fiscal year. For example, in fiscal year 2010, NTSB recovered $2 million in operating costs, making the fiscal year 2011 goal to recover at least $1.8 million of the Training Center’s costs. NTSB achieved its goal in fiscal years 2011 and 2012 by which time the agency was recovering about half of the Training Center’s operating costs, reducing the operating deficit at the Training Center to $2.1 million, one-third of what it was in 2004. (See fig. 1 for changes in the Training Center’s expenses and revenues.) The automation of NTSB’s recommendation follow-up process has reduced the amount of time it takes to formally respond to agencies about whether planned actions to implement an NTSB recommendation are acceptable. In fiscal year 2010, NTSB deployed the previously described CNS to manage the Board’s correspondence, including accident reports, safety studies, recommendation transmittals, and public notice responses. CNS allows for the relevant modal offices and the Research and Engineering Office to simultaneously review and assess planned actions to address NTSB recommendations. According to NTSB officials, prior to the implementation of CNS, the average time NTSB took to respond to an agency’s proposals to address an NTSB recommendation was 216 days. After CNS was implemented, that figure dropped to 115 days—a reduction of 47 percent. At the same time, the number of responses the agency put out each quarter also increased. (See fig. 2.) NTSB officials have indicated that they have an internal goal to further reduce the response time to 90 days on average. Our analysis of improvements in NTSB’s employee and management communication related to NTSB’s efforts indicated uneven results; specifically we observed improvements in some but not all measures. We reviewed NTSB employees’ responses to the three federal survey questions we determined related to employees’ perceptions about managers’ communication, as described below: Managers communicate the goals and priorities of the organization. NTSB respondents increased their positive responses to this question, from about 49 percent in 2004 to 57 percent in 2012. (See fig. 3.) We compared NTSB employees’ responses with employees in a group of small federal and independent agencies and found that NTSB employees’ satisfaction level increased while the proportion of employees from small agencies responding positively to this question during the same period was relatively unchanged from 57 percent in 2004 to 59 percent in 2012. How satisfied are you with the information you receive from management on what’s going on in your organization? Responses to this question indicated an increase in the level of satisfaction, from 44 percent in 2004 to 49 percent in 2012. NTSB employees’ responses were similar to the positive responses by employees from small agencies that showed an increase from 44 percent to 50 percent. Managers promote communication among different work units (for example, about projects, goals, needed resources). The proportion of respondents reporting positive responses on this survey question from 2004 to 2012 was relatively unchanged from 48 percent to 50 percent. Similarly, there was little change in the proportion of positive responses reported by federal employees from small agencies from 50 percent in 2004 and 49 percent in 2012. NTSB officials stated that their internal communication surveys, which the agency administered 2009 through 2011, provided information that helped them identify continuing barriers to employee and management communication. In 2012, NTSB developed an action plan in this area that included detailed activities, target dates, and regular status reports. Furthermore, because of lingering concerns, NTSB continues to monitor employees’ views about employee and management communication to address any remaining weaknesses. Our analysis of outcomes associated with NTSB’s efforts to improve its diversity management program indicated uneven results, with indications of improvements in some measures but not all. We reviewed NTSB employees’ responses to the three federal survey questions that we determined related to employees’ perceptions about managers’ diversity and inclusiveness efforts, as described below: My supervisor/team leader is committed to a workforce representative of all segments of society. The federal survey of NTSB employees indicated an increase in the positive responses, from 54 percent to about 71 percent. (See fig. 4.) We compared NTSB employees’ responses to those of employees in a group of small federal and independent agencies and found that NTSB employees were more positive in 2012 than the employees from small agencies, whose positive responses rose from 57 percent in 2004 to 69 percent in 2012. Policies and programs promote diversity in the workplace. Again, NTSB employees’ responses to this question indicated an increase from 2004 to 2012, from 55 percent to 73 percent. NTSB employees’ satisfaction level exceeded that of employees from small agencies, whose positive responses to this question stayed about the same, from 56 percent in 2004 to 57 percent in 2012. Managers/supervisors/team leaders work well with employees of different backgrounds. The proportion of NTSB employees reporting positive responses on this survey question from 2004 to 2012 declined from 66 percent to 60 percent. The decline in NTSB employees’ level of satisfaction was greater than that shown by employees from small agencies, whose positive responses also declined from 66 percent in 2004 to 63 percent in 2012. One of the potential outcomes of a robust diversity management program is an increase in the diversity of the workforce. Based on our analysis of NTSB’s workforce diversity data, we found that the proportion of white employees in NTSB’s workforce declined from 77 percent in 2008 (289 employees) to 73 percent (293 employees) in 2012. (See table 1.) NTSB’s total workforce increased 6 percent over the same period, from 378 in 2008 to 402 in fiscal year 2012. The proportion of women remained roughly the same at about 40 percent, as did the proportion of African American employees at 17 percent, and Hispanic employees at 2 percent of the total NTSB workforce. Conversely, NTSB increased the number of employees who are Native American although these employees represent only 2 percent of the overall workforce. We compared these figures to those representing comparative groups in the civilian labor force and found that NTSB’s labor force had a larger proportion of some minority groups (e.g., African American) and smaller proportion of other groups (e.g., Hispanic) than the civilian labor force. Roughly half of NTSB’s workforce performs investigations and investigation-related work—work directly related to NTSB’s core mission. In 2008 there were 190 investigators and related staff; in 2010, there were 198, and in 2012 there were 206. Based on our analysis, we found that the proportion of investigator and investigation-related staff that were white was about 90 percent over the period of 2008 to 2012 and the proportion of women was about 19 percent. We compared these figures to those representing comparative groups in the civilian labor force and found that NTSB’s investigator and investigation-related workforce had a smaller proportion of minority and other groups, including African American, Asian, Hispanic, and women than the civilian labor force. In addition, NTSB reported that from fiscal years 2008 to 2012, it had no minority group members among its 15 senior executives although the number of women increased from 3 to 4. Despite its efforts, NTSB has not been able to appreciably change its diversity profile for minority group members and women. However, as mentioned previously, NTSB has taken steps to implement initiatives as a result of its diversity management strategy, including its recently completed diversity and inclusiveness survey, which the agency plans to use to identify gaps in its diversity and inclusiveness efforts and to benchmark future progress. It is too soon to tell whether initiatives, such as its recruitment strategies, will lead to additional changes in its workforce diversity profile. We were unable to determine if NTSB’s cost accounting system had improved the agency’s ability to make operational decisions because it has not yet fully utilized the system for its intended purposes. For the implementation of a cost accounting system to be effective, it must be tailored to the needs of the organization, be a tool managers can use to make everyday decisions, and be based on sound data that captures time spent on all activities, such as investigations and training. In 2011, NTSB implemented a cost accounting system that includes a time and attendance program in response to a GAO recommendation. This program allows an investigator to assign his or her time to specific investigations through a series of codes, allowing NTSB to assess the cost of conducting investigations rather than simply tracking and managing a budget. NTSB officials stated that the time and attendance system has allowed it to obtain information about the cost of investigations more efficiently than its prior method. In a May 2011 advisory, NTSB management envisioned that the cost accounting system would enable NTSB to measure and compare performance with other organizations, and the data from the system would help the agency monitor and improve productivity and mission effectiveness by better utilizing personnel resources. However, officials provided no time frame for when the data might be used by management for making resource and operational decisions. NTSB officials stated that they are currently focused on ensuring the quality of the time and attendance data before developing goals, targets, and management tools or using such information to make resource or operational decisions. While ensuring the quality of data is a necessary step in fully implementing a cost accounting system, it has been over 2 years since NTSB first began collecting time and attendance data to establish the cost of conducting investigations. As a result, NTSB is not using the system’s full capabilities. Thus, NTSB has not yet fully achieved its vision of using the data to improve labor productivity and mission effectiveness. NTSB has implemented a cost accounting system, but effective utilization is required to achieve the long-term rewards of those efforts. Although NTSB has been collecting data from this system to account for costs of investigations, it has not yet developed a management strategy that would allow the agency to maximize the utility of the cost accounting system. This has prevented NTSB from using that information to make the decisions necessary to better manage its labor resources. Without fully utilizing the cost accounting system, NTSB will not achieve the intended benefits of improving labor productivity and mission effectiveness. NTSB needs to continue its improvement efforts in each of the five areas discussed in this report. Further, to improve financial management and provide information to managers for operational decisions, we recommend that the Chairman of the NTSB direct senior management to develop a strategy for maximizing the utility of NTSB’s cost accounting system. We provided a draft of this report to NTSB for its review and comment. The agency provided written comments (see app. II). NTSB agreed with our recommendation and provided technical clarifications that we incorporated as appropriate. We are sending copies of this report to the appropriate congressional committees and to the Chairman of the National Transportation Safety Board. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-2834. Contact points for Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix III. Our objective in this review was to assess whether there have been management and operational improvements associated with the National Transportation Safety Board’s (NTSB) actions in areas where GAO had conducted previous work and made recommendations. In that previous work, GAO made 21 recommendations within 12 management and operational areas. Some areas have more than one recommendation. For example, the communication area has two related recommendations; the first called for NTSB to develop mechanisms to facilitate communication from staff to management and the other called for NTSB to report to Congress on the status of the GAO recommendations. To determine whether there were any associated improvements based on NTSB’s actions, we first identified a subset of the 12 original management and operational areas that were included in NTSB’s 2013-2016 Strategic Plan. Being highlighted in NTSB’s current strategic plan indicates that these areas were relevant and important areas for the agency. We identified 7 of the 12 original management and operational areas in NTSB’s current strategic plan. These 7 areas contained 13 of the 21 original recommendations. Many of the 13 recommendations within these areas required only a single action by NTSB to implement (e.g. develop a strategic plan that follows performance based practices, articulate risk- based criteria for selecting which accidents to investigate, or obtain authority to use appropriations to make lease payments in order to correct its violation of the Anti-Deficiency Act) while other of those recommendations included strategies involving continual action and monitoring overtime in order to achieve desired improvements (e.g., increased utilization of the training center and improve the recommendation close out process). It is these latter types of recommendations that we focused on in this review, and based on these criteria, we identified 5 for review: increase the utilization of the Training Center, improve the process for changing the status of recommendations through computerization and concurrent review, develop mechanisms to facilitate communication from staff to develop strategies for diversity management as part of the human develop a full cost accounting system to track time employees spend on each investigation in training. We identified a sixth recommendation—maximize the delivery of core curriculum for each mode at the Training Center—based on our selection criteria. However, we did not include this recommendation in our review because NTSB lacked information about employee training and we could not identify an outcome measure without such data. To identify the outcome measures to assess changes associated with NTSB’s actions, we used information from prior GAO reports, information from NTSB including strategic plans, workforce reports, and financial reports as well as information gathered from interviews with NTSB officials. The measures we identified for each recommendation are: (1) Training Center utilization—utilization of classroom and non-classroom space and operating deficit; (2) recommendation close-out process— average time to respond to agency proposals; (3) employee and management communication—employee responses to Office of Personnel Management’s (OPM) federal employee surveys; (4) diversity management—employee responses to OPM’s federal employee surveys and NTSB employment levels of women and members of racial and ethnic groups; and (5) financial management—cost accounting reports used to measure performance. To measure whether there was improvement in the outcomes and results associated with NTSB’s actions, we compared current conditions with the initial conditions at the time we performed our previous work or earlier in some instances in order to establish a baseline before actions occurred. We used NTSB’s financial and program data, employee survey data from OPM, and workforce data from NTSB and the Bureau of Labor and Statistics (BLS). To ensure the data used were of sufficient reliability for our analysis, we examined program reporting procedures and quality assurance controls, and discussed various data elements with knowledgeable agency officials. We also spoke with NTSB officials who were knowledgeable about operations and management in these five selected areas. In addition to the contact named above, H. Brandon Haller (Assistant Director), Christopher Jones, Gail Marnik, Josh Ormond, Amy Rosewarne, and Jack Warner made key contributions to this report. National Transportation Safety Board: Implementation of GAO Recommendations. GAO-12-306R. Washington, D.C.: January 6, 2012 National Transportation Safety Board: Issues related to the 2010 Reauthorization. GAO-10-366T. Washington, D.C.: January 27, 2010 National Transportation Safety Board: Reauthorization Provides an Opportunity to Focus on Implementing Leading Management Practices and Addressing Human Capital and Training Center Issues. GAO-10-183T. Washington, D.C.: October 29, 2009 National Transportation Safety Board: Progress Made in Management Practices, Investigation Priorities, Training Center Use, and Information Security, But These Areas Continue to Need Improvement. GAO-08-652T. Washington, D.C.: April 23, 2008 National Transportation Safety Board: Observations on the Draft Business Plan for NTSB’s Training Center. GAO-07-886R. Washington, D.C.: June 14, 2007 National Transportation Safety Board: Progress Made, Yet Management Practices, Investigation Priorities, and Training Center Use Should Be Improved. GAO-07-118. Washington D.C.: November 22, 2006.

Summary: The NTSB plays a vital role in transportation safety. It is charged with investigating all civil aviation accidents in the United States and selected accidents in other transportation modes, determining the probable cause of these accidents, and making appropriate recommendations, as well as performing safety studies. In 2006, NTSB's reauthorization legislation mandated GAO to annually evaluate its programs. From 2006 to 2008, GAO made 21 recommendations to NTSB aimed at improving management and operations across several areas. Since that time, NTSB has taken action to address all 21 recommendations. Some of these were completed by requiring only a single action, whereas others required continuing effort to achieve operational improvement. For this review, GAO examined the extent to which desired outcomes are being achieved in five areas where continuing effort was necessary. GAO analyzed workforce, financial, and program data, and interviewed agency officials about actions NTSB has taken. GAOs analysis found varying degrees of improvement associated with the National Transportation Safety Boards (NTSB) actions in areas selected for review.  Training Center utilization . NTSB increased utilization of its Training Centerboth non-classroom and classroom spacesince 2006. NTSB has also set and achieved its cost recovery goal at the Training Center in the last 2 fiscal years, allowing NTSB to recover half of its operating costs.  Recommendation close-out process . By automating the recommendation follow-up process, NTSB has reduced by about 3 months the amount of time it takes to respond to agencies on whether planned actions to implement NTSB recommendations are acceptable; this allows agencies to move forward with approved actions sooner than under NTSBs former paper-driven process.  Communication . NTSB employees responses on federal employee surveys from 2004 to 2012 indicated an increase from 49 to 57 percent in employees positive responses regarding managers communication about agency goals, and from 44 to 49 percent regarding the amount of information received. We compared NTSB employees responses to those of employees from a group of small agencies and found that NTSB employees satisfaction level was about the same or more positive depending on the question. NTSB officials continue to monitor employees views about communication to address any remaining concerns.  Diversity management . NTSB employees positive responses to the federal employee survey questions about managers commitment to diversity and NTSBs diversity policies and programs increased from about 54 percent to over 70 percent from 2004 to 2012. However, employees positive responses to the question about managers ability to work well with employees with different backgrounds declined 6 percentage points over the same period. In addition, the proportion of minority and women employees in NTSBs workforce, including in its investigator staff, showed little appreciable change over the period 2008 to 2012. NTSBs workforce had a smaller proportion of some minority groups than the civilian labor force. NTSB officials are using results from their recent diversity survey to identify gaps in their diversity management efforts and to benchmark future progress. It is too soon to tell whether NTSBs actions will lead to additional changes in its workforce diversity profile.  Financial management . To improve operational effectiveness, NTSB has implemented a cost accounting system that includes a time and attendance program to track staff hours and costs related to accident investigations. NTSB is currently focused on ensuring the quality of the time and attendance data, but has not yet developed a strategy to maximize the utility of its cost accounting system for making resource and operational decisions. Thus, NTSB has not yet fully achieved its vision of using the data to improve labor productivity and mission effectiveness. In each of the five areas NTSB needs to continue its improvement efforts. Further, GAO recommends that NTSB senior managers develop a strategy for maximizing the utility of NTSB's cost accounting system. GAO provided a draft of this report to officials at NTSB. NTSB officials concurred with the recommendation and provided technical comments, which GAO incorporated as appropriate.