prompt_front: You are given two reports by the government agency.



Question: Write a one-page summary of the report 1.

Report 1: Safety defect vehicle recalls (auto recalls) are initiated when a defect in a vehicle or vehicle equipment creates an unreasonable safety risk, as determined by NHTSA or a manufacturer  After a recall is initiated, manufacturers are required to provide written notification to vehicle owners via First-Class Mail within 60 days and remedy the defect  Franchised dealers—which sell or lease an auto manufacturer’s new vehicles—perform the recall remedy  Before manufacturers send recall notification letters to affected vehicle owners, NHTSA reviews draft letters and envelopes to ensure they include required information about the safety defect  Required information includes, among other things, a clear description of the safety defect, an evaluation of the risk to vehicle safety, and a statement that the manufacturer will remedy the defect without charge  See appendix II for an example of a notification letter  The number of vehicles affected by safety defect vehicle recalls has increased dramatically since 2011 (see fig  1)  The increase reflects, in part, several large-scale recalls  For example, in 2014, General Motors initiated a recall of over 8 million vehicles with faulty ignition switches  Similarly, according to NHTSA in 2014 and 2015, some passenger vehicle manufacturers began recalling Takata air bag inflators, recalls that have grown to include approximately 34 million vehicles and 19 auto manufacturers  For the Takata recall, NHTSA issued various orders and established a Coordinated Remedy Program under which the agency oversees the supply of remedy parts and risk-based prioritization of vehicles for repair, and manages related recalls with the assistance of an Independent Monitor  The Independent Monitor assesses compliance with the applicable orders issued by NHTSA and makes recommendations aimed at enhancing the remedy program  According to NHTSA’s Strategic Plan 2016–2020, this unprecedented recall activity encouraged the agency to improve its system for identifying and addressing defective vehicles  For example, the plan states that NHTSA’s “vision is to achieve a 100-percent completion rate for every recall by improving communication at every level, at every step of the way ” Thus, according to the plan, NHTSA and the auto industry have committed to identifying and implementing effective strategies to inform consumers of safety defects and envision that their coordination will bolster recall efforts to improve completion rates  NHTSA reported that annual completion rates for passenger vehicle recalls have remained relatively flat, ranging from 63 to 67 percent between calendar year 2011 and calendar year 2014  See appendix III for completion rates by vehicle component and vehicle type  In part, to improve communication and encourage consumers to complete repairs, NHTSA and manufacturers provide auto recall information to the public on their websites  For example, certain motor vehicle manufacturers are required to allow consumers to search a vehicle’s recall remedy status on the Internet using the vehicle identification number (VIN)  NHTSA also provides publicly available auto recall information on its website, including examples of recall notification letters  In December 2016, NHTSA began consolidating its websites into NHTSA gov to provide a single access point for its auto recall content  One of these websites, safercar gov, was once NHTSA’s primary method of communicating auto recall information to consumers; however, the agency is in the process of moving this information to NHTSA gov  NHTSA’s Strategic Plan 2016–2020 states that the agency wants NHTSA gov to be a comprehensive user-friendly platform that serves as the premier source of vehicle safety information by, for example, improving the website’s search capabilities  NHTSA also aims to encourage consumers to use its website’s auto recall information through its communications program  NHTSA’s Office of Communications and Consumer Information (OCCI) is the primary office responsible for implementing the agency’s public communication efforts  OCCI intends to increase public engagement with the agency’s information through its social media channels, such as Instagram, Twitter, and Facebook  The amount OCCI obligated to support the agency’s auto recall efforts has increased from nearly $ 5 million in fiscal year 2011 to about $2 5 million in fiscal year 2016  According to NHTSA officials, these obligations supported various efforts, including public awareness campaigns, an auto recall hotline, advertising agencies, exhibits at auto shows, and NHTSA’s mobile application  As part of our focus group discussion sessions, consumers selected safety risk and convenience as the two most influential factors they considered when using auto recall information to decide whether to complete repairs  All factors considered: During each session, we first asked consumers to describe all the factors they considered  Across the sessions, consumers shared a wide variety of factors including availability of a loaner vehicle, time to schedule and complete the repair, safety risk, and other factors  For example, some consumers had not yet repaired their vehicles because they were “just waiting” for parts to become available  Other consumers considered their previous customer service experiences at the franchised dealership or the distance they would need to travel to complete the repair  For example, one consumer at our rural focus group location told us it would take roughly 2 hours to reach the dealership’s repair shop  Most influential factors considered: After the discussion of all factors, we then asked each consumer to select the single most influential factor they considered  Consumers in the sessions overwhelmingly selected safety risk and convenience as the two most influential factors (see table 1)  More than half of consumers in our focus group discussion sessions selected safety risk as the most influential factor they considered when making repair decisions  They told us that their perception of the risk influenced whether or not they repaired their vehicle  For instance, some consumers stated that they completed repairs immediately, because the risks “sounded serious” or that they considered the defect a “safety concern ” Conversely, some consumers said they did not complete the repairs because the defect “didn’t sound very urgent ” While each recall notification letter is required to include an evaluation of the risk to vehicle safety reasonably related to the defect, consumers in our focus group sessions shared mixed opinions about the quality and clarity of safety risk information included in the notification letter they received  For example, some consumers told us the letter’s safety risk information seemed vague  For instance, one consumer told us the letter’s description of the safety defect did not clearly state the chances of an increased risk of injury and so he “had to figure out on his own ” In addition, some consumers commented that the safety risk information could be more prominent in the notification letter, that the letter could emphasize the severity of the risk, or that the letter could describe the risk in simpler language  However, other consumers stated the notification letter they received adequately described the recall’s safety risk  In June 2011, we recommended that NHTSA modify the requirements for defect notification letters to include additional information to obtain readers’ attention  In 2013, NHTSA responded to our recommendation by requiring manufacturers to include the statement “IMPORTANT SAFETY RECALL” at the top of auto recall notification letters  Focus Group Participant’s Comment “I don't want to be without a car for half the day or stay with my kids all day ” Consumers in our focus group discussion sessions selected convenience as the second most influential factor they considered in making repair decisions  While some consumers described the “hassle” of the repair and being “too busy” to schedule and fix the defect, other consumers told us they repaired their vehicles more easily because, for example, they could take advantage of previously scheduled service appointments to also repair the defect  Also, some consumers in our sessions stated that the letter or notification they received could better address the inconvenience of the recall, for example by including better estimates of how long repairs might take  In addition, some consumers recommended the letter include options for scheduling needed repairs  As we discuss later in the report, NHTSA officials told us they continue to work with auto manufacturers to identify ways to encourage consumers to complete needed repairs, while representatives from some manufacturers we met with described specific steps they have taken to address some of the inconveniences consumers may experience in completing repairs  For example, one manufacturer facilitated a pilot program for a third-party service provider in conjunction with dealers to repair vehicles at the owner’s home or place of work, while another manufacturer told us they work with individual dealers to hold events specifically for recall repairs when consumers can come in to have repairs performed after normal business hours  Industry stakeholders’ use of auto recall information varies because these stakeholders play different roles in the auto recall process  Auto manufacturers are primarily responsible for providing auto recall information to the public and others, including NHTSA and auto dealers  Franchised dealers are responsible for performing the recall remedy for manufacturers and therefore use manufacturer-provided information for that purpose  Specifically, all of the franchised dealers we interviewed told us they identify recalls on new vehicles in their inventory primarily by accessing auto recall information through internal manufacturer databases  These franchised dealers may also use information from third-party providers or publicly available auto recall information on NHTSA’s website to identify recalls affecting used vehicles  Independent dealers—which are not generally authorized by manufacturers to perform recall remedies—may use publicly available auto recall information to identify open recalls  Specifically, 2 of the 3 independent dealers we met with told us they use NHTSA’s VIN look-up tool to search for open recalls affecting vehicles in their inventory before selling them to consumers  However, these dealers told us that the current design of the tool takes too much time to use, because it requires users to search each VIN individually  For example, one dealer told us each search took about 15 seconds to perform, resulting in significant time and cost because the dealership has tens of thousands of vehicles in its inventory  These dealers told us being able to search multiple VINs in a single search (i e , VIN-batch search) could save them time or money  Representatives from the Alliance of Automobile Manufacturers stated they—in coordination with other industry stakeholders—are working with a third-party provider to develop a search tool that would address this concern by enabling VIN-batch searches for use by government agencies, such as state departments of motor vehicles, and commercial entities  The group anticipates the tool will be available in the first half of 2018  Although the vast majority of consumers who participated in our focus group discussion sessions reported a preference to receive auto recall notification by mail, most preferred to receive notifications by at least one additional electronic means such as e-mail, phone calls, and text messages  Eighty of the 94 consumers in our sessions reported a preference for receiving notification by mail, and all but 4 reported actually receiving mailed notification (see fig  2)  However, 69 of the 94 consumers in our sessions also reported a preference for receiving recall notification by electronic means, but only 7 reported actually receiving at least one type of electronic notification  This result suggests a gap between industry recall notification practices and notification preferences for most consumers in our focus groups, especially for younger consumers who were more likely to report a preference for notification by electronic means  For complete results of the questionnaire we administered to consumers for the discussion session, see appendix IV  As we discuss later in this report, in September 2016, NHTSA issued a Notice of Proposed Rulemaking (NPRM) that proposes to require auto manufacturers to notify consumers about auto recalls by electronic means in addition to First-Class Mail  NHTSA officials told us the agency is working with the administration on NHTSA’s regulatory portfolio and priorities, including this rulemaking  Some manufacturers told us they use additional methods to reach consumers, including notifying consumers by electronic means and translating recall information into Spanish  For example, representatives from one manufacturer told us they always notify consumers by e-mail before sending out the required First-Class Mail letter notification  These representatives told us using multiple recall notification means resulted in higher recall completion rates  In addition, eight of the remaining nine manufacturers told us they use supplemental electronic means notification on a case-by-case basis—generally using additional means to improve recall completion rates—while four manufacturers stated they consider safety risk severity when deciding when or how to use additional notification means for individual recalls  Also, representatives from 3 of the 10 manufacturers we spoke with told us they translate the entire mailed notification letter into Spanish  In late 2016, NHTSA launched its redesigned NHTSA gov website, including the auto recall areas consumers assessed during our testing sessions  According to responses to a questionnaire we administered during our testing sessions, 78 of the 94 consumers found the auto recall areas of NHTSA gov either “somewhat” or “very easy” to use (see fig  3)  See appendix V for complete participant responses to the questionnaire we administered to each consumer  To inform the development of the redesigned website, NHTSA worked with a contractor to conduct a usability study in 2015 to evaluate users’ reactions to the agency’s websites, including NHTSA gov  According to agency officials, NHTSA implemented several changes based on the findings from the usability study, including: the creation of a dedicated “recalls” area of NHTSA gov, and the ability for users to access the VIN look-up tool in three different ways—on the homepage, in the “recalls” area, and through direct links either in a NHTSA e-mail for subscribers or from an external website  In addition, NHTSA officials told us that Department of Transportation (DOT) and NHTSA staff meet as needed to discuss the website and consider improvements  For example, the officials said they monitor user searches for the relevance and accuracy of results and adjust the search software to better assist users in finding auto recall information  Officials also told us the agency collects a variety of other information about how visitors use NHTSA gov, including how visitors access the website, and makes adjustments accordingly  For instance, NHTSA incorporated responsive web design as part of the agency’s ongoing consolidation effort—meaning the site is optimized for viewing on desktop, tablet, and mobile devices  In addition to monitoring searches and how visitors access NHTSA gov, NHTSA officials told us they collect and consider online survey data to make website improvements and use web-analytic software to monitor, for example, where visitors choose to exit the website  Officials stated that such monitoring activities have allowed NHTSA to identify and correct problems with NHTSA gov  We did not directly evaluate the accessibility of the auto recall areas of NHTSA gov to ensure the ability of people with physical or mental disabilities to use the website  However, NHTSA officials provided us with an overview of several steps the agency takes to ensure NHTSA gov complies with website accessibility requirements  For example, according to officials, NHTSA subscribes to a service that provides monthly accessibility scans of the agency’s websites  While most consumers in our usability testing sessions generally found the auto recall areas of NHTSA’s website easy to use, some consumers experienced difficulties completing tasks we asked them to perform (see table 2)  Specifically, during each testing session we asked participants to perform tasks using the primary means NHTSA gov provides for consumers to access information about auto recalls affecting their vehicles: searching for auto recalls using their vehicle’s VIN; searching for auto recalls using their vehicle’s year, make, and model; and locating NHTSA’s auto recall notification e-mail subscription service  In addition, an evaluation we requested to corroborate the results of our consumer usability testing, identified similar issues  As discussed below, consumers experienced these difficulties because the auto recall areas of NHTSA gov do not always reflect federal and industry key website usability practices, which describe standards and guidelines for making websites easy to use  Following such practices can assist agencies in creating quality websites while providing the flexibility necessary to meet organizational needs  Website usability is particularly important for agencies, such as NHTSA, that are responsible for conveying safety information to the public  Federal standards for internal control state that agencies should communicate quality information externally and select appropriate methods for communicating with the public  While most consumers in our usability testing sessions found searching for recalls by VIN somewhat or very easy, some consumers found the search results did not provide the information they were seeking  When we asked consumers to perform VIN searches, they generally found the VIN look-up tool easy to use—88 of 94 consumers found searching with a VIN either somewhat or very easy  But some consumers experienced difficulties performing this task  Specifically, some consumers who had had their vehicles repaired expected to find the completed recall on the search results page  However, they were confused because the page is designed to display only open (i e , unrepaired) recalls, not completed (i e , repaired) recalls—leading these consumers to question the accuracy of the results  In addition, the evaluation conducted by website usability professionals found that, when an error occurred during a VIN search, the error message was too difficult to locate on the search results page  The evaluation recommended the error message have greater weight and more prominence on the page  Federal key website usability practices state that agencies should ensure that results of user searches provide the precise information being sought, and in a format matching users’ expectations  When users are confused by search results, or do not immediately find what they are searching for, they become frustrated and may abandon the search or the website entirely  Since NHTSA launched the VIN look-up tool in August 2014, the number of VIN searches performed has increased (see fig  4)  According to NHTSA officials, major increases occurred in mid-2015— when the Takata air bag inflator recalls were announced—and in early 2017, when NHTSA made the VIN look-up tool search function available on NHTSA gov and displayed it prominently on the website  Ensuring the usability of NHTSA’s VIN look-up tool is particularly important because it is the only way on NHTSA gov for a consumer to determine whether their specific vehicle has an open safety recall  Recall Search Using Vehicle Year, Make, and Model Some consumers’ vehicle year, make, and model searches were hampered by the information required to conduct an accurate search, as the content on the website is not always in plain language  We asked consumers to perform a recall search using their vehicles’ year, make, and model, and 78 of 94 consumers found the task to be either somewhat or very easy  However, some consumers found that they did not know enough information about their specific vehicles to feel confident that they were searching for the correct vehicle  For example, a year, make, and model search for a 2009 Toyota Tacoma may ask the consumer to choose among vehicle options, including “2009 TOYOTA TACOMA REGULAR CAB W/SAB RWD/AWD ” Acronyms such as “W/SAB”— which stands for “with side air bags”—may be confusing to consumers  Federal key website usability practices state that federal agencies should write website content using plain language, so website visitors can easily find and use what they need  Focus Group Participant’s Comment “I think [the Recall Notification E-mail System Sign-Up is] poorly placed  I had to scroll to find it  I had to search for it  You want at the top  ” Recall Notification E-mail System Sign-Up Some consumers suggested improvements to make the Recall Notification E-mail System Sign-Up easier to locate on the homepage  NHTSA first made its Recall Notification E-mail System Sign-Up available in March 2008  Of the 94 consumers in our testing sessions, 66 found it either “somewhat” or “very easy” to find the Recall Notification E-mail System Sign-Up—making this the least easy of the three tasks we asked consumers to perform  Specifically, several consumers said the Recall Notification E-mail System Sign-Up should include a clearer description, be easier to find, and be located at the top of the homepage (see fig  5)  These improvements are particularly important because some consumers in our focus group sessions told us that the ability to sign up for auto recall e-mail notifications was the most useful part of the auto recall areas of NHTSA gov  The website evaluation conducted by website usability professionals recommended that NHTSA streamline its homepage with more of a focus on primary website tasks  The evaluation also found that users must move through too many pages to sign up for recall e-mails  Federal key website design and usability practices state that agencies should put important items closer to the top of the page, where users can better locate the information  Key practices also state that agencies should design their websites so users can successfully complete the most common tasks in the fewest number of steps  The website usability difficulties that consumers in our focus groups experienced may be due to the fact that NHTSA has not studied the website’s usability since the agency redesigned NHTSA gov in late 2016 and, therefore, may have been unaware of these difficulties prior to our review  NHTSA plans to conduct a website usability study with consumers after the consolidation effort, discussed above, is complete  However, NHTSA could not provide a general time frame for conducting the study because it has not yet determined when the consolidation effort will be complete  We have previously reported that it is essential for organizations to effectively guide their information technology efforts by establishing timelines to complete them, among other strategic planning best practices  Without establishing a completion date for its website consolidation effort, the website usability difficulties we identified may persist and limit the effectiveness of NHTSA’s primary means of providing consumers with safety recall information about their vehicles on NHTSA gov  In January 2016, NHTSA launched a national advertising campaign encouraging consumers to check for open recalls using the agency’s VIN and year, make, and model look-up tools  Through March 2017, NHTSA spent about $1 million on its Safe Cars Save Lives campaign, which sponsors advertisements on Google, Facebook, and other media platforms  For example, Google might place NHTSA’s advertisement above other search results, when a consumer typed certain keywords— such as “recall,” “airbag recalls,” or “safercar gov”—into the search  NHTSA evaluated the campaign’s effectiveness by monitoring website traffic performance reports to determine how frequently consumers clicked on NHTSA-sponsored advertisements and ultimately searched for open recalls using the agency’s look-up tools  NHTSA also compared results across media platforms and adjusted the campaign’s strategy to improve performance  For example, NHTSA optimized advertisements on mobile devices, since mobile-device users performed more recall searches than other users  According to NHTSA data, the awareness campaign resulted in consumers performing 1 1 million recall searches through March 2017—a cost of about $0 90 per search  Agency data indicate that this cost generally decreased as NHTSA improved the campaign’s strategy  Agency officials told us NHTSA plans to spend another approximately $1 8 million on Safe Cars Save Lives from September 2017 through September 2018 due to the campaign’s effectiveness in raising the public’s awareness about auto recalls  NHTSA began implementing a mandated 2-year pilot grant program intended to evaluate the feasibility and effectiveness of informing consumers about open auto recalls during state vehicle registration  In September 2016, NHTSA solicited applications to participate in the program, wherein selected states would inform consumers—at no charge—about open recalls using all means that permit consumers to register vehicles within the state (e g , in person, Internet, and mail)  According to NHTSA, only one state applied for the grant  In September 2017, NHTSA awarded the sole applicant $223,000  Under the program, the grantee needs to collect and report program performance data, including the extent to which open recalls have been identified and repaired  In addition, the grantee must report whether certain notification means were more effective than others and what could be done to improve the program  Upon completion of the pilot program, NHTSA is required to evaluate the extent to which open recalls identified have been remedied  Auto manufacturers we met with were generally supportive of the program  Specifically, representatives from 9 of the 10 manufacturers told us that notifying consumers about open recalls during vehicle registration can raise consumer awareness or improve recall completion rates  In September 2016, NHTSA issued a Notice of Proposed Rulemaking (NPRM), which proposes to require auto manufacturers to notify consumers about open recalls by electronic means—such as e-mails, phone calls, and text messages, in addition to First-Class Mail  As we described earlier, auto manufacturers are currently required to notify consumers about safety recalls affecting their vehicles via First-Class Mail  According to NHTSA, the NPRM aims to aid in efficiently and effectively improving recall completion rates, by proposing that manufacturers provide notification using electronic means in addition to First-Class Mail  Consumers in our focus groups as well as auto manufacturers and consumer associations we interviewed generally supported additional notification using electronic means  Consumers in our focus groups: As we discussed earlier, 69 of the consumers in our focus group discussion sessions reported they would prefer to receive additional notification by at least one type of electronic means  However, only 7 consumers actually received such notifications—suggesting a gap between industry notification practices and notification preferences for these consumers  Auto manufacturers: Representatives from 9 of 10 manufacturers we interviewed told us they generally support providing notification using electronic means  Although the NPRM proposes a broad definition of electronic means to give manufacturers flexibility to determine the most effective means, these representatives also shared implementation concerns  For example, representatives from 1 of the 9 manufacturers told us that—although the company collects e- mail addresses from some customers for other purposes—not all customers provide e-mail addresses, and those collected are not always accurate  As we discussed previously, most manufacturers we met with currently use supplemental electronic means notification on a case-by-case basis  Consumer associations: Similarly, both consumer associations we interviewed told us additional electronic notification can help reach consumers who do not complete repairs after receiving initial mailed notification  NHTSA’s proposal would maintain manufacturer reporting requirements, though it may result in additional reporting  This additional information could help the agency evaluate the effectiveness of various means of consumer notification  We previously found that NHTSA may be able to use manufacturers’ data to identify what factors make some recalls more or less successful than others  We recommended that NHTSA use the recall data it collects to analyze particular patterns or trends that may characterize successful recalls and determine whether these factors represent best practices  If the NPRM is finalized, manufacturers would provide NHTSA with representative copies of the newly required electronic notifications, in addition to mailed notifications, and would specify the electronic means used, such as e-mail or text message  According to NHTSA officials, this information could allow the agency to track and evaluate the effectiveness of various notification means used by manufacturers by, for example, comparing completion rates across means—a key step in identifying best practices that could encourage consumers to complete repairs  However, it is too early for NHTSA to conduct such an evaluation, because the agency has not issued a final rule  NHTSA officials told us the agency is working with the administration on NHTSA’s regulatory portfolio and priorities, including this rulemaking  NHTSA has also taken steps to collaborate with industry stakeholders and explore consumer education best practices  For example, in April 2015 NHTSA hosted a day-long workshop that brought together auto industry stakeholders to examine public education of the recall process  During the workshop, participants identified current barriers to the public’s awareness of auto recalls and discussed potential solutions to address them, such as using text messages and social media to communicate with younger consumers and using different delivery methods for recall notices  Similarly, in January 2016 NHTSA and 18 auto manufacturers adopted a set of Proactive Safety Principles to explore and employ new ways to increase safety recall participation rates  For example, NHTSA and auto manufacturers agreed to share industry best practices and policies based on lessons learned from ongoing safety recalls  The Independent Monitor of Takata in conjunction with NHTSA has also issued a set of coordinated communications recommendations based on consumer research, best practices observed during the Takata recall, and discussions with manufacturers  For example, the recommendations encourage manufacturers to: pursue a “multi-touch” communications strategy that employs non- traditional means, such as e-mail and text messages; convey risk in clear, accurate and urgent terms; and include a clear “call to action” designed to facilitate prompt and efficient scheduling of repairs  According to NHTSA officials, the agency relies on auto manufacturers to evaluate the effectiveness of these efforts  However, agency officials told us NHTSA reviews manufacturers communication plans as part of the Takata recall’s Coordinated Remedy Program and provides ongoing recommendations on manufacturers’ communication language, approach, and strategies  With the recent steep increase in safety defect vehicle recalls and continued low recall completion rates, it is vital for consumers to be able to easily access and use publicly available auto recall information  NHTSA has taken important steps to improve its website—which provides safety recall information to consumers—resulting in most consumers in our focus groups finding the website easy to use  However, the difficulties some experienced in attempting to complete essential auto recall tasks demonstrated that NHTSA gov does not always reflect key website usability practices for website design  Although NHTSA plans to conduct a website usability study with consumers after consolidating its websites, it has not determined a completion date for this effort—an essential step for organizations to effectively guide their information technology efforts  Without such a date, the website usability difficulties may persist and limit the effectiveness of NHTSA gov in providing consumers with recall information about their vehicles  By addressing these difficulties in the interim, NHTSA can better assure that consumers obtain this information, which can be vital to their safety  We are making the following two recommendations to NHTSA: The Administrator of NHTSA should determine a completion date for the agency’s website consolidation effort  (Recommendation 1) The Administrator of NHTSA should, while the agency continues its website consolidation effort, take interim steps to improve the usability of the auto recall areas of NHTSA gov by addressing the website usability difficulties we identified  (Recommendation 2) We provided a draft of this report to DOT for review and comment  In its written comments, reproduced in appendix VI, DOT stated that it concurred with our recommendations  The department also provided technical comments, which we incorporated as appropriate  We are sending copies of this report to relevant congressional committees, the Secretary of Transportation, and the Administrator of NHTSA  In addition, the report is available at no charge on the GAO website at http://www gao gov  If you or your staff have any questions about this report, please contact me at (202) 512-2834 or flemings@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report  GAO staff who made key contributions to this report are listed in appendix VII  This report examines the use of publicly available auto recall information for safety defects affecting passenger vehicles  The report addresses the following objectives: (1) How do consumers and industry stakeholders use publicly available auto recall information? (2) How easy or difficult to use do consumers find the auto recall areas of NHTSA gov? (3) What steps, if any, has the National Highway Traffic Safety Administration (NHTSA) taken to raise consumer awareness about auto recalls and how has NHTSA evaluated the effectiveness of these steps? We define publicly available auto recall information to include information on the auto recall areas of NHTSA gov, such as examples of notification letters that manufacturers mail to consumers  This report focuses on safety defect vehicle recalls affecting passenger vehicles that are initiated when a defect in a vehicle or vehicle equipment creates an unreasonable safety risk, as determined by NHTSA or a manufacturer  To determine how consumers use publicly available auto recall information, we conducted and analyzed transcripts and questionnaires from 12 consumer focus groups we conducted with used and new vehicle owners who had experienced an auto recall in the last 24 months  Each focus group was split into two sessions: (1) a discussion session to explore participants’ thoughts, experiences, and preferences about auto recall information and (2) a website usability testing session  Also, we administered a questionnaire as part of each of these sessions  For the discussion session, we asked consumers about the recall notification process and how they used the recall information, and for the website usability testing session, we asked consumers to fill in a questionnaire during the session itself as they assessed the usability of the auto recall areas of NHTSA’s website  We conducted the 12 focus groups at six locations across the country, with each group including 7 or 8 consumers for a total of 94 participants  Half of the focus groups were comprised of consumers who had completed the repair and the remaining half included consumers who had not completed the repair  We selected the six focus group locations to provide population and geographic dispersion  To ensure geographic dispersion, we selected at least one location in each U S  Census region (see table 3)  To ensure population dispersion, we selected Metropolitan Statistical Areas representing a range of population sizes based on 2015 U S  Census estimates  To ensure our selection included the perspectives of vehicle owners in geographically distant or isolated communities, we also selected a rural location, which we defined as a city or town that has a population of less than 50,000 inhabitants and is not an urbanized area contiguous and adjacent to a city or town that has a population of greater than 50,000 inhabitants  Using information provided by the participants, we selected focus group participants based on age, income, gender, education level, race, and ethnicity to ensure we collected a range of perspectives on auto recall information use  However, since we did not select a representative sample of participants, focus group results are not generalizable to all vehicle owners  During focus group discussion sessions, we asked participants to discuss factors they considered when deciding whether to repair their recalled vehicle and then to select the single most influential factor  Each of the 12 focus group sessions was audio recorded and transcriptions were created; transcripts served as the record for each group  We then evaluated those transcripts using systematic content analysis to identify the factors consumers considered when deciding whether to complete repairs and any suggested improvements to the auto recall communication process  The analysis was conducted in three steps  First, two analysts independently developed a code framework and then worked together to resolve any discrepancies  Second, each transcript was coded independently by analysts using the framework and any discrepancies were resolved by both analysts agreeing on the coding of the associated statement by a participant  Third, if needed, another analyst adjudicated any continued disagreement between coders  Because the transcripts did not include a unique identifier for each focus group participant, we conducted our analysis of focus group session discussions at the group level (i e , of the 12 focus groups we conducted)  We also administered and analyzed a questionnaire as part of each discussion session to quantify responses regarding consumers’ use of auto recall information, including how they received and preferred to receive auto recall notifications  Our analysis of the questionnaire responses was conducted at the individual consumer level (i e , of the 94 consumers who participated)  These focus group sessions were structured, guided by a moderator who used a standardized list of questions to encourage participants to share their thoughts, experiences, and preferences  We also conducted two pretest focus groups at our headquarters and made some revisions to the focus group guide prior to beginning the sessions with consumers  Methodologically, focus groups are not designed to demonstrate the extent of a problem or to generalize results to a larger population or provide statistically representative samples or reliable quantitative estimates  Instead, they are intended to generate in-depth information about the reasons for the focus group participants’ thoughts, experiences, and preferences on specific topics  The projectability of the information produced by our focus group sessions is limited  For example, the information includes only the responses from the vehicle owners from the 12 selected groups and their individual responses to questions we asked  The experiences and preferences expressed may not reflect other vehicle owners’ thoughts and preferences  In addition, while the composition of the groups was designed to ensure a range of age and education levels, among the other criteria mentioned previously, the groups were not constructed using a random sampling method  To determine how industry stakeholders use auto recall information, we interviewed selected auto manufacturers, selected franchised and independent auto dealerships, NHTSA program officials, and other industry stakeholders  Specifically, we interviewed representatives from the following 10 auto manufacturers, selected based on each manufacturer’s sales market share (small, medium, and large), place of ownership (foreign and domestic), and experience with auto recalls (lower to higher based on the average annual number of auto recall campaigns and average market share of each manufacturer from 2010 to 2014) to collect a range of perspectives on how manufacturers use auto recall information: Tesla Motors, Inc  To understand the perspective of auto dealers, we interviewed four franchised dealerships, one in each of the four U S  Census regions where we conducted focus groups with consumers  We also interviewed three independent auto dealerships in two U S  Census regions  The results of these interviews are not generalizable to all auto manufacturers and dealerships, but provide insights about how some industry stakeholders use auto recall information  We conducted interviews with NHTSA program officials to understand NHTSA’s role in the auto recall process  In addition, we interviewed other stakeholders, including the Independent Monitor of Takata, which assists NHTSA in overseeing the Takata recall, as well as officials from consumer associations and other industry groups (see table 4)  To evaluate how easy or difficult consumers find the auto recall areas of NHTSA gov to use, we reviewed various website usability resources to understand federal and industry key website usability practices for making websites easy to use, such as focusing on design and how easily users can find information  In addition, we reviewed federal standards for internal control related to communicating quality information externally  During our usability testing sessions, we asked consumers to attempt to complete auto recall tasks—the primary means NHTSA gov provides for consumers to access information about auto recalls affecting their vehicles—and discuss their experiences  We then compared consumers’ experiences with the usability of the website against these practices  To identify key website usability practices, we analyzed guidance documents from NHTSA and other federal agencies  For example, we analyzed the General Services Administration’s (GSA) and the Department of Health and Human Services’ Research-Based Web Design & Usability Guidelines, which includes quantified, peer-reviewed guidelines intended to help federal agencies improve the design and usability of their information-based websites  We also analyzed GSA’s Requirements for Federal Websites and Digital Services, and the U S  Digital Services Playbook to identify key practices for making websites easy to use  Identified key practices are: (1) design and content— focusing on the layout, headers, and design; (2) navigation—how easily users can find information; (3) clarity—the ability to read and digest content; (4) identity and purpose—whether the site clearly presents its purpose; and (5) accessibility—the ability of people with physical or mental disabilities to use the site  To analyze the results of focus group website testing sessions, we performed a systematic content analysis of the session transcripts using the same content analysis methods described above and an analysis of the questionnaire we administered to each participant during the website usability sessions  Specifically, we analyzed the transcripts from the website usability testing sessions to account for consumers’ experiences, including their initial impressions of the website and any suggested usability improvements  We also analyzed the results of the questionnaire that each participant completed where participants were asked to mark responses regarding their experience including an assessment of the usability of the auto recall areas of NHTSA gov  Our analysis of the results from the questionnaire responses was conducted at the individual consumer level (i e , of the 94 consumers who participated) while our analysis of focus group session discussions was conducted at the group level (i e , of the 12 focus groups we conducted)  To corroborate the results of usability testing sessions we conducted with the consumers in our focus groups, we requested that five website usability professionals from GSA’s Federal User Experience Community conduct an independent evaluation of the auto recall areas of NHTSA gov against federal and industry key website usability practices (described above)  The website usability professionals developed a website usability evaluation form, which they used to individually evaluate the auto recall areas of NHTSA’s website  The website usability professionals then met to form a consensus and provided us with one final group evaluation of the website usability of the auto recall areas of NHTSA gov  Also, although neither our usability testing nor the website usability evaluation conducted by website usability professionals directly addressed accessibility, we interviewed responsible agency officials about how the agency assesses the accessibility of NHTSA gov  We also requested and analyzed website data provided by NHTSA to understand how consumers access and use NHTSA gov  Requested data included the number of subscribers to NHTSA’s Recall Notification E-mail System Sign-up; the number of weekly vehicle identification number (VIN) searches performed on NHTSA gov from August 2014 through May 2017; and NHTSA gov usage data by device (i e , usage by mobile devices, tablets, and desktop computers)  We assessed the reliability of these data by reviewing any supporting documents provided by the agency and interviewing responsible NHTSA officials, and concluded the data were sufficiently reliable for our reporting purposes  While we did not independently review the usability of auto manufacturers’ auto recall websites, we requested and reviewed the results of any audits that NHTSA performed of these websites, including whether the websites met statutory and regulatory requirements for providing auto recall information to the public  We then corroborated any audit findings by reviewing the auto recall websites of the selected auto manufacturers that we interviewed  To determine any steps NHTSA has taken to raise consumer awareness about auto recalls and how NHTSA evaluates the effectiveness of any steps, we reviewed relevant statutes, regulations, and proposed rules, including the Fixing America’s Surface Transportation Act and a Notice of Proposed Rulemaking related to recall notification methods  We also reviewed agency and other documents that describe or evaluate NHTSA’s public awareness activities  For example, we analyzed NHTSA’s strategic planning documents—such as NHTSA’s Strategic Plan 2016–2020—to identify ongoing public awareness activities along with their related goals, objectives, or performance metrics  Similarly, we requested and analyzed any documents NHTSA uses to evaluate the effectiveness of its public awareness activities, including performance reports for NHTSA’s ongoing Safe Cars Save Lives campaign  To assess the reliability of data included in these performance reports— such as VIN searches performed—we reviewed agency documentation and interviewed agency officials about the reliability, accuracy, and completeness of the data and determined the data were sufficiently reliable for our reporting purposes  We reviewed performance management practices as provided in the Government Performance and Results Act of 1993 (GPRA), the GPRA Modernization Act of 2010, and standards for internal control in the federal government to identify any opportunities for improvement  We also performed a literature review to identify any related published articles and research studies  To understand how NHTSA implements and evaluates any public awareness activities, we also interviewed responsible agency officials from NHTSA’s Office of Communications and Consumer Information and other offices  In addition, we discussed NHTSA’s public awareness efforts during interviews with industry stakeholders, including selected auto manufacturers, selected franchised and independent auto dealerships, and other industry stakeholders  We analyzed the results of these interviews along with the focus group discussions we conducted with consumers (discussed above) to identify perspectives on the effectiveness of NHTSA’s public awareness steps  We conducted this performance audit from October 2016 to December 2017 in accordance with generally accepted government auditing standards  Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives  We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives  The National Highway Traffic Safety Administration (NHTSA) is required to conduct a biennial analysis of vehicle safety recall completion rates and submit the results of its analysis in a report to certain congressional committees  The report must include, among other things, the annual recall completion rate by vehicle type and vehicle component (such as brakes, fuel systems, and air bags) for each of the 5 years preceding the year the report is submitted  According to NHTSA’s May 2017 report, completion rates for all vehicles combined ranged between 63 percent and 67 percent between calendar year 2011 and calendar year 2014 (see table 5)  However, NHTSA reported wider variation when the recall completion rates are broken down by vehicle type  Similarly, the report found that completion rates for most component categories fall within a range of 60 percent to 75 percent (see table 6)  The annual completion rate is a volume-based, weighted metric, such that the more vehicles affected by the recall, the more weight or influence it has on the computed rate  Focus group participants responded to a questionnaire we administered to collect information on consumers’ auto recall notification preferences during our discussion sessions  Table 7 shows participants’ responses to the administered questionnaire, by age group  We present these responses by age group, because consumers’ notification preferences may vary according to their ages  Focus group participants responded to a questionnaire we administered to collect information on the usability of NHTSA gov during our usability testing sessions  Table 8 shows focus group participants’ responses to the administered questionnaire, by age group  We present these responses by age group, because consumers’ website usability needs or preferences may vary according to their ages  In addition to the individual named above, H  Brandon Haller (Assistant Director); Katherine Blair; Jason Blake; Melissa Bodeau; Alicia Cackley; William Colwell (Analyst in Charge); Lacey Coppage; Elizabeth Dretsch; Jaci Evans; Marcia Fernandez; Sarah Kaczmarek; Malika Rice; Todd Schartung; and Andrew Stavisky made key contributions to this report 


Report 2: Cybersecurity incidents continue to impact federal entities and the information they maintain  According to OMB’s 2018 annual FISMA report to Congress, agencies reported 35,277 information security incidents to DHS’s U S  Computer Emergency Readiness Team (US-CERT) in fiscal year 2017  As shown in figure 1, these incidents involved threat vectors, such as web-based attacks, phishing attacks, and the loss or theft of computer equipment, among others  These incidents and others like them can pose a serious challenge to economic, national, and personal privacy and security  The following examples highlight the impact of such incidents: In March 2018, the Department of Justice reported that it had indicted nine Iranians for conducting a massive cybersecurity theft campaign on behalf of the Islamic Revolutionary Guard Corps  According to the department, the Iranians allegedly stole more than 31 terabytes of documents and data from more than 140 American universities, 30 U S  companies, and 5 federal government agencies, among other entities  In March 2018, a joint alert from DHS and the Federal Bureau of Investigation stated that, since at least March 2016, Russian government actors had targeted U S  government entities and critical infrastructure sectors, including the energy, nuclear, water, aviation, and critical manufacturing sectors  In June 2015, the Office of Personnel Management reported that an intrusion into its systems had affected the personnel records of about 4 2 million current and former federal employees  Then, in July 2015, the agency reported that a separate but related incident had compromised its systems and the files related to background investigations for at least 21 5 million individuals  The federal approach and strategy for securing information systems is prescribed by federal law and policy  FISMA sets requirements for effectively securing federal systems and information  In addition, the Federal Cybersecurity Enhancement Act of 2015 requires protecting federal networks through the use of federal intrusion prevention and detection capabilities  Further, Executive Order 13800, Strengthening the Cybersecurity of Federal Networks and Critical Infrastructure, directs agencies to manage cybersecurity risks to the federal enterprise by, among other things, using the NIST Framework for Improving Critical Infrastructure Cybersecurity (cybersecurity framework)  FISMA was enacted to improve federal cybersecurity and clarify government-wide responsibilities  The law is intended to provide for improved oversight of federal agencies’ information security programs  Specifically, the law provides a comprehensive framework for ensuring the effectiveness of information security controls over information resources that support federal operations and assets  The law is also intended to ensure the effective oversight of information security risks, including those throughout civilian, national security, and law enforcement agencies  FISMA assigns OMB and DHS oversight roles in ensuring federal agencies’ compliance with the law  Among other things, FISMA requires OMB to develop and oversee the implementation of policies, principles, standards, and guidelines on information security in federal agencies, except with regard to national security systems  The law also assigns OMB the responsibility of requiring agencies to identify and provide information security protections commensurate with assessments of risk to their information and information systems  The law further requires DHS to administer the implementation of agency information security policies and practices for non-national security information systems, in consultation with OMB, by developing, issuing, and overseeing implementation of binding operational directives; monitoring agency implementation of information security policies and practices; and convening meetings with senior agency officials to help ensure their effective implementation of information security policies and practices, among other things  FISMA assigned to NIST the responsibility for developing standards and guidelines that include minimum information security requirements  To this end, NIST has issued several publications to provide guidance for agencies in implementing an information security program  For example, NIST Special Publication (SP) 800-53 provides guidance to agencies on the selection and implementation of information security and privacy controls for systems  FISMA also assigns to the head of each executive branch agency, responsibility for providing information security protections commensurate with the risk and magnitude of harm resulting from unauthorized access, use, disclosure, disruption, modification, or destruction of information systems used or operated by an agency or by a contractor of an agency or other organization on behalf of an agency  The law also delegates to the agency chief information officer (CIO), or comparable official, the authority to ensure compliance with FISMA requirements  The CIO is responsible for designating a senior agency information security officer whose primary duty is information security  In addition, the law requires agencies to develop, document, and implement an agency-wide information security program to secure federal information systems  Specifically, these information security programs are to provide risk-based protections for the information and information systems that support the operations and assets of the agency  Further, FISMA requires agencies to comply with DHS binding operational directives, OMB policies and procedures, and NIST federal information processing standards  FISMA also has reporting requirements for OMB and federal agencies  Specifically, OMB is to report annually, in consultation with DHS, on the effectiveness of agency information security policies and practices, including a summary of major agency information security incidents and an assessment of agency compliance with NIST standards  Further, the law requires agencies to report annually to OMB, DHS, certain congressional committees, and the Comptroller General of the United States on the adequacy and effectiveness of their information security policies, procedures, and practices, as well as their compliance with FISMA  The Federal Cybersecurity Enhancement Act of 2015, among other things, sets forth authority for enhancing federal intrusion prevention and detection capabilities among federal entities  The act contains several provisions for DHS and OMB  Specifically, the act requires that DHS deploy, operate, and maintain capabilities to prevent and detect cybersecurity risks in network traffic traveling to or from an agency’s information system  DHS is to make these capabilities available for use by any agency  In addition, the act requires DHS to improve intrusion detection and prevention capabilities, as appropriate, by regularly deploying new technologies and modifying existing technologies  The act also requires OMB and DHS, in consultation with appropriate agencies, to review and update government-wide policies and programs to ensure appropriate prioritization and use of network security monitoring tools within agency networks, and to brief appropriate congressional committees  In May 2017, the President signed Executive Order 13800, which sets policy for managing cybersecurity risk as an executive branch enterprise  Specifically, it outlines actions to enhance cybersecurity across federal agencies and critical infrastructure to improve the nation’s cyber posture and capabilities against cybersecurity threats  To this end, the order states that the President will hold executive agency heads accountable for managing agency-wide cybersecurity risk and directs each executive agency to use the NIST cybersecurity framework to manage those risks  The cybersecurity framework, which provides guidance for cybersecurity activities, is based on five core security functions: Identify: Develop an organizational understanding to manage cybersecurity risk to systems, people, assets, data, and capabilities  Protect: Develop and implement appropriate safeguards to ensure delivery of critical services  Detect: Develop and implement appropriate activities to identify the occurrence of a cybersecurity event  Respond: Develop and implement appropriate activities to take action regarding a detected cybersecurity incident  Recover: Develop and implement appropriate activities to maintain plans for resilience and to restore capabilities or services that were impaired due to a cybersecurity incident  According to NIST, these five functions should be performed concurrently and continuously to address cybersecurity risk  In addition, when considered together, they provide a high-level, strategic view of the life cycle of an organization’s management of cybersecurity risk  Within the five functions are 23 categories and 108 subcategories that include controls for achieving the intent of each function  Appendix II provides a description of the cybersecurity framework categories and subcategories of controls  In February 2013, we reported that the government had issued a variety of strategy-related documents that addressed priorities for enhancing cybersecurity within the federal government, as well as for encouraging improvements in the cybersecurity of critical infrastructure within the private sector  However, we noted that no overarching cybersecurity strategy had been developed that articulated priority actions, assigned responsibilities for performing them, and set time frames for their completion  Accordingly, we recommended that the White House Cybersecurity Coordinator in the Executive Office of the President develop an overarching federal cybersecurity strategy that included all key elements of the desirable characteristics of a national strategy  These characteristics would include, among other things, milestones and performance measures for major activities to address stated priorities; cost and resources needed to accomplish stated priorities; and specific roles and responsibilities of federal organizations related to the strategy’s stated priorities  Since that time, the executive branch has made progress toward outlining a federal strategy for confronting cyber threats  For example, in September 2018, we reported that recent executive branch initiatives that identify cybersecurity priorities for the federal government provide a good foundation toward establishing a more comprehensive strategy  Nevertheless, we pointed out that additional efforts were needed to address all of the desirable characteristics of a national strategy that we recommended  Specifically, recently issued executive branch strategy documents did not include key elements of desirable characteristics that can enhance the usefulness of a national strategy as guidance for decision makers in allocating resources, defining policies, and helping to ensure accountability  For example, these strategy documents did not generally include: milestones and performance measures to gauge results; resources needed to carry out the goals and objectives; and clearly defined roles and responsibilities for key agencies, such as DHS, the Department of Defense, and OMB  Ultimately, we determined that a more clearly defined, coordinated, and comprehensive approach to planning and executing an overall strategy would likely lead to significant progress in furthering strategic goals and lessening persistent weaknesses  Subsequent to our September 2018 report, the President issued the National Cyber Strategy on September 20, 2018  The strategy builds upon Executive Order 13800 and describes actions that federal agencies and the administration are to take to, among other things, secure federal information systems  For example, the strategy states that the administration is expected to further enable DHS to secure federal department and agency networks, to include ensuring that DHS has appropriate access to agency information systems for cybersecurity purposes and can take and direct action to safeguard systems  In addition, the strategy states that the administration plans to continue with its existing efforts underway to transition agencies to shared services and infrastructure and that DHS is to have appropriate visibility into those services and infrastructure to improve cybersecurity posture  DHS’s Network Security Deployment (NSD) division manages cybersecurity programs that are intended to improve the cybersecurity posture of the federal government  Among these programs, NCPS provides a capability to detect and prevent potentially malicious network traffic from entering agencies’ networks  In addition, the Continuous Diagnostics and Mitigation (CDM) program provides tools to agencies intended to identify and resolve cyber vulnerabilities on an ongoing basis  Operated by DHS’s US-CERT, NCPS is intended to detect and prevent cyber intrusions into agency networks, analyze network data for trends and anomalous data, and share information with agencies on cyber threats and incidents  Deployed in stages, this system, operationally known as EINSTEIN, has provided increasing capabilities to detect and prevent potential cyberattacks involving the network traffic entering or exiting the networks of participating federal agencies  Table 1 provides an overview of the EINSTEIN deployment stages to date  In January 2016, we reported the projected total life-cycle cost of the program was approximately $5 7 billion through fiscal year 2018  In addition, according to the Federal CIO, Congress appropriated $468 million in fiscal year 2017 and $402 million in fiscal year 2018 for NCPS  In that report, we also noted that NCPS was partially, but not fully, meeting most of its stated system objectives  Although the system’s intrusion detection capabilities provided the ability to detect known patterns of malicious activity on agency networks, it was limited in its capabilities to identify potential threats using anomaly-based detection  We also reported that although DHS had developed metrics for measuring the performance of NCPS, the metrics did not gauge the quality, accuracy, or effectiveness of the system’s intrusion detection and prevention capabilities  The department had also identified needs for future capabilities, but had not defined requirements for the capability to detect threats entering and exiting cloud service providers  Further, DHS had not considered specific vulnerability information for agency information systems in making risk- based decisions about future intrusion prevention capabilities  Accordingly, we made nine recommendations to DHS to, among other things, enhance the NCPS capabilities for meeting its objectives and better define requirements for future capabilities  DHS agreed with each of our nine recommendations and indicated that it would take steps to address them  DHS’s CDM program provides federal agencies with tools and services that have the intended capability to automate network monitoring, correlate and analyze security-related information, and enhance risk- based decision making at agency and government-wide levels  These tools include sensors that perform automated scans or searches for known cyber vulnerabilities, the results of which can feed into a dashboard that, at an agency level, is intended to alert network managers and enable the agency to allocate resources based on the risk  Summary data from each participating agency’s dashboard is expected to be transmitted to the Federal Dashboard where the data can be used to inform decisions about cybersecurity risks across the federal government  There are four phases of CDM implementation: Phase 1—involves deploying products to automate hardware and software asset management, configuration settings, and common vulnerability management capabilities  According to the Cybersecurity Strategy and Implementation Plan, DHS purchased phase 1 tools and integration services for all participating agencies in fiscal year 2015  DHS plans to have all phase 1 tools deployed at participating agencies by the end of the second quarter of fiscal year 2019  Phase 2—intends to address privilege management and infrastructure integrity by allowing agencies to monitor users on their networks and to detect whether users are engaging in unauthorized activity  According to the Cybersecurity Strategy and Implementation Plan, DHS was to provide agencies with additional phase 2 capabilities throughout fiscal year 2016, with the full suite of CDM phase 2 capabilities delivered by the end of that fiscal year  However, according to the OMB FISMA Annual Report to Congress for Fiscal Year 2017, the CDM program began deploying Phase 2 tools and sensors during fiscal year 2017  DHS plans to have all phase 2 tools deployed at participating agencies by the end of fiscal year 2019  Phase 3—includes detection capabilities that are intended to assess agency network activity and identify any anomalies that may indicate a cybersecurity compromise  Full operating capability for phases 1, 2, and 3 is planned to be achieved by the end of fiscal year 2022  Phase 4—intends to provide tools to (1) protect data at rest, in transit, and in use; (2) prevent loss of data; and (3) manage and mitigate data breaches  According to CDM program officials, phase 4 has not been approved and no tools have been selected  An approach for protecting systems against cybersecurity compromise is for federal agencies to build successive layers of defense mechanisms at strategic points in their information technology infrastructures  This approach, commonly referred to as defense in depth, entails implementing a series of protective mechanisms so that if one mechanism fails to detect and prevent an attack, another will provide a backup defense  By utilizing defense in depth, federal agencies can reduce the risk of a successful cyberattack by implementing intrusion detection and prevention capabilities  NIST has developed guidelines for protecting agency information systems using intrusion detection and prevention capabilities  For example, NIST SP 800-53 recommends that agencies strategically deploy capabilities and perform monitoring of their systems to include observation of events occurring on their network and at the external boundary of their network  In addition, NIST SP 800-94 provides agencies with guidance in designing, implementing, configuring, securing, monitoring, and maintaining such capabilities  As part of their defense-in-depth approach and, as recommended by the NIST guidelines, agencies can deploy the following list of capabilities, among others, on their networks to detect and prevent an attack: Protecting email from intrusions: According to OMB, email, by way of phishing attacks, remains one of the most common threat vectors across the government  Methods for protecting email include encryption, false email alerts, and anti-spear-phishing training  Monitoring cloud services: Cloud vendors provide services to agencies, including Software as a Service, Platform as a Service, and Infrastructure as a Service  As agencies increasingly rely on cloud services, monitoring traffic to and from their cloud service providers helps to ensure that agencies detect malicious traffic  Using host-based intrusion prevention: Host-based intrusion prevention systems provide defense at an individual system or device level by protecting against malicious activities  Host-based capabilities include memory-based protection and application whitelisting  Monitoring external and internal traffic: Agencies can monitor external and internal traffic, including: encrypted traffic, traffic between workstations and servers on the network, and direct connections to outside entities such as universities  Monitoring traffic helps to ensure that agencies detect malicious activity  Using security information and event management: A security information and event management capability produces real-time alerts and notifications of significant security events  Security alerts and notifications can provide the agency with better situational awareness regarding possible intrusion activity  According to inspectors general, agency CIOs, and OMB reports on federal information security practices, many agencies were not effectively implementing the federal government’s approach and strategy to securing information systems as of fiscal year 2017  Agencies’ inspectors general determined that most of the 23 civilian CFO Act agencies did not have effective agency-wide information security programs  They also reported that agencies did not have effective information security controls in place, leading to deficiencies in internal control over financial reporting  In addition, the CIOs demonstrated that, during fiscal years 2016 and 2017, most agencies had not met all targets for the cybersecurity CAP goal for improving cybersecurity performance  Further, based on FISMA metrics reported for fiscal year 2017, OMB determined that 13 of the 23 agencies were managing risks to their enterprise, while the other 10 agencies were at risk of ineffectively identifying, protecting, detecting, responding to, and if necessary, recovering from cyber intrusions  Figure 2 summarizes agencies’ efforts to implement the government’s approach and strategy for securing information systems as of fiscal year 2017  Appendix III includes a table that provides an additional overview of the effectiveness of each agency’s implementation of the government’s approach and strategy to securing information systems  Inspectors general determined that more than half of the 23 civilian CFO Act agencies did not have effective agency-wide information security programs as of fiscal year 2017  Further, in agency financial statement audit reports for fiscal year 2017, inspectors general reported that, despite improvements being made in information security practices, most of the civilian CFO Act agencies continued to exhibit deficiencies in information security controls  As a result of these deficiencies, inspectors general reported material weaknesses or significant deficiencies in internal control over financial reporting  FISMA requires inspectors general to determine the effectiveness of their respective agencies’ information security programs  To do so, FISMA reporting instructions direct inspectors general to provide a maturity rating for agency information security policies, procedures, and practices related to the five core security functions established in the NIST cybersecurity framework, as well as for the agency-wide information security program  The ratings used to evaluate the effectiveness of agencies’ information security programs are based on a five-level maturity model, as described in table 2  According to this maturity model, Level 4 (managed and measurable) represents an effective level of security  Therefore, if an inspector general rates the agency’s information security program at Level 4 or Level 5, then that agency is considered to have an effective information security program  For fiscal year 2017, the inspectors general for 6 of the 23 civilian CFO Act agencies reported that their agencies had an effective agency-wide information security program  More specifically, for the 5 core security functions, most inspectors general reported that their agency was at Level 3 (consistently implemented) for the identify, protect, and recover functions, and at Level 2 (defined) for the detect and respond functions, as shown in figure 3  Inspectors general report on the effectiveness of agencies’ information security controls as part of the annual audits of the agencies’ financial statements  The reports resulting from these audits include a description of information security control deficiencies related to the five major control categories defined by the Federal Information System Controls Audit Manual (FISCAM)—access controls, configuration management, segregation of duties, contingency planning, and security management  The reports also identify the inspectors general’s designation of information security as a significant deficiency or material weakness in internal control over financial reporting systems  For fiscal year 2017, inspectors general continued to identify information security control deficiencies in each of the five major control categories across the 23 civilian CFO Act agencies  The number of agencies with deficiencies in the access control and contingency planning information security control categories decreased between fiscal years 2016 and 2017, according to the inspectors general  Nevertheless, the inspectors general reported that agencies continued to exhibit deficiencies in these two control categories  In addition, the number of agencies with deficiencies in the security management and segregation of duties control categories increased from the prior year  The number of agencies reported as having deficiencies in the configuration management control category remained the same  Figure 4 shows the number of agencies that reported deficiencies in each of the information security control categories for fiscal years 2016 and 2017  Overall, inspectors general for the 23 civilian CFO Act agencies reported progress in agencies’ information security practices for fiscal year 2017  Specifically, during that time, 17 inspectors general designated information security as either a significant deficiency (11) or material weakness (6) in internal control over financial reporting systems for their agencies  This is a decrease from the previous fiscal year when 19 inspectors general designated information security as a significant deficiency (12) or material weakness (7)  Reporting instructions contained in the fiscal year 2017 FISMA metrics directed CIOs to assess their agencies’ progress toward achieving outcomes that strengthen federal cybersecurity  To do this, CIOs evaluated their agencies’ performance in reaching targets for specific FISMA reporting metrics  According to the reporting instructions, certain metrics were selected to represent the administration’s cybersecurity CAP goal  These selected metrics allowed CIOs to evaluate their agencies progress in meeting targets for that goal  The cybersecurity CAP goal for fiscal years 2015 through 2017 was to improve cybersecurity performance by having an ongoing awareness of information security, vulnerabilities, and threats impacting the operating information environment; ensuring that only authorized users have access to resources and information; and implementing technologies and processes that reduce the risk of malware  The cybersecurity CAP goal consisted of three priority areas with a total of nine performance indicators  Each of the nine performance indicators had an expected level of performance, or target, for implementation  Table 3 shows the three priority areas and related performance indicators and targets of the cybersecurity CAP goal for fiscal years 2015 through 2017  According to agency CIO assessments for fiscal year 2017, 6 of the 23 agencies met all 9 targets for the cybersecurity CAP goal  More specifically, 8 agencies met all four targets for the information security continuous 16 agencies met the two targets for the identity, credential, and access management priority area; and 17 agencies met all three targets for the anti-phishing and malware defense priority area  In addition, CIOs reported that agencies were making progress in meeting the targets for the nine performance indicators for fiscal years 2016 and 2017, with increases in the number of agencies meeting the targets within each of the three priority areas  However, although the number of agencies that met the targets in individual priority areas showed a net increase, not all agencies maintained their status  For example, the CIO for one agency reported meeting all three targets for the anti-phishing and malware defense priority area in fiscal year 2016, but reported that the agency only met two of the three targets in fiscal year 2017  Figure 5 shows the number of agencies that reported meeting each of the targets within the individual cybersecurity CAP goal priority areas for fiscal years 2016 and 2017  Although the CIOs for only six agencies reported meeting each of the targets associated with all nine performance indicators for the three cybersecurity CAP goal priority areas, the CIOs at an additional eight agencies reported meeting each target for two of the three priority areas  Specifically, one CIO reported that its agency met each of the targets for the (1) information security continuous monitoring and (2) identity, credential, and access management priority areas; another CIO reported that its agency met each of the targets for the (1) information security continuous monitoring and (2) anti-phishing and malware defense priority areas; and the CIOs at six other agencies met each of the targets for the (1) identity, credential, and access management and (2) anti-phishing and malware defense priority areas  In fiscal year 2018, the President’s Management Agenda replaced the three cybersecurity-focused CAP goal priority areas with updated performance indicators, most of which are to be met by 2020: 1  the manage asset security priority area is similar to the information security continuous monitoring priority area from the previous CAP goal and has a focus on understanding the assets and users on agency networks  In addition to hardware asset and software asset management, this priority area includes performance indicators for authorization and mobile device management  2  the limit personnel access priority area focuses on issues of access management  This area includes performance indicators for using automated access management and managing access for privileged network and high-impact system users  The privileged network access management performance indicator is a continuation of the identity, credential, and access management priority area of the previous cybersecurity CAP goal  Therefore, agencies are expected to complete this metric by the end of the fiscal year 2018 FISMA reporting year  3  the protect networks and data priority area, which is similar to the anti-phishing and malware defense priority area from the previous cybersecurity CAP goal, has three new performance indicators: intrusion detection and prevention, exfiltration and enhanced defenses, and data protection  Appendix IV describes the updated cybersecurity-focused CAP priority areas and performance indicators in more detail  In Executive Order 13800, the President directed OMB, in coordination with DHS, to assess and report to the executive branch on the sufficiency and appropriateness of federal agencies’ processes for managing cybersecurity risks  For these risk management assessments, OMB leveraged the FISMA metrics reported by agency CIOs and inspectors general for fiscal year 2017  The metrics addressed domains that correspond with the five core security functions identified in the cybersecurity framework  Table 4 lists these domains and their relationship to the core functions  Based on OMB’s evaluation of these domains, agency risk management processes related to the five core security functions and overall agency enterprise fell into one of the following three rating categories: managing risk: required cybersecurity policies, procedures, and tools are in use and the agency actively manages cybersecurity risks; at risk: some essential policies, processes, and tools are in place to mitigate overall cybersecurity risk, but significant gaps remain; and high risk: key fundamental cybersecurity policies, processes, and tools are either not in place or not deployed sufficiently  For fiscal year 2017, OMB reported that not all agencies were managing risk  When considering each of the five core security functions, OMB reported that most of the 23 agencies were at risk or at high risk with regard to the identify and protect core security functions  Less than half of the 23 agencies were at risk with regard to the detect, respond, and recover core security functions  Overall, OMB determined that 13 agencies were managing risk and that the remaining 10 agencies were at risk of not effectively identifying, protecting, detecting, responding to, and if necessary, recovering from cyber intrusions  Figure 6 shows OMB’s risk management assessment ratings by core security function across the 23 agencies for fiscal year 2017  DHS and OMB, as required by law and policy, have taken various actions to facilitate the agencies’ use of intrusion detection and prevention capabilities to secure federal systems  For example, DHS has developed an intrusion assessment plan, deployed NCPS to offer intrusion detection and prevention capabilities to agencies, and is providing tools and services to agencies to monitor their networks through its CDM program  However, NCPS still had limitations in detecting certain types of traffic and agencies were not sending all appropriate traffic through the system  Further, CDM was behind at meeting planned implementation dates, and agencies have requested additional training and guidance for these services  OMB has taken steps to improve upon agencies’ capabilities, but has not completed a policy and strategy to do so, or fully reported on its assessment of agencies’ capabilities  The Federal Cybersecurity Enhancement Act of 2015 requires DHS, in coordination with OMB, to develop and implement an intrusion assessment plan to proactively detect, identify, and remove intruders in agency information systems on a routine basis  The act also requires that the plan be updated, as necessary  In December 2016, DHS documented its Intrusion Assessment Plan  In the plan, DHS outlined tools, platforms, resources, and ongoing work that the department provides, and that are intended to help agencies detect, identify, and remove intruders on their networks and systems  The intrusion assessment plan also outlines a defense-in-depth strategy, which utilizes multiple layers of cybersecurity and deploys multiple capabilities in combination, to secure agencies’ networks and information systems  For example, the plan calls for DHS to implement NCPS to provide a perimeter defense for the networks of federal civilian executive branch agencies, while the agencies are to deploy their own intrusion detection and prevention capabilities inside their networks  DHS submitted its intrusion assessment plan to OMB in January 2017  The Federal Cybersecurity Enhancement Act of 2015 also requires DHS to deploy, operate, and maintain a capability to detect cybersecurity risks and prevent network traffic associated with such risks from transiting to or from an agency information system  In addition, the act requires that DHS make regular improvements to intrusion detection and prevention capabilities by deploying new technologies and modifying existing technologies  Further, the act requires agencies to use this capability on all information traveling between their information systems and any information system other than an agency information system  DHS developed NCPS, operationally known as EINSTEIN, to provide the capabilities to detect and prevent potentially malicious network traffic from entering agency networks  Consistent with recommendations we made to DHS in January 2016, DHS has taken actions to improve these capabilities and has other actions underway  For example, the department determined that enhancing NCPS’s current intrusion detection approach to include functionality that would detect deviations from normal network behavior baselines would be feasible  In addition, according to DHS officials, the department was operationalizing functionality intended to identify malicious activity in network traffic otherwise missed by signature-based methods  determined that developing enhancements to current intrusion detection capabilities to facilitate the scanning of Internet Protocol Version 6 (IPv6) traffic would be feasible  According to DHS officials, the department has developed plans to fully support IPv6 for several of its NCPS intrusion detection capabilities  Further, the department has developed implementation schedules and begun roll-out of the enhancements  updated the tool it uses to manage and deploy intrusion detection signatures to include a mechanism to clearly link signatures to publicly available, open-source information  developed clearly defined requirements for detecting threats on agency internal networks and at cloud service providers to help better ensure effective support of information security activities  According to DHS officials, the department was also continuing pilot activities with cloud service providers to enhance protections of agency assets  developed processes and procedures for using vulnerability information, such as data from the CDM program as it becomes available, to help ensure the department is using a risk-based approach for the selection/development of future NCPS intrusion prevention capabilities  Nevertheless, NCPS continues to have known limitations in its ability to identify potential threats  For example: NCPS does not have the ability to effectively detect intrusions across multiple types of traffic  Specifically, DHS determined that developing enhancements to current intrusion detection capabilities to facilitate the scanning of traffic related to supervisory control and data acquisition (SCADA) control systems would not be feasible  However, according to DHS officials, the department is exploring capabilities that are intended to provide critical, cross-sector, real-time visibility into critical infrastructure companies that utilize SCADA systems  In addition, DHS determined that the scanning of encrypted traffic would not be feasible  Nevertheless, according to its officials, the department performed research on potential architectural, technical, and policy mitigation strategies that could provide both the protection and situational awareness for encrypted traffic  The department has actions under way to continue its research in this area  DHS does not always explicitly ask agencies for feedback or confirmation of receipt of NCPS-related notification  While the department had drafted a standard operating procedure related to its incident notification process, the policy did not instruct DHS analysts specifically to include a solicitation of feedback from agencies within the notification  Further, US-CERT could not provide any information regarding the timetable for when these procedures would take effect  Metrics for NCPS, as provided by DHS, do not provide information about how well the system is enhancing government information security or the quality, efficiency and accuracy of supporting actions  Without the deployment of comprehensive measures, DHS cannot appropriately articulate the value provided by NCPS  While the department had taken actions to develop new measures, these measures did not provide a qualitative or quantitative assessment of the system’s ability to fulfill the system’s objectives  NSD did not provide guidance to agencies on how to securely route their information to their Internet service providers  Without providing network routing guidance, NSD has no assurance that the traffic it sees constitutes all or only a subset of the traffic the customer agencies intend to send  As shown in table 5, as of October 2018, the department had implemented five of the nine recommendations and was in the process of implementing the remainder  However, until DHS completes implementation of the remaining recommendations, the effectiveness of NCPS’s intrusion detection and prevention capabilities may be hindered  In addition, the 23 civilian CFO Act agencies had implemented NCPS capabilities to varying degrees  In a March 2018 report, OMB reported that 21 (about 91 percent) of the 23 agencies had implemented the first two iterations of the NCPS capabilities  In addition, 15 (about 65 percent) of the 23 agencies had implemented all three NCPS capabilities, as shown in table 6 below  However, agencies did not route all network traffic for all information traveling between their information systems and any information system other than an agency information system through NCPS sensors  For example, officials at 13 of 23 agencies stated that not all of their agency external network traffic flowed through NCPS  To illustrate, officials at one agency estimated that 20 percent of their external network traffic did not flow through the system  In addition, 4 of the agencies in our review previously cited several challenges in routing all of their traffic through NCPS intrusion detection sensors, including capacity limitations of the sensors, agreements with external business partners that use direct network connections, interagency network connections that do not route through Internet gateways, use of encrypted communications mechanisms, and backup network circuits that are not used regularly  NSD officials stated that agencies are responsible for routing their traffic to the intrusion detection sensors, and DHS does not have a role in that aspect of NCPS implementation  As a result, potential cyberattacks may not be detected or prevented for a portion of the external traffic at federal agencies  As noted above, we previously recommended that DHS work with agencies to better ensure the complete routing of information to NCPS sensors  The Federal Cybersecurity Enhancement Act of 2015 requires DHS to include, in the efforts of the department to continuously diagnose and mitigate cybersecurity risks, advanced network security tools to improve the visibility of network activity and to detect and mitigate intrusions and anomalous activity  According to DHS officials, the department is addressing the requirement to improve the visibility of network activity by including advanced network security tools as a part of CDM phase 3  In April 2018, we testified that DHS had previously planned to provide 97 percent of federal agencies with the services they needed for CDM phase 3 in fiscal year 2017  In addition, according to OMB’s annual FISMA report for fiscal year 2017, the CDM program was to continue to incorporate additional capabilities, including phase 3, in fiscal year 2018  However, DHS now expects initial operational capabilities to be in place for phase 3 in fiscal year 2019  The department has awarded contracts of approximately $3 26 billion to support its Dynamic and Evolving Federal Enterprise Network Defense (also known as DEFEND) aspect of the CDM program, which is to include phase 3  DEFEND also is to provide coverage for existing agency deployments

ground_truth: The number of vehicles affected by safety defect recalls increased sharply in recent years—from nearly 13 million in 2011 to over 51 million in 2016. Once a defect is identified, auto manufacturers are required to send written notification to vehicle owners by mail. NHTSA also aims to enhance awareness of auto recalls by providing information on its website, NHTSA.gov . The Fixing America's Surface Transportation Act includes a provision requiring GAO to study the use of publicly available safety recall information. This report addresses: (1) how consumers and industry stakeholders use such information and (2) how easy to use do consumers find the auto recall areas of NHTSA.gov, among other objectives. To understand consumers' use of auto recall information and to test website usability, GAO conducted 12 focus groups with 94 consumers who had a recall. Focus groups were held in six locations selected for population and geographic variation. GAO identified key website usability practices and requested an evaluation by website usability professionals. GAO reviewed statutes, regulations, and NHTSA documents, and interviewed industry stakeholders—including 10 manufacturers selected based on sales market share and other factors. Consumers, manufacturers, and auto dealers use publicly available auto recall information differently. For example, the 94 consumers in 12 focus groups that GAO conducted used this information to decide whether to repair their vehicles. These consumers overwhelmingly cited safety risk and convenience as the two most influential factors they considered. Most consumers reported a preference for receiving recall notification by at least one electronic means, such as by e-mail or text message, in addition to mail. However, only 7 of 94 consumers reported receiving electronic notifications, suggesting a gap between the industry's auto recall notification practices and consumers' preferences. (See fig.). In response to a mandate in law, in September 2016, the National Highway Traffic Safety Administration (NHTSA) issued a proposed rule that, if finalized, would require manufacturers to notify consumers about auto recalls by electronic means in addition to mail. Most consumers in GAO's focus group website usability tests found the auto recall areas of NHTSA's website—NHTSA.gov—easy to use; however, some consumers experienced difficulties when asked to complete auto recall related tasks. For example, when consumers attempted to search for recalls affecting their specific vehicles, some found the search results confusing, leading them to question the accuracy of the results. Similarly, some consumers were hampered in searching for recalls by their vehicles' year, make, and model because the website did not always display model options using plain language. GAO found that the auto recall areas of NHTSA.gov do not always reflect federal and industry key website usability practices, and that an independent evaluation conducted by website usability professionals at GAO's request identified similar issues. NHTSA is in the process of consolidating its websites and plans to conduct a website usability study of NHTSA.gov with consumers after the consolidation is complete. However, the agency has not determined a completion date for the consolidation effort—an essential step for organizations to effectively guide their information technology efforts. Without establishing a completion date and taking interim steps to improve the usability of NHTSA.gov, consumers will likely continue to experience difficulties, which may limit the effectiveness of the website's primary means of providing consumers with information about recalls affecting their vehicles. GAO recommends that NHTSA determine a completion date for its website consolidation effort and take interim steps to improve the usability of NHTSA.gov by addressing the website usability difficulties GAO identified. The Department of Transportation concurred with the recommendations.

prompt_front: You are given two reports by the government agency.



Question: Write a one-page summary of the report 2.



Report 2:  According to DHS documentation, the task orders associated with DEFEND are to be issued between the second quarter of fiscal year 2018 and the second quarter of fiscal year 2024  FISMA requires that DHS provide operational and technical assistance to agencies in implementing policies, principles, standards, and guidelines on information security  Toward this end, DHS has available training and guidance related to the implementation of the capabilities of NCPS (i e , EINSTEIN) and CDM  Specifically: According the DHS officials, the department offers training and guidance to agencies on EINSTEIN 1 implementation  For example, DHS established a program in which the Software Engineering Institute will provide training and mentoring to agencies looking to enhance their understanding of, and proficiency with, the EINSTEIN 1 capability (e g , network traffic information)  NCPS program officials stated that agencies can use this service, which is available at no charge to them, on an unlimited basis as long as the requests relate to EINSTEIN 1  According to the officials, training and guidance related to EINSTEIN 2 and EINSTEIN 3 Accelerated is limited because DHS intentionally restricts the amount of data provided to agencies  According to DHS officials, the department also offers training and guidance to assist agencies with the implementation and use of resources associated with the CDM program, including webinars, guides, and computer-based training  The DEFEND contracts that the department awarded also include a mechanism for agencies to procure specialized tailored training, such as on the use of CDM tools  The department also offers customer advisory forums every other month that agencies are invited to attend  According to CDM program officials, the program’s governance, among other topics, is commonly discussed during these forums  Further, the department provides agencies with guidance, such as various governance documents, best practices, and frequently asked questions, through a web portal that is made available by OMB  In addition, US-CERT offers the CDM training program, which is to provide CDM implementation resources  Nevertheless, most agencies told us that they wanted DHS to provide more training and guidance as it relates to their implementation of the capabilities made available by NCPS and CDM  Specifically, Officials from 16 of 23 agencies reported that they wanted to receive additional training on NCPS capabilities  For example, officials at 5 agencies stated that they would like to receive training related to using network traffic information, understanding alerts, or implementing capabilities for cloud services  The officials also stated that they wanted training specific to agency security personnel  Officials from 19 of 23 agencies stated that they wanted to receive additional guidance related to NCPS’s capabilities, but not all of the 19 provided specific details  For example, officials from at least 3 agencies stated that they wanted additional guidance such as, “how to” documents, descriptions of architecture details, or guidance documents that explain NCPS’s capabilities so that agencies can gauge the gap between the security that the system provides and the security being provided by their own agency’s capabilities  Officials from 21 of 23 agencies reported that they wanted to receive additional training on implementing CDM at their agencies  For example, officials from 7 agencies suggested that additional training on the use of the tools would be beneficial  Officials from 22 of 23 agencies stated that they wanted additional guidance as it relates to CDM implementation  For example, officials from one agency stated that they would like examples of best practices and successful deployments  These requests for additional training and guidance demonstrate that agencies are either unaware of the available training and guidance, or that the training may not meet their needs  Until DHS coordinates with agencies to determine if additional training and guidance are needed, agencies may not be able to fully realize the benefits of the capabilities provided by the NCPS and CDM programs  Although OMB took steps to report on agencies’ implementation of intrusion detection and prevention capabilities, it did not report on all required actions  For example, the office did not submit DHS’s intrusion plan to Congress as required by the Federal Cybersecurity Enhancement Act of 2015  In addition, OMB provided various reports to Congress that described agencies’ intrusion detection and prevention capabilities, but the reports did not always include all information required by the act  Further, OMB developed a draft policy and strategy that were intended to improve agency capabilities, but it had not finalized these documents  The Federal Cybersecurity Enhancement Act of 2015 requires OMB to submit the intrusion assessment plan developed by DHS to the appropriate congressional committees no later than 6 months after the date of enactment of the act  The act also required OMB to submit to Congress a description of the implementation of the intrusion assessment plan and the findings of the intrusion assessments conducted pursuant to the intrusion assessment plan no later than 1 year after the date of enactment of the act, and annually thereafter  Although DHS developed and documented an intrusion assessment plan, which described a defense-in-depth approach to security, OMB did not submit the plan to Congress, as called for in the act  Even though DHS submitted the plan to OMB in January 2017, OMB had not submitted it to Congress as of October 2018 (21 months after DHS submitted the plan and 28 months past the due date)  On the other hand, OMB did submit its own reports to Congress which generally described elements of the implementation of DHS’s intrusion assessment plan and intrusion assessment findings  In September 2017, OMB issued its analysis of agencies’ implementation of intrusion detection and prevention capabilities, or more specifically, agencies’ implementation of the various versions of NCPS  In addition, the office’s annual FISMA report, issued most recently in March 2018, generally covered elements of the intrusion assessment plan  OMB personnel within the Office of the Federal CIO believed that these two reports, along with a process the office had initiated to validate incidents across the government, addressed the requirement for OMB to submit to Congress a description of the implementation of the intrusion assessment plan and the findings of the intrusion assessments conducted pursuant to the plan  However, the September 2017 and March 2018 reports did not address other elements described in DHS’s intrusion assessment plan  For example, OMB did not describe agency roles associated with segmenting their networks, identifying key servers based on threat and impact, ensuring all applications are appropriately tracked and configured, and categorizing and tagging data based on threat and impact  While OMB has provided important information to congressional stakeholders through its own reports, until it submits the plan and addresses all elements described in DHS’s intrusion assessment plan, it will continue to be remiss in providing timely and sufficiently detailed information regarding the intrusion assessment plan to congressional stakeholders to support their oversight responsibilities  The Federal Cybersecurity Enhancement Act of 2015 also required that OMB submit an analysis of agencies’ application of the intrusion detection and prevention capabilities to Congress no later than 18 months after the date of enactment of the act, and annually thereafter  OMB was to include a list of federal agencies and the degree to which each agency had applied the intrusion detection and prevention capabilities in this analysis  As discussed previously in this report, OMB issued its analysis of agencies’ implementation of intrusion detection and prevention capabilities in September 2017  However, the analysis did not include the degree to which agencies had applied the intrusion detection and prevention capabilities  For example, the analysis did not reflect that not all agencies were using this capability on all information traveling between their systems and any system other than an agency system, as required by the act  Until OMB includes the degree to which agencies have applied intrusion detection and prevention capabilities in its analysis, it cannot provide congressional stakeholders with an accurate portrayal of the extent to which the capabilities are detecting and preventing potential intrusions  The Federal Cybersecurity Enhancement Act of 2015 further required that the Federal Chief Information Officer, within OMB, submit a report to Congress no earlier than 18 months after the date of enactment, but no later than 2 years after that date, assessing the intrusion detection and intrusion prevention capabilities that DHS made available to agencies  The act required that the report address (1) the effectiveness of DHS’s system used for detecting, disrupting, and preventing cyber-threat actors, including advanced persistent threats, from accessing agency information and agency information systems; (2) whether the intrusion detection and prevention capabilities, continuous diagnostics and mitigation, and other systems deployed are effective in securing federal information systems; (3) the costs and benefits of the intrusion detection and prevention capabilities, including as compared to commercial technologies and tools, and including the value of classified cyber threat indicators; and (4) the capability of agencies to protect sensitive cyber threat indicators and defensive measures if they were shared through unclassified mechanisms for use in commercial technologies and tools  In a report issued in September 2018 (about 8 months past the required due date), the Federal Chief Information Officer provided Congress an assessment of intrusion detection and intrusion prevention capabilities across the federal enterprise  The report pointed out, among other things, that agencies did not possess or properly deploy capabilities to detect or prevent intrusions or minimize the impact of intrusions when they occur  In addition, the report acknowledged the need to improve the effectiveness of intrusion detection and intrusion prevention capabilities and stated that OMB would track performance through the CAP goal and annual FISMA reports  However, the report did not address all of the requirements specified in the act  For example, the report did not address whether DHS’s system (i e , NCPS) was effective in detecting advanced persistent threats  In addition, the report did not include a comparison of the costs and benefits of the intrusion detection and prevention capabilities versus commercial technologies and tools, or the value of classified cyber threat indicators  Further, the report did not address the capability of agencies to protect sensitive cyber threat indicators and defensive measures  Until OMB updates the Federal CIO report to address all of the requirements specified in the act, it will continue to be remiss in providing timely and sufficiently detailed information, such as that related to costs and benefits, among other elements in the act, to congressional stakeholders to support their oversight responsibilities  In addition to OMB’s responsibilities in the Federal Cybersecurity Enhancement Act of 2015, OMB has initiated plans for further improving agencies’ intrusion detection and prevention capabilities  In response to a tasking in Executive Order 13800, the Director of the American Technology Council coordinated the development of a report to the President from the Secretary of DHS, the Director of OMB, and the Administrator of the General Services Administration, regarding the modernization of federal information technology (IT)  The report, Report to the President on Federal IT Modernization, identified actions that OMB should take for (1) prioritizing the modernization of high-risk, high-value assets and (2) modernizing the Trusted Internet Connection (TIC) program and NCPS to improve protections, remove barriers, and enable commercial cloud migration  For example, OMB was to take the following actions subsequent to the December 13, 2017 report issuance date: Within 60 days: Update a TIC policy to address challenges with agencies’ perimeter-based architectures, such as the modernization of NCPS  In addition, introduce a “90 day sprint” during which approved projects would pilot proposed changes in TIC requirements  Within 90 days: Update the annual FISMA and CAP goal metrics to focus on those critical capabilities that were most commonly lacking among agencies and focus oversight assessments on high-value assets  Within 120 days: In conjunction with DHS, develop a strategy for optimally realigning resources across agencies to reduce the risk to high-value assets and respond to cybersecurity incidents for those assets  OMB has taken steps toward implementing several, but not all, of these actions  For example, it introduced a “90 day sprint” and, according to knowledgeable OMB staff, the outcomes of this action are directly informing changes in TIC requirements  In addition, OMB updated the annual FISMA and CAP goal metrics by including several metrics that focus on high-value assets  The updated FISMA and CAP goal metrics went into effect in April 2018  However, while OMB had taken steps toward updating the TIC policy and developing a strategy for optimally realigning resources, the policy and strategy were in draft and had not yet been finalized as of October 2018  The agency did not specify a time frame for finalizing the policy and strategy  Until OMB finalizes the TIC policy and the strategy for optimally realigning resources, the enhancements offered through the policy and strategy are unlikely to be realized  FISMA requires agencies to provide information security protections to prevent unauthorized access to their systems and information  Officials from the 23 selected agencies reported to us that they generally took steps to meet this requirement by augmenting the tools and services provided by DHS with their own intrusion detection and prevention capabilities  However, agencies did not consistently implement five key capabilities specified by DHS and NIST guidance  In addition, most of the agencies did not fully implement any of the phases of DHS’s CDM program that is intended to improve their capabilities to detect and prevent intrusions  Binding Operational Directive (BOD) 18-01 instructs agencies to enhance email security  These enhancements include enabling encrypted email transmission, ensuring that receiving mail servers know what actions the agency would like taken when an email falsely claims to have originated from the agency, and removing certain insecure protocols, among others  The final deadline for implementing all BOD 18-01 requirements was October 16, 2018  Additionally, NIST SP 800-53 Revision 4 recommends that security awareness training include training on how to recognize and prevent spear-phishing attempts  As of September 2018, only 2 of the 23 agencies reported implementing all of the email requirements  For the remaining 21 agencies: 9 agencies stated that their agency had plans to implement all enhancements by the October 2018 deadline, 1 agency was uncertain whether it would meet the deadline, and 11 stated they would not be able to meet the deadline  By contrast, the majority of agencies (22 of 23) reported that they had trained staff on spear-phishing exercises, as recommended by NIST SP 800-53 Revision 4  Officials at the remaining agency told us that the agency planned to have spear-phishing exercises in fiscal year 2019  Such training should help ensure that phishing will be a less effective attack vector against the majority of agencies  While agencies benefit from secure protocols and spear-phishing training, implementing the remaining BOD 18-01 email requirements would provide additional protection to agency information systems  NIST recommends that federal agencies deploy intrusion detection and prevention capabilities  These capabilities include monitoring cloud services, using host-based intrusion prevention systems, monitoring external and internal network traffic, and using a security information and event management (SIEM) system  However, in our semi-structured interviews of the 23 agencies, officials told us that they often had not implemented many of these capabilities  Such inconsistent implementation exposes federal systems and the information they contain to additional risk  As part of their continuing oversight efforts, OMB and DHS can use the information below to work with agencies to identify obstacles and impediments affecting the agencies’ abilities to implement these capabilities  NIST SP 800-53 Revision 4 states that agencies should monitor and control communications at the external boundary of the network  However, as of June 2018, fewer than half of the agencies that used cloud computing services were monitoring cloud traffic  Specifically: 10 of 22 agencies that used Infrastructure as a Service were monitoring inbound and outbound Infrastructure as a Service traffic, 7 of 21 agencies that used Platform as a Service were monitoring inbound and outbound Platform as a Service traffic, and 10 of 23 agencies that used Software as a Service were monitoring inbound and outbound Software as a Service traffic  Without monitoring traffic to and from cloud service providers, agencies risk a greater chance of malicious cloud activity detrimentally affecting agency information security  NIST SP 800-53 Revision 4 states that agency internal monitoring may be achieved by utilizing intrusion prevention capabilities  These capabilities include using host-based intrusion prevention systems to provide defense at an individual system or device level by protecting against malicious activities  Host-based capabilities include memory-based protection and application whitelisting  As of June 2018, officials at the 23 agencies reported the following to us: 16 agencies used host-based intrusion prevention capabilities, 15 agencies used memory-based protection, and 8 agencies used host-based application whitelisting  Until host-based intrusion protections are fully deployed, agencies will be at greater risk of malicious activity adversely affecting agency operations  NIST SP 800-53 Revision 4 also states that agencies should monitor and control communications at the external boundary of the network and at key internal boundaries (e g , network traffic)  NIST guidance also stated that an agency should deploy monitoring devices strategically within the network to detect essential information  However, the agencies in our review did not always monitor external and internal traffic  For example, of the 23 agencies: 5 reported that they were not monitoring inbound or outbound direct connections to outside entities  11 reported that they were not persistently monitoring inbound encrypted traffic  8 reported that they were not persistently monitoring outbound encrypted traffic  In addition, 13 agencies reported they were not using a network-based session capture solution  Of the 10 agencies that reported using this solution, officials from 2 agencies stated that they were not capturing workstation-to-workstation connections  Without thorough monitoring of external and internal traffic, agencies will have less assurance that they are aware of compromised or potentially compromised traffic within their network  NIST SP 800-53 Revision 4 states that agencies should establish enhanced monitoring capabilities  Such capabilities should include automated mechanisms that collect and analyze incident data for increased threat and situational awareness  According to NIST, a security information and event management (SIEM) system analyzes data from different sources and identifies and prioritizes significant events  Sources of data used by SIEM systems include logs from database systems, network devices, security systems, web applications, and workstation operating systems  Of the 23 agencies that we reviewed, 21 reported using a SIEM capability  Over half of the agencies employing a SIEM used one or more of their logs to match against known vulnerabilities and advanced persistent threats, as well as to create real-time alerts  For example, of the 21 agencies: 14 agencies reported collecting database logs, but only 7 agencies reported using the logs to match against known vulnerabilities and persistent threats and to create real-time alerts; 20 agencies reported collecting network logs, but only 13 agencies reported using them to match against known vulnerabilities and persistent threats and to create real-time alerts; 21 agencies reported collecting security logs, but only 13 reported using them to match against known vulnerabilities and persistent threats and to create real-time alerts; 15 agencies reported collecting web application logs, but only 9 agencies reported using them to match against known vulnerabilities and persistent threats and to create real-time alerts; and 13 agencies reported collecting workstation logs, but only 8 agencies reported using them to match against known vulnerabilities and persistent threats and to create real-time alerts  Only 5 agencies collected all 5 types of logs and used them to match against known vulnerabilities and persistent threats and to create real- time alerts  By not fully using SIEM capabilities, agencies will have less assurance that relevant personnel will be aware of possible weaknesses or intrusions  To further enhance their intrusion detection and prevention capabilities, the 23 civilian CFO Act agencies were in the process of implementing DHS’s CDM program  As previously discussed, Phase 1 of the program involves deploying products to automate hardware and software asset management, configuration settings, and common vulnerability management capabilities  Phase 2 intends to address privilege management and infrastructure integrity by allowing agencies to monitor users on their networks and to detect whether users are engaging in unauthorized activity  Phase 3 is intended to assess agency network activity and identify any anomalies that may indicate a cybersecurity compromise  As of June 2018, most agencies had not fully implemented any of the three phases  As shown in Figure 7, 15 agencies had partially implemented phase 1, 21 had partially or not yet begun to implement phase 2, and none of the agencies had fully implemented phase 3  Agencies’ implementation status has been affected, at least in part, due to delays in DHS’s deployment of the program phases  As a result, federal systems will remain at risk until the program is fully deployed  Many agencies have not effectively implemented the federal approach and strategy for securing information systems  For example, the inspectors general for 17 of the 23 selected agencies reported that their agencies had not effectively implemented their information security programs and had significant information security deficiencies associated with internal control over financial reporting  In addition, CIOs for 17 agencies reported not meeting all nine targets for the cybersecurity cross- agency priority goal  Further, OMB determined that that only 13 of the 23 agencies were managing risks to their overall enterprise, while the other 10 agencies were at risk  Until agencies more effectively implement the government’s approach and strategy, federal systems will remain at risk  DHS and OMB have initiatives underway that are intended to further improve agencies’ security posture  However, although DHS had provided training and guidance for NCPS and CDM, agencies expressed the need for more  In addition, OMB had also not finalized its policy and strategy aimed at addressing challenges with perimeter security and protecting high value assets, respectively  OMB had also not provided useful information to Congress, such as a description of agencies’ implementation of DHS’s intrusion assessment plan, the degree to which agencies are using NCPS, a complete analysis of agencies’ implementation of DHS’s intrusion assessment plan, or the costs and benefits of using commercial tools  Although agencies’ officials reported various efforts underway to enhance their agency’s intrusion detection and prevention capabilities, implementation efforts across the federal government were not consistent  OMB and DHS can use the information provided in this report to work with agencies to identify obstacles and impediments affecting the agencies’ abilities to implement these capabilities  We are making a total of nine recommendations, including two to DHS and seven to OMB  Specifically: The Secretary of DHS should direct the Network Security Deployment division to coordinate further with federal agencies to identify training and guidance needs for implementing NCPS and CDM  (Recommendation 1) The Secretary of DHS should direct the appropriate staff to work with OMB to follow up with agencies to identify obstacles and impediments affecting their abilities to implement intrusion detection and prevention capabilities  (Recommendation 2) The Director of OMB should submit the intrusion assessment plan to the appropriate congressional committees  (Recommendation 3) The Director of OMB should report on implementation of the defense- in-depth strategy described in the intrusion assessment plan, including all elements described in the plan  (Recommendation 4) The Director of OMB should update the analysis of agencies’ intrusion detection and prevention capabilities to include the degree to which agencies are using NCPS  (Recommendation 5) The Director of OMB should direct the Federal CIO to update her report to Congress to include required information, such as detecting advanced persistent threats, a comparison of the costs and benefits of the capabilities versus commercial technologies and tools, and the capability of agencies to protect sensitive cyber threat indicators and defense measures  (Recommendation 6) The Director of OMB should establish a time frame for finalizing the Trusted Internet Connections policy intended to address challenges with agencies’ perimeter-based architectures and issue it when finalized  (Recommendation 7) The Director of OMB should establish a time frame for finalizing the strategy for realigning resources across agencies to protect high- value assets and issue it when finalized  (Recommendation 8) The Director of OMB should direct the Federal CIO to work with DHS to follow-up with agencies to identify obstacles and impediments affecting their abilities to implement intrusion detection and prevention capabilities  (Recommendation 9) We provided a draft of this report to OMB and the 23 civilian CFO Act agencies, including DHS, covered by our review  In response, OMB provided comments via email, and DHS and three other agencies (the Department of Commerce, Social Security Administration, and U S  Agency for International Development) provided written comments, which are reprinted in appendices V through VIII, respectively  The 19 remaining agencies (the Departments of Agriculture, Education, Energy, Health and Human Services, Housing and Urban Development, the Interior, Justice, Labor, State, Transportation, the Treasury, and Veterans Affairs; as well as the Environmental Protection Agency, General Services Administration, National Aeronautics and Space Administration, National Science Foundation, Nuclear Regulatory Commission, Office of Personnel Management, and Small Business Administration) stated via email that they had no comments  In its comments, which the OMB liaison provided to GAO via email on December 7, 2018, OMB did not state whether it agreed or disagreed with the seven recommendations that we made to it  Rather, according to the liaison, OMB agreed with the facts in our draft report, but found that the report did not reflect the agency’s rationale for not submitting the DHS intrusion assessment plan to Congress and a report on the implementation of the plan, as required by the Federal Cybersecurity Enhancement Act of 2015  The liaison stated that OMB is working closely with DHS to provide strategic direction in assessing gaps in, and modernizing, the manner in which intrusion detection and prevention capabilities are delivered to the federal government  Further, in a subsequent email on December 10, 2018, OMB said it believes the Federal CIO’s September 2018 report to Congress, along with data provided in OMB’s fiscal year 2017 FISMA report to Congress, achieves the outcomes sought by the Federal Cybersecurity Enhancement Act of 2015 and demonstrates OMB's continuous engagement with DHS across the evolution of the intrusion detection and prevention program  As stated in our report, we acknowledge that OMB has provided important information to congressional stakeholders through its reports  However, OMB’s reports did not cover all outcomes described in the act  For example, as we pointed out, these reports did not fully address implementation of the defense-in-depth strategy described in DHS’s intrusion assessment plan  In addition, although OMB reported on several elements required by the Federal Cybersecurity Enhancement Act of 2015, it did not report on all of the required elements  For example, the reports did not address whether DHS’s NCPS was effective in detecting advanced persistent threats  The reports also did not include a comparison of the costs and benefits of the intrusion detection and prevention capabilities versus commercial technologies and tools, or the value of classified cyber threat indicators  Further, the reports did not address the capability of agencies to protect sensitive cyber threat indicators and defensive measures  Accordingly, we maintain that our recommendations for OMB to report on required elements in the Federal Cybersecurity Enhancement Act of 2015 are warranted  In addition, OMB suggested that we revise our recommendations to the agency to include a shared responsibility with DHS to help drive desired outcomes  However, six of the seven recommendations we are making to OMB are related to specific OMB responsibilities cited in either the Federal Cybersecurity Enhancement Act of 2015 or the Report to the President on Federal IT Modernization  As such, we believe the recommendations are appropriately addressed to OMB  Furthermore, our recommendations do not prevent OMB from working with DHS to implement them  Our seventh recommendation to OMB—to work with DHS to follow up with agencies to identify obstacles and impediments affecting their abilities to implement intrusion detection and prevention capabilities—includes a shared responsibility with DHS  OMB also provided technical comments, which we incorporated into the report, as appropriate  Subsequent to providing initial comments on our draft report, OMB issued a memorandum intended to provide a strategy for realigning resources across agencies to protect high-value assets  This action addresses our recommendation 8, which called for the Director of OMB to establish a time frame for finalizing the strategy for realigning resources across agencies to protect high-value assets, and to issue the strategy when finalized  In its comments, DHS stated that it concurred with the two recommendations we made to the department  DHS stated that it expects to implement the recommendations in 2019  The Department of Commerce commented that the report was reasonable and that the department agreed with the findings and recommendations  In its comments, the Social Security Administration stated that protecting its networks and information is a critical priority  According to the agency, it continued to make improvements in fiscal year 2018, such as improvements and progress in securing applications, leveraging the cloud, managing its assets and vulnerabilities, strengthening its network and incident response capabilities, improving its security training, and enhancing the overall effectiveness of its cybersecurity program  Finally, the U S  Agency for International Development commented that its inspector general had improved the agency’s capability maturity ratings for core security functions in fiscal year 2018  The agency also pointed out that it was the only selected agency in which fiscal year 2017 indicators of effectiveness in implementing the federal approach and strategy for securing information systems were all positive (as noted in Appendix III)  We are sending copies of this report to appropriate congressional committees, the Director of OMB, the heads of the 23 civilian CFO Act agencies and their inspectors general, and other interested congressional parties  In addition, the report is available at no charge on the GAO website at http://www gao gov  If you or your staff have any questions about this report, please contact Gregory C  Wilshusen at (202) 512-6244 or wilshuseng@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report  GAO staff who made key contributions to this report are listed in appendix IX  The Federal Cybersecurity Enhancement Act of 2015, which was enacted December 18, 2015, included a provision for GAO to report on the effectiveness of the federal government’s approach and strategy for securing agency information systems, including intrusion detection and prevention capabilities  The objectives of our review were to assess: (1) the reported effectiveness of selected agencies’ implementation of the federal government’s approach and strategy to securing agency information systems; (2) the extent to which the Office of Management and Budget (OMB) and the Department of Homeland Security (DHS) have facilitated the use of intrusion detection and prevention capabilities to secure federal agency information systems; and (3) the extent to which selected agencies reported implementing intrusion detection and prevention capabilities  Selected agencies for our review were the 23 civilian agencies covered by the Chief Financial Officers Act of 1990 (CFO Act)  We did not include the Department of Defense because the Federal Cybersecurity Enhancement Act of 2015 only pertains to civilian agencies  Because we focused our work on the 23 civilian agencies, results from these reviews are not generalizable to the entire federal government  To assess the reported effectiveness of agencies’ implementation of the federal government’s approach and strategy to securing agency information systems, we described the federal government’s approach and strategy by summarizing the Federal Information Security Modernization Act of 2014 (FISMA), Executive Order 13800, Strengthening the Cybersecurity of Federal Networks and Critical Infrastructure, and the National Institute of Standards and Technology’s (NIST) Framework for Improving Critical Infrastructure Cybersecurity (cybersecurity framework)  assessed the reported effectiveness of agencies’ implementation of the approach and strategy by reviewing annual reports from OMB and the inspectors general (IG) of the 23 civilian CFO Act agencies regarding the reported implementation of FISMA for fiscal year 2017  We described the IG reported maturity levels, including the Office of Inspectors General FISMA Reporting Metrics definition of “effectiveness ” These maturity levels are based on security domains aligned with the five core functions in NIST’s cybersecurity framework  We also summarized IG reported conclusions on the effectiveness of agencies’ information security programs for fiscal year 2017  reviewed the fiscal year 2016 and 2017 financial statement audit reports for each of the 23 civilian CFO Act agencies to identify the extent to which any significant deficiencies or material weaknesses related to information security over financial systems had been reported and to identify information security control weaknesses reported by the IGs  identified whether agencies had met the targets for the cybersecurity- focused cross-agency priority goal for fiscal years 2016 and 2017 by examining agency-reported performance metrics for fiscal years 2016 and 2017  evaluated OMB’s agency risk management assessment ratings to make a determination on how agencies were managing risk to their enterprise  These conclusions were based on FISMA metrics, and are aligned with the five core security functions defined in the cybersecurity framework  interviewed knowledgeable OMB officials and staff to obtain their views on the reported effectiveness of the federal government’s approach and strategy to securing agency information systems  To assess the extent to which OMB and DHS have facilitated the use of intrusion detection and prevention capabilities to secure federal agency information systems, we determined the extent OMB and DHS fulfilled their requirements described in the Federal Cybersecurity Enhancement Act of 2015 by collecting and reviewing artifacts from OMB and DHS and comparing them to the provisions outlined in the act  We also interviewed knowledgeable officials from OMB and DHS regarding their efforts to fulfill their requirements described in the act  determined the effectiveness of corrective actions taken by DHS to address nine previously reported recommendations we made in our report related to NCPS  Specifically, we collected appropriate artifacts and assessed the artifacts against the criteria used in that report, and determined the extent to which the actions taken by DHS met the intent of the recommendations, and we met with DHS staff responsible for the remediation activities and obtained their views of the status of actions taken to address the recommendations  held semi-structured interviews with knowledgeable officials from the 23 civilian CFO Act agencies  During these interviews, we obtained the agency’s views on whether they need more training and guidance from DHS for NCPS and CDM  We also interviewed knowledgeable officials and staff at DHS to obtain their views on how DHS had improved the intrusion detection and prevention capabilities it provides to federal agencies  We also interviewed DHS officials to obtain their views on the training and guidance that the department makes available to agencies  To assess the extent to which selected agencies reported implementing intrusion detection and prevention capabilities, we described the reported intrusion detection and prevention capabilities implemented by the 23 civilian CFO Act civilian agencies by summarizing implemented intrusion detection and prevention capability information obtained from the semi-structured interviews at the 23 civilian CFO Act agencies described above; identifying the extent to which the 23 civilian CFO Act agencies were in compliance with DHS’s binding operating directive (BOD) pertaining to enhanced email and web security (BOD 18-01) by collecting and summarizing Cyber Hygiene Trustworthy Email reports from the 23 agencies and determining the extent to which the agencies had taken required actions to implement the BOD  During the semi-structured interviews, we also obtained the agency’s views and experiences with other programs and services provided by DHS, including the extent to which agencies had implemented the tools offered by the department’s Continuous Diagnostics and Mitigation (CDM) program  To determine the reliability of submitted data and obtain clarification about agencies’ processes to ensure the accuracy and completeness of data used in their respective FISMA reports, we analyzed documents and conducted interviews with officials from 6 of the 23 civilian CFO Act agencies  To select these six agencies, we sorted agency fiscal year 2017 information technology budget data from highest to lowest amount and then divided the data into three tiers: high spending, medium spending, and low spending  We then randomly selected two agencies from each of the three tiers  The selected agencies were the Departments of Agriculture, Commerce, Housing and Urban Development, Transportation, and Veterans Affairs, and the U S  Agency for International Development  While not generalizable to all agencies, the information we collected and analyzed about the six selected agencies provided insights into various processes in place to produce FISMA reports  Based on this assessment, we determined that the data were sufficiently reliable for the purposes of our reporting objectives  We conducted this performance audit from December 2017 to December 2018 in accordance with generally accepted government auditing standards  Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives  We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives  The National Institute of Standards and Technology established the cybersecurity framework to provide guidance for cybersecurity activities within the private sector and government agencies at all levels  The cybersecurity framework consists of five core functions: identify, protect, detect, respond, and recover  Within the five functions are 23 categories and 108 subcategories that define discrete outcomes for each function, as described in table 7  The federal approach and strategy for securing information systems is prescribed by federal law and policy, including the Federal Information Security Modernization Act of 2014 and the presidential executive order on Strengthening the Cybersecurity of Federal Networks and Critical Infrastructure  Accordingly, federal reports describing agency implementation of this law and policy, and reports of related agency information security activities, indicated the effectiveness of agencies’ efforts to implement the federal approach and strategy  Table 8 summarizes the reported effectiveness of the 23 civilian Chief Financial Officers Act of 1990 agencies to implement the government’s approach and strategy to securing information systems  The President’s Management Agenda identifies cross-agency priority (CAP) goals to target areas where multiple agencies must collaborate to effect change  The agenda issued in fiscal year 2018 established an information technology modernization goal that includes a cybersecurity objective with specific priority areas and performance indicators  This cybersecurity-focused goal is intended to drive progress in the government’s efforts to modernize information technology to increase productivity and security  Figure 8 describes the 3 updated cybersecurity- focused cross-agency priority areas and 10 performance indicators  Each federal agency is expected to meet one of the 10 new performance indicators by the end of fiscal year 2018 and the remainder by 2020  In addition to the individual named above, Jeffrey Knott (assistant director), Daniel Swartz (analyst-in-charge), David Blanding, Chris Businsky, Kristi Dorsey, Di’Mond Spencer, Priscilla Smith, and Edward Varty made key contributions to this report  West Coile, Franklin Jackson, and Chris Warweg also provided assistance 
Report 2: L  116-6 ), did not provide an exact funding guidance for Industry and Analysis  The mission of ITA's Enforcement and Compliance unit is to enforce U S  trade laws and ensure compliance with negotiated international trade agreements  It is responsible for enforcing U S  antidumping and countervailing duty (AD/CVD) laws; overseeing a variety of programs and policies regarding the enforcement and administration of U S  trade remedy laws; assisting U S  industry and businesses with unfair trade matters; and administering the Foreign Trade Zone program and other U S  import programs  For FY2019, the Administration requested $90 6 million for the Enforcement and Compliance unit, an increase of $3 1 million (3 6%) from the FY2018-enacted amount  For the requested increase in funding, ITA cited the unit's increasing number of AD/CVD investigations, its new focus on self-initiating AD/CVD cases, and the increased workload due to the tariffs and investigations initiated through Section 232 of the Trade Expansion Act of 1962  The House Appropriations Committee proposed $85 5 million for Enforcement and Compliance  The House committee proposal represented $2 1 million less (-2 3%) than the Administration's request, and $1 0 million (1 1%) more than the FY2018-enacted amount   The Senate Committee on Appropriations recommended $91 5 million for Enforcement and Compliance  The Senate recommendation represented $0 9 million more than the Administration's request and $4 0 million more than the 2018-enacted amount  The Senate committee report noted that the committee was supportive of the Administration's request to self-initiate AD/CVD cases  For FY2019 appropriations, the conference report accompanying the Consolidated Appropriations Act, 2019 ( P L  116-6 ), included $88 5 million for Enforcement and Compliance  This amount was $1 0 million more (1 1%) than the FY2018 funding, and $2 1 million less (-2 1%) than the Administration's request  USITC is an independent, quasi-judicial agency responsible for conducting trade-related investigations and providing independent technical advice related to U S  international trade policy to Congress, the President, and the USTR  The commission (1) investigates and determines whether imports injure a domestic industry or violate U S  intellectual property rights; (2) provides independent tariff, trade, and competitiveness-related analysis to the President and Congress; and (3) maintains the U S  tariff schedule  USITC also serves as a federal resource for trade data and other trade policy information  It makes most of its information and analyses available to the public to promote understanding of competitiveness, international trade issues, and the role that international trade plays in the U S  economy  In addition to the President's budget request for the commission, USITC also has bypass authority to submit its budget directly to Congress without revision by the President, pursuant to Section 175 of the Trade Act of 1975  For FY2019, the President requested $87 6 million for USITC, which represented a $6 1 million decrease (-6 5%) from the FY2018-enacted amount ($93 7 million)  While the President requested a decrease in funding for USITC, the commission's independent budget submission—sent directly to Congress without revision by the President—requested $97 5 million for FY2019, an increase of $3 8 million (4 0%) from the FY2018-enacted amount  USITC cited the increasing number of import injury cases in the previous five years and projected that the caseload would increase further in FY2019  Both the House and Senate committee-reported bills recommended $95 0 million for USITC  This amount was $7 4 million (8 4%) more than the President's request and $1 3 million (1 4%) more than the FY2018-enacted amount  The Consolidated Appropriations Act, 2019 ( P L  116-6 ), provided $95 0 million for USITC, which was $7 4 million (8 4%) more than the President's request and $1 3 million (1 4%) more than the FY2018-enacted amount  USTR, located in the Executive Office of the President, is responsible for developing and coordinating U S  international trade and direct investment policies  USTR is the President's chief negotiator for international trade agreements, including commodity and direct investment negotiations  It negotiates directly with foreign governments to create trade agreements, resolve disputes, and participate in global trade policy organizations such as the World Trade Organization  It also meets with business groups, policymakers, and public interest groups on trade policy issues  In 2018, USTR led the negotiations for the modernization of the North American Free Trade Agreement (NAFTA) and the investigations into Chinese intellectual property practices  In addition to direct appropriations for USTR, supplementary funding for the agency is available through the congressionally established Trade Enforcement Trust Fund  For more detail on the trust fund, see section " Trade Enforcement Trust Fund ," below  For FY2019, the Administration requested $63 0 million for USTR's salaries and expenses, and no additional funding from the Trade Enforcement Trust Fund  The request represents a $9 6 million decrease (-13 2%) from the FY2018-enacted amounts ($72 6 million)  In the Administration's budget request, USTR outlined the Trump Administration's "aggressive trade agenda," and its goals of "(1) defending U S  national sovereignty over trade policy; (2) strictly enforcing U S  trade laws; (3) using all possible sources of leverage to encourage other countries to open their markets to U S  exports of goods and services, and protecting U S  intellectual property rights; and (4) negotiating better trade deals with countries in key markets around the world " Both the House and Senate committee-reported bills recommended a total of $72 6 million for USTR for FY2019  These proposals included $57 6 million for USTR's salaries and expenses and $15 0 million from the Trade Enforcement Trust Fund for enforcement activities authorized in Section 611 of the Trade Facilitation and Trade Enforcement Act of 2015 ( P L  114-125 )  The total proposals were $9 6 million (15 2%) more than the Administration's request, and were equal to the FY2018-enacted amount  The Consolidated Appropriations Act, 2019 ( P L  116-6 ), provided a total of $63 0 million for USTR, which included $53 0 million in salaries and expenses for USTR and an additional $15 0 million to be derived from the Trade Enforcement Trust Fund  The total funding was $4 6 million less (-6 3%) than the FY2018 appropriated amount  Over the past decade, Congress has provided funding for specific trade-related programs, including (1) China trade enforcement and compliance activities; (2) trade promotion and attracting foreign direct investment to the United States through ITA's SelectUSA program; and (3) trade enforcement activities through the Trade Enforcement Trust Fund and the Interagency Center on Trade Implementation, Monitoring, and Enforcement (ICTIME, formerly the Interagency Trade Enforcement Center (ITEC))  Since 2004, Congress has dedicated some of ITA's funding to AD/CVD enforcement and compliance activities with respect to China and other nonmarket economies  ITA's Office of China Compliance was established by the Consolidated Appropriations Act of 2004 ( P L  108-199 )  Its primary role has been to enforce U S  AD/CVD laws and to develop and implement other policies and programs aimed at countering unfair foreign trade practices in China  ITA's China Countervailing Duty Group was established in FY2009 to accommodate the workload that resulted from the application of countervailing duty law to imports from nonmarket economy countries  The Office of China Compliance is within the Enforcement and Compliance unit at ITA  ITA's FY2019 budget justification did not provide a breakdown of funding for its China AD/CVD activities  Both the House and Senate committee-reported bills included $16 4 million from ITA's funding for China AD/CVD enforcement and compliance activities for FY2019  The Consolidated Appropriations Act, 2019 ( P L  116-6 ), provided $16 4 million from ITA's funding for China AD/CVD enforcement and compliance activities for FY2019  This amount was equal to the FY2018-enacted amount  SelectUSA was created in 2011 and is now part of ITA's Global Markets unit  It coordinates investment-related resources across more than 20 federal agencies to (1) promote the United States as an investment market and (2) address investor climate concerns that may impede investment in the United States  The program serves as an information resource for international investors and advocates for U S  cities, states, and regions as investment destinations  ITA's budget justification did not provide a breakdown for requested funding for SelectUSA  The House committee-reported bill did not propose a specific funding amount for SelectUSA  The Senate committee report recommended $10 0 million in funding for SelectUSA, an amount equal to the FY2018-enacted amount  The Senate Committee on Appropriation proposed making funding contingent on (1) SelectUSA updating its protocol to ensure that its programs did not encourage foreign investments by state-owned entities into the United States and (2) SelectUSA reporting its updated protocol to the committee within 30 days of enactment of the bill  According to the conference report accompanying the Consolidated Appropriations Act, 2019 ( P L  116-6 ), the final agreement adopted the Senate committee report language regarding SelectUSA  ITA's Survey of International Air Travelers (SIAT) gathers statistics about air passenger travelers in the United States  These statistics are used across federal agencies, for a variety of purposes, such as to estimate the contribution of international travel to the economy, develop public policy on the travel industry, and forecast staffing needs at consulates and ports of entry  The Administration requested $5 0 million for FY2019 for SIAT to expand the survey and data collection  The Administration proposed that "$5 million in fee revenues collected from the surcharge on international travelers utilizing the Electronic System for Travel Authorization (ESTA) be redirected to fully fund the SIAT " The House committee-reported bill did not include a specific recommendation for SIAT  According to the Senate committee report, the Senate Committee on Appropriations did not adopt the Administration's proposal to seek alternative funding sources for SIAT and "direct[ed] ITA to continue funding SIAT out of its base budget  Within funds provided, ITA [was] encouraged to increase the sample size for SIAT " The Consolidated Appropriations Act, 2019 ( P L  116-6 ), and the accompanying conference report did not provide specific language regarding SIAT  In order to provide additional funding for USTR's trade enforcement activities, Congress established the Trade Enforcement Trust Fund (TETF) in 2016  In Section 611 of the Trade Facilitation and Trade Enforcement Act of 2015 ( P L  114-125 ), Congress set up the trust fund and outlined authorized uses of the funds  According to Section 611(d), USTR can use funds from the TETF to monitor and enforce trade agreements and WTO commitments and to support trade capacity-building assistance to help partner countries meet their free-trade agreement obligations and commitments  USTR can also transfer funds to select federal agencies for trade enforcement activities authorized in Section 611(d) of the Trade Facilitation and Trade Enforcement Act of 2015  For FY2019, the Administration requested no funding to be derived from the TETF; the FY2018-enacted amount was $15 0 million  Both the House and Senate committee bills proposed $15 0 million from the TETF for enforcement activities authorized in Section 611 of the Trade Facilitation and Trade Enforcement Act of 2015  These proposals were equal to the FY2018-enacted amount  The Consolidated Appropriations Act, 2019 ( P L  116-6 ), provided $15 0 million to be derived from the TETF for USTR, for enforcement activities authorized in Section 611 of the Trade Facilitation and Trade Enforcement Act of 2015  This amount is equal to the FY2018-enacted amount  ITEC was established by executive order in 2012 to take a "whole-of-government" approach to monitoring and enforcing U S  trade rights by using expertise from across the federal government  In 2016, the ITEC was succeeded by ICTIME, which Congress established through the Trade Facilitation and Trade Enforcement Act of 2015 ( P L  114-125 )  The executive-established ITEC received its funding through ITA; funding for ICTIME is now appropriated through USTR  ICTIME's purpose is to advance U S  trade policy through strengthened and coordinated enforcement of U S  trade rights  ICTIME investigates potential disputes under the auspices of the World Trade Organization; inspects potential disputes pursuant to bilateral and regional trade agreements to which the United States is a party; and carries out the functions of USTR with respect to the monitoring and enforcement of trade agreements to which the United States is a party  USTR and ITA work closely within the ICTIME to identify issues and develop information in areas of economic importance to U S  industries  The USTR's budget justification did not provide a breakdown for requested funding for ICTIME  The House and Senate committee-reported bills did not include a specific funding amount for ICTIME  The Senate committee report did note that the Senate committee supports ICTIME within the funds provided for USTR  The Consolidated Appropriations Act, 2019 ( P L  116-6 ) and the accompanying conference report also did not provide specific language regarding ICTIME 

ground_truth: Federal agencies are dependent on information systems to carry out operations. The risks to these systems are increasing as security threats evolve and become more sophisticated. To reduce the risk of a successful cyberattack, agencies can deploy intrusion detection and prevention capabilities on their networks and systems. GAO first designated federal information security as a government-wide high-risk area in 1997. In 2015, GAO expanded this area to include protecting the privacy of personally identifiable information. Most recently, in September 2018, GAO updated the area to identify 10 critical actions that the federal government and other entities need to take to address major cybersecurity challenges. The federal approach and strategy for securing information systems is grounded in the provisions of the Federal Information Security Modernization Act of 2014 and Executive Order 13800. The act requires agencies to develop, document, and implement an agency-wide program to secure their information systems. The Executive Order, issued in May 2017, directs agencies to use the National Institute of Standards and Technology's cybersecurity framework to manage cybersecurity risks. The Federal Cybersecurity Enhancement Act of 2015 contained a provision for GAO to report on the effectiveness of the government's approach and strategy for securing its systems. GAO determined (1) the reported effectiveness of agencies' implementation of the government's approach and strategy; (2) the extent to which DHS and OMB have taken steps to facilitate the use of intrusion detection and prevention capabilities to secure federal systems; and (3) the extent to which agencies reported implementing capabilities to detect and prevent intrusions. To address these objectives, GAO analyzed OMB reports related to agencies' information security practices including OMB's annual report to Congress for fiscal year 2017. GAO also analyzed and summarized agency-reported security performance metrics and IG-reported information for the 23 civilian CFO Act agencies. In addition, GAO evaluated plans, reports, and other documents related to DHS intrusion detection and prevention programs, and interviewed OMB, DHS, and agency officials. The 23 civilian agencies covered by the Chief Financial Officers Act of 1990 (CFO Act) have often not effectively implemented the federal government's approach and strategy for securing information systems (see figure below). Until agencies more effectively implement the government's approach and strategy, federal systems will remain at risk. To illustrate: As required by Office of Management and Budget (OMB), inspectors general (IGs) evaluated the maturity of their agencies' information security programs using performance measures associated with the five core security functions—identify, protect, detect, respond, and recover. The IGs at 17 of the 23 agencies reported that their agencies' programs were not effectively implemented. IGs also evaluated information security controls as part of the annual audit of their agencies' financial statements, identifying material weaknesses or significant deficiencies in internal controls for financial reporting at 17 of the 23 civilian CFO Act agencies. Chief information officers (CIOs) for 17 of the 23 agencies reported not meeting all elements of the government's cybersecurity cross-agency priority goal. The goal was intended to improve cybersecurity performance through, among other things, maintaining ongoing awareness of information security, vulnerabilities, and threats; and implementing technologies and processes that reduce malware risk. Executive Order 13800 directed OMB, in coordination with the Department of Homeland Security (DHS), to assess and report on the sufficiency and appropriateness of federal agencies' processes for managing cybersecurity risks. Using performance measures for each of the five core security functions, OMB determined that 13 of the 23 agencies were managing overall enterprise risks, while the other 10 agencies were at risk. In assessing agency risk by core security function, OMB identified a few agencies to be at high risk (see figure at the top of next page). DHS and OMB facilitated the use of intrusion detection and prevention capabilities to secure federal agency systems, but further efforts remain. For example, in response to prior GAO recommendations, DHS had improved the capabilities of the National Cybersecurity Protection System (NCPS), which is intended to detect and prevent malicious traffic from entering agencies' computer networks. However, the system still had limitations, such as not having the capability to scan encrypted traffic. The department was also in the process of enhancing the capabilities of federal agencies to automate network monitoring for malicious activity through its Continuous Diagnostics and Mitigation (CDM) program. However, the program was running behind schedule and officials at most agencies indicated the need for additional training and guidance. Further, the Federal CIO issued a mandated report assessing agencies' intrusion detection and prevention capabilities, but the report did not address required information, such as the capability of NCPS to detect advanced persistent threats, and a cost/benefit comparison of capabilities to commercial technologies and tools. Selected agencies had not consistently implemented capabilities to detect and prevent intrusions into their computer networks. Specifically, the agencies told GAO that they had not fully implemented required actions for protecting email, cloud services, host-based systems, and network traffic from malicious activity. For example, 21 of 23 agencies had not, as of September 2018, sufficiently enhanced email protection through implementation of DHS' directive on enhanced email security. In addition, less than half of the agencies that use cloud services reported monitoring these services. Further, most of the selected 23 agencies had not fully implemented the tools and services available through the first two phases of DHS's CDM program. Until agencies more thoroughly implement capabilities to detect and prevent intrusions, federal systems and the information they process will be vulnerable to malicious threats. GAO is making two recommendations to DHS, to among other things, coordinate with agencies to identify additional needs for training and guidance. GAO is also making seven recommendations to OMB to, among other things, direct the Federal CIO to update the mandated report with required information, such as detecting advanced persistent threats. DHS concurred with GAO's recommendations. OMB did not indicate whether it concurred with the recommendations or not.