prompt_front: You are given two reports by the government agency.



Question: Write a one-page summary of the report 1.

Report 1: Since 2001, DOD’s total workforce has changed in size and composition  DOD’s military, civilian, and contractor workforces peaked around 2011 and have since decreased in size, as shown in figure 1  Several factors have contributed to changes in the size of the workforces including varying levels of U S  involvement in the conflicts in Iraq and Afghanistan, military to civilian and contractor conversions, contractor insourcing, and the growth in certain workforces such as acquisition and cyber  DOD’s management of its workforce is governed by several workforce management statutes, including sections 129, 129a, and 2463 of Title 10 of the United States Code  Section 129 directs that DOD civilian personnel be managed each fiscal year on the basis of, and consistent with, total-force management policies and procedures established under section 129a, the workload required to carry out the functions and activities of the department, and the funds made available to the department each fiscal year  Section 129a directs the Secretary of Defense to establish policies and procedures for determining the most appropriate and cost-efficient mix of military, civilian, and contracted services to perform the mission of the department  Finally, Section 2463 directs the Under Secretary of Defense for Personnel and Readiness to devise and implement guidelines and procedures to ensure that consideration is given to using, on a regular basis, DOD civilian employees to perform new functions and functions performed by contractors that could be performed by DOD civilian employees  DOD Instruction 1100 22, Policy and Procedures for Determining Workforce Mix (April 12, 2010) (Change 1, Dec  1, 2017) establishes policy, assigns responsibilities, and prescribes procedures for determining the appropriate workforce mix of the military, civilian, and contracted services  The instruction provides criteria for workforce-mix decisions and directs DOD components to conduct a cost comparison to determine the low-cost provider for all new or expanding mission requirements and for functions that have been contracted but could be performed by DOD civilian employees  In addition, over the past 10 years DOD has taken steps to better understand the costs associated with its workforces  For example, we found in September 2013 that DOD had improved its methodology for estimating and comparing the full cost of work performed by military and civilian personnel and contractor support, but the methodology continued to have certain limitations, such as the lack of guidance for certain cost elements related to overhead  We made five recommendations, including for DOD to assess the advantages and disadvantages of allowing the continued use of different cost-estimation tools across the department or directing department-wide application of one tool, and revise its guidance in accordance with the findings of its assessment  DOD implemented this recommendation but has not yet implemented the other four recommendations although it concurred or generally concurred with them  DOD’s Cost-Comparison Report addressed three elements and partially addressed one element concerning the accounting for the fully-burdened, or full, cost of federal civilian and service contractor personnel performing functions at the selected installations, as shown in table 1  DOD concluded that for the 21,000 federal civilians and service contractors compared, neither federal civilians nor service contractors were predominately more or less expensive, with the costs being dependent upon the function being performed, location, and level of expertise  DOD noted that the results were not generalizable across the department  Each of the elements and our assessment are discussed below  We believe that DOD addressed the reporting element to assess performance of functions performed by civilian and contractor personnel by developing a methodology to assess performance of functions performed by federal civilians and service contractors at organizations within nine geographic regions including two locations outside the continental United States  Organizations included in DOD’s methodology include the following: Fort Belvoir Community Hospital Defense Threat Reduction Agency US Army Intelligence and Security Command Aviation and Missile Research, Development, and Engineering Center Naval Medical Center San Diego Space and Naval Warfare Systems Command Ogden Air Logistics Complex 75th Air Base Wing Naval Facilities Engineering Command Tripler Army Medical Center DOD’s methodology included the following: Selecting installations and organizations: DOD used data from the Defense Civilian Personnel Data System to identify military installations with large reported numbers of federal civilians  According to DOD officials, they eliminated from consideration those installations that had no reported contractors  From this subset of installations, DOD selected organizations to represent all three military departments and diverse geographical locations, to include two locations outside the continental United States  Assessing the functions performed by civilians and contractors to identify federal civilians and service contractors performing similar functions: DOD assessed the performance of functions at these selected locations to identify federal civilians and service contractors performing similar functions as there is no direct mapping or perfect match between existing taxonomies used to quantify federal civilian positions and contracted services  Further, DOD reported that the day-to-day functions performed by federal civilian employees do not always directly correlate to the designated occupational series or the job title for their position  For example, an individual with an occupational series assigned as an accountant may actually perform work more consistent with that of a financial analyst  According to DOD’s Cost-Comparison Report, DOD did not rely on occupational series names or job titles alone to determine the actual work being performed by federal civilians  Specifically, DOD conducted site visits with each organization and relied on local managers’ direct knowledge of the actual tasks that their federal civilians and service contractor personnel performed  According to DOD’s Cost-Comparison Report, DOD determined that personnel need to perform at least 80 percent common tasks to be able to make a comparison  For the organizations selected, DOD compared the costs of all federal civilians and service contractors identified as performing similar functions  The challenges DOD identified in DOD’s Cost-Comparison Report on determining the functions performed by contractor personnel are similar to those we encountered in our prior work on DOD’s efforts to compile and review of an inventory of contracted services  Section 2330a of Title 10 of the U S  Code directs the Secretary of Defense to annually prepare an inventory of activities performed during the preceding fiscal year pursuant to staff augmentation contracts  Section 2330a also directs the secretary of each military department and head of each defense agency responsible for activities in the inventory to, within 90 days after the Secretary of Defense submits the inventory, review the contracts and activities in the inventory for which that secretary or agency head is responsible, in part to identify activities that should be considered for conversion  Our prior work has identified, among other issues, that the absence of a complete and accurate inventory of contracted services hinders DOD’s management of these services  According to DOD officials, the Office of the Under Secretary of Defense (Personnel and Readiness) has also recognized the challenges associated with the various taxonomies and lexicons associated with articulating the size and composition of federal civilian, military, and contracted services workforces, and has efforts underway with the goal of better aligning those to enable more holistic total force management of all sources of labor  According to DOD officials, by improving available workforce data, DOD can support better-informed leadership decisions, improve accuracy of analyses, and provide consistent explanations of the department’s workforce resources  DOD officials told us that this effort has an estimated completion of December 2018  We believe that DOD partially addressed the reporting element to account for the full cost of civilian and contractor personnel by providing an accounting of the labor costs of selected federal civilian and service contractor full-time equivalents for personnel performing similar functions at government-owned facilities during calendar year 2015, but excluding certain non-labor costs from its cost calculations  According to DOD officials, 2015 was the last year for which complete data were available  DOD developed a methodology for identifying labor costs associated with federal civilian and service contractor full-time equivalents during calendar year 2015 at government-owned facilities for its cost comparisons  Based on reviews of applicable guidance and consultations with the Office of the Under Secretary of Defense – Comptroller, DOD included numerous federal civilian costs collected from several sources in DOD’s Cost Comparison Report, as shown in table 2  In addition, to assure data quality, DOD officials told us that they took steps to identify data errors in the data collected including identifying missing data fields and data entries that might indicate data errors  For example, DOD officials told us that they verified that they had pay records for every pay period in calendar year 2015 by identifying potential errors and outliers and sharing these with the Defense Finance and Accounting Service and the selected DOD organizations for review  Officials also stated that DOD sent its complete calculated data sets to each organization for review against their own pay records and that all errors were corrected or outliers were explained  Additionally, according to our analysis, DOD excluded overtime from its costs related to federal civilians in accordance with Office of Management and Budget Circular A-11  However, DOD included overtime pay in its report separately for context and noted that overtime pay is a significant part of civilian compensation for some organizations  Officials noted that those funded via a working capital fund arrangements, such as depots, use overtime to handle surges in demand throughout the year  DOD noted in its report that selected service contracts at government facilities and developed three methodologies to identify labor costs associated with service contractor full-time equivalents during calendar year 2015, as shown in table 3  DOD stated in its report that identifying service contractor full-time equivalents is a significant challenge because the level of detail available in each contract varied such that DOD could not employ a single methodology, and unlike federal civilian pay data, there is no centralized database on service-contractor pay  DOD reported that contracts are rarely written to address the cost-per-contractor as a full-time equivalent, and some contracts do not differentiate between labor and non-labor costs  DOD noted in its report that the negotiated price of the contract includes direct costs, including labor and non-labor costs, and indirect costs such as overhead  Further, the contract costs include service contractor profit  Based on our review of DOD’s Cost-Comparison Report, DOD used non- excludable contract costs as a basis in two of its methodologies  These costs to DOD are associated with labor, and include pay and benefits provided to service contractor personnel, contractor profit, and overhead the contractor included in the cost of the contract  When the number of service contractor full-time equivalents and full costs for a contract was known, DOD used the first method, dividing contract costs by the number of service contractor full-time equivalents to arrive at the cost per service contractor full-time equivalent  When the number of billable hours was known, DOD used the second method, multiplying the ratio of contract costs divided by billable hours by a standard number of annual billable hours  For contracts in which the labor rate was known but costs could not be disaggregated, DOD multiplied the labor rate by a standard 1,880 annual billable hours unless a contract specified the labor rate as a number of annual billable hours  For example, Defense Logistics Agency contractor-labor rates for wage grade equivalent contractor full-time equivalents are based on 2,080 annual labor hours  We assessed DOD’s report as partially addressing the reporting element to account for the full cost of federal civilian and contractor personnel because DOD excluded certain non-labor costs from its costs calculations—(1) direct non-labor costs for government owned facilities and government provided supplies, (2) indirect costs for general and administrative and overhead for civilians, and (3) costs to manage contracts—from its costs calculations  Senate Report 114-49 stated that DOD is to include an accounting of the full cost of DOD federal civilian and service contractors performing similar functions, including facility overhead  DOD stated in its report that the methodology utilized to compare the costs of federal civilian and service contractor full-time equivalents was consistent with DOD Instruction (DODI) 7041 04, Estimating and Comparing the Full Costs of Civilian and Active Duty Military Manpower and Contract Support (July 3, 2013), hereafter referred to DODI 7041 04  However, DODI 7041 04 states that the full cost of personnel should include direct and indirect non-labor costs, such as those referenced previously  DOD officials stated that they considered including non-labor costs in their calculations but did not because they believe these costs would add approximately the same to both federal civilian and service contractor costs  DODI 7041 04 instructs that if a function is performed on government property, the costs of goods, services, and benefits that are common costs may be excluded provided the number of government and contactor personnel is equivalent  DODI 7041 04 further instructs that when the number of government and contractor personnel differs, adjustments must be made to the cost estimates to account for the difference in number of government and contractor personnel  While there were some instances where it was the case that DOD’s cost estimates involved an equal number of civilian and contractor personnel performing functions on government property, there were many instances in where the personnel numbers differed and common costs should not have been excluded  For example, in DOD’s comparisons of federal civilians and service contractors at Fort Belvoir Community Hospital, DOD conducted comparisons of 19 functions where 2 functions had equal numbers of federal civilian and service contractors and 17 functions had differing numbers of federal civilian and service contractors  In one comparison, the number of contractors was over three times the number of civilians  DOD officials also stated that they believe their methodology is in accordance with DODI 7041 04 because DODI 7041 04 states that the cost elements in the instruction can be modified or augmented in each specific case as necessary, but DOD components should be prepared to support such decisions with sufficient justification  We acknowledge that DODI 7041 04 states that the cost elements can be modified, but by excluding non-labor costs in its cost comparisons, DOD did not account for the full cost of federal civilians and service contractors as requested in the mandate  We believe that DOD addressed the reporting element to compare costs by comparing its calculated costs of selected federal civilians and service contractors performing similar functions at selected installations  DOD reported that its results represent selected personnel performing functions within selected organizations and are not generalizable across the department  DOD concluded that for the federal civilian and contractor full-time equivalents included in the study, the costs varied by organization, location, and function being performed  DOD presented comparisons of federal civilian and service contractor full-time equivalents costs and expressed these results as a cost ratio  However, it is not clear how the results would be different if all costs that encompass full costs of personnel would have been included in DOD’s comparisons  See tables 4 and 5 below for examples of greater costs for the performance of functions by federal civilians or service contractors at Fort Belvoir Community Hospital in Fort Belvoir, Virginia  We believe that DOD addressed the reporting element by assessing the flexible employment authorities for the employment and retention of federal civilian employees at the same 17 organizations used for the cost comparison  Specifically, DOD sent questionnaires to DOD hiring officials and human resource professionals to collect information on flexible employment authorities  DOD included a broad spectrum of organizational missions in its query of management and human resource officials regarding the use and availability of flexible hiring authorities  Noting that this assessment is more subjective than the others in DOD’s Cost-Comparison Report, DOD queried senior leaders, middle managers, front-line supervisors and human resource professionals regarding which authorities are being used and the effectiveness of each  According to DOD’s report, in this way, DOD was able to gauge the extent to which each type of authority was used as well as the satisfaction with and effectiveness of each  DOD’s Cost-Comparison Report made several conclusions regarding flexible hiring authorities and made one recommendation  The findings included that there was a variance in the authorities used between organizations, management unfamiliarity with all available authorities, and a belief among managers that expanded use of some authorities is needed to produce more quality hires  DOD’s Cost- Comparison Report recommended DOD and OPM should explore opportunities to refine, consolidate, or reduce unused, inefficient, or cumbersome hiring authorities  We provided a draft of this report to DOD for review and comment  In written comments, DOD non-concurred with our assessment that DOD partially addressed the mandated reporting element to provide an accounting of the fully-burdened cost of federal civilian and service contractor personnel performing functions at the selected installations to include training, benefits, reimbursable costs, and facility overhead  DOD’s comments are reproduced in their entirely in appendix I  DOD also provided technical comments, which we incorporated as appropriate  DOD stated that we presented the three reporting elements identified in the congressional mandate absent the full context and congressional intent  Specifically, DOD stated that in the congressional mandate, the list of elements to be included in the report is not a stand-alone list and DOD stated that we present the elements as a stand-alone list  DOD further stated that the list of elements in the mandate is preceded by a paragraph that we did not reproduce in our report, but which provides context and congressional intent for the reporting elements  We do not believe that the language omitted from our report changes the meaning of the reporting elements to be included in DOD’s cost comparison report because the paragraph omitted clearly states that the purpose of the report is to provide the results of a study that includes a comparison of the fully-burdened cost of the performance of functions by DOD civilian personnel with the fully-burdened cost of the performance by DOD contractors  The paragraph preceding the reporting elements and the elements reads as follows: The committee directs the Secretary of Defense to submit to the Committees on Armed Services of the Senate and the House of Representatives, and to the Comptroller General of the United States, a report setting forth the results of a study, conducted by the Secretary for the purposes of the report, of a comparison of the fully-burdened cost of performance of functions by Department of Defense (DOD) civilian personnel with the fully-burdened cost of the performance of functions by DOD contractors by no later the February 1, 2016  The study shall include: (1) An assessment of performance of such functions at six DOD installations selected by the Secretary for purposes of the study from among DOD installations at which functions are performed by an appropriate mix of civilian personnel and contractors, with four such installations to be located in the continental United States and two such installations to be located outside the continental United States; (2) An accounting of the fully-burdened cost of DOD civilian personnel and contractors performing functions for DOD (including costs associated with training, benefits, reimbursable costs under chapter 43 of title 41, United States Code, and facility overhead) in order to permit a direct comparison between the cost of performance of functions by DOD civilian personnel and the cost of the performance of functions by contractors; (3) A comparison of the cost of performance of the full range of functions, required expertise, and managerial qualities required to adequately perform the function to be compared, including: a  Secretarial, clerical, or administrative duties, including data entry; b  Mid-level managers and other personnel possessing special expertise or professional qualifications; c  Managers and other leadership; and d  Personnel responsible for producing congressionally-directed reports  The committee recommends that, in conducting the study, the Secretary should take into account the policy that inherently governmental functions vital to the national security of the United States may not be performed by contractor personnel  The report required shall include an assessment of the flexible employment authorities available to the Secretary for the employment and retention of civilian employees of the DOD, including an identification of such additional flexible employment authorities as the Secretary considers appropriate to shape the civilian personnel workforce of the DOD  Not later than 120 days after receipt of such report, the Comptroller General shall submit to Congress a report that includes an assessment of the adequacy and sufficiency of the report submitted by the Secretary, including any recommendations for policy or statutory change as the Comptroller deems appropriate  As we reported, DOD noted in its cost comparison report that it identified labor costs used in its comparisons  However, DOD did not include direct and indirect non-labor costs and DODI 7041 04 states that the full cost of personnel should include these non-labor costs as we discussed earlier in the report  Therefore, DOD only partially addressed the reporting provision  In addition, DOD stated that we omit relevant language related to congressional intent for the second reporting element (i e , an accounting of the fully-burdened cost of DOD civilian personnel and contractors)  DOD stated that the text, “      in order to permit a direct comparison between the cost of performance of functions by DOD civilian personnel and the cost of the performance of the function by contractors,” conveys the congressional intent that the study is for comparison and our exclusion of the text in our restatement of the element omitted language indicating relevant Congressional intent  We do not believe that the language omitted in our report changed the meaning of the reporting element, which is that DOD was to include an accounting of the fully- burdened costs of federal civilians and service contractors in its cost comparisons  DOD further stated that we did not assess the second reporting element (i e , an accounting of the fully-burdened cost of DOD civilian personnel and contractors) as it is directly stated but rather that we assessed the element by redefining it and then asserting that DOD partially addressed it  DOD noted that the direct language of the second reporting element is for DOD to include an “accounting” of the fully burdened cost of DOD civilian personnel and contractors  DOD asserted that we misinterpreted the meaning of “accounting” when we determined that DOD partially addressed the mandate because it did not “calculate” certain non-labor costs  We disagree  As we discuss in our report, DOD did account for the labor costs associated with federal civilian and service contractors by gathering labor cost data from several sources, but it did not include non- labor costs in its cost calculations  In order to account for the fully burdened costs of federal civilians and service contractors, as directed to do so by the preamble to the reporting elements, as well as the second reporting element, DOD should have included all labor and non-labor costs in the cost calculations  DOD also stated that our assessment incorrectly implies that to “account” for costs is equivalent to “calculating” costs as evidenced by the following quote from our draft report, "We acknowledge that DODI 7041 04 states that the cost elements can be modified, but by excluding non-labor costs in its cost comparisons, DOD did not account for the full cost of federal civilians and service contractors as requested in the mandate " DOD stated that although DOD did not “calculate” some non-labor costs, they did “account” for them in accordance with DODI 7041 04 and as directed in the congressional mandate  DOD asserted that in multiple places, DODI 7041 04 states that common costs "are excluded" and "may be excluded" from cost comparisons  DOD provided facility costs as an example of non-labor costs accounted for but not calculated in its cost comparisons  DOD stated that in its report, all of the civilian positions and contractor functions are performed at government-owned facilities  Thus, facility costs are common costs and may be excluded  DOD stated that their report accounted for facility costs by recognizing that such costs exist and are common costs, thus, DOD properly excluded such costs in accordance with DODI 7041 04, and their report satisfied the Congressional mandate  We disagree  As mentioned above, the preamble to the mandated reporting elements and the second reporting element specifically directed that DOD account for the fully-burdened cost of DOD civilian and contractor personnel  Because there are multiple costs associated with civilian and contractor personnel, calculations are necessary in order to account for the full cost of these workforces  DODI 7041 04 instructs that if a function is performed on government property, the costs of goods, services, and benefits that are common costs may be excluded provided the number of government and contactor personnel is equivalent  While there were some instances where it was the case that DOD’s cost estimates involved an equal number of civilian and contractor personnel performing functions on government property, there were many instances where the personnel numbers differed and common costs should not have been excluded  For example, in DOD’s comparisons of federal civilians and service contractors at Fort Belvoir Community Hospital, DOD conducted comparisons of 19 functions where 2 functions had equal numbers of federal civilian and service contractors and 17 functions had differing numbers of federal civilian and service contractors  In one comparison, the number of contractors was over three times the number of civilians  DODI 7041 04 further instructs that when the number of government and contractor personnel differs, adjustments must be made to the cost estimates to account for the difference in number of government and contractor personnel  DOD did not make these adjustments in is calculations and as result non-labor costs should not have been excluded; therefore, DOD did not account for the fully- burdened costs, as directed by Congress  We are sending copies of this report to the appropriate congressional committees  We are also sending copies to the Secretary of Defense, the Director of the Office of Cost assessment and Program Evaluation and other interested parties  This report will also be available at no charge on our Web site at http://www gao gov  Should you or your staff have any questions concerning this report, please contact Brenda S  Farrell at (202) 512-3604 or farrellb@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report  GAO staff who made key contributions to this report are listed in appendix II  In addition to the contact named above, Vincent Balloon, Assistant Director; Timothy Carr, Felicia Lopez, Clarice Ransom, Michael Silver, and Norris “Traye” Smith made key contributions to this report  Department of Defense: Actions Needed to Address Five Key Mission Challenges, GAO-17-369 (Washington, D C : June 13, 2017) DOD Civilian and Contractor Workforces: Additional Cost Savings Data and Efficiencies Plan Are Needed, GAO-17-128 (Washington, D C : October 12, 2016) Federal Hiring: OPM Needs to Improve Management and Oversight of Hiring Authorities, GAO-16-521 (Washington, D C : August 2, 2016) DOD Service Acquisition: Improved Use of Available Data Needed to Better Manage and Forecast Service Contract Requirements, GAO-16-119 (Washington, D C : February 18, 2016) Civilian and Contractor Workforces: Complete Information Needed to Assess DOD’s Progress for Reductions and Associated Savings, GAO-16-172 (Washington, D C : December 23, 2015) DOD Inventory of Contracted Services: Actions Needed to Help Ensure Inventory Data Are Complete and Accurate, GAO-16-46 (Washington, D C : November 18, 2015) Sequestration: Comprehensive and Updated Cost Savings Would Better Inform DOD Decision Makers if Future Civilian Furloughs Occur, GAO-14-529 (Washington, D C : June 17, 2014) Human Capital: Opportunities Exist to Further Improve DOD’s Methodology for Estimating the Costs of Its Workforces, GAO-13-792 (Washington, D C : September 25, 2013) Human Capital: Additional Steps Needed to Help Determine the Right Size and Composition of DOD’s Total Workforce, GAO-13-470 (Washington, D C : May 29, 2013) Defense Outsourcing: Better Data Needed to Support Overhead Rates for A-76 Studies, GAO/NSIAD-98-62 (Washington, D C : Feb  27, 1998)


Report 2: IT systems supporting federal agencies and our nation’s critical infrastructures are inherently at risk  These systems are highly complex and dynamic, technologically diverse, and often geographically dispersed  This complexity increases the difficulty in identifying, managing, and protecting the numerous operating systems, applications, and devices comprising the systems and networks  Compounding the risk, federal systems and networks are also often interconnected with other internal and external systems and networks, including the Internet  This increases the number of avenues of attack and expands their attack surface  As systems become more integrated, cyber threats will pose an increasing risk to national security, economic well-being, and public health and safety  Advancements in technology, such as data analytics software for searching and collecting information, have also made it easier for individuals and organizations to correlate data (including PII) and track it across large and numerous databases  For example, social media has been used as a mass communication tool where PII can be gathered in vast amounts  In addition, ubiquitous Internet and cellular connectivity makes it easier to track individuals by allowing easy access to information pinpointing their locations  These advances—combined with the increasing sophistication of hackers and others with malicious intent, and the extent to which both federal agencies and private companies collect sensitive information about individuals—have increased the risk of PII being exposed and compromised  Cybersecurity incidents continue to impact entities across various critical infrastructure sectors  For example, in its 2018 annual data breach investigations report, Verizon reported that 53,308 security incidents and 2,216 data breaches were identified across 65 countries in the 12 months since its prior report  Further, the report noted that cybercriminals can often compromise a system in just a matter of minutes—or even seconds, but that it can take an organization significantly longer to discover the breach  Specifically, the report stated nearly 90 percent of the reported breaches occurred within minutes, while nearly 70 percent went undiscovered for months  These concerns are further highlighted by the number of information security incidents reported by federal executive branch civilian agencies to DHS’s U S  Computer Emergency Readiness Team (US-CERT)  For fiscal year 2017, 35,277 such incidents were reported by the Office of Management and Budget (OMB) in its 2018 annual report to Congress, as mandated by the Federal Information Security Modernization Act (FISMA)  These incidents include, for example, web-based attacks, phishing, and the loss or theft of computing equipment  Different types of incidents merit different response strategies  However, if an agency cannot identify the threat vector (or avenue of attack), it could be difficult for that agency to define more specific handling procedures to respond to the incident and take actions to minimize similar future attacks  In this regard, incidents with a threat vector categorized as “other” (which includes avenues of attacks that are unidentified) made up 31 percent of the various incidents reported to US-CERT  Figure 1 shows the percentage of the different types of incidents reported across each of the nine threat vector categories for fiscal year 2017, as reported by OMB  These incidents and others like them can pose a serious challenge to economic, national, and personal privacy and security  The following examples highlight the impact of such incidents: In March 2018, the Mayor of Atlanta, Georgia, reported that the city was victimized by a ransomware cyberattack  As a result, city government officials stated that customers were not able to access multiple applications that are used to pay bills or access court related information  In response to the attack, the officials noted that they were working with numerous private and governmental partners, including DHS, to assess what occurred and determine how best to protect the city from future attacks  In March 2018, the Department of Justice reported that it had indicted nine Iranians for conducting a massive cybersecurity theft campaign on behalf of the Islamic Revolutionary Guard Corps  According to the department, the nine Iranians allegedly stole more than 31 terabytes of documents and data from more than 140 American universities, 30 U S  companies, and five federal government agencies, among other entities  In March 2018, a joint alert from DHS and the Federal Bureau of Investigation (FBI) stated that, since at least March 2016, Russian government actors had targeted the systems of multiple U S  government entities and critical infrastructure sectors  Specifically, the alert stated that Russian government actors had affected multiple organizations in the energy, nuclear, water, aviation, construction, and critical manufacturing sectors  In July 2017, a breach at Equifax resulted in the loss of PII for an estimated 148 million U S  consumers  According to Equifax, the hackers accessed people’s names, Social Security numbers (SSN), birth dates, addresses and, in some instances, driver’s license numbers  In April 2017, the Commissioner of the Internal Revenue Service (IRS) testified that the IRS had disabled its data retrieval tool in early March 2017 after becoming concerned about the misuse of taxpayer data  Specifically, the agency suspected that PII obtained outside the agency’s tax system was used to access the agency’s online federal student aid application in an attempt to secure tax information through the data retrieval tool  In April 2017, the agency began notifying taxpayers who could have been affected by the breach  In June 2015, OPM reported that an intrusion into its systems had affected the personnel records of about 4 2 million current and former federal employees  Then, in July 2015, the agency reported that a separate, but related, incident had compromised its systems and the files related to background investigations for 21 5 million individuals  In total, OPM estimated 22 1 million individuals had some form of PII stolen, with 3 6 million being a victim of both breaches  Safeguarding federal IT systems and the systems that support critical infrastructures has been a long-standing concern of GAO  Due to increasing cyber-based threats and the persistent nature of information security vulnerabilities, we have designated information security as a government-wide high-risk area since 1997  In 2003, we expanded the information security high-risk area to include the protection of critical cyber infrastructure  At that time, we highlighted the need to manage critical infrastructure protection activities that enhance the security of the cyber and physical public and private infrastructures that are essential to national security, national economic security, and/or national public health and safety  We further expanded the information security high-risk area in 2015 to include protecting the privacy of PII  Since then, advances in technology have enhanced the ability of government and private sector entities to collect and process extensive amounts of PII, which has posed challenges to ensuring the privacy of such information  In addition, high- profile PII breaches at commercial entities, such as Equifax, heightened concerns that personal privacy is not being adequately protected  Our experience has shown that the key elements needed to make progress toward being removed from the High-Risk List are top-level attention by the administration and agency leaders grounded in the five criteria for removal, as well as any needed congressional action  The five criteria for removal that we identified in November 2000 are as follows: Leadership Commitment  Demonstrated strong commitment and top leadership support  Capacity  The agency has the capacity (i e , people and resources) to resolve the risk(s)  Action Plan  A corrective action plan exists that defines the root cause, solutions, and provides for substantially completing corrective measures, including steps necessary to implement solutions we recommended  Monitoring  A program has been instituted to monitor and independently validate the effectiveness and sustainability of corrective measures  Demonstrated Progress  Ability to demonstrate progress in implementing corrective measures and in resolving the high-risk area  These five criteria form a road map for efforts to improve and ultimately address high-risk issues  Addressing some of the criteria leads to progress, while satisfying all of the criteria is central to removal from the list  Figure 2 shows the five criteria and illustrative actions taken by agencies to address the criteria  Importantly, the actions listed are not “stand alone” efforts taken in isolation from other actions to address high- risk issues  That is, actions taken under one criterion may be important to meeting other criteria as well  For example, top leadership can demonstrate its commitment by establishing a corrective action plan including long-term priorities and goals to address the high-risk issue and using data to gauge progress—actions which are also vital to monitoring criteria  As we reported in the February 2017 high-risk report, the federal government’s efforts to address information security deficiencies had fully met one of the five criteria for removal from the High-Risk List— leadership commitment—and partially met the other four, as shown in figure 3  We plan to update our assessment of this high-risk area against the five criteria in February 2019  Based on our prior work, we have identified four major cybersecurity challenges: (1) establishing a comprehensive cybersecurity strategy and performing effective oversight, (2) securing federal systems and information, (3) protecting cyber critical infrastructure, and (4) protecting privacy and sensitive data  To address these challenges, we have identified 10 critical actions that the federal government and other entities need to take (see figure 4)  The four challenges and the 10 actions needed to address them are summarized following the table  In addition, we also discuss in more detail each of the 10 actions in appendices II through XI  The federal government has been challenged in establishing a comprehensive cybersecurity strategy and in performing effective oversight as called for by federal law and policy  Specifically, we have previously reported that the federal government has faced challenges in establishing a comprehensive strategy to provide a framework for how the United States will engage both domestically and internationally on cybersecurity related matters  We have also reported on challenges in performing oversight, including monitoring the global supply chain, ensuring a highly skilled cyber workforce, and addressing risks associated with emerging technologies  The federal government can take four key actions to improve the nation’s strategic approach to, and oversight of, cybersecurity  Develop and execute a more comprehensive federal strategy for national cybersecurity and global cyberspace  In February 2013 we reported that the government had issued a variety of strategy- related documents that addressed priorities for enhancing cybersecurity within the federal government as well as for encouraging improvements in the cybersecurity of critical infrastructure within the private sector; however, no overarching cybersecurity strategy had been developed that articulated priority actions, assigned responsibilities for performing them, and set time frames for their completion  In October 2015, in response to our recommendation to develop an overarching federal cybersecurity strategy that included all key elements of the desirable characteristics of a national strategy, the Director of OMB and the Federal Chief Information Officer issued a Cybersecurity Strategy and Implementation Plan for the Federal Civilian Government  The plan directed a series of actions to improve capabilities for identifying and detecting vulnerabilities and threats, enhance protections of government assets and information, and further develop robust response and recovery capabilities to ensure readiness and resilience when incidents inevitably occur  The plan also identified key milestones for major activities, resources needed to accomplish milestones, and specific roles and responsibilities of federal organizations related to the strategy’s milestones  Since that time, the executive branch has made progress toward outlining a federal strategy for confronting cyber threats  For example, a May 2017 presidential executive order required federal agencies to take a variety of actions, including better manage their cybersecurity risks and coordinate to meet reporting requirements related to cybersecurity of federal networks, critical infrastructure, and the nation  Additionally, the December 2017 National Security Strategy cites cybersecurity as a national priority and identifies related needed actions, such as including identifying and prioritizing risk, and building defensible government networks  Further, DHS issued a cybersecurity strategy in May 2018, which articulated seven goals the department plans to accomplish in support of its mission related to managing national cybersecurity risks  The strategy is intended to provide DHS with a framework to execute its cybersecurity responsibilities during the next 5 years to keep pace with the evolving cyber risk landscape by reducing vulnerabilities and building resilience; countering malicious actors in cyberspace; responding to incidents; and making the cyber ecosystem more secure and resilient  These efforts provide a good foundation toward establishing a more comprehensive strategy, but more effort is needed to address all of the desirable characteristics of a national strategy that we have previously recommended  The recently issued executive branch strategy documents did not include key elements of desirable characteristics that can enhance the usefulness of a national strategy as guidance for decision makers in allocating resources, defining policies, and helping to ensure accountability  Specifically, the documents generally did not include milestones and performance measures to gauge results, nor did they describe the resources needed to carry out the goals and objective  Further, most of the strategy documents lacked clearly defined roles and responsibilities for key agencies, such as DHS, the Department of Defense (DOD), and OMB, who contribute substantially to the nation’s cybersecurity programs  Ultimately, a more clearly defined, coordinated, and comprehensive approach to planning and executing an overall strategy would likely lead to significant progress in furthering strategic goals and lessening persistent weaknesses  For more information on this action area, see appendix II  Mitigate global supply chain risks  The global, geographically disperse nature of the producers and suppliers of IT products is a growing concern  We have previously reported on potential issues associated with IT supply chain and risks originating from foreign- manufactured equipment  For example, in July 2017, we reported that the Department of State had relied on certain device manufacturers, software developers, and contractor support which had suppliers that were reported to be headquartered in a cyber-threat nation (e g , China and Russia)  We further pointed out that the reliance on complex, global IT supply chains introduces multiple risks to federal agencies, including insertion of counterfeits, tampering, or installation of malicious software or hardware  In July 2018, we testified that if such global IT supply chain risks are realized, they could jeopardize the confidentiality, integrity, and availability of federal information systems  Thus, the potential exists for serious adverse impact on an agency’s operations, assets, and employees  These factors highlight the importance and urgency of federal agencies appropriately assessing, managing, and monitoring IT supply chain risk as part of their agency-wide information security programs  For more information on this action area, see appendix III  Address cybersecurity workforce management challenges  The federal government faces challenges in ensuring that the nation’s cybersecurity workforce has the appropriate skills  For example, in June 2018, we reported on federal efforts to implement the requirements of the Federal Cybersecurity Workforce Assessment Act of 2015  We determined that most of the Chief Financial Officers (CFO) Act agencies had not fully implemented all statutory requirements, such as developing procedures for assigning codes to cybersecurity positions  Further, we have previously reported that DHS and DOD had not addressed cybersecurity workforce management requirements set forth in federal laws  In addition, we have reported in the last 2 years that federal agencies (1) had not identified and closed cybersecurity skills gaps, (2) had been challenged with recruiting and retaining qualified staff, and (3) had difficulty navigating the federal hiring process  A recent executive branch report also discussed challenges associated with the cybersecurity workforce  Specifically, in response to Executive Order 13800, the Department of Commerce and DHS led an interagency working group exploring how to support the growth and sustainment of future cybersecurity employees in the public and private sectors  In May 2018, the departments issued a report that identified key findings, including: the U S  cybersecurity workforce needs immediate and sustained improvements; the pool of cybersecurity candidates needs to be expanded through retraining and by increasing the participation of women, minorities, and veterans; a shortage exists of cybersecurity teachers at the primary and secondary levels, faculty in higher education, and training instructors; and comprehensive and reliable data about cybersecurity workforce position needs and education and training programs are lacking  The report also included recommendations and proposed actions to address the findings, including that private and public sectors should (1) align education and training with employers’ cybersecurity workforce needs by applying the National Initiative for Cybersecurity Education Cybersecurity Workforce Framework; (2) develop cybersecurity career model paths; and (3) establish a clearinghouse of information on cybersecurity workforce development education, training, and workforce development programs and initiatives  In addition, in June 2018, the executive branch issued a government reform plan and reorganization recommendations that included, among other things, proposals for solving the federal cybersecurity workforce shortage  In particular, the plan notes that the administration intends to prioritize and accelerate ongoing efforts to reform the way that the federal government recruits, evaluates, selects, pays, and places cyber talent across the enterprise  The plan further states that, by the end of the first quarter of fiscal year 2019, all CFO Act agencies, in coordination with DHS and OMB, are to develop a critical list of vacancies across their organizations  Subsequently, OMB and DHS are to analyze these lists and work with OPM to develop a government-wide approach to identifying or recruiting new employees or reskilling existing employees  Regarding cybersecurity training, the plan notes that OMB is to consult with DHS to standardize training for cybersecurity employees, and should work to develop an enterprise-wide training process for government cybersecurity employees  For more information on this action area, see appendix IV  Ensure the security of emerging technologies  As the devices used in daily life become increasingly integrated with technology, the risk to sensitive data and PII also grows  Over the last several years, we have reported on weaknesses in addressing vulnerabilities associated with emerging technologies, including: IoT devices, such as fitness trackers, cameras, and thermostats, that continuously collect and process information are potentially vulnerable to cyber-attacks; IoT devices, such as those acquired and used by DOD employees or that DOD itself acquires (e g , smartphones), may increase the security risks to the department; vehicles that are potentially susceptible to cyber-attack through technology, such as Bluetooth; the unknown impact of artificial intelligence cybersecurity; and advances in cryptocurrencies and blockchain technologies  Executive branch agencies have also highlighted the challenges associated with ensuring the security of emerging technologies  Specifically, in a May 2018 report issued in response to Executive Order 13800, the Department of Commerce and DHS issued a report on the opportunities and challenges in reducing the botnet threat  The opportunities and challenges are centered on six principal themes, including the global nature of automated, distributed attacks; effective tools; and awareness and education  The report also provides recommended actions, including that federal agencies should increase their understanding of what software components have been incorporated into acquired products and establish a public campaign to support awareness of IoT security  For more information on this action area, see appendix V  In our previously discussed reports related to this cybersecurity challenge, we made a total of 50 recommendations to federal agencies to address the weaknesses identified  As of August 2018, 48 recommendations had not been implemented  These outstanding recommendations include 8 priority recommendations, meaning that we believe that they warrant priority attention from heads of key departments and agencies  These priority recommendations include addressing weaknesses associated with, among other things, agency-specific cybersecurity workforce challenges and agency responsibilities for supporting mitigation of vehicle network attacks  Until our recommendations are fully implemented, federal agencies may be limited in their ability to provide effective oversight of critical government-wide initiatives, address challenges with cybersecurity workforce management, and better ensure the security of emerging technologies  In addition to our prior work related to the federal government’s efforts to establish key strategy documents and implement effective oversight, we also have several ongoing reviews related to this challenge  These include reviews of: the CFO Act agencies’ efforts to submit complete and reliable baseline assessment reports of their cybersecurity workforces; the extent to which DOD has established training standards for cyber mission force personnel, and efforts the department has made to achieve its goal of a trained cyber mission force; and selected agencies’ ability to implement cloud service technologies and notable benefits this might have on agencies  The federal government has been challenged in securing federal systems and information  Specifically, we have reported that federal agencies have experienced challenges in implementing government-wide cybersecurity initiatives, addressing weaknesses in their information systems and responding to cyber incidents on their systems  This is particularly concerning given that the emergence of increasingly sophisticated threats and continuous reporting of cyber incidents underscores the continuing and urgent need for effective information security  As such, it is important that federal agencies take appropriate steps to better ensure they have effectively implemented programs to protect their information and systems  We have identified three actions that the agencies can take  Improve implementation of government-wide cybersecurity initiatives  Specifically, in January 2016, we reported that DHS had not ensured that the National Cybersecurity Protection System (NCPS) had fully satisfied all intended system objectives related to intrusion detection and prevention, information sharing, and analytics  In addition, in February 2017, we reported that the DHS National Cybersecurity and Communications Integration Center’s (NCCIC) functions were not being performed in adherence with the principles set forth in federal laws  We noted that, although NCCIC was sharing information about cyber threats in the way it should, the center did not have metrics to measure that the information was timely, relevant and actionable, as prescribed by law  For more information on this action area, see appendix VI  Address weaknesses in federal information security programs  We have previously identified a number of weaknesses in agencies’ protection of their information and information systems  For example, over the past 2 years, we have reported that: most of the 24 agencies covered by the CFO Act had weaknesses in each of the five major categories of information system controls (i e , access controls, configuration management controls, segregation of duties, contingency planning, and agency-wide security management); three agencies—the Securities Exchange Commission, the Federal Deposit Insurance Corporation, and the Food and Drug Administration—had not effectively implemented aspects of their information security programs, which resulted in weaknesses in these agencies’ security controls; information security weaknesses in selected high-impact systems at four agencies—the National Aeronautics and Space Administration, the Nuclear Regulatory Commission, OPM, and the Department of Veterans Affairs—were cited as a key reason that the agencies had not effectively implemented elements of their information security programs; DOD’s process for monitoring the implementation of cybersecurity guidance had weaknesses and resulted in the closure of certain tasks (such as completing cyber risk assessments) before they were fully implemented; and agencies had not fully defined the role of their Chief Information Security Officers, as required by FISMA  We also recently testified that, although the government had acted to protect federal information systems, additional work was needed to improve agency security programs and cyber capabilities  In particular, we noted that further efforts were needed by agencies to implement our prior recommendations in order to strengthen their information security programs and technical controls over their computer networks and systems  For more information on this action area, see appendix VII  Enhance the federal response to cyber incidents  We have reported that certain agencies have had weaknesses in responding to cyber incidents  For example, as of August 2017, OPM had not fully implemented controls to address deficiencies identified as a result of its 2015 cyber incidents; DOD had not identified the National Guard’s cyber capabilities (e g , computer network defense teams) or addressed challenges in its exercises; as of April 2016, DOD had not identified, clarified, or implemented all components of its support of civil authorities during cyber incidents; and as of January 2016, DHS’s NCPS had limited capabilities for detecting and preventing intrusions, conducting analytics, and sharing information  For more information on this action area, see appendix VIII  In the public versions of the reports previously discussed for this challenge area, we made a total of 101 recommendations to federal agencies to address the weaknesses identified  As of August 2018, 61 recommendations had not been implemented  These outstanding recommendations include 14 priority recommendations to address weaknesses associated with, among other things, the information security programs at the National Aeronautics and Space Administration, OPM, and the Security Exchange Commission  Until these recommendations are implemented, these federal agencies will be limited in their ability to ensure the effectiveness of their programs for protecting information and systems  In addition to our prior work, we also have several ongoing reviews related to the federal government’s efforts to protect its information and systems  These include reviews of: Federal Risk and Authorization Management Program (FedRAMP) implementation, including an assessment of the implementation of the program’s authorization process for protecting federal data in cloud environments; the Equifax data breach, including an assessment of federal oversight of credit reporting agencies’ collection, use, and protection of consumer PII; the Federal Communication Commission’s Electronic Comment Filing System security, to include a review of the agency’s detection of and response to a May 2017 incident that reportedly impacted the system; DOD’s efforts to improve the cybersecurity of its major weapon DOD’s whistleblower program, including an assessment of the policies, procedures, and controls related to the access and storage of sensitive and classified information needed for the program; IRS’s efforts to (1) implement security controls and the agency’s information security program, (2) authenticate taxpayers, and (3) secure tax information; and the federal approach and strategy to securing agency information systems, to include federal intrusion detection and prevention capabilities and the intrusion assessment plan  The federal government has been challenged in working with the private sector to protect critical infrastructure  This infrastructure includes both public and private systems vital to national security and other efforts, such as providing the essential services that underpin American society  As the cybersecurity threat to these systems continues to grow, federal agencies have millions of sensitive records that must be protected  Specifically, this critical infrastructure threat could have national security implications and more efforts should be made to ensure that it is not breached  To help address this issue, the National Institute of Standards and Technology (NIST) developed the cybersecurity framework—a voluntary set of cybersecurity standards and procedures for industry to adopt as a means of taking a risk-based approach to managing cybersecurity  However, additional action is needed to strengthen the federal role in protecting the critical infrastructure  Specifically, we have reported on other critical infrastructure protection issues that need to be addressed  For example: DHS did not track vulnerability reduction from the implementation and verification of planned security measures at the high-risk chemical facilities that engage with the department, as a basis for assessing performance  Entities within the 16 critical infrastructure sectors reported encountering four challenges to adopting the cybersecurity framework, such as being limited in their ability to commit necessary resources towards framework adoption and not having the necessary knowledge and skills to effectively implement the framework  DOD and the Federal Aviation Administration identified a variety of operations and physical security risks that could adversely affect DOD missions  Major challenges existed to securing the electricity grid against cyber threats  These challenges included monitoring implementation of cybersecurity standards, ensuring security features are built into smart grid systems, and establishing metrics for cybersecurity  DHS and other agencies needed to enhance cybersecurity in the maritime environment  Specifically, DHS did not include cyber risks in its risk assessments that were already in place nor did it address cyber risks in guidance for port security plans  Sector-specific agencies were not properly addressing progress or metrics to measure their progress in cybersecurity  For more information on this action area, see appendix IX  We made a total of 21 recommendations to federal agencies to address these weaknesses and others  These recommendations include, for example, a total of 9 recommendations to 9 sector-specific agencies to develop methods to determine the level and type of cybersecurity framework adoption across their respective sectors  As of August 2018, all 21 recommendations had not been implemented  Until these recommendations are implemented, the federal government will continue to be challenged in fulfilling its role in protecting the nation’s critical infrastructure  In addition to our prior work related to the federal government’s efforts to protect critical infrastructure, we also have several ongoing reviews focusing on: the physical and cybersecurity risks to pipelines across the country responsible for transmitting oil, natural gas, and other hazardous liquids; the cybersecurity risks to the electric grid; and the privatization of utilities at DOD installations  The federal government has been challenged in protecting privacy and sensitive data  Advances in technology, including powerful search technology and data analytics software, have made it easy to correlate information about individuals across large and numerous databases, which have become very inexpensive to maintain  In addition, ubiquitous Internet connectivity has facilitated sophisticated tracking of individuals and their activities through mobile devices such as smartphones and fitness trackers  Given that access to data is so pervasive, personal privacy hinges on ensuring that databases of PII maintained by government agencies or on their behalf are protected both from inappropriate access (i e , data breaches) as well as inappropriate use (i e , for purposes not originally specified when the information was collected)  Likewise, the trend in the private sector of collecting extensive and detailed information about individuals needs appropriate limits  The vast number of individuals potentially affected by data breaches at federal agencies and private sector entities in recent years increases concerns that PII is not being properly protected  Federal agencies should take two types of actions to address this challenge area  In addition, we have previously proposed two matters for congressional consideration aimed toward better protecting PII  Improve federal efforts to protect privacy and sensitive data  We have issued several reports noting that agencies had deficiencies in protecting privacy and sensitive data that needed to be addressed  For example: The Department of Health and Human Services’ (HHS) Centers for Medicare and Medicaid Services (CMS) and external entities were at risk of compromising Medicare Beneficiary Data due to a lack of guidance and proper oversight  The Department of Education’s Office of Federal Student Aid had not properly overseen its school partners’ records or information security programs  HHS had not fully addressed key security elements in its guidance for protecting the security and privacy of electronic health information  CMS had not fully protected the privacy of users’ data on state- based marketplaces  Poor planning and ineffective monitoring had resulted in the unsuccessful implementation of government initiatives aimed at eliminating the unnecessary collection, use, and display of SSNs  For more information on this action area, see appendix X  Appropriately limit the collection and use of personal information and ensure that it is obtained with appropriate knowledge or consent  We have issued a series of reports that highlight a number of the key concerns in this area  For example: The emergence of IoT devices can facilitate the collection of information about individuals without their knowledge or consent; Federal laws for smartphone tracking applications have not generally been well enforced; The FBI has not fully ensured privacy and accuracy related to the use of face recognition technology  For more information on this action area, see appendix XI  We have previously suggested that Congress consider amending laws, such as the Privacy Act of 1974 and the E-Government Act of 2002, because they may not consistently protect PII  Specifically, we found that while these laws and guidance set minimum requirements for agencies, they may not consistently protect PII in all circumstances of its collection and use throughout the federal government and may not fully adhere to key privacy principles  However, revisions to the Privacy Act and the E-Government Act have not yet been enacted  Further, we also suggested that Congress consider strengthening the consumer privacy framework and review issues such as the adequacy of consumers’ ability to access, correct, and control their personal information; and privacy controls related to new technologies such as web tracking and mobile devices  However, these suggested changes have not yet been enacted  We also made a total of 29 recommendations to federal agencies to address the weaknesses identified  As of August 2018, 28 recommendations had not been implemented  These outstanding recommendations include 6 priority recommendations to address weaknesses associated with, among other things, publishing privacy impact assessments and improving the accuracy of the FBI’s face recognition services  Until these recommendations are implemented, federal agencies will be challenged in their ability to protect privacy and sensitive data and ensure that its collection and use is appropriately limited  In addition to our prior work, we have several ongoing reviews related to protecting privacy and sensitive data  These include reviews of: IRS’s taxpayer authentication efforts, including what steps the agency is taking to monitor and improve its authentication methods; the extent to which the Department of Education’s Office of Federal Student Aid’s policies and procedures for overseeing non-school partners’ protection of federal student aid data align with federal requirements and guidance; data security issues related to credit reporting agencies, including a review of the causes and impacts of the August 2017 Equifax data breach; the extent to which Equifax assessed, responded to, and recovered from its August 2017 data breach; federal agencies’ efforts to remove PII from shared cyber threat indicators; and how the federal government has overseen Internet privacy, including the roles of the Federal Communications Commission and the Federal Trade Commission, and strengths and weaknesses of the current oversight authorities  In conclusion, since 2010, we have made over 3,000 recommendations to agencies aimed at addressing the four cybersecurity challenges  Nevertheless, many agencies continue to be challenged in safeguarding their information systems and information, in part because many of these recommendations have not been implemented  Of the roughly 3,000 recommendations made since 2010, nearly 1,000 had not been implemented as of August 2018  We have also designated 35 as priority recommendations, and as of August 2018, 31 had not been implemented  The federal government and the nation’s critical infrastructure are dependent on IT systems and electronic data, which make them highly vulnerable to a wide and evolving array of cyber-based threats  Securing these systems and data is vital to the nation’s security, prosperity, and well-being  Nevertheless, the security over these systems and data is inconsistent and urgent actions are needed to address ongoing cybersecurity and privacy challenges  Specifically, the federal government needs to implement a more comprehensive cybersecurity strategy and improve its oversight, including maintaining a qualified cybersecurity workforce; address security weaknesses in federal systems and information and enhance cyber incident response efforts; bolster the protection of cyber critical infrastructure; and prioritize efforts to protect individual’s privacy and PII  Until our recommendations are addressed and actions are taken to address the four challenges we identified, the federal government, the national critical infrastructure, and the personal information of U S  citizens will be increasingly susceptible to the multitude of cyber-related threats that exist  We are sending copies of this report to the appropriate congressional committees  In addition, the report is available at no charge on the GAO website at http://www gao gov  If you or your staff have any questions about this report, please contact Nick Marinos at (202) 512-9342 or marinosn@gao gov or Gregory C  Wilshusen at (202) 512-6244 or wilshuseng@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report  GAO staff who made key contributions to this report are listed in appendix XII  Critical Infrastructure Protection: DHS Should Take Actions to Measure Reduction in Chemical Facility Vulnerability and Share Information with First Responders  GAO-18-538  Washington, D C : August 8, 2018  High-Risk Series: Urgent Actions Are Needed to Address Cybersecurity Challenges Facing the Nation  GAO-18-645T  Washington, D C : July 25, 2018  Information Security: Supply Chain Risks Affecting Federal Agencies  GAO-18-667T  Washington, D C : July 12, 2018  Information Technology: Continued Implementation of High-Risk Recommendations Is Needed to Better Manage Acquisitions, Operations, and Cybersecurity  GAO-18-566T  Washington, D C : May 23, 2018  Cybersecurity: DHS Needs to Enhance Efforts to Improve and Promote the Security of Federal and Private-Sector Networks, GAO-18-520T  Washington, D C : April 24, 2018  Electronic Health Information: CMS Oversight of Medicare Beneficiary Data Security Needs Improvement  GAO-18-210  Washington, D C : March 6, 2018  Technology Assessment: Artificial Intelligence, Emerging Opportunities, Challenges, and Implications  GAO-18-142SP  Washington, D C : March 28, 2018  GAO Strategic Plan 2018-2023: Trends Affecting Government and Society  GAO-18-396SP  Washington, D C : February 22, 2018  Critical Infrastructure Protection: Additional Actions Are Essential for Assessing Cybersecurity Framework Adoption  GAO-18-211  Washington, D C : February 15, 2018  Cybersecurity Workforce: Urgent Need for DHS to Take Actions to Identify Its Position and Critical Skill Requirements  GAO-18-175  Washington, D C : February 6, 2018  Homeland Defense: Urgent Need for DOD and FAA to Address Risks and Improve Planning for Technology That Tracks Military Aircraft  GAO-18-177  Washington, D C : January 18, 2018  Federal Student Aid: Better Program Management and Oversight of Postsecondary Schools Needed to Protect Student Information  GAO-18-121  Washington, D C : December 15, 2017  Defense Civil Support: DOD Needs to Address Cyber Incident Training Requirements  GAO-18-47  Washington, D C : November 30, 2017  Federal Information Security: Weaknesses Continue to Indicate Need for Effective Implementation of Policies and Practices  GAO-17-549  Washington, D C : September 28, 2017  Information Security: OPM Has Improved Controls, but Further Efforts Are Needed  GAO-17-614  Washington, D C : August 3, 2017  Defense Cybersecurity: DOD’s Monitoring of Progress in Implementing Cyber Strategies Can Be Strengthened  GAO-17-512  Washington, D C : August 1, 2017  State Department Telecommunications: Information on Vendors and Cyber-Threat Nations  GAO-17-688R  Washington, D C : July 27, 2017  Internet of Things: Enhanced Assessments and Guidance Are Needed to Address Security Risks in DOD  GAO-17-668  Washington, D C : July 27, 2017  Information Security: SEC Improved Control of Financial Systems but Needs to Take Additional Actions  GAO-17-469  Washington, D C : July 27, 2017  Information Security: Control Deficiencies Continue to Limit IRS’s Effectiveness in Protecting Sensitive Financial and Taxpayer Data  GAO-17-395  Washington, D C : July 26, 2017  Social Security Numbers: OMB Actions Needed to Strengthen Federal Efforts to Limit Identity Theft Risks by Reducing Collection, Use, and Display  GAO-17-553  Washington, D C : July 25, 2017  Information Security: FDIC Needs to Improve Controls over Financial Systems and Information  GAO-17-436  Washington, D C : May 31, 2017  Technology Assessment: Internet of Things: Status and implications of an increasingly connected world  GAO-17-75  Washington, D C : May 15, 2017  Cybersecurity: DHS’s National Integration Center Generally Performs Required Functions but Needs to Evaluate Its Activities More Completely  GAO-17-163  Washington, D C : February 1, 2017  High-Risk Series: An Update  GAO-17-317  Washington, D C : February 2017  IT Workforce: Key Practices Help Ensure Strong Integrated Program Teams; Selected Departments Need to Assess Skill Gaps  GAO-17-8  Washington, D C : November 30, 2016  Electronic Health Information: HHS Needs to Strengthen Security and Privacy Guidance and Oversight  GAO-16-771  Washington, D C : September 26, 2016  Defense Civil Support: DOD Needs to Identify National Guard’s Cyber Capabilities and Address Challenges in Its Exercises  GAO-16-574  Washington, D C : September 6, 2016  Information Security: FDA Needs to Rectify Control Weaknesses That Place Industry and Public Health Data at Risk  GAO-16-513  Washington, D C : August 30, 2016  Federal Chief Information Security Officers: Opportunities Exist to Improve Roles and Address Challenges to Authority  GAO-16-686  Washington, D C : August 26, 2016  Federal Hiring: OPM Needs to Improve Management and Oversight of Hiring Authorities  GAO-16-521  Washington, D C : August 2, 2016  Information Security: Agencies Need to Improve Controls over Selected High-Impact Systems  GAO-16-501  Washington, D C : May 18, 2016  Face Recognition Technology: FBI Should Better Ensure Privacy and Accuracy  GAO-16-267  Washington, D C : May 16, 2016  Smartphone Data: Information and Issues Regarding Surreptitious Tracking Apps That Can Facilitate Stalking  GAO-16-317  Washington, D C : May 9, 2016  Vehicle Cybersecurity: DOT and Industry Have Efforts Under Way, but DOT Needs to Define Its Role in Responding to a Real-world Attack  GAO-16-350  Washington, D C : April 25, 2016  Civil Support: DOD Needs to Clarify Its Roles and Responsibilities for Defense Support of Civil Authorities during Cyber Incidents  GAO-16-332  Washington, D C : April 4, 2016  Healthcare gov: Actions Needed to Enhance Information Security and Privacy Controls  GAO-16-265  Washington, D C : March 23, 2016  Information Security: DHS Needs to Enhance Capabilities, Improve Planning, and Support Greater Adoption of Its National Cybersecurity Protection System  GAO-16-294  Washington, D C : January 28, 2016  Critical Infrastructure Protection: Sector-Specific Agencies Need to Better Measure Cybersecurity Progress  GAO-16-79  Washington, D C : November 19, 2015  Critical Infrastructure Protection: Cybersecurity of the Nation’s Electricity Grid Requires Continued Attention  GAO-16-174T  Washington, D C : October 21, 2015  Maritime Critical Infrastructure Protection: DHS Needs to Enhance Efforts to Address Port Cybersecurity  GAO-16-116T  Washington, D C : October 8, 2015  Cybersecurity: National Strategy, Roles, and Responsibilities Need to Be Better Defined and More Effectively Implemented  GAO-13-187  Washington, D C : February 14, 2014  Information Resellers: Consumer Privacy Framework Needs to Reflect Changes in Technology and the Marketplace  GAO-13-663  Washington, D C : September 25, 2013  Privacy: Alternatives Exist for Enhancing Protection of Personally Identifiable Information  GAO-08-536  Washington, D C : May 19, 2008  Federal law and policy call for a risk-based approach to managing cybersecurity within the government, as well as globally  We have previously reported that the federal government has faced challenges in establishing a comprehensive strategy to provide a framework for how the United States will engage both domestically and internationally on cybersecurity related matters  More specifically, in February 2013, we reported that the government had issued a variety of strategy-related documents that addressed priorities for enhancing cybersecurity within the federal government as well as for encouraging improvements in the cybersecurity of critical infrastructure within the private sector; however, no overarching cybersecurity strategy had been developed that articulated priority actions, assigned responsibilities for performing them, and set time frames for their completion  Accordingly, we recommended that the White House Cybersecurity Coordinator in the Executive Office of the President develop an overarching federal cybersecurity strategy that included all key elements of the desirable characteristics of a national strategy including, among other things, milestones and performance measures for major activities to address stated priorities; cost and resources needed to accomplish stated priorities; and specific roles and responsibilities of federal organizations related to the strategy’s stated priorities  In response to our recommendation, in October 2015, the Director of OMB and the Federal Chief Information Officer, issued a Cybersecurity Strategy and Implementation Plan for the Federal Civilian Government  The plan directed a series of actions to improve capabilities for identifying and detecting vulnerabilities and threats, enhance protections of government assets and information, and further develop robust response and recovery capabilities to ensure readiness and resilience when incidents inevitably occur  The plan also identified key milestones for major activities, resources needed to accomplish milestones, and specific roles and responsibilities of federal organizations related to the strategy’s milestones  Since that time, the executive branch has made progress toward outlining a federal strategy for confronting cyber threats  Table 1 identifies these recent efforts and a description of their related contents  These efforts provide a good foundation toward establishing a more comprehensive strategy, but more effort is needed to address all of the desirable characteristics of a national strategy that we recommended  The recently issued executive branch strategy documents did not include key elements of desirable characteristics that can enhance the usefulness of a national strategy as guidance for decision makers in allocating resources, defining policies, and helping to ensure accountability  Specifically: Milestones and performance measures to gauge results were generally not included in strategy documents  For example, although the DHS Cybersecurity Strategy stated that its implementation would be assessed on an annual basis, it did not describe the milestones and performance measures for tracking the effectiveness of the activities intended to meet the stated goals (e g , protecting critical infrastructure and responding effectively to cyber incidents)  Without such performance measures, DHS will lack a means to ensure that the goals and objectives discussed in the document are accomplished and that responsible parties are held accountable  According to officials from DHS’s Office of Cybersecurity and Communications, the department is developing a plan for implementing the DHS Cybersecurity Strategy and expects to issue the plan by the end of calendar year 2018  The officials stated that the plan is expected to identify milestones, roles, and responsibilities across DHS to inform the prioritization of future efforts  The strategy documents generally did not include information regarding the resources needed to carry out the goals and objectives  For example, although the DHS Cybersecurity Strategy identified a variety of actions the agency planned to take to perform their cybersecurity mission, it did not articulate the resources needed to carry out these actions and requirements  Without information on the specific resources needed, federal agencies may not be positioned to allocate such resources and investments and, therefore, may be hindered in their ability meet national priorities  Most of the strategy documents lacked clearly defined roles and responsibilities for key agencies, such as DHS, DOD, and OMB  These agencies contribute substantially to the nation’s cybersecurity programs  For example, although the National Security Strategy discusses multiple priority actions needed to address the nation’s cybersecurity challenges (e g , building defensible government networks, and deterring and disrupting malicious cyber actors), it does not describe the roles, responsibilities, or the expected coordination of any specific federal agencies, including DHS, DOD, or OMB, or other non-federal entities needed to carry out those actions  Without this information, the federal government may not be able foster effective coordination, particularly where there is overlap in responsibilities, or hold agencies accountable for carrying out planned activities  Ultimately, a more clearly defined, coordinated, and comprehensive approach to planning and executing an overall strategy would likely lead to significant progress in furthering strategic goals and lessening persistent weaknesses  The exploitation of information technology (IT) products and services through the supply chain is an emerging threat  IT supply chain-related threats can be introduced in the manufacturing, assembly, and distribution of hardware, software, and services  Moreover, these threats can appear at each phase of the system development life cycle, when an agency initiates, develops, implements, maintains, and disposes of an information system  As a result, the compromise of an agency’s IT supply chain can degrade the confidentiality, integrity, and availability of its critical and sensitive networks, IT-enabled equipment, and data  Federal regulation and guidance issued by the National Institute of Standards and Technology (NIST) set requirements and best practices for mitigating supply chain risks  The Federal Acquisition Regulation established codification and publication of uniform policies and procedures for acquisition by all executive branch agencies  Agencies are required by the Federal Acquisition Regulation to ensure that contracts include quality requirements that are determined necessary to protect the government’s interest  In addition, the NIST guidance on supply chain risk management practices for federal information systems and organizations intends to assist federal agencies with identifying, assessing, and mitigating information and communications technology supply chain risks at all levels of their organizations  We have previously reported on risks to the IT supply chain and risks originating from foreign-manufactured equipment  For example: In July 2018, we testified that if global IT supply chain risks are realized, they could jeopardize the confidentiality, integrity, and availability of federal information systems  Thus, the potential exists for serious adverse impact on an agency’s operations, assets, and employees  We further stated that in 2012 we determined that four national security-related agencies—the Departments of Defense, Justice, Energy, Homeland Security (DHS)—varied in the extent to which they had addressed supply chain risks  We recommended that three agencies take eight actions, as needed, to develop and document policies, procedures, and monitoring capabilities that address IT supply chain risk  The agencies generally concurred with the recommendations and subsequently implemented seven recommendations and partially implemented the eighth recommendation  In July 2017, we reported that, based on a review of a sample of organizations within the Department of State’s telecommunications supply chain, we were able to identify instances in which device manufacturers, software developers and contractor support were reported to be headquartered in a leading cyber-threat nation  For example, of the 52 telecommunications device manufacturers and software developers in our sample, we were able to identify 12 that had 1 or more suppliers that were reported to be headquartered in a leading cyber-threat nation  We noted that the reliance on complex, global IT supply chains introduces multiple risks to federal agencies, including insertion of counterfeits, tampering, or installation of malicious software or hardware  Figure 5 illustrates possible manufacturing locations of typical network components  Although federal agencies have taken steps to address IT supply chain deficiencies that we previously identified, this area continues to be a potential threat vector for malicious actors to target the federal government  For example, in September 2017, DHS issued a binding operating directive which calls on departments and agencies to identify any use or presence of Kaspersky products on their information systems and to develop detailed plans to remove and discontinue present and future use of the products  DHS expressed concern about the ties between certain Kaspersky officials and Russian intelligence and other government agencies, and requirements under Russian law that allow Russian intelligence agencies to request or compel assistance from Kaspersky and to intercept communications transiting Russian networks  On May 11, 2017, the President issued an executive order on strengthening the cybersecurity of federal networks and critical infrastructure  The order makes it the policy of the United States to support the growth and sustainment of a workforce that is skilled in cybersecurity and related fields as the foundation for achieving our objectives in cyberspace  It directed the Secretaries of Commerce and Homeland Security (DHS), in consultation with other federal agencies, to assess the scope and sufficiency of efforts to educate and train the American cybersecurity workforce of the future, including cybersecurity- related education curricula, training, and apprenticeship programs, from primary through higher education  Nevertheless, the federal government continues to face challenges in addressing the nation’s cybersecurity workforce  Agencies had not effectively conducted baseline assessments of their cybersecurity workforce or fully developed procedures for coding positions  In June 2018, we reported that 21 of the 24 agencies covered by the Chief Financial Officer’s Act had conducted and submitted to Congress a baseline assessment identifying the extent to which their cybersecurity employees held professional certifications, as required by the Federal Cybersecurity Workforce Assessment Act of 2015  However, we found that the results of these assessments may not have been reliable because agencies did not address all of the reportable information and agencies were limited in their ability to obtain complete and consistent information about their cybersecurity employees and the certifications they held  We determined that this was because agencies had not yet fully identified all members of their cybersecurity workforces or did not have a consistent list of appropriate certifications for cybersecurity positions  Further, 23 of the agencies reviewed had established procedures for identifying and assigning the appropriate employment codes to their civilian cybersecurity positions, as called for by the act  However, 6 of the 23 did not address one or more of 7 activities required by OPM in their procedures, such as reviewing all filled and vacant positions and annotating reviewed position descriptions with the appropriate employment code  Accordingly, we made 30 recommendations to 13 agencies to fully implement two of the act’s requirements on baseline assessments and coding procedures  The extent to which these agencies agreed with the recommendations varied  DHS and the Department of Defense (DOD) had not addressed cybersecurity workforce management requirements set forth in federal laws  In February 2018, we reported that, while DHS had taken actions to identify, categorize, and assign employment codes to its cybersecurity positions, as required by the Homeland Security Cybersecurity Workforce Assessment Act of 2014, its actions were not timely and complete  For example, DHS did not establish timely and complete procedures to identify, categorize, and code its cybersecurity position vacancies and responsibilities  Further, DHS had not yet completed its efforts to identify all of its cybersecurity positions and accurately assign codes to all filled and vacant cybersecurity positions  Table 2 shows DHS’s progress in implementing the requirements of the Homeland Security Cybersecurity Workforce Assessment Act of 2014, as of December 2017  Accordingly, we recommended that DHS take six actions, including ensuring that its cybersecurity workforce procedures identify position vacancies and responsibilities; reported workforce data are complete and accurate; and plans for reporting on critical needs are developed  DHS agreed with our six recommendations, but had not implemented them as of August 2018  Regarding DOD, in November 2017, we reported that instead of developing a comprehensive plan for U S  Cyber Command, the department submitted a report consisting of a collection of documents that did not fully address the required six elements set forth in Section 1648 of the National Defense Authorization Act for Fiscal Year 2016  More specifically, DOD’s 1648 report did not address an element related to cyber incident training  In addition to not addressing the training element in the report, DOD had not ensured that staff were trained as required by the Presidential Policy Directive on United States Cyber Incident Coordination or DOD’s Significant Cyber Incident Coordination Procedures  Accordingly, we made two recommendations to DOD to address these issues  DOD agreed with one of the recommendations and partially agreed with the other, citing ongoing activities related to cyber incident coordination training it believed were sufficient  However, we continued to believe the recommendation was warranted  As of August 2018, both recommendations had not yet been implemented  Agencies had not identified and closed cybersecurity skills gaps  In November 2016, we reported that five selected agencies had made mixed progress in assessing their information technology (IT) skill gaps  These agencies had started focusing on identifying cybersecurity staffing gaps, but more work remained in assessing competency gaps and in broadening the focus to include the entire IT community  Accordingly, we made a total of five recommendations to the agencies to address these issues  Four agencies agreed and one, DOD, partially agreed with our recommendations citing progress made in improving its IT workforce planning  However, we continued to believe our recommendation was warranted  As of August 2018, all five of the recommendations had not been implemented  Agencies had been challenged with recruiting and retaining qualified staff  In August 2016, we reported on the current authorities chief information security officers (CISO) at 24 agencies  Among other things, CISOs identified key challenges they faced in fulfilling their responsibilities  Several of these challenges were related to the cybersecurity workforce, such as not having enough personnel to oversee the implementation of the number and scope of security requirements  In addition, CISOs stated that they were not able to offer salaries that were competitive with the private sector for candidates with high-demand technical skills  Furthermore, CISOs stated that certain security personnel lacked the skill sets needed or were not sufficiently trained  To assist CISOs in carrying out their responsibilities and better define their roles, we made a total of 34 recommendations to the Office of Management and Budget (OMB) and 13 agencies in our review  Agency responses to the recommendations varied; as of August 2018, 18 of the 34 recommendations had not been implemented  Agencies have had difficulty navigating the federal hiring process  In August 2016, we reported on the extent to which federal hiring authorities were meeting agency needs  Although competitive hiring has been the traditional method of hiring, agencies can use additional hiring authorities to expedite the hiring process or achieve certain public policy goals  Among other things, we noted that agencies rely on a relatively small number of hiring authorities (as established by law, executive order, or regulation) to fill the vast majority of hires into the federal civil service  Further, while OPM collects a variety of data to assess the federal hiring process, neither it nor agencies used this information to assess the effectiveness of hiring authorities  Conducting such assessments would be a critical first step in making more strategic use of the available hiring authorities to more effectively meet their hiring needs  Accordingly, we made three recommendations to OPM to work with agencies to strengthen hiring efforts  OPM generally agreed with the recommendations; however, as of August 2018, two of them had not been implemented  The emergence of new technologies can potentially introduce security vulnerabilities for those technologies which were previous unknown  As we have previously reported, additional processes and controls will need to be developed to potentially address these new vulnerabilities  While some progress has been made to address the security and privacy issues associated with these technologies, such as the Internet of Things (IoT) and vehicle networks, there is still much work to be done  For example: IoT devices that continuously collect and process information are potentially vulnerable to cyber-attacks  In May 2017, we reported that the IoT has become increasingly used to communicate and process vast amounts of information using “smart” devices (such as fitness trackers, cameras, and thermostats)  However, we noted that this emerging technology also presents new issues in areas such as information security, privacy, and safety  For example, IoT devices, networks, or the cloud servers where they store data can be compromised in a cyberattack  Table 3 provides examples of cyber- attacks that could affect IoT devices and networks  IoT devices may increase the security risks to federal agencies  In July 2017, we reported that IoT devices, such as those acquired and used by Department of Defense (DOD) employees or that DOD itself acquires (e g , smartphones), may increase the security risks to the department  We noted that these risks can be divided into two categories, risks with the devices themselves, such as limited encryption, and risks with how they are used, such as unauthorized communication of information  The department has also identified notional threat scenarios, based on input from multiple DOD entities, which exemplify how these security risks could adversely impact DOD operations, equipment, or personnel  Figure 6 highlights a few examples of these scenarios  In addition, we reported that DOD had started to examine the security risks of IoT devices, but that the department had not conducted required assessments related to the security of its operations  Further, DOD had issued policies and guidance for these devices, but these did not clearly address all of the risks relating to these devices  To address these issues, we made two recommendations to DOD  The department agreed with our recommendations; however, as of August 2018, they had not yet been implemented  Vehicles are potentially susceptible to cyber-attack through networks, such as Bluetooth  In March 2016, we reported that many stakeholders in the automotive industry acknowledge that in-vehicle networks pose a threat to the safety of the driver, as an external attacker could gain control to critical systems in the car  Further, these industry stakeholders agreed that critical systems and other vehicle systems, such as a Bluetooth connection, should be separate in-vehicle networks so they could not communicate or interfere with one another  Figure 7 identifies the key interfaces that could be exploited in a vehicle cyber-attack  To enhance the Department of Transportation’s ability to effectively respond in the event of a real-world vehicle cyberattack, we made one recommendation to the department to better define its roles and responsibilities  The department agreed with the recommendation but, as of August 2018, had not yet taken action to implement it  Artificial intelligence holds substantial promise for improving cybersecurity, but also posed new risks  In March 2018, we reported on the results of a forum we convened to discuss emerging opportunities, challenges, and implications associated with artificial intelligence  At the forum, participants from industry, government, academia, and nonprofit organizations discussed the potential implications of this emerging technology, including assisting with cybersecurity by helping to identify and patch vulnerabilities and defending against attacks; creating safer automated vehicles; improving the criminal justice system’s allocation of resources; and improving how financial services govern investments  However, forum participants also highlighted a number of challenges and risks related to artificial intelligence  For example, if the data used by artificial intelligence are biased or become corrupted by hackers, the results could be biased or cause harm  Moreover, the collection and sharing of data needed to train artificial intelligence systems, a lack of access to computing resources, and adequate human capital were also challenges facing the development of artificial intelligence  Finally, forum participants noted that the widespread adoption raises questions about the adequacy of current laws and regulations  Cryptocurrencies provide an alternative to traditional government-issued currencies, but have security implications  In February 2018, we reported on trends affecting government and society, including the increased use of cryptocurrencies—digital representations of value that are not government-issued—that operate online and verify transactions using a public ledger called blockchain  We highlighted the potential benefits of this technology, such as anonymity and lower transaction costs, as well as drawbacks, including making it harder to detect money laundering and other financial crimes  Because of these capabilities and others, we noted the potential for virtual currencies and blockchain technology to reshape financial services and affect the security of critical financial infrastructures  Lastly, we pointed out that the use of blockchain technology could have more security vulnerabilities as computing power increases as a result of new advancements in quantum computing, an area of quantum information science  In January 2008, the President issued National Security Presidential Directive 54/Homeland Security Presidential Directive 23  The directive established the Comprehensive National Cybersecurity Initiative, a set of projects with the objective of safeguarding federal executive branch government information systems by reducing potential vulnerabilities, protecting against intrusion attempts, and anticipating future threats against the federal government’s networks  Under the initiative, the Department of Homeland Security (DHS) was to lead several projects to better secure civilian federal government networks  Specifically, the agency established the National Cybersecurity and Communications Integration Center (NCCIC), which functions as the 24/7 cyber monitoring, incident response, and management center  Figure 8 depicts the Watch Floor, which functions as a national focal point of cyber and communications incident integration  The United States Computer Emergency Readiness Team (US-CERT), one of several subcomponents of the NCCIC, is responsible for operating the National Cybersecurity Protection System (NCPS), which provides intrusion detection and prevention capabilities to entities across the federal government  Although DHS is fulfilling its statutorily required mission by establishing the NCCIC and managing the operation of NCPS, we have identified challenges in the agency’s efforts to manage these programs: DHS had not ensured that NCPS has fully satisfied all intended system objectives  In January 2016, we reported that NCPS had a limited ability to detect intrusions across all types of network types  In addition, we reported that the system’s intrusion prevention capability was limited and its information-sharing capability was not fully developed  Furthermore, we reported that DHS’s current metrics did not comprehensively measure the effectiveness of NCPS  Accordingly, we made nine recommendations to DHS to address these issues and others  The department agreed with our recommendations and has taken action to address one of them  However, as of August 2018, eight of these recommendations had not been implemented  DHS had been challenged in measuring how the NCCIC was performing its functions in accordance with mandated implementing principles  In February 2017, we reported instances where, with certain products and services, NCCIC had implemented its functions in adherence with one or more of its principles, as required by the National Cybersecurity Protection Act of 2014 and Cybersecurity Act of 2015  For example, consistent with the principle that it seek and receive appropriate consideration from industry sector-specific, academic, and national laboratory expertise, NCCIC coordinated with contacts from industry, academia, and the national laboratories to develop and disseminate vulnerability alerts  However, we also identified instances where the cybersecurity functions were not performed in adherence with the principles  For example, NCCIC is to provide timely technical assistance, risk management support, and incident response capabilities to federal and nonfederal entities, but it had not established measures or other procedures for ensuring the timeliness of these assessments  Further, we reported that NCCIC faces impediments to performing its cybersecurity functions more efficiently, such as tracking security incidents and working across multiple network platforms  Accordingly, we made nine recommendations to DHS related to implementing the requirements identified in the National Cybersecurity Protection Act of 2014 and the Cybersecurity Act of 2015  The department agreed with our recommendations and has taken action to address two of them  However, as of August 2018, the remaining seven recommendations had not been implemented  The Federal Information Security Modernization Act of 2014 (FISMA) requires federal agencies in the executive branch to develop, document, and implement an information security program and evaluate it for effectiveness  The act retains many of the requirements for federal agencies’ information security programs previously set by the Federal Information Security Management Act of 2002  These agency programs should include periodic risk assessments; information security policies and procedures; plans for protecting the security of networks, facilities, and systems; security awareness training; security control assessments; incident response procedures; a remedial action process, and continuity plans and procedures  In addition, Executive Order 13800 states that the President will hold agency heads accountable for managing cybersecurity risk to their enterprises  In addition, according to the order, it is the policy of the United States to manage cybersecurity risk as an executive branch enterprise because risk management decisions made by agency heads can affect the risk to the executive branch as a whole, and to national security  Over the past several years, we have performed numerous security control audits to determine how well agencies are managing information security risk to federal information systems and data through the implementation of effective security controls  These audits have resulted in the identification of hundreds of deficiencies related to agencies’ implementation of effective security controls  Accordingly, we provided agencies with limited official use only reports identifying technical security control deficiencies for their respective agency  In these reports, we made hundreds of recommendations related to improving agencies’ implementation of those security control deficiencies  In addition to systems and networks maintained by federal agencies, it is also important that agencies ensure the security of federal information systems operated by third party providers, including cloud service providers  Cloud computing is a means for delivering computing services via information technology networks  Since 2009, the government has encouraged agencies to use cloud-based services to store and process data as a cost-savings measure  In this regard, the Office of Management and Budget (OMB) established the Federal Risk and Authorization Management Program (FedRAMP) to provide a standardized approach to security assessment, authorization, and continuous monitoring for cloud products and services  FedRAMP is intended to ensure that cloud computing services have adequate information security, eliminate duplicative efforts, and reduce costs  Although there are requirements and government-wide programs to assist with ensuring the security of federal information systems maintained by federal agencies and third party providers, we have identified weaknesses in agencies’ implementation of information security programs  Federal agencies continued to experience weaknesses in protecting their information and information systems due to ineffective implementation of information security policies and practices  In September 2017, we reported that most of the 24 agencies covered by the Chief Financial Officers (CFO) Act had weaknesses in each of the five major categories of information system controls (i e , access controls, configuration management controls, segregation of duties, contingency planning, and agency-wide security management)  Weaknesses in these security controls indicate that agencies did not adequately or effectively implement information security policies and practices during fiscal year 2016  Figure 9 identifies the number of agencies with information security weaknesses in each of the five categories  In addition, we found that several agencies had not effectively implemented some aspects of its information security program, which resulted in weaknesses in these agencies’ security controls  In July 2017, we reported that the Security Exchange Commission did not always keep system security plans complete and accurate or fully implement continuous monitoring, as required by agency policy  We made two recommendations to the Security Exchange Commission to effectively manage its information security program  The agency agreed with our recommendations; however, as of August 2018, they had not been implemented  In another July 2017 report, we noted that the Internal Revenue Service (IRS) did not effectively support a risk-based decision to accept system deficiencies; fully develop, document, or update information security policies and procedures; update system security plans to reflect changes to the operating environment; perform effective tests and evaluations of policies, procedures, and controls; or address shortcomings in the agency’s remedial process  Accordingly, we made 10 recommendations to IRS to more effectively implement security-related policies and plans  The agency neither agreed nor disagreed with the recommendations; as of August 2018, all 10 recommendations had not been implemented  In May 2017, we reported that the Federal Deposit Insurance Corporation did not include all necessary information in procedures for granting access to a key financial application; fully address its Inspector General findings that security control assessments of outsourced service providers had not been completed in a timely manner; fully address key previously identified weaknesses related to establishing agency-wide configuration baselines and monitoring changes to critical server files; or complete actions to address the Inspector General’s finding that the Federal Deposit Insurance Corporation had not ensured that major security incidents are identified and reported in a timely manner  We made one recommendation to the agency to more fully implement its information security program  The agency agreed with our recommendation and has taken steps to implement it  In August 2016, we reported that the Food and Drug Administration did not fully implement certain security practices involved with assessing risks to systems; complete or review security policies and procedures in a timely manner; complete and review system security plans annually; always track and fully train users with significant security responsibilities; fully test controls or monitor them; remediate identified security weaknesses in a timely fashion based on risk; or fully implement elements of its incident response program  Accordingly, we issued 15 recommendations to the Food and Drug Administration to fully implement its agency-wide information security program  The agency agreed with our recommendations  As of August 2018, all 15 recommendations had been implemented  In May 2016, we reported that a key reason for the information security weaknesses in selected high-impact systems at four agencies—National Aeronautics and Space Administration, Nuclear Regulatory Commission, the Office of Personnel Management, and Department of Veterans Affairs—was that they had not effectively implemented elements of their information security programs  For example, most of the selected agencies had conducted information security control assessments for systems, but not all assessments were comprehensive  We also reported that remedial action plans developed by the agencies did not include all the required elements, and not all agencies had developed a continuous monitoring strategy  Table 4 identifies the extent to which the selected agencies implemented key aspects of their information security programs  Accordingly, we made 19 recommendations to the four selected agencies to correct these weaknesses  Agency responses to the recommendations varied  Further, as of August 2018, 16 of the 19 recommendations had not been implemented  DOD’s monitoring of progress in implementing cyber strategies varied  In August 2017, we reported that the DOD’s progress in implementing key strategic cybersecurity guidance—the DOD Cloud Computing Strategy, DOD Cyber Strategy, and DOD Cybersecurity Campaign—has varied  More specifically, we determined that the department had implemented the cybersecurity objectives identified in the DOD Cloud Computing Strategy and had made progress in implementing the DOD Cyber Strategy and DOD Cybersecurity Campaign  However, the department’s process for monitoring implementation of the DOD Cyber Strategy had resulted in the closure of tasks as implemented before the tasks were fully implemented  In addition, the DOD Cybersecurity Campaign lacked time frames for completion and a process to monitor progress, which together provide accountability to ensure implementation  We made two recommendations to improve DOD’s process of ensuring its cyber strategies are effectively implemented  The department partially concurred with these recommendations and identified actions it planned to take to address them  We noted that, if implemented, the actions would satisfy the intent of our recommendations  However, as of August 2018, DOD had not yet implemented our recommendations  Agencies had not fully defined the role of their Chief Information Security Officers (CISO), as required by FISMA  In August 2016, we reported that 13 of 24 agencies covered by the CFO Act had not fully defined the role of their CISO  For example, these agencies did not always identify a role for the CISO in ensuring that security controls are periodically tested; procedures are in place for detecting, reporting, and responding to security incidents; or contingency plans and procedures for agency information systems are in place  Thus, we determined that the CISOs’ ability to effectively oversee these agencies’ information security activities can be limited  To assist CISOs in carrying out their responsibilities and better define their roles, we made a total of 34 recommendations to OMB and 13 agencies in our review  Agency responses to the recommendations varied; as of August 2018, 18 of the 34 recommendations had not been implemented  Presidential Policy Directive-41 sets forth principles governing the federal government’s response to any cyber incident, whether involving government or private sector entities  According to the directive, federal agencies shall undertake three concurrent lines of effort when responding to any cyber incident: threat response; asset response; and intelligence support and related activities  In addition, when a federal agency is an affected entity, it shall undertake a fourth concurrent line of effort to manage the effects of the cyber incident on its operations, customers, and workforce  We have reviewed federal agencies’ preparation and response to cyber incidents and have identified the following weaknesses: The Office of Personnel Management (OPM) had not fully implemented controls to address deficiencies identified as a result of a cyber incident  In August 2017, we reported that OPM did not fully implement the 19 recommendations made by the Department of Homeland Security’s (DHS) United States Computer Emergency Readiness Team (US-CERT) after the data breaches in 2015  Specifically, we noted that, after breaches of personnel and background investigation information were reported, US-CERT worked with the agency to resolve issues and develop a comprehensive mitigation strategy  In doing so, US-CERT made 19 recommendations to OPM to help the agency improve its overall security posture and, thus, improve its ability to protect its systems and information from security breaches  In our August 2017 report, we determined that OPM had fully implemented 11 of the 19 recommendations  For the remaining 8 recommendations, actions for 4 were still in progress  For the other 4 recommendations, OPM indicated that it had completed actions to address them, but we noted further improvements were needed  Further, OPM had not validated actions taken to address the recommendations in a timely manner  As a result of our review, we made five other recommendations to OPM to improve its response to cyber incidents  The agency agreed with four of these and partially concurred with the one related to validating its corrective action  The agency did not cite a reason for its partial concurrence and we continued to believe that the recommendation was warranted  As of August 2018, three of the five recommendations had not been implemented  The Department of Defense (DOD) had not identified the National Guard’s cyber capabilities (e g , computer network defense teams) or addressed challenges in its exercises  In September 2016, we reported that DOD had not identified the National Guard’s cyber capabilities or addressed challenges in its exercises  Specifically, DOD had not identified and did not have full visibility into National Guard cyber capabilities that could support civil authorities during a cyber incident because the department has not maintained a database that identifies National Guard cyber capabilities, as required by the National Defense Authorization Act for Fiscal Year 2007  In addition, we identified three types of challenges with DOD’s cyber exercises that could limit the extent to which DOD is prepared to support civilian authorities in a cyber incident: limited access because of classified exercise environments; limited inclusion of other federal agencies and critical infrastructure owners; and inadequate incorporation of joint physical-cyber scenarios  In our September 2016 report, we noted that DOD had not addressed these challenges  Furthermore, we stated that DOD had not addressed its goals by conducting a “tier 1” exercise (i e , an exercise involving national-level organizations and combatant commanders and staff in highly complex environments), as stated in the DOD Cyber Strategy  Accordingly, we recommended that DOD (1) maintain a database that identifies National Guard cyber capabilities and (2) conduct a tier 1 exercise to prepare its forces in the event of a disaster with cyber effects  The department partially agreed with our recommendations, stating that its current mechanisms and exercises are sufficient to address the issues highlighted in our report  However, we continued to believe the recommendations were valid  As of August 2018, our two recommendations had not been implemented  DOD had not identified, clarified, or implemented all components of its incident response program  In April 2016, we also reported that DOD had not clarified its roles and responsibilities for defense support of civil authorities during cyber incidents  Specifically, we found that DOD’s overarching guidance about how it is to support civil authorities as part of its Defense Support of Civil Authorities mission did not clearly define the roles and responsibilities of key DOD entities, such as DOD components, the supported command, or the dual-status commander, if they are requested to support civil authorities in a cyber incident  Further, we found that, in some cases, DOD guidance provides specific details on other types of Defense Support of Civil Authorities-related responses, such as assigning roles and responsibilities for fire or emergency services support and medical support, but does not provide the same level of detail or assign roles and responsibilities for cyber support  Accordingly, we recommended that DOD issue or update guidance that clarifies DOD roles and responsibilities to support civil authorities in a domestic cyber incident  DOD concurred with the recommendation and stated that the department will issue or update guidance  However, as of August 2018, the department had not implemented our recommendation  DHS’s NCPS had limited capabilities for detecting and preventing intrusions, conducting analytics, and sharing information  In January 2016, we reported that NCPS had a limited ability to detect intrusions across all types of network types  In addition, we reported that the system’s intrusion prevention capability was limited and its information-sharing capability was not fully developed  Furthermore, we reported that DHS’s current metrics did not comprehensively measure the effectiveness of NCPS  Accordingly, we made nine recommendations to DHS to address these issues and others  The department agreed with our recommendations and has taken action to address one of them  However, as of August 2018, eight of these recommendations had not been implemented  The nation’s critical infrastructure include both public and private systems vital to national security and other efforts including providing the essential services, such as banking, water, and electricity—that underpin American society  The cyber threat to critical infrastructure continues to grow and represents a national security challenge  To address this cyber risk, the President issued Executive Order 13636 in February 2013 to enhance the security and resilience of the nation’s critical infrastructure and maintain a cyber environment that promotes safety, security, and privacy  In accordance with requirements in the executive order which were enacted into law in 2014, the National Institute of Standards and Technology (NIST) facilitated the development of a set of voluntary standards and procedures for enhancing cybersecurity of critical infrastructure  This process, which involved stakeholders from the public and private sectors, resulted in NIST’s Framework for Improving Critical Infrastructure Cybersecurity  The framework is to provide a flexible and risk-based approach for entities within the nation’s 16 critical infrastructure sectors to protect their vital assets from cyber-based threats  Since then, progress has been made to protect the critical infrastructure of the nation but we have reported that challenges to ensure the safety and security of our infrastructure exist  The Department of Homeland Security (DHS) had not measured the impact of its efforts to support cyber risk reduction for high- risk chemical sector entities  In August 2018, we reported that DHS had strengthened its processes for identifying high-risk chemical facilities and assigning them to tiers under its Chemical Facility Anti- Terrorism Standards program  However, we found that DHS’s new performance measure methodology did not measure reduction in vulnerability at a facility resulting from the implementation and verification of planned security measures during the compliance inspection process  We concluded that doing so would provide DHS an opportunity to begin assessing how vulnerability is reduced—and by extension, risk lowered—not only for individual high-risk facilities but for the Chemical Facility Anti-Terrorism Standards program as a whole  We also determined that, although DHS shares some Chemical Facility Anti-Terrorism Standards program information, first responders and emergency planners may not have all of the information they need to minimize the risk of injury or death when responding to incidents at high-risk facilities  This was due to first responders at the local level not having access or widely using a secure interface that DHS developed (known as the Infrastructure Protection Gateway) to obtain information about high-risk facilities and the specific chemicals they process  To address the weaknesses we identified, we recommended that DHS take actions to (1) measure reduction in vulnerability of high-risk facilities and use that data to assess program performance, and (2) encourage access to and wider use of the Infrastructure Protection Gateway among first responders and emergency planners  DHS concurred with both recommendations and outlined efforts underway or planned to address them  The federal government had identified major challenges to the adoption of the cybersecurity framework  In February 2018, we reported that there were four different challenges to adopting the cybersecurity framework, including limited resources and competing priorities, reported by entities within their sectors  We further reported that none of the 16 sector-specific agencies were measuring the implementation by these entities, nor did they have qualitative or quantitative measures of framework adoption  While research had been done to determine the use of the framework in the sectors, these efforts had yielded no real results for sector wide adoption  We concluded that, until sector-specific agencies understand the use of the framework by the implementing entities, their ability to understand implementation efforts would be limited  Accordingly, we made a total of nine recommendations to nine sector-specific agencies to address these issues  Five agencies agreed with the recommendations, while four others neither agreed nor disagreed; as of August 2018, all five recommendations had not been implemented  Agencies had not addressed risks to their systems and the information they maintain  In January 2018, we reported that the Department of Defense (DOD) and Federal Aviation Administration (FAA) identified a variety of operations and physical security risks related to Automatic Dependent Surveillance-Broadcast Out technology that could adversely affect DOD missions  These risks came from information broadcast by the system itself, as well as from potential vulnerabilities to electronic warfare- and cyber-attacks, and from the potential divestment of secondary-surveillance radars  However, DOD and FAA had not approved any solutions to address the risks they identified to the system  Accordingly, we recommended that DOD and FAA, among other things, take action to approve one or more solutions to address Automatic Dependent Surveillance- Broadcast Out-related security risks  DOD and FAA generally agreed with our recommendations; however, as of August 2018, they had not been implemented  Major challenges existed to securing the electricity grid against cyber threats  In October 2015, we testified on the status of the electricity grid’s cybersecurity, reporting that entities associated with the grid have encountered several challenges  We noted that these challenges included implementation monitoring, built-in security features in smart grid systems, and establishing metrics for cybersecurity  We concluded that continued attention to these issues and cyber threats in general was required to help mitigate these risks to the electricity grid  DHS and other agencies needed to enhance cybersecurity in the maritime environment  In October 2015, we testified on the status of the cybersecurity of our nation’s ports, concluding that steps needed to be taken to enhance their security  Specifically, we noted that DHS needed to include cyber risks in its risk assessments that are already in place as well as addressing cyber risks in guidance for port security plans  We concluded that, until DHS and the other stakeholders take steps to address cybersecurity in the ports, risk of a cyber-attack with serious consequences are increased  Sector-specific agencies were not properly addressing progress or metrics to measure their progress in cybersecurity  In November 2015, we reported that sector-specific agencies were not comprehensively addressing the cyber risk to the infrastructure, as 11 of the 15 sectors had significant cyber risk  Specifically, we noted that these entities had taken actions to mitigate their cyber risk; however, most had not identified incentives to promote cybersecurity in their sectors  We concluded that while the sector-specific agencies have successfully disseminated the information they possess, there was still work to be done to properly measure cybersecurity implementation progress  Accordingly, we made seven recommendations to six agencies to address these issues  Four of these agencies agreed with our recommendation, while two agencies did not comment on the recommendations  As of August 2018, all seven recommendations had not been implemented  Advancements in technology, such as new search technology and data analytics software for searching and collecting information, have made it easier for individuals and organizations to correlate data and track it across large and numerous databases  In addition, lower data storage costs have made it less expensive to store vast amounts of data  Also, ubiquitous Internet and cellular connectivity make it easier to track individuals by allowing easy access to information pinpointing their locations  the effectiveness of these procedures  Based on a survey of the schools, the majority of the schools had policies in place for records retention but the way these policies were implemented was highly varied for paper and electronic records  We also found that the oversight of the school’s programs was lacking, as Federal Student Aid conducts reviews but does not consider information security as a factor for selecting schools  out provisions of the Patient Protection and Affordable Care Act  We made three recommendations to CMS related to defining procedures for overseeing the security of state-based marketplaces and requiring continuous monitoring of state marketplace controls  HHS concurred with our recommendations  As of August 2018, two of the recommendations had not yet been implemented  Poor planning and ineffective monitoring had resulted in the unsuccessful implementation of government initiatives designed to protect federal data  In July 2017, we reported that government initiatives aimed at eliminating the unnecessary collection, use, and display of Social Security numbers (SSN) have had limited success  Specifically, in agencies’ response to our questionnaire on SSN reduction efforts, the 24 agencies covered by the Chief Financial Officers Act reported successfully curtailing the collection, use, and display of SSNs  Nevertheless, all of the agencies continued to rely on SSNs for important government programs and systems, as seen in figure 10  Given that access to data is so pervasive, personal privacy hinges on ensuring that databases of personally identifiable information (PII) maintained by government agencies or on their behalf are protected both from inappropriate access (i e , data breaches) as well as inappropriate use (i e , for purposes not originally specified when the information was collected)  Likewise, the trend in the private sector of collecting extensive and detailed information about individuals needs appropriate limits  The vast number of individuals potentially affected by data breaches at federal agencies and private sector entities in recent years increases concerns that PII is not being properly protected  The emergence of IoT devices can facilitate the collection of information about individuals without their knowledge or consent  In May 2017, we reported that the IoT has become increasingly used to communicate and process vast amounts of information using “smart” devices (such as a fitness tracker connected to a smartphone)  However, we noted that this emerging technology also presents new issues in areas such as information security, privacy, and safety  Smartphone tracking apps can present serious safety and privacy risks  In April 2016, we reported on smartphone applications that facilitated the surreptitious tracking of a smartphone’s location and other data  Specifically, we noted that some applications could be used to intercept communications and text messages, essentially facilitating the stalking of others  While it is illegal to use these applications for these purposes, stakeholders differed over whether current federal laws needed to be strengthened to combat stalking  We also noted that stakeholders expressed concerns over what they perceived to be limited enforcement of laws related to tracking apps and stalking  In particular, domestic violence groups stated that additional education of law enforcement officials and consumers about how to protect against, detect, and remove tracking apps is needed  The Federal Bureau of Investigation (FBI) has not ensured privacy and accuracy related to the use of face recognition technology  In May 2016, we reported that the Department of Justice had not been timely in publishing and updating privacy documentation for the FBI’s use of face recognition technology  Publishing such documents in a timely manner would better assure the public that the FBI is evaluating risks to privacy when implementing systems  Also, the FBI had taken limited steps to determine whether the face recognition system it was using was sufficiently accurate  We recommended that the department ensure required privacy-related documents are published and that the FBI test and review face recognition systems to ensure that they are sufficiently accurate  Of the six recommendations we made, the Department of Justice agreed with one, partially agreed with two, and disagreed with three  We continued to believe all the recommendations made were valid  As of August 2018, the six recommendations had not been implemented  In addition to the contacts named above, Jon Ticehurst, Assistant Director; Kush K  Malhotra, Analyst-In-Charge; Chris Businsky; Alan Daigle; Rebecca Eyler; Chaz Hubbard; David Plocher; Bradley Roach; Sukhjoot Singh; Di’Mond Spencer; and Umesh Thakkar made key contributions to this report 

ground_truth: In addition to more than 2.2 million active duty and reserve personnel, DOD employs about 760,000 federal civilians and more than 560,000 contractors. In the Senate Report 114-49 accompanying a bill for the National Defense Authorization Act for Fiscal Year 2016 included a provision for DOD to issue a report (1) assessing functions performed by federal civilian and service contractor personnel, (2) accounting for the full costs of federal civilian and service contractor personnel performing these functions, (3) comparing these costs, and (4) assessing available hiring and retention authorities for federal civilians. The Senate report also included a provision for GAO to assess DOD's report, which DOD submitted to Congress in April 2017. This report examines the extent to which DOD's report addressed the prescribed congressional elements. GAO reviewed DOD's report and compared it to the prescribed elements, examined documents relevant to DOD's cost estimating and comparison methodology, and interviewed DOD officials, including those in its Office of Cost Assessment and Program Evaluation responsible for the calculations in DOD's report. In response to Congressional direction, the Department of Defense (DOD) issued a report in April 2017 comparing the costs of federal civilian and service contractor personnel at select installations. The report addressed three out of four provision elements and partially addressed one, as discussed below. DOD concluded that neither federal civilians nor service contractors were predominately more or less expensive, with costs being dependent on position, location, and level of seniority. DOD noted that it used a non-probability based sample of personnel for its report, and the results are not generalizable. An assessment of performance of functions being performed by federal civilian and service contractor personnel at six military installations, with four being in the continental United States and two being outside the continental United States. GAO believes that DOD addressed this requirement because it developed a methodology to assess performance of functions performed by federal civilians and service contractors at 17 organizations within nine geographic regions including two locations outside the continental United States. DOD used data from the Defense Civilian Personnel Data System to identify military installations with large reported numbers of federal civilians. DOD determined that personnel need to perform at least 80 percent common tasks to be able to make a comparison. An accounting of the fully-burdened, or full, cost of federal civilian and service contractor personnel performing functions at the selected installations including training, benefits, reimbursable costs, and facility overhead. GAO believes that DOD partially addressed this requirement because while it calculated the labor costs of selected federal civilian and service contractor full-time equivalents performing similar functions for organizations at government-owned facilities, it excluded certain non-labor costs from its calculations. A comparison of the costs of performance of these functions by federal civilians and service contractor personnel at the selected installations. GAO believes that DOD addressed this requirement because it compared calculated costs for selected federal civilians and service contractors performing similar functions at selected installations and included those comparisons in its report. An assessment of the flexible employment authorities for the employment and retention of federal civilian employees. GAO believes that DOD addressed this requirement because it sent questionnaires to DOD hiring officials and human resource professionals to collect information on flexible employment authorities and conducted interviews with these and human resource professionals at the same 17 organizations used for the cost comparison. Based on an analysis of the information collected, DOD's report included several conclusions regarding flexible hiring authorities and made one recommendation. GAO is not making any recommendations; however, DOD non-concurred with GAO's assessment that DOD partially addressed the element to account for the full cost of personnel. GAO believes the assessment is correct as discussed in the report.

prompt_front: You are given two reports by the government agency.



Question: Write a one-page summary of the report 2.

Report 1: The federal government has recognized 573 Indian tribes as distinct, independent political communities with tribal sovereignty  There are different categories of tribal lands, with differing implications with respect to ownership and administration  Reservations are defined geographic areas with established boundaries recognized by the United States  Tribal lands vary in size, demographics, and location  For example, those lands smallest in size are less than one square mile, and the largest, the Navajo Nation, is more than 24,000 square miles (the size of West Virginia)  Tribal land locations can range from extremely remote, rural locations to urban areas  Figure 1 shows tribal lands in the United States according to the 2010 Census  The term “broadband” commonly refers to Internet access that is high speed and provides an “always-on” connection, so users do not have to reestablish a connection each time they access the Internet  Broadband service may be “fixed”—that is, providing service to a single location, such as a customer’s home—or “mobile,” that is, providing service wherever a customer has access to a mobile wireless network, including while on the move, through a mobile device, such as a smartphone  Fixed and mobile broadband providers deploy and maintain infrastructure to connect consumers to the Internet  Providers offer fixed Internet service through a number of technologies, such as copper phone lines, fiber-optic lines, coaxial cables, wireless antennas, satellites, or a mix of technologies (see fig  2)  To install fixed or wireless infrastructure, providers must obtain permits from government entities with jurisdiction over the land or permission from public utilities to deploy infrastructure on existing utility poles  The federal government has emphasized the importance of ensuring Americans have access to broadband, and a number of agencies, including FCC, currently provide funding to subsidize broadband deployment in areas in which the return on investment has not attracted private investment  The Communications Act of 1934, as amended by the Telecommunications Act of 1996, specifies that consumers in “rural, insular, and high-cost areas” should have access to telecommunication services and rates that are “reasonably comparable” to consumers in urban areas  To achieve this goal, FCC administers the High-Cost program, which provides subsidies to providers of phone service in rural, insular, and other remote areas  In 2011, FCC launched a series of reforms to its High-Cost program, including adding support for broadband services, and created the Connect America Fund, which provides subsidies to fixed and mobile providers of telecommunications and broadband services in rural, insular, and other remote areas where the costs of providing service is high  To be eligible for Universal Service Fund support from FCC, a provider must be designated an Eligible Telecommunications Carrier by the appropriate state or by FCC and must meet certain service obligations  The Connect America Fund has distributed approximately $4 5 billion per year, and has separate funding mechanisms targeted to specific goals  For example, there are funds for fixed-phone and broadband service and funds for mobile service, including a Tribal Mobility Fund (Phase 1) that awarded nearly $50 million in 2014 for the provision of 3G and 4G service to unserved tribal areas  In addition to FCC, a number of other agencies provide funding for broadband deployment in unserved or underserved areas  For example, the United States Department of Agriculture’s Community Connect Program, which provides grants to rural communities to provide high- speed Internet service to unserved areas  The American Recovery and Reinvestment Act of 2009 (Recovery Act) mandated the development of a nationwide map of broadband availability  To implement the act, the National Telecommunications & Information Administration (NTIA)—an agency within the Department of Commerce—established a grant program to enable U S  states and territories to collect state-level broadband mapping data  NTIA used these data to launch the National Broadband Map (www broadbandmap gov) in February 2011  As the funding for the NTIA’s program came to an end in 2014, NTIA stopped collecting data to update the map and, according to FCC officials, created a memorandum of understanding with FCC through which FCC agreed to maintain public access to the last version of the map  FCC issued rules in 2013 to begin collecting broadband deployment data, in addition to the broadband subscription data it had collected from providers since 2000  FCC sought, but did not receive, $3 million to update the National Broadband Map in its fiscal year 2015 and fiscal year 2016 budgets  In 2018, Congress directed FCC to develop a report by March 23, 2019, evaluating broadband coverage in certain tribal lands (to include an assessment of areas that have adequate broadband coverage, as well as an assessment of unserved areas), and to complete a proceeding to address unserved areas by September 23, 2020  Currently, FCC requires broadband providers to report on their broadband deployment by filing a form twice a year (Form 477)  Fixed broadband providers submit a list of the census blocks in which their broadband service is available, and mobile providers submit “shapefiles”—a geospatial depiction of the coverage area, which FCC refers to as “polygons”—of their coverage areas  FCC uses providers’ 477 data to develop a statutorily mandated annual report on advanced telecommunications capability  In addition, in 2016, FCC began publishing its own maps of broadband deployment, using the information from providers’ Form 477 filings  In February 2018, FCC launched an updated map of fixed broadband deployment (https://broadbandmap fcc gov/#/)  This map allows users to search for broadband deployment by address and provides summary-level statistics regarding broadband deployment in specific tribal lands (see fig  3)  According to FCC officials, this new map format will support more frequent data updates  FCC also provides national maps of mobile LTE coverage; these maps do not allow users to access data at the same level of granularity as the maps of fixed broadband (see fig  4)  FCC collects and uses data that capture broadband availability to measure broadband access on tribal lands, leading to overstatements of broadband access on tribal lands  Specifically, FCC’s method of collecting mobile and fixed broadband data from providers (the Form 477) does not accurately or completely capture broadband access on tribal lands because it (1) captures nationwide broadband availability data— areas where providers may have broadband infrastructure—but does so in a way that leads to overstatements of availability, and (2) does not capture information on factors that FCC and tribal stakeholders have stated can affect broadband access on tribal lands, such as affordability, service quality, and denials of service  Nonetheless, FCC uses its Form 477 broadband availability data in annual broadband deployment reports to measure the percentage of Americans living on tribal lands with or without access to broadband, and to measure progress toward FCC’s strategic goal of increasing all Americans’ access to affordable broadband  By using broadband availability data to measure broadband access on tribal lands, FCC overstates broadband access on tribal lands  FCC’s Form 477, its primary method of collecting nationwide broadband data, collects information on broadband availability, which identifies where providers have broadband infrastructure and could potentially provide broadband services but not where consumers can actually access those services  Moreover, the Form 477’s mobile broadband data- collection methods are not standardized, and its fixed broadband data- collection methods are not sufficiently granular to provide information about broadband availability on tribal lands  FCC’s Form 477 requires mobile broadband providers to report their coverage areas by submitting geospatial data depicting the areas in which consumers could expect to receive the minimum advertised speed  FCC has previously noted the importance of collecting nationally standardized, uniform broadband data from providers to assess broadband availability and allow for easy comparison across providers  However, the Form 477 does not require that providers use a standardized method with defined technical parameters (such as signal strength, or amount of interference) when determining their coverage area, resulting in data that cannot be meaningfully compared across providers, according to FCC  To map their coverage areas, providers may use predictive models based on different measurement methods and a variety of factors known to affect mobile broadband service such as topography, tree cover, and buildings, among other factors  Providers and tribal stakeholders have expressed concern with the accuracy of FCC’s mobile broadband data, and FCC has acknowledged concerns that the lack of a standardized method resulted in data that were unreliable for the purposes of determining mobile broadband coverage for specific geographic areas, such as tribal lands  About half of the tribal government representatives we interviewed told us that they believe FCC’s data overstate mobile LTE broadband availability on their lands  For example, a few representatives expressed concerns with the accuracy of the mobile data in areas with varied terrain, such as mountains and valleys  In comments to FCC, broadband providers have also raised concerns regarding the accuracy of the mobile coverage data generated by the Form 477 for the purposes of identifying areas eligible for funding through FCC’s Mobility Fund Phase II program, which provides federal funding to increase mobile broadband services in unserved areas  In 2017, in response to such concerns, FCC reversed its prior decision to use the Form 477 data to identify specific areas eligible for federal funding through the Mobility Fund Phase II program  Instead, FCC undertook a one-time special data collection, for which it required providers to measure their coverage based on a common set of standards, in order to better identify unserved areas that would be presumptively eligible for funding  FCC plans to allow parties, including tribal governments, to challenge the data where they believe the data overstate mobile broadband coverage through August, 2018  Additionally, in an August 2017 Notice of Proposed Rulemaking, FCC requested comment on potential changes to modernize its Form 477 data collection, including whether it should require all providers to use a standardized method when submitting mobile coverage data on the form  FCC officials told us that they do not have a timeline for the development of a final rule, and as of August 2018, FCC had not yet issued a final rule on modernizing the Form 477  The Form 477 collects fixed broadband data that are not sufficiently granular to accurately depict broadband availability on tribal lands  Specifically, FCC directs fixed broadband providers to submit a list of census blocks where service is available on the Form 477  FCC defines “available” as whether the provider does—or could, within a typical service interval or without an extraordinary commitment of resources— provide service to at least one end-user premises in a census block  Thus, in its annual reports and maps of fixed broadband service, FCC considers an entire block to be served if a provider reports that it does, or could offer, service to at least one household in the census block  FCC does not define a typical service interval or an extraordinary commitment of resources in its Form 477 instructions  However, FCC officials stated that providers should not report service in areas in which major construction would be required to provide service  A few providers told us that the lack of clear guidance from FCC regarding how to determine where broadband is available has led different providers to interpret the Form 477 directions in different ways, which can affect the accuracy and consistency of reporting from provider to provider  For example, in a filing with FCC, one provider stated that it had misapplied the definition of “available” and, as a result, overstated the availability of its services by almost 3,000 census blocks  As shown in figure 5, FCC’s definition of availability leads to overstatements of fixed broadband availability on tribal lands by: (1) counting an entire census block as served if only one location has broadband, and (2) allowing providers to report availability in blocks where they do not have any infrastructure connecting homes to their networks if the providers determine they could offer service to at least one household  Almost all the providers and private companies, and most of the representatives of tribal governments and organizations we spoke with told us that due to these issues, FCC’s definition of availability results in data that overstate broadband availability  According to FCC officials, FCC requires providers to report fixed broadband availability where they could provide service within a “typical service interval” and without “an extraordinary commitment of resources” in order to: (1) ensure that it captures instances in which a provider has a network nearby but has not installed the last connection to the homes, and (2) identify where service is connected to homes, but homes have not subscribed  FCC officials also told us that FCC measures availability at the census block level because sub-census block data may be costly to collect  In 2013, FCC considered collecting more granular nationwide data on broadband deployment but decided against collecting these data because it determined that the burden would outweigh the benefit  However, FCC, tribal stakeholders, and providers have noted that FCC’s approach leads to overstatements of availability  For example, in its 2017 Notice of Proposed Rulemaking on modernizing the Form 477 data collection, FCC acknowledged that by requiring a provider to report where it could provide service, it is impossible to tell whether the provider would be unable or unwilling to take on additional subscribers in a census block it lists as served  According to FCC, this limits the value of the data to inform FCC policies  In addition, several providers and tribal stakeholders we interviewed said that some “digital subscriber line” (DSL) and fixed wireless providers may overstate their service areas on the Form 477 because they may not take into account technological or terrain limitations that would affect their ability to actually provide service  FCC has also recognized that by measuring availability at the census block level, not every person will have access to broadband in a block that the data show as served, and FCC has noted that in rural areas, such as tribal lands, census blocks can be large and providers may only deploy service to a portion of the census block  A few representatives for tribal governments and organizations noted that the use of census blocks may uniquely overstate broadband availability on tribal lands when census blocks contain both tribal and non-tribal areas, because availability in the non-tribal portion of the block can result in the tribal area of the census block also being counted as served  FCC is considering requiring providers to report whether they are willing and able to serve additional customers in a census block and collecting sub-census block data in its 2017 proposed rulemaking on modernizing the Form 477  About one-third of the parties that commented on FCC’s proposals were not in favor of FCC collecting these more granular data on the Form 477, stating that the data would be less accurate and more burdensome for providers to collect and report, among other reasons, and questioned whether more detailed information on nationwide broadband availability is necessary  We heard similar concerns from a few of the providers and trade associations we interviewed  However, about one- third of the parties that commented on FCC’s proposals were in favor of collecting more granular data, stating that such data would be more useful for policymakers and more accurate  Additionally, a few tribally owned and non-tribal providers we interviewed told us that providers already maintain data for business purposes that would allow them to report more granular information on broadband availability  One stakeholder we spoke with pointed out that, as the federal government and states work to ensure the last remaining unserved areas—rural, low- population density areas including tribal lands—have service, sub- census-block-level data are needed to ensure that governments are making wise and accurate investments  FCC does not collect information on several factors that FCC and tribal stakeholders have stated can affect broadband access  FCC and tribal stakeholders have noted that broadband access can be affected by factors such as the affordability and quality of the broadband services being offered, and the extent to which providers deny service to those who request it  By collecting and using data on factors that can affect broadband access, FCC would have more complete information on the extent to which Americans living on tribal lands have access to broadband Internet services  Affordability: FCC has noted that affordability of broadband services can affect broadband access but does not collect information on the cost of broadband service on tribal lands on the Form 477  For example, in the National Broadband Plan, FCC cited affordable access to robust broadband service as a long-term goal, and in its Strategic Plan 2018–2022, FCC acknowledged that affordability is an important factor affecting broadband access and a key driver of the digital divide  Moreover, most of the representatives of tribal governments and organizations we spoke to told us that the affordability of broadband services is an important factor for understanding whether or not people on tribal lands could realistically access broadband services  Tribal government officials from one tribe we spoke with told us that residents on their lands cannot access broadband because it is too costly  For example, a provider that advertises services on the tribe’s land charges $130 per month for broadband services, approximately one-and-a-half times the average rate providers charge for comparable services in urban areas, according to FCC (see fig  6)  In the 2018 Broadband Deployment Report, FCC acknowledged that affordability can influence a consumer’s decision on whether to purchase broadband, but FCC did not consider cost in its assessment of broadband access on tribal lands, stating that pricing does not go to the congressional requirement to assess deployment and availability in conducting its inquiry as required by Congress under section 706 of the Telecommunications Act and also citing a lack of reliable comprehensive data on this issue  In addition, FCC officials we interviewed acknowledged that while broadband service may be technically available, it may be prohibitively expensive for some, which may make availability alone an incomplete indicator of broadband access  Quality of Service: In the Telecommunications Act of 1996 Congress recognized the importance of service quality by defining advanced telecommunications capability as any technology that enables users to originate and receive high-quality voice, data, graphics, and video telecommunications  In keeping with this legislation, FCC has consistently set thresholds for speeds that qualify as broadband services and has stated that “latency” and consistency of service figure prominently into whether a broadband service is able to provide advanced capabilities and thus whether users can access high-quality telecommunications  Likewise, almost all of the representatives for tribal governments or organizations we interviewed told us that quality of service is a key component of access to broadband and that routine outages, slow speeds, and high latency keep people on tribal lands from consistently accessing the Internet  Most tribal stakeholders and a few providers we interviewed told us that factors such as terrain, weather, and type of technology can all affect the quality of service an end user receives and, ultimately, the subscribers’ ability to access the Internet (see fig  7)  For example, some representatives of tribal governments and organizations told us issues like oversubscription— when a provider signs up more customers than its equipment can handle—and outdated or limited infrastructure result in low-quality services that cannot support advanced and, in some cases, basic functions  Though FCC uses the Form 477 to collect some data on advertised speeds from providers, FCC does not collect data on actual speeds, service outages, and latency on the form  In its 2018 Broadband Deployment Report, FCC stated that it did not consider FCC data on actual speed, latency, or consistency of service when evaluating broadband access due to the lack of appropriate data  FCC noted that the lack of Form 477 data on actual speeds in particular constrained evaluation of mobile broadband access  Service Denials: FCC has recognized that information on denials of service is pertinent to understanding actual broadband access but does not collect data on service denials in the Form 477  Specifically, in the National Broadband Plan, FCC recommended that FCC collect data to determine whether broadband service is being denied to potential residential customers based on the income of the residents in a particular geographic area  Some representatives of the tribal governments or organizations told us that that they were aware of a provider denying service to residents of tribal lands, despite the provider reporting broadband availability on at least a portion of those lands, according to our analysis of the Form 477 data  These representatives told us that they believed service was denied because of disputes with the tribal government, low demand for service, or the high costs of extending services to the home on tribal lands  Some representatives of tribal governments or organizations we spoke with also told us that providers may have denied service because their equipment was at capacity and could not accommodate new users (see fig  8)  For example, on three of the tribal lands we visited, we observed fiber optic cable located close to government and residential structures that did not have broadband access via fiber  According to tribal government officials, despite the physical proximity of the fiber optic cable, the tribal government and residents could not access it because the provider was not offering service or was unwilling or unable to build to the structures  A few providers we interviewed stated that they may not provide services to individuals who request them because of high-costs, administrative barriers, or technical limitations  However, FCC does not collect data on service denials on the Form 477  In its Strategic Plan 2018–2022 and the National Broadband Plan, FCC identified increasing all Americans’ access to affordable broadband as a long-term, strategic goal  Congress has similarly directed FCC to develop policies and programs aimed at increasing access to affordable broadband in all regions of the United States, including tribal lands, and required FCC to report annually on its progress  According to the Government Performance and Results Act (GPRA), as enhanced by the GPRA Modernization Act of 2010 (GPRAMA), agencies should use accurate and reliable data to measure progress toward achieving their goals  Additionally, Standards for Internal Control in the Federal Government state that agencies should use quality information— information that is complete, appropriate, and reliable—to inform decision-making processes and evaluate the agency’s performance in achieving goals  According to these standards, agencies should also communicate quality information externally to achieve the agency’s goals  However, FCC has used its Form 477 data, which do not accurately or completely measure broadband access on tribal lands, as its primary source to evaluate progress toward FCC’s strategic goal of increasing broadband access and to develop maps and reports intended to depict broadband access on tribal lands  For example, in its 2018 Broadband Deployment Report, FCC found that 64 6 percent of Americans residing on tribal lands have access to fixed broadband services  By using these data, FCC has overstated the extent to which Americans living on tribal lands can actually access broadband Internet services and FCC’s progress toward increasing broadband access  As a result, the digital divide may appear less significant as a national challenge, and FCC and tribal stakeholders working to target broadband funding to unserved or underserved tribal lands will be limited in their ability to make informed decisions  This increases the risk that residents living on tribal lands will continue to lack broadband access  Some tribal officials stated that inaccurate data have affected their ability to plan their own broadband networks and obtain federal broadband funding, and most of the tribal stakeholders we interviewed identified a pressing need for accurate data on the gaps in broadband access on tribal lands in order to ensure that tribes can qualify for federal funding and to effectively target the areas that need it most  For example, representatives for one tribal government that is providing broadband services said the government will not be able to use a federal grant to build broadband infrastructure in areas of their reservation that lack access, because the Form 477 data overstate actual access on the tribe’s land  As more than three quarters of the tribal governments we spoke to are working to provide broadband services on their lands in some capacity, overstating broadband access on tribal lands could affect the ability of a number of tribes to access federal funding to increase broadband access on their lands  As previously discussed, FCC is considering proposals to modify its Form 477 data collection as part of a 2017 Notice of Proposed Rulemaking, but FCC officials told us that the Commission does not have a timeline for issuance of a final rule  While some of FCC’s proposals could help address some of the limitations identified above by, for example, collecting more granular nationwide broadband availability data, FCC has not addressed specifically the collection of more accurate and complete data on broadband access for tribal lands in this proceeding  FCC has identified the need to improve broadband data for tribal lands in particular, and as previously noted, in 2018 Congress directed FCC to develop a report evaluating broadband coverage in certain tribal lands and initiate a proceeding to address the unserved areas identified in the report  FCC officials told us that FCC has not determined how it will address this requirement, but it is currently considering its options, including potentially addressing the requirement as part of its ongoing proposed rulemaking on modernizing the Form 477 data collection  An evaluation of broadband coverage on tribal lands that relies on the current Form 477 data would be subject to the limitations described above, including the overstatement of broadband access on tribal lands  Additionally, FCC has demonstrated that it is possible in some circumstances to collect more granular data when such data collection is targeted to a specific need or area  For example, in 2017 FCC began requiring certain providers that receive funding through the Connect America Fund to report the latitude and longitude of locations where broadband is available, and FCC has noted that these more granular data are extremely useful to the Commission, especially for rural areas where census blocks can be quite large  A few large providers and trade associations similarly stated in public comments on FCC’s proposed rulemaking to modernize the Form 477 process that FCC should target its collection of more granular broadband data to areas where the data are most likely to be overstated—specifically, large, rural census blocks with low population densities, such as those on tribal lands  Additionally, as discussed above, FCC undertook a one-time special data collection for Mobility Fund II to ensure that the mobile broadband data it collected would be reliable for the intended use  By developing and implementing methods for collecting and reporting accurate and complete data on broadband access specific to tribal lands, FCC would be able to better identify tribal areas without access to broadband and to target federal broadband funding to the tribal areas most in need


Report 2:  FCC uses data submitted by broadband providers via the Form 477 process to develop maps and datasets depicting broadband services nationwide, and in specific locations, such as tribal lands, but does not have a formal process to obtain input from tribes on the accuracy of the broadband data  FCC’s 2010 National Broadband Plan noted the need for the federal government to improve the quality of data regarding broadband on tribal lands and recommended that FCC work with tribes to ensure that any information collected is accurate and useful  It also noted that tribal representatives should have the opportunity to review mapping data about tribal lands and offer supplemental data or corrections  Similarly, federal internal control standards note the need for federal agencies to communicate with external entities, such as tribal governments, and to enable these entities to provide quality information to the agency that will help it achieve its objectives  FCC officials told us that they address questions and concerns regarding provider coverage claims submitted to the Office of Native Affairs and Policy, which will work with tribal governments to help them identify inaccurate broadband data for tribal lands, and share tribal questions and concerns with the appropriate FCC bureaus  However, FCC does not have a formal process for tribes (or other governmental entities) to provide input to ensure that the broadband data FCC collects through the 477 process, or the resulting maps that FCC creates to depict broadband on tribal lands, are accurate  Similarly, FCC does not use other methods to verify provider-submitted Form 477 data on tribal lands against other sources of information, such as on-site tests or data collected by other agencies  When discussing the lack of a formal process for tribal representatives or other governmental entities to provide feedback on the accuracy of the 477 broadband data, FCC officials noted that if consumers and local officials have information on individual locations that lack broadband service, such information does not indicate that the entire census block lacks broadband service  Additionally, FCC officials noted that providers attest to the accuracy of the data and that FCC staff validate the data by conducting internal checks to identify possible errors, such as unlikely changes in a providers’ coverage area, and may follow-up with a provider to discuss such changes  However, these checks do not include soliciting input from tribes  About half of the tribal stakeholders we spoke to raised concerns that FCC’s broadband deployment data rely solely on unverified information submitted by providers  Additionally, most tribal stakeholders we interviewed told us that consistent with the recommendations in the National Broadband Plan, FCC should work directly with tribes to obtain information from them to improve the accuracy of its broadband deployment data for tribal lands  These stakeholders identified several ways in which FCC could work with tribes on this issue, including: conducting on-site visits with tribal stakeholders to observe the extent to which broadband infrastructure and services are present; conducting outreach and technical assistance for tribal stakeholders to raise awareness and use of FCC’s broadband data; and providing opportunities for the tribes to collect their own data or submit feedback regarding the accuracy of FCC’s data  FCC’s National Broadband Plan notes the importance of supporting tribal efforts to build technical expertise with respect to broadband issues, and federal internal control standards state that federal agencies should obtain quality information from external entities  Officials we interviewed in FCC’s Office of Native Affairs and Policy told us that they provide some outreach and technical assistance to tribal officials at regional and national workshops, and FCC officials stated that they conducted specific outreach to tribal entities regarding the Mobility Fund Phase II challenge process, while, about half of the tribal representatives we spoke to stated that they were not aware of the Form 477 data or corresponding maps, or raised concerns about a lack of outreach from FCC to inform tribes about the data  Some tribal stakeholders stated that if FCC were to solicit tribal input as part of its verification of the broadband data and maps, technical training and assistance could help tribes use and provide feedback on the data, or improve the collection and submission of their own data  A few of the stakeholders we interviewed noted that tribes can face difficulties when they attempt to challenge FCC’s broadband availability data  For example, in 2013, prior to the auction that distributed Tribal Mobility Fund Phase 1 support, FCC allowed interested parties to challenge FCC’s preliminary determinations regarding which census blocks lacked 3G or better service and would be eligible for support in the auctions  However, all of the tribal entities that challenged the accuracy of FCC’s data were unsuccessful in increasing the number of eligible areas  According to FCC officials, the tribal entities did not provide sufficient or sufficiently verifiable information to support their challenges  A few tribal stakeholders provided varying reasons for this, one of which was the need for more technical expertise to help the tribe meet FCC’s requirements  Because FCC lacks a formal process to obtain tribal input on its broadband data, FCC is missing an important source of information regarding areas in which the data may overstate broadband service on tribal lands  Tribal stakeholders are able to provide a first-hand perspective on the extent to which service is available within their lands and the extent to which factors like affordability, service quality, and service denials affect residents’ ability to access broadband  FCC plans to award nearly $2 billion in support from the Connect America Fund to areas that it has identified as lacking broadband, including tribal lands  Any inaccuracies in its broadband data could affect FCC’s funding decisions and the ability of tribal lands to access broadband in the future  Additionally, in its 2017 report on tribal infrastructure, the National Congress of American Indians stressed the importance of including tribal governments in a leadership role with respect to collecting data on local infrastructure needs  Specifically, it stressed the need for the federal government to invest in tribal data systems and researchers to generate useful, locally specific data that can inform the development and implementation of infrastructure development projects and assess the effectiveness of those projects over time  By establishing a process to obtain input from tribal governments on the accuracy of provider- submitted broadband data that includes outreach and technical assistance, as recommended in the National Broadband Plan, FCC could help tribes develop and share locally specific information on broadband access, which would in turn improve the accuracy of FCC’s broadband data for tribal lands  The success of such an effort may rely on the tribes’ knowledge of, and technical ability to participate in, the process  When discussing the need to improve data regarding broadband on tribal lands, FCC’s 2010 National Broadband Plan recommended that FCC develop a process for tribes to receive information from providers about broadband services on tribal lands  In 2011, FCC required that Eligible Telecommunications Carriers (providers receiving Universal Service Funds from FCC) serving tribal lands meaningfully engage with tribes regarding communications services (including broadband)  Specifically, the providers must file an annual report documenting that this engagement included a discussion of, among other things, a needs assessment and deployment planning for communications services, including broadband  FCC’s 2012 guidance on fulfilling the engagement obligations, which FCC officials confirmed is still in effect, noted that the stated goal of the engagement requirement was to benefit tribal government leaders, providers, and consumers by fostering a dialogue between tribal governments and providers that would lead to improved services on tribal lands  The guidance further noted that the tribal engagement process “cannot be viewed as simply another ‘check the box’ requirement by either party,” and states that a provider should “demonstrate repeated good faith efforts to meaningfully engage with the tribal government ” Finally, FCC noted in its 2012 guidance that the guidance would evolve over time based on the feedback of both tribal governments and broadband providers and that FCC would develop further guidance and best practices  This approach is consistent with federal internal control standards, which call for agencies to communicate with, and obtain quality information from, external parties  About half of the tribal stakeholders we interviewed raised concerns about difficulties accessing information from providers regarding broadband deployment on their tribe’s lands, a key part of the provider engagement process, according to FCC’s guidance  For example, a representative from one tribe stated that a provider declined his requests to meet more than once a year to discuss the provider’s deployment of broadband services on the tribe’s land  A representative from another tribal government stated that some providers are very focused and transparent about their broadband plans and work with the tribe, while other providers treat tribal engagement as a “box to check” and send the tribe broadband deployment information that is not useful because it is redacted  Similarly, some tribal stakeholders stated that providers heavily redacted deployment information (which providers may consider proprietary) or required the tribe sign non-disclosure agreements to access deployment data  According to one tribal stakeholder, these non-disclosure agreements could possibly require tribes to waive tribal sovereign immunity in order to view the data  Some of the industry stakeholders we interviewed stated that they attempt to engage with tribes but the level of responsiveness from tribes varies  For example, some stakeholders stated that they send letters and do not hear back from tribes  One stakeholder stated that they make repeated attempts to contact tribes when they do not hear back after their initial contact, while another stated that a provider meets regularly with some tribes  Although FCC stated in its 2012 guidance that it would update the tribal engagement guidance and develop best practices based on feedback from tribal governments and broadband providers, it has taken limited steps to obtain such feedback from providers and tribal governments to determine whether its guidance is enabling meaningful tribal engagement  Additionally, FCC has not updated the guidance or issued best practices  Thus, FCC has limited information regarding whether its tribal engagement requirement is fulfilling its intended purpose  FCC officials we interviewed said that the Office of Native Affairs and Policy (ONAP) provided information and, in some cases, held training sessions about the tribal engagement obligation during workshops with tribal representatives, and encouraged representatives to contact ONAP with any concerns  ONAP officials also noted that they handle complaints from tribes regarding a lack of provider engagement and reach out to providers to address tribal concerns  ONAP officials stated that they have had internal discussions about whether the guidance is clear or needs revision, but this has not gone beyond internal discussions  A few of the tribal stakeholders provided examples of the benefits of providers engaging with tribes to ensure tribal representatives have access to information regarding broadband availability on their lands  For example, one representative stated that this information could help the tribes plan deployments by focusing on areas that they know the provider does not plan to serve  Another representative stated that tribal engagement could help improve the accuracy of FCC’s broadband maps  By obtaining feedback from both tribal stakeholders and providers on the effectiveness of FCC’s tribal engagement guidance to determine whether changes are needed, FCC would be better positioned to ensure that tribal governments and providers are sharing information in a manner that will lead to improved services on tribal lands  FCC has collected data and developed maps and reports depicting broadband on tribal lands and has noted the lower levels of broadband access on tribal lands, in comparison to other areas  However, limitations in FCC’s existing process for collecting and reporting broadband data have led FCC to overstate broadband access on tribal lands  By taking steps to address these limitations and to collect data that more accurately and completely depict broadband access on tribal lands, FCC would have greater assurance that it is making progress on reducing the digital divide on tribal lands and targeting broadband funding to tribal lands most in need  Without taking these steps, FCC increases the risk that residents living on tribal lands will continue to lack broadband access  Compounding the limitations in FCC’s data collection process is FCC’s lack of a formal process to obtain tribal input on the accuracy of provider- submitted broadband data for tribal lands  By developing a process to solicit tribal input and ensuring that tribes know about the process and are equipped with the technical skills and abilities necessary to provide this information, FCC would be better able to ensure the accuracy of its broadband data for tribal lands  Moreover, FCC would be able to obtain firsthand, locally specific information on broadband access that could inform FCC’s policies and funding decisions and help FCC achieve its goal of increasing broadband access for all Americans, including those living on tribal lands  Finally, by obtaining feedback from providers and tribal stakeholders on the effectiveness of FCC’s tribal engagement guidance, FCC would be better positioned to assess whether its guidance is helping providers meet requirements and ultimately whether providers’ engagement is fulling its intended purpose of fostering a dialogue between tribal governments and providers that would lead to improved services on tribal lands  We are making the following three recommendations to the Chairman of the Federal Communications Commission  The Chairman of the Federal Communications Commission should develop and implement methods—such as a targeted data collection—for collecting and reporting accurate and complete data on broadband access specific to tribal lands  (Recommendation 1) The Chairman of the Federal Communications Commission should develop a formal process to obtain tribal input on the accuracy of provider-submitted broadband data that includes outreach and technical assistance to help tribes participate in the process  (Recommendation 2) The Chairman of the Federal Communications Commission should obtain feedback from tribal stakeholders and providers on the effectiveness of FCC’s 2012 statement to providers on how to fulfill their tribal engagement requirements to determine whether FCC needs to clarify the agency’s tribal engagement statement  (Recommendation 3) We provided a draft of this report to FCC for review and comment  In written comments provided by FCC (reproduced in appendix III), FCC agreed with our findings and recommendations  In its written comments, FCC described efforts, some of which are already under way, that it felt would address each recommendation and stated its intent to build upon those efforts  For example, FCC explained that it is exploring methods to collect more granular broadband deployment data and noted the need to balance the burden on Form 477 filers  FCC also noted that it is starting work to address a statutorily-required evaluation of broadband coverage on certain tribal lands  We agree that increasing the granularity of deployment data is helpful in addressing data accuracy issues, but we also note that it is important to collect data related to factors that affect broadband access on tribal lands  FCC also described informal efforts to collect tribal feedback on providers’ broadband data and stated it would explore options for a formal process to collect feedback  Regarding our recommendation related to providers’ engagement efforts, FCC outlined its existing methods by which tribal stakeholders can provide feedback on providers’ engagement efforts and agreed that seeking additional feedback from tribal stakeholders and providers would be desirable  We agree that improving feedback in these ways could help FCC determine whether it needs to clarify its tribal engagement statement  FCC also provided technical comments, which we incorporated as appropriate  We are sending copies of this report to the appropriate congressional committees, the Chairman of the Federal Communications Commission, and other interested parties  In addition, the report is available at no charge on the GAO website at http://www gao gov  If you or your staff have any questions about this report, please contact me at (202) 512-2834 or GoldsteinM@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report  GAO staff who made key contributions to this report are listed in appendix IV  Appendix I: List of Interviewees Representatives from tribal governments or tribally owned broadband providers Choctaw Nation of Oklahoma (OK) Confederated Tribes of the Colville Reservation (WA) Fond du Lac Band of Lake Superior Chippewa (MN) Fort Belknap Indian Community (MT) Gila River Telecommunications, Inc  (AZ) Hopi Telecommunications, Inc  (AZ) Jamestown S’Klallam Tribe (WA) Karuk Tribe (CA) Leech Lake Band of Ojibwe (MN) Makah Tribe (WA) Navajo Tribal Utility Authority (AZ, NM, UT) Nez Perce Tribe (ID) Osage Nation (OK) Pueblo of Acoma (NM) Pueblo of Pojoaque (NM) Pueblo of San Ildefonso (NM) Taos Pueblo (NM) Red Spectrum Communications (Coeur d’Alene Tribe (ID)) Saint Regis Mohawk Tribe and Mohawk Networks, LLC (NY) San Carlos Apache Telecommunications Utility, Inc  (AZ) Southern California Tribal Chairmen’s Association - Tribal Digital Village Network (CA) Spokane Tribe of Indians and Spokane Tribe Telecom Exchange (WA) Standing Rock Telecommunications, Inc  (ND, SD) Warm Springs Telecommunications Co  (OR) Yurok Tribe and Yurok Connect (CA) Representatives from tribal associations/consortiums that include tribes Affiliated Tribes of Northwest Indians Middle Rio Grande Pueblo Consortium National Congress of American Indians Native American Finance Officers Association (NAFOA) REDINet Representatives from companies/academic groups that work with tribes AMERIND Risk Arizona State University, American Indian Policy Institute and School of Public Affairs Turtle Island Communications Representatives from providers/trade associations (non-tribally owned) AT&T Representatives from companies that collect broadband data Alexicon Connected Nation Government Agencies (non-tribal) Census Bureau U S  Department of Agriculture’s Rural Utilities Service Department of Interior’s Bureau of Indian Affairs National Telecommunications and Information Administration Minnesota Office of Broadband Development One broadband provider we interviewed did not want to be included in this appendix  This report discusses the extent to which: (1) the Federal Communications Commission’s (FCC) approach to collecting broadband availability data accurately captures the ability of Americans living on tribal lands to access broadband Internet services and (2) FCC obtains tribal input on the accuracy of provider-submitted broadband data for tribal lands  To address both objectives, we analyzed FCC’s December 2016 fixed and mobile broadband availability data—the most recent data at the time of our review—to identify the speeds, technologies, and availability providers reported for federally recognized tribal lands  Providers currently report this information to FCC by filing a “Form 477,” twice a year  We also used 2010 U S  Census data to identify census blocks completely or partially on tribal lands  To assess the reliability of FCC’s data and 2010 U S  Census data, we reviewed a previous GAO reliability assessment, and for FCC’s data we conducted electronic testing and analysis of the data, reviewed FCC guidance and documentation, and interviewed FCC officials  Based on the results of our analysis, we determined the data to be reliable for our purposes, which were: (1) to inform our selection of tribal governments and providers for interviews and visits, as described below, and (2) to develop maps depicting fixed and mobile broadband availability for the nine tribal lands we selected for visits, in order to obtain tribal representatives’ feedback on the data  Specifically, we mapped; fixed broadband data according to speed and technology, and mobile data for long-term evolution (LTE) services by provider for each tribal land  We used those maps during our visits to discuss the accuracy of the data with representatives for each tribal government or tribally owned provider  Though we analyzed all up and download speeds that providers reported in the Form 477, for the purposes of this report we defined “broadband” as fixed Internet service reaching at least 25 megabits per second (Mbps) download and 3 Mbps upload speeds, in accordance with FCC’s advanced telecommunications capability benchmark in its 2018 Broadband Deployment Report  We also report on the availability of mobile broadband, which, for the purposes of this report, does not have a speed threshold and refers to long-term evolution (LTE) services  To address both objectives and obtain tribal government representatives’ feedback on the accuracy of FCC’s broadband data for their lands, we interviewed representatives from 25 tribal governments or tribally owned providers, including visits to 9 tribal lands  We considered a range of factors when we selected tribal governments and tribally owned providers for interviews, including our analysis of Form 477 data, recommendations from tribal, industry, or government stakeholders regarding tribal and non- tribal representatives familiar with broadband data issues, and demographic and geographic characteristics, among others  For example, we considered demographic characteristics such as unemployment rate from the 2011– 2015 American Community Survey data, and geographic characteristics such as rurality from the United States Department of Agriculture (USDA) Rural-Urban Commuting Area Codes data  The tribes included in our review vary with respect to location, level of broadband availability according to FCC, land mass, and population size and density  The results of our interviews are not generalizable to all tribal governments or tribally owned broadband providers  In addition to tribal governments and tribally owned providers, we interviewed six tribal organizations and four stakeholders who work with tribes on broadband issues  For reporting purposes, we developed the following series of indefinite quantifiers to describe the tribal responses from the 35 entities representing tribal stakeholders we interviewed: 3 to 7 is defined as “a few;” 8 to 15 is described as “some;” 16 to 20 is described as “about half;” 21 to 27 is described as “most;” and 28 to 34 is described as “almost all ” A full list of the tribal stakeholders we interviewed can be found in appendix I  Further, to obtain industry perspectives, we reviewed public comments submitted by providers and industry associations in FCC’s ongoing 2017 Notice of Proposed Rulemaking on Modernizing the Form 477 Data Program  We also interviewed 10 non-tribally owned fixed and mobile broadband providers and three industry associations to understand providers’ views on the Form 477 and how providers interact with tribal governments  When selecting providers for interviews, we included providers that reported serving the lands of tribal governments we interviewed and selected providers that varied in the percentage of tribal lands they reported serving  The providers we interviewed represent large, nationwide carriers as well as small, local carriers, and offer broadband via a variety of technologies, including fiber optics, digital subscriber line (DSL), fixed wireless, and mobile LTE  The results of our interviews with providers are not generalizable to all broadband providers  In addition, to address both objectives, we interviewed representatives from other government entities, as well as private companies that collect and report broadband data  A full list of the industry stakeholders we interviewed can be found in appendix I  To identify the extent to which FCC’s approach to collecting broadband availability data reflects the ability of Americans living on tribal lands to actually access broadband Internet services, we reviewed documentation of the Form 477 process, including submission guidance, and FCC’s proposals and public comments in its 2017 Notice of Proposed Rulemaking on Modernizing the Form 477 Data Program and Mobility Fund Phase II proceedings  We also interviewed FCC officials, industry stakeholders, and tribally owned broadband providers to understand FCC’s current process for collecting broadband data  To understand the purpose of the Form 477 data collection process and FCC’s strategic goals, we reviewed relevant statutes, and FCC documents, including FCC’s Strategic Plan 2018––2022, the National Broadband Plan, and FCC’s broadband deployment and progress reports  Given the importance placed on broadband access in these documents, we interviewed tribal stakeholders, as described above and reviewed FCC documents to identify factors affecting the ability of Americans living on tribal lands to access broadband Internet services  We also reviewed previous GAO work that identified barriers to broadband access on tribal lands  We compared the Form 477 process to FCC’s strategic goals and to factors affecting broadband access to determine the extent to which the Form 477 was designed to collect information on those factors and to meet FCC’s goals  We further evaluated this information against the Government Performance and Results Act, as enhanced by the GPRA Modernization Act of 2010 and Standards for Internal Control in the Federal Government  We also reviewed documentation for other FCC data collection programs, including the Measuring Broadband America program and the Urban Rate Survey, to determine the extent to which FCC collected data on factors affecting broadband access outside of the Form 477 process  To determine the extent to which FCC obtains tribal input on the accuracy of provider-submitted broadband data for tribal lands, we interviewed FCC officials and analyzed FCC documents regarding the collection procedures for the Form 477 data and FCC’s policies for working with tribal governments, as well as Connect America Fund documents regarding requirements for providers to share information with tribal governments  We also reviewed documents on past FCC Universal Service Fund processes to challenge broadband data and identified prior instances in which tribal governments or tribally owned providers challenged FCC’s broadband data and the outcomes of those challenges  Additionally, we interviewed tribal stakeholders, as described above, to understand the extent to which: (1) FCC involves tribal governments and other stakeholders in the validation of Form 477 broadband data, (2) tribal governments can access broadband data from FCC or providers, and (3) FCC’s Form 477 data accurately reflected broadband access on their lands  For the nine tribal lands we visited, we asked tribal governments or tribally owned providers to identify where the data do or do not accurately reflect broadband access on maps of FCC’s data  Further, to identify how providers complied with FCC’s tribal engagement requirement and obtain their perspectives, we interviewed providers and industry associations  We compared FCC’s data validation procedures and tribal stakeholders’ feedback on the process to FCC’s policies for working with tribal governments, FCC recommendations from the National Broadband Plan and Standards for Internal Control in the Federal Government  We also interviewed and received written comments from officials from other federal agencies that have broadband programs, including USDA Rural Utilities Service, the National Telecommunications and Information Administration (NTIA), and others, in addition to a state agency and three private companies that collect and report broadband data to understand how other entities collect and validate broadband data  We conducted this performance audit from June 2017 to September 2018 in accordance with generally accepted government auditing standards  Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives  We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives  Mark L  Goldstein, (202) 512-2834 or GoldsteinM@gao gov  In addition to the contact named above, Keith Cunningham (Assistant Director); Crystal Huggins (Analyst in Charge); Katherine Blair; Lilia Chaidez; Camilo Flores; Adam Gomez; Serena Lo; Jeffery Malcolm; John Mingus; Joshua Ormond; Jay Spaan; James Sweetman, Jr ; Elaine Vaurio; and Michelle Weathers made key contributions to this report 

ground_truth: Federal agencies and the nation's critical infrastructures—such as energy, transportation systems, communications, and financial services—are dependent on information technology systems to carry out operations. The security of these systems and the data they use is vital to public confidence and national security, prosperity, and well-being. The risks to these systems are increasing as security threats evolve and become more sophisticated. GAO first designated information security as a government-wide high-risk area in 1997. This was expanded to include protecting cyber critical infrastructure in 2003 and protecting the privacy of personally identifiable information in 2015. This report provides an update to the information security high-risk area. To do so, GAO identified the actions the federal government and other entities need to take to address cybersecurity challenges. GAO primarily reviewed prior work issued since the start of fiscal year 2016 related to privacy, critical federal functions, and cybersecurity incidents, among other areas. GAO also reviewed recent cybersecurity policy and strategy documents, as well as information security industry reports of recent cyberattacks and security breaches. GAO has identified four major cybersecurity challenges and 10 critical actions that the federal government and other entities need to take to address them. GAO continues to designate information security as a government-wide high-risk area due to increasing cyber-based threats and the persistent nature of security vulnerabilities. GAO has made over 3,000 recommendations to agencies aimed at addressing cybersecurity shortcomings in each of these action areas, including protecting cyber critical infrastructure, managing the cybersecurity workforce, and responding to cybersecurity incidents. Although many recommendations have been addressed, about 1,000 have not yet been implemented. Until these shortcomings are addressed, federal agencies' information and systems will be increasingly susceptible to the multitude of cyber-related threats that exist. GAO has made over 3,000 recommendations to agencies since 2010 aimed at addressing cybersecurity shortcomings. As of August 2018, about 1,000 still needed to be implemented.