Personnel Security Clearances: Plans Needed to Fully Implement and Oversee Continuous Evaluation of Clearance Holders
Released: 2017-11-21T00:00:00

Letter

Background

Overview of Personnel Security Clearance Process

ODNI estimates that, as of October 1, 2015, approximately 4.2 million government and contractor employees were eligible to hold a security clearance. Personnel security clearances are required for access to certain national security information. National security information may be classified at one of three levels: confidential, secret, or top secret. The level of classification denotes the degree of protection required for information and the amount of damage that unauthorized disclosure could reasonably be expected to cause to national security. Specifically, unauthorized disclosure could reasonably be expected to cause (1) “damage,” in the case of confidential information; (2) “serious damage,” in the case of secret information; and (3) “exceptionally grave damage,” in the case of top secret information. According to the Office of Personnel Management (OPM) Federal Investigations Notice 16-02, tier 3 investigations are required for eligibility for access to secret and confidential information, or for noncritical sensitive positions, or “L” access. OPM Federal Investigations Notice 16-07 indicates that tier 5 investigations are required for eligibility for access to top secret or sensitive compartmented information, or for critical sensitive or special sensitive positions, or “Q” access.

Once an executive branch agency determines that a position requires a certain level of access to classified information, the employee in that position completes a questionnaire for national security positions, which the requesting agency sends to an investigative service provider. NBIB— the bureau within OPM with responsibility for conducting personnel background investigations—conducts background investigations for most of the federal government; however, some agencies have authority delegated to them to conduct their own investigations. The investigative service provider then conducts a background investigation and submits an investigative report to the requesting agency. Adjudicators from the requesting agency use the information from the investigative report to determine whether to grant or deny the employee eligibility for a security clearance by considering guidelines in 13 specific areas that address (1) conduct that could raise security concerns and (2) factors that could allay those security concerns and permit granting a clearance. Individuals granted security clearances are investigated periodically—for as long as they remain in a position requiring access to classified information—to ensure their continued eligibility. The 2012 Federal Investigative Standards changed the frequency of periodic reinvestigations for certain clearance holders.

Continuous Evaluation Is Intended to Supplement the Personnel Security Clearance Process

According to Executive Order 13467, as amended, continuous evaluation is a vetting process to review the background of an individual who has been determined to be eligible for access to classified information or to hold a sensitive position at any time during the period of eligibility. Continuous evaluation is intended to fill the gap that exists between periodic reinvestigations in which issues relevant to an individual’s continued eligibility for a security clearance may go unreported or unknown. For example, while the Federal Investigative Standards have allowed for periodic reinvestigations to be conducted at any time following the completion of the previous investigation or reinvestigation, agencies have not been required to conduct them more frequently than every 5 years, at most, depending on the clearance level and investigative standards in effect. Like periodic reinvestigations, the purpose of continuous evaluation is to assist agencies in evaluating an individual’s continued eligibility for access to classified information.

Continuous evaluation involves automated record checks conducted on a more frequent basis, whereas periodic reinvestigations are conducted less frequently and may include, among other things, subject and reference interviews. The types of records checked as part of continuous evaluation are the same as those checked for other personnel security purposes. Security-relevant information discovered in the course of continuous evaluation is to be investigated and adjudicated under the existing standards. According to ODNI, implementation of continuous evaluation will not alter clearance holders’ existing rights or responsibilities and it will incorporate protections for privacy and civil liberties.

Continuous Evaluation Is a Key Initiative of the Personnel Security Clearance Reform Effort

The enactment of the Intelligence Reform and Terrorism Prevention Act of 2004 initiated a reform effort including goals and requirements for improving the personnel security clearance process government-wide. In June 2008, Executive Order 13467 established the PAC as the government-wide governance structure responsible for driving the implementation of and overseeing security and suitability reform efforts. The PAC presently has four principal members: the Deputy Director for Management of OMB; the Director of National Intelligence, who is the Security Executive Agent; the Director of OPM, who is the Suitability Executive Agent; and the Under Secretary of Defense for Intelligence. The Executive Order also designated the Deputy Director for Management of OMB as the chair of the PAC. Among other things, the PAC is to work with agencies to implement continuous performance improvement programs, policies, and procedures; establish annual goals and progress metrics; and prepare annual reports on results. It is also to develop and continuously reevaluate and revise outcome-based metrics that measure the quality, efficiency, and effectiveness of the vetting enterprise. In April 2014, the PAC established the Program Management Office to implement security clearance reforms. This office includes subject-matter experts with knowledge of personnel security clearances and suitability determinations from OMB, ODNI, OPM, DOD, the Department of Homeland Security, the Department of Justice, the Department of the Treasury, and the Federal Bureau of Investigation.

In March 2014, OMB established Insider Threat and Security Clearance Reform as a government-wide, cross-agency priority goal to improve interagency coordination and implementation within the area of personnel security clearances. Through this goal, the PAC and executive-branch agencies are to work to improve oversight to ensure that investigations and adjudications meet government-wide quality standards. Included among the goal’s key milestones are implementing a continuous evaluation policy for the executive branch that regularly assesses trusted insiders who have been granted, or are eligible for, access to classified national security information, and overseeing the establishment of continuous evaluation capabilities. ODNI is identified as the lead agency for achieving both of these milestones. In addition, continuous evaluation is identified as a key initiative in the PAC’s strategic framework for fiscal years 2017 through 2021 as part of an effort to modernize the vetting process.

While the PAC is responsible for driving the implementation of and overseeing the overall government-wide reform effort, individual agencies are responsible for various aspects of the effort. For example, as the Security Executive Agent, ODNI is responsible for developing and issuing uniform and consistent policies and procedures to ensure the effective, efficient, timely, and secure completion of investigations, polygraphs, and adjudications relating to determinations of eligibility for access to classified information or eligibility to hold a sensitive position. In addition, Executive Order 12968, as amended, indicates that ODNI is responsible for setting the standards for continuous evaluation of those individuals who have access to classified information. According to ODNI, under these Executive Orders, it has responsibility for and oversight of continuous evaluation, as it is an investigative activity that supports eligibility determinations. As such, ODNI established a program office within the National Counterintelligence and Security Center to, among other things, establish policy, guidance, and standards for the implementation of continuous evaluation across the executive branch.

DOD Has Conducted Research on Continuous Evaluation for More Than a Decade

DOD has been piloting aspects of continuous evaluation for more than a decade—with pilot tests of automated record checks conducted as early as 2002. Specifically, PERSEREC has conducted several studies dating back to 2001 that have informed and evaluated DOD’s continuous evaluation pilots, including the utility of and costs associated with various data sources. These studies have focused on the technical capability to conduct automated record checks from over 40 government and commercial databases, the value and utility of automated record checks in tier 5 investigations, and investigative alternatives to the traditional periodic reinvestigation, among other things. The studies have also included recommendations to further improve DOD’s continuous evaluation program, as well as areas for future research. PERSEREC noted that it undertook these studies to identify ways to make the personnel security system more efficient, fair, and effective. According to PERSEREC, starting in 2004 with the formation of the government-wide security clearance reform effort, it began to plan for a broader application of its research beyond the department.

Using this body of knowledge, DOD has incrementally improved its automated record check capabilities and therefore its ability to implement a continuous evaluation program, which it did in 2014 at the recommendation of the Secretary of Defense. Specifically, following the September 2013 shooting at the Washington Navy Yard, the Secretary of Defense directed concurrent internal and independent reviews to identify and recommend actions to address any gaps or deficiencies in DOD programs, policies, and procedures regarding, among other things, the granting and renewing of security clearances for department and contractor personnel. In March 2014, the Secretary of Defense identified four key recommendations based on the findings and recommendations from those reviews. One of those recommendations was to implement continuous evaluation to provide automated record checks of personnel with access to DOD facilities or classified information. In addition, DOD Instruction 5200.02, which was also issued in March 2014, states that all personnel in national security positions shall be subject to continuous evaluation. Consistent with the recommendation and the DOD Instruction, the department implemented a continuous evaluation pilot in October 2014, the details of which are discussed later in the report.

ODNI Has Taken an Initial Step to Implement Continuous Evaluation across the Executive Branch but Has Not Determined Key Program Aspects or How it Will Monitor and Measure Performance

In October 2016, ODNI took an initial step to implement continuous evaluation across the executive branch in a phased approach, but as of May 2017, it had not yet formalized the program in policy. The seven agencies we spoke with have been limited in their abilities to plan for the implementation of continuous evaluation, including developing estimated costs, in accordance with ODNI’s phased approach. This is due, in part, to the fact that ODNI has not yet determined key aspects of the program, such as when the future phases of implementation will occur or what they will entail, and none of the agencies has completed implementation plans. Further, ODNI lacks plans for monitoring and measuring the performance of continuous evaluation across the executive branch.

ODNI Has Taken an Initial Step to Implement Continuous Evaluation across Executive Branch Agencies, but Has Not Yet Formalized the Program in Policy

ODNI has taken an initial step to implement continuous evaluation across all executive branch agencies in a phased approach, but it has not yet formalized the program in policy. Specifically, in October 2016, ODNI initiated the first phase of continuous evaluation and outlined requirements for this phase in interim guidance distributed to implementing agencies in December 2016. For the first phase of implementation, executive branch agencies are to conduct certain continuous evaluation record checks against a portion of their national security population. Specific details of the requirements were omitted from this report because the information is sensitive. According to OPM Federal Investigations Notice 17-03, the first phase of continuous evaluation is to be implemented by the end of fiscal year 2017. These checks are conducted in addition to any initial investigations or periodic reinvestigations occurring in fiscal year 2017. ODNI provided agencies with prioritization guidance to help them select individuals for continuous evaluation. Nearly 80 executive branch agencies are subject to the requirements for this first phase of implementation.

ODNI has taken steps to establish the executive branch-wide continuous evaluation program in coordination with key stakeholders. For example, in June 2013, ODNI established a Continuous Evaluation Working Group— consisting of 12 core voting member agencies—as a mechanism to effectively coordinate continuous evaluation implementation among executive branch departments and agencies. According to the group’s charter, it meets on at least a quarterly basis and is responsible for coordinating the development of continuous evaluation standards, policies, and procedures, among other things. Since January 2015, ODNI has also issued interim guidance to executive branch agencies that are subject to its continuous evaluation requirements informing them about the purpose of continuous evaluation and providing them with some details of the program. Further, to inform the establishment of the executive branch-wide program, ODNI itself began a 1-year continuous evaluation pilot in September 2016, according to ODNI officials. Specific details of ODNI’s pilot were omitted from this report because the information is sensitive.

In addition to developing standards for continuous evaluation and its oversight role, ODNI is also developing a system that agencies can use to conduct continuous evaluation. According to ODNI, its system is under development and will be available to all executive branch agencies with a full suite of continuous evaluation data sources. Agencies may opt to: (1) use ODNI’s system, (2) develop their own technical solution, (3) partner with another agency to fulfill their continuous evaluation requirements, or (4) some combination of the above options. ODNI asked agencies in December 2016 to provide a preliminary determination as to how they will satisfy future automated records checks requirements to allow ODNI’s continuous evaluation program to adequately plan for system enrollee volume and data usage. Specific details regarding the response of executive branch agencies to this request were omitted from this report because the information is sensitive. Some executive branch agencies stated the following:

Department of Justice and State officials stated that they plan to use ODNI’s system once its development is complete;

DOD officials stated that they plan to use their own internal system that they are developing to conduct continuous evaluation, but that they may use ODNI’s system to conduct certain checks; and

Department of Homeland Security officials noted that they plan to use a combination of existing internal agency systems and ODNI’s system.

Standards for Internal Control in the Federal Government states that management should externally communicate the necessary information to achieve an entity’s objectives. Effective information and communication are vital for enabling an entity to achieve its objectives, which can be accomplished through written guidance.

While ODNI has provided some details of the program to implementing executive branch agencies through interim guidance, it has not yet formalized the continuous evaluation program through a Security Executive Agent Directive. Specifically, in May 2017, ODNI officials stated that ODNI had not yet issued a Security Executive Agent Directive for continuous evaluation, but that a draft directive was undergoing interagency coordination. ODNI officials stated that the directive will contain a definition of continuous evaluation that is consistent with, but expands upon, the definition contained in the relevant Executive Order. These officials stated that the expanded definition will help to clarify continuous evaluation and ensure that agencies have a common understanding of the program. In addition, ODNI officials stated that they have developed draft implementation guidelines, which they plan to issue after the directive is finalized. ODNI officials stated that the interim guidance will remain in effect until the Security Executive Agent Directive or follow-on interim guidance is issued.

DOD’s continuous evaluation program—which it began in October 2014, in advance of implementation of continuous evaluation executive branch- wide by ODNI—identified, in a requirements document for its continuous evaluation IT system, that the most important gap in the development of the department’s program was the lack of a national or DOD-level policy. Specifically, the requirements document notes the lack of a policy that fully describes the continuous evaluation process or purpose, or the end uses of data. The requirements document further notes that there are multiple definitions of continuous evaluation and, due to the lack of policy, there is not a common lexicon of terms used in the continuous evaluation program, thereby creating an additional gap.

While ODNI reports that the policy is under review, it has not prioritized the implementation of continuous evaluation and, as a result, has missed numerous milestones in issuing the policy since 2014. Specifically, the original Insider Threat and Security Clearance Reform cross-agency priority goal milestone for ODNI to issue a continuous evaluation policy was July 2014. This milestone was not attained, and it was adjusted to September 2016, a milestone that was also missed. The current milestone for issuing the policy is October 2017. Additionally, ODNI has missed other milestones for implementing a continuous evaluation program, as discussed later in the report.

Furthermore, ODNI has initiated the first phase of continuous evaluation without a government-wide issued policy or an expanded definition of continuous evaluation. As a result, agencies may develop inconsistent approaches to implementing continuous evaluation. For example, DOD officials stated that DOD has developed its own path for continuous evaluation from ODNI’s limited guidance and that in the absence of a government-wide policy, DOD is developing its own internal guidance. As a result, the approach to continuous evaluation taken by DOD—the executive branch agency with the majority of security clearance holders— may differ from that of other executive branch agencies once fully implemented. Ultimately, such inconsistent approaches to continuous evaluation could affect reciprocity among agencies—another key objective of government-wide security clearance reform efforts. Without issuing a Security Executive Agent Directive in advance of the next phase of implementation—the timeframe for which ODNI has not yet determined—that includes, among other things, an expanded definition of continuous evaluation, agencies may develop inconsistent approaches to continuous evaluation, resulting in an uneven and perhaps ineffective implementation across the federal government.

ODNI Has Not Yet Determined Key Aspects of the Continuous Evaluation Program, and Executive Branch Agencies Have Been Limited in Their Ability to Plan for Implementation

ODNI has not yet determined key aspects of its continuous evaluation program, which has limited the ability of executive branch agencies to plan for implementation in accordance with ODNI’s phased approach. For example, while ODNI has initiated the first phase of continuous evaluation in coordination with implementing executive branch agencies, it has not yet determined what the future phases of implementation will entail, or when they will occur. Specifically, ODNI officials stated that they have not set any further timeframes for implementing continuous evaluation or determined agency requirements for future phases.

Moreover, the timeframes for the implementation of continuous evaluation across the executive branch have been extended over time. For example, the original milestone set by the government-wide reform effort for implementing continuous evaluation was the 4th quarter of fiscal year 2010, and it was not attained. The PAC subsequently set an Insider Threat and Security Clearance Reform cross-agency priority goal milestone for developing an initial continuous evaluation capability for the most sensitive top secret clearance holders by September 2014—which was extended to December 2014—and a milestone for implementing the capability for additional clearance holders by December 2016. These milestones were also missed. As of May 2017, continuous evaluation had not yet been fully implemented, and ODNI had not set a milestone for when it would occur.

Although ODNI is one of the goal leaders for the Insider Threat and Security Clearance Reform cross-agency priority goal, a senior ODNI official stated that the milestones were arbitrarily set, and that implementing continuous evaluation has proven to be challenging as a result of several technical and legal issues that need to be resolved. Further, ODNI officials highlighted the complexities associated with developing a whole-of-government continuous evaluation program and noted that a number of challenges have come to light as they have been developing the program, which have contributed to missed milestones. However, ODNI has not prioritized the setting of internal milestones for the future phases of implementation that it considers to be reasonable. ODNI officials stated that because continuous evaluation is a new initiative, no realistic timeline for full implementation will be set until the initial results of implementation are analyzed and technical capabilities have matured. Further, they stated that although they are unable to develop a timeline for full implementation at this time, they are actively working to implement the program. In addition, as previously discussed, ODNI’s milestone for issuing a continuous evaluation policy has also been adjusted over time. Figure 1 shows the adjusted executive branch milestones for issuing a continuous evaluation policy and implementing a continuous evaluation program, including developing a technical capability.

The uncertainty regarding the requirements and timeframes for the future phases of the program has affected the ability of executive branch agencies to plan to implement continuous evaluation and estimate the associated costs. First, although OPM Federal Investigations Notice 17- 03 notes that the first phase of continuous evaluation is to be implemented by the end of fiscal year 2017, none of the seven executive branch agencies we spoke with has completed an agency-specific implementation plan. While some agencies, such as DOD and State— both of which have established continuous evaluation programs in advance of implementation across the executive branch—have developed concepts of operations or standard operating procedures for continuous evaluation, all seven agencies we spoke with stated that they are waiting for additional information from ODNI before completing their implementation plans. Department of Homeland Security officials stated that they are waiting for ODNI to define and schedule the future phases of implementation and to finish developing its continuous evaluation IT system, because there could be unknown policy implications that would affect the Department’s planning efforts. In August 2017, ODNI officials described plans to distribute information to executive branch agencies regarding continuous evaluation requirements for fiscal year 2018. Specific details of these plans were omitted from this report because the information is sensitive.

Second, six of the seven agencies we spoke with noted challenges associated with estimating the costs of implementation. For example, while the Federal Bureau of Investigation has developed some cost estimates for implementing continuous evaluation, officials noted that it is challenging to estimate the full costs of the program until they receive additional information from ODNI, such as the requirements for future phases of implementation, as well as information about record check, technology, and personnel requirements. DOD officials stated that the number of individuals enrolled in continuous evaluation directly relates to the amount of agency resources required, for example, to validate, respond to, and adjudicate alerts. Two agencies we spoke with stated that they had not yet taken any steps to estimate costs because they are waiting for additional information from ODNI. In August 2017, ODNI officials stated that they plan to leverage an upcoming OMB budget data request, administered through the PAC, to obtain agency funding estimates for expenses related to conducting continuous evaluation from fiscal years 2017 through 2019.

We have previously identified weaknesses associated with estimating the costs of personnel security clearance reform. Specifically, in April 2015 we found, among other things, that long-term costs of implementing the 2012 Federal Investigative Standards—including the implementation of continuous evaluation—were not addressed in personnel security clearance background investigation reform planning documentation. Further, we found that OMB did not have current and detailed cost- estimate information from executive-branch agencies, because it did not begin to solicit the information from the agencies until almost 2 years after the updated standards were approved. As such, we recommended in April 2015, among other things, that the Deputy Director for Management of OMB, in the capacity as Chair of the PAC, develop long-term funding estimates for changes to the federal government’s investigation practices resulting from the implementation of the standards, including but not limited to costs related to: (1) information technology adjustments to enable government-wide data sharing; (2) implementation of continuous evaluation of clearance holders; and (3) additional personnel resources for twice-as-frequent reinvestigations. OMB concurred with the recommendation. However, as of October 2017, this recommendation remained open. We continue to believe that this recommendation is valid and should be implemented.

In addition, the seven executive branch agencies we spoke with identified other areas related to agency expectations for which they need information from ODNI. For example, officials from the Department of Justice; the Bureau of Alcohol, Tobacco, Firearms, and Explosives; and the Federal Bureau of Investigation stated that while they would like to use ODNI’s IT system to conduct all or at least some of the record checks that will be required, they will need to develop an interface with ODNI’s system to do so. However, these officials stated that they were unaware of ODNI’s technical requirements for that interface. These officials further stated that without information related to the technical requirements, they are unable to sufficiently plan or budget for continuous evaluation. ODNI officials stated that although ODNI’s IT system remains under development, information on technical interface requirements is available to all stakeholders and that they meet with agencies to discuss agency- specific IT requirements. According to ODNI, several executive branch agencies have expressed an interest in using ODNI’s IT system to conduct at least some, if not all, of the checks that will be required once continuous evaluation is fully implemented. ODNI officials acknowledged that agencies will need to develop an interface to use the system, and that agencies will be responsible for the associated costs.

The Project Management Institute’s Guide to the Project Management Body of Knowledge (PMBOK® Guide) provides guidelines for managing individual projects, including developing a project management plan—in advance of executing the project—that describes how the project will be executed, monitored, and controlled. The plan should include, among other things, project schedules and stakeholder roles and responsibilities. The guide notes that updates may be made to the project management plan as changes may occur as the project progresses.

ODNI officials managing the continuous evaluation program stated that they have not developed a project management plan for the implementation of continuous evaluation, to include an implementation schedule, because they are still in the planning stage. However, ODNI has already started to implement the program. Without a plan that, among other things, identifies reasonable milestones for the future phases of implementation, ODNI does not have a schedule against which it can track its progress or to which it is accountable. Further, without a plan for implementing continuous evaluation executive branch-wide that includes a schedule and agency requirements for future implementation phases, full implementation—which has been delayed for almost 7 years—may be further delayed.

While a phased approach to implementation provides agencies time to adapt their personnel security clearance programs to changing requirements, without an implementation plan outlining ODNI’s expectations of agencies’ roles and responsibilities, agencies are unable to sufficiently plan for the implementation of continuous evaluation, including identifying required resources and estimating potential costs. Further, without clearly defining expectations for agencies—including information such as the planned requirements for future phases of implementation—continuous evaluation may not be fully implemented across the executive branch. Incomplete implementation could potentially prevent the federal government from identifying security-relevant information in a timely manner, thereby exposing it to further national security risks, such as unauthorized disclosures of classified information. Limited planning, both by ODNI and at the agency level, ultimately puts the success of the continuous evaluation program—a key aspect of the security clearance reform effort—at risk.

ODNI Lacks Plans for Monitoring and Measuring Continuous Evaluation Program Performance

ODNI lacks a plan to monitor and measure the performance of continuous evaluation across executive branch agencies. Specifically, ODNI officials stated that ODNI has not developed a plan to monitor or assess the performance of continuous evaluation across the executive branch, including for the first phase of implementation, which is underway. ODNI officials stated that, ideally, agencies will report that they have met the fiscal year 2017 requirements for the first phase of implementation, and that ODNI will follow up with agencies that do not report. The officials added that, in the long term, ODNI would like to incorporate continuous evaluation into its Security Executive Agent National Assessment Program, through which it conducts oversight of the security clearance process at executive branch agencies, but that continuous evaluation is not currently included in the oversight program. As previously discussed, according to Executive Order 13467, ODNI, as the Security Executive Agent, is to direct the oversight of investigations, reinvestigations, adjudications, and, as applicable, polygraphs for individuals’ eligibility for access to classified information, or eligibility to hold a sensitive position made by any agency. Similarly, Executive Order 12968, as amended, indicates that ODNI is responsible for determining standards for continuous evaluation. According to ODNI, its authorities under the Executive Orders include responsibility for and oversight of continuous evaluation as it is an investigative activity that supports eligibility determinations.

Standards for Internal Control in the Federal Government emphasizes the importance of assessing performance over time, noting that ongoing monitoring should be built into operations, performed continually, and responsive to change. The PMBOK® Guide also states that project management includes monitoring and controlling work to meet performance objectives.

Without developing a plan to monitor continuous evaluation—including assessing continuous evaluation at various phases of implementation— ODNI cannot ensure that continuous evaluation is being conducted consistently across the executive branch, and it may experience challenges in identifying any needed modifications to the program. Further, ODNI cannot ensure that continuous evaluation is effectively meeting its critical purpose of filling the information gap between investigative cycles to identify risks to national security.

Additionally, we reported in 2012 that federal agencies engaging in large projects can use performance measures to determine how well they are achieving their goals and to identify any areas for improvement. Reporting on these measures can help key decision makers within agencies, as well as stakeholders, to obtain feedback for improving both policy and operational effectiveness. Moreover, performance measures need to provide managers and other stakeholders with timely, action- oriented information in a format that helps them make decisions that improve program performance. Throughout our body of work on leading performance management practices we have identified several attributes of successful performance measures, which include, among other things, measures that are clear, quantifiable, and objective, and that are linked to measurable goals.

However, ODNI has not developed and distributed to executive branch agencies performance measures to assess the effectiveness of continuous evaluation once it is implemented executive branch-wide. ODNI officials stated that they would like to collect metrics in order to determine the potential effects of continuous evaluation, in particular on agency resources. Although these officials stated that they have had some discussions with DOD about the types of metrics it might want to collect, such as the number of false positives and the resources required to address the workload, ODNI has not prioritized the development of performance measures. In February 2017, ODNI officials stated that they had not developed—or distributed to DOD or other agencies conducting continuous evaluation—any performance measures for continuous evaluation. These officials stated that once continuous evaluation has matured, ODNI plans to identify appropriate measures and determine a mechanism to collect and analyze them. In August 2017, ODNI officials stated that they had developed a draft list of metrics for fiscal year 2017.

Once the metrics are finalized, these officials stated that they would issue guidance to agencies requesting them to report these metrics to ODNI.

However, since ODNI initiated the first phase of continuous evaluation in October 2016, without developing and distributing performance measures to executive branch agencies, it is unclear whether agencies are positioned to collect and report the information to ODNI for fiscal year 2017. Developing performance measures before the program fully matures could help it to identify potential program modifications needed prior to the next phase of implementation, as well as prior to full implementation. Further, without developing clear, quantifiable, and objective performance measures that are linked to measurable goals for agencies to track, and without determining a process and schedule for agencies to regularly report those measures, ODNI cannot ensure that the first phase of the program it has already initiated is effective or achieving similar results at all agencies, which could ultimately affect reciprocity.

DOD and State Have Designed, Piloted, and Evaluated Continuous Evaluation to Varying Extents

DOD and State have designed, piloted, and evaluated continuous evaluation, although their respective approaches have varied in scope, size, and duration—with DOD’s pilot involving the most record checks, the largest population, and the longest duration. As previously discussed, DOD’s efforts to design, pilot, and evaluate continuous evaluation have been ongoing for more than a decade, and they pre-date efforts at ODNI to develop and implement an executive branch-wide continuous evaluation program. According to ODNI officials, as of February 2017, DOD and State were the only agencies, other than ODNI, that had piloted continuous evaluation. ODNI officials stated that DOD and State’s pilots were conducted at the discretion of those agencies, and that while ODNI did not oversee them, the results of the pilots have helped inform ODNI’s development of an executive branch-wide program. These pilots were ongoing prior to ODNI’s December 2016 interim guidance outlining the fiscal year 2017 continuous evaluation requirements for executive branch agencies, and as a result, both DOD and State have taken different approaches to developing their programs.

DOD’s Continuous Evaluation Pilot

In October 2014, consistent with the Secretary of Defense’s March 2014 recommendation to implement continuous evaluation and DOD Instruction 5200.02, DOD initiated a continuous evaluation pilot that included approximately 100,000 military, civilian, and contractor clearance holders, using a limited set of trusted commercial and government data sources. DOD has conducted this pilot in a phased approach, increasing the number of cleared individuals enrolled over time, in accordance with enrollment milestones set as part of the Insider Threat and Security Clearance Reform cross-agency priority goal. Specifically, the department expanded enrollment to 225,000 DOD clearance holders in December 2015 and 500,000 in December 2016, and it plans to increase the enrolled population to 1 million by the end of calendar year 2017. The department has also set an internal goal to enroll all clearance holders department-wide by the end of fiscal year 2021.

DOD has developed its own continuous evaluation IT system—which is called Mirador, and is separate from the IT system that ODNI is developing—to conduct automated record checks of commercial and government data sources on the enrolled population, with the goal of near real-time identification of adverse information to be considered in the evaluation of an individual’s continued eligibility for access to classified information. DOD officials developing the system stated that while they are currently using Mirador to conduct automated record checks for continuous evaluation, the system remains under development, and they are integrating additional data sources and user requirements as those are identified. As of February 2017, the department had implemented seven data sources in Mirador, which provide information about suspicious financial and criminal activity, among other things. Another nine sources were undergoing testing or were otherwise in progress. The department expects Mirador to reach initial operating capacity in fiscal year 2018. DOD officials stated that aspects of Mirador are still manual, such as enrolling individuals, but that they plan to take steps to automate them.

DOD officials stated that, depending on the data source, they run record checks on enrolled individuals daily, monthly, quarterly, or annually. According to DOD officials, if a record check results in an alert, such as for criminal activity, Mirador forwards the alert to DOD’s continuous evaluation validation cell—within the Defense Security Service, which manages the department’s continuous evaluation program—to ensure that: (1) the alert applies to the correct individual; (2) the issue was not previously known; and (3) the issue is adjudicatively relevant. DOD officials stated that if an analyst determines that an alert is valid— meaning that all three of the above statements are believed to be true— then the analyst generates a report and forwards it to the individual’s designated security manager. Alerts are prioritized for analyst review according to business rules designed around the severity of the alert, and according to DOD officials, all alerts are reviewed by a supervisor following an analyst’s initial determination. The officials stated that currently, if additional investigative work is required based on the alert, the results of that investigation are forwarded to an adjudicator to make a determination as to whether the alert affects the individual’s continued eligibility for a security clearance. The officials added that the due process safeguards in place for periodic reinvestigations are also in place for continuous evaluation. Figure 2 provides an overview of DOD’s continuous evaluation process.

DOD has collected and analyzed metrics on the results of its current pilot. For example, according to DOD data, as of February 2017, continuous evaluation had identified 12,400 alerts. Of those alerts, 2,064—pertaining to 1,816 individuals—were determined to be valid, meaning that they were adjudicatively relevant and not previously known. According to DOD, action has been completed on 1,307 of those cases. Specifically, 859 cases were closed with a favorable decision, but context was added to the individuals’ records; in 375 cases the subject separated and/or no longer needed access; and 62 cases involved a clearance revocation, condition, or warning. For DOD’s secret-eligible population, continuous evaluation helped to identify risk, on average, 6 years 7 months sooner than the traditional 10-year periodic reinvestigation model, and 1 year 5 months earlier for the top secret-eligible population, which is to be reinvestigated every 5 years. DOD officials stated that these metrics are presently tracked manually by the Consolidated Adjudications Facility, and they identified a need to automate the process, going forward.

In addition, DOD officials stated that they have shared the results of the pilot and lessons learned with ODNI through the Continuous Evaluation Working Group. For example, DOD identified lessons learned related to identifying the right data sources, eliminating duplicate alerts, the frequency of record checks, methods for achieving identity resolution, and the need for operational access to reporting data. Most recently, DOD issued Department of Defense Manual 5200.02 in April 2017, which includes continuous evaluation among the responsibilities and procedures of the DOD Personnel Security Program.

State’s Continuous Evaluation Pilot

State began its continuous evaluation pilot in January 2015 to evaluate the coverage and reliability of public records information, using a public records service provider. Specifically, it compared information received from public record checks, such as criminal and financial activity, against information contained in personnel security files for approximately 8,600 personnel. State found, among other things, that while public records can provide coverage beyond the traditional scope of investigations, the quality of the information varies, and not all jurisdictions participate. State continued its pilot in 2016 and expanded the enrolled population to include its entire tier 5 population. Additionally, the focus of the pilot shifted from evaluating the usefulness of public records information to evaluating the alerts received. State officials stated that the results of the public record checks are reviewed by the department’s continuous evaluation team, which determines whether the information is new, accurate, and relevant, and if so, whether it needs further review and investigation. These officials stated that because State has authority to conduct its own investigations, it is easy to conduct investigative follow- up. According to officials, minor issues, such as traffic violations, are added to personnel files for consideration during the individual’s next periodic reinvestigation.

According to State officials, as of March 2017, they had not revoked any clearances as a result of the identification of derogatory information through continuous evaluation. As of April 2017, State had invested approximately $2.4 million in its continuous evaluation pilot for contract costs and personnel to administer the program, and, according to State officials, ODNI provided approximately one-third of that funding. State officials stated that because ODNI provided funding, State has voluntarily shared some lessons learned with ODNI, although it was not tasked to do so. Some details of State’s pilot were omitted because the information is sensitive.

Number of Agencies Meeting Periodic Reinvestigation Timeliness Goals Decreased from Fiscal Years 2012- 2016, and Potential Continuous Evaluation Effects Are Unknown

The number of executive branch agencies meeting established timeliness goals for completing periodic reinvestigations decreased from fiscal years 2012 through 2016. Additionally, while executive branch agencies have already initiated the first phase of continuous evaluation, the potential effects of continuous evaluation on periodic reinvestigations and agency resources are unknown, as they have not been assessed.

Executive Branch Agencies Meeting Established Timeliness Goals for Completing Periodic Reinvestigations Decreased from Fiscal Years 2012 through 2016

Our analysis of timeliness data for specific executive branch agencies showed that the percent of agencies meeting timeliness goals decreased from fiscal year 2012 through 2016. As part of the Insider Threat and Security Clearance Reform cross-agency priority goal, since the second quarter of fiscal year 2014, the PAC has reported quarterly on agency timeliness. Among other things, the PAC reports on the average number of days taken, for the executive branch as a whole, to complete the end- to-end process for periodic reinvestigations, as compared with the following goals for the fastest 90 percent of periodic reinvestigations: 15 days to initiate a case, 150 days to conduct the investigation, and 30 days to adjudicate—totaling 195 days to complete the end-to-end processing of the periodic reinvestigation.

For fiscal year 2016, the PAC reported that the executive branch as a whole: did not meet the goal of conducting the investigative portion of periodic reinvestigations within 150 days for the fastest 90 percent of cases for any quarter. The average number of days ranged from 175 days to 192 days. did not meet the goal of completing periodic reinvestigations—the end-to-end goal—within 195 days for any quarter of fiscal year 2016. The average ranged from 209 days to 227 days.

Our analysis of timeliness data for specific executive branch agencies showed that the percent of agencies that reported meeting timeliness goals decreased from fiscal year 2012 through 2016. Specifically, while 84 percent of the executive branch agencies met the 150-day investigative goal for at least three of four quarters for the fastest 90 percent of periodic reinvestigations in fiscal year 2012, only 18 percent of the agencies met the investigative goal in fiscal year 2016. while 84 percent of the executive branch agencies met the end-to-end processing goal of 195 days for at least three of four quarters for the fastest 90 percent of periodic reinvestigations in fiscal year 2012, only 22 percent of the agencies completed their fastest 90 percent of periodic reinvestigations within 195 days for at least three of four quarters in fiscal year 2016.

Of the agencies we reviewed, we found that agencies which use NBIB as their investigative service provider and agencies with delegated authority to conduct their own investigations both experienced challenges in meeting established timeliness goals for periodic reinvestigations in fiscal years 2015 and 2016. For example, 50 percent of the agencies with delegated authority completed investigations for at least three of four quarters for the fastest 90 percent of periodic reinvestigations within 150 days in fiscal year 2015, and 44 percent of agencies with delegated authority met the timeliness goal in fiscal year 2016. Of the executive branch agencies for which we obtained timeliness data from ODNI and which use NBIB as their investigative service provider, NBIB completed the investigative portion within 150 days for 0 percent of the agencies in fiscal year 2015, and completed it within that timeframe for 6 percent of the agencies in fiscal year 2016 for at least three of four quarters for the fastest 90 percent of reinvestigations. Of the executive branch agencies we reviewed, 67 percent met the adjudication timeliness goal of 30 days in fiscal year 2016 for at least three of four quarters for the fastest 90 percent of reinvestigations. Specific details on the timeliness of individual executive branch agencies’ periodic reinvestigations were omitted from this report because the information is sensitive.

According to NBIB officials, as of June 2017, NBIB’s investigation backlog totaled approximately 673,000 cases—about 183,000 of which were periodic reinvestigations for both tier 3 and tier 5 clearances. NBIB cited the September 2014 decision to not exercise the option of one of its investigative fieldwork contracts—which led to a loss in capacity and an increase in the program’s contract costs—and difficulties attracting and retaining investigative resources as two main challenges to timeliness. NBIB officials stated that they are taking steps to address the backlog for background investigations, including periodic reinvestigations. These steps include hiring additional federal and contract investigators, implementing a number of workload management initiatives, and conducting a business process reengineering review to identify potential process efficiencies. Additionally, executive branch agencies noted the increased requirements stemming from the 2012 Federal Investigative Standards, such as continuous evaluation and more frequent periodic reinvestigations for certain clearance holders, as additional challenges to meeting timeliness goals.

In 2008, the Joint Security and Suitability Reform Team issued Security and Suitability Process Reform, a report to the President that, among other things, includes OMB-issued interim government-wide processing goals for security clearances for calendar year 2008. The calendar year 2008 government-wide goal for the fastest 90 percent of periodic reinvestigations is the same as the goal currently in place: 195 days to complete the end-to-end processing of the periodic reinvestigation. The report states that OMB issued the interim goal to assist agencies in projecting workload and resource requirements.

However, the timeliness goals on which the PAC currently reports for periodic reinvestigations are the same as those identified by OMB as interim goals for calendar year 2008. Unlike initial investigations, for which timeliness objectives are established by statute, the 195-day goal for the end-to-end timeliness of periodic reinvestigations was an interim goal set by OMB for calendar year 2008. The 2008 report to the President does not detail how the goals were developed or what data, if any, were used to establish them. ODNI officials initially stated that they did not know how the 195-day goal was developed or where it was documented, and did not know whether subsequent, finalized goals were ever established, but they later provided a copy of the 2008 report. A senior NBIB official stated that OMB’s interim calendar year 2008 timeliness goals were developed based on the average timeliness of the fastest 90 percent of periodic reinvestigations at that time. Since the establishment of OMB’s interim goals, the executive branch has measured periodic reinvestigation timeliness against those goals, and it has not conducted an evidence-based review to ensure that 195 days— and the associated goals of the different phases of periodic reinvestigations—are realistic goals for periodic reinvestigations.

Standards for Internal Control in the Federal Government states that management evaluates and, if necessary, revises defined objectives so that they are consistent with requirements and expectations. Without conducting an evidence-based review of the goals, the executive branch will continue to compare the timeliness of its periodic reinvestigations against goals that it established almost a decade ago and that may no longer be appropriate. Further, without ensuring that 195 days, along with the associated goals of the different phases of periodic reinvestigations, are appropriate goals, agencies may not be adequately planning for the amount of time and resources actually required to conduct periodic reinvestigations, and, as a result, they may experience further timeliness delays. Moreover, if an agency does not plan for sufficient time to conduct periodic reinvestigations, it may allow individuals to retain access to sensitive documents when it has not yet confirmed those individuals’ continued eligibility, which could have potential repercussions for national security.

Potential Effects of Continuous Evaluation on Periodic Reinvestigations Are Unknown

The potential effects of continuous evaluation on periodic reinvestigations, such as possible changes to their frequency or scope, remain unknown. In addition, the executive branch’s plans for replacing periodic reinvestigations with continuous evaluation have evolved over time. For example, the 2008 Security and Suitability Process Reform report to the President outlined plans to replace the periodic reinvestigation model with continuous evaluation, conducting continuous evaluation annually or at least once every 5 years, depending on an individual’s security clearance level. The report identified a June 2009 milestone to develop an implementation plan to transition from periodic reinvestigations to continuous evaluation, and as previously discussed, an estimated operational date of the fourth quarter of fiscal year 2010 (see figure 1). The purpose of the change was to reveal security-relevant information earlier and to provide increased scrutiny on populations that could potentially represent risk to the government because they already have access to classified information. However, ODNI documentation states that continuous evaluation supplements and enhances, but does not replace, established personnel security processes.

Executive branch agencies have expressed varying views about potential changes to the periodic reinvestigation model. For example, DOD officials stated that with workload and funding issues, they see no alternative but to replace periodic reinvestigations for certain clearance holders with continuous evaluation, as the record checks conducted are the same for both processes. In addition, DOD officials stated that they believe continuous evaluation will not only result in the more timely identification of security-relevant information, but will also help to change individuals’ behaviors—for example, that individuals will be more likely to self-report such information once they are enrolled in the program. DOD officials also noted that if changes are not made to the periodic reinvestigation process, the investigation backlog will persist, because continuous evaluation alerts will continue to add to the investigative workload.

In addition, in September 2016, PERSEREC issued a report on a study it conducted on the effectiveness, timeliness, and cost of various automated record checks-based investigative strategies as compared with traditional periodic reinvestigations. The analysis found that some of the automated record checks strategies were effective, improved the timeliness of issue detection, and lowered costs. However, DOD officials noted that because ODNI is the Security Executive Agent, it must approve the change to the investigative process. These officials stated that they hope to influence this change by demonstrating the effectiveness of continuous evaluation at DOD. Additionally, NBIB officials stated that continuous evaluation will increase their workload and costs, since it is an additional layer to the personnel security clearance process. Accordingly, they hope that ODNI will identify efficiencies that can be made to the process. Further, PAC Program Management Office officials stated that there may be changes to the periodic reinvestigation model in the future, but that any changes to the model will be determined by data and will be made under the authority of ODNI and OPM as the Security Executive Agent and the Suitability Executive Agent, respectively.

Other agencies, such as State, do not share DOD’s view. For example, State officials stated that although a reduction in costs would result from replacing periodic reinvestigations with continuous evaluation, they have concerns that relevant information, such as state and local law enforcement records that are not yet automated, would be missed if they did not conduct periodic reinvestigations. Similarly, officials from the Department of Justice and the Department of Homeland Security stated that they do not intend to replace periodic reinvestigations, and that continuous evaluation is to be a supplement to the personnel security clearance process. However, officials from all three of these agencies stated that it may be possible to change the frequency or scope of periodic reinvestigations at some point in the future. ODNI officials stated that, at this time, they have no intention of replacing periodic reinvestigations with continuous evaluation, and that the Security Executive Agent Directive for continuous evaluation, once issued, will clarify that continuous evaluation is intended to supplement and not replace periodic reinvestigations.

In May 2017, ODNI officials stated that ODNI is not opposed to further improving the security clearance process, and that once continuous evaluation is operational, it plans to determine the efficiencies and mitigation of risks associated with the approach. Specifically, these officials stated that once continuous evaluation is further implemented and ODNI has gathered sufficient data—which they estimated would take about a year from May 2017—they can perform analysis and research to determine whether any changes are needed to the periodic reinvestigation model.

While executive branch agencies have different views about potential changes to the periodic reinvestigation process, officials from five of the seven executive branch agencies we spoke with identified the potential expenditure of increased resources, such as workload and costs, as a risk associated with the implementation of continuous evaluation. Specifically, all five agencies stated that continuous evaluation will increase their workloads—and therefore costs—if no other changes are made to the personnel security process. For example, DOD officials noted that adjudicator workloads will increase as new investigative leads—identified through continuous evaluation—require adjudication. Senior DOD officials stated that DOD cannot afford to conduct both continuous evaluation and periodic reinvestigations. Specifically, DOD estimates that implementing the 2012 Federal Investigative Standards requirement to conduct more frequent periodic reinvestigations for certain clearance holders will cost approximately $1.8 billion for fiscal years 2018 through 2022. In addition, State officials stated that they anticipate that continuous evaluation will increase their personnel security workload because alerts will have to be validated, and potentially investigated, and then adjudicated.

Standards for Internal Control in the Federal Government states that management should identify, analyze, and respond to risks related to achieving defined objectives. Risk assessment is the identification and analysis of risks related to achieving defined objectives to form a basis for designing risk responses. In addition, the PMBOK® Guide states that entities should perform a quantitative risk analysis to numerically analyze the effect of identified risks on overall project objectives. The key benefit of this process is that it produces quantitative risk information to support decision-making in order to reduce project uncertainty.

Although executive branch agencies have identified increased resources as a risk associated with implementing continuous evaluation, and ODNI has acknowledged that risk, ODNI, in coordination with the PAC, has not assessed the potential effects of continuous evaluation on an agency’s resources. Further, ODNI has not developed a plan, in consultation with implementing agencies, to address such effects, to include modifying the scope or frequency of periodic reinvestigations or replacing periodic reinvestigations for certain clearance holders. While ODNI is implementing continuous evaluation in a phased approach, having a plan in place to address the increased workload once continuous evaluation is fully implemented is critical to ensuring the sustainability and effectiveness of executive branch agencies’ personnel security programs. Further, without assessing the potential impacts on agency resources and developing a plan to address them—once ODNI has further defined the program—implementing continuous evaluation could further increase the periodic reinvestigation backlog and agency costs. With delays in determining continued eligibility, executive branch agencies are assuming greater risk, which runs counter to the purpose of continuous evaluation.

Conclusions

Continuous evaluation has been a key and long-standing initiative of security clearance reform efforts, intended to assist agencies in the timely identification of security-relevant information that may affect an individual’s continued eligibility for access to classified information. However, ODNI has not demonstrated the leadership necessary to make continuous evaluation a priority. Accordingly, the program’s implementation has been delayed for almost 7 years. Although ODNI has taken an initial step to implement it in a phased approach, it has not yet formalized the program in policy or provided an expanded definition of continuous evaluation to implementing agencies. In addition, ODNI has not yet determined key aspects of the program, including future phases of implementation and agency requirements. Key executive branch agencies have deemed information about the future phases necessary to plan for the implementation of continuous evaluation and to estimate potential costs. The absence of this information has limited their ability to prepare for the next phases of implementation. This could further delay the full implementation of continuous evaluation executive branch-wide and result in inconsistencies among agencies’ approaches. Specifically, in the absence of ODNI policy and comprehensive guidance, DOD and State continue to develop their current continuous evaluation programs. The ultimate effects of such inconsistencies could negatively affect reciprocity—another key government-wide security clearance reform effort.

Although ODNI is to have oversight of continuous evaluation, it has not incorporated it into its oversight program or developed a plan to ensure that agencies implement it. Without a Security Executive Agent Directive for continuous evaluation that provides an expanded definition of continuous evaluation and relevant terms to help ensure consistent use; a plan for implementing continuous evaluation across the executive branch, that includes future phases of implementation and expectations for agencies; and a plan for monitoring program performance throughout the implementation process, as well as performance measures by which to track and report progress, ODNI is not well-positioned to ensure the success and effectiveness of the continuous evaluation initiative. Further, ODNI does not know whether it is meeting the critical purpose of filling the information gap between investigative cycles to identify risks to national security.

Executive branch timeliness in completing periodic reinvestigations has declined over the past five years. Further, the executive branch does not know whether the timeliness goals—set nearly a decade ago—are still relevant and appropriate, given changes to the personnel security clearance process. Without conducting an evidence-based review to ensure that goals for the timely completion of periodic reinvestigations are appropriate, executive branch agencies may not be planning sufficient time and resources to complete periodic reinvestigations and therefore may be challenged to ensure the continued eligibility of the entire national security workforce. Finally, executive branch agencies have identified increased resources, such as workload and costs, as a challenge to implementing continuous evaluation. However, the executive branch has not determined the potential expected effects of continuous evaluation on periodic reinvestigations, and agencies have varying views about what, if any, additional changes should be made to the personnel security clearance process. Without an assessment of the potential effects of continuous evaluation and a plan to address those effects—once ODNI has further defined the program—agencies may not be able to effectively integrate continuous evaluation into their personnel security clearance programs, which in turn could lead to further delays in the clearance process.

Recommendations for Executive Action

We are making the following six recommendations to ODNI:

The Director of National Intelligence should issue a Security Executive Agent Directive for continuous evaluation to formalize the program, which includes, among other things, an expanded definition of continuous evaluation in advance of the next phase of implementation. (Recommendation 1)

The Director of National Intelligence should, in coordination with the Continuous Evaluation Working Group, develop an implementation plan for continuous evaluation across the executive branch that includes a schedule with timeframes and expectations for agencies, such as the requirements (e.g., the size of the enrolled population in continuous evaluation) for future phases of implementation. (Recommendation 2)

The Director of National Intelligence should develop a plan for monitoring continuous evaluation performance, to include assessing continuous evaluation at various phases of implementation. (Recommendation 3)

The Director of National Intelligence should develop performance measures for continuous evaluation that agencies must track and determine a process and schedule for agencies to regularly report those measures to ODNI. At minimum, these performance measures should be clear, quantifiable, objective, and linked to measurable goals. (Recommendation 4)

The Director of National Intelligence should, in coordination with the Deputy Director for Management of the Office of Management and Budget in the capacity as Chair of the Security, Suitability, and Credentialing Performance Accountability Council, conduct an evidence-based review of the timeliness goal of 195 days for completing the fastest 90 percent of periodic reinvestigations and the associated goals for the different phases of periodic reinvestigations, and adjust the goal if appropriate, taking into consideration available resources, the additional workload of continuous evaluation, and the risks associated with individuals retaining access to classified information without determining their continued eligibility. (Recommendation 5)

The Director of National Intelligence should, once ODNI has further defined the continuous evaluation program, to include issuing a Security Executive Agent Directive and developing an implementation plan, in coordination with the Deputy Director for Management of the Office of Management and Budget in the capacity as Chair of the Security, Suitability, and Credentialing Performance Accountability Council, assess the potential effects of continuous evaluation on agency resources and develop a plan, in consultation with implementing agencies, to address those effects, such as modifying the scope of periodic reinvestigations, changing the frequency of periodic reinvestigations, or replacing periodic reinvestigations for certain clearance holders. (Recommendation 6)

Agency Comments and Our Evaluation

We provided a draft of this report to ODNI, DOD, OMB, State, NBIB, the Department of Justice, and the Department of Homeland Security for review and comment. Written comments from ODNI are reprinted in their entirety in appendix I. DOD, OMB, NBIB, and the Department of Homeland Security did not provide comments. ODNI, State, and the Department of Justice provided technical comments, which we incorporated in the report as appropriate.

In its written comments, ODNI stated that it generally concurred, with comments, with our six recommendations. However, ODNI stated that it did not concur with aspects of our overall conclusions and provided observations in four specific areas. We continue to believe that our conclusions are valid, as discussed below.

First, ODNI disagreed with our conclusion that it has not demonstrated the leadership necessary to make continuous evaluation a priority. ODNI noted that it has taken recent actions to better prioritize the implementation of continuous evaluation. While these recent steps are positive and may help position ODNI for success, historically ODNI has not demonstrated the leadership necessary to make the implementation of a continuous evaluation program a priority. Specifically, while ODNI refers to continuous evaluation as a new initiative, the original milestone for implementing the program was the fourth quarter of fiscal year 2010, which was not attained. Since then, as discussed in the report, a number of revised milestones for implementing the program have been missed. For example, the PAC, of which ODNI is a principal member, subsequently set a milestone for developing an initial continuous evaluation capability for other clearance holders by September 2014— which was extended to December 2014—and a milestone for implementing the capability for other clearance holders by December 2016. These milestones were also missed. As of August 2017, continuous evaluation has not yet been fully implemented, and ODNI has not set a milestone for when full implementation would occur. As such, we recommended specific actions that are needed to better position ODNI for success, including issuing a Security Executive Agent Directive for continuous evaluation, developing plans for implementing the program and monitoring its performance, and developing performance measures.

Second, ODNI disagreed with our conclusion that it has not yet determined key aspects of the continuous evaluation program, including future phases of implementation and agency requirements. ODNI stated that the Security Executive Agent Directive for continuous evaluation is undergoing interagency coordination and that it has provided executive branch agencies with interim guidance until that process is completed, which we acknowledge in the report. While ODNI has provided interim guidance for continuous evaluation, it only details the requirements for fiscal year 2017 and not for the future phases of implementation. In August 2017, after receiving a draft of our report, ODNI officials stated that they planned to provide additional guidance to agencies clarifying that the requirements for fiscal year 2018 will be the same as those for fiscal year 2017. While this correspondence, once issued, will help agencies with their immediate program planning, ODNI officials stated that they have not yet determined the requirements for fiscal year 2019 or beyond, which limits agencies’ abilities to plan beyond the next fiscal year for the future phases of implementation.

Additionally, ODNI stated that the technical development milestones of the Continuous Evaluation System it is developing are well-established, tracked, and shared with stakeholders. As discussed in the report, according to ODNI officials, they have established technical milestones for the development of ODNI’s Continuous Evaluation System. While this is an important step in implementing the program, ODNI has not developed similar programmatic milestones for the overall implementation of the program, such as when future phases of implementation will occur, to include full implementation. As discussed in the report, this has limited the ability of executive branch agencies to plan for implementation in accordance with ODNI’s phased approach. As a result, full implementation—which has been delayed for almost 7 years—may be further delayed.

Third, ODNI did not agree with our conclusion that although it is to have oversight of continuous evaluation, it has not incorporated it into its oversight program or developed a plan to ensure agencies implement it. In its response, ODNI identified its intention to take certain actions and future mechanisms that could position it to monitor continuous evaluation. Specifically, ODNI stated that continuous evaluation metrics will be collected and analyzed when the initial phase of continuous evaluation implementation ends on September 30, 2017. Additionally, ODNI stated that it will leverage a pending OMB budget data request and that its Security Executive Agent National Assessments Program will be responsible for analysis and oversight of agency implementation and operation of continuous evaluation.

However, as we note in the report, ODNI has not developed and distributed plans to monitor or assess the performance of continuous evaluation across the executive branch, including for the first phase of implementation. As we note in our report, ODNI officials stated that ODNI did not oversee the pilots that were conducted by DOD and State, as they were performed at the discretion of those agencies. State officials noted that while they have shared lessons learned on their continuous evaluation pilot, they were not tasked to do so. While ODNI stated in its written comments that it has specific expertise in researching, measuring, analyzing, and monitoring personnel security performance across the executive branch, it has not yet demonstrated these actions with regard to continuous evaluation. For example, DOD—the executive branch agency with the majority of security clearance holders—has conducted research on continuous evaluation since 2001, piloted its program since October 2014, and plans to increase the number of personnel enrolled in the program to 1 million by the end of calendar year 2017. However, ODNI, in the capacity as the Security Executive Agent, has not overseen DOD’s pilot.

Moreover, as discussed in the report, as of August 2017—10 months into fiscal year 2017—ODNI has not yet developed and distributed to executive branch agencies continuous evaluation performance measures. At the end of our review, in August 2017, ODNI officials stated that they have developed a draft list of metrics for continuous evaluation for fiscal year 2017 and that once the metrics are finalized, they will issue guidance to executive branch agencies requesting them to report these metrics to ODNI. While metrics can help to establish a baseline and inform aspects of a program’s status—and ODNI’s development of draft metrics is a positive step—performance measures are linked to a goal and inform how well an agency is doing against that goal. As ODNI has not developed and distributed performance measures that are clear, quantifiable, and objective, and that are linked to measurable goals prior to initiating, or earlier in the first phase of implementation, executive branch agencies may not be positioned to collect and report these metrics at the end of the fiscal year. Additionally, as discussed in the report, according to ODNI officials, while they would like to incorporate continuous evaluation into their Security Executive Agent National Assessments Program, it is not currently part of the program. While ODNI has identified steps that could position it to monitor continuous evaluation in the future, it has not yet implemented mechanisms to monitor and measure program performance.

Fourth, ODNI did not agree with our conclusions that it is not well- positioned to ensure the success and effectiveness of the continuous evaluation initiative, and that it does not know if it is meeting the critical purpose of filling the information gap between investigative cycles to identify risks to national security. However, in its written comments, ODNI stated that successful implementation of continuous evaluation across the executive branch requires formal Security Executive Agent policy guidance, implementation and technical guidance and milestones, performance measures, and a monitoring program, which we recommended in the report. ODNI states that it is well-postured to achieve these goals, and refers to its intention to apply Security Executive Agent National Assessments Program best practices as a mechanism to use to monitor and ensure compliance. Although this action could be a step in better positioning ODNI as continuous evaluation implementation further proceeds, as noted above and in our report, ODNI has not yet finalized, distributed, and implemented these and other actions to ensure that it is currently positioned to ensure success, even while it has initiated the first phase of continuous evaluation implementation.

As noted in our report, although ODNI has taken steps to implement continuous evaluation in a phased approach, executive branch efforts to implement continuous evaluation have been a long-standing component of overall security clearance reform. The actions ODNI intends to take as it further implements continuous evaluation, as well as the mechanisms it identified, may better position it and the implementing agencies for success. However, given the challenges that the executive branch has faced in implementing continuous evaluation thus far and the continued delays it has faced, without a fully defined program in place, we believe that our conclusions remain valid.

Finally, in its written comments, ODNI suggested a revision to our sixth recommendation. Specifically, ODNI suggested adding an explicit timeframe for completing the action. We believe that ODNI is best positioned to set an appropriate timeframe for completion based on its familiarity with the progress of the program and, as such, did not incorporate this change in our report. We agree with ODNI that establishing such a timeframe is a positive step.

We are sending copies of this report to the appropriate congressional committees, the Director of National Intelligence, the Secretary of Defense, the Director of OMB, the Secretary of State, the Secretary of Homeland Security, the Director of OPM, the Director of NBIB, the Attorney General, the Director of the Federal Bureau of Intelligence, and the Director of the Bureau of Alcohol, Tobacco, Firearms, and Explosives. In addition, this report will also be available at no charge on the GAO website at http://www.gao.gov.

If you or your members of your staff have any questions regarding this report, please contact me at (202) 512-3604 or farrellb@gao.gov. GAO staff who made significant contributions to this report are listed in appendix II.

Appendix I: Comments from the Office of the Director of National Intelligence

Appendix II: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Kimberly C. Seay (Assistant Director), Chris Businsky, Molly Callaghan, Jenny Chanley, Dawn Godfrey, Saida Hussain, James Krustapentus, Michael Shaughnessy, Rachel R. Stoiko, John Van Schaik, Cheryl Weissman, and Jina Yu made significant contributions to this report.

Related GAO Products

High-Risk Series: Progress on Many High-Risk Areas, While Substantial Efforts Needed on Others. GAO-17-317. Washington, D.C.: February 15, 2017.

Personnel Security Clearances: Funding Estimates and Government- Wide Metrics Are Needed to Implement Long-Standing Reform Efforts. GAO-15-179SU. Washington, D.C.: April 23, 2015.

Personnel Security Clearances: Additional Guidance and Oversight Needed at DHS and DOD to Ensure Consistent Application of Revocation Process. GAO-14-640. Washington, D.C.: September 8, 2014.

Personnel Security Clearances: Actions Needed to Ensure Quality of Background Investigations and Resulting Decisions. GAO-14-138T. Washington, D.C.: February 11, 2014.

Personnel Security Clearances: Opportunities Exist to Improve Quality Throughout the Process. GAO-14-186T. Washington, D.C.: November 13, 2013.

Personnel Security Clearances: Full Development and Implementation of Metrics Needed to Measure Quality of Process. GAO-14-157T. Washington, D.C.: October 31, 2013.

Personnel Security Clearances: Further Actions Needed to Improve the Process and Realize Efficiencies. GAO-13-728T. Washington, D.C.: June 20, 2013.

Managing for Results: Agencies Should More Fully Develop Priority Goals under the GPRA Modernization Act. GAO-13-174. Washington, D.C.: April 19, 2013.

Security Clearances: Agencies Need Clearly Defined Policy for Determining Civilian Position Requirements. GAO-12-800. Washington, D.C.: July 12, 2012.

Personnel Security Clearances: Continuing Leadership and Attention Can Enhance Momentum Gained from Reform Effort. GAO-12-815T. Washington, D.C.: June 21, 2012. 2012 Annual Report: Opportunities to Reduce Duplication, Overlap and Fragmentation, Achieve Savings, and Enhance Revenue. GAO-12-342SP. Washington, D.C.: February 28, 2012.

Background Investigations: Office of Personnel Management Needs to Improve Transparency of Its Pricing and Seek Cost Savings. GAO-12-197. Washington, D.C.: February 28, 2012.

GAO’s 2011 High-Risk Series: An Update. GAO-11-394T. Washington, D.C.: February 17, 2011.

High-Risk Series: An Update. GAO-11-278. Washington, D.C.: February 16, 2011.

Personnel Security Clearances: Overall Progress Has Been Made to Reform the Governmentwide Security Clearance Process. GAO-11-232T. Washington, D.C.: December 1, 2010.

Personnel Security Clearances: Progress Has Been Made to Improve Timeliness but Continued Oversight Is Needed to Sustain Momentum. GAO-11-65. Washington, D.C.: November 19, 2010.

DOD Personnel Clearances: Preliminary Observations on DOD’s Progress on Addressing Timeliness and Quality Issues. GAO-11-185T. Washington, D.C.: November 16, 2010.

Personnel Security Clearances: An Outcome-Focused Strategy and Comprehensive Reporting of Timeliness and Quality Would Provide Greater Visibility over the Clearance Process. GAO-10-117T. Washington, D.C.: October 1, 2009.

Personnel Security Clearances: Progress Has Been Made to Reduce Delays but Further Actions Are Needed to Enhance Quality and Sustain Reform Efforts. GAO-09-684T. Washington, D.C.: September 15, 2009.

Personnel Security Clearances: An Outcome-Focused Strategy Is Needed to Guide Implementation of the Reformed Clearance Process. GAO-09-488. Washington, D.C.: May 19, 2009.

DOD Personnel Clearances: Comprehensive Timeliness Reporting, Complete Clearance Documentation, and Quality Measures Are Needed to Further Improve the Clearance Process. GAO-09-400. Washington, D.C.: May 19, 2009.

High-Risk Series: An Update. GAO-09-271. Washington, D.C.: January 2009.

Personnel Security Clearances: Preliminary Observations on Joint Reform Efforts to Improve the Governmentwide Clearance Eligibility Process. GAO-08-1050T. Washington, D.C.: July 30, 2008.

Personnel Clearances: Key Factors for Reforming the Security Clearance Process. GAO-08-776T. Washington, D.C.: May 22, 2008.

Employee Security: Implementation of Identification Cards and DOD’s Personnel Security Clearance Program Need Improvement. GAO-08-551T. Washington, D.C.: April 9, 2008.

Personnel Clearances: Key Factors to Consider in Efforts to Reform Security Clearance Processes. GAO-08-352T. Washington, D.C.: February 27, 2008.

DOD Personnel Clearances: DOD Faces Multiple Challenges in Its Efforts to Improve Clearance Processes for Industry Personnel. GAO-08-470T. Washington, D.C.: February 13, 2008.

DOD Personnel Clearances: Improved Annual Reporting Would Enable More Informed Congressional Oversight. GAO-08-350. Washington, D.C.: February 13, 2008.

DOD Personnel Clearances: Delays and Inadequate Documentation Found for Industry Personnel. GAO-07-842T. Washington, D.C.: May 17, 2007.

High-Risk Series: An Update. GAO-07-310. Washington, D.C.: January 2007.

DOD Personnel Clearances: Additional OMB Actions Are Needed to Improve the Security Clearance Process. GAO-06-1070. Washington, D.C.: September 28, 2006.

DOD Personnel Clearances: Some Progress Has Been Made but Hurdles Remain to Overcome the Challenges That Led to GAO’s High-Risk Designation. GAO-05-842T. Washington, D.C.: June 28, 2005.

High-Risk Series: An Update. GAO-05-207. Washington, D.C.: January 2005.