Various statutes, regulations, and presidential policy protect different groups of DOD civilians and contractors from acts of reprisal for whistleblowing. Specifically, these groups comprise (1) DOD appropriated-fund employees, (2) DOD NAFI employees, (3) DOD contractor, subcontractor, grantee, and subgrantee employees, and (4) DCIPS employees of the defense intelligence community and any employee with eligibility for access to classified information. Table 1 summarizes the statutory and policy authorities covering each employee group, along with selected protected disclosures and prohibited personnel actions—which are two required elements of the test for determining whether there was reprisal against a complainant for whistleblowing. A protected disclosure is a disclosure of wrongdoing by a whistleblower to a party that is an eligible recipient of that disclosure, while prohibited personnel actions are those actions that are taken or threatened in response to a protected disclosure, such as termination, reassignment, or a significant change in duties, responsibilities, or working conditions. DODIG, the Office of Special Counsel, and the IGs of the defense intelligence components have responsibility for investigating whistleblower reprisal complaints brought by DOD civilian, contractor, subcontractor, grantee, and subgrantee employees. Among other matters, DODIG is responsible for investigating and overseeing component IG investigations of complaints alleging reprisal against certain DOD civilian employees and for conducting investigations of complaints alleging reprisal against DOD contractor, subcontractor, grantee, and subgrantee employees. Specifically: Civilian NAFI employees—DODIG is to investigate all reprisal complaints from these employees under 10 U.S.C. § 1587. DOD Directive 1401.03 requires that DODIG submit investigation reports involving NAFI employees to the Director of Administration and Management, who has responsibility for final adjudication and the direction of corrective action. DOD contractor, subcontractor, grantee and subgrantee employees— DODIG is responsible for investigating all complaints of reprisal under 10 U.S.C. § 2409. Civilian appropriated-fund employees—DODIG may investigate, on a discretionary basis, reprisal complaints from these employees that are of particular interest to DODIG under the Inspector General Act of 1978, as amended. DODIG typically retains for investigation those appropriated-fund civilian cases received through the DODIG hotline that involve retaliation for making a sexual-assault complaint, misconduct by senior DOD officials, and other high-profile issues that may be of interest to DODIG. While DODIG may exercise its discretionary authority to investigate appropriated-fund cases it receives, the Office of Special Counsel has primary jurisdiction to investigate the majority of civilian whistleblower reprisal cases across the federal government, including those involving DOD appropriated- fund civilians. The Office of Special Counsel has the authority to seek corrective actions, including through mediation with employing agencies. DCIPS employees and any employees with eligibility for access to classified information—DODIG is to conduct investigations of reprisal involving these employees and oversee the investigation of such complaints by the defense intelligence components under PPD-19 Part A and Directive-Type Memorandum 13-008, DOD’s implementing guidance. The Under Secretary of Defense for Intelligence issued Directive-Type Memorandum 13-008 to establish DOD policy, assign responsibilities, and provide procedures for the implementation of PPD-19 by the DOD components, which include the defense intelligence components and their IGs. Under PPD-19 Part B, DODIG is to investigate reprisal allegations involving actions affecting eligibility for access to classified information. Under the Intelligence Authorization Act for Fiscal Year 2014 and the Intelligence Authorization Act for Fiscal Year 2010, the defense intelligence component IGs—the IGs of the Defense Intelligence Agency, the National Geospatial-Intelligence Agency, the National Reconnaissance Office, and the National Security Agency—have independent statutory authority to conduct investigations of reprisal complaints brought by DCIPS employees. DOD Directive-Type Memorandum 13-008, which implements PPD-19 within DOD, requires the defense intelligence component IGs to notify DODIG of whistleblower reprisal allegations they receive, and grants DODIG the authority to retain such cases for investigation or refer them back to the component IG, in which case DODIG conducts oversight of the investigation. Upon receipt of a whistleblower reprisal complaint from a DOD civilian, contractor, subcontractor, grantee, or subgrantee employee, DODIG follows a standard intake process to analyze the complaint for a prima facie allegation. According to DODIG’s AI Investigations Manual, the elements of prima facie generally consist of: (1) whether a protected whistleblowing disclosure was made, (2) whether the alleged reprisal action followed the disclosure, (3) whether the subjects alleged to have taken the alleged reprisal action could reasonably have had knowledge of the protected disclosure, and (4) whether the alleged facts support an inference of reprisal. If the complaint is retained by DODIG for investigation, DODIG employs a standard investigative process. Figure 1 provides a summary of DODIG’s intake and investigation process from complaint receipt through case closure. CIGIE’s Quality Standards for Federal Offices of Inspector General and Quality Standards for Investigations collectively provide a set of overarching principles that IGs should adhere to in conducting their operations, including whistleblower reprisal investigations. Specifically, CIGIE’s Quality Standards for Federal Offices of Inspector General sets forth the overall quality framework for managing, operating, and conducting the work of IGs by providing guidance on ethics, independence, confidentiality, internal control, quality assurance, and professional standards, among other things. CIGIE’s Quality Standards for Investigations provides a framework for conducting high-quality investigations through general and qualitative investigative standards. General standards, among other things, address the qualifications of investigators, independence, and the concept of due professional care. Qualitative standards focus on investigative planning, execution, reporting, and information management. DODIG generally did not meet statutory and internal timeliness goals for conducting and overseeing DOD civilian and contractor reprisal cases from fiscal years 2013 through 2015, but improved its timeliness for some categories of cases, as well as its intake process and oversight reviews. As a result of our review, DODIG implemented a process to ensure that information is updated in its case-management system so that it can accurately determine the total length of NAFI civilian reprisal cases. Further, DODIG has taken and plans to take additional steps to improve the timeliness of conducting and overseeing investigations but has not assessed the feasibility of collecting key workload data that would enable it to more fully assess its personnel requirements and has not reported regularly to Congress on the timeliness of civilian and contractor reprisal investigations. DODIG generally did not meet its statutory and internal timeliness goals for the DOD civilian and contractor whistleblower reprisal investigations that it closed in fiscal years 2013 through 2015, but improved timeliness for some categories of cases, as well as its intake process and oversight reviews during that time. Table 2 shows the statutory and internal timeliness goals for DODIG civilian and contractor employee reprisal investigations, intake reviews, and oversight reviews. Our review found that DODIG received 1,208 civilian and contractor complaints from fiscal year 2013 through fiscal year 2015 and closed 1,155 cases by declining (i.e., declining to investigate), dismissing, or investigating complaints. Of the 1,155 cases DODIG closed, 103 were fully investigated. Overall, DODIG did not meet statutory or internal timeliness goals in about 83 percent of the investigations it closed in fiscal years 2013 and 2014, and about 84 percent of the investigations it closed in fiscal year 2015. However, the average length of all closed investigations improved by about 20 percent (99 days), decreasing from 505 days in fiscal year 2013 to 406 days in fiscal year 2015. Further, the aggregate median case length of investigations decreased by about 34 percent (170 days), from 503 days in fiscal year 2013 to 333 days in fiscal year 2015. Figure 2 shows the average and median case length by employee group across each fiscal year, as well as the range of these cases and the percentage of cases meeting the timeliness target. Our review found that DODIG’s timeliness in conducting intake reviews and oversight reviews also improved across the 3 fiscal years, although DODIG did not always meet timeliness goals. Specifically, DODIG’s average time for conducting intake reviews improved from 62 days in fiscal year 2013 to 49 days in fiscal years 2014 and 2015, a decrease of about 22 percent. DODIG met its timeliness goal in 50 percent of intake reviews in fiscal year 2013, 42 percent in fiscal year 2014, and 37 percent in fiscal year 2015. The time that DODIG took to conduct oversight reviews of investigations conducted by the defense intelligence components similarly decreased, from an average of 32 days in fiscal year 2013 to 24 days in fiscal year 2015, a decrease of 25 percent. DODIG met its 60-day timeliness goal in two of three such oversight reviews in fiscal year 2013 and all four reviews in fiscal year 2015. DODIG did not close any oversight reviews of investigations conducted by the defense intelligence components in fiscal year 2014. Appendix II provides information on the general characteristics of DOD civilian and contractor whistleblower reprisal cases. DODIG officials and investigators cited a number of factors that have affected DODIG’s ability to meet its statutory and internal timeliness goals, including case volume, the complexity of cases, DODIG’s internal review process, a focus on completing intake reviews, and an influx of new investigators, who require training and cannot initially handle the same caseload as more experienced investigators. Chief among these, according to DODIG officials, is case volume—particularly in relation to existing program resources. In an October 2016 letter to members of Congress, the Acting DOD Inspector General described DODIG’s increasing caseload, stating that DODIG’s budget had not kept pace with the growth in the department’s budget and that the DODIG budget clearly had not grown commensurate with its increase in responsibilities, particularly in the whistleblower area. Specifically, the Acting DOD Inspector General stated that in fiscal year 2016, DODIG received 1,594 whistleblower reprisal complaints and closed 1,492 cases, constituting 30 percent and 27 percent increases over fiscal year 2015, respectively. He added that while DODIG had increased its staff in the whistleblower reprisal unit from 28 in 2010 to 54 in 2016 (about 93 percent), these increases were not sufficient to keep up with the “burgeoning” workload. Specifically, DODIG analysis showed that over the same period, the number of incoming complaints and cases closed increased by 158 percent and 136 percent, respectively. In addition, 18 of the 24 investigators we interviewed stated that their intake and investigation caseload make it less feasible to meet timeliness goals, with 8 stating that the high number of complaints they process through intake can slow investigations. As a result of our review, DODIG implemented a process to ensure that information is updated in its case-management system so that it can accurately determine the total length of NAFI civilian reprisal cases. Per DOD policy, the Director of Administration and Management is the final adjudicative authority for NAFI civilian reprisal cases within DOD. Accordingly, upon completion of a NAFI investigation, DODIG forwards a copy of its report of investigation to the Director of Administration and Management for final adjudication. Once the case has been adjudicated, the Director of Administration and Management notifies the complainant of the investigation outcome and informs DODIG that the case is closed by sending to DODIG a signed copy of the closure letter. DODIG’s case- management system user guide requires that these closure letters be uploaded into the case-management system and that corresponding changes be made to the case status in the case-management system in order to accurately track the status and timeliness of these investigations. During the course of our case-file review, we reviewed case-closure information for each of the 11 NAFI investigations closed in fiscal year 2015 and identified discrepancies between the closure documentation sent to DODIG and the data in DODIG’s case-management system. These discrepancies can result in an inaccurate calculation of total case length for closed NAFI investigations. In working with DODIG officials to resolve these discrepancies, we asked that they provide us information on all 33 NAFI civilian investigations closed in fiscal years 2013 and 2014—resulting in a total of 44 of the 103 civilian and contractor investigations closed over the 3 fiscal years. Our analysis of this information showed that in 35 of the 44 NAFI investigations closed by DODIG from fiscal year 2013 through fiscal year 2015, DODIG had not updated data in its case-management system to reflect the receipt of case-closure documentation from the Director of Administration and Management indicating final case closure. As a result, DODIG was able to monitor its own timeliness in completing these investigations, but could not accurately determine the full length of NAFI reprisal cases as defined by the time a complainant files an allegation until the case is finally adjudicated by the Director of Administration and Management and the complainant is informed of the outcome. DODIG had not consistently updated its case-management system to accurately capture and track the length of closed NAFI civilian investigations because it did not have a process in place to ensure that these data are regularly updated in accordance with its guidance. CIGIE standards state that an investigative organization’s database should reflect accurate processing of information and final case information, while Standards for Internal Control in the Federal Government similarly state that management should obtain high-quality information based on relevant and reliable data. In addition, DODIG’s AI Investigations Manual states that, to comply with CIGIE standards, it is critical that investigators ensure that data fields are complete and accurate, including date fields such as case closure. During the course of our review, DODIG officials corrected the discrepancies we found and, in April 2017, provided us with documentation of a process they implemented to ensure that such updates are made in the future. Specifically, DODIG now requests that the Director of Administration and Management send the closure documentation it provides to complainants to a specific DODIG e- mail address. DODIG has assigned its investigative support specialists the task of retrieving NAFI closure documentation sent to this e-mail address and making corresponding updates to the case-management system. In addition, DODIG has developed a dashboard view in its case- management system that will allow the investigative support specialists to monitor the status of NAFI cases that are missing closure documentation from the Director of Administration and Management. By establishing a process for consistently updating its case-management system upon the receipt of closure documentation from the Director of Administration and Management, DODIG will be able to more accurately assess and report on the timeliness of NAFI investigations, as well as the effect of the length of these investigations, which is of importance to NAFI employees. DODIG has taken and has planned steps to improve the timeliness of its DOD civilian and contractor employee whistleblower reprisal investigations, but it has not collected key workload data that would enable it to more fully assess its personnel requirements. DODIG steps taken have included the following: Increasing its staff, as stated previously, from 28 in fiscal year 2010 to 54 in fiscal year 2016. As of April 2017, the whistleblower reprisal unit had 56 authorized civilian positions. Streamlining its oversight review process by reviewing only the report of an investigation instead of the report and its supporting documentation, unless the report of an investigation raises concerns. Specifically, DODIG officials stated that instead of reviewing all of the evidence in every case, DODIG now looks at the evidence if the report appears to be inconsistent, the report’s conclusions are not supported by the facts presented, or the report appears deficient in another respect. DODIG officials stated that the goal for completing this process has been reduced from 70 days or less to an average of 10 days. Modifying its case-management system in April 2016 by (1) launching an automated tool to help ensure compliance with statutory notification requirements for military and contractor reprisal cases in response to our 2015 recommendation that it implement such a tool, and (2) establishing new milestones to better track the timeliness of investigative phases. Developing templates for documentation such as routine correspondence and reports of investigations to standardize report elements and writing formats. Prioritizing investigations involving allegations of reprisal by senior officials in response to direction from the Office of the Secretary of Defense. DODIG is also working to lower the average case length for these cases to 120 days by 2017, according to DODIG officials. In addition to the steps that DODIG has taken, DODIG has also planned steps to further improve the timeliness of its investigations and oversight reviews. Chief among these is DODIG’s plan to further increase its staff by 29 personnel. According to DODIG officials, this increase was submitted as part of DODIG’s fiscal year 2018 budget request in order to address its “burgeoning” caseload. Specifically, these officials stated that the additional staff will help improve timeliness by creating two new investigative teams to backfill the team that was retasked in July 2016 to handle all DOD employee reprisal complaints involving sexual assault, and to reduce the caseload carried by investigators. The additional personnel will also allow DODIG to add another oversight team and to create an alternative dispute-resolution program to mediate complaints, which could improve both the timeliness and outcome of reprisal complaints received by DODIG, according to DODIG officials. In 2016, DODIG conducted an assessment of its personnel requirements in support of its fiscal year 2018 budget request for additional personnel, concluding, among other things, that the additional staff requested would enable DODIG to further improve timeliness and meet statutory and internal timeliness requirements. Specifically, DODIG’s assessment analyzed the average number of intakes, investigations, and oversight reviews per investigator, along with growth in complaints received and closed DOD-wide and the trend in DODIG incoming complaints, intakes, investigations, oversight reviews, and closed cases from fiscal years 2013 through 2015. In this analysis, DODIG found an increase in the volume of complaints and projected that the number of incoming complaints would continue to rise, thus affecting overall workload—a point borne out by additional data for fiscal year 2016 and the first half of fiscal year 2017 provided to us by DODIG officials. Specifically, DODIG projected that the number of incoming complaints would increase from 1,595 in fiscal year 2016 to 1,802 in fiscal year 2017 (13 percent) and the number of cases closed would increase from 1,495 to 1,854 (24 percent), consisting of an increase in cases closed at intake from 697 to 886 (27 percent) and an increase in closed oversight reviews, from 755 to 930 (23 percent). The number of closed investigations was projected to decrease from 43 in fiscal year 2016 to 38 in fiscal year 2017 (12 percent), which DODIG officials attributed to the steady increase in complaints received and intakes conducted. DODIG officials stated that the increasing caseload both affects timeliness and constitutes the basis for the fiscal year 2018 budget request for additional staff. In addition, these officials told us that they have taken action to continually monitor personnel requirements in response to increasing caseload and make personnel adjustments. For instance, DODIG officials told us that, based on data showing an increase in oversight reviews of the military service IGs, they created a dedicated oversight team in fiscal year 2015, and later added a second team. Although DODIG has assessed its personnel requirements and requested additional staff to, in part, improve timeliness, it has not collected workload data that could help it identify factors affecting timeliness and enable it to more fully assess its personnel requirements. CIGIE standards for investigations state that an IG’s management information system should collect workload data, including the amount of direct and indirect labor hours expended on each case, where appropriate. Further, DOD guidance states that personnel requirements should be driven by workload and established at the minimum levels necessary to accomplish mission and performance objectives. However, the data DODIG relied on in its budget analysis, such as the number of incoming complaints, closed cases, and intake reviews, may not fully indicate workload levels or where DODIG is focusing its resources because DODIG does not collect related data on the number of direct and indirect labor hours associated with different types of cases and investigative activities. For example, many of the complaints DODIG receives and ultimately closes are either military or appropriated-fund civilian cases that may be less resource-intensive than other cases because most are declined or referred to another organization for investigation. From fiscal year 2013 through fiscal year 2015, DODIG received 666 complaints from appropriated-fund civilians but also declined 622 appropriated-fund civilian cases. Also, our analysis shows differences in case length and timeliness improvements for civilian and contactor investigations by covered employee group, suggesting that resources may be expended differently across different case types. Relatedly, the investigators with whom we spoke stated that case complexity can be driven by statutory requirements, with several noting that contractor cases present unique challenges that can affect timeliness. Finally, process changes may also drive the resources associated with different activities and therefore affect personnel needs. As an example, officials stated that the streamlined oversight review process was intended to support improved timeliness, and, according to its data, DODIG reduced the average length of oversight reviews from an average of 16 days in the first quarter of fiscal year 2017 to an average of 7 days in the second quarter despite an increase in the number of reviews closed. DODIG officials stated that collecting more detailed workload data, such as by tracking labor hours, would be burdensome on its investigators. However, there are several ways DODIG could track its labor hours through existing systems and processes—such as through its time and attendance system. One of the supervisory investigators with whom we spoke stated that he already requires investigators on his team to track their hours for each case. Additionally, DODIG’s current budget request only includes notional allocations for its requested personnel that DODIG officials acknowledged may need to be adjusted. By assessing the feasibility of collecting more detailed workload data, such as the labor hours associated with each case, and including such data in future personnel requirements assessments, DODIG would be better positioned to evaluate the effectiveness of process changes it has implemented, assess its personnel requirements, and allocate personnel in the most efficient manner in order to accomplish its mission. DODIG does not regularly report to Congress on the timeliness of civilian and contractor reprisal cases. Under the Inspector General Act of 1978, as amended, DODIG is an independent and objective component within DOD and subject to oversight by Congress. Under the act, DODIG is required to keep Congress fully informed through, among other things, its semiannual reports to Congress. Similarly, Standards for Internal Control in the Federal Government state that it is important for oversight bodies, such as Congress, to have information to carry out their responsibilities. In prior work, we have also found that DODIG was not reporting to Congress on the timeliness of its military whistleblower reprisal cases, and in 2012 we recommended that DOD regularly do so, to include information on those military reprisal cases exceeding the 180-day time frame. DOD concurred and, in October 2016 began to report to Congress semiannually on the timeliness of its military whistleblower reprisal cases, including the number of military reprisal cases exceeding 180 days. DODIG did not include information in these reports on the timeliness of civilian and contractor cases, including the number of contactor and subcontractor cases exceeding the 180-day statutory time frame. According to DODIG officials, this information was not included because they were being responsive to our 2012 recommendation. We believe that while DODIG’s actions were responsive to our 2012 recommendation, reporting on the timeliness of civilian and contractor whistleblower reprisal cases is also important. This conclusion is because DODIG data show that while military reprisal cases constitute the majority of those that DODIG oversees, civilian and contractor cases are the majority of cases that DODIG investigates. For instance, DODIG reported to Congress that, in fiscal years 2015 and 2016, about 72 percent of its closed whistleblower reprisal investigations were civilian and contractor cases. Without regularly reporting information on the timeliness of civilian and contractor reprisal cases, Congress will not have complete visibility over the timeliness of a substantial portion of DODIG’s whistleblower reprisal investigations, including DODIG’s performance in meeting the 180-day statutory goal for contractor and subcontractor reprisal cases. DODIG has established several processes to help ensure the independence and thoroughness of the civilian and contractor reprisal cases it handles, including a training program, a staff recusal process to help safeguard independence, and an internal controls process to help ensure the accuracy of case-file information. However, a lack of documentation on recusals and conflicts of interest may limit its ability to fully evaluate threats to its independence, and its practice of declining cases is not fully consistent with its complaint-intake process. Further, we found that some key documentation needed to demonstrate compliance with DODIG investigative and quality-assurance processes was missing from its fiscal year 2015 case files. DODIG has implemented a training program that provides instruction to its investigators on CIGIE standards and professional competencies. CIGIE general standards include independence and due professional care, which includes thoroughness, among other things. In addition, these standards require that organizations ensure that investigators possess the necessary knowledge, skills, and abilities through entry-level training and periodic training on topics that affect operations, among other things. DODIG hired a training manager in 2012, in response to a recommendation made during a peer audit by the Department of Justice in 2009. According to the training manager, the training program aims to develop core competencies needed by investigators—such as interviewing techniques and writing—to enhance professional development and to support organizational goals related to standardized processes and efficient operations. To achieve these goals, DODIG delivers training through internally developed courses, courses available from external vendors, semiannual internally organized symposia, all- hands meetings, and job tools. For example: Within 3 months of hire, all whistleblower reprisal unit investigators are to be assigned to an internally developed entry-level whistleblower reprisal investigations course. This course covers CIGIE standards, investigative skills and procedures, and relevant whistleblower statutes. External courses cover a variety of topics, including, among other things, writing effectively and advanced interviewing techniques. Internal all-hands meetings and training are typically used to convey recent changes to policy, process, technology, and legislation, according to DODIG officials. For example, DODIG conducted an all- hands training session in May 2016 covering reprisal complaints related to sexual assault; has provided training on its case- management system to coincide with new system releases; and provided training when changes were made to section 2409 of Title 10 of the United States Code to include protections for DOD subcontractors, among other things. DODIG’s job tools include the AI Investigations Manual, a user guide for its case-management system, and quick-reference guides that contain, among other things, requirements for managing case information and statutory guidance. According to DODIG officials, investigators are required to take one formal professional development course per year, but, depending on fiscal year budget constraints, each will attend at least one formal course, with most attending four or five. Courses are reflected in investigators’ individual development plans, which are developed annually and approved by each investigator’s supervisory investigator. As part of the individual development plan process, investigators choose courses based on guidance provided by the training manager—including a career-paths training list covering required and optional courses—as well as input from their supervisory investigators. Course completion is tracked by the training manager, who stated that this process helps to ensure that investigators continually build on needed competencies. DODIG has established processes for safeguarding independence and objectivity, but a lack of documentation on recusals and conflicts of interest may limit its ability to fully evaluate threats to its independence, which our work indicates may be present. CIGIE standards state that in all matters relating to investigative work, the investigative organization must be free, both in fact and appearance, from impairments to independence and maintain an independent attitude. Impairments to independence include preconceived opinions of individuals or groups that could bias the investigation; professional or personal relationships that might weaken the investigative work; financial interest in an individual, entity, or program being investigated; and biases, including those induced by political or social convictions. Further, CIGIE standards state that IGs should identify and evaluate threats to independence both individually and in the aggregate because threats can have a cumulative effect on an IG employee’s independence, and that IGs should have policies and procedures in place to resolve, report, and document impairments to independence when they exist. Similarly, DODIG guidance states that employees are responsible for informing their supervisors in writing of any potential impairment to their independence or objectivity and that, at minimum, applicable documentation should be retained with the project files. DODIG has established processes to safeguard independence, including a quality-assurance process for reviewing final reports of investigation and a recusal process whereby investigators with a real or perceived potential conflict of interest can be removed from a case. Specifically: DODIG’s quality assurance review process is designed to meet the requirements set forth by DODIG Instruction 7600.1, Quality Standards for Department of Defense Office of Inspector General Reports. Specifically, this process includes the following: 1. Peer review of the report to identify where facts are missing or where facts presented do not logically flow to conclusions. 2. Supervisor review of the report and supporting evidence. 3. Director or deputy director review of the report. 4. Editor review of the report to check for compliance with internal and external writing guidelines. 5. Review of the report by a quality-assurance program analyst— who is organizationally independent of the investigations unit—to ensure compliance with CIGIE standards for accuracy, documentation, and clarity. 6. Deputy Inspector General for Administrative Investigations review of the report and accompanying correspondence. 7. Office of General Counsel review of the report for legal sufficiency. DODIG’s AI Investigations Manual notes that, as part of this process, investigators have a responsibility to be impartial, remain objective, and be receptive to evidence that is exculpatory as well as incriminating. Further, the manual states that investigators should not allow conjecture, unsubstantiated opinion, bias, or personal observations or conclusions to affect their work. DODIG officials cited the quality-assurance process as a key step in ensuring objectivity in whistleblower reprisal investigations. DODIG’s recusal process is designed to help ensure independence by requiring that investigators recuse themselves from cases in which they may have a real or perceived conflict of interest in the outcome of an investigation, such as personal financial interests or relationships with the subject or complainant. Specifically, DODIG’s AI Investigations Manual states that if at any point investigators believe that they cannot be impartial in a particular case, or the matter raises the appearance of a conflict of interest, the investigator is to notify his or her supervisor immediately. DODIG officials stated that when such an instance arises, the case will be reassigned to another investigator and that recusals typically occur when a case is first assigned. These officials noted also that the recusal policy extends to management of the whistleblower reprisal investigations unit as well. However, we found that DODIG may be limited in its ability to fully evaluate threats to its independence because it does not have a process for documenting recusals and associated conflicts of interest. DODIG officials stated that they generally do not document or track the number recusals or associated conflicts of interest because cases are assigned to another investigator when such instances occur. Further, these officials stated that the robustness of the peer-review process helps ensure independence and objectivity and that DODIG’s status as a statutory IG creates a climate favorable to conducting independent and objective investigations. However, without a process to document recusals and conflicts, DODIG may lack institutional awareness of potential impairments to its independence, including bias, and may be unable to evaluate such threats both individually and in the aggregate. For example, our interviews with the 24 investigators and four supervisory investigators from investigative teams indicate the presence of potential threats to the independence of DODIG staff. Specifically, 8 of the 28 investigators and supervisory investigators we interviewed (approximately 29 percent) reported observing acts that they perceived to demonstrate bias on the part of one or more whistleblower reprisal unit staff or management. Investigators stated examples of perceived bias that, if true, indicate a climate that may not be consistently favorable to independent and objective investigations. In response, DODIG officials stated that when issues involving potential bias are brought to their attention, they take appropriate action, and they provided us with an example of an incident in which an investigator was issued a letter of counseling for interacting with a complainant in a manner that could be perceived as biased. DODIG officials also stated—and we agree—that the DODIG Office of Professional Responsibility handles the type of allegations reported to us; that DODIG’s required entry-level whistleblower reprisal investigations training course addresses the CIGIE standard on independence, including different types of impairments; and that additional DODIG guidance covering the topic of independence is available to investigators. CIGIE’s Quality Standards for Federal Offices of Inspector General state that IGs and IG staff have a responsibility to maintain independence so that opinions, conclusions, and recommendations will be impartial and be viewed as impartial by knowledgeable third parties. Without documenting recusals and impairments to employee independence and evaluating threats to independence both individually and in the aggregate, DODIG may be unable to fully identify threats to its independence. Further, a full evaluation of potential threats to independence could decrease the potential for bias—thereby increasing assurance that civilians and contractors receive the whistleblower protections afforded by law—and help to ensure that investigations are independent in both fact and appearance. DODIG has a complaint intake process to ensure due process and management oversight when determining whether complaints alleging reprisal provide sufficient evidence to warrant an investigation, but in practice it has declined some nondiscretionary cases—that is, those cases for which it has primary investigative authority—without completing the intake process. Also, about 25 percent of the declined nondiscretionary cases that we reviewed were declined for reasons inconsistent with DODIG guidance and the reasons DODIG officials told us may warrant a case being declined. Moreover, this guidance is not consistent with DODIG policy on performing the complaint intake process. As previously described in this report, the purpose of DODIG’s complaint intake process is to determine whether the alleged facts of a case, if proven, would raise an inference of reprisal and therefore warrant investigation. According to DODIG’s AI Investigations Manual, this process consists of five steps, including an interview of the complainant to clarify the complaint and analysis of the alleged facts against the elements of reprisal. The manual also states that contacting the complainant and conducting the intake interview are important steps toward obtaining a thorough understanding of the allegation in order to determine whether the elements of reprisal are present, and that a complainant’s assertions should be viewed in the light most favorable to the complainant. DODIG officials, supervisory investigators, and investigators we spoke to also emphasized the importance of conducting an intake interview, in part because complainants are oftentimes not familiar with the requirements for demonstrating whistleblower reprisal, making it difficult to discern whether a complaint involves reprisal. For example, one supervisory investigator stated that it is difficult to determine whether a complaint involves reprisal without speaking to the complainant because it is difficult to provide all of the necessary information in the written complaint. Also, an investigator told us that the investigator asks about the required elements and read definitions to complainants, because only about half of complainants are knowledgeable about the elements of reprisal. According to DODIG’s AI Investigations Manual, all decisions to investigate or not investigate complaints at the conclusion of the intake process require approval from the director or deputy director of the whistleblower reprisal investigations unit. Under limited circumstances, DODIG may decline a case—that is, not accept it—without completing the intake process and obtaining management approval. According to DODIG officials, declined cases usually involve complaints filed by DOD appropriated-fund civilians, which are typically referred to the Office of Special Counsel—which has primary jurisdiction over these cases—unless they involve issues of particular interest to DODIG. DODIG officials also told us that DODIG may sometimes decline nondiscretionary cases involving DOD NAFI civilians, DOD contractors and subcontractors, and DCIPS employees if the case clearly does not involve reprisal, the complaint is anonymous, or the complaint is a duplicate. Unlike cases routed through the intake process, such decisions can be made by supervisory investigators without approval from the director or deputy director from the whistleblower reprisal investigations unit. In his July 2016 letter to Congress, the Acting DOD Inspector General stated that for statutes for which DODIG has primary or sole jurisdiction (i.e., nondiscretionary cases), DODIG’s long-established intake process normally requires that complainants be interviewed to clarify each complaint before a determination is made to investigate or dismiss a case, but that DODIG may decline to take action on a complaint without conducting an intake interview in cases involving appropriated-fund civilians. Our analysis of DODIG data shows that DODIG declined 63 nondiscretionary reprisal cases from fiscal years 2013 through 2015 without completing the intake process. We found that about 25 percent—16 of 63—of these cases were declined for reasons that were inconsistent with those described above as being appropriate for declination without completing the intake process. Specifically, in 15 of the declined nondiscretionary cases across these 3 fiscal years, the documented reason for declining the case was the perceived lack of a reprisal element, such as a protected disclosure, and in 1 case there were no notes describing why it had been declined. In response to our analysis, DODIG officials stated that it is sometimes necessary to decline cases when they are frivolous or clearly do not involve reprisal and, separately, that all but two of the declined nondiscretionary cases they examined from fiscal years 2013 through 2015 either adhered to prior policy, procedure, or practice, or were miscoded. Specifically, to support their rationale in most cases, DODIG officials cited criteria from the 2016 Data Entry Guide, which states that cases may be declined when the complaint makes no explicit or implicit reference to reprisal or military restriction. However, DODIG’s Data Entry Guide did not contain such criteria until July 2016, nearly a year after the most recent cases we reviewed were closed. Moreover, 15 of the 16 cases we identified were declined due to the absence of an element of reprisal—such as a protected disclosure—which is not a stated reason for declining cases in DODIG’s 2016 Data Entry Guide, does not align with DODIG’s cited criterion of no explicit or implicit reference to reprisal, and is something that DODIG officials, investigators, and supervisory investigators told us can be difficult to ascertain without conducting an intake clarification interview. Finally, DODIG officials reviewed the declined nondiscretionary cases we identified from fiscal years 2013 through 2015 and concluded that two of the cases should have gone through intake, and stated that another case was later opened as an investigation after additional information was provided by the complainant. This event indicates that conducting the full intake process—including interviewing the complainant—might have uncovered relevant information for this case that did not appear in the written complaint. Federal internal control standards state that management should implement control activities through policies that should be documented in the appropriate level of detail to effectively monitor the activity and communicated such that personnel can implement the activities for their assigned responsibilities. DODIG officials stated that the criteria in DODIG’s 2016 Data Entry Guide constitute policy for declining cases and that they have communicated such guidance to investigative staff. However, the criteria DODIG cited as policy—with which some of the rationales we reviewed did not align, as discussed—are not consistent with DODIG’s intake policy, which requires that complainants be interviewed and that either the director or deputy director approve all decisions to investigate or not investigate complaints. Also, documentation provided by DODIG to demonstrate its prior communication of guidance for declining nondiscretionary cases either did not clearly address this issue or appeared to be provided to staff that do not have responsibility for declining such cases. Specifically, an October 2015 e-mail to supervisory investigators addressed the practice of declining discretionary cases (i.e., appropriated-fund civilian cases) but not nondiscretionary cases like the ones we identified in our analysis; an e-mail officials provided referencing an all-hands meeting that they told us addressed declinations, among other topics, states only that the meeting would cover questions about handling different types of statutes; and the slides for a training course that DODIG officials said coincided with the release of the 2016 Data Entry Guide were addressed to oversight investigators, not the supervisory investigators who have responsibility for declining incoming complaints. Finally, we also found, based on our interviews with all four supervisory investigators that have responsibility for declining cases, that there is an inconsistent understanding of why and when cases should be declined. For example, three of the four supervisory investigators we interviewed were unsure whether any policy or criteria for declining cases existed, one stated that nondiscretionary cases should never be declined, one stated that nondiscretionary cases may be declined when the complaint clearly does not pertain to reprisal, and two of the four told us that they currently decline some cases for perceived lack of a reprisal element. One of the four supervisory investigators also stated that the practice of declining cases arose to improve timeliness and that the act of declining nondiscretionary cases is “fostered” by management. Without establishing and clearly communicating a declination policy for nondiscretionary cases in the AI Investigations Manual or other guidance, and aligning this policy with its intake policy, DODIG may not have reasonable assurance that decisions to decline cases consistently align with policy, and that complainants are afforded the same due process and fair and equal treatment as those whose cases are routed through the intake process. Moreover, delegating final responsibility for declining nondiscretionary cases to supervisory investigators without management approval may also result in differential treatment, since NAFI, contractor and subcontractor, and DCIPS employees do not have the same recourse to the Office of Special Counsel as do DOD appropriated-fund civilians, whose cases may also be declined. DODIG has established investigative, quality-assurance, and internal controls processes to help ensure the thoroughness, accuracy, and completeness of its investigations and case files. However, our review of case files closed by DODIG in fiscal year 2015 found that some key documentation or data needed to demonstrate compliance with these processes was missing or was not uploaded to DODIG’s case-file system in a timely manner. Further, our interviews with DODIG’s 24 investigators from investigative teams showed that some steps required to help ensure thoroughness during the investigative process are not routinely followed. DODIG has established investigative, quality-assurance, and internal controls processes to help ensure the thoroughness, accuracy, and completeness of its investigations. Collectively, these processes encompass the planning and conduct of investigations, review of the report of investigation, and checks on the completeness of case files. As described below, each of these processes is designed to align with CIGIE quality standards. DODIG’s investigative process is intended to facilitate the planning and conduct of investigations in accordance with the CIGIE quality standards of planning and thoroughness by (1) directing investigative teams to complete an investigative plan prior to beginning fieldwork and (2) prescribing subsequent, required steps of the investigative process. According to DODIG’s AI Investigations Manual, a good investigative plan provides a roadmap for conducting focused, thorough, and efficient investigations. The key elements of an investigative plan include the identification of investigation subjects; witnesses to be interviewed and questions relevant to the allegations; legal and regulatory standards; evidence to be collected; travel; investigation milestones; and necessary investigative steps. Investigative steps are the strategy or the steps through which the investigation plans to proceed to complete the case and include notifying the proper parties of the investigation; interviewing complainants, subjects, and witnesses; and collecting documentary evidence. DODIG’s manual includes detailed guidance for conducting these and other steps of the process and provides investigative tools such as a diagram that can be used to graphically depict the evidence collected, along with its weight and preponderance. Additionally, the manual describes quality checks and roundtables, which are discussions between various personnel such as investigators, supervisors, attorneys, and management of the whistleblower reprisal investigations unit. According to DODIG’s AI Investigations Manual, these discussions should occur before commencing fieldwork to facilitate investigative planning and both prior to and after interviews with the subject of the investigation to facilitate an “interactive and write-as-you-go investigative process.” DODIG’s quality-assurance review process, as previously described in this report, is a multistep review process to help ensure that final reports of investigation are thorough, factually accurate, and legally sufficient. This process is designed to address the minimum quality-assurance standards requirements set forth by DODIG Instruction 7600.1, Quality Standards for Department of Defense Office of Inspector General Reports, which are based in part on CIGIE quality standards. DODIG’s internal controls process is designed to test compliance with CIGIE professional standards related to the thoroughness of investigations; the adequacy of case-file documentation; and the currency, accuracy, and completeness of data maintained in DODIG’s case-management system. According to DODIG, this process was developed in response to our prior recommendation in 2012 to develop and implement performance metrics to ensure the quality and effectiveness of the investigative process, such as by ensuring that case files contain evidence sufficient to support the conclusions. Specifically, the internal controls process requires that investigators complete a criteria-based checklist at the conclusion of an investigation to ensure that certain documentation and data are present in the case-management system. For example, this checklist includes items to check case files for the presence of letters of notification to parties involved in an investigation, an approved investigative plan, interview records, evidence of legal review, and corrective-action documentation. In addition, the checklist includes items to ensure that certain data fields have been populated in the case-management system, such as information about complainants and subjects; alleged protected disclosures and personnel actions; and dates associated with investigative steps. According to DODIG’s internal controls process description, these checklists were completed on a quarterly basis prior to fiscal year 2016, at which point they began to be completed at the close of each case. Our review of case files closed by DODIG in fiscal year 2015 found that some documentation was consistently present, but that some other key documentation or data were missing or were not uploaded to DODIG’s case-file system in a timely manner. This documentation and these data are needed to demonstrate compliance with or execution of investigative, quality-assurance, and internal controls processes, including the internal controls checklist—which we found did not include checks for some key documentation and required investigative events. Further, our interviews with the 24 investigators from investigative teams showed that some steps required to help ensure thoroughness during the investigative process are not routinely followed. The version of DODIG’s investigations manual in place during the period covered by our case-file review states that case files should be complete from case initiation until case closure and that upon case closure, investigators will organize the final records in the file as soon as possible to ensure readiness for Freedom of Information Act or other requests for investigation documents. The manual further states that closed case files should be ready to withstand scrutiny by an outside peer review or oversight authority. Similarly, CIGIE standards state that the degree to which an organization efficiently achieves its goals is affected by the quality and relevance of information that is collected, stored, retrieved, and analyzed, and that the results of investigative activities should be accurately and completely documented in the case file. Examples of documents consistently present—Our review of 178 case files for cases closed in fiscal year 2015 found that some documentation was consistently present. For example: 100 percent of the cases that were fully investigated included evidence of a sworn recorded testimony with the complainant. 100 percent of the cases that were fully investigated had a final report of investigation. We estimate that about 98 percent of all cases included the incoming complaint. We estimate that about 98 percent of the cases that were dismissed had a closure letter to the complainant. Examples of documents or data that were missing or not uploaded— Conversely, our review of case files for cases closed in fiscal year 2015 found that some other key documentation or data that are needed to demonstrate compliance with or execution of the investigative, quality- assurance, or internal controls processes were either missing or were not uploaded to DODIG’s case-file system in a timely manner. For example: Approximately 13 percent of the cases that were fully investigated did not have documentation of legal sufficiency. In response to our review, DODIG located documentation of legal sufficiency for these cases and uploaded it to the relevant case files. Approximately 13 percent of the cases that were fully investigated did not include an intake worksheet. Approximately 19 percent of the cases that were fully investigated were missing the notice of investigation sent to the complainant. DODIG inconsistently populated mandatory investigative event milestone dates related to management approval of the report of investigation. For example, of the cases that were fully investigated, approximately 36 percent were missing the date that the staff package was sent to the Director or Deputy IG for Administrative Investigations and approximately 13 percent were missing the date that the Deputy IG for Administrative Investigations approved the report. The absence of these dates does not necessarily indicate that the actions did not occur; rather, it hinders DODIG’s ability to analyze the timeliness of specific investigative phases, such as management review, as DODIG officials stated they intend to do in order to further improve investigation timeliness. In regard to the missing Deputy IG for Administrative Investigations approvals, DODIG officials stated that the missing dates were data-entry errors and that no case moves beyond this step in the approval process without the Deputy IG’s approval or her designee’s approval. DODIG also provided evidence to show that either the Director or Deputy IG for Administrative Investigations did approve the reports. Approximately 48 percent of the cases that were fully investigated included documents that were modified or were newly uploaded at least 30 days after case closure. These changes related to documents such as the final report of investigation, closure letters, evidence of legal review, the investigative plan, congressional correspondence, and interview transcripts. On average, the most recent change occurred 228 days (median 148 days) after case closure. In response to this analysis, DODIG officials stated that we had selected an arbitrary measure of 30 days after case closure to determine whether document uploads were appropriate. We applied the 30-day threshold as a grace period, recognizing that it may not always be practical for case files to be complete from case initiation to closure, as DODIG’s then-guidance required. DODIG officials also stated that because document uploads were performed in the majority of cases in accordance with the DODIG internal controls checklist process or related to follow-up activities, our inclusion of these cases implies that it is inappropriate to have internal controls in place to assure that key documentation is uploaded unless those reviews can be performed in less than 30 days after case closure. Further, DODIG officials stated that while it would be ideal if documentation was captured contemporaneously, internal guidance states that documentation gaps should be corrected when found, such as when found through the internal controls process. However, as previously stated, DODIG’s internal policy required that case files be complete upon case closure and the most recent change occurred on average 228 days (median 148 days) after case closure, far exceeding the internal policy and the prior practice of completing internal controls checklists on a quarterly basis. We believe that DODIG’s current practice of completing internal controls checklists at case closure is a positive step towards complying with its requirement for complete case files upon case closure and that it should help improve the timeliness of its case-file documentation. In approximately 16 percent of the cases that were fully investigated, it was not clear whether the investigative plan on file was the approved version because it did not include the approval milestone date in the document and did not appear to be a complete plan based on one or more missing or incomplete key elements. For example, one plan was missing investigative milestones after “investigation start,” while another was missing investigative milestones, did not specify the documentary evidence to be gathered, and did not include witnesses to be interviewed. DODIG’s 2012 Administrative Investigations Manual required that investigative plans be approved prior to the start of fieldwork and it identified key elements of the investigative plan as including documentary and other relevant evidence to be collected, witnesses to be interviewed, and investigation milestones, among other things. Further, DODIG’s internal controls checklist requires that the approved investigative plan corresponding to the plan’s approval date in the database be uploaded to the case file. DODIG officials agreed that these plans were either missing key elements or had incomplete investigative milestones, but also stated that each plan was the approved plan as evidenced by the date of the document corresponding to the approval date in the database. Of the plans we identified as incomplete, we noted that the date of one document matched the approval date in the database, while the dates of the other documents preceded the approval date in the database by between 3 business days and over a month—making it further unclear as to whether these incomplete plans were the approved plans. DODIG officials also stated that initial investigative plans sometimes contain less than the full complement of witnesses and documentary evidence. However, the plans we identified as being incomplete in this regard did not contain any documentary evidence and one did not contain any witnesses. In addition, all of the plans we identified with missing or incomplete key elements either did not include investigative milestones or had incomplete investigative milestones. Two out of three fully investigated cases with congressional inquiries were missing correspondence to the Member of Congress providing a summary of the findings, as required by DODIG’s 2012 investigations manual. In response to this finding, DODIG officials stated in regard to one of the cases that the draft summary finding letter in the case file was sufficient to address documentation requirements because DODIG’s Office of Legislative Affairs and Communications is responsible for facilitating signatures of, transmitting, and maintaining records of congressional responses. However DODIG’s 2012 investigations manual states that acknowledgment, interim, and final letters to members of Congress should be saved to the case file, and its internal controls checklist includes items to verify the presence of this correspondence. Because the letter saved to the case file was not the letter sent by DODIG’s Office of Legislative Affairs and Communications, it is unclear whether it was the final correspondence. Approximately 81 percent of the 413 testimony transcripts and voice recordings we reviewed did not address one or more points of the standard read-in/read-out process. This process consists of a series of questions and statements at the beginning and end of each interview intended to ensure that all witnesses are treated equally and that they are afforded the proper notifications of authorities and due process. In response to this finding, DODIG officials stated that it was “clearly permissible” to not cover the entire read-in/read-out in cases where a witness had already been interviewed and provided with the standard read-in/read-out information. During the course of our case- file review, we noted some cases where complainants, subjects, or witnesses were interviewed more than once. However, DODIG did not identify the interviews where it determined that the full read-in/read- out did not need to be covered or provide us with policy or other guidance to support its statement that read-in/read-out language could be abbreviated. In addition to missing case-file documentation or case data, our interviews with each of the 24 DODIG investigators from the investigative teams showed staff are not routinely holding roundtable meetings, which are discussions between investigators, supervisors, attorneys, and management of the whistleblower reprisal investigations unit, as previously mentioned. Specifically, for the roundtables required by DODIG’s AI Investigations Manual, 18 of the 24 investigators we interviewed stated that investigative planning roundtables occur infrequently or not at all, 16 of the 24 investigators stated that the pre-subject-interview roundtables occur infrequently or not at all, and 19 of the 24 investigators stated that post-subject-interview roundtables occur infrequently or not at all. DODIG officials stated that roundtables are an important part of the review process in that they help to keep cases on track and identify necessary adjustments. Federal internal control standards state that management should monitor its internal control system, evaluate the results, and remediate internal control deficiencies on a timely basis. Further, these standards state that a control cannot be effectively implemented if it was not effectively designed and that an existing control is not properly designed if the control objective would not be met when the control operates as designed. However, we noted, and DODIG officials acknowledged, that DODIG’s internal controls checklist does not capture the full range of key case-file documentation and required investigative events, including documentation of (1) an intake worksheet, (2) required roundtable discussions, and (3) the program analyst for quality assurance review. Without a checklist that captures the full range of key case-file documentation and data associated with required investigative steps, DODIG will be limited in its ability to ensure compliance with CIGIE standards related to thoroughness and adequacy of case-file documentation, as well as the currency, accuracy, and completeness of data maintained in its case-management system. DODIG has conducted oversight of investigations involving DCIPS employees that were conducted by the some of the defense intelligence component IGs, which consist of the IGs of the Defense Intelligence Agency, National Geospatial-Intelligence Agency, National Reconnaissance Office, and National Security Agency. However, DODIG and the defense intelligence components have not fully addressed requirements related to DODIG’s review of all DCIPS employee allegations, determinations, and investigations handled by the component IGs. DOD Directive-Type Memorandum 13-008—which implements Presidential Policy Directive 19 (PPD-19)—requires that DODIG provide oversight of reprisal allegations involving DCIPS employees that are handled by the defense intelligence component IGs, and that the component IGs correspondingly furnish information to DODIG. Table 3 highlights key oversight requirements specified by Directive-Type Memorandum 13-008 for DODIG and the defense intelligence component IGs. DODIG’s AI Investigations Manual implements DODIG’s oversight requirements in accordance with Directive-Type Memorandum 13-008 by describing a process for DODIG to review (1) defense intelligence component IG determinations to not investigate allegations involving DCIPS employees, and (2) reports of investigation for completed investigations involving DCIPS employees submitted by the defense intelligence component IGs. To conduct these reviews, among other duties, DODIG has established an oversight branch comprising 17 personnel across two teams, each of which has a supervisory investigator, according to DODIG officials. Specifically, DODIG’s oversight team is to review determinations to not investigate an allegation to ascertain whether the elements of a prima facie allegation were adequately addressed, including alleged protected disclosures and personnel actions; whether the alleged responsible management official knew of the protected disclosures before taking, withholding, or threating the personnel actions; and whether there is an inference of causation between the protected disclosures and personnel actions. According to DODIG’s AI Investigations Manual, investigations are to be reviewed for compliance with CIGIE standards, including independence, objectivity, thoroughness, documentation, and legal sufficiency. These reviews are to be documented in a worksheet, and the results of the review are to be communicated to the component IG. If the determination to not investigate or the investigation is sufficient, an approval letter stating such is to be sent to the component IG; if not, DODIG is to work with the component IG to correct any deficiencies. As of April 2017, DODIG had received and reviewed investigations from three of the four defense intelligence component IGs as well as notifications of allegation receipt from one of the four component IGs. DODIG had not received determinations to not investigate allegations from any of the four component IGs. Specifically, DODIG officials stated that they received investigations for review from the IGs of the Defense Intelligence Agency, the National Geospatial-Intelligence Agency, and the National Security Agency. According to DODIG officials, DODIG had not received investigations for review from the National Reconnaissance Office IG. Further, these same officials stated that DODIG had received notifications of allegation receipt from the National Geospatial-Intelligence Agency IG only, and that DODIG had not received determinations to not investigate an allegation from any of the four defense intelligence component IGs. DODIG officials provided us with three notifications from the National Geospatial-Intelligence Agency IG that they stated constituted notification of allegation receipt. In reviewing these notifications, we noted that each notification was received by DODIG after the 10-working day period prescribed by Directive-Type Memorandum 13-008 and that each notification also informed DODIG of the National Geospatial-Intelligence Agency IG’s intent to investigate the allegation. As previously mentioned, DODIG has the authority to retain such cases for investigation or refer them back to the component IG. Specifically, two notifications stated that the National Geospatial-Intelligence Agency IG planned to investigate the associated allegations on the basis of preliminary review. One of these notifications was received by DODIG more than a month after the National Geospatial-Intelligence Agency IG’s receipt of the allegation, and the other was received by DODIG more than 2 months after initial receipt of the allegation. The third notification—received by DODIG more than 3 months after initial receipt of the allegation—stated that the National Geospatial-Intelligence Agency IG would continue its investigation of the matter in the absence of notification from DODIG that DODIG intended to assume investigative jurisdiction over the matter. DODIG officials stated the defense intelligence component IGs are likely receiving numerous complaints of reprisal, and the DODIG official responsible for managing the oversight branch further stated that it is unlikely that the investigations DODIG has received for review constitute all of the reprisal investigations conducted by the component IGs. The defense intelligence component IG officials we contacted provided varying perspectives on the oversight requirements, as well as the degree to which they currently comply with the requirements of Directive-Type Memorandum 13-008, and intend to do so in the future. For example, National Security Agency IG officials told us that they are in compliance with the memorandum’s provisions because they notify DODIG whenever they commence an investigation of a credible allegation, and because all credible allegations have been investigated. However, these officials also noted that they receive other reprisal allegations for which they do not notify DODIG because the allegations are not deemed to be credible, or because they do not “facially meet” the criteria for reprisal. As previously mentioned, Directive-Type Memorandum 13-008 requires that defense intelligence component IGs notify DODIG within 10 working days of receiving any allegation of reprisal from a DCIPS employee and provide a copy of the written complaint to DODIG. Similarly, officials from the Defense Intelligence Agency IG stated that they notify DODIG of new investigations and provide the resulting investigations for review, but that they do not notify DODIG of all allegations received or provide DODIG with determinations to not investigate allegations for DODIG’s review, as is required. These same officials also stated that the Defense Intelligence Agency IG works collaboratively with DODIG to respond to complaints regarding Defense Intelligence Agency personnel that are reported to DODIG. According to a senior DODIG official in February 2017, the Defense Intelligence Agency IG director of whistleblower reprisal investigations stated to DODIG officials that the Defense Intelligence Agency IG intends to start complying with all oversight requirements. Additionally, DODIG officials stated that DODIG provided officials from the Defense Intelligence Agency IG with a sample template that can be used to notify DODIG of determinations to not investigate cases. However, Defense Intelligence Agency IG officials stated that senior Defense Intelligence Agency IG officials met with DODIG to discuss a way forward on the oversight requirements and came to no conclusion, and that no Defense Intelligence Agency IG official committed to complying with all of the oversight requirements. Additionally, a Defense Intelligence Agency IG official told us that the memorandum is problematic, that our inquiry regarding the Defense Intelligence Agency IG’s compliance with the memorandum’s provisions has prompted them to reevaluate their reporting processes, and that they are considering reducing future reprisal reporting to DODIG. DODIG officials stated some of the defense intelligence component IGs objected to the memorandum’s oversight provisions during its development, and that some of the component IGs’ status as presidentially appointed, Senate-confirmed “statutory” IGs may influence their views with regard to the appropriateness of DODIG oversight. However, DODIG officials noted that they view the requirements of Directive-Type Memorandum 13-008 as being clearly delineated and binding throughout the department, and that while the four defense intelligence component IGs have specific responsibilities as intelligence organizations, they remain part of the larger DOD and are therefore subject to department-wide duties, responsibilities, and authorities specified in the Inspector General Act of 1978, as amended. Moreover, they noted that DODIG remains the department’s principal IG over all components within the department—with responsibility to ensure coordination, cooperation, and efficiency in the audits and investigations of the defense intelligence components—and that PPD-19, DOD regulation, and the Inspector General Act provide the basis for DODIG to approve determinations and investigations of DOD component IGs. DODIG has taken some steps to coordinate with the defense intelligence component IGs in relation to the oversight requirements. For example, DODIG officials stated that they have conveyed their expectations with regard to the oversight process to representatives from the National Security Agency IG and the Defense Intelligence Agency IG, and that the memorandum’s requirements have been addressed during a symposium attended by representatives from defense intelligence component IGs and during DODIG’s basic whistleblower reprisal course, which is offered quarterly and is regularly attended by investigative personnel from the component IGs. DODIG officials also stated that in March 2017, they presented oversight requirements to an audience that included representatives from the component IGs. National Security Agency IG officials stated that oversight expectations were constructively discussed in a November 2015 meeting that concluded with an agreement to continue the excellent working relationship that had already been established, and that the symposium and reprisal course—while valuable—do not spend much time on dealing with intelligence community cases, PPD-19, and Directive-Type Memorandum13-008. Additionally, these same officials stated that the March 2017 oversight requirements briefing was provided to a Defense Council on Integrity and Efficiency Administrative Investigations Committee meeting, and that the National Security Agency IG did not receive the invitation or presentation. Similarly, Defense Intelligence Agency IG officials stated that in a December 2016 meeting with DODIG, they explained their statutory independence and objectivity, indicated the extent that any oversight requirements interfered with the agency’s statutory responsibilities, and stated that full compliance required additional discussion. These officials also stated that the Defense Intelligence Agency IG was not represented at the March 2017 oversight requirements briefing. However, DODIG has not established processes with the defense intelligence component IGs—such as standard operating procedures—to ensure that the components provide the allegation notifications, determinations to not investigate, and investigations prescribed by Directive-Type Memorandum 13-008. As the principal advisor for DOD IGs, DODIG is to assist the defense IGs—including the IGs of the defense intelligence components—by coordinating and clarifying DOD policy, issuing implementing instructions, and resolving conflicting or inconsistent IG policy involving defense IG duties, responsibilities, and functions. Without a process to fully implement the requirements of Directive-Type Memorandum 13-008 that includes (1) receiving notification of all allegations received by the defense intelligence component IGs, (2) reviewing their determinations to not investigate allegations, and (3) reviewing all investigations that they conduct, DODIG and the defense intelligence components will continue to not fully adhere to the prescribed roles related to the oversight of component determinations and investigations, as defined in agency directives. In April 2017, DODIG developed six performance measures to assess the timeliness of its investigations and oversight reviews for fiscal year 2017 that demonstrate many, but not all, key attributes of successful performance measures. However, DODIG does not have performance measures to assess the quality of its investigations and oversight reviews for fiscal year 2017 and beyond. Prior to fiscal year 2017, DODIG had developed several different timeliness and quality performance measures, but these measures were inconsistently documented and reported. Standards for Internal Control in the Federal Government emphasize the importance of assessing performance over time. By using performance measures, decision makers can obtain feedback for improving both policy and operational effectiveness. Further, we found in our previous work that measures on both timeliness and quality—such as the completeness of investigative reports and adequacy of internal controls—can enhance the ability of organizations to provide reasonable assurance that they are exercising appropriate safeguards for federal programs. Our body of work on leading performance-management practices identified 10 attributes of successful performance measures (see table 4). While these attributes may not represent all the attributes of successful performance measures, we believe they address important areas. In April 2017, more than halfway through the fiscal year, DODIG developed and provided us with six performance measures for assessing the timeliness of fiscal year 2017 investigations and oversight reviews. We compared the performance measures against the 10 key attributes of successful performance measures identified in our prior work, as shown in figure 3. In our analysis of DODIG’s six performance measures for timeliness of whistleblower reprisal investigations and oversight reviews, we found the following. Linkage. Five of the six measures we assessed addressed the attribute of linkage, and one measure partially addressed this attribute. A measure demonstrates linkage when it is aligned with division and agency-wide goals and mission and is clearly communicated throughout the organization. We found that all six timeliness measures aligned with DODIG’s strategic objective to provide timely and relevant feedback to the department through high impact DODIG products that result in improvements in program and operational performance. Additionally, we found that all six measures aligned with DODIG’s vision to improve the timeliness of its investigative products. According to DODIG officials, the timeliness measures have been communicated to whistleblower reprisal unit staff through briefings and all-hands training. However, we found that DODIG’s measure of the “average number of days to complete an oversight review” partially addressed the attribute of linkage because DODIG’s target to close its oversight investigations is less than or equal to 10 days, but its parameter for success for this measure is less than or equal to 60 days. This inconsistency may hinder DODIG’s ability to clearly communicate the measure. Clarity. Five of the six measures we assessed addressed the attribute of clarity, and one measure partially addressed this attribute. A measure achieves clarity when it is clearly stated and its name and definition are consistent with the methodology used to calculate it. Further, a measure that is clearly stated should not contain extraneous information or omit key data elements. DODIG’s timeliness measures are consistent with the methodology for calculating them, which uses standard dates entered for each case in its case-management system. However, the “average days for closing oversight investigations” measure partially addressed the attribute of clarity because its target to close oversight investigations in less than or equal to 10 days is inconsistent with its parameter for success, which is less than or equal to 60 days. Measurable Target. Five of the six measures we assessed addressed the attribute of having a measurable target, and one measure partially addressed this attribute. A measure with a measurable target has a numerical goal. DODIG has set a quantifiable, numerical target for each of the six performance measures we assessed. For instance, DODIG established a target of completing 51 percent or more of its civilian whistleblower reprisal investigations in less than or equal to 180 days or 240 days, depending upon the investigation authority. However, the “average days for closing oversight investigations” measure partially addressed the attribute of measurable target because DODIG’s target to close oversight investigations is less than or equal to 10 days, but its parameter for success for this measure is less than or equal to 60 days. Objectivity. All six measures we assessed addressed the attribute of objectivity. A measure demonstrates objectivity when it is reasonably free from significant bias or manipulation. Specifically, DODIG officials stated that the methodology for calculating its timeliness measures is objectively determined using standard dates entered for each case in its case- management system. We agree that this methodology does not allow for subjective considerations or judgments, as these measures by name indicate what is to be observed (e.g., closed investigations), in which population or conditions, and in what time frame (e.g., quarterly). Reliability. All six measures we assessed addressed the attribute of reliability. A measure demonstrates reliability when it produces the same result under similar conditions. DODIG officials stated that the same methodology will be used to evaluate performance on a quarterly basis across the fiscal year. In addition, DODIG has procedures to verify and validate each measure through its internal control checks of the case- management system. Baseline and Trend Data. All six measures we assessed partially addressed the attribute of baseline and trend data. A measure with a baseline and trend data associated with it identifies, monitors, and reports changes in performance and helps ensure that performance is viewed in context. DODIG’s measures provided data on year-to-date performance for the first two quarters of fiscal year 2017, allowing decision makers to identify and monitor trends in performance over time. In addition, DODIG officials stated that they established a baseline for their prior timeliness measures based on a finding from our prior work in 2012 and that they have since set goals to reduce the average days to conduct investigations by a certain percentage each year. However, DODIG did not provide baselines for its fiscal year 2017 whistleblower reprisal timeliness performance measures, and it has not consistently reported on all of its past timeliness performance measures, as discussed below, thus inhibiting baseline and trend analysis. Core Program Activities. Collectively, the six measures we assessed partially addressed the attribute of core program activities. Measures that reflect core program activities cover the activities that an entity is expected to perform to support the intent of the program. The whistleblower reprisal investigations unit’s mission is to objectively and thoroughly investigate whistleblower reprisal complaints and to provide oversight of military service and component IG investigations of such complaints. DODIG’s AI Investigations Manual specifies major activities associated with this mission, such as conducting intake reviews to determine whether an allegation warrants investigation, investigating whistleblower reprisal complaints, and conducting oversight reviews of investigations performed by the component IGs. DODIG’s fiscal year 2017 timeliness measures address program activities such as the timely completion of investigations and oversight reviews of investigations, but they do not assess whether other core program activities, such as intake reviews, are completed in a timely fashion. Limited Overlap. Collectively, the six measures we assessed addressed the attribute of limited overlap. Measures with limited overlap provide new information beyond that provided by other measures. DODIG’s fiscal year 2017 measures all relate to the timeliness of case-closure efforts, but the measures address both investigations and oversight reviews, and provide data that can supplement one another. For example, the measure of average days to complete civilian investigations may be supplemented by the percentage of completed civilian investigations meeting the 240-day timeliness goal, because the latter could provide context for the average days measure by indicating whether there may have been one or more long or short cases that affected the average. Balance. Collectively, the six measures we assessed did not address the attribute of balance. Measures that demonstrate balance ensure that an organization’s various priorities are covered. Performance-measurement efforts that overemphasize one or two priorities at the expense of others may skew the agency’s performance and keep its managers from understanding the effectiveness of their programs in supporting the overall mission and goals. Taken together, DODIG’s fiscal year 2017 timeliness measures address the whistleblower reprisal unit’s priority of improving the timeliness of its whistleblower reprisal investigations and oversight reviews, but they do not address the unit’s priority of conducting thorough investigations that adhere to CIGIE standards. Government-Wide Priorities. Collectively, the six measures we assessed partially addressed the attribute of government-wide priorities. Measures should cover a range of government-wide priorities, such as quality, timeliness, efficiency, cost of service, and outcome. A range is important because most program activities require managers to balance these priorities among other demands, and measures that assess some priorities but neglect others could place the program’s success at risk. DODIG’s fiscal year 2017 timeliness measures are focused on the agency’s priority to improve the timeliness of its reprisal investigations and oversight reviews, and do not address the quality of investigations. Without addressing the government-wide priority of quality, an overemphasis on timeliness could undermine quality. In addition, DODIG’s suite of timeliness measures does not address the efficiency or cost of service associated with investigations and oversight reviews. Although DODIG has not developed performance measures for fiscal year 2017 to specifically assess the quality of its reprisal investigations and oversight reviews, DODIG officials stated in April 2017 that they were in the process of doing so. Prior to fiscal year 2017, DODIG had developed several performance measures to assess both the timeliness and quality of its reprisal investigations, but the evaluation of these measures was inconsistently documented and reported, and some measures changed over time. For example, DODIG variously reported to senior DODIG management from fiscal year 2013 through fiscal year 2015 on timeliness measures such as the percentage of military and contractor cases meeting the statutory goal of 180 days, the average and median length of these investigations, as well as the percentage of NAFI, appropriated-fund civilian, and PPD-19 investigations meeting the internal 240-day goal. However, according to DODIG officials, in fiscal year 2016, DODIG reported to the Acting Inspector General on one measure alone— the percentage of oversight reviews meeting the internal goal of 70 days. DODIG also developed three quality measures in 2014 in response to our 2012 recommendation that it develop and implement performance measures to ensure the quality and effectiveness of the investigative process, such as by ensuring that case files contain evidence sufficient to support conclusions. DODIG’s quality measures specifically addressed thoroughness, documentation, and data integrity and completeness for DODIG investigations and oversight reviews. However, those measures were last briefed to the DOD Inspector General in fiscal year 2014. DODIG officials stated in February 2017 that changes in DODIG’s leadership and internal processes have contributed to the inconsistency in reporting on timeliness performance measures, but that they continued to monitor aspects of timeliness and quality throughout fiscal years 2013 through 2016. Subsequently, these officials disagreed with our conclusion that their measures had been inconsistently documented and reported, stating that they had implemented a robust set of performance measures for quality and timeliness, which they had monitored and documented on a quarterly basis from fiscal years 2013 through 2016. They also stated that they reported on fiscal year 2013–2016 average and median days for investigations against the 180- and 240-day benchmarks and on a sample of oversight reviews in annual assurance statements, and on fiscal year 2013–2015 average days for investigations in annual organizational assessments. DODIG officials also provided us with a written narrative explaining how their prior performance measures addressed the 10 key attributes of successful performance measures identified in our prior work. Our analysis of the prior measures differed from DODIG’s analysis in regard to the extent that DODIG’s prior measures addressed the attributes, but we do not present the results of our analysis because DODIG’s prior performance measures were not carried forward to fiscal year 2017. In discussing their efforts to develop performance measures, DODIG officials noted the difficulty of developing performance measures for investigative work, and stated that the Government Performance and Results Act (GPRA) Modernization Act of 2010 (GPRAMA) should provide the basis for any assessment of DODIG’s performance measures. These officials also stated that GAO’s identified key attributes of successful performance measures are accompanied by scant justification or explanation as to why they were chosen, and included no guidance as to how agencies should implement them. We believe that our prior work has demonstrated that the attributes—which are based in part on the earlier GPRA—are broadly applicable as leading practice for developing successful performance measures across various types of federal programs. Additionally, the attributes are similar to requirements set forth in GPRAMA that DODIG officials cited as the standard to which they should be held. For example, GPRAMA’s requirement that agencies establish performance goals that are objective, quantifiable, and measurable aligns with our attributes of objectivity and measurable target. Similarly, GPRAMA’s requirement that performance goals should describe how they contribute to the general goals and objectives in an agency’s strategic plan aligns with our attribute of linkage. Finally, we believe that the utility of each attribute is articulated by the potential adverse consequences of not meeting it, while the definition for each attribute conveys its purpose in order to guide implementation, as shown previously in table 4. Ultimately, by developing both quality and timeliness performance measures that fully reflect the key attributes of successful measures, DODIG decision makers would be better positioned to (1) assess the quality and timeliness of DODIG’s whistleblower reprisal investigations and oversight reviews and (2) determine whether past, ongoing, and future improvement initiatives are on track to achieve desired outcomes. Maintaining a timely, independent, and thorough process for investigating whistleblower reprisal complaints is essential to executing DODIG’s mission and to fulfilling the whistleblower reprisal unit’s vision of being the model whistleblower-protection program in the federal government. To these ends, DODIG has taken and has planned steps to improve the timeliness of its investigations, and has established several processes to help ensure their independence and thoroughness. However, without regularly reporting on the timeliness of all civilian and contractor investigations, decision makers’ ability to effectively oversee the whistleblower reprisal program is limited. DODIG also does not have key workload data that would enable it to more fully assess its personnel requirements in support of its planned steps to improve timeliness. By assessing the feasibility of collecting more detailed workload data, such as the labor hours associated with its cases, DODIG would be better positioned to identify the resource investment associated with different types of cases and investigative activities and to then allocate existing and future personnel in order to accomplish its mission. Enhancements are also needed for existing processes designed to help ensure the independence and thoroughness of DODIG’s investigations. By documenting investigators’ recusals and conflicts of interest—and evaluating these and other threats in the aggregate—DODIG will have increased institutional awareness of threats such as bias, thereby better enabling it to fully evaluate such threats. Also, by establishing and clearly communicating a declination policy for nondiscretionary cases, DODIG will have better assurance that these complainants are afforded the same due process as those whose cases are routed through the intake process. Additionally, developing an internal controls checklist that captures the full range of key case-file documentation and data associated with required investigative steps will help DODIG ensure compliance with CIGIE standards related to the thoroughness and adequacy of case-file documentation. Moreover, it will be better positioned to withstand scrutiny by outside authorities, and address concerns expressed by members of Congress regarding the integrity of its investigations. Another area in which DODIG’s process can be improved is in its oversight of cases involving DCIPS employees that are handled by the defense intelligence component IGs. While DODIG is reviewing investigations conducted by some of the defense intelligence component IGs, it has not established processes with the defense intelligence component IGs that fully address requirements to receive notification of all allegations involving DCIPS employees that are received by the components, review component determinations to not investigate allegations, and review all the investigations the components conduct. Without a process for doing so, DODIG and the defense intelligence components are unable to fulfill their prescribed roles related to the oversight of component determinations and investigations. Moreover, without fully executing its oversight responsibilities, DODIG cannot achieve its vision of being the model whistleblower-protection program in the federal government. Finally, given the improvement initiatives DODIG is planning and has under way—and the anticipated increase in cases—it is important that there be a reliable means by which to gauge progress in the timeliness and quality of both investigations and oversight reviews. DODIG developed performance measures for its timeliness for fiscal year 2017, more than halfway through the fiscal year, and these measures address many, but not all, key attributes of successful performance measures. However, DODIG has not developed measures for quality for fiscal year 2017 and beyond. By developing quality performance measures and improving existing timeliness performance measures to be consistent with all attributes of successful measures, senior DODIG leaders will have better information to assess progress in improving timeliness and completeness of investigations as well as to guide ongoing and future improvement efforts. We are making the following seven recommendations to the Department of Defense Office of Inspector General (DODIG): The DOD Inspector General should assess the feasibility of collecting additional workload data, such as the amount of direct and indirect labor hours associated with each case, and including such data in future personnel requirements assessments, as appropriate. (Recommendation 1) The DOD Inspector General should report regularly to Congress on the timeliness of civilian and contractor investigations, including those contractor and subcontractor cases exceeding the 180-day timeliness requirement. (Recommendation 2) The DOD Inspector General should implement a process to document employee recusals and impairments to independence and incorporate such information into an aggregate-level evaluation of threats to DODIG’s independence. (Recommendation 3) The DOD Inspector General should establish and clearly communicate a declination policy for nondiscretionary cases in the AI Investigations Manual or other guidance, and align this policy with the intake policy. (Recommendation 4) The DOD Inspector General should revise the existing internal controls checklist to include all key case-file documentation and required investigative events. (Recommendation 5) The DOD Inspector General should work in coordination with the Secretary of Defense, the Under Secretary of Defense for Intelligence, and the inspectors general of the defense intelligence components to establish a process to fully implement the requirements of Directive- Type Memorandum 13-008 so that DODIG (1) receives notifications of all allegations received by the components, (2) reviews all component determinations to not investigate allegations, and (3) reviews all investigations conducted by the components. (Recommendation 6) The DOD Inspector General should develop quality performance measures and enhance existing timeliness performance measures to reflect key attributes of successful performance measures. At minimum, these measures should be clear, quantifiable, and objective, and they should include a baseline assessment of current performance. (Recommendation 7) We provided a draft of this report to DODIG for review and comment. In written comments, DODIG concurred with each of the seven recommendations and cited actions it plans to take to address them. DODIG’s comments are reprinted in appendix III. As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 11 days from the report date. At that time, we will send copies of this report to the congressional defense committees, the Secretary of Defense, the Department of Defense Inspector General, the Inspectors General of the Defense Intelligence Agency and the National Security Agency, the Office of Special Counsel, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-3604 or farrellb@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix IV. To determine the extent to which the Department of Defense Office of Inspector General (DODIG) has met and taken steps to achieve key timeliness goals for civilian and contractor whistleblower reprisal investigations, we obtained data on all Department of Defense (DOD) civilian and contractor employee whistleblower reprisal cases closed by DODIG and appropriated-fund DOD civilian employee cases closed by the Office of Special Counsel from October 1, 2012, through September 30, 2015, and on all DODIG civilian and contractor employee cases open as of September 2016. We selected data from this period because they constituted the most complete and recent data available in DODIG’s and the Office of Special Counsel’s case-management systems during the time we were doing our analysis. Using these data, we calculated the timeliness of DODIG and Office of Special Counsel cases in relation to time frames prescribed by statute and internal goals. Specifically, we assessed the timeliness of closed DODIG cases against the 180-day statutory timeliness requirement for contractor cases; DODIG’s 240-day internal goal for all civilian cases; DODIG’s 30-day internal goal for the intake of complaints; and DODIG’s 60-day internal goal for completing oversight reviews that was in place during the years under review. We also assessed the length of DODIG’s open cases involving DOD civilian and contractor employees. We assessed the timeliness of closed Office of Special Counsel cases involving DOD appropriated-fund civilians against the statutory 240-day goal for completing these investigations. To identify other characteristics of DODIG cases involving DOD civilians and contractors we also analyzed case data by fiscal year to determine statute type, case disposition, the number of complaints received, the number of complaints investigated, the frequency and type of alleged personnel actions, and rates of substantiation for investigated cases. We assessed the reliability of DODIG and Office of Special Counsel data by administering questionnaires to officials familiar with the data systems, interviewing DODIG and Office of Special Counsel officials, and reviewing the queries used to retrieve the data, along with system documentation and quality-assurance procedures. For DODIG data, we also compared electronic data to fiscal year 2015 case-file documentation associated with our review of a stratified random sample of fiscal year 2015 case files, discussed below, to determine whether dates had been properly recorded in the system, and we conducted internal logic and range checks to assess completeness and accuracy. We notified DODIG officials of data anomalies found during our review. For example, we found negative values for days in intake for six cases and discrepancies in dates between case-file documentation and database entries for 7 of 11 fiscal year 2015 non-appropriated-fund instrumentality (NAFI) civilian employee cases. DODIG made corrections to these data, which we incorporated into our analysis, as well as to fiscal year 2013 and 2014 NAFI cases. We reviewed such case-data inaccuracies against Council of the Inspectors General on Integrity and Efficiency (CIGIE) standards related to the accuracy of database information and Standards for Internal Control in the Federal Government related to obtaining and presenting reliable information. Overall, we determined that DODIG and Office of Special Counsel data were sufficiently reliable for the purpose of our objectives. Specifically, we determined that DODIG data were reliable to report on the timeliness of DODIG cases by fiscal year and case type, including case-length mean, median, and range; case disposition, including cases declined, dismissed, withdrawn, and investigated, whether substantiated or not substantiated; case outcomes; and other case characteristics, including the number and type of personnel actions reported. Office of Special Counsel data were determined to be sufficiently reliable for limited uses, including the case-length average and median, and range of cases that went through some degree of investigation and were not closed in the Office of Special Counsel’s complaints examining unit and did not go through the Office of Special Counsel’s alternative dispute-resolution process. To identify factors affecting case timeliness and assess DODIG’s steps to improve timeliness, we reviewed documentation pertaining to DODIG and Office of Special Counsel investigations and case management and interviewed cognizant agency officials. We also interviewed the four supervisory investigators of investigative teams, and conducted semistructured interviews with the 24 investigators who collectively are responsible for conducting civilian and contractor investigations. We analyzed the results of the semistructured interviews both quantitatively and qualitatively—as described below. We also compared DODIG’s completed and planned efforts to improve the timeliness of its investigations against DOD guidance related to manpower management, CIGIE standards related to workload data, and Standards for Internal Control in the Federal Government related to the importance of information to oversight bodies. To determine the extent to which DODIG has established processes to ensure that whistleblower reprisal cases are handled independently and thoroughly, we reviewed DODIG investigative policies, processes, practices, guidance, and quality-assurance mechanisms and safeguards pertaining to investigative independence and thoroughness against relevant statutes, CIGIE standards on independence and thoroughness, and federal internal control standards related to control activities and performance measurement. Using the case data we obtained to assess timeliness from fiscal years 2013 through 2015, we also selected a stratified random sample of 187 case files from the population of 409 civilian and contractor whistleblower reprisal cases that were closed by DODIG in fiscal year 2015, from October 1, 2014, through September 30, 2015. We chose to review cases from this period as they constituted the most recent and complete year of data available and would most accurately reflect the extent to which DODIG’s cases were thorough. We stratified the population into 12 strata according to employee type and disposition. We randomly selected cases within 3 of the 12 strata and selected all cases with certainty from the remaining 9 strata as shown in table 5 below. Specifically, we calculated a sample size for stratum 1 (civilian dismissed cases where DODIG declined to take action) to achieve a desired precision of plus or minus 13 percentage points. We calculated the sample size for strata 8 and 9 (contractor dismissed cases) to achieve a desired precision of plus or minus 10 percentage points or fewer. We reviewed all investigated cases. As a result, data collected from our case-file review and presented in this report involving investigated cases do not have a sampling error. During the course of our review, we removed 9 out-of-scope cases, reducing the original sample size from 187 to 178. These cases included 8 cases involving complainants that were military servicemembers and 1 case involving a contractor employee that was a duplicate case. These removals did not materially affect the confidence intervals of our sample and were not replaced because they all occurred in strata where we selected all of the cases for review. We also identified and replaced one case that would have required significant work on the part of DODIG in order to obtain approval to release classified case data. Table 5 describes the 12 strata by adjusted population and sample size. Because we followed a probability procedure based on random selections, our sample is only one of a large number of samples that we might have drawn. Since each sample could have provided different estimates, we express our confidence in the precision of our particular sample’s results as a 95 percent confidence interval (e.g., plus or minus 7 percentage points). This is the interval that would contain the actual population value for 95 percent of the samples we could have drawn. As a result, we are 95 percent confident that each of the confidence intervals in this report will include the true values in the study population. All percentage estimates in this report have a margin of error of plus or minus 7 percentage points at the 95 percent confidence level. To conduct the case-file review, we developed and used a data-collection instrument to capture information regarding general case characteristics, the reliability of database information, and the presence of information and documentation required by law, regulations, policy, and best practices such as standards for investigations established by CIGIE. Core elements of this instrument were shared with DODIG officials to ensure alignment with the policies and practices in place during the cases’ period of investigation, and we incorporated their feedback into our instrument before commencing the file review. These core elements represented individual documents and data elements, and sometimes comprised multiple documents. Examples of elements in our review that represent key data in DODIG’s database, demonstrate compliance with statute, or constitute documentation of key steps of the investigative, case-management, and oversight processes include the following: interview documentation or recording; closure letters to complainants, subjects, and responsible management officials; report reviews and approvals, including management, quality- assurance, and General Counsel; key correspondence, including between DODIG and the complainant, for contractor cases exceeding 180 days; oversight worksheet (oversight reviews); and oversight approval memorandum (oversight reviews). To validate the data-collection instrument and ensure consistency in its application, we developed and used standard operating procedures to review a test sample of 16 case files that were selected randomly from each stratum to ensure that each stratum of the full random sample was tested at least once. We adjusted the relevant case-file elements for each case based on its stratum and circumstances (e.g., by not looking for certain elements for withdrawn cases) and captured responses in our data-collection instrument accordingly. To ensure the reliability of the results of the full case-file review, two analysts reviewed each case file and coded for the presence of required information using the data- collection instrument, with one analyst conducting the initial coding and the second analyst reviewing the first analyst’s work. In the event that disagreement between the two analysts occurred, the analysts discussed and resolved the disagreement by identifying and reviewing supporting database information or documentation, and obtained the input of a third analyst, if necessary, until a final resolution was made. We did not question DODIG’s judgment in these cases. We generalized the results of our sample to the population of 400 in-scope cases closed in fiscal year 2015 with a margin of error of plus or minus 7 percentage points or fewer. To assess the implementation of DODIG independence and thoroughness policies, and to identify supplementary investigative and case-management practices, we interviewed the 24 investigators and four supervisory investigators responsible for conducting civilian and contractor investigations, as well as whistleblower reprisal investigations unit management. To interview the 24 investigators, we used a standardized semistructured interview questionnaire. As part of the questionnaire’s development, a survey specialist helped develop the questionnaire, and another survey specialist provided independent feedback on the questionnaire to ensure that content necessary to understand the questions was included and that the questions could be answered accurately and completely. To minimize errors that might occur from respondents interpreting our questions differently than we intended, we pretested our questionnaire with four investigators (including one senior investigator) to ensure the clarity and reasonableness of the questions. During the pretests, conducted in person and by phone, we read the instructions and each question out loud to the interviewees and asked them to tell us whether (1) the instructions and questions were clear and unambiguous, (2) the terms we used were accurate, and (3) they could offer a potential solution to any problems identified. We noted any potential problems and modified the questionnaire based on the feedback received from the reviewers and pretests as appropriate. We conducted interviews between September 12, 2016, and October 20, 2016. We interviewed all 24 investigators assigned to investigative teams, for a response rate of 100 percent. Each interview was conducted by at least two analysts, who then discussed interview responses to ensure consistency in their interpretation. Because we collected data from every investigator there was no sampling error. However, the practical difficulties of conducting any semistructured interview may introduce errors, commonly referred to as nonsampling errors. For example, differences in how a particular question is interpreted, the sources of information available to interviewees, how the responses were processed and analyzed, or the types of people who do not respond can influence the accuracy of the survey results. We took steps in the development of the interview questionnaire, the data collection, and the data analysis to minimize these nonsampling errors and help ensure the accuracy of the answers that were obtained. For example, a social-science survey specialist designed the questionnaire, in collaboration with analysts having subject-matter expertise. Then, as noted earlier, the draft questionnaire was pretested to ensure that questions were relevant, clearly stated, and easy to comprehend. The questionnaire was also reviewed by another specialist with expertise in survey development, as mentioned above. We calculated the frequency of responses to our bounded questions and performed content analysis on the open-ended questions to identify common themes from across the responses and to determine their frequencies. To perform these analyses, data were manually extracted from the questionnaires into a computer program, and data-entry accuracy was verified. The quantitative analysis was performed in this program by one analyst and independently reviewed by another analyst. For the qualitative analysis, a standard coding scheme was developed to identify common themes and determine their frequencies. We also used professional judgment to identify other themes that did not meet the chosen quantitative threshold, but were determined to be important based on our review of case files, discussions with DODIG management, and review of guidance and relevant standards. This analysis was conducted by one analyst and independently reviewed by another analyst. The pair of analysts then discussed any discrepancies in their coding until they reached a consensus. A selection of key survey questions whose results are described in this report are listed below. Have you received training during your time as a whistleblower reprisal investigations unit investigator? (Interviewer recorded: “Yes,” “No.”) (If yes, we asked a series of questions regarding training on the case- management system, the various civilian and contractor whistleblower statutes, CIGIE standards, federal ethics laws, conflicts of interest, and independence, among other topics. The following question is an example of the typical wording of those questions.) Have you ever received training on the case management system? (Interviewer recorded: “Yes,” “No.”) (If yes) Was this training mandatory? How frequently have you received this training? How was it delivered? Do you have an individual development plan? (Interviewer recorded: “Yes,” “No.”) (If yes) How do you identify training to include in your plan? Is your plan updated? (If yes) How often? Is your plan tracked to ensure that training has been completed? (If yes) Who is responsible for tracking it? Would a standardized training curriculum with material specific to your core responsibilities as a whistleblower reprisal investigations unit investigator help you execute your responsibilities as an investigator? (Interviewer recorded: “Yes,” “No.”) (If yes) Why?’ (If no) Why not? During intake, do you document any information from the intake interview with the complainant in the case-management system? (Interviewer recorded: “Yes,” “No.”) (If yes) How is this information documented? In your opinion, is it generally feasible to complete contractor investigations within 180 days? (This question was repeated for civilian investigations within 240 days.) (Interviewer recorded: “Yes,” “No,” “Don’t Know.”) (If yes) Why do you think it is feasible? (If no) Why do you think it is not feasible? Are there factors that have typically contributed to any delays you’ve encountered in meeting investigative time frames? (Interviewer recorded: “Yes,” “No,” “Don’t Know,” “No delays.”) (If yes) What are the factors? As an investigation unfolds, how often, if at all, do investigative planning roundtables occur? (This question was repeated for five other types of roundtables.) Who typically attends investigative planning roundtables? Have you ever observed bias by another investigator, supervisory investigator, whistleblower reprisal investigations unit management, or other staff involved in the investigative or review processes? (Interviewer recorded: “Yes,” “No,” “Don’t Know.”) (If yes) Please provide an example of bias without identifying the individuals involved. In your opinion, what, if anything, could be done to improve the handling of whistleblower reprisal investigations for DOD civilians and contractors? Separately, we also reviewed training materials, guidance, and requirements for DODIG investigators and attended three DODIG Administrative Investigations training symposia, which included sessions on recent developments in whistleblower reprisal investigations, investigator competencies, hotline best practices, and protecting investigative source identities. During the course of our semistructured interviews, we also discussed training practices, requirements, and experiences with the 24 investigators responsible for conducting investigations, and separately discussed these same topics and other key survey questions with the four supervisory investigators responsible for supervising the investigative teams. To determine the extent to which DODIG has conducted oversight of civilian reprisal cases handled by the defense intelligence components, we assessed DODIG, National Security Agency IG, and Defense Intelligence Agency IG oversight policies, processes, practices, and guidance against DOD policy that implements Presidential Policy Directive 19 (PPD-19) and governs the whistleblower reprisal case oversight process. We also reviewed the four investigations completed by the defense intelligence components—specifically, the National Security Agency and Defense Intelligence Agency—in fiscal year 2015 and submitted to DODIG for review; and interviewed cognizant officials from DODIG, the National Security Agency IG, and the Defense Intelligence Agency IG to discuss existing oversight practices and to obtain relevant perspectives on the requirements for case oversight articulated by DOD policy. To determine the extent to which DODIG has developed performance measures to assess the timeliness and quality of its investigations, we reviewed documentation including quarterly briefing materials, annual reports, and internal control processes and checklists, and interviewed DODIG officials, to identify timeliness and quality performance measures for investigations and oversight reviews that have been variably used by DODIG since fiscal year 2013. We compared this information to federal internal control standards related to performance assessment, including standards related to defining measures for assessing an entity’s performance in achieving quantitative and qualitative objectives. DODIG defined timeliness measures for fiscal year 2017 in April 2017 but it had not identified quality performance measures for fiscal year 2017 as of that time. As a result, we assessed DODIG’s fiscal year 2017 timeliness measures against 10 key attributes of successful measures identified in our prior work, including clarity, measurable targets, and baseline and trend data. To perform this analysis, two analysts each used a scorecard methodology to review the selected measures and determine whether the measures “addressed,” “partially addressed,” or “did not address” the attributes. We scored an attribute as “addressed” if all aspects of an attribute were addressed; “partially addressed” if some, but not all, aspects of an attribute were addressed; and “not addressed” if the measure did not address any aspects of an attribute. To reconcile disagreements, the two analysts discussed and resolved the differences with the input of a third analyst, as necessary. We conducted this performance audit from February 2016 to September 2017 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. This appendix provides information on the characteristics of civilian and contractor or subcontractor whistleblower reprisal cases based on our analysis of fiscal year 2013 through fiscal year 2015 case data from the Department of Defense Office of Inspector General (DODIG) case- management system and our review of a sample of 178 cases closed from October 1, 2014, through September 30, 2015. The majority of civilian and contractor and subcontractor complaints closed by DODIG in fiscal year 2013 through fiscal year 2015—1,094 of 1,197, or about 91 percent—were closed without investigation, while 103, or about 9 percent, were fully investigated. Specifically, DODIG closed 29 fully investigated cases in fiscal year 2013, 42 fully investigated cases in fiscal year 2014, and 32 fully investigated cases in fiscal year 2015. Our analysis of data from DODIG’s case-management system showed that the overall number of declined cases—those that do not go through the intake process—as a proportion of total closed civilian and contractor cases rose from fiscal year 2013 to fiscal year 2015. Specifically, there were 176 declined cases, or 50 percent, out of a total of 353 closed cases in fiscal year 2013; 270, or 60 percent, out of 447 in fiscal year 2014; and 243, or about 61 percent, out of 397 in fiscal year 2015. While appropriated-fund civilian complaints resulted in the most declinations for each fiscal year as, according to an official, these are generally referred to the Office of Special Counsel, DODIG also declined complaints involving non-appropriated-fund instrumentality (NAFI) civilians, contractors or subcontractors, and Defense Civilian Intelligence Personnel System (DCIPS) employees and any employees with eligibility for access to classified information, as discussed in our report. Figure 4 shows the percentage of cases closed in each fiscal year, by case disposition. Substantiation rates of closed investigations varied by statute across the 3 fiscal years. NAFI civilian cases constituted the greatest proportion of closed investigations—44 of 103, or about 43 percent—but the highest substantiation rate was for investigations of complaints made by DOD appropriated-fund civilians. As previously noted, DODIG retains for investigation those cases involving appropriated-fund civilians that are deemed to be of interest to DODIG or to the Secretary of Defense. Table 6 shows the number of closed investigations for each covered employee group, the number of substantiated investigations, and the associated substantiation rates across the 3 fiscal years. In fiscal years 2013 through 2015, DODIG received a total of 1,208 complaints from appropriated-fund civilians; NAFI civilians; contractors, subcontractors, grantees, and subgrantees; and DCIPS employees or employees with eligibility for access to classified information alleging whistleblower reprisal. The majority of complaints, 666, were from appropriated-fund civilians. An additional 376 complaints came from contractors, subcontractors, grantees, and subgrantees. NAFI civilians brought 123 complaints, and the fewest complaints, 43, came from DCIPS employees or employees with eligibility for access to classified information. Figure 5 shows the percentage of whistleblower reprisal complaints received by DODIG from each covered group across the 3 fiscal years. Of the 1,208 complaints, 758, or 63 percent, were filed by civilians and contractors or subcontractors affiliated with the military services. Among these, civilians and contractors or subcontractors affiliated with the Army made the highest number of complaints, with 304; the Air Force had 219; the Navy had 189; and the Marine Corps had 46. Defense agencies had another 203, and the rest were spread across the combatant commands, joint commands, and other entities. Figure 6 provides the percentages of civilian and contractor complaints received across all 3 fiscal years, by organization. We also received data on the prohibited personnel actions alleged by complainants who filed whistleblower reprisal complaints in fiscal year 2013 through fiscal year 2015. Some prohibited personnel actions are specific to certain statutes or to Presidential Policy Directive 19 (PPD-19). For example, prohibited personnel actions involving the restriction of one’s access to classified information are specific to PPD-19. Conversely, other personnel actions are common across more than one statute. For example, a change in duties or responsibilities is defined as a prohibited personnel action under both 5 U.S.C. § 2302 and 10 U.S.C. § 1587. Our analysis of data from DODIG’s case-management system showed that the complaints received across the 3 fiscal years within our scope alleged 935 prohibited personnel actions. Of those 935, 927 were aligned with prohibited personnel actions in the DOD civilian and contractor whistleblower statutes or PPD-19. To determine the frequency of alleged prohibited personnel actions, we grouped similar actions from each statute and PPD-19. Across all 3 fiscal years, we found that the greatest proportion of alleged prohibited personnel actions, at 24 percent, was demotion, suspension, or termination, which was mainly liked to contractor, subcontractor, grantee, and subgrantee complainants. However, two indeterminate categories—unspecified and other— constituted the greatest proportion of alleged prohibited personnel actions, with about 36 percent, when combined. Figure 7 provides the percentages of prohibited personnel actions falling under the broader categories alleged in the civilian and contractor or subcontractor whistleblower reprisal complaints received by DODIG from fiscal year 2013 through fiscal year 2015 and provided to us. Our review of a representative sample of 178 cases closed by DODIG in fiscal year 2015 showed that the majority of protected disclosures described in whistleblower reprisal complaints alleged a violation of law, rule, or regulation. Specifically, of 348 protected disclosures recorded in the case files we reviewed, 251 alleged a violation of law, rule, or regulation. The next largest category was a composite field we created to capture those disclosures recorded in the system that DODIG determined were not protected. This category, other/none/not determined to be a protected disclosure, included 41 disclosures. Table 7 provides the estimated percentages of alleged protected disclosures for all cases closed in fiscal year 2015, based on our sample. The recipients of the protected disclosures in our sample of closed cases from fiscal year 2015 varied somewhat due to differences in the statutes and PPD-19. For instance, 10 U.S.C. §1587 states that NAFI civilians can disclose whistleblowing to anyone and to any civilian employee or member of the armed forces designated by law or by the Secretary of Defense to receive disclosures, which together accounted for 75 of 79 NAFI disclosure recipients in our sample. Table 8 provides estimated percentages of the recipients of protected disclosures for all cases closed in fiscal year 2015, based on our sample. In addition to the contact named above, Lori Atkinson, Assistant Director; James Ashley; Tracy Barnes; David Ballard; Ryan D’Amore; Taylor Hadfield; Cale Jones; Amie Lesser; Marc Molino; Richard Powelson; Cynthia Saunders; Michael Silver; and Erik Wilkins-McKee made key contributions to this report. Contractor Whistleblower Protections Pilot Program: Improvements Needed to Ensure Effective Implementation. GAO-17-227. Washington, D.C.: March 2, 2017. Whistleblower Protection: Additional Actions Would Improve Recording and Reporting of Appeals Data. GAO-17-110. Washington, D.C.: November 28, 2016. Whistleblower Protection: DOD Has Improved Oversight for Reprisal Investigations but Can Take Additional Actions to Standardize Process and Reporting. GAO-16-860T. Washington, D.C.: September 7, 2016. Department of Energy: Whistleblower Protections Need Strengthening. GAO-16-618. Washington, D.C.: July 11, 2016. Whistleblower Protection: DOD Needs to Enhance Oversight of Military Whistleblower Reprisal Investigations. GAO-15-477. Washington, D.C.: May 7, 2015. Whistleblower Protection: Additional Actions Needed to Improve DOJ’s Handling of FBI Retaliation Complaints. GAO-15-112. Washington, D.C.: January 23, 2015. Whistleblower Protection Program: Opportunities Exist for OSHA and DOT to Strengthen Collaborative Mechanisms. GAO-14-286. Washington, D.C.: March 19, 2014. Whistleblower Protection: Actions Needed to Improve DOD’s Military Whistleblower Reprisal Program. GAO-12-362. Washington, D.C.: February 22, 2012. Tax Whistleblowers: Incomplete Data Hinders IRS’s Ability to Manage Claim Processing Time and Enhance External Communication. GAO-11-683. Washington, D.C.: August 10, 2011. Criminal Cartel Enforcement: Stakeholder Views on Impact of 2004 Antitrust Reform Are Mixed, but Support Whistleblower Protection. GAO-11-619. Washington, D.C.: July 25, 2011. Whistleblower Protection: Sustained Management Attention Needed to Address Long-Standing Program Weaknesses. GAO-10-722. Washington, D.C.: August 17, 2010. Defense Contracting Integrity: Opportunities Exist to Improve DOD’s Oversight of Contractor Ethics Programs. GAO-09-591. Washington, D.C.: September 22, 2009. Whistleblower Protection Program: Better Data and Improved Oversight Would Help Ensure Program Quality and Consistency. GAO-09-106. Washington, D.C.: January 27, 2009. Justice and Law Enforcement: Office of Special Counsel Needs to Follow Structured Life Cycle Management Practices for Its Case Tracking System. GAO-07-318R. Washington, D.C.: February 16, 2007. U.S. Office of Special Counsel: Strategy for Reducing Persistent Backlog of Cases Should Be Provided to Congress. GAO-04-36. Washington, D.C.: March 8, 2004.

Summary: Whistleblowers play an important role in safeguarding the federal government against waste, fraud, and abuse. However, whistleblowers also risk reprisal, such as demotion, reassignment, and firing. GAO was asked to review DODIG's reprisal investigations program for DOD civilians and contractors. This report examines the extent to which DODIG has (1) met and taken steps to achieve key timeliness goals for civilian and contractor reprisal investigations, (2) established processes to ensure that civilian and contractor reprisal cases are handled independently and thoroughly, (3) conducted oversight of civilian reprisal cases handled by the defense intelligence components, and (4) developed performance measures to assess the timeliness and quality of its investigations. GAO analyzed DODIG data for cases closed from fiscal years 2013 through 2015, reviewed a generalizable random sample of 178 cases closed in 2015, which included all fully investigated cases, and interviewed cognizant officials and investigators. The Department of Defense Office of Inspector General (DODIG) did not meet statutory or internal timeliness goals for more than 83 percent of the Department of Defense (DOD) civilian and contractor employee whistleblower reprisal investigations it closed in fiscal years 2013 through 2015. DODIG has taken steps to improve timeliness and has reduced the average length of its investigations, intake process, and oversight reviews. Although the average length of all closed investigations improved by about 20 percent over the 3 fiscal years, it was significantly longer than the established timeliness goals. For example, DODIG's timeliness goal is 240 days for DOD appropriated-fund and non-appropriated-fund civilians, but in fiscal year 2015 the average length of these investigations was 608 and 402 days, respectively. Similarly, the statutory timeliness goal for DOD contractors and subcontractors is 180 days, and in fiscal year 2015 the average length for those investigations was 285 days. To continue to improve timeliness, DODIG requested funds to increase its personnel, but it has yet to determine the feasibility of collecting key workload data such as labor hours that would enable it to strengthen its assessment of personnel requirements and allocate personnel in the most efficient manner. DODIG has established processes to help ensure the independence and thoroughness of the DOD civilian and contractor cases it handles, including a quality-assurance process and an internal controls process. However, a lack of documentation may limit its ability to fully evaluate threats to its independence, and it does not always follow its complaint intake process. GAO's review of case files closed by DODIG in fiscal year 2015 found that some key documentation or data needed to demonstrate compliance with these processes were missing or were not uploaded to DODIG's case-management system in a timely manner. GAO found, and DODIG officials acknowledged, that DODIG's internal controls checklist for assessing case-file completeness does not capture all key documentation and investigative events, thus limiting DODIG's ability to ensure the completeness and accuracy of case-file documentation and data. DODIG has conducted oversight of reprisal cases provided by some of the defense intelligence component inspectors general—such as the Defense Intelligence Agency and the National Security Agency. However, DODIG and the components have not fully addressed requirements related to DODIG's oversight of component cases, and there is not a process to do so. As a result, DODIG and the components are unable to completely fulfill their prescribed roles related to the oversight of component cases. In April 2017, DODIG developed performance measures to assess the timeliness of its investigations and oversight reviews for fiscal year 2017 that demonstrate many, but not all, attributes of successful performance measures. However, DODIG has not established performance measures to assess the quality of its investigations and oversight reviews for fiscal year 2017 and beyond. By developing performance measures that fully reflect these key attributes, DODIG will be better positioned to assess the timeliness and quality of its investigations and oversight reviews and determine whether initiatives are on track to achieve desired outcomes. GAO is making seven recommendations, including that DODIG assess the feasibility of collecting key workload data, document threats to independence and incorporate such information into an evaluation of independence threats, strengthen its internal controls checklist, develop a process to oversee all defense intelligence component cases, and develop performance measures to assess its quality and timeliness. DODIG concurred with the recommendations.