Whistleblower Protection: Opportunities Exist for DOD to Improve the Timeliness and Quality of Civilian and Contractor Reprisal Investigations
Published: 2017-09-29T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Whistleblowers play an important role in safeguarding the federal government against waste, fraud, and abuse. However, whistleblowers also risk reprisal, such as demotion, reassignment, and firing.
GAO was asked to review DODIG's reprisal investigations program for DOD civilians and contractors. This report examines the extent to which DODIG has (1) met and taken steps to achieve key timeliness goals for civilian and contractor reprisal investigations, (2) established processes to ensure that civilian and contractor reprisal cases are handled independently and thoroughly, (3) conducted oversight of civilian reprisal cases handled by the defense intelligence components, and (4) developed performance measures to assess the timeliness and quality of its investigations. GAO analyzed DODIG data for cases closed from fiscal years 2013 through 2015, reviewed a generalizable random sample of 178 cases closed in 2015, which included all fully investigated cases, and interviewed cognizant officials and investigators.
section_title: What GAO Found
paragraphs: The Department of Defense Office of Inspector General (DODIG) did not meet statutory or internal timeliness goals for more than 83 percent of the Department of Defense (DOD) civilian and contractor employee whistleblower reprisal investigations it closed in fiscal years 2013 through 2015. DODIG has taken steps to improve timeliness and has reduced the average length of its investigations, intake process, and oversight reviews. Although the average length of all closed investigations improved by about 20 percent over the 3 fiscal years, it was significantly longer than the established timeliness goals. For example, DODIG's timeliness goal is 240 days for DOD appropriated-fund and non-appropriated-fund civilians, but in fiscal year 2015 the average length of these investigations was 608 and 402 days, respectively. Similarly, the statutory timeliness goal for DOD contractors and subcontractors is 180 days, and in fiscal year 2015 the average length for those investigations was 285 days. To continue to improve timeliness, DODIG requested funds to increase its personnel, but it has yet to determine the feasibility of collecting key workload data such as labor hours that would enable it to strengthen its assessment of personnel requirements and allocate personnel in the most efficient manner.
DODIG has established processes to help ensure the independence and thoroughness of the DOD civilian and contractor cases it handles, including a quality-assurance process and an internal controls process. However, a lack of documentation may limit its ability to fully evaluate threats to its independence, and it does not always follow its complaint intake process. GAO's review of case files closed by DODIG in fiscal year 2015 found that some key documentation or data needed to demonstrate compliance with these processes were missing or were not uploaded to DODIG's case-management system in a timely manner. GAO found, and DODIG officials acknowledged, that DODIG's internal controls checklist for assessing case-file completeness does not capture all key documentation and investigative events, thus limiting DODIG's ability to ensure the completeness and accuracy of case-file documentation and data.
DODIG has conducted oversight of reprisal cases provided by some of the defense intelligence component inspectors general—such as the Defense Intelligence Agency and the National Security Agency. However, DODIG and the components have not fully addressed requirements related to DODIG's oversight of component cases, and there is not a process to do so. As a result, DODIG and the components are unable to completely fulfill their prescribed roles related to the oversight of component cases.
In April 2017, DODIG developed performance measures to assess the timeliness of its investigations and oversight reviews for fiscal year 2017 that demonstrate many, but not all, attributes of successful performance measures. However, DODIG has not established performance measures to assess the quality of its investigations and oversight reviews for fiscal year 2017 and beyond. By developing performance measures that fully reflect these key attributes, DODIG will be better positioned to assess the timeliness and quality of its investigations and oversight reviews and determine whether initiatives are on track to achieve desired outcomes.
section_title: What GAO Recommends
paragraphs: GAO is making seven recommendations, including that DODIG assess the feasibility of collecting key workload data, document threats to independence and incorporate such information into an evaluation of independence threats, strengthen its internal controls checklist, develop a process to oversee all defense intelligence component cases, and develop performance measures to assess its quality and timeliness. DODIG concurred with the recommendations.

section_title: Letter
paragraphs: Whistleblowers play an important role in safeguarding the federal  government against waste, fraud, and abuse, and their willingness to  come forward can contribute to improvements in government operations.  However, whistleblowers also risk reprisal, such as demotion,  reassignment, and firing. Congress and the former administration had  established a statutory and policy framework to protect whistleblowers  from reprisal for disclosing information concerning, among other things,  fraud, waste, and abuse to designated persons such as an inspector  general (IG). Under this framework and its implementing directives, the  Department of Defense Office of Inspector General (DODIG) is  responsible for investigating and overseeing the investigation of  complaints alleging reprisal against certain DOD civilian employees and  for conducting investigations of complaints alleging reprisal against DOD  contractor employees. Specifically, DODIG is the primary investigative  authority for reprisal complaints involving Department of Defense (DOD)  civilian non-appropriated-fund instrumentality (NAFI) employees, and  DOD contractor, subcontractor, grantee and subgrantee employees.  DODIG may also investigate, on a discretionary basis, reprisal complaints  it receives from DOD civilian appropriated-fund employees. Finally,  DODIG is responsible for investigating reprisal complaints involving DOD civilian employees under the Defense Civilian Intelligence Personnel  System (DCIPS), overseeing the investigation of such complaints by the  defense intelligence components, and investigating reprisal complaints  involving actions affecting DOD civilian and contractor employees’ access  to classified information. In recent years, members of Congress have  expressed concerns regarding the timeliness and integrity of DODIG’s  reprisal investigations, along with its interpretation of statutory protections  for contractor employees, among other things.
In February 2012 and May 2015, we reported that DOD faced challenges  in overseeing the military services’ whistleblower reprisal investigations  and made 18 recommendations to help improve the timeliness of military  servicemembers’ whistleblower reprisal investigations and to improve  investigation and oversight processes, among other things. For example,  we reported that DODIG was not consistently or accurately recording key  dates to track the length of investigations, did not report the timeliness of  its investigations to Congress, and had outdated guidance about the  investigation process. DOD concurred with all of our recommendations  and as of September 2017 had taken action to address 17 of them.
You asked us to examine DODIG’s conduct and oversight of  whistleblower reprisal investigations involving DOD civilian and contractor  employees. This report examines the extent to which DODIG has: (1) met  and taken steps to achieve key timeliness goals for civilian and contractor  whistleblower reprisal investigations; (2) established processes to ensure  that civilian and contractor whistleblower reprisal cases are handled  independently and thoroughly; (3) conducted oversight of civilian reprisal  cases handled by the defense intelligence components; and (4)  developed performance measures to assess the timeliness and quality of  its investigations.
For the first objective, we obtained data on all civilian and contractor  whistleblower reprisal cases closed by DODIG and appropriated-fund  DOD civilian cases closed by the Office of Special Counsel from October  1, 2012, through September 30, 2015, and on all DODIG cases open as  of September 2016. We selected data from this period because they  constituted the most complete and recent data available in DODIG’s and  the Office of Special Counsel’s case-management systems during the  time we were doing our analysis. Using these data, we calculated the  timeliness of DODIG and Office of Special Counsel cases in relation to  time frames prescribed by statute and internal DOD goals, and assessed  other characteristics including case type and rates of substantiation. We  assessed the reliability of DODIG and Office of Special Counsel data by  administering questionnaires, interviewing cognizant officials, and  reviewing system documentation and quality-assurance procedures. For  DODIG data, we also compared electronic data to fiscal year 2015 case- file documentation and conducted internal checks. After correcting  several anomalies in DODIG’s data, we determined that DODIG and  Office of Special Counsel data were sufficiently reliable for the purposes  of our objectives. To identify factors affecting case timeliness, we  reviewed documentation pertaining to DODIG and Office of Special  Counsel investigations and case management, and interviewed agency  officials. We also compared DODIG’s completed and planned efforts to  improve the timeliness of its investigations against DOD guidance,  Council of the Inspectors General on Integrity and Efficiency (CIGIE)  standards, and Standards for Internal Control in the Federal  Government related to the importance of information to oversight  bodies.
For the second objective, we assessed DODIG investigative policies,  processes, practices, guidance, and quality-assurance mechanisms  pertaining to investigative independence and thoroughness against  relevant statutes, CIGIE standards, and federal internal control standards  related to control activities and performance measurement. To assess  the implementation of DODIG independence and thoroughness policies,  and to identify supplementary practices, we interviewed the 24  investigators and four supervisory investigators responsible for  conducting civilian and contractor investigations, as well as whistleblower  reprisal investigations unit management. We also selected a stratified  random sample of 178 case files from the population of 409 civilian and  contractor whistleblower reprisal cases that were closed by DODIG in  fiscal year 2015, from October 1, 2014, through September 30, 2015.  This sample included all fully investigated cases. We chose to review  cases from this period as they constituted the most recent and complete  year of data available and would most accurately reflect the extent to  which DODIG’s cases were thorough. To conduct this review, we  developed and used a data-collection instrument to capture information  regarding general case characteristics and the presence of information  and documentation required by law, regulations, policies, and best  practices. Core elements of this instrument were shared with DODIG  officials to ensure alignment with the policies and practices in place  during the cases’ period of investigation. To help ensure the reliability of  the results of the case-file review, two analysts independently reviewed  each case file and coded for the presence of required information using  the data-collection instrument. In the event of disagreement between the  two analysts, the analysts discussed and resolved the disagreement by  identifying and reviewing supporting database information or  documentation, and obtained the input of a third analyst, if necessary. We  did not question DODIG’s judgment in these cases. We generalized the  results of our sample to the population of 400 cases closed in fiscal year  2015 with a margin of error of plus or minus 7 percentage points or  fewer.
For the third objective, we reviewed DODIG, National Security Agency IG,  and Defense Intelligence Agency IG oversight policies, processes,  practices, and guidance against DOD policy that implements Presidential  Policy Directive 19 (PPD-19) and governs the whistleblower reprisal case  oversight process. We also reviewed 4 investigations completed by the  defense intelligence components in fiscal year 2015 and submitted to  DODIG for review, and interviewed cognizant officials from DODIG, the  National Security Agency IG, and the Defense Intelligence Agency IG to  discuss their oversight practices.
For the fourth objective, we reviewed documentation and interviewed  officials on DODIG’s current and past timeliness and quality performance  measures for investigations and oversight reviews, and compared this  information to federal internal control standards related to performance  assessment. We also assessed DODIG’s fiscal year 2017 timeliness  performance measures against key attributes of successful performance  measures that GAO has previously identified. At the time of our review,  DODIG had not developed quality performance measures for fiscal year  2017. To perform this analysis, two analysts used a scorecard  methodology to independently review the measures and to determine  whether the measures that we assessed “addressed,” “partially  addressed,” or “did not address” the attributes. We scored an attribute as  “addressed” if all aspects of an attribute were addressed; “partially  addressed” if some, but not all, aspects of an attribute were addressed;  and “not addressed” if the measure did not address any aspects of an  attribute. To reconcile disagreements, the two analysts discussed and  resolved the differences with the input of a third analyst, as necessary.  Appendix I provides additional details about our scope and methodology.
We conducted this performance audit from February 2016 to September  2017 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
subsections: section_title: Whistleblower Protections for DOD Civilian and Contractor Employees Are Provided by Statutes and Presidential Policy
paragraphs: Various statutes, regulations, and presidential policy protect different  groups of DOD civilians and contractors from acts of reprisal for  whistleblowing. Specifically, these groups comprise (1) DOD  appropriated-fund employees, (2) DOD NAFI employees, (3) DOD  contractor, subcontractor, grantee, and subgrantee employees, and (4)  DCIPS employees of the defense intelligence community and any  employee with eligibility for access to classified information. Table 1  summarizes the statutory and policy authorities covering each employee  group, along with selected protected disclosures and prohibited personnel  actions—which are two required elements of the test for determining  whether there was reprisal against a complainant for whistleblowing. A  protected disclosure is a disclosure of wrongdoing by a whistleblower to a  party that is an eligible recipient of that disclosure, while prohibited  personnel actions are those actions that are taken or threatened in  response to a protected disclosure, such as termination, reassignment, or  a significant change in duties, responsibilities, or working conditions.
section_title: Several Entities Are Responsible for Investigating DOD Civilian and Contractor Employee Whistleblower Reprisal Complaints
paragraphs: DODIG, the Office of Special Counsel, and the IGs of the defense  intelligence components have responsibility for investigating  whistleblower reprisal complaints brought by DOD civilian, contractor,  subcontractor, grantee, and subgrantee employees. Among other  matters, DODIG is responsible for investigating and overseeing  component IG investigations of complaints alleging reprisal against  certain DOD civilian employees and for conducting investigations of  complaints alleging reprisal against DOD contractor, subcontractor,  grantee, and subgrantee employees. Specifically:
Civilian NAFI employees—DODIG is to investigate all reprisal  complaints from these employees under 10 U.S.C. § 1587. DOD  Directive 1401.03 requires that DODIG submit investigation reports  involving NAFI employees to the Director of Administration and  Management, who has responsibility for final adjudication and the  direction of corrective action.
DOD contractor, subcontractor, grantee and subgrantee employees— DODIG is responsible for investigating all complaints of reprisal under  10 U.S.C. § 2409.
Civilian appropriated-fund employees—DODIG may investigate, on a  discretionary basis, reprisal complaints from these employees that are  of particular interest to DODIG under the Inspector General Act of  1978, as amended. DODIG typically retains for investigation those  appropriated-fund civilian cases received through the DODIG hotline  that involve retaliation for making a sexual-assault complaint,  misconduct by senior DOD officials, and other high-profile issues that  may be of interest to DODIG. While DODIG may exercise its  discretionary authority to investigate appropriated-fund cases it  receives, the Office of Special Counsel has primary jurisdiction to  investigate the majority of civilian whistleblower reprisal cases across  the federal government, including those involving DOD appropriated- fund civilians. The Office of Special Counsel has the authority to  seek corrective actions, including through mediation with employing  agencies.
DCIPS employees and any employees with eligibility for access to  classified information—DODIG is to conduct investigations of reprisal  involving these employees and oversee the investigation of such  complaints by the defense intelligence components under PPD-19  Part A and Directive-Type Memorandum 13-008, DOD’s implementing  guidance. The Under Secretary of Defense for Intelligence issued  Directive-Type Memorandum 13-008 to establish DOD policy, assign  responsibilities, and provide procedures for the implementation of  PPD-19 by the DOD components, which include the defense  intelligence components and their IGs. Under PPD-19 Part B, DODIG  is to investigate reprisal allegations involving actions affecting  eligibility for access to classified information. Under the Intelligence  Authorization Act for Fiscal Year 2014 and the Intelligence  Authorization Act for Fiscal Year 2010, the defense intelligence  component IGs—the IGs of the Defense Intelligence Agency, the  National Geospatial-Intelligence Agency, the National  Reconnaissance Office, and the National Security Agency—have  independent statutory authority to conduct investigations of reprisal  complaints brought by DCIPS employees. DOD Directive-Type  Memorandum 13-008, which implements PPD-19 within DOD,  requires the defense intelligence component IGs to notify DODIG of  whistleblower reprisal allegations they receive, and grants DODIG the  authority to retain such cases for investigation or refer them back to  the component IG, in which case DODIG conducts oversight of the  investigation.
section_title: DODIG Employs a Standard Investigation Process for DOD Civilian and Contractor Cases
paragraphs: Upon receipt of a whistleblower reprisal complaint from a DOD civilian,  contractor, subcontractor, grantee, or subgrantee employee, DODIG  follows a standard intake process to analyze the complaint for a prima  facie allegation. According to DODIG’s AI Investigations Manual, the  elements of prima facie generally consist of: (1) whether a protected  whistleblowing disclosure was made, (2) whether the alleged reprisal  action followed the disclosure, (3) whether the subjects alleged to have  taken the alleged reprisal action could reasonably have had knowledge of  the protected disclosure, and (4) whether the alleged facts support an  inference of reprisal. If the complaint is retained by DODIG for  investigation, DODIG employs a standard investigative process. Figure 1  provides a summary of DODIG’s intake and investigation process from  complaint receipt through case closure.
section_title: Council of the Inspectors General on Integrity and Efficiency Standards
paragraphs: CIGIE’s Quality Standards for Federal Offices of Inspector General and  Quality Standards for Investigations collectively provide a set of  overarching principles that IGs should adhere to in conducting their  operations, including whistleblower reprisal investigations. Specifically,  CIGIE’s Quality Standards for Federal Offices of Inspector General sets  forth the overall quality framework for managing, operating, and  conducting the work of IGs by providing guidance on ethics,  independence, confidentiality, internal control, quality assurance, and  professional standards, among other things. CIGIE’s Quality Standards  for Investigations provides a framework for conducting high-quality  investigations through general and qualitative investigative standards.  General standards, among other things, address the qualifications of  investigators, independence, and the concept of due professional care.  Qualitative standards focus on investigative planning, execution,  reporting, and information management.
section_title: Timeliness Has Improved, but DODIG Generally Did Not Meet Timeliness Goals between Fiscal Years 2013 and 2015 and Has Not Collected Key Workload Data or Reported Regularly to Congress
paragraphs: DODIG generally did not meet statutory and internal timeliness goals for  conducting and overseeing DOD civilian and contractor reprisal cases  from fiscal years 2013 through 2015, but improved its timeliness for some  categories of cases, as well as its intake process and oversight reviews.  As a result of our review, DODIG implemented a process to ensure that  information is updated in its case-management system so that it can  accurately determine the total length of NAFI civilian reprisal cases.  Further, DODIG has taken and plans to take additional steps to improve  the timeliness of conducting and overseeing investigations but has not  assessed the feasibility of collecting key workload data that would enable  it to more fully assess its personnel requirements and has not reported  regularly to Congress on the timeliness of civilian and contractor reprisal  investigations.
subsections: section_title: In Fiscal Years 2013– 2015, DODIG Generally Did Not Meet Statutory and Internal Goals for Civilian and Contractor Reprisal Cases, but Timeliness Improved
paragraphs: DODIG generally did not meet its statutory and internal timeliness goals  for the DOD civilian and contractor whistleblower reprisal investigations  that it closed in fiscal years 2013 through 2015, but improved timeliness  for some categories of cases, as well as its intake process and oversight  reviews during that time. Table 2 shows the statutory and internal  timeliness goals for DODIG civilian and contractor employee reprisal  investigations, intake reviews, and oversight reviews.
Our review found that DODIG received 1,208 civilian and contractor  complaints from fiscal year 2013 through fiscal year 2015 and closed  1,155 cases by declining (i.e., declining to investigate), dismissing, or  investigating complaints. Of the 1,155 cases DODIG closed, 103 were fully investigated. Overall, DODIG did not meet statutory or internal  timeliness goals in about 83 percent of the investigations it closed in fiscal  years 2013 and 2014, and about 84 percent of the investigations it closed  in fiscal year 2015. However, the average length of all closed  investigations improved by about 20 percent (99 days), decreasing from  505 days in fiscal year 2013 to 406 days in fiscal year 2015. Further, the  aggregate median case length of investigations decreased by about 34  percent (170 days), from 503 days in fiscal year 2013 to 333 days in fiscal  year 2015. Figure 2 shows the average and median case length by  employee group across each fiscal year, as well as the range of these  cases and the percentage of cases meeting the timeliness target.
Our review found that DODIG’s timeliness in conducting intake reviews  and oversight reviews also improved across the 3 fiscal years, although  DODIG did not always meet timeliness goals. Specifically, DODIG’s  average time for conducting intake reviews improved from 62 days in  fiscal year 2013 to 49 days in fiscal years 2014 and 2015, a decrease of  about 22 percent. DODIG met its timeliness goal in 50 percent of intake  reviews in fiscal year 2013, 42 percent in fiscal year 2014, and 37 percent  in fiscal year 2015. The time that DODIG took to conduct oversight  reviews of investigations conducted by the defense intelligence  components similarly decreased, from an average of 32 days in fiscal  year 2013 to 24 days in fiscal year 2015, a decrease of 25 percent.  DODIG met its 60-day timeliness goal in two of three such oversight  reviews in fiscal year 2013 and all four reviews in fiscal year 2015.  DODIG did not close any oversight reviews of investigations conducted  by the defense intelligence components in fiscal year 2014. Appendix II  provides information on the general characteristics of DOD civilian and  contractor whistleblower reprisal cases.
DODIG officials and investigators cited a number of factors that have  affected DODIG’s ability to meet its statutory and internal timeliness  goals, including case volume, the complexity of cases, DODIG’s internal  review process, a focus on completing intake reviews, and an influx of  new investigators, who require training and cannot initially handle the  same caseload as more experienced investigators. Chief among these,  according to DODIG officials, is case volume—particularly in relation to  existing program resources. In an October 2016 letter to members of  Congress, the Acting DOD Inspector General described DODIG’s  increasing caseload, stating that DODIG’s budget had not kept pace with the growth in the department’s budget and that the DODIG budget clearly  had not grown commensurate with its increase in responsibilities,  particularly in the whistleblower area. Specifically, the Acting DOD  Inspector General stated that in fiscal year 2016, DODIG received 1,594  whistleblower reprisal complaints and closed 1,492 cases, constituting 30  percent and 27 percent increases over fiscal year 2015, respectively.  He added that while DODIG had increased its staff in the whistleblower  reprisal unit from 28 in 2010 to 54 in 2016 (about 93 percent), these  increases were not sufficient to keep up with the “burgeoning” workload.  Specifically, DODIG analysis showed that over the same period, the  number of incoming complaints and cases closed increased by 158  percent and 136 percent, respectively. In addition, 18 of the 24  investigators we interviewed stated that their intake and investigation  caseload make it less feasible to meet timeliness goals, with 8 stating that  the high number of complaints they process through intake can slow  investigations.
section_title: DODIG Has Implemented a Process to Address Incomplete Data That Prevented It from Accurately Determining the Total Length of NAFI Reprisal Cases
paragraphs: As a result of our review, DODIG implemented a process to ensure that  information is updated in its case-management system so that it can  accurately determine the total length of NAFI civilian reprisal cases. Per  DOD policy, the Director of Administration and Management is the final  adjudicative authority for NAFI civilian reprisal cases within DOD.  Accordingly, upon completion of a NAFI investigation, DODIG forwards a  copy of its report of investigation to the Director of Administration and  Management for final adjudication. Once the case has been adjudicated,  the Director of Administration and Management notifies the complainant  of the investigation outcome and informs DODIG that the case is closed  by sending to DODIG a signed copy of the closure letter. DODIG’s case- management system user guide requires that these closure letters be  uploaded into the case-management system and that corresponding  changes be made to the case status in the case-management system in  order to accurately track the status and timeliness of these  investigations.
During the course of our case-file review, we reviewed case-closure  information for each of the 11 NAFI investigations closed in fiscal year  2015 and identified discrepancies between the closure documentation  sent to DODIG and the data in DODIG’s case-management system.  These discrepancies can result in an inaccurate calculation of total case  length for closed NAFI investigations. In working with DODIG officials to  resolve these discrepancies, we asked that they provide us information  on all 33 NAFI civilian investigations closed in fiscal years 2013 and  2014—resulting in a total of 44 of the 103 civilian and contractor  investigations closed over the 3 fiscal years. Our analysis of this  information showed that in 35 of the 44 NAFI investigations closed by  DODIG from fiscal year 2013 through fiscal year 2015, DODIG had not  updated data in its case-management system to reflect the receipt of  case-closure documentation from the Director of Administration and  Management indicating final case closure. As a result, DODIG was able  to monitor its own timeliness in completing these investigations, but could  not accurately determine the full length of NAFI reprisal cases as defined  by the time a complainant files an allegation until the case is finally  adjudicated by the Director of Administration and Management and the  complainant is informed of the outcome.
DODIG had not consistently updated its case-management system to  accurately capture and track the length of closed NAFI civilian  investigations because it did not have a process in place to ensure that  these data are regularly updated in accordance with its guidance. CIGIE  standards state that an investigative organization’s database should  reflect accurate processing of information and final case information,  while Standards for Internal Control in the Federal Government similarly  state that management should obtain high-quality information based on  relevant and reliable data. In addition, DODIG’s AI Investigations  Manual states that, to comply with CIGIE standards, it is critical that  investigators ensure that data fields are complete and accurate, including  date fields such as case closure. During the course of our review,  DODIG officials corrected the discrepancies we found and, in April 2017,  provided us with documentation of a process they implemented to ensure  that such updates are made in the future. Specifically, DODIG now  requests that the Director of Administration and Management send the  closure documentation it provides to complainants to a specific DODIG e- mail address. DODIG has assigned its investigative support specialists  the task of retrieving NAFI closure documentation sent to this e-mail  address and making corresponding updates to the case-management  system. In addition, DODIG has developed a dashboard view in its case- management system that will allow the investigative support specialists to  monitor the status of NAFI cases that are missing closure documentation  from the Director of Administration and Management. By establishing a  process for consistently updating its case-management system upon the  receipt of closure documentation from the Director of Administration and  Management, DODIG will be able to more accurately assess and report  on the timeliness of NAFI investigations, as well as the effect of the length  of these investigations, which is of importance to NAFI employees.
section_title: DODIG Has Taken and Has Planned Steps to Improve Timeliness but Has Not Collected Key Workload Data That Would Enable It to More Fully Assess Personnel Requirements
paragraphs: DODIG has taken and has planned steps to improve the timeliness of its  DOD civilian and contractor employee whistleblower reprisal  investigations, but it has not collected key workload data that would  enable it to more fully assess its personnel requirements. DODIG steps  taken have included the following:  Increasing its staff, as stated previously, from 28 in fiscal year 2010 to  54 in fiscal year 2016. As of April 2017, the whistleblower reprisal unit  had 56 authorized civilian positions.
Streamlining its oversight review process by reviewing only the report  of an investigation instead of the report and its supporting  documentation, unless the report of an investigation raises concerns.  Specifically, DODIG officials stated that instead of reviewing all of the  evidence in every case, DODIG now looks at the evidence if the  report appears to be inconsistent, the report’s conclusions are not  supported by the facts presented, or the report appears deficient in  another respect. DODIG officials stated that the goal for completing  this process has been reduced from 70 days or less to an average of  10 days.
Modifying its case-management system in April 2016 by (1) launching  an automated tool to help ensure compliance with statutory  notification requirements for military and contractor reprisal cases in  response to our 2015 recommendation that it implement such a tool,  and (2) establishing new milestones to better track the timeliness of  investigative phases.
Developing templates for documentation such as routine  correspondence and reports of investigations to standardize report  elements and writing formats.
Prioritizing investigations involving allegations of reprisal by senior  officials in response to direction from the Office of the Secretary of  Defense. DODIG is also working to lower the average case length for  these cases to 120 days by 2017, according to DODIG officials.
In addition to the steps that DODIG has taken, DODIG has also planned  steps to further improve the timeliness of its investigations and oversight  reviews. Chief among these is DODIG’s plan to further increase its staff  by 29 personnel. According to DODIG officials, this increase was  submitted as part of DODIG’s fiscal year 2018 budget request in order to  address its “burgeoning” caseload. Specifically, these officials stated that  the additional staff will help improve timeliness by creating two new  investigative teams to backfill the team that was retasked in July 2016 to  handle all DOD employee reprisal complaints involving sexual assault,  and to reduce the caseload carried by investigators. The additional  personnel will also allow DODIG to add another oversight team and to  create an alternative dispute-resolution program to mediate complaints,  which could improve both the timeliness and outcome of reprisal  complaints received by DODIG, according to DODIG officials.
In 2016, DODIG conducted an assessment of its personnel requirements  in support of its fiscal year 2018 budget request for additional personnel,  concluding, among other things, that the additional staff requested would  enable DODIG to further improve timeliness and meet statutory and  internal timeliness requirements. Specifically, DODIG’s assessment  analyzed the average number of intakes, investigations, and oversight  reviews per investigator, along with growth in complaints received and  closed DOD-wide and the trend in DODIG incoming complaints, intakes,  investigations, oversight reviews, and closed cases from fiscal years 2013  through 2015. In this analysis, DODIG found an increase in the volume of  complaints and projected that the number of incoming complaints would  continue to rise, thus affecting overall workload—a point borne out by  additional data for fiscal year 2016 and the first half of fiscal year 2017  provided to us by DODIG officials. Specifically, DODIG projected that the  number of incoming complaints would increase from 1,595 in fiscal year  2016 to 1,802 in fiscal year 2017 (13 percent) and the number of cases  closed would increase from 1,495 to 1,854 (24 percent), consisting of an  increase in cases closed at intake from 697 to 886 (27 percent) and an  increase in closed oversight reviews, from 755 to 930 (23 percent). The  number of closed investigations was projected to decrease from 43 in  fiscal year 2016 to 38 in fiscal year 2017 (12 percent), which DODIG  officials attributed to the steady increase in complaints received and  intakes conducted. DODIG officials stated that the increasing caseload  both affects timeliness and constitutes the basis for the fiscal year 2018  budget request for additional staff. In addition, these officials told us that  they have taken action to continually monitor personnel requirements in  response to increasing caseload and make personnel adjustments. For  instance, DODIG officials told us that, based on data showing an increase  in oversight reviews of the military service IGs, they created a dedicated  oversight team in fiscal year 2015, and later added a second team.
Although DODIG has assessed its personnel requirements and requested  additional staff to, in part, improve timeliness, it has not collected  workload data that could help it identify factors affecting timeliness and  enable it to more fully assess its personnel requirements. CIGIE  standards for investigations state that an IG’s management information  system should collect workload data, including the amount of direct and  indirect labor hours expended on each case, where appropriate. Further,  DOD guidance states that personnel requirements should be driven by  workload and established at the minimum levels necessary to accomplish  mission and performance objectives. However, the data DODIG relied  on in its budget analysis, such as the number of incoming complaints,  closed cases, and intake reviews, may not fully indicate workload levels  or where DODIG is focusing its resources because DODIG does not  collect related data on the number of direct and indirect labor hours  associated with different types of cases and investigative activities. For  example, many of the complaints DODIG receives and ultimately closes  are either military or appropriated-fund civilian cases that may be less  resource-intensive than other cases because most are declined or  referred to another organization for investigation. From fiscal year 2013  through fiscal year 2015, DODIG received 666 complaints from  appropriated-fund civilians but also declined 622 appropriated-fund  civilian cases. Also, our analysis shows differences in case length and  timeliness improvements for civilian and contactor investigations by  covered employee group, suggesting that resources may be expended  differently across different case types. Relatedly, the investigators with  whom we spoke stated that case complexity can be driven by statutory  requirements, with several noting that contractor cases present unique  challenges that can affect timeliness. Finally, process changes may also  drive the resources associated with different activities and therefore affect  personnel needs. As an example, officials stated that the streamlined  oversight review process was intended to support improved timeliness,  and, according to its data, DODIG reduced the average length of  oversight reviews from an average of 16 days in the first quarter of fiscal  year 2017 to an average of 7 days in the second quarter despite an  increase in the number of reviews closed.
DODIG officials stated that collecting more detailed workload data, such  as by tracking labor hours, would be burdensome on its investigators.  However, there are several ways DODIG could track its labor hours  through existing systems and processes—such as through its time and  attendance system. One of the supervisory investigators with whom we  spoke stated that he already requires investigators on his team to track  their hours for each case. Additionally, DODIG’s current budget request  only includes notional allocations for its requested personnel that DODIG  officials acknowledged may need to be adjusted. By assessing the  feasibility of collecting more detailed workload data, such as the labor  hours associated with each case, and including such data in future  personnel requirements assessments, DODIG would be better positioned  to evaluate the effectiveness of process changes it has implemented,  assess its personnel requirements, and allocate personnel in the most  efficient manner in order to accomplish its mission.
section_title: DODIG Does Not Regularly Report to Congress on the Timeliness of Civilian and Contractor Whistleblower Reprisal Cases
paragraphs: DODIG does not regularly report to Congress on the timeliness of civilian  and contractor reprisal cases. Under the Inspector General Act of 1978,  as amended, DODIG is an independent and objective component within  DOD and subject to oversight by Congress. Under the act, DODIG is  required to keep Congress fully informed through, among other things, its  semiannual reports to Congress. Similarly, Standards for Internal  Control in the Federal Government state that it is important for oversight  bodies, such as Congress, to have information to carry out their  responsibilities.
In prior work, we have also found that DODIG was not reporting to  Congress on the timeliness of its military whistleblower reprisal cases,  and in 2012 we recommended that DOD regularly do so, to include  information on those military reprisal cases exceeding the 180-day time  frame. DOD concurred and, in October 2016 began to report to  Congress semiannually on the timeliness of its military whistleblower  reprisal cases, including the number of military reprisal cases exceeding  180 days. DODIG did not include information in these reports on the  timeliness of civilian and contractor cases, including the number of  contactor and subcontractor cases exceeding the 180-day statutory time  frame. According to DODIG officials, this information was not included  because they were being responsive to our 2012 recommendation.
We believe that while DODIG’s actions were responsive to our 2012  recommendation, reporting on the timeliness of civilian and contractor  whistleblower reprisal cases is also important. This conclusion is because  DODIG data show that while military reprisal cases constitute the majority  of those that DODIG oversees, civilian and contractor cases are the  majority of cases that DODIG investigates. For instance, DODIG  reported to Congress that, in fiscal years 2015 and 2016, about 72  percent of its closed whistleblower reprisal investigations were civilian  and contractor cases. Without regularly reporting information on the  timeliness of civilian and contractor reprisal cases, Congress will not have  complete visibility over the timeliness of a substantial portion of DODIG’s  whistleblower reprisal investigations, including DODIG’s performance in  meeting the 180-day statutory goal for contractor and subcontractor  reprisal cases.
section_title: DODIG Has Several Processes to Help Ensure the Independence and Thoroughness of Reprisal Cases, but Some Weaknesses Exist
paragraphs: DODIG has established several processes to help ensure the  independence and thoroughness of the civilian and contractor reprisal  cases it handles, including a training program, a staff recusal process to  help safeguard independence, and an internal controls process to help  ensure the accuracy of case-file information. However, a lack of  documentation on recusals and conflicts of interest may limit its ability to  fully evaluate threats to its independence, and its practice of declining  cases is not fully consistent with its complaint-intake process. Further, we  found that some key documentation needed to demonstrate compliance  with DODIG investigative and quality-assurance processes was missing  from its fiscal year 2015 case files.
subsections: section_title: DODIG Has Implemented a Training Program to Help Ensure the Independence and Thoroughness of Its Investigations
paragraphs: DODIG has implemented a training program that provides instruction to  its investigators on CIGIE standards and professional competencies.  CIGIE general standards include independence and due professional  care, which includes thoroughness, among other things. In addition, these  standards require that organizations ensure that investigators possess  the necessary knowledge, skills, and abilities through entry-level training  and periodic training on topics that affect operations, among other things.
DODIG hired a training manager in 2012, in response to a  recommendation made during a peer audit by the Department of Justice  in 2009. According to the training manager, the training program aims to  develop core competencies needed by investigators—such as  interviewing techniques and writing—to enhance professional  development and to support organizational goals related to standardized  processes and efficient operations. To achieve these goals, DODIG  delivers training through internally developed courses, courses available  from external vendors, semiannual internally organized symposia, all- hands meetings, and job tools. For example:
Within 3 months of hire, all whistleblower reprisal unit investigators  are to be assigned to an internally developed entry-level whistleblower  reprisal investigations course. This course covers CIGIE standards,  investigative skills and procedures, and relevant whistleblower  statutes.
External courses cover a variety of topics, including, among other  things, writing effectively and advanced interviewing techniques.
Internal all-hands meetings and training are typically used to convey  recent changes to policy, process, technology, and legislation,  according to DODIG officials. For example, DODIG conducted an all- hands training session in May 2016 covering reprisal complaints  related to sexual assault; has provided training on its case- management system to coincide with new system releases; and  provided training when changes were made to section 2409 of Title 10  of the United States Code to include protections for DOD  subcontractors, among other things.
DODIG’s job tools include the AI Investigations Manual, a user guide  for its case-management system, and quick-reference guides that  contain, among other things, requirements for managing case  information and statutory guidance.
According to DODIG officials, investigators are required to take one  formal professional development course per year, but, depending on  fiscal year budget constraints, each will attend at least one formal course,  with most attending four or five. Courses are reflected in investigators’  individual development plans, which are developed annually and  approved by each investigator’s supervisory investigator. As part of the  individual development plan process, investigators choose courses based  on guidance provided by the training manager—including a career-paths  training list covering required and optional courses—as well as input from  their supervisory investigators. Course completion is tracked by the  training manager, who stated that this process helps to ensure that  investigators continually build on needed competencies.
section_title: DODIG Has Established Processes for Safeguarding Independence, but a Lack of Documentation May Limit Its Ability to Fully Evaluate Threats to Independence
paragraphs: DODIG has established processes for safeguarding independence and  objectivity, but a lack of documentation on recusals and conflicts of  interest may limit its ability to fully evaluate threats to its independence,  which our work indicates may be present. CIGIE standards state that in  all matters relating to investigative work, the investigative organization  must be free, both in fact and appearance, from impairments to  independence and maintain an independent attitude. Impairments to  independence include preconceived opinions of individuals or groups that  could bias the investigation; professional or personal relationships that  might weaken the investigative work; financial interest in an individual,  entity, or program being investigated; and biases, including those induced  by political or social convictions. Further, CIGIE standards state that IGs  should identify and evaluate threats to independence both individually  and in the aggregate because threats can have a cumulative effect on an  IG employee’s independence, and that IGs should have policies and  procedures in place to resolve, report, and document impairments to  independence when they exist. Similarly, DODIG guidance states that  employees are responsible for informing their supervisors in writing of any  potential impairment to their independence or objectivity and that, at  minimum, applicable documentation should be retained with the project  files.
DODIG has established processes to safeguard independence, including  a quality-assurance process for reviewing final reports of investigation  and a recusal process whereby investigators with a real or perceived  potential conflict of interest can be removed from a case. Specifically:
DODIG’s quality assurance review process is designed to meet the  requirements set forth by DODIG Instruction 7600.1, Quality  Standards for Department of Defense Office of Inspector General  Reports. Specifically, this process includes the following:  1.  Peer review of the report to identify where facts are missing or  where facts presented do not logically flow to conclusions.  2.  Supervisor review of the report and supporting evidence.  3.  Director or deputy director review of the report.  4.  Editor review of the report to check for compliance with internal  and external writing guidelines.  5.  Review of the report by a quality-assurance program analyst— who is organizationally independent of the investigations unit—to  ensure compliance with CIGIE standards for accuracy,  documentation, and clarity.  6.  Deputy Inspector General for Administrative Investigations review  of the report and accompanying correspondence.  7.  Office of General Counsel review of the report for legal sufficiency.
DODIG’s AI Investigations Manual notes that, as part of this process,  investigators have a responsibility to be impartial, remain objective,  and be receptive to evidence that is exculpatory as well as  incriminating. Further, the manual states that investigators should not  allow conjecture, unsubstantiated opinion, bias, or personal  observations or conclusions to affect their work. DODIG officials cited  the quality-assurance process as a key step in ensuring objectivity in  whistleblower reprisal investigations.
DODIG’s recusal process is designed to help ensure independence  by requiring that investigators recuse themselves from cases in which  they may have a real or perceived conflict of interest in the outcome of  an investigation, such as personal financial interests or relationships  with the subject or complainant. Specifically, DODIG’s AI  Investigations Manual states that if at any point investigators believe  that they cannot be impartial in a particular case, or the matter raises  the appearance of a conflict of interest, the investigator is to notify his  or her supervisor immediately. DODIG officials stated that when  such an instance arises, the case will be reassigned to another  investigator and that recusals typically occur when a case is first  assigned. These officials noted also that the recusal policy extends to  management of the whistleblower reprisal investigations unit as well.
However, we found that DODIG may be limited in its ability to fully  evaluate threats to its independence because it does not have a process  for documenting recusals and associated conflicts of interest. DODIG  officials stated that they generally do not document or track the number  recusals or associated conflicts of interest because cases are assigned to  another investigator when such instances occur. Further, these officials  stated that the robustness of the peer-review process helps ensure  independence and objectivity and that DODIG’s status as a statutory IG  creates a climate favorable to conducting independent and objective  investigations. However, without a process to document recusals and  conflicts, DODIG may lack institutional awareness of potential  impairments to its independence, including bias, and may be unable to  evaluate such threats both individually and in the aggregate. For  example, our interviews with the 24 investigators and four supervisory  investigators from investigative teams indicate the presence of potential  threats to the independence of DODIG staff. Specifically, 8 of the 28  investigators and supervisory investigators we interviewed (approximately  29 percent) reported observing acts that they perceived to demonstrate  bias on the part of one or more whistleblower reprisal unit staff or  management. Investigators stated examples of perceived bias that, if  true, indicate a climate that may not be consistently favorable to  independent and objective investigations. In response, DODIG officials  stated that when issues involving potential bias are brought to their  attention, they take appropriate action, and they provided us with an  example of an incident in which an investigator was issued a letter of  counseling for interacting with a complainant in a manner that could be  perceived as biased. DODIG officials also stated—and we agree—that  the DODIG Office of Professional Responsibility handles the type of  allegations reported to us; that DODIG’s required entry-level  whistleblower reprisal investigations training course addresses the CIGIE standard on independence, including different types of impairments; and  that additional DODIG guidance covering the topic of independence is  available to investigators.
CIGIE’s Quality Standards for Federal Offices of Inspector General state  that IGs and IG staff have a responsibility to maintain independence so  that opinions, conclusions, and recommendations will be impartial and be  viewed as impartial by knowledgeable third parties. Without documenting  recusals and impairments to employee independence and evaluating  threats to independence both individually and in the aggregate, DODIG  may be unable to fully identify threats to its independence. Further, a full  evaluation of potential threats to independence could decrease the  potential for bias—thereby increasing assurance that civilians and  contractors receive the whistleblower protections afforded by law—and  help to ensure that investigations are independent in both fact and  appearance.
section_title: When Declining Cases for Review, DODIG Does Not Always Follow Its Intake Process
paragraphs: DODIG has a complaint intake process to ensure due process and  management oversight when determining whether complaints alleging  reprisal provide sufficient evidence to warrant an investigation, but in  practice it has declined some nondiscretionary cases—that is, those  cases for which it has primary investigative authority—without completing  the intake process. Also, about 25 percent of the declined  nondiscretionary cases that we reviewed were declined for reasons  inconsistent with DODIG guidance and the reasons DODIG officials told  us may warrant a case being declined. Moreover, this guidance is not  consistent with DODIG policy on performing the complaint intake process.  As previously described in this report, the purpose of DODIG’s complaint  intake process is to determine whether the alleged facts of a case, if  proven, would raise an inference of reprisal and therefore warrant  investigation. According to DODIG’s AI Investigations Manual, this  process consists of five steps, including an interview of the complainant to  clarify the complaint and analysis of the alleged facts against the  elements of reprisal. The manual also states that contacting the  complainant and conducting the intake interview are important steps  toward obtaining a thorough understanding of the allegation in order to  determine whether the elements of reprisal are present, and that a  complainant’s assertions should be viewed in the light most favorable to  the complainant.
DODIG officials, supervisory investigators, and investigators we spoke to  also emphasized the importance of conducting an intake interview, in part  because complainants are oftentimes not familiar with the requirements  for demonstrating whistleblower reprisal, making it difficult to discern  whether a complaint involves reprisal. For example, one supervisory  investigator stated that it is difficult to determine whether a complaint  involves reprisal without speaking to the complainant because it is difficult  to provide all of the necessary information in the written complaint. Also,  an investigator told us that the investigator asks about the required  elements and read definitions to complainants, because only about half of  complainants are knowledgeable about the elements of reprisal.  According to DODIG’s AI Investigations Manual, all decisions to  investigate or not investigate complaints at the conclusion of the intake  process require approval from the director or deputy director of the  whistleblower reprisal investigations unit.
Under limited circumstances, DODIG may decline a case—that is, not  accept it—without completing the intake process and obtaining  management approval. According to DODIG officials, declined cases  usually involve complaints filed by DOD appropriated-fund civilians, which  are typically referred to the Office of Special Counsel—which has primary  jurisdiction over these cases—unless they involve issues of particular  interest to DODIG. DODIG officials also told us that DODIG may  sometimes decline nondiscretionary cases involving DOD NAFI civilians,  DOD contractors and subcontractors, and DCIPS employees if the case  clearly does not involve reprisal, the complaint is anonymous, or the  complaint is a duplicate. Unlike cases routed through the intake  process, such decisions can be made by supervisory investigators  without approval from the director or deputy director from the  whistleblower reprisal investigations unit. In his July 2016 letter to  Congress, the Acting DOD Inspector General stated that for statutes for  which DODIG has primary or sole jurisdiction (i.e., nondiscretionary  cases), DODIG’s long-established intake process normally requires that  complainants be interviewed to clarify each complaint before a  determination is made to investigate or dismiss a case, but that DODIG  may decline to take action on a complaint without conducting an intake  interview in cases involving appropriated-fund civilians.
Our analysis of DODIG data shows that DODIG declined 63  nondiscretionary reprisal cases from fiscal years 2013 through 2015  without completing the intake process. We found that about 25  percent—16 of 63—of these cases were declined for reasons that were  inconsistent with those described above as being appropriate for  declination without completing the intake process. Specifically, in 15 of  the declined nondiscretionary cases across these 3 fiscal years, the  documented reason for declining the case was the perceived lack of a  reprisal element, such as a protected disclosure, and in 1 case there were  no notes describing why it had been declined. In response to our  analysis, DODIG officials stated that it is sometimes necessary to decline  cases when they are frivolous or clearly do not involve reprisal and,  separately, that all but two of the declined nondiscretionary cases they  examined from fiscal years 2013 through 2015 either adhered to prior  policy, procedure, or practice, or were miscoded. Specifically, to support  their rationale in most cases, DODIG officials cited criteria from the 2016  Data Entry Guide, which states that cases may be declined when the  complaint makes no explicit or implicit reference to reprisal or military  restriction. However, DODIG’s Data Entry Guide did not contain such  criteria until July 2016, nearly a year after the most recent cases we  reviewed were closed. Moreover, 15 of the 16 cases we identified were  declined due to the absence of an element of reprisal—such as a  protected disclosure—which is not a stated reason for declining cases in  DODIG’s 2016 Data Entry Guide, does not align with DODIG’s cited  criterion of no explicit or implicit reference to reprisal, and is something  that DODIG officials, investigators, and supervisory investigators told us  can be difficult to ascertain without conducting an intake clarification  interview. Finally, DODIG officials reviewed the declined nondiscretionary  cases we identified from fiscal years 2013 through 2015 and concluded  that two of the cases should have gone through intake, and stated that  another case was later opened as an investigation after additional  information was provided by the complainant. This event indicates that  conducting the full intake process—including interviewing the  complainant—might have uncovered relevant information for this case  that did not appear in the written complaint.
Federal internal control standards state that management should  implement control activities through policies that should be documented in  the appropriate level of detail to effectively monitor the activity and  communicated such that personnel can implement the activities for their  assigned responsibilities. DODIG officials stated that the criteria in  DODIG’s 2016 Data Entry Guide constitute policy for declining cases and  that they have communicated such guidance to investigative staff.  However, the criteria DODIG cited as policy—with which some of the  rationales we reviewed did not align, as discussed—are not consistent  with DODIG’s intake policy, which requires that complainants be  interviewed and that either the director or deputy director approve all  decisions to investigate or not investigate complaints. Also,  documentation provided by DODIG to demonstrate its prior  communication of guidance for declining nondiscretionary cases either  did not clearly address this issue or appeared to be provided to staff that  do not have responsibility for declining such cases. Specifically, an October 2015 e-mail to supervisory investigators addressed the  practice of declining discretionary cases (i.e., appropriated-fund  civilian cases) but not nondiscretionary cases like the ones we  identified in our analysis; an e-mail officials provided referencing an all-hands meeting that they  told us addressed declinations, among other topics, states only that  the meeting would cover questions about handling different types of  statutes; and  the slides for a training course that DODIG officials said coincided  with the release of the 2016 Data Entry Guide were addressed to  oversight investigators, not the supervisory investigators who have  responsibility for declining incoming complaints.
Finally, we also found, based on our interviews with all four supervisory  investigators that have responsibility for declining cases, that there is an  inconsistent understanding of why and when cases should be declined.  For example, three of the four supervisory investigators we interviewed  were unsure whether any policy or criteria for declining cases existed,  one stated that nondiscretionary cases should never be declined, one  stated that nondiscretionary cases may be declined when the complaint  clearly does not pertain to reprisal, and two of the four told us that they  currently decline some cases for perceived lack of a reprisal element.  One of the four supervisory investigators also stated that the practice of  declining cases arose to improve timeliness and that the act of declining  nondiscretionary cases is “fostered” by management.
Without establishing and clearly communicating a declination policy for  nondiscretionary cases in the AI Investigations Manual or other guidance,  and aligning this policy with its intake policy, DODIG may not have  reasonable assurance that decisions to decline cases consistently align  with policy, and that complainants are afforded the same due process and  fair and equal treatment as those whose cases are routed through the  intake process. Moreover, delegating final responsibility for declining  nondiscretionary cases to supervisory investigators without management  approval may also result in differential treatment, since NAFI, contractor  and subcontractor, and DCIPS employees do not have the same  recourse to the Office of Special Counsel as do DOD appropriated-fund  civilians, whose cases may also be declined.
section_title: DODIG Has Established Processes to Help Ensure the Thoroughness of Investigations, but Its Case Files Were Missing Some Key Documentation
paragraphs: DODIG has established investigative, quality-assurance, and internal  controls processes to help ensure the thoroughness, accuracy, and  completeness of its investigations and case files. However, our review of  case files closed by DODIG in fiscal year 2015 found that some key  documentation or data needed to demonstrate compliance with these  processes was missing or was not uploaded to DODIG’s case-file system  in a timely manner. Further, our interviews with DODIG’s 24 investigators  from investigative teams showed that some steps required to help ensure  thoroughness during the investigative process are not routinely followed.
subsections: section_title: DODIG Has Established Investigative, Quality- Assurance, and Internal Controls Processes
paragraphs: DODIG has established investigative, quality-assurance, and internal  controls processes to help ensure the thoroughness, accuracy, and  completeness of its investigations. Collectively, these processes  encompass the planning and conduct of investigations, review of the  report of investigation, and checks on the completeness of case files. As  described below, each of these processes is designed to align with CIGIE  quality standards.
DODIG’s investigative process is intended to facilitate the planning and  conduct of investigations in accordance with the CIGIE quality standards  of planning and thoroughness by (1) directing investigative teams to  complete an investigative plan prior to beginning fieldwork and (2)  prescribing subsequent, required steps of the investigative process.  According to DODIG’s AI Investigations Manual, a good investigative plan  provides a roadmap for conducting focused, thorough, and efficient  investigations. The key elements of an investigative plan include the  identification of investigation subjects; witnesses to be interviewed and  questions relevant to the allegations; legal and regulatory standards;  evidence to be collected; travel; investigation milestones; and necessary  investigative steps. Investigative steps are the strategy or the steps  through which the investigation plans to proceed to complete the case  and include notifying the proper parties of the investigation; interviewing  complainants, subjects, and witnesses; and collecting documentary  evidence. DODIG’s manual includes detailed guidance for conducting  these and other steps of the process and provides investigative tools  such as a diagram that can be used to graphically depict the evidence  collected, along with its weight and preponderance. Additionally, the  manual describes quality checks and roundtables, which are discussions  between various personnel such as investigators, supervisors, attorneys,  and management of the whistleblower reprisal investigations unit.  According to DODIG’s AI Investigations Manual, these discussions should  occur before commencing fieldwork to facilitate investigative planning and  both prior to and after interviews with the subject of the investigation to  facilitate an “interactive and write-as-you-go investigative process.”
DODIG’s quality-assurance review process, as previously described in  this report, is a multistep review process to help ensure that final reports  of investigation are thorough, factually accurate, and legally sufficient.  This process is designed to address the minimum quality-assurance  standards requirements set forth by DODIG Instruction 7600.1, Quality  Standards for Department of Defense Office of Inspector General  Reports, which are based in part on CIGIE quality standards.
DODIG’s internal controls process is designed to test compliance with  CIGIE professional standards related to the thoroughness of  investigations; the adequacy of case-file documentation; and the  currency, accuracy, and completeness of data maintained in DODIG’s  case-management system. According to DODIG, this process was  developed in response to our prior recommendation in 2012 to develop  and implement performance metrics to ensure the quality and  effectiveness of the investigative process, such as by ensuring that case  files contain evidence sufficient to support the conclusions. Specifically,  the internal controls process requires that investigators complete a  criteria-based checklist at the conclusion of an investigation to ensure that  certain documentation and data are present in the case-management  system. For example, this checklist includes items to check case files for  the presence of letters of notification to parties involved in an  investigation, an approved investigative plan, interview records, evidence  of legal review, and corrective-action documentation. In addition, the  checklist includes items to ensure that certain data fields have been  populated in the case-management system, such as information about  complainants and subjects; alleged protected disclosures and personnel  actions; and dates associated with investigative steps. According to  DODIG’s internal controls process description, these checklists were  completed on a quarterly basis prior to fiscal year 2016, at which point  they began to be completed at the close of each case.
section_title: DODIG’s Case files Were Missing Some Key Documentation
paragraphs: Our review of case files closed by DODIG in fiscal year 2015 found that  some documentation was consistently present, but that some other key  documentation or data were missing or were not uploaded to DODIG’s  case-file system in a timely manner. This documentation and these data  are needed to demonstrate compliance with or execution of investigative,  quality-assurance, and internal controls processes, including the internal  controls checklist—which we found did not include checks for some key  documentation and required investigative events. Further, our interviews  with the 24 investigators from investigative teams showed that some  steps required to help ensure thoroughness during the investigative  process are not routinely followed. The version of DODIG’s investigations  manual in place during the period covered by our case-file review states  that case files should be complete from case initiation until case closure  and that upon case closure, investigators will organize the final records in  the file as soon as possible to ensure readiness for Freedom of  Information Act or other requests for investigation documents. The  manual further states that closed case files should be ready to withstand  scrutiny by an outside peer review or oversight authority. Similarly, CIGIE  standards state that the degree to which an organization efficiently  achieves its goals is affected by the quality and relevance of information  that is collected, stored, retrieved, and analyzed, and that the results of  investigative activities should be accurately and completely documented  in the case file.
Examples of documents consistently present—Our review of 178 case  files for cases closed in fiscal year 2015 found that some documentation  was consistently present. For example:
100 percent of the cases that were fully investigated included  evidence of a sworn recorded testimony with the complainant.
100 percent of the cases that were fully investigated had a final report  of investigation.
We estimate that about 98 percent of all cases included the incoming  complaint.
We estimate that about 98 percent of the cases that were dismissed  had a closure letter to the complainant.
Examples of documents or data that were missing or not uploaded— Conversely, our review of case files for cases closed in fiscal year 2015  found that some other key documentation or data that are needed to  demonstrate compliance with or execution of the investigative, quality- assurance, or internal controls processes were either missing or were not  uploaded to DODIG’s case-file system in a timely manner. For example:
Approximately 13 percent of the cases that were fully investigated did  not have documentation of legal sufficiency. In response to our  review, DODIG located documentation of legal sufficiency for these  cases and uploaded it to the relevant case files.
Approximately 13 percent of the cases that were fully investigated did  not include an intake worksheet.
Approximately 19 percent of the cases that were fully investigated  were missing the notice of investigation sent to the complainant.
DODIG inconsistently populated mandatory investigative event  milestone dates related to management approval of the report of  investigation. For example, of the cases that were fully investigated,  approximately 36 percent were missing the date that the staff package  was sent to the Director or Deputy IG for Administrative Investigations  and approximately 13 percent were missing the date that the Deputy  IG for Administrative Investigations approved the report. The absence  of these dates does not necessarily indicate that the actions did not  occur; rather, it hinders DODIG’s ability to analyze the timeliness of  specific investigative phases, such as management review, as DODIG  officials stated they intend to do in order to further improve  investigation timeliness. In regard to the missing Deputy IG for  Administrative Investigations approvals, DODIG officials stated that  the missing dates were data-entry errors and that no case moves  beyond this step in the approval process without the Deputy IG’s  approval or her designee’s approval. DODIG also provided evidence  to show that either the Director or Deputy IG for Administrative  Investigations did approve the reports.
Approximately 48 percent of the cases that were fully investigated  included documents that were modified or were newly uploaded at  least 30 days after case closure. These changes related to documents  such as the final report of investigation, closure letters, evidence of  legal review, the investigative plan, congressional correspondence,  and interview transcripts. On average, the most recent change  occurred 228 days (median 148 days) after case closure. In  response to this analysis, DODIG officials stated that we had selected  an arbitrary measure of 30 days after case closure to determine  whether document uploads were appropriate. We applied the 30-day  threshold as a grace period, recognizing that it may not always be  practical for case files to be complete from case initiation to closure,  as DODIG’s then-guidance required. DODIG officials also stated that  because document uploads were performed in the majority of cases in  accordance with the DODIG internal controls checklist process or  related to follow-up activities, our inclusion of these cases implies that  it is inappropriate to have internal controls in place to assure that key  documentation is uploaded unless those reviews can be performed in  less than 30 days after case closure. Further, DODIG officials stated  that while it would be ideal if documentation was captured  contemporaneously, internal guidance states that documentation gaps  should be corrected when found, such as when found through the  internal controls process. However, as previously stated, DODIG’s  internal policy required that case files be complete upon case closure  and the most recent change occurred on average 228 days (median  148 days) after case closure, far exceeding the internal policy and the  prior practice of completing internal controls checklists on a quarterly  basis. We believe that DODIG’s current practice of completing internal  controls checklists at case closure is a positive step towards  complying with its requirement for complete case files upon case  closure and that it should help improve the timeliness of its case-file  documentation.
In approximately 16 percent of the cases that were fully investigated,  it was not clear whether the investigative plan on file was the  approved version because it did not include the approval milestone  date in the document and did not appear to be a complete plan based  on one or more missing or incomplete key elements. For example,  one plan was missing investigative milestones after “investigation  start,” while another was missing investigative milestones, did not  specify the documentary evidence to be gathered, and did not include  witnesses to be interviewed. DODIG’s 2012 Administrative  Investigations Manual required that investigative plans be approved  prior to the start of fieldwork and it identified key elements of the  investigative plan as including documentary and other relevant  evidence to be collected, witnesses to be interviewed, and  investigation milestones, among other things. Further, DODIG’s  internal controls checklist requires that the approved investigative plan  corresponding to the plan’s approval date in the database be  uploaded to the case file. DODIG officials agreed that these plans  were either missing key elements or had incomplete investigative  milestones, but also stated that each plan was the approved plan as  evidenced by the date of the document corresponding to the approval  date in the database. Of the plans we identified as incomplete, we  noted that the date of one document matched the approval date in the  database, while the dates of the other documents preceded the  approval date in the database by between 3 business days and over a  month—making it further unclear as to whether these incomplete  plans were the approved plans. DODIG officials also stated that initial  investigative plans sometimes contain less than the full complement  of witnesses and documentary evidence. However, the plans we  identified as being incomplete in this regard did not contain any  documentary evidence and one did not contain any witnesses. In  addition, all of the plans we identified with missing or incomplete key  elements either did not include investigative milestones or had  incomplete investigative milestones.
Two out of three fully investigated cases with congressional inquiries  were missing correspondence to the Member of Congress providing a  summary of the findings, as required by DODIG’s 2012 investigations  manual. In response to this finding, DODIG officials stated in regard to  one of the cases that the draft summary finding letter in the case file  was sufficient to address documentation requirements because  DODIG’s Office of Legislative Affairs and Communications is  responsible for facilitating signatures of, transmitting, and maintaining  records of congressional responses. However DODIG’s 2012  investigations manual states that acknowledgment, interim, and final  letters to members of Congress should be saved to the case file, and  its internal controls checklist includes items to verify the presence of  this correspondence. Because the letter saved to the case file was not  the letter sent by DODIG’s Office of Legislative Affairs and  Communications, it is unclear whether it was the final  correspondence.
Approximately 81 percent of the 413 testimony transcripts and voice  recordings we reviewed did not address one or more points of the  standard read-in/read-out process. This process consists of a series  of questions and statements at the beginning and end of each  interview intended to ensure that all witnesses are treated equally and  that they are afforded the proper notifications of authorities and due  process. In response to this finding, DODIG officials stated that it was  “clearly permissible” to not cover the entire read-in/read-out in cases  where a witness had already been interviewed and provided with the  standard read-in/read-out information. During the course of our case- file review, we noted some cases where complainants, subjects, or  witnesses were interviewed more than once. However, DODIG did not  identify the interviews where it determined that the full read-in/read- out did not need to be covered or provide us with policy or other  guidance to support its statement that read-in/read-out language  could be abbreviated.
In addition to missing case-file documentation or case data, our  interviews with each of the 24 DODIG investigators from the investigative  teams showed staff are not routinely holding roundtable meetings, which  are discussions between investigators, supervisors, attorneys, and  management of the whistleblower reprisal investigations unit, as  previously mentioned. Specifically, for the roundtables required by  DODIG’s AI Investigations Manual,
18 of the 24 investigators we interviewed stated that investigative  planning roundtables occur infrequently or not at all,
16 of the 24 investigators stated that the pre-subject-interview  roundtables occur infrequently or not at all, and
19 of the 24 investigators stated that post-subject-interview  roundtables occur infrequently or not at all.
DODIG officials stated that roundtables are an important part of the  review process in that they help to keep cases on track and identify  necessary adjustments.
Federal internal control standards state that management should monitor  its internal control system, evaluate the results, and remediate internal  control deficiencies on a timely basis. Further, these standards state  that a control cannot be effectively implemented if it was not effectively  designed and that an existing control is not properly designed if the  control objective would not be met when the control operates as  designed. However, we noted, and DODIG officials acknowledged, that  DODIG’s internal controls checklist does not capture the full range of key  case-file documentation and required investigative events, including  documentation of (1) an intake worksheet, (2) required roundtable  discussions, and (3) the program analyst for quality assurance review.  Without a checklist that captures the full range of key case-file  documentation and data associated with required investigative steps,  DODIG will be limited in its ability to ensure compliance with CIGIE  standards related to thoroughness and adequacy of case-file  documentation, as well as the currency, accuracy, and completeness of  data maintained in its case-management system.
section_title: DODIG Has Conducted Oversight of DCIPS Employee Cases Provided by Some of the Defense Intelligence Components but Has Not Fully Addressed Oversight Requirements
paragraphs: DODIG has conducted oversight of investigations involving DCIPS  employees that were conducted by the some of the defense intelligence  component IGs, which consist of the IGs of the Defense Intelligence  Agency, National Geospatial-Intelligence Agency, National  Reconnaissance Office, and National Security Agency. However,  DODIG and the defense intelligence components have not fully  addressed requirements related to DODIG’s review of all DCIPS  employee allegations, determinations, and investigations handled by the  component IGs. DOD Directive-Type Memorandum 13-008—which  implements Presidential Policy Directive 19 (PPD-19)—requires that  DODIG provide oversight of reprisal allegations involving DCIPS  employees that are handled by the defense intelligence component IGs,  and that the component IGs correspondingly furnish information to  DODIG. Table 3 highlights key oversight requirements specified by  Directive-Type Memorandum 13-008 for DODIG and the defense  intelligence component IGs.
DODIG’s AI Investigations Manual implements DODIG’s oversight  requirements in accordance with Directive-Type Memorandum 13-008 by  describing a process for DODIG to review (1) defense intelligence  component IG determinations to not investigate allegations involving  DCIPS employees, and (2) reports of investigation for completed  investigations involving DCIPS employees submitted by the defense  intelligence component IGs. To conduct these reviews, among other  duties, DODIG has established an oversight branch comprising 17  personnel across two teams, each of which has a supervisory  investigator, according to DODIG officials. Specifically, DODIG’s  oversight team is to review determinations to not investigate an allegation  to ascertain whether the elements of a prima facie allegation were  adequately addressed, including alleged protected disclosures and  personnel actions; whether the alleged responsible management official  knew of the protected disclosures before taking, withholding, or threating  the personnel actions; and whether there is an inference of causation  between the protected disclosures and personnel actions. According to  DODIG’s AI Investigations Manual, investigations are to be reviewed for  compliance with CIGIE standards, including independence, objectivity,  thoroughness, documentation, and legal sufficiency. These reviews are  to be documented in a worksheet, and the results of the review are to be  communicated to the component IG. If the determination to not  investigate or the investigation is sufficient, an approval letter stating such  is to be sent to the component IG; if not, DODIG is to work with the  component IG to correct any deficiencies.
As of April 2017, DODIG had received and reviewed investigations from  three of the four defense intelligence component IGs as well as  notifications of allegation receipt from one of the four component IGs.  DODIG had not received determinations to not investigate allegations  from any of the four component IGs. Specifically, DODIG officials stated  that they received investigations for review from the IGs of the Defense  Intelligence Agency, the National Geospatial-Intelligence Agency, and the  National Security Agency. According to DODIG officials, DODIG had  not received investigations for review from the National Reconnaissance  Office IG. Further, these same officials stated that DODIG had received  notifications of allegation receipt from the National Geospatial-Intelligence  Agency IG only, and that DODIG had not received determinations to not  investigate an allegation from any of the four defense intelligence  component IGs.
DODIG officials provided us with three notifications from the National  Geospatial-Intelligence Agency IG that they stated constituted notification  of allegation receipt. In reviewing these notifications, we noted that each  notification was received by DODIG after the 10-working day period  prescribed by Directive-Type Memorandum 13-008 and that each  notification also informed DODIG of the National Geospatial-Intelligence  Agency IG’s intent to investigate the allegation. As previously mentioned,  DODIG has the authority to retain such cases for investigation or refer  them back to the component IG. Specifically, two notifications stated that  the National Geospatial-Intelligence Agency IG planned to investigate the  associated allegations on the basis of preliminary review. One of these  notifications was received by DODIG more than a month after the  National Geospatial-Intelligence Agency IG’s receipt of the allegation, and  the other was received by DODIG more than 2 months after initial receipt  of the allegation. The third notification—received by DODIG more than 3  months after initial receipt of the allegation—stated that the National  Geospatial-Intelligence Agency IG would continue its investigation of the  matter in the absence of notification from DODIG that DODIG intended to  assume investigative jurisdiction over the matter. DODIG officials stated  the defense intelligence component IGs are likely receiving numerous  complaints of reprisal, and the DODIG official responsible for managing  the oversight branch further stated that it is unlikely that the investigations  DODIG has received for review constitute all of the reprisal investigations  conducted by the component IGs.
The defense intelligence component IG officials we contacted provided  varying perspectives on the oversight requirements, as well as the degree  to which they currently comply with the requirements of Directive-Type  Memorandum 13-008, and intend to do so in the future. For example,  National Security Agency IG officials told us that they are in compliance  with the memorandum’s provisions because they notify DODIG whenever  they commence an investigation of a credible allegation, and because all  credible allegations have been investigated. However, these officials also  noted that they receive other reprisal allegations for which they do not  notify DODIG because the allegations are not deemed to be credible, or  because they do not “facially meet” the criteria for reprisal. As previously  mentioned, Directive-Type Memorandum 13-008 requires that defense  intelligence component IGs notify DODIG within 10 working days of  receiving any allegation of reprisal from a DCIPS employee and provide a  copy of the written complaint to DODIG. Similarly, officials from the  Defense Intelligence Agency IG stated that they notify DODIG of new  investigations and provide the resulting investigations for review, but that  they do not notify DODIG of all allegations received or provide DODIG  with determinations to not investigate allegations for DODIG’s review, as  is required. These same officials also stated that the Defense Intelligence  Agency IG works collaboratively with DODIG to respond to complaints  regarding Defense Intelligence Agency personnel that are reported to  DODIG. According to a senior DODIG official in February 2017, the  Defense Intelligence Agency IG director of whistleblower reprisal  investigations stated to DODIG officials that the Defense Intelligence  Agency IG intends to start complying with all oversight requirements.  Additionally, DODIG officials stated that DODIG provided officials from  the Defense Intelligence Agency IG with a sample template that can be  used to notify DODIG of determinations to not investigate cases.  However, Defense Intelligence Agency IG officials stated that senior  Defense Intelligence Agency IG officials met with DODIG to discuss a  way forward on the oversight requirements and came to no conclusion,  and that no Defense Intelligence Agency IG official committed to  complying with all of the oversight requirements. Additionally, a Defense  Intelligence Agency IG official told us that the memorandum is  problematic, that our inquiry regarding the Defense Intelligence Agency  IG’s compliance with the memorandum’s provisions has prompted them  to reevaluate their reporting processes, and that they are considering  reducing future reprisal reporting to DODIG.
DODIG officials stated some of the defense intelligence component IGs  objected to the memorandum’s oversight provisions during its  development, and that some of the component IGs’ status as  presidentially appointed, Senate-confirmed “statutory” IGs may influence  their views with regard to the appropriateness of DODIG oversight.  However, DODIG officials noted that they view the requirements of  Directive-Type Memorandum 13-008 as being clearly delineated and  binding throughout the department, and that while the four defense  intelligence component IGs have specific responsibilities as intelligence  organizations, they remain part of the larger DOD and are therefore  subject to department-wide duties, responsibilities, and authorities  specified in the Inspector General Act of 1978, as amended. Moreover,  they noted that DODIG remains the department’s principal IG over all  components within the department—with responsibility to ensure  coordination, cooperation, and efficiency in the audits and investigations  of the defense intelligence components—and that PPD-19, DOD  regulation, and the Inspector General Act provide the basis for DODIG to  approve determinations and investigations of DOD component IGs.
DODIG has taken some steps to coordinate with the defense intelligence  component IGs in relation to the oversight requirements. For example,  DODIG officials stated that they have conveyed their expectations with  regard to the oversight process to representatives from the National  Security Agency IG and the Defense Intelligence Agency IG, and that the  memorandum’s requirements have been addressed during a symposium  attended by representatives from defense intelligence component IGs  and during DODIG’s basic whistleblower reprisal course, which is offered  quarterly and is regularly attended by investigative personnel from the  component IGs. DODIG officials also stated that in March 2017, they  presented oversight requirements to an audience that included  representatives from the component IGs. National Security Agency IG  officials stated that oversight expectations were constructively discussed  in a November 2015 meeting that concluded with an agreement to  continue the excellent working relationship that had already been  established, and that the symposium and reprisal course—while  valuable—do not spend much time on dealing with intelligence  community cases, PPD-19, and Directive-Type Memorandum13-008.  Additionally, these same officials stated that the March 2017 oversight  requirements briefing was provided to a Defense Council on Integrity and  Efficiency Administrative Investigations Committee meeting, and that the  National Security Agency IG did not receive the invitation or  presentation. Similarly, Defense Intelligence Agency IG officials stated  that in a December 2016 meeting with DODIG, they explained their  statutory independence and objectivity, indicated the extent that any  oversight requirements interfered with the agency’s statutory  responsibilities, and stated that full compliance required additional  discussion. These officials also stated that the Defense Intelligence  Agency IG was not represented at the March 2017 oversight  requirements briefing.
However, DODIG has not established processes with the defense  intelligence component IGs—such as standard operating procedures—to  ensure that the components provide the allegation notifications,  determinations to not investigate, and investigations prescribed by  Directive-Type Memorandum 13-008. As the principal advisor for DOD  IGs, DODIG is to assist the defense IGs—including the IGs of the  defense intelligence components—by coordinating and clarifying DOD  policy, issuing implementing instructions, and resolving conflicting or  inconsistent IG policy involving defense IG duties, responsibilities, and  functions. Without a process to fully implement the requirements of  Directive-Type Memorandum 13-008 that includes (1) receiving  notification of all allegations received by the defense intelligence  component IGs, (2) reviewing their determinations to not investigate  allegations, and (3) reviewing all investigations that they conduct, DODIG  and the defense intelligence components will continue to not fully adhere  to the prescribed roles related to the oversight of component  determinations and investigations, as defined in agency directives.
section_title: DODIG’s Timeliness Performance Measures Demonstrate Key Attributes of Successful Measures, but It Lacks Measures to Assess the Quality of Investigations and Oversight Reviews
paragraphs: In April 2017, DODIG developed six performance measures to assess the  timeliness of its investigations and oversight reviews for fiscal year 2017  that demonstrate many, but not all, key attributes of successful  performance measures. However, DODIG does not have performance  measures to assess the quality of its investigations and oversight reviews  for fiscal year 2017 and beyond. Prior to fiscal year 2017, DODIG had  developed several different timeliness and quality performance measures,  but these measures were inconsistently documented and reported.
Standards for Internal Control in the Federal Government emphasize the  importance of assessing performance over time. By using performance  measures, decision makers can obtain feedback for improving both policy  and operational effectiveness. Further, we found in our previous work that  measures on both timeliness and quality—such as the completeness of  investigative reports and adequacy of internal controls—can enhance the  ability of organizations to provide reasonable assurance that they are  exercising appropriate safeguards for federal programs. Our body of  work on leading performance-management practices identified 10  attributes of successful performance measures (see table 4). While  these attributes may not represent all the attributes of successful  performance measures, we believe they address important areas.
In April 2017, more than halfway through the fiscal year, DODIG  developed and provided us with six performance measures for assessing  the timeliness of fiscal year 2017 investigations and oversight reviews.  We compared the performance measures against the 10 key attributes of  successful performance measures identified in our prior work, as shown  in figure 3.
In our analysis of DODIG’s six performance measures for timeliness of  whistleblower reprisal investigations and oversight reviews, we found the  following.
Linkage. Five of the six measures we assessed addressed the attribute  of linkage, and one measure partially addressed this attribute. A measure  demonstrates linkage when it is aligned with division and agency-wide  goals and mission and is clearly communicated throughout the  organization. We found that all six timeliness measures aligned with  DODIG’s strategic objective to provide timely and relevant feedback to  the department through high impact DODIG products that result in  improvements in program and operational performance. Additionally, we  found that all six measures aligned with DODIG’s vision to improve the  timeliness of its investigative products. According to DODIG officials, the  timeliness measures have been communicated to whistleblower reprisal  unit staff through briefings and all-hands training. However, we found that  DODIG’s measure of the “average number of days to complete an  oversight review” partially addressed the attribute of linkage because  DODIG’s target to close its oversight investigations is less than or equal  to 10 days, but its parameter for success for this measure is less than or  equal to 60 days. This inconsistency may hinder DODIG’s ability to clearly  communicate the measure.
Clarity. Five of the six measures we assessed addressed the attribute of  clarity, and one measure partially addressed this attribute. A measure  achieves clarity when it is clearly stated and its name and definition are  consistent with the methodology used to calculate it. Further, a measure  that is clearly stated should not contain extraneous information or omit  key data elements. DODIG’s timeliness measures are consistent with the  methodology for calculating them, which uses standard dates entered for  each case in its case-management system. However, the “average days  for closing oversight investigations” measure partially addressed the  attribute of clarity because its target to close oversight investigations in  less than or equal to 10 days is inconsistent with its parameter for  success, which is less than or equal to 60 days.
Measurable Target. Five of the six measures we assessed addressed  the attribute of having a measurable target, and one measure partially  addressed this attribute. A measure with a measurable target has a  numerical goal. DODIG has set a quantifiable, numerical target for each  of the six performance measures we assessed. For instance, DODIG  established a target of completing 51 percent or more of its civilian  whistleblower reprisal investigations in less than or equal to 180 days or  240 days, depending upon the investigation authority. However, the  “average days for closing oversight investigations” measure partially  addressed the attribute of measurable target because DODIG’s target to  close oversight investigations is less than or equal to 10 days, but its  parameter for success for this measure is less than or equal to 60 days.
Objectivity. All six measures we assessed addressed the attribute of  objectivity. A measure demonstrates objectivity when it is reasonably free  from significant bias or manipulation. Specifically, DODIG officials stated  that the methodology for calculating its timeliness measures is objectively  determined using standard dates entered for each case in its case- management system. We agree that this methodology does not allow for  subjective considerations or judgments, as these measures by name  indicate what is to be observed (e.g., closed investigations), in which  population or conditions, and in what time frame (e.g., quarterly).
Reliability. All six measures we assessed addressed the attribute of  reliability. A measure demonstrates reliability when it produces the same  result under similar conditions. DODIG officials stated that the same  methodology will be used to evaluate performance on a quarterly basis  across the fiscal year. In addition, DODIG has procedures to verify and  validate each measure through its internal control checks of the case- management system.
Baseline and Trend Data. All six measures we assessed partially  addressed the attribute of baseline and trend data. A measure with a  baseline and trend data associated with it identifies, monitors, and reports  changes in performance and helps ensure that performance is viewed in  context. DODIG’s measures provided data on year-to-date performance  for the first two quarters of fiscal year 2017, allowing decision makers to  identify and monitor trends in performance over time. In addition, DODIG  officials stated that they established a baseline for their prior timeliness  measures based on a finding from our prior work in 2012 and that they  have since set goals to reduce the average days to conduct investigations  by a certain percentage each year. However, DODIG did not provide  baselines for its fiscal year 2017 whistleblower reprisal timeliness  performance measures, and it has not consistently reported on all of its  past timeliness performance measures, as discussed below, thus  inhibiting baseline and trend analysis.
Core Program Activities. Collectively, the six measures we assessed  partially addressed the attribute of core program activities. Measures that  reflect core program activities cover the activities that an entity is  expected to perform to support the intent of the program. The  whistleblower reprisal investigations unit’s mission is to objectively and  thoroughly investigate whistleblower reprisal complaints and to provide  oversight of military service and component IG investigations of such  complaints. DODIG’s AI Investigations Manual specifies major activities  associated with this mission, such as conducting intake reviews to  determine whether an allegation warrants investigation, investigating  whistleblower reprisal complaints, and conducting oversight reviews of  investigations performed by the component IGs. DODIG’s fiscal year  2017 timeliness measures address program activities such as the timely  completion of investigations and oversight reviews of investigations, but  they do not assess whether other core program activities, such as intake  reviews, are completed in a timely fashion.
Limited Overlap. Collectively, the six measures we assessed addressed  the attribute of limited overlap. Measures with limited overlap provide new  information beyond that provided by other measures. DODIG’s fiscal year  2017 measures all relate to the timeliness of case-closure efforts, but the  measures address both investigations and oversight reviews, and provide  data that can supplement one another. For example, the measure of  average days to complete civilian investigations may be supplemented by  the percentage of completed civilian investigations meeting the 240-day  timeliness goal, because the latter could provide context for the average  days measure by indicating whether there may have been one or more  long or short cases that affected the average.
Balance. Collectively, the six measures we assessed did not address the  attribute of balance. Measures that demonstrate balance ensure that an  organization’s various priorities are covered. Performance-measurement  efforts that overemphasize one or two priorities at the expense of others  may skew the agency’s performance and keep its managers from  understanding the effectiveness of their programs in supporting the  overall mission and goals. Taken together, DODIG’s fiscal year 2017  timeliness measures address the whistleblower reprisal unit’s priority of  improving the timeliness of its whistleblower reprisal investigations and  oversight reviews, but they do not address the unit’s priority of conducting  thorough investigations that adhere to CIGIE standards.
Government-Wide Priorities. Collectively, the six measures we  assessed partially addressed the attribute of government-wide priorities.  Measures should cover a range of government-wide priorities, such as  quality, timeliness, efficiency, cost of service, and outcome. A range is  important because most program activities require managers to balance  these priorities among other demands, and measures that assess some  priorities but neglect others could place the program’s success at risk.  DODIG’s fiscal year 2017 timeliness measures are focused on the  agency’s priority to improve the timeliness of its reprisal investigations  and oversight reviews, and do not address the quality of investigations.  Without addressing the government-wide priority of quality, an  overemphasis on timeliness could undermine quality. In addition,  DODIG’s suite of timeliness measures does not address the efficiency or  cost of service associated with investigations and oversight reviews.
Although DODIG has not developed performance measures for fiscal  year 2017 to specifically assess the quality of its reprisal investigations  and oversight reviews, DODIG officials stated in April 2017 that they were  in the process of doing so. Prior to fiscal year 2017, DODIG had  developed several performance measures to assess both the timeliness  and quality of its reprisal investigations, but the evaluation of these  measures was inconsistently documented and reported, and some  measures changed over time. For example, DODIG variously reported to  senior DODIG management from fiscal year 2013 through fiscal year  2015 on timeliness measures such as the percentage of military and  contractor cases meeting the statutory goal of 180 days, the average and  median length of these investigations, as well as the percentage of NAFI,  appropriated-fund civilian, and PPD-19 investigations meeting the internal  240-day goal. However, according to DODIG officials, in fiscal year 2016,  DODIG reported to the Acting Inspector General on one measure alone— the percentage of oversight reviews meeting the internal goal of 70 days.  DODIG also developed three quality measures in 2014 in response to our  2012 recommendation that it develop and implement performance  measures to ensure the quality and effectiveness of the investigative  process, such as by ensuring that case files contain evidence sufficient to  support conclusions. DODIG’s quality measures specifically addressed  thoroughness, documentation, and data integrity and completeness for  DODIG investigations and oversight reviews. However, those measures  were last briefed to the DOD Inspector General in fiscal year 2014.
DODIG officials stated in February 2017 that changes in DODIG’s  leadership and internal processes have contributed to the inconsistency  in reporting on timeliness performance measures, but that they continued  to monitor aspects of timeliness and quality throughout fiscal years 2013  through 2016. Subsequently, these officials disagreed with our conclusion  that their measures had been inconsistently documented and reported,  stating that they had implemented a robust set of performance measures  for quality and timeliness, which they had monitored and documented on  a quarterly basis from fiscal years 2013 through 2016. They also stated  that they reported on fiscal year 2013–2016 average and median days for  investigations against the 180- and 240-day benchmarks and on a  sample of oversight reviews in annual assurance statements, and on  fiscal year 2013–2015 average days for investigations in annual  organizational assessments. DODIG officials also provided us with a  written narrative explaining how their prior performance measures  addressed the 10 key attributes of successful performance measures  identified in our prior work. Our analysis of the prior measures differed  from DODIG’s analysis in regard to the extent that DODIG’s prior  measures addressed the attributes, but we do not present the results of  our analysis because DODIG’s prior performance measures were not  carried forward to fiscal year 2017.
In discussing their efforts to develop performance measures, DODIG  officials noted the difficulty of developing performance measures for  investigative work, and stated that the Government Performance and  Results Act (GPRA) Modernization Act of 2010 (GPRAMA) should  provide the basis for any assessment of DODIG’s performance  measures. These officials also stated that GAO’s identified key  attributes of successful performance measures are accompanied by scant  justification or explanation as to why they were chosen, and included no  guidance as to how agencies should implement them. We believe that our  prior work has demonstrated that the attributes—which are based in part  on the earlier GPRA—are broadly applicable as leading practice for  developing successful performance measures across various types of  federal programs. Additionally, the attributes are similar to  requirements set forth in GPRAMA that DODIG officials cited as the  standard to which they should be held. For example, GPRAMA’s  requirement that agencies establish performance goals that are objective,  quantifiable, and measurable aligns with our attributes of objectivity and  measurable target. Similarly, GPRAMA’s requirement that performance  goals should describe how they contribute to the general goals and  objectives in an agency’s strategic plan aligns with our attribute of  linkage. Finally, we believe that the utility of each attribute is articulated  by the potential adverse consequences of not meeting it, while the  definition for each attribute conveys its purpose in order to guide  implementation, as shown previously in table 4. Ultimately, by developing  both quality and timeliness performance measures that fully reflect the  key attributes of successful measures, DODIG decision makers would be  better positioned to (1) assess the quality and timeliness of DODIG’s  whistleblower reprisal investigations and oversight reviews and (2)  determine whether past, ongoing, and future improvement initiatives are  on track to achieve desired outcomes.
section_title: Conclusions
paragraphs: Maintaining a timely, independent, and thorough process for investigating  whistleblower reprisal complaints is essential to executing DODIG’s  mission and to fulfilling the whistleblower reprisal unit’s vision of being the  model whistleblower-protection program in the federal government. To  these ends, DODIG has taken and has planned steps to improve the  timeliness of its investigations, and has established several processes to  help ensure their independence and thoroughness. However, without  regularly reporting on the timeliness of all civilian and contractor  investigations, decision makers’ ability to effectively oversee the  whistleblower reprisal program is limited. DODIG also does not have key  workload data that would enable it to more fully assess its personnel  requirements in support of its planned steps to improve timeliness. By  assessing the feasibility of collecting more detailed workload data, such  as the labor hours associated with its cases, DODIG would be better  positioned to identify the resource investment associated with different  types of cases and investigative activities and to then allocate existing  and future personnel in order to accomplish its mission.
Enhancements are also needed for existing processes designed to help  ensure the independence and thoroughness of DODIG’s investigations.  By documenting investigators’ recusals and conflicts of interest—and  evaluating these and other threats in the aggregate—DODIG will have  increased institutional awareness of threats such as bias, thereby better  enabling it to fully evaluate such threats. Also, by establishing and clearly  communicating a declination policy for nondiscretionary cases, DODIG  will have better assurance that these complainants are afforded the same  due process as those whose cases are routed through the intake  process. Additionally, developing an internal controls checklist that  captures the full range of key case-file documentation and data  associated with required investigative steps will help DODIG ensure  compliance with CIGIE standards related to the thoroughness and  adequacy of case-file documentation. Moreover, it will be better  positioned to withstand scrutiny by outside authorities, and address  concerns expressed by members of Congress regarding the integrity of  its investigations.
Another area in which DODIG’s process can be improved is in its  oversight of cases involving DCIPS employees that are handled by the  defense intelligence component IGs. While DODIG is reviewing  investigations conducted by some of the defense intelligence component  IGs, it has not established processes with the defense intelligence  component IGs that fully address requirements to receive notification of  all allegations involving DCIPS employees that are received by the  components, review component determinations to not investigate  allegations, and review all the investigations the components conduct.  Without a process for doing so, DODIG and the defense intelligence  components are unable to fulfill their prescribed roles related to the  oversight of component determinations and investigations. Moreover,  without fully executing its oversight responsibilities, DODIG cannot  achieve its vision of being the model whistleblower-protection program in  the federal government.
Finally, given the improvement initiatives DODIG is planning and has  under way—and the anticipated increase in cases—it is important that  there be a reliable means by which to gauge progress in the timeliness  and quality of both investigations and oversight reviews. DODIG  developed performance measures for its timeliness for fiscal year 2017,  more than halfway through the fiscal year, and these measures address  many, but not all, key attributes of successful performance measures.  However, DODIG has not developed measures for quality for fiscal year  2017 and beyond. By developing quality performance measures and  improving existing timeliness performance measures to be consistent with  all attributes of successful measures, senior DODIG leaders will have  better information to assess progress in improving timeliness and  completeness of investigations as well as to guide ongoing and future  improvement efforts.
section_title: Recommendations for Executive Action
paragraphs: We are making the following seven recommendations to the Department  of Defense Office of Inspector General (DODIG):
The DOD Inspector General should assess the feasibility of collecting  additional workload data, such as the amount of direct and indirect  labor hours associated with each case, and including such data in  future personnel requirements assessments, as appropriate.  (Recommendation 1)
The DOD Inspector General should report regularly to Congress on  the timeliness of civilian and contractor investigations, including those  contractor and subcontractor cases exceeding the 180-day timeliness  requirement. (Recommendation 2)
The DOD Inspector General should implement a process to document  employee recusals and impairments to independence and incorporate  such information into an aggregate-level evaluation of threats to  DODIG’s independence. (Recommendation 3)
The DOD Inspector General should establish and clearly  communicate a declination policy for nondiscretionary cases in the AI  Investigations Manual or other guidance, and align this policy with the  intake policy. (Recommendation 4)
The DOD Inspector General should revise the existing internal  controls checklist to include all key case-file documentation and  required investigative events. (Recommendation 5)
The DOD Inspector General should work in coordination with the  Secretary of Defense, the Under Secretary of Defense for Intelligence,  and the inspectors general of the defense intelligence components to  establish a process to fully implement the requirements of Directive- Type Memorandum 13-008 so that DODIG (1) receives notifications of  all allegations received by the components, (2) reviews all component  determinations to not investigate allegations, and (3) reviews all  investigations conducted by the components. (Recommendation 6)
The DOD Inspector General should develop quality performance  measures and enhance existing timeliness performance measures to  reflect key attributes of successful performance measures. At  minimum, these measures should be clear, quantifiable, and  objective, and they should include a baseline assessment of current  performance. (Recommendation 7)
section_title: Agency Comments
paragraphs: We provided a draft of this report to DODIG for review and comment. In  written comments, DODIG concurred with each of the seven  recommendations and cited actions it plans to take to address them.  DODIG’s comments are reprinted in appendix III.
As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 11 days from the  report date. At that time, we will send copies of this report to the  congressional defense committees, the Secretary of Defense, the  Department of Defense Inspector General, the Inspectors General of the  Defense Intelligence Agency and the National Security Agency, the Office  of Special Counsel, and other interested parties. In addition, the report is  available at no charge on the GAO website at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-3604 or farrellb@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix IV.
section_title: Appendix I: Scope and Methodology
paragraphs: To determine the extent to which the Department of Defense Office of  Inspector General (DODIG) has met and taken steps to achieve key  timeliness goals for civilian and contractor whistleblower reprisal  investigations, we obtained data on all Department of Defense (DOD)  civilian and contractor employee whistleblower reprisal cases closed by  DODIG and appropriated-fund DOD civilian employee cases closed by  the Office of Special Counsel from October 1, 2012, through September  30, 2015, and on all DODIG civilian and contractor employee cases open  as of September 2016. We selected data from this period because they  constituted the most complete and recent data available in DODIG’s and  the Office of Special Counsel’s case-management systems during the  time we were doing our analysis. Using these data, we calculated the  timeliness of DODIG and Office of Special Counsel cases in relation to  time frames prescribed by statute and internal goals. Specifically, we  assessed the timeliness of closed DODIG cases against the 180-day  statutory timeliness requirement for contractor cases; DODIG’s 240-day  internal goal for all civilian cases; DODIG’s 30-day internal goal for the  intake of complaints; and DODIG’s 60-day internal goal for completing  oversight reviews that was in place during the years under review. We  also assessed the length of DODIG’s open cases involving DOD civilian  and contractor employees. We assessed the timeliness of closed Office  of Special Counsel cases involving DOD appropriated-fund civilians  against the statutory 240-day goal for completing these investigations.  To identify other characteristics of DODIG cases involving DOD civilians  and contractors we also analyzed case data by fiscal year to determine  statute type, case disposition, the number of complaints received, the  number of complaints investigated, the frequency and type of alleged  personnel actions, and rates of substantiation for investigated cases.
We assessed the reliability of DODIG and Office of Special Counsel data  by administering questionnaires to officials familiar with the data systems,  interviewing DODIG and Office of Special Counsel officials, and reviewing  the queries used to retrieve the data, along with system documentation  and quality-assurance procedures. For DODIG data, we also compared  electronic data to fiscal year 2015 case-file documentation associated  with our review of a stratified random sample of fiscal year 2015 case  files, discussed below, to determine whether dates had been properly  recorded in the system, and we conducted internal logic and range  checks to assess completeness and accuracy. We notified DODIG  officials of data anomalies found during our review. For example, we  found negative values for days in intake for six cases and discrepancies  in dates between case-file documentation and database entries for 7 of  11 fiscal year 2015 non-appropriated-fund instrumentality (NAFI) civilian  employee cases. DODIG made corrections to these data, which we  incorporated into our analysis, as well as to fiscal year 2013 and 2014  NAFI cases. We reviewed such case-data inaccuracies against Council of  the Inspectors General on Integrity and Efficiency (CIGIE) standards  related to the accuracy of database information and Standards for  Internal Control in the Federal Government related to obtaining and  presenting reliable information. Overall, we determined that DODIG and  Office of Special Counsel data were sufficiently reliable for the purpose of  our objectives. Specifically, we determined that DODIG data were reliable  to report on the timeliness of DODIG cases by fiscal year and case type,  including case-length mean, median, and range; case disposition,  including cases declined, dismissed, withdrawn, and investigated, whether substantiated or not substantiated; case outcomes; and other  case characteristics, including the number and type of personnel actions  reported. Office of Special Counsel data were determined to be  sufficiently reliable for limited uses, including the case-length average and  median, and range of cases that went through some degree of  investigation and were not closed in the Office of Special Counsel’s  complaints examining unit and did not go through the Office of Special  Counsel’s alternative dispute-resolution process.
To identify factors affecting case timeliness and assess DODIG’s steps to  improve timeliness, we reviewed documentation pertaining to DODIG and  Office of Special Counsel investigations and case management and  interviewed cognizant agency officials. We also interviewed the four  supervisory investigators of investigative teams, and conducted  semistructured interviews with the 24 investigators who collectively are  responsible for conducting civilian and contractor investigations. We  analyzed the results of the semistructured interviews both quantitatively  and qualitatively—as described below. We also compared DODIG’s  completed and planned efforts to improve the timeliness of its  investigations against DOD guidance related to manpower management,  CIGIE standards related to workload data, and Standards for Internal  Control in the Federal Government related to the importance of  information to oversight bodies.
To determine the extent to which DODIG has established processes to  ensure that whistleblower reprisal cases are handled independently and  thoroughly, we reviewed DODIG investigative policies, processes,  practices, guidance, and quality-assurance mechanisms and safeguards  pertaining to investigative independence and thoroughness against  relevant statutes, CIGIE standards on independence and thoroughness,  and federal internal control standards related to control activities and  performance measurement. Using the case data we obtained to assess  timeliness from fiscal years 2013 through 2015, we also selected a  stratified random sample of 187 case files from the population of 409  civilian and contractor whistleblower reprisal cases that were closed by  DODIG in fiscal year 2015, from October 1, 2014, through September 30,  2015. We chose to review cases from this period as they constituted the  most recent and complete year of data available and would most  accurately reflect the extent to which DODIG’s cases were thorough. We  stratified the population into 12 strata according to employee type and  disposition. We randomly selected cases within 3 of the 12 strata and  selected all cases with certainty from the remaining 9 strata as shown in  table 5 below. Specifically, we calculated a sample size for stratum 1  (civilian dismissed cases where DODIG declined to take action) to  achieve a desired precision of plus or minus 13 percentage points. We  calculated the sample size for strata 8 and 9 (contractor dismissed cases)  to achieve a desired precision of plus or minus 10 percentage points or  fewer. We reviewed all investigated cases. As a result, data collected  from our case-file review and presented in this report involving  investigated cases do not have a sampling error.
During the course of our review, we removed 9 out-of-scope cases,  reducing the original sample size from 187 to 178. These cases included  8 cases involving complainants that were military servicemembers and 1  case involving a contractor employee that was a duplicate case. These  removals did not materially affect the confidence intervals of our sample  and were not replaced because they all occurred in strata where we  selected all of the cases for review. We also identified and replaced one  case that would have required significant work on the part of DODIG in  order to obtain approval to release classified case data. Table 5 describes  the 12 strata by adjusted population and sample size.
Because we followed a probability procedure based on random  selections, our sample is only one of a large number of samples that we  might have drawn. Since each sample could have provided different  estimates, we express our confidence in the precision of our particular  sample’s results as a 95 percent confidence interval (e.g., plus or minus 7  percentage points). This is the interval that would contain the actual  population value for 95 percent of the samples we could have drawn. As  a result, we are 95 percent confident that each of the confidence intervals  in this report will include the true values in the study population. All  percentage estimates in this report have a margin of error of plus or  minus 7 percentage points at the 95 percent confidence level.
To conduct the case-file review, we developed and used a data-collection  instrument to capture information regarding general case characteristics,  the reliability of database information, and the presence of information  and documentation required by law, regulations,  policy, and best practices such as standards for investigations  established by CIGIE. Core elements of this instrument were shared  with DODIG officials to ensure alignment with the policies and practices in  place during the cases’ period of investigation, and we incorporated their  feedback into our instrument before commencing the file review. These  core elements represented individual documents and data elements, and  sometimes comprised multiple documents. Examples of elements in our  review that represent key data in DODIG’s database, demonstrate  compliance with statute, or constitute documentation of key steps of the  investigative, case-management, and oversight processes include the  following:  interview documentation or recording;  closure letters to complainants, subjects, and responsible  management officials;  report reviews and approvals, including management, quality- assurance, and General Counsel;  key correspondence, including between DODIG and the complainant,  for contractor cases exceeding 180 days; oversight worksheet (oversight reviews); and oversight approval memorandum (oversight reviews).
To validate the data-collection instrument and ensure consistency in its  application, we developed and used standard operating procedures to  review a test sample of 16 case files that were selected randomly from  each stratum to ensure that each stratum of the full random sample was  tested at least once. We adjusted the relevant case-file elements for each  case based on its stratum and circumstances (e.g., by not looking for  certain elements for withdrawn cases) and captured responses in our  data-collection instrument accordingly. To ensure the reliability of the  results of the full case-file review, two analysts reviewed each case file  and coded for the presence of required information using the data- collection instrument, with one analyst conducting the initial coding and  the second analyst reviewing the first analyst’s work. In the event that  disagreement between the two analysts occurred, the analysts discussed  and resolved the disagreement by identifying and reviewing supporting  database information or documentation, and obtained the input of a third  analyst, if necessary, until a final resolution was made. We did not  question DODIG’s judgment in these cases. We generalized the results of  our sample to the population of 400 in-scope cases closed in fiscal year  2015 with a margin of error of plus or minus 7 percentage points or  fewer.
To assess the implementation of DODIG independence and  thoroughness policies, and to identify supplementary investigative and  case-management practices, we interviewed the 24 investigators and four  supervisory investigators responsible for conducting civilian and  contractor investigations, as well as whistleblower reprisal investigations  unit management. To interview the 24 investigators, we used a  standardized semistructured interview questionnaire. As part of the  questionnaire’s development, a survey specialist helped develop the  questionnaire, and another survey specialist provided independent  feedback on the questionnaire to ensure that content necessary to  understand the questions was included and that the questions could be  answered accurately and completely. To minimize errors that might occur  from respondents interpreting our questions differently than we intended,  we pretested our questionnaire with four investigators (including one  senior investigator) to ensure the clarity and reasonableness of the  questions. During the pretests, conducted in person and by phone, we  read the instructions and each question out loud to the interviewees and  asked them to tell us whether (1) the instructions and questions were  clear and unambiguous, (2) the terms we used were accurate, and (3)  they could offer a potential solution to any problems identified. We noted  any potential problems and modified the questionnaire based on the  feedback received from the reviewers and pretests as appropriate.
We conducted interviews between September 12, 2016, and October 20,  2016. We interviewed all 24 investigators assigned to investigative teams,  for a response rate of 100 percent. Each interview was conducted by at  least two analysts, who then discussed interview responses to ensure  consistency in their interpretation. Because we collected data from every  investigator there was no sampling error. However, the practical  difficulties of conducting any semistructured interview may introduce  errors, commonly referred to as nonsampling errors. For example,  differences in how a particular question is interpreted, the sources of  information available to interviewees, how the responses were processed  and analyzed, or the types of people who do not respond can influence  the accuracy of the survey results. We took steps in the development of  the interview questionnaire, the data collection, and the data analysis to  minimize these nonsampling errors and help ensure the accuracy of the  answers that were obtained. For example, a social-science survey  specialist designed the questionnaire, in collaboration with analysts  having subject-matter expertise. Then, as noted earlier, the draft  questionnaire was pretested to ensure that questions were relevant,  clearly stated, and easy to comprehend. The questionnaire was also  reviewed by another specialist with expertise in survey development, as  mentioned above.
We calculated the frequency of responses to our bounded questions and  performed content analysis on the open-ended questions to identify  common themes from across the responses and to determine their  frequencies. To perform these analyses, data were manually extracted  from the questionnaires into a computer program, and data-entry  accuracy was verified. The quantitative analysis was performed in this  program by one analyst and independently reviewed by another analyst.  For the qualitative analysis, a standard coding scheme was developed to  identify common themes and determine their frequencies. We also used  professional judgment to identify other themes that did not meet the  chosen quantitative threshold, but were determined to be important based  on our review of case files, discussions with DODIG management, and  review of guidance and relevant standards. This analysis was conducted  by one analyst and independently reviewed by another analyst. The pair  of analysts then discussed any discrepancies in their coding until they  reached a consensus.
A selection of key survey questions whose results are described in this  report are listed below.
Have you received training during your time as a whistleblower  reprisal investigations unit investigator? (Interviewer recorded: “Yes,”  “No.”)  (If yes, we asked a series of questions regarding training on the case- management system, the various civilian and contractor whistleblower  statutes, CIGIE standards, federal ethics laws, conflicts of interest,  and independence, among other topics. The following question is an  example of the typical wording of those questions.) Have you ever  received training on the case management system? (Interviewer  recorded: “Yes,” “No.”)  (If yes) Was this training mandatory?
How frequently have you received this training?
How was it delivered?
Do you have an individual development plan? (Interviewer recorded:  “Yes,” “No.”) (If yes) How do you identify training to include in your plan?
Is your plan updated?  (If yes) How often?
Is your plan tracked to ensure that training has been completed?  (If yes) Who is responsible for tracking it?
Would a standardized training curriculum with material specific to your  core responsibilities as a whistleblower reprisal investigations unit  investigator help you execute your responsibilities as an investigator?  (Interviewer recorded: “Yes,” “No.”)  (If yes) Why?’ (If no) Why not?
During intake, do you document any information from the intake  interview with the complainant in the case-management system?  (Interviewer recorded: “Yes,” “No.”)  (If yes) How is this information documented?
In your opinion, is it generally feasible to complete contractor  investigations within 180 days? (This question was repeated for  civilian investigations within 240 days.) (Interviewer recorded: “Yes,”  “No,” “Don’t Know.”)  (If yes) Why do you think it is feasible?  (If no) Why do you think it is not feasible?
Are there factors that have typically contributed to any delays you’ve  encountered in meeting investigative time frames? (Interviewer  recorded: “Yes,” “No,” “Don’t Know,” “No delays.”)  (If yes) What are the factors?
As an investigation unfolds, how often, if at all, do investigative  planning roundtables occur? (This question was repeated for five  other types of roundtables.)
Who typically attends investigative planning roundtables?
Have you ever observed bias by another investigator, supervisory  investigator, whistleblower reprisal investigations unit management, or  other staff involved in the investigative or review processes?  (Interviewer recorded: “Yes,” “No,” “Don’t Know.”)  (If yes) Please provide an example of bias without identifying the  individuals involved.
In your opinion, what, if anything, could be done to improve the  handling of whistleblower reprisal investigations for DOD civilians and  contractors?
Separately, we also reviewed training materials, guidance, and  requirements for DODIG investigators and attended three DODIG  Administrative Investigations training symposia, which included sessions  on recent developments in whistleblower reprisal investigations,  investigator competencies, hotline best practices, and protecting  investigative source identities. During the course of our semistructured  interviews, we also discussed training practices, requirements, and  experiences with the 24 investigators responsible for conducting  investigations, and separately discussed these same topics and other key  survey questions with the four supervisory investigators responsible for  supervising the investigative teams.
To determine the extent to which DODIG has conducted oversight of  civilian reprisal cases handled by the defense intelligence components,  we assessed DODIG, National Security Agency IG, and Defense  Intelligence Agency IG oversight policies, processes, practices, and  guidance against DOD policy that implements Presidential Policy  Directive 19 (PPD-19) and governs the whistleblower reprisal case  oversight process. We also reviewed the four investigations completed  by the defense intelligence components—specifically, the National  Security Agency and Defense Intelligence Agency—in fiscal year 2015  and submitted to DODIG for review; and interviewed cognizant officials  from DODIG, the National Security Agency IG, and the Defense  Intelligence Agency IG to discuss existing oversight practices and to  obtain relevant perspectives on the requirements for case oversight  articulated by DOD policy.
To determine the extent to which DODIG has developed performance  measures to assess the timeliness and quality of its investigations, we  reviewed documentation including quarterly briefing materials, annual  reports, and internal control processes and checklists, and interviewed  DODIG officials, to identify timeliness and quality performance measures  for investigations and oversight reviews that have been variably used by  DODIG since fiscal year 2013. We compared this information to federal  internal control standards related to performance assessment, including  standards related to defining measures for assessing an entity’s  performance in achieving quantitative and qualitative objectives. DODIG  defined timeliness measures for fiscal year 2017 in April 2017 but it had  not identified quality performance measures for fiscal year 2017 as of that  time. As a result, we assessed DODIG’s fiscal year 2017 timeliness  measures against 10 key attributes of successful measures identified in  our prior work, including clarity, measurable targets, and baseline and  trend data. To perform this analysis, two analysts each used a  scorecard methodology to review the selected measures and determine  whether the measures “addressed,” “partially addressed,” or “did not  address” the attributes. We scored an attribute as “addressed” if all  aspects of an attribute were addressed; “partially addressed” if some, but  not all, aspects of an attribute were addressed; and “not addressed” if the  measure did not address any aspects of an attribute. To reconcile  disagreements, the two analysts discussed and resolved the differences  with the input of a third analyst, as necessary.
We conducted this performance audit from February 2016 to September  2017 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
section_title: Appendix II: General Characteristics of Civilian and Contractor Whistleblower Reprisal Cases
paragraphs: This appendix provides information on the characteristics of civilian and  contractor or subcontractor whistleblower reprisal cases based on our  analysis of fiscal year 2013 through fiscal year 2015 case data from the  Department of Defense Office of Inspector General (DODIG) case- management system and our review of a sample of 178 cases closed  from October 1, 2014, through September 30, 2015.
subsections: section_title: Case Dispositions for Complaints Closed in Fiscal Year 2013 through Fiscal Year 2015
paragraphs: The majority of civilian and contractor and subcontractor complaints  closed by DODIG in fiscal year 2013 through fiscal year 2015—1,094 of  1,197, or about 91 percent—were closed without investigation, while 103,  or about 9 percent, were fully investigated. Specifically, DODIG closed  29 fully investigated cases in fiscal year 2013, 42 fully investigated cases  in fiscal year 2014, and 32 fully investigated cases in fiscal year 2015.
Our analysis of data from DODIG’s case-management system showed  that the overall number of declined cases—those that do not go through  the intake process—as a proportion of total closed civilian and contractor  cases rose from fiscal year 2013 to fiscal year 2015. Specifically, there  were 176 declined cases, or 50 percent, out of a total of 353 closed cases  in fiscal year 2013; 270, or 60 percent, out of 447 in fiscal year 2014; and  243, or about 61 percent, out of 397 in fiscal year 2015. While  appropriated-fund civilian complaints resulted in the most declinations for  each fiscal year as, according to an official, these are generally referred  to the Office of Special Counsel, DODIG also declined complaints  involving non-appropriated-fund instrumentality (NAFI) civilians,  contractors or subcontractors, and Defense Civilian Intelligence  Personnel System (DCIPS) employees and any employees with eligibility  for access to classified information, as discussed in our report. Figure 4  shows the percentage of cases closed in each fiscal year, by case  disposition.
Substantiation rates of closed investigations varied by statute across the  3 fiscal years. NAFI civilian cases constituted the greatest proportion of  closed investigations—44 of 103, or about 43 percent—but the highest  substantiation rate was for investigations of complaints made by DOD  appropriated-fund civilians. As previously noted, DODIG retains for  investigation those cases involving appropriated-fund civilians that are  deemed to be of interest to DODIG or to the Secretary of Defense. Table  6 shows the number of closed investigations for each covered employee  group, the number of substantiated investigations, and the associated  substantiation rates across the 3 fiscal years.
section_title: Whistleblower Reprisal Complaints Received by DODIG in Fiscal Year 2013 through Fiscal Year 2015
paragraphs: In fiscal years 2013 through 2015, DODIG received a total of 1,208  complaints from appropriated-fund civilians; NAFI civilians; contractors,  subcontractors, grantees, and subgrantees; and DCIPS employees or  employees with eligibility for access to classified information alleging  whistleblower reprisal. The majority of complaints, 666, were from  appropriated-fund civilians. An additional 376 complaints came from  contractors, subcontractors, grantees, and subgrantees. NAFI civilians  brought 123 complaints, and the fewest complaints, 43, came from  DCIPS employees or employees with eligibility for access to classified  information. Figure 5 shows the percentage of whistleblower reprisal  complaints received by DODIG from each covered group across the 3  fiscal years.
Of the 1,208 complaints, 758, or 63 percent, were filed by civilians and  contractors or subcontractors affiliated with the military services. Among  these, civilians and contractors or subcontractors affiliated with the Army  made the highest number of complaints, with 304; the Air Force had 219;  the Navy had 189; and the Marine Corps had 46. Defense agencies had  another 203, and the rest were spread across the combatant commands,  joint commands, and other entities. Figure 6 provides the percentages of  civilian and contractor complaints received across all 3 fiscal years, by  organization.
section_title: Prohibited Personnel Actions Alleged by Complainants in Complaints Filed with DODIG from Fiscal Year 2013 through Fiscal Year 2015
paragraphs: We also received data on the prohibited personnel actions alleged by  complainants who filed whistleblower reprisal complaints in fiscal year  2013 through fiscal year 2015. Some prohibited personnel actions are  specific to certain statutes or to Presidential Policy Directive 19 (PPD-19).  For example, prohibited personnel actions involving the restriction of  one’s access to classified information are specific to PPD-19. Conversely,  other personnel actions are common across more than one statute. For  example, a change in duties or responsibilities is defined as a prohibited  personnel action under both 5 U.S.C. § 2302 and 10 U.S.C. § 1587.
Our analysis of data from DODIG’s case-management system showed  that the complaints received across the 3 fiscal years within our scope  alleged 935 prohibited personnel actions. Of those 935, 927 were  aligned with prohibited personnel actions in the DOD civilian and  contractor whistleblower statutes or PPD-19. To determine the frequency  of alleged prohibited personnel actions, we grouped similar actions from  each statute and PPD-19. Across all 3 fiscal years, we found that the  greatest proportion of alleged prohibited personnel actions, at 24 percent,  was demotion, suspension, or termination, which was mainly liked to  contractor, subcontractor, grantee, and subgrantee complainants.  However, two indeterminate categories—unspecified and other— constituted the greatest proportion of alleged prohibited personnel  actions, with about 36 percent, when combined.
Figure 7 provides the percentages of prohibited personnel actions falling  under the broader categories alleged in the civilian and contractor or  subcontractor whistleblower reprisal complaints received by DODIG from  fiscal year 2013 through fiscal year 2015 and provided to us.
section_title: Protected Disclosures Alleged by Complainants in Cases Closed by DODIG in Fiscal Year 2015
paragraphs: Our review of a representative sample of 178 cases closed by DODIG in  fiscal year 2015 showed that the majority of protected disclosures  described in whistleblower reprisal complaints alleged a violation of law,  rule, or regulation. Specifically, of 348 protected disclosures recorded in  the case files we reviewed, 251 alleged a violation of law, rule, or  regulation. The next largest category was a composite field we created to  capture those disclosures recorded in the system that DODIG determined  were not protected. This category, other/none/not determined to be a  protected disclosure, included 41 disclosures. Table 7 provides the  estimated percentages of alleged protected disclosures for all cases  closed in fiscal year 2015, based on our sample.
The recipients of the protected disclosures in our sample of closed cases  from fiscal year 2015 varied somewhat due to differences in the statutes  and PPD-19. For instance, 10 U.S.C. §1587 states that NAFI civilians can  disclose whistleblowing to anyone and to any civilian employee or  member of the armed forces designated by law or by the Secretary of  Defense to receive disclosures, which together accounted for 75 of 79  NAFI disclosure recipients in our sample. Table 8 provides estimated  percentages of the recipients of protected disclosures for all cases closed  in fiscal year 2015, based on our sample.
section_title: Appendix III: Comments from the Department of Defense
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Lori Atkinson, Assistant Director;  James Ashley; Tracy Barnes; David Ballard; Ryan D’Amore; Taylor  Hadfield; Cale Jones; Amie Lesser; Marc Molino; Richard Powelson;  Cynthia Saunders; Michael Silver; and Erik Wilkins-McKee made key  contributions to this report.
section_title: Related GAO Products
paragraphs: Contractor Whistleblower Protections Pilot Program: Improvements  Needed to Ensure Effective Implementation. GAO-17-227. Washington,  D.C.: March 2, 2017.
Whistleblower Protection: Additional Actions Would Improve Recording  and Reporting of Appeals Data. GAO-17-110. Washington, D.C.:  November 28, 2016.
Whistleblower Protection: DOD Has Improved Oversight for Reprisal  Investigations but Can Take Additional Actions to Standardize Process  and Reporting. GAO-16-860T. Washington, D.C.: September 7, 2016.
Department of Energy: Whistleblower Protections Need Strengthening.  GAO-16-618. Washington, D.C.: July 11, 2016.
Whistleblower Protection: DOD Needs to Enhance Oversight of Military  Whistleblower Reprisal Investigations. GAO-15-477. Washington, D.C.:  May 7, 2015.
Whistleblower Protection: Additional Actions Needed to Improve DOJ’s  Handling of FBI Retaliation Complaints. GAO-15-112. Washington, D.C.:  January 23, 2015.
Whistleblower Protection Program: Opportunities Exist for OSHA and  DOT to Strengthen Collaborative Mechanisms. GAO-14-286.  Washington, D.C.: March 19, 2014.
Whistleblower Protection: Actions Needed to Improve DOD’s Military  Whistleblower Reprisal Program. GAO-12-362. Washington, D.C.:  February 22, 2012.
Tax Whistleblowers: Incomplete Data Hinders IRS’s Ability to Manage  Claim Processing Time and Enhance External Communication.  GAO-11-683. Washington, D.C.: August 10, 2011.
Criminal Cartel Enforcement: Stakeholder Views on Impact of 2004  Antitrust Reform Are Mixed, but Support Whistleblower Protection.  GAO-11-619. Washington, D.C.: July 25, 2011.
Whistleblower Protection: Sustained Management Attention Needed to  Address Long-Standing Program Weaknesses. GAO-10-722.  Washington, D.C.: August 17, 2010.
Defense Contracting Integrity: Opportunities Exist to Improve DOD’s  Oversight of Contractor Ethics Programs. GAO-09-591. Washington,  D.C.: September 22, 2009.
Whistleblower Protection Program: Better Data and Improved Oversight  Would Help Ensure Program Quality and Consistency. GAO-09-106.  Washington, D.C.: January 27, 2009.
Justice and Law Enforcement: Office of Special Counsel Needs to Follow  Structured Life Cycle Management Practices for Its Case Tracking  System. GAO-07-318R. Washington, D.C.: February 16, 2007.
U.S. Office of Special Counsel: Strategy for Reducing Persistent Backlog  of Cases Should Be Provided to Congress. GAO-04-36. Washington,  D.C.: March 8, 2004.