The National Defense Authorization Act (NDAA) for Fiscal Year 1995 authorized the Secretary of Defense to conduct personnel demonstration projects at the department’s laboratories designated as Science and Technology Reinvention Laboratories. The demonstration projects were established to give laboratory managers more authority and flexibility in managing their civilian personnel. These projects function as the vehicles through which the department can determine whether changes in personnel management concepts, policies, or procedures, such as flexible pay or hiring authorities, would result in improved performance and would contribute to improved DOD or federal personnel management. Table 1 presents a list of the 15 defense laboratories included in the scope of our review. The Defense Laboratories Office—within the Office of the Undersecretary of Defense for Research and Engineering (Research and Engineering)— carries out a range of core functions related to the defense labs, including the aggregation of data, analysis of capabilities, and alignment of activities, as well as advocacy for the defense labs. The National Defense Authorization Act for Fiscal Year 2017 gave authority to conduct and evaluate defense laboratory personnel demonstration projects to the Under Secretary of Defense for Research and Engineering and, accordingly, the Defense Laboratories Office. The Defense Laboratories Office supports the Research and Engineering mission by helping to ensure comprehensive department-level insight into the activities and capabilities of the defense laboratories. The LQEP was chartered on April 15, 1994 to improve productivity and effectiveness of the defense laboratories through changes in, among other things, personnel management and contracting processes. The NDAA for Fiscal Year 2017 established a new organizational structure for the program, adding two new panels while also specifying that two previously existing subpanels on personnel and infrastructure would continue to meet. The NDAA for Fiscal Year 2017 requires the department to maintain a LQEP Panel on Personnel, Workforce Development, and Talent Management—one of the four panels established by a February 14, 2018 charter signed by the Under Secretary of Defense for Research and Engineering. The purpose of the panel is to help the LQEP achieve the following goals: (1) review and make recommendations to the Secretary of Defense on current policies and new initiatives affecting the defense laboratories; (2) support implementation of quality enhancement initiatives; and (3) conduct assessments and data analysis. The LQEP Panel on Personnel, Workforce, Development, and Talent Management includes representatives from each of the defense laboratories, as well as from the Army, Navy, Air Force, appropriate defense agencies, and Office of the Under Secretary of Defense for Research and Engineering. A hiring authority is the law, executive order, or regulation that allows an agency to hire a person into the federal civil service. Among other roles, hiring authorities determine the rules (or a subset of rules within a broader set) that agencies must follow throughout the hiring process. These rules may include whether a vacancy must be announced, who is eligible to apply, how the applicant will be assessed, whether veterans preference applies, and how long the employee may stay in federal service. Hiring authorities may be government-wide or granted to specific agencies. Competitive (Delegated) Examining. This is the traditional method for making appointments to competitive service positions, and it requires adherence to Title 5 competitive examining requirements. The competitive examining process requires agencies to notify the public that the government will accept applications for a job, screen applications against minimum qualification standards, apply selection priorities such as veterans preference, and assess applicants’ relative competencies or knowledge, skills, and abilities against job-related criteria to identify the most qualified applicants. Federal agencies typically assess applicants by rating and ranking them based on their experience, training, and education. Figure 1 depicts the Office of Personnel Management’s (OPM) 80-day standard roadmap for hiring under the competitive process. Governmentwide (Title 5) Direct Hire Authority. This authority allows agencies to appoint candidates to positions without regard to certain requirements in Title 5 of the United States Code, with OPM approval. A direct hire authority expedites hiring by eliminating specific hiring rules. In order for an agency to use direct hire, OPM must determine that there is either a severe shortage of candidates or a critical hiring need for a position or group of positions. When using the direct hire authority, agencies must adhere to certain public notice requirements. The Pathways Programs. These programs were created to ensure that the federal government continues to compete effectively for students and recent graduates. The current Pathways Programs consist of the Internship Program, the Recent Graduates Program, and the Presidential Management Fellows Program. Initial hiring is made in the excepted service, but it may lead to conversion to permanent positions in the competitive service. Veterans-Related Hiring Authorities. These include both the Veterans Recruitment Appointment Authority and the Veterans Employment Opportunities Act authority. The Veterans Recruitment Appointment authority allows for certain exceptions from the competitive examining process. Specifically, agencies may appoint eligible veterans without competition under limited circumstances or otherwise through excepted service hiring procedures. The Veterans Employment Opportunities Act authority is a competitive service appointment authority that allows eligible veterans to apply for positions announced under merit promotion procedures when an agency accepts applications from outside of its own workforce. The Defense Laboratory Direct Hire Authorities. These include the following four types of direct hire authorities granted to the defense laboratories by Congress for hiring STEM personnel: (1) direct hire authority for candidates with advanced degrees; (2) direct hire authority for candidates with bachelor’s degrees; (3) direct hire authority for veterans; and (4) direct hire authority for students currently enrolled in a graduate or undergraduate STEM program. The purpose of these direct hire authorities is to provide a streamlined and accelerated hiring process to allow the labs to successfully compete with private industry and academia for high-quality scientific, engineering, and technician talent. The Expedited Hiring Authority for Acquisition Personnel. This authority permits the Secretary of Defense to designate any category of positions in the acquisition workforce as positions for which there exists a shortage of candidates or there is a critical hiring need; and to utilize specific authorities to recruit and appoint qualified persons directly to positions so designated. The Science, Mathematics, and Research for Transformation (SMART) Scholarship-for-Service Program. This program was established pursuant to 10 USC §2192a, as amended, and is funded through the National Defense Education Program. The SMART scholarship for civilian service program provides academic funding in exchange for completing a period of full-time employment with DOD upon graduation. The labs have used the defense laboratory-specific direct hire authorities more than any other category of agency-specific or government-wide hiring authority. Defense laboratory officials we surveyed reported that these direct hire authorities had been the most helpful to the labs’ efforts to hire highly qualified candidates for STEM positions, and also reported that the use of certain incentives had been helpful in this effort. However, even with access to the authorities, these defense laboratory officials identified challenges associated with the hiring process that affected their ability to hire highly qualified candidates. For fiscal years 2015 through 2017, the defense laboratories used laboratory-specific direct hire authorities more often than any other category of hiring authorities when hiring STEM personnel. Moreover, the defense laboratories’ use of these direct hire authorities increased each year from fiscal year 2015 through fiscal year 2017. Of the 11,562 STEM hiring actions in fiscal years 2015 through 2017, approximately 46 percent were completed using one of the defense laboratory direct hire authorities. The second and third most used hiring authorities were internal hiring actions and the expedited hiring authority for acquisition personnel, each of which comprised approximately 12 percent of the hiring actions during the time period. Table 2 provides information on the overall number of hiring actions by hiring authority for fiscal years 2015 through 2017. The laboratory-specific direct hire authorities include the direct hire authorities for candidates with advanced degrees, candidates with bachelor’s degrees, and candidates who are veterans—authorities were granted by Congress in prior legislation. Among the defense laboratory direct hire authorities, the direct hire authority for candidates with bachelor’s degrees was used for 55 percent of all direct hires, for a total of 2,920 hiring actions for fiscal years 2015 through 2017. During the same time frame, the labs used the direct hire authority for candidates with advanced degrees for approximately 36 percent (1,919 hiring actions) of all direct hires, and the direct hire authority for veteran candidates for approximately 9 percent (455 hiring actions). In addition, for less than one percent of the direct hires, either the labs used another category of laboratory-specific direct hire authority or we were unable to determine which type of direct hire authority was used during those same three fiscal years. See table 3 for information on the defense labs’ use of the defense laboratory-specific direct hire authorities for fiscal years 2015 through 2017. In fiscal year 2017 the defense labs used the defense laboratory direct hire authorities for 54 percent of STEM hiring actions completed, representing an increase of approximately 16 percentage points relative to fiscal year 2015, when 38 percent were hired under defense lab direct hire authorities. For additional information on the labs’ use of hiring authorities in fiscal years 2015 through 2017, as well as hiring authority data by laboratory, see appendix IV. One laboratory official explained that the increased use of the direct hire authorities could be a result of the NDAA for Fiscal Year 2016, which increased the laboratories’ allowable use of the direct hire authority for candidates with bachelor’s degrees from 3 percent to 6 percent, and use of the direct hire authority for veterans from 1 percent to 3 percent, of the total number of scientific and engineering positions at each laboratory at the end of the preceding fiscal year. The direct hire authority for candidates with bachelor’s degrees was used most often—for 1,151 out of 1,835 hiring actions—as compared with the other direct hire authorities in fiscal year 2017. See table 4 for more information on the laboratories’ use of all hiring authorities in fiscal year 2017. In addition, table 5 provides more information on the labs’ use of the direct hire authorities in fiscal year 2017. Defense laboratory officials we surveyed most frequently identified the three defense laboratory-specific direct hire authorities as having helped to hire highly qualified candidates (see figure 2) and to hire quickly (see figure 3). Specifically, 15 of 16 respondents to our survey stated that each of the three direct hire authorities had been helpful in hiring highly qualified candidates, and that the direct hire authorities for veterans and for candidates with an advanced degree had helped them to hire quickly. Moreover, all 16 survey respondents stated that the direct hire authorities for candidates with a bachelor’s degree had helped them to hire quickly. Among the three direct hire authorities, the one for candidates with bachelor’s degrees was reported to be the most helpful to the laboratories’ hiring efforts, according to our survey results. A majority of the laboratory officials we surveyed also stated that the Expedited Hiring Authority and the Science, Mathematics, and Research for Transformation (SMART) Program had both helped facilitate their efforts to hire highly qualified candidates and to hire them quickly. According to our survey, the least helpful hiring authority that lab officials reported using was the delegated examining unit authority. Six of 16 survey respondents stated that the delegated examining unit authority had helped them to hire highly qualified candidates, while 9 of 16 stated that the authority had hindered this effort. Three of 16 survey respondents stated that the delegated examining unit authority had helped them to hire quickly, while 12 of 16 stated that the use of this authority had hindered their ability to hire quickly. During our interviews with laboratory officials, hiring officials and supervisors described the defense laboratory direct hire authorities as being helpful in their hiring efforts. For example, hiring officials from one lab stated that the direct hire authorities were the easiest authorities to use, and that since their lab had started using them, job offer acceptance rates had increased and their workload related to hiring had decreased. A hiring official from another laboratory stated that the use of direct hire authorities had allowed their lab to be more competitive with the private sector in hiring, which is useful due to the high demand for employees in research fields. A supervisor from one lab stated that the use of direct hire authorities was not only faster than the competitive hiring process, but it also allowed supervisors a greater ability to get to know candidates early in the process to determine whether they met the needs of a position. In comparison, hiring managers we interviewed at one laboratory stated that the Pathways Program is not an effective means of hiring students because the program requires a competitive announcement. Supervisors also stated that the application process for Pathways can be cumbersome and confusing for applicants and may cause quality applicants to be screened out early. Defense laboratory officials who responded to our survey also stated that the process takes too long and that quality applicants may drop out of the process due to the length of the process. Defense laboratory hiring data also indicated that use of the defense laboratory direct hire authorities resulted in faster than median hiring times. As shown in table 6, the median time to hire for STEM positions at the defense laboratories in fiscal year 2017 was 88 days. The median time to hire when using the defense laboratories’ direct hire authorities, Pathways, or the SMART program authority was faster than that of the median for all categories combined. The median time to hire when using the competitive hiring process was approximately twice as long as when using the labs’ direct hire authorities. Our full analysis of defense laboratory hiring data, including the time to hire by hiring authority category, for fiscal years 2015 through 2017 can be found in appendix V. Defense laboratory officials also cited the use of incentives as helpful in hiring highly qualified candidates, as shown in figure 4. According to our survey results, the defense laboratories’ flexibility in pay setting under their demonstration project authority was generally considered to be the most helpful incentive, with 13 of 16 survey respondents stating that this incentive had very much helped them to hire highly qualified candidates. During interviews, laboratory officials described the use of these incentives as being particularly helpful if a candidate is considering multiple job offers because the incentives can help make the lab’s offer more competitive with offers from other employers. Multiple hiring officials stated that they would generally not include such incentives in an initial offer, but that if the candidate did not accept that offer, they would consider increasing the salary or offering a bonus. A hiring official from one lab stated that his lab has not offered many recruitment bonuses in recent years, because their acceptance rate has been sufficiently high without the use of that incentive. Many of the recently hired lab employees whom we interviewed also cited incentives, including bonuses and student loan repayment, as factoring into their decisions to accept the employment offers for their current positions. For example, one recently hired employee stated that the lab’s student loan repayment program was a significant factor in his decision to accept employment at the lab rather than with private industry. Recently hired employees also cited less tangible benefits of working at the labs, including the work environment, job stability, and type of work performed, as key factors in their decisions to accept their current positions. One newly hired employee stated that, while she could earn more money in a private-sector job, the defense laboratory position would afford her the freedom to pursue the type of work she is currently doing, and that this was a major consideration in her decision to accept it. Another newly hired employee similarly stated that he was interested in the type of research conducted at the lab where he now works, and that he was attracted to the opportunity to contribute to the national defense, while also taking advantage of benefits that support the pursuit of higher education. Defense laboratory officials we surveyed reported that, although the available hiring authorities and incentives are helpful, they experience a range of challenges to their ability to hire highly qualified candidates, as shown in figure 5, ranging in order from the most to the least frequently cited. In addition, figure 6 shows the extent to which officials reported selected top challenges that hindered their respective labs’ abilities to hire highly qualified candidates. Defense laboratory officials described how hiring challenges identified in our survey affect their ability to hire high quality candidates. Specifically, these challenges are as follows: Losing quality candidates to the private sector: Fifteen of 16 survey respondents stated that this was a challenge, and 12 of the 15 stated that this challenge had somewhat or very much hindered their lab’s ability to hire highly qualified candidates for STEM positions since October 2015. Hiring officials and supervisors we interviewed stated that private-sector employers can make on-the-spot job offers to candidates at college career fairs or other recruiting events, whereas the labs are unable to make a firm job offer until later in the hiring process. Government-wide hiring freeze: Fifteen of 16 survey respondents identified this as a challenge, with 13 of those reporting that it had either somewhat or very much hindered their lab’s ability to hire highly qualified candidates for STEM positions since October 2015. Multiple hiring officials and supervisors we interviewed stated that they had lost candidates whom they were in the process of hiring because the candidates had accepted other offers due to the delays created by the hiring freeze. In addition, some officials stated that, although the freeze had been lifted, their labs’ hiring efforts were still affected by backlogs created by the freeze, or were adapting to new processes that were implemented as a result of the freeze. Delays with the processing of security clearances: Fifteen of 16 survey respondents cited this as a challenge; 12 of the 15 stated that this challenge had somewhat or very much hindered their lab’s ability to hire highly qualified candidates for STEM positions since October 2015. A supervisor from one lab stated that he was in the process of trying to hire two employees whose hiring actions had been delayed due to the security clearance process. The supervisor stated that he had been told it could potentially take an additional 6 months to 1 year to complete the process, and that he believed this may cause the candidates to seek other employment opportunities. In other cases, hiring officials stated that employees may be able to begin work prior to obtaining a clearance, but that they may be limited in the job duties they can perform while waiting for their clearance to be granted. The government-wide personnel security clearance process was added to GAO’s High Risk List in 2018, based on our prior work that identified, among other issues, a significant backlog of background investigations and delays in the timely processing of security clearances. Inability to extend a firm job offer until a final transcript is received: Fourteen of 16 survey respondents stated that this was a challenge, with 10 of the officials responding that it had somewhat or very much hindered their lab’s ability to hire highly qualified candidates. One hiring official stated that top candidates will often receive 5 to 10 job offers prior to graduation, and that his lab’s may be the only one of those offers that is characterized as tentative. Multiple officials noted that career fairs can often occur several months prior to graduation, so the lab would have to wait for the duration of this time before extending a firm offer to a candidate who has been identified. Delays with processing personnel actions by the external human resources office: Thirteen of 16 survey respondents stated that this presented a challenge, and 9 of the 13 stated that this challenge had somewhat or very much hindered their lab’s ability to hire highly qualified candidates for STEM positions since October 2015. Multiple hiring officials stated that employees at their human resource offices may not have an understanding of either the technical nature of the positions being filled at the lab or the lab’s unique hiring authorities, and that this lack of knowledge could create delays. Other officials noted that their servicing human resource offices seemed to be inflexible regarding certain paperwork requirements. For example, officials at one lab stated that their human resource office requires candidates’ resumes to be formatted in a particular way, and that they have been required to ask candidates to make formatting changes to their resumes. An official at another lab stated that the lab has faced similar challenges with regard to the formatting of transcripts and has had to request clarifying documentation from the university. In both cases, the officials described these requirements as embarrassing, and as a delay to the hiring process. Further, both a supervisor and a newly hired employee we interviewed noted that it is difficult to learn the status of an application when it is being processed by the human resource office. Overall length of the hiring process: Twelve of 16 survey respondents cited this as a challenge; 11 of the 12 stated that this challenge had somewhat or very much hindered their lab’s ability to hire highly qualified candidates for STEM positions since October 2015. Hiring officials and supervisors we interviewed stated that their lab had lost candidates due to the length of the hiring process. One supervisor we interviewed stated that he has encountered candidates who really wanted to work at his lab but had had to pursue other opportunities because they could not afford to wait to be hired by the lab. Multiple newly hired employees we interviewed described the process as slow or lengthy, but described reasons why they were willing to wait. For example, some employees were already working at their lab in a contractor or post-doctoral fellowship position, and accordingly they were able to continue in these positions while completing the hiring process for the permanent positions they now hold. One employee stated that if the process had gone on any longer, he likely would have accepted another offer he had received, while another employee stated that he knew of at least two post- doctoral fellows at his lab who chose not to continue in the hiring process for a permanent position at the lab due to the length of the hiring process. The department and the defense laboratories track hiring data that can be used to evaluate some aspects of the individual labs’ hiring efforts, but the Defense Laboratories Office has not routinely obtained or monitored these data or evaluated the effectiveness of hiring, including the use of hiring authorities, across the defense laboratories as a whole. Laboratory hiring data are captured at the department level in the Defense Civilian Personnel Data System (DCPDS)—the department’s system of record for personnel data. In addition, the individual defense laboratories track hiring data, including the type of hiring authority used and certain milestone dates that can be used to measure the length of the hiring process, known as time to hire. According to OPM guidance and our prior work, time to hire is a measure that may inform about the effectiveness of the hiring process, and federal agencies are required to report time to hire for certain types of hiring actions to OPM. Defense laboratory officials stated that, from their perspectives, the time- to-hire metric does not sufficiently inform about the effectiveness of the use of specific authorities, particularly when using the most commonly tracked milestones—from the initiation of a request for personnel action to an employee’s entrance-on-duty date. For example, officials stated that when a direct hire authority is used to hire a candidate who is completing the final year of his or her educational program, the lab may identify and provide a tentative offer to this candidate several months prior to graduation, consistent with private- sector recruitment methods. In this case, officials stated that the length of time between the initiation of the request for personnel action and the candidate’s entrance-on-duty date, following his or her graduation, could span a period of several months. According to defense laboratory officials, the total number of days for this hiring action gives the appearance that the use of the hiring authority was not efficient in this case; however, officials stated that it would have been effective from the supervisor’s perspective, because the use of the hiring authority resulted in the ability to recruit a highly qualified candidate in a manner that was more competitive with the private sector. Further, time-to-hire data, as reflected by the milestone dates that are currently tracked across the defense laboratories, may not reflect a candidate’s perception of the length of the hiring process. More specifically, a candidate may consider the hiring process to be completed upon receiving a job offer (either tentative or final), which could occur weeks or months before the candidate’s entrance-on-duty date, the commonly used end-point for measuring time to hire. According to officials, the length of time from when the offer is extended to entrance on duty can be affected by a candidate’s individual situation and preferences, such as the need to complete an educational program or fulfill family or professional responsibilities prior to beginning work in the new position. In other cases, certain steps of the hiring process, such as completing the initial paperwork or obtaining management approval, may occur after a candidate has been engaged but prior to the initiation of a request for personnel action—the commonly used start-point for measuring time to hire. In this situation, the candidate’s perception of the length of the hiring process may be longer than what is reflected by the time-to-hire data. For the reasons described above, some defense laboratories measure time to hire using milestones that they have determined more appropriately reflect the effectiveness of their hiring efforts. For example, officials from one lab stated that they have sought to measure the length of the hiring process that occurs prior to the request for personnel action, while officials from some labs stated that they measure time to hire using the tentative offer date as an end-point. In addition, some laboratories informally collect other types of data that they use in an effort to evaluate their hiring efforts, such as the reasons why candidates decline a job offer or feedback on the hiring process from newly hired employees. However, officials from the Defense Laboratories Office stated that their office has not conducted any review of the effectiveness of defense laboratory hiring, including the use of hiring authorities, across the labs. The National Defense Authorization Action for Fiscal Year 2017 gave authority to conduct and evaluate defense laboratory personnel demonstration projects to the Office of the Under Secretary of Defense for Research and Engineering, under which the Defense Laboratories Office resides. Defense Laboratories Office officials stated that the office has not evaluated the effectiveness of defense laboratory hiring because it does not have access to defense laboratory hiring data, has not routinely requested these data from the labs or at the department level to monitor the data, and has not developed performance measures to evaluate the labs’ hiring. As noted, laboratory hiring data are captured at the department level in DCPDS and in a variety of service- and laboratory- specific systems and tools. However, the Defense Laboratories Office does not have access to these data and, according to one official, the office would not have access to defense laboratory hiring data unless officials specifically requested them from the labs or from the Defense Manpower Data Center, which maintains DCPDS. According to the official, the Defense Laboratories Office has not routinely requested such data in the past, in part because their role did not require evaluation of such data. In addition, the Defense Laboratories Office has not developed performance measures to evaluate the effectiveness of hiring across the defense laboratories or the labs’ use of hiring authorities. An official from the Defense Laboratories Office stated that the office may begin to oversee the effectiveness of the defense laboratories’ hiring efforts and, in doing so, may consider establishing performance measures to be used consistently across the labs, which could include time-to-hire or other measures. However, as of March 2018, the office had not established such measures for use across the defense laboratories nor provided any documentation on any planned efforts. Standards for Internal Control in the Federal Government states that management should design appropriate types of control activities to achieve the entity’s objectives, including top-level reviews of actual performance and the comparison of actual performance with planned or expected results. Further, consistent with the principles embodied in the GPRA Modernization Act of 2010, establishing a cohesive strategy that includes measurable outcomes can provide agencies with a clear direction for implementation of activities in multi-agency cross-cutting efforts. We have previously reported that agencies are better equipped to address management and performance challenges when managers effectively use performance information for decision making. Without routinely obtaining and monitoring defense laboratory hiring data and developing performance measures, the Defense Laboratories Office cannot effectively oversee the effectiveness of hiring, including the use of hiring authorities, at the defense laboratories. Specifically, without performance measures for evaluating the effectiveness of the defense laboratories’ hiring, and more specifically the use of hiring authorities, the department lacks reasonable assurance that these authorities—in particular, those granted by Congress to the defense laboratories—are resulting in improved hiring outcomes. In addition, without evaluating the effectiveness of the defense laboratories’ hiring efforts, the department cannot understand any challenges experienced by the labs or determine appropriate strategies for mitigating these challenges. As a result, the department and defense laboratories may be unable to demonstrate that they are using their authorities and flexibilities effectively, or that such authorities and flexibilities should be maintained or expanded for future use. DOD does not have clear time frames for its process for approving and implementing new hiring authorities for the defense laboratories. Section 1105 of the Carl Levin and Howard P “Buck” McKeon National Defense Authorization Act for Fiscal Year 2015 established a direct hire authority for students enrolled in a scientific, technical, engineering, or mathematics course of study at institutions of higher education on a temporary or term basis. Officials from the Defense Laboratories Office stated that the labs were unable to use the authority because the department’s current process—the publication of a federal register notice—for allowing the laboratories to use the hiring authority took longer than anticipated. On June 28, 2017—2 ½ years after the authority was granted in the NDAA for Fiscal Year 2015—the department published a federal register notice allowing the defense laboratories the authority to use the direct hire for students. DOD officials stated that the department has typically published a federal register notice whenever the defense laboratories are granted a new hiring authority in legislation—for example, when an NDAA is issued, or when certain modifications to the demonstration projects are made. The Defense Civilian Personnel Advisory Service—through its personnel policymaking role for the department—at the time required that the federal register notice process be used to implement any hiring authorities granted to the defense labs by Congress in legislation. These procedures were published in DOD Instruction 1400.37. DOD officials identified coordination issues that occurred during the approval process of the federal register notice across the relevant offices as the cause of the delay associated with this federal register notice. Changes to DOD organizational structures further complicated the process of implementing new hiring authorities for defense laboratories. Specifically, in late 2016 a provision in the NDAA for Fiscal Year 2017 shifted the authority to conduct and evaluate defense laboratory personnel demonstration projects from the Office of the Under Secretary of Defense for Personnel and Readiness to the Office of the Under Secretary of Defense for Research and Engineering. Within the Office of the Under Secretary of Defense for Research and Engineering, the Defense Laboratories Office has been tasked with the responsibility for matters related to the defense laboratories. According to the Director of the Defense Laboratories Office, informal discussions about the transition began shortly after the NDAA for Fiscal Year 2017 was passed in late 2016. According to that official, despite the shift in oversight responsibility, coordination between the offices of the Under Secretaries for Research and Engineering and for Personnel and Readiness is required on issues related to civilian personnel, including defense laboratory federal register notices. Although a formal process for coordination did not exist at the start of our review, officials from the Defense Laboratories Office stated that representatives from the offices have met approximately five times since December 2016 and were taking steps to establish a coordination process for implementing new authorities. According to officials from the Defense Laboratories Office, during those meetings as well as during other, less formal interactions, officials have taken steps to formalize the roles and responsibilities of the relevant offices. According to officials from the Defense Laboratories Office, as of May 2018 the office was drafting a memorandum to formalize the roles and responsibilities of the Defense Laboratories Office and the Office of the Under Secretary of Defense for Personnel and Readiness to correspond to the federal register notice approval process; however, officials did not provide a completion date. The Defense Laboratories Office established and documented its own federal register approval process in spring 2017 and updated it in early 2018. The aforementioned memorandum would further describe the roles and responsibilities for the Offices of the Under Secretary for Research and Engineering and the Deputy Assistant Secretary of Defense for Civilian Personnel Policy in carrying out the updated process. According to officials, this is the process the office will use moving forward for coordination and approval of any future federal register notices. On March 6, 2018, the Office published a federal register notice that rescinds the earlier instruction published by the Defense Civilian Personnel Advisory Service of the Office of the Under Secretary of Personnel and Readiness. By rescinding that instruction—including the earlier process for approving requests from the labs and federal register notices—the Defense Laboratories Office can, according to officials, publish its own process and guidance. In a 2016 presentation to the Joint Acquisition/Human Resources Summit on the defense laboratories, the Chair of the Laboratory Quality Enhancement Program Personnel Subpanel stated that a renewed and streamlined approval process would be beneficial to the creation of new authorities, among other things. Although Defense Laboratories Office officials provided a flowchart of the office’s updated federal register approval process for coordination, this process did not include time frames for specific stages of the coordination. Officials stated that they cannot arbitrarily assign time frames or deadlines for a review process because any time frames will be contingent on the other competing priorities of each office, and other tasks may take priority and thus push review of a federal register notice down in order of priority. Our prior work has found that other federal agencies identify milestones, significant events, or stages in the agency-specific rulemaking process, and track data associated with these milestones. That work also found that, despite variability across federal agencies in the length of time taken by the federal rulemaking process, scheduling and budgeting for rulemaking are useful tools for officials to manage regulation development and control the resources needed to complete a rule. Standards for Internal Control in the Federal Government further establishes that management should design control activities to achieve objectives and respond to risks. Further, management should also establish an organizational structure, assign responsibility, and delegate authority to achieve the entity’s objectives. Moreover, documentation is a necessary part of an effective internal control system. The level and nature of documentation may vary based on the size and complexity of the organization and its processes. The standards also underscore that specific terms should be fully and clearly set forth such that they can be easily understood. Our prior work on interagency collaboration has also found that overarching plans can help agencies overcome differences in missions, cultures, and ways of doing business, and can help agencies better align their activities, processes, and resources to collaborate effectively to accomplish a commonly defined outcome. Without establishing and documenting clear time frames for its process for departmental coordination efforts related to the approval and implementation of new hiring authorities, the department cannot be certain that it is acting in the most efficient or effective manner possible. Moreover, the defense laboratories may not promptly benefit from the use of congressionally granted hiring authorities, relying instead on other existing authorities. Doing so could, according to officials, have the unintended consequence of complicating the hiring process, increasing hiring times, or resulting in the loss of highly qualified candidates. The future of the department’s technological capabilities depends, in large part, on its investment in its people—the scientists and engineers who perform research, development, and engineering. To that end, Congress has granted the defense laboratories specific hiring authorities meant to encourage experimentation and innovation in their approaches to building and strengthening their workforces. The defense laboratories have used most of these authorities as a part of their overall hiring efforts. However, without obtaining and monitoring hiring data and developing performance measures, the Defense Laboratories Office may not be in a position to provide effective oversight of the defense laboratories’ hiring, including the use of hiring authorities, or to evaluate the effectiveness of specific hiring authorities. Moreover, the absence of clear time frames to facilitate timely decision-making and implementation of any new hiring authorities may impede the laboratories’ ability to make use of future authorities when authorized by Congress. Until the department addresses these issues, it lacks reasonable assurance that the defense laboratories are taking the most effective approach toward hiring a workforce that is critical to the military’s technological superiority and ability to address existing and emerging threats. We are making three recommendations to DOD. The Secretary of Defense should ensure that the Defense Laboratories Office routinely obtain and monitor defense laboratory hiring data to improve the oversight of the defense laboratories’ use of hiring authorities. (Recommendation 1) The Secretary of Defense should ensure that the Defense Laboratories Office develop performance measures to evaluate the effectiveness of the defense laboratories’ use of hiring authorities as part of the labs’ overall hiring to better inform future decision making about hiring efforts and policies. (Recommendation 2) The Secretary of Defense should ensure that the Defense Laboratories Office, in collaboration with the Under Secretary of Defense for Personnel and Readiness and the Laboratory Quality Enhancement Panel’s Personnel Subpanel, establish and document time frames for its coordination process to direct efforts across the relevant offices and help ensure the timely approval and implementation of hiring authorities. (Recommendation 3) We provided a draft of this report to DOD for review and comment. In its written comments, reproduced in appendix VI, DOD concurred with our recommendations, citing steps the department has begun and plans to take to improve oversight and coordination of the defense laboratories’ hiring efforts. DOD also provided technical comments, which we incorporated as appropriate. We are sending copies of this report to the appropriate congressional committees, the Secretary of Defense, and other interested parties, including the Defense Laboratories Office and defense laboratories. In addition, this report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions regarding this report, please contact Brenda Farrell at (202) 512-3604 or farrellb@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix VII. The term “STEM” refers to the fields of science, technology, engineering, and mathematics. The following figure identifies the Department of Defense’s broad categories of STEM occupations, as well as the specific occupational series within each category. This report examines (1) the defense laboratories use of existing hiring authorities and what officials view as the benefits of authorities and incentives and the challenges in hiring; (2) the extent to which the Department of Defense (DOD) evaluates the effectiveness of hiring, including hiring authorities, at the defense laboratories; and (3) the extent to which DOD has time frames for approving and implementing new hiring authorities. To address these objectives, we included in the scope of our review science, technology, engineering, and mathematics (STEM) hiring at the 15 defense laboratories designated as Science and Technology Reinvention Laboratories (STRL) that were implemented at the time of our review within the Army, Navy, and Air Force. We included 9 Army laboratories: Armament Research, Development, and Engineering Center; Army Research Laboratory; Aviation and Missile Research, Development, and Engineering Center; Communications-Electronics Research, Development, and Engineering Center; Edgewood Chemical and Biological Center; Engineer Research and Development Center; Medical Research and Materiel Command; Natick Soldier Research, Development, and Engineering Center; and Tank Automotive Research, Development, and Engineering Center. We included 5 Navy laboratories: Naval Air Systems Command Warfare Centers, Weapons Division and Aircraft Division; Naval Research Laboratory; Naval Sea Systems Command Warfare Centers, Naval Surface and Undersea Warfare Centers; Office of Naval Research; and Space and Naval Warfare Systems Command, Space and Naval Warfare Systems Center, Atlantic and Pacific. We included 1 Air Force laboratory: Air Force Research Laboratory. We excluded 2 additional defense laboratories within the Army—the Army Research Institute and the Space and Missile Defense Command—because these defense laboratories were in the process of being implemented at the time of our review. For our first objective, we obtained and analyzed documentation, including past National Defense Authorization Acts (fiscal years 1995 through 2017), guidance related to government-wide hiring authorities, and federal register notices on existing hiring authorities used by the defense laboratories to hire STEM personnel. We obtained data that were coordinated by the Defense Manpower Data Center and prepared by the Defense Civilian Personnel Advisory Service’s Planning and Accountability Directorate. These data included, among other things, hiring process milestone dates and type of hiring authority used for each civilian hire at the defense laboratories for fiscal years 2015 through 2017. We selected these years because they were the three most recent years for which hiring data were available, and because doing so would allow us to identify any trends in the use of hiring authorities or the length of time taken to hire. The data we obtained were extracted from DCPDS using the Corporate Management Information System. The team refined the data to include only those hiring actions that were made by the 15 defense laboratories included within the scope of our review. In addition, we excluded hiring actions that used a 700-series nature of action code, which denotes actions that relate to position changes, extensions, and other changes, which we determined should not be included in our analysis. We included actions that used nature of action codes in the 100-series (appointments) and 500-series (conversions to appointments). For the purpose of calculating time to hire, we also excluded records with missing dates and those for which the time-to-hire calculation resulted in negative number (that is, the record’s request for personnel action initiation date occurred after the enter-on- duty date). Specifically, we excluded 92 actions for which no request for personnel action initiation date was recorded and 205 actions for which the date occurred after the enter-on-duty date, for a total of 2.57 percent of all hiring actions. We included in our calculation 7 actions for which the request for personnel action initiation date was the same date as the enter-on-duty date, resulting in a time to hire of zero days. To determine the extent to which the defense laboratories use existing hiring authorities, based on the department’s data, we analyzed the current appointment authority codes identified for individual hiring actions. Current appointment authority codes are designated by the Office of Personnel Management and are used to identify the law, executive order, rule, regulation, or other basis that authorizes an employee’s most recent conversion or accession action. Based on our initial review of the data, we determined that, in some cases, more than one distinct current appointment authority code could be used to indicate the use of a certain hiring authority. Alternately, a single current appointment authority code could in some cases be used for indicating more than one type of authority. In these cases, the details of the specific type of hiring authority that was used for the hiring action can be recorded in the description field associated with the current appointment authority code field. For this reason, in order to determine the type of hiring authority used, it was necessary to analyze the description fields for the current appointment authority code when certain codes were used. Two analysts independently reviewed each description and identified the appropriate hiring authority. Following this process, the two analysts compared their work and resolved any instances in which the results of their analyses differed. A data analyst used the results to produce counts of the number of times various categories of hiring authorities were used, as well as the average time to hire for each hiring authority category. For those instances where the analysts could not identify a hiring authority on the basis of the three digit codes or the description fields, the hiring actions were assigned to an “unknown” category. We note that the “unknown” category included 591 hiring actions, or approximately 5 percent of the total data for fiscal years 2015 through 2017. In addition, within the laboratory-specific direct hire authority category, if a determination could not be made about the specific type of laboratory- specific direct hire authority used, the hiring action was captured in the “direct hire authority, unspecified” category because the action was clearly marked as one of the laboratory-specific direct hire authorities but the type of authority (for example, direct hire for veterans) was unclear. Of the 5,303 hiring actions identified as a laboratory-specific direct hire authority, 0.1 percent of the hiring actions fell into the unspecified category. Based on the aforementioned steps and discussions with officials from the Defense Civilian Personnel Advisory Service and the Defense Manpower Data Center and reviews of additional documentation provided to support the data file, as well as interviews with officials from 13 of the laboratories about their data entry and tracking, we determined that these data were sufficiently reliable for the purposes of reporting the frequency with which the labs used specific hiring authorities and calculating the time it takes the labs to hire, or time to hire, for fiscal years 2015 through 2017. To describe officials’ views of hiring authorities and other incentives, we conducted a survey of officials at each of the defense laboratories on (1) their perceptions of the various hiring authorities and incentives, (2) whether those authorities and incentives have helped or hindered hiring efforts, (3) the extent to which they experienced barriers to using hiring authorities, and (4) any challenges during the hiring process, among other things. We administered the survey to the official at each defense laboratory who was identified as the Laboratory Quality Enhancement Program Personnel, Workforce Development, and Talent Management Panel point of contact, because we determined that this individual would be the most knowledgeable about his or her lab’s hiring process and use of hiring authorities. One laboratory—the Space and Naval Warfare Systems Command Centers—had two designated Laboratory Quality Enhancement Program Personnel, Workforce Development, and Talent Management Panel points of contact, one for each of its command centers (Atlantic and Pacific). Because the contacts would each be knowledgeable about his or her lab’s hiring processes for their respective command centers, we chose to include both command centers in our survey. As a result, we included a total of 16 laboratory officials in our survey. We drafted our questionnaire based on the information obtained from our initial interviews with department, service, and laboratory personnel. We conducted pretests to check that (1) the questions were clear and unambiguous, (2) terminology was used correctly, (3) the questionnaire did not place an undue burden on agency officials, (4) the information could feasibly be obtained, and (5) the survey was comprehensive and unbiased. We conducted five pretests to include representatives from each of the three services, as well as from corporate research laboratories and from research, development, and engineering centers. We conducted the pretests—with the assistance of a GAO survey specialist—by telephone and made changes to the content and format of the questionnaire after each pretest, based on the feedback we received. Key questions from the questionnaire used for this study are presented in appendix II. We sent a survey notification email to each laboratory’s identified point of contact on July 6, 2017. On July 10, 2017, we sent the questionnaire by email as a Microsoft Word attachment that respondents could return electronically after marking checkboxes or entering responses into open answer boxes. One week later, we sent a reminder email, attaching an additional copy of the questionnaire, to everyone who had not responded. We sent a second reminder email and copy of the questionnaire to those who had not responded 2 weeks following the initial distribution of the questionnaire. We received questionnaires from all 16 participants by August 4, 2017, for a 100 percent response rate. Between July 26 and October 5, 2017, we conducted additional follow-up with 11 of the respondents via email to resolve missing or problematic responses. Because we collected data from every lab, there was no sampling error. However, the practical difficulties of conducting any survey may introduce errors, commonly referred to as non-sampling errors. For example, differences in how a particular question is interpreted, the sources of information available to respondents, how the responses were processed and analyzed, or the types of people who do not respond can influence the accuracy of the survey results. We took steps in the development of the survey, the data collection, and the data analysis to minimize these non-sampling errors and help ensure the accuracy of the answers that were obtained. For example, a survey specialist designed the questionnaire, in collaboration with analysts having subject matter expertise. Then, as noted earlier, the draft questionnaire was pretested to ensure that questions were relevant, clearly stated, and easy to comprehend. The questionnaire was also reviewed by internal subject matter experts and an additional survey specialist. Data were electronically extracted from the Microsoft Word questionnaires into a comma-delimited file that was then imported into a statistical program for quantitative analyses and Excel for qualitative analyses. We examined the survey results and performed computer analyses to identify inconsistencies and other indications of error, and we addressed such issues as necessary. Quantitative data analyses were conducted by a survey specialist using statistical software. An independent data analyst checked the statistical computer programs for accuracy. To obtain information on department- and service-level involvement in and perspectives of defense laboratory hiring, we interviewed officials at the Defense Personnel Advisory Service, Defense Laboratories Office, Army Office of the Assistant G-1 for Civilian Personnel, and Navy Office of Civilian Human Resources. In addition, we interviewed hiring officials, first-line supervisors, and newly hired employees from a non- generalizable sample of six defense laboratories or subordinate level entities within a laboratory (for example, division or directorate) to obtain their perspectives on the hiring process. We selected the six laboratories based on the following two criteria: (1) two laboratories from each of the three services, and (2) a mix of both corporate research laboratories and research and engineering centers. In addition, because some hiring activities can occur at subordinate levels within a laboratory—such as a division or directorate—we included at least one subordinate level entity for each service. In total, we selected: Army Research Laboratory Sensors and Electron Devices directorate; Aviation and Missile Research, Development, and Engineering Center (Army); Naval Research Laboratory; Naval Air Warfare Center Weapons Division; Air Force Research Laboratory Information directorate; and Air Force Research Laboratory Space Vehicles directorate. For each lab, we requested to interview the official(s) most knowledgeable about the lab’s hiring process, supervisors who had recently hired, and newly hired employees. We initially requested to interview one group each of supervisors and newly hired employees. Following our first round of interviews at one laboratory, we requested to interview two groups each of supervisors and newly hired employees. Subsequent to this request, at one lab we were able to conduct one supervisor interview and at a second lab we were able to conduct one newly hired employee interview, due to scheduling constraints. The views obtained from these officials, supervisors, and recent hires are not generalizable and are presented solely for illustrative purposes. For our second and third objectives, we reviewed guidance and policies for collecting and analyzing laboratory personnel data related to the implementation and use of hiring authorities by these labs. We interviewed DOD, military service, and defense laboratory officials to discuss and review their hiring processes and procedures for STEM personnel, the use of existing hiring authorities, and efforts to document and evaluate time-to-hire metrics. We also met with DOD officials from the Office of the Under Secretary of Defense for Personnel and Readiness and the Office of the Under Secretary of Defense for Research and Engineering to discuss processes and procedures for implementing new hiring authorities granted by Congress. We evaluated their efforts to determine whether they met federal internal control standards, including that management should design appropriate types of control activities to achieve the entity’s objectives, including top-level reviews of actual performance, and should establish an organizational structure, assigning responsibilities and delegating authority to achieve an organization’s objectives. We conducted this performance audit from November 2016 to May 2018 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. We analyzed three years of Department of Defense hiring data obtained from the Defense Civilian Personnel Data System to identify the defense laboratories’ use of hiring authorities. We found that the defense laboratories completed a total of 11,562 STEM hiring actions in fiscal years 2015 through 2017 and used the defense laboratory direct hire authorities the most often when hiring STEM personnel. Table 7 provides information on the laboratories’ use of hiring actions by hiring authority for fiscal years 2015, 2016, and 2017. Table 8 provides a breakdown of the individual labs’ use of hiring authorities in fiscal years 2015 through 2017. We analyzed three years of the DOD hiring data to identify time to hire using various types of hiring authorities when hiring for Science, Technology, Engineering, and Math (STEM) occupations at the defense laboratories. Tables 9, 10, 11, and 12 below show the frequency of actions for each hiring authority category and the average, minimum, maximum, median, 25th percentile, and 75th percentile of the number of days to hire for each category in fiscal years 2015 through 2017 and for all three years combined. In addition to the contact named above, Vincent Balloon (Assistant Director), Isabel Band, Vincent Buquicchio, Joseph Cook, Charles Culverwell, Serena Epstein, Christopher Falcone, Robert Goldenkoff, Cynthia Grant, Chelsa Gurkin, Amie Lesser, Oliver Richard, Michael Silver, John Van Schaik, Jennifer Weber, and Cheryl Weissman made key contributions to this report.

Summary: DOD's defense labs help sustain, among other things, U.S. technological superiority and the delivery of technical capabilities to the warfighter. Over time Congress has granted unique flexibilities—such as the ability to hire qualified candidates who meet certain criteria using direct hire authorities—to the defense labs to expedite the hiring process and facilitate efforts to compete with the private sector. Senate Report 114-255 included a provision for GAO to examine the labs' hiring structures and effective use of hiring authorities. This report examines (1) the defense labs use of existing hiring authorities and officials' views on the benefits of authorities and challenges of hiring; (2) the extent to which DOD evaluates the effectiveness of hiring, including hiring authorities at the defense labs; and (3) the extent to which DOD has time frames for approving and implementing new hiring authorities. GAO analyzed DOD hiring policies and data; conducted a survey of 16 defense lab officials involved in policy-making; interviewed DOD and service officials; and conducted nongeneralizable interviews with groups of officials, supervisors, and new hires from 6 labs—2 from each of the 3 military services, selected based on the labs' mission. The Department of Defense's (DOD) laboratories (defense labs) have used the laboratory-specific direct hire authorities more than any other category of agency-specific or government-wide hiring authority for science, technology, engineering, and mathematics personnel. As shown below, in fiscal years 2015—2017 the labs hired 5,303 personnel out of 11,562 total hires, or 46 percent using these direct hire authorities. Lab officials, however, identified challenges to hiring highly qualified candidates, such as delays in processing security clearances, despite the use of hiring authorities such as direct hire. Source: GAO analysis of Department of Defense data. | GAO-18-417 . a Other includes all other defense laboratory-specific direct hiring authorities used. b All other includes remaining five categories of hiring authorities. c Percentages may not sum to total due to rounding. DOD and the defense labs track hiring data, but the Defense Laboratories Office (DLO) has not obtained or monitored these data or evaluated the effectiveness of the labs' hiring, including the use of hiring authorities. While existing lab data can be used to show the length of time of the hiring process, effectiveness is not currently evaluated. According to lab officials, timeliness data do not sufficiently inform about the effectiveness of the authorities and may not reflect a candidate's perception of the length of the hiring process. Further, the DLO has not developed performance measures to evaluate the effectiveness of hiring across the defense laboratories. Without routinely obtaining and monitoring hiring data and developing performance measures, DOD lacks reasonable assurance that the labs' hiring and use of hiring authorities—in particular, those granted by Congress to the labs—result in improved hiring outcomes. DOD does not have clear time frames for approving and implementing new hiring authorities. The defense labs were unable to use a direct hire authority granted by Congress in fiscal year 2015 because it took DOD 2½ years to publish a federal register notice—the process used to implement new hiring authorities for the labs—for that authority. DOD officials identified coordination issues associated with the process as the cause of the delay and stated that DOD is taking steps to improve coordination—including meeting to formalize roles and responsibilities for the offices and developing a new approval process—between offices responsible for oversight of the labs and personnel policy. However, DLO's new federal register approval process does not include time frames for specific stages of coordination. Without clear time frames for its departmental coordination efforts related to the approval and implementation of new hiring authorities, officials cannot be certain they are taking action in a timely manner. GAO recommends that DOD (1) routinely obtain and monitor defense lab hiring data to improve oversight; (2) develop performance measures for evaluating the effectiveness of hiring; and (3) establish time frames to guide hiring authority approval and implementation. DOD concurred with the recommendations.