Defense Pilot Programs: DOD Needs to Improve Implementation Process for Pilot Programs
Released: 2003-07-28T00:00:00

Letter

Background

The United States has a long history of military research and development. To help conduct and manage this research, DOD has a diverse network of 80 in-house laboratories and 26 test centers. Their missions range from basic scientific research to direct technical support to operational commands. The management, operations, and funding for these disparate laboratories and test centers also vary among the services.

Over the past decade, several organizations, panels, and commissions have identified significant personnel and resource problems facing the laboratories and test centers. For example, several studies found that the laboratories needed more flexibility in personnel rules governing the scientific workforce in order to attract and retain staff. Similarly, several recent studies identified problems with declines in investment and infrastructure, resulting in outdated facilities and technical equipment.

To help the laboratories and test centers with these problems, the Congress enacted legislation in fiscal years 1999 and 2000 establishing pilot programs for laboratories and test centers to propose innovative partnerships, business-like practices, and human capital initiatives. The 1999 pilot program focused on partnerships and business-like practices, while the 2000 program focused more on human capital initiatives. Together, the two pilot programs authorized the Secretary of Defense to provide one laboratory and one test center in each service the authority to explore innovative methods for partnering with universities and private sector entities to conduct defense research and development; attract a workforce balance between permanent and temporary personnel and with an appropriate skill and experience level; develop or expand innovative methods of operation that provide more defense research for the dollar; and waive any restrictions on these methods that are not required by law.

A total of 10 laboratories and test centers from all 3 services participated in the pilot programs. They are listed in appendix I.

Both programs were authorized for 3 years. The 1999 pilot expired in March 2002; the 2000 pilot, in March 2003. For both programs, DOD was required to submit preliminary and final reports to the Congress on program activities. The preliminary report for the 1999 program was submitted in July 1999. However, as of the date of this report, the three other reports have not been submitted.

In fiscal year 2003, the Congress authorized another 3-year pilot program and extended the 1999 and 2000 pilot programs until 2005. Under the new 2003 pilot program, the Secretary of Defense is to provide one laboratory and one test center in each service the authority to use innovative personnel management methods to ensure that the participants can employ and retain an appropriately balanced workforce, and effectively shape the workforce to fulfill the organization mission; develop or expand innovative methods of using cooperative agreements with private sector and educational organizations to promote the technological industrial base for critical defense technologies and facilitate the training of a future scientific and technical workforce; and waive any restrictions not required by law.

As of May 2003, DOD had not identified any participants for the 2003 pilot program.

The 2003 legislation also requires DOD to issue three reports, including a January 2003 report on its experience with the 1999 and 2000 pilot programs, barriers to implementation of these programs, and proposed solutions to overcome these barriers. According to DOD officials, this report has been drafted, but as of May 2003, it had not been submitted to the Congress.

Many Initiatives Were Proposed but Few Were Implemented under Pilot Programs

Since the inception of the pilot programs in 1999, 178 initiatives have been proposed, but only 4—or 2 percent—have been implemented under the pilot programs. Participating laboratories and test centers proposed initiatives covering a variety of areas, including business-like practices, partnerships with industry and academia, and human capital innovations. We found that laboratories focused many of their proposals on human capital innovations, while test centers tended to concentrate on business-like practices and partnerships.

Range and Volume of Proposed Initiatives

Over the course of the 1999 and 2000 pilot programs, the laboratories and test centers proposed 178 human capital, business, and partnership initiatives. As shown in table 1, slightly over half of the initiatives dealt with human capital and the remainder dealt with business-like practices and partnerships.

Overall, the laboratories proposed substantially more initiatives than did the test centers. Furthermore, the laboratories and test centers focused on different types of initiatives. The laboratories more often proposed human capital initiatives, while the test centers overwhelmingly focused on business and partnership initiatives. Laboratory officials told us that they are especially concerned about attracting top-quality scientists to replace a retiring workforce. Test center officials told us that they are focused on modernizing their infrastructure and developing new methods of sharing the cost of operations.

Proposals for business-like practices included many initiatives to streamline or improve local operations. Some initiatives focused on expanding the use of innovative techniques such as other transactions or cooperative agreements. Several other proposals sought the authority to reinvest fees or revenues into facilities revitalization. For example, one Navy laboratory proposed imposing a surcharge for its services and using that revenue to fund capital investments, and an Air Force laboratory proposed using facility construction as a valid in-kind contribution under cooperative agreements.

Partnership proposals included initiatives such as collaborative research agreements with Arnold Engineering Development Center and the University of Tennessee Space Institute to create a formal business bond to pursue research in laser-induced surface improvement technology and university flight research.

The Army’s Aberdeen Test Center proposed a limited liability company. Under this concept, industry, academia, and government would form a profit-making company to conduct research and testing at the installation. The test center proposed using its share of the profits to reinvest in the infrastructure at Aberdeen.

Several human capital initiatives focused on recruiting and retention flexibilities as well as additional voluntary separation incentives. These proposals included initiatives to streamline hiring of experts and consultants; accelerate promotions for scientists and engineers; provide retention bonuses for key scientists; and hire students directly after graduation. Several participants submitted proposals for direct hire authority to allow faster hiring of scientists, and several submitted proposals for voluntary retirement incentives as a mechanism for reshaping the workforce.

Few Proposals Were Implemented under Pilot Program Authorities

Almost none of the 178 proposed initiatives were approved and implemented using the pilot programs’ authorities. As figure 1 shows, only 2 percent—or 4 proposals—were implemented under the pilot programs. In contrast, 74 percent were blocked or dropped during the review process or remain on hold awaiting resolution.

The four implemented initiatives were donating laboratory equipment directly to local schools, waiving top-level certification of certain service agreements with streamlining cooperative agreements to facilitate collaborative work agreements with outside activities, and granting temporary relief from some mandatory personnel placement reviews.

Officials at the laboratories that proposed these initiatives told us that they were considered minor changes with little impact on the larger problems facing the laboratories.

Twelve times as many initiatives—24 percent—were implemented using different authorities than the pilot programs. For example, several laboratories requested the authority to appoint retired military members to civilian positions without having to wait the required 180 days. This requirement was waived using a different authority than the pilot programs. Another human capital initiative—to appoint senior scientists from private industry—was authorized by subsequent legislation. In the business/partnership category, the 46th Test Group at Holloman Air Force Base used other authorities to negotiate a complex leasing arrangement with industry to install a radar test facility at White Sands Missile Range. This effort took several years and overcame many contractual and regulatory barriers. In addition, a Navy laboratory streamlined foreign license applications using another authority.

The Pilot Programs Were Not Effective for Two Primary Reasons

The low level of implementation of the proposed initiatives occurred for two primary reasons. First, DOD did not develop an effective process for implementing the pilot programs. Second, DOD determined that proposed human capital initiatives—for example, requests for the authority to hire directly or offer voluntary retirement incentives—were in conflict with statutory provisions.

Lack of an Effective Implementation Process

DOD did not provide standardized guidance on proposal requirements or feedback for improving proposals; coordinate or prioritize proposals; or clarify decision-making authority for proposal review and approval. DOD also did not designate a strong focal point to coordinate the pilot programs, advocate process improvements, and provide assistance and advice to participants. The lack of a strong focal point exacerbated other process gaps.

Lack of Guidance and Coordination

According to officials at DOD laboratories, test centers, and headquarters, DOD did not provide standardized guidance on proposal requirements or feedback for improving proposals (or, in many cases, information on the status of proposals submitted for approval). Proposals often lacked specificity and detail. Many were broadly conceptual or generic in nature and lacked a detailed business case that linked their contribution to overall objectives for the pilot programs. For example, a proposal to permit scientists to serve in a leadership role in professional societies failed to include details of the problems encountered, and the potential to improve operations. Similarly, several proposals for direct hire authority failed to include a business case to explain what specific needs this authority would address or how it would address them. Lack of specificity and business case detail led to the failure of many initiatives to win approval. DOD attorneys told us that many proposals were so vague that it was impossible to determine whether or not the proposed initiatives could meet legal requirements.

At a department level, DOD also did not coordinate or prioritize proposals, thereby precluding decisions on how best to pursue common interests and issues such as direct hiring authority or forming partnerships with universities. Instead, each participant submitted proposals individually, and thus multiple independent proposals were often submitted for the same or similar issues. DOD attorneys pointed out that it would have been more effective to group proposals by common theme and prioritize them. They believed a unified approach and prioritized proposals with clearly written, specific plans for solving well-defined problems would have enabled them to more effectively assist participants with resolving legal issues.

Unclear Decision-Making Authority

DOD did not clarify decision-making authority for proposal review and approval. Many organizations and individuals were stakeholders in proposal review and approval, and they often had differing management structures, concerns, and interests. Stakeholders included military and civilian leaders, attorneys, and human capital and personnel staff at several levels: the local installation where participating laboratories and test centers were housed; the individual service; and OSD. The roles and decision-making authority of the various stakeholders were never negotiated and clarified. As a result, many players at multiple organizational levels had—and took—an opportunity to say “no” to a particular proposal, but it remained unclear who had the authority to say “yes.”

For example, some participants believed that the pilot program legislation gave the director of a participating laboratory or test center the authority to approve a proposed initiative. OSD officials, however, believed that the proposed initiatives had to be approved at higher levels. The role of the services was also unclear. Some laboratory and test center directors initially sent proposals directly to OSD’s Directorate of Defense Research and Engineering (DDR&E), bypassing their service headquarters. Others sent proposals to their service headquarters for approval before submitting the proposals to DDR&E. Eventually, however, each of the service headquarters decided to become more heavily involved in the approval process and provide service-level responses to proposals. These service-level responses often came into play after proposals had been sent directly to DDR&E for approval, further complicating the approval process.

Within OSD, both DDR&E and Personnel and Readiness (P&R) had substantial stakes in the human capital proposals—DDR&E because it is charged with oversight and management of defense laboratories and P&R because it has the authority within DOD for human capital issues. However, DDR&E and P&R never agreed on a process for approving proposals. In addition, for the past year P&R’s attention has been focused primarily on developing DOD’s proposed new civilian human capital management system, the National Security Personnel System (NSPS), which the Secretary of Defense recently submitted to the Congress. DOD officials believe that, if enacted, NSPS will provide flexibility to make necessary human capital changes. The Undersecretary of Defense P&R directed that implementation of new personnel initiatives be placed on hold during the development of NSPS so that the existing system could be studied to identify needs and best practices. Consequently, P&R officials believed it would be premature for DOD to implement new personnel initiatives during this time.

No Strong Focal Point

DOD did not designate a strong focal point to coordinate the pilot programs, advocate process improvements, and provide assistance and advice to participants. This exacerbated the other process gaps. Without such a focal point, participants found their own individual ways to develop proposals and get them reviewed. Several officials agreed that a strong focal point would be helpful. For example, DOD attorneys stated that the laboratories or someone acting as their focal point needed to define the issues they wanted to resolve. The attorneys noted that a focal point could have more successfully drawn upon their expertise and experience with addressing legal challenges in other innovative programs (e.g., demonstration projects). Some pilot program participants also agreed a strong focal point was needed, but they had some concerns regarding the amount of influence and authority he or she should have.

Human Capital Proposals Were in Conflict with Existing Statutory Provisions

According to officials at DOD laboratories, test centers, and headquarters, human capital initiatives were generally in conflict with title 5 of the United States Code. Title 5 provides the framework for standard and equitable personnel practices across the federal government and is the current foundation for management of the DOD civilian workforce. Over time, the Office of Personnel Management has added implementing rules and regulations to the framework. Proposed human capital initiatives often sought relief from these provisions, for example, requests for the authority to hire directly or offer voluntary retirement incentives.

However, after reviewing the legislation, the DOD Office of General Counsel advised that the 1999 and 2000 legislation did not provide the authority to waive personnel rules based on title 5 provisions. Rather, the office advised that the pilot programs’ authorities allow only for changes that could already be accomplished under existing DOD regulations. In other words, the pilot programs did not provide any new or additional authority to waive existing personnel rules and regulations grounded in title 5. Consequently, absent statutory authority beyond that provided by the pilot programs, human capital proposals in conflict with title 5 and its implementing rules and regulations could not be implemented. Many initiatives fell into this category.

The 2003 Pilot Program Faces Implementation Challenges

The 2003 pilot program faces several implementation challenges. First, as of May 2003, DOD had not addressed implementation problems. Thus, proposals made via the 2003 pilot program will face the same obstacles as previous proposals.

Second, human capital initiatives will continue to face title 5 challenges. Like the earlier legislation, the 2003 legislation does not provide DOD any new authority. Hence, initiatives proposed under the 2003 pilot program will encounter the same statutory restrictions as previous initiatives. P&R officials believe that, if implemented, NSPS will provide the flexibility to make necessary human capital changes, thereby eliminating the need for the pilot programs in this area. However, NSPS has not yet been enacted, and if enacted, it will still require an implementation process.

Finally, laboratories and test centers may be reluctant to participate in the new pilot program. Many participants in the earlier pilots told us they were discouraged by their experience and consequently unwilling to repeat it. Some expressed frustration with the lack of guidance and feedback on their proposals; others questioned whether management was really committed to the pilot program. Even those few participants that had proposals approved were wary of expending additional resources on another pilot program.

Conclusions

While DOD appears to recognize a need to address human capital and business operations issues specific to laboratories and test centers, it has not effectively managed the pilot programs. If DOD intends to use the pilot programs to address laboratory and test center issues, it will have to address the factors—both process and statutory—that blunted previous proposals made through the pilot programs. The small volume of approved proposals, coupled with DOD’s not providing status reports required by the Congress, has left the Congress uninformed about what objectives DOD would like to achieve with the laboratories and test centers, how it plans to achieve those objectives, and what vehicles it plans to use. This information will be important to the success of any future actions.

Recommendations for Executive Action

We recommend that by March 31, 2004, the Secretary of Defense inform the Congress of DOD’s objectives regarding human capital and business operations in the laboratories and test centers, how it plans to meet these objectives, and what vehicles it will use to meet them.

We also recommend that by March 31, 2004, the Secretary of Defense develop a process for proposing, evaluating, and implementing human capital, business, and partnership initiatives for the laboratories and test centers, regardless whether by the pilot authority or by some other vehicle. Such a process should include instructions for proposal requirements such as linking to overall goals and measurable objectives and the need for a business case, and specification of procedures for proposal submission and review and providing feedback on proposal quality and scope.

Finally, we recommend that the Secretary of Defense designate a strong focal point to receive, evaluate, and prioritize all proposals and work with laboratory and test center directors, legal counsel, personnel and other specialists to develop sound and well-developed business cases and strategies to obtain needed changes.

Agency Comments and Our Evaluation

In written comments on a draft of this report, DOD states that it does not concur with our recommendations because it has already taken actions that in effect implement them. While the actions DOD cites that it has taken are important to implementing our recommendations, they are not sufficiently specific to address the problems identified in our report. DOD’s written comments are contained in appendix II.

Regarding our first recommendation—that DOD inform the Congress of its human capital and business objectives for the laboratories and test centers and the strategies it will employ to meet them—DOD did not concur. DOD discusses various high-level, agencywide initiatives it has taken to address human capital and business issues in general and stated that the Congress has been made aware of these initiatives, obviating the need for additional reporting. We continue to believe that additional reporting is necessary. We recognize that the general initiatives DOD discusses may provide ways of helping the laboratories and test centers; however, to be effective, they must be made specific, that is, developed into targeted strategies and plans that address the particular problems the laboratories and test centers face. DOD has not provided the Congress sufficient details on how the general initiatives will be used to address laboratories’ and test centers’ objectives and problems.

Regarding our second recommendation—that DOD develop a process for proposing, evaluating, and implementing human capital and business-like practices initiatives for the laboratories and test centers—DOD did not concur. DOD states that it has already introduced new agencywide management processes—the Business Initiative Council and the submission of the NSPS proposal to the Congress—to address human capital and business issues in general. However, DOD has not detailed how these general initiatives will apply to the laboratories and test centers or address our process concerns. For example, while the Business Initiative Council may have an effective process for proposing, evaluating, and implementing laboratory and test center business-like practices initiatives, DOD has not provided sufficient information for us to make such a determination. We also recognize that NSPS may address some of the human capital problems faced by the laboratories and test centers, but this system is still under consideration by the Congress. Until it becomes law, we believe it is premature to cite it as an effective management tool.

With regard to our third recommendation—that DOD designate a strong focal point to work with the laboratories and test centers to develop, evaluate, prioritize, and coordinate proposed initiatives—DOD did not concur. DOD states that the recently created position of Undersecretary for Laboratories and Basic Sciences has oversight responsibility for all laboratory initiatives and that it is establishing a new Defense Test Resources Management Center that will oversee the test centers. DOD asserts that these two organizations will perform as focal points. However, DOD has not detailed how these organizations will fulfill this role and work with the laboratories and test centers to overcome the many barriers noted in our report.

Scope and Methodology

During our review, we met with officials from the following organizations in the Office of the Secretary of Defense: the Director, Defense Research and Engineering; the Director, Operational Test and Evaluation; the General Counsel, and the Deputy Undersecretary of Defense for Personnel and Readiness. We also met with officials from the Army Research Laboratory, Aberdeen Test Center, Army Medical Research and Materiel Command, Naval Research Laboratory, Naval Undersea Warfare Center, Air Force Research Laboratory, Air Force Research Laboratory’s Space Vehicles Directorate, and 46th Test Wing. We also discussed pilot program issues with each participating laboratory or center.

To determine the initiatives proposed to date and their status, we obtained records from OSD and service officials. From these records and from discussions with each participant, we compiled a listing of initiatives proposed by each participating laboratory and test center. We verified the listing and the current status of each initiative with cognizant service officials.

To determine what obstacles inhibited DOD’s implementation of the pilot programs, we obtained documentation and data from pilot program participants as well as from OSD officials. We also discussed statutory obstacles with the officials from DOD’s Office of General Counsel and Undersecretary of Defense for Personnel and Readiness. We discussed management and procedural obstacles with officials from the Director, Operational Test and Evaluation and Defense Research and Engineering. In addition, we discussed all obstacles with the participating laboratories and test centers.

The problems facing the laboratories and test centers have been documented by many organizations, panels, and commissions. We did not independently verify these problems or the findings and conclusions of these entities. We conducted our review from July 2002 to April 2003 in accordance with generally accepted government auditing standards.

We are sending copies of this report to the Secretary of Defense; the Secretaries of the Army, Navy, and Air Force; and interested congressional committees. We will also make copies available to others upon request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov.

Major contributors to this report were Catherine Baltzell, Arthur Cobb, Christopher Durbin, Rae Ann Sapp, Sylvia Schatz, and Katrina Taylor. If you have any questions regarding this report, please call me at (202) 512-4841.

Appendix I: Fiscal Year 1999 and 2000 Pilot Program Participants

Appendix II: Comments from the Department of Defense