Federal Research Opportunities: DOE, DOD, and HHS Need Better Guidance for Participant Activities
Released: 2016-01-20T00:00:00

Letter

Background

The purpose of DOE’s contract with ORAU is to provide management and direction of programs through ORISE that maintain and advance science and education capabilities supporting DOE’s strategic goals in the areas of defense, energy, science, and the environment. To support these goals, ORAU carries out a range of activities for DOE, including administering workforce development programs to help ensure the future availability of scientists and engineers. These workforce development programs are intended to encourage individuals to enter STEM careers, complement students’ academic programs, and provide faculty with state- of-the-art information to use in the classroom, as well as developing a pool of talent from which federal agencies can draw for future employment. ORAU groups its workforce development activities into the following three categories:

Research participation program: This program provides research experiences to students, postgraduates, faculty, and other participants. These activities make up the ORISE program.

Fellowships and scholarships: Among other things, these programs provide financial assistance for students to obtain academic degrees in areas related to the sponsoring agency’s mission.

Events, academies, and competitions: These programs, such as the National Science Bowl, a nationwide middle- and high-school science and mathematics competition, are designed to encourage participation in scientific and technological fields.

In fiscal year 2014, most of the federal agency expenditures on workforce development activities administered by ORAU were for the ORISE program. In that year, federal agencies expended $193.8 million on the ORISE program, followed by $3.2 million for fellowships and scholarships, and $1.7 million for events, academies, and competitions. These expenditures supported 5,854 research participation appointments, 72 scholarships and fellowships, and 1,191 special event participants.

ORISE research participants engage in a variety of subject areas, such as climate change, weather impacts on military projects, infectious and chronic diseases, computer simulations of potential terrorist attacks or natural disasters, and conservation measures for fish and wildlife. (See app.I for additional examples, by sponsoring agency, of project subject areas in fiscal year 2014.) Research participants may also be involved in developing briefing materials on their research for agency leadership, publishing the results of their research, participating in conferences, and obtaining other research-related experiences. Research participants are not considered federal employees or federal contractors and do not receive a salary. They instead receive stipends and certain other expenses to defray their costs of living during their appointments. Participant appointments can be full- or part-time and can last for weeks, such as in the case of a 10- to 12-week summer program, or years, such as in the case of a 1-year postgraduate program renewable for up to 4 additional years.

Agency Expenditures for the ORISE Program Increased from 2010 through 2014, with Stipends Making Up the Largest Portion of Expenses

From fiscal year 2010 through fiscal year 2014, DOE and other sponsoring agencies expended a total of $776.4 million for the ORISE program, with DOD, DOE, and HHS accounting for the majority of the expenditures. Over that period, annual program expenditures increased by 73 percent, and the number of annual appointments rose by 42 percent. Stipends accounted for the largest portion of agencies’ expenditures. Sponsoring agency expenditures per appointment varied, affected by factors such as the length of research participants’ appointments and the program support services sponsoring agencies had ORAU perform.

DOD, DOE, and HHS Expended the Most for the ORISE Program from 2010 through 2014, as Overall Expenditures and Participation Increased

During fiscal years 2010 through 2014, sponsoring agencies, which included 11 departments and other federal agencies, expended a total of $776.4 million for the ORISE program. DOD, HHS, and DOE collectively had the highest expenditures for the program (over 87 percent) over that period and had the highest number of appointments in fiscal year 2014 (over 88 percent). Within DOD, the Army was the primary component that sponsored ORISE research participants, accounting for 77 percent of DOD expenditures over the 5-year period and 70 percent of appointments in fiscal year 2014. Within HHS over the same time periods, the Food and Drug Administration (FDA) and Centers for Disease Control and Prevention (CDC) accounted for about 59 percent and 32 percent of expenditures, respectively, and about 53 percent and 36 percent of appointments. See figure 1 below and appendix II for further information on agencies’ expenditures and numbers of appointments.

Sponsoring agencies’ total annual expenditures increased from $112.3 million in fiscal year 2010 to $193.8 million in fiscal year 2014, a 73 percent increase (61 percent when adjusted for inflation), and the number of appointments grew from 4,128 to 5,854, a 42 percent increase (see fig. 2 below and app. II for further information).

An ORAU official who maintains data on appointments attributed the growth in the number of appointments to an increase in the program’s popularity, which led to the addition of new sponsoring agencies and increases in the number of appointments per sponsoring agency. Agency component officials we interviewed cited a variety of reasons for wanting to sponsor ORISE research participants, including: access to the ORISE program’s recruiters and network of connections administrative support from the ORISE program that the sponsoring agencies could not easily supply themselves, and the speed, flexibility, and relatively low overhead cost of the ORISE program.

For example, an official who managed the research participation program at the U.S. Army Medical Research Institute of Infectious Diseases told us that the cost of hiring and managing staff to administer their own program would cost more than the overhead that they pay for the ORISE program.

The average total expenditure per appointment in the ORISE program also increased from fiscal year 2010 through fiscal year 2014, from about $27,200 per appointment in fiscal year 2010 to about $33,100 per appointment in fiscal year 2014. Expenditures per appointment may have risen for a variety of reasons, such as changes in the average education level of research participants and the average length of their appointments. For example, the proportions of appointments at different education levels in fiscal year 2014 shifted compared to the proportions in fiscal year 2010, with recent graduate and postdoctoral appointments increasing 65 percent and 68 percent, respectively, while undergraduate appointments increased 12 percent. An ORAU official said that postgraduate appointments generally command higher stipends than undergraduate appointments. For example, according to information provided by FDA’s Center for Drug Evaluation and Research, monthly stipends at their center could be as high as $2,897 for currently enrolled undergraduate students and as high as $7,569 for postgraduates with PhD degrees. The official said that postgraduate appointments at their center also last longer than undergraduate appointments, resulting in higher expenditures per appointment.

Most Expenses Were for Stipends, and Several Factors Contributed to Variations in Agencies’ Expenditures per Appointment

From fiscal year 2010 through fiscal year 2014, stipends—funds paid to research participants to defray their costs of living during their appointments—comprised the majority of agencies’ expenditures for the ORISE program. Sponsoring agencies’ other expenditures for the program included the following categories of expenses:

Travel and other research participant expenses: Funds paid to research participants to cover particular expenses not covered by their stipends, such as expenses for travel to conferences or other appointment-related destinations.

Program support and overhead: Funds paid to DOE to cover ORAU’s expenses for administering the appointment of research participants at agencies. These expenses included (1) program support expenses—direct expenses for services ORAU provides to agencies, such as managing recruitment activities, and (2) general and administrative expenses—indirect expenses such as building expenses, paid by agencies as a fixed percentage (negotiated by DOE and ORAU) of total expenditures on the ORISE program.

Federal administrative and security charges: Fees paid to DOE by other sponsoring agencies, including (1) a federal administrative charge of 3 percent of an agency’s total expenditures on the ORISE program to offset DOE’s administrative expenses for work conducted on behalf of other agencies and (2) a charge applied to Strategic Partnership Projects to supplement DOE support for safeguards and security expenses.

Figure 3 shows the percentage of expenditures for each category of expense.

Sponsoring agencies’ expenditures per appointment in the ORISE program varied among agencies. In each year from fiscal year 2010 through fiscal year 2014, the lowest average expenditure per appointment for a sponsoring agency was $14,396 or less, and the highest average expenditure per appointment was $42,996 or more. For example, in fiscal year 2014, the Department of the Interior expended an average of $12,246 per appointment, while the Environmental Protection Agency expended an average of $44,099 per appointment. The proportions expended for different categories of expenses also varied. For example, data provided by ORAU showed that the proportion agencies expended on stipends ranged from 69 percent to 88 percent in fiscal year 2014.

We identified the following factors that contributed to per-appointment expenditures varying among agencies:

Research participants’ appointment terms. Differences in the terms that sponsoring agency components set for research participants’ appointments contributed to variation in expenditures per appointment. Some appointments lasted for days, weeks, or months, while others lasted for a full year or more. For example, FDA’s National Center for Toxicological Research’s Summer Student Research Program placed research participants in a 10-week summer program. In contrast, the National Library of Medicine’s Associate Fellowship Program placed research participants in 1- or 2-year residency programs. In addition, some appointments were full-time, while others were part-time.

Methods of setting stipends. Officials at sponsoring agency components reported that they used differing methods to set research participants’ stipends. ORAU officials said that they sometimes provided advice to the agencies, but that the agencies ultimately set their own stipends. Almost all of the officials we interviewed at sponsoring agency components said that they considered applicants’ education levels when setting stipends, but they varied in the other factors they considered. For example, some used the Office of Personnel Management’s General Schedule pay scale, but others did not. The officials also differed in the extent to which they considered other factors, including prior work experience, salaries in the private and government sectors, stipends received by research participants in other programs, and geographic location. Some of the officials said that they set fixed stipends for all research participants, but others said that they determined stipends individually or made exceptions to fixed stipends when attempting to fill particular appointments.

Other expenses covered. Sponsoring agency components chose to reimburse their research participants for different types and amounts of expenses not covered by their stipends. For example, in fiscal year 2014, the Air Force expended an average of $492 per appointment to pay for participants’ travel expenses, while the Environmental Protection Agency expended an average of $1,387 per appointment for that purpose. Other expenses that can vary among sponsoring agency components include payment of research participants’ tuition and fees at their academic institutions; reimbursement for the costs of moving to a research site; allowances for housing at research sites; payment of visa processing fees for foreign research participants; and purchase of safety equipment, books, and research supplies.

Services performed by ORAU. Sponsoring agency components selected from and paid for many different services performed by ORAU. These services included managing recruitment activities, processing applications, making and monitoring appointments, designing and implementing program enhancements, paying stipends, administering other research participant expenses and insurance, managing domestic and foreign travel, analyzing and providing financial reports, developing and administering program goals and objectives, handling immigration status issues, and other tasks. Agencies’ selections of these services determined the amount that they paid in program support expenses for each of their appointments.

Assessment of the ORISE Program and Guidance on Inherently Governmental Functions Varies Among Sponsoring Agency Components

According to DOE officials, the ORISE program consists of a set of distinct activities, or separate programs, that ORAU carries out on behalf of DOE and other sponsoring agency components. As a result, DOE considers responsibility for assessing the effectiveness of ORISE program activities to be dispersed among the sponsoring agencies, each of which may have separate objectives for sponsoring research participants. Sponsoring agency components we reviewed use questionnaires and other methods to assess how well the program is working. Responsibility for ensuring research participants do not perform inherently governmental functions is also dispersed among sponsoring agencies. However, documents provided by DOE, DOD, and HHS components to research participants, coordinators, and mentors contain varying levels of detail on the prohibition on nonfederal employees performing inherently governmental functions. Without detailed guidance, sponsoring agencies have limited assurance that the prohibition is being followed.

Sponsoring Agency Components Chose Whether and How to Assess the ORISE Program

In May 2013, the National Science and Technology Council, which coordinates executive branch science and technology policy, released its 5-year strategic plan for STEM education, which stated that federal agencies would focus on building and using evidence-based approaches to evaluate the federal investment in STEM education. DOE officials told us that, because the ORISE program consists of separate activities that ORAU carries out on behalf of DOE and other sponsoring agencies, these agencies choose whether to assess the effectiveness of ORISE program activities as part of their other investments in STEM education. As a result, other than periodically evaluating ORAU’s performance (with input from sponsoring agencies) under its contract to determine ORAU’s award fee, DOE does not assess the overall effectiveness of the activities that ORAU carries out under the ORISE program, according to a DOE official. For example, the official said DOE does not assess how ORISE program activities at other sponsoring agencies contribute to the ORISE program’s objective to enhance the quantity, quality, and diversity of the future scientific and engineering workforce and to increase the scientific and technical literacy of the U.S. citizenry.

Sponsoring agency components establish their own objectives for sponsoring research participants and decide whether and how to assess the extent to which the ORISE program meets those objectives, according to DOE officials. Some but not all DOE, DOD, and HHS components have used questionnaires, and some components have used other methods to assess how well the ORISE program is working in the short term, such as over the course of a research participant’s appointment. In particular, some components use questionnaires developed with assistance from ORAU and administered to research participants, and sometimes to mentors. ORISE program coordinators and other officials at sponsoring agency components described other methods they use to assess the program, such as asking research participants about their experiences and monitoring the progress of research participants’ research projects, research participants’ publications and presentations related to their research, and the number of current agency employees who were past ORISE research participants. In addition, one of the program support functions that ORAU can offer to sponsoring agencies at the cost of the service is performing an assessment of ORISE program effectiveness. In response to a request from the sponsoring agency component, ORAU performed such an assessment for the Joint Prisoner of War/Missing in Action Accounting Command and issued a report in August 2014.

The methods being used by the sponsoring components we reviewed assess how well the program is meeting the short-term needs of research participants and mentors. For example, some research participant questionnaires included questions about research participants’ satisfaction with their assignment, training, mentoring, stipends, and program administration. DOD mentor questionnaires include questions on reasons for renewing a research participant’s appointment and research participants’ skills and knowledge.

A DOE Office of Science official told us that DOE is working with other agencies to develop methods for assessing the long-term outcomes of STEM education efforts, such as the ORISE program increasing the diversity of the STEM workforce. The official noted that, without such methods, they face challenges in assessing the long-term effectiveness of the ORISE program. For example, according to the official, such challenges include developing methods to track research participants over the course of their careers and determining the extent to which a participant’s degree of success in a STEM field is a result of the ORISE program as opposed to other educational experiences.

Documents Used by DOE, DOD, and HHS for Research Participants and Mentors Provide Varying Levels of Detail on Inherently Governmental Functions

In 2011, OMB’s Office of Federal Procurement Policy issued guidance to assist agency officers and employees in ensuring that only federal employees perform work that is inherently governmental or otherwise needs to be reserved to the public sector. This guidance directs agencies to develop and maintain internal procedures; take appropriate steps to help employees understand and meet their responsibilities; and periodically evaluate the effectiveness of their internal management controls for reserving work for federal employees. In accordance with OMB’s guidance, agencies that sponsor ORISE research participants are responsible for ensuring that research participants at their agencies do not perform inherently governmental functions.

Documents we reviewed that are issued by DOE, DOD, and HHS, regarding research participants at their agencies, and that are used by sponsoring agency components’ coordinators, mentors, and research participants varied in level of detail on activities considered inherently governmental functions. For example, within HHS, ORISE program handbooks from FDA’s Center for Veterinary Medicine and Center of Drug Evaluation and Research included examples of activities research participants should not perform, such as serving as a drug, device, safety, or facilities reviewer. Similarly, within DOD, the research participant appointment letters used by the Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics included detailed guidance, such as a statement that research participants should not accept policy, budget, or program management authority. In contrast, sample appointment letters that we reviewed used by HHS’s CDC and DOD’s U.S. Army Environmental Command stated only that the research participant will not enter into an employee/employer relationship with ORISE, DOE, or any other office or agency, and did not specifically cite the inherently governmental activities prohibition. The terms of appointment developed by ORAU and used by DOE, DOD, and HHS to make research participant appointments states only that the appointment is an educational experience and not a contract of employment.

Statements of work agreed upon as part of interagency agreements between DOE and sponsoring agencies also varied in their level of detail about activities considered to be inherently governmental functions. For example, a statement of work for the CDC stated that ORISE research participation projects should not include activities reserved for federal employees, such as those involving budget or program management authority. In contrast, a statement of work for the National Institutes of Health did not include this level of detail, stating only that individuals selected for appointments do not become employees.

DOE and other sponsoring agency officials noted that ORISE research participants are assigned to research projects that generally do not involve inherently governmental functions. A DOE Office of Science official said that the research focus of most ORISE appointments reduced the risk of those research participants performing inherently governmental functions. However, GAO found that some research participants’ projects involve activities that are closely associated with inherently governmental functions, such as participating in policy and strategic planning meetings, which may increase the risk of the participants performing inherently governmental functions. The DOE Office of Science official described how, in such cases, DOE provided more detailed briefings on inherently governmental functions for certain research participants, as well as briefings for their mentors. However, officials at other sponsoring agency components we interviewed did not describe providing such briefings as a standard practice for coordinators, mentors, or research participants. For example, the position description for a research participant in the DOD Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics included participating in policy and strategic planning meetings, but officials at this DOD component did not describe providing briefings on inherently governmental functions, increasing the importance of written guidance.

Not having detailed guidance increases the risk that coordinators responsible for managing overall participation in the program and mentors responsible for directing research participants’ day-to-day activities may overlook the possibility of research participants engaging in inherently governmental functions, especially in cases where participants’ activities are closely associated with inherently governmental functions. Development of detailed guidance could help sponsoring agencies fulfill their responsibilities as identified in OMB’s Office of Federal Procurement Policy guidance on inherently governmental functions.

Conclusions

By providing hands-on research experiences in government agencies for students, postgraduates, and faculty, the ORISE research participation program makes an important contribution to federal efforts to help prepare students and teachers for careers in STEM fields. Responsibility for administering the program is dispersed among agencies that sponsor research participants. In particular, agencies are responsible for ensuring that research participants do not perform inherently governmental functions—for example, by developing guidance and other documents for research participants, coordinators, and mentors. Having this responsibility allows agencies to tailor guidance on inherently governmental functions to the features of the ORISE program at their agencies, such as the types of projects to which research participants are assigned. However, the level of detail in documents currently used by DOE, DOD, and HHS varies, with some documents describing specific types of activities that are inherently governmental functions and others only providing general statements that research participants are not federal government employees. More detailed guidance can help ORISE coordinators, mentors, and research participants ensure that they are adhering to the prohibition on research participants as nonfederal government employees performing inherently government functions.

Recommendation for Executive Action

We recommend that the Secretaries of Energy, Defense, and Health and Human Services develop detailed guidance to ensure that ORISE program coordinators, mentors, and research participants are fully informed of the prohibition on nonfederal employees performing inherently governmental functions.

Agency Comments and Our Evaluation

We provided a draft of this report to DOE, DOD, and HHS for their review and comment. In their written comments, reproduced in appendices III through V, DOE, DOD and HHS concurred with our recommendation. DOE and HHS also provided technical comments, which we incorporated as appropriate.

In their written comments, DOE, DOD, and HHS described the measures they will take to implement our recommendation on inherently governmental functions. In particular, DOE stated that it plans to provide detailed guidance to all relevant parties involved in DOE-sponsored research participation activities administered through ORISE within 180 days, following consultation with relevant DOE offices. DOD stated detailed guidance will be developed to further ensure those connected with the ORISE program are fully informed of the prohibition on non- federal employees performing inherently governmental functions. HHS stated that they are developing an agency-wide policy, including a section on inherently governmental functions that will provide guidance to agency program coordinators, mentors, and research participants.

In its letter and technical comments, DOE stated that the draft report did not reflect detailed discussions we had with DOE officials regarding inherently governmental functions. In addition, DOE stated that the draft report significantly understated the extent to which DOE communicates the prohibition of inherently governmental functions to sponsored participants and agency mentors. We do not believe our report understates DOE’s efforts. For example, our report includes a discussion of the detailed briefings that DOE Office of Science officials provide on inherently governmental functions to research participants selected for a program designed to expose the participants to federal policymaking. Other DOE, DOD, and HHS sponsoring agency components we interviewed did not describe a similar practice for their coordinators, mentors, or research participants. Our report’s discussion of these briefings, as well as of documents issued by DOE for the ORISE program, reflect the extent of communications on inherently governmental functions that DOE provided to us.

We are sending copies of this report to the appropriate congressional committees; the Secretaries of Energy, Defense, and Health and Human Services; and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff members have any questions about this report, please contact me at (202) 512-3841 or neumannj@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix VI.

Appendix I: Examples of Subject Areas for ORISE Research Participant Projects in Fiscal Year 2014

Appendix I: Examples of Subject Areas for ORISE Research Participant Projects in Fiscal Year 2014 Examples of subject areas Improving prevention and treatments of emerging foreign animal diseases, climate change impacts on forests, porcine epidemic diarrhea virus, and sensor networks on variable rate irrigation systems.

Laser systems, nanomaterials, acoustics, neurobiology, additive manufacturing, civil engineering, cognitive modeling, intelligent sensors, exercise science, and visual analytics.

Weather impacts on military projects, cognitive function and psychological performance of soldiers, health promotion and wellness, environmental medicine, improving military field equipment, and science and technology policy.

Infectious disease or deployment health surveillance, clinical and health care epidemiology, optical coherence tomography, and aeromedical studies.

Forensic sciences, human immunodeficiency virus/acquired immune deficiency syndrome prevention, public health and preventive medicine studies, and science and technology policy.

Neutron scattering, fusion energy, efficiency of renewable energy sources, computational sciences, materials sciences, process controls of advanced power systems, gas sensors and high temperatures, improving extraction of earth elements, quantum computing, biofilms and biotechnology, advanced manufacturing (carbon fiber), climate change, and science and technology policy.

Infectious diseases (e.g., influenza, sexually transmitted, food borne, vector borne, respiratory), chronic diseases (e.g., heart, obesity, cancer), environmental health, toxic substances, health statistics, and public health preparedness.

Toxicology, food safety, drug evaluation and testing, biological therapeutics, tobacco products, blood products, medical devices, biotechnology products, translational sciences, women’s health, vaccines, cell and gene therapies, and regulatory science.

Localization of proteins using molecular markers, gene regulatory effects in cancer, medical informatics, and central nervous system injuries.

Public health economics, population based model testing, clinical care models, minority health, women’s health, tobacco prevention initiatives, national human immunodeficiency virus/acquired immune deficiency syndrome strategies, and geospatial analysis of underserved populations.

Encryption for criminal databases, improving materials for coastal bridges, computer simulations of potential terrorist attacks or natural disasters, brain-like modeling systems, searchable databases of potential threats, human trafficking, and detecting and identifying explosive-related threats.

Data analysis of housing and urban development impacts and value on communities.

Department of the Interior Data collection and surveys related to conservation measures for fish and wildlife.

Clean energy and climate change policy and analyses in the international economy, and building efficiencies.

Climate change, software codes for aerial sampling systems, urban ecosystems, nanoparticles and surface coating, waste disposal, safety of water supply, and biomarkers for environmental contaminants.

Juvenile prostitution and child abduction, causes of postmortem hair root banding, forensic applications of isotopes, crimes against adults, and identification of facial phenotypic markers.

Includes Office of the Secretary, Office of Diversity Management and Equal Opportunity, National Geospatial-Intelligence Agency, Defense Threat Reduction Agency, Defense Prisoner of War/Missing in Action Accounting Command, and U.S. Southern Command. Includes Office of the Secretary and Health Resources and Services Administration, and Center for Medicare and Medicaid Innovation.

Appendix II: Federal Expenditures for and Appointments in the ORISE Research Participation Program

The following tables detail federal agencies’ expenditures for and research participant appointments they sponsored as part of the Oak Ridge Institute for Science and Education (ORISE) research participation program for fiscal years 2010 through 2014. Table 1 identifies agencies’ total annual expenditures for their involvement in the ORISE program. Table 2 identifies the numbers of appointments at each agency for each year. Table 3 details each agency’s total expenditures for the ORISE program for fiscal years 2010 through 2014 by type of expense, including stipends, travel, other research participant expenses, program support and overhead, and federal administrative and security charges. In each table, the three agencies that account for the largest share of expenditures and appointments—the Department of Health and Human Services (HHS), the Department of Defense (DOD), and the Department of Energy (DOE)—are broken out into component agencies that sponsored research participants.

Appendix III: Comments from the Department of Energy

Appendix IV: Comments from the Department of Defense

Appendix V: Comments from the Department of Health and Human Services

Appendix VI: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the individual named above, Joseph Cook (Assistant Director), Sherri Doughty, Ellen Fried, Tobias Gillett, Kirsten Lauber, Gerald Leverich, Cynthia C. Norris, Stephanie Shipman, Kathryn Smith, Jeanette Soares, Sara Sullivan, and Thema Willette made key contributions to this report.