Military Education: Improved Oversight and Management Needed for DOD's Fellowship and Training-with-Industry Programs
Published: 2012-04-20T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Department of Defense (DOD), which includes the military services, selects mid- to upper-career-level military officers to participate in fellowship and training-with-industry programs conducted at non-DOD organizations such as universities, think tanks, private corporations, federal agencies, and Congress. For some fellowships, the military departments pay a fee or tuition to the host organization. GAO was directed to review DODs use of these programs. GAOs objectives were to determine: (1) the statutory provisions that authorize DODs fellowship and training-with-industry programs for military officers, (2) the extent of the Office of the Secretary of Defenses (OSD) visibility over these programs, and (3) the extent to which the services are able to determine that they derive benefits from these programs. GAO analyzed relevant laws and DOD policies, collected data, and interviewed OSD and military service officials on their oversight and management roles and responsibilities for these programs.
section_title: What GAO Found
paragraphs: GAO determined that DOD primarily uses two explicit statutory authoritiessection 2603 of Title 10 of the United States Code, which authorizes servicemembers to accept fellowships from certain organizations, and section 2013 of Title 10 of the United States Code, which authorizes the training of servicemembers at nongovernmental facilitiesfor its fellowships and training-with-industry programs for military officers. For two specific types of fellowshipsLegislative and Interagencythe underlying authorities are less explicit than they are for the others.
OSD has limited visibility over its fellowship and training-with-industry programs for several reasons. First, OSD has not developed a mission statement that would clearly define the respective key purposes for these programs. Having a clear mission statement is critical because it defines an organizations purpose in language that states desired outcomes. Additionally, OSD has not consistently enforced its requirement for the military departments to provide an annual report on fellowship and training-with-industry programs. Further, not all fellowship and training-with-industry programs have a designated office within each department for preparing the annual report. OSDs visibility is also limited by not having a reliable inventory of these various programs, and by not having a clear and commonly shared definition of a fellowship. Without improved oversight, OSDs visibility over the military departments compliance with its requirements governing these programs will remain limited. Additionally, visibility is limited over the legislative fellowship program in particular because oversight responsibilities are not clearly delineated, and because OSD does not have documented criteria for the placement of DOD fellows with the offices of congressional committees and members. OSD officials agree that such criteria would be helpful since it does not have enough available fellows to meet the full congressional demand.
The military services are not well positioned to determine the extent of the benefits they are deriving from their participation in these programs for four principal reasons. First, not all of the services conduct periodic program reviews, as are required for some programs. In addition, the reviews that are conducted are not comprehensive in that they do not assess the program against program goals using quantifiable performance measures, review the needs that prompted the program, incorporate feedback from fellows into the review, or document the results of the review. Second, they do not have clear guidance as to what qualifies as a postfellowship assignmentan assignment that uses the skills and knowledge developed during the fellowship programor criteria for when such assignments can be postponed or waived, thus limiting the extent the services are able to determine they are deriving benefits from these programs. Third, the services do not know their overall program costs, so it is difficult to know whether these programs are cost-effective. Finally, some of the services do not have memoranda of understanding with the non-DOD host organizations, such as think tanks, so they cannot be assured that expectations are clearly understood and the intended benefits are obtained. Without better management controls, the services ability to determine the benefits of these programs will remain limited. However, service officials believe that they obtain benefits from fellowships and training-with-industry programs.
section_title: What GAO Recommends
paragraphs: GAO is making 11 recommendations to DOD for improving oversight and management of DODs fellowship and training-with-industry programsfor example, submitting DOD-required annual reports and performing service-required program reviewsthat would enhance OSDs visibility over the programs and better position DOD to determine the extent to which it derives benefits from them. In response to a draft of this report, DOD concurred with the 11 recommendations and stated its action plan to implement the recommendations.

section_title: Letter
paragraphs: The Department of Defense (DOD), which includes the military services,  selects mid- to upper-career-level military officers to participate in  fellowship and training-with-industry programs at non-DOD organizations  such as universities, think tanks, private corporations, federal agencies,  and Congress. For some fellowships, the military departments pay a fee  or tuition to the host organization. According to DOD, the department’s  intent for its fellowship program is to help fulfill a present need,  anticipated requirement, or future capability that contributes to the  effectiveness of the participating Military Department and DOD.  Congressional interest in these programs has grown, in part due to an  increase in the number of military officers participating in one particular   We were  program—the Legislative Fellowship—starting in 2008.directed, in the Senate report accompanying a bill for the National  Defense Authorization Act for Fiscal Year 2011, to review DOD  educational and legislative fellowships as well as training-with-industry  programs and to report to the congressional defense committees.  Accordingly, we examined these programs to determine: (1) the statutory  provisions that authorize DOD’s fellowship and training-with-industry  programs for military officers, (2) the extent of the Office of the Secretary  of Defense’s (OSD) visibility over these programs, and (3) the extent to  which the military services are able to determine that they derive benefits  from these programs.
To address our first objective, we researched and analyzed relevant legal  authorities for DOD to use fellowship and training-with-industry programs,  reviewed DOD and service regulations and guidance related to these  programs, and interviewed relevant DOD attorneys and received written  responses to questions on DOD’s use of these authorities. For our  second objective, we collected and analyzed available service data on the  numbers and types of fellowships and training-with-industry programs  offered from fiscal years 2007 through 2011; interviewed officials in OSD,  particularly with the offices of the Under Secretary of Defense for  Personnel and Readiness (USD P&R), and the Assistant Secretary of  Defense for Legislative Affairs; and collected and reviewed DOD-required  service annual reports. For the third objective, we interviewed officials  within the four military services and reviewed DOD and service guidance  related to these programs. Most of our interviews centered on obtaining  information about roles and responsibilities related to overseeing and  managing fellowship and training-with-industry programs, and  particularly on activities that provide means of ensuring that DOD  received benefits from these programs. More specifically, we collected  information on service program goals, program reviews, costs of the  programs, and memoranda of understanding between the services and  host organizations. We found the fellowship and training-with-industry  data provided to us by the services on the number of participants in these  programs to be sufficiently reliable regarding contextual information on  the minimum number of participants; however, these data do not allow us  to provide actual totals. We conducted this performance review from  March 2011 through April 2012 in accordance with generally accepted  government auditing standards. Those standards require that we plan  and perform the audit to obtain sufficient, appropriate evidence to provide  a reasonable basis for our findings and conclusions based on our audit  objectives. We believe that the evidence obtained provides a reasonable  basis for our findings and conclusions based on our audit objectives.   (See app. I for a more detailed description of our scope and  methodology.)
subsections: section_title: Background
paragraphs: DOD’s fellowship and training-with-industry programs comprise a few of  the several vehicles DOD uses to provide training and education to its  military officers to develop and advance their careers while in the military.  Officers have educational opportunities throughout their careers, both  within and outside of the military educational system. These include  attendance at intermediate- and senior-level military schools, as well as  participation in graduate programs to obtain advanced degrees.  Fellowship and training-with-industry programs constitute another type of  professional development that officers may receive in addition to or in lieu  of attendance at a military school.
subsections: section_title: Fellowship and Training- with-Industry Programs
paragraphs: To provide guidance to the services on participation in a fellowship or  training-with-industry program, USD P&R issued DOD Instruction   Under this instruction, a fellowship is defined as an assignment  1322.06.in which selected DOD personnel work away from DOD to gain education  or experience of value to both the DOD component and the gaining  organization. Military officers can participate in a variety of types of  fellowships at host organizations such as universities, think tanks,  corporations, federal agencies, and congressional committees or member  offices. Most of these fellowships are limited to a duration of no longer  than 12 months. The following types of organizations can host fellowship  and training-with-industry participants:
Universities. For example, Tufts University, a private university in  Massachusetts, hosts fellows from the military services within the  International Security Studies Program at its Fletcher School, Tufts’  graduate school in international affairs. Fellows at the Fletcher School  focus on research and writing requirements and participate in  activities both inside and outside of the classroom. The Marine Corps  Fellows may receive a master’s degree through this program.
Think tanks. For example, the Brookings Institution, a nonprofit think  tank based in Washington, D.C., has hosted fellows under its 21st  Century Defense Initiative—one of Brookings’ research projects.
This initiative focuses on the future of war, U.S. defense needs and  priorities, and the defense system. Fellows from the military services  spend a year at Brookings researching and publishing on defense  topics and provide policy feedback to help craft realistic, applicable  policy recommendations for the military services. Fellows at Brookings  may also elect to attend courses to obtain an executive certificate in  public leadership, for an additional fee.
Corporations. For example, as part of the Secretary of Defense  Corporate Fellowship Program, these fellows are trained at the  executive level to learn how the host corporations use innovative  practices and technology to plan, organize, and manage, and how  these business practices could be applied within the military services.  The fellows shadow corporate executives and write a paper on their  observations and experiences upon completion of the fellowship.
Federal agencies. For example, the United States Agency for  International Development is one of the federal agencies that host  fellows from the Army Interagency Fellowship program and from the  Commandant of the Marine Corps Fellowships program. Federal  agency fellows are placed with departments or agencies to develop a  more thorough understanding of the host agency’s mission, culture,  capabilities, and procedures. The Army, Marine Corps, and Air Force  currently send fellows to other federal departments or agencies.
Congressional committees or members’ offices, as part of DOD’s  Legislative Fellowship program. This program offers officers an  educational opportunity to learn more about the legislative process by  being placed with various committees, or with the staff of the House or  Senate Majority or Minority Leader, or with the staff of the Speaker of  the House.George Washington University to obtain Masters’ degrees in  Legislative Affairs. The Navy, Marine Corps, and Air Force offer their  officers the opportunity to take courses for credit or to receive a   In addition, the Army sends its legislative fellows to  certificate in Legislative Affairs at the Georgetown’s Government  Affairs Institute.
Fellows pursue individual research and writing projects in consultation  with host organization experts or faculty, and they may elect to become  involved in host organization projects and participate in conferences,  seminars, or classes. In addition, fellows share operational and service  insights with host organization staff. Some university and think tank host  organizations charge the military departments a fee or tuition for hosting  fellows.
In addition to fellowship programs, DOD offers training-with-industry  programs. The DOD instruction defines training-with-industry as a  nondegree-producing program designed to provide training or skills in  best business procedures and practices not available through existing  military or advanced civilian schooling for identifiable DOD requirements.  The instruction states that there must be an existing need or desired  future capability fulfilled by the gained training-with-industry experience.  The military departments generally do not pay corporations a fee or tuition  to host training-with-industry participants.
DOD continues to provide officers participating in both fellowships and  training-with-industry programs their normal pay and allowances.   In exchange, after their participation, officers participating in most of the  fellowships discussed in this report incur an active duty service  commitment of three times the length of the term they spent in the  fellowship or training-with-industry program.
section_title: Professional Military Education
paragraphs: Within DOD’s professional military educational system, officers are  afforded several key opportunities. At an officer level of O3, selected  officers could attend primary level education, generally at the Branch,  warfare, or staff specialty schools. At the level of O4, selected officers  attend intermediate level education, generally at the service’s command  and staff college or an equivalent military school. At the level of O5 or   selected officers attend senior-level education, generally at the  O6,service’s war college or at an equivalent military school. Officers may  earn graduate degrees upon completing their intermediate- or senior-level  education at a military school. They additionally have other opportunities  throughout their careers to earn graduate degrees; for example, officers  may apply to attend a college or university to receive a master’s or PhD  degree in a graduate educational program.
Military officers may participate in fellowships as either an alternative or a  supplement to their professional military education, as shown in figure 1.  Three of the services—the Army, the Marine Corps, and the Air Force— grant professional military educational credit for some of their fellowship  programs at the intermediate or senior level. The Air Force, for example,  grants professional military education credit for all of its intermediate- and  senior-level fellowships. These fellowships replace an officer’s   in-residence intermediate or senior professional military education.
One service program—the Army Legislative Fellowship Program—grants  a degree.
When fellowship programs are provided in lieu of attendance at a military  school, the participants are generally prohibited from also attending the  corresponding in-residence military school program. For example, a  fellow who attends a senior-level fellowship and is granted senior-level  professional military education credits is prohibited from attending a  senior in-resident War College program. Unlike most fellowships, DOD’s  training-with-industry programs exist outside of the professional military  educational system. These opportunities do not grant professional military  education credits; instead, they are considered a supplemental  professional development and training experience in private sector  practices that are not available through existing military education and  training programs.
section_title: Roles, Responsibilities, and Guidance
paragraphs: USD P&R has overall responsibility for fellowship and training-with- industry policy for fellowship programs that fall under DOD Instruction  1322.06. This Instruction covers most of the fellowship and training- with-industry programs in our review. As specified in the instruction,  each military departmental secretary has oversight and management  responsibilities for that department’s programs, with the exception of the  Legislative Fellowship Program. For that program, each department  selects nominees, whose names are then sent to USD P&R for   The Office of the Assistant Secretary of Defense for  approval.Legislative Affairs places these officers with congressional committees or  with various congressional staffs. Additionally, some of DOD’s fellowship  programs are administered separately; for example, the Secretary of  Defense Corporate Fellowship Program and the Medical Professional  Fellowship Program are administered under two other, separate DOD  instructions, with different OSD offices having oversight and policy  responsibilities for them. The Secretary of Defense Corporate Fellowship  Program is managed by a director who is part of the National Defense  University, under the authority, direction, and control of DOD’s Deputy  Chief Management Officer, subject to certain exceptions. The Assistant  Secretary of Defense for Health Affairs, under USD P&R, is responsible  for monitoring compliance with DOD Instruction 6000.13 and providing  additional guidance for medical fellowships, and the secretaries of the  military departments are responsible for administering these programs.
DOD Instruction 1322.06, Fellowships, Scholarships, Training-with-Industry (TWI), and  Grants for DOD Personnel (Nov. 15, 2007). assignments. A fellow at a host organization with a cyber or nuclear focus  may be assigned to a utilization tour involving the development of  strategies for cyber or nuclear warfare. Training-with-industry participants  are generally used in positions directly related to a corporation’s area of  expertise. For example, a training-with-industry participant at a private  company that specializes in logistics and the transportation of items may  be later assigned to a DOD position in supply chain management.
The military departments are responsible for the management of their  respective fellowship and training-with-industry programs (see app. II on  the life-cycle phases of a fellowship program for a more detailed  description). In addition to statutory authorities and the DOD instruction,  the military services also have their own guidance or regulations covering  their respective legislative fellowship, nonlegislative fellowship, and  training-with-industry programs (see app. III for a list of service guidance).
section_title: DOD Uses Several Statutory Authorities for Fellowship and Training-with-Industry Programs
paragraphs: We determined that DOD uses several statutory authorities for its  fellowships and training-with-industry programs for military officers.  Primarily, it uses two explicit statutory authorities. However, for two types  of fellowships—legislative and interagency—the underlying authorities are  less explicit than they are for the others.
subsections: section_title: DOD Primarily Uses Two Explicit Statutory Authorities
paragraphs: We determined that DOD’s authority to pursue fellowships at non-DOD  educational institutions, foundations, and corporations derives primarily  from section 2603 of Title 10 of the United States Code, which authorizes  servicemembers to accept fellowships from certain organizations, and  section 2013 of Title 10 of the United States Code, which authorizes the  training of servicemembers at nongovernmental facilities. Generally,  DOD’s fellowships and training-with-industry programs are conducted  under one or both of these authorities.
Section 2603 allows servicemembers, subject to regulations,a fellowship offered by a corporation, fund, foundation, or educational  institution that is organized primarily for scientific, literary, or educational  purposes, and the benefits may be accepted by the member in addition to  the member’s military pay and allowances subject to certain conditions.  The member is required to agree in writing to serve on active duty after  completion of the fellowship for a period of at least three times the length  of the period of the education or training.
To create fellowship and training-with-industry programs with a variety of  nongovernmental facilities, DOD also uses provisions in section 2013 that  authorize it to enter into agreements or other arrangements for the  training of servicemembers with nongovernmental organizations such as  medical, scientific, technical, educational, research, or professional  institutions, foundations, or organizations—in addition to the business,  commercial, or industrial firms, corporations, or other nongovernmental  facilities, as defined by section 2013. These agreements or other  arrangements are not subject to certain requirements normally applicable  to government contracts. Section 2013 also authorizes DOD to pay expenses in connection with training at nongovernmental facilities in  addition to the member’s military pay.
section_title: DOD Uses Less Explicit Authorities for Two Specific Fellowship Programs
paragraphs: Although most of DOD’s fellowships or training-with-industry programs  are authorized under section 2603 and section 2013, we determined that  the authorities DOD uses for its military personnel to participate in two  specific types of fellowships—legislative and interagency—are less  explicit. According to OSD attorneys, DOD’s legislative and interagency  fellowships for servicemembers are conducted using a combination of  more general departmental authorities to train personnel. For example,  these attorneys noted that the secretaries of the military departments  have the authority to train members of the military services within their  respective military departments, subject to the authority, direction, and  control of the Secretary of Defense. They also cited other authorities  related to detailing personnel outside of DOD in support of these  fellowships. These attorneys cited section 4301 of Title 10 of the United  States Code as an example that authorizes (among other things) the  Secretary of the Army to detail members of the Army as students or  observers at locations such as industrial plants, hospitals, and other  places, where they would be best suited to acquire knowledge or   These attorneys noted that, because  experience in certain specialties.legislative and interagency fellowships involve detailing servicemembers  to “other places” best suited to acquire relevant knowledge, these  authorities could also be used to support fellowships. OSD attorneys  further indicated that these statutory sections provide authority to conduct  legislative and interagency fellowships, but agreed that the authority for  those specific programs is not as explicit as the authority provided for  other types of fellowships, or for intra-governmental training of civilians.
section_title: OSD Has Limited Visibility over Fellowship and Training-with-Industry Programs
paragraphs: OSD has limited visibility over its fellowship and training-with-industry  programs, because (1) OSD has not developed a clear mission statement  for these programs that defines the programs’ purpose, (2) OSD has not  consistently enforced its requirement for the military departments to  provide annual reports on fellowship and training-with-industry programs,  and (3) not all fellowship and training-with-industry programs have a  designated office within each department to be responsible for preparing  information for these programs. In turn, OSD does not have a reliable  inventory of the various fellowship and training-with-industry programs to  educate its military officers. The absence of a reliable inventory is due, in  part, to OSD’s not having a clear definition of a fellowship and the lack of  a common reporting requirement for the annual report on fellowships and  training-with-industry to OSD, both of which complicate OSD’s ability to  develop a reliable inventory. Additionally, OSD has limited visibility over  one type of fellowship program—legislative—because OSD has not  clearly delineated roles and responsibilities for overseeing this program  and has not developed documented criteria for the placement of DOD  fellows with congressional committees and members. OSD officials agree  that such placement criteria would be helpful, since DOD does not have  enough legislative fellows to meet the full congressional demand.
subsections: section_title: OSD Has Not Developed a Clear Mission Statement for Fellowship and Training Programs
paragraphs: OSD has not developed a mission statement that would clearly define the  respective key purposes for the legislative and nonlegislative fellowship  and training-with-industry programs. Having a clear mission statement is  critical because it defines an organization’s purpose in language that  states desired outcomes. A mission statement ultimately describes why  an organization exists and constitutes an important element in an  oversight structure.serves as a basis for having quantifiable performance goals. DOD’s  instruction on fellowships and training-with-industry provides a very  broadly written mission statement for the legislative and training-with- industry programs, but does not provide a mission statement for the  nonlegislative fellowship program. For the legislative fellowship program,  the instruction identifies the need for servicemembers to learn the  operative process of the legislative branch. For the training-with-industry   Further, a mission statement is important because it  program, the instruction identifies the need to develop skills in private  sector procedures and practices not available through existing military   or advanced civilian educational and training programs. However, these  definitions do not identify the purpose of providing participants with  additional skills and knowledge in terms of enhancing their value to DOD  throughout their careers. An OSD official stated that establishing a clear  mission statement would improve its ability to conduct policy oversight of  DOD’s fellowship and training programs. Without a clearly defined  mission statement, OSD is not in a position to know the extent to which  desired program outcomes are being achieved.
section_title: OSD Has Not Consistently Enforced an Annual Reporting Requirement on the Respective Military Departments’ Programs
paragraphs: Prior to our review, USD P&R did not enforce its requirement that the  military departments submit an annual report on their program reviews,  and has not received annual reports for fiscal years 2008 and 2009. Such  annual reports can enable USD P&R to have increased visibility over  these programs and can assist in confirming that DOD policy is being  implemented as expected. DOD Instruction 1322.06 directs USD P&R to  maintain overall responsibility for DOD fellowship and training-with- industry policy. This instruction also directs the military departments to  conduct an annual review of their respective fellowship and training-with- industry programs to ensure that they are in compliance, and to submit  the results of their reviews to USD P&R by January 31 of each year.
Since our review began, USD P&R has taken steps toward enforcing this  requirement with regard to the fiscal year 2010 reports.  However, two  of the departments’ fiscal year 2010 reports were incomplete in that some  covered only a portion of the programs, and the third department’s report  was submitted significantly after the deadline. Specific details on these  reports follow.
The Army reported only on one type of fellowship for fiscal year  2010—legislative. According to some Army officials, they did not have  any records indicating that the requested reports should include their  nonlegislative and training-with-industry programs—a requirement  about which they said they did not know until our review brought it to  their attention.
The Navy and the Marine Corps, conversely, reported on their  nonlegislative and training-with-industry programs for fiscal year 2010,  but not on their legislative fellowships. Further, according to Navy  program officials, USD P&R did not ask for the annual report in 2008  and 2009. Navy officials stated that they noticed this requirement in  2010 while seeking approval for a separate task and brought it to the  attention of the Assistant Secretary of the Navy for Manpower and  Reserve Affairs, who then tasked the program office to complete the  report for the nonlegislative fellowship and training-with-industry  programs. The Navy and Marine Corps provided their reports to USD  P&R for the first time in February 2011.
The Air Force report for fiscal year 2010 was submitted to USD P&R  in December 2011 (approximately 11 months after the deadline), but  only after USD P&R officials told Air Force officials in November 2011  to provide them with a report. The report included general information  on legislative and nonlegislative fellowships, and on training-with- industry programs.
USD P&R officials acknowledged that they did not request these reports  from the services, although USD P&R had been requiring these annual  reports as of the November 2007 revision to their instruction. Unless it  consistently enforces the requirement for the services to submit the  annual reports, USD P&R does not have the visibility to fully review the  services’ fellowship and training-with-industry programs and limits its  ability to perform its oversight responsibilities.
section_title: Not All Programs Have a Designated Office for Preparing Annual Reports
paragraphs: Certain offices have been designated by their military departments as  responsible for compiling information on some fellowship programs for the  OSD-required annual report; however, not all fellowship programs have a  designated office for compiling this report. Each service has separate  organizations that manage their legislative fellowship, nonlegislative  fellowship, and training-with-industry programs, respectively. We  contacted many different organizations to obtain a comprehensive picture  of the various phases of the fellowship and training-with-industry  programs, as shown in table 1 below,. (See app. II on the life-cycle  phases of a fellowship program, such as monitoring the fellows’ progress  during the fellowship, ensuring completion, and assigning the fellows to a  follow-on utilization or postfellowship tour.) For example, we had to gather  information from seven different organizations within the Army to obtain a  comprehensive view for just one of that service’s programs—the Army’s  Senior Service College Fellowships Program.
Although we interviewed officials in numerous offices within each military  service to obtain a more comprehensive understanding of the various  programs, only some fellowship program offices acknowledged having  been designated as having responsibility for compiling information on  their program for the annual report. Three service fellowship programs  have included the requirement for the annual report in their program  guidance—the Army Legislative Fellowship Program, the Navy Cyber  Federal Executive Fellowship Program, and the Legislative Fellowship  Program. However, the Air Force and Marine Corps legislative fellowship,  nonlegislative fellowship, and training-with-industry programs and the  Army and Navy nonlegislative fellowship and training-with-industry  programs have not formally designated program offices for the annual  preparation of these reports.
Officials in the policy office of the Air Force and the Navy told us that  while they have not been formally designated as responsible for compiling  this report, they anticipate being given the responsibility for reaching out  to the fellowship and training-with-industry programs to obtain information  for the report. Prior to the reports being submitted in response to  congressional inquiries in 2009 and 2010, some program officials said  that they were not aware that they were responsible for submitting a  report to USD P&R because they were not tasked as the office  responsible for this requirement. Without having a designated office within  each department to take responsibility for reporting requirements for their  respective fellowship and training-with-industry programs, USD P&R’s  visibility over departmental compliance with its instruction and relevant  laws will continue to be limited. In the course of this review, we identified  an important consequence of USD P&R’s limited visibility over DOD’s  various fellowship and training programs. By law, servicemembers in  certain fellowships are required to complete a service obligation for a  period of no less than three times the length of a fellowship. However,  one Army regulation for certain nonlegislative fellowship programs  stipulated a service obligation length that was not in compliance with the  law or with the DOD instruction governing that program. In that instance,  the Army was allowing certain officers to meet the service obligation  requirement for a 1-year fellowship by committing to serve 2 years rather  than the statutory 3 years. Subsequently, Army officials have informed us  that they are in the process of taking corrective action to change the Army  guidance based on our finding. USD P&R officials with whom we met  were unaware of this instance of noncompliance.
section_title: OSD Has Not Developed a Reliable Inventory of Fellowship and Training- with-Industry Programs and Participants
paragraphs: USD P&R has not developed a reliable inventory of DOD’s fellowship and  training-with-industry programs—an inventory that would be useful for  enabling DOD to know how often, at what cost, and in what capacity it is  using these alternative educational and training programs for its military  officers to meet new and emerging skill needs, and to better fulfill its  mission. We have previously reported on the importance of maintaining  a complete inventory of the type of skills an agency needs to better  position it to properly assess gaps in its capabilities and to appropriately  assess risk so it can make informed decisions about the future direction,  scope, and nature of its efforts and investments in support of emerging  skill needs. Although USD P&R has collected some of the DOD  instruction-required annual reports on fellowships and training-with- industry programs, these do not enable it to determine the totality of the  fellowship participants and programs, or of the needs these programs  address. We found that these required annual reports did not provide  consistent and detailed information linking the fellowships with current or  emerging needs, which are the basis for offering the various types of  fellowship opportunities. Such information would be useful in helping USD  P&R develop a robust inventory and identify training and educational  solutions to fill identified gaps. We collected information, which USD P&R  was unable to provide us, on program participants from the services   and we found that, at a minimum, 1,797 mid- and  dating back to 2007,senior-rank officers participated in fellowships and training-with-industry  programs from fiscal years 2007 through 2011, as shown in table 2.
The DOD instruction on fellowships and training-with-industry defines the  term fellowship, but we found that usage of the term to describe various  educational opportunities varies significantly from service to service, thus  complicating the compilation of a reliable and complete inventory. For  example, the Air Force increased its total number of fellowships offered  when it changed the designation of some of its “internships” to  “fellowships” after adding an interagency rotation requirement to the  program.more aligned with the fellowship description in the DOD instruction, and  thus changed the designation. Army officials, on the other hand, said that  their interagency fellowships—similar to those of the Air Force—are  called fellowships, but are not counted as fellowships for reporting  purposes. USD P&R officials did not know about the Air Force change  and stated that they disagreed with it. They observed that the instruction’s  definition of a fellowship could be made more descriptive to avoid  ambiguities such as this. In another example, the Navy included the  Secretary of Defense Corporate Fellows Program in its annual report for  2010—a program that is governed by a different instruction, overseen by  the Deputy Chief Management Officer, and managed in coordination with  the National Defense University director. In this same report, however,  the Navy did not include its Medical Professional Fellowship Programs,  which are governed by a separate instruction and managed through the  Navy Medicine Professional Development Center.
Air Force officials said that they viewed the internship as being  USD P&R officials said that inconsistent views as to which fellowships  should be counted—views that reflect the absence of a clear and  commonly shared definition of what constitutes a fellowship—could hinder  the development of an accurate inventory. Officials agreed that a more  specific, DOD-wide definition of a fellowship, which would explain which  opportunities warrant using the word “fellowship” in the program title,  could be helpful. They said they anticipate conducting a review of each  service’s policies or regulations that focus on fellowships and training- with-industry programs, to better ensure that all have the same definitions  and are counting the programs and participants comparably, and that the  services’ policies or regulations are in line with DOD Instruction 1322.06.  This review will become even more important given that DOD will be  drawing down its force structure and given that program officials within  the services informed us that they are interested in increasing the number  of fellowship and training-with-industry opportunities in the future.
The responsibilities of the Office of the USD P&R include ensuring that  personnel and readiness policies and programs are designed and  managed to improve standards of performance, economy, and efficiency,  which would facilitate visibility into its various programs, including DOD  fellowship and training-with-industry programs. Without a clear definition  of fellowship and training-with-industry programs, USD P&R’s ability to  maintain a reliable inventory of participants and thus oversee the service  obligations associated with these educational fellowship and training-with- industry programs will remain limited.
With regard to the military departments’ required reports on their annual  reviews of the education and training-with-industry fellowship programs,  USD P&R has not provided specific instructions as to the information the  military departments should include. As noted previously, these reports  did not contain consistent and detailed information on these programs.  Service officials said that they would benefit from having more detailed  reporting guidelines to help determine what information would be most  useful for USD P&R. USD P&R officials said that when they update their  Instruction in late 2012, they anticipate providing specifics to facilitate  collection of more consistent information on the number of fellows, on  their assignments during and after the fellowships, and on the timing of  their returns to their official duty stations. Detailed guidelines could better  position USD P&R to fulfill its oversight responsibility for fellowship and  training-with-industry policy, and to help ensure service adherence to  policies. Without USD P&R specifying in the DOD instruction common  reporting requirements for the annual reports, DOD will continue to  experience challenges in collecting consistent information and developing  a reliable inventory on these programs.
section_title: OSD Has Not Clearly Delineated Oversight Responsibilities for Its Legislative Fellowship Program
paragraphs: DOD’s Instruction 1322.06 does not delineate specific and clear roles and  responsibilities for overseeing the fellowship and training programs for the  Legislative Fellowship Program, and this absence of delineations has in  some cases resulted in mutually contradictory assumptions by different  OSD offices with respect to oversight responsibilities. We have previously  reported that having clearly defined roles and responsibilities can be  beneficial to address management challenges. The absence of defined  oversight roles and responsibilities for the Legislative Fellowship Program  has created some conflicting perspectives for two separate offices within  OSD. For example, officials within the Office of the Assistant Secretary of  Defense for Legislative Affairs said that although they assign DOD fellows  to various congressional offices and serve as liaisons between the fellows  and Congress, they do not have policy oversight responsibility for the  program. However, officials within USD P&R said that they have  responsibility for all reporting requirements for this program, but that  otherwise the legislative affairs office manages the program. Our  questions about roles and responsibilities concerning the Legislative  Fellowship Program elicited contradictory information from the two offices,  with each asserting that the other had policy oversight responsibility.  Although officials within USD P&R have subsequently revised their  position and said that they do have oversight responsibility, we note that  this important role is not clearly delineated in the DOD instruction. In the  absence of clearly delineated roles and responsibilities in DOD’s  governing instruction, OSD is at risk of not fulfilling its oversight  responsibilities. USD P&R officials stated that they agree with our  observation and indicated that they plan to address oversight roles and  responsibilities for legislative fellowships when they revise the instruction  in late 2012.
section_title: OSD Does Not Have Placement Criteria for Its Legislative Fellows
paragraphs: The Office of the Assistant Secretary of Legislative Affairs does not have  documented criteria for the placement of DOD legislative fellows to   that agencies  congressional offices. We have previously reportedshould assign employees to host organizations to make the best use of  their skills and help them to develop professionally while addressing host  organizations’ needs. However, officials in OSD’s legislative affairs office  said that the placement of DOD fellows with host congressional  committees or individual congressional member offices is a complicated  and very challenging process. These officials said that placing DOD  fellows with congressional offices involves balancing a number of factors,  such as (1) the fellows’ educational and career needs; (2) the availability  of fellows each year to a bipartisan and bicameral assembly of  congressional offices and members; and (3) any specified interests of  individual congressional members regarding a requested fellow’s military  service, educational background, and particular field of specialization. While these considerations help inform placement decisions, they do not  enable DOD to systematically decide and explain its rationale for  decisions on which congressional offices will receive DOD fellows. These  officials stated that their primary intent is to provide military officers a  broadening educational experience. In doing so, however, they have  found an additional challenge in meeting demand when placing DOD  legislative fellows in congressional offices because the number of  legislative fellows requested by congressional committees and members  has been exceeding the total number of available fellows, as shown in  table 3.
OSD legislative affairs officials said that because OSD does not have  documented placement criteria, decisions as to which congressional  offices receive DOD fellows are made on a case-by-case basis and are  ultimately a judgment call, which can lead to inconsistent dealings with  congressional offices. They said that they do their best to match the  educational needs of the DOD fellow with the needs of the congressional  office, but that there are not enough available fellows to meet the full  demand. They further stated that these decisions can be difficult to  defend to a congressional member who is denied a request for a DOD  fellow, and that it is always difficult to turn down congressional requests.  Without documented placement criteria, OSD’s legislative affairs office  will continue to have difficulty in ensuring consistency in the department’s  dealings with interested committees and members of Congress  requesting DOD fellows. Officials in OSD’s legislative affairs office said  that having documented placement criteria would help in the decision- making process and would better position the office to defend its  decisions with congressional members who were denied a request for a  DOD fellow.
section_title: Military Services Are Not Well Positioned to Determine the Extent of the Benefits from These Programs
paragraphs: The military services believe that they derive benefits from their  fellowships and training-with-industry programs, but they are not well  positioned to determine the extent of these benefits for four main reasons.  First, not all of the services conduct periodic program reviews, as required  by their service guidance, for some or all of their programs; and the few  reviews that have been conducted have not been comprehensive. For  example, they do not ensure that the needs that prompted the program  and the goals of the program are being met. Second, the services do not  have clear guidance as to what qualifies as a follow-on utilization tour, or  criteria for when such a tour can be postponed or waived, and this limits  their ability to know the extent they are deriving a return on their  educational investment. Third, the services do not know their overall  program costs, including both direct and indirect costs, and therefore it is  difficult to know whether these programs are cost-effective. Fourth, some  of the services do not have written agreements or memoranda of  understanding with the non-DOD host organizations providing the  educational opportunity that spell out both parties’ roles and  responsibilities—thus contributing to unclear expectations about the DOD  benefit to be derived from these programs. Collectively, these limitations  diminish the services’ assurances about the benefits they derive from  their fellowship and training-with-industry programs. The services  observed, however, that they use certain other program management  practices to ensure that they derive benefits from these programs.
subsections: section_title: Services Have Not Conducted Periodic and Comprehensive Program Reviews
paragraphs: While each of the services has taken some steps to review its programs,  none has conducted both periodic and comprehensive reviews of its  fellowship and training-with-industry programs. For some of the programs,  there is no requirement to conduct periodic program reviews. With  respect to fellowship programs, service review requirements vary, and  compliance with them has been uneven. Further, for those services that  have conducted program reviews, the reviews have not been  comprehensive and have not included assessments of the status of the  program against program goals. As a result, some services have limited  assurances that their fellowship or training-with-industry opportunities  meet desired program outcomes, address emerging mission  requirements, or provide the best venue for developing needed personnel  skills. We have previously reported on the importance of having  systematically planned evaluations of training and development  programs. In addition, evaluating programs through periodic and  comprehensive reviews enables an agency to identify problems and  improve a program, as needed. The services’ variance in review  requirements for these programs is shown in table 4.
Of the service programs that require a periodic review, Navy officials said  that they conducted program reviews for their nonlegislativelegislative programs, and Marine Corps officials said they have conducted  one nonlegislative program review. Two of the services—the Army and  the Air Force—have not fully conducted periodic reviews of their   and  programs as required in their program guidance. Although the Army is  required to review its nonlegislative fellowships, it has not conducted  reviews recently. In 2011, the Army began reviewing a portion of one type  of nonlegislative fellowship—the Senior Service College Fellowships— and officials said they expect to complete this portion of the review in  early 2012, and to review the other fellowship programs in the near future.  Air Force officials, on the other hand, informed us that their requirement  for a biennial review of fellowships is an error in their instruction, and they  plan to write this requirement out of the next version of the instruction.  Further, these officials said that the review requirement applies only to  graduate education programs—not fellowships. Therefore, the Air Force  has not periodically reviewed its fellowship programs, although Air Force  officials said that information on fellowships is incorporated into their  annual process for designating educational and developmental  opportunities.
None of the services has comprehensively reviewed its programs.   A comprehensive review, as we have previously reported, may include  the following four elements: quantifiably measuring the extent to which  the program is meeting program goals; validating that fellowships and  training-with-industry programs meet current and emerging requirements;  incorporating feedback from program participants and host organizations  to improve the program; and documenting the results of the review to  enable the service to modify its programs, as needed. However, we found  that not all services are including these four elements in their reviews, as  described below:
Using quantifiable performance measures for meeting their stated  goals for their programs. The services have not measured the extent  to which their programs are meeting their goals,  because they have  not developed quantifiable performance measures for most of their  program goals. We have previously reported on the importance of  agencies developing and using performance measures to ensure  accountability and assess progress toward achieving results aligned  with goals. The services vary in the extent to which they review their  programs to measure against program goals. For example, an Army  official said that the Army has not assessed the program using  quantifiable performance measures for its legislative fellowship  program, but the official believed that a measure that tracks the  completion of utilization tours could be used to assess the extent to  which they are meeting the goal of having a pool of officers from  which some may be selected for future utilization in the field of  congressional liaison. Similarly, a Navy official with the Cyber Federal  Executive Fellowship Program said that the utilization tour could be  used as a performance measure for meeting the program’s goals,   but they plan to use it as a qualitative, not quantifiable, performance  measure. The Marine Corps Legislative and Commandant of the  Marine Corps Fellowship Programs have not created quantifiable  performance measures either, although officials said they use  qualitative measures, such as discussions with fellows and senior  leaders, to assess the programs. The Army and Air Force have taken  some steps to develop quantifiable measures for some programs.   For example, in 2011 the Army surveyed fellows in the Senior Service  College Fellowship Program to assess the extent to which these  fellowships met their objectives. The Air Force Fellowship Program  surveys its host organizations on the percentage of time the fellows  spend on the Air Force’s service-specific competencies—used by Air  Force officials as indicators that the program is meeting its goals and  objectives. However, the Air Force has not developed quantifiable  goals with which these survey results can be compared. Without  reviewing their programs to assess progress against program goals,  the services are limited in their ability to determine the effectiveness of  their programs.
Validating that programs meet current or future mission requirements.
Some of the services have not periodically validated that the  fellowship or training-with-industry assignments continue to address  current or emerging mission requirements. The DOD instruction states  that the secretaries of the military departments are responsible for  ensuring that the current assignments using fellowships and training- with-industry positions meet the intent of the program and continue to  meet military department and DOD requirements or anticipated  needs. We have reported that successful organizations match training  and development programs to their specific needs and capabilities.While the Navy and the Marine Corps have periodically reviewed their  programs to ensure that they meet current and emerging needs, the  Army and the Air Force have not. The Marine Corps has informally  validated its programs through a process that looks at the costs of the  program, updates the list of fellowship host organizations, and  determines which fellowships are available for the upcoming year.  Although the Army’s instruction for nonlegislative fellowships requires  that each fellowship program be reviewed for continued relevancy and  to determine that it meets stated criteria and objectives, the Army has  not conducted this review regularly. We note that Army officials stated  that in their current review of a portion of the Senior Service College  fellowships that they plan to revalidate the need for each or the  uniqueness of each host organization. While the Air Force has not  revalidated that its fellowships meet current or emerging needs  through periodic program reviews, it stated in its December 2011  report to USD P&R on nonlegislative fellowships that program  oversight and management ensure validation that the programs fulfill  a present need, anticipated requirement, or future capability.
Incorporating feedback. While all of the services have collected some  feedback from fellows and host organizations, they have not  consistently incorporated this feedback into their program reviews.   We have previously reported on the importance of agencies  measuring training participants’ reactions to and satisfaction with their  programs, and of measuring changes in their knowledge, skills, and  abilities after their participation in the programs. The Army and the  Air Force have conducted postfellowship surveys for some fellowships  to gather information on the fellows’ experiences. While these surveys  provide information on the experience of the fellowship, they do not  provide information on how the fellowship experience has been of  benefit to the fellows’ careers from a perspective of several years  later. We note that the Marine Corps has conducted surveys to collect  feedback from both recent and earlier fellows. In addition, some of the  services gather feedback from fellows on their experiences through  periodic reports to the program office. All of the services monitor their  fellows and collect feedback through intermittent meetings with them.  The Air Force also collects feedback from host organizations through  surveys and periodic site visits, and the Army has recently conducted  site visits to some of its host organizations. Across the services,  however, there has not been consistent incorporation of feedback in  program reviews.
Documenting program accomplishments. Although some of the  services said that they document a portion of their review efforts, no  service has documented accomplishments that covered an entire  program review, or the entirety of its programs. We have previously  reported on the importance of using program evaluations to make  fact-based determinations of the program’s impact, and to reallocate  or redirect resources accordingly. Service officials said they use or  plan to use the information they collect to make changes to their  fellowship and training-with-industry programs. Army officials said  they plan to use their ongoing review of a portion of the Senior  Service College fellowships to refine the Army’s portfolio of senior- level fellowships. Prior to the review, the Army had identified problems  with the fellowship programs primarily by incident, either with the  fellow or with the host organization. Marine Corps officials stated they  have used feedback from the fellows to decide upon which host  organizations to keep or to discontinue in their portfolio. Air Force  officials said they have analyzed feedback from fellows and host  organizations to make decisions on which host organizations to  discontinue when overall education funding was reduced.  Documentation of these results would enable the services to  demonstrate that they have conducted the review and what steps,   if any, they may take to improve the program.
Some of the services informed us that they intend to do more to review  their programs. For example, as part of its ongoing review the Army plans  to match fellows with host organizations that will enable them to research  issues identified by Army leadership as critical problems facing the Army.  A Navy official said that although the Navy has not yet conducted its first  review of the Cyber Federal Executive Fellowship, it plans to survey 2011  fellows in the fall of 2012, a year into their utilization tour, to assess how  the fellowship benefited their careers. Marine Corps officials said that they  are examining a more formal process to supplement their current informal  annual fellowship reevaluation practices. Without conducting periodic and  comprehensive performance reviews, the services’ ability to determine  the benefit they derive from fellowship and training-with-industry  programs will remain limited.
section_title: Services Do Not Have Clear Guidance on Utilization Tours
paragraphs: The services inconsistently implement a DOD requirement to have  utilization tours—assignments where program participants can apply  knowledge and skills gained through fellowship and training-with-industry  programs—because they do not have clear guidance as to what qualifies  as a follow-on utilization tour, and they do not have criteria for when to  postpone or waive this requirement. After a participant’s completion of the  fellowship or training-with industry opportunity, the services are expected  to assign the participant to a tour that enables him or her to make use of  the newly gained skills, knowledge, and abilities, according to DOD  guidance.
Service officials described utilization tours to us as a benefit that DOD  gains from conducting its fellowship and training-with-industry programs.  For example, both Army fellowship policy documents and officials with  whom we spoke described utilization tours as a mechanism for  maximizing the benefit derived by the Army. Navy officials described  utilization tours as a way to receive a return on their investment and to  meet the intent of the programs, given that they reinforce skills, promote  continued learning, and serve to meet the requirements of the Navy. Air  Force policy documents and officials described utilization tours as a way  to ensure that officers use the skills they have developed.
However, the services do not implement utilization tours for fellowships  consistently, either within or across services. For example, within the  same Air Force fellowship program, some fellows have a mandatory  utilization tour, while for others it is a recommendation. According to Air  Force officials, the decision as to whether a utilization tour is mandatory  or recommended is made by the Air Force office responsible for  fellowship policy on a case-by-case basis at the time it initiates the  fellowship. Air Force officials said they do not have established criteria for  determining whether a utilization tour is mandatory or recommended.  Similarly, the Army requires utilization tours for some of its fellowship  programs, such as the legislative fellowship, but not for others. In  contrast, the Navy and the Marine Corps generally require utilization tours  for all of their fellowship programs, and all of the services generally  require utilization tours for their training-with-industry programs.
Service officials said that inconsistency across the services is attributable,  in part, to the absence of a clear definition as to what constitutes a follow- on utilization tour in the DOD instruction. A Navy official said that the  differences in how the services implement utilization tours are a result of  not having a clear definition of the requirement. Similarly, Marine Corps  officials described fulfilling utilization tours as a judgment call, and one  Marine Corps official said there has been an internal debate as to what  assignments should count as utilization tours, and within what time frame  such tours should be assigned.
Also, the services’ postponement or waivers of the requirement to  perform utilization tours occur inconsistently, because the services do not  have criteria to determine when it is appropriate to postpone or waive this  requirement. As a result, the services have different implementation  practices for postponing or waiving the utilization tour requirement.  Generally, the services postpone or waive the utilization tour requirement  on a case-by-case basis. For example, according to officials the Army  postpones or waives the requirement for most, but not all, of its  fellowships. For one Army fellowship program—the Senior Service  College Fellowship Program—the former Army Chief of Staff provided  written guidance in 2010 indicating that specific requirements for  utilization tours upon completion of fellowships should not be imposed.  Army officials stated that the utilization tour requirement was viewed as  adding more stress and burdens to the officers during a period of high  operational tempo and frequent deployments, and that utilization tours for  these fellows were viewed simply as a recommendation for postfellowship  assignments. In the case of one Navy fellowship program, on the other  hand, Navy guidance generally requires a utilization tour, and that a  high-level Navy Personnel Command official must approve any  exceptions; the regulation does not provide any criteria for such  exceptions. Service officials from all of the services stated that they  postpone assigning a utilization tour if the tour would interfere with  another career-advancing assignment, such as being selected for a  command position.
Some service officials said that improving the consistency of their  implementation of the utilization tour requirement could increase  assurances that they derive benefits from fellowship and training-with- industry programs, even though their reasons for not requiring utilization  tours vary. Army officials informed us that they are examining steps to  better strategically link utilization tours with the specific research focus of  the fellow, and might begin to require utilization tours for their Senior  Service College fellows now that they expect fewer Army officers to be  deploying overseas. Navy officials said they are hoping to develop a  capability to better track utilization tour completion to better ensure that  they are getting a good return on investment. Marine Corps officials said  they also plan to improve their process for tracking legislative fellowship  utilization tours in 2012. In addition, Air Force officials said that  implementing the utilization tour requirement more consistently would  benefit their service.
We have previously reported on the importance of having agency training  and development managers remove barriers that can impede the use of  knowledge and skills gained in training, so as to improve performance on  the job. Without having a common understanding of the follow-on  utilization tour requirement and its waiver criteria, the services’ ability to  determine the benefit they derive from fellows and training-with-industry  participants using their newly developed or enhanced skills in these tours  will remain limited, and the circumstances surrounding each waiver will  remain uncertain. In addition, having assurances that utilization tour  requirements are being met can better position the services to justify the  use of appropriated funds for these programs and can better ensure that  the services are meeting the intent of the programs by filling positions that  meet current and anticipated needs.
section_title: Services Do Not Know Overall Program Costs and Cannot Determine Cost-Effectiveness
paragraphs: The services do not know the overall costs of their fellowship and training- with-industry programs, including both direct and indirect costs. We have  previously reported on the importance of ensuring that training and  development efforts are cost-effective relative to their anticipated  benefits. Without having reviewed the overall costs of the programs, the  services cannot be assured that they are cost-effective, and they will not  have the information necessary to aid decision makers in managing  scarce training, development, and education resources, particularly due  to anticipated lower levels of defense spending.
The services are not required by the DOD instruction to report on the  overall costs of the program. However, while not uniformly required  across the board, some of the service instructions do require reviewing a  portion of the costs of their respective programs. The Army requires that  host candidate organizations, within their proposals to become hosts,   As part of a periodic review  present the estimated costs to the service. for two of the Navy’s fellowship programs—the Cyber Federal Executive  Fellowship and the Federal Executive Fellowship—the Navy reviews the  cost of execution, utilization, and any fiscal surpluses or constraints that  might affect future execution of the program, to help inform future  decisions about it. For one of its fellowship programs—the Commandant  of the Marine Corps Fellowships—the Marine Corps requires that a cost  estimate is determined for tuition, fees, and travel costs to aid in budget  planning. One Air Force instruction requires that all fellowships be  reviewed for their cost efficiency, but, as previously mentioned, Air Force  officials stated that this requirement to review fellowships is an error in the  Instruction.
The services know some of the direct costs associated with their  fellowship programs, such as the tuition and fees paid to the universities  and think tanks that host fellows, but not all of them. Service officials  said that the tuition or fee paid to a host organization was used for the  administrative, academic, and technological support provided to the  fellow, as well as for the facilities and resources at the host organization.  Some of the direct costs of the program include those associated with  travel, research, and orientation. We collected information from the  services on the tuition or fee amounts paid to host organizations for  fellowships. We estimated that for the academic year 2010-2011, the  services paid approximately $2.9 million in tuition and fees for  approximately 270 fellows, of which the Army paid approximately   $1.2 million; the Navy paid approximately $208,000; the Marine Corps  paid approximately $378,000; and the Air Force paid approximately   $1.1 million.fellowship host organizations.
The services paid a tuition or fee to more than half of their  For two of the services, officials said that knowing the direct costs of their  fellowship and training-with-industry programs is difficult, given that the  programs are financially managed by several different organizations. For  example, funding for the cost of a fellowship may come from the  proponent that sponsors a fellowship, or from general student and  educational funding, or from other components, such as the National  Guard or Reserve, depending on the fellowship and the service. In  addition, the costs of the program may extend across more than 1 fiscal  year, as the programs are based on academic or calendar rather than  fiscal years.
Service officials said that they do not know the indirect costs of the  programs, such as administrative support, faculty, and permanent change  of station costs, and that there are several challenges in determining  those costs. In addition, some officials said that for their programs indirect  costs may include the salary and benefits of the officer while participating  in the fellowship or training-with-industry program. Service officials said  that the indirect costs of the fellowship are not easily identified. For  example, an Army official said the Army does not have a financial  management system that would allow it to distinguish the administrative  and faculty costs directed toward Army War College students from those  directed toward fellows. For many of the programs, management and  faculty support of fellows and training-with-industry participants is an  additional duty, not the sole duty, of a service official.
Without knowing the direct and indirect costs of these programs, the  services are unable to compare the cost-effectiveness of attending one  type of educational opportunity versus that of another type. Service  officials said they experience similar challenges in knowing the overall  costs of servicemembers attending military schools, or other professional  development and training opportunities. Some service officials said that  having more information about the overall costs of the programs would be  beneficial because it would better inform them on how many fellowship  and training-with-industry opportunities they can afford to devote  educational and training resources to, and would provide better  information for budgeting.
section_title: Some of the Services Have Not Established or Reviewed Memoranda of Understanding with Host Organizations
paragraphs: Some services have not established or reviewed agreements or  memoranda of understanding that would outline the responsibilities of  the services and the think tank, university, and federal agency host  organizations for fellowships, but several service officials we interviewed  said this would be a best practice to incorporate. We have previously  reported on the importance of having management controls to provide  accountability for the use of government resources, and to outline the  responsibilities of each party. We have also reported on the importance  of having policies and procedures to ensure that training and  development efforts and expectations are discussed and understood by  managers, supervisors, employees, training coordinators, and others.
The requirement for having written memoranda of understanding between  each party varies across the services and by the type of training involved.  DOD’s Instruction 1322.06 does not require written memoranda of  understanding for fellowships, but it does require a written agreement for  training-with-industry programs. The instruction states that before the  start of a training-with-industry assignment there must be a written  agreement shared by the private sector host, the employee, and the DOD  component concerned. The services required agreements with host corporations. Although not required by the  instruction for fellowships, the services have varied in the extent to which  they have established memoranda of understanding with host  organizations. The Army and the Marine Corps have established such   In contrast, the  memoranda for a small proportion of their fellowships.Navy and the Air Force have established memoranda of understanding  with most of their fellowship host organizations, stating that these  documents help them to ensure that the host organization understands  the roles and responsibilities of each party as well as the key learning  objectives of the program.
The Marine Corps does not have a training-with-industry program.
The services have also varied in the extent to which service guidance  require these documents to be reviewed. The respective guidance for one  Navy and one Army fellowship program require that memoranda of   More specifically, the Navy’s Federal  understanding be reviewed.Executive Fellowship requires that its memoranda of understanding be  reviewed every 2 years; and the Commandant of the U.S. Army War  College is required to review and make recommendations on all  fellowship proposals and memoranda of understanding for the Senior  Service College Fellowships to ensure that the senior-level educational  criteria are met. However, since the Army does not have memoranda of  understanding for a majority of its fellowships, such reviews have rarely  occurred.
Several service officials cited establishing and reviewing memoranda of  understanding as a good management practice. One Navy official said  that establishing and reviewing memoranda of understanding is a best  practice that should be encouraged, saying these agreements would clear  up any misunderstanding about the structure of the individual program.  Another Navy official had positive comments about establishing and  reviewing memoranda of understanding, saying that establishing these  memoranda constitutes a good practice because they define roles and  responsibilities, provide protections for ethics and payments, and make  clear the objectives of the fellowship. Furthermore, the Army and the  Marine Corps both agreed that establishing and reviewing memoranda of  understanding with host organizations would be beneficial. Army officials  said they plan to establish and update the memoranda for all their  fellowships and to update their policy accordingly. Army officials also said  that these memoranda would provide clarity on the costs of fellowships  each year and thus facilitate better accounting and budgeting for their  program. Further, Army officials said the memoranda would provide  clarity on what makes each host organization unique, the purpose of the  fellowship, and the roles and responsibilities of the Army and the host  organization.
Memoranda of understanding serve as a management control and  document key information and expectations between the service and the  host organization. As a feature of documenting key information and  expectations, memoranda of understanding can provide protections  against potential conflicts of interest. Several Navy memoranda of  understanding we reviewed contained provisions that pertained to fellows  working on certain federal projects. For example, a memorandum of  understanding for the Federal Executive Fellowships says that the fellow  may not work on any contract or procurement studies that would violate  federal standards of conduct, especially those pertaining to conflicts of  interest. Without establishing and reviewing these memoranda, the  services cannot be assured that expectations, including safeguards  against potential conflicts of interests and criteria for appropriate fees and  projects, are clearly understood by the host organizations and the  fellowship participants.
section_title: The Services Use Certain Management Practices as a Means of Obtaining Benefits from Fellowship and Training-with-Industry Programs
paragraphs: While the military services are not well positioned to determine the extent  of their benefit from fellowship and related training programs, service  officials believe that they do obtain benefits from these programs. For  example, officials said these programs provide officers with a career- broadening developmental experience; enable them to conduct research  on topics that are important to the military; help them develop critical and  strategic thinking skills; help them gain perspectives from the civilian  sector; build civilian-military relationships; and develop more responsive  leaders during times of significant change in the global security  environment. Officials also believe that the use of fellowship and training- with-industry programs affords DOD flexibility in meeting new or emerging  educational and training needs that sometimes cannot be met within the  military educational structure—for example, in the rapidly developing new  approaches and techniques of the cyber security area. Service officials  said they use several program management practices to enhance the  benefits derived from fellowship and training-with-industry programs,  including the following:
Competitive selection: Service officials said that having a highly  competitive selection process that rank-orders officers who are  eligible to compete for many fellowship and training-with-industry  opportunities helps to produce a selection of highly qualified officers  for these opportunities. Officials said the process better ensures that  the right officer is selected for a given fellowship or training-with- industry program, which ultimately benefits the officer’s career as well  as accomplishes DOD’s mission though the experience gained. In our  review of the selection process, we found that the preferences of  career field managers, senior leaders, and the officers themselves  were considered, for many of the services, in the selection of officers  for fellowship or training-with-industry programs.
Orientation and ethics counseling: Service officials said that  conducting orientation before the start of a fellowship helps to prepare  fellows for their program by introducing them to key contacts, program  requirements, and expectations for the year. In addition, officials  stated, the ethics briefings given at orientation and availability of  subsequent counseling provide assurance that fellows understand  what is expected of them and are informed as to how to handle  potential conflicts of interest. We reviewed seven ethics briefings and  found them to contain essential elements, such as guidance on  accepting gifts and information on contacting DOD legal officials with  questions about ethics issues. (See app. IV for our review of these  briefings.)
Research: Service officials said they have assurances that they  receive a benefit by requiring many of the fellows to conduct research  and write papers on topics that are important to each respective  service, as identified by senior leaders. At orientation, some of the  services have senior leaders speak about emerging topics that could  be better researched to benefit the service. For academic year   2010-2011, for example, DOD fellows researched enhancing DOD’s  acquisition and logistics processes, as well as countering irregular  threats such as piracy.
Service obligations: Service officials informed us that they had general  procedures to provide some measure of benefits obtained when  officers complete their additional service obligation as required by  statute or DOD guidance for participating in certain fellowships or  training-with-industry programs—that is, that the fellow will serve in  the military for a period at least three times the length of the period of  the fellowship education or training-with-industry opportunity.
We found that all of the services generally have (1) policies on the  requirement; (2) special coding capabilities for personnel record- keeping and monitoring; (3) a formalized waiver process to better  ensure that servicemembers do not retire or separate without fulfilling  service obligations; and (4) the potential ability to recoup certain  educational costs. While many service officials asserted that these  practices increase the likelihood that benefits are derived from  fellowships and training-with-industry opportunities, some also said  they could still improve upon these practices to have greater  assurances that DOD derives benefits from these programs.
section_title: Conclusions
paragraphs: According to DOD, the intent for fellowships and training-with-industry  programs is to help fulfill a present need, anticipated requirement, or  future capability that contributes to the effectiveness of the department’s  mission. DOD has used these programs for at least 1,797 mid- to senior- level officers across the services over the past 5 fiscal years. Even as the  war in Afghanistan is winding down and reductions in personnel end- strengths occur, servicemembers will still be deployed globally, and DOD  will have to carefully balance operational assignments with identified  needs for professional military education, both within DOD’s professional  military educational system and outside of DOD, in fellowship and training  with industry programs. However, until OSD gains improved visibility into  these programs and the military services are better positioned to know  that they achieve their intended benefits and are cost-effective, DOD will  continue to face challenges in ensuring that it is deriving sufficient value  from these programs.
section_title: Recommendations for Executive Action
paragraphs: To improve oversight and management of DOD’s fellowship and training- with-industry programs, we are making 11 recommendations to the  Secretary of Defense.
To help ensure compliance with DOD Instruction 1322.06 and thus  enhance DOD’s visibility over all of the fellowship and training-with- industry programs, and to promote a shared understanding across the  military services of what is expected in meeting the instruction, the  Secretary of Defense should direct the Under Secretary of Defense for  Personnel and Readiness to take the following five actions:  (1) develop a mission statement that clearly defines the respective  purposes of the legislative and nonlegislative fellowship and training-with- industry programs to be in a better position to know the extent to which  desired program outcomes are being achieved;  (2) more consistently enforce the DOD instruction’s requirement on the  submission of annual reviews from the military services on these  programs; and  (3) collaborate with the military departments to ensure that each service  has designated an office to be responsible for compiling information on  the legislative and nonlegislative fellowship and training-with-industry  programs for the annual reports required in the DOD instruction.
In addition, to enable DOD to develop a more reliable inventory for these  programs, the Under Secretary of Defense for Personnel and Readiness  should:  (4) clarify in the DOD instruction the definition of a fellowship; and  (5) specify in the DOD instruction common reporting requirements for the  annual report, and clarify which fellowship and training-with-industry  opportunities should be included in this report, to facilitate the collection of  consistent information on these programs across the military services.
Additionally, to facilitate OSD’s ability to perform its oversight  responsibility for the legislative fellowship program and to better ensure  consistency in the department’s dealings with interested committees and  members of Congress, the Secretary of Defense should direct the  Assistant Secretary of Defense for Legislative Affairs to take the following  two actions:  (6) clearly delineate in the DOD instruction the roles and responsibilities  for overseeing DOD’s Legislative Fellowship Program, and  (7) develop documented placement criteria for legislative fellows.
Finally, to better position DOD to determine the extent of the benefits it  derives from legislative and nonlegislative fellowship and training-with- industry programs and better assess whether fellowship and training-with- industry programs offer the best venues for developing needed personnel  skills, the Secretary of Defense should direct the Secretaries of the  Department of the Army, the Department of the Navy, and the  Department of the Air Force to take the following four actions:  (8) perform periodic and comprehensive program reviews that assess the  progress using quantifiable measures, validate that programs continue to  meet current or emerging needs, incorporate feedback from program  participants and host organizations, and document the results of reviews;  (9) clarify guidance for determining what qualifies as a follow-on utilization  tour, and establish criteria to determine when a utilization tour is needed  or, conversely, when it can be postponed or waived;  (10) determine the direct costs of these programs by periodically  obtaining and analyzing overall direct program costs, and explore the  feasibility of estimating indirect program costs; and  (11) establish and periodically review fellowship written agreements or  memoranda of understanding to document key information and  expectations between the services and the host organizations, such as  fellowship objectives, criteria for evaluating the appropriateness of fees or  tuition charged to the military departments , and criteria for evaluating the  appropriateness of the projects involved.
section_title: Agency Comments and Our Evaluation
paragraphs: In written comments on a draft of this report, DOD concurred with the   11 recommendations we made to improve OSD oversight and strengthen  the military services’ management of its fellowships and training-with- industry programs. DOD also outlined actions that it plans to take for each  recommendation, which it stated will improve the issues we identified in  the report. DOD’s comments appear in their entirety in appendix V.   We are encouraged by the department’s action plans that carry the intent  to implement our recommendations.  For example, we made   7 recommendations to OSD designed to enhance OSD’s oversight and  visibility over these programs, for which OSD provided its intended  actions such as making planned revisions to its instruction governing  these programs by including a mission statement, a standard format for  the military services to follow when completing the required annual report,  and a date when this report is due to OSD—all actions we believe meet  the intent of these recommendations.  We also made 4 recommendations  to the military service secretaries to strengthen its management of these  programs, which again OSD provided a series of steps it plans to take in  response to our recommendations.  OSD’s action plans included such  steps as working with the military services to develop qualitative or  quantitative measures that the department believes will best meet the  services needs for these programs.  Again, we are encouraged by the  department’s action plans as stated in OSD’s comments to our draft  report in meeting the intent of our recommended actions to strengthen  these fellowship and training-with-industry programs.  Finally, DOD  provided technical comments, which we considered and incorporated  where appropriate.
We are sending copies of this report to the appropriate congressional  committees. We are also sending copies to the Secretary of Defense; the  Under Secretary of Defense (Comptroller); the Under Secretary of  Defense for Personnel and Readiness; the Secretaries of the Army, Navy,  and Air Force; and the Commandant of the Marine Corps. This report will  also be available at no charge on our website at http://www.gao.gov.
Should you or your staff have any questions concerning this report,  please contact me at (202) 512-3604 or farrellb@gao.gov. Contact points  for our Offices of Congressional Relations and Public Affairs may be  found on the last page of this report. Key contributors are listed in  appendix VI.
section_title: Appendix I: Scope and Methodology
paragraphs: To determine the statutory provisions that authorize the Department of  Defense (DOD) fellowship and training-with-industry programs, we  reviewed and assessed relevant laws and regulations that authorize and  govern these programs. We also interviewed officials with the Office of  the Under Secretary of Defense for Personnel and Readiness (USD P&R)  and attorneys from OSD’s Office of General Counsel for their  perspectives on laws and regulations governing these programs and  obtained written responses from OSD General Counsel on these  authorities.
To determine the extent of OSD’s visibility over these programs, we  obtained and assessed DOD instructions and service guidance governing  its fellowship and training-with-industry programs to determine the policy  oversight responsibility and other requirements. We interviewed officials  from USD P&R and the Office of the Assistant Secretary of Legislative  Affairs to obtain information about their roles and responsibilities with  respect to the Legislative Fellowship Program. We also interviewed  officials from USD P&R with knowledge of the service reporting  requirements from the DOD instruction on fellowships and training-with- industry programs. We collected available service reports that  corresponded to these requirements, and assessed the extent to which  USD P&R has established guidelines for information to include in these  reports. We obtained information on the extent to which USD P&R was  aware of the services’ compliance with statutory and regulatory  requirements, such as the service obligation commitments and reporting  requirements. Also, we assessed the extent to which USD P&R had a  complete inventory of its fellowship and training-with-industry programs.  We collected data on the number of military officers and the types of  fellowship and training-with-industry programs from each of the military  services and from OSD. We relied primarily on data from the military  services, as they were able to provide more detailed information on these  programs. To assess the reliability of each of the military services’  program data, we obtained information on (1) the systems used to  maintain their data and these systems’ ability to record and report on  these data, and (2) the quality control measures in place to ensure that  the data were reliable for our reporting purposes. We also interviewed  some service officials who were responsible for compiling these data. We  identified some data limitations within these data and cross referenced  with other documentation where available. We found the fellowship and  training-with-industry data provided to us by the services to be sufficiently  reliable for providing contextual information on the minimum number of  participants; however, these data do not allow us to provide the actual  totals.
To determine the extent to which the military services are able to  determine that they derive benefits from these programs, we obtained  and assessed service guidance, collected and reviewed information on  service processes and practices used to manage their programs, and  interviewed service officials. We collected and analyzed information on  the extent to which the services reviewed their programs, including  completion of periodic program reviews and whether the reviews  assessed the program against program goals using quantifiable  performance measures; validated that the program continued to address  current or emerging mission requirements; incorporated feedback from  program participants or host organizations; and documented the results of  the review. We collected data and information on utilization tours and on  the extent to which the fellows are assigned to them. We also collected  information on the extent to which the services tracked the costs of these  programs and established and maintained memoranda of understanding  with host organizations. We obtained information from each of the military  services on the tuition or fee paid to fellowship host organizations for  fellowships in academic year 2010-2011. We also obtained and  evaluated seven ethics briefings that DOD provided to fellows and  training-with-industry participants prior to the start of their selected  program with a non-DOD host organization. We identified the key  elements in these briefings and evaluated them for comprehensiveness.  In doing so, our Office of General Counsel staff from the Ethics Office— an office that provides guidance to our employees on ethics, conflicts of  interest, impairments to independence, and related conduct rules and  issues—reviewed and assessed the briefings we obtained from the  military services. We also selected and interviewed a non-probability  sample of four legislative fellows and six think tank fellows to obtain a  firsthand understanding about the benefits of these educational and  training opportunities from the perspective of the officer. We selected  fellows from each service who were either current or had recently  completed the fellowship at the time of our interviews, who were available  for interviews; and who were situated in proximity to the Washington,  D.C., area.
We selected fellows from all four services and from a variety of different  host organizations, focusing on think tanks and on legislative fellows in  response to the interest expressed by Senate Armed Services Committee  staff. In addition, we interviewed a non-probability sample of nine non- DOD host organizations to obtain their perspectives on the educational  benefits provided to military officers. We selected host organizations that  were hosting a fellow at the time of our interviews; that were available for  interviews; and that were situated in proximity to the Washington, D.C.,  area. To provide more in-depth examples of think tank fellowships, we  selected three of the host organizations that hosted a fellow from each of  the services and reported on the fees associated with the fellowships, and  on the extent to which the services established memoranda of  understanding with those organizations.
We visited or contacted the following organizations during our review:  Office of the Secretary of Defense
Under Secretary of Defense for Personnel and Readiness
Office of the Deputy General Counsel
Assistant Secretary of Defense for Legislative Affairs
National Defense University
Manpower and Reserve Affairs, Military Personnel Policy
Congressional Operations Division
Combined Arms Center
Department of Academic Affairs, Army War college
Interagency Student Division
Leadership Development Division
Senior Leaders Division
Strategic Leadership Division
Retirements and Separations Division
Manpower and Reserve Affairs, Military Personnel Policy
Education Branch
Distribution Management
Graduate Education and Training Placement
Information Dominance Corp
Legislative Liaison Office
Office of Supply Corps Personnel
Congressional Fellows Program Office
Marine Corps University
Officer Assignments Branch  Department of the Air Force
Congressional Support Branch
Colonel Management Office
Force Development
Fellowships Program Office
Developmental Education Branch
Officer Promotions, Appointments, and Selective Continuation Branch
Learning Division
Acquisitions Career Management
Air Force Research Institute, Air University
American Enterprise Institute
Atlantic Council
Brookings Institution
Center for New American Security
Center for Strategic and International Studies.
Council on Foreign Relations.
Washington Institute for Near East Studies  Institute of World Politics  Joint Center for Political and Economic Studies  We conducted this performance review from March 2011 through April  2012 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
section_title: Appendix II: Life-cycle Phases of a Fellowship Program
paragraphs: For each DOD fellowship program, there are various phases that  comprise the program, depending on its life-cycle as shown in figure 2.
Typical phases of a life-cycle for fellowships are as follows:  1.  Different organizations within each service identify present and  emerging needs of the service that the officers should be educated in  and trained on. For example, a service identifies an emerging need for  more cyber security based on recent events.  2.  The service policy or program offices establish the fellowship  opportunity that provides the training and skills to meet the identified  need. For example, once the cyber security emerging need was  identified, the Navy policy office established a fellowship program with  leading organizations in that field outside of DOD.  3.  Generally, the personnel command or center selects officers to  participate in the fellowships through a competitive selection process  that ranks eligible officers for selection.  4.  The fellows start their fellowship with a university, think tank,  corporation, federal agency, or congressional committee or member  office, generally with DOD or the service providing orientation and  ethics counseling.  5.  During the fellowship, the program office with primary responsibility for  the fellowship monitors the fellows’ progress, assisting them, when  applicable, with research associated with the fellowship.  6.  At the completion of some fellowships, the fellows are generally  required to submit a research paper or article. After completion, the  fellow’s personnel command or center is responsible for assigning the  participant to the postfellowship utilization tour, and for ensuring the  fellow does not separate or retire from the military before completing  the service obligation.  7.  Reviewing the fellowship program is generally conducted by either the  policy office or the program office. The review considers whether  there are any necessary changes to incorporate into the program for  the upcoming year.
Although we discussed fellowships in terms of a life-cycle, the life-cycle  for training-with-industry programs is similar to that of fellowships.
section_title: Appendix III: Military Service Guidance on Fellowship and Training-with-Industry Programs
paragraphs: The military services each have their own guidance or regulations that  covers fellowships and training-with-industry programs, as shown in   table 5.
section_title: Appendix IV: Ethics Guidance Provided to Fellows
paragraphs: DOD legal officials presented ethics briefings to fellows during their  orientation to the fellowship program. Each service provided a multiple- day orientation to fellows prior to the start of the fellowship and included a  segment that covered ethics issues. Some services provided a general  ethics briefing to all fellowship program participants at one time, and other  services provided the ethics briefing separately to each fellowship  program. Fellows participating in the Secretary of Defense Corporate  Fellowship Program received two ethics briefings, one from their service  and the other at the Secretary of Defense Corporate Fellowship Program  orientation.
In consultation with GAO’s Ethics Office, we identified key elements that  would provide fellows with guidance on potential ethical issues and  conflicts of interest they may face during their fellowship with a non-DOD  host organization. We obtained seven examples of ethics briefings: two  from the Army, two from the Navy, one from the Air Force, one from the  Marine Corps, and one from the Secretary of Defense Corporate  Fellowship Program. The key elements in the ethics briefings included:
Relevant laws and DOD policies, including the Joint Ethics Regulation  (JER), DOD 5500.7-R, the Standards of Ethical Conduct for Executive  Branch Employees 5 C.F.R. Part 2635, and relevant sections of 18  United States Code , Chapter 11, Bribery, Graft, and Conflicts of  Interest.
Standards of Ethical Conduct for Executive Branch Employees 5  C.F.R. Part 2635 include
Conflicts of interest
Acceptance of gifts
Unauthorized commitments purporting to bind the government
Using public office for private gain
Seeking outside employment  Some of the briefings provided guidance on teaching, speaking, and  writing, such as guidance on the proper use of disclaimers in those  activities for which the opinions of the fellow do not represent DOD, and  guidance on being compensated for those activities. For example, fellows  were instructed not to take compensation for teaching, speaking, and  writing if the activity was undertaken as part the fellow’s official position,  or if the topic dealt with an ongoing or announced policy, program, or  operation of DOD. In addition, some briefings provided guidance on  engaging in political activities. For example, fellows were instructed that  they were not permitted to use official authority or influence for interfering  with an election; to be a candidate for civil office except as authorized; to  participate in partisan political management, campaigns, speeches,  articles, or conventions; or to promote or attend political dinners or  fundraising events as an official representative of the Armed Forces.  Furthermore, DOD provided the fellows with a point of contact for future  questions and advice.
section_title: Appendix V: Comments from the Department of Defense
section_title: Appendix VI: GAO Contact and Staff Acknowledgments
subsections: section_title: Contact
section_title: Acknowledgments
paragraphs: In addition to the contact above, Laura Talbott, Assistant Director;  Darreisha Bates; Maria McCollester; Erin Preston; Sara Olds; Terry  Richardson; Amie Steele; Cheryl Weissman; Allen Westheimer; and  Michael Willems made key contributions to this report.