DOD’s fellowship and training-with-industry programs comprise a few of the several vehicles DOD uses to provide training and education to its military officers to develop and advance their careers while in the military. Officers have educational opportunities throughout their careers, both within and outside of the military educational system. These include attendance at intermediate- and senior-level military schools, as well as participation in graduate programs to obtain advanced degrees. Fellowship and training-with-industry programs constitute another type of professional development that officers may receive in addition to or in lieu of attendance at a military school. To provide guidance to the services on participation in a fellowship or training-with-industry program, USD P&R issued DOD Instruction Under this instruction, a fellowship is defined as an assignment 1322.06.in which selected DOD personnel work away from DOD to gain education or experience of value to both the DOD component and the gaining organization. Military officers can participate in a variety of types of fellowships at host organizations such as universities, think tanks, corporations, federal agencies, and congressional committees or member offices. Most of these fellowships are limited to a duration of no longer than 12 months. The following types of organizations can host fellowship and training-with-industry participants: Universities. For example, Tufts University, a private university in Massachusetts, hosts fellows from the military services within the International Security Studies Program at its Fletcher School, Tufts’ graduate school in international affairs. Fellows at the Fletcher School focus on research and writing requirements and participate in activities both inside and outside of the classroom. The Marine Corps Fellows may receive a master’s degree through this program. Think tanks. For example, the Brookings Institution, a nonprofit think tank based in Washington, D.C., has hosted fellows under its 21st Century Defense Initiative—one of Brookings’ research projects. This initiative focuses on the future of war, U.S. defense needs and priorities, and the defense system. Fellows from the military services spend a year at Brookings researching and publishing on defense topics and provide policy feedback to help craft realistic, applicable policy recommendations for the military services. Fellows at Brookings may also elect to attend courses to obtain an executive certificate in public leadership, for an additional fee. Corporations. For example, as part of the Secretary of Defense Corporate Fellowship Program, these fellows are trained at the executive level to learn how the host corporations use innovative practices and technology to plan, organize, and manage, and how these business practices could be applied within the military services. The fellows shadow corporate executives and write a paper on their observations and experiences upon completion of the fellowship. Federal agencies. For example, the United States Agency for International Development is one of the federal agencies that host fellows from the Army Interagency Fellowship program and from the Commandant of the Marine Corps Fellowships program. Federal agency fellows are placed with departments or agencies to develop a more thorough understanding of the host agency’s mission, culture, capabilities, and procedures. The Army, Marine Corps, and Air Force currently send fellows to other federal departments or agencies. Congressional committees or members’ offices, as part of DOD’s Legislative Fellowship program. This program offers officers an educational opportunity to learn more about the legislative process by being placed with various committees, or with the staff of the House or Senate Majority or Minority Leader, or with the staff of the Speaker of the House.George Washington University to obtain Masters’ degrees in Legislative Affairs. The Navy, Marine Corps, and Air Force offer their officers the opportunity to take courses for credit or to receive a In addition, the Army sends its legislative fellows to certificate in Legislative Affairs at the Georgetown’s Government Affairs Institute. Fellows pursue individual research and writing projects in consultation with host organization experts or faculty, and they may elect to become involved in host organization projects and participate in conferences, seminars, or classes. In addition, fellows share operational and service insights with host organization staff. Some university and think tank host organizations charge the military departments a fee or tuition for hosting fellows. In addition to fellowship programs, DOD offers training-with-industry programs. The DOD instruction defines training-with-industry as a nondegree-producing program designed to provide training or skills in best business procedures and practices not available through existing military or advanced civilian schooling for identifiable DOD requirements. The instruction states that there must be an existing need or desired future capability fulfilled by the gained training-with-industry experience. The military departments generally do not pay corporations a fee or tuition to host training-with-industry participants. DOD continues to provide officers participating in both fellowships and training-with-industry programs their normal pay and allowances. In exchange, after their participation, officers participating in most of the fellowships discussed in this report incur an active duty service commitment of three times the length of the term they spent in the fellowship or training-with-industry program. Within DOD’s professional military educational system, officers are afforded several key opportunities. At an officer level of O3, selected officers could attend primary level education, generally at the Branch, warfare, or staff specialty schools. At the level of O4, selected officers attend intermediate level education, generally at the service’s command and staff college or an equivalent military school. At the level of O5 or selected officers attend senior-level education, generally at the O6,service’s war college or at an equivalent military school. Officers may earn graduate degrees upon completing their intermediate- or senior-level education at a military school. They additionally have other opportunities throughout their careers to earn graduate degrees; for example, officers may apply to attend a college or university to receive a master’s or PhD degree in a graduate educational program. Military officers may participate in fellowships as either an alternative or a supplement to their professional military education, as shown in figure 1. Three of the services—the Army, the Marine Corps, and the Air Force— grant professional military educational credit for some of their fellowship programs at the intermediate or senior level. The Air Force, for example, grants professional military education credit for all of its intermediate- and senior-level fellowships. These fellowships replace an officer’s in-residence intermediate or senior professional military education. One service program—the Army Legislative Fellowship Program—grants a degree. When fellowship programs are provided in lieu of attendance at a military school, the participants are generally prohibited from also attending the corresponding in-residence military school program. For example, a fellow who attends a senior-level fellowship and is granted senior-level professional military education credits is prohibited from attending a senior in-resident War College program. Unlike most fellowships, DOD’s training-with-industry programs exist outside of the professional military educational system. These opportunities do not grant professional military education credits; instead, they are considered a supplemental professional development and training experience in private sector practices that are not available through existing military education and training programs. USD P&R has overall responsibility for fellowship and training-with- industry policy for fellowship programs that fall under DOD Instruction 1322.06. This Instruction covers most of the fellowship and training- with-industry programs in our review. As specified in the instruction, each military departmental secretary has oversight and management responsibilities for that department’s programs, with the exception of the Legislative Fellowship Program. For that program, each department selects nominees, whose names are then sent to USD P&R for The Office of the Assistant Secretary of Defense for approval.Legislative Affairs places these officers with congressional committees or with various congressional staffs. Additionally, some of DOD’s fellowship programs are administered separately; for example, the Secretary of Defense Corporate Fellowship Program and the Medical Professional Fellowship Program are administered under two other, separate DOD instructions, with different OSD offices having oversight and policy responsibilities for them. The Secretary of Defense Corporate Fellowship Program is managed by a director who is part of the National Defense University, under the authority, direction, and control of DOD’s Deputy Chief Management Officer, subject to certain exceptions. The Assistant Secretary of Defense for Health Affairs, under USD P&R, is responsible for monitoring compliance with DOD Instruction 6000.13 and providing additional guidance for medical fellowships, and the secretaries of the military departments are responsible for administering these programs. DOD Instruction 1322.06, Fellowships, Scholarships, Training-with-Industry (TWI), and Grants for DOD Personnel (Nov. 15, 2007). assignments. A fellow at a host organization with a cyber or nuclear focus may be assigned to a utilization tour involving the development of strategies for cyber or nuclear warfare. Training-with-industry participants are generally used in positions directly related to a corporation’s area of expertise. For example, a training-with-industry participant at a private company that specializes in logistics and the transportation of items may be later assigned to a DOD position in supply chain management. The military departments are responsible for the management of their respective fellowship and training-with-industry programs (see app. II on the life-cycle phases of a fellowship program for a more detailed description). In addition to statutory authorities and the DOD instruction, the military services also have their own guidance or regulations covering their respective legislative fellowship, nonlegislative fellowship, and training-with-industry programs (see app. III for a list of service guidance). We determined that DOD uses several statutory authorities for its fellowships and training-with-industry programs for military officers. Primarily, it uses two explicit statutory authorities. However, for two types of fellowships—legislative and interagency—the underlying authorities are less explicit than they are for the others. We determined that DOD’s authority to pursue fellowships at non-DOD educational institutions, foundations, and corporations derives primarily from section 2603 of Title 10 of the United States Code, which authorizes servicemembers to accept fellowships from certain organizations, and section 2013 of Title 10 of the United States Code, which authorizes the training of servicemembers at nongovernmental facilities. Generally, DOD’s fellowships and training-with-industry programs are conducted under one or both of these authorities. Section 2603 allows servicemembers, subject to regulations,a fellowship offered by a corporation, fund, foundation, or educational institution that is organized primarily for scientific, literary, or educational purposes, and the benefits may be accepted by the member in addition to the member’s military pay and allowances subject to certain conditions. The member is required to agree in writing to serve on active duty after completion of the fellowship for a period of at least three times the length of the period of the education or training. To create fellowship and training-with-industry programs with a variety of nongovernmental facilities, DOD also uses provisions in section 2013 that authorize it to enter into agreements or other arrangements for the training of servicemembers with nongovernmental organizations such as medical, scientific, technical, educational, research, or professional institutions, foundations, or organizations—in addition to the business, commercial, or industrial firms, corporations, or other nongovernmental facilities, as defined by section 2013. These agreements or other arrangements are not subject to certain requirements normally applicable to government contracts. Section 2013 also authorizes DOD to pay expenses in connection with training at nongovernmental facilities in addition to the member’s military pay. Although most of DOD’s fellowships or training-with-industry programs are authorized under section 2603 and section 2013, we determined that the authorities DOD uses for its military personnel to participate in two specific types of fellowships—legislative and interagency—are less explicit. According to OSD attorneys, DOD’s legislative and interagency fellowships for servicemembers are conducted using a combination of more general departmental authorities to train personnel. For example, these attorneys noted that the secretaries of the military departments have the authority to train members of the military services within their respective military departments, subject to the authority, direction, and control of the Secretary of Defense. They also cited other authorities related to detailing personnel outside of DOD in support of these fellowships. These attorneys cited section 4301 of Title 10 of the United States Code as an example that authorizes (among other things) the Secretary of the Army to detail members of the Army as students or observers at locations such as industrial plants, hospitals, and other places, where they would be best suited to acquire knowledge or These attorneys noted that, because experience in certain specialties.legislative and interagency fellowships involve detailing servicemembers to “other places” best suited to acquire relevant knowledge, these authorities could also be used to support fellowships. OSD attorneys further indicated that these statutory sections provide authority to conduct legislative and interagency fellowships, but agreed that the authority for those specific programs is not as explicit as the authority provided for other types of fellowships, or for intra-governmental training of civilians. OSD has limited visibility over its fellowship and training-with-industry programs, because (1) OSD has not developed a clear mission statement for these programs that defines the programs’ purpose, (2) OSD has not consistently enforced its requirement for the military departments to provide annual reports on fellowship and training-with-industry programs, and (3) not all fellowship and training-with-industry programs have a designated office within each department to be responsible for preparing information for these programs. In turn, OSD does not have a reliable inventory of the various fellowship and training-with-industry programs to educate its military officers. The absence of a reliable inventory is due, in part, to OSD’s not having a clear definition of a fellowship and the lack of a common reporting requirement for the annual report on fellowships and training-with-industry to OSD, both of which complicate OSD’s ability to develop a reliable inventory. Additionally, OSD has limited visibility over one type of fellowship program—legislative—because OSD has not clearly delineated roles and responsibilities for overseeing this program and has not developed documented criteria for the placement of DOD fellows with congressional committees and members. OSD officials agree that such placement criteria would be helpful, since DOD does not have enough legislative fellows to meet the full congressional demand. OSD has not developed a mission statement that would clearly define the respective key purposes for the legislative and nonlegislative fellowship and training-with-industry programs. Having a clear mission statement is critical because it defines an organization’s purpose in language that states desired outcomes. A mission statement ultimately describes why an organization exists and constitutes an important element in an oversight structure.serves as a basis for having quantifiable performance goals. DOD’s instruction on fellowships and training-with-industry provides a very broadly written mission statement for the legislative and training-with- industry programs, but does not provide a mission statement for the nonlegislative fellowship program. For the legislative fellowship program, the instruction identifies the need for servicemembers to learn the operative process of the legislative branch. For the training-with-industry Further, a mission statement is important because it program, the instruction identifies the need to develop skills in private sector procedures and practices not available through existing military or advanced civilian educational and training programs. However, these definitions do not identify the purpose of providing participants with additional skills and knowledge in terms of enhancing their value to DOD throughout their careers. An OSD official stated that establishing a clear mission statement would improve its ability to conduct policy oversight of DOD’s fellowship and training programs. Without a clearly defined mission statement, OSD is not in a position to know the extent to which desired program outcomes are being achieved. Prior to our review, USD P&R did not enforce its requirement that the military departments submit an annual report on their program reviews, and has not received annual reports for fiscal years 2008 and 2009. Such annual reports can enable USD P&R to have increased visibility over these programs and can assist in confirming that DOD policy is being implemented as expected. DOD Instruction 1322.06 directs USD P&R to maintain overall responsibility for DOD fellowship and training-with- industry policy. This instruction also directs the military departments to conduct an annual review of their respective fellowship and training-with- industry programs to ensure that they are in compliance, and to submit the results of their reviews to USD P&R by January 31 of each year. Since our review began, USD P&R has taken steps toward enforcing this requirement with regard to the fiscal year 2010 reports. However, two of the departments’ fiscal year 2010 reports were incomplete in that some covered only a portion of the programs, and the third department’s report was submitted significantly after the deadline. Specific details on these reports follow. The Army reported only on one type of fellowship for fiscal year 2010—legislative. According to some Army officials, they did not have any records indicating that the requested reports should include their nonlegislative and training-with-industry programs—a requirement about which they said they did not know until our review brought it to their attention. The Navy and the Marine Corps, conversely, reported on their nonlegislative and training-with-industry programs for fiscal year 2010, but not on their legislative fellowships. Further, according to Navy program officials, USD P&R did not ask for the annual report in 2008 and 2009. Navy officials stated that they noticed this requirement in 2010 while seeking approval for a separate task and brought it to the attention of the Assistant Secretary of the Navy for Manpower and Reserve Affairs, who then tasked the program office to complete the report for the nonlegislative fellowship and training-with-industry programs. The Navy and Marine Corps provided their reports to USD P&R for the first time in February 2011. The Air Force report for fiscal year 2010 was submitted to USD P&R in December 2011 (approximately 11 months after the deadline), but only after USD P&R officials told Air Force officials in November 2011 to provide them with a report. The report included general information on legislative and nonlegislative fellowships, and on training-with- industry programs. USD P&R officials acknowledged that they did not request these reports from the services, although USD P&R had been requiring these annual reports as of the November 2007 revision to their instruction. Unless it consistently enforces the requirement for the services to submit the annual reports, USD P&R does not have the visibility to fully review the services’ fellowship and training-with-industry programs and limits its ability to perform its oversight responsibilities. Certain offices have been designated by their military departments as responsible for compiling information on some fellowship programs for the OSD-required annual report; however, not all fellowship programs have a designated office for compiling this report. Each service has separate organizations that manage their legislative fellowship, nonlegislative fellowship, and training-with-industry programs, respectively. We contacted many different organizations to obtain a comprehensive picture of the various phases of the fellowship and training-with-industry programs, as shown in table 1 below,. (See app. II on the life-cycle phases of a fellowship program, such as monitoring the fellows’ progress during the fellowship, ensuring completion, and assigning the fellows to a follow-on utilization or postfellowship tour.) For example, we had to gather information from seven different organizations within the Army to obtain a comprehensive view for just one of that service’s programs—the Army’s Senior Service College Fellowships Program. Although we interviewed officials in numerous offices within each military service to obtain a more comprehensive understanding of the various programs, only some fellowship program offices acknowledged having been designated as having responsibility for compiling information on their program for the annual report. Three service fellowship programs have included the requirement for the annual report in their program guidance—the Army Legislative Fellowship Program, the Navy Cyber Federal Executive Fellowship Program, and the Legislative Fellowship Program. However, the Air Force and Marine Corps legislative fellowship, nonlegislative fellowship, and training-with-industry programs and the Army and Navy nonlegislative fellowship and training-with-industry programs have not formally designated program offices for the annual preparation of these reports. Officials in the policy office of the Air Force and the Navy told us that while they have not been formally designated as responsible for compiling this report, they anticipate being given the responsibility for reaching out to the fellowship and training-with-industry programs to obtain information for the report. Prior to the reports being submitted in response to congressional inquiries in 2009 and 2010, some program officials said that they were not aware that they were responsible for submitting a report to USD P&R because they were not tasked as the office responsible for this requirement. Without having a designated office within each department to take responsibility for reporting requirements for their respective fellowship and training-with-industry programs, USD P&R’s visibility over departmental compliance with its instruction and relevant laws will continue to be limited. In the course of this review, we identified an important consequence of USD P&R’s limited visibility over DOD’s various fellowship and training programs. By law, servicemembers in certain fellowships are required to complete a service obligation for a period of no less than three times the length of a fellowship. However, one Army regulation for certain nonlegislative fellowship programs stipulated a service obligation length that was not in compliance with the law or with the DOD instruction governing that program. In that instance, the Army was allowing certain officers to meet the service obligation requirement for a 1-year fellowship by committing to serve 2 years rather than the statutory 3 years. Subsequently, Army officials have informed us that they are in the process of taking corrective action to change the Army guidance based on our finding. USD P&R officials with whom we met were unaware of this instance of noncompliance. USD P&R has not developed a reliable inventory of DOD’s fellowship and training-with-industry programs—an inventory that would be useful for enabling DOD to know how often, at what cost, and in what capacity it is using these alternative educational and training programs for its military officers to meet new and emerging skill needs, and to better fulfill its mission. We have previously reported on the importance of maintaining a complete inventory of the type of skills an agency needs to better position it to properly assess gaps in its capabilities and to appropriately assess risk so it can make informed decisions about the future direction, scope, and nature of its efforts and investments in support of emerging skill needs. Although USD P&R has collected some of the DOD instruction-required annual reports on fellowships and training-with- industry programs, these do not enable it to determine the totality of the fellowship participants and programs, or of the needs these programs address. We found that these required annual reports did not provide consistent and detailed information linking the fellowships with current or emerging needs, which are the basis for offering the various types of fellowship opportunities. Such information would be useful in helping USD P&R develop a robust inventory and identify training and educational solutions to fill identified gaps. We collected information, which USD P&R was unable to provide us, on program participants from the services and we found that, at a minimum, 1,797 mid- and dating back to 2007,senior-rank officers participated in fellowships and training-with-industry programs from fiscal years 2007 through 2011, as shown in table 2. The DOD instruction on fellowships and training-with-industry defines the term fellowship, but we found that usage of the term to describe various educational opportunities varies significantly from service to service, thus complicating the compilation of a reliable and complete inventory. For example, the Air Force increased its total number of fellowships offered when it changed the designation of some of its “internships” to “fellowships” after adding an interagency rotation requirement to the program.more aligned with the fellowship description in the DOD instruction, and thus changed the designation. Army officials, on the other hand, said that their interagency fellowships—similar to those of the Air Force—are called fellowships, but are not counted as fellowships for reporting purposes. USD P&R officials did not know about the Air Force change and stated that they disagreed with it. They observed that the instruction’s definition of a fellowship could be made more descriptive to avoid ambiguities such as this. In another example, the Navy included the Secretary of Defense Corporate Fellows Program in its annual report for 2010—a program that is governed by a different instruction, overseen by the Deputy Chief Management Officer, and managed in coordination with the National Defense University director. In this same report, however, the Navy did not include its Medical Professional Fellowship Programs, which are governed by a separate instruction and managed through the Navy Medicine Professional Development Center. Air Force officials said that they viewed the internship as being USD P&R officials said that inconsistent views as to which fellowships should be counted—views that reflect the absence of a clear and commonly shared definition of what constitutes a fellowship—could hinder the development of an accurate inventory. Officials agreed that a more specific, DOD-wide definition of a fellowship, which would explain which opportunities warrant using the word “fellowship” in the program title, could be helpful. They said they anticipate conducting a review of each service’s policies or regulations that focus on fellowships and training- with-industry programs, to better ensure that all have the same definitions and are counting the programs and participants comparably, and that the services’ policies or regulations are in line with DOD Instruction 1322.06. This review will become even more important given that DOD will be drawing down its force structure and given that program officials within the services informed us that they are interested in increasing the number of fellowship and training-with-industry opportunities in the future. The responsibilities of the Office of the USD P&R include ensuring that personnel and readiness policies and programs are designed and managed to improve standards of performance, economy, and efficiency, which would facilitate visibility into its various programs, including DOD fellowship and training-with-industry programs. Without a clear definition of fellowship and training-with-industry programs, USD P&R’s ability to maintain a reliable inventory of participants and thus oversee the service obligations associated with these educational fellowship and training-with- industry programs will remain limited. With regard to the military departments’ required reports on their annual reviews of the education and training-with-industry fellowship programs, USD P&R has not provided specific instructions as to the information the military departments should include. As noted previously, these reports did not contain consistent and detailed information on these programs. Service officials said that they would benefit from having more detailed reporting guidelines to help determine what information would be most useful for USD P&R. USD P&R officials said that when they update their Instruction in late 2012, they anticipate providing specifics to facilitate collection of more consistent information on the number of fellows, on their assignments during and after the fellowships, and on the timing of their returns to their official duty stations. Detailed guidelines could better position USD P&R to fulfill its oversight responsibility for fellowship and training-with-industry policy, and to help ensure service adherence to policies. Without USD P&R specifying in the DOD instruction common reporting requirements for the annual reports, DOD will continue to experience challenges in collecting consistent information and developing a reliable inventory on these programs. DOD’s Instruction 1322.06 does not delineate specific and clear roles and responsibilities for overseeing the fellowship and training programs for the Legislative Fellowship Program, and this absence of delineations has in some cases resulted in mutually contradictory assumptions by different OSD offices with respect to oversight responsibilities. We have previously reported that having clearly defined roles and responsibilities can be beneficial to address management challenges. The absence of defined oversight roles and responsibilities for the Legislative Fellowship Program has created some conflicting perspectives for two separate offices within OSD. For example, officials within the Office of the Assistant Secretary of Defense for Legislative Affairs said that although they assign DOD fellows to various congressional offices and serve as liaisons between the fellows and Congress, they do not have policy oversight responsibility for the program. However, officials within USD P&R said that they have responsibility for all reporting requirements for this program, but that otherwise the legislative affairs office manages the program. Our questions about roles and responsibilities concerning the Legislative Fellowship Program elicited contradictory information from the two offices, with each asserting that the other had policy oversight responsibility. Although officials within USD P&R have subsequently revised their position and said that they do have oversight responsibility, we note that this important role is not clearly delineated in the DOD instruction. In the absence of clearly delineated roles and responsibilities in DOD’s governing instruction, OSD is at risk of not fulfilling its oversight responsibilities. USD P&R officials stated that they agree with our observation and indicated that they plan to address oversight roles and responsibilities for legislative fellowships when they revise the instruction in late 2012. The Office of the Assistant Secretary of Legislative Affairs does not have documented criteria for the placement of DOD legislative fellows to that agencies congressional offices. We have previously reportedshould assign employees to host organizations to make the best use of their skills and help them to develop professionally while addressing host organizations’ needs. However, officials in OSD’s legislative affairs office said that the placement of DOD fellows with host congressional committees or individual congressional member offices is a complicated and very challenging process. These officials said that placing DOD fellows with congressional offices involves balancing a number of factors, such as (1) the fellows’ educational and career needs; (2) the availability of fellows each year to a bipartisan and bicameral assembly of congressional offices and members; and (3) any specified interests of individual congressional members regarding a requested fellow’s military service, educational background, and particular field of specialization. While these considerations help inform placement decisions, they do not enable DOD to systematically decide and explain its rationale for decisions on which congressional offices will receive DOD fellows. These officials stated that their primary intent is to provide military officers a broadening educational experience. In doing so, however, they have found an additional challenge in meeting demand when placing DOD legislative fellows in congressional offices because the number of legislative fellows requested by congressional committees and members has been exceeding the total number of available fellows, as shown in table 3. OSD legislative affairs officials said that because OSD does not have documented placement criteria, decisions as to which congressional offices receive DOD fellows are made on a case-by-case basis and are ultimately a judgment call, which can lead to inconsistent dealings with congressional offices. They said that they do their best to match the educational needs of the DOD fellow with the needs of the congressional office, but that there are not enough available fellows to meet the full demand. They further stated that these decisions can be difficult to defend to a congressional member who is denied a request for a DOD fellow, and that it is always difficult to turn down congressional requests. Without documented placement criteria, OSD’s legislative affairs office will continue to have difficulty in ensuring consistency in the department’s dealings with interested committees and members of Congress requesting DOD fellows. Officials in OSD’s legislative affairs office said that having documented placement criteria would help in the decision- making process and would better position the office to defend its decisions with congressional members who were denied a request for a DOD fellow. The military services believe that they derive benefits from their fellowships and training-with-industry programs, but they are not well positioned to determine the extent of these benefits for four main reasons. First, not all of the services conduct periodic program reviews, as required by their service guidance, for some or all of their programs; and the few reviews that have been conducted have not been comprehensive. For example, they do not ensure that the needs that prompted the program and the goals of the program are being met. Second, the services do not have clear guidance as to what qualifies as a follow-on utilization tour, or criteria for when such a tour can be postponed or waived, and this limits their ability to know the extent they are deriving a return on their educational investment. Third, the services do not know their overall program costs, including both direct and indirect costs, and therefore it is difficult to know whether these programs are cost-effective. Fourth, some of the services do not have written agreements or memoranda of understanding with the non-DOD host organizations providing the educational opportunity that spell out both parties’ roles and responsibilities—thus contributing to unclear expectations about the DOD benefit to be derived from these programs. Collectively, these limitations diminish the services’ assurances about the benefits they derive from their fellowship and training-with-industry programs. The services observed, however, that they use certain other program management practices to ensure that they derive benefits from these programs. While each of the services has taken some steps to review its programs, none has conducted both periodic and comprehensive reviews of its fellowship and training-with-industry programs. For some of the programs, there is no requirement to conduct periodic program reviews. With respect to fellowship programs, service review requirements vary, and compliance with them has been uneven. Further, for those services that have conducted program reviews, the reviews have not been comprehensive and have not included assessments of the status of the program against program goals. As a result, some services have limited assurances that their fellowship or training-with-industry opportunities meet desired program outcomes, address emerging mission requirements, or provide the best venue for developing needed personnel skills. We have previously reported on the importance of having systematically planned evaluations of training and development programs. In addition, evaluating programs through periodic and comprehensive reviews enables an agency to identify problems and improve a program, as needed. The services’ variance in review requirements for these programs is shown in table 4. Of the service programs that require a periodic review, Navy officials said that they conducted program reviews for their nonlegislativelegislative programs, and Marine Corps officials said they have conducted one nonlegislative program review. Two of the services—the Army and the Air Force—have not fully conducted periodic reviews of their and programs as required in their program guidance. Although the Army is required to review its nonlegislative fellowships, it has not conducted reviews recently. In 2011, the Army began reviewing a portion of one type of nonlegislative fellowship—the Senior Service College Fellowships— and officials said they expect to complete this portion of the review in early 2012, and to review the other fellowship programs in the near future. Air Force officials, on the other hand, informed us that their requirement for a biennial review of fellowships is an error in their instruction, and they plan to write this requirement out of the next version of the instruction. Further, these officials said that the review requirement applies only to graduate education programs—not fellowships. Therefore, the Air Force has not periodically reviewed its fellowship programs, although Air Force officials said that information on fellowships is incorporated into their annual process for designating educational and developmental opportunities. None of the services has comprehensively reviewed its programs. A comprehensive review, as we have previously reported, may include the following four elements: quantifiably measuring the extent to which the program is meeting program goals; validating that fellowships and training-with-industry programs meet current and emerging requirements; incorporating feedback from program participants and host organizations to improve the program; and documenting the results of the review to enable the service to modify its programs, as needed. However, we found that not all services are including these four elements in their reviews, as described below: Using quantifiable performance measures for meeting their stated goals for their programs. The services have not measured the extent to which their programs are meeting their goals, because they have not developed quantifiable performance measures for most of their program goals. We have previously reported on the importance of agencies developing and using performance measures to ensure accountability and assess progress toward achieving results aligned with goals. The services vary in the extent to which they review their programs to measure against program goals. For example, an Army official said that the Army has not assessed the program using quantifiable performance measures for its legislative fellowship program, but the official believed that a measure that tracks the completion of utilization tours could be used to assess the extent to which they are meeting the goal of having a pool of officers from which some may be selected for future utilization in the field of congressional liaison. Similarly, a Navy official with the Cyber Federal Executive Fellowship Program said that the utilization tour could be used as a performance measure for meeting the program’s goals, but they plan to use it as a qualitative, not quantifiable, performance measure. The Marine Corps Legislative and Commandant of the Marine Corps Fellowship Programs have not created quantifiable performance measures either, although officials said they use qualitative measures, such as discussions with fellows and senior leaders, to assess the programs. The Army and Air Force have taken some steps to develop quantifiable measures for some programs. For example, in 2011 the Army surveyed fellows in the Senior Service College Fellowship Program to assess the extent to which these fellowships met their objectives. The Air Force Fellowship Program surveys its host organizations on the percentage of time the fellows spend on the Air Force’s service-specific competencies—used by Air Force officials as indicators that the program is meeting its goals and objectives. However, the Air Force has not developed quantifiable goals with which these survey results can be compared. Without reviewing their programs to assess progress against program goals, the services are limited in their ability to determine the effectiveness of their programs. Validating that programs meet current or future mission requirements. Some of the services have not periodically validated that the fellowship or training-with-industry assignments continue to address current or emerging mission requirements. The DOD instruction states that the secretaries of the military departments are responsible for ensuring that the current assignments using fellowships and training- with-industry positions meet the intent of the program and continue to meet military department and DOD requirements or anticipated needs. We have reported that successful organizations match training and development programs to their specific needs and capabilities.While the Navy and the Marine Corps have periodically reviewed their programs to ensure that they meet current and emerging needs, the Army and the Air Force have not. The Marine Corps has informally validated its programs through a process that looks at the costs of the program, updates the list of fellowship host organizations, and determines which fellowships are available for the upcoming year. Although the Army’s instruction for nonlegislative fellowships requires that each fellowship program be reviewed for continued relevancy and to determine that it meets stated criteria and objectives, the Army has not conducted this review regularly. We note that Army officials stated that in their current review of a portion of the Senior Service College fellowships that they plan to revalidate the need for each or the uniqueness of each host organization. While the Air Force has not revalidated that its fellowships meet current or emerging needs through periodic program reviews, it stated in its December 2011 report to USD P&R on nonlegislative fellowships that program oversight and management ensure validation that the programs fulfill a present need, anticipated requirement, or future capability. Incorporating feedback. While all of the services have collected some feedback from fellows and host organizations, they have not consistently incorporated this feedback into their program reviews. We have previously reported on the importance of agencies measuring training participants’ reactions to and satisfaction with their programs, and of measuring changes in their knowledge, skills, and abilities after their participation in the programs. The Army and the Air Force have conducted postfellowship surveys for some fellowships to gather information on the fellows’ experiences. While these surveys provide information on the experience of the fellowship, they do not provide information on how the fellowship experience has been of benefit to the fellows’ careers from a perspective of several years later. We note that the Marine Corps has conducted surveys to collect feedback from both recent and earlier fellows. In addition, some of the services gather feedback from fellows on their experiences through periodic reports to the program office. All of the services monitor their fellows and collect feedback through intermittent meetings with them. The Air Force also collects feedback from host organizations through surveys and periodic site visits, and the Army has recently conducted site visits to some of its host organizations. Across the services, however, there has not been consistent incorporation of feedback in program reviews. Documenting program accomplishments. Although some of the services said that they document a portion of their review efforts, no service has documented accomplishments that covered an entire program review, or the entirety of its programs. We have previously reported on the importance of using program evaluations to make fact-based determinations of the program’s impact, and to reallocate or redirect resources accordingly. Service officials said they use or plan to use the information they collect to make changes to their fellowship and training-with-industry programs. Army officials said they plan to use their ongoing review of a portion of the Senior Service College fellowships to refine the Army’s portfolio of senior- level fellowships. Prior to the review, the Army had identified problems with the fellowship programs primarily by incident, either with the fellow or with the host organization. Marine Corps officials stated they have used feedback from the fellows to decide upon which host organizations to keep or to discontinue in their portfolio. Air Force officials said they have analyzed feedback from fellows and host organizations to make decisions on which host organizations to discontinue when overall education funding was reduced. Documentation of these results would enable the services to demonstrate that they have conducted the review and what steps, if any, they may take to improve the program. Some of the services informed us that they intend to do more to review their programs. For example, as part of its ongoing review the Army plans to match fellows with host organizations that will enable them to research issues identified by Army leadership as critical problems facing the Army. A Navy official said that although the Navy has not yet conducted its first review of the Cyber Federal Executive Fellowship, it plans to survey 2011 fellows in the fall of 2012, a year into their utilization tour, to assess how the fellowship benefited their careers. Marine Corps officials said that they are examining a more formal process to supplement their current informal annual fellowship reevaluation practices. Without conducting periodic and comprehensive performance reviews, the services’ ability to determine the benefit they derive from fellowship and training-with-industry programs will remain limited. The services inconsistently implement a DOD requirement to have utilization tours—assignments where program participants can apply knowledge and skills gained through fellowship and training-with-industry programs—because they do not have clear guidance as to what qualifies as a follow-on utilization tour, and they do not have criteria for when to postpone or waive this requirement. After a participant’s completion of the fellowship or training-with industry opportunity, the services are expected to assign the participant to a tour that enables him or her to make use of the newly gained skills, knowledge, and abilities, according to DOD guidance. Service officials described utilization tours to us as a benefit that DOD gains from conducting its fellowship and training-with-industry programs. For example, both Army fellowship policy documents and officials with whom we spoke described utilization tours as a mechanism for maximizing the benefit derived by the Army. Navy officials described utilization tours as a way to receive a return on their investment and to meet the intent of the programs, given that they reinforce skills, promote continued learning, and serve to meet the requirements of the Navy. Air Force policy documents and officials described utilization tours as a way to ensure that officers use the skills they have developed. However, the services do not implement utilization tours for fellowships consistently, either within or across services. For example, within the same Air Force fellowship program, some fellows have a mandatory utilization tour, while for others it is a recommendation. According to Air Force officials, the decision as to whether a utilization tour is mandatory or recommended is made by the Air Force office responsible for fellowship policy on a case-by-case basis at the time it initiates the fellowship. Air Force officials said they do not have established criteria for determining whether a utilization tour is mandatory or recommended. Similarly, the Army requires utilization tours for some of its fellowship programs, such as the legislative fellowship, but not for others. In contrast, the Navy and the Marine Corps generally require utilization tours for all of their fellowship programs, and all of the services generally require utilization tours for their training-with-industry programs. Service officials said that inconsistency across the services is attributable, in part, to the absence of a clear definition as to what constitutes a follow- on utilization tour in the DOD instruction. A Navy official said that the differences in how the services implement utilization tours are a result of not having a clear definition of the requirement. Similarly, Marine Corps officials described fulfilling utilization tours as a judgment call, and one Marine Corps official said there has been an internal debate as to what assignments should count as utilization tours, and within what time frame such tours should be assigned. Also, the services’ postponement or waivers of the requirement to perform utilization tours occur inconsistently, because the services do not have criteria to determine when it is appropriate to postpone or waive this requirement. As a result, the services have different implementation practices for postponing or waiving the utilization tour requirement. Generally, the services postpone or waive the utilization tour requirement on a case-by-case basis. For example, according to officials the Army postpones or waives the requirement for most, but not all, of its fellowships. For one Army fellowship program—the Senior Service College Fellowship Program—the former Army Chief of Staff provided written guidance in 2010 indicating that specific requirements for utilization tours upon completion of fellowships should not be imposed. Army officials stated that the utilization tour requirement was viewed as adding more stress and burdens to the officers during a period of high operational tempo and frequent deployments, and that utilization tours for these fellows were viewed simply as a recommendation for postfellowship assignments. In the case of one Navy fellowship program, on the other hand, Navy guidance generally requires a utilization tour, and that a high-level Navy Personnel Command official must approve any exceptions; the regulation does not provide any criteria for such exceptions. Service officials from all of the services stated that they postpone assigning a utilization tour if the tour would interfere with another career-advancing assignment, such as being selected for a command position. Some service officials said that improving the consistency of their implementation of the utilization tour requirement could increase assurances that they derive benefits from fellowship and training-with- industry programs, even though their reasons for not requiring utilization tours vary. Army officials informed us that they are examining steps to better strategically link utilization tours with the specific research focus of the fellow, and might begin to require utilization tours for their Senior Service College fellows now that they expect fewer Army officers to be deploying overseas. Navy officials said they are hoping to develop a capability to better track utilization tour completion to better ensure that they are getting a good return on investment. Marine Corps officials said they also plan to improve their process for tracking legislative fellowship utilization tours in 2012. In addition, Air Force officials said that implementing the utilization tour requirement more consistently would benefit their service. We have previously reported on the importance of having agency training and development managers remove barriers that can impede the use of knowledge and skills gained in training, so as to improve performance on the job. Without having a common understanding of the follow-on utilization tour requirement and its waiver criteria, the services’ ability to determine the benefit they derive from fellows and training-with-industry participants using their newly developed or enhanced skills in these tours will remain limited, and the circumstances surrounding each waiver will remain uncertain. In addition, having assurances that utilization tour requirements are being met can better position the services to justify the use of appropriated funds for these programs and can better ensure that the services are meeting the intent of the programs by filling positions that meet current and anticipated needs. The services do not know the overall costs of their fellowship and training- with-industry programs, including both direct and indirect costs. We have previously reported on the importance of ensuring that training and development efforts are cost-effective relative to their anticipated benefits. Without having reviewed the overall costs of the programs, the services cannot be assured that they are cost-effective, and they will not have the information necessary to aid decision makers in managing scarce training, development, and education resources, particularly due to anticipated lower levels of defense spending. The services are not required by the DOD instruction to report on the overall costs of the program. However, while not uniformly required across the board, some of the service instructions do require reviewing a portion of the costs of their respective programs. The Army requires that host candidate organizations, within their proposals to become hosts, As part of a periodic review present the estimated costs to the service. for two of the Navy’s fellowship programs—the Cyber Federal Executive Fellowship and the Federal Executive Fellowship—the Navy reviews the cost of execution, utilization, and any fiscal surpluses or constraints that might affect future execution of the program, to help inform future decisions about it. For one of its fellowship programs—the Commandant of the Marine Corps Fellowships—the Marine Corps requires that a cost estimate is determined for tuition, fees, and travel costs to aid in budget planning. One Air Force instruction requires that all fellowships be reviewed for their cost efficiency, but, as previously mentioned, Air Force officials stated that this requirement to review fellowships is an error in the Instruction. The services know some of the direct costs associated with their fellowship programs, such as the tuition and fees paid to the universities and think tanks that host fellows, but not all of them. Service officials said that the tuition or fee paid to a host organization was used for the administrative, academic, and technological support provided to the fellow, as well as for the facilities and resources at the host organization. Some of the direct costs of the program include those associated with travel, research, and orientation. We collected information from the services on the tuition or fee amounts paid to host organizations for fellowships. We estimated that for the academic year 2010-2011, the services paid approximately $2.9 million in tuition and fees for approximately 270 fellows, of which the Army paid approximately $1.2 million; the Navy paid approximately $208,000; the Marine Corps paid approximately $378,000; and the Air Force paid approximately $1.1 million.fellowship host organizations. The services paid a tuition or fee to more than half of their For two of the services, officials said that knowing the direct costs of their fellowship and training-with-industry programs is difficult, given that the programs are financially managed by several different organizations. For example, funding for the cost of a fellowship may come from the proponent that sponsors a fellowship, or from general student and educational funding, or from other components, such as the National Guard or Reserve, depending on the fellowship and the service. In addition, the costs of the program may extend across more than 1 fiscal year, as the programs are based on academic or calendar rather than fiscal years. Service officials said that they do not know the indirect costs of the programs, such as administrative support, faculty, and permanent change of station costs, and that there are several challenges in determining those costs. In addition, some officials said that for their programs indirect costs may include the salary and benefits of the officer while participating in the fellowship or training-with-industry program. Service officials said that the indirect costs of the fellowship are not easily identified. For example, an Army official said the Army does not have a financial management system that would allow it to distinguish the administrative and faculty costs directed toward Army War College students from those directed toward fellows. For many of the programs, management and faculty support of fellows and training-with-industry participants is an additional duty, not the sole duty, of a service official. Without knowing the direct and indirect costs of these programs, the services are unable to compare the cost-effectiveness of attending one type of educational opportunity versus that of another type. Service officials said they experience similar challenges in knowing the overall costs of servicemembers attending military schools, or other professional development and training opportunities. Some service officials said that having more information about the overall costs of the programs would be beneficial because it would better inform them on how many fellowship and training-with-industry opportunities they can afford to devote educational and training resources to, and would provide better information for budgeting. Some services have not established or reviewed agreements or memoranda of understanding that would outline the responsibilities of the services and the think tank, university, and federal agency host organizations for fellowships, but several service officials we interviewed said this would be a best practice to incorporate. We have previously reported on the importance of having management controls to provide accountability for the use of government resources, and to outline the responsibilities of each party. We have also reported on the importance of having policies and procedures to ensure that training and development efforts and expectations are discussed and understood by managers, supervisors, employees, training coordinators, and others. The requirement for having written memoranda of understanding between each party varies across the services and by the type of training involved. DOD’s Instruction 1322.06 does not require written memoranda of understanding for fellowships, but it does require a written agreement for training-with-industry programs. The instruction states that before the start of a training-with-industry assignment there must be a written agreement shared by the private sector host, the employee, and the DOD component concerned. The services required agreements with host corporations. Although not required by the instruction for fellowships, the services have varied in the extent to which they have established memoranda of understanding with host organizations. The Army and the Marine Corps have established such In contrast, the memoranda for a small proportion of their fellowships.Navy and the Air Force have established memoranda of understanding with most of their fellowship host organizations, stating that these documents help them to ensure that the host organization understands the roles and responsibilities of each party as well as the key learning objectives of the program. The Marine Corps does not have a training-with-industry program. The services have also varied in the extent to which service guidance require these documents to be reviewed. The respective guidance for one Navy and one Army fellowship program require that memoranda of More specifically, the Navy’s Federal understanding be reviewed.Executive Fellowship requires that its memoranda of understanding be reviewed every 2 years; and the Commandant of the U.S. Army War College is required to review and make recommendations on all fellowship proposals and memoranda of understanding for the Senior Service College Fellowships to ensure that the senior-level educational criteria are met. However, since the Army does not have memoranda of understanding for a majority of its fellowships, such reviews have rarely occurred. Several service officials cited establishing and reviewing memoranda of understanding as a good management practice. One Navy official said that establishing and reviewing memoranda of understanding is a best practice that should be encouraged, saying these agreements would clear up any misunderstanding about the structure of the individual program. Another Navy official had positive comments about establishing and reviewing memoranda of understanding, saying that establishing these memoranda constitutes a good practice because they define roles and responsibilities, provide protections for ethics and payments, and make clear the objectives of the fellowship. Furthermore, the Army and the Marine Corps both agreed that establishing and reviewing memoranda of understanding with host organizations would be beneficial. Army officials said they plan to establish and update the memoranda for all their fellowships and to update their policy accordingly. Army officials also said that these memoranda would provide clarity on the costs of fellowships each year and thus facilitate better accounting and budgeting for their program. Further, Army officials said the memoranda would provide clarity on what makes each host organization unique, the purpose of the fellowship, and the roles and responsibilities of the Army and the host organization. Memoranda of understanding serve as a management control and document key information and expectations between the service and the host organization. As a feature of documenting key information and expectations, memoranda of understanding can provide protections against potential conflicts of interest. Several Navy memoranda of understanding we reviewed contained provisions that pertained to fellows working on certain federal projects. For example, a memorandum of understanding for the Federal Executive Fellowships says that the fellow may not work on any contract or procurement studies that would violate federal standards of conduct, especially those pertaining to conflicts of interest. Without establishing and reviewing these memoranda, the services cannot be assured that expectations, including safeguards against potential conflicts of interests and criteria for appropriate fees and projects, are clearly understood by the host organizations and the fellowship participants. While the military services are not well positioned to determine the extent of their benefit from fellowship and related training programs, service officials believe that they do obtain benefits from these programs. For example, officials said these programs provide officers with a career- broadening developmental experience; enable them to conduct research on topics that are important to the military; help them develop critical and strategic thinking skills; help them gain perspectives from the civilian sector; build civilian-military relationships; and develop more responsive leaders during times of significant change in the global security environment. Officials also believe that the use of fellowship and training- with-industry programs affords DOD flexibility in meeting new or emerging educational and training needs that sometimes cannot be met within the military educational structure—for example, in the rapidly developing new approaches and techniques of the cyber security area. Service officials said they use several program management practices to enhance the benefits derived from fellowship and training-with-industry programs, including the following: Competitive selection: Service officials said that having a highly competitive selection process that rank-orders officers who are eligible to compete for many fellowship and training-with-industry opportunities helps to produce a selection of highly qualified officers for these opportunities. Officials said the process better ensures that the right officer is selected for a given fellowship or training-with- industry program, which ultimately benefits the officer’s career as well as accomplishes DOD’s mission though the experience gained. In our review of the selection process, we found that the preferences of career field managers, senior leaders, and the officers themselves were considered, for many of the services, in the selection of officers for fellowship or training-with-industry programs. Orientation and ethics counseling: Service officials said that conducting orientation before the start of a fellowship helps to prepare fellows for their program by introducing them to key contacts, program requirements, and expectations for the year. In addition, officials stated, the ethics briefings given at orientation and availability of subsequent counseling provide assurance that fellows understand what is expected of them and are informed as to how to handle potential conflicts of interest. We reviewed seven ethics briefings and found them to contain essential elements, such as guidance on accepting gifts and information on contacting DOD legal officials with questions about ethics issues. (See app. IV for our review of these briefings.) Research: Service officials said they have assurances that they receive a benefit by requiring many of the fellows to conduct research and write papers on topics that are important to each respective service, as identified by senior leaders. At orientation, some of the services have senior leaders speak about emerging topics that could be better researched to benefit the service. For academic year 2010-2011, for example, DOD fellows researched enhancing DOD’s acquisition and logistics processes, as well as countering irregular threats such as piracy. Service obligations: Service officials informed us that they had general procedures to provide some measure of benefits obtained when officers complete their additional service obligation as required by statute or DOD guidance for participating in certain fellowships or training-with-industry programs—that is, that the fellow will serve in the military for a period at least three times the length of the period of the fellowship education or training-with-industry opportunity. We found that all of the services generally have (1) policies on the requirement; (2) special coding capabilities for personnel record- keeping and monitoring; (3) a formalized waiver process to better ensure that servicemembers do not retire or separate without fulfilling service obligations; and (4) the potential ability to recoup certain educational costs. While many service officials asserted that these practices increase the likelihood that benefits are derived from fellowships and training-with-industry opportunities, some also said they could still improve upon these practices to have greater assurances that DOD derives benefits from these programs. According to DOD, the intent for fellowships and training-with-industry programs is to help fulfill a present need, anticipated requirement, or future capability that contributes to the effectiveness of the department’s mission. DOD has used these programs for at least 1,797 mid- to senior- level officers across the services over the past 5 fiscal years. Even as the war in Afghanistan is winding down and reductions in personnel end- strengths occur, servicemembers will still be deployed globally, and DOD will have to carefully balance operational assignments with identified needs for professional military education, both within DOD’s professional military educational system and outside of DOD, in fellowship and training with industry programs. However, until OSD gains improved visibility into these programs and the military services are better positioned to know that they achieve their intended benefits and are cost-effective, DOD will continue to face challenges in ensuring that it is deriving sufficient value from these programs. To improve oversight and management of DOD’s fellowship and training- with-industry programs, we are making 11 recommendations to the Secretary of Defense. To help ensure compliance with DOD Instruction 1322.06 and thus enhance DOD’s visibility over all of the fellowship and training-with- industry programs, and to promote a shared understanding across the military services of what is expected in meeting the instruction, the Secretary of Defense should direct the Under Secretary of Defense for Personnel and Readiness to take the following five actions: (1) develop a mission statement that clearly defines the respective purposes of the legislative and nonlegislative fellowship and training-with- industry programs to be in a better position to know the extent to which desired program outcomes are being achieved; (2) more consistently enforce the DOD instruction’s requirement on the submission of annual reviews from the military services on these programs; and (3) collaborate with the military departments to ensure that each service has designated an office to be responsible for compiling information on the legislative and nonlegislative fellowship and training-with-industry programs for the annual reports required in the DOD instruction. In addition, to enable DOD to develop a more reliable inventory for these programs, the Under Secretary of Defense for Personnel and Readiness should: (4) clarify in the DOD instruction the definition of a fellowship; and (5) specify in the DOD instruction common reporting requirements for the annual report, and clarify which fellowship and training-with-industry opportunities should be included in this report, to facilitate the collection of consistent information on these programs across the military services. Additionally, to facilitate OSD’s ability to perform its oversight responsibility for the legislative fellowship program and to better ensure consistency in the department’s dealings with interested committees and members of Congress, the Secretary of Defense should direct the Assistant Secretary of Defense for Legislative Affairs to take the following two actions: (6) clearly delineate in the DOD instruction the roles and responsibilities for overseeing DOD’s Legislative Fellowship Program, and (7) develop documented placement criteria for legislative fellows. Finally, to better position DOD to determine the extent of the benefits it derives from legislative and nonlegislative fellowship and training-with- industry programs and better assess whether fellowship and training-with- industry programs offer the best venues for developing needed personnel skills, the Secretary of Defense should direct the Secretaries of the Department of the Army, the Department of the Navy, and the Department of the Air Force to take the following four actions: (8) perform periodic and comprehensive program reviews that assess the progress using quantifiable measures, validate that programs continue to meet current or emerging needs, incorporate feedback from program participants and host organizations, and document the results of reviews; (9) clarify guidance for determining what qualifies as a follow-on utilization tour, and establish criteria to determine when a utilization tour is needed or, conversely, when it can be postponed or waived; (10) determine the direct costs of these programs by periodically obtaining and analyzing overall direct program costs, and explore the feasibility of estimating indirect program costs; and (11) establish and periodically review fellowship written agreements or memoranda of understanding to document key information and expectations between the services and the host organizations, such as fellowship objectives, criteria for evaluating the appropriateness of fees or tuition charged to the military departments , and criteria for evaluating the appropriateness of the projects involved. In written comments on a draft of this report, DOD concurred with the 11 recommendations we made to improve OSD oversight and strengthen the military services’ management of its fellowships and training-with- industry programs. DOD also outlined actions that it plans to take for each recommendation, which it stated will improve the issues we identified in the report. DOD’s comments appear in their entirety in appendix V. We are encouraged by the department’s action plans that carry the intent to implement our recommendations. For example, we made 7 recommendations to OSD designed to enhance OSD’s oversight and visibility over these programs, for which OSD provided its intended actions such as making planned revisions to its instruction governing these programs by including a mission statement, a standard format for the military services to follow when completing the required annual report, and a date when this report is due to OSD—all actions we believe meet the intent of these recommendations. We also made 4 recommendations to the military service secretaries to strengthen its management of these programs, which again OSD provided a series of steps it plans to take in response to our recommendations. OSD’s action plans included such steps as working with the military services to develop qualitative or quantitative measures that the department believes will best meet the services needs for these programs. Again, we are encouraged by the department’s action plans as stated in OSD’s comments to our draft report in meeting the intent of our recommended actions to strengthen these fellowship and training-with-industry programs. Finally, DOD provided technical comments, which we considered and incorporated where appropriate. We are sending copies of this report to the appropriate congressional committees. We are also sending copies to the Secretary of Defense; the Under Secretary of Defense (Comptroller); the Under Secretary of Defense for Personnel and Readiness; the Secretaries of the Army, Navy, and Air Force; and the Commandant of the Marine Corps. This report will also be available at no charge on our website at http://www.gao.gov. Should you or your staff have any questions concerning this report, please contact me at (202) 512-3604 or farrellb@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors are listed in appendix VI. To determine the statutory provisions that authorize the Department of Defense (DOD) fellowship and training-with-industry programs, we reviewed and assessed relevant laws and regulations that authorize and govern these programs. We also interviewed officials with the Office of the Under Secretary of Defense for Personnel and Readiness (USD P&R) and attorneys from OSD’s Office of General Counsel for their perspectives on laws and regulations governing these programs and obtained written responses from OSD General Counsel on these authorities. To determine the extent of OSD’s visibility over these programs, we obtained and assessed DOD instructions and service guidance governing its fellowship and training-with-industry programs to determine the policy oversight responsibility and other requirements. We interviewed officials from USD P&R and the Office of the Assistant Secretary of Legislative Affairs to obtain information about their roles and responsibilities with respect to the Legislative Fellowship Program. We also interviewed officials from USD P&R with knowledge of the service reporting requirements from the DOD instruction on fellowships and training-with- industry programs. We collected available service reports that corresponded to these requirements, and assessed the extent to which USD P&R has established guidelines for information to include in these reports. We obtained information on the extent to which USD P&R was aware of the services’ compliance with statutory and regulatory requirements, such as the service obligation commitments and reporting requirements. Also, we assessed the extent to which USD P&R had a complete inventory of its fellowship and training-with-industry programs. We collected data on the number of military officers and the types of fellowship and training-with-industry programs from each of the military services and from OSD. We relied primarily on data from the military services, as they were able to provide more detailed information on these programs. To assess the reliability of each of the military services’ program data, we obtained information on (1) the systems used to maintain their data and these systems’ ability to record and report on these data, and (2) the quality control measures in place to ensure that the data were reliable for our reporting purposes. We also interviewed some service officials who were responsible for compiling these data. We identified some data limitations within these data and cross referenced with other documentation where available. We found the fellowship and training-with-industry data provided to us by the services to be sufficiently reliable for providing contextual information on the minimum number of participants; however, these data do not allow us to provide the actual totals. To determine the extent to which the military services are able to determine that they derive benefits from these programs, we obtained and assessed service guidance, collected and reviewed information on service processes and practices used to manage their programs, and interviewed service officials. We collected and analyzed information on the extent to which the services reviewed their programs, including completion of periodic program reviews and whether the reviews assessed the program against program goals using quantifiable performance measures; validated that the program continued to address current or emerging mission requirements; incorporated feedback from program participants or host organizations; and documented the results of the review. We collected data and information on utilization tours and on the extent to which the fellows are assigned to them. We also collected information on the extent to which the services tracked the costs of these programs and established and maintained memoranda of understanding with host organizations. We obtained information from each of the military services on the tuition or fee paid to fellowship host organizations for fellowships in academic year 2010-2011. We also obtained and evaluated seven ethics briefings that DOD provided to fellows and training-with-industry participants prior to the start of their selected program with a non-DOD host organization. We identified the key elements in these briefings and evaluated them for comprehensiveness. In doing so, our Office of General Counsel staff from the Ethics Office— an office that provides guidance to our employees on ethics, conflicts of interest, impairments to independence, and related conduct rules and issues—reviewed and assessed the briefings we obtained from the military services. We also selected and interviewed a non-probability sample of four legislative fellows and six think tank fellows to obtain a firsthand understanding about the benefits of these educational and training opportunities from the perspective of the officer. We selected fellows from each service who were either current or had recently completed the fellowship at the time of our interviews, who were available for interviews; and who were situated in proximity to the Washington, D.C., area. We selected fellows from all four services and from a variety of different host organizations, focusing on think tanks and on legislative fellows in response to the interest expressed by Senate Armed Services Committee staff. In addition, we interviewed a non-probability sample of nine non- DOD host organizations to obtain their perspectives on the educational benefits provided to military officers. We selected host organizations that were hosting a fellow at the time of our interviews; that were available for interviews; and that were situated in proximity to the Washington, D.C., area. To provide more in-depth examples of think tank fellowships, we selected three of the host organizations that hosted a fellow from each of the services and reported on the fees associated with the fellowships, and on the extent to which the services established memoranda of understanding with those organizations. We visited or contacted the following organizations during our review: Office of the Secretary of Defense Under Secretary of Defense for Personnel and Readiness Office of the Deputy General Counsel Assistant Secretary of Defense for Legislative Affairs National Defense University Manpower and Reserve Affairs, Military Personnel Policy Congressional Operations Division Combined Arms Center Department of Academic Affairs, Army War college Interagency Student Division Leadership Development Division Senior Leaders Division Strategic Leadership Division Retirements and Separations Division Manpower and Reserve Affairs, Military Personnel Policy Education Branch Distribution Management Graduate Education and Training Placement Information Dominance Corp Legislative Liaison Office Office of Supply Corps Personnel Congressional Fellows Program Office Marine Corps University Officer Assignments Branch Department of the Air Force Congressional Support Branch Colonel Management Office Force Development Fellowships Program Office Developmental Education Branch Officer Promotions, Appointments, and Selective Continuation Branch Learning Division Acquisitions Career Management Air Force Research Institute, Air University American Enterprise Institute Atlantic Council Brookings Institution Center for New American Security Center for Strategic and International Studies. Council on Foreign Relations. Washington Institute for Near East Studies Institute of World Politics Joint Center for Political and Economic Studies We conducted this performance review from March 2011 through April 2012 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. For each DOD fellowship program, there are various phases that comprise the program, depending on its life-cycle as shown in figure 2. Typical phases of a life-cycle for fellowships are as follows: 1. Different organizations within each service identify present and emerging needs of the service that the officers should be educated in and trained on. For example, a service identifies an emerging need for more cyber security based on recent events. 2. The service policy or program offices establish the fellowship opportunity that provides the training and skills to meet the identified need. For example, once the cyber security emerging need was identified, the Navy policy office established a fellowship program with leading organizations in that field outside of DOD. 3. Generally, the personnel command or center selects officers to participate in the fellowships through a competitive selection process that ranks eligible officers for selection. 4. The fellows start their fellowship with a university, think tank, corporation, federal agency, or congressional committee or member office, generally with DOD or the service providing orientation and ethics counseling. 5. During the fellowship, the program office with primary responsibility for the fellowship monitors the fellows’ progress, assisting them, when applicable, with research associated with the fellowship. 6. At the completion of some fellowships, the fellows are generally required to submit a research paper or article. After completion, the fellow’s personnel command or center is responsible for assigning the participant to the postfellowship utilization tour, and for ensuring the fellow does not separate or retire from the military before completing the service obligation. 7. Reviewing the fellowship program is generally conducted by either the policy office or the program office. The review considers whether there are any necessary changes to incorporate into the program for the upcoming year. Although we discussed fellowships in terms of a life-cycle, the life-cycle for training-with-industry programs is similar to that of fellowships. The military services each have their own guidance or regulations that covers fellowships and training-with-industry programs, as shown in table 5. DOD legal officials presented ethics briefings to fellows during their orientation to the fellowship program. Each service provided a multiple- day orientation to fellows prior to the start of the fellowship and included a segment that covered ethics issues. Some services provided a general ethics briefing to all fellowship program participants at one time, and other services provided the ethics briefing separately to each fellowship program. Fellows participating in the Secretary of Defense Corporate Fellowship Program received two ethics briefings, one from their service and the other at the Secretary of Defense Corporate Fellowship Program orientation. In consultation with GAO’s Ethics Office, we identified key elements that would provide fellows with guidance on potential ethical issues and conflicts of interest they may face during their fellowship with a non-DOD host organization. We obtained seven examples of ethics briefings: two from the Army, two from the Navy, one from the Air Force, one from the Marine Corps, and one from the Secretary of Defense Corporate Fellowship Program. The key elements in the ethics briefings included: Relevant laws and DOD policies, including the Joint Ethics Regulation (JER), DOD 5500.7-R, the Standards of Ethical Conduct for Executive Branch Employees 5 C.F.R. Part 2635, and relevant sections of 18 United States Code , Chapter 11, Bribery, Graft, and Conflicts of Interest. Standards of Ethical Conduct for Executive Branch Employees 5 C.F.R. Part 2635 include Conflicts of interest Acceptance of gifts Unauthorized commitments purporting to bind the government Using public office for private gain Seeking outside employment Some of the briefings provided guidance on teaching, speaking, and writing, such as guidance on the proper use of disclaimers in those activities for which the opinions of the fellow do not represent DOD, and guidance on being compensated for those activities. For example, fellows were instructed not to take compensation for teaching, speaking, and writing if the activity was undertaken as part the fellow’s official position, or if the topic dealt with an ongoing or announced policy, program, or operation of DOD. In addition, some briefings provided guidance on engaging in political activities. For example, fellows were instructed that they were not permitted to use official authority or influence for interfering with an election; to be a candidate for civil office except as authorized; to participate in partisan political management, campaigns, speeches, articles, or conventions; or to promote or attend political dinners or fundraising events as an official representative of the Armed Forces. Furthermore, DOD provided the fellows with a point of contact for future questions and advice. In addition to the contact above, Laura Talbott, Assistant Director; Darreisha Bates; Maria McCollester; Erin Preston; Sara Olds; Terry Richardson; Amie Steele; Cheryl Weissman; Allen Westheimer; and Michael Willems made key contributions to this report.

Summary: The Department of Defense (DOD), which includes the military services, selects mid- to upper-career-level military officers to participate in fellowship and training-with-industry programs conducted at non-DOD organizations such as universities, think tanks, private corporations, federal agencies, and Congress. For some fellowships, the military departments pay a fee or tuition to the host organization. GAO was directed to review DODs use of these programs. GAOs objectives were to determine: (1) the statutory provisions that authorize DODs fellowship and training-with-industry programs for military officers, (2) the extent of the Office of the Secretary of Defenses (OSD) visibility over these programs, and (3) the extent to which the services are able to determine that they derive benefits from these programs. GAO analyzed relevant laws and DOD policies, collected data, and interviewed OSD and military service officials on their oversight and management roles and responsibilities for these programs. GAO determined that DOD primarily uses two explicit statutory authoritiessection 2603 of Title 10 of the United States Code, which authorizes servicemembers to accept fellowships from certain organizations, and section 2013 of Title 10 of the United States Code, which authorizes the training of servicemembers at nongovernmental facilitiesfor its fellowships and training-with-industry programs for military officers. For two specific types of fellowshipsLegislative and Interagencythe underlying authorities are less explicit than they are for the others. OSD has limited visibility over its fellowship and training-with-industry programs for several reasons. First, OSD has not developed a mission statement that would clearly define the respective key purposes for these programs. Having a clear mission statement is critical because it defines an organizations purpose in language that states desired outcomes. Additionally, OSD has not consistently enforced its requirement for the military departments to provide an annual report on fellowship and training-with-industry programs. Further, not all fellowship and training-with-industry programs have a designated office within each department for preparing the annual report. OSDs visibility is also limited by not having a reliable inventory of these various programs, and by not having a clear and commonly shared definition of a fellowship. Without improved oversight, OSDs visibility over the military departments compliance with its requirements governing these programs will remain limited. Additionally, visibility is limited over the legislative fellowship program in particular because oversight responsibilities are not clearly delineated, and because OSD does not have documented criteria for the placement of DOD fellows with the offices of congressional committees and members. OSD officials agree that such criteria would be helpful since it does not have enough available fellows to meet the full congressional demand. The military services are not well positioned to determine the extent of the benefits they are deriving from their participation in these programs for four principal reasons. First, not all of the services conduct periodic program reviews, as are required for some programs. In addition, the reviews that are conducted are not comprehensive in that they do not assess the program against program goals using quantifiable performance measures, review the needs that prompted the program, incorporate feedback from fellows into the review, or document the results of the review. Second, they do not have clear guidance as to what qualifies as a postfellowship assignmentan assignment that uses the skills and knowledge developed during the fellowship programor criteria for when such assignments can be postponed or waived, thus limiting the extent the services are able to determine they are deriving benefits from these programs. Third, the services do not know their overall program costs, so it is difficult to know whether these programs are cost-effective. Finally, some of the services do not have memoranda of understanding with the non-DOD host organizations, such as think tanks, so they cannot be assured that expectations are clearly understood and the intended benefits are obtained. Without better management controls, the services ability to determine the benefits of these programs will remain limited. However, service officials believe that they obtain benefits from fellowships and training-with-industry programs. GAO is making 11 recommendations to DOD for improving oversight and management of DODs fellowship and training-with-industry programsfor example, submitting DOD-required annual reports and performing service-required program reviewsthat would enhance OSDs visibility over the programs and better position DOD to determine the extent to which it derives benefits from them. In response to a draft of this report, DOD concurred with the 11 recommendations and stated its action plan to implement the recommendations.