Joint Professional Military Education: Opportunities Exist for Greater Oversight and Coordination of Associated Research Institutions
Released: 2014-03-10T00:00:00

Letter

Background

Overview of Professional Military Education

The primary purpose of PME is to develop military personnel, throughout their careers, for the intellectual demands of complex contingencies and major conflicts. The military services provide PME at their respective staff and war colleges. Each service educates service members in their core competencies according to service needs. Air Force colleges, for example, primarily teach air and space warfare. Similarly, Army, Navy, and Marine Corps colleges focus on land, maritime, and expeditionary warfare, respectively. DOD depends on the services’ PME institutions to develop personnel with these service-specific skills. However, the JPME program places emphasis on preparing leaders to conduct operations as a coherently joint force in complex operating environments.

Following the passage of the Goldwater-Nichols Act (the Act) in 1986, DOD developed JPME as a subset of learning within the PME program, to comply with the “joint” requirements outlined in the Act and subsequent legislation. Currently, JPME is provided at multiple sites across the country, including the services’ staff and war colleges and NDU. Together, PME and JPME, prepare service members in successive stages throughout their careers to engage intellectual challenges appropriate to increases in their ranks and responsibilities. See figure 1 for a map of service and joint colleges and universities where JPME is provided.

The military services are primarily responsible for overseeing PME at their respective staff and war colleges. As part of their oversight efforts, the military services’ leader-development efforts are included in education programs. For example, the Army’s Training and Doctrine Command serves as the executive agent for ensuring leader development is integrated into PME courses at the Army War College and the Army Command and Staff College. In contrast, JPME is overseen by the Joint Staff. The Joint Staff is responsible for developing the learning objectives for JPME and for accrediting the service staff and war colleges and the joint institutions to provide JPME coursework. The Joint Staff also has oversight responsibility of NDU. The Chairman of the Joint Chiefs of Staff is statutorily responsible for formulating policies for coordinating the military education and training of members of the armed forces.Military Education Coordination Council, which consists of representatives from the Joint Staff, the service and joint colleges and universities, and other JPME-accredited institutions, serves as an advisory body to the Joint Staff on joint education issues. The purpose of the council is to address key educational issues of interest to the joint educational community, promote cooperation and collaboration among the colleges and universities certified to grant JPME degrees, and coordinate joint education initiatives.

The Joint Staff conducts periodic assessments of the three statutorily mandated levels of officer JPME to ensure that the curricula being taught at service staff and war colleges and the joint institutions meet the prescribed joint educational requirements outlined in the Officer Professional Military Education Policy. JPME program includes curriculum components that JPME colleges and universities should follow to develop the knowledge, analytical skills, perspectives, and values that are essential for U.S. servicemembers to function effectively in joint, interagency, intergovernmental, and multinational operations. Moreover, Enclosure A of the policy states that senior officer studies at JPME-degree-granting colleges and universities should emphasize analysis, foster critical examination, and provide progressively broader educational experiences.

DOD Funds Various Types of Research, Including Research Conducted at JPME Research Institutions

CJCSI 1800.01D. subject-matter experts in their field of study. Table 1 provides a list of the 20 research institutions that are associated with JPME colleges and universities and are within the scope of our review.

Other DOD-funded organizations also conduct studies and analysis research. For example:

Federally Funded Research and Development Centers, such as the Center for Naval Analyses and the Institute for Defense Analyses, maintain capabilities to conduct research in core competencies in areas of importance to DOD, such as analysis, acquisition support, and research and development. According to a May 2011 memorandum from the Under Secretary of Defense for Acquisition, Technology and Logistics, the mission of Federally Funded Research and Development Centers is to provide DOD with unique capabilities in many areas where the government cannot attract and retain personnel in sufficient depth and numbers. The memorandum further explains that Federally Funded Research and Development Centers are a vital component of the department’s overall acquisition workforce because they operate in the public interest, free from organizational conflicts of interest.

Service-affiliated organizations, such as the Naval Postgraduate School and the Center for Army Analysis, provide research products to their parent services to help with decision making and analysis on critical issues facing the service.

DOD’s Regional Centers for Security Studies support DOD’s objective to build the defense capacity of partner nations. In our prior work, we reported that the Regional Centers’ activities include education, exchanges, research, and information sharing. For example, the George C. Marshall European Center for Security Studies conducts research on European security issues relevant to U.S. interests.

JPME research institutions receive funding through their colleges and universities and other departmental offices for their operations, which include research activities. Specifically, most JPME colleges and universities receive direct funding from their respective military service to fund their PME and JPME programs, and some of those resources are used to fund their JPME research institutions. For example, the Naval War College receives operation and maintenance and military personnel funding from the Department of the Navy as well as funds in the form of monetary gifts from the Naval War College Foundation. In turn, the college allocates some of those resources to fund its associated research institutions, such as the China Maritime Studies Institute. However, NDU receives operation and maintenance funding from defense-wide appropriations for its JPME program and research institutions. With these funds, JPME research institutions can support PME and JPME programs as well as the research needs of those entities that provide their funding.

For example, the Air University’s Center for Strategy and Technology produces research that is responding to key questions and topics of interest posed by the Chief of Staff of the Air Force.

Additionally, some JPME research institutions receive funding on a reimbursable basis from other departmental offices, such as the directorates within the Office of the Secretary of Defense, the Joint Staff, and the military services. These offices receive their own funding, such as research, development, test, and evaluation funds and operation and maintenance funds, which may be used in part to fund annual requirements for research projects. To fulfill these annual research requirements, funding may be allocated to JPME research institutions, Federally Funded Research and Development Centers, or think tanksconduct individual research projects in support of those offices’ annual research requirements. For example, NDU’s research institutions have received funding from the Office of the Secretary of Defense to study and build subject-matter expertise on issues related to Afghanistan and Pakistan.

DOD also funds science and technology–related research. According to testimony in April 2013 by Acting Assistant Secretary of Defense for Defense Research & Engineering, science and technology research is funded to mitigate new or emerging capabilities that could degrade U.S. capabilities, enable new or extended capabilities in existing military systems, and develop new concepts and technologies through science and engineering applications to military problems. Science and technology research is conducted under the auspices of the Office of the Assistant Secretary of Defense for Research and Engineering. According to DOD guidance, this office develops the strategies and supporting plans for utilizing technology to respond to DOD needs and ensures U.S. technological superiority. The office is the executive secretary for DOD’s Research and Engineering Executive Committee. This committee brings together leadership from the DOD components that have science and technology research investments for the purpose of strengthening coordination and enhancing the efficiency of research and engineering investments in areas that cannot be addressed adequately by any single component. Science and technology research comprises basic research, applied research, and advanced technology development.research institutions do not conduct science and technology research. Science and technology research is generally conducted by DOD laboratories associated with the military services, such as the Army Research Laboratory, and some Federally Funded Research and Development Centers, such as the Massachusetts Institute of Technology Lincoln Laboratory, among other organizations.

JPME Research Institutions Expanded from 2007 through 2011 Due to Several Factors, but Have Subsequently Contracted

JPME research institutions, particularly at NDU, experienced considerable growth in number, funding, and size in terms of staffing levels from fiscal year 2007 through fiscal year 2011 but have declined over the past 2 years. Several factors contributed to JPME research institution growth, including increases in reimbursable funding from outside offices sponsoring JPME research, the creation of new research institutions, and the realignment of institutions at some JPME colleges and universities. While a variety of factors contributed to the expansion of JPME research institutions, it has primarily been department-wide budget reductions that contributed to their decreases in number, funding, and size since 2011.

Number, Funding, and Size of JPME Research Institutions Grew from 2007 through 2011, Then Declined

JPME research institutions, particularly at NDU, experienced considerable growth in number, funding, and size in terms of staffing levels from fiscal year 2007 through fiscal year 2011, but have declined over the past 2 years. The following sections discuss overall trends in the number of research institutions at JPME colleges and universities from fiscal year 2007 through fiscal year 2013, as well as overall trends in funding and staffing levels for this same period. Appendix II provides more-detailed information for each of the 20 JPME research institutions from fiscal year 2004 through fiscal year 2013, as available.

Number of JPME Research Institutions

From 2007 through 2011, the number of JPME research institutions grew from 14 to 20. During this period, the number of research institutions at NDU increased by 3. At Marine Corps University and Air University, the number of research institutions increased by 2 and 1, respectively. Since 2011, however, the number of research institutions has slightly declined due to the disestablishment of the Center for Transatlantic Security Studies at NDU in 2012. Figure 2 shows the total number of JPME research institutions for fiscal years 2007 through 2013.

Funding for research institutions at JPME colleges and universities experienced growth from fiscal year 2007 through 2011. Specifically, total funding for JPME research institutions increased from about $31.0 million in fiscal year 2007 to about $47.7 million in fiscal year 2011. Much of the growth took place at NDU, where research institutions’ total funding increased by about 78 percent. Other JPME colleges and universities also experienced considerable increases in funding for the operation of their associated research institutions. For example, with the establishment of the Middle East Studies institute and the Center for Advanced Operational Culture Learning’s Translational Research Group in 2007 and 2010, respectively, funding for Marine Corps University’s research institutions increased from $156,000 in fiscal year 2007 to about $4.9 million in fiscal year 2011.

Since fiscal year 2011, funding for JPME research institutions decreased overall. Specifically, total funding for JPME research institutions fell by about 15 percent from fiscal year 2011 through 2013, from about $47.7 million to about $40.6 million. Much of the decline reflects decreases at NDU, where research institutions experienced a 21 percent decrease in total funding from about $21.4 million in fiscal year 2011 to about $16.8 million in fiscal year 2013. The Army Command and General Staff College’s Combat Studies Institute and the Center for Army Leadership also experienced considerable declines during this period, as total funding for both decreased by about 19 percent. Figure 3 provides total funding for JPME research institutions by JPME college and university for fiscal years 2007 through 2013.

Staffing levels at JPME research institutions also increased considerably from fiscal year 2007 through fiscal year 2011. Specifically, staffing levels, in terms of full-time equivalents, increased from 207 in fiscal year 2007 to 384 in fiscal year 2011, about an 86 percent increase. In particular, total staffing levels at NDU’s research institutions increased by about 58 percent during this period while other JPME colleges and universities also experienced growth in staffing levels. For example, total staffing levels at Air University’s research institutions increased from 19 to 97.

Since 2011, total staffing levels at JPME research institutions decreased from 384 to 310, about a 19 percent decrease. Much of the decrease is the result of a decline in staffing levels at NDU, where research institutions experienced a 31 percent decline during this period. Figure 4 shows staffing levels for JPME research institutions by JPME college and university for fiscal years 2007 through 2013.

Several Factors Contributed to JPME Research Institution Expansion and Subsequent Contraction

Several factors contributed to JPME research institution growth from fiscal year 2007 through fiscal year 2011, including increases in reimbursable funding provided by outside offices sponsoring JPME research, the creation of new research institutions, the realignment of institutions such that they were incorporated into JPME colleges and universities, and an increase in resources dedicated to research at some JPME colleges and universities. According to DOD officials, these increases occurred within the context of the then-ongoing operations in Iraq and Afghanistan, for which the Joint Staff and the military services expected the JPME research institutions to provide increased support to the warfighter. In particular, these factors led to an expansion at NDU’s research institutions during this period. For example, NDU’s research budget grew primarily due to increases in reimbursable research funded by outside offices such as the Office of the Secretary of Defense and the Joint Staff. Reimbursable funding provided to NDU’s research institutions increased from about $5.6 million in 2007 to about $14.6 million in fiscal year 2011, as shown in figure 5. Specifically, reimbursable funding for NDU’s Center for Technology and National Security Policy’s research increased from about $3.8 million in fiscal year 2007 to about $6.9 million in fiscal year 2011. Additionally, funding for NDU’s Center for Study of Weapons of Mass Destruction, which is funded entirely on a reimbursable basis, more than doubled from $1.5 million in fiscal year 2007 to $3.1 million in fiscal year 2011.

NDU officials attributed the growth in reimbursable funding to the fact that research had evolved into a key area of emphasis for the university. These officials also noted that, as a result, faculty members at the research institutions were encouraged to pursue research directly funded by other DOD entities and other U.S. government agencies. To meet the increased demands for reimbursable research, particularly when in-house expertise did not exist, NDU increased the number of contractor and noncontractor researchers at its research institutions.noted that because a significant portion of the NDU workforce, including that of its research institutions, is made up of senior-level positions, researcher salaries contributed to the growth in NDU’s research budget.

We also found that funding for NDU’s research institutions increased as the result of the transfer of research institutions to NDU as well as the broadening of missions of other research institutions. For example, according to DOD officials, in an effort to better deliver education to DOD and other U.S. government personnel on issues related to ongoing operations in Iraq and Afghanistan, such as irregular warfare, counterinsurgency, and stability and reconstruction operations, the Center for Complex Operations was transferred from the Defense Security Cooperation Agency to NDU in 2009. Furthermore, funding for the Center for the Study of Weapons of Mass Destruction at NDU increased due to DOD’s decision to broaden the Center’s counterproliferation focus government-wide. Moreover, the Chairman of the Joint Chiefs of Staff designated the center as the focal point for weapons of mass destruction education for JPME. As a result, the Joint Staff began to fund the center in 2008 to perform that mission.

Other JPME colleges and universities also experienced considerable increases in research institution funding and staffing levels from fiscal year 2007 through fiscal year 2011 due to factors such as the creation of new research institutions and the realignment of others within JPME colleges and universities. For instance, according to Air Force documentation and officials, Air University’s increase in staffing levels for its research institutions can largely be attributed to the establishment of the Air Force Research Institute in 2008. According to an Air University official, the establishment of the Air Force Research Institute resulted from the consolidation and realignment of personnel from existing Air Force institutions as well as the creation of 18 new positions. Air Force officials also stated that staffing levels at the Air Force Counterproliferation Center and the Air Force Center for Strategy and Technology increased due to increased research requests from the Air Staff on nuclear and strategic-level research projects, respectively. Marine Corps University’s increase in staffing levels at its three research institutions can be attributed to the establishment of the Translational Research Group in 2010 and increases in staff positions within the History Division and Middle East Studies center.

While a variety of factors contributed to the expansion of JPME research institutions, it has primarily been department-wide budget reductions, including the implementation of sequestration in fiscal year 2013, that contributed to their decreases in number, funding, and size. For example, officials stated that decreases in funding for NDU’s research institutions and staffing levels resulted from overall reductions at the university due to declining budgets. Furthermore, according to officials, NDU’s budget for its research institutions came under increased scrutiny in 2011 with the issuance of a new mission statement for NDU by the Chairman of the Joint Chiefs of Staff that prioritized research that more-directly supported education over reimbursable research. Moreover, according to officials, the Joint Staff established full-time equivalent caps for both direct and reimbursable funding in 2012. As a result of actions taken to reduce NDU’s budget, NDU’s Center for Transatlantic Studies was disestablished in September 2012, and the decision was made in early 2013 to defund the Conflict Records Research Center beginning in fiscal year 2014.

Similarly, DOD-wide budget reductions contributed to decreases in the funding and size of other JPME research institutions. For example, Naval War College officials stated that its research institutions absorbed a majority of the college’s budget cuts since fiscal year 2011 because the college prioritized funds to support its principal education mission. Air University has also experienced decreasing budgets and staffing level reductions beginning in fiscal year 2011 through fiscal year 2013. Specifically, we found that the Air Force Research Institute’s total staffing levels decreased from a high of 81 in fiscal year 2011 to 61 in fiscal year 2013 as a result of overall reductions in Air Force civilian personnel.

DOD Is Limited in Its Ability to Assess the Performance of JPME Research Institutions

The extent to which DOD can assess the performance of JPME research institutions is limited by the lack of a comprehensive framework to systematically assess their performance in meeting PME and other departmental goals and objectives. With limited exceptions, the JPME colleges and universities, which have broad latitude in overseeing their associated research institutions, have not consistently established measurable goals or objectives linked with metrics to assess the performance of their associated research institutions. However, best practices state that achieving results in government requires a comprehensive framework that includes measurable goals and objectives and metrics for assess progress, consistent with the framework identified in the Government Performance and Results Act. Further, while there are mechanisms in place for overseeing JPME colleges and universities, such as the Joint Staff’s JPME accreditation process, these are focused on the quality of academic programs and not on the research institutions’ performance. There is no DOD-wide guidance that addresses the intended role of the research institutions in supporting PME or other departmental goals or assigns responsibilities for conducting reviews of them, leaving the department without a basis to assess the institutions’ stated mission and actual performance against planned or expected results. This is inconsistent with the Standards for Internal Control in the Federal Government, which state that agencies should conduct reviews by management at the functional or activity level, which in this case would be the JPME research institutions, and compare actual performance to planned or expected results. Clearly establishing linkages between significant activities, their intended role in meeting agency-wide goals and objectives, and assigning oversight responsibilities underpins an agency’s ability to conduct such reviews.

DOD Does Not Have a Framework to Assess the Performance of JPME Research Institutions

According to officials representing the Joint Staff and JPME colleges and universities, DOD has provided JPME colleges and universities with broad latitude in overseeing their associated research institutions. In doing so, the Joint Staff and the military services have not provided guidance to assist the JPME colleges and universities in developing a comprehensive oversight framework for assessing the performance of JPME research institutions. As a result, we found that JPME colleges and universities have not consistently established measurable goals and objectives linked with performance metrics to assess the performance of their associated research institutions and therefore are unable to comprehensively assess their performance to determine whether they are furthering JPME or other departmental goals. According to best practices, achieving results in government requires a comprehensive oversight framework that includes measurable goals and objectives, and metrics for assessing progress, consistent with the framework identified in the Government Performance and Results Act.

In April 2012, the Joint Staff conducted a management control review of NDU, the purpose of which was to assess the administrative and fiscal control processes that were in place to ensure proper stewardship of NDU’s resources. As part of that review, the Joint Staff noted that throughout NDU there appeared to be a fundamental disagreement regarding how its research supported the JPME mission and courses. Accordingly, the Chairman of the Joint Chiefs of Staff provided NDU with a new mission statement emphasizing that its research should support its academic mission. teaching faculty to bring together expertise in national security studies. The NDU President approved the strategic plan for research in January 2014.

We identified other examples where JPME colleges and universities identified broad goals and objectives for research. However, the linkage between these goals and objectives and JPME research institutions was unclear as the goals and objectives are not specifically assigned to associated JPME research institutions. For example, the Naval War College’s strategic plan contains a guiding principle to keep the college’s research and scholarly activities relevant to the needs of the Navy and the nation. Similarly, Marine Corps University’s strategic plan contains a goal related to strengthening professional scholarship and outreach. However, neither of these goals makes reference to the college’s or university’s research institutions. Additionally, JPME colleges and universities, such as Air University and Army War College, have developed lists of research priorities on an annual basis. According to officials, these lists are developed to reflect the priorities of senior leadership within their service and have been used to guide the research activities of JPME students. In 2013, the Army War College completed a strategic review of its academic programs and, as a result of this review, has aligned the development of its Key Strategic Issues List with a specific strategic goal of influencing national security decision-making. However, the Army War College has not clearly linked its strategic issues list with the education goals of the college.

Furthermore, we found that JPME colleges and universities have not consistently established metrics to assess the performance of JPME research institutions in meeting PME or other departmental needs. Based on our review, we identified some examples where JPME colleges and universities had established performance metrics for their associated research institutions. For example, Air University established a performance measure for the Air Force Research Institute that includes a count of the requested versus the delivered research studies for senior Air Force staff. Similarly, Marine Corps University established several measures to assess the progress its research institutions have made in achieving desired outcomes. For example, the university established a measure for the History Division intended to assess its responsiveness to research inquiries.

Officials from JPME colleges and universities, including JPME research institutions, told us that they recognize the need to establish measures for assessing the research institutions’ performance. They explained, however, that they have faced difficulties in developing them for research institutions. For example, officials representing JPME colleges and universities stated that it is challenging to compile quantitative data that represent the value or the usefulness of research. Although we recognize that it is difficult to establish performance measures for outcomes that are not readily observable or in some cases systematic, the department does use metrics to assess the performance of other DOD-funded organizations that conduct studies and analysis research. For example, DOD guidance directs organizations that sponsor a Federally Funded Research and Development Center to assess their performance.

According to the guidance, sponsoring organizations must develop procedures to annually monitor the value, quality and responsiveness of their work. For instance, officials we spoke with within the Office of the Under Secretary of Defense for Acquisition, Technology and Logistics stated that their office compiles data on metrics for the RAND National Defense Research Institute—a Federally Funded Research and Development Center—that are primarily based upon quantitative ratings and comments gathered from surveying organizations that contracted for research projects with it. For those research projects that received low ratings or negative comments, the RAND National Defense Research Institute is required to follow up with sponsors to understand what had happened and provide a plan for corrective actions.

Without a framework that includes measurable goals and objectives linked with metrics, DOD, including its JPME colleges and universities, does not have a systematic basis to comprehensively assess the performance of JPME research institutions. Best practices state that a framework that consists of measurable goals and objectives linked with metrics for assessing progress would better enable DOD to determine whether JPME research institutions are achieving results. Moreover, it would provide the DOD with a sounder basis for making resource determinations to ensure that these research institutions are furthering JPME and other departmental goals and that JPME continues to provide servicemembers with the expertise necessary for their careers.

Established Oversight Processes Do Not Assess the Performance of JPME Research Institutions

Standards for Internal Control in the Federal Government state that agencies should conduct reviews by management at the functional or activity level and compare actual performance to planned or expected results. Clearly establishing linkages between significant activities, their intended role in helping meet agency-wide goals and objectives, and assigning oversight responsibilities underpins an agency’s ability to conduct such reviews. According to these standards, such controls are an integral part of an agency’s planning, implementing, reviewing, and accountability for stewardship of government resources and achieving effective results. However, the oversight conducted by the Joint Staff and by external accrediting bodies reviews the quality of JPME academic programs and not the JPME research institutions’ performance. For example, the Joint Staff’s Process for Accreditation of Joint Education process is DOD’s primary mechanism of oversight, assessment, and improvement of JPME academic programs. The instruction governing this process lays out seven educational standards common to all PME colleges and universities, including JPME colleges and universities, which the Chairman of the Joint Chiefs of Staff considers essential. Officials from the Joint Staff JPME Division stated that this office conducts reviews of the JPME colleges and universities every 6 years to determine how well their academic programs are meeting these education standards.

Although the Chairman of the Joint Chiefs of Staff is statutorily responsible for overseeing the officer joint education system, the Joint Staff instruction that serves as the primary guidance for JPME-related policy does not address what the intended role of research institutions should be at JPME colleges and universities and it does not assign responsibilities for conducting oversight of their activities. Without clarity as to the intended role of research institutions in support of JPME academic programs or for another purpose, there is no basis by which to compare the research institutions’ respective stated missions as well as actual performance to planned or expected results, as required by Standards for Internal Control. Specifically, the instruction contains standards to evaluate the quality of JPME academic programs, but it does not define the role of JPME research institutions and it contains no educational standards, learning areas, or specific objectives for research that would enable DOD to assess the performance of research institutions at JPME colleges and universities. Further, no organization is assigned specific responsibility for overseeing the performance of the research institutions.

Joint Staff officials agreed that their accreditation process is focused specifically on JPME academic programs and pertains only to academic curricula development and quality and the assurance of uniformity in course content across the different JPME colleges and universities. As a result, JPME research institutions are not reviewed as part of the Joint Staff’s accreditation process. Moreover, according to Joint Staff officials, while they are statutorily responsible for overseeing the quality of JPME academic programs, there is no statutory responsibility for the Joint Staff to oversee the performance of JPME research institutions. One JPME official with whom we met noted that if the department had goals, objectives, and performance measures for JPME research institutions as part of the instruction governing the Joint Staff accreditation process, it would strengthen the department’s oversight process for JPME research institutions. In the absence of DOD-wide guidance that defines the role of research institutions as part of the JPME system and establishes roles and responsibilities for conducting oversight of JPME research institutions, the department and JPME colleges and universities cannot systematically assess the performance of JPME research institutions and whether they are furthering JPME.

In addition to the Joint Staff’s accreditation process, oversight of JPME academic quality is performed by external accrediting bodies. Accreditation is a means of self-regulation and peer review to ensure agreed upon standards are met. The regional accreditation process is intended to examine academic institutions as a whole. While the accreditation process may review the extent to which research is conducted at a JPME college or university among a number of other activities, these evaluations are not intended to assess the performance of JPME research institutions in meeting JPME and other departmental goals and objectives.

JPME colleges and universities, as Master’s Degree–granting institutions, are accredited by the following four regional accreditation bodies:

Middle States Commission on Higher Education accredits NDU and the Army War College;

Higher Learning Commission accredits the Army Command and

New England Association of Schools and Colleges accredits the Naval War College; and

Southern Association of Colleges and Schools Commission on Colleges accredits Air University, including its Staff and War Colleges, and Marine Corps University, including its Staff and War Colleges.

JPME colleges and universities are subject to the regional accreditation processes every 10 years and these processes are intended to strengthen and sustain the quality and integrity of higher education. For example, according to the Middle States Commission on Higher Education, accreditation by the commission is based on the results of institutional reviews by peers and colleagues and attests to the judgment that the institution has met certain criteria, such as that it has a mission appropriate to higher education; it is guided by well-defined and appropriate goals, including goals for student learning; and it has established conditions and procedures under which its mission and goals can be realized.

While the accreditation process reviews the quality at JPME colleges and universities, it does not specifically assess the performance of JPME research institutions. Our review of reports prepared through the regional accreditation process as well as interviews with JPME officials knowledgeable about the accreditation processes confirmed that their reviews are generally focused on the curriculum of JPME academic programs and not the performance of JPME research institutions. Therefore, the accrediting processes also do not provide DOD or JPME colleges and universities with a means for evaluating the performance of JPME research institutions and whether they are furthering JPME and other departmental goals.

DOD Does Not Formally Coordinate Studies and Analysis Research Requests Performed by JPME and Other DOD- Funded Research Institutions, Although Some Conduct Work in Similar Topic Areas

DOD does not formally coordinate requests for studies and analysis research conducted by JPME research institutions and other DOD-funded research organizations, even though many of these organizations have missions to conduct work in similar topic areas. Our analysis found that multiple organizations, including JPME research institutions and other DOD-funded research organizations, such as Federally Funded Research and Development Centers, have missions to conduct work in similar topic areas. However, DOD relies on a variety of separate processes to manage research requests that can be conducted at either JPME research institutions or other DOD-funded research organizations. Specifically, offices within the Office of the Secretary of Defense and the military departments have their own separate internal processes to request such research. Because there is no requirement for them to do so, these offices do not have mechanisms in place to participate in one another’s processes, thereby limiting opportunities to share information on DOD-wide priorities and collective research efforts, and to identify any areas of potentially similar research. Although there are notable differences even among the JPME research institutions and other DOD- funded organizations that have missions to conduct work in similar topic areas, we note that, as we concluded in September 2009, organizations involved in similar missions should coordinate to avoid unnecessary duplication of work. Furthermore, results-oriented management practices call for establishing a means to operate across organizational boundaries to enhance and sustain coordination.

Some DOD Research Organizations Have Missions to Conduct Work in Similar Topic Areas

Although we did not identify specific instances of duplication through our analyses of mission statements and the research project titles of 20 JPME and 14 other DOD-funded research institutions, we identified similarities in their research topic areas. Through our analysis of the mission statements, we identified multiple instances in which several DOD research organizations conduct work in similar topic areas. For example, we found that

11 JPME research institutions, 5 Federally Funded Research and Development Centers, 3 regional centers and 2 service-affiliated research organizations have missions to conduct research related to DOD strategy, policy, and doctrine;

5 JPME research institutions, 1 Federally Funded Research and Development Center, and 3 regional centers have missions that include researching issues related to civilian-military issues and irregular warfare; and

2 JPME research institutions, 5 Federally Funded Research and Development Centers, and 4 service-affiliated research organizations have missions that include researching technology, acquisition, and systems issues.

Our analysis of mission statements also identified instances in which more-limited numbers of DOD research organizations conduct work in similar topic areas. For example, we found that

2 JPME research institutions and 1 service-affiliated research organization have missions that include researching issues related to cyber issues; and

2 JPME research institutions and 2 regional centers have missions that include researching issues related to Africa.

Figure 6 summarizes the results of our analysis of similarities in research topic areas for the 20 JPME research institutions and 14 other DOD- funded research organizations, according to 23 areas of concentration.A checkmark indicates a research institution’s mission statement identified that category is a topic area in which it conducts research.

The similarities among the DOD research organizations are also illustrated in the titles of the research projects conducted by JPME research institutions and other DOD-funded research organizations. By categorizing 2,217 research project titles provided to us for 2012 and 2013 from both JPME research institutions and other DOD-funded research organizations, we found that multiple organizations’ project titles were grouped in related topic areas. For example, project titles from 13 JPME research institutions, 5 Federally Funded Research and Development Centers, and 2 regional centers were related to the Middle East; project titles from 11 JPME research institutions, 5 Federally Funded Research and Development Centers, 2 regional centers, and 1 service-affiliated research organization concerned Asia studies; and project titles from 10 JPME research institutions, 5 Federally Funded Research and Development Centers, 1 regional center, and 1 service- affiliated research organization concerned force structure and operational issues.

Our analysis also identified limited instances of similarities of specific research project titles within topic areas. For example, we identified four research project titles that focused on the “Arab Spring” and two research project titles specifically related to China’s development-assistance efforts. However, given our objective’s focus on research organizations as opposed to projects, we did not review the content of individual research projects and their respective methodologies. As a result, we did not assess the extent to which individual research projects and their findings overlapped or were duplicative with other research projects. Appendix III provides more-detailed results of our analysis of research project titles conducted by JPME research institutions and other DOD-funded research organizations in 2012 and 2013.

There Are Notable Differences among JPME and Other DOD-funded Research Institutions

While there are similarities in the research topic areas of JPME research institutions and other DOD-funded research organizations, DOD officials also identified notable differences among these organizations. One such difference is that some JPME research institutions are required to support the PME mission at their respective colleges and universities, whereas that is not part of the mission of other DOD-funded research organizations. For example, the mission statement of NDU’s Institute for National Strategic Studies, which comprises its research institutions, currently includes advancing the strategic thinking of NDU and the JPME community through research. Also along these lines, Air University’s JPME research institutions seek to use research publications to enhance strategic thought within the Air Force and in Air University academic curricula. To carry out their mission to support JPME academic programs, the research institutions engage in efforts not required of other-DOD funded research organizations. For example, officials at Air University noted that the research products developed by the Counterproliferation Center are used to update Air Force PME curriculum. As another example, faculty from Naval War College JPME research institutions teach elective courses in the JPME academic program. Conversely, the mission of the Center for Naval Analyses, a Federally Funded Research and Development Center, is to provide independent, authoritative research, analysis, and technical support to the Navy and other DOD organizations, and this research is not tied to JPME academic programs.

We identified two additional factors that differentiate JPME research institutions among themselves and other DOD-funded research organizations. The first pertains to differences among the JPME research institutions as to which office primarily sponsors the work of research institutions. For example, two JPME research institutions have missions to conduct research on China issues—the China Maritime Studies Institute at the Naval War College and the Center for the Study of Chinese Military Affairs at NDU. However, the two institutions conduct research on different aspects of China, reflecting the interests of their primary sponsors. Specifically, the China Maritime Studies Institute conducts research on Chinese maritime issues primarily for the Navy, while the Center for the Study of Chinese Military Affairs conducts broader research on Chinese strategic-level issues for the Joint Staff and Office of the Secretary of Defense.

According to officials from JPME colleges and universities and other DOD-funded research organizations, a second factor that differentiates JPME research institutions among themselves and with other DOD- funded research organizations is the level of technical expertise provided by some research organizations. Specifically, officials explained that Federally Funded Research and Development Centers can produce research with a more scientific and technical focus than that of JPME institutions. For example, while both the Institute for Defense Analyses, a Federally Funded Research and Development Center, and NDU’s Center for Technology and National Security Policy have missions related to researching technology, the Institute for Defense Analyses conducts tests and evaluations of technologies, requiring staff to have specialized scientific and technical skills, while the Center for Technology and National Security Policy’s research discusses the effect of technology on defense policy.

Multiple Offices Use Fragmented Processes to Request Studies and Analysis Research

Although multiple organizations, including JPME research institutions and other DOD-funded research organizations, have missions to conduct work in similar topic areas, offices throughout DOD use separate processes to request studies and analysis research. This fragmentation across DOD occurs in the absence both of a DOD requirement to coordinate studies and analysis research requirements among the military departments and of other DOD offices and mechanisms to facilitate such coordination. In September 2009, we concluded that offices involved in similar missions should coordinate and share relevant information to avoid unnecessary duplication of work. Furthermore, results-oriented management practices call for establishing a means to operate across organizational boundaries to enhance and sustain coordination.

We identified several separate processes used by JPME research institutions or DOD offices to manage requests for studies and analysis research, but DOD has not established formal mechanisms to coordinate requests. JPME research institutions, for example, individually manage their own research activities. According to Joint Staff officials, JPME research activities are not typically coordinated with other departmental offices that request studies and analysis research. At JPME research institutions, researchers have the discretion to determine whether research has been or is being conducted on a given topic. For example, JPME research institution officials told us that while it is not a requirement, they may contact other subject-matter experts to determine whether similar work is being conducted at another JPME research institution. Officials also said researchers may conduct a literature review to understand the existing research on a topic as part of the research process, or they may review completed research projects that are contained in the Defense Technical Information Center database to see whether DOD has funded past studies.not contain information on ongoing research efforts, and no other formal mechanism for sharing information on ongoing studies and analysis research activities within DOD was identified.

However, that database does Within the Office of the Secretary of Defense, multiple offices generate requests annually for studies and analysis research, but these research requests are determined based on individual offices’ research requirements and are not formally coordinated with other departmental offices. Office of the Secretary of Defense research requests may be fulfilled by contracting with other DOD-funded research organizations or JPME research institutions to conduct the research. For example, research requests for the Office of the Under Secretary of Defense for Acquisition, Technology and Logistics are managed at the Office of the Secretary of Defense, Studies and Federally Funded Research and Development Center Management office. In doing so, officials with this office explained that they do not formally coordinate with other DOD offices to determine whether similar research requests are being funded by other departmental offices. Separately, the Office of the Under Secretary of Defense for Policy uses a different process to manage requests for studies and analysis research. Specifically, the Office of the Deputy Under Secretary of Defense for Strategy, Plans, and Forces reviews requests from within the Office of the Under Secretary of Defense for Policy on an annual basis. These research requests are not formally coordinated with other Office of the Secretary of Defense offices or other departmental offices, such as service-level studies and analysis offices, to determine whether similar work is being conducted or funded elsewhere.

Similarly, the military departments have their own respective internal processes for requesting studies and analysis research, but absent a DOD requirement to do so, these processes are not used to formally coordinate research requests among the military departments or with other DOD offices. In general, these processes are used as a mechanism to coordinate requests for studies and analysis research within each of the military departments. For example:

According to a senior Air Force official, the Air Staff’s Analyses, Assessments and Lessons Learned directorate is responsible for collecting annual research requests from across the Air Force and for ensuring that the contracted studies are not duplicative. A senior official within this office told us that the Air Force has experienced challenges with regard to its oversight over the number of studies it has funded. In response, the Air Force has developed a policy to track all Air Force funded studies in an internal database. However, according to this official, the Analyses, Assessments and Lessons Learned directorate generally does not formally coordinate with offices outside the Air Force on annual research requests.

The Army Study Program Management Office within Headquarters, Department of the Army, issues an annual call for research requests from Army commands, and in turn funds the research requests according to Army priorities. A senior Army official in this office said that its process is focused on reviewing Army-specific research requests and does not include other DOD offices that request or conduct studies and analysis research.

The Navy’s annual research requests are administered through the Office of the Chief of Naval Operations, which compiles and prioritizes research needs identified from across the Navy. The Navy’s studies and analysis program guidance says that the Navy should coordinate analytic efforts with the Marine Corps, but according to an official in this office, the Navy generally does not coordinate with other DOD- funded research organizations or JPME research institutions with regard to these annual research requests.

DOD officials within the studies and analysis research community observed that there are both costs and benefits to the department’s decentralized approach to requesting studies and analysis research. One official told us that limited coordination among the multiple offices that request studies and analysis research may put DOD at risk for funding overlapping research activities. Furthermore, a senior Air Force official in the Air Staff’s Analyses, Assessments and Lessons Learned directorate stated that the current approach makes it difficult for DOD to have a complete picture of how much money is being spent on studies and analysis research. While DOD officials identified costs to the current approach for coordinating studies and analysis research requests, officials also acknowledged that DOD’s decentralized approach may result in several benefits. For example, a senior Air Force official in the Air Staff’s Analyses, Assessments and Lessons Learned directorate stated that decentralization generates creativity and diversity of thought in DOD’s studies and analysis community, which can prove useful in informing DOD decision makers. Furthermore, a senior Navy official in the Office of the Chief of Naval Operations stated that the current approach allows each office to be concerned with its own area of functional expertise, which varies widely across the services and DOD. For example, the Office of the Chief of Naval Operations is responsible for funding studies related to naval functional areas such as designing, building, and maintaining ships, which is unique when compared to other service-level studies and analysis offices such as the Air Force Analyses, Assessments and Lessons Learned office or the Army Study Program Management Office.

In contrast to how it manages requests for studies and analysis research, DOD has established mechanisms to coordinate science and technology– specific research efforts across multiple departmental offices engaged in similar missions. Specifically, the science and technology research community has governing bodies, such as executive committees to facilitate such coordination. These committees are intended to better manage DOD’s science and technology research by bringing together the multiple departmental offices that sponsor such research for the purpose of sharing information. A senior official responsible for coordinating science and technology research efforts explained that the executive committees do not require additional resources. Rather, they are intended to share offices’ existing annual research plans and provide opportunities to leverage resources in a fiscally constrained environment. Some DOD officials we spoke with who are responsible for managing studies and analysis research requests also said that a mechanism that provided greater information on what studies and analysis research other departmental offices were sponsoring would improve their ability to identify potential overlap in research requests. Without a mechanism for coordinating research requests and sharing information on studies and analysis research activities among multiple offices, DOD cannot ensure that it minimizes potentially unnecessary overlap in research activities. Furthermore, making information on department-wide annual research requests available to JPME research institutions would provide the institutions an opportunity to further understand research needs and align some of the institutions’ research with strategic priorities identified by DOD leadership.

Conclusions

Given the ongoing and unique role of JPME research institutions in the development of DOD’s future leaders, DOD’s oversight of these institutions is important for helping the department to make the best use of the resources it devotes to the colleges and universities that provide PME and JPME and for decreasing fragmentation of research requests and the risk of potential overlap in research activities. As fiscal pressures facing DOD continue to mount, so too does the need for the department to prioritize resources for JPME research institutions to most-effectively meet the JPME mission. Considering the overall growth of JPME research institutions that occurred between fiscal year 2007 and fiscal year 2011 in number, funding, and size, it is paramount that DOD maintains oversight of these institutions. Best practices state that achieving results in government requires a comprehensive oversight framework that includes measurable goals and objectives, and metrics for assessing progress. Yet, with limited exceptions, the research institutions lack such goals, objectives, and associated metrics. Additionally, while DOD has some oversight mechanisms in place for JPME, DOD does not have clear guidance establishing the role of JPME research institutions in furthering PME or other departmental goals that would provide a basis for evaluating their performance and helping ensure that intended results are achieved. Further, no entity within DOD is assigned responsibility for overseeing the performance of JPME research institutions. Consequently, DOD cannot ensure the effectiveness of JPME research institutions and lacks a sound basis for making resource determinations. Furthermore, some JPME research institutions and other DOD funded research organizations have missions to conduct research in similar topic areas, but DOD uses a variety of separate processes for requesting studies and analysis research. Results-oriented management practices call for establishing a means to operate across organizational boundaries to enhance and sustain coordination. DOD, however, does not have a mechanism in place to coordinate studies and analysis research requests and minimize fragmentation. While DOD officials believe that their current decentralized approach to requesting studies and analysis research has its benefits, they also recognize that it has its costs. It is not clear that the benefits of DOD’s current approach outweigh the risks of fragmentation and potential duplication, particularly in a budget-constrained environment. DOD’s science and technology research community provides one mechanism for a coordination mechanism, but by no means is it the only mechanism that could meet the needs of the studies and analysis research community as it seeks to support department-wide priorities. Without a mechanism to facilitate coordination and reduce fragmentation among offices requesting studies and analysis research, DOD cannot ensure that it minimizes potential overlap in research activities and that its resources are used efficiently in support of department-wide priorities at its JPME research institutions and other research organizations.

Recommendations for Executive Action

To enhance the performance of JPME research institutions, we are recommending that the Secretary of Defense direct the Chairman of the Joint Chiefs of Staff and the Secretaries of the military departments for their respective PME and JPME colleges and universities to take the following three actions: define the role of JPME research institutions to provide a basis for assign responsibilities for conducting performance reviews of JPME establish a framework that includes measurable goals and objectives linked with metrics to assess the performance of JPME research institutions.

To improve the coordination of requests for studies and analysis research within the department and to reduce the risk of potential overlap in research activities, we recommend that the Secretary of Defense establish and implement a departmental mechanism that requires leadership from the military services and departmental offices responsible for managing requests for studies and analysis research to coordinate their annual research requests and ongoing research efforts.

Agency Comments and Our Evaluation

In written comments on a draft of this report, DOD concurred with our recommendations. The full text of DOD’s written comments is reprinted in appendix IV.

In concurring with our first recommendation, DOD noted that some work is already in progress to clarify organizational goals and establish metrics of success at each of the research institutions. DOD stated that, for example, the Joint Staff has been collaborating with NDU to refine its research enterprise. DOD noted that our recommendation is reflected in NDU’s Strategic Plan for Research 2014-2019 and a revision of the Chairman’s policy document for the university. According to the department, our recommendation should be fully implemented when the next academic year begins in the fall of 2014. We agree that these are positive steps towards establishing a comprehensive framework at NDU to systematically assess the performance of its JPME research institutions in meeting PME and other departmental goals and objectives. Notwithstanding this effort, as noted in our report there remains no DOD- wide guidance that addresses the intended role of research institutions in supporting PME, including JPME, or other departmental goals or assigns responsibilities for conducting performance reviews of them. This leaves the department without a sound basis to assess NDU’s and the other research institutions’ stated missions and actual performance against planned or expected results. Clearly establishing linkages between significant activities, their intended role in meeting agency-wide goals and objectives, and assigning oversight responsibilities underpins DOD’s ability to conduct such reviews.

In its concurrence with our second recommendation, DOD stated that to improve coordination of research requests, it plans to establish a Studies and Analysis Executive Committee by the end of fiscal year 2014 with regional and topical “communities of interest.” DOD noted that the committee will be a combined effort organized through the Office of the Under Secretary of Defense for Policy and the Assistant Secretary of Defense for Research and Engineering, with other representation from the JPME and PME community, as appropriate.

DOD also provided technical comments on a draft of our report, which we have incorporated into the report, as appropriate.

We are sending copies of this report to the appropriate congressional committees. We are also sending copies to the Secretary of Defense, the Chairman of the Joint Chiefs of Staff, the Secretary of the Air Force, the Secretary of the Army, the Secretary of the Navy, and the Commandant of the Marine Corps. In addition, the report will also be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have questions about this report, please contact me at (202) 512-5741 or ayersj@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix V.

Appendix I: Objectives, Scope, and Methodology

The National Defense Authorization Act for Fiscal Year 2013 mandated that we review the work performed by joint professional military education (JPME) research institutions in support of professional military education In this report, and the Department of Defense’s (DOD) broader mission.we (1) describe how JPME research institutions have changed in number, funding, and size, and the factors that contributed to any changes; (2) evaluate the extent to which DOD is assessing the performance of JPME research institutions in meeting professional military education and other departmental goals and objectives; and (3) evaluate the extent to which DOD coordinates research requests for JPME research institutions and other DOD-funded research organizations.

For the purposes of this report, we refer to service and joint colleges and universities that are accredited by the Joint Staff to provide JPME certification as JPME colleges and universities.

JPME colleges and universities. Based on this evidence, we determined that 20 institutions were conducting research as their primary mission and had dedicated personnel assigned to them and therefore were included in the scope of our review.

For our first objective of determining the extent to which JPME research institutions have changed in number, funding, and size, and the factors contributing to any changes, we obtained questionnaire responses and other documentation on the number of research institutions that existed at JPME colleges and universities from fiscal years 2000 through 2013 and collected and analyzed available funding and staffing data for these years for the JPME research institutions. We assessed the reliability of the funding and staffing data collected by analyzing questionnaire responses from JPME colleges and universities, which included information on their data-system management, data quality-assurance processes, and potential sources of errors and mitigations of those errors. Based on our review of the data provided and our review of the questionnaire responses, we concluded that the systems used to provide the data, and thus the data they provide, are sufficiently reliable for our audit purposes. However, based on this evidence, we determined that we were unable to report consistent data on JPME research institution funding and staffing levels prior to fiscal year 2007, and therefore we are providing trend data on JPME research institutions from fiscal years 2007 through 2013. Furthermore, although we have identified a number of factors that could affect data quality, we concluded that these were the best available data on JPME research institutions. We also concluded that the data would not lead to an incorrect or unintentional message since it is corroborated through interviews with cognizant officials at the National Defense University, Army War College, Army Command and General Staff College, Air University, Naval War College, and Marine Corps University. We also discussed the reasons for any trends in these budget and staffing data with knowledgeable officials in DOD and at the JPME colleges and universities. Specifically, we conducted interviews with officials from the Joint Staff; Office of the Under Secretary of Defense (Comptroller); National Defense University; Army War College; Army Command and General Staff College; Air University; Naval War College; and Marine Corps University.

For our second objective of determining the extent to which DOD is assessing the performance of JPME research institutions in meeting professional military education and other departmental goals and objectives, we obtained and reviewed documentation from the Joint Staff and the JPME colleges and universities that identify any goals, objectives, or performance measures for JPME research institutions. Specifically, we reviewed current strategic plans, mission statements, and other documentation describing activities of the JPME research institutions. We also reviewed documentation describing oversight mechanisms that monitor the academic quality of JPME colleges and universities. Specifically, we reviewed Chairman of the Joint Chiefs of Staff Instruction (CJCSI) 1800.01D, Officer Professional Military Education Policy, which is the DOD policy that governs the Joint Staff’s Accreditation of Joint Education process. We also reviewed reports prepared by external regional accrediting bodies.processes used to assess the performance of JPME research institutions, we used a standard set of questions to interview officials with the Joint Staff; Army, Navy, Marine Corps, and Air Force; National Defense University; Army War College; Army Command and General Staff College; Air University; Naval War College; and Marine Corps University. We then reviewed the results of the interviews and related documents to develop summary findings. In reviewing this documentation and testimonial evidence, we referred to our prior work on best practices that identifies elements that constitute a comprehensive oversight framework.

To further our understanding of any Specifically, best practices state that such a framework should include measurable goals and objectives linked with metrics for assessing progress, which is consistent with the framework identified in the Government Performance and Results Act (GPRA), as amended by the GPRA Modernization Act of 2010. We also reviewed this evidence in light of key internal control standards, which state that federal agencies should conduct reviews by management at the functional or activity level and compare actual performance to planned or expected results.

For our third objective of determining the extent to which DOD coordinates research requests for JPME research institutions and other DOD-funded research organizations, we included the 20 JPME research institutions discussed above and 14 other DOD-funded research organizations. To identify the other DOD-funded research institutions to include in the scope of our review, we gathered documentation from the Assistant Secretary of Defense for Research and Engineering and canvassed knowledgeable DOD officials in offices responsible for requesting research, such as the military departments, science and technology executive agents, studies and analysis research program managers, and the Office of the Secretary of Defense, including the offices of the Under Secretary of Defense for Acquisition, Technology and Logistics and the Under Secretary of Defense for Policy. Based on this work, we determined that we would include the following 14 other DOD- funded research organizations for the purposes of our review:

DOD’s Regional Centers for Security Studies:

George C. Marshall European Center for Security Studies, the Near East South Asia Center for Strategic Studies, the William J. Perry Center for Hemispheric Defense Studies, and the Africa Center for Strategic Studies; the Naval Postgraduate School Modeling, Virtual Environments and Simulation Institute, the Naval Postgraduate School’s Center for Interdisciplinary Remotely Piloted Aircraft Studies, the Naval Postgraduate School’s Cebrowski Institute for Information and Innovation, the Army’s Center for Army Analysis, and

U.S. Army Training and Doctrine Command Analysis Center;

Federally Funded Research and Development Centers that were identified as studies and analysis research centers:

Center for Naval Analyses,

RAND Project Air Force,

RAND National Defense Research Institute, and the Institute for Defense Analyses.

To further our understanding on the processes used to request studies and analysis research from JPME institutions and other DOD-funded research organizations, we reviewed documentation from and interviewed officials at the military services, Office of the Secretary of Defense, JPME research institutions and other DOD-funded research organizations. We also reviewed documentation and interviewed knowledgeable agency officials about DOD’s approach to coordinate research requests among DOD organizations. Specifically, we used a standard set of questions to interview officials with the Office of the Secretary of Defense and the military services about the processes used to share information with other offices that also request studies and analysis research. We then reviewed the results of the interviews and related documents to develop summary findings. We reviewed the documentary and testimonial evidence in light of key practices for enhancing and sustaining coordination as described in best practices. Specifically, best practices state that organizations involved in similar missions should coordinate and share information to avoid unnecessary duplication of work. Further, we assessed whether there were any similarities or dissimilarities among the missions of the JPME research institutions and other DOD-funded research organizations. We did this in four ways:

First, we assessed whether the JPME and DOD-funded research institutions were conducting science and technology–related research or studies and analysis–related research, using missions statements and other documentation provided by DOD to make this determination. We did not conduct further analysis on other DOD- funded organizations that conduct science and technology research as that type of research represented a notable difference from the JPME research institutions that primarily conduct studies and analysis research.

Second, for all the JPME and DOD-funded research institutions conducting studies and analysis research, we reviewed mission statements and other mission-related information provided by DOD, and categorized the organization’s mission as falling primarily into 1 or more of 23 areas of concentration—for example, Asia studies or leadership and ethics studies. To create the 23 areas of concentration that were used to categorize mission statements and research project titles, we reviewed documentation from the JPME research institutions that identified general topic areas in which the institutions conducted research. We also reviewed documentation from Federally Funded Research and Development Centers that identified the core topic areas within which the Federally Funded Research and Development Centers were authorized to conduct work. The areas of concentration are identified as follows: Africa; Asia; Europe; Middle East; Western Hemisphere; civilian-military issues and irregular warfare; cyber; energy and environment; force structure and operational issues; historical; intelligence; leadership and ethics; legal; logistics; nuclear and weapons of mass destruction; other; personnel and training; public affairs and communication; resource management; strategy, policy, and doctrine; technology, acquisition, and systems; war gaming; and unable to code. We determined that these areas of concentration we selected were appropriate for comparing JPME research institutions and other DOD-funded research institutions because they explain the focus of each organization’s primary studies and analysis efforts. To complete the content analysis, two GAO analysts independently reviewed the mission statements and other mission-related information provided by DOD and coded them into one or more of the 23 areas of concentration. When the coding was completed, both analysts reviewed every code made by the other analyst and indicated whether they agreed or disagreed with the code. The analysts then met to discuss their coding determinations and to reach agreement where there were any discrepancies. The results of our analysis are not generalizable beyond the 20 JPME research institutions and 14 other DOD-funded research institutions included in the scope of our review.

Third, for all the JPME and DOD-funded research institutions conducting studies and analysis research, we collected a list of research projects they conducted for fiscal years 2012 and 2013. To complete the content analysis, one GAO analyst independently reviewed each of the 2,217 research project titles and coded them into one or more of the 23 areas of concentration. When the coding was completed, two GAO analysts shared responsibility to review the coding made by the first analyst and indicated whether they agreed or disagreed with the code. The analysts then met to discuss their coding determinations and to reach agreement where there were any discrepancies. The results of our analysis are not generalizable beyond the 20 JPME research institutions and 14 other DOD-funded research institutions included in the scope of our review.

Fourth, we reviewed documentation about the offices that request research from the JPME and other DOD-funded research institutions, along with testimonial evidence gathered during our interviews with DOD officials to provide context for any similarities or dissimilarities we identified through the mission statement and project title analysis.

To further our understanding of DOD’s processes for requesting research and of the similarities and differences among research organizations, we conducted interviews with officials from the Joint Staff; Office of the Under Secretary of Defense (Comptroller); Office of the Under Secretary of Defense for Policy; Office of the Secretary of Defense, Studies and Federally Funded Research and Development Center Management Office; Office of the Assistant Secretary of Defense for Research and Engineering; Army Study Program Management Office; Air Force Analyses, Assessments and Lessons Learned office; Office of the Chief of Naval Operations; Marine Corps Analysis Directorate; National Defense University; Army War College; Army Command and General Staff College; Air University; Naval War College; and Marine Corps University.

We conducted this performance audit from February 2013 through March 2014 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Joint Professional Military Education Research Institutions

This appendix contains more-detailed information for each of the 20 joint professional military education (JPME) research institutions included in the scope of our review. For each research institution, we provide a one- page summary that includes information on the following elements:

Location: the institution’s associated JPME college or university and geographical location.

Background: the date that the institution was established and information on its establishment, as well as any relevant information such as changes in the research institution’s name.

Mission: the institution’s mission, either self-reported or as derived from relevant documentation.

Customers: a list of entities, as reported by the research institutions and other documents that represent the principal requesters and users of the research institutions’ research products.

Nature of research and publications: a summary description of the types of research and studies conducted by the research institution, as well as the names of publications produced by the institution, if any. the institution’s total funding, depicted in thousands of dollars, for fiscal years 2004 to 2013, as available. Total funding is subdivided into two categories—direct funding and reimbursable funding. Direct funding includes federal appropriations made available for JPME colleges or universities. Reimbursable funding refers to amounts earned or collected from outside offices for research services furnished by the institution.

Total staffing: the institution’s total number of personnel, expressed as full-time equivalents for fiscal years 2004 through 2013, as available. Full-time equivalents are calculated as the total hours worked divided by the number of compensable hours in a full-time schedule.

The Air Force Research Institute augments Air University’s and the Air Force’s research capacity and supports airpower research inquiries from the Chief of Staff of the Air Force, as well as other decision makers throughout DOD. The institute conducts research on topics related to air, space, and cyberspace opportunities, threats, and capabilities; evaluation of operational and strategic issues; conducts regional strategic assessments; estimates long-term strategic and technical capabilities; analyses of logistical constraints and basing issues, among other issues. The institute also serves as the focal point and provides support for Air University’s “Call for Topics,” which makes potential research topics of interest to Air Force leaders available to student researchers. The institute also operates the Air University Press, and publishes the Department of the Air Force’s Air and Space Power Journal, and the Strategic Studies Quarterly.

Background: Established in 2008 by a Special Order from Headquarters Air Force, the Air Force Research Institute’s research roots extend back to the Airpower Research Institute of the late 1970s and even the 1930s in the Air Corps Tactical School. The organizational functions were previously embedded in the Air War College and later the College of Aerospace Doctrine, Research and Education.

Location: Air University, Montgomery, AL Background: The Center for Strategy and Technology was established at the Air University in 1996. looking 20 to 30 years into the future to provide a vision to prepare the Air Force for future challenges. Research conducted under the auspices of the center is briefed to the Air Staff, published as occasional papers and disseminated to senior military and political officials, think tanks, educational institutions, and other interested parties.

Mission: To engage in long-term strategic thinking about the implications of technological change and its implications for U.S. national security. The Center for Strategy and Technology focuses on education, research, and publications that support the integration of technology into national strategy and policy.

- Appendix II: Joint Professional Military Education Research Institutions -

Nature of Research and Publications The center provides research articles, papers, and monographs addressing issues pertinent to U.S. military-response options for dealing with nuclear, biological, and chemical threats and attacks. Research topics include military and diplomatic policy and concepts related to weapons of mass destruction; international nonproliferation diplomacy; nonproliferation and arms control treaty regimes; counterterrorist activities; and nuclear deterrence of conflicts. The center develops a strategic-issues list that provides potential research topics to student and faculty researchers. Additionally, it conducts outreach on issues related to counterproliferation and nuclear operations through its publication of Counterproliferation Center Outreach Journal, and the Trinity Site Papers series, and through an annual conference on countering weapons of mass destruction.

- Appendix II: Joint Professional Military Education Research Institutions -

The Center for Army Leadership conducts research and studies to identify leader development trends and requirements, and to develop and promote leadership and leader development practices and techniques for the Army. Further, the Center accomplishes these outcomes by contributing to Army doctrine and policy by informing leadership on best practices for developing leader competencies, and by producing the Annual Survey of Army Leadership, which is a survey- based study that assesses Army-leader attitudes regarding leader education, including the quality of leadership, and the contribution of leadership to accomplishing the Army’s overall mission.

Background: The Center for Army Leadership was established in 2001 in response to a study chartered by the Chief of Staff of the Army to identify the characteristics and skills required for Army leaders in light of changes to the operational environment.

The Combat Studies Institute researches, writes, and publishes, through the Combat Studies Institute Press, original interpretive works on doctrinal and operational issues of relevance to the U.S. Army and policymakers. The center also implements U.S. Army Training and Doctrine Command’s program of military history instruction throughout the Army, conducts an oral history research program that targets Command and General Staff College students and faculty, as well as visitors of the Combined Arms Center, focusing on compiling their past operational experiences, and provides oversight responsibilities for the Combined Arms Center Command History program, the Staff Ride team—which offers live and virtual battlefield tours—and the Frontier Army Museum.

Background: The Combat Studies Institute was established in 1979 to provide a range of military historical and educational support to the Combined Arms Center, Training and Doctrine Command, and the United States Army.

Mission: To provide military historical and educational support to the Combined Arms Center, Training and Doctrine Command, and the United States Army.

Location: Army War College, Carlisle Barracks, PA Background: Established in 1954 by the Commandant of the Army War College to create an advanced study group to undertake a program of long-range thinking on strategy and land power.

The Strategic Studies Institute conducts research on topics such as the future of American strategy; geostrategic analyses; strategic landpower; Army forward presence in the Pacific; cyber security; energy security; Army’s role in missile defense; effects of war on leadership; and Army profession and public trust. The institute also compiles a Key Strategic Issues List based on input from the U.S. Army War College faculty, the Army Staff, the Joint Staff, the unified and specified commands, and other Army organizations. This is designed to guide the research of the Strategic Studies Institute, the U.S. Army War College, and other Army- related strategic analysts.

Mission: To conduct and disseminate independent strategic analysis that develops recommendations for addressing key national security issues.

Examples of Customers - Appendix II: Joint Professional Military Education Research Institutions - Nature of Research and Publications The Translational Research Group at the Center for Advanced Operational Culture Learning aims to link the findings of scientists with the needs of Marine Corps soldiers and leadership by helping the two sides understand each other’s needs and capabilities.

Location: Marine Corps University, Quantico, VA Background: The Translational Research Group was established in 2010 by the Director of the Center for Advanced Operational Culture and Learning at the Marine Corps University and by the Executive Deputy of the Marine Corps Training and Education Command.

Mission: To identify practical applications for social and behavioral scientific research that will help address pressing challenges facing the Marine Corps.

The History Division’s primary task is to research and write the Marine Corps’ official history. The division provides assistance through their reference branch and deploying field historians to record history in the making during operations. The History Division and also conducts research through an oral history program, in which it obtains, catalogs, transcribes, and preserves personal narrative experiences and observations of historic value from active-duty and retired Marines for use as reference source material. The division prepares a wide variety of official publications that tell the Marine Corps story. Publications include articles, monographs, occasional papers, and definitive histories. It also creates material for and publishes Fortitudine, an online bulletin of the Marine Corps history program.

Background: Established in 1919 by Marine Corps Headquarters to record, preserve, and distribute the Corps’ history, the History Division was transferred in 2005 to Marine Corps University.

Mission: To write, document and track the history of the Marine Corps across the entire spectrum of time; to collect documents and accounts of permanent value to the history of the Marine Corps and preserve them for future use; and distribute the history of the Corps through publications, papers and other programs, in order to preserve history, aid combat and noncombat decision making, support Professional Military Education, motivate Marines, and inform the American public.

Examples of Customers - Appendix II: Joint Professional Military Education Research Institutions - Nature of Research and Publications In an effort to improve the Marine Corps’ understanding of the complex security environment of the Middle East, the center began three forms of publications. The Middle East Studies Occasional Paper Series, with the first issue published in June 2011, aims to disseminate original, peer-reviewed research papers on a wide variety of subjects pertaining to the Middle East, Afghanistan, and Pakistan. The MES Monograph Series, with the first issue published in August 2011, focuses on subjects of strategic relevance to the current and future U.S. professional military education community and is meant to be published quickly to address fast- developing situations. Finally, the Middle East Studies institute publishes Insights, which is produced bimonthly as the newsletter of the center. It features short analytical pieces as well as information on events organized by the center and provides a forum for debate with readers.

- Appendix II: Joint Professional Military Education Research Institutions -

Nature of Research and Publications The center collects and analyzes interagency lessons from the field on overseas contingency operations, including stabilization, irregular warfare, and security assistance, and integrates them into joint military doctrine on such topics as counterinsurgency, stability operations, security cooperation, and interagency coordination; as well as into education, policy, training, and joint military/interagency exercises. The center also analyzes interagency aspects of overseas operations on behalf of Department of Defense, the Intelligence Community, and several federal agencies. The center’s principal journal, PRISM, serves to inform members of U.S. federal agencies, allies, and other partners on complex and integrated national security operations; reconstruction and nation- building; relevant policy and strategy; lessons learned; and developments in training and education. The center also produces publications on issues of importance to interagency stakeholders and JPME, such as its recent book Convergence: Illicit Networks in the Age of Globalization.

Fiscal Year 2004-2013 Total Funding

Office of the Under Secretary

Combatant Commands

U.S. Department of State

U.S. Agency for International - Appendix II: Joint Professional Military Education Research Institutions - Nature of Research and Publications To fulfill its mission, the Conflict Records Research Center was charged with organizing, encouraging, and facilitating greater analytic and academic access to digitized copies of captured documents; coordinating the translation of captured documents of interest; conducting sponsored research and analysis on captured documents; developing and delivering training programs and providing research assistance; informing researchers of the collections in its custody; and publishing research in books, reports, journal articles, conference papers, newsletters, or other media.

Location: National Defense University, Washington, DC Background: Established in 2009 at the direction of the Office of the Under Secretary of Defense for Policy as a subelement of the Institute for National Strategic Studies. The center was defunded for fiscal year 2014, but in September 2013, the Office of the Under Secretary of Defense for Policy provided funding to keep the center open. Section 1071 of the National Defense Authorization Act for Fiscal Year 2014 provided statutory authorization for the center.

About the Center Location: National Defense University, Washington, DC Background: Established in 1984 by the Secretary of Defense as the Research Directorate of National Defense University’s Institute for National Security Studies, the center was originally charged to provide independent advice to Secretary of Defense, the Chairman of the Joint Chiefs of Staff, and the combatant commands for the formulation of national security policy and strategy. The center was renamed in 2010 during the university’s research reorganization.

The Center for Strategic Research performs research and educational activities in support of joint professional military education and explores strategic and regional topics to offer advice and strategic support to the Office of the Under Secretary of Defense for Policy, the Joint Staff, and other senior DOD officials. The center also conducts outreach to share its research with policymakers through studies, reports, briefings, and memorandums. Strategic studies encompass national security and military strategy, to include defense policy and organization, deterrence, arms control and counter proliferation, peace operations and small-scale contingencies, transnational security problems, command and control, and future warfare. Regional studies encompass national security strategy, defense policy, defense cooperation, and military strategy issues as they relate to significant countries or geographic areas of the world such as Asia and the Middle East. The center’s publication product line includes books, Occasional Papers, Strategic Perspectives, Strategic Forum (policy papers), conference papers, and journal articles.

Mission: To provide educational support to joint professional military education and advice to the Secretary of Defense, Chairman of the Joint Chiefs of Staff, and the combatant commands through studies, reports, briefings, and memorandums; conducts directed research and analysis in the areas of strategic studies and regional studies; and engages in independent and leading- edge research and analysis in related areas.

Location: National Defense University, Washington, DC Background: Established in 2000 as a part of National Defense University’s Institute for National Strategic Studies, pursuant to the National Defense Authorization Act for Fiscal Year 2000.

The center’s research focuses on documenting China’s expanding international interests, understanding China’s development and employment of new economic, military, and diplomatic capabilities, and analyzing Chinese debates about how these capabilities should be employed to advance national goals. It also explores the implications of these developments for U.S.-China relations and for the U.S. role in Asia. The center also conducts outreach to share its research with policymakers and informs the public debate through books, articles, memorandums, briefings and conferences. For example, the center cosponsors an annual conference on the People’s Liberation Army with the Council for Advanced Policy Studies (a Taiwanese think tank), and RAND, a nonprofit institution that conducts research and analysis. The center’s publication product line includes books, Occasional Papers, Strategic Perspectives, Strategic Forum (policy papers), conference papers, and journal articles.

Mission: To serve as a national focal point and resource center for multidisciplinary research and analytic exchanges on the national goals and strategic posture of the People’s Republic of China and the ability of that nation to develop, field, and deploy an effective military instrument in support of its national strategic objectives.

- Appendix II: Joint Professional Military Education Research Institutions -

University, Washington, DC Background: Established in 1994 as the Center for Counterproliferation Research, pursuant to memoranda of understanding among the Assistant Secretary of Defense for Nuclear Security and Counterproliferation, the Director of Strategic Plans and Policy, the Joint Staff, and the President, National Defense University. The center was renamed the Center for the Study of Weapons of Mass Destruction in 2004.

The Center for the Study of Weapons of Mass Destruction performs research on the full spectrum of issues related to weapons of mass destruction, engages in educational activities, and collaborates with partners across the government. The center conducts directed and self-initiated research of the following types: (1) operational and policy support; (2) traditional academic research; and (3) research undertaken to support the joint professional military education program at the National Defense University. Research topics related to studies in weapons of mass destruction include deterrence; counterproliferation operations; and policy and doctrinal development regarding weapons of mass destruction. The center also conducts outreach to share its research through papers, and planning and participating in various venues, including conferences and dialogues with participants from U.S. and foreign partner entities. For example, the center organizes the annual meeting of the Weapons of Mass Destruction Education Consortium.

Mission: To prepare the joint warfighter and select others to address the challenges posed by weapons of mass destruction through education and professional development, scholarship, and outreach and collaboration activities across the full spectrum of issues related to weapons of mass destruction and to become one of the preeminent institutions in the United States for weapons of mass destruction expertise.

Nature of Research and Publications The Center for Technology and National Security Policy conducts research on a reimbursable basis by means of memorandums of understanding with sponsoring organizations on science and technology (chemical/biological defense, human hardiness research, counter–improvised explosive devices, policing, and counterinsurgency); civilian-military integration (transformative innovation for development and emergency support, social media in strategic communication); emerging challenges (anticipatory governance concerning cyber security; climate change; vulnerability to severe space weather); and advanced education initiatives. The center’s publication product line includes books, Defense Technology Papers, Defense Horizons (policy papers), conference papers, and journal articles.

- Appendix II: Joint Professional Military Education Research Institutions -

The Center for Transatlantic Studies provided senior Department of Defense and other U.S. government leaders with the North Atlantic Treaty Organization (NATO) and Transatlantic policy advice, research, and outreach, notably in the run-up to the 2012 NATO Summit and beyond. Specifically, the center conducted research on capabilities studies, evaluating transatlantic bargain and dialogue, and, NATO–Russia relations, and NATO’s countering hybrid threats. It also published Transatlantic Currents, CTSS Flash notes, and Transatlantic Perspectives.

Background: The Center for Transatlantic Security Studies was formed in 2010 to support the Under Secretary of Defense for Policy in North Atlantic Treaty Organization/European–related policy development. The center incorporated the NATO Orientation Program that had provided training to NATO assigned officers since at least 1990 as was mandated by the Chairman of the Joint Chiefs of Staff. As of October 2012, both the center and the NATO Orientation Program were dissolved.

Mission: To be the focal point for national and international collaboration on issues related to transatlantic security, defense policy, and military strategy through research, education, and outreach. Develops and conducts education and orientation programs for U.S. and allied military officers, government civilians, and international partners on issues relating to NATO and transatlantic security community.

Examples of Customers

Under Secretary of Defense for - Appendix II: Joint Professional Military Education Research Institutions - Nature of Research and Publications The China Maritime Studies Institute has four primary areas of activity: (1) broad, multidisciplinary research on China’s maritime activity as it relates to its strategic orientation; (2) annual conferences and speaker series; (3) publications, ranging from short assessments and think pieces to monographs and books; and (4) support for U.S. Navy and joint commands. The center conducts research in areas related to China’s maritime development, including energy, global commerce, law of the sea, maritime technologies, merchant marine, naval development, naval diplomacy, and shipbuilding.

Background: The China Maritime Studies Institute was established as a subcomponent of the Strategic Research Department on October 1, 2006, in accordance with a Program Objective Memorandum by the Chief of Naval Operations.

Research Department, it receives dedicated funding on an annual basis for its research activities, including to fund its own researchers. As a result, we have categorized the China Maritime Studies Institute as a separate joint professional military education research institution for the purposes of this report. However, because the center is a subcomponent of the Strategic Research Department, Naval War College officials stated that it shares budgetary, personnel, and administrative functions with the Strategic Research Department.

- Appendix II: Joint Professional Military Education Research Institutions -

Location: Naval War College, Newport, RI Background: The International Law Department was founded in 1984 and opened its doors in 1986.

The International Law Department serves as the Naval War College’s focal point for the study of international and maritime law and oceans policy as they affect U.S. military policy, strategy and operations. As part of its research efforts, the department compiles, edits, and publishes the International Law Studies Series that provides a forum for prominent legal scholars to publish articles that contribute to the broader understanding of international law. Recently, in response to discussion with the Joint Staff, the department initiated an Information Paper Series. These are short papers that break down legal issues for further consideration by senior military leaders. Furthermore, in addition to the Information Paper Series and International Law Studies, the individual staffers of the department engage in independent research and writing. In addition to legal research and scholarly writing, Naval War College officials state that staff members actively support the Naval War College’s core intermediate and senior- level Navy PME courses as professors, lecturers, and moderators, while hosting several operational-law electives throughout the academic year.

Mission: To conduct and disseminate advanced international law research and analysis.

The Strategic Research Department’s research projects, including those of the China Maritime Studies Institutes, fall into three broad categories: (1) projects assigned by the Navy or another U.S. national security organization; (2) sustained projects that do not depend on year-to-year tasking but rather constitute multiyear, multideliverable, multiclient investments serving long-term U.S. national security and Navy interests (these projects focus on strategic regions such as Eurasia, Africa, the greater Middle East and the Asia-Pacific region, as well as on functional topics such as maritime strategy, cyber conflict, and sea- based ballistic missile defense); (3) self-sponsored projects conducted in consultation with the leadership of the Center for Naval Warfare Studies (these projects have no specific tasking or set of clients, rather, they address emerging issues that officials believe will garner substantial national or naval attention in the foreseeable future).

Background: Formally established in 1987 by the Dean of the Center for Naval Warfare Studies, and in consultation with the Chief of Naval Operations. However, the Strategic Research Department dates back to the origins of the Center for Naval Warfare Studies in 1981.

Mission: To produce innovative strategic research and analysis for the U.S. Navy, Department of Defense, and the broader national security community.

Secretary of the Navy

Chief of Naval Operations

U.S. Marine Corps leadership

Combatant commands

Federal agencies - Appendix II: Joint Professional Military Education Research Institutions - Nature of Research and Publications Each year, the Chief of Naval Operations selects a broad governing theme for the Strategic Studies Group’s research. The 2013 topic is Undersea Dominance out to 2030. The Strategic Studies Group is responsible for keeping the Chief of Naval Operations informed of progress throughout the year and produces a summary briefing and written report of actionable concepts with recommendations that can be executed by the Chief of Naval Operations in the near term. The products, while encompassing long-term views, are designed to help inform the Chief of Naval Operations on near- and mid-term program decisions.

Appendix III: Research Project Title Analysis

This appendix contains the results of our research project title analysis. We reviewed 2,217 research project titles from 20 joint professional military education research institutions and 14 other DOD-funded research organizations. These titles are associated with projects conducted in 2012 and 2013. Based on a project’s title, we coded it into one or more areas of concentration. The table below presents the number of research project titles coded into these 23 areas of concentration. The results do not add to 2,217 because some research project titles could be coded into more than one area of concentration and two areas of concentration are not included in the final results. For example, a research project title on North Korea’s nuclear future would be coded into both the “Asia” and “nuclear and weapons of mass destruction” topic areas.

Appendix IV: Comments from the Department of Defense

Appendix V: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Matthew Ullengren, Assistant Director; Erin Behrmann; Richard Burkard; Gabrielle A. Carrington; Alberto Leff; Marcus Lloyd Oliver; Michael Silver; and Cheryl Weissman made key contributions to this report.