The Director of National Intelligence serves as head of the IC and acts as the principal adviser to the President and National Security Council on intelligence matters related to national security. The IC is comprised of 17 executive branch agencies and organizations, generally referred to as IC elements. These IC elements include two independent agencies, eight elements within the Department of Defense, and seven elements across five other executive departments. Table 1 provides a list of the 17 IC elements. In its first National Intelligence Strategy, issued in 2005, ODNI highlighted the importance of a diverse talent pool to address the complex challenges the IC faced. In its most recent strategy released in 2019, ODNI reaffirmed and emphasized the IC’s commitment to developing and retaining a diverse workforce to address enduring and emerging mission requirements. The 2019 National Intelligence Strategy defines diversity as a collection of individual attributes that include, but are not limited to national origin, language, race, color, mental or physical disability, ethnicity, sex, age, religion, sexual orientation, gender identity or expression, socioeconomic status, veteran status, and family structure. The Intelligence Authorization Act for Fiscal Year 2004 directed the Director of Central Intelligence to develop a pilot project to test and evaluate alternative innovative methods to promote equality of employment opportunities in the IC for women, minorities, and individuals with diverse ethnic and cultural backgrounds, skills, language proficiency, and expertise. The first pilot was initiated at Trinity Washington University in Washington, D.C., with a 1 year contract totaling $250,000. The college developed and designed curricular components to align with IC mission skills sets and competencies and competitively selected students to participate in the college’s IC CAE Scholars Program. In the first year, the program sponsored nine students who were selected as IC CAE scholars. After the initial pilot year at Trinity Washington University, the pilot program was expanded to three additional colleges—Tennessee State University in Nashville, Tennessee; Florida International University in Miami, Florida; and Clark Atlanta University in Atlanta, Georgia. In 2005 ODNI, on behalf of the IC, established the IC CAE program. ODNI reported that by 2007, 65 scholars participated in the program from these four CAE colleges. By 2008, ODNI had expanded the pilot to six additional colleges. Overall, the 10 participating colleges increased the student population to 338 IC CAE scholars. During the 2008 to 2009 academic year, ODNI established a continuity strategy with the initial 10 IC CAE pilot colleges and the program continued to expand its academic outreach to additional colleges. In 2009, a total of 17 colleges were participating in the program and these colleges had arrangements with academic consortia that increased the total outreach to 31 colleges. During ODNI’s management of the program from 2005 through 2011, ODNI established general goals and oversaw the program’s implementation by defining and collecting performance measures on a range of IC CAE activities and working with a contractor to summarize this information in annual reports. We describe ODNI’s management of the program in more detail in appendix I. The Intelligence Authorization Act for Fiscal Year 2010 codified the IC CAE program to authorize the Director of National Intelligence to carry out grant programs to enhance the recruitment and retention of an ethnically and culturally diverse IC workforce with capabilities critical to the national security interests of the United States. In 2010, ODNI launched an Intelligence Community Efficiency Studies Initiative that included an examination of the size, structure, and functions of the ODNI. One recommendation was to consolidate and streamline the education and training programs in the IC by transferring the functions and responsibilities of the IC CAE program from ODNI to DIA. DIA began managing the program on October 1, 2011. The memorandum of understanding between ODNI and DIA implementing the decision of the transfer established that while DIA would manage the IC CAE program, ODNI would continue to provide periodic strategic guidance and regular budgetary oversight for program. Figure 1 shows various IC CAE program milestones, such as grant announcements and program transition dates, among other details. According to ODNI and DIA officials, program management and oversight of the IC CAE program is currently transitioning from DIA back to ODNI, following a DIA roles and mission review in 2018. According to ODNI and DIA officials, officials are working to complete the transition in fiscal year 2020 to enable ODNI to assume responsibility for the program. According to ODNI officials, as of April 2019, the transition plans were still in progress and ODNI was still in the planning stage of the transition. For example, officials noted they were drafting an implementation plan for the transition as well as a transfer memorandum to document the transfer. According to DIA officials, during this process, ODNI and DIA officials were also holding weekly coordination meetings and sharing program documents, such as college reports collected by DIA, and program guidance. ODNI officials also stated that in February 2019, they hired a contractor to conduct a study of the program prior to the final transition date. According to ODNI officials, the study, along with their interactions with DIA, will help ODNI determine how to manage the program, identify any challenges or successes of the program, and consolidate the data collected on the program to date. Officials expect the study to be completed by October 2019. The IC CAE Senior Advisory Board consists of representatives of the IC elements and key organizations that may include representatives from the National Intelligence University, a U.S. Combatant Command (rotating basis), and the Office of the Undersecretary of Defense for Intelligence. The board, which meets quarterly, was created to provide policy and guidance for the IC CAE program and ensure that participating IC elements are included in discussions of policy matters. As outlined in the board’s official charter and business rules, board members are responsible for attending board meetings, voting on issues before the board, evaluating colleges for grant funding, acting as points of contact for the program, and promoting the program as leverage to affect future IC missions. According to DIA officials, the board advises the IC CAE program manager on standards for the IC CAE program relating to college selection, strategies to foster collaboration, and other issues as needed. The IC CAE program awards grants to colleges on a competitive basis. IC CAE grants help colleges establish new intelligence-related programs and support existing programs at selected colleges. The grants can be issued for up to 5 years. From fiscal years 2004 through 2018, a total of 29 colleges have received 46 IC CAE grants. Of these 29 colleges, 13 have formed a consortium with one or more colleges to enhance collaboration with resources from other colleges in the same geographic area. The IC considers colleges with active grants as active IC CAE colleges, and those colleges that sustain the program after grant funding ends are called legacy colleges. Figure 2 shows the location of IC CAE colleges and which colleges led an academic consortium. See appendix II for additional details on the years that grants were awarded, grant funding amount, and a list of consortium colleges. Since 2011, DIA has issued grants for the IC CAE program through a process initiated by an announcement published online by the DIA grants officer. Grant announcements vary by year, but generally include guidelines for colleges to follow in completing their grant proposal. For example, the 2014 grant announcement listed eight program components a college’s proposal would be evaluated on, to include study abroad opportunities and annual colloquium or speaker series on intelligence and national security issues, along with other requirements such as cost program management and sustainment plans. Following submission, a grants officer reviews colleges’ grant proposals for technical and financial sufficiency. The IC CAE program office then reviews grant proposals for program sufficiency. From there, the IC CAE Senior Advisory Board’s Source Selection Board reviews applications deemed sufficient and makes a recommendation on which should be funded and at what funding level. The DIA CAE program office then forwards the selected proposals to a grant officer who notifies the college of the award. The grant announcements we reviewed may add specific program components as an area of focus for a specific year. For example, the 2019 grant announcement added a program component that required colleges submitting a proposal for a grant to offer courses or programs in three or more listed science, technology, engineering, and mathematics topics of interest to the IC. Examples of some other program components included in grant announcements since 2014 include the following: IC Curriculum. A key objective of the program is to strengthen academic programs in intelligence or national security in minority- serving, historically rural and under-resourced population colleges. Specifically, colleges shall explain how they plan to creatively expand, upgrade, enrich, or integrate undergraduate and graduate course offerings to better prepare students to perform work in intelligence or national security. Foreign Language. Colleges should demonstrate a capability to offer language study programs or courses in one or more specified languages of interest to the IC. Facilitate Student Participation in Academic Programs. IC CAE students shall be involved in the program and aware of the numerous benefits. Colleges are required to facilitate student participation in on- campus programs and activities such as workshops, seminars, and other off-campus activities such as national security or intelligence conferences, seminars, or workshops. Annual Colloquium. IC CAE colleges are required to hold annual colloquium or speaker series on intelligence or national security issues. These events should invite rural and under-resourced regional colleges and universities, government speakers, and industry partners with a primary goal of maximizing relationships and outreach. The colloquium should be at least 1 day in length, or a speaker series may include shorter presentations scheduled over weeks or months, which equates in the number of hours to a daylong colloquium. Program Management and Sustainment Plans. IC CAE colleges are required to have both program management and sustainment plans. The program management plan must detail the responsibilities of personnel to attain explicitly stated, measureable, and achievable program objectives. The sustainment plan must detail what the college will do during the grant period to build sustainability of the IC CAE program at that institution after the funding expires. The IC CAE program is especially interested in colleges with diverse populations of talent and in geographic diversity—specifically, Historically Black Colleges and Universities, Hispanic-Serving Institutions, Tribal Colleges and Universities, Asian American and Pacific Islander-Serving Institutions, and majority serving institutions with significant populations of minorities or women. The IC CAE program is also interested in majority serving institutions with significant populations of minorities and women that possess credentials in disciplines and specializations that meet IC core mission requirements. Figure 3 shows the minority designation of the 29 colleges receiving grants and figure 4 shows the minority designation of the 43 consortium colleges. See appendix II for a list of schools and their minority designations. As part of the IC CAE program, DIA also administers other programs that provide intelligence-related learning experiences to IC CAE students and to increase advanced capabilities in national defense. For example: IC CAE Professional Development Summit. These annual summits allow the IC to interact with the principal investigators—the individuals responsible for the IC CAE program at their respective colleges—to provide them with relevant and up-to-date information to support the creation and teaching of IC-centric curricula. According to DIA, the summit is intended to foster collaboration with the IC and college representatives by providing DIA with a platform to meet the needs of the IC. According to DIA, IC CAE Senior Advisory Board members are an integral part of the summit and provide context and perspective from the agencies they represent. National Security and Analysis Intelligence Summer Seminar. This 2-week seminar is designed to provide IC CAE students with knowledge about the intelligence career field in general, and analytic tradecraft in particular. The seminar is intended to provide students from across the IC CAE colleges an opportunity to engage directly with intelligence professionals in both seminar learning and scenario- based simulation training, focusing on threats to the U.S. homeland by extremist terrorists. According to DIA officials, the seminar is only open to a limited number of IC CAE students from active and legacy colleges. For example, two sessions were held during 2017 and a total of 80 students were competitively selected by their respective colleges to attend. According to ODNI officials, the summer seminar also holds a career fair and provides mentoring opportunities for the participating students so that those interested in an IC career have an opportunity to interact with recruiters. IC CAE Summer Internship. In the summer of 2017, the IC CAE program held its first IC CAE summer internship program. According to DIA officials, rather than establish a new IC CAE internship program, DIA leveraged the IC elements’ existing internship programs and tracked IC CAE student participation in these programs. The IC CAE internship offers IC CAE students additional opportunities, such as an opening and closing ceremony for the internship, an IC career fair at the National Security and Analysis Intelligence Summer Seminar event, and IC mentors upon request. DIA identified a total of 141 IC interns from colleges that had an IC CAE program in 2017 and 2018. However, according to ODNI officials, not all IC interns identified participated in their school’s IC CAE program. The internship opportunities among the IC elements vary. For example, according to FBI officials, their internship program is a primary pipeline for entry-level positions and, in 2017, they had 1,200 interns with 300 hired into entry-level positions. According to DIA data, the FBI identified 31 IC CAE scholars in its 2017 internship program and 21 scholars in 2018. According to Department of State’s Bureau of Intelligence and Research officials, their office has approximately 15 to 20 summer interns each year. According to Department of State officials, two of their interns were IC CAE scholars since the program began in 2017. While DIA has continued to implement the IC CAE program by issuing grants to colleges, DIA has not sufficiently planned or overseen the program since the transition from ODNI in 2011. Specifically, we found that DIA did not fully implement five of the six key practices of sound planning that we have identified in our prior work. While DIA continued the program’s mission to increase the pool of diverse applicants for the IC, it lacked results-oriented goals, an overall strategy for the program, an evaluation of external factors, performance measures, and a plan to assess the program’s performance in order to determine the appropriateness of the goals and effectiveness of implemented strategies. Our assessment of the extent to which DIA incorporated these key practices of sound strategic management planning into the IC CAE program is reflected in table 2. DIA annual reports for the IC CAE program and IC CAE grant announcements emphasize that the overall mission of the program is to increase the pool of diverse applicants for the IC. DIA’s annual reports describe the program’s mission as developing national security and intelligence education programs in order to increase the pool of culturally, geographically, and ethnically diverse, multidisciplinary job applicants who possess highly desired skills and competencies in areas of critical need to the IC. This mission statement is also contained in IC CAE grant funding opportunity announcements for 2014, 2017, 2018, and 2019, which also refer to broader IC human capital and diversity guidance. For example, one goal from the IC’s Equal Employment Opportunity and Diversity Enterprise Strategy (2015-2020) is to recruit from groups with lower than expected participation rates and diverse candidates who will meet the IC’s current and future mission requirements. Since 2011, DIA has not established results-oriented goals for the IC CAE program or an overall strategy that details the agency resources and processes that are required to achieve the program’s mission. First, DIA failed to document specific policy, programmatic, or management goals for the IC CAE program. DIA developed a business plan for the program in 2011; however, this plan describes short-term goals for program management, outreach, and education and most of these goals were intended to be complete by mid-2012. DIA’s documentation does not indicate whether these goals were achieved or whether DIA continued to use the goals to guide the program after 2012. Current DIA internal guidance states that the IC CAE program office carries out the program’s mission by providing grants to colleges to support the establishment of intelligence-centric curricula. However, this guidance fails to provide results-oriented goals that are defined in measurable terms to guide the program. For example: DIA has not described the number of potential IC employees it expects to be able to educate or make aware of IC careers by supporting intelligence programs at IC CAE colleges. This could include specific goals for targeting underrepresented populations within the IC, such as women and minorities. According to several IC element officials, IC elements use the percentage of women and minorities in the U.S. civilian labor force as a target for their own diversity recruitment efforts. However, DIA has not developed any results-oriented goals that include specific targets or milestones for recruiting potential IC employees who have participated in the IC CAE program. In addition, DIA has not developed specific goals for the program that identify how to prioritize among program requirements contained in IC CAE grant announcements . Specifically, it is not clear from IC CAE program documentation how gender and ethnic diversity is prioritized relative to other IC needs, such as the IC’s long-standing need for technical and language skills. For example, IC CAE grant announcements state a general goal of increasing the pool of qualified women and racial and ethnic minorities to the IC. At the same time, IC CAE grants have supported training in science, technology, engineering, and math, and critical languages, but DIA has not established specific targets or milestones that would allow it to track the program’s development of a diverse pool of applicants with the skills that the IC requires. Second, while DIA has developed some plans and continues to award grants for the IC CAE program, we found that DIA has not documented an overall strategy that details the agency resources and processes required to achieve the program’s mission. In 2016, DIA officials stated they began developing a document outlining the general structure of the IC CAE program, but as of May 2019, the document has not been issued. DIA has also documented its standard operating procedures for monitoring colleges’ implementation of grants in part to ensure that all programmatic goals are met, but it is not a strategic document that describes processes for achieving the program’s mission or goals. Further, DIA continues to award IC CAE grants to colleges based on program components or criteria that have changed over time, but these changes are not clearly linked to an overall program strategy. For example, in 2014, DIA added the diversity of a college’s student population as one of the criteria it used to select grant proposals. Colleges with a minority-serving designation or with a student population that is more than 75 percent ethnically and culturally diverse are given an excellent rating, while colleges with a student population that is less than 25 percent diverse are given a poor rating. In 2017, DIA then added criteria requiring colleges to be part of a consortium in a manner that promotes diversity. These two diversity criteria have been given more weight than all other criteria since 2017, while previous announcements gave greater weight to the development of national security curricula. This change in approach may align with the program’s overall mission to increase diversity in the IC, but DIA has not outlined an overall strategy that explains how such changes to the grant selection criteria would achieve a results-oriented goal like increasing the number of minority applicants to the IC. Two interconnected sound planning practices are to establish results- oriented goals and strategies to achieve those goals. These goals should be documented in measurable terms that are focused on results so that the agency can determine how it will achieve its mission. Once goals are established, strategies explain how these goals would be achieved. Since assuming responsibility for the program in 2011, DIA officials stated that their focus for managing the IC CAE program has been tactical, focusing on tasks like awarding, executing, and monitoring grants to IC CAE colleges, rather than strategic planning. In addition, DIA officials highlighted staff turnover as a challenge to managing the program and stated DIA has had five IC CAE program directors in its 8 years of program management. DIA officials stated that DIA has received little guidance about the goals of the IC CAE program from ODNI, and they instead rely on the IC CAE Senior Advisory Board to define goals and strategies that reflect the needs of IC elements. DIA officials stated that their only source of guidance from ODNI for the IC CAE program was the 2011 memorandum of understanding between DIA and ODNI, which DIA officials characterized as being high level and lacking specificity. DIA officials also said that they do not have the authority to create a strategic recruitment plan or set recruiting targets for the IC. The board only meets quarterly to advise the IC CAE program office on standards and strategies and board members occasionally review grant proposals. The IC CAE program managers are responsible for the program, and therefore, defining and documenting its goals and strategies. As the IC CAE program transitions back to ODNI, ODNI will not be able to determine whether the program is meeting the diversity goals of the 2019 National Intelligence Strategy without results-oriented goals for the program and a documented strategy showing how those goals are to be achieved. DIA has identified external factors that could affect the IC CAE program’s success, such as program branding and the ability of colleges to sustain the program after the grant period ends, but has not developed a process to fully evaluate them. One example of an external factor that could affect the IC CAE program’s success is the fact that not all students are aware of their participation in an IC CAE program. Colleges participating in the IC CAE program have not always featured participation in the program prominently, based on our analysis of selected websites, which are often managed by an academic department or institute. This can limit the visibility of the program and the IC’s support of it for both current and potential students. Since at least 2014, DIA has required colleges to demonstrate how they plan to promote their program as an IC CAE program to ensure that students, faculty, and administrators are aware of it. Colleges are also required to feature up-to-date program information on the college’s website. However, in November 2018, a DIA official noted that some colleges continue to use the IC CAE brand without oversight and accountability to provide intelligence-related courses. According to officials from selected IC CAE colleges and IC elements, students graduating from these programs are not always aware that they have participated in an IC CAE program. One college official stated that the certificates or degrees do not necessarily indicate that the student graduated from an IC CAE program. Another college official stated the college needs to directly inform students who apply to the program that they are participating in an IC CAE program. NGA and NSA officials stated that some employees at their agencies first became aware there was an IC CAE program at their college after being informed directly by their respective agency. While DIA requires that colleges develop marketing plans, it does not have a process to evaluate external factors such as the long-term effect of colleges’ efforts to advertise their programs’ connection with the IC CAE program. Without adequately advertising IC CAE programs, IC CAE colleges may not be able to recruit a strong pool of qualified students with the skills that the IC requires. Another example of external factors that could affect program success is the ability of colleges to sustain their IC CAE program. The intent of the IC CAE program has been to enable colleges to continue the program beyond the end of the grant period and maintain a continuous talent pool for the IC. However, DIA has not fully evaluated the challenges colleges may encounter if they are not able to secure continuous funding for the IC CAE program. When DIA awards grants, colleges are awarded a base year of funding and renewable up to 4 additional option years. It may take time for a college to develop intelligence-related courses and have students graduate from the IC CAE program. Colleges then need to apply for another grant in order to continue to receive federal funding following expiration of any additional option years. Since 2011, colleges have been required to demonstrate a plan to sustain their programs after the initial grant period ends. However, according to some IC CAE college officials, it is nonetheless difficult to continue the program and secure external funding once the grant is over. Some college officials have also said that the loss of grant funding can result in colleges discontinuing key aspects of the IC CAE program and can limit consortium college participation in activities. We have also observed that some colleges may have suspended their programs entirely. Specifically: Colleges may be able to sustain some, but not all components of their program once grant funding ends. For example, one college has sustained an IC CAE program since 2005 even though the college did not receive grant funding from 2008 through 2012. According to college officials, loss of grant funding resulted in the college suspending professional development activities. The program received additional IC CAE grants in 2012 and in 2017 and college officials stated they hold professional development workshops and one-on-one mentoring sessions between students and representatives from IC elements. Without grant funding, consortium colleges may not have funding for student travel to IC CAE events. Consortium colleges face a specific challenge since many of the IC CAE events are hosted by the lead IC CAE college. We spoke with faculty at two consortium colleges who said that grant funding from the program helps reduce the cost of their students’ travel to off-campus IC CAE events, such as annual colloquiums at the lead college that are attended by subject matter experts from IC elements. The distance students may need to travel can be especially challenging for colleges that are not located near the lead college, including one community college that is 400 miles away from the lead consortium college according to an IC CAE college official. DIA identifies some programs as legacy colleges, but some colleges have not updated their IC CAE program websites. For example, we reviewed the IC CAE program websites for two colleges that had received a grant from DIA after 2012, but the colleges had stopped updating their websites in 2014 and 2016. DIA has identified sustainment of the IC CAE program following termination of grant funding at colleges as a significant challenge. At a recent meeting with the IC CAE Program’s Senior Advisory Board, the head of DIA’s program office stated that the sustainment of IC CAE programs after federal funding ends tended to be a systematic failure, especially for many smaller colleges that may lack the resources of larger colleges, and that there have been no consequences for failure. While DIA acknowledges this problem, it does not have a process to systematically evaluate this issue or consider alternative approaches for colleges that may need additional support to maintain relevant curricula or professional development activities. For example, DIA has not evaluated whether some colleges’ difficulty with sustaining their IC CAE program may invalidate underlying assumptions about how the program is structured, including whether awarding grants to colleges to develop and maintain an intelligence-focused curriculum is the most effective means of establishing long-term relationships with those colleges and fostering a diverse talent pool for the IC. A key practice of sound planning is to fully evaluate key factors external to the organization that are beyond its control. IC CAE colleges decide how to brand the program as well as how to allocate resources in order to sustain their IC CAE program. These decisions could significantly affect the achievement of the IC CAE program’s mission and goals. Both ODNI and DIA officials are aware of some external factors that could affect the success of the IC CAE program, such as branding and program sustainment. As of March 2019, ODNI officials have stated that they are developing plans to address branding and sustainment as the program transitions to ODNI. DIA drafted a plan for post-grant requirements for colleges in order to maintain their IC CAE designation, though this draft plan does not address the sustainment challenges that may make it difficult for those colleges to follow these additional requirements. However, DIA internal guidance and the most recent Senior Advisory Board charter do not outline a process to identify and continuously evaluate external factors that could affect program performance. As the new program manager, ODNI may be unable to assess whether factors like program branding or sustainment might affect the IC CAE program’s implementation and potential for success without a process in place to evaluate the effect of these and other potential external factors. DIA lacks comprehensive performance measures for the IC CAE program that would allow DIA to measure program success. Specifically, DIA has not (1) clearly and consistently defined performance measures to be reported and collected, (2) collected on or reported complete information on the program, and (3) determined whether data collected may be incomplete or unreliable due to reporting challenges. DIA has not clearly and consistently defined the performance measures that need to be reported by the colleges in order to determine the IC CAE program’s success. DIA required colleges to provide reports on significant accomplishments related to the objectives in their grant proposals. However, we reviewed final grant reports that colleges submitted to DIA from 2014 to 2018 that revealed differences in how colleges reported measures. For example: Two colleges reported that a total of 664 students received an IC CAE certificate, 99 completed an internship, and 128 received a conditional offer of employment between 2012 and 2017. However, the report did not indicate whether these offers of employment were from IC elements or the number actually hired. A legacy college reported that 49 students received a conditional offer of employment or were hired by an IC element, but it did not indicate the total number of program participants. The final report from a legacy college that had an IC CAE program from 2013 to 2015 reported the total number of internships, but it did not report conditional offers of employment or total program participation. In 2017, DIA revised the reporting template for colleges to require progress on the goals and objectives in the approved grant proposal. However, the information colleges provided varies because DIA’s performance measures are not clearly stated so that colleges can report them consistently, and they are not scoped to evaluate specific program outcomes. For example, IC CAE programs are required to report their progress in developing critical language studies, but there is no minimum requirement on the type of information that a college should report in the updated template. Comparing the reporting template for two colleges from 2018, one college’s narrative provided a high level overview of its foreign language options at the college and reported that IC CAE scholars will be encouraged to participate in the language courses, whereas another college’s narrative provided details on the number of students participating in the foreign language program and details on stipends provided to students who studied abroad. DIA’s updated reporting template also required IC CAE colleges to report the aggregated totals of IC CAE participants, conditional offers of employment, internships, and hires into the IC. However, some colleges track different types of information for these metrics. For example, the way colleges count student participants in the IC CAE program varies. Some colleges only track students enrolled in the IC CAE certificate or degree program, while other colleges report much larger totals of participants, including those who are not enrolled in an IC CAE certificate or degree program but may participate in some IC CAE events. In addition, DIA’s updated reporting template did not clearly describe the hiring data that colleges are required to report. For example, colleges are required to report the total number of conditional offers that IC CAE scholars receive, but it does not specify whether this number is for all employers or just IC elements. Furthermore, it is not clear whether students that received a conditional job offer in one semester are being reported again as a hire in the following semester. Without clearly defined performance measures, decision makers may not be able to clearly identify the accomplishments of the program among the various participating colleges. DIA is responsible for reporting on the IC CAE program’s performance to ODNI, but DIA has not collected complete performance measures that cover the entire program and has not reported a complete summary of the performance measures it has collected. Since 2011, the DIA program office has collected some information from IC CAE colleges in order to monitor compliance with the colleges’ grant proposals. This information was reported by IC CAE colleges in their interim and final reports that include narrative descriptions of IC CAE program activities and descriptive data about program participants. However, DIA has not collected complete information that captured relevant performance measures for the IC CAE program. For example, between 2011 and 2016, DIA officials stated colleges provided DIA a spreadsheet of information on IC CAE program activities, including descriptions of IC CAE courses and events, study abroad program participation, IC element interaction, and information about individual IC CAE scholars. However, the data provided by the colleges varied. For example, based on a review of spreadsheets that DIA provided from the fall of 2014, some colleges provided details on IC CAE sponsored events, IC element interaction, and student employment, while other colleges did not provide any information in these areas. We also found that colleges summarized this information in their final grant. DIA’s annual reports to ODNI from 2012 to 2017 reported little of the information that DIA collected over this time period. The annual reports described financial data and provided some description of select college activities, but they did not summarize information related to any of the program’s core requirements such as curriculum development, critical language study, or professional development. For example, DIA has not collected or reported data on the number of IC CAE scholars who have studied a critical language from 2012 to 2017. The reports also did not include the total number of IC internships, conditional job offers, or hires after 2012. Moreover, college officials stated they do not report on performance measures after the grant period ends, which may limit DIA’s ability to provide comprehensive data for both active grant colleges and legacy colleges each year. DIA officials stated that legacy IC CAE colleges that have sustained the program but no longer receive a federal grant are not obligated to provide reports to DIA. According to DIA officials, DIA is currently developing a plan that would require colleges to report information in order to maintain their IC CAE designation after the grant period ends. For example, a college official from a legacy program that first received a grant in 2006 stated that the college no longer shares information with DIA because DIA had not requested it do so after the grant ended. The official noted that the college is no longer receiving support to facilitate IC recruitment of its students. DIA officials stated they have relied on colleges rather than the IC elements themselves to report data on IC CAE scholars. DIA informs colleges through its reporting template that data on internships, conditional job offers, and hires into the IC are definitive evidence of the success and sustainability of a college’s IC CAE program. However, due to challenges with collecting these data, the information being provided to DIA by the colleges may be incomplete and unreliable. While DIA has not reported on the total number of IC CAE scholars that have been hired from 2012 to 2017, it has collected some information from IC CAE colleges. For example, three colleges from our sample reported that a total of 23 IC CAE scholars were hired by the IC between the beginning of the fall semester of 2017 and the end of the fall semester in 2018. However, according to officials at these colleges, it is difficult to provide complete data on students’ employment as they no longer have direct contact with students after they graduate and some IC elements discourage applicants from discussing their employment offers with others. As a result, the information the colleges report to DIA may be incomplete because they are not able to track all the students who have graduated from the IC CAE program. ODNI also reported similar challenges when it managed the program from 2005 through 2011. ODNI reported a total of 61 IC CAE scholars were hired into the IC between 2005 and 2011 based on IC CAE college data, but noted that the hiring data from IC elements was higher than the total reported by colleges. Further, IC elements have noted that there are security risks associated with tracking the number of IC CAE scholars that receive a conditional offer of employment or have been hired into the IC. At the February 2019 IC CAE professional development summit, for example, Senior Advisory Board members from the CIA and the FBI advised IC CAE colleges that storing or sharing information about potential IC applicants on unsecured college systems is a security risk. Some IC element officials have suggested that the best way to track applicants would be to obtain a list of IC CAE scholars from the colleges and match the names against IC element applicants. However, according to officials, the IC elements would need an individual’s full legal name and college, and some IC CAE college officials raised privacy concerns with sharing student information. An IC CAE college official stated that even during the grant period, the college only provided DIA aggregated totals on student data because of privacy concerns. DIA and ODNI have collected some data on the number of applicants from IC CAE colleges and new hires from the IC elements, but they have only recently done so in a systematic manner. Officials from DIA’s IC CAE program office said they cannot force IC elements to report employment information and that the burden is on the IC elements to track and report that data. According to ODNI officials, in response to a provision in the Intelligence Authorization Act for Fiscal Year 2017, ODNI sent out a request to IC elements for data on hiring and demographic information that included questions about the number of IC CAE graduates hired by the IC. As of April 2019, officials stated that they have collected hiring and demographic information from six of the largest IC elements that includes data about the number of IC CAE graduates hired by the IC. The officials said they expect this to be a large enough sample to report in June 2019. However, according to ODNI officials, ODNI has not yet determined how it will define performance measures for the IC CAE program or how it will continue to collect and report these performance measures. A key practice of sound planning requires the development of a set of performance measures that will be applied to gauge progress toward attainment of the plan’s goals. We have also established that key attributes of successful performance measures, which include measures that cover core program activities, are that they are clearly defined and consistent and can be reliably produced. Furthermore, Standards for Internal Control in the Federal Government state that management should use relevant data from reliable sources; process this data into high-quality information that is complete, accurate, and valid; and communicate high-quality information to all levels of the department. Comprehensive performance measures would allow DIA to gauge the success of the IC CAE program in developing a pool of diverse talent with skills needed in the IC, but DIA has not defined performance measures in program guidance and documentation. In its 2012 annual report, DIA stated that it intended to redesign ODNI’s data collection tool in order to simplify reporting. However, DIA did not report data collected with this tool and stopped collecting these data altogether in 2016 after informing IC CAE colleges that the collection effort required a lengthy approval process from the Office of Management and Budget. DIA officials continued to require colleges to report performance measures after 2016 through a reporting template. In April 2019 DIA officials stated that they intended to make improvements to this template given that the way colleges have tracked student participation has varied. However, DIA did not clearly and consistently define performance measures for all aspects of the program, process them via a data system or spreadsheet, or report them to ODNI. As the new IC CAE program manager, ODNI will not be able to gauge the success of the IC CAE program in achieving its mission without defining, collecting, and reporting on comprehensive performance measures. Since 2012, DIA has not conducted a comprehensive assessment of the IC CAE program. According to a 2013 amendment to the memorandum of understanding with ODNI, DIA was responsible for providing ODNI with an annual review of the program’s performance and including possible outcomes, such as specific benefits to the IC. ODNI was responsible for evaluating this information to ensure the appropriate and efficient expenditure of IC resources and performance improvement. However, DIA’s annual reports to ODNI from 2012 to 2017 did not comprehensively assess the program’s performance or the extent to which the program had achieved its mission. These reports only provide a few details about IC CAE program activities and summarize grant expenditures. For example, the 2016 annual report for the IC CAE program provided information on the number of grants awarded, a list of IC CAE colleges participating in the program, funding and execution data, and a sample of IC CAE program events from three colleges. However, the report did not provide complete details on the status of the program at each IC CAE college, such as a summary of the performance metrics it had collected from all of the colleges with an active grant. DIA officials said that they only included the information in annual reports that ODNI requested in the memorandum of understanding and lacked resources to provide a comprehensive assessment. However, the memorandum of understanding requires DIA to provide an annual review of the IC CAE program’s performance to possibly include outcomes such as the number of students who completed IC CAE coursework and specific benefits to the IC. As ODNI officials work with DIA to transition the IC CAE program back to ODNI, ODNI officials began working with the MITRE Corporation in February 2019 to evaluate the IC CAE program. ODNI officials said they will rely on MITRE’s findings and their own interactions with IC CAE colleges to determine how to manage the program. Officials stated they expect the evaluation to be complete by October 2019. However, ODNI has not yet developed a plan to conduct continuous and comprehensive assessments of the IC CAE program. A key practice of sound strategic planning is the use of assessments, through objective measurement and systematic analysis. For example, an evaluation plan can assist an agency in determining the appropriateness of a program’s goals, the effectiveness of implemented strategies, and the potential need for corrective action. The memorandum of understanding between DIA and ODNI in 2011 and amended in 2013, designated performance reporting as a DIA responsibility, but DIA did not identify performance assessment as a responsibility in program guidance. The IC CAE program office’s standard operating procedures provide that the grant officer’s representative monitors an IC CAE college’s compliance with its grant assistance agreement and collects performance and financial data reports. However, there is no mention of a systematic, outcomes-based assessment of these reports or the program as a whole. Without such assessments, the IC will not be able to determine whether the IC CAE program is effectively increasing the pool of diverse applicants. Congress will also be unable to determine the return on investment in this long-standing program. IC elements participate in the IC CAE program in a variety of ways, including by attending IC CAE college workshops and recruitment events and participating in the annual IC CAE program meeting, among other events. Table 3 shows the varying levels of participation in the IC CAE program among the eight selected IC elements, as reported by IC element officials. IC elements’ participation in the IC CAE program varies according to the specific organizational needs of each IC element. Some IC elements do not participate actively in the program because they do not directly hire employees into their intelligence office or because they conduct only limited hiring. For example, according to officials from the Department of Energy’s Office of Intelligence and Counterintelligence, the office is small and hiring is therefore limited. Further, officials stated the office often hires specialized personnel with advanced degrees and would not hire IC CAE scholars from undergraduate programs. Similarly, State Department officials from the Bureau of Intelligence and Research stated that they do not participate in events since they do not have direct hiring authority. Further, these officials stated that the State Department’s participation in IC CAE events is also constrained by limited personnel and financial resources. Other IC elements, such as the CIA and the NSA, have developed separate relationships with colleges and programs to address their specific hiring needs. CIA. The CIA has reduced its involvement with the IC CAE program to better align its needs according to CIA officials. In 2009, CIA selected senior officers to serve as advisors to 16 IC CAE colleges. The CIA advisors were directed to make a minimum of two visits per year and conducted a wide range of activities to include presenting at colleges events, counseling IC CAE scholars, and discussing CIA and IC career opportunities. However, about a third of the advisors were pulled back because, according to CIA officials, the IC CAE colleges were not meeting hiring expectations. Since 2014, CIA has focused its efforts on only six of the IC CAE colleges based on the return on investment from these colleges and alignment with CIA hiring needs. In addition, according to CIA officials, CIA has designated five universities as signature colleges to recruit skilled applicants from a range of cultures and backgrounds. According to CIA officials, the signature college program targets large, diverse colleges where the CIA has received a significant number of applications. Its criteria for selection of signature colleges include high diversity, the size of the college, and potential for developing a deep relationship. Two of the five CIA signature colleges are also in the IC CAE program and are currently receiving or have received grant funding. NSA. According to NSA officials, NSA has been involved in the IC CAE program since its inception, and its involvement includes participating in a variety of events such as colloquium, summer seminars, and recruitment events. In addition, NSA has also sponsored two types of Centers of Academic Excellence, one for cyber defense and one for cyber operations. The goal of these programs is to develop technical skills by promoting higher education and research in cyber defense and producing professionals with cyber defense expertise. In addition, the programs also aim to broaden the pool of skilled workers capable of supporting a cyber-secure nation. The programs involve awarding a designation as a Center of Academic Excellence in Cyber Defense or Cyber Operations to U.S. universities based on criteria. No funding is provided to the U.S. universities. According to NSA officials, these programs are independent of the IC CAE program and have different goals from the IC CAE program. Officials stated NSA’s CAE programs are focused specifically on increasing the pipeline of cyber talent. Further, some IC elements’ recruitment strategies incorporate the IC CAE program as part of their strategy, but it is not the only aspect of the elements’ approach to recruiting. For example, according to NGA’s Campus Recruitment Strategy, the agency targets high-quality colleges that provide access to diverse applicants in high-quality, mission-aligned degree programs across a broader geographic reach. The strategy has 31 designated colleges that were selected based on a variety of criteria, including demographic diversity and academic programs that align with the agency’s mission areas. According to NGA officials, they continue to recruit from at least seven IC CAE colleges; however, being an IC CAE college was not part of the primary selection criteria for colleges in NGA’s campus recruitment strategy. As program manager, DIA has relied on the IC CAE Senior Advisory Board and its charter as a means to engage IC elements in the program. However, not all IC elements participate on the Senior Advisory Board or in the IC CAE program. For example, in the November 2017 board meeting, only 9 of the 17 elements attended the meeting and a quorum was not established. Without a quorum, votes held during a meeting are not valid and actions cannot be approved. Moreover, during board meetings, members have raised concerns about limited attendance, citing concerns that only about half of the members regularly attend. According to some IC element officials, they do not attend IC CAE program events, including the Senior Advisory Board meetings, because the program does not meet their IC element’s organizational needs. For example, as discussed above, some IC elements have developed separate relationships with colleges not in the IC CAE program. Further, as discussed above, some IC elements have developed separate relationships with colleges and programs to address their specific hiring needs. As a result, some IC element officials have stated they have intentionally reduced their recruitment at some IC CAE colleges. Since not all IC elements participate in the IC CAE program or attend the board meetings, DIA has had to conduct other outreach to engage IC elements. According to DIA officials, since 2017 the IC CAE program office has conducted additional ad hoc outreach to engage with IC elements. For example, DIA officials have stated the IC CAE program office has utilized ODNI forums, such as the IC Recruitment Council and IC Chief Human Capital Office Council to engage with IC elements on the IC CAE program. However, DIA officials also stated that not all IC elements attend these ODNI council meetings because different offices within the IC elements are responsible for attending the meetings. Some IC elements are set up differently with regard to which office within the IC element participates in the IC Recruitment Council, so the IC element representatives to the IC CAE Board can differ from those who attend the IC Recruitment Council. While these ad hoc outreach efforts are likely a positive step to improving coordination, there remains a lack of engagement by all IC elements. Standards for Internal Control in the Federal Government state that management should establish and operate monitoring activities, to include a determination of when to revise the program baseline to address program needs. Further, the standards state that management should evaluate and document the results of ongoing monitoring and separate evaluations to identify issues. As program manager, DIA has not established a process for monitoring and assessing IC elements’ participation in the IC CAE program, and the board’s charter does not describe such a process. As result, DIA does not fully understand the reasons for the lack of engagement on the part of IC elements. IC elements that do not attend board meetings are not engaged in the discussions and decisions being made about the program. Similarly, IC elements that do not participate actively in the program have limited contact and interaction with IC CAE colleges, which has hampered the effectiveness of the IC CAE program. Without a process for monitoring and assessing IC elements’ participation in the IC CAE program, ODNI will not be able to tailor the program to meet the needs of the IC and address the overall program goal of creating a diverse pool of applicants for the IC. Assessing and addressing IC elements’ reasons for not participating in the program would increase ODNI’s understanding of the factors that inhibit participation and inform an approach to mitigating these factors and achieving program goals. The IC CAE program is a collaborative effort that allows IC elements to participate at college events, such as colloquia, speaker series, and campus recruitment events. The IC CAE Senior Advisory Board was created to provide policy and guidance for the IC CAE program and ensure that participating IC elements are included in decisions related to policy matters. The board’s charter states the Senior Advisory Board members are responsible for attending board meetings, voting on issues before the board, acting as points of contact for the program, and promoting the program. However, the charter does not define the expected or required level of participation of IC elements at IC CAE colleges. The IC CAE program manager, DIA, has communicated the schedule of IC CAE college events during Senior Advisory Board meetings and also asked for IC elements to participate in various events. Through the IC CAE grant process, IC CAE colleges are required to host a variety of events to educate IC CAE colleges about the IC. Based on the IC CAE grant announcements, these events are predicated on IC element participation. Specifically, recruitment fairs at colleges are facilitated by IC elements and IC element officials are speakers at colloquia events, with a primary goal of maximizing relationships and outreach. However, some colleges have experienced challenges with engaging with IC elements to attend these events. For example: An official from a legacy IC CAE college noted that it has been difficult to get IC elements to attend college events or recruit from the college. The official stated that IC element participation has been ad hoc and based on personal relationships with the IC elements rather than assistance from the IC CAE program office. For example, the official noted that at recent events the college was only able to attract 8 IC elements to a recruiting event compared to the 20 representatives across 12 IC elements who attended the events in the past. An official from an active IC CAE college also noted that some IC elements are not well informed about the IC CAE program. For example, the official noted that the college would like more IC elements to attend IC CAE college events. However, the official stated that the responsibility of developing relationships with IC elements has been placed on the college. According to the official, the IC elements should be more aware of which colleges have IC CAE programs and should be the first stop for IC element recruitment. The official also stated IC CAE colleges would like the IC element to drive the relationships with colleges. Our leading collaboration practices include (1) having participating agencies clarify roles and responsibilities and (2) ensuring that participating agencies document how they are collaborating in a written agreement and develop ways to continuously update and monitor these agreements. Roles and responsibilities can be defined through laws, policies, memorandums of understanding, or other requirements. The IC has defined the mission for the IC CAE program, but the current program manager, DIA, has not clarified IC element roles and responsibilities for program participation and the Senior Advisory Board charter does not clarify what is expected of the IC elements regarding participation at IC CAE events. According to DIA officials currently managing the program, the Senior Advisory Board charter is the key to getting IC element participation in the program and overall program success. An update to the Senior Advisory Board charter could include all relevant participants and define roles and responsibilities. Without clearly defined roles and responsibilities, the IC elements are not taking full advantage of what the IC CAE program has to offer, including participation in events and college engagement. Thus, the IC CAE colleges will not be able to fully execute their IC CAE programs and the program may not be able to meet its goal of creating a pool of diverse applicants for the IC. In 2005, ODNI established the IC CAE program with a goal of creating an increased pool of culturally and ethnically diverse, multi-disciplinary job applicants for the IC. However, the current program manager, DIA, has not sufficiently planned and overseen the program and the IC is unable to determine whether the program has been successful in meeting its goal to create an increased pool of culturally and ethnically diverse job applicants for the IC. Specifically, DIA has not developed results-oriented goals or documented an overall strategy for the program, evaluated external factors that could significantly affect the program’s success, defined and collected comprehensive metrics, or conducted an assessment of the program’s performance. As ODNI takes over the program, it needs to address these sound planning practices in order to determine whether the program is being implemented successfully and to help ensure the IC has a trusted, diverse workforce with the right expertise. Further, without sufficient planning and oversight, decision makers will also be unable to determine the return on investment in this long-standing program. In addition, ODNI also needs to improve IC element participation in the program. The IC CAE program is a collaborative effort that encourages participation among all IC elements. However, DIA has not established a process to monitor and assess IC element participation in the program or clearly defined IC elements roles and responsibilities for the IC CAE program. A process for monitoring and assessing IC element participation and addressing IC elements’ reasons for not participating in the program will increase understanding of the factors that inhibit participation and inform ODNI’s approach to mitigating these factors and achieving its goal for the program. Further, without clearly defined roles for IC element participation in the program, IC CAE colleges may not be most effectively executing their IC CAE programs and the program overall may not be able to meet its goals. We are making the following seven recommendations to the Director of National Intelligence as the IC CAE program transitions to ODNI: The Director of National Intelligence should establish and document results-oriented goals that include specific targets or milestones for the IC CAE program. (Recommendation 1) The Director of National Intelligence should establish and document strategies to achieve the results-oriented goals that are established for the IC CAE program. (Recommendation 2) The Director of National Intelligence should develop and document a process to identify and continuously evaluate external factors that could affect the program’s ability to achieve identified goals. This should include, but not be limited to, a consideration of program branding and post-grant sustainment. (Recommendation 3) The Director of National Intelligence should define and document comprehensive performance measures for the IC CAE program, collect and evaluate the completeness and reliability of information it receives from grant recipients and IC elements, and report this information on a regular basis. (Recommendation 4) The Director of National Intelligence should establish a requirement for and develop a plan to periodically evaluate the IC CAE program’s performance through objective measurement and systematic analysis. (Recommendation 5) The Director of National Intelligence should develop a process for assessing why some IC elements are not participating in the IC CAE program and address these reasons in order to ensure the program is structured to meet the needs of IC elements. (Recommendation 6) The Director of National Intelligence should clearly define IC elements’ roles and responsibilities for participation in the IC CAE program to better facilitate interagency collaboration in support of the program. (Recommendation 7) We provided a draft of this report to ODNI for review and comment. In written comments, ODNI concurred with all seven of our recommendations but did not identify the steps it plans to take to address the recommendations as the IC CAE program transitions to ODNI. ODNI’s comments are reprinted in their entirety in appendix III. ODNI also provided technical comments, which we incorporated as appropriate. We also provided a draft of this report to the CIA, Department of Defense, DIA, FBI, NGA, NRO, NSA, the Department of State’s Bureau of Intelligence and Research, and the Department of Energy’s Office of Intelligence and Counterintelligence for review and comment. These agencies concurred without providing comments on the draft report. NGA provided technical comments, which we incorporated as appropriate. We are sending copies of this report to the appropriate congressional committees. We are also sending copies to the Secretaries of Defense, Energy, and State; the Directors of National Intelligence, DIA, CIA, NGA, NRO, and NSA; and the Attorney General. In addition, this report will be available at no charge on our website at http://www.gao.gov. If you or your staff have any questions concerning this report, please contact Brian M. Mazanec at (202) 512-5130 or mazanecb@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix IV. The Office of the Director of National Intelligence (ODNI) was the Intelligence Community (IC) Centers for Academic Excellence (CAE) program manager from 2005 through 2011. Internal documents and grant announcements from that period state that the program’s mission was to increase the pool of eligible applicants in core skills areas, specifically targeting women, racial and ethnic minorities and individuals with varied cultural backgrounds, regional expertise, and language proficiency. ODNI outlined four goals in its 2008 guidance for the program, including a focus on developing relationships with colleges, providing resources and grants to competitively selected colleges, providing technical assistance in the design and implementation of colleges’ IC CAE programs, and documenting results to improve the efficacy of the IC CAE program. Each of these goals included supporting objectives. For example, the goal of providing support, resources, and grants to competitively selected colleges included four supporting objectives, such as instituting long-term practices to increase relationships with minority-serving institutions and providing access to IC internships, co-ops, and graduate fellowships. These goals and objectives were aligned with the program’s overall mission, but they were not defined in measurable terms that would allow future assessments of whether they were being achieved. For example, ODNI did not establish targets for the goals or supporting objectives listed above that would have allowed it to determine how successful it had been at supporting long- term programs at minority-serving institutions or providing access to IC employment opportunities. In addition, ODNI defined a strategy to support its program goals, and the strategy included the following four elements: outreach to high schools; operations at colleges, including curriculum development; infrastructure at the colleges to support these operations such as faculty and administrators; and relationships between IC CAE programs and IC elements. These elements of ODNI’s strategy described specific operational requirements for the program. For example, IC CAE grant announcements in 2006, 2009, and 2011 supported a wide range of academic activities that prioritized the development of curricula in national security studies, science and technology programs, study abroad programs, courses in critical languages, and pre-collegiate outreach through activities like summer camps to raise awareness and interest in IC careers. ODNI also defined assessment and evaluation as an overarching part of the program’s strategy, as shown in figure 5. ODNI worked with a contractor to conduct annual performance evaluations through 2012. The contractor developed an evaluation methodology and reviewed colleges’ interim reports, collected and verified performance data, and developed findings and recommendations. For example, the contractor recommended that IC CAE colleges broaden their critical language offerings and increase the number of IC CAE Scholars enrolled in foreign languages courses in each of the annual reports from 2007 to 2010. ODNI defined performance measures and reported data on activities, including the number of IC CAE courses and events, demographic information, and employment outcomes. Specifically, IC CAE colleges were required to report these data quarterly, and the contractor compiled the data annually into its program reviews. Table 4 shows selected performance measures outlined in ODNI’s final report that summarized information collected from 2004 through 2011. Table 5 and table 6 show the 46 grants managed by Office of the Director of National Intelligence (ODNI) and the Defense Intelligence Agency (DIA). The total amount of grant funding projected to be obligated from fiscal year 2005 and fiscal year 2021 is $69,053,618, not including a $250,000 contract in September 2004 to initiate a pilot Intelligence Community (IC) Centers for Academic Excellence (CAE) program at Trinity Washington University. Tables 7 and 8 list the IC CAE colleges by designation of eligibility for Department of Education funding as a minority serving institution under various statutory grant programs including programs authorized by the Higher Education Act of 1965, as amended. Eligibility for grant funding under these statutory programs as determined by the Department of Education in 2018 does not designate or certify any college as a particular type of institution, for example, as a Hispanic Serving Institution. The colleges listed in tables 5 and 6 are listed in the order that they received a grant by fiscal year and some IC CAE colleges received multiple grants. Grants fund a base year and up to 4 additional option years. The consortium colleges below are listed alongside the IC CAE college that received a grant. ODNI and DIA awarded IC CAE grants to colleges following an announcement for proposals in fiscal years 2006, 2009, 2011, 2014, 2017, 2018, and 2019. In addition to the individual named above, Kristy Williams, Assistant Director; Jason Bair; Tracy Barnes; John Bumgarner; Meeta Engle; Gina Hoover; Amie Lesser; Benjamin Licht; Ned Malone; Parke Nicholson; Alice Paszel; Sarah Veale; and Lillian Yob made key contributions to this report.

Summary: A trusted, diverse workforce with the right expertise is critical to ensuring the IC achieves its mission of delivering distinctive, timely insights with clarity, objectivity, and independence. ODNI established the IC CAE program in 2005 to educate highly qualified students of diverse backgrounds and encourage them to pursue careers in the IC. ODNI and DIA have provided 29 colleges a total of 46 IC CAE grants through fiscal year 2018, totaling approximately $69 million through fiscal year 2021. This report evaluates the extent to which (1) DIA has planned and overseen the IC CAE program since 2011 and (2) selected IC elements are participating in the IC CAE program and have clearly defined roles. GAO reviewed IC CAE documentation related to DIA program planning and oversight from 2011 through 2019 and applied key practices of sound planning to evaluate DIA's management of the program. GAO interviewed selected IC elements and IC CAE college officials and reviewed related documentation to assess program planning and implementation. The Defense Intelligence Agency (DIA) has not sufficiently planned and overseen the Intelligence Community (IC) Centers for Academic Excellence (CAE) program—intended to create an increased pool of culturally and ethnically diverse job applicants for the IC—after the program transitioned from the Office of the Director of National Intelligence (ODNI) to DIA in 2011. Specifically, DIA has not applied most of GAO's key practices of sound planning in overseeing the program (see table), thus challenging decision makers' ability to determine the program's return on investment. Specifically, while DIA has developed some short-term goals and plans for the program, DIA has not established results-oriented program goals or an overall strategy that details the agency resources and processes required to achieve the program's mission. Similarly, DIA collected some data for the program and required colleges to provide reports on significant program accomplishments, but these data are not complete or reliable and have not been used to comprehensively evaluate the program's success. As oversight responsibility for the IC CAE program transitions back to ODNI in fiscal year 2020, ODNI will not be able to determine the extent to which the program has been successful in achieving its mission without establishing and documenting goals with targets and milestones; developing strategies to achieve those goals; and defining, collecting, and reporting comprehensive performance measures. Selected IC elements are participating in the IC CAE program to varying degrees, but DIA has not established a process for monitoring and assessing IC elements' participation or clearly defining IC elements' role in the program. The IC CAE program is a collaborative effort that allows IC elements to participate in college events, such as IC CAE recruitment events. However, not all IC elements participate in the program. As IC CAE program manager, DIA has engaged with IC elements in a variety of ways, but this engagement has not resulted in consistent participation among the IC elements. Moreover, program documentation has not clearly defined IC elements' roles and responsibilities for participation. Without a process for monitoring and assessing IC elements' participation and clearly defining roles and responsibilities, ODNI will neither be able to identify reasons for the lack of IC element engagement nor ensure that IC elements are taking advantage of the IC CAE program and its goal of creating a diverse pool of applicants for the IC. GAO is making seven recommendations to the Director of National Intelligence, including that ODNI establish and document results-oriented goals and strategies for the IC CAE program; define, collect, and report comprehensive performance measures; and clearly define the roles and responsibilities of the IC elements for participation in the program. ODNI concurred with the recommendations but did not identify steps it plans to take to implement them.