DOD’s military Tuition Assistance Program helps active duty service members—some of whom may regularly be reassigned to another location, including overseas—pursue an education. Through partnership agreements between DOD and more than 3,000 schools, during off-duty hours, service members are able to take undergraduate, graduate, vocational, licensure, certificate, and language courses. They may also complete their high school education, if necessary.service members who participate in the program are enrolled in undergraduate courses. The Undersecretary of Defense for Personnel and Readiness within DOD is responsible for implementing the Tuition Assistance Program, which includes the provision of educational counseling for service members.However, each military service is responsible for establishing and operating its own program. Through 195 education centers located on U.S. military bases, advisors are available to provide assistance and information to service members pursuing their education. These bases also make classroom space available for service members to take classes on base, although the majority of service members enroll in online classes. To participate in the program, service members must meet certain requirements. In consultation with an advisor, they must develop an education goal and education plan, and maintain a 2.0 grade point average (GPA) for undergraduate-level courses and a 3.0 GPA for graduate-level courses. Service members can receive up to $250 in tuition assistance per credit hour, with a maximum of $4,500 each year. If the cost of tuition exceeds the amount that the program provides, service members are eligible for other federal financial aid, such as federal grants and loans, to cover their expenses. Tuition is paid directly to the schools by DOD, and if a service member fails to complete a course or receives a failing grade, the student must pay back the money for those courses. Schools participating in the military Tuition Assistance Program must sign DOD’s Voluntary Education Partnership memorandum of understanding (MOU), which requires, among other things, that the schools (1) be accredited by a national or regional accrediting agency recognized by the Department of Education (Education); (2) comply with state authorization requirements consistent with Education regulations; (3) be certified to participate in federal student aid programs authorized under Title IV of the Higher Education Act of 1965; (4) disclose basic information about the school’s programs and costs, including tuition, fees, and other charges to service members; and (5) undergo, when requested, an evaluation of the quality of the education programs it is providing to service members. DOD contracts with an independent entity to assess the quality of postsecondary educational programs and services used by service members to assist in the improvement of these educational programs and services. In accordance with contract requirements, the contractor was to conduct evaluations of individual postsecondary schools. Also, for selected military bases that have a school on the base, the contractor was to evaluate all of the schools located on a single base and the facilities and operations of that base that support these schools and the delivery of education services to service members. Specifically, each year DOD required: four evaluations of military bases (with limited scope evaluations of the schools located on the base); two evaluations of distance learning (or online) schools; and four evaluations of schools located in close proximity to the base. Education is responsible for the administration of all federal student aid under Title IV of the Higher Education Act of 1965. Under that act, Education has oversight of the more than 7,200 postsecondary schools that participate in federal student aid programs, including those that participate in DOD’s Tuition Assistance Program (but not with respect to compliance with DOD’s requirements). Specifically, Education must certify a school’s eligibility to participate in federal student aid by determining that the school is accredited by an accrediting agency it recognizes, is authorized to operate within a state, and meets certain administrative and financial requirements. In addition, postsecondary schools that provide federal student aid are subject to program reviews by Education, which are made available on its website. Education also maintains websites that provide publicly available information about schools that participate in federal student aid programs authorized under Title IV of the Higher Education Act of 1965, including graduation rates, default rates, and costs. In addition to Education reviews, these schools are subject to compliance and financial audits by independent auditors. 38 U.S.C. §§ 3301-3325. housing and book payments to eligible students. As part of this program, VA is required to conduct annual compliance reviews to assess whether postsecondary schools that receive VA educational benefits adhere to applicable laws and regulations. Postsecondary schools that participate in Education, VA, and DOD education benefits programs include (1) public schools, which are operated and funded by state or local governments; (2) nonprofit schools, which are owned and operated by nonprofit organizations whose net earnings do not benefit any shareholder or individual; and (3) for-profit schools, which are privately owned and whose net earnings can benefit individuals or shareholders. In fiscal year 2013, 571 advisors were available to provide a range of information on programs and educational support to nearly 280,000 service members taking courses funded under the Tuition Assistance Program (see fig. 1). Each military service determines the number of advisors it will allocate to support service members’ education, based on competing priorities that balance education support with readiness needs. Across all four services, there was a net increase of 27 advisors from fiscal year 2011 through 2013. That increase, however, was driven primarily by the Air Force, which added 48 advisors to its program. According to one Air Force official, the increase was largely the result of provisions in law requiring that service members separating from the military be provided with transition assistance services (including education advice). Conversely, the Navy reduced the number of its advisors by 25. A Navy official said this decrease was due to budget cuts resulting from sequestration. This official said that the decrease in the number of advisors also reflects a shift within the Navy to reduce support for off-duty education and direct it towards training that directly affects military readiness—such as training in operating and maintaining complex radar and fire control systems for ballistic missile defense. (See app. I for statistics on service members and advisors participating in the Tuition Assistance Program, fiscal years 2011 through 2013.) From fiscal year 2011 through 2013, the number of service members taking courses using the Tuition Assistance Program declined by 8,819 (about 3 percent), according to DOD data. DOD officials said the decrease was due in part to a temporary suspension of the program resulting from the automatic spending reductions to federal budgets as part of the 2013 sequestration. Further, according to DOD officials, the drawdown of forces in 2013 accounted for some of this decrease. As the forces continue to draw down, the demand for tuition assistance could drop as service members leave active duty service and are no longer eligible for the program. Decreases in the number of active duty service members could potentially affect the number of advisors needed in the future. To help service members pursue educational opportunities, DOD advisors are available to provide a range of assistance and information and service members can also receive assistance from other sources (see fig. 2). DOD advisors, who are required to have specific training in educational advising, are available to help service members determine their educational goals, advise them about the range and types of schools they can attend, and the types of courses and degrees these schools offer. (See app. II for a description of DOD advisor qualifications.) In addition, advisors provide information on possible sources of funding for education, such as federal grants and loans, and discuss the program requirements. Each of the services has guidance that describes the roles and responsibilities of their respective DOD advisors. (See app. III for information on guidance the services provided about the role of DOD advisors.) At Joint Base Andrews—where 2,223 service members were enrolled in courses during fiscal year 2013—the two advisors we interviewed said that as service members arrive on base they are provided with information about Tuition Assistance Program funding, eligibility, and the schools available. These advisors reported holding information sessions, distributing brochures and pamphlets, posting information on the military base’s Facebook™ page, and holding office hours for service members who want to meet in person to receive this overview. Once a service member decides to register for the Tuition Assistance Program, the advisors said they are available to answer questions about the schools, financing education, and transfer of credits if reassigned to another location. In addition, they said that they might also explain the differences in the types of programs schools offer and share information such as graduation rates and other school statistics available from Education websites. Further, the advisors we spoke with said they also discuss a range of financing options available, including other military aid, federal financial aid (grants and loans), and private student loans. With respect to private student loans, which sometimes have significantly higher interest rates than federal student loans, the advisors told us they advise service members to exhaust all federal options to cover college costs before pursuing private loans. Once a service member has completed a certain number of course hours, as required by the program, advisors we interviewed told us they work with that individual to develop, and then approve, an education plan for completing a specific course of study. These plans are to help students and advisors track the student’s progress toward fulfilling course requirements that will ultimately lead to a degree, certificate, or license. All services require the development of an education plan for service members. When service members are registering for classes, advisors we interviewed said they are the busiest. They also told us that requests for their services vary by service member, with some requiring little assistance. Advisors added that other support staff at the education center on the base are available to assist service members when an advisor is not needed. For example, support staff may answer routine questions about the program; mail or e-mail the service member materials; and confirm that the service member has dropped or withdrawn from a course, or that course grades have been entered into the system. All of the services have similar staff to perform these functions. Further, as required by DOD, all participating schools have their own counselors available to help service members, for example, by recommending courses and explaining how to register for classes. At Joint Base Andrews, counselors representing each of the five schools operating on that base told us that they help students decide which courses they will need to complete a degree program and help transfer the students’ credits if they are reassigned to another duty station. These counselors also told us that while they are able to answer routine questions about the Tuition Assistance Program, they refer the students to the DOD advisors for more detailed information. Lastly, we found that DOD and each of the services provide online information and tools about the Tuition Assistance Program through multiple websites. For example, one of the DOD websites provides a list of all of the schools eligible to participate in the program. In addition, all of the services report that they are increasingly using online tools to deliver information and support to service members. Also, the Navy relies heavily on its online resources to help answer service members’ questions. In addition, the Army website allows service members to register for classes, and access information about other financial aid programs, such as federal grants, loans, and GI bill funding, which provides funding for veterans’ education. The evaluations of schools by DOD’s contractor have provided a range of different information on schools participating in the Tuition Assistance Program, according to DOD officials, but they do not provide the information DOD needs to assess the schools. This is because DOD lacked a specific plan to frame and guide the evaluations, and did not require the contractor to develop one. According to federal standards, an evaluation plan should clearly define the evaluation questions and methodology and address the collective knowledge, skills, and experience needed by the entity conducting the evaluations. According to DOD’s contract, evaluations were to assess school quality. However, our review of the contract found that the 15 areas for evaluation that DOD provided to the contractor were often broad, not clearly defined, and lacked specificity (see app. IV for list of 15 areas). For example, one area was simply stated as “the methods whereby academic institutions are invited on the installation.” Another simply stated “the degree of congruence among various missions (the military, installation, and the institutions), the education plan of the installation, the educational programs provided by the institutions which have a MOU with the installation and the distance learning providers.” Based on our review of these 15 areas, it was not always clear what DOD was asking the contractor to evaluate and how the 15 areas would be measured. For example, in asking the contractor to assess the “degree of congruence,” it was not clear what DOD meant and how the contractor would measure this area. According to the contractor, DOD did not clearly define quality and the 15 areas to be assessed were the only formal guidance provided to them. Further, evaluation questions are critical because they frame the scope of the assessment and drive the evaluation design, the selection of data to collect, and the study results. For this reason, evaluation questions must be clear and specific and use terms that can be readily defined and measured. Had DOD developed a specific plan to frame and guide the evaluations, such a plan could have better positioned DOD to fully assess the skills needed by the contractor before awarding the evaluation contract. Specifically, DOD did not initially include requirements in its contract to ensure that the contractor provided personnel with the requisite education and experience needed to conduct the evaluations. Thus, DOD had to modify its contract to obtain the needed skills (see app. V for contract modifications). For example, in a 2013 modification, DOD required that the contractor acquire staff with a working knowledge of measurement methods and tools; sufficient experience in postsecondary education; and expertise in education theories, principles, and practices; among other skills. Although the contractor hired additional staff in time for the fiscal year 2014 evaluations, we identified problems in some fiscal year 2013 evaluation reports that were also present in the more recent 2014 evaluation reports we reviewed, where data from student surveys were misinterpreted and erroneously reported. Specifically, the contractor made broad generalizations about student satisfaction with their school based on survey responses from a non-representative sample of students whose responses may not have been the prevailing view of the other students who did not respond. In providing technical comments on a draft of this report, the contractor said that they made no attempt to draw general conclusions about the student body as a whole based on the limited survey responses. We found, however, that several of the reports did contain broad generalizations. For example, a recent evaluation report stated that there was a “high overall degree of student satisfaction” with the institution based on responses from a non-representative sample of students. Lastly, in some instances, we found that the contractor made recommendations that only tangentially related to quality. For example, a fiscal year 2012 report of the schools on one military base recommended replacing the artwork in the classrooms with pictures of students in the hallways to foster a positive climate. In addition, a fiscal year 2013 evaluation of one school included a recommendation that for clarity and consistency, the school change the names of the academic terms to read “mid-Fall,” “mid-Winter,” “mid-Spring,” and “mid-Summer.” In a fiscal year 2014 report, the contractor recommended that the school change the term “Military Assistance” to “Tuition Assistance” in the listing of funding sources within the school’s online application process. In providing technical comments on a draft of this report, the contractor stated that their contract with DOD required them to evaluate all aspects of the course offerings provided by educational institutions under review. As a result, the contractor said in total it made over 300 recommendations, which were both tangential and more central to quality. For example, the contractor cited several recommendations which they viewed as more central to quality including recommendations that the close working relationship between one education office and a school be maintained; additional professional development opportunities be offered to adjunct faculty members; and an advisor position that had recently been cut be reinstated and filled with an experienced advisor. By not fully considering the skills and expertise needed by the contractor before awarding the contract, DOD risked receiving information provided from the contractor that would not meet its needs. According to the American Evaluation Association, agencies should ensure that contractors conducting evaluations possess the education, skills, and experience necessary to undertake the evaluations. After concluding that the evaluations are not providing the information needed, DOD decided not to renew its school evaluation contract. The agency is suspending the evaluations and plans to refocus its evaluations in accordance with a recently issued Executive Order. Under that Executive Order, DOD, along with Education and VA, are to ensure that schools provide meaningful information to service members about the cost and quality of programs to help them make informed choices about how to use their tuition benefits; prevent abusive and deceptive recruiting practices that target service members; and ensure that schools provide high-quality academic and student support services to service members, among other things. Further, the Order requires that DOD, Education, and VA work collaboratively to develop a comprehensive strategy for developing student outcome measures (e.g., graduation and retention rates) and strengthen their oversight of schools to protect service members participating in educational programs, such as the Tuition Assistance Program. According to DOD officials, they are meeting with Education and VA to discuss how they might coordinate efforts and leverage information from these agencies. DOD officials have been discussing a process for determining the number of school evaluations DOD will conduct each year and how they will select the schools for evaluation. DOD officials are also exploring whether these evaluations will be conducted under contract or through an interagency agreement. Although DOD has several efforts in place, it does not have a plan guiding these efforts. Fundamentally, the agency has not yet developed a plan that includes specific questions to frame the evaluations and the qualification requirements for those conducting the evaluations. Unless DOD addresses these issues, it risks receiving future evaluations that will not provide all of the information necessary to evaluate schools. Although the federal investment in service member education is substantial, DOD’s current approach has left the department without the information it needs about the quality of schools that served about 280,000 service members in fiscal year 2013. By not having a plan in place to guide the evaluation of schools, DOD’s ability to effectively assess the schools has been limited. Without a plan, it will be difficult for DOD to obtain information on the quality of the schools and determine whether any adjustments are needed in the program. To improve the usefulness of information from school evaluations, we recommend that the Secretary of Defense direct the Undersecretary of Defense for Personnel and Readiness to develop a plan for future school evaluations that includes, among other things, clearly-defined evaluation questions and an assessment of the experience, expertise, and skills needed by the personnel from the entity or entities conducting the school evaluations. We provided a draft copy of this report to the Department of Defense for review and comment. We received written comments from DOD, which are reproduced in appendix VI. In its comments, DOD agreed with our recommendation that a plan was needed to guide future school evaluations. However, DOD expressed concern that we had not made recommendations about whether the number of advisors was sufficient relative to the number of service members they serve. As stated in our report, each military service determines the number of advisors it will allocate to support service members’ education, based on competing priorities that balance education support with readiness needs. We made no recommendations in this area in acknowledgement of the difficulty inherent in weighing readiness needs against education needs. Further, with respect to the advisors, DOD stated that the numbers cited in our report can be misleading because its advisors perform other education functions unrelated to the Tuition Assistance Program. Our report, however, already acknowledges that advisors also support additional service members who are participating in DOD’s other voluntary education programs. We also provided a draft copy of this report to the Department of Education and the DOD contractor for review and comment, and received technical comments, which we incorporated into the report, as appropriate. We are sending copies of this report to the appropriate congressional committees, the Secretaries of Defense and Education, and other interested parties. In addition, this report will also be available at no charge on GAO’s website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (617) 788-0534 or emreyarrasm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix VII. Tests and measurement: Study of the selection, evaluation, administration, scoring, interpretation, and uses of group and individual aptitude, proficiency, interest, and other tests. Adult education: Study of the adult as a learner, teaching-learning theories for adults, models and procedures for planning, designing, managing, and evaluating adult learning activities. Educational program administration: Study of the foundation and methods in organizing for adult and continuing education programs. Curriculum development or design: Study of the principles and techniques for development of curricula for adult or vocational education programs. Teaching methods: Study of teaching strategies and learning styles of the adult learner. Guidance and counseling: Study of the purposes and methods in counseling and guidance, the role of the counselor in various settings, approaches to counseling, and the uses of tests in the counseling situation. Career planning: Study of career development, learning activities, systems, approaches, program coordination, use of educational and community resources, and vocational counseling systems. Occupational information: Study of theories of occupational choice and vocational development and their application to the guidance process. Identification and utilization of various types of occupational information and resources. College or university-sponsored practicum in counseling. At least one course in Test and Measurement or Adult Education. Assist soldiers in determining appropriate educational goals (for example, occupational certificates/ diplomas, terminal, or transferable college degrees). Counsel (face-to-face or electronic medium) all soldiers before Tuition Assistance is initially approved to ensure that soldiers understand their degree plan and their responsibilities regarding Tuition Assistance use. Counsel all soldiers about the requirement to have a documented degree plan after completion of nine semester hours of college credit with one school. Provide information on alternative funding, such as the use of in-service GI Bill benefits and federal financial aid. Approve an active duty soldier’s request to use Tuition Assistance for more than 8 semester hours at one time when the soldier’s prior academic history indicates likelihood for success. Discuss comparative cost effectiveness of similar programs when assisting the soldier in choosing a degree program. Explain the Tuition Assistance reimbursement requirement. Advise active duty soldiers of their responsibility to accomplish all Tuition Assistance-related actions through the GoArmyEd portal. Advise soldiers with regard to enrollment with schools not accredited by a regional accrediting body recognized by the Department of Education. Credits earned may not be accepted in transfer by regionally accredited colleges. Assist personnel in establishing an educational goal based on the individual’s academic background, aptitudes, work experience, and career objectives. Establish education plans to enable sailors to pursue their educational goals by providing information on available education institutions, degrees, and courses. Brief sailors on requirements for using Tuition Assistance benefits. Recommend and/or administer appropriate or required examinations. Assist with enrollment in schools and programs. Provide information on financial aid programs and procedures to include assisting sailors in applying on line to the Free Application for Federal Student Aid, http://www.fafsa.ed.gov. Conduct regularly scheduled Education Service Officers’ education workshops for the purpose of training military education services personnel, career counselors, and command master chiefs. Advise enlisted and officer airmen on academic and career development from the time they enter active duty until the time they retire or separate. Provide counseling prior to authorization of Military Tuition Assistance for first time use on a specific education goal. Additional counseling will be provided to meet specific needs as they arise during an airman’s progress toward an education goal. Counsel and assist students with application for the education benefits programs available under existing Department of Veterans Affairs programs of title 38 of the U.S. Code to include, but not limited to, the following: Chapter 30, Montgomery GI Bill; Chapter 32, Post-Vietnam Era Veterans Educational Assistance Program; and Chapter 33, Post-9/11 GI Bill. Further assistance with GI Bill-related issues should be referred to the Regional Department of Veterans Affairs Office, as appropriate. Provide counseling in both group and individual venues so students can make informed decisions on their eligibility for, and use of, GI Bill benefits. Counseling is appropriate when requested, but may also be provided prior to separation or retirement in, or apart from, Transition Assistance briefings. Emphasize potential benefits available to airmen who are scheduled to be involuntarily separated. Guidance to advisors Officers and enlisted marines appointed as top echelon education officers at installations, division or wing level will provide educational guidance and counseling as follows: Provide counseling at the first permanent duty station, at each new duty station, prior to separation, and at other suitable intervals during their military career. Identify and counsel, individually, those enlisted marines who do not possess a high school credential and those officers who do not possess a baccalaureate degree. Identify and screen all eligible Military Academic Skills Program personnel and provide for enrollment opportunity. Provide assistance to marines applying for Military Academic Skills Program. Maintain official Lifelong Learning program files, records, and data. Prepare a Lifelong Learning program education plan for all Lifelong Learning program participants. Publicize and promote the opportunities available through the Lifelong Learning program, using a variety of appropriate media. Appendix IV: Fifteen Areas Covered by the DOD Contractor Evaluations Evaluation Area The character of the academic partnership of the institution with the service to include but not limited to programs offered, demographics, and facilities (for non-distance learning delivery), student services for all delivery learning programs, learning resource support, library resources, institutional support for military education needs, coordination of academic institution’s satellite offices with home campus, and the working relationship between the service’s education staff and the academic institution personnel. The educational needs assessment by the service. The methods whereby academic institutions are invited on the installation. The methods by which the installation and the academic institution assess institutional effectiveness in meeting the educational needs of the installation and monitoring institutional compliance with the MOU or contract (if overseas). The support provided by various levels of the military, the Undersecretary of Defense for Personnel and Readiness, service headquarters, major commands and installations for voluntary education. The degree of congruence among various missions (the military, installation, and the institutions), the education plan of the installation, the educational programs provided by the institutions which have a MOU with the installation and the distance learning providers. The degree of consistency of the distance learning programs for both military and civilian students and traditional in-the- classroom courses, if offered, and the consistency of satellite campus courses to home campus courses. The responsiveness and flexibility toward service members with regard to programmatic, administrative, and academic processes—such as flexibility in areas of admissions, credit transfer, and academic residency requirements. The means by which students are given the opportunity to evaluate the learning they receive and how the institutions respond to those evaluations. The institution’s disclosure in its marketing and communication regarding its mission, accreditation, courses and programs, services, transfer credit, tuition and fees, recruitment incentives/commissions in its enrollment and recruitment processes. The institutional outcomes, on-campus students compared to military students, regarding retention, graduation, subject course completion, and withdrawal rates of the students. The resources institutions offer to support student learning. All faculty members meet the same academic qualifications and standards in accord with the institution’s accrediting and oversight bodies. The institution has clear, consistent policies, measures, and procedures to evaluate the performance and needs of faculty members. The institution’s academic and administrative student services—standards and processes to ensure timely response to student’s questions and concerns. Appendix V: Evaluation Contract Modifications to Address Skills Required Description of DOD’s Modifications to the Contract for School Evaluations Required that one member on the team evaluating schools have postsecondary or tuition assistance program experience. Working knowledge of measurement methods and tools. Expert in education theories, principles, and practices and in roles of federal and state governments sufficient to plan, evaluate, and advise DOD voluntary education agencies and other relevant stakeholders. 2 to 3 years experience in postsecondary adult education, including one of following: voluntary education for the military, accreditation assessment or evaluation, institutional self-studies, academic counseling or administration on postsecondary level from an accredited institution and academic instruction, with a degree in adult education or instructional/curriculum design, with an emphasis on designing Distance Learning Coursework. One assessor on each assessment team must have two to three years experience in voluntary education with the military. Assessors who have worked on previous evaluations but do not meet the qualification requirements may have requirements waived on a case-by-case basis. The first item was modified on October 19, 2012; the next four items were modified on September 25, 2013. The contract for assessment services was awarded August 31, 2011, at a cost of $645,438. At the time of our report, DOD had extended the contract twice, each time for a year, before it allowed the contract to expire. As of June 2014, DOD had paid a total of $2,069,329 for evaluation services provided by the contractor. In addition to the contact named above, Sherri Doughty (Assistant Director), Sandra Baxter, Kurt Burgeson, and Linda Siegel made key contributions to this report. Also contributing to this report were Susan Bernstein, Deborah Bland, Jessica Botsford, Holly Dye, Mimi Nguyen, Michael Silver, and Craig Winslow.

Summary: DOD's military Tuition Assistance Program includes partnership agreements with about 3,000 schools through which service members can pursue a postsecondary education. Through this program, service members' tuition is paid directly to participating schools and in fiscal year 2013, the program spent $540 million. The program also provides service members with education advisors, and conducts evaluations of schools to assess quality. Congress mandated that GAO provide information on the role of these advisors and on the DOD contractor evaluations of schools participating in the program. GAO examined (1) the number of advisors and the type of advice they provide, and (2) the information collected through evaluations of schools participating in the military Tuition Assistance Program. For this work, GAO analyzed DOD data on the program from fiscal year 2011 through 2013; reviewed all DOD contractor evaluations for fiscal years 2012 and 2013; and interviewed officials from DOD and the military services, contractor staff responsible for the evaluations, and advisors at Joint Base Andrews, Maryland. GAO visited this base because many of its service members participate in the program, and some of the participating schools were evaluated by DOD's contractor in 2013. In fiscal year 2013, 571 Department of Defense (DOD) education advisors were available to provide information and educational support to the nearly 280,000 service members taking courses funded through the military Tuition Assistance Program. This program accommodates service members, who may regularly be reassigned to another location (including overseas), by allowing them to take classes online, directly on base, or at nearby schools. DOD advisors offer a range of services to service members such as helping them understand the types of degrees and courses schools offer and helping them develop educational goals and plans. DOD used a contractor to conduct evaluations of schools participating in the Tuition Assistance Program, however, according to DOD, the evaluations did not provide the agency the information it needed to assess schools. This is because DOD lacked a specific plan to frame the evaluations, which according to federal standards, should clearly define the evaluation questions and methodology and address the collective knowledge, skills, and experience needed by the entity conducting the evaluations. According to DOD's contract, evaluations were to assess school quality, but the 15 areas DOD provided the contractor for evaluation were often not clearly defined and it was not clear what the contractor was to evaluate. For example, one of the areas was the “degree of congruence” among various entities involved in delivering educational services, which DOD provided the contractor without further specificity. Further, because DOD's contract did not specify all the skills needed by the contractor, DOD had to modify its contract to require such skills. However, still lacking information it needs, DOD recently decided not to renew the contract. DOD has suspended the evaluations and is exploring alternative options for evaluating schools, but does not yet have a plan to guide future efforts. Absent a plan, it will be difficult for DOD to have all of the information it needs to effectively evaluate schools. GAO recommends that DOD develop a plan for future school evaluations. DOD agreed with the recommendation.