Higher Education: Use of New Data Could Help Improve Oversight of Distance Education
Published: 2011-11-17T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: As the largest provider of financial aid in higher education, with about $134 billion in Title IV funds provided to students in fiscal year 2010, the Department of Education (Education) has a considerable interest in distance education. Distance education--that is, offering courses by the Internet, video, or other forms outside the classroom--has been a growing force in postsecondary education and there are questions about quality and adequate oversight. GAO was asked to determine (1) the characteristics of distance education today, (2) the characteristics of students participating in distance education, (3) how the quality of distance education is being assessed, and (4) how Education monitors distance education in its stewardship of federal student aid funds. GAO reviewed federal laws and regulations, analyzed Education data and documents, and interviewed Education officials and industry experts. GAO also interviewed officials from accrediting and state agencies, as well as 20 schools--which were selected based on a variety of factors to represent diverse perspectives.
section_title: What GAO Found
paragraphs: While distance education can use a variety of technologies, it has grown most rapidly online with the use of the Internet. Online distance education is currently being offered in various ways to students living on campus, away from a campus, and across state lines. School offerings in online learning range from individual classes to complete degree programs. Courses and degree programs may be a mix of face-to-face and online instruction--"hybrid" or "blended" instruction. Online asynchronous instruction--whereby students participate on their own schedule--is most common because it provides students with more convenience and flexibility, according to school officials. In the 2009-2010 academic year, almost half of postsecondary schools offered distance education opportunities to their students. Public 2- and 4-year schools were most likely to offer distance education, followed closely by private for-profit 4-year schools. Students in distance education enroll mostly in public schools, and they represent a diverse population. While they tend to be older and female, and have family and work obligations, they also include students of all races, current and former members of the military, and those with disabilities. According to the most current Education data (2007-2008), students enrolled in distance education studied a range of subjects, such as business and health. Accrediting agencies and schools assess the academic quality of distance education in several ways, but accreditors reported some oversight challenges. Federal law and regulations do not require accrediting agencies to have separate standards for reviewing distance education. As such, accreditors GAO spoke with have not adopted separate review standards, although they differed in the practices they used to examine schools offering distance education. Officials at two accreditors GAO spoke with cited some challenges with assessing quality, including keeping pace with the number of new online programs. School officials GAO interviewed reported using a range of design principles and student performance assessments to hold distance education to the same standards as face-to-face education. Some schools reported using specialized staff to translate face-to-face courses to the online environment, as well as standards developed by distance education experts to design their distance education courses. Schools also reported collecting outcome data, including data on student learning, to improve their courses. Education has increased its monitoring of distance education but lacks sufficient data to inform its oversight activities. In 2009, Education began selecting 27 schools for distance education monitoring based on an analysis of risk factors, but it did not have data to identify schools with high enrollments in distance education, which may have impeded its ability to accurately identify high-risk schools. Between 2011 and 2013, Education's National Center for Education Statistics (NCES) will start collecting survey data on the extent to which schools offer distance education, as well as enrollment levels. However, the department's Office of Federal Student Aid (FSA), responsible for monitoring Title IV compliance, was not involved in the process of deciding what distance education information would be collected; therefore, it did not provide input on what types of data could be helpful in oversight. Further, FSA officials said they do not yet have a plan on how they will use the new data in monitoring. To improve its oversight and monitoring of federal student aid funds, Education should develop a plan on how it could best use the new distance education data NCES is collecting and provide input to NCES on future data collections. Education agreed with the recommendation.

section_title: Letter
paragraphs: As the largest provider of financial aid for postsecondary education, with  about $134 billion provided to students during fiscal year 2010 under Title  IV of the Higher Education Act of 1965, as amended (HEA), the U.S.  Department of Education (Education) has a considerable interest in  distance education—education that uses technology to provide  instruction to students who are separated from the instructor. Students  participating in distance education are eligible for federal financial aid in  the same way as students taking traditional courses. This aid is available  through Education in the form of grants, loans, and work study wages.  While distance education has extended the opportunity to continue higher  education learning and complete degrees to many students, including  nontraditional students with work and family obligations as well as military  personnel, its rapid growth in popularity presents challenges for ensuring  both its quality and the oversight of federal funding. Congress has  included a number of provisions in the HEA, as amended, such as  oversight requirements for accrediting agencies, in recognition of the  increasing role of distance education in higher education and to address  concerns regarding the academic quality of distance education given the  rapid growth in this area. Education’s Office of the Inspector General  (OIG) has also raised concerns over the past few years about the ability  of schools to verify student identities and ensure enrolled students are  engaged in academic activities given the limited or no face-to-face  contact with distance education students.
To provide a national perspective on the nature of distance education and  its current level of oversight, we were asked to describe (1) the  characteristics of distance education today, (2) the characteristics of  students participating in distance education, (3) how the quality of  distance education is being assessed, and (4) how Education monitors  distance education in its stewardship of federal student aid funds.
To conduct our work, we reviewed and analyzed relevant federal laws  and regulations, literature, studies, and program documents, and  consulted with subject matter experts. We analyzed data from  Education’s Integrated Postsecondary Education Data System (IPEDS)  and the National Postsecondary Student Aid Study (NPSAS) databases  to determine the school and student characteristics involved in distance  education. We also reviewed information from a 2008 report by  Education’s National Center for Education Statistics (NCES) and a 2010  industry report to obtain a national perspective on distance education  practices and offerings at postsecondary schools. We conducted site  visits to Florida, Minnesota, and Puerto Rico to visit schools and interview  state agency officials. We selected these sites based on various factors,  including the level of state data collected and an industry summary of  states’ policies for approving distance education. We interviewed officials  from higher education organizations, accrediting agencies (three regional,  two national, and one specialized), and Education to determine their role  in overseeing the delivery of distance education. Finally, we interviewed  school officials from a nongeneralizable sample of 20 postsecondary  schools to obtain more information on current practices in and the specific  types of programs and coursework being offered through distance  education. Our criteria for school selection included total enrollment,  change in enrollment over time, school sector (public, private nonprofit,  and private for-profit), and geographic diversity, among other factors. For  more information on our scope and methodology, see appendix I.
We conducted this performance audit from November 2010 to November  2011 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: Distance education is not a new concept, but in recent years, it has  assumed markedly new forms and greater prominence. In the past,  distance education generally took the form of correspondence courses— home study courses completed by mail. Distance education today can  take many forms and is defined by federal law and regulation as  education that uses one or more technologies (such as the Internet or  audio conferencing) to deliver instruction to students who are separated  from the instructor and to support regular and substantive interaction  between the students and the instructor. Instruction provided through the  Internet—or online—may be synchronous (simultaneous or “real time”) or  asynchronous, whereby students and the instructor need not be present  and available at the same time (see fig. 1).
In general, for their students to be eligible for federal student aid funds  under Title IV programs, schools must be legally authorized by a state,  accredited by an agency recognized by Education, and be found eligible  and certified by Education. State governments, accrediting agencies, and  Education form the program integrity triad established by Title IV of the  HEA to oversee postsecondary education. The state authorization role is  primarily one of providing consumer protection through the state licensing  process, while the accrediting agencies are intended to function as a  quality assurance mechanism. In certifying a school for participation,  Education is responsible for determining the financial responsibility and  administrative capability of schools and is also responsible for monitoring  to ensure compliance with Title IV requirements.
Accrediting agencies, private educational associations set up to review  the qualifications of member schools, are the primary overseers of  schools’ academic quality. Accreditation is a peer review process that  evaluates a school against the accrediting agency’s established  standards. An institutional accrediting agency assesses a school in its  entirety, including resources, admissions requirements, services offered,  and the quality of its degree programs, while a programmatic accrediting  agency reviews specific programs or single-purpose schools. A school’s  accreditation is re-evaluated every 3 to 10 years, depending on the  accrediting agency. If a school makes a substantive change to its  educational programs or method of delivery from those that were offered  when the agency last evaluated the school, the agency must ensure the  change continues to meet standards. Schools may lose accreditation if  their accrediting agency determines that they no longer meet the  established standards. While Education does not have the authority to  dictate the specifics of an agency’s standards, the department recognizes  accrediting agencies by reviewing and assessing their standards in  various areas required by statute, such as student achievement, curricula,  and student support services.
Education’s Office of Federal Student Aid (FSA) is responsible for  monitoring the over 6,000 postsecondary schools participating in Title IV  programs to ensure their compliance with applicable statutory and  regulatory provisions and to ensure that only eligible students receive  federal student aid. The postsecondary school types include the following:    Public schools—schools operated and funded by state or local  governments, including state universities and community colleges.    Private nonprofit schools—schools owned and operated by nonprofit  organizations whose net earnings do not benefit any shareholder or  individual.    For-profit schools—schools that are privately owned or owned by a  publicly traded company and whose net earnings can benefit a  shareholder or individual.
Education fulfills its school monitoring responsibilities through four main  activities. First, it determines the initial eligibility of schools to participate in  the federal student aid programs, as well as recertifies that eligibility  periodically. Second, as part of ensuring compliance, FSA staff conduct  program reviews of a select number of schools each year where they  examine school records, interview school staff and students, and review  relevant student information, among other things. FSA issues reports on  these reviews, which include information on areas where a school was found  to be in violation of the Title IV requirements. Third, schools are required to  employ independent auditors to conduct annual compliance reviews and  financial audits, which are then submitted to Education. Finally, Education’s  OIG conducts its own audits and investigations of schools to identify and  combat fraud, waste, and abuse and makes recommendations to the  department. Education may assess liabilities and/or impose fines or other  sanctions on schools found in violation of Title IV requirements.
subsections: section_title: Brief History of Statutory Provisions Related to Distance Education
paragraphs: This change was made following Education’s completion of a mandated distance  education demonstration project. The project was undertaken to (1) test the quality and  viability of expanded distance education programs, (2) provide increased student access  to higher education, and (3) determine the specific statutory and regulatory requirements  that should be altered to provide greater access to high-quality distance education  programs. In 2005, Education reported to Congress that waivers of the 50 percent rule did  not lead to increases in fraud and abuse of Title IV funds.
20 U.S.C. § 1009b(c)(2).
Recognized accrediting agencies that do not already have distance education within  their scope of review may add distance education to their scope by notifying Education in  writing. Such agencies must monitor the head count enrollment at each school they  accredit, and if any school experiences an increase of 50 percent or more within 1 year,  the agency must report that information to Education and also submit a report outlining the  circumstances of the increased enrollment and how the agency evaluates the capacity of  the school. Education submits that report to NACIQI for consideration in reviewing the  agency’s change in scope. 20 U.S.C. §§ 1099b(a)(4)(B)(i)(II) and 1099b(q); 34 C.F.R. §§  602.19(e), 602.31(d), and 602.34(c)(1).
section_title: Distance Education Has Become Common in All Sectors and Is Offered through a Range of Programs and Courses
subsections: section_title: Online Distance Education Has Grown Dramatically and Is Offered in a Variety of Ways
paragraphs: While distance education can use various technologies, it has grown most  rapidly online with the use of the Internet to support interaction among  users. With the emergence of the Internet and expansion of Internet- based communication technologies, distance education today is a  common phenomenon and widely used throughout higher education.  Moreover, the term “distance education” no longer connotes only  instruction separated by physical distance, since many distance  education courses—specifically online courses—are offered to students  living on campus as well as away from a campus and across state lines.  School offerings in online learning range from individual classes to  complete degree programs. Individual courses as well as degree  programs may also be a mix of face-to-face and online instruction—often  referred to as “hybrid” or “blended” instruction. Furthermore, an online  class may be synchronous (simultaneous, real-time instruction), or  asynchronous, where students and the instructor are not present and  available at the same time.
According to a 2008 study on distance education conducted by  Education, postsecondary schools of all types offer a variety of distance  education courses. Specifically, for the 2006-2007 school year, 61  percent of 2-year and 4-year schools reported offering online courses, 35  percent reported hybrid/blended courses, and 26 percent reported other  types of distance courses. The study also suggests that the majority of  schools offering distance education used asynchronous Internet  technologies. Specifically, 92 percent of the degree-granting  postsecondary institutions offering distance education in 2006-2007  reported using asynchronous Internet technologies to a moderate or large  extent, compared with 31 percent of schools that reported using  synchronous technologies to a moderate or large extent. In our interviews  at the schools we selected, officials said that online, asynchronous  instruction was also their predominate method for providing distance  education and that this type of instruction meets students’ need for  flexible schedules. For example, over half of the school officials we  interviewed noted that many students taking classes online are working  adults or active duty military service members who would otherwise be  unable to continue or complete their studies.
The use of distance education, particularly online learning, has grown  dramatically in recent years. According to a 2010 industry survey, online  enrollment in degree-granting postsecondary schools has continued to  grow at rates far in excess of the growth for total enrollment in higher  education. Survey results indicate that over 5.6 million students were  taking at least one online course during the fall 2009 term—an increase of  nearly 1 million students over the number reported the previous year and  an increase of 21 percent, as compared with the less than 2 percent  growth in the overall higher education student population. The survey also  suggests that nearly 30 percent of higher education students were taking  at least one course online.
Such remarkable growth may be attributed to institutional efforts to  expand access to more students, alleviate constraints on campus  capacity, and the desire to capitalize on emerging market opportunities  and compete with other schools. According to Education’s 2008 study on  distance education, which includes online and other forms of distance  education, the top four factors affecting postsecondary schools’ decisions  regarding distance education offerings are (1) meeting student demand  for flexible schedules; (2) providing access to college for students who  otherwise would not have access due to geographic, family, or work- related reasons; (3) making more courses available; and (4) seeking to  increase student enrollment. Several of these factors, such as providing  access to more students, were also cited by school officials we  interviewed. For example, one school we visited had increased access to  education by establishing over 20 “cyber-centers,” including one on a  National Guard base and another in a shopping mall where students can  access computers with Internet capabilities and participate in online  courses as well as complete assignments and take exams. Additionally,  officials at two of the schools we interviewed noted that on-campus  students were registering for online classes, instead of face-to-face  classes that were otherwise full or scheduled for times of day that  conflicted with their personal schedules. Furthermore, one school we  interviewed provided flexibility to its students by allowing them to begin  and complete courses at their own pace.
While cost savings might be a factor, none of the school officials we  spoke with cited cost savings as the primary reason for providing online  distance education courses and programs. Moreover, they said students  taking distance education courses, including online courses, are generally  charged the same tuition and fees as students taking face-to-face  courses. These officials cited various costs associated with developing  and expanding online distance education offerings, such as the purchase  of hardware and software (which includes a learning management  system), course development, faculty training and salaries, and the  provision of student support services. They also said online instruction is  not necessarily less expensive to provide, in part, because schools have  to provide similar support services to both online students and classroom  students—such as tutoring, library access, and (virtual) faculty office  hours. For example, officials at three schools mentioned one of the major  expenses associated with online distance education is providing off-hours  library access or tutoring. Also, almost all the officials said it is often  difficult to isolate the costs of online courses from the costs of providing  traditional courses. Professors generally teach both online and face-to- face course sections, and the infrastructure developed for online distance  education, such as the online learning management systems, can also be  used by students and instructors participating in face-to-face instruction.
section_title: A Wide Variety of Schools Provide a Range of Distance Education Courses and Programs
paragraphs: Schools of all types reported offering distance education, according to  data collected by Education through its annual IPEDS survey.  Specifically, during the 2009-2010 school year, 46 percent of all Title IV  eligible schools reported that they offered distance education  opportunities to their students. Figure 2 shows the variation among these  schools by sector and program length.
As shown in figure 2, public schools, both 2- and 4-year, were more likely  to offer distance education opportunities than private nonprofit or for-profit  schools. Among public schools, distance education was more likely to be  offered at 2-year schools rather than 4-year schools. One school official  we spoke with attributed this likelihood to the increased number of  students at 2-year schools, given the weak economy and limited capacity  at 4-year public schools.
With regard to minority-serving institutions and institutions with specific  high minority concentrations, IPEDS data indicate that these institutions  are as likely or more likely to offer some distance education than all  schools combined, with the exception of Hispanic-serving institutions. For  the 2009-2010 school year, more than 60 percent of Historically Black  Colleges and Universities and Tribal Colleges and Universities offered  distance education opportunities to their students, compared with about  46 percent of institutions overall. Furthermore, 49 percent of Asian/Pacific  Islander/Native Hawaiian-serving institutions offered distance education  to their students. Among Hispanic-serving institutions, just over 30  percent of these schools were offering distance education (see fig. 3).
With regard to the size of schools that offer some distance education, the  IPEDS data suggest that larger schools—as defined by enrollment—are  more likely to offer distance education opportunities than smaller schools.  Specifically, 23 percent of schools with fewer than 1,000 students offered  distance education, while 96 percent of larger schools—those with 20,000  or more students—did so (see fig. 4).
The 2008 distance education study by Education provided additional  insights on the extent and nature of distance education offerings by  school type, sector, and size. In terms of full degree and certificate  programs, the study indicated that in the 2006-2007 academic year, about  a third of all degree-granting schools offered entire degree programs or  certificate programs through distance education. Additionally, public  schools were more likely to offer a degree or certificate program entirely  through distance education than were private schools. Larger schools  were also more likely to offer a degree or certificate program entirely  through distance education than smaller schools (see table 1).
section_title: Students in Distance Education Enroll Mostly in Public Schools and Represent a Diverse Population
subsections: section_title: Most Distance Education Students Attend Public Schools and Study a Range of Subjects
paragraphs: Our analysis of the NPSAS data for the 2007-2008 academic year  showed that of the estimated 5 million postsecondary students who  have taken distance education, participation was most common among  students attending public schools. These students enrolled in a range of  academic fields of study.
Most distance education students enroll at public schools. As might be  expected, most undergraduate and graduate students taking distance  education courses or programs were enrolled at public schools, followed  by private nonprofit and private for-profit schools (see fig. 5).
Distance education students enroll in a variety of fields of study. Both  undergraduate and graduate students taking distance education courses  or programs had higher rates of enrollment in the fields of business and  health. Undergraduates taking distance education courses and programs  also often majored in the humanities (liberal arts), while graduate  students often studied education.
section_title: While Students in Distance Education Tend to Be Older and Female, and Have Family and Work Obligations, They Are Also a Diverse Population
paragraphs: According to our analysis of 2007-2008 NPSAS data, distance education  students varied somewhat from students who did not enroll in distance  education in that they tended to be somewhat older and female, and have  family and work obligations. Moreover, students who are participating in  distance education represent a diverse population that includes students  of all races, current and former members of the military, and students with  disabilities. Some of these characteristics are consistent with what we  reported in our 2002 testimony on distance education and also were  corroborated in our recent interviews with selected schools for this report.
Distance education students tend to be older. As figure 6 shows,  undergraduate and graduate students who took distance education  courses or programs were about 3 years older, on average, when  compared with students who did not take any distance education courses.
Distance education students are more often female. Women represented  about 61 percent of undergraduate students who took distance education  courses or programs, compared with about 56 percent of undergraduates  who took no distance education, and about 57 percent of undergraduates  overall. For graduate students, the percentage of students taking distance  education courses or programs who were female was about 65 percent,  which was higher than those who took no distance education (59 percent)  and the overall percentage of graduate students who were female (61  percent).
Distance education students more often have family obligations. Figure 7  shows that undergraduate and graduate students who took distance  education courses or programs were more often married and had  dependents than those taking no distance education courses.
Distance education students more often work full time. A higher  percentage of students who took distance education courses or programs  worked full time when compared with students who did not take any  distance education courses. This difference was greatest among  graduate students—about 74 percent of the students who took distance  education courses or programs worked full time compared with 57  percent of students who did not take any distance education courses.
For undergraduates, the figures were 45 percent and 31 percent,  respectively.
Students of all races and ethnicities participate in distance education to  some extent. Postsecondary students of various races and ethnicities  participated in distance education (see fig. 8).
Current and former members of the military enrolled in postsecondary  education participate in distance education. Forty-five percent of active  duty service members, 29 percent of reservists, and 30 percent of  veterans enrolled in postsecondary education took distance education  courses or programs. In addition, of those enrolled in postsecondary  education, 42 percent of active duty service members with a disability and  29 percent of veterans with a disability took distance education courses or  programs. Taken together, active duty service members, reservists, and  veterans represented about 7 percent of all students taking distance  education courses and programs, compared with 4 percent of students  who took no distance education.
Students with disabilities participate in distance education. Twenty-one  percent of all students with disabilities, including members of the military  and civilians, enrolled in distance education courses or programs.  Further, 25 percent of students with disabilities affecting their mobility  took distance education courses or programs. Students with disabilities  represented 10 percent of all students taking distance education courses  and programs, while students with mobility disabilities represented about  3 percent.
Many of these student characteristics were also noted by school officials  we interviewed. These school officials reported that they collect data such  as age, gender, and race and ethnicity of their students. The  demographic data provided from schools generally showed similar  student characteristics as that suggested by the 2007-2008 NPSAS  data—that distance education students tend to be older and female, and  have work and family obligations. Officials of at least three of the schools  we selected indicated that many of their students taking classes online  are veterans or students serving in the military. While at least three  schools reported tracking students who identified themselves as having  disabilities, at the time of our interviews, none of these schools indicated  that they had determined how many of these students were taking online  distance education classes. Officials at one of these schools, however,  conducted some analysis after our interview and reported that about 3  percent of their students enrolled in the past year had documented  disabilities. These students took, on average, 15 percent of their classes  online.
While most of the schools where we conducted interviews collected  demographic data on their students, including those taking courses  online, less than half of these schools have compared the demographics  of students taking completely online courses with those taking face-to- face courses. Officials at five schools mentioned that comparing data on  students can be difficult, in part, because students can take courses or  degrees through a mix of instructional modalities—including completely  online, hybrid/blended (mix of online and face-to-face), and completely  face-to-face. For example, officials from one private nonprofit 4-year  school that offers completely online as well as blended courses and  degrees said that it is difficult to collect comparison data because the  school’s administrative records do not differentiate online students from  those who enroll in both online and campus-based courses.
section_title: Accreditors and Schools Assess the Academic Quality of Distance Education in Several Ways, but Accreditors Reported Some Oversight Challenges
subsections: section_title: Accrediting Agencies Examine the Quality of Distance Education in Various Ways but Reported Some Challenges
paragraphs: Accreditors we interviewed have various procedures to examine schools’  distance education programs, but some accreditors reported they face  challenges. Federal law and regulations require accrediting agencies to  have standards that address student achievement, curricula, faculty, and  student support services, among other areas. In addition, accreditors  must ensure that schools have a process in place to verify registered  students are doing their own work by using methods such as secure  logins, passwords, proctored examinations, or other technologies.  However, accrediting agencies are not required to have separate  standards for distance education. As such, accreditors we spoke with  who accredit both distance education and face-to-face programs use the  same standards for both, although they differed in the practices they used  to examine schools offering distance education.
The accreditors we spoke with conduct reviews of schools’ distance  education programs according to the accreditors’ own standards. For  example, to address the effectiveness of a program, accreditors may  review such measures as student retention rates, completion/graduation  rates, student satisfaction, placement rates (if applicable), and various  measures of student learning. The three regional accreditors we spoke  with give schools the responsibility for determining the best way to assess  student learning for both face-to-face and distance education programs.  However, both national accrediting agencies and the specialized  accreditor we spoke with have specific quantitative thresholds as  minimum standards on various outcomes. For example, one national  accreditor requires that their member schools meet specific thresholds for  student retention and placement rates. Officials at this agency said they  could sanction schools whose programs fall below these standards. The  other national accreditor we spoke with also requires its schools to meet  thresholds established for outcomes such as course completion rates,  program graduation rates, student satisfaction rates, and student learning  (as measured by professional licensing exams such as those for physical  therapists and lawyers). One regional accreditor said it was exploring  including standardized learning outcomes in its accreditation standards.
As part of their periodic site visits to schools to assess the quality of  academic programs, accreditors have to adapt their approach when  reviewing schools with distance education. Accreditors are required to  employ staff who are well-trained and knowledgeable about distance  education, for example, when performing on-site reviews of schools  providing distance education. Officials at all six accrediting agencies we  spoke with said they include such experts on their on-site review teams.  At one regional accreditor we interviewed, distance education experts are  tasked with specifically reviewing the quality of a school’s distance  education learning infrastructure, as well as the educational effectiveness  of its programs, and receive specific training to do so. To review schools’  student supports, faculty supports, and educational effectiveness, officials  at another regional accreditor told us their distance education experts  may use video teleconferences or e-mails to communicate with  administrative staff, faculty, and students not located on campus. These  experts also remotely observe interactions between students and faculty  in online classes.
In addition to the periodic on-site accreditation reviews to reassess a  school’s accreditation status that are required by statute, accreditors are  to be notified if schools make substantive changes to academic programs  or their schools. The main purpose of this substantive change policy is  to ensure that when schools make changes, they are maintaining the  same level of quality they had when last reviewed. While there are a  number of circumstances that can trigger the substantive change  requirement, the one most applicable to distance education is the addition  of courses or programs that represent a significant departure from the  existing offerings of educational programs, including method of delivery,  from those that were offered when the accreditor last evaluated the  school. A shift to distance education courses that constitute more than 50  percent of a program’s offerings was the substantive change threshold  used by four of the six accrediting agencies we interviewed. Officials at  one regional accrediting agency reported that, in calendar year 2010, the  agency turned down 34 percent of initial substantive change requests for  new distance education programs because of weak student learning  assessments or inadequately trained faculty, among other reasons.  However, they said this figure has since come down to about 16 percent  because schools have had more training on how to develop a substantive  change proposal.
To ensure academic integrity, the six accrediting agencies we interviewed  require schools to provide evidence that they verify registered students  are doing their own work. For example, officials at one regional accreditor  we spoke with said they require schools to use a student identification  number and password as the minimum for verifying student identity. This  accreditor said most institutions also verify student identity through  student interaction during the course. In addition, one national accreditor  we spoke with said some schools design tests that require a login and  password, and may also feature pop-up questions during tests, prompting  students to enter verification information such as their address or  mother’s maiden name.
While the accreditors we interviewed have a range of activities to assess  the quality of distance education, a few accrediting agency officials and  industry experts we spoke with also expressed some concerns and  reported challenges involved in assessing the quality of distance  education. These challenges were mostly related to accreditors’ capacity  to keep pace with substantive changes and conduct follow-up quality  reviews with schools. Officials at one regional and one national  accrediting agency said they have had some difficulty keeping up with the  high number of substantive change applications for new online programs.  Officials representing the national accreditor said these applications have  increased by about 30 percent and that the officials have had to double  the number of evaluators on staff over the last 5 years. According to  officials with the regional accreditor, they have increased the number of  follow-up reviews to ensure that schools address concerns about meeting  quality standards identified during the initial site visits. These officials  reported that they withdrew one school’s accreditation for failure to  demonstrate that its distance education programs met the same  standards as its face-to-face programs, with respect to curriculum,  resources, support, and student learning outcomes. Industry experts also  acknowledged that some accreditors have limited resources and have  had problems training their peer reviewers in distance education.
section_title: Schools Use a Range of Course Design Principles and Student Performance Assessments to Hold Distance Education to the Same Quality Standards as Traditional Courses
paragraphs: To assure that their distance education programs are accredited by  federally recognized accreditors and that their students qualify for Title IV  funding, officials we interviewed at 20 selected schools reported that they  generally apply certain course design principles and use student  performance assessments to assess the quality of the courses that make  up these programs. The accreditors we spoke with require schools to  have standards that address the quality of degree programs with respect  to such things as student achievement, which could include such  measures as course completion, licensing exams, and job placement  rates, as well as student support services. A majority of school officials  reported that they assess their distance education courses by the same  standards they use for their traditional courses.
Officials at most of the schools we spoke with said they used instructional  teams to design their distance education courses according to the  schools’ standards. These teams varied in their composition and  activities. Some teams include specialized staff who work with faculty to  translate traditional face-to-face courses to the online environment. For  example, one school we visited in Florida has a 20-member instructional  design team that includes instructional designers, graphic artists,  multimedia technicians, and quality control coordinators. Officials at this  school said the design team considers which instructional methods are  most appropriate for the material delivered in each online course. For  example, a psychology course may use mostly text-based storytelling,  while an anthropology class may rely more heavily on video clips.  Officials at an online school we spoke with stressed the need to replace  face-to-face course instructors’ body language and tone of voice cues  with appropriate text and video media. Besides assisting professors with  designing online courses, school officials said instructional design teams  also train professors in the pedagogical differences of teaching online and  on the online technology used by the school.
Officials at over half of the 20 schools we interviewed also reported that,  to ensure quality in the design of their courses, they had used standards  and best practices, some of which were developed by distance education  industry experts. For example, 5 schools subscribe to Quality Matters, a  nonprofit organization that lays out principles for designing quality online  and blended courses. This organization sets specific standards for  learning objectives, technology, faculty-student interaction, student  supports, and assessment that online courses must meet in order to  receive Quality Matters certification.
In addition, school officials reported that their schools collect outcome  data to help them assess the quality of courses. The types of learning  outcomes that the schools reported tracking include end-of-course  grades, course completion rates, and results of national professional  licensing assessments. Officials at most schools we spoke with said they  also used outcome data to make improvements to their courses. Officials  at two schools told us they employ staff to analyze these data and make  recommendations for course updates. For example, officials at one fully  online school we spoke with noticed their students were performing below  the national average on a section of a third-party end-of-course criminal  justice test. The officials used the results of this test to strengthen the  related material. According to these officials, their criminal justice  students’ performance improved on that section of the exam subsequent  to their course improvements. In addition to using outcome data to  improve their courses, one school we spoke with in Florida had collected  these types of data on their online and hybrid courses over a period of 15  years to determine which factors most influenced student success.
To meet accreditors’ requirement to verify the identities of students  enrolled in their distance education courses or programs, officials at most  of the schools reported using various methods. For example, most of the  school officials we interviewed said they issue students a secure login  and password and some also use other methods, such as proctored  exams. Officials at one school said they are also starting to use  audiovisual software that works as a web cam to verify the student taking  an exam is the one enrolled in the course and to ensure the student is not  receiving assistance. In addition to technological safeguards, officials  from one school said the interaction between students and faculty is key  to ensuring students are doing their own work. They said instructors  become familiar with a student’s writing or communication style through  online discussions or the completion of assignments, and the instructor  recognizes if that style changes. Officials at one school said they cannot  be completely sure that distance education students are doing all of their  own work even when using these methods; officials also noted that  similar challenges exist for face-to-face courses. A few schools  mentioned taking further steps to combat potential fraud in their online  programs. Specifically, officials at two of the completely online schools we  interviewed said they conduct reviews of or request further documentation  from students who register with the same e-mail addresses or telephone  numbers. Officials at one school we spoke with said they would like  more guidance, either from Education or their institutional accrediting  agency, on examples of verification and authentication systems for  student identity to improve the school’s monitoring of the verification  process.
section_title: Education Has Increased Its Monitoring of Distance Education but Lacks Sufficient Data to Inform Its Oversight
paragraphs: Education’s Office of Federal Student Aid (FSA) has recently increased  its monitoring of distance education by updating its program review  procedures and undertaking a risk analysis project. These efforts are in  response to the expansion of distance education and the Education OIG’s  identification of distance education as a high-risk area for managing  student aid dollars. To better monitor distance education, FSA updated  and issued new program review procedures. The previous set of FSA’s  procedures, issued in 2008, did not provide in-depth guidance for  assessing whether a school was approved to offer distance education or  if there was regular and substantive interaction between instructors and  students. The new procedures on distance education provide staff with  expanded guidance for assessing a school’s compliance with these  requirements. FSA officials said staff have been trained on the new  procedures and, as of June 2011, have been using them for program  reviews. All program reviews will include at least routine testing to  determine basic program eligibility for schools that offer distance  education, according to Education officials. Schools that offer more than  half of any of their programs through distance education will also be  required to undergo expanded testing for regular and substantive  interaction.
Compliance with federal student aid requirements by schools offering  distance education programs is difficult to assess because many of the  violations Education identifies through its program reviews are not  specific to distance education; for those that are, Education does not  necessarily identify or code the violations as such in its database,  according to an Education official. For example, violations such as a  school not appropriately returning Title IV funds when a student  withdraws are coded in Education’s database based on the type of  violation rather than whether this violation occurred in traditional or  distance education. Violations specific to distance education that are  tracked by Education are related to a lack of regular and substantive  interaction between instructor and students and certain accreditation  issues, such as an accrediting agency being ineligible because it does not  have distance education in its scope. Education reported that from  October 2005 through May 2011, no program reviews or audits identified  any lack of regular and substantive interaction or distance education  accreditation violations.
FSA used several indicators to identify a school’s risk, including a change in school  sector (e.g., from proprietary to private nonprofit or from private nonprofit to proprietary),  an audit or investigation by the OIG, and the distribution of a high percentage of full  student loans, as this may be an indicator that a school is not appropriately monitoring  student withdrawals for return of student aid funds.
FSA officials said they conducted 25 reviews and the OIG is conducting 2 audits. conducting this project in conjunction with others in the department. FSA  officials were not able to estimate a date when all final project reports will  be issued, but said their last program review was conducted in early  August 2011. They said the results of the project, including its methods  for identifying high-risk schools and the procedures used, will be  evaluated to determine if any changes need to be made to FSA’s annual  program reviews.
While the objective of the project was to review high-risk distance  education schools, Education lacked data to adequately identify schools’  level of risk based on the extent to which they offered distance education  and the amount of federal student aid they received for those programs or  courses. For example, to identify high-risk schools that may be offering  distance education courses and programs, one indicator Education relied  on was the Department of Defense’s enrollment information on its military  members. Because distance education provides the flexibility needed to  fit active duty service members’ duty schedules and location, many  military members are enrolled in distance education courses and  programs. Therefore, in its risk analysis, Education included schools  that had 200 or more military members receiving tuition assistance from  the Department of Defense. While Education’s IPEDS database can show  which schools offer distance education, it lacks information on the extent  of a school’s offerings and enrollment levels. Despite using data from  multiple sources, one of the 27 schools Education originally selected for  review through the risk analysis did not actually offer distance education.  As a result, FSA officials said they had to substitute another school for the  study. While the project is not yet complete, officials reported confidence  that their study is currently based on an appropriate selection of  schools. Nevertheless, they acknowledged that, in selecting their target  schools, they lacked sufficient data to help them identify the extent to  which a school was offering distance education as well as the amount of  federal dollars being spent for distance education at each school, both of  which would have been significant in evaluating a school’s risk.
The Office of Federal Student Aid has plans to collect more information  on distance education, but complete information on all schools may not  be available for several years. Under its new Integrated Partner  Management (IPM) system, which will consolidate data systems on  schools receiving Title IV funds, FSA will collect information about how a  school’s programs are offered. Specifically, FSA officials said when  schools apply for Title IV initial certification or recertification, they will be  asked to indicate whether a program is predominantly (more than 50  percent) delivered via the classroom, distance education,  correspondence, or independent study. They said the IPM system is  expected to be implemented in November 2012 and would eventually  allow them to analyze comprehensive data about a school. For example,  they will be able to match the extent to which schools offer distance  education with Title IV violations identified during program reviews.  However, because schools are generally required to recertify only every 6  years, officials acknowledged that it could be several years before the  IPM system will contain information on all schools’ distance education  offerings. Therefore, distance education information on all schools may  not be available through IPM until 2018.
In the meantime, Education’s NCES is expanding its IPEDS survey to  provide a more in-depth picture of distance education offerings and  enrollment patterns. The plan by NCES to expand the IPEDS survey  with regard to distance education was the result of a decision by its  technical review panel to better describe postsecondary education offered  throughout the nation, allow schools to compare their distance education  activities with those of their peer schools, and provide valuable  information to parents and students on available college programs. This  expanded data collection will be conducted in phases. The 2011-2012  survey used the definition of distance education as established in 2008  and collected information about whether schools offer their programs  completely through distance education. Additional new distance  education questions will be added to the 2012-2013 survey. The new  survey questions ask for information such as the range of a school’s  offerings in distance education, the number of students enrolled either  partially or entirely in distance education, and whether the students are  located in or out of state in relation to the school (see fig. 9). An NCES  official said the new IPEDS data are expected to be available 1 year after  the survey closes but may be available earlier. For example, early release  data collected during the 2011-2012 survey may be available as early as  February 2012 and available publicly by November 2012.
Despite the prospect of more comprehensive data on schools and their  distance education offerings being collected through IPEDS, FSA does  not yet have specific plans to use these data for monitoring school  compliance with federal student aid requirements. According to FSA  officials, they intend to wait and see what information the survey yields  before deciding how to make use of it. Moreover, FSA indicated it was  not aware of NCES’s efforts to expand the IPEDS distance education  data collection and, therefore, was not involved in the planning and did  not provide input during comment periods. According to NCES officials,  the NCES technical review panel process engages a number of  stakeholders and is open to federal officials who are interested in  participating.
section_title: Conclusions
paragraphs: Distance education, specifically online education, has been developing for  a number of years and has become a part of the mainstream of higher  education. This delivery mode of instruction has provided some new  opportunities and access, particularly for nontraditional students and  working adults who are looking to advance their careers. Moreover, it is  likely to continue growing, as schools across all sectors and levels see it  as a critical educational tool in meeting student needs and demand.
The growth in distance education and the sizable federal investment in  higher education will challenge all segments of the triad responsible for  the oversight of higher education—the states, accreditation agencies, and  the federal government—in their capacity to provide consumer protection,  ensure academic quality, and protect the federal investment. In response  to this challenge, Education has taken steps to increase its oversight by  providing its staff with expanded guidance for assessing a school’s  compliance with distance education requirements and participating in the  OIG/FSA risk project, which identified potential risk indicators. However, a  key factor in Education’s ability to properly focus oversight on the areas of  greatest risk will be the availability and use of pertinent, up-to-date data  on both the extent to which schools offer distance education and the  extent to which students use federal aid to attend those programs.
While FSA’s IPM system may eventually be helpful in providing Education  with the opportunity to monitor distance education with better information,  the expanded IPEDS data would provide relevant information much  sooner. However, without a plan on how to use the new IPEDS data to  identify and monitor high-risk schools, FSA may lose the opportunity to  strengthen its oversight of distance education in the near term. Moreover,  if FSA does not coordinate with NCES going forward, it stands to lose the  opportunity to provide input on any additionally needed data that may  strengthen oversight and ensure accountability in the long term.
section_title: Recommendation for Executive Action
paragraphs: To help Education strengthen its oversight of distance education, the  Secretary of Education should direct FSA to develop a plan on how best  to use the new IPEDS distance education data and provide input to  NCES on future IPEDS survey work with regard to distance education.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to officials at Education for their review  and comment. Education provided comments, which are reproduced in  appendix III of this report, and technical comments, which we  incorporated as appropriate. In its comments, Education agreed with our  recommendation and noted that FSA will update its School Participation  Team procedures to include consideration of IPEDS data on distance  education for monitoring schools. Education also stated that FSA will  provide input to NCES on the design and results of any future IPEDS  surveys that include distance education.
We are sending copies of this report to relevant congressional  committees, the Secretary of Education, and other interested parties. In  addition, this report will also be available at no charge on GAO’s website  at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-7215 or scottg@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix IV.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: This appendix discusses in detail our methodology for addressing the  following research objectives: (1) the characteristics of distance education  today, (2) the characteristics of students participating in distance  education, (3) how the quality of distance education is being assessed,  and (4) how Education monitors distance education in its stewardship of  federal student aid funds.
To address these research questions, we reviewed relevant federal laws  and regulations, literature, studies, and reports; interviewed officials from  Education, representatives from all types of postsecondary schools,  accreditation agencies, and distance education and industry experts; and  conducted site visits to Florida, Minnesota, and Puerto Rico to interview  state agency and school officials. We selected these sites based on  various factors, including the level of state data collected and an industry  summary of states’ policies for approving distance education. We also  analyzed data from Education’s Integrated Postsecondary Education  Data System (IPEDS) and the National Postsecondary Student Aid Study  (NPSAS) databases to determine the school and student characteristics  involved in distance education. We determined that IPEDS and NPSAS  data were sufficiently reliable for the purposes of this report based on  prior testing of the data from these systems in 2011. The data were  tested for accuracy and completeness, documentation about the data and  systems used to produce the data was reviewed, and agency officials  were interviewed.
To determine the current characteristics of distance education, we  analyzed 2009-2010 data from Education’s IPEDS and also from a 2008  report by Education’s National Center for Education Statistics (NCES) to  obtain a national perspective on distance education practices and  offerings at postsecondary schools. Specifically, we analyzed IPEDS data  to provide information on the size, number, sector, and program length of  schools offering distance education courses and programs. We used the  2008 distance education report to describe how schools are providing  distance education to students, including the type of technology (Internet,  video, audio, etc.) and instructional methods (asynchronous and  synchronous) used, and the various types of degrees, certificates, and  courses offered, including the percentage of courses offered online. In  addition, we analyzed the 2010 Sloan Consortium report on online  education to show updated enrollment figures specific to online courses.  We supplemented the nationally representative data with information  obtained from our interviews with industry experts and representatives at  a nongeneralizable sample of postsecondary schools regarding the range  of delivery and instructional techniques being used, and the type of  programs and coursework offered through distance education.
To select our sample of postsecondary schools, we used enrollment data  from Education’s 2009-2010 IPEDS to identify schools that were offering  distance education and had significant increases in total enrollment,  which may be due, in part, to increased enrollment in distance education  classes or programs. Based on the schools’ percentage change in  enrollment, we then selected schools by size—as defined by enrollment— as well as by sector and program length. We also considered the  following factors in selecting our sample of schools:    geographic dispersion by state,    minority serving school status (e.g., Historically Black Colleges and  Universities and Hispanic-serving institutions),  selectivity in accepting students,  industry expert or stakeholder recommendations,    extent to which distance education programs and courses are offered  (totally online schools versus schools offering both campus-based and  online instruction), and    whether the schools are regionally or nationally accredited.
Based on these considerations, we selected 20 schools representing all  sectors and program lengths, for site visits or phone interviews (see app.  II for a list of colleges and universities we interviewed). Our selected  schools break out as follows:    4 public 2-year schools.    5 public 4-year schools.    6 private nonprofit schools.    5 private for-profit schools.
After our interviews with officials from the selected schools, we conducted  a content analysis on the information gathered. Interview responses and  comments from officials were categorized to identify common themes.  The themes were reviewed by a methodologist before all comments were  categorized. One analyst coded the information and a second analyst  assessed the accuracy of the coding. Disagreements between coders  were resolved through discussion. We used the information gathered  from these schools for illustrative purposes only. Because the schools  were not selected to be representative of all postsecondary schools, the  interview results are not generalizable to other postsecondary schools,  including groups of schools in the same sector or program length.
To determine the characteristics of students participating in distance  education courses and programs, as well as those who do not participate,  we analyzed Education’s 2007-2008 NPSAS data, the most current  available data. These data allowed us to compare distance education  students to nondistance education students on the following  characteristics: age, gender, marital status, dependent status, and  employment status. The data also allowed us to describe the  characteristics of students enrolled in distance education, in terms of type  of school attended, field of study, race, veteran status, and disability  status. We supplemented this analysis with information from our  interviews with selected postsecondary schools and student demographic  data provided by school officials.
To determine how the quality of distance education programs is being  assessed, we obtained information from accrediting agency and school  officials and reviewed and analyzed federal laws and regulations related  to accreditation. We interviewed officials from six accrediting agencies  (three regional, two national, and one specialized) and reviewed their  standards and policies to determine how they are assessing the quality of  distance education courses and programs. In addition, we reviewed  documents from the Council for Higher Education Accreditation (CHEA)  website, to gain a broader understanding of accreditation. We also  interviewed officials from schools in our sample to describe the specific  quality assurance frameworks and the outcomes they use to assess the  performance of students engaged in distance education. In addition, we  interviewed an official from Quality Matters and reviewed quality  standards documents provided at the interview.
To determine the extent to which Education is monitoring distance  education programs to ensure the protection of federal student aid funds,  we reviewed relevant federal laws and regulations regarding distance  education oversight requirements. We interviewed officials from  Education’s Federal Student Aid office and the Office of Postsecondary  Education to determine their roles in the monitoring and governance of  Title IV programs, specifically with respect to distance education. In  addition, we interviewed officials from NCES to learn about their IPEDS  data collection efforts and Education’s Office of the Inspector General to  learn about their distance education monitoring activities and findings.  Finally, we reviewed agency documents, including plans to add distance  education variables to the IPEDS survey, OIG testimonies and reports,  and an interim status memorandum issued by the OIG/FSA Risk Project.
We conducted this performance audit from November 2010 to November  2011 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
section_title: Appendix II: List of Colleges and Universities GAO Interviewed
section_title: Appendix III: Comments from the Department of Education
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Tranchau Nguyen, Assistant  Director; Susan Chin, Analyst-in-Charge; Amy Anderson; Jeffrey G.  Miller; and Jodi Munson Rodríguez made significant contributions to this  report in all aspects of the work. Susan Bernstein contributed to writing  this report. Michael Silver, Christine San, and John Mingus provided  technical support, and Jessica Botsford provided legal support. Mimi  Nguyen assisted with report graphics.
section_title: Related GAO Products
paragraphs: Distance Education: Growth in Distance Education Programs and  Implications for Federal Education Policy, GAO-02-1125T. Washington,  D.C.: Sept. 26, 2002.
Distance Education: Improved Data on Program Costs and Guidelines on  Quality Assessments Needed to Inform Federal Policy, GAO-04-279.  Washington, D.C.: Feb. 26, 2004.
Higher Education: Institutions’ Reported Data Collection Burden is Higher  Than Estimated but Can Be Reduced through Increased Coordination,  GAO-10-871. Washington, D.C.: Aug. 13, 2010.
DOD Education Benefits: Increased Oversight of Tuition Assistance  Program Is Needed, GAO-11-300. Washington, D.C.: March 1, 2011.
DOD Education Benefits: Further Actions Needed to Improve Oversight of  Tuition Assistance Program, GAO-11-389T. Washington, D.C.: March 2,  2011.
VA Education Benefits: Actions Taken, but Outreach and Oversight Could  Be Improved, GAO-11-256. Washington, D.C.: Feb. 28, 2011.