Teacher Preparation Programs: Education Should Ensure States Identify Low-Performing Programs and Improve Information-Sharing
Published: 2015-07-23T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: TPPs play a vital role in preparing teachers, including helping them teach to new K-12 college- and career-ready standards recently adopted or under development in all states. Under Title II of the Higher Education Act, states collect information on TPPs and report it to Education, which reports it to the public. Education also administers competitive grant programs related to teacher preparation. In light of new K-12 standards and questions about TPP quality, GAO was asked to review TPP, state, and federal efforts.
This report examines: (1) state oversight activities, (2) state and TPP actions related to new K-12 standards, and (3) the extent to which Education shares information about TPP quality. GAO reviewed relevant federal laws and documents, surveyed all state oversight offices (with a 100 percent response rate), and interviewed Education officials and various stakeholders, as well as a non-generalizable sample of officials in five states with varied approaches to oversight and 14 TPPs in those states.
section_title: What GAO Found
paragraphs: State oversight officials reported that they approve teacher preparation programs (TPP) by assessing the quality of program design and analyzing candidate data such as program graduation rates, according to GAO’s 2014-2015 survey of states and the District of Columbia. However, some states reported that they do not assess whether TPPs are low-performing, as required by federal law. To receive funding under the Higher Education Act, states are required to conduct an assessment to identify TPPs that are low-performing. Seven states reported to GAO that they do not have a process to do so. State officials who reported not having a process in GAO’s survey cited several reasons including that they believed other oversight procedures were sufficient to ensure quality. Education officials told GAO they have not verified states’ processes to identify low-performing TPPs. In accordance with federal internal control standards, Education should provide reasonable assurance of compliance with applicable laws. If states fail to assess whether TPPs are low-performing, potential teaching candidates may have difficulty identifying low-performing TPPs. This could result in teachers who are not fully prepared to educate children.
Officials in most surveyed states and all 14 of the TPPs GAO interviewed reported making changes to prepare teaching candidates for new state K-12 standards. Thirty-seven states reported providing TPPs with guidance about the new standards and a similar number of states reported adjusting their process for approving TPPs. Most states also required prospective teachers to pass licensing tests that have been modified in response to the new standards. Officials from all of the 14 TPPs GAO interviewed reported making changes that generally fell within the following three categories: (1) increasing subject-matter knowledge of teachers, (2) modifying coursework related to teaching techniques, and (3) using classroom training to provide real world experience.
Education missed opportunities to share information about TPP quality internally and with state oversight entities. Federal internal controls standards highlight the value of effective information-sharing with internal and external stakeholders. However, Education does not have mechanisms in place to promote regular, sustained information-sharing among its various program offices that support TPP quality because the workgroup that used to facilitate such information-sharing was discontinued. Without such a mechanism, Education cannot fully leverage information about TPP quality gathered by its various programs. Furthermore, Education's current efforts to share information about TPP quality with states only reach about a third of states, according to GAO's survey, although about half of all states reported that they wanted more of such information. Gaps in the agency's efforts to disseminate information result from information-sharing being left to individual offices' initiative rather than an agency-wide mechanism. Education officials acknowledged that more could be done to share information with states and other stakeholders. Without such efforts, Education may miss opportunities to support state efforts to improve TPP quality. For example, states may be unaware of good practices identified by Education that could assist them in their oversight.
section_title: What GAO Recommends
paragraphs: Among other things, GAO recommends that Education monitor states to ensure their compliance with requirements to assess whether any TPPs are low-performing and develop mechanisms to share information about TPP quality within the agency and with states. Education agreed with our recommendations.

section_title: Letter
paragraphs: Teacher preparation programs (TPP) play a vital role in helping ready the  nation’s K-12 teachers for the classroom. The role of TPPs is especially  important as all states begin using or developing college- and career- ready standards for math and English that require more complex subject- matter understanding from K-12 students and their teachers than many  previous state standards. According to data from the U.S. Department of  Education (Education), approximately 500,000 teaching candidates were enrolled in over 2,000 TPPs during the 2012-2013 academic year.Further, the federal government makes a substantial investment in  preparing teachers. Students who completed bachelor’s degree  requirements between July 1, 2007 and June 30, 2008 and were teaching  or considering teaching by June 2009 received an estimated $12.7 billion  in Federal Pell Grants and federal student loans for their undergraduate  and graduate education.
States have the primary responsibility for overseeing TPPs, including  defining the types of TPPs that may operate in a given state and  reviewing and approving TPP operations. States must also comply with  federal reporting requirements under Title II of the Higher Education Act  of 1965, as amended (Higher Education Act), which include assessing  whether any TPPs are low-performing and reporting a list of low- performing and at-risk TPPs to Education and the public. plays an indirect role in supporting TPPs through various methods. In  particular, Education collects and disseminates the data reported by  states about TPPs, such as the number of teaching candidates enrolled in  TPPs, pass rates for the tests candidates are required to take to be  licensed to teach, and states’ procedures for identifying low-performing  TPPs. Education also administers several competitive grant programs  that fund TPP reforms, such as the Teacher Quality Partnership Grant  Program.
Title II of the Higher Education Act, as amended by the Higher Education Opportunity  Act, is codified at 20 U.S.C. §§ 1021-1041. Each institution of higher education that  conducts a TPP and enrolls students receiving federal student aid is required to annually  report certain information to the state and the general public, and each state that receives  funds under the Higher Education Act must report annually to Education and the general  public certain information on the TPPs in the state. See 20 U.S.C. §§ 1022d-1022f for  state and institutional reporting requirements. For the purposes of this report, we refer to  these requirements collectively as Higher Education Act Title II reporting requirements. candidates attending many of these programs, you and Representative  George Miller, in his role as Ranking Member of the House Committee on  Education and the Workforce, asked us to review oversight and support  provided to TPPs. This report examines:  1.  how states oversee TPPs;  2.  what states and select TPPs are doing to prepare new teachers for  new K-12 standards; and,  3.  the extent to which Education’s data reporting and other information- sharing support and encourage high quality TPPs.
To obtain information about all three objectives, we conducted a survey of  all 50 states and the District of Columbia, obtaining a 100 percent  response rate. The surveys and a more complete tabulation of the results  can be viewed at GAO-15-599SP. To obtain more detailed examples  about states’ and TPPs’ actions and experiences, we conducted case  studies of five states: Arizona, California, Maine, Tennessee and Virginia.  We selected these states because they represent a variety of sizes (i.e.,  numbers of TPPs operating in the state) and approaches to overseeing  TPPs (including how they use data about graduate effectiveness to  evaluate TPP performance), and to reflect different types of college- and  career-ready K-12 standards. For each case study state, we reviewed  documents and interviewed officials from the state agency or agencies  that oversee TPPs, two to three TPPs, and at least one K-12 school  district. In total, we interviewed 14 TPPs in our 5 case study states. We  ensured that this non-generalizable sample varied in a number of ways,  including whether they were located within or outside colleges and  universities; were alternative route or traditional; and offered in-person or  online coursework. We also interviewed U.S. Department of Education  officials and contractors, teacher preparation researchers, organizations  that represent TPPs, companies that test teaching candidates for state  licensure, and other stakeholders. In addition, we reviewed relevant  federal laws, regulations, and other documents; analyzed data collected  pursuant to the Higher Education Act Title II reporting requirements; and  compared Education’s actions to federal standards for internal controls  and leading practices for data-driven program management. We also  reviewed the reliability of select data fields of the Higher Education Act  Title II reports published by Education regarding the number of TPPs that  states reported identifying as low-performing or at risk of being low- performing by: (1) reviewing agency documents, (2) interviewing relevant  agency officials, and (3) testing the accuracy of these reporting fields for a  non-generalizable sample of states. We determined that Education’s  compilation of state data about the number of low-performing TPPs was  sufficiently reliable to include in our report.
We conducted our work from March 2014 to July 2015 in accordance with  generally accepted government auditing standards. Those standards  require that we plan and perform the audit to obtain sufficient, appropriate  evidence to provide a reasonable basis for our findings and conclusions  based on our audit objectives. We believe that the evidence obtained  provides a reasonable basis for our findings and conclusions based on  our audit objectives.
subsections: section_title: Background
subsections: section_title: Structure of TPPs
paragraphs: TPPs prepare teaching candidates to employ effective teaching  techniques and gain real-world experience in the classroom. TPPs take  many forms and may be operated by a variety of organizations (see table  1). For example, the structure of TPPs can vary widely, from “traditional”  TPPs such as four-year undergraduate programs with student teaching  requirements, to “alternative route” TPPs such as those wherein  candidates serve as a classroom teacher while concurrently completing  their coursework.
section_title: State Oversight
paragraphs: State oversight responsibilities related to TPPs may be held by one or  more state agencies, including the state department of education, the  state board of education, or a state independent standards board. States  have discretion in how they conduct oversight of TPP quality, by, for  example: defining the types of TPPs that may operate in the state, such as  undergraduate or post-baccalaureate TPPs, or alternative route TPPs;  reviewing and approving individual TPPs to operate and periodically  assessing them for renewal; assessing whether any TPPs in the state are low-performing, as  required under the Higher Education Act, using criteria of the state’s  choosing; and,  setting licensing requirements that teaching candidates must satisfy  that often include completing an approved TPP and passing licensing  tests that assess subject-matter knowledge or other skills.
section_title: College- and Career- Ready Standards for K-12 Students
paragraphs: States are also responsible for adopting academic content standards for  K-12 students. To address concerns about inadequately prepared  students, all states are now using or developing academic standards that  are explicitly tied to college and career preparation (referred to in this  report as new K-12 standards). In 2010, the Council of Chief State School  Officers and the National Governors Association spearheaded the effort  to help states develop common college- and career-ready standards for  grades K-12 in math and English, which resulted in the Common Core  State Standards. As of the beginning of the 2014-15 academic year, 44  states and the District of Columbia were using the Common Core  Standards that were developed and published in 2010, and the remaining  states were using or developing their own college- and career-ready  standards.
section_title: Federal Reporting Requirements and Support
paragraphs: Education does not have direct oversight authority over TPPs.main ways it influences the quality of TPPs are (1) implementing the  Higher Education Act Title II reporting requirements and (2) awarding and  administering several competitive grants.
subsections: section_title: Higher Education Act Title II Reporting Requirements
paragraphs: Title II of the Higher Education Act requires states and institutions of  higher education (referred to in this report as “colleges and universities”) that conduct TPPs to annually report specific information. States and  most colleges or universities that offer TPPs submit the required data  using reporting templates developed by Education (see table 2).
In this report we use the term “college or university” to mean “institution of higher  education” as defined by the act.
This table presents a summary of the templates developed by the Department of Education for  reporting purposes and is not intended to provide an exhaustive list of the Higher Education Act Title  II reporting requirements for states or institutions of higher education. For the statutory reporting  requirements, see 20 U.S.C. §§ 1022d-1022f.
Education is responsible for ensuring that states and colleges and  universities offering TPPs provide the required information. The agency  also compiles and disseminates the information to the public in annual  reports, webpages and data spreadsheets. Education contracts with a  private research organization (Westat) to provide states and colleges and  universities with technical assistance in collecting the required information  and to assist the agency in compiling, analyzing, and publishing the  resulting data. On December 2, 2014, Education published a notice of  proposed rulemaking, which among other things, proposed to modify the   The proposed rule was available for  Title II reporting requirements.public comment through February 2, 2015. As part of this rulemaking  effort, Education has also proposed revisions to the templates for  reporting the Title II data.
section_title: Competitive Grant Programs
paragraphs: Education administers several competitive grant programs that provide  funding for TPP reforms. Of these competitive grants, the largest that is  focused specifically on improving TPP quality is the Teacher Quality  Partnership Grant program. In September 2014, Education selected 24  partnerships, including TPPs and partnering school districts, to receive a  combined $35 million in Teacher Quality Partnership grant funds to  improve teacher preparation primarily for science, technology,  engineering, and math teachers. The Transition to Teaching grant  program also provides grants to recruit and retain teachers in high-need  schools and encourage the development and expansion of alternative  route TPPs. Aside from these two grant programs, applicants for grants  from the Race to the Top Fund, Investing in Innovation Fund, and  Supporting Effective Educator Development program may also choose to  develop proposals related to teacher preparation programs and activities,  among other topics. For example, Investing in Innovation grants funded  25 projects related to TPPs out of the 143 projects the program has  funded during fiscal years 2010-2014.
section_title: States Assessed TPPs before Approving Them to Prepare Teachers but Some States Reported Not Having a Process to Identify Low-Performing TPPs as Required by Statute
subsections: section_title: Almost All States Assessed TPP Program Design and Information about Candidates and More Than Half Assessed TPPs Based on Graduates’ Effectiveness
paragraphs: All states reported that they review traditional TPPs before approving  them to prepare new teachers and may renew approval on a periodic  basis. To do this, nearly all states reported that they review TPP  program design and data about candidates before approving them to  prepare new teachers, and more than half also use one or more types of  information to assess graduates’ effectiveness as teachers. To assess  program design, 49 states and the District of Columbia reported in our  survey that they assess whether TPPs seeking approval are meeting  standards for program quality.specify that teaching candidates should be trained to identify the   For example, these standards may  appropriate teaching techniques for particular learning needs or achieve a  particular threshold of subject matter knowledge. States also reported that  they typically reviewed program design for traditional TPPs by reviewing  syllabi or other course material (43 states), and interviewing TPP faculty  or staff (41 states). To conduct these reviews, states reported that they  may use teams made up of peers from other TPPs, state staff, national  accreditation organization staff, or a combination, and make  determinations using professional judgment. For example, when  conducting these reviews, 43 states reported that they consider  information collected by an external TPP accreditation organization— such as the Council for the Accreditation of Educator Preparation  (CAEP)—for at least some traditional TPPs.
In addition to reviewing program design, nearly all states reported in our  survey that they examine data about teaching candidates. Most states  reported using data about the proportion of candidates who obtain a  teaching license (48 states) and the proportion of candidates who  graduate (29 states) as part of their approval process for all traditional  TPPs (see fig. 1).
These results are based on our survey of all 50 states and the District of Columbia. In some  instances, the wording of the original survey question has been modified for brevity and to remove  technical terminology. See the related e-supplement, GAO-15-599SP, for the original language.  Additionally, for information about the teaching candidate data that states used as part of their  approval process for alternative route TPPs, see questions 9a, 9f, 9h, and 9i in the related e- supplement.  “Pre-service assessments” could include evaluations of candidates’ performance as student  teachers.
More than half of states reported that they also review certain information  about graduate effectiveness when assessing TPPs for approval or  renewal. States that incorporate this information reported doing so in  several ways. Most commonly, 30 states reported using surveys that  assessed principals’ and other district personnel’s satisfaction with recent  traditional TPP graduates. Fifteen states reported assessing traditional  TPPs based on other outcomes data such as the test scores (i.e., K-12  student assessment results) of public school students taught by recent  TPP graduates (see fig. 2). For example, one of our case study states  uses such data to help TPPs identify potential problem areas in the  training they provide to teaching candidates. Officials in this state told us  they used these data to help a TPP identify shortcomings in its social  science program.
As of the 2014-2015 academic year, at least 10 additional states reported  that they planned to begin using graduate effectiveness information or  expand their current use of such information as part of their approval  process, according to our case study state interviews and several survey  responses. For example, officials in Tennessee told us that they currently  review some effectiveness data in their approval process and plan to  begin reviewing recent TPP graduates’ teacher evaluation results in the  future. Additionally, officials in Arizona told us they plan to begin  reviewing data on graduate effectiveness as part of their approval  process. They told us that the impetus behind adding this new data is to  better align the state’s TPP quality standards with recommended  requirements presented by CAEP, so that TPPs that already have CAEP  approval may receive an expedited state approval process.
States and TPPs reported challenges collecting information on graduates’  effectiveness. Officials from 3 of our 5 case study states and 7 of the 14  TPPs we spoke to said that collecting this type of data is difficult. For  example, state oversight offices or TPPs would need to obtain key  information about TPP graduates—such as performance evaluations or  employer survey responses—from local districts and several of these  officials noted districts may be difficult to identify or not be willing to  provide such information. Officials in one of our case study states noted  that it was especially challenging to obtain data on teachers who work in  another state or in private schools. As shown in figure 2 above, states  more commonly reported using information about the effectiveness of  graduates teaching in public schools within their state versus information  about those teaching in private schools or in other states.
section_title: Some States Use Different Types of Information to Approve Alternative Route TPPs versus Traditional TPPs
paragraphs: When deciding whether to approve or renew TPPs for operation, 22  states reported using fewer sources of information for alternative route  TPPs compared to traditional TPPs. For example, eight states reported  that they assessed alternative route TPPs against state developed  standards less frequently than traditional TPPs and seven states reported  using observations of alternative route TPP courses or experiences less  frequently (see table 3).
The differences in how states approve alternative route TPPs compared  to traditional TPPs may be a consequence of several factors, including  how states define alternative route TPPs and differences in state  requirements for alternative route and traditional TPPs. For example,  some alternative route TPPs may not include a student teaching  requirement, so information about student teachers’ performance would  not be relevant for making approval decisions. Additionally, in response to  one of our survey questions, a representative from one state explained  that the requirements for alternative route and traditional TPPs are  different because the requirements for traditional TPPs are set by state  regulations while some alternative TPPs may be approved through other  mechanisms.
section_title: Some States Reported Not Having a Process for Identifying Low-Performing TPPs, and Education Does Not Verify State Procedures
paragraphs: Some states reported not having a process for identifying low-performing  TPPs, as required under the Higher Education Act. In order to receive  funds under the Higher Education Act, states are required to conduct an  assessment to identify low-performing TPPs in the state and provide  Education with an annual list of such programs and any programs at risk  of being placed on the list. States have flexibility in the criteria they use  to identify low-performing TPPs, but are required to describe those criteria  to Education as part of their annual Title II reports. This provides an  avenue for states to assess TPP quality and make their determinations  public. However, officials from seven states reported in our survey that  they did not have a process for identifying low-performing TPPs in  academic year 2014-2015.
In response to our follow-up inquiries, state officials who reported not  having a process in our survey told us they believed their other oversight  procedures are sufficient to ensure quality without having a process to  identify low-performing TPPs or that they were in the middle of changing  their state’s process for identifying low-performing TPPs, among other  reasons. For example, officials in 2 of the 7 states that reported they do  not have a process to identify low-performing programs in our survey told  us having a process for identifying low-performing TPPs was not  necessary to ensure that all TPPs are performing sufficiently. While it is  possible that all TPPs are meeting states’ performance criteria and do not  merit a low-performance designation, states are still required to conduct  an assessment. Officials from two other states told us that they are  developing or planning to develop a new process for identifying low- performing TPPs, but do not have a process in the interim. Education  officials told us that while states can choose to change their process from  time to time, they are expected to use the previous process until the new  one is implemented.
Education does not verify whether states use the process they describe in  their Title II reports to identify low-performing programs or ensure that all  states have such a process. According to Standards for Internal Control in  the Federal Government, an agency’s management should provide  reasonable assurance of compliance with applicable laws and  regulations. In addition, under the Higher Education Act, Education has  responsibility for ensuring the quality of the data submitted in Title II  reports. Education officials told us that state officials are expected to  certify the accuracy of the Title II data they submit to Education and  Education reviews the state reports for obvious instances of non- compliance. However, agency officials were not aware that two states did  not describe a process to identify low-performing programs in their most  recent Title II reports. Education officials also said that Education does  not verify that states are in fact implementing the procedures they  describe in their Title II reports, due to financial constraints. All seven  states that reported in our survey that they did not have a process for  identifying low-performing TPPs described a process to Education in the   Without a monitoring  Title II report that was reported in October, 2014.process to verify the accuracy of this information in state reports,  Education may miss instances of noncompliance. If states are failing to  comply with the federal requirement to conduct an assessment to identify  low-performing TPPs, struggling TPPs may not receive the technical  assistance they need and potential teaching candidates or hiring school  districts will have difficulty identifying struggling TPPs. This may impact  the quality of training provided to new teachers and result in their being  inadequately prepared to educate children.
The majority of states (43) reported to us that they had a process for  identifying low-performing TPPs or TPPs that were at risk of becoming  low-performing. The most common criteria they used to identify low- performing TPPs were failure to meet the state’s TPP or teaching  standards (used by 35 states) and denial or conditional approval during  the state approval or renewal process (used by 34 states).used teacher evaluations (9) or student assessments (8) to identify low- performing TPPs.
According to data that states submitted as part of their Title II report, 6  states identified one or more TPPs as low-performing and 13 states  identified one or more TPPs as at risk of becoming low-performing in  2013 or 2014. Education Act requires that states provide them with technical  assistance. Of the 6 states that identified low-performing TPPs in 2013  or 2014, all reported in our survey that they provided technical support  and informed the TPPs of their status. Most also publicized this status (4)  and increased their monitoring of the TPPs (5).
Two of the six states that identified low-performing programs also identified programs at  risk of becoming low-performing. U.S. Department of Education, At-Risk and Low- Performing Programs by State for 2013 and 2014, in the Title II Data Tools, accessed  April 2015, https://title2.ed.gov/Public/DataTools/Tables.aspx.
section_title: Most States Provided Some Support and Oversight As TPPs Modified Their Programs Based on New K-12 Standards
subsections: section_title: New K-12 Standards Led Most States to Offer TPPs Information, Adjust Their Approval Process, and Use Modified Licensing Tests
paragraphs: As states shifted to new K-12 standards, most reported taking steps to  help TPPs prepare prospective teachers to teach lessons aligned with the  new standards. To help TPPs understand the standards, 37 states  reported in our survey that they provided TPPs with written resources,  information sessions, or both. All of our five case study states offered  TPPs information about the standards using various approaches. For  example, one state convened a half-day workshop that included  information about the new standards, related changes to TPP oversight  processes, and examples of how TPPs might choose to adapt to the new  standards. Officials then posted a recording of the session and additional  resources about the standards online. Three states reported inviting TPPs  to K-12 conferences that discussed the new standards. Officials in one of  those states said that this approach allowed them to foster  communication and coordination among K-12 districts and TPPs.
Apart from offering information, most states reported that they modified  their oversight activities to verify that TPPs were aligning with new K-12  standards. In particular, 34 states reported deliberately modifying their  TPP approval process to assess such alignment, by, for example,  modifying state standards for TPP quality to align them with new K-12  standards. Twelve other states did not report modifying their approval  process for this specific reason, but did report assessing some or all  TPPs against standards for TPP quality that may nonetheless provide  information about alignment. For example, the Interstate Teacher  Assessment and Support Consortium (InTASC) standards are commonly  used standards for TPP quality and were designed to align with the  Common Core State Standards.reported placing a TPP under conditional approval due in part to limited  alignment with new K-12 standards during the 2013-2014 academic year,  and no states reported denying approval for this reason.
According to our survey, two states  Fewer states reported modifying their process for identifying low- performing TPPs to assess alignment with new K-12 standards.  Specifically, in our survey, 27 states reported taking steps to modify the  process for identifying low-performing TPPs. For example, California  reported modifying its process for approving TPPs to assess alignment  with new K-12 standards. It also continued its previous practice of  identifying TPPs as low-performing if they receive conditional renewal  decisions. In our survey, no states reported identifying a TPP as low- performing during the 2013-2014 academic year because of limited  alignment with K-12 standards.
States also used modified licensing tests designed to assess individual  teaching candidates’ preparation for the new K-12 standards, according  to officials from the companies that develop the tests. Two national  companies—Educational Testing Service and Pearson—developed tests  related to math and English language arts and reported that such tests  are used by 43 states. The testing companies both reported modifying  those tests to align with new K-12 standards, although we did not  independently evaluate the extent of this alignment. Among the 8 states  that do not use such tests, 4 reported in our survey that they modified  their licensure requirements in other ways in response to the new K-12  standards. For example, all four of these states contract with testing  companies to design customized tests and may request revisions to align  with the new K-12 standards. Pearson officials described working with  one such state to modify its custom tests to align with new K-12  standards by adding more questions that measure teaching candidates’  ability to teach non-fiction texts and address how to help hypothetical K- 12 students understand complex subject-matter information.
section_title: To Varying Degrees, Select TPPs Took Steps to Incorporate New K-12 Standards
paragraphs: All 14 TPPs we interviewed made changes that ranged from large-scale  reforms to more modest modifications.the following three categories: (1) increasing subject-matter knowledge,  (2) modifying coursework related to teaching techniques, and (3) using  classroom training to provide real world experience. Examples of such  changes, and related challenges, are listed below.
The changes generally fell within  Increasing subject-matter knowledge: Officials from 11 of 14 TPPs  described changing coursework or coursework requirements to  ensure that teaching candidates had sufficient subject-matter  knowledge to teach to the new K-12 standards. The three TPPs that  did not make such changes were graduate-level programs or  otherwise required teaching candidates to obtain a bachelor’s degree  before attending the TPP and officials said most or all subject-matter  knowledge should be obtained prior to starting the TPP. The TPPs  that did make changes sometimes coordinated with other  departments, such as math and English. For example, one TPP  began offering courses that were co-taught by subject-area and TPP  faculty. Officials from a few TPPs stated that some academic  departments were more receptive to modifying their curriculum than  others, due to department priorities or other factors. For example,  officials from one TPP observed that the new K-12 standards for  English require the participation of teachers from multiple academic  disciplines, but TPP faculty members who were not in the English  department were sometimes reluctant to modify their courses  accordingly.
Modifying coursework related to teaching techniques: Officials  from all but one of the 14 TPPs we contacted described modifying  coursework related to teaching techniques. For example, officials from  one TPP said that candidates should learn teaching techniques that  focus on collaboration and officials from another said that it was  important for candidates to make connections between different  subject areas. Officials from five TPPs also said the new K-12  standards led them to start or expand courses on teaching techniques  that are subject-matter specific.
Using classroom training to provide real world experience:  Officials from all 14 TPPs we spoke with highlighted the importance of  providing candidates with ample opportunities to apply the new K-12  standards in real classrooms and receive feedback or support from  mentor teachers or TPP staff. For example, officials from one TPP  and a school district said that mentor teachers can be important role  models for teaching candidates because they can illustrate how to  apply new teaching techniques and adapt to changing expectations.  Officials from 9 of the 14 TPPs described assessing candidates’  preparedness for the new K-12 standards when reviewing their  performance or soliciting district feedback about teaching candidates’  performance. However, officials from half of the TPPs we interviewed  acknowledged the difficulty of training new teachers in real classroom  settings. In particular, officials from 6 TPPs said that school districts  are training veteran teachers in the new standards at varying rates,  and officials from several TPPs observed that teaching candidates  may not always be paired with a veteran teacher who knows the  standards as well as the teaching candidate.
As states continue to implement the new K-12 standards, several TPPs  we spoke with said that they planned to make further modifications. For  example, one TPP that made a number of modifications to its program  recently surveyed faculty and administrators to evaluate its efforts and  identify any ongoing needs. In addition, many states are beginning to  adopt assessments to measure K-12 students’ performance on the new  standards, and officials from five TPPs said it will be important to  incorporate information about these assessments into their programs.
section_title: Education Missed Opportunities to Support TPP Improvement through Its Data Reporting and Information- Sharing
subsections: section_title: Some Title II Data Have Not Been Useful to States and TPPs, but Education Has Not Assessed Whether Less Useful Elements Should be Eliminated
paragraphs: The current Title II data requirements may be of limited use in helping to  improve the quality of TPPs and Education has not taken steps to  evaluate whether any of them should be eliminated. Each state collects  Title II information from colleges and universities that offer TPPs in the  state and submits information about TPPs and some information about  state oversight processes to Education annually. For example, they report  the number and demographics of teaching candidates enrolled in and  completing TPPs. As described below, states, TPPs, and other  stakeholders often reported to us that, while they may use some data  elements, others are not useful. This difference between the Title II  information states and colleges and universities are required to report  about TPPs and the information they and other stakeholders ultimately  use to make decisions is contrary to leading practices for data-driven  management, which state that measures should be selected based on  their relevance and ability to inform decisions by key stakeholders.
Moreover, state oversight entities are important potential users of the Title  II data, but they reported mixed views about whether the data were useful  for their oversight, even though it takes a relatively large number of staff  hours to prepare the data. In our survey, states most frequently reported  spending between 21-100 staff hours completing the annual Title II state- level reports and another 21-100 hours assisting with institution and  program-level Title II reports. (See fig. 3).
After devoting this time to the task, 5 states said the Title II data were  “very useful,” 19 said they were “moderately useful,” and 25 states said  they were “neither useful nor not useful,” “slightly useful,” or “not useful.”  States most frequently reported using the Title II data to inform the  approval process or inform state agency discussions about proposed TPP  requirements. Very few states reported using the data to inform state  funding decisions or to provide information to school districts that are  hiring teachers from TPPs. Nearly every state reported some cases  where certain requirements may not be helpful. For example, less than  half the states told us that they use the goals and assurances section of  the Title II reports, which includes information about institutions’ progress  toward their goals for addressing teacher shortage areas. Further, 48  states told us either that they are not using some sections of the Title II  reports or that they already collect most of the useful Title II data  elements through other mechanisms. These results suggest that, even  among states that find the Title II data generally useful, states are  frequently required to complete some reporting requirements that they  report are not contributing to their oversight activities.
Most of the TPPs and K-12 districts and several other stakeholders with  whom we spoke questioned the usefulness of some Title II data to  themselves and other stakeholders. Officials from 6 of the 14 TPPs with  whom we spoke reported spending at least a month completing the  annual Title II reports. Yet, 8 of the 14 TPPs told us that very little of the  Title II data was useful to them for assessing the performance of their  own programs. Officials from several TPPs also expressed confusion  about the overarching goal of the Title II reporting requirements. The TPP  officials said they have not seen any indication that other stakeholders,  such as state and federal regulators or prospective teaching candidates,  were using the Title II data to inform decisions. Further, none of the  officials in the six K-12 school districts we spoke with said they use the  Title II data when comparing TPP performance or recruiting new  teachers. Similarly, seven researchers and stakeholder organizations we  spoke with questioned the usefulness of some of the Title II data for  research purposes.
Education officials said they have to continue to collect the current data,  due to statutory requirements. According to Education officials, the  agency can require states to submit new Title II data elements, but they  do not have authority to remove existing data elements that are required  by statute. Education described some benefits to the existing data,  including that it provides a robust picture of the demographics of teaching  candidates in each state, as well as the number of individuals who  complete a program. However, Education also noted in the preamble to  its December 2014 proposed rule that “data that are collected and  reported have not led to an identification of significant improvements in  teacher preparation program performance in part because the data are  not based on meaningful indicators of program effectiveness.”  Education officials told us they have not conducted a study to inform  Congress or the agency about whether any current reporting  requirements are not useful. Therefore, Education has an incomplete  picture of the usefulness of various data fields and states and colleges  and universities will continue submitting data that are time-consuming to  gather and may not be contributing to state oversight, TPP improvement,  or the public’s knowledge about TPPs.
section_title: Education Has Not Identified and Publicized Key Limitations in How Data Should Be Interpreted
paragraphs: Education permits states to report some Title II data in different ways to  account for their differing approaches to overseeing TPPs and the  differing structures of the TPPs themselves. This affects the consistency  and clarity of the data that Education ultimately disseminates. Examples  of key elements that may vary include:
Alternative route TPPs: States may use different definitions of  alternative route TPPs when reporting information such as the number  of alternative route TPPs in their state and the number of teaching  candidates who enroll in or complete such TPPs each year.example, according to Education officials, one of the top teacher- producing states defines some TPPs as traditional TPPs that most  states would consider alternative route TPPs. Education officials said  that such decisions are within the states’ authority, but have  repercussions for the consistency of national Title II data.
Teaching candidate enrollment: TPPs define when a teaching  candidate is formally enrolled and these definitions may range from  when a teaching candidate first takes a course to after they have  completed other requirements such as a certain sequence of courses.  Moreover, one large online TPP defines most of its teaching  candidates as enrolled in the state where the TPP is headquartered,  even though they live throughout the country and most likely plan to  teach in other states. Consequently, the Title II report lists that state  as one of the nation’s top teacher-producing states, while state  officials told us that in fact the state faces teacher shortages.
Program Completers: Education’s Title II guidance defines a TPP  completer as someone who has met all the educational or training  requirements in a state-approved course of study for initial teacher  certification or licensure. However, this definition allows for states or  TPPs to choose whether they require their students to take and pass  all state licensing tests before they can complete all of those  educational or training requirements. As a result, different definitions  of completer can lead to inconsistent data and make it difficult to  make comparisons across TPPs or obtain a national picture regarding  TPP completers.
These varying ways of calculating and reporting key Title II data have  persisted despite Education’s efforts to clarify guidance and improve  reporting tools. Such efforts, according to Education and its Title II  contractor, have included clarified definitions, new guidance documents,  some on-site technical assistance visits, and additional data checks in the  online Title II reporting system. Several states and TPPs with whom we  spoke praised Education’s efforts to facilitate the Title II submission  process, noting particularly that they have received excellent technical  assistance when they have questions about the process. However, states  and TPPs also noted remaining challenges, such as how to interpret the  Title II requirements within the context of each state or TPP’s specific  circumstances.
Education identified some potential data inconsistencies, such as  differences in state definitions of alternate route TPPs, in its most recent  Title II annual report, which presented information about the 2009-2010  academic year. However, it has not provided similar information about  limitations of data it has disseminated since then. In more recent Title II  data in published spreadsheets and on its website, Education does not  include clarifications about potentially inconsistent data elements, such as  alternative route TPPs or definitions of enrollees and completers.  Education officials told us they did not include such explanatory material  in these other formats because they considered the explanations in the  previous annual report to be sufficient. However, these officials also noted  that the agency may consider adding such material in the future. By not  providing the explanatory material, the data may be potentially misleading  and make it difficult for users to compare across states or programs. In  addition, by not providing these explanations, Education’s approach is not  consistent with federal internal control standards, which require that  pertinent information should be identified, captured, and distributed in a  form that permits users to perform their duties efficiently.
section_title: Education Has Taken Few Steps to Share Information about Teacher Preparation Program Quality Internally and with States
paragraphs: Education has made few efforts to share expertise about TPP quality  among its offices. Various offices and programs within Education  influence TPP quality, including the Office of Postsecondary Education,  which administers Higher Education Act Title II reporting requirements,  and several offices that administer competitive grants. However, the  agency does not have mechanisms in place to promote regular, sustained  information-sharing among these offices. Education officials said the  agency and its Title II contractor occasionally create custom Title II data  runs for Education program offices, but there is no systematic effort to  share Title II data within the agency. Further, Education officials in one  office that administers competitive grants related to TPP quality described  discussing grant results internally, but did not systematically discuss TPP  quality with staff in other offices and programs. Education formerly  convened a teacher quality workgroup on a regular basis that included  opportunities to share information across the agency regarding issues  related to TPP quality. Education officials noted that such workgroups,  particularly when operating out of high-level offices in the agency, have  been very helpful for systematically sharing information across offices.  However, the teacher quality workgroup has been inactive since the office  in which it was housed reorganized in the fall of 2014 and the agency has  not subsequently resumed these information-sharing efforts. This  represents a missed opportunity to use relevant information to bolster the  effectiveness of several Education programs, such as Race to the Top  and Supporting Effective Educator Development, which fund TPP  improvements, among other priorities. Federal internal controls standards  emphasize that effective information-sharing efforts are those that flow  broadly across an agency in a form that is helpful for those who need it to  carry out their responsibilities. Without such mechanisms to promote  information-sharing, programs and offices within Education may not have  access to clear and useful information about TPP quality.
Furthermore, Education’s efforts to support or enhance TPP quality reach  a limited number of states. For example, according to Education officials,  several technical assistance and research entities and the office that  administers competitive grants such as the Teacher Quality Partnership  Grant Program have recently undertaken research or disseminated good  practices about teacher preparation. However, in our survey, only about  one third of states reported receiving information from Education about  TPP oversight or enhancing TPP quality, and about half of all states said  they would like additional support from Education on this topic. This  suggests that Education is also missing opportunities to support states  that could use relevant research and assistance from Education to  enhance TPP quality. Additionally, 15 states reported in our survey that  they oversee TPPs through an independent standards board, oversee the  licensing of new teachers through such boards, or both. Among the eight  such states that responded to our follow-up inquires, officials from four  expressed concern about their access to Education resources for TPP  improvements because they are independent from the primary state  educational agency that has formalized relationships with Education’s  technical assistance providers. Gaps in the agency’s efforts to  disseminate information result from information- sharing being left to  individual offices’ initiative rather than an agency-wide mechanism, and  Education officials noted that more could be done to share information  with states and other stakeholders. Education officials also noted that  sharing information with states can be challenging for some competitive  grant programs, because program funds are not always available for  Education to use for national activities, including providing technical  assistance to non-grantees. However, this also underscores the  importance of Education systematically leveraging existing resources to  disseminate knowledge about enhancing TPP quality. Federal internal  controls standards emphasize the importance of agencies ensuring  adequate information-sharing with key stakeholders. Without such an  approach, Education may be missing opportunities to support state efforts  to enhance TPP quality. For example, states may be unaware of  information about good practices for TPP quality that could assist them in  their oversight.
section_title: Conclusions
paragraphs: TPPs serve a vital role in preparing new teachers—and thereby, K-12  students—for future success. The recent shift to college- and career- ready K-12 standards further highlights the importance of such programs.  Education is responsible for collecting and disseminating Higher  Education Act Title II data and administering grant programs that provide  an opportunity to contribute to TPPs’ continuous improvement. However,  unless Education ensures that all states assess whether TPPs are low- performing as required by the Higher Education Act, low-performing TPPs  may not be identified, potentially resulting in new teachers being ill- prepared to teach K-12 students. Further, if Higher Education Act Title II  reports include data elements that are not useful, states and colleges and  universities that offer TPPs will expend unnecessary effort on collecting  information that is unlikely to improve TPP quality. Among reporting  requirements that are useful, if Title II reports do not include important  limitations on how the data should be used, policymakers and  practitioners could draw incorrect conclusions based on the data. Finally,  without increasing information-sharing within the Department and with  states, Education may miss opportunities to disseminate information that  could enhance TPP quality.
section_title: Recommendations for Executive Action
paragraphs: We recommend that the Secretary of Education take the following four  actions:  1.  Develop a risk-based, cost-effective strategy to verify that states are  implementing a process for assessing whether any teacher  preparation programs are low-performing.  2.  Study the usefulness of Title II data elements for policymakers and  practitioners, and, if warranted, develop a proposal for Congress to  eliminate or revise any statutorily-required elements that are not  providing meaningful information.  3.  Identify potential limitations in the Title II data and consistently  disclose these limitations in the reports, websites, and data tables the  agency uses to distribute the results. This could include more detailed  information about data elements where definitions vary substantially  from state to state or teacher preparation program to teacher  preparation program.  4.  Develop and implement mechanisms to systematically share  information about teacher preparation program quality with relevant  Department of Education program offices and states (including state  Independent Standards Boards).
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of the report to the Department of Education for  review and comment. Education’s comments are reproduced in Appendix  II. Education agreed with our four recommendations. Regarding our first  recommendation, Education noted that its proposed regulations include  new requirements for how states report on TPP performance, including  whether any TPPs are low-performing. Education anticipates that its final  regulations will guide the agency’s future efforts to monitor states’  processes for identifying any low-performing TPPs. While it finalizes its  regulations, Education also plans to work closely with select states to help  ensure they comply with Title II requirements related to identifying low- performing TPPs. We believe that interim monitoring will be important,  particularly if the regulations are not finalized prior to the next Title II  reporting cycle. Education also agreed with our other three  recommendations, stating that the agency would: examine statutorily- defined Title II reporting requirements and make recommendations to  Congress to remove or revise requirements, as warranted; identify  potential limitations in the Title II data and disclose such limitations in its  reports, websites, and data tables; and, enhance information-sharing  about TPP quality within Education and relevant state agencies.
We are sending copies of this report to the appropriate congressional  committees, the Secretary of the Department of Education, and other  interested parties. In addition, the report is available at no charge on the  GAO website at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (617) 788-0534 or EmreyArrasM@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix III.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The objectives of this study were to answer the following questions: (1)  How do states oversee teacher preparation programs (TPP); (2) What are  states and select TPPs doing to prepare new teachers for new K-12  standards; and (3) To what extent does the U.S. Department of  Education’s (Education) data reporting and other information-sharing  support and encourage high quality TPPs?
We used a variety of methods to examine all three objectives. We  reviewed relevant federal laws and regulations; federal internal controls  standards; and leading practices for program management. We also  reviewed documents from Education, select states, and research  organizations. We also conducted a web-based survey of the state  entities responsible for overseeing TPPs in all 50 states and the District of  Columbia (see below for more information about this survey). In addition,  we conducted interviews with a wide range of stakeholders including  Education officials and contractors; companies responsible for  administering state licensing tests; the national accreditation organization  for teacher preparation programs; researchers; as well as organizations  representing teachers, teacher preparation programs, officials who head  state departments of education, and governors. We also conducted case  studies in 5 states. In each of these states we interviewed officials from  state oversight entities, officials from two or three TPPs, and officials in at  least one K-12 school district (see below for more information about these  case study interviews). Finally, we analyzed data collected to fulfill Higher  Education Act Title II reporting requirements and Education’s related  reports and technical assistance guidance (see below for more  information about this analysis).
We conducted our work from March 2014 to July 2015 in accordance with  generally accepted government auditing standards. Those standards  require that we plan and perform the audit to obtain sufficient, appropriate  evidence to provide a reasonable basis for our findings and conclusions  based on our audit objectives. We believe that the evidence obtained  provides a reasonable basis for our findings and conclusions based on  our audit objectives.
subsections: section_title: Survey of 50 States and the District of Columbia
paragraphs: To identify how states oversee TPPs, modified their practices in response  to new K-12 standards, and used Higher Education Act Title II data and  Department of Education information about TPPs, we surveyed state  oversight entities from all 50 states and the District of Columbia,  achieving a 100 percent response rate. The survey was administered  from November 2014 to March 2015. The survey used self-administered,  electronic questionnaires that were posted on the Internet. We sent the  survey to the official responsible for submitting each state’s Higher  Education Act Title II reporting requirements and requested that this  person consult with other officials in order to provide an official state  response. We reviewed state responses and followed up by e-mail or  telephone with select states for additional clarification and obtained  corrected information for our final survey analysis. We also published  survey responses in an e-publication supplemental to this report,  “Teacher Preparation Programs: Survey of State Entities that Oversee  Teacher Preparation Programs, an E-Supplement to GAO-15-598”  (GAO-15-599SP, July, 2015).
The quality of survey data can be affected by nonsampling error.  Nonsampling error includes variations in how respondents interpret  questions, respondents’ willingness to offer accurate responses, and data  collection and processing errors. We included steps in developing the  survey, and collecting, editing, and analyzing survey data, to minimize  such nonsampling error. In developing the Web survey, we pretested  draft versions of the instrument with state officials in four states and  consulted with officials from the Department of Education to check the  clarity of the questions and the flow and layout of the survey. On the basis  of the pretests, we made revisions to the survey. Further, using a web- based survey also helped remove error in our data collection effort. By  allowing state officials to enter their responses directly into an electronic  instrument, this method automatically created a record for each state  official in a data file and eliminated the errors associated with a manual  data entry process. In addition, the program used to analyze the survey  data was independently verified to ensure the accuracy of this work.
section_title: Case Studies
paragraphs: To obtain more detailed information about how states oversee teacher  preparation; the steps that states and TPPs have taken in response to  new K-12 standards; and, state and TPP views about Higher Education  Act Title II reporting requirements and Department of Education grants  and information-sharing related to TPPs, we conducted case studies of  five states: Arizona, California, Maine, Tennessee and Virginia. We  selected this non-generalizable sample of states because they represent  a variety of sizes (i.e., numbers of TPPs operating in the state) and  approaches to overseeing TPPs (including how they use data about  graduate effectiveness to evaluate TPP performance) and to reflect  different types of college- and career-ready K-12 standards. As part of  these case studies, we reviewed state documents and interviewed  officials from the state oversight agency, two to three TPPs, and at least  one K-12 school district. In total, we interviewed 14 TPPs. We ensured  that this non-generalizable sample of TPPs varied in a number of ways  including whether they were located within or separate from colleges and  universities; were alternative route or traditional; and offered in-person or  online coursework. We also ensured that the TPPs represented a range  of sizes and urban, suburban, and rural locations. We selected K-12  districts that worked with these TPPs, by providing student teaching  experiences, hiring graduates, or both. Interviews with state, TPP, and K- 12 district officials from Arizona, California and Maine were conducted in  person and interviews with officials from Tennessee and Virginia were  primarily conducted by telephone.
section_title: Analysis of Higher Education Act Title II Reports
paragraphs: To assess the extent that Education’s data reporting supports and  encourages high quality TPPs, we reviewed select state Title II reports;  Education’s reporting templates for the state reports and institutional and  program reports; Education’s December 2014 proposed rule; technical  assistance documents developed by Education and its contractor; and  Education’s reports, data spreadsheets and web pages that compile data  from Title II state reports. We interviewed Education officials and  contractors responsible for collecting, analyzing and reporting such data.  We also compared Education’s reports, spreadsheets and web pages— including their definitions and descriptions of the appropriate use of  reported data—against federal standards for internal controls and leading  practices for data-driven program management.reliability of fields in Education’s Title II data spreadsheets regarding the  number of TPPs that states reported identifying as low-performing or at  risk of being low-performing by: (1) reviewing agency documents, (2)  interviewing relevant agency officials, and (3) testing the accuracy of  these reporting fields for a non-generalizable sample of states. We  determined that Education’s compilation of state data about TPPs that  have been identified as low-performing was sufficiently reliable to include  in our report.
section_title: Appendix II: Comments from the Department of Education
section_title: Appendix III: GAO Contact and Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Scott Spicer (Assistant Director);  Barbara Steel-Lowney and Daren Sweeney (analysts-in-charge); Lucas  Alvarez, and Hedieh Fusfield made key contributions to this report. Also  contributing to this report were: Deborah Bland, Kate Blumenreich,  Joanna Chan, Sarah Cornetto, Holly Dye, Kirsten Lauber, Ashley McCall,  Sheila McCoy, Jennifer McDonald, John Mingus, Mimi Nguyen, and Tom  James.