Teacher Preparation Programs: Education Should Ensure States Identify Low-Performing Programs and Improve Information-Sharing
Released: 2015-07-23T00:00:00

Letter

Background

Structure of TPPs

TPPs prepare teaching candidates to employ effective teaching techniques and gain real-world experience in the classroom. TPPs take many forms and may be operated by a variety of organizations (see table 1). For example, the structure of TPPs can vary widely, from “traditional” TPPs such as four-year undergraduate programs with student teaching requirements, to “alternative route” TPPs such as those wherein candidates serve as a classroom teacher while concurrently completing their coursework.

State Oversight

State oversight responsibilities related to TPPs may be held by one or more state agencies, including the state department of education, the state board of education, or a state independent standards board. States have discretion in how they conduct oversight of TPP quality, by, for example: defining the types of TPPs that may operate in the state, such as undergraduate or post-baccalaureate TPPs, or alternative route TPPs; reviewing and approving individual TPPs to operate and periodically assessing them for renewal; assessing whether any TPPs in the state are low-performing, as required under the Higher Education Act, using criteria of the state’s choosing; and, setting licensing requirements that teaching candidates must satisfy that often include completing an approved TPP and passing licensing tests that assess subject-matter knowledge or other skills.

College- and Career- Ready Standards for K-12 Students

States are also responsible for adopting academic content standards for K-12 students. To address concerns about inadequately prepared students, all states are now using or developing academic standards that are explicitly tied to college and career preparation (referred to in this report as new K-12 standards). In 2010, the Council of Chief State School Officers and the National Governors Association spearheaded the effort to help states develop common college- and career-ready standards for grades K-12 in math and English, which resulted in the Common Core State Standards. As of the beginning of the 2014-15 academic year, 44 states and the District of Columbia were using the Common Core Standards that were developed and published in 2010, and the remaining states were using or developing their own college- and career-ready standards.

Federal Reporting Requirements and Support

Education does not have direct oversight authority over TPPs.main ways it influences the quality of TPPs are (1) implementing the Higher Education Act Title II reporting requirements and (2) awarding and administering several competitive grants.

Higher Education Act Title II Reporting Requirements

Title II of the Higher Education Act requires states and institutions of higher education (referred to in this report as “colleges and universities”) that conduct TPPs to annually report specific information. States and most colleges or universities that offer TPPs submit the required data using reporting templates developed by Education (see table 2).

In this report we use the term “college or university” to mean “institution of higher education” as defined by the act.

This table presents a summary of the templates developed by the Department of Education for reporting purposes and is not intended to provide an exhaustive list of the Higher Education Act Title II reporting requirements for states or institutions of higher education. For the statutory reporting requirements, see 20 U.S.C. §§ 1022d-1022f.

Education is responsible for ensuring that states and colleges and universities offering TPPs provide the required information. The agency also compiles and disseminates the information to the public in annual reports, webpages and data spreadsheets. Education contracts with a private research organization (Westat) to provide states and colleges and universities with technical assistance in collecting the required information and to assist the agency in compiling, analyzing, and publishing the resulting data. On December 2, 2014, Education published a notice of proposed rulemaking, which among other things, proposed to modify the The proposed rule was available for Title II reporting requirements.public comment through February 2, 2015. As part of this rulemaking effort, Education has also proposed revisions to the templates for reporting the Title II data.

Competitive Grant Programs

Education administers several competitive grant programs that provide funding for TPP reforms. Of these competitive grants, the largest that is focused specifically on improving TPP quality is the Teacher Quality Partnership Grant program. In September 2014, Education selected 24 partnerships, including TPPs and partnering school districts, to receive a combined $35 million in Teacher Quality Partnership grant funds to improve teacher preparation primarily for science, technology, engineering, and math teachers. The Transition to Teaching grant program also provides grants to recruit and retain teachers in high-need schools and encourage the development and expansion of alternative route TPPs. Aside from these two grant programs, applicants for grants from the Race to the Top Fund, Investing in Innovation Fund, and Supporting Effective Educator Development program may also choose to develop proposals related to teacher preparation programs and activities, among other topics. For example, Investing in Innovation grants funded 25 projects related to TPPs out of the 143 projects the program has funded during fiscal years 2010-2014.

States Assessed TPPs before Approving Them to Prepare Teachers but Some States Reported Not Having a Process to Identify Low-Performing TPPs as Required by Statute

Almost All States Assessed TPP Program Design and Information about Candidates and More Than Half Assessed TPPs Based on Graduates’ Effectiveness

All states reported that they review traditional TPPs before approving them to prepare new teachers and may renew approval on a periodic basis. To do this, nearly all states reported that they review TPP program design and data about candidates before approving them to prepare new teachers, and more than half also use one or more types of information to assess graduates’ effectiveness as teachers. To assess program design, 49 states and the District of Columbia reported in our survey that they assess whether TPPs seeking approval are meeting standards for program quality.specify that teaching candidates should be trained to identify the For example, these standards may appropriate teaching techniques for particular learning needs or achieve a particular threshold of subject matter knowledge. States also reported that they typically reviewed program design for traditional TPPs by reviewing syllabi or other course material (43 states), and interviewing TPP faculty or staff (41 states). To conduct these reviews, states reported that they may use teams made up of peers from other TPPs, state staff, national accreditation organization staff, or a combination, and make determinations using professional judgment. For example, when conducting these reviews, 43 states reported that they consider information collected by an external TPP accreditation organization— such as the Council for the Accreditation of Educator Preparation (CAEP)—for at least some traditional TPPs.

In addition to reviewing program design, nearly all states reported in our survey that they examine data about teaching candidates. Most states reported using data about the proportion of candidates who obtain a teaching license (48 states) and the proportion of candidates who graduate (29 states) as part of their approval process for all traditional TPPs (see fig. 1).

These results are based on our survey of all 50 states and the District of Columbia. In some instances, the wording of the original survey question has been modified for brevity and to remove technical terminology. See the related e-supplement, GAO-15-599SP, for the original language. Additionally, for information about the teaching candidate data that states used as part of their approval process for alternative route TPPs, see questions 9a, 9f, 9h, and 9i in the related e- supplement. “Pre-service assessments” could include evaluations of candidates’ performance as student teachers.

More than half of states reported that they also review certain information about graduate effectiveness when assessing TPPs for approval or renewal. States that incorporate this information reported doing so in several ways. Most commonly, 30 states reported using surveys that assessed principals’ and other district personnel’s satisfaction with recent traditional TPP graduates. Fifteen states reported assessing traditional TPPs based on other outcomes data such as the test scores (i.e., K-12 student assessment results) of public school students taught by recent TPP graduates (see fig. 2). For example, one of our case study states uses such data to help TPPs identify potential problem areas in the training they provide to teaching candidates. Officials in this state told us they used these data to help a TPP identify shortcomings in its social science program.

As of the 2014-2015 academic year, at least 10 additional states reported that they planned to begin using graduate effectiveness information or expand their current use of such information as part of their approval process, according to our case study state interviews and several survey responses. For example, officials in Tennessee told us that they currently review some effectiveness data in their approval process and plan to begin reviewing recent TPP graduates’ teacher evaluation results in the future. Additionally, officials in Arizona told us they plan to begin reviewing data on graduate effectiveness as part of their approval process. They told us that the impetus behind adding this new data is to better align the state’s TPP quality standards with recommended requirements presented by CAEP, so that TPPs that already have CAEP approval may receive an expedited state approval process.

States and TPPs reported challenges collecting information on graduates’ effectiveness. Officials from 3 of our 5 case study states and 7 of the 14 TPPs we spoke to said that collecting this type of data is difficult. For example, state oversight offices or TPPs would need to obtain key information about TPP graduates—such as performance evaluations or employer survey responses—from local districts and several of these officials noted districts may be difficult to identify or not be willing to provide such information. Officials in one of our case study states noted that it was especially challenging to obtain data on teachers who work in another state or in private schools. As shown in figure 2 above, states more commonly reported using information about the effectiveness of graduates teaching in public schools within their state versus information about those teaching in private schools or in other states.

Some States Use Different Types of Information to Approve Alternative Route TPPs versus Traditional TPPs

When deciding whether to approve or renew TPPs for operation, 22 states reported using fewer sources of information for alternative route TPPs compared to traditional TPPs. For example, eight states reported that they assessed alternative route TPPs against state developed standards less frequently than traditional TPPs and seven states reported using observations of alternative route TPP courses or experiences less frequently (see table 3).

The differences in how states approve alternative route TPPs compared to traditional TPPs may be a consequence of several factors, including how states define alternative route TPPs and differences in state requirements for alternative route and traditional TPPs. For example, some alternative route TPPs may not include a student teaching requirement, so information about student teachers’ performance would not be relevant for making approval decisions. Additionally, in response to one of our survey questions, a representative from one state explained that the requirements for alternative route and traditional TPPs are different because the requirements for traditional TPPs are set by state regulations while some alternative TPPs may be approved through other mechanisms.

Some States Reported Not Having a Process for Identifying Low-Performing TPPs, and Education Does Not Verify State Procedures

Some states reported not having a process for identifying low-performing TPPs, as required under the Higher Education Act. In order to receive funds under the Higher Education Act, states are required to conduct an assessment to identify low-performing TPPs in the state and provide Education with an annual list of such programs and any programs at risk of being placed on the list. States have flexibility in the criteria they use to identify low-performing TPPs, but are required to describe those criteria to Education as part of their annual Title II reports. This provides an avenue for states to assess TPP quality and make their determinations public. However, officials from seven states reported in our survey that they did not have a process for identifying low-performing TPPs in academic year 2014-2015.

In response to our follow-up inquiries, state officials who reported not having a process in our survey told us they believed their other oversight procedures are sufficient to ensure quality without having a process to identify low-performing TPPs or that they were in the middle of changing their state’s process for identifying low-performing TPPs, among other reasons. For example, officials in 2 of the 7 states that reported they do not have a process to identify low-performing programs in our survey told us having a process for identifying low-performing TPPs was not necessary to ensure that all TPPs are performing sufficiently. While it is possible that all TPPs are meeting states’ performance criteria and do not merit a low-performance designation, states are still required to conduct an assessment. Officials from two other states told us that they are developing or planning to develop a new process for identifying low- performing TPPs, but do not have a process in the interim. Education officials told us that while states can choose to change their process from time to time, they are expected to use the previous process until the new one is implemented.

Education does not verify whether states use the process they describe in their Title II reports to identify low-performing programs or ensure that all states have such a process. According to Standards for Internal Control in the Federal Government, an agency’s management should provide reasonable assurance of compliance with applicable laws and regulations. In addition, under the Higher Education Act, Education has responsibility for ensuring the quality of the data submitted in Title II reports. Education officials told us that state officials are expected to certify the accuracy of the Title II data they submit to Education and Education reviews the state reports for obvious instances of non- compliance. However, agency officials were not aware that two states did not describe a process to identify low-performing programs in their most recent Title II reports. Education officials also said that Education does not verify that states are in fact implementing the procedures they describe in their Title II reports, due to financial constraints. All seven states that reported in our survey that they did not have a process for identifying low-performing TPPs described a process to Education in the Without a monitoring Title II report that was reported in October, 2014.process to verify the accuracy of this information in state reports, Education may miss instances of noncompliance. If states are failing to comply with the federal requirement to conduct an assessment to identify low-performing TPPs, struggling TPPs may not receive the technical assistance they need and potential teaching candidates or hiring school districts will have difficulty identifying struggling TPPs. This may impact the quality of training provided to new teachers and result in their being inadequately prepared to educate children.

The majority of states (43) reported to us that they had a process for identifying low-performing TPPs or TPPs that were at risk of becoming low-performing. The most common criteria they used to identify low- performing TPPs were failure to meet the state’s TPP or teaching standards (used by 35 states) and denial or conditional approval during the state approval or renewal process (used by 34 states).used teacher evaluations (9) or student assessments (8) to identify low- performing TPPs.

According to data that states submitted as part of their Title II report, 6 states identified one or more TPPs as low-performing and 13 states identified one or more TPPs as at risk of becoming low-performing in 2013 or 2014. Education Act requires that states provide them with technical assistance. Of the 6 states that identified low-performing TPPs in 2013 or 2014, all reported in our survey that they provided technical support and informed the TPPs of their status. Most also publicized this status (4) and increased their monitoring of the TPPs (5).

Two of the six states that identified low-performing programs also identified programs at risk of becoming low-performing. U.S. Department of Education, At-Risk and Low- Performing Programs by State for 2013 and 2014, in the Title II Data Tools, accessed April 2015, https://title2.ed.gov/Public/DataTools/Tables.aspx.

Most States Provided Some Support and Oversight As TPPs Modified Their Programs Based on New K-12 Standards

New K-12 Standards Led Most States to Offer TPPs Information, Adjust Their Approval Process, and Use Modified Licensing Tests

As states shifted to new K-12 standards, most reported taking steps to help TPPs prepare prospective teachers to teach lessons aligned with the new standards. To help TPPs understand the standards, 37 states reported in our survey that they provided TPPs with written resources, information sessions, or both. All of our five case study states offered TPPs information about the standards using various approaches. For example, one state convened a half-day workshop that included information about the new standards, related changes to TPP oversight processes, and examples of how TPPs might choose to adapt to the new standards. Officials then posted a recording of the session and additional resources about the standards online. Three states reported inviting TPPs to K-12 conferences that discussed the new standards. Officials in one of those states said that this approach allowed them to foster communication and coordination among K-12 districts and TPPs.

Apart from offering information, most states reported that they modified their oversight activities to verify that TPPs were aligning with new K-12 standards. In particular, 34 states reported deliberately modifying their TPP approval process to assess such alignment, by, for example, modifying state standards for TPP quality to align them with new K-12 standards. Twelve other states did not report modifying their approval process for this specific reason, but did report assessing some or all TPPs against standards for TPP quality that may nonetheless provide information about alignment. For example, the Interstate Teacher Assessment and Support Consortium (InTASC) standards are commonly used standards for TPP quality and were designed to align with the Common Core State Standards.reported placing a TPP under conditional approval due in part to limited alignment with new K-12 standards during the 2013-2014 academic year, and no states reported denying approval for this reason.

According to our survey, two states Fewer states reported modifying their process for identifying low- performing TPPs to assess alignment with new K-12 standards. Specifically, in our survey, 27 states reported taking steps to modify the process for identifying low-performing TPPs. For example, California reported modifying its process for approving TPPs to assess alignment with new K-12 standards. It also continued its previous practice of identifying TPPs as low-performing if they receive conditional renewal decisions. In our survey, no states reported identifying a TPP as low- performing during the 2013-2014 academic year because of limited alignment with K-12 standards.

States also used modified licensing tests designed to assess individual teaching candidates’ preparation for the new K-12 standards, according to officials from the companies that develop the tests. Two national companies—Educational Testing Service and Pearson—developed tests related to math and English language arts and reported that such tests are used by 43 states. The testing companies both reported modifying those tests to align with new K-12 standards, although we did not independently evaluate the extent of this alignment. Among the 8 states that do not use such tests, 4 reported in our survey that they modified their licensure requirements in other ways in response to the new K-12 standards. For example, all four of these states contract with testing companies to design customized tests and may request revisions to align with the new K-12 standards. Pearson officials described working with one such state to modify its custom tests to align with new K-12 standards by adding more questions that measure teaching candidates’ ability to teach non-fiction texts and address how to help hypothetical K- 12 students understand complex subject-matter information.

To Varying Degrees, Select TPPs Took Steps to Incorporate New K-12 Standards

All 14 TPPs we interviewed made changes that ranged from large-scale reforms to more modest modifications.the following three categories: (1) increasing subject-matter knowledge, (2) modifying coursework related to teaching techniques, and (3) using classroom training to provide real world experience. Examples of such changes, and related challenges, are listed below.

The changes generally fell within Increasing subject-matter knowledge: Officials from 11 of 14 TPPs described changing coursework or coursework requirements to ensure that teaching candidates had sufficient subject-matter knowledge to teach to the new K-12 standards. The three TPPs that did not make such changes were graduate-level programs or otherwise required teaching candidates to obtain a bachelor’s degree before attending the TPP and officials said most or all subject-matter knowledge should be obtained prior to starting the TPP. The TPPs that did make changes sometimes coordinated with other departments, such as math and English. For example, one TPP began offering courses that were co-taught by subject-area and TPP faculty. Officials from a few TPPs stated that some academic departments were more receptive to modifying their curriculum than others, due to department priorities or other factors. For example, officials from one TPP observed that the new K-12 standards for English require the participation of teachers from multiple academic disciplines, but TPP faculty members who were not in the English department were sometimes reluctant to modify their courses accordingly.

Modifying coursework related to teaching techniques: Officials from all but one of the 14 TPPs we contacted described modifying coursework related to teaching techniques. For example, officials from one TPP said that candidates should learn teaching techniques that focus on collaboration and officials from another said that it was important for candidates to make connections between different subject areas. Officials from five TPPs also said the new K-12 standards led them to start or expand courses on teaching techniques that are subject-matter specific.

Using classroom training to provide real world experience: Officials from all 14 TPPs we spoke with highlighted the importance of providing candidates with ample opportunities to apply the new K-12 standards in real classrooms and receive feedback or support from mentor teachers or TPP staff. For example, officials from one TPP and a school district said that mentor teachers can be important role models for teaching candidates because they can illustrate how to apply new teaching techniques and adapt to changing expectations. Officials from 9 of the 14 TPPs described assessing candidates’ preparedness for the new K-12 standards when reviewing their performance or soliciting district feedback about teaching candidates’ performance. However, officials from half of the TPPs we interviewed acknowledged the difficulty of training new teachers in real classroom settings. In particular, officials from 6 TPPs said that school districts are training veteran teachers in the new standards at varying rates, and officials from several TPPs observed that teaching candidates may not always be paired with a veteran teacher who knows the standards as well as the teaching candidate.

As states continue to implement the new K-12 standards, several TPPs we spoke with said that they planned to make further modifications. For example, one TPP that made a number of modifications to its program recently surveyed faculty and administrators to evaluate its efforts and identify any ongoing needs. In addition, many states are beginning to adopt assessments to measure K-12 students’ performance on the new standards, and officials from five TPPs said it will be important to incorporate information about these assessments into their programs.

Education Missed Opportunities to Support TPP Improvement through Its Data Reporting and Information- Sharing

Some Title II Data Have Not Been Useful to States and TPPs, but Education Has Not Assessed Whether Less Useful Elements Should be Eliminated

The current Title II data requirements may be of limited use in helping to improve the quality of TPPs and Education has not taken steps to evaluate whether any of them should be eliminated. Each state collects Title II information from colleges and universities that offer TPPs in the state and submits information about TPPs and some information about state oversight processes to Education annually. For example, they report the number and demographics of teaching candidates enrolled in and completing TPPs. As described below, states, TPPs, and other stakeholders often reported to us that, while they may use some data elements, others are not useful. This difference between the Title II information states and colleges and universities are required to report about TPPs and the information they and other stakeholders ultimately use to make decisions is contrary to leading practices for data-driven management, which state that measures should be selected based on their relevance and ability to inform decisions by key stakeholders.

Moreover, state oversight entities are important potential users of the Title II data, but they reported mixed views about whether the data were useful for their oversight, even though it takes a relatively large number of staff hours to prepare the data. In our survey, states most frequently reported spending between 21-100 staff hours completing the annual Title II state- level reports and another 21-100 hours assisting with institution and program-level Title II reports. (See fig. 3).

After devoting this time to the task, 5 states said the Title II data were “very useful,” 19 said they were “moderately useful,” and 25 states said they were “neither useful nor not useful,” “slightly useful,” or “not useful.” States most frequently reported using the Title II data to inform the approval process or inform state agency discussions about proposed TPP requirements. Very few states reported using the data to inform state funding decisions or to provide information to school districts that are hiring teachers from TPPs. Nearly every state reported some cases where certain requirements may not be helpful. For example, less than half the states told us that they use the goals and assurances section of the Title II reports, which includes information about institutions’ progress toward their goals for addressing teacher shortage areas. Further, 48 states told us either that they are not using some sections of the Title II reports or that they already collect most of the useful Title II data elements through other mechanisms. These results suggest that, even among states that find the Title II data generally useful, states are frequently required to complete some reporting requirements that they report are not contributing to their oversight activities.

Most of the TPPs and K-12 districts and several other stakeholders with whom we spoke questioned the usefulness of some Title II data to themselves and other stakeholders. Officials from 6 of the 14 TPPs with whom we spoke reported spending at least a month completing the annual Title II reports. Yet, 8 of the 14 TPPs told us that very little of the Title II data was useful to them for assessing the performance of their own programs. Officials from several TPPs also expressed confusion about the overarching goal of the Title II reporting requirements. The TPP officials said they have not seen any indication that other stakeholders, such as state and federal regulators or prospective teaching candidates, were using the Title II data to inform decisions. Further, none of the officials in the six K-12 school districts we spoke with said they use the Title II data when comparing TPP performance or recruiting new teachers. Similarly, seven researchers and stakeholder organizations we spoke with questioned the usefulness of some of the Title II data for research purposes.

Education officials said they have to continue to collect the current data, due to statutory requirements. According to Education officials, the agency can require states to submit new Title II data elements, but they do not have authority to remove existing data elements that are required by statute. Education described some benefits to the existing data, including that it provides a robust picture of the demographics of teaching candidates in each state, as well as the number of individuals who complete a program. However, Education also noted in the preamble to its December 2014 proposed rule that “data that are collected and reported have not led to an identification of significant improvements in teacher preparation program performance in part because the data are not based on meaningful indicators of program effectiveness.” Education officials told us they have not conducted a study to inform Congress or the agency about whether any current reporting requirements are not useful. Therefore, Education has an incomplete picture of the usefulness of various data fields and states and colleges and universities will continue submitting data that are time-consuming to gather and may not be contributing to state oversight, TPP improvement, or the public’s knowledge about TPPs.

Education Has Not Identified and Publicized Key Limitations in How Data Should Be Interpreted

Education permits states to report some Title II data in different ways to account for their differing approaches to overseeing TPPs and the differing structures of the TPPs themselves. This affects the consistency and clarity of the data that Education ultimately disseminates. Examples of key elements that may vary include:

Alternative route TPPs: States may use different definitions of alternative route TPPs when reporting information such as the number of alternative route TPPs in their state and the number of teaching candidates who enroll in or complete such TPPs each year.example, according to Education officials, one of the top teacher- producing states defines some TPPs as traditional TPPs that most states would consider alternative route TPPs. Education officials said that such decisions are within the states’ authority, but have repercussions for the consistency of national Title II data.

Teaching candidate enrollment: TPPs define when a teaching candidate is formally enrolled and these definitions may range from when a teaching candidate first takes a course to after they have completed other requirements such as a certain sequence of courses. Moreover, one large online TPP defines most of its teaching candidates as enrolled in the state where the TPP is headquartered, even though they live throughout the country and most likely plan to teach in other states. Consequently, the Title II report lists that state as one of the nation’s top teacher-producing states, while state officials told us that in fact the state faces teacher shortages.

Program Completers: Education’s Title II guidance defines a TPP completer as someone who has met all the educational or training requirements in a state-approved course of study for initial teacher certification or licensure. However, this definition allows for states or TPPs to choose whether they require their students to take and pass all state licensing tests before they can complete all of those educational or training requirements. As a result, different definitions of completer can lead to inconsistent data and make it difficult to make comparisons across TPPs or obtain a national picture regarding TPP completers.

These varying ways of calculating and reporting key Title II data have persisted despite Education’s efforts to clarify guidance and improve reporting tools. Such efforts, according to Education and its Title II contractor, have included clarified definitions, new guidance documents, some on-site technical assistance visits, and additional data checks in the online Title II reporting system. Several states and TPPs with whom we spoke praised Education’s efforts to facilitate the Title II submission process, noting particularly that they have received excellent technical assistance when they have questions about the process. However, states and TPPs also noted remaining challenges, such as how to interpret the Title II requirements within the context of each state or TPP’s specific circumstances.

Education identified some potential data inconsistencies, such as differences in state definitions of alternate route TPPs, in its most recent Title II annual report, which presented information about the 2009-2010 academic year. However, it has not provided similar information about limitations of data it has disseminated since then. In more recent Title II data in published spreadsheets and on its website, Education does not include clarifications about potentially inconsistent data elements, such as alternative route TPPs or definitions of enrollees and completers. Education officials told us they did not include such explanatory material in these other formats because they considered the explanations in the previous annual report to be sufficient. However, these officials also noted that the agency may consider adding such material in the future. By not providing the explanatory material, the data may be potentially misleading and make it difficult for users to compare across states or programs. In addition, by not providing these explanations, Education’s approach is not consistent with federal internal control standards, which require that pertinent information should be identified, captured, and distributed in a form that permits users to perform their duties efficiently.

Education Has Taken Few Steps to Share Information about Teacher Preparation Program Quality Internally and with States

Education has made few efforts to share expertise about TPP quality among its offices. Various offices and programs within Education influence TPP quality, including the Office of Postsecondary Education, which administers Higher Education Act Title II reporting requirements, and several offices that administer competitive grants. However, the agency does not have mechanisms in place to promote regular, sustained information-sharing among these offices. Education officials said the agency and its Title II contractor occasionally create custom Title II data runs for Education program offices, but there is no systematic effort to share Title II data within the agency. Further, Education officials in one office that administers competitive grants related to TPP quality described discussing grant results internally, but did not systematically discuss TPP quality with staff in other offices and programs. Education formerly convened a teacher quality workgroup on a regular basis that included opportunities to share information across the agency regarding issues related to TPP quality. Education officials noted that such workgroups, particularly when operating out of high-level offices in the agency, have been very helpful for systematically sharing information across offices. However, the teacher quality workgroup has been inactive since the office in which it was housed reorganized in the fall of 2014 and the agency has not subsequently resumed these information-sharing efforts. This represents a missed opportunity to use relevant information to bolster the effectiveness of several Education programs, such as Race to the Top and Supporting Effective Educator Development, which fund TPP improvements, among other priorities. Federal internal controls standards emphasize that effective information-sharing efforts are those that flow broadly across an agency in a form that is helpful for those who need it to carry out their responsibilities. Without such mechanisms to promote information-sharing, programs and offices within Education may not have access to clear and useful information about TPP quality.

Furthermore, Education’s efforts to support or enhance TPP quality reach a limited number of states. For example, according to Education officials, several technical assistance and research entities and the office that administers competitive grants such as the Teacher Quality Partnership Grant Program have recently undertaken research or disseminated good practices about teacher preparation. However, in our survey, only about one third of states reported receiving information from Education about TPP oversight or enhancing TPP quality, and about half of all states said they would like additional support from Education on this topic. This suggests that Education is also missing opportunities to support states that could use relevant research and assistance from Education to enhance TPP quality. Additionally, 15 states reported in our survey that they oversee TPPs through an independent standards board, oversee the licensing of new teachers through such boards, or both. Among the eight such states that responded to our follow-up inquires, officials from four expressed concern about their access to Education resources for TPP improvements because they are independent from the primary state educational agency that has formalized relationships with Education’s technical assistance providers. Gaps in the agency’s efforts to disseminate information result from information- sharing being left to individual offices’ initiative rather than an agency-wide mechanism, and Education officials noted that more could be done to share information with states and other stakeholders. Education officials also noted that sharing information with states can be challenging for some competitive grant programs, because program funds are not always available for Education to use for national activities, including providing technical assistance to non-grantees. However, this also underscores the importance of Education systematically leveraging existing resources to disseminate knowledge about enhancing TPP quality. Federal internal controls standards emphasize the importance of agencies ensuring adequate information-sharing with key stakeholders. Without such an approach, Education may be missing opportunities to support state efforts to enhance TPP quality. For example, states may be unaware of information about good practices for TPP quality that could assist them in their oversight.

Conclusions

TPPs serve a vital role in preparing new teachers—and thereby, K-12 students—for future success. The recent shift to college- and career- ready K-12 standards further highlights the importance of such programs. Education is responsible for collecting and disseminating Higher Education Act Title II data and administering grant programs that provide an opportunity to contribute to TPPs’ continuous improvement. However, unless Education ensures that all states assess whether TPPs are low- performing as required by the Higher Education Act, low-performing TPPs may not be identified, potentially resulting in new teachers being ill- prepared to teach K-12 students. Further, if Higher Education Act Title II reports include data elements that are not useful, states and colleges and universities that offer TPPs will expend unnecessary effort on collecting information that is unlikely to improve TPP quality. Among reporting requirements that are useful, if Title II reports do not include important limitations on how the data should be used, policymakers and practitioners could draw incorrect conclusions based on the data. Finally, without increasing information-sharing within the Department and with states, Education may miss opportunities to disseminate information that could enhance TPP quality.

Recommendations for Executive Action

We recommend that the Secretary of Education take the following four actions: 1. Develop a risk-based, cost-effective strategy to verify that states are implementing a process for assessing whether any teacher preparation programs are low-performing. 2. Study the usefulness of Title II data elements for policymakers and practitioners, and, if warranted, develop a proposal for Congress to eliminate or revise any statutorily-required elements that are not providing meaningful information. 3. Identify potential limitations in the Title II data and consistently disclose these limitations in the reports, websites, and data tables the agency uses to distribute the results. This could include more detailed information about data elements where definitions vary substantially from state to state or teacher preparation program to teacher preparation program. 4. Develop and implement mechanisms to systematically share information about teacher preparation program quality with relevant Department of Education program offices and states (including state Independent Standards Boards).

Agency Comments and Our Evaluation

We provided a draft of the report to the Department of Education for review and comment. Education’s comments are reproduced in Appendix II. Education agreed with our four recommendations. Regarding our first recommendation, Education noted that its proposed regulations include new requirements for how states report on TPP performance, including whether any TPPs are low-performing. Education anticipates that its final regulations will guide the agency’s future efforts to monitor states’ processes for identifying any low-performing TPPs. While it finalizes its regulations, Education also plans to work closely with select states to help ensure they comply with Title II requirements related to identifying low- performing TPPs. We believe that interim monitoring will be important, particularly if the regulations are not finalized prior to the next Title II reporting cycle. Education also agreed with our other three recommendations, stating that the agency would: examine statutorily- defined Title II reporting requirements and make recommendations to Congress to remove or revise requirements, as warranted; identify potential limitations in the Title II data and disclose such limitations in its reports, websites, and data tables; and, enhance information-sharing about TPP quality within Education and relevant state agencies.

We are sending copies of this report to the appropriate congressional committees, the Secretary of the Department of Education, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (617) 788-0534 or EmreyArrasM@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

The objectives of this study were to answer the following questions: (1) How do states oversee teacher preparation programs (TPP); (2) What are states and select TPPs doing to prepare new teachers for new K-12 standards; and (3) To what extent does the U.S. Department of Education’s (Education) data reporting and other information-sharing support and encourage high quality TPPs?

We used a variety of methods to examine all three objectives. We reviewed relevant federal laws and regulations; federal internal controls standards; and leading practices for program management. We also reviewed documents from Education, select states, and research organizations. We also conducted a web-based survey of the state entities responsible for overseeing TPPs in all 50 states and the District of Columbia (see below for more information about this survey). In addition, we conducted interviews with a wide range of stakeholders including Education officials and contractors; companies responsible for administering state licensing tests; the national accreditation organization for teacher preparation programs; researchers; as well as organizations representing teachers, teacher preparation programs, officials who head state departments of education, and governors. We also conducted case studies in 5 states. In each of these states we interviewed officials from state oversight entities, officials from two or three TPPs, and officials in at least one K-12 school district (see below for more information about these case study interviews). Finally, we analyzed data collected to fulfill Higher Education Act Title II reporting requirements and Education’s related reports and technical assistance guidance (see below for more information about this analysis).

We conducted our work from March 2014 to July 2015 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Survey of 50 States and the District of Columbia

To identify how states oversee TPPs, modified their practices in response to new K-12 standards, and used Higher Education Act Title II data and Department of Education information about TPPs, we surveyed state oversight entities from all 50 states and the District of Columbia, achieving a 100 percent response rate. The survey was administered from November 2014 to March 2015. The survey used self-administered, electronic questionnaires that were posted on the Internet. We sent the survey to the official responsible for submitting each state’s Higher Education Act Title II reporting requirements and requested that this person consult with other officials in order to provide an official state response. We reviewed state responses and followed up by e-mail or telephone with select states for additional clarification and obtained corrected information for our final survey analysis. We also published survey responses in an e-publication supplemental to this report, “Teacher Preparation Programs: Survey of State Entities that Oversee Teacher Preparation Programs, an E-Supplement to GAO-15-598” (GAO-15-599SP, July, 2015).

The quality of survey data can be affected by nonsampling error. Nonsampling error includes variations in how respondents interpret questions, respondents’ willingness to offer accurate responses, and data collection and processing errors. We included steps in developing the survey, and collecting, editing, and analyzing survey data, to minimize such nonsampling error. In developing the Web survey, we pretested draft versions of the instrument with state officials in four states and consulted with officials from the Department of Education to check the clarity of the questions and the flow and layout of the survey. On the basis of the pretests, we made revisions to the survey. Further, using a web- based survey also helped remove error in our data collection effort. By allowing state officials to enter their responses directly into an electronic instrument, this method automatically created a record for each state official in a data file and eliminated the errors associated with a manual data entry process. In addition, the program used to analyze the survey data was independently verified to ensure the accuracy of this work.

Case Studies

To obtain more detailed information about how states oversee teacher preparation; the steps that states and TPPs have taken in response to new K-12 standards; and, state and TPP views about Higher Education Act Title II reporting requirements and Department of Education grants and information-sharing related to TPPs, we conducted case studies of five states: Arizona, California, Maine, Tennessee and Virginia. We selected this non-generalizable sample of states because they represent a variety of sizes (i.e., numbers of TPPs operating in the state) and approaches to overseeing TPPs (including how they use data about graduate effectiveness to evaluate TPP performance) and to reflect different types of college- and career-ready K-12 standards. As part of these case studies, we reviewed state documents and interviewed officials from the state oversight agency, two to three TPPs, and at least one K-12 school district. In total, we interviewed 14 TPPs. We ensured that this non-generalizable sample of TPPs varied in a number of ways including whether they were located within or separate from colleges and universities; were alternative route or traditional; and offered in-person or online coursework. We also ensured that the TPPs represented a range of sizes and urban, suburban, and rural locations. We selected K-12 districts that worked with these TPPs, by providing student teaching experiences, hiring graduates, or both. Interviews with state, TPP, and K- 12 district officials from Arizona, California and Maine were conducted in person and interviews with officials from Tennessee and Virginia were primarily conducted by telephone.

Analysis of Higher Education Act Title II Reports

To assess the extent that Education’s data reporting supports and encourages high quality TPPs, we reviewed select state Title II reports; Education’s reporting templates for the state reports and institutional and program reports; Education’s December 2014 proposed rule; technical assistance documents developed by Education and its contractor; and Education’s reports, data spreadsheets and web pages that compile data from Title II state reports. We interviewed Education officials and contractors responsible for collecting, analyzing and reporting such data. We also compared Education’s reports, spreadsheets and web pages— including their definitions and descriptions of the appropriate use of reported data—against federal standards for internal controls and leading practices for data-driven program management.reliability of fields in Education’s Title II data spreadsheets regarding the number of TPPs that states reported identifying as low-performing or at risk of being low-performing by: (1) reviewing agency documents, (2) interviewing relevant agency officials, and (3) testing the accuracy of these reporting fields for a non-generalizable sample of states. We determined that Education’s compilation of state data about TPPs that have been identified as low-performing was sufficiently reliable to include in our report.

Appendix II: Comments from the Department of Education

Appendix III: GAO Contact and Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Scott Spicer (Assistant Director); Barbara Steel-Lowney and Daren Sweeney (analysts-in-charge); Lucas Alvarez, and Hedieh Fusfield made key contributions to this report. Also contributing to this report were: Deborah Bland, Kate Blumenreich, Joanna Chan, Sarah Cornetto, Holly Dye, Kirsten Lauber, Ashley McCall, Sheila McCoy, Jennifer McDonald, John Mingus, Mimi Nguyen, and Tom James.