1

SUPPORTING	STATEMENT	B:

REQUEST	FOR	CLEARANCE	OF	INFORMATION	

COLLECTION	FORMS	FOR

“Understanding	the	Impact	of	Providing	

Information	to	Parents	about	the	Role	of	Algebra	

II:	An	Opportunistic	Study”

January 2014

Submitted to: Submitted by:

U.S. Department of Education SEDL

Institute of Education Sciences 4700 Mueller Blvd.

555 New Jersey Ave. NW, Rm. 308 Austin, TX 78723

Washington, DC 20208 Phone: (800) 476-6861

4700 Mueller Blvd. Austin, TX 78723

800-476-6861

www.relsouthwest.org

This publication was prepared for the Institute of Education Sciences (IES) under contract ED-IES-12-C-00012 by 

Regional Educational Laboratory Southwest, administered by SEDL. The content of the publication does not 

necessarily reflect the views or policies of IES or the U.S. Department of Education, nor does mention of trade names, 

commercial products, or organizations imply endorsement by the U.S. government. The publication is in the public 

domain. Authorization to reproduce in whole or in part for educational purposes is granted.
2

Contents

1. Respondent Universe and Sampling Methods ........................................................................... 6

2. Description of Procedures for the Collection of Information..................................................... 8

Data Sources................................................................................................................................ 8

3. Description of Procedures for Maximizing Response Rates....................................................... 9

4. Description of Tests, Procedures, and Methods....................................................................... 10

5. Individuals Consulted on Statistical Aspects of the Design and Key Staff................................ 11
3
4

The U.S. Department of Education (ED) requests clearance for data collection under the Office 

of Management and Budget (OMB) clearance agreement (OMB number [IES to complete]) for 

activities related to the Regional Educational Laboratory (REL) Program. ED, in consultation with 

SEDL, intends to conduct a timely, policy relevant opportunistic experiment to determine if 

directly providing parents/guardians, prior to students’ selection of their courses, with 

information on the importance of completing algebra II for college access and success has an 

impact on the percentage of students who enroll in and complete algebra II by the end of their 

junior year.

In 2013, the governor of Texas signed House Bill (HB) 5—the Foundation High School Program—

into law. HB changed the high school graduation requirements and reduced the number of 

required state secondary school exams for public high school students in Texas. Prior to 

enactment of HB 5, high school students were required take four courses each in English, 

mathematics (including algebra II), science, and social studies—earning the credits they needed 

to be admitted to most state universities and colleges. Now, only one of the three new 

graduation plans1, created for the Foundation High School Program requires completion of 

algebra II in order to graduate. Despite the change in high school graduation requirements, the 

public universities in Texas will still require students to have completed algebra I, geometry, 

and algebra II in order to gain admission

Thus, a relevant question is whether providing parents/guardians with information about the 

importance of completing algebra II will help prevent students who aspire to postsecondary 

degrees from opting to skip algebra II in high school. This study builds on research being 

conducted in a companion study that investigates the trends in the percentage of students in a 

school who complete algebra II by the end of their junior year for the graduating classes of 

2009-10 through 2016-17. 

The results of this study will provide Texas Education Agency (TEA) and the Texas Higher 

Education Coordinating Board (THECB) with information about the role that information 

dissemination can play in influencing students’ course selection and completion. Information 

from this study will provide TEA with valuable information that will inform their attempts to 

stem the number of students who wish to go on to postsecondary education, but may not be 

aware of the importance completing algebra II has toward this goal. The intervention is 

relatively inexpensive, so if it is successful, it would demonstrate the benefit of low-cost 

interventions for influencing student course selection. TEA will be able to use this information 

to answer questions about changes in course taking patterns and course failure rates that may 

 

1

The STEM endorsement included in the Foundation plus Endorsement Plan also requires completion of algebra II.
5

arise from parents, education practitioners and administrators, policymakers, and researchers. 

It will also allow them to respond to inquiries regarding any variation in how districts responded 

to the HB 5 graduation requirements.

Additionally, if the results of the study show that schools in which students’ parents were 

provided with information about the role of algebra II in college access and success have higher 

algebra II completion rates, TEA could recommend that districts conduct an information 

campaign similar to this one. The Texas Association for School Administrators has already 

expressed an interest in providing the materials to its members—all school principals in Texas. 

Similarly, if this study shows positive results, TEA could produce materials for districts to 

disseminate to parents when statewide policy changes occur. This would allow for a clear and 

consistent message that is currently lacking, as schools and districts must create their own 

materials based on their own interpretations of the policy. 

Finally, results of our study will provide evidence to other states about the impact of providing 

parents/guardians with information about course completion. Due to the size and demographic 

makeup, many states look to Texas for guidance when making decisions about implementing 

statewide policies and programs. The findings of this study should be of particular interest to 

those states. 

OMB approval is being requested to recruit public high schools in Texas to participate in this 

study, as well as to collect extant student data records from TEA.

This study will address the following research questions: 

Confirmatory Research Questions

1. Does providing information about the role of algebra II in college access and success

to the parents/guardians of students entering grade 9 after the enactment of House 

Bill 5 have an impact on:

a. The percentage of students in a school who enroll in algebra II by the end of 

their junior year?

b. The percentage of students in a school who complete algebra II by the end of 

their junior year?

Exploratory Research Questions

2. Are there differential impacts of providing information about the role of algebra II in 

college access and success on students’ algebra II enrollment for:
6

a. Schools with a high percentage of minority students? 

b. Schools with a high percentage of low-income students?

c. Schools with a high percentage of low-achieving students?

3. Are there differential impacts of providing information about the role of algebra II in 

college access and success on students’ algebra II completion for:

a. Schools with a high percentage of minority students? 

b. Schools with a high percentage of low-income students?

c. Schools with a high percentage of low-achieving students?

4. To what extent are schools implementing the intervention with fidelity?

Data collection for this project consists of recruitment activities (e.g., telephone calls, email 

messages) and collection of extant data files from TEA. Specifically, in this OMB clearance 

package, ED is requesting clearance for the following data collection approach:

 Recruitment activities, including telephone calls and email messages, targeted at public 

school districts and public high schools in Texas

 Extant data collection consisting of student records data to be obtained from the Texas 

Education Agency

ED believes that the data collections for which clearance is being requested represent the bare

minimum necessary to determine if providing parents/guardians, prior to students’ selection of 

their courses, with information on the importance of completing algebra II for college access 

and success has an impact on the percentage of students who enroll in and complete algebra II 

by the end of their junior year.

1. Respondent Universe and Sampling Methods

Respondent Universe

In this study, ED’s contractor, REL Southwest will investigate differences in algebra II enrollment 

and completion by the end of the junior year between students in study schools who receive 

information about the role of algebra II for college access (treatment group) and students in 

study schools who do not (control group). To do so, ED’s contractor will follow the grade 9 

students who begin high school in study schools during the 2014–15 school year—the first 

cohort of students required to graduate under the state’s new diploma plans—and assess 
7

differences in algebra II enrollment and completion by the end of junior year between students 

in treatment schools and students in control schools. All grade 9 students, including special 

education and English learners who enroll in either algebra I or geometry will be included in the 

study. All public high schools in the state of Texas that are not automatically placing their 

incoming grade 9 students into the Distinguished graduation plan2 will be recruited to 

participate in the study. 

Power

For this study, ED’s contractor wishes to be able to detect at least a 5 percentage point 

difference in algebra II completion between our treatment and control groups—it is not

believed that a difference smaller than this would be meaningful. We used Optimal Design 

version 3.01 (Spybrook, Bloom, Congdon, Hill, Liu, Martinez & Raudenbush, 2013) to determine 

the number of schools needed for our study. Within the software package, ED’s contractor 

estimated a two-level cluster RCT design with binary outcomes3, with the following 

assumptions:

Probability of success in treatment group: 0.70

Probability of success in control group: 0.65

Number of students in each school: 170 (median number of students in grade 9 in Texas 

public high schools)

Plausible value range: 0.60—0.90

Power: 0.80

Thus, for this study, ED’s contractor determined that it will need to include a total of 144 

schools in the RCT, with 50 percent of the schools allocated to the treatment and control 

groups, to be able to detect a 5 percentage point difference between treatment and control 

schools4. 

Random Assignment

Once schools have been consented into the study, ED’s contractor will use cluster analysis to 

assign schools to strata for randomization. Cluster analysis allows for creation clusters of 

schools that are similar across multiple variables—rather than just one or two—which should 

increase precision. Conducting random assignment within strata reduces the likelihood of 

obtaining treatment and control groups that differ significantly on covariates of interest. For 

this study, we will use k-means clustering, which creates k strata and then assigns each unit to 

 

2

The Distinguished graduation plan requires students to complete algebra II in order to earn a Distinguished Level 

of Achievement. 

3

The Optimal Design software does not allow for inclusion of estimates of the amount of variation taken into 

account by either level-1 or level-2 variables. As such, there is no ICC estimate, and the power for this study is likely 

an underestimate. 

4

A difference of less than 5 percentage points between treatment and control groups is not meaningful.
8

one stratum, so that a measure of similarity is maximized. Since the covariates are both 

categorical and continuous, how close the units are to one another will be determined using 

the general similarity measure (distance) outlined in Glower (1971). To determine the optimal 

number of strata, ED’s contractor will generate and evaluate results for up to 20 strata and then 

plot the between-cluster variability in an elbow plot, which is similar to a scree plot in principal 

components analysis. ED’s contractor will use this plot to determine where the rate of change 

in the proportion of variance accounted for by each additional strata slows. There are 6 schoollevel covariates that are likely to predict whether or not students will complete algebra II. These 

covariates, shown in Table 1, will be used in the cluster analysis to define strata.

Table 1. Covariates used for cluster analysis

Percent Black, White, 

Hispanic, Other

School size Percent special education

Average incoming test scores 

in math and reading (that is, 

grade 8)

School region (such as

suburban, rural, urban)

Percent English language 

learners

After the schools have been placed into strata, one-half of the high schools selected 

from each strata will be randomly assigned to serve in the treatment group, while the other 

half of the schools within each strata will serve as the control group for the impact study.

2. Description of Procedures for the Collection of Information

ED’s contractor will recruit schools, manage data collection and ensure quality and timeliness. 

ED’s contractor proposes to collect secondary data only. This study will utilize extant studentlevel data collected and archived by TEA. OMB clearance is being sought for extant data 

collection, as well as study recruitment activities. 

Data Sources

To answer research questions 1 through 3, ED’s contractor will utilize longitudinal student- and 

school-level data sets available from TEA. Specifically, the study will use data from TEA’s Public 

Education Information Management System (PEIMS), statewide assessment files, and Texas 

Academic Performance Report (TAPR) files. PEIMS contains student-level data on student 

enrollment and demographic characteristics, special program participation, and course 

completion data. TAPR contains organizational data for schools and districts. To answer 

research question 4, ED’s contractor will use the envelopes mailed to the contractor by 

participating high schools. 
9

3. Description of Procedures for Maximizing Response Rates

ED is committed to obtaining complete data for this evaluation. This impact study relies on 

administrative data. ED’s contractor anticipates a 95-percent response rate from TEA on 

student variables in the administrative data. That is, ED’s contractor assumes 5 percent or less 

of the data will be missing on key variables. ED’s contractor has a strong working relationship 

with representatives from TEA. All administrative data files will be reviewed for consistency and 

completeness. The response rate from schools regarding the returned envelopes will be, by 

definition, 100 percent. 

Based on its prior experience with recruiting schools to participate in RCTs, ED’s contractor 

expects to have minimal difficulty recruiting high schools to participate in the study. ED’s 

contractor will utilize a two-part process to make schools aware of the project, invite them to 

participate, and answer questions that schools might have about the study. 

Recruitment will be conducted via email and telephone. Around the third week of October, ED’s 

contractor will send an email message to all high school principals in the state introducing the 

study and inviting them to participate. The emails will outline the goals of the study, the 

content of the informational materials, and the school’s role in information dissemination if 

randomized into the treatment condition. Principals will be made aware that this study is a 

randomized controlled trial (RCT) and that only half of participating schools will receive the 

informational materials and be involved in information dissemination initially; however, they 

will also be informed that their schools will receive the materials, if desired, after the 

conclusion of the study—the 2017-18 school year. Interested principals will be directed to 

contact REL Southwest for additional information. A follow-up email will be sent to all nonresponding principals one week after the initial email, with a second follow-up e-mail sent to all 

non-responding principals one week after the first follow-up. One week after distribution of the

second follow-up email, if enough schools have not agree to participate in the study, REL 

Southwest will begin conducting telephone recruiting calls. REL Southwest anticipates that it

will be able to recruit about 80 high schools via email. REL Southwest also anticipates needing 

to contact approximately 150 high school principals via telephone in order to recruit an 

additional 64 high schools. To recruit additional high schools, they will begin by contacting 

other high schools within the same districts as those that have already agreed to participate. If 

this does not result in a sufficient number of high schools, random numbers will be assigned to 

each of the high schools in Texas that have not responded to the recruitment emails, and the 

schools will be sorted in chronological order by these random numbers. Starting with the 

smallest number, high schools will be contacted via telephone and asked to participate in the 

study.
10

During the recruiting process, REL Southwest researchers will ensure that principals understand 

the nature of the study and the responsibilities of participating schools. Once principals have 

agreed to participate in the study, they will be asked to sign a memorandum of 

understanding/consent form indicating that their school agrees to participate in the study. 

While ED’s contractor does not intend to provide principals or schools with incentives to 

participate in the study, the cost of materials and material dissemination will be reimbursed. As 

part of the recruiting process principals will be informed that they will be provided with all of 

the materials necessary to distribute the informational brochures. In January, principals will be 

asked to provide ED’s contractor with the number of current students who entered grade 9 

during the 2014–15 school year. Principals will be provided with enough study materials for 

that number of students, plus a few extra. That is, ED’s contractor will send each participating 

school a box of postage-paid envelopes containing the informational brochures, and we will ask 

principals to have their administrative staff create mailing labels, affix the labels to the 

envelopes, and have the envelopes picked up during the regular mail service. Schools will not 

need to supply any resources other than staff time. 

Along with these materials, schools will be provided with one postage-paid envelope addressed 

to ED’s contractor. The envelope will contain the name of the school from which it was sent, 

and schools will be asked to mail this envelope at the same time as the informational materials. 

ED’s contractor will use these envelopes to determine which high schools have mailed the 

informational materials and when they were sent. If schools have not mailed the informational 

materials within one week of the mailing deadline, ED’s contractor will conduct reminder 

telephone calls to high school principals. 

4. Description of Tests, Procedures, and Methods

Data analyses for this study will consist of a confirmatory analysis examining the impact of 

providing parents/guardians with information about the role of algebra II in college access and 

success on algebra II course enrollment and completion by the end of junior year and a set of 

exploratory analyses investigating differences in the effect of providing information about the 

role of algebra II for college access and success on students’ mathematics course completion for 

various subgroups. 

To answer our confirmatory research question, we will utilize multilevel models to compare 

algebra II enrollment and completion rates by the end of junior year for students in schools that 

received information about the role of algebra II in college access and success and students in 

schools that did not receive this information. To answer our exploratory research questions, we 

will include interaction terms into the model used to analyze our confirmatory research 
11

question in order to look for differential impacts of providing information about the role of 

algebra II for college access and success for schools with high minority populations and high 

low-income populations. We will also assess whether there are differential impacts in schools 

with a higher percentage of low-achieving schools.

Although the specific email messages and telephone scripts that will be used in this study have 

not been pilot tested, they are modeled email messages and telephone scripts that ED’s 

contractor has used for recruitment in other similar randomized controlled trials. Only minor 

changes have been made to reflect the nuances of this study. The time burden estimate used 

here is based on experiences from these similar studies. 

The informational materials to be used in the study were pilot tested with parents of students 

who were enrolled in grade 9 during the 2013-14 school year—these parents and students 

cannot be included in this study. Several different versions of the brochure were pilot tested 

were pilot tested with nine parents. These parents were asked to review the brochure. Then, 

they were asked to identify which brochure was the easiest to read, easiest to understand, 

most eye catching, and most informative. We used feedback from these parents to create the 

final version of the brochure. 

5. Individuals Consulted on Statistical Aspects of the Design and Key Staff

The following individuals were consulted on the statistical, data collection, and analytic aspects 

of this study through REL Southwest’s Technical Working Group (TWG):

Dan Goldhaber, Ph.D.

Director, CALDER (National Center for Analysis of Longitudinal Data in Education Research)

Vice President, American Institutes for Research (AIR) 

Director, Center for Education Data & Research (CEDR), University of Washington Bothell

Co-Editor, Education Finance and Policy

3876 Bridge Way N,Suite 201

Seattle, WA 98103

Ph: 206-547-1562

Fax: 206-547-1641

E-mail: dgoldhab@uw.edu

Geoffrey Borman, Ph.D.

Professor of Education, University of Wisconsin—Madison

Deputy Director of the University of Wisconsin's Predoctoral Interdisciplinary Research Training 

Program
12

Senior Researcher, Consortium for Policy Research in Education.

348 Education Building

1000 Bascom Mall

Madison, WI 53706-1326

Ph: 608-263-3688

Fax: 608-265-3135

E-mail: gborman@education.wisc.edu

Johannes M. (Hans) Bos, Ph.D.

Vice President and Program Director, International Development, Evaluation, and Research 

(IDER) Program

American Institutes for Research

2800 Campus Drive, Suite 200

San Mateo, CA 94403

Ph: 650-843-8100

Fax: 650-843-8200

E-mail: jbos@air.org

W. Steven Barnett, Ph.D.

Board of Governors Professor and Director of the National Institute for Early Education 

Research

Rutgers University

73 Easton Avenue

New Brunswick, NJ 08901

Ph: 848-932-4350 x23132

Fax: 732-932-4360

E-mail: sbarnett@nieer.org