1

THE IMPACT OF PROFESSIONAL DEVELOPMENT IN FRACTIONS FOR FOURTH GRADE 

TEACHERS ON STUDENT ACHIEVEMENT AND TEACHER KNOWLEDGE IN GEORGIA 

AND SOUTH CAROLINA

OMB Clearance Request

Supporting Statement Part B

November 26, 2013

Prepared for:

NCEE Contracting Officer’s Representative: Sandra Garcia

U.S. Department of Education

Institute of Education Sciences

555 New Jersey Ave., NW, Rm. 308

Washington, DC 20208

(202) 208-7078

Submitted by:

Regional Educational Laboratory Southeast at the Florida Center for Reading Research – Florida State 

University

2010 Levy Avenue, Suite 100

Tallahassee, FL 32310

(850) 644-9352

Lab Director:

Barbara Foorman, Ph.D.

Florida State University

2010 Levy Avenue Suite 100

Tallahassee, FL 32310

(850) 644-9352

rel-se@fsu.edu

http://rel-se.fsu.edu

Principal Investigator:

Russell Gersten, Ph.D.

Instructional Research Group

4821 Katella Avenue Suite 205

Los Alamitos, CA 90720

Phone: (714) 826-9600

Fax: (714) 826-9610

http://www.inresg.org
2

SUPPORTING STATEMENT Part B

FOR PAPERWORK REDUCTION ACT SUBMISSION

Contents

B. Data Collection Procedures and Statistical Methods....................................................................4

B1. Describe (including a numerical estimate) the potential respondent universe and any 

sampling or other respondent selection methods to be used. Data on the number of 

entities (e.g., establishments, State and local government units, households, or persons) 

in the universe covered by the collection and in the corresponding sample are to be 

provided in tabular form for the universe as a whole and for each of the strata in the 

proposed sample. Indicate expected response rates for the collection as a whole. If the 

collection had been conducted previously, include the actual response rate achieved 

during the last collection .................................................................................................................4

B2. Describe the procedures for the collection of information ............................................................5

B3. Describe methods to maximize response rates and to deal with issues of non-response. 

The accuracy and reliability of information collected must be shown to be adequate for 

intended uses. For collections based on sampling, a special justification must be provided 

for any collection that will not yield "reliable" data that can be generalized to the 

universe studied ...............................................................................................................................6

B4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as 

an effective means of refining collections of information to minimize burden and improve 

utility. Tests must be approved if they call for answers to identical questions from 10 or 

more respondents. A proposed test or set of tests may be submitted for approval 

separately or in combination with the main collection of information............................................6

B5. Provide the name and telephone number of individuals consulted on statistical aspects of 

the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) 

who will actually collect and/or analyze the information for the agency........................................7
3

Appendices Referenced in Part B

Appendix F: Details for Question B1

Appendix G: Details for Question B2
4

SECTION B:

B1. Describe (including a numerical estimate) the potential respondent universe and any 

sampling or other respondent selection methods to be used. Data on the number of entities 

(e.g., establishments, State and local government units, households, or persons) in the 

universe covered by the collection and in the corresponding sample are to be provided in 

tabular form for the universe as a whole and for each of the strata in the proposed sample. 

Indicate expected response rates for the collection as a whole. If the collection had been 

conducted previously, include the actual response rate achieved during the last collection.

Full details of the recruiting and sampling plan can be found in the attached design document 

included in Appendix F. The study design was approved by NCEE/IES at the end of June 2013.

This study is not intended to formally generalize to any population beyond the sample included 

in the study itself. The study focuses on grade 4 teachers and their students from Georgia and 

South Carolina. The sample consists of a non-representative group of rural, small-town and 

urban schools from 6 districts in these 2 states. Schools will be recruited from each of those three 

urbanicity categories. It is estimated that at least 80 schools, 240 teachers, and 4560 students 

will be enrolled in the study. (Initial power estimates suggest than a minimum of 80 schools is 

necessary for an MDES of g = .12. However, 84 schools will be recruited to participate in the

study to account for possible school-level attrition.)

Eligible schools will be restricted to those below the state median in achievement as based on the 

prior-year school-wide state assessment results. By definition eligible schools represent 50% of 

the schools in each of the two states. Local Education Agencies (LEAs) will be recruited based 

on meeting these very basic criteria and on the perceived level of commitment of school 

administrators to the study. This commitment will be evidenced by the willingness of district and 

school administrators to sign MOUs (See Appendix D for District/School Memorandum of 

Understanding). These schools (and their teachers and students) represent a sample of 

convenience and are not intended to generalize beyond the sample itself.

In schools which have agreed to participate, all grade 4 teachers will be invited in May – June 

2014 to participate in the study. Those returning a signed consent form will be enrolled in the 

study and be eligible for random assignment. For students, a waiver of informed consent was 

granted from the IRB, as the intervention is focused on teachers, the intervention represents a 

professional development activity typical of general LEA practice, and students will only

participate in a short post-test at the end of the year. All students in classrooms where teachers

have consented will participate, unless parents opt-out. Information forms will be distributed to 

all parents at the beginning of the school-year with the opportunity to opt out (See Appendix E 

for Parent/Guardian Information Letter and Opt-out). If parents return the signed form, their 
5

child will be excluded from the fractions assessment. The information and opt-out forms will be 

translated and made available in Spanish as well.

B2. Describe the procedures for the collection of information including:

・ Statistical methodology for stratification and sample selection,

・ Estimation procedure,

・ Degree of accuracy needed for the purpose described in the justification,

・ Unusual problems requiring specialized sampling procedures, and

・ Any use of periodic (less frequent than annual) data collection cycles to reduce

burden.

Full details of the sampling and analytic plan can be found in the attached design document 

included in Appendix G. 

Design 

ED intends to conduct a multi-level analysis (confirmatory) of student outcomes, comparing 

students whose teachers participated in DMI, versus students whose teachers participated in 

business-as-usual professional development activities. Multi-level models will appropriately 

account for the natural clustering of students within classrooms and teachers within schools. 

Another multi-level analysis (exploratory) of teacher knowledge outcomes will be conducted to 

compare teachers who participated in DMI versus teachers who participated in business-as-usual 

professional development activities. This multi-level model will account for clustering of 

teachers within schools. Matched school-pairs will be created (i.e. blocking schools within pairs), 

then a school in each pair will be randomly assigned to the treatment condition.

A What Works Clearinghouse-type (WWC) literature review was conducted in the domain of 

mathematics professional development to determine an appropriate minimum detectable effect 

size (MDES)1to design the study to detect. Using other parameters from the research literature 

for R2between pretest covariates and the outcome measure, and ICCs at different levels (i.e. 

teachers/schools), a power analysis was conducted and resulted in an estimate of approximately

80 schools required to achieve the desired MDES (g ≈ .12). However, at least 84 schools will be 

recruited to allow for some school-level attrition. Assuming 80 schools for the student outcome 

and three participating teachers per school (N=240 teachers), an MDES of g ≈ .33 was estimated 

for the teacher outcome.

 

1 Minimum detectable effect size (MDES) stands for the minimally detectable differences in 

means between two groups, expressed in standard deviation units.
6

Full details (Appendix G) about the multi-level model, and assumptions made for the power 

analysis are included from the study proposal approved by NCEE/IES.

B3. Describe methods to maximize response rates and to deal with issues of non-response. The 

accuracy and reliability of information collected must be shown to be adequate for intended 

uses. For collections based on sampling, a special justification must be provided for any 

collection that will not yield "reliable" data that can be generalized to the universe studied.

Districts and schools will be enrolled in the study based on perceived commitment, and 

completion of MOUs which explicitly delineate LEA commitments. Teachers will sign consent 

forms and be informed that it represents a contract, a commitment to participate in an important 

scientific activity. In our experience, this requirement helps retain teachers at rates of over 90%. 

The IRB for this study has granted a waiver of active consent for students. Parents will be sent a 

letter informing them of the study, of its importance, and of the very small amount of time (30-

40 minutes) their child will be asked to participate. Under such circumstances, in previous

professional development (PD) research, the research team has experienced a parent refusal rate 

of < 5%.

Study staff will carefully monitor teacher engagement through the completion of monthly teacher 

logs. Methods to reduce attrition and nonresponse will include invitation emails prompting 

teachers to complete monthly logs, use of a monthly PD online survey to maximize convenience, 

and gathering of students in group settings to administer post-test . Follow-up e-mails and 

reminder calls will be sent to all non-respondents. Teachers in both the treatment and control 

group will receive a remuneration of $150 for completion of the consent/demographic form, 

teacher knowledge assessments and monthly PD surveys, which should help reduce attrition.

B4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an 

effective means of refining collections of information to minimize burden and improve utility. 

Tests must be approved if they call for answers to identical questions from 10 or more 

respondents. A proposed test or set of tests may be submitted for approval separately or in 

combination with the main collection of information.

A small feasibility study was performed with five grade 4 teachers to evaluate both instructions 

and testing time for the teacher measures.

Teacher Measures:

Five grade 4 teachers were recruited from Southern California school districts via a Craigslist 

posting. The teacher consent and demographic form and teacher PD log were administered in 

sequence in a paper format. Teachers who piloted the forms were paid approximately $150 upon 

completion of the pilot.
7

Teachers attended proctored test sessions individually. With the exception of requesting timing 

information for the demographic form, proctors followed the protocol intended for the largescale study. Teachers were first given the demographic form to complete. Teachers wrote their 

own start and completion time on the form. After writing the completion time they were asked to 

write down any components they found confusing, or any general comments. The demographic 

forms were returned to the proctors.

The teacher professional development (PD) survey form was distributed. Proctors noted the time 

of distribution. Teachers read the instructions on the form and completed them individually. 

The MKT teacher knowledge test was not piloted, as it has been used extensively in other studies 

and consistently takes under an hour to complete.

This information was used to estimate burden hours and future payments in A12 (see Table A2).

The burden on teachers for the data collection instruments is approximately .2 hours for the 

Teacher Consent Forms with Demographics, 4 hours for the Teacher Assessment (Pre & Post), 

and 1.8 hours for the monthly PD survey. The total burden hours are 6 for each teacher, totaling 

approximately 1512 hours for the 252 teachers in the study.

Student Measure (TUF):

The student measure has not been piloted as the final TUF form will not be available from the 

Center for Improving Learning of Fractions (CILF) until June 2014. CILF has scheduled two

iterative pilots (beginning Winter 2014) for the development process where items, instructions,

and timing will be repeatedly revised and refined.

B5. Provide the name and telephone number of individuals consulted on statistical aspects of 

the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who 

will actually collect and/or analyze the information for the agency.

The following individuals were consulted on the statistical aspects of this proposal:

Dr. Sybilla Beckman, University of Georgia – ph# 706-542-2548

Dr. John Deke, Mathematica Policy Research – ph# 609-275-2230

Dr. Mike Garet, American Institutes for Research – ph# 202-403-5000

Dr. Russell Gersten, Instructional Research Group – ph# 714-826-9600

Dr. Nathan Jones, Boston University – ph# 617-353-3295

Dr. Yaacov Petscher, Florida State University – ph# 850-644-0327

Dr. Eric Rolfhus, Instructional Research Group – ph# 714-826-9600

Dr. Mengli Song, American Institutes for Research – ph# 202-403-5000

The following individual designed the data collection:

Dr. Russell Gersten, Instructional Research Group
8

The following individual will oversee the data collection:

Dr. Russell Gersten, Instructional Research Group

The following individual(s) will analyze the data:

Dr. Tran Keys, Instructional Research Group – ph# 714-826-9600

Dr. Rebecca Newman-Gonchar, Instructional Research Group – ph# 714-826-9600

Dr. Madhavi Jayanthi, Instructional Research Group – ph# 714-826-9600

Ms. Kelly Haymond, Instructional Research Group – ph# 714-826-9600

Dr. Mengli Song, American Institutes for Research – ph# 202-403-5000