1

THE IMPACT OF PROFESSIONAL DEVELOPMENT IN FRACTIONS FOR FOURTH GRADE 

TEACHERS ON STUDENT ACHIEVEMENT AND TEACHER KNOWLEDGE IN GEORGIA 

AND SOUTH CAROLINA

OMB Clearance Request

Supporting Statement Part B

February 21, 2014

Prepared for:

NCEE Contracting Officer’s Representative: Sandra Garcia

U.S. Department of Education

Institute of Education Sciences

555 New Jersey Ave., NW, Rm. 308

Washington, DC 20208

(202) 208-7078

Submitted by:

Regional Educational Laboratory Southeast at the Florida Center for Reading Research – Florida State 

University

2010 Levy Avenue, Suite 100

Tallahassee, FL 32310

(850) 644-9352

Lab Director:

Barbara Foorman, Ph.D.

Florida State University

2010 Levy Avenue Suite 100

Tallahassee, FL 32310

(850) 644-9352

rel-se@fsu.edu

http://rel-se.fsu.edu

Principal Investigator:

Russell Gersten, Ph.D.

Instructional Research Group

4821 Katella Avenue Suite 205

Los Alamitos, CA 90720

Phone: (714) 826-9600

Fax: (714) 826-9610

http://www.inresg.org
2

SUPPORTING STATEMENT Part B

FOR PAPERWORK REDUCTION ACT SUBMISSION

Contents

B. Data Collection Procedures and Statistical Methods....................................................................4

B1. Describe (including a numerical estimate) the potential respondent universe and any 

sampling or other respondent selection methods to be used. Data on the number of 

entities (e.g., establishments, State and local government units, households, or persons) 

in the universe covered by the collection and in the corresponding sample are to be 

provided in tabular form for the universe as a whole and for each of the strata in the 

proposed sample. Indicate expected response rates for the collection as a whole. If the 

collection had been conducted previously, include the actual response rate achieved 

during the last collection .................................................................................................................4

B2. Describe the procedures for the collection of information ............................................................5

B3. Describe methods to maximize response rates and to deal with issues of non-response. 

The accuracy and reliability of information collected must be shown to be adequate for 

intended uses. For collections based on sampling, a special justification must be provided 

for any collection that will not yield "reliable" data that can be generalized to the 

universe studied ...............................................................................................................................6

B4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as 

an effective means of refining collections of information to minimize burden and improve 

utility. Tests must be approved if they call for answers to identical questions from 10 or 

more respondents. A proposed test or set of tests may be submitted for approval 

separately or in combination with the main collection of information............................................6

B5. Provide the name and telephone number of individuals consulted on statistical aspects of 

the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) 

who will actually collect and/or analyze the information for the agency........................................7
3

Appendices

Appendix A: Approved Teacher Consent & Demographic Form .................................................... A-1

Appendix B: Mathematical Knowledge for Teaching (MKT) Sample Items....................................B-1

Appendix C: Teacher Professional Development Survey .................................................................C-1

Appendix D: District/School Memorandum of Understanding (MOU) .......................................... D-1

Appendix E: Frequently Asked Questions.........................................................................................E-1

Appendix F: Approved Parent/Guardian Information Letter and Opt-out ........................................F-1

Appendix G: Approved Student Assent Form.................................................................................. G-1

Appendix H: Process for selecting DMI from possible programs.................................................... H-1

Appendix I: Placeholder for 60-Day and 30-Day Federal Register Notices ......................................I-1

Appendix J: Details for Question B1 .................................................................................................J-1

Appendix K: Details for Question B2 .............................................................................................. K-1
4

SECTION B:

B1. Describe (including a numerical estimate) the potential respondent universe and any 

sampling or other respondent selection methods to be used. Data on the number of entities 

(e.g., establishments, State and local government units, households, or persons) in the 

universe covered by the collection and in the corresponding sample are to be provided in 

tabular form for the universe as a whole and for each of the strata in the proposed sample. 

Indicate expected response rates for the collection as a whole. If the collection had been 

conducted previously, include the actual response rate achieved during the last collection.

Full details of the recruiting and sampling plan can be found in the attached design document 

included in Appendix J. The study design was approved by NCEE/IES at the end of June 2013.

This study is not intended to formally generalize to any population beyond the sample included 

in the study itself. The study focuses on grade 4 teachers and their students from Georgia and 

South Carolina. The sample consists of a non-representative group of rural, small-town and 

urban schools from 4-5 districts in each of these 2 states. Schools will be recruited from each of 

those three urbanicity categories. It is estimated that at least 82 schools, 246 teachers, and 6150

students will be enrolled in the study.

1

Eligible schools will be restricted to those with at least two fourth grade classes. By definition 

eligible schools represent 50% of the schools in each of the two states. Local Education Agencies

(LEAs) will be recruited based on meeting these very basic criteria and on the perceived level of 

commitment of school administrators to the study. This commitment will be evidenced by the 

willingness of district and school administrators to sign MOUs (See Appendix D for 

District/School Memorandum of Understanding). These schools (and their teachers and students) 

represent a sample of convenience and are not intended to generalize beyond the sample itself.

In schools which have agreed to participate, all grade 4 teachers will be invited in May – June 

2014 to participate in the study. Those returning a signed consent form will be enrolled in the 

study and be eligible for random assignment. For students, a waiver of informed consent was 

granted from the IRB, as the intervention is focused on teachers, the intervention represents a 

professional development activity typical of general LEA practice, and students will only

participate in a short post-test at the end of the year. All students in classrooms where teachers

have consented will be asked to participate, unless parents opt-out. If they agree to participate, 

 

1

Initial power estimates suggest than a minimum of 80 schools is necessary for an MDES of g = .12. 

However, 84 schools will be recruited to participate in the study to account for possible school-level 

attrition. We expect that 82 schools will remain in the study after attrition.
5

they will be included in the study (see Appendix G for the Student Assent Form). Information 

forms will be distributed to all parents at the beginning of the school-year with the opportunity to 

opt out (See Appendix F for Parent/Guardian Information Letter and Opt-out). If parents return 

the signed form, their child will be excluded from the fractions assessment. The information and 

opt-out forms will be translated and made available in Spanish as well.

B2. Describe the procedures for the collection of information including:

・ Statistical methodology for stratification and sample selection,

・ Estimation procedure,

・ Degree of accuracy needed for the purpose described in the justification,

・ Unusual problems requiring specialized sampling procedures, and

・ Any use of periodic (less frequent than annual) data collection cycles to reduce

burden.

Full details of the sampling and analytic plan can be found in the attached design document 

included in Appendix K. 

Design 

ED intends to conduct a multi-level analysis (confirmatory) of student outcomes, comparing 

students whose teachers participated in DMI, versus students whose teachers participated in 

business-as-usual professional development activities. Multi-level models will appropriately 

account for the natural clustering of students within classrooms and teachers within schools. 

Another multi-level analysis (exploratory) of teacher knowledge outcomes will be conducted to 

compare teachers who participated in DMI versus teachers who participated in business-as-usual 

professional development activities. This multi-level model will account for clustering of 

teachers within schools. Matched school-pairs will be created (i.e. blocking schools within pairs), 

then a school in each pair will be randomly assigned to the treatment condition.

A What Works Clearinghouse-type (WWC) literature review was conducted in the domain of 

mathematics professional development to determine an appropriate minimum detectable effect 

size (MDES)2to design the study to detect. Using other parameters from the research literature 

for R

2

between pretest covariates and the outcome measure, and ICCs at different levels (i.e. 

teachers/schools), a power analysis was conducted and resulted in an estimate of approximately 

80 schools required to achieve the desired MDES (g ≈ .12). However, at least 84 schools will be 

recruited to allow for some school-level attrition. Assuming 82 schools for the student outcome 

 

2 Minimum detectable effect size (MDES) stands for the minimally detectable differences in 

means between two groups, expressed in standard deviation units.
6

and three participating teachers per school (N=246 teachers), an MDES of g ≈ .33 was estimated 

for the teacher outcome.

Full details (Appendix K) about the multi-level model, and assumptions made for the power 

analysis are included from the study proposal approved by NCEE/IES.

B3. Describe methods to maximize response rates and to deal with issues of non-response. The 

accuracy and reliability of information collected must be shown to be adequate for intended 

uses. For collections based on sampling, a special justification must be provided for any 

collection that will not yield "reliable" data that can be generalized to the universe studied.

Districts and schools will be enrolled in the study based on perceived commitment, and 

completion of MOUs which explicitly delineate LEA commitments. Teachers will sign consent 

forms and be informed that it represents a contract, a commitment to participate in an important 

scientific activity. In our experience, this requirement helps retain teachers at rates of over 90%. 

The IRB for this study has granted a waiver of active consent for students. Parents will be sent a 

letter informing them of the study, of its importance, and of the very small amount of time (30-

40 minutes) their child will be asked to participate. Under such circumstances, in previous

professional development (PD) research, the research team has experienced a parent refusal rate 

of < 5%.

Study staff will carefully monitor teacher engagement through the completion of monthly teacher 

logs. Methods to reduce attrition and nonresponse will include invitation emails prompting

teachers to complete monthly logs, use of a monthly PD online survey to maximize convenience, 

and gathering of students in group settings to administer post-test. Follow-up e-mails and 

reminder calls will be sent to all non-respondents. Teachers in both the treatment and control 

group will receive a remuneration of $150 for completion of the consent/demographic form, 

teacher knowledge assessments and monthly PD surveys, which should help reduce attrition.

B4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an 

effective means of refining collections of information to minimize burden and improve utility. 

Tests must be approved if they call for answers to identical questions from 10 or more 

respondents. A proposed test or set of tests may be submitted for approval separately or in 

combination with the main collection of information.

A small feasibility study was performed with five grade 4 teachers to evaluate both instructions 

and testing time for the teacher measures.

Teacher Measures:

Five grade 4 teachers were recruited from Southern California school districts via a Craigslist 

posting. The teacher consent and demographic form and teacher PD survey were administered in 
7

sequence in a paper format. Teachers who piloted the forms were paid approximately $150 upon 

completion of the pilot.

Teachers attended proctored test sessions individually. With the exception of requesting timing 

information for the demographic form, proctors followed the protocol intended for the largescale study. Teachers were first given the demographic form to complete. Teachers wrote their 

own start and completion time on the form. After writing the completion time they were asked to 

write down any components they found confusing, or any general comments. The demographic 

forms were returned to the proctors.

The teacher professional development (PD) survey form was distributed. Proctors noted the time 

of distribution. Teachers read the instructions on the form and completed them individually. 

The MKT teacher knowledge test was not piloted, as it has been used extensively in other studies 

and consistently takes under an hour to complete.

This information was used to estimate burden hours in A12 (see Table A2). The burden on 

teachers for the data collection instruments is approximately .2 hours for the Teacher Consent 

Forms with Demographics and .2 hours for each of 9 monthly PD surveys (total 1.8 hours). For 

these measures, the total burden hours are 2 for each teacher, totaling approximately 492 hours 

for the 246 teachers in the study. Annualized, the burden on teachers is approximately 164 hours 

per year. 

Note that the MKT teacher knowledge test is strictly an assessment and not included in the 

burden estimates. A sample of MKT items is provided in Appendix B.

Student Measure (TUF):

The student measure has not been piloted as the final TUF form will not be available from the 

Center for Improving Learning of Fractions (CILF) until June 2014. CILF has scheduled a pilot 

of their measure for the Spring of 2014 for the development process where items, instructions,

and timing will be revised and refined.

Note that the student TUF measure is strictly an assessment and not included in the burden 

estimates.

B5. Provide the name and telephone number of individuals consulted on statistical aspects of 

the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who 

will actually collect and/or analyze the information for the agency.

The following individuals were consulted on the statistical aspects of this proposal:

Dr. Sybilla Beckman, University of Georgia – ph# 706-542-2548
8

Dr. John Deke, Mathematica Policy Research – ph# 609-275-2230

Dr. Mike Garet, American Institutes for Research – ph# 202-403-5000

Dr. Russell Gersten, Instructional Research Group – ph# 714-826-9600

Dr. Nathan Jones, Boston University – ph# 617-353-3295

Dr. Yaacov Petscher, Florida State University – ph# 850-644-0327

Dr. Eric Rolfhus, Instructional Research Group – ph# 714-826-9600

Dr. Mengli Song, American Institutes for Research – ph# 202-403-5000

The following individual designed the data collection:

Dr. Russell Gersten, Instructional Research Group

The following individual will oversee the data collection:

Dr. Russell Gersten, Instructional Research Group

The following individual(s) will analyze the data:

Dr. Tran Keys, Instructional Research Group – ph# 714-826-9600

Dr. Rebecca Newman-Gonchar, Instructional Research Group – ph# 714-826-9600

Dr. Madhavi Jayanthi, Instructional Research Group – ph# 714-826-9600

Ms. Kelly Haymond, Instructional Research Group – ph# 714-826-9600

Dr. Mengli Song, American Institutes for Research – ph# 202-403-5000