1

THE IMPACT OF PROFESSIONAL DEVELOPMENT IN FRACTIONS FOR FOURTH GRADE 

TEACHERS ON STUDENT ACHIEVEMENT AND TEACHER KNOWLEDGE IN GEORGIA 

AND SOUTH CAROLINA

OMB Clearance Request

Supporting Statement Part A

February 21, 2014

Prepared for:

NCEE Contracting Officer’s Representative: Sandra Garcia

U.S. Department of Education

Institute of Education Sciences

555 New Jersey Ave., NW, Rm. 506C

Washington, DC 20208

(202) 219-1597

Submitted by:

Regional Educational Laboratory Southeast at the Florida Center for Reading Research – Florida State 

University

2010 Levy Avenue, Suite 100

Tallahassee, FL 32310

(850) 644-9352

Lab Director:

Barbara Foorman, Ph.D.

Florida State University

2010 Levy Avenue Suite 100

Tallahassee, FL 32310

(850) 644-9352

rel-se@fsu.edu

http://rel-se.fsu.edu

Principal Investigator:

Russell Gersten, Ph.D.

Instructional Research Group

4821 Katella Avenue Suite 205

Los Alamitos, CA 90720

Phone: (714) 826-9600

Fax: (714) 826-9610

http://www.inresg.org
2

Contents

Supporting Statement for Paperwork Reduction Act Submission

A. Justification ....................................................................................................................................

A1. Explain the circumstances that make the collection of information necessary. Identify 

any legal or administrative requirements that necessitate the collection. Attach a copy of 

the appropriate section of each statute and regulation mandating or authorizing the 

collection of information .......................................................................................................... 5

A2. Indicate how, by whom, and for what purpose the information is to be used. Except for a 

new collection, indicate the actual use the agency has made of the information received 

from the current collection....................................................................................................... 7

A3. Describe whether, and to what extent, the collection of information involves the use of 

automated, electronic, mechanical, or any other technological collection techniques or 

forms of information technology, e.g. permitting electronic submission of responses, and 

the basis for the decision of adopting the means of collection. Also describe any 

consideration of using information technology to reduce burden ........................................... 12

A4. Describe efforts to identify duplication. Show specifically why any similar information 

already available cannot be used or modified for use of the purposes described in Item 2 

above ........................................................................................................................................ 12

A5. If the collection of information impacts small businesses or other small entities (Item 5 of 

OMB Form 83-1), describe any methods used to minimize burden......................................... 13

A6. Describe the consequences to Federal program or policy activities if the collection is not 

conducted or is conducted less frequently, as well as any technical or legal obstacles to 

reducing burden ....................................................................................................................... 13

A7. Explain any special circumstances that would cause an information collection to be 

conducted in a manner inconsistent with Section 1320.5(d)(2) of the federal regulations ..... 14

A8. Federal Register Comments and Persons Consulted Outside of the Agency........................... 15

A9. Explain any decision to provide any payment or gift to respondents, other than 

remuneration of contractors or grantees................................................................................. 16

A10. Describe any assurances of confidentiality provided to respondents and the basis for the 

assurance in statute, regulation, or agency policy................................................................... 16
3

A11. Provide additional justification for any questions of a sensitive nature, such as sexual 

behavior and attitudes, religious beliefs, and other matters that are commonly considered 

private. The justification should include the reasons why the agency considers the 

questions necessary, the specific uses to be made of the information, the explanation to 

be given to persons from whom the information is requested, and any steps to be taken to 

obtain their consent.................................................................................................................. 17

A12. Provide estimates of the hour burden of the collection of information.................................... 17

A13. Describe any other costs to respondents or record keepers .................................................... 19

A14. Provide estimates of annualized cost to the Federal government. Also provide a 

description of the method used to estimate cost, which should include quantification of 

hours, operational expenses (such as equipment, overhead, printing, and support staff), 

and any other expenses that would not have been incurred without this collection 

information. Agencies may aggregate cost estimates from Items 12, 13, and 14 in a single 

table.......................................................................................................................................... 19

A15. Describe any changes in the burden from prior approvals ..................................................... 20

A16. For collections of information whose results will be published, outline plans for 

tabulation and publication. Address any complex analytical techniques that will be used. 

Provide the time schedule for the entire project, including beginning and ending dates of 

the collection of information, completion of the report, publication dates, and other 

actions...................................................................................................................................... 20

A17. Describe arrangements for displaying the number provided by OMB and its expiration 

date........................................................................................................................................... 21

A18. Exceptions to Certification Statement...................................................................................... 22

References............................................................................................................................................23
4

Appendices

Appendix A: Approved Teacher Consent & Demographic Form .................................................... A-1

Appendix B: Mathematical Knowledge for Teaching (MKT) Sample Items....................................B-1

Appendix C: Teacher Professional Development Survey .................................................................C-1

Appendix D: District/School Memorandum of Understanding (MOU) .......................................... D-1

Appendix E: Frequently Asked Questions.........................................................................................E-1

Appendix F: Approved Parent/Guardian Information Letter and Opt-out ........................................F-1

Appendix G: Approved Student Assent Form.................................................................................. G-1

Appendix H: Process for selecting DMI from possible programs.................................................... H-1

Appendix I: Placeholder for 60-Day and 30-Day Federal Register Notices ......................................I-1
5

SUPPORTING STATEMENT

FOR PAPERWORK REDUCTION ACT SUBMISSION

A. Justification

A1. Explain the circumstances that make the collection of information necessary. Identify 

any legal or administrative requirements that necessitate the collection. Attach a copy of the 

appropriate section of each statute and regulation mandating or authorizing the collection of 

information.

The dismal state of student mathematics achievement, especially in fractions, has been well 

documented in research literature (e.g., Ma, 1999; Yanik, Helding, & Baek, 2006); national 

reports (e.g., National Mathematics Advisory Panel [NMAP], 2008); the IES practice guide on 

fractions (Siegler et al., 2010); and the results of national assessments (National Assessment of 

Educational Progress [NAEP], 2011). The National Mathematics Advisory Panel (NMAP) 

reported that “difficulty with fractions (including decimals and percent) is pervasive and is a 

major obstacle to further progress in mathematics, including algebra” (2008, p. xix). In fact, 

Siegler and his colleagues (2012) recently found that knowledge of fractions at age 10 is 

predictive of algebra knowledge and overall mathematics achievement in high school, above and 

beyond the effects of general intellectual ability, other mathematical knowledge, and family 

background.1There is, in general, a consensus among researchers, educators, and policy makers 

that improving American students’ knowledge of fractions should be made a major educational 

priority. In fact, the National Center of Educational Research at IES recently awarded a $10 

million dollar grant to fund a five-year research and development center aimed at understanding 

difficulties students have with fractions and developing effective interventions for struggling 

learners in this area (Center for Improving Learning of Fractions, 2013).

Unfortunately, research examining teacher knowledge in mathematics has repeatedly 

documented that elementary teachers often have limited mathematical knowledge, particularly in 

the area of fractions (Hill et al., 2008; Hill, Kapitula, & Umland, 2011; Hill, Rowan, & Ball, 

2005; Ma, 1999). Clearly, providing professional development that focuses on addressing 

teachers’ mathematical content knowledge underlying the Common Core State Standards

(CCSS) is a necessary step for full and effective implementation of CCSS. With the roll out of 

 

1

Fractions knowledge at age 10-12 was found to be uniquely associated with a 0.17 SD increase (p < .05) 

in algebra knowledge and a 0.18 SD increase (p < .01) in total math achievement at age 15-17. This is 

based on a U.S. sample of 599 children who were tested in 1997 as 10-12 year-olds and again in 2002 as 

15-17 year-olds. Nearly identical results were found in a UK sample (Siegler et al., 2012).
6

the CCSS in mathematics scheduled for 2014–2015 in Georgia and South Carolina2, 

understanding the mathematical ideas and concepts involved with fractions and decimals–as well 

as computational algorithms–is a critical professional development need in the region, as well as 

the nation. 

There is a small body of high quality experimental and quasi-experimental research that has 

examined the impact of intensive mathematics professional development (PD) programs on 

student mathematics outcomes. The research team conducted a What Works Clearinghouse-type 

review of rigorous research evaluating mathematics PD approaches. A search of the research 

literature published between January 2006 and July 2012 (or rigorous studies that were identified 

in a previous review by Yoon, Duncan, Lee, Scarloss, & Shapley, 2007) was conducted. All 

studies that potentially could provide causal evidence of the effectiveness of teacher PD on 

student achievement were analyzed. Based on the five studies that were of acceptable technical 

quality, two professional development approaches, namely Lesson Study and intensive math 

content courses yielded either statistically significant outcomes on some mathematics measures 

or indicated relatively large impacts on mathematics outcomes (effect size greater than .25). 

There is clearly a need for additional research conducted under more typical conditions and with 

a clear topical focus. Thus, the purpose of the proposed study is to assess the impact of a 

professional development program that focuses on developing teachers’ knowledge of the formal 

mathematics that underlies fractions as well as pedagogical knowledge of techniques that 

promote development of student understanding of the mathematical ideas as well as 

computational proficiency. The study team, in consultation with experts in the field of 

mathematics, determined that fourth grade will be the focus of this study as it is a critical year for 

fractions instruction, especially as the foundation is laid for clearly understanding the meaning of 

fractions in later grades. 

For this project, the study team will use a randomized controlled trial design to study the 

Developing Mathematical Ideas (DMI; Schifter, Bastable, & Russell, 2010) program to 

determine if such an approach to professional development is effective in producing positive 

impacts on student achievement in mathematics. DMI was selected after a comprehensive

examination of professional development programs, as it has a robust infrastructure for 

implementing on a large scale.

 

2 This is based on the information retrieved from the South Carolina State Department of Education and 

Georgia Department of Education websites (South Carolina State Department of Education, 2013; 

Georgia Department of Education, 2012).
7

DMI was selected in partnership with a research alliance sponsored by REL Southeast. Members 

of the alliance helped review and select DMI from other candidate programs. This process is 

presented in more detail in Appendix H.

DMI was created and field-tested by the Education Development Center (EDC) through a grant 

funded by the National Science Foundation. The primary objectives of the DMI program are for 

teachers to: a) spend time on the actual teaching of mathematical topics to students in their 

classes, and b) explore students’ thinking about mathematics, including the misconceptions and 

hazy conceptions of mathematical ideas that develop during early phases of instruction. 

Using a random assignment design for this study helps ensure that–all else equal–this study will 

yield the strongest, most reliable evidence possible on which to base policy and practice. 

Note that burden estimates do not include the time teachers spend attending the PD sessions or 

completing the PD-related homework because the PD in this study will be replacing PD the 

school/district already require of their teachers. The PD in this study will not require an 

additional burden on the teachers. Teachers in the PD condition will be granted release days to 

participate in the PD sessions.

The current authorization for the Regional Educational Laboratories program is under the 

Education Sciences Reform Act of 2002, Part D, Section 174, (20 U.S.C. 9564), administered by 

the Department of Education (ED), Institute of Education Sciences (IES), National Center for 

Education Evaluation and Regional Assistance (NCEE).

A2. Indicate how, by whom, and for what purpose the information is to be used. Except for a 

new collection, indicate the actual use the agency has made of the information received from 

the current collection.

Research Questions and Overview of the Project Plan

This study is a large-scale evaluation of a mathematics professional development program for 

grade 4 teachers. Specifically, it is an evaluation of Developing Mathematical Ideas (DMI). The 

goal of the proposed study is to assess the impact of a professional development program that 

focuses on developing teachers’ knowledge of the formal mathematics that underlies fractions in 

order to promote development of student understanding of the mathematical ideas as well as 

computational proficiency. Information will be collected by the study team at the Instructional 

Research Group (IRG) and Florida State University, the prime contractor of the Regional 

Education Laboratory Southeast (REL-SE) and will be analyzed and used to inform the research 

questions and build upon existing IES initiatives. IES will review all results and release a report 

describing the study’s findings. 
8

The study team will address the following research questions, the first confirmatory3, and the 

second exploratory:

RQ1. What is the impact of teacher participation in Developing Mathematical Ideas for 

one year on students’ proficiency in the area of fractions when compared with those 

students whose teachers receive existing professional development activities provided by

schools and districts? 

RQ2. What is the impact of teacher participation in Developing Mathematical Ideas for 

one year on teacher knowledge of mathematics relevant for teaching fractions to 4th 

graders when compared with teachers who only participate in existing professional 

development activities provided by schools and districts?

To answer these questions, the study team has developed an experimental research design 

wherein data will be collected over the course of one year using a sample of fourth grade 

teachers from at least 80 schools4in diverse districts (city/town/rural) in Georgia and South 

Carolina. Only schools with two or more fourth grade classrooms will be recruited. Schools will 

be randomly assigned to treatment and control conditions (Shadish, Cook, & Campbell, 2002) 

and all fourth grade teachers in each participating school will be asked to participate. Schoollevel random assignment will reduce the potential for contamination between intervention and 

control group teachers. Assuming there are 82 schools participating, this will yield 41 treatment 

and 41 control schools. The study team will need to collect new information from the 

participating districts and schools through student and teacher testing, analysis of audio 

recordings of DMI professional development sessions, and teacher surveys. Data routinely 

collected and compiled by school districts will also help answer these questions. The data 

collection measures for each research question are summarized in Table A1. 

Note that burden estimates do not include the time teachers spend attending the PD sessions or 

completing the PD-related homework because the PD in this study will be replacing PD the 

school/district already require of their teachers. The PD in this study will not require an 

additional burden on the teachers. Teachers in the PD condition will be granted release days to 

participate in the PD sessions.

 

3 The term confirmatory refers to the primary research question for which the study was specifically 

designed and powered to address.

4 We will recruit 84 schools in order to account for any school-level attrition. We expect our final sample 

to include 82 schools. 
9

TABLE A1

PROPOSED DATA COLLECTION PLAN

Data Collection 

Sources

Mode Timeline Key Data

Student Achievement 

testing of all 

students

Post-test only

May–June 2015

Student achievement in 

mathematics (RQ1)

School/District Student records July 2014 Student grade 3 state 

assessment scores to use as 

a pretest covariate and 

student demographic 

information (RQ1)

Teacher Consent Form May–June 2014 Teacher demographic 

information (RQ2)

Teacher knowledge

of mathematics

testing of all 

teachers

Pretest: Aug.–Sept. 

2014

Post-test: May–June 

2015

Teacher knowledge of 

mathematics –

(RQ2)

Monthly online 

surveys of DMI 

activities (all 

teachers)

September 2014 –

April 2015

Professional development 

activities in treatment and 

control schools (RQ2)

PD Facilitator Procedural fidelity

checklists to use 

when analyzing 

audio tapes of DMI 

sessions (treatment 

PD only)

August 2014 –

November 2014

DMI fidelity (RQ1 & 2)
10

Student and Teacher Assessment

Student Assessment: For the student-level confirmatory analysis, grade 4 students will be 

assessed on a test, Test for Understanding of Fractions (TUF), that is currently being developed 

by the IES/NCSER (National Center for Special Education Research) Center for Improving 

Learning of Fractions. TUF is being developed in conjunction with the PIs of the Center (Nancy 

Jordan, Lynn Fuchs, and Robert Siegler) and will include, in part, measures that have been fieldtested in their current longitudinal and intervention research with fourth graders. These existing

measures focus on both procedural and conceptual knowledge, and include, for example, items 

from NAEP, batteries used in the research of Hecht, Close, and Santisi (2003) and measures 

specifically developed by Jordan (e.g. Jordan et al., 2013).

In summary, items aligned with the fourth grade objectives for fractions in the CCSS will be 

selected or adapted. As items from various sources will be incorporated into the measure for use 

in this study, scales will be constructed and evaluated using both classical test theory and itemresponse theory.

Teacher Assessment: The impact of the DMI intervention on teacher knowledge of mathematics 

relevant for teaching fractions to 4th graders will be assessed using the Mathematical Knowledge 

for Teaching (MKT) (Ball & Hill, 2008). The IRT reliability for MKT at the elementary level is 

.94. (Hill, Rowan, & Ball, 2005). The MKT consists of a large pool of items and relevant items 

on fractions will be selected under the guidance of Kristin Umland and Jim Lewis, two

mathematicians. Possible items are included in Appendix B. The final measure is expected to 

take less than 1 hour to complete. 

Note that the MKT teacher knowledge test is strictly an assessment and not included in the 

burden estimates. A sample of MKT items is provided in Appendix B.
11

Data Collection Methods

In each district, data will be collected from both treatment and control schools at the same time, 

by assessors blind to condition, to guard against bias entering the data collection process. 

Collecting Teacher Data: Active consent of individual teachers for participation will be 

acquired, before assignment of individual schools. Prior to the start of the intervention (August 

2014), the teacher demographics form will be collected (e.g., teacher experience and education) 

as well as the teacher knowledge of mathematics (MKT)

5

assessment. The demographic survey

is included in our calculations of respondent burden. Demographic information will be collected 

at the outset of the study as part of the teacher consent form. A copy of this form has been 

included with this submission as Appendix A. Immediately after the end of the professional 

development intervention teachers will complete the MKT measure (for RQ2). Note that the 

MKT teacher knowledge test is strictly an assessment and not included in the burden estimates. 

Sample MKT items have been included as Appendix B. A monthly online measure assessing the 

type and amount of professional development teachers participate in will be sent to all teachers 

using SurveyMonkeyTM. This survey is included in our calculations of respondent burden. A copy 

of the monthly teacher survey is included with this document as Appendix C. 

Collecting Student Data: Student prior year pretest data on the third grade state assessment will 

be collected from school districts when it becomes available in 2014. Student demographic data 

will be gathered from the school databases upon completion of the parent opt-out phase. The 

study team has a waiver of active consent from the IRB, as direct engagement of students in the 

study is limited strictly to the fractions outcome assessment. A 30-40 minute group assessment 

(very similar to the state assessments) appears to represent a very low risk to students.

Post-test data will be collected in the spring after the professional development has ended, 

around 4-6 weeks prior to the end of the school year. Students will be given the Test for 

Understanding of Fractions (TUF) measure by trained evaluation staff in large groups during 

independent seatwork to maximize teacher instructional time during the school day (RQ1). 

The administration of an achievement test like the TUF is not considered for information 

collection burden under 5 CFR 1320.3(h) (7). 

Collecting School Data: School level data (e.g., free and reduced lunch, school achievement 

scores) will be collected from the prior year school report cards for purposes of matching 

schools. 

 

5 Mathematical Knowledge for Teaching (MKT) (Ball and Hill, 2008). Note that the MKT teacher 

knowledge test is strictly an assessment and not included in the burden estimates. A sample of MKT 

items is provided in Appendix B.
12

Prior year student achievement data are pre-existing, as they have already been collected by

schools and districts. The time district personnel spend locating and organizing this data for us is 

included in our calculation of burden. Using existing school records data helps to minimize 

overall burden because it avoids the collection of these same data items from individual 

respondents at the school level.

Collecting Fidelity of Implementation Data: DMI facilitators at each site will be asked to 

audiotape each session they have with participating teachers. Two research staff with training in 

observation systems will listen to each tape and use the fidelity of implementation instrument to 

measure adherence to critical core components of the DMI program using checklists developed 

in conjunction with the developers. Discrepancies will be resolved by senior staff. It is our 

understanding that the collection of this audio data is not considered in calculations of paperwork 

burden.

A3. Describe whether, and to what extent, the collection of information involves the use of 

automated, electronic, mechanical, or any other technological collection techniques or forms 

of information technology, e.g. permitting electronic submission of responses, and the basis 

for the decision of adopting the means of collection. Also describe any consideration of using 

information technology to reduce burden. 

Wherever possible the research team will use information technologies to maximize the 

efficiency and completeness of the information gathered for this evaluation and to minimize the 

burden on respondents. In particular, student demographic and prior year assessment data will be 

gathered from existing electronic school administrative records. Brief monthly teacher surveys 

regarding the type and amount of professional development teachers participate in will be sent to 

all teachers using the online application SurveyMonkeyTM. 

A4. Describe efforts to identify duplication. Show specifically why any similar information 

already available cannot be used or modified for use of the purposes described in Item 2 

above. 

The data collection effort planned for this project will produce data that are unique and 

specifically target the research questions identified for this project. A rigorous literature review 

was conducted along the standards outlined by the What Works Clearinghouse that identified the 

need for the kind of experimental research that will be conducted in this study. Furthermore, 

there is no current evidence from studies using randomized controlled designs available on the 

DMI program.

To complete this research, the study team is planning to administer the TUF to all students in the 
13

study sample because the data collected with such an instrument are not available from any other 

source on a national basis; the state-administered assessments vary in what they measure, and 

how precisely they can measure student achievement in mathematics. Thus, administering our 

own assessment is necessary because it provides site-to-site consistency and assures a reliable 

and common mathematics achievement score. 

Data collection through the teacher assessment will provide a measure of teacher knowledge of 

mathematics relevant for teaching fractions to 4th graders as well as the treatment contrast 

between DMI (treatment) and non-DMI (control) schools. Surveying all of the teachers on a 

monthly basis as to professional development activities will provide descriptive data to augment 

the discussion of treatment versus control schools, and taking audio recordings of DMI 

professional development sessions will provide useful descriptive data regarding fidelity of 

implementation of the DMI program. In every case, these data do not currently exist. 

Note that burden estimates do not include the student or teacher knowledge measures as these are 

strictly assessments. The estimates do not include the time teachers spend attending the PD 

sessions or completing the PD-related homework because the PD in this study will be replacing 

PD the school/district already require of their teachers. The PD in this study will not require an 

additional burden on the teachers. Teachers in the PD condition will be granted release days to 

participate in the PD sessions.

A5. If the collection of information impacts small businesses or other small entities (Item 5 of 

OMB Form 83-1), describe any methods used to minimize burden.

Not applicable. No small businesses will be burdened by this data collection. The focus of this 

study is on school districts and the attendant schools–which includes the students, teachers, and 

administrative staff–within these districts. The study team has reduced burden for respondents 

using a data collection plan that requests the minimum information needed to successfully 

execute this study. To minimize burden on respondents, the study design requests information 

already collected by schools and districts, and any new collection instruments have been 

designed to ask questions that cannot be answered through any other available sources. In some

cases, these data will be collected electronically. The study team will conduct all data collection 

activities.

A6. Describe the consequences to Federal program or policy activities if the collection is not 

conducted or is conducted less frequently, as well as any technical or legal obstacles to 

reducing burden.

The systematic collection, analysis, and reporting of the data described in this submission are 

required to accomplish the goals of the research project approved by IES. Participation in all data 
14

collection activities is voluntary but the study team will work to assure high participation rates. 

Multiple data collection strategies are planned and these data are necessary to measure impacts 

on academic achievement and teacher knowledge. 

The TUF will be administered to students at treatment and control schools once as an outcome 

measure at the end of the 2014–2015 school year. This is the primary outcome measure for 

mathematics skills and if it is not collected the study would lack a common measure of students’ 

mathematical ability. There is no other common measure across the two states and multiple 

districts.

The teacher measure will be administered to teachers at treatment and control schools prior to 

beginning the DMI program and after professional development at the end of the academic year. 

This is the principal outcome measure for teacher knowledge and if it is not collected, the study 

would lack a common measure of teacher knowledge at baseline and after receipt of professional 

development. Pretesting teacher knowledge is necessary to boost the statistical power, and is 

assumed in the power analysis, because of the inherently smaller number of teachers in the study. 

If the pretest is not given, the study may not be as sensitive to effects on teacher knowledge that 

may be present. 

During the course of the study year, treatment and control teachers will be surveyed monthly 

regarding any professional development in the field of mathematics they received. These surveys 

will provide the study team with valuable descriptive information about the professional 

development activities going on in all of the study schools. If the study team surveys the teacher 

sample less frequently there is a risk that the teachers might have less valid memories of the PD 

they received.

All DMI professional development sessions with teachers will be audio recorded (8 sessions 

delivered per site) and these recordings will be analyzed by two independent raters to ensure 

implementation fidelity of the DMI program. Without these recordings, the study team will not 

have means to make certain the program is being implemented correctly.

High response rates are anticipated given that states, districts, and even schools will be selected 

based on their willingness to fully participate in the project. The study team is confident that the 

use of multiple measures of data collection will ensure accurate analyses and results.

Finally, if this study is not conducted, REL Southeast may not meet the requirements for 

conducting rigorous research that meets regional needs. REL (NCEE) contracts are awarded 

contingent on a plan to partner in research alliances with educational policymakers from the 

region to identify research needs and to conduct studies that address them. These researcher-
15

practitioner alliances are mandated to conduct rigorous studies (typically RCTs) wherever 

possible. The proposed study meets this contractual requirement.

A7. Explain any special circumstances that would cause an information collection to be 

conducted in a manner inconsistent with Section 1320.5(d)(2) of the federal regulations.

The proposed study meets all guidelines listed under A7 except for a requirement that 

respondents report information more than quarterly. As explained in section A6, the study team 

will survey all teachers about their professional development activities on a monthly basis. This 

survey is brief (approximately 12 minutes per month) and administered electronically. Not 

surveying the teachers as regularly could result in less valid responses owing to potential 

inaccuracy of memory over longer periods.

A8. Federal Register Comments and Persons Consulted Outside of the Agency 

A notice to solicit public comments was published in the Federal Register in order to provide the 

opportunity for public comment. No comments were submitted.

The following individuals were consulted on the statistical aspects of this proposal:

Dr. Sybilla Beckman, University of Georgia

Expertise: mathematician, mathematics education

Email: sybilla@math.uga.edu

Phone: 706-542-2548

Dr. John Deke, Mathematica Policy Research

Expertise: methods for large-scale randomized trials

Email: jdeke@mathematica-mpr.com

Phone: 609-275-2230

Dr. Mike Garet, American Institutes for Research

Expertise: teacher professional development, research methods, large-scale randomized trials

Email: MGaret@air.org

Phone: 202-403-5000

Dr. Nathan Jones, Boston University

Expertise: measurement of teacher quality, teacher evaluation, special education

Email: ndjones@bu.edu

Phone: 617-353-3295
16

Dr. Yaacov Petscher, Florida State University

Expertise: research design, measurement, and statistical methods

Email: ypetscher@fcrr.org

Phone: 850-644-0327

Dr. Mengli Song, American Institutes for Research

Expertise: research design, advanced quantitative methods, and evaluations of educational 

programs and policy

Email: MSong@air.org

Phone: 202-403-5000

The first five individuals in the list above attended a full-day meeting in Washington, D.C. on 

October 12, 2012 to discuss the proposal. Of particular focus were the statistical assumptions 

informing the power analysis, advantages and disadvantages of the mathematics PD approaches

under consideration at that time, covariates to be included in the HLM models and the proposed 

outcome measures.

The sixth individual, Dr. Song, reviewed the power analyses and HLM models independently.

A9. Explain any decision to provide any payment or gift to respondents, other than 

remuneration of contractors or grantees.

All teachers in the PD and the control condition will be compensated for their time spent 

completing the measures for this study (see section A12 – Table A2). These measures include the 

teacher consent/demographics form, the pre- and post-test of teacher knowledge of mathematics 

relevant for teaching fractions to 4th graders, and the monthly professional development activity

survey. Therefore both treatment and control teachers will be remunerated $10 for the 

consent/demographics form, $50 for two assessments ($25 each), $90 for 9 monthly PD reports

($10 each), for a sum of $150.

Treatment teachers will not be compensated for DMI participation time. If teachers attend the PD 

during the work week, teachers will be provided with one full-release day from instruction for 

each DMI session. The release time will cover both the PD session and the time required to 

complete the homework. Note that burden estimates do not include the time teachers spend 

attending the PD sessions or completing the PD-related homework because the PD in this study

will be replacing PD the school/district already require of their teachers. The PD in this study

will not require an additional burden on the teachers. Teachers in the PD condition will be 

granted release days to participate in the PD sessions.
17

Budget limitations preclude us from offering the PD to control teachers in a subsequent year.

A10. Describe any assurances of confidentiality provided to respondents and the basis for the 

assurance in statute, regulation, or agency policy.

REL Southeast will be following the new policies and procedures required by the Education 

Sciences Reform Act of 2002, Title I, Part E, Section 183 requires "All collection, maintenance, 

use, and wide dissemination of data by the Institute" to "conform with the requirements of 

section 552 of title 5, United States Code, the confidentiality standards of subsection (c) of this 

section, and sections 444 and 445 of the General Education Provision Act (20 U.S.C. 1232g, 

1232h)." These citations refer to the Privacy Act, the Family Educational Rights and Privacy 

Act, and the Protection of Pupil Rights Amendment. 

In addition for student information, "The Director shall ensure that all individually identifiable 

information about students, their academic achievements, their families, and information with 

respect to individual schools, shall remain confidential in accordance with section 552a of title 5, 

United States Code, the confidentiality standards of subsection (c) of this section, and sections 

444 and 445 of the General Education Provision Act.”

Subsection (c) of section 183 referenced above requires the Director of IES to "develop 

and enforce standards designed to protect the confidentiality of persons in the collection, 

reporting, and publication of data".

Subsection (d) of section 183 prohibits disclosure of individually identifiable information 

as well as making any the publishing or communicating of individually identifiable 

information by employees or staff a felony.

REL Southeast will protect the confidentiality of all information collected for the study, as 

permitted by law, and will use it for research purposes only. Information collected for this study 

comes under the confidentiality and data protection requirements of the Institute of Education 

Sciences (The Education Sciences Reform Act of 2002, Title I, Part E, Section 183). Responses 

to this data collection will be used only for statistical purposes. The reports prepared for the 

study will summarize findings across the sample and will not associate responses with a specific 

district, school, or individual. We will not provide information that identifies the participants 

(teachers and their students), the schools or the districts to anyone outside the study team, except 

as required by law.

No information that identifies any study participant will be released. Information from 

participating institutions and respondents will be presented at aggregate levels (across schools or 

groups) in reports. Staff working on the project have signed confidentiality pledges and been 
18

screened for data security through the U.S. Department of Education e-QIP system. All paper 

protocols will be stored in a locked facility and data stored in digital files will be maintained on a 

secure server that is backed up daily. Only persons conducting this study and maintaining its 

records will have access to the records collected that contain individually identifying 

information.Three years after the end of the project, the data will be destroyed.

A11. Provide additional justification for any questions of a sensitive nature, such as sexual 

behavior and attitudes, religious beliefs, and other matters that are commonly considered 

private. The justification should include the reasons why the agency considers the questions 

necessary, the specific uses to be made of the information, the explanation to be given to 

persons from whom the information is requested, and any steps to be taken to obtain their 

consent.

None of the questions utilized to collect data for this study, including interviews, concern topics 

commonly considered private or sensitive, such as religious beliefs or sexual practices. 

IRB approval has been granted.

A12. Provide estimates of the hour burden of the collection of information.

The estimated burden on respondents for completing the activities included in the study’s data 

collection activities is listed in Table A2. Respondents will include teachers at treatment and 

control schools.

Table A2 summarizes reporting burden on respondents for the data collection instruments, 

including total and annualized estimates. The annual burden has been averaged across the 3 years 

of the clearance. Annualized, the total number of respondents is 4,185 per year and the total 

number of responses is 4,923 per year. The total burden across all respondents is expected 595

burden hours per year.

Note that burden estimates do not include the time teachers spend attending the PD sessions or 

completing the PD-related homework because the PD in this study will be replacing PD the 

school/district already require of their teachers. The PD in this study will not require an 

additional burden on the teachers. Teachers in the PD condition will be granted release days to 

participate in the PD sessions.

The burden estimates do not include the student assessment (administered at the end of the 

study) or the teacher assessments (pre and post). These measures are strictly assessments and do 

not impact burden estimates.
19

Table A2. Burden Estimates

Data Collection 

Activity

Respondents 

Per 

School

Number 

of 

Schoolsa

Total 

Respondents

Responses 

per 

Respondent

Total 

Responses

Average 

Burden 

Hours 

per 

Response

Total 

Burden 

Hours

Teacher 

Consent Forms 

with 

Demographics

(Appendix A)

3 82 246 teachers 1 246 .2 49.2

Teacher 

Professional 

Development 

Survey 

(Appendix C)

3 82 246 teachers 9 

(monthly)

2214 .2 442.8

Parent Opt-Out 

Form for 

Student 

Participation

(Appendix F)

75b82 6150 parents 1 6150 .1 615

Student Assent 

Form for 

Student 

Participation 

(Appendix G)

75c82 6150 students 1 6150 .1 615

School records 

data collection: 

Student 

information

na na 8 districts 1 8 8 64

TOTAL 

ESTIMATES 12,554 14,768 1,786

ANNUAL 

ESTIMATESd 4,185 4,923 595

a The number of schools needed to detect an effect for this study is 80; however, we will over-recruit up 

to 84 schools to account for possible school-level attrition. We expect that 82 schools will remain in the 

study after attrition.

b We estimate 25 parents in each classroom will be asked to read the parent opt-out form. 

c We estimate 25 students in each classroom will be asked to complete the assent form; however, on 

average 19 students (of 25) will participate in the study from each classroom (assuming 10% of parents 

opt to remove their child from the study and 15% of students attrite).

d The annual figures have been averaged over the 3 years of the clearance.
20

A13. Describe any other costs to respondents or record keepers.

Not applicable. The information collection activities do not place any capital cost or cost of 

maintaining capital requirements on respondents. 

A14. Provide estimates of annualized cost to the Federal government. Also provide a 

description of the method used to estimate cost, which should include quantification of hours, 

operational expenses (such as equipment, overhead, printing, and support staff), and any 

other expenses that would not have been incurred without this collection information. 

Agencies may aggregate cost estimates from Items 12, 13, and 14 in a single table. 

The proposed data collection is being supported through Department of Education contract EDIES-12-C-0011. This project is considered Task 4.1.5 of the NCEE REL Southeast contract 

awarded to Florida State University.

The annualized cost of data collection for this project is calculated simply as the planned annual 

fixed price allocations for the study during the two calendar years (2014 and 2015) that this study

is gearing up for data collection, actually being in the field, and wrapping up data collection.

Total cost of data collection for this project is estimated to be: $1,408,400. The annual cost of the 

data collection over the three years is estimated to be: $469,467. This represents a fixed price 

contract to conduct these activities in the two calendar years that involve data collection.

2014: Budget = $690,400

 January – April 2014: Approximate date-range to receive OMB clearance

 January – June 2014: After OMB clearance: Recruiting of districts/schools/teachers – sign 

district/school MOUs

 January – June 2014: Work with DMI developers to plan/refine program as needed

 January – June 2014: Collaborate with the Center for Improving Learning of Fractions on 

development of student outcome measure.

 May – June 2014: Distribute/acquire teacher consent forms

 June – August 2014: Recruit & train teacher knowledge assessors

 July 2014: Conduct random assignment

 August – September 2014: Conduct teacher knowledge pretests 

 August – November 2014: Conduct eight DMI sessions with 240 teachers in 8-10 districts

 September – Dec 2014: Collect monthly teacher PD logs

2015: Budget = $718,000

 January – April 2015: Collect monthly teacher PD logs

 April – May 2015: Recruit/train assessors

 May – June 2015: Conduct teacher and student post-tests

 June 2015: Data entry & cleanup

 July – October 2015: Conduct data analysis

 September – Dec. 2015: Report Preparation
21

 December 2015: Submit first draft of technical report to IES

Funding includes paying for REL Southeast staff, as well as covering costs of subcontractors on 

this study who will assist in study design, data collection, and data analysis. Also included are 

costs associated with delivering the intervention to treatment schools, and collecting data from 

both treatment and control schools. There are no additional costs, other than staff collection time 

and duplication, for the student and teacher outcome measures.

A15. Describe any changes in the burden from prior approvals.

This submission to OMB is a new request for approval of data collection plans. 

A16. For collections of information whose results will be published, outline plans for 

tabulation and publication. Address any complex analytical techniques that will be used. 

Provide the time schedule for the entire project, including beginning and ending dates of the 

collection of information, completion of the report, publication dates, and other actions. 

The project schedule is as follows:

2014: Budget

 January – April 2014: Approximate date-range to receive OMB clearance

 January – June 2014: After OMB clearance: Recruiting of districts/schools/teachers – sign 

district/school MOUs

 January – June 2014: Work with DMI developers to plan/refine program as needed

 January – June 2014: Collaborate with the Center for Improving Learning of Fractions on 

development of student outcome measure.

 May – June 2014: Distribute/acquire teacher consent forms

 June – August 2014: Recruit & train teacher knowledge assessors

 July 2014: Conduct random assignment

 August – September 2014: Conduct teacher knowledge pretests 

 August – September 2014: Conduct eight DMI sessions 

 September – Dec 2014: Collect monthly teacher PD logs

2015: Budget

 January – April 2015: Collect monthly teacher PD logs 

 April – May 2015: Recruit/train assessors

 May – June 2015: Conduct teacher and student post-tests

 June 2015: Data entry & cleanup

 July – October 2015: Conduct data analysis

 September – Dec. 2015: Report Preparation

 December 2015: Submit first draft of technical report to IES

2016:

 January – June 2016: Approximate date range to respond iteratively to IES reviews of drafts of 

report
22

 September 2016: Estimated date of final report release by IES

The design, data collection, analysis, and reporting for this study is driven by the primary

(confirmatory) research question: What is the impact of teacher participation in Developing 

Mathematical Ideas on students’ proficiency in the area of fractions when compared with those 

students whose teachers receive existing professional development activities provided by schools 

and districts? 

A secondary (exploratory) research question will also be addressed, although the study was not 

specifically powered to do so: What is the impact of teacher participation in Developing 

Mathematical Ideas for one year on teacher knowledge of mathematics relevant for teaching 

fractions to 4th graders when compared with teachers who only participate in existing

professional development activities provided by schools and districts?

Our approach to estimating the effects of DMI has the following core features:

1) A focus on student-level impacts based directly on the experimental design.

2) Estimation of impacts in ways that account for clustering of students within 

classrooms, within schools, and within districts.

The basic logic of our analysis strategy is to compare the schools that are randomly assigned to 

receive the treatment to those that are not. As random assignment occurs at the school level, 

schools are the primary unit of analysis. Schools will be paired within district based on overall 

demographic characteristics to increase the probability of baseline equivalence.

The research team will use multi-level models in the impact analysis to account for the clustering 

of students within classrooms, and teachers within schools. The analytic methods are discussed 

in further detail in Part B2 and Appendix J.

All results for REL rigorous studies will be made available to the public through peer-reviewed 

evaluation reports that are published by IES. The datasets from these rigorous studies will be 

turned over to the REL’s IES project officer. These data will become IES restricted use datasets 

requiring a user’s license that is applied for through the same process as NCES restricted use 

datasets. Even the REL contractor would be required to obtain a restricted use license to conduct 

any work with the data beyond the original evaluation.

A17. Describe arrangements for displaying the number provided by OMB and its expiration 

date.
23

The approval number provided by OMB and its expiration date will appear in the heading on all 

instruments for this project.

A18. Exceptions to Certification Statement 

No exceptions are necessary for this information collection.
24

References

Ball, D. L., & Hill, H. C. (2008). Mathematical knowledge for teaching (MKT) measures: 

Mathematics released items 2008. Ann Arbor, MI: University of Michigan. Retrieved 

from http://sitemaker.umich.edu/lmt/files/LMT_sample_items.pdf

Center for Improving Learning of Fractions. (2013). Center for Improving Learning of 

Fractions. Retrieved from https://sites.google.com/a/udel.edu/fractions/

Georgia Department of Education. (2012). Common core Georgia performance standards. 

Retrieved from http://www.doe.k12.ga.us/Curriculum-Instruction-andAssessment/Curriculum-and-Instruction/Pages/CCGPS.aspx

Hecht, S. A., Close, L., & Santisi, M. (2003). Sources of individual differences in fraction skills. 

Journal of Experimental Child Psychology, 86(4), 277-302. 

Hill, H., Blunk, M., Charalambous, C., Lewis, J., Phelps, G., Sleep, L., & Ball, D. L. (2008). 

Mathematical knowledge for teaching and the mathematical quality of instruction: An 

exploratory study. Cognition and Instruction, 26(4), 430-511.

Hill, H., Kapitula, L., & Umland, K. (2011). A validity argument approach to evaluating teacher 

value-added scores. American Educational Research Journal, 48(3), 794-831.

Hill, H., Rowan, B., & Ball, D. (2005). Effects of teachers' mathematical knowledge for teaching 

on student achievement. American Educational Research Journal, 42(2), 371-406.

Jordan, N. C., Hansen, N., Fuchs, L. S., Siegler, R. S., Gersten, R., & Micklos, D. (2013). 

Developmental predictors of fraction concepts and procedures. Journal of Experimental 

Child Psychology, 116 (1), 45-58.

Ma, L. (1999). Knowing and teaching elementary mathematics: Teacher's understanding of 

fundamental mathematics in China and the United States. New York, NY: Routledge.

National Assessment of Educational Progress (2011). The nation’s report card: Mathematics 

2011. (NCES: 2012-458). Washington, DC: Institute of Education Sciences, U.S. 

Department of Education. Retrieved from 

http://nces.ed.gov/nationsreportcard/pdf/main2011/2012458.pdf

National Mathematics Advisory Panel. (2008). Foundations for success: The final report of the 

National Mathematics Advisory Panel. Washington, DC: U.S. Department of Education. 

Retrieved from http://www2.ed.gov/about/bdscomm/list/mathpanel/report/finalreport.pdf 

Perry, R. R., & Lewis, C. C. (2011). A randomized trial of lesson study with mathematical 

resources: Measuring the impact on fractions knowledge. Manuscript submitted for 

publication, School of Education, Mills College, Oakland, California. 

Schifter, D., Bastable, V. & Russell, S. J. (2010). Making meaning for operations in the domains 

of whole numbers and fractions: Facilitator’s guide. Boston, MA: Pearson Education, 

Inc.
25

Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental 

designs for generalized causal inference. Boston, MA: Houghton Mifflin.

Siegler, R., Carpenter, T., Fennell, F., Geary, D., Lewis, J., Okamoto, Y., . . . Wray, J. (2010). 

Developing effective fractions instruction for kindergarten through 8th grade: A practice 

guide (NCEE #2010-4039). Washington, DC: National Center for Education Evaluation 

and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. 

Retrieved from http://whatworks.ed.gov/publications/practiceguides

Siegler, R. S., Duncan, G. J., Davis-Kean, P. E., Duckworth, K., Claessens, A., Engle, M., . . . 

Chen, M. (2012). Early predictors of high school mathematics achievement. 

Psychological Science, 23(7), 691-697. doi: 10.1177/0956797612440101

South Carolina State Department of Education. (2013). Common Core State Standards.

Retrieved from http://ed.sc.gov/agency/programs-services/190/

What Works Clearinghouse. (2013). What works clearinghouse: Procedures and standards 

handbook (Version 3.0). Retrieved from 

http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_procedures_v3_0_draft_standar

ds_handbook.pdf

Yanik, H. B., Helding, B., & Baek, J. M. (2006). Students’ difficulties in understanding fractions 

as measures. Problem Solving, 2, 323-325. 

Yoon, K. S., Duncan, T., Lee, S. W.-Y., Scarloss, B., & Shapley, K. (2007). Reviewing the 

evidence on how teacher professional development affects student achievement (Issues & 

Answers Report, REL 2007–No. 033). Washington, DC: U.S. Department of Education, 

Institute of Education Sciences, National Center for Education Evaluation and Regional 

Assistance, Regional Educational Laboratory Southwest. Retrieved from 

http://ies.ed.gov/ncee/edlabs