Appendix A:

National Evaluation of i3 (NEi3) 

Data Collection Survey

Partners:

Abt Associates Inc.

Dillon-Goodson 

Research 

Associates

Chesapeake 

Research 

Associates

ANALYTICA, Inc.

Westat

Century Analytics

American Institutes 

for Reserach

This work is supported by the U.S. Department of Education under 

contract number ED-IES-10-C-0064, ED-IES-13-C-0005 and/or EDIES-14-C-0007, Abt Associates, Prime Contractor.
National Evaluation of i3 (NEi3) Data Collection Survey Page 2

Welcome [First_Name Last_Name]!

The National Evaluation of Investing in Innovation (NEi3) is being conducted by Abt Associates Inc. on 

behalf of the U.S. Department of Education (ED) Institute of Education Sciences (IES). A primary goal of 

the NEi3 is to assess the strength of evidence generated by independent local evaluations of i3 grants and 

to provide summaries of evaluation findings. The purpose of this data collection is to collect information 

from evaluators like you regarding the findings of your i3 independent evaluation. We look forward to 

working with you on this important endeavor!

We would like you to know that...

Your response is critical for producing valid and reliable data. You may skip any questions you do not wish to 

answer; however, we hope that you answer as many questions as you can. Your answers to questions will not 

affect your grant—now or in future. Participation in the evaluator survey will not impose any risks to you as a 

respondent. If you have any questions about your rights as a research participant, you can contact Teresa 

Doksum, the Chairperson of Abt’s IRB at 617-349-2896 or by email at irb@abtassoc.com. 

Responses to this data collection will be used to disseminate information about your study findings. 

This survey is authorized by law:American Recovery and Reinvestment Act of 2009 (ARRA), Section 

14007, Title XIV (Public Law (P.L.) 111-5.

OMB Clearance Number: xxxx-xxxx Expiration Date: xx/xx/xxxx

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a 

valid OMB control number. The valid OMB control number for this information collection is xxxx-xxxx. The time required to complete this 

information collection is estimated to average approximately 3 hours per respondent, including the time to review instructions, gather the 

data needed, and complete and review the information collected. If you have any comments concerning the accuracy of the time 

estimate(s) or suggestions for improving this form, please write to: U.S. Department of Education, Washington, DC 20202. If you have 

comments or concerns regarding the status of your individual submission of this form, write directly to: U.S. Department of Education, 

Institute of Education Sciences, 555 New Jersey Avenue, N.W., Washington, DC 20208.
National Evaluation of i3 (NEi3) Data Collection Survey Page 3

Overview of the NEi3 Data Collection Survey

Throughout the data collection survey, the NEi3 study team has pre-populated information available from 

the following sources: 

 Documents you produced to fulfill requirements of ED’s cooperative agreements (your analysis 

plan and your contrast tool )

 Reports or other documents on interim or final results you’ve provided us

Please review this information carefully to ensure its accuracy (you’ll be asked to actively confirm the 

accuracy of all pre-populated information). 

We realize that it may take you a few visits to this website to complete entering your findings into this 

data collection survey. Therefore, you may log into and out of the survey as many times as necessary to 

complete the entire data collection survey by MM/DD/YYYY. The information you enter into this survey 

will be automatically saved once you navigate to a different survey section. 

The data collection website is organized into three main sections: 

1. Background Section

2. Fidelity of Implementation Section

3. Impact Study Section

After you enter the survey, you can access each of the above sections by clicking on the appropriate 

navigation button at the top of your screen. Below, we briefly describe the content of each of these 

sections. 

The Background Section asks you about the parties involved in developing, implementing and evaluating 

the i3-funded intervention on which you worked. It also asks some general questions about the grant, 

intervention, and evaluation. 

The Fidelity of Implementation Section asks you to describe the key components of the i3-funded 

intervention you evaluated, the measures constructed to evaluate fidelity of implementation for those 

components, and the findings from your implementation study. 

The Impact Study Section asks you to describe each of your evaluation’s contrasts. Specifically, the 

survey will ask you to provide information about the results from the analysis for each contrast, including 

the estimated effects of the intervention and the size of the analysis sample. 

Getting Help

For Help with technical issues using this website, please call 617-520-2447 For Help with selecting 

appropriate data for entry and/or for further guidance on survey items, please call 301-347-5638.
National Evaluation of i3 (NEi3) Data Collection Survey Page 4

Background Information

The following section asks you about the parties involved in developing, implementing and evaluating the 

i3-funded intervention on which you worked. It also asks some general questions about the grant, 

intervention, and evaluation. 

1. What is the name of the grantee for which you conducted/are conducting the evaluation?

___________________________________________________________

2. What is the name of the grant under which you conducted/are conducting the evaluation?

________________________________________________________

3. What is the i3 grant type of this grant? 

 Development  Scale-Up  Validation

4. How many impact studies/evaluations are funded under this grant? ______________________

5. If there are multiple impact studies/evaluations funded under this grant, what is the name of 

the study reported in this survey? 

____________________________________________________________________________

6. What is/are the name(s) of the organization(s) that conducted/is conducting the impact study? 

____________________________________________________________________________

7. What is/are the name(s) of the principal investigator(s) that conducted/is conducting the study? 

____________________________________________________________________________

8. What is the name of the i3-funded intervention? 

____________________________________________________________________________
National Evaluation of i3 (NEi3) Data Collection Survey Page 5

9. Please review the following summary description of the i3-funded intervention, and indicate 

its accuracy by selecting the appropriate box below. If the description requires any revision, 

please insert a corrected description in the area provided. Please note that this description is 

intended for inclusion in the NEi3 Findings Report; however, the NEi3 may condense, or seek 

clarifications for, proposed revisions. 

NEi3 Description: This section will be pre-populated by the NEi3 team with a summary of the 

intervention in sufficient detail to help readers understand what makes this intervention similar to 

or different from other interventions. The level of detail will be similar to what would be provided 

in an introductory section of a typical impact evaluation report. It will include the length of the 

intervention and the dosage, as well as information about the content, delivery, and 

implementation of the intervention. 

 The intervention description is correct. (If this box is checked, the NEi3 Description that 

appears above will be used in the NEi3 Findings Report.) 

 The intervention description requires some revision. (If this box is checked, please enter a 

revision below.)

Evaluator Revision: 

10. What are the names of the individual(s) and organization(s) that developed the intervention? 

Individual Organizational affiliation

11. What are the name(s) of the individuals and organization(s) that implemented and/or are 

implementing the intervention being evaluated under this grant? 

Individual Organizational affiliation

Please insert a revised description here if needed.
National Evaluation of i3 (NEi3) Data Collection Survey Page 6

12. Did the following entities review the study findings prior to their release and/or submission to 

NEi3: 

Yes No

The individual/organization implementing the intervention (from 

question 11)  

The individual/organization developing the intervention (from 

question 10)  

Participating School District(s)  

Participating School Principal(s)  

Other, please specify: ____________________________  

If you answered “yes” to any question above, please explain: 

______________________________________________________________________________

Programming note: if “yes” is checked above, flag for NEi3 reviewer to discuss which contrasts 

are implicated if warranted based on open-ended response.

13. Did the following entities provide comments on the study findings prior to their release and/or 

submission to NEi3? 

Yes No

The individual/organization implementing the intervention (from 

question 11)  

The individual/organization developing the intervention (from 

question 10)  

Participating School District(s)  

Participating School Principal(s)  

Other, please specify: _________________________________  

If you answered “yes” to any question above, please explain how comments were addressed: 

______________________________________________________________________________

Programming note: if “yes” is checked above, flag for NEi3 reviewer to discuss which contrasts 

are implicated if warranted based on open-ended response.
National Evaluation of i3 (NEi3) Data Collection Survey Page 7

14. For each entity below, did the entity request changes to the analysis and reporting prior to their 

release and/or submission to NEi3? 

Yes No

The individual/organization implementing the intervention (from 

question 11)  

The individual/organization developing the intervention (from 

question 10)  

Participating School District(s)  

Participating School Principal(s)  

Other, please specify: _________________________________  

If you answered “yes” to any question above, please explain whether the requested changes were 

made, and why: 

______________________________________________________________________________

Programming note: if “yes” is checked above, flag for NEi3 reviewer to discuss which contrasts 

are implicated if warranted based on open-ended response.

15. For each of the following entities, did the entity conduct any of the statistical analysis for the 

findings submitted to NEi3? 

Yes No

The individual/organization implementing the intervention (from 

question 11)  

The individual/organization developing the intervention (from 

question 10)  

Participating School District(s)  

Participating School Principal(s)  

Other, please specify: _________________________________  

If you answered “yes” to any question above, please explain: 

______________________________________________________________________________

Programming note: if “yes” is checked above, flag for NEi3 reviewer to discuss which contrasts 

are implicated if warranted based on open-ended response.
National Evaluation of i3 (NEi3) Data Collection Survey Page 8

16. Please review the following description of the population served by this grant’s i3-funded 

intervention and indicate its accuracy by selecting the appropriate box below. If the 

description requires any revision, please insert a corrected description in the area provided. 

NEi3 Description: This section will be prepopulated with a description of the population served 

by this grant’s i3-funded intervention(s). The description will include the target age/grade, or 

teacher type (for example, fifth grade teacher, elementary school counselor, high school science 

teacher, etc.), of the intervention. It will also include the location of the intervention and a 

description of the type and number of schools served. 

 The intervention description is correct. (If this box is checked, the NEi3 Description that 

appears above will be used in the NEi3 Findings Report.) 

 The intervention description requires some revision. (If this box is checked, please enter a 

revision below.)

Evaluator Revision: 

17. Please review the following description of the sample eligibility and inclusion for this study’s 

impact evaluation. If the description requires any revision, please insert a corrected 

description in the area provided. 

NEi3 Description: [Describe how the districts/schools were chosen to be in the study (e.g. 

universe, random sample from a defined population, convenience sample , etc...). Please 

articulate all inclusion and exclusion criteria. For example, “The five largest school districts in 

XYZ County, Texas, were recruited to participate in this study because of their size, and because 

of the high percentage of ELL students in XYZ County. Within each of the five school districts, two 

to four elementary schools will be recruited and half will be randomly assigned to treatment and 

half will be assigned to control. The intervention is a school-wide intervention. All recruited 

schools will be included in the impact evaluation. Outcomes will be assessed for students in grades 

three through five. The primary research question concerns the impact on school average student 

test scores after the intervention has been in schools for three years.” Finally, please ensure it is 

clear in the summary to whom the sample was designed to generalize.]

 The intervention description is correct. (If this box is checked, the NEi3 Description that 

appears above will be used in the NEi3 Findings Report.) 

 The intervention description requires some revision. (If this box is checked, please enter a 

revision below.)

Evaluator Revision: 

Please insert a revised description here if needed.
National Evaluation of i3 (NEi3) Data Collection Survey Page 9

18. For both the full population served by this grant’s i3-funded intervention, and the sample of 

the population included in this study’s impact estimates, please provide the following statistics 

at the student-level if available and at the school-level if the student-level is not available:

Full Population Served by this 

Grant’s i3-Funded Intervention

Sample of the Population 

Included in this Study’s 

Impact Estimates

Location 

City n_____ (___%) n_____ (___%)

Suburban n_____ (___%) n_____ (___%)

Town n_____ (___%) n_____ (___%)

Rural n_____ (___%) n_____ (___%)

Demographics

American Ind/Alaskan n_____ (___%) n_____ (___%)

Asian/Pacific Islander n_____ (___%) n_____ (___%)

Black n_____ (___%) n_____ (___%)

Hispanic n_____ (___%) n_____ (___%)

White n_____ (___%) n_____ (___%)

Two or more races n_____ (___%) n_____ (___%)

Region 

Northeast n_____ (___%) n_____ (___%)

Southeast n_____ (___%) n_____ (___%)

Midwest n_____ (___%) n_____ (___%)

Southwest n_____ (___%) n_____ (___%)

West n_____ (___%) n_____ (___%)

Grade Level 

Before Kindergarten n_____ (___%) n_____ (___%)

K-5 n_____ (___%) n_____ (___%)

6-8 n_____ (___%) n_____ (___%)

9-12 n_____ (___%) n_____ (___%)

School Ranking 

Student attending Priority 

Schools/ Priority Schools

n_____ (___%) n_____ (___%)

Students attending Focus 

Schools/Focus Schools

n_____ (___%) n_____ (___%)

Other i3 Priority Groups

English Language Learner n_____ (___%) n_____ (___%)

Free/Reduced-Price Lunch n_____ (___%) n_____ (___%)

Overall Number n_____ n_____

Please indicate the level at which the above data are reported: 

Please insert a revised description here if needed.
National Evaluation of i3 (NEi3) Data Collection Survey Page 10

Full Population Served by this 

Grant’s i3-Funded Intervention

Sample of the Population 

Included in this Study’s 

Impact Estimates

 student-level 

 school-level 
National Evaluation of i3 (NEi3) Data Collection Survey Page 11

Please complete a row for each of the key components of the i3 intervention that is included in your logic model 

and fidelity measurement system. The following instructions define the data that should be entered in each of 

the columns. 

Column Column Header Instructions for completing corresponding cells 

A Key Component

Please enter the name of the key component. 

Key components are the activities and inputs that are under the direct control of the individual or organization 

responsible for program implementation (e.g., program developer, grant recipient), and are considered by the 

developer to be essential for implementing the intervention. Key components may include financial resources, 

professional development for teachers, curricular materials, or technology products.

B Indicators Characterizing Key Component

Using your study’s logic model, design summary, and fidelity matrix as a guide (whenever applicable), please 

provide the definition of each key component, i.e., a list of the indicators that represent each key component. 

Indicators should be quantified where possible. Depending upon how you defined each key component, 

indicators might encompass a combination of activities at the teacher, classroom, school, or sample level. 

Example: Fidelity of implementation for the key component “professional development” is based on four 

professional development indicators: a one-week summer institute, five webinars, two coaching sessions, and 

ongoing PD.

C

Year 1: Component Level Threshold for 

Fidelity of Implementation for the Unit that 

is the Basis for the Sample-Level 

Representation of Fidelity (i.e. definition of 

implementation with fidelity)

Please provide the definition of implementation with fidelity for the unit that is the basis for the sample-level 

representation of fidelity in Year 1 for each key component; i.e., the component level threshold that delineates 

implementation with fidelity from implementation without fidelity for a particular sample unit. 

Example of unit-level: If the sample-level fidelity threshold is defined as the percent of schools that have 

received an adequate score on a key component, the unit that is the basis for the sample-level representation of 

fidelity is “school”. The definition of implementation with fidelity could be: “a single school will be considered to 

have implemented the key component with fidelity if at least 90% of teachers at the school: attended the oneweek summer institute, attended four of five webinars, attended two of two coaching sessions, and received 

ongoing PD as needed (as self-reported in teacher survey)."
National Evaluation of i3 (NEi3) Data Collection Survey Page 12

Column Column Header Instructions for completing corresponding cells 

D

Year 1: Component Level Threshold for 

Fidelity of Implementation at the Sample 

Level (i.e. definition of implementation with 

fidelity)

Please provide the definition of implementation with fidelity at the sample-level in Year 1 for each key component; 

i.e., the component level threshold that delineates implementation with fidelity from implementation without 

fidelity across sample units. 

Examples of sample-level: “At least 80% of schools implement with fidelity” or “60% or more of districts 

implement with fidelity”.

The sample-level threshold (e.g., across schools) should be a summary measure of fidelity score ranges and 

thresholds established at level of the unit that is the basis for the sample-level representation of fidelity (e.g., 

teacher, classroom, school). 

E

Year 1: Component Level Fidelity Score for 

the Entire Sample

Please report the component level fidelity scores achieved in Year 1 of the intervention for each key component. 

Examples: “85% of schools implemented with fidelity” or “70% of districts implemented with fidelity”

F

Year 1: Component Level Implementation 

with Fidelity at the Sample Level? (Yes/No)

Note: This field is automatically calculated based on the two prior fields.

G

Year 2: Component Level Threshold for 

Fidelity of Implementation for the Unit that 

is the Basis for the Sample-Level 

Representation of Fidelity (i.e. definition of 

implementation with fidelity)

Same instructions as in column C will appear in survey instrument, replacing year one with year two.

H

Year 2: Component Level Threshold for 

Fidelity of Implementation at the Sample 

Level (i.e. definition of implementation with 

fidelity)

Same instructions as in column D will appear in survey instrument, replacing year one with year two.

I

Year 2: Component Level Fidelity Score for 

the Entire Sample

Same instructions as in column E will appear in survey instrument, replacing year one with year two.

J

Year 2: Component Level Implementation 

with Fidelity at the Sample Level? (Yes/No)

Same instructions as in column F will appear in survey instrument, replacing year one with year two.

K

Year 3: Component Level Threshold for 

Fidelity of Implementation for the Unit that 

is the Basis for the Sample-Level 

Representation of Fidelity (i.e. definition of 

implementation with fidelity)

Same instructions as in column C will appear in survey instrument, replacing year one with year three.

L

Year 3: Component Level Threshold for 

Fidelity of Implementation at the Sample 

Level (i.e. definition of implementation with 

fidelity)

Same instructions as in column D will appear in survey instrument, replacing year one with year three.
National Evaluation of i3 (NEi3) Data Collection Survey Page 13

Column Column Header Instructions for completing corresponding cells 

M

Year 3: Component Level Fidelity Score for 

the Entire Sample

Same instructions as in column E will appear in survey instrument, replacing year one with year three.

N

Year 3: Component Level Implementation 

with Fidelity at the Sample Level? (Yes/No)

Same instructions as in column F will appear in survey instrument, replacing year one with year three.

O

Year 4: Component Level Threshold for 

Fidelity of Implementation for the Unit that 

is the Basis for the Sample-Level 

Representation of Fidelity (i.e. definition of 

implementation with fidelity)

Same instructions as in column C will appear in survey instrument, replacing year one with year four.

P

Year 4: Component Level Threshold for 

Fidelity of Implementation at the Sample 

Level (i.e. definition of implementation with 

fidelity)

Same instructions as in column D will appear in survey instrument, replacing year one with year four.

Q

Year 4: Component Level Fidelity Score for 

the Entire Sample

Same instructions as in column E will appear in survey instrument, replacing year one with year four.

R

Year 4: Component Level Implementation 

with Fidelity at the Sample Level? (Yes/No)

Same instructions as in column F will appear in survey instrument, replacing year one with year four.

S

Year 5: Component Level Threshold for 

Fidelity of Implementation for the Unit that 

is the Basis for the Sample-Level 

Representation of Fidelity (i.e. definition of 

implementation with fidelity)

Same instructions as in column C will appear in survey instrument, replacing year one with year five.

T

Year 5: Component Level Threshold for 

Fidelity of Implementation at the Sample 

Level (i.e. definition of implementation with 

fidelity)

Same instructions as in column D will appear in survey instrument, replacing year one with year five.

U

Year 5: Component Level Fidelity Score for 

the Entire Sample

Same instructions as in column E will appear in survey instrument, replacing year one with year five.

V

Year 5: Component Level Implementation 

with Fidelity at the Sample Level? (Yes/No)

Same instructions as in column F will appear in survey instrument, replacing year one with year five.
National Evaluation of i3 (NEi3) Data Collection Survey Page 14

Please complete a row for each contrast measured by your impact study. The following instructions define the 

data that should be entered in each of the columns. 

Column Column Header

Instructions for completing corresponding cells (for each contrast)

INTERNAL USE ONLY: 

Programming notes about skip 

patterms

Contrast Description

A Contrast Name

Please enter the names of each of the evaluation’s confirmatory and exploratory 

contrasts. This field does not affect the assessment or reporting of the contrast; 

the contrast name simply facilities communication between the evaluator and the 

NEi3 AR Team, and facilitates referencing contrasts between the data-entry 

fields of this survey. The contrast name should uniquely identify the contrast and 

differentiate among similar contrasts: e.g. “Math Achievement-confirmatory” and 

“Math Achievement-cohort 1” only might be the names of two similar contrasts.

B

Confirmatory or 

Exploratory?

Indicate whether this contrast is confirmatory or exploratory. Please note that 

any pre-specified contrasts that were registered as confirmatory will be prepopulated as such and cannot be changed to be exploratory. All other contrasts 

will be listed as exploratory. 

Contrast Description: Key features that define a contrast

C Outcome Domain

For confirmatory contrasts, this field is populated with the registered domain 

name of your pre-specified confirmatory contrast. As the domain is a defining 

feature of a contrast, it cannot be edited.

For exploratory contrasts, please enter the name of the domains measured. If 

the exploratory contrast measures the same construct as one of the confirmatory 

contrasts, please use the same domain name. 
National Evaluation of i3 (NEi3) Data Collection Survey Page 15

Column Column Header

Instructions for completing corresponding cells (for each contrast)

INTERNAL USE ONLY: 

Programming notes about skip 

patterms

D Treatment Condition

For confirmatory contrasts, this field is populated with the registered description 

of the intervention tested by the pre-specified confirmatory contrast. As the 

treatment condition is a defining feature of a contrast, it cannot be edited.

For exploratory contrasts, please name the intervention(s) experienced by the 

treatment group in the study. Do not use acronyms. If the condition is the same 

as the one tested by a confirmatory contrast, please use the same name.

If the contrast involves comparing one treatment to another, please identify one 

treatment as the condition and another as the comparison condition for the 

purposes of responding to this survey. Use this identification throughout the 

survey when asked about the treatment/intervention group and the comparison 

group.

E Comparison Condition

For confirmatory contrasts, this field is populated with the registered description 

of the counterfactual condition in the pre-specified confirmatory contrast. As the 

treatment condition is a defining feature of a contrast, it cannot be edited.

For exploratory contrasts, please name the intervention(s) experienced by the 

comparison group in the study. This may be the name of another intervention or 

Business as Usual, if no specific intervention. Do not use acronyms. If the 

condition is the same as the one tested by a confirmatory contrast, please use 

the same name.

If the contrast involves comparing one treatment to another, please identify one 

treatment as the condition and another as the comparison condition for the 

purposes of responding to this survey. Use this identification throughout the 

survey when asked about the treatment/intervention group and the comparison 

group

F Educational Level

For confirmatory contrasts, this field is populated with the educational level that 

was registered in the pre-specified confirmatory contrast. As the educational level 

is a defining feature of a contrast, it cannot be edited.

For exploratory contrasts, please report the educational level (e.g. elementary, 

middle or high school) of the students that experience the intervention. For 

teacher outcomes, report the educational level of the students in the teachers’ 

classrooms.
National Evaluation of i3 (NEi3) Data Collection Survey Page 16

Column Column Header

Instructions for completing corresponding cells (for each contrast)

INTERNAL USE ONLY: 

Programming notes about skip 

patterms

Contrast Description: Approach to addressing contrast

G Design

Please select one of the following designs: RCT, RDD, QED, Comparison Group 

Design Without a Pretest, ITS, Pre-post.

For confirmatory contrasts, this field is populated with the registered design for 

the contrast. If you edit the pre-populated contents of this field – which suggests 

a change to the study plan registered with the NEi3 – please explain why this 

change was made using the pop-up box that will appear.

Definitions:

Randomized controlled trial (RCT). Under this design, an evaluation uses a 

random process to assign study units to an intervention groups and to a control 

group, and measures the effect of the intervention by comparing the outcomes of 

units assigned to the two groups. 

Regression discontinuity design (RDD). Under this design, an evaluation would 

use a continuous “scoring” rule to assign study units to the intervention. Units 

with scores above a pre-set cutoff value are assigned to the treatment group and

units with scores below the cutoff value are assigned to the comparison group, or 

vice versa. The evaluation then compares the outcomes of the two groups close 

to the cutoff to measure the effect of the intervention. 

Quasi-Experimental Design (QED). A QED compares outcomes for intervention 

participants with outcomes for a comparison group chosen through methods 

other than randomization (RCT) or a scoring rule with a well-defined threshold 

(RDD) and includes one or more pre-intervention measures. Comparative 

Interrupted Time Series designs are treated as QEDs by the NEi3.

Comparison Group Design Without a Pretest. A comparison group design 

without a pre-test compares outcomes for intervention participants with outcomes 

for a comparison group chosen through methods other than randomization (RCT) 

or a scoring rule with a well-defined threshold (RDD) and does not include preintervention observations.

Interrupted Time Series Design (ITS). An ITS design uses observations at 

multiple points in time both before and after the intervention was implemented, 
National Evaluation of i3 (NEi3) Data Collection Survey Page 17

Column Column Header

Instructions for completing corresponding cells (for each contrast)

INTERNAL USE ONLY: 

Programming notes about skip 

patterms

and compares the outcomes after the intervention with the trend established prior 

to the intervention. These designs do not use a comparison group.

Pre-post Design. A pre-post design compares outcomes for intervention 

participants after receiving the intervention to their outcomes before receiving the 

intervention. These designs do not use a comparison group.

H Unit of Assignment

For each contrast, please report the unit of assignment or the level at which 

assignment to the treatment and comparison conditions occur (e.g., 

student, teacher, classroom, or school). If you edit the pre-populated contents 

of this field – which suggests a change to the study plan registered with the NEi3 

– please explain why this change was made using the pop-up box that will 

appear.

I

Intervention Group 

[alternately, “Treatment 

Group”]: Description

For confirmatory contrasts, this cell is prepopulated with the registered 

description of the intervention group for the contrast. If you edit the pre-populated 

contents of this field – which suggests a change to the study plan registered with 

the NEi3 – please explain why this change was made using the pop-up box that 

will appear.

For exploratory contrasts and for confirmatory contrasts you wish to change, 

please describe which schools, teachers and/or students will be included in 

the intervention (“treatment”) group for this analysis.

Example 1 (Candy Valley): “2012 first-graders in all Candy Valley District 

Elementary Schools reading below grade level who do not have an IEP and were 

randomly assigned to the intervention.” 

Example 2 (Apple Valley): “This contrast pools data across all treated cohorts 

and schools. All students in treated grades in Fuji Elementary (K-2), Granny 

Smith Elementary (K-1) and Red Delicious Elementary (K-2) who were present 

for data collection in Spring 2014 are included in the treatment group.”

J

Age/Grade during the 

Intervention

For confirmatory contrasts, this cell is prepopulated with the registered 

description of the age/grade of the students in the treatment group for the 

contrast. If you edit the pre-populated contents of this field – which suggests a 

change to the study plan registered with the NEi3 – please explain why this 

change was made using the pop-up box that will appear.
National Evaluation of i3 (NEi3) Data Collection Survey Page 18

Column Column Header

Instructions for completing corresponding cells (for each contrast)

INTERNAL USE ONLY: 

Programming notes about skip 

patterms

For exploratory contrasts and for confirmatory contrasts you wish to change, 

please describe the age or grade-level of students in the intervention group 

during their exposure to the intervention. For teacher outcomes, please enter 

the grade(s) taught by the teachers.

Example 1 (Candy Valley): “The intervention was fielded to first grade students.” 

Example 2 (Apple Valley): “Both Fuji and Red Delicious Elementary Schools 

introduced the intervention in the 2011-2012 school year to all kindergarten 

students, offered the intervention to all kindergarten and first graders in 2012-

2013 and to kindergarten through second graders in 2013-2014. Granny Smith 

Elementary introduced the intervention in the 2012-2013 school year to all 

kindergarten students and offered the intervention to all kindergarten and first 

graders in 2013-2014.”

K

Length of Student 

Exposure to Intervention

For confirmatory contrasts, this cell is prepopulated with the registered 

description of the age/grade of the students in the treatment group for the 

contrast. If you edit the pre-populated contents of this field – which suggests a 

change to the study plan registered with the NEi3 – please explain why this 

change was made using the pop-up box that will appear.

For exploratory contrasts and for confirmatory contrasts you wish to change, 

please describe how long (no. of semesters/years) the students in the 

intervention group were engaged in the intervention at the time the 

outcome was measured for this contrast. If you edit the pre-populated 

contents of this field – which suggests a change to the study plan registered with 

the NEi3 – please explain why this change was made using the pop-up box that 

will appear.

Example 1 (Candy Valley): “Students received one year of treatment” 

Example 2 (Apple Valley): “When outcomes were measured in Spring 2014, 

both Fuji and Red Delicious Elementary Schools students had the following 

pattern of exposure to treatment: second grade students had three years of 

treatment, first grade students had two years of treatment and kindergarten 

students had one year of treatment. When outcomes were measured in Spring 

2014, Granny Smith Elementary first graders had two years of treatment and 

kindergarten students had one year of treatment.”
National Evaluation of i3 (NEi3) Data Collection Survey Page 19

Column Column Header

Instructions for completing corresponding cells (for each contrast)

INTERNAL USE ONLY: 

Programming notes about skip 

patterms

L

Comparison Group: 

Description

For confirmatory contrasts, this cell is prepopulated with the registered 

description of the comparison group for the contrast. If you edit the pre-populated 

contents of this field – which suggests a change to the study plan registered with 

the NEi3 – please explain why this change was made using the pop-up box that 

will appear.

For exploratory contrasts and for confirmatory contrasts you wish to change, 

please describe which schools, teachers and/or students will be included in 

the comparison group for this analysis.

Example 1 (Candy Valley): “2012 first-graders in all Candy Valley District 

Elementary Schools reading below grade level who do not have an IEP and were 

randomly assigned to the control group.”

Example 2 (Apple Valley): “This contrast pools data across all cohorts and 

schools matched to treated cohorts and schools. K-2 students from Honey Crisp 

Elementary (comparison for Fuji Elementary), K-1 students from Ginger Gold 

Elementary serves (comparison for Granny Smith Elementary), and K-2 students 

Jonagold Elementary (comparison for Red Delicious Elementary) who were 

present for data collection in Spring 2014 are included in the treatment group.”

M

Outcome or Instrument 

Name (full names, not 

acronyms)

For confirmatory contrasts, this cell is prepopulated with the registered outcome 

measure for the contrast. If you edit the pre-populated contents of this field –

which suggests a change to the study plan registered with the NEi3 – please 

explain why this change was made using the pop-up box that will appear.

For exploratory contrasts and for confirmatory contrasts you wish to change, 

please report the full name of the outcome measure, including the name of the 

subtest or subscale if applicable. If the instrument for the outcome measure is not 

used in one of the confirmatory contrasts, please describe (1) what the 

instrument is designed to measure and (2) how the outcome measure is 

constructed. 
National Evaluation of i3 (NEi3) Data Collection Survey Page 20

Column Column Header

Instructions for completing corresponding cells (for each contrast)

INTERNAL USE ONLY: 

Programming notes about skip 

patterms

N

Construction of Outcome 

Measure (binary, ordinal, 

or continuous)

For confirmatory contrasts, this cell is prepopulated with the registered scale of 

the outcome measure for the contrast. If you edit the pre-populated contents of 

this field – which suggests a change to the study plan registered with the NEi3 –

please explain why this change was made using the pop-up box that will appear.

For exploratory contrasts and for confirmatory contrasts you wish to change, 

please indicate whether the outcome measure is binary (dichotomous), ordinal 

(ordered, e.g., “failing, provisionally passing, proficient, advanced”), or 

continuous.

O

Level at which Outcomes 

are Observed

For confirmatory contrasts, this cell is prepopulated with the registered level at 

which outcomes are observed, called level of analysis in some NEi3 

correspondence. If you edit the pre-populated contents of this field – which 

suggests a change to the study plan registered with the NEi3 – please explain 

why this change was made using the pop-up box that will appear.

For exploratory contrasts and for confirmatory contrasts you wish to change, 

please report the level at which outcomes are observed in your data (e.g. 

student, teacher, classroom, school, district).

Example: If your data are school-level averages of student test scores, enter 

“School”.

P

Timing of Outcome

Measure (year, Fall/Spring) 

for the Intervention Group

For confirmatory contrasts, this cell is prepopulated with the registered timing of 

measurement for the contrast. If you edit the pre-populated contents of this field –

which suggests a change to the study plan registered with the NEi3 – please 

explain why this change was made using the pop-up box that will appear.

For exploratory contrasts and for confirmatory contrasts you wish to change,

please report when during the calendar year the outcome was measured.

Q

Timing of Outcome

Measure (age/grade) for 

the Intervention Group

For confirmatory contrasts, this cell is prepopulated with the registered timing of 

measurement for the contrast. If you edit the pre-populated contents of this field –

which suggests a change to the study plan registered with the NEi3 – please 

explain why this change was made using the pop-up box that will appear.

For exploratory contrasts and for confirmatory contrasts you wish to change,

please report the age or grade level for which the outcome was measured. 
National Evaluation of i3 (NEi3) Data Collection Survey Page 21

Column Column Header

Instructions for completing corresponding cells (for each contrast)

INTERNAL USE ONLY: 

Programming notes about skip 

patterms

R

Timing of Outcome

Measure (year, Fall/Spring) 

for the Comparison Group

For confirmatory contrasts, this cell is prepopulated with the registered timing of 

measurement for the contrast. If you edit the pre-populated contents of this field –

which suggests a change to the study plan registered with the NEi3 – please 

explain why this change was made using the pop-up box that will appear.

For exploratory contrasts and for confirmatory contrasts you wish to change,

please report when during the calendar year the outcome was measured. 

S

Timing of Outcome

Measure (age/grade) for 

the Comparison Group

For confirmatory contrasts, this cell is prepopulated with the registered timing of 

measurement for the contrast. If you edit the pre-populated contents of this field –

which suggests a change to the study plan registered with the NEi3 – please 

explain why this change was made using the pop-up box that will appear.

For exploratory contrasts and for confirmatory contrasts you wish to change,

please report the age or grade level for which the outcome was measured. 

T Key Baseline Measures

For confirmatory contrasts, this cell is prepopulated with the registered baseline 

measure for the contrast. If you edit the pre-populated contents of this field –

which suggests a change to the study plan registered with the NEi3 – please 

explain why this change was made using the pop-up box that will appear.

For exploratory contrasts and for confirmatory contrasts you wish to change,

please report the baseline variable(s) on which you are establishing baseline 

equivalence.

(Field is not applicable to the following designs: comparison group designs 

without a pretest, ITS designs and pre-post designs.)

Programming notes:If the Design is 

Comparison Group without a Pretest, 

ITS or Pre-post, display “Not Applicable”

U

Construction of Baseline 

Measure (binary, ordinal, 

or continuous)

For confirmatory contrasts, this cell is prepopulated with the registered scale of 

the baseline measure for the contrast. If you edit the pre-populated contents of 

this field – which suggests a change to the study plan registered with the NEi3 –

please explain why this change was made using the pop-up box that will appear.

For exploratory contrasts and for confirmatory contrasts you wish to change,

please indicate whether the baseline measure is binary (dichotomous), ordinal 

(ordered, e.g., “failing, provisionally passing, proficient, advanced”), or 

continuous. 

(Field is not applicable to the following designs: comparison group designs 

without a pretest, ITS designs and pre-post designs.)

Programming notes: If the Design is 

Comparison Group without a Pretest, 

ITS or Pre-post, display “Not Applicable”
National Evaluation of i3 (NEi3) Data Collection Survey Page 22

Column Column Header

Instructions for completing corresponding cells (for each contrast)

INTERNAL USE ONLY: 

Programming notes about skip 

patterms

V

Level of at which the 

Baseline Measure is 

Observed

For confirmatory contrasts, this cell is prepopulated with the registered level of 

observation for the baseline measure for the contrast. If you edit the prepopulated contents of this field – which suggests a change to the study plan 

registered with the NEi3 – please explain why this change was made using the 

pop-up box that will appear.

For exploratory contrasts and for confirmatory contrasts you wish to change, 

please report the level at which baseline measures are observed in your data 

(e.g. student, teacher, classroom, school, district).

Example: If your data are school-level averages of student test scores, enter 

“School”.

(Field is not applicable to the following designs: comparison group designs 

without a pretest, ITS designs and pre-post designs.)

Programming notes: If the Design is 

Comparison Group without a Pretest, 

ITS or Pre-post, display “Not Applicable”

W

Timing of Baseline 

Measure (year, 

Fall/Spring,) for the 

Intervention Group

For confirmatory contrasts, this cell is prepopulated with the registered timing of 

baseline measurement for the contrast.

For exploratory contrasts and for confirmatory contrasts you wish to change, 

please report when during the calendar year the data you will use to test 

baseline equivalence was measured. 

(Field is not applicable to the following designs: comparison group designs 

without a pretest, ITS designs and pre-post designs.)

Programming notes: If the Design is 

Comparison Group without a Pretest, 

ITS or Pre-post, display “Not Applicable”

X

Timing of Baseline 

Measure (age/grade) for 

the Intervention Group

For confirmatory contrasts, this cell is prepopulated with the registered timing of 

baseline measurement for the contrast. If you edit the pre-populated contents of 

this field – which suggests a change to the study plan registered with the NEi3 –

please explain why this change was made using the pop-up box that will appear.

For exploratory contrasts and for confirmatory contrasts you wish to change, 

please report the age or grade level for which the data you will use to test 

baseline equivalence was measured. 

(Field is not applicable to the following designs: comparison group designs 

without a pretest, ITS designs and pre-post designs.)

Programming notes: If the Design is 

Comparison Group without a Pretest, 

ITS or Pre-post, display “Not Applicable”
National Evaluation of i3 (NEi3) Data Collection Survey Page 23

Column Column Header

Instructions for completing corresponding cells (for each contrast)

INTERNAL USE ONLY: 

Programming notes about skip 

patterms

Y

Timing of Baseline 

Measure (year, Fall/Spring) 

for the Comparison Group

For confirmatory contrasts, this cell is prepopulated with the registered timing of 

baseline measurement for the contrast. If you edit the pre-populated contents of 

this field – which suggests a change to the study plan registered with the NEi3 –

please explain why this change was made using the pop-up box that will appear.

For exploratory contrasts and for confirmatory contrasts you wish to change, 

please report when during the calendar year the data you will use to test 

baseline equivalence was measured. 

(Field is not applicable to the following designs: comparison group designs 

without a pretest, ITS designs and pre-post designs.)

Programming notes: If the Design is 

Comparison Group without a Pretest, 

ITS or Pre-post, display “Not Applicable”

Z

Timing of Baseline 

Measure (age/grade) for 

the Comparison Group

For confirmatory contrasts, this cell is prepopulated with the registered timing of 

baseline measurement for the contrast. If you edit the pre-populated contents of 

this field – which suggests a change to the study plan registered with the NEi3 –

please explain why this change was made using the pop-up box that will appear.

For exploratory contrasts and for confirmatory contrasts you wish to change, 

please report when the data you will use to test baseline equivalence was 

measured. 

(Field is not applicable to the following designs: comparison group designs 

without a pretest, ITS designs and pre-post designs.)

Programming notes:. If the Design is 

Comparison Group without a Pretest, 

ITS or Pre-post, display “Not Applicable”

Confounds

AA Treatment Condition -

Other Activities

To your knowledge, were any other interventions (other than your project’s i3-

funded intervention) targeting the outcome domain listed in [C] that were offered 

to the treatment group but not the comparison group? If so, please describe the 

other intervention.

AB Comparison Condition -

Other Activities

To your knowledge, were any other interventions (other than your project’s i3-

funded intervention) targeting the outcome domain listed in [C] that were offered 

to the comparison group but not the treatment group? If so, please describe the 

other intervention.

AC

Fundamental Confound at 

the District Level: 

Treatment Condition

Does the treatment condition include schools/teachers/students from a single 

school district which does not also include comparison group members?
National Evaluation of i3 (NEi3) Data Collection Survey Page 24

Column Column Header

Instructions for completing corresponding cells (for each contrast)

INTERNAL USE ONLY: 

Programming notes about skip 

patterms

AD

Fundamental Confound at 

the District Level: 

Comparison Condition

Does the comparison condition include schools/teachers/students from a 

single school district which does not also include treatment group members?

AE

Fundamental Confound at 

the School Level: 

Treatment Condition

Does the treatment condition include teachers/students from a single school 

which does not also include comparison group members?

AF

Fundamental Confound at 

the School Level: 

Comparison Condition

Does the comparison condition include teachers/students from a single 

school which does not also include treatment group members?

Outcomes

AG

Outcome or Instrument 

Name (full names, not 

acronyms)

This field displays the outcome measure you entered previously. This field is not 

editable. If you wish to edit this entry, go to <reference location>.

AH

Who Provided the 

Outcome Data?

For each contrast, please report the name of the entity that provided the 

outcome data. 

AI

Value and Type of 

Reliability Measure of 

Instrument for Outcome 

Data

Please provide a narrative description of the outcome and discuss the reliability 

and validity of the measure, including the value and type of reliability measure. 

For example, you may report an internal consistency, test-retest reliability, or 

inter-rater reliability measure.

For outcomes for which reliability statistics cannot be obtained, please describe 

why reliability statistics are not available.

Example 1: “The Orange Gulch District writing assessment was scored by 

trained reviewers according to a holistic rubric. A random subset of essays was 

double scored and demonstrated an inter-rater reliability of 89%.”

Example 2: “Attendance is the proportion of school days for which an individual 

student was present for at least one course at Orange Gulch High School. 

Reliability statistics cannot be calculated for attendance. However, it is a face 

valid measure of the pre-specified domain: High School Attendance.” 
National Evaluation of i3 (NEi3) Data Collection Survey Page 25

Column Column Header

Instructions for completing corresponding cells (for each contrast)

INTERNAL USE ONLY: 

Programming notes about skip 

patterms

AJ Consistency of Definition

Please describe any differences in the definition of the outcome measure 

between the groups.

To be consistently defined, an outcome measure must be defined using the same 

rules or procedures in both groups. Standardized tests (e.g., the Terra Nova or 

Stanford 10) and state-required tests are assumed to be consistently defined. For 

other outcomes (e.g., measures of grades or GPA, course credits, and 

graduation), the evaluator should provide an assessment of whether the outcome 

is consistently defined in all study groups. 

AK Consistency of Collection

Please describe any differences in the process for collecting data on the outcome 

measure between the groups.

To be consistently collected, an outcome measure must be collected using the 

same rules or procedures in both groups. For example, data would not be 

consistently collected if treatment students take the post-test in January, but 

control students take the test in March – two months later.

AL Treatment of Missing Data Describe how the study addressed missing values for the outcome measure.

Assessing Attrition

AM Level of Assignment

This field displays the unit of assignment you entered previously. This field is not 

editable. If you wish to edit this entry, go to <reference location>.

(Section is not applicable to the following designs: QEDs, comparison group 

designs without a pretest, ITS designs and pre-post designs.)

Programming note: Display “Not 

Applicable” if the Design is not RCT. 

AN Level at which Outcomes 

are Observed

This field displays the level at which outcomes are observed entered previously. 

This field is not editable. If you wish to edit this entry, go to <reference location>.

(Section is not applicable to the following designs: QEDs, comparison group 

designs without a pretest, ITS designs and pre-post designs.)

Programming note: Display “Not 

Applicable” if the Design is not RCT.

AO Cluster Randomized 

Design?

Select “Yes” if the contrast is from a cluster randomized design.

Select “No”, otherwise 

For NEi3, all cluster randomized designs are those in which clusters (of 

individuals) were assigned by a process that was functionally random. These 

clusters may be associated with, for instance, particular teachers, classrooms, 

schools, or districts. 

Programming note: Display “Not 

Applicable” if the Design is not RCT.
National Evaluation of i3 (NEi3) Data Collection Survey Page 26

Column Column Header

Instructions for completing corresponding cells (for each contrast)

INTERNAL USE ONLY: 

Programming notes about skip 

patterms

(Section is not applicable to the following designs: QEDs, comparison group 

designs without a pretest, ITS designs and pre-post designs.)

AP Analytic Sample Includes 

Joiners?

Select “not applicable” if your design is not a cluster randomized design.

Select “yes” if your analytic sample includes individuals who were not present in 

the cluster at the time of random assignment.

Select “no” if your analytic sample is restricted to individuals who were present at 

the time of random assignment. 

(Section is not applicable to the following designs: QEDs, comparison group 

designs without a pretest, ITS designs and pre-post designs.)

Programming Note: Display “Not 

Applicable” if the Design is not 

RCT.Display “Not applicable” if the 

contrast is NOT a cluster randomized 

design. 

AQ

Unit of Assignment 

Baseline Sample: 

Intervention Group

Enter the number of clusters randomly assigned to the intervention group.

Complete this field if the contrast is from a cluster randomized design AND the 

unit of assignment differs from the unit of analysis.

(Section is not applicable to the following designs: QEDs, comparison group 

designs without a pretest, ITS designs and pre-post designs.)

Programming Note: Display “Not 

Applicable” if the Design is not 

RCT.Display “Not applicable” if the 

contrast is (a) NOT a cluster randomized 

design OR (b) the level of observation is 

the same as the level of assignment

AR

Unit of Assignment 

Baseline Sample: 

Comparison Group

Enter the number of clusters randomly assigned to the comparison group.

Complete this field if the contrast is from a cluster randomized design AND the 

unit of assignment differs from the unit of analysis

(Section is not applicable to the following designs: QEDs, comparison group 

designs without a pretest, ITS designs and pre-post designs.)

Programming Note: Display “Not 

Applicable” if the Design is not 

RCT.Display “Not applicable” if Design is 

not RCT. Display “Not applicable” if the 

contrast is (a) NOT a cluster randomized 

design OR (b) the level of observation is 

the same as the level of assignment

AS

Unit of Assignment 

Analytic Sample: 

Intervention Group

Enter the number of clusters for the intervention group for the analytic sample.

Complete this field if the contrast is from a cluster randomized design AND the 

unit of assignment differs from the unit of analysis

(Section is not applicable to the following designs: QEDs, comparison group 

designs without a pretest, ITS designs and pre-post designs.)

Programming Note: Display “Not 

Applicable” if the Design is not 

RCT.Display “Not applicable” if the 

contrast is (a) NOT a cluster randomized 

design OR (b) the level of observation is 

the same as the level of assignment
National Evaluation of i3 (NEi3) Data Collection Survey Page 27

Column Column Header

Instructions for completing corresponding cells (for each contrast)

INTERNAL USE ONLY: 

Programming notes about skip 

patterms

AT

Unit of Assignment 

Analytic Sample: 

Comparison Group

Enter the number of clusters for the comparison group for the analytic sample.

Complete this field if the contrast is from a cluster randomized design AND the 

unit of assignment differs from the unit of analysis

(Section is not applicable to the following designs: QEDs, comparison group 

designs without a pretest, ITS designs and pre-post designs.)

Programming Note: Display “Not 

Applicable” if the Design is not RCT. 

Display “Not applicable” if Design is not 

RCT. Display “Not applicable” if the 

contrast is (a) NOT a cluster randomized 

design OR (b) the level of observation is 

the same as the level of assignment

AU

Unit of Analysis Baseline 

Sample: Intervention 

Group

Enter the sample size for the intervention group at the time of random 

assignment

Complete this field the contrast is from:

(a)a cluster randomized design AND the unit of assignment differs from the unit 

of analysis AND the contrast does not include joiners

(b)any randomized design where the unit of analysis is the same as the unit of 

assignment

(Section is not applicable to the following designs: QEDs, comparison group 

designs without a pretest, ITS designs and pre-post designs.)

Programming Note: Display “Not 

Applicable” if the Design is not RCT. 

Display “Not applicable” if the contrast is 

a cluster randomized design AND 

includes joiners AND the level of 

observation does not equal the level of 

assignment

AV

Unit of Analysis Baseline 

Sample: Comparison 

Group

Enter the sample size for the comparison group at the time of random 

assignment

Complete this field the contrast is from:

(c)a cluster randomized design AND the unit of assignment differs from the unit 

of analysis AND the contrast does not include joiners

(d)any randomized design where the unit of analysis is the same as the unit of 

assignment

(Section is not applicable to the following designs: QEDs, comparison group 

designs without a pretest, ITS designs and pre-post designs.)

Programming Note: Display “Not 

Applicable” if the Design is not RCT. 

Display “Not applicable” if the contrast is 

a cluster randomized design AND 

includes joiners AND the level of 

observation does not equal the level of 

assignment

AW

Unit of Analysis Analytic 

Sample: Intervention 

Group

Enter the sample size for the intervention group for the analytic sample

Complete this field the contrast is from:

(e)a cluster randomized design AND the unit of assignment differs from the unit 

of analysis AND the contrast does not include joiners

(f)any randomized design where the unit of analysis is the same as the unit of 

assignment

Programming Note: Display “Not 

Applicable” if the Design is not RCT. 

Display “Not applicable” if the contrast is 

a cluster randomized design AND 

includes joiners AND the level of 

observation does not equal the level of 

assignment
National Evaluation of i3 (NEi3) Data Collection Survey Page 28

Column Column Header

Instructions for completing corresponding cells (for each contrast)

INTERNAL USE ONLY: 

Programming notes about skip 

patterms

(Section is not applicable to the following designs: QEDs, comparison group 

designs without a pretest, ITS designs and pre-post designs.)

AX

Unit of Analysis Analytic 

Sample: Comparison 

Group

Enter the sample size for the comparison group for the analytic sample

Complete this field the contrast is from:

(g)a cluster randomized design AND the unit of assignment differs from the unit 

of analysis AND the contrast does not include joiners

(h)any randomized design where the unit of analysis is the same as the unit of 

assignment

(Section is not applicable to the following designs: QEDs, comparison group 

designs without a pretest, ITS designs and pre-post designs.)

Programming Note: Display “Not 

Applicable” if the Design is not RCT. 

Display “Not applicable” if the contrast is 

a cluster randomized design AND 

includes joiners AND the level of 

observation does not equal the level of 

assignment

Baseline Measurement

AY Key Baseline Measures

This field displays the key baseline measure(s) you entered previously. This field 

is not editable. If you wish to edit this entry, go to <reference location>.

(Section is not applicable to the following designs: comparison group designs 

without a pretest, ITS designs and pre-post designs.)

Programming note: Display “Not 

Applicable” if the Design is not an RCT 

or QED.

AZ

Construction of Baseline 

Measure (binary, ordinal, 

or continuous)

This field displays the description of the construction of the baseline measure you 

entered previously. This field is not editable. If you wish to edit this entry, go to 

<reference location>.

(Section is not applicable to the following designs: comparison group designs 

without a pretest, ITS designs and pre-post designs.)

Programming note: Display “Not 

Applicable” if the Design is not an RCT 

or QED.

BA Level of Observation for

Baseline Equivalence

This field displays the level of observation for the baseline measure you entered 

previously. This field is not editable. If you wish to edit this entry, go to <reference 

location>.

(Section is not applicable to the following designs: comparison group designs 

without a pretest, ITS designs and pre-post designs.)

Programming note: Display “Not 

Applicable” if the Design is not an RCT 

or QED.

BB

Treatment of Missing Data 

for Baseline Balance 

Testing

Describe how the study addressed missing values for the baseline measure for 

baseline balance testing.

Programming note: Display “Not 

Applicable” if the Design is not an RCT 

or QED.
National Evaluation of i3 (NEi3) Data Collection Survey Page 29

Column Column Header

Instructions for completing corresponding cells (for each contrast)

INTERNAL USE ONLY: 

Programming notes about skip 

patterms

BC

Narrative Description of 

Sample for Baseline 

Balance Testing

Describe the eligibility criteria for inclusion in the sample for baseline balance 

testing. If balance is established using cluster-level averages (e.g. school-level 

average student test score), describe which individual units are included in the 

average.

Programming note: Display “Not 

Applicable” if the Design is not an RCT 

or QED.

BD

Type of Information Used 

to Assess Baseline 

Equivalence

Select one option that best represents the information generated by your 

baseline balance testing:

Unadjusted Mean and Standard Deviation

T-statistic

Dichotomous

This field determines which of the following fields you need to complete. Fields 

that do not need to be completed are populated with “Not applicable”

(Section is not applicable to the following designs: comparison group designs 

without a pretest, ITS designs and pre-post designs.)

Programming note: Display “Not 

Applicable” if the Design is not an RCT 

or QED.

BE

Baseline Balance Testing 

Sample Size: Intervention 

Group

Enter the sample size for the intervention group for baseline balance testing

(Section is not applicable to the following designs: comparison group designs 

without a pretest, ITS designs and pre-post designs.)

Programming note: Display “Not 

Applicable” if the Design is not an RCT 

or QED.

BF

Mean of 

Baseline Measure: 

Intervention Group

Enter the pre-intervention mean for the intervention group. For binary measures, 

enter a decimal value.

(Section is not applicable to the following designs: comparison group designs 

without a pretest, ITS designs and pre-post designs.)

Programming note: Display “Not 

Applicable” if the Design is not an RCT 

or QED. Display “Not applicable” if the 

type of information used to assess 

baseline equivalence is “T-statistic”

BG

Standard Deviation of 

Baseline Measure: 

Intervention Group

Enter the pre-intervention standard deviation for the intervention group.

(Section is not applicable to the following designs: comparison group designs 

without a pretest, ITS designs and pre-post designs.)

Programming note: Display “Not 

Applicable” if the Design is not an RCT 

or QED. Display “Not applicable” if the 

type of information used to assess 

baseline equivalence is “T-statistic”

BH

Baseline Balance Testing 

Sample Size: Comparison 

Group

Enter the sample size for the comparison group for baseline balance testing

(Section is not applicable to the following designs: comparison group designs 

without a pretest, ITS designs and pre-post designs.)

Programming note: Display “Not 

Applicable” if the Design is not an RCT 

or QED.
National Evaluation of i3 (NEi3) Data Collection Survey Page 30

Column Column Header

Instructions for completing corresponding cells (for each contrast)

INTERNAL USE ONLY: 

Programming notes about skip 

patterms

BI

Mean of 

Baseline Measure: 

Comparison Group

Enter the pre-intervention mean for the comparison group. For binary measures, 

enter a decimal value.

(Section is not applicable to the following designs: comparison group designs 

without a pretest, ITS designs and pre-post designs.)

Programming note: Display “Not 

Applicable” if the Design is not an RCT 

or QED. Display “Not applicable” if the 

type of information used to assess 

baseline equivalence is “T-statistic”

BJ

Standard Deviation of 

Baseline Measure: 

Comparison Group

Enter the pre-intervention standard deviation for the comparison group.

(Section is not applicable to the following designs: comparison group designs

without a pretest, ITS designs and pre-post designs.)

Programming note: Display “Not 

Applicable” if the Design is not an RCT 

or QED. Display “Not applicable” if the 

type of information used to assess 

baseline equivalence is “T-statistic”

BK T-Statistic for Baseline 

Balance Test

Enter the t-statistic for the pre-intervention difference in means between the 

treatment and the comparison

(Section is not applicable to the following designs: comparison group designs 

without a pretest, ITS designs and pre-post designs.)

Programming note: Display “Not 

Applicable” if the Design is not an RCT 

or QED. Display “Not applicable” if the 

type of information used to assess 

baseline equivalence is “T-statistic”

Reported Findings

BL Type of Findings Reported

Select one of the following types of data to use to report findings:

Unadjusted means 

Adjusted means, e.g. ANOVA

T-test

Coefficient from a linear regression model, e.g. OLS, HLM, etc. 

Coefficient from a non-linear regression model, e.g. logit or probit 

This field determines which of the following fields you need to complete. Fields 

that do not need to be completed are populated with “Not applicable”

BM

Adjusted for Baseline/PreTest Measure? (If 

applicable)

Please describe whether and how your analysis accounted for pre-test 

measures.

Programming Note: Display “Not 

Applicable” if type of findings is 

“Unadjusted means”

BN Adjustment for Clustering 

Used in Analysis?

Please indicate whether or not the analysis accounted for the clustering of 

sample units in larger units, and if applicable describe the type of method used to 

adjust for this clustering (e.g., multi-level modeling or robust standard errors). 

Programming Note: Display “Not 

Applicable” if type of findings is 

“Unadjusted means”

BO Intervention Group: 

Sample Size

Please report the size of the intervention group sample for the analysis.

BP

Intervention Group: 

Number of Clusters in 

Analytic Sample

Please report the number of clusters assigned to the intervention group that 

appear in the analysis sample. 
National Evaluation of i3 (NEi3) Data Collection Survey Page 31

Column Column Header

Instructions for completing corresponding cells (for each contrast)

INTERNAL USE ONLY: 

Programming notes about skip 

patterms

Although your analysis may account for clustering at multiple levels, this field only 

asks about clusters at the level of assignment: e.g. number of schools, if schools 

were assigned.

BQ Comparison Group: 

Sample Size

Please report the size of the comparison group sample for the analysis.

BR

Comparison Group: 

Number of Clusters in 

Analytic Sample

Please report the number of clusters assigned to the comparison group that 

appear in the analysis sample. 

Although your analysis may account for clustering at multiple levels, this field only 

asks about clusters at the level of assignment: e.g. number of schools, if schools 

were assigned.

Programming Note: Display “Not 

applicable” if the level of observation is 

the same as the level of assignment

BS Intraclass Correlation 

Coefficient

Please report the intraclass correlation coefficient to describe the similarity of 

units at the level of observation within the cluster at the level of assignment 

Although your analysis may account for clustering at multiple levels, this field only 

asks about clusters at the level of assignment, e.g. the correlation between 

outcomes of students within the same school, if schools were assigned.

Programming Note: Display “Not 

applicable” if the level of observation is 

the same as the level of assignment

BT

Unadjusted Outcome 

Mean for the Intervention 

Group

Please provide the unadjusted mean of the outcome measure in the intervention 

group.

(You do not need to complete this field if it is automatically populated with “Not 

applicable”. However, you can choose to provide this information if you wish.)

Programming Note: Display “Not 

Applicable” if type of findings is NOT 

“Unadjusted means”

BU Adjusted Outcome Mean 

for the Intervention Group

Please provide the adjusted mean of the outcome measure in the intervention 

group. 

(You do not need to complete this field if it is automatically populated with “Not 

applicable”. However, you can choose to provide this information if you wish.)

Programming Note: Display “Not 

Applicable” if type of findings is NOT 

“Adjusted means”

BV

Unadjusted Standard 

Deviation in the Outcome 

Measure of the 

Intervention Group

Please provide the unadjusted standard deviation of the outcome measure in the 

intervention group.

BW

Unadjusted Outcome 

Mean for the Comparison 

Group

Please provide the unadjusted mean of the outcome measure in the comparison 

group. 

(You do not need to complete this field if it is automatically populated with “Not 

applicable”. However, you can choose to provide this information if you wish.)

Programming Note: Display “Not 

Applicable” if type of findings is NOT 

“Unadjusted means”
National Evaluation of i3 (NEi3) Data Collection Survey Page 32

Column Column Header

Instructions for completing corresponding cells (for each contrast)

INTERNAL USE ONLY: 

Programming notes about skip 

patterms

BX Adjusted Outcome Mean 

for the Comparison Group

Please provide the unadjusted mean of the outcome measure in the comparison 

group. 

(You do not need to complete this field if it is automatically populated with “Not 

applicable”. However, you can choose to provide this information if you wish.)

Programming Note: Display “Not 

Applicable” if type of findings is NOT 

“Adjusted means”

BY

Unadjusted Standard 

Deviation in the Outcome 

Measure of the 

Comparison Group

Please provide the unadjusted standard deviation of the outcome measure in the 

comparison group.

BZ

Impact Estimate (e.g., 

coefficient from OLS or 

HLM)

Please report the regression coefficient that estimates of the effect of the 

intervention. If your impact estimate is the sum of multiple regression 

coefficients, please enter this sum here. 

(You do not need to complete this field if it is automatically populated with “Not 

applicable”.)

Programming Note: Display “Not 

applicable” if the type of findings is NOT 

a coefficient from a linear or non-linear 

regression model.

CA Standard Error of Impact 

Estimate

Please insert the estimated standard error of the regression coefficient here. 

If your impact estimate is the sum of multiple coefficients, please enter the 

standard deviation of the summed variable. 

(You do not need to complete this field if it is automatically populated with “Not 

applicable”.)

Programming Note: Display “Not 

applicable” if the type of findings is NOT 

a coefficient from a linear or non-linear 

regression model.

CB Two-tailed p-value

Please insert the associated p-value that has not been adjusted for multiple 

comparisons here. If your impact estimate is the sum of multiple coefficients, 

please enter the associated p-value here.

(You do not need to complete this field if it is automatically populated with “Not 

applicable”.)

Programming Note: Display “Not 

applicable” if the type of findings is NOT 

a coefficient from a linear or non-linear 

regression model.

CC Degrees of Freedom

Please enter the degrees of freedom of your statistical analysis. 

(You do not need to complete this field if it is automatically populated with “Not 

applicable”.)

Programming Note: Display “Not 

applicable” if the type of findings is 

“unadjusted means”.

CD

Number of Time Periods in 

the Model (ITS or CITS)

For each contrast, please indicate the number of time periods in the model. 

(You do not need to complete this field if it is automatically populated with “Not 

applicable”. However, you may report this information if it is available and you 

wish to do so)

Programming Note: Display “Not 

applicable” if the design is NOT “ITS”