## B

- 2. Draft Protocol

— School Data Cu lture Survey

This survey will be administered to both treatment and control schools to estimate the impact of

EWIMS on school level outcomes.

This attachment provides additional information on the draft protocols to measure School Data

Culture. Table B

- 1 presents the reliability of the subscales and Key Dimensions of School Data Culture for principals and the reliability of the subscales and sca les measuring principals’ report of School Data Culture. Cronbach’s alpha was calculated for each subscale and scale. It is generally accepted that Cronbach’s alphas above 0.70 indicate adequate internal consistency among the items in a scale.

The reliabi lity statistics of each scale and the number of survey items in each scale are shown below. The alpha statistic for all scales and subscales measuring School Data Culture (Context, Supports for Data Use, and Barriers to Data Use) is above 0.70, suggesting adequate reliability. Although ED’s contractor will revise items to fit the needs of the EWIMS evaluation, these Cronbach’s alphas are promising for the internal consistency of the revised School Data Culture

Scale.

### Table B

- 1. Reliability Estimates of Pr incipals’ Key Dimensions of

Data Use Scales and Subscales

Cronbach’s Alpha

Key
Dimension

Scale Subscales within each Key

Dimension scale Survey

i tem N General

p rincipal

r esponses Reading

i nstruction

–

s pecific Math

i nstruction

–

s pecific

Context 32 0.87 0.87

Assessment/Instructional Context 23 0.82 0.81

State, District, and School Data

Culture 9 0.84

Supports for Data Use 51 0.92

Data Infrastructure 8 0.82

Organizational Supports 31 0.90

Staffing/Human Resources 12 0.93

Barriers to Data Use 13 0.76

Sample size: N = 212 Principals.

Note: The Supports for Data Use scale and Barriers to Data Use scale did not contain survey items that were reading or mathematics specific.

School

- Level Instructional Response is a subscale specific to the Principal Survey.



## Paperwork Burden Statement

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid Office of Management and Budget (OMB) control

number. The valid OMB control number for this information collection is XXXX

- XXXX . The time required to complete this information collection is estimated to average 60 minutes per response. This information collection is voluntary. If you have any comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: U.S. Department of Education, Washington, DC 20202

– 4651. If you have comments or concerns regarding the status of your individual

submission o f this form, write directly to: Christopher Boccanfuso, U.S. Department of Education, Institute of Education Sciences, Room 506D, 555 New Jersey Ave. NW, Washington, DC 20208

- 5500.

Per the policies and procedures required by the Education Sciences Reform A ct of 2002, Title I, Part E, Section 183, responses to this data collection will be used only for statistical purposes. The reports prepared for this study will summarize findings across the sample and will not associate responses with a specific district or individual. Any willful disclosure of such information for nonstatistical purposes, except as required by law, is a class E felony.

## Survey Items

1. How much do you agree or disagree with the following statements about your school’s priorities about using data?

Strongly

d isagree Disagree Agree Strongly

a gree

a. This school has clear goals for using data to improve student outcomes ⃝ ⃝ ⃝ ⃝

b. The school’s data use policies help us address

student needs at our school. ⃝ ⃝ ⃝ ⃝

c. This school has adequate resources (e.g. time, staff, money) to facilitate teachers’ use of

data. ⃝ ⃝ ⃝ ⃝

d. This school has adequate professional development to give teachers the skills to access, interpret

, and make decisions about

data. ⃝ ⃝ ⃝ ⃝



### 2. Now consider the professional climate in your school. To what extent do you agree or disagree with each of the following statements?

Strongly

d isagree Disagree Agree Strongly

a gree

a. Educators in this school are continually learning and seeking new ideas. ⃝ ⃝ ⃝ ⃝

b. Educators are engaged in systematic analysis

of student data. ⃝ ⃝ ⃝ ⃝

c. Educators in this school work hard to match

students with interventions that will meet

their individual needs ⃝ ⃝ ⃝ ⃝

d. Assessment of student performance leads to changes in programming. ⃝ ⃝ ⃝ ⃝

e. Educators in this school regularly examine

student data. ⃝ ⃝ ⃝ ⃝



### 3. To what extent do you agree or disagree with the following statements about your school leadership team (e.g. principal, assistant principals, other key administrators)?

The school leadership team

... Strongly

d isagree Disagree Agree Strongly

a gree

a. Encourages teachers to make decisions based

on data. ⃝ ⃝ ⃝ ⃝

b. F acilitates conversations about using student

data ⃝ ⃝ ⃝ ⃝

c. Commits adequate resources to help teachers interpret and use the student data ⃝ ⃝ ⃝ ⃝

d. Places too much emphasis on using the

student data ⃝ ⃝ ⃝ ⃝

### 4. Please read the following response options and pick the one that best reflects the degree of

support for data use provided by the school leadership team:

o Not at all supportive: Does not make the student data a priority. There is limited

discussion of the data with staff.

o Not very supportive: Occasional support for the student data use in faculty and staff discussions, but administrators do not see using the student data as central to the school’s

mission.

o Supportive: Administration is supportive of teac her’s efforts, speaks positively about the student data with staff, problem

- solves obstacles to using the student data, uses the system themselves.

o Very supportive: Administration is a “cheerleader” for the student data, effectively supports staff use of the system, and sees it as central to school mission.

5. About how often does your school have scheduled meeting time to review student data (e.g.

, in staff meetings, in data team meetings)?

About

once a week 1

– 2

times

per month 1

– 2

times

per quarter On c e

a year My

school

does provide

time for

this

a. Review student data (e.g., state test scores, student work, curriculum

- based unit tests) ⃝ ⃝ ⃝ ⃝ ⃝

b. Discuss individual student achievement ⃝ ⃝ ⃝ ⃝ ⃝

c. Discuss student achievement by subgroup (e.g.

students with disabilities, ELL/LEP, gender, race/ethnicity) ⃝ ⃝ ⃝ ⃝ ⃝

d. Discuss and share instructional strategies ⃝ ⃝ ⃝ ⃝ ⃝

e. Assign students to interventions ⃝ ⃝ ⃝ ⃝ ⃝

f. Discuss students’ progress in interventions ⃝ ⃝ ⃝ ⃝ ⃝



### 6. In the last 12 months, how much professional development was offered to your staff that focused on using data to inform educational decisions?

No PD provided

on this topic Minor

e mphasis Moderate

e mphasis Major

e mphasis

a. Linking student data to classroom practice ⃝ ⃝ ⃝ ⃝

b. Analyzing student data ⃝ ⃝ ⃝ ⃝

c. Identifying strengths or weaknesses of their own teaching ⃝ ⃝ ⃝ ⃝

d. Incorporating student data into lesson planning ⃝ ⃝ ⃝ ⃝No PD provided

on this topic Minor

e mphasis Moderate

e mphasis Major

e mphasis

e. Using data to target interventions for low

performing students ⃝ ⃝ ⃝ ⃝

f. Using data to target interventions for high

performing students ⃝ ⃝ ⃝ ⃝

g. Understanding how data can be used to guide

instruction ⃝ ⃝ ⃝ ⃝



### 7. How many teachers at your school have the following characteristics?

None Some About

h alf Most All

a. The ability to use data from student

assessments ⃝ ⃝ ⃝ ⃝ ⃝

b. The ability to analyze trends in individual student performance over

time ⃝ ⃝ ⃝ ⃝ ⃝

c. The ability to analyze trends in

classroom

- level performance over

time ⃝ ⃝ ⃝ ⃝ ⃝

d. The ability to translate data into knowledge about student strengths and

weaknesses ⃝ ⃝ ⃝ ⃝ ⃝

e. The ability to make instructional changes based on data ⃝ ⃝ ⃝ ⃝ ⃝



### 8. This question concerns how teachers interact with each other in your school. Please indicate the extent to which your teachers do each of the following:

Not at all To a

s light

e xtent To a

m oderate

e xtent To a

g reat

e xtent

a. Meet together to look at trends in the data (or analyze data) ⃝ ⃝ ⃝ ⃝

b. Share ideas about using data to improve teaching with other teachers ⃝ ⃝ ⃝ ⃝

c. Share and discuss student work with other

teachers ⃝ ⃝ ⃝ ⃝

d. Discuss particular lessons that were not very

successful ⃝ ⃝ ⃝ ⃝

e. Discuss beliefs about teaching and learning ⃝ ⃝ ⃝ ⃝

### 9. Please rate how much you agree or disagree with the following statements.

Not at all To a s light

e xtent To a

m oderate

e xtent To a g reat

e xtent

a. Teachers and administrators work together to

review student data

. ⃝ ⃝ ⃝ ⃝

b. Decisions about students’ enrollment in interventions and supports are made as a

team

. ⃝ ⃝ ⃝ ⃝

c. Teachers work together to identify their students strengths and weaknesses

. ⃝ ⃝ ⃝ ⃝

d. Administrators and teachers work together to support their at

- risk students

. ⃝ ⃝ ⃝ ⃝

e. Teachers prefer to review data on their students independently

. ⃝ ⃝ ⃝ ⃝

f. At this school, data review is a team effort

. ⃝ ⃝ ⃝ ⃝



### 10. To what extent do the following factors hinder your ability to use student data to inform

instruction and interventions?

Not at

all To a

minor

extent To a

moderate

extent To a

great

extent

a. Lack of time to study and think about available data ⃝ ⃝ ⃝ ⃝

b. Lack of time to collaborate with others in analyzing and interpreting data ⃝ ⃝ ⃝ ⃝

c. Lack or professional development or training on how

to use the dashboards ⃝ ⃝ ⃝ ⃝

d. Personal discomfort with data analysis ⃝ ⃝ ⃝ ⃝

e. Lack of technology (e.g.

, access to computer with reliable internet connection) ⃝ ⃝ ⃝ ⃝

f. Insufficient amount of data ⃝ ⃝ ⃝ ⃝

g. Data provided too late for use ⃝ ⃝ ⃝ ⃝

h. The data in the system are inaccurate ⃝ ⃝ ⃝ ⃝

i. The pacing guides are too rigid to really allow me to

re

- teach or adapt my instruction to data ⃝ ⃝ ⃝ ⃝

j. Other ( p lease specify) ⃝ ⃝ ⃝ ⃝

### 11. Which of the following best describes your role at your school?

o Principal

o Assistant p rincipal

o Guidance c ounselor

o Other s chool a dministrator (please specify)

_______________________________

12. How long have you been teaching? If this is your first year teaching please select “ one. ”

o Total n umber of y ears:_____(drop down menu 1 through 25 or more)

13. What is the highest level of education you have completed?

o Bachelor's d egree

o Master's d egree

o Professional d egree (Ed

. D

. , Ph

. D

. )

o Other (please specify)

_________________________________________________