CONTRACT: ED-IES-12-C-0003

Supporting Statement for OMB Clearance of a Survey of 

Principals of Rural Schools Receiving School Improvement 

Grants and Using the Transformation Model 

Section B

August 16, 2013

Revised September 6, 2013

Revised October 8, 2013

Revised October 16, 2013
ii

Contents

Section B: Collection of Information........................................................................................................ 1

B1. Respondent Universe and Sampling Methods................................................................................ 1

B2. Procedures for the Collection of Information.................................................................................. 2

B3. Methods To Maximize Response Rates and Deal With Nonresponse......................................... 4

B4. Tests of Procedures or Methods to be Undertaken......................................................................... 4

B5. Individuals Consulted on Statistical Aspects of the Design.......................................................... 5

References ................................................................................................................................................... 5

List of Tables

Table 1. Characteristics of rural SIG transformation schools vary in Cohort 1..........................2

The following attachments are stored in the “NW S07” folder in the “REL Northwest Memos 

and Correspondence” library:

Attachment B: Copy of Survey and Online Survey Screen Shots

Attachment C: Intro Letter Invitation and Follow-Up Emails
Supporting Statement for OMB Clearance 1

Supporting Statement for OMB Clearance of a Survey of Principals of 

Rural Schools Receiving School Improvement Grants and Using the 

Transformation Model

Section B: Collection of Information

B1. Respondent Universe and Sampling Methods

The Regional Educational Laboratory (REL) Northwest, with assistance from Policy Studies 

Associates (PSA), plans to examine how rural schools implement School Improvement Grants 

(SIGs). These competitive grants have provided two cohorts of low-performing schools with

federal funding for school improvement and require schools to implement one of four school 

improvement models: the transformation model, which involves a variety of reforms; the 

turnaround model, which involves primarily replacing staff; the restart model, which involves

becoming a charter school; and the closure model, which closes the school. Collecting data from 

rural SIG schools is essential to this study, which seeks to inform regional and national supports 

for future school improvement efforts.

REL Northwest was established under the 2012–2017 Regional Education Laboratories Program. 

The current authorization for the program is under the Education Sciences Reform Act of 2002, 

Part D, Section 174, (20 U.S.C. 9564), administered by the Institute of Education Sciences’ (IES’) 

National Center for Education Evaluation and Regional Assistance. IES has approved this study 

plan that collects principal (or proxy) survey data about the implementation of the 

transformation model under SIGs.

The universe for this descriptive study is relatively small (i.e., 211 schools across the nation). In 

order to have a large enough number of respondents to examine the data disaggregated by 

school characteristics, such as geographic location and student demographics, this study 

surveys the entire universe of rural Cohort 1 SIG schools (N = 211) and, therefore, does not use 

any sampling procedures. We identified these 211 Cohort 1 rural schools across the nation that 

implemented the SIG transformation model using the SIG baseline database and the National 

Center on Education Statistics preliminary 2011 data. Hurlburt, Le Floch, Therriault, and Cole 

(2011) compiled the SIG baseline database based on publicly available information. The SIG 

baseline database is available on the U.S. Department of Education website. We defined rural 

schools as those given locale codes 31 through 43 by the National Center for Education Statistics 

(NCES)1.

 

1 NCES has developed locale codes to describe a school’s location ranging from “large urban” to “remote rural.” The 

codes are based on the physical location represented by an address that is matched against a geographic database 

maintained by the Census Bureau.
Supporting Statement for OMB Clearance 2

Because SIG funding goes to schools serving large numbers of low-income students (i.e., those 

eligible for free or reduced-price lunch), rural SIG transformation schools have high levels of 

poverty; however, these schools vary considerably by ethnicity (table 1).

Table 1. Characteristics of rural SIG transformation schools vary in Cohort 1

Number of 

schools

Percentage

non-White

Percentage

Native American

Percentage

free and reducedprice lunch

31 Town, Fringe 6 41% 0% 47%

32 Town, Distant 33 60% 5% 68%

33 Town, Remote 16 75% 18% 81%

41 Rural, Fringe 48 70% 7% 72%

42 Rural, Distant 62 55% 16% 71%

43 Rural, Remote 46 77% 72% 78%

31–43 All Schools 211 65% 24% 72%

Source: Author’s analysis of SIG baseline database.

The schools are spread throughout the United States. Of the 50 states, 42 had Cohort 1 rural SIG 

transformation schools. California and three southern states had the largest number of rural SIG 

transformation schools: 19 schools in Florida, 12 in California, 12 in South Carolina, and 10 in 

North Carolina. The remaining states had fewer than 10 rural SIG transformation schools.

These survey data have not been collected previously. We expect a response rate of 75 percent. 

More details about the response rate are included in B3. The survey is included in attachment B.

B2. Procedures for the Collection of Information

This one-time data collection by REL Northwest and PSA seeks responses from the universe of 

211 rural SIG schools in Cohort 1 implementing the transformation model. Therefore, we will 

not use any sampling methods. Our procedure for collecting information includes several steps: 

a pilot survey, an initial invitation letter, the actual survey, and up to five follow-up contacts.

By the end of the 60 days of public notice through the Federal Register, we will have already 

conducted a pilot of the survey instrument with nine or fewer individuals from rural SIG 

schools that are using the transformation model and received Cohort 1 funding. This pilot, to be 

conducted in October 2013, will allow us to analyze pilot participant responses item-by-item 

and revise the survey to ensure that all items and instructions are relevant and easily 

understood. Cohort 1 principals who participate in the pilot will have the opportunity to update 

their online pilot survey responses when the actual survey is administered.

After we receive OMB clearance, we will begin actual data collection. To conduct the survey, we 

will first send an introductory letter to superintendents and school turnaround leaders in state 

departments of education, superintendents of districts, and principals of rural SIG 

transformation schools (attachment C). The letter emphasizes the importance of the study. To 
Supporting Statement for OMB Clearance 3

provide further incentive to complete the survey, the letter informs recipients that those who 

respond to the survey will receive a link to the published study and will be invited to 

participate in multiple webinars to discuss the study results.

The letter will also inform respondents that REL Northwest/PSA will mail and email principals 

an online survey link within one week. Principals who do not complete the online survey will 

be offered a Word document after several reminder emails, in case they prefer responding to 

this version of the survey. They may return the word document via fax or email. If participants 

wish, they may request a paper document in the mail, and we will provide a paper document 

that is saddle stitched to avoid the pages becoming separated. Either version should take 

approximately 20 minutes to complete.

Finally, the letter informs recipients that principals who are unfamiliar with SIG at their current 

schools will only fill out the beginning of the survey. This portion of the survey asks if the 

principal is familiar with SIG implementation at his or her school. If the principal is not familiar 

with SIG implementation, the survey provides instructions for designating a proxy to complete 

the survey and ends without asking any remaining survey items. The letter also explains that 

current principals who were at their schools during the time period being studied will also be 

allowed to designate a proxy if it will help the school complete the survey promptly. They can 

do so by responding to the email.

One week after sending the introductory letter, we will email an invitation to Cohort 1 rural SIG 

school principals to inviting them to complete the online survey. The online survey will be 

available via Survey Gizmo, a survey administration software program. We will use procedures 

allowing respondents to log in and out of their survey as often as needed to complete it. Two 

weeks later after three reminder emails, nonrespondents will be offered a Word doc version of 

the survey that they can return via fax or email. We will maintain all survey data in our 

password-protected servers under secure conditions. We will conduct up to five follow-up 

emails and/or telephone calls to nonrespondents to achieve at least a 75 percent response rate. If 

the principal is willing we will conduct the survey via phone, if we reach that principal by 

phone in order to reach our 75 percent response rate target.

To clean the survey data, we will import all data into SPSS software. Our initial cleaning efforts 

will focus on identifying principals for whom we have missing data. We expect missing data to 

be minimal since the online version of the survey will prompt principals to fill in any missing 

responses before they submit the survey. However, some principals may opt to take the paper 

version, so these surveys may contain missing data. If this is the case, we will contact principals 

to fill in these missing data. Although we will encourage principals to complete all items on the 

survey and will explain its importance, their participation is voluntary. We understand that 

some principals may decline to respond to all of the survey items.

Once the missing data are retrieved, we will continue our data processing. Our survey dataprocessing procedures will include a visual review of any word document surveys, data editing
Supporting Statement for OMB Clearance 4

as necessary, and a review of initial frequencies to ensure that responses are within acceptable 

ranges and there are no data entry errors.

B3. Methods To Maximize Response Rates and Deal With Nonresponse

Our target response rate is 75 percent. We have set this target based on other similar studies. 

For example, the most recent Schools and Staffing Survey (SASS) of principals by the National 

Center of Education Statistics had a response rate of 79 percent for both public school and 

Bureau of Indian Education principals; however, this estimate may be high since the SASS had 

local survey coordinators (Battle, 2009). In addition, this report did not disaggregate the 

response rates for rural school principals. We suspect that response rates would be somewhat 

lower for rural school principals. For example, PSA assisted with data collection for a survey of 

rural district administrators, which achieved a response rate of 74 percent (Zhang, 2008). Our 

population is principals in low-performing rural schools. These principals may have more job 

stress and be less likely to respond than those in nonrural, higher performing schools. 

Therefore, we have set our target at 75 percent, a lower rate than the SASS and similar to Zhang 

(2008).

We will use several methods to maximize response rates and address nonresponse. First, we 

have minimized the burden for respondents by (1) requesting information that is generally 

available to school leadership, including principals, other administrators, and teacher-leaders 

directly involved in SIG implementation and (2) requesting the minimum amount of 

information required to answer the study questions.

As discussed in B2, we will send an initial introductory letter to superintendents and school 

turnaround leaders in state departments of education, superintendents in districts, and 

principals in selected rural SIG transformation schools. Several other studies have found these 

introductory letters improved survey response rates (Cook, Heath, & Thompson, 2000;

Kaplowitz, Hadlock, & Levine, 2004). We will also send reminder emails as described 

previously. If needed after multiple reminders, we are prepared to administer the survey by 

phone.

As discussed previously, principals who are new to their schools or who are too busy to 

complete the survey may designate a proxy. At some SIG schools, district or teacher leaders are 

very involved with SIG implementation (Scott, McMurrer, McIntosh, & Dibner, 2012). These 

individuals may have more time to respond to the survey and, therefore, increase return rates.

B4. Tests of Procedures or Methods to be Undertaken

Survey items in this data collection are adapted from a previously conducted survey as part of 

an evaluation of SIG in Oregon (Scott, Davis, & Krasnoff, 2012). In addition, as described

previously, we will conduct a pilot of the survey instrument with nine or fewer participants
Supporting Statement for OMB Clearance 5

during the 60-day period of public notice through the Federal Register. We will randomly select 

nine principals of rural SIG schools that received their SIG funds in cohort 1. The pilot will 

follow the survey procedures described for the actual study. Additionally, pilot participants 

will be asked to comment on whether the survey correspondence clearly conveyed the purpose 

of the study and whether the survey instructions and items were clear and easy to follow.

Cohort 1 principals who participate in the pilot will have the opportunity to update their online 

pilot survey responses when the actual survey is administered.

B5. Individuals Consulted on Statistical Aspects of the Design

Two technical experts will play an important role in providing insight and guidance in support 

of this study of the rural SIG transformation model. They include:

 Tom Dee, Professor, Graduate School of Education, Stanford University, 650.723.6847

 Hans Bos, Vice President, Education, American Institutes for Research, 707.701.3004

Both individuals are members of REL Northwest’s Technical Working Group (TWG). The main 

responsibility of TWG members is to provide written feedback on study plans and/or study 

reports and to meet as a group via telephone with the research team to clarify feedback and 

provide additional recommendations. The TWG members also are available to advise the 

researchers as questions come up, speak with REL staff about future research project ideas, and 

generally lend their expertise as needed. All TWG members are experts who are distinguished 

in their fields and extremely knowledgeable about research design and methods.

References

Battle, D. (2009). Characteristics of public, private, and Bureau of Indian Education elementary and 

secondary school principals in the United States: Results from the 2007–08 Schools and Staffing 

Survey (First Look, NCES 2009-323). Retrieved from U.S. Department of Education, 

National Center for Education Statistics website: 

http://nces.ed.gov/pubs2009/2009323.pdf

Cook, C., Heath, F., & Thompson, R.L. (2000). A meta-analysis of response rates in web- or 

Internet-based surveys. Educational and Psychological Measurement, 60(6), 821–836.

Hurlburt, S., Le Floch, K.C., Therriault, S.B., & Cole, S. (2011). Baseline analyses of SIG applications 

and SIG-eligible and SIG-awarded schools (NCEE 2011-4019). Retrieved from U.S. 

Department of Education, Institute of Education Sciences, National Center for Education 

Evaluation and Regional Assistance website: 

http://ies.ed.gov/ncee/pubs/20114019/pdf/20114019.pdf

Kaplowitz, M.D., Hadlock, T.D., & Levine, R. (2004). A comparison of web and mail survey 

response rates. Public Opinion Quarterly, 68(1), 94–101.
Supporting Statement for OMB Clearance 6

Scott, C., Davis, A., & Krasnoff, B. (2012). Oregon School Improvement Grants: Status report 2011–

2012. Portland, OR: Education Northwest.

Scott, C., McMurrer, J., McIntosh, S., & Dibner, K. (2012). Opportunities and obstacles: 

Implementing stimulus-funded School Improvement Grants in Maryland, Michigan, and Idaho. 

Washington, DC: Center on Education Policy.

Zhang, Y. (2008). Some perspectives from rural school districts on the No Child Left Behind Act. 

Retrieved from Center on Education Policy website: http://www.cepdc.org/displayDocument.cfm?DocumentID=308