Attachment F: Pilot Survey Report

REL Northwest conducted a pilot test of the survey between October 23 and November 8. The 

initial invitation to complete the online survey went out to the sample of nine principals on 

October 23, 2013. Principals were instructed to complete the survey by October 30, 2013. After 

the initial email invitations were sent, two of the emails were undeliverable and one principal 

declined to participate. Through follow-up calls to schools and districts, REL Northwest

discovered that one email was undeliverable because the school had closed and the other 

because the principal was no longer at the school. We then drew two new schools for our 

sample. Staff sent invitations to the principals of these two new schools, as well as to the new 

principal of the third school.

By October 28, two days before the pilot test deadline, only two of the nine principals in the 

pilot study had responded. We then provided two reminders to complete the survey. First, we

sent an email reminder to the seven principals to complete the survey and extended the 

submission deadline to November 1. Second, we also called schools to urge principals to 

complete the survey and provide feedback, but it had little success directly reaching principals. 

In most cases, REL Northwest staff spoke with school secretaries and left voicemail messages 

for principals.

By October 31, one day before the extended deadline, four principals had completed the survey 

(a return rate of 44 percent) and one had partially completed it. None of these principals had 

provided feedback as requested. REL Northwest staff decided to follow up with the principals 

who had already completed the survey to ask them for their feedback and to provide $50 

Amazon gift cards as an incentive. (Note that this amount is slightly higher than the $20 we will 

offer as an incentive during the actual survey, because the principals in the pilot not only 

completed the survey but also provided feedback, which took more time than the actual survey 

will take). In addition to emailing responding principals, we also emailed the nonresponding 

principals offering the incentive and asking them to complete the survey and provide feedback 

via email by November 8. The principal who had partially completed the survey was asked to 

complete it and to provide feedback.

Several principals responded positively to the incentive. Of the five principals who had 

completed or started the survey when we decided to offer the incentive, three provided 

feedback by email or by phone. In addition, the principal who had only partially completed the 

survey, finished it. Two others completed the survey a few days later, one of whom provided 

helpful feedback.
2

REL Northwest staff closed the survey on November 8. Seven of nine principals responded to 

the survey pilot test, for a response rate of 77 percent. Four of the seven principals provided the 

following comments about their experience taking the survey:

1. The SIG-related jargon was not a problem.

2. The response options were complete and appropriate.

3. One principal was confused about how to answer the questions about receiving 

technical assistance from a university, as they received assistance from an organization 

affiliated with a university.

4. One principal felt that the survey should have provided opportunities to share other 

school improvement activities. For example, work addressing literacy, school climate, 

and mainstreaming of special needs students are important school improvement efforts, 

but there was not an opportunity in the survey to highlight such work.

5. Two principals had difficulty finding the time to complete the survey. The length of the 

survey was not burdensome but their other responsibilities came first.

Principal comments one and two required no revisions to the survey. Principal comment three 

required no revision, since the survey already allowed the principals to respond about the 

technical assistance form the organization affiliated with a university under the “other” 

category. To address principal comment four, we moved an open-ended item from the end of 

the survey to near the beginning, which gives principals a more direct opportunity to enter 

information about other school improvement activities. Principal comment five (drawn from 

two principals) required no response; however, we have added an incentive to complete the 

survey, which we believe will increase response rates, as it did in the pilot survey.