# NATIONAL ASSESSMENT OF

EDUCATIONAL PROGRESS

Wave 2 Submittal for 2014

VOLUME II

Part 3

Assessment Feedback Questions

Part 3 contains:

School Coordinator Debriefing Interview Questions Post Assessment Follow-Up Survey Questions Teacher/School Survey Feedback Question

The amount of time estimated to complete these forms: School Coordinator Debriefing In terview Questions: 7 minutes Post Assessment Follow Up Survey Questions: 10 minutes Teacher/School Survey Feedback Question (No additional burden-part of Teacher/School Questionnaires)

# School Coordinator Debriefing Interview Questions

## School Coordinator Debriefing Interview

Directions: Complete with the school coordinator before leaving the school on assessment day.



## 1. Overall, how do you think the assessment(s) went in your school? Would you say:

Read aloud and check the appropriate box.



##  Very well



##  Satisfactorily



##  Unsatisfactorily



## 2. How well did the instruction brochure you received with the Preassessment Packet prepare and direct you in performing the school coordinator tasks? Would you say:

Read aloud and check the appropriate box.



##  Very well



##  Satisfactorily



##  Unsatisfactorily



## 3. NAEP is interested in the feedback you have received from other school staff members about this assessment. Would you say it has been:

Read aloud and check the appropriate box.



##  Positive



##  Neutral



##  Negative



##  No feedback



##  Record comments:

_____________________________________________________________________

_____________________________________________________________________

## 4. Did anyone meet with the students to explain the purpose of the assessment?



##  Yes



##  No



##  Don’t know



## 5. Did the students receive any special instructions to prepare them for the

assessment?



##  Yes



##  No



##  Don’t know



##  If yes, specify type of instructions received:

_____________________________________________________________________

_____________________________________________________________________



## 6. Will the students receive (or have they already received) something from the school for participating in the assessment?



##  Yes



##  No



##  Don’t know



##  If yes, specify what (e.g., food, candy, money, party):

_____________________________________________________________________

_____________________________________________________________________



## 7. Do you have any feedback about the material you received from NAEP (publications, letters, etc.)?



##  Yes



##  No



##  Don’t know



##  Record comments:

_____________________________________________________________________

_____________________________________________________________________

## 8. High School Students Strategies Form

In order to obtain more information on high school student participation and motivation, we would like to know which strategies were used in the school

. Please answer the following questions:

Prepare and Schedule NAEP

1. Which of the following methods did the school use to prepare for NAEP ?



###  Add NAEP to school calendar


###  Use the NAEP MySchool website to track assessment activities for NAEP


###  Post information about NAEP on school website


###  Use the NAEP Appointment Card


###  Send notification letter to students selected for NAEP before assessment day


###  NAEP announced prior to assessment day to parents, students, and/or faculty


###  NAEP announced on the day o f the assessment


###  Give teachers a list of students with testing time and location


###  Provide NAEP staff with a master list of where students are at the beginning of

the assessment


###  Flexible scheduling for students


###  Other (please specify) ______________________

_ _______________________ 

###  Did not do anything extra to prepare for NAEP

Work With Teachers

2. Which of the following methods did the school use to inform teachers about the importance of NAEP?



###  Announce NAEP at a school faculty meeting


###  Hold a meeting with teachers for the sole purpose of discussing NAEP

assessment 

###  Resources used at any time:


###  PowerPoint presentation

– Introducing NAEP to Teachers


###  Video

– Introducing NAEP to Teachers


###  Talking points to answer teacher questions about NAEP


###  Other NAEP materials (i.e., Measure Up and Teacher’s Guide )


###  Other (please specify) ______________________

________________________ 

###  Did not do anything extra to inform teachers about the importance of NAEP

## 8.

(cont) High School Students Strategies Form (continued)

Motivate Students

3. Which of the following methods did the school use to motivate the selected

students and be sure they attended?

 Met with students to explain the importance of NAEP

. If a meeting was held
with students, what resources were used at the meeti ng?


###  PowerPoint presentation

– Introducing NAEP to Students


###  Video

– Introducing NAEP to Students


###  Talking points to answer student questions about NAEP


###  References to the NCES website to find additional resources and past

results

 Publish an article about NAEP in school newspaper

 Senior class advisors, teachers, counselors, and/or administration urge the students to participate

 Invite staff to attend assessment sessions

 Importance of NAEP stressed at beginning of assessment session by school

staff

 Thank student s for participation in NAEP (please specify):

 Informal

– (e.g., verbal thank

- you from school coordinator, principal, counselor)

 Formal

– (e.g., letter, certificate, community service hours, recognition at assembly)

 Other (please specify)

______________________ ________________________  Did not do anything extra to motivate the students

Provide Incentives

4. Which of the following incentives did the school give to the students who participated?



###  NAEP Certificate of Community Service


###  Recognition at a school/awards assembly or other announcement


###  Food incentive before or after the assessment (for example pizza, donuts, cookies, etc.)


###  Lottery for items (yearbook, prom tickets, entertainment, restaurant coupon, parking spot, hall pass, etc .)


###  All students given item for participating (lunch line pass, key chain, early release, restaurant coupon, etc.)


###  Other (please specify) ______________________

________________________ 

###  Did not do anything extra to provide incentives for the students

## 9. Discuss any issues documented on the Session Debriefing Form that should be brought to the school coordinator’s attention.



## 10. I have placed copies of all documents used in the assessment in the NAEP Storage Envelope. NAEP would like you to retain the envelope until June 1 or the end of the school year in case there should be questions about the
assessment. A postage

- paid postcard is stapled to the outside as a reminder for you to destroy the contents after the “Destroy By” date. Please drop the

car d in the mail when the package has been destroyed.



## 11. Thank you for everything you have done to make this a successful assessment. I have a Certificate of Appreciation I would like to give the school as a token of our appreciation for your participation.

Give the school coordinator the School Certificate of Appreciation.

PRIVATE SCHOOLS THAT HAVE BEEN PROMISED ATLASES. I brought NAEP atlases your school was offered this past summer.

 Distribute atlases.



## 12. Offer the school coordinator any used mathematics ancillary foam shapes, rulers, protractors, and spinner squares.

# School Post Assessment Follow Up Survey

### 1 INSTRUCTIONS

After the interview, assign a summary rating on the front cover that best reflects the school coordinator’s responses, as

well as any unsolicited comments he/she makes during your conversation that are relevant to the team’s performance.

The rating should factor in the school coordinator’s overall evaluation of the team, as well as any favorable or unfavorable

reports about individual team members. In assigning your rating, be sure to differentiate, when possible, a negative

attitude about the NAEP program versus a negative evaluation of the assessment team’s performance.

The rating scale for the overall evaluation consists of a five-category scale, with values ranging f rom “1,” signifying

excellent, to “5,” meaning the performance was unacceptable. A “Not Enough Information” (“8”) option is provided, but

unless the school coordinator consistently answers your questions with “don’t know,” assign a rating based on the

opinions the school coordinator was able to express. Ratings of “4” and “5” should be extremely rare and, when assigned,

you will need to provide more details about the team’s performance. Promptly discuss with your field manager if a team

receives an overall rating of “4” or “5.”

The rating scale is explained in more detail below.

Excellent (1) The team was totally prepared, organized, and professional. The school coordinator had nothing but a glowing review for each question asked of them. Almost all questions were answered either “very well” or “very good.”

Good (2) The team was prepared and professional, but there were a few small missteps that kept this from being an excellent assessment.

Satisfactory / Adequate (3) The team was adequate and got the job done, but the school coordinator answered most of the questions as “satisfactorily” or “adequate” (as opposed to very well/very good). Additional briefing or training may be in order.

Unsatisfactory (4) The team was unprepared and/or unprofessional at times.

The team’s performance definitely requires improvement. Additional briefing or training may be in order.

Unacceptable

/ Problem (5) The team was totally unprepared and/or unprofessional. This performance was unacceptable, and immediate remedial action is definitely required.

Not Enough

Information to

Rate (8) Due to the session scheduling, the school coordinator was unable to observe any portion of the team’s performance or enough of the performance to rate the team.

Please consult the NAEP Assessment Team Quality Control Form Telephone Version 2013 MAIN/TEL Question- by

Question Specifications for additional guidelines on what you should consider when assigning these ratings.

### 2 THIS PAGE INTENTIONALLY LEFT BLANK

### 3 INTRODUCTION TO THE SCHOOL COORDINAT OR

Hello, this is ______ from the National Assessment of Educational Progress. I work with the NAEP Supervisor/Assessment Coordinator for this area. A few days ago, our assessment team conducted (a/#

_______ ) session(s) at your school. NAEP would like to thank you for all of your work in making the assessment at _________________________ (SCHOOL NAME) possible. As part of our efforts to improve the performance of our assessment teams, we routinely call a random sample of school coordinators to find out how the assessment and preassessment visit went. This call should only take a

few minutes.

Just as a reminder, the assessment team at your school was led by _________________________ ( SV / AC NAME). When responding to these questions, please feel free to speak frankly

. Your comments will help improve the quality of the assessments.

QUESTIONS ABOUT THE PREASSESSMENT VISIT

First, I have a few questions about the preassessment visit that

__________ ____ ( SV /AC NAME) conducted with you on _______________ (PREASSESSMENT DATE).

1. Overall, how would you rate the preassessment visit at your school? Would you say the visit went very well, satisfactorily, or unsatisfactorily?

VERY WELL

................................... 1 (Q2)

SATISFACTORILY

......................... 2 (Q2)

UNSATISFACTORILY

.................... 3 (Q1a)

1a. IF UNSATISFACTORILY: Why do you feel the preassessment visit went unsatisfactorily? (PROBE FOR SPECIFICS)

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

2. Overall, how would you rate the process of updating the demographic information on the Student Information Reports? Would you say the process worked very well, satisfactorily, or unsatisfactorily?

VERY WELL

................................... 1 (Q3)

SATISFACTORILY

......................... 2 (Q3)

UNSATISFACTORILY

.................... 3 (Q2a)

2a. IF UNSATISFACTORILY: Why do you feel the process of updating the demographic information worked unsatisfactorily? (PROBE FOR SPECIFICS)

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

### 4 QUESTIONS ABOUT THE PREASSESSMENT VISIT (CONTINUED)

3. Overall, how would you rate the process of determining accommodations for students with disabilities and English language learners? Would you say the process of determining accommodations worked very well, satisfactorily, or unsatisfactorily?

VERY WELL

................................... 1 (Q4)

SATISFACTORILY

......................... 2 (Q4)

UNSATISFACTORILY

.................... 3 (Q3a)

3a. IF UNSATISFACTORILY: Why do you feel the process worked unsatisfactorily? (PROBE FOR SPECIFICS)

__________________________________________________________________________

__________________________________________________________________________

__________________________________________________________________________

__________________________________________________________________________________________

4. Do you have any other comments or suggestions about the preassessment visit?

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

QUESTIONS ABOUT THE ASSESSMENT

Now I have a few questions about the assessment that was conducted at your school

On

___________________________ (ASSESSMENT DATE).

5. Overall, how would you say the NAEP team handled the assessment at your school? Would you say very well, satisfactorily, or unsatisfactorily?

VERY WELL

................................... 1 (Q6)

SATISFACTORILY

......................... 2 (Q6)

UNSATISFACTORILY

.................... 3 (Q5a)

5a. IF UNSATISFACTORILY: Why do you feel the assessment team handled things unsatisfactorily? (PROBE FOR SPECIFICS)

__________________________________________________________________________

__________________________________________________________________________

__________________________________________________________________________

__________________________________________________________________________

### 5 QUESTIONS ABOUT THE ASSESSMENT (CONTINUE D)

6. Did the team arrive on time?

YES

................................................ 1 (Q7)

NO

.................................................. 2 (Q6a)

DON’T KNOW

.................................. 8 (Q7)

6a. IF NO, RECORD DETAILS ABOUT THE LATE ARRIVAL. (PROBE: Could you tell me more about that or why they were late?)

__________________________________________________________________________

__________________________________________________________________________

7. As far as you know, did (all of) the session(s) start on time?

YES

................................................ 1 (Q8 )

NO (SOME OR ALL WERE DELAYED) 2 (Q7 a)

DON’T KNOW

.................................. 8 (Q8 )

7a. IF NO, RECORD ANY DETAILS OFFERED ABOUT THE DELAY. (PROBE: Could you tell me more about why the session(s) was/were delayed?)

__________________________________________________________________________

__________________________________________________________________________

__________________________________________________________________________

7b. DELAY WAS CAUSED BY (CODE ONE):

TEAM

.............................................. 1

OTHER REASON

............................ 2

COULD NOT DETERMINE

.............. 8

8. Did you have the opportunity to observe (any of) the assessment session(s)?

YES

................................................ 1 (Q8 a)

NO

.................................................. 2 (Q9 )

8 a. IF YES: How would you rate the way the assessment administrator(s) handled the session(s) you observed? Would you say very well, adequately, or poorly?

VERY WELL

.................................... 1 (Q9 )
ADEQUATELY

................................ . 2 (Q8 b)

POORLY

......................................... 3 (Q8 b)

DON’T KNOW

.................................. 8 (Q9 )

8 b. IF ADEQUATELY OR POORLY, RECORD ANY DETAILS MENTIONED. (PROBE: Could you tell me more about that or how the assessment administrators handled the sessions?)

__________________________________________________________________________

__________________________________________________________________________

__________________________________________________________________________

### 6 QUESTIONS ABOUT THE ASSESSMENT (CONTINUE D)

9. How would you rate the overall organization of the assessment team? Would you say it was very good, adequate, or poor?

VERY GOOD

.................................. 1 (Q10)

ADEQUATE

.................................... 2 (Q9 a)

POOR

............................................. 3 (Q9 a)

DON’T KNOW

.................................. 8 (Q10)

9 a. IF ADEQUATE OR POOR, RECORD ANY DETAILS MENTIONED. (PROBE: C ould you tell me more about the team’s organization ?)

__________________________________________________________________________

__________________________________________________________________________

__________________________________________________________________________

10 . How would you rate the overall professionalism of the assessment team? Would you say it was very good, adequate, or poor?

VERY GOOD

.................................. 1 (Q11)
ADEQUATE

.................................... 2 (Q10 a)

POOR

............................................. 3 (Q10 a)

DON’T KNOW

.................................. 8 (Q11)

10a. IF ADEQUATE OR POOR, RECORD ANY DETAILS MENTIONED. (PROBE: C ould you tell me more about the team’s professionalism ?)

__________________________________________________________________________

__________________________________________________________________________

__________________________________________________________________________

11. How would you rate the way the assessment team interacted with the other school staff? Would you say the interaction was very good, adequate, or poor ?

VERY GOOD

.................................. 1 (Q12)
ADEQUATE

.................................... 2 (Q11a)

POOR

............................................. 3 (Q11a)

DON’T KNOW

.................................. 8 (Q12)

11a. IF ADEQUATE OR POOR, RECORD ANY DETAILS MENTIONED. (PROBE: C ould you tell me more about the team’s interaction with other school staff ?)

__________________________________________________________________________

__________________________________________________________________________

__________________________________________________________________________________________

### 7 QUES TIONS ABOUT THE ASSE SSMENT (CONTINUED)

12. How would you rate the way the assessment team interacted with the students? Would you say the interaction was very good, adequate, or poor?

VERY GOOD

.................................. 1 (Q13)

ADEQUATE

.................................... 2 (Q12a)

POOR

............................................. 3 (Q12a)

DON’T KNOW

.................................. 8 (Q13)

12a. IF ADEQUATE OR POOR, RECORD ANY DETAILS MENTIONED. (PROBE: C ould you tell me more about the team’s interaction with students ?)

__________________________________________________________________________

__________________________________________________________________________

__________________________________________________________________________

13. At the end of the session(s), did

________________ ( SV /AC name) give you the red NAEP Storage Envelope and tell you when the materials should be destroyed? (REVIEW IF NECESSARY).

YES

................................................ 1

NO

.................................................. 2

DON’T KNOW ............................ 8

14. Is there anything (else) you would like to say about the assessment team or any of its members?

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

SUGGESTIONS

15. NAEP is conducted regularly and we continue to refine the materials and procedures for future assessments. Do you have any suggestions on how we can make it easier for schools to participate in

the future?

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

These are all of the questions I have. On behalf of the national assessment, I want to thank you again for all of your help.

### 8 FEEDBACK GIVEN TO SV / AC /AA AND THEIR RESPON SE *

School ID __________________ School Name ____________________________________ Assessment Date _______________

SV / AC /AA Name: Feedback/Response:

________________________________________________________________________________________________________

________________________________________________________________________________________________________

________________________________________________________________________________________________________

________________________________________________________________________________________________________

________________________________________________________________________________________________________

________________________________________________________________________________________________________

________________________________________________________________________________________________________

________________________________________________________________________________________________________

________________________________________________________________________________________________________

________________________________________________________________________________________________________

________________________________________________________________________________________________________

________________________________________________________________________________________________________

________________________________________________________________________________________________________

* PLACE A COPY OF THIS SECTION IN THE SV’S / AC’S PERSONNEL FOLDER AND EACH AA’S PERSONNEL FOLDER, IF

APPROPRIATE. ALSO, INCLUDE A COPY WHEN THE SV , AC OR AA RECEIVED AN EXCEPTIONALLY POSTIVE REVIEW.

# Teacher/School Online Evaluation Question

(Part of Teacher and School Questionnaires)Evaluation Question Regarding Online Completion

If given the choice, would you have preferred to respond to these survey questions on paper or

online?

A) On Paper
B) Online
C) No preference