Appendix A

Data Collection Matrix and Data Source
Appendix A Data Collection Matrix and Data Source 1

Evaluation Questions

Doc. 

Review

Interviews

ObserSEA District Principal vationsK 

teacher

Test 

admin. PreK

I. How have KEAs been developed and adopted by four states at the forefront of using KEAs?

1. How were KEAs developed or chosen (e.g., timeline, 

participants, content decisions)? 1,2,9 1a-e,g,h,2c 12

a. How were various stakeholders engaged in selecting a KEA? 3,10 3

b. For what key purposes was the KEA selected or developed? 4 2 11

c. What key criteria were used in the selection of the KEA (e.g., 

cost, burden, use with certain populations, connection to a 

PreK or K–3 assessment)?

5 4, 18,19

d. Why did the state choose an existing KEA measure or decide 

to develop its own (or some components of its own)? 6-8 4

2. What are the characteristics of the specific KEAs selected? 11,15 3 13

a. What skills and domains are assessed? 10

b. What evidence or types of information are collected (e.g., 

observation, direct assessments, parent report)? 15 11 2a 3 4 3

c. Is use of the KEA mandatory or optional for districts? What 

factors led to that decision? 1f, 8

3. Was the KEA pilot- and field-tested? If yes, how, and what was 

learned about its validity and reliability? 7

4. How is the KEA funded (both the up-front cost and the annual 

cost)? What is the overall and per-student cost to implement 

the KEA? How will funding be sustained?

9 9a

II. How are KEAs being implemented by 12 school districts within four states at the forefront of using KEAs?

5. What general information and training on the administration of 

the KEA are provided to teachers, administrators, and parents? 12 13,14 3,4,5 5,6,7 5,6 4,5 2 5-15

a. What types of training models are used? 13 14a 5a 7a 5a

b. How well prepared were people to administer the KEA? What 

additional support would have been helpful? 5c-f 7b-e 6a-c 5b,c
Appendix A Data Collection Matrix and Data Source 2

Evaluation Questions

Doc. 

Review

Interviews

ObserSEA District Principal vationsK 

teacher

Test 

admin. PreK

6. How are KEAs administered? 12 2 1 1,2 1 14-16

a. When are KEAs administered? 14 1 2 3 2 15,

b. Who administers the KEAs, and what training and support did 

those individuals receive? 15 12a, 14 2b,5 3a,6,7 4,6 3a, 5 15,18

c. How much time do the KEAs require of district staff, school 

administrators, and teachers to implement? 9 3d 4d

d. Are families involved in providing information? If so, how? 12b 2c 3b 4b 3b 15

e. How is technology used to support KEA administration and 

scoring? 16 12c, 17b 2d 3c 4c 3c 15

f. What strategies are used to help ensure that children with 

disabilities and English learners are accurately assessed? 17 18, 19 10, 11 11,12 9,10 8,9 14

g. What aspects of the KEA implementation are standardized 

across districts and schools, and what aspects involve local 

discretion?

13,15 15 6 8 15

7. How are KEA data shared with the state? (e.g., Are results added 

to a student longitudinal data system or another permanent 

student record?)

18 17 8 10 8 7 15

8. How, if at all, do states and districts monitor and evaluate the 

implementation of KEAs? 19 1e,16 7 9 7 6 16

III. How do these four states and their districts and schools use KEA results to inform policy and practice?

9. Who receives the results of the KEAs (e.g., parents, PreK 

educators, kindergarten teachers, principals and early childhood 

administrators, the SEA)? 

20 22 14,20 4,15 13,16 13 4

a. What information do they receive about the results? 20 23 15a-c 16 13,16 10 4a

b. Who has access to results and in what form and on what 

schedule? 20 23a-c 15a-d 4b

10. What supporting information or training do stakeholders 

receive to help them interpret or use the results? 21 23d,25

15d,e,17, 

18a,19a,

20b

18,20 14 11 5 16-18
Appendix A Data Collection Matrix and Data Source 3

Evaluation Questions

Doc. 

Review

Interviews

Observati

SEA District Principal ons

K 

teacher

Test 

admin. PreK

11. How do various stakeholders use KEA results? 23,24,26 16,18,19, 20 17,19 15 12

a. How aware of KEAs are PreK programs and teachers 

(including PreK, Head Start, community-based PreK 

programs, child care)? Do KEAs inform PreK program 

improvement? 

20a 1,3,6,7

b. How, if at all, do teachers use KEA results to inform their 

classroom practice? 19 19 15

c. How, if at all, do principals use KEA results to inform decisions 

at the school level (e.g., classroom assignment for individual 

children, budget allocations, curriculum, or professional 

development)?

18 17

d. How, if at all, do districts use KEA results to inform curriculum 

or professional development? 26 16

e. How, if at all, do states review and use KEA results by student 

demographic characteristics, school district, or by PreK

program? 

23,24

f. How, if at all, do states use KEA results to inform PreK policy, 

curriculum, or other programmatic decisions? 24

12.Are there any policies regarding how KEA data should not be 

used (e.g., determine promotion or prevent kindergarten entry 

in any way)? If so, how are these policies communicated and 

enforced?

27 21 21 17 14 17

IV. What lessons did states, districts, and schools learn about KEA adoption, implementation, and uses?

13. What lessons can states learn about KEA adoption? For 

example, what processes were helpful, and what challenges 

needed to be overcome?

5,6,30

14.What lessons can states, districts, and schools learn about KEA 

implementation? For example, what implementation 

challenges were reported by state, district, and school staff, 

and what efforts did staff undertake to overcome them?

20,21,30 12,13,24

13,14,

22,23, 

24

11,12,20 15, 16 10

15. What lessons can states, districts, and schools learn about 

using KEA results to inform policy and practice? 28,29,30 22,23,24 24 18,19,20 15, 16,17 8, 9, 10
Appendix B

Copy of Federal Register Notice

(Placeholder)
Appendix C

Consent Form
Appendix C Consent Form 1

Case Studies of the Implementation and Use of Kindergarten Entry Assessments

PARTICIPANT CONSENT FORM

Purpose 

This study, Case Studies of the Implementation and Use of Kindergarten Entry Assessments (KEAs), is 

sponsored by the U.S. Department of Education and is being carried out by SRI International (SRI), an 

independent, nonprofit research institute, and Decision Information Resources, Inc.

The purpose of the KEA implementation case studies is to document the processes, accomplishments, 

challenges, and solutions of 24 schools in four states. Findings will be shared with federal and state 

policymakers and the public.

As part of the study, we are inviting you to participate in an interview that will include questions about 

[tailored to respondent: KEA selection, development, adoption, and characteristics; KEA implementation; 

use of KEA results; and lessons learned]. The interviews are designed to last [tailored to respondent: 45 

minutes with school staff, 60 minutes with district-level respondents, and 90 minutes with state-level 

respondents]. The interviews are intended to provide information about the overall KEA process and not 

to evaluate individuals’ capabilities or performance. 

Risks and Discomfort 

We do not know of any risks or discomforts associated with this interview. Your participation in the 

interview is voluntary. If you decide to participate, you may discontinue your participation in the 

interview at any time without penalty of any kind.

Benefits

While there is no direct benefit to you, your participation in the study will contribute to the 

identification of effective KEA implementation practices that can then be disseminated to the many 

states engaged in or interested in implementing KEAs and support the technical assistance efforts of the 

U.S. Department of Education.

Confidentiality

We will treat the information you supply in a confidential manner. Only research staff will have access to 

your interview responses. You will not be identified by name in any report, and data will be reported in a 

manner that does not reveal your identify. Data will be aggregated across states, districts, and schools. 

States, however, will be identified by name.

We would like your consent to record the interview. Recordings will be kept in a secure location by SRI 

International and will not be accessed by anyone outside the study team. The audio recordings will be 

destroyed at the conclusion of the study. You can participate in the interview but decline to have it 

recorded. Additionally, if you elect to have the interview recorded, you may stop the recording at any 

time.
Appendix C Consent Form 2

As part of the study, the study team will share its findings with the U.S. Department of Education and 

other federal agencies. The results of the evaluation will be summarized at the state level as well as 

across all the states and districts participating in the study. A publicly available final report integrating 

and consolidating the results from all the research activities is scheduled to be available in February 

2016.

More Information

If this explanation leaves you with any unanswered questions, please ask and obtain answers before 

signing below. If you have questions later, please call the Project Director, Dr. Tracy Huang, at 650-859-

2338. For questions about your rights as a human participating in this research, you may contact SRI’s 

Human Subjects Committee at 650-859-2686 and reference HS case number [XXXX].

Informed Consent

By signing this form you are indicating that you have read and understood the information provided to 

you about your participation in this interview, agree to participate in this study, and received a signed 

and dated copy of this consent form.

Please check one:

____I am willing to participate in this interview and be audio recorded.

____I am willing to participate in this interview and not be audio recorded.

I understand that this is voluntary and is part of a study sponsored by the U.S. Department of Education 

and conducted by SRI International and Decision Information Resources, Inc.

_______________________________________ _______________________

Name (please print) Date

_______________________________________ __________________________________

Signature Job Title
Appendix D

Recruitment Materials
Appendix D Recruitment Materials 1

KEA Case Studies Sample (Interview) Recruitment Letter

State	Recruiting	Letter	
Appendix D Recruitment Materials 2

[Department of Education letterhead]

Dear [State Superintendent]:

The U.S. Department of Education is sponsoring a study to learn about the development, 

implementation, and use of Kindergarten Entry Assessments (KEAs). [Insert name of state] is one of 

four selected to participate. Each state’s case study will consist of document reviews, observations of 

one or two professional development sessions on the administration and use of KEAs, interviews with 

state-level staff, interviews with district-level staff in three districts, and site visits and interviews at two 

schools within each district. The result of this effort will be a report that identifies cross-site lessons, 

revealing how practices work on the ground and implementation challenges and solutions.

States are approaching KEAs in various ways and your state was chosen because the U.S. Department of 

Education believes that your experiences will help other states think through their decisions about 

which KEA to use, how to the KEA, and how to use the KEA results. 

SRI International and Decision Information Resources, Inc. (DIR) are conducting the study. A 

representative from SRI or DIR will contact [Insert name of SEA representative], [Insert role of 

representative], soon to discuss the study and your state’s plan and schedule for KEA training and 

implementation. If you have questions, you may contact the Project Director, Dr. Tracy Huang, at 650-

859-2338 or Erica Lee at the U.S. Department of Education at erica.lee@ed.gov.

We appreciate your cooperation in helping us gather and disseminate information about how states, 

districts, and schools are developing, implementing, and using KEAs in order to strengthen these efforts 

across the country. We look forward to learning more about the interesting work that you are doing.

Sincerely,

[Name], Director [Name], Deputy Assistant Secretary for 

Policy and Program Studies Service Policy and Early Learning

Office of Planning, Evaluation and Policy Development Office of Elementary and Secondary Education

cc: [SEA representative(s)]
Appendix D Recruitment Materials 3

KEA Case Studies Sample (Interview) Recruitment Email

State	Version	for	Key	Informants

Dear [State representative],

The U.S. Department of Education is sponsoring a study to learn about the development, 

implementation, and use of Kindergarten Entry Assessments (KEAs). [Insert name of state] is one of four 

selected to participate. [Insert name of SEA contact] identified you as a key informant in your state to 

talk with about your experiences with and lessons learned about the selection and use of a KEA. We are 

asking for your participation in a phone interview. This email tells you about the purpose of the 

interview and how the responses will be used.

The study is being carried out by SRI International and Decision Information Resources, Inc. (DIR). Our 

study team is inviting state and district administrators as well as local educators to share with us what 

they have learned about the processes, accomplishments, and challenges of implementing KEAs. We 

believe information about your experiences will be valuable for other states interested in implementing 

KEAs and for federal and state policy makers.

The interview will take approximately 90 minutes of your time. We can schedule it at your convenience 

during [proposed time period]. The interviews are intended to provide information about the overall 

KEA system and not to evaluate individuals’ capabilities or performance. Questions cover topics such as 

the purpose of your state’s KEA, the reasons for choosing your state’s KEA, funding of the KEA, 

administration of the KEA, and uses of KEA results.

Our study team will summarize our findings from your state’s case study as well as integrate and 

consolidate the results across all the states and districts in the study. We will present the results to the 

U.S. Department of Education and produce a public final report planned to be available in 2016. Your 

responses to the interview questions will be kept confidential, as names and other identifying 

information will not be included in any reports.

Please let me know when you are available for an interview during [propose time period]. If you have 

questions about the interview, please email me or contact me at the number below. We appreciate your 

cooperation in helping us gather and disseminate information about how states, districts, and schools 

are developing, implementing, and using KEAs in order to strengthen these efforts across the country. 

We look forward to learning more about the interesting work that you are doing.

Respectfully,

[SRI Interviewer name and contact information]
Appendix D Recruitment Materials 4

Follow-up email reminder about scheduled interview

Dear [State representative],

This is a friendly reminder about our 90-minute phone interview about the selection, implementation, 

and use of your kindergarten entry assessment (KEA). We are scheduled to talk on [Day], [Date], at

[Time]. I will call you at this number [Phone Number] unless there is a more convenient place to reach 

you. 

In preparation for the interview, I am sending a few sample questions and a list of documents that might 

be helpful to have on hand during the call. The sample questions are: 

 For what key purposes was the KEA selected or developed in your state?

 What were the primary reasons you chose the particular KEA(s) for your state?

o How, if at all, does the KEA align with the domains of your state’s early learning standards?

 How is the KEA currently funded? Is there a plan for sustaining future implementation?

 How is the KEA administered across the state?

 How are KEA results shared and used to inform policy and practice (at the state, district, and 

school levels)?

 What are the most important lessons learned about KEA selection, implementation, and use of 

results?

These are the documents that may be helpful to have on hand: 

 Summary of the KEA selection process in your state

 Pilot and/or field test findings

 Map of KEA domains/skills to your state’s early learning standards

 General information provided to parents, teachers, and administrators about the KEA

 Example reports of KEA results.

If these documents are available electronically, please send them to me before the interview.

If you have any questions about the interview, please don’t hesitate to contact me. I look forward to our 

discussion.

Sincerely,

[SRI Interviewer name and contact information]
Appendix D Recruitment Materials 5

KEA Case Studies Sample (Interview) Recruitment Letter

District	Recruiting	Letter	
Appendix D Recruitment Materials 6

[Department of Education letterhead]

Dear [District Superintendent]:

The U.S. Department of Education is sponsoring a study to learn about the development, 

implementation, and use of Kindergarten Entry Assessments (KEAs). [Insert name of state] is one of only 

four in the United States selected to participate in case studies sponsored by the U.S. Department of 

Education to learn about the implementation and use of kindergarten entry assessments (KEAs). 

Further, [Insert name of SEA contact] identified your district as a good one to learn from about how KEA 

implementation works on the ground. 

The study is being carried out by SRI International and Decision Information Resources, Inc. (DIR). In 

each district, the study team will conduct phone interviews with three district staff supporting KEA 

implementation. The team will also visit two schools per district to interview school administrators and 

teachers (six staff members per school), collect and review state and local documents, and possibly 

observe one professional development session on how to administer the KEA or how to use the 

resulting data. The result of this effort will be a report that identifies cross-site lessons, revealing how 

practices work on the ground and implementation challenges and solutions.

A representative from SRI or DIR will contact you soon to discuss the study and your district’s plan for 

KEA implementation. If you have questions, you may contact the Project Director, Dr. Tracy Huang, at 

650-859-2338 or Erica Lee at the U.S. Department of Education at erica.lee@ed.gov.

We appreciate your cooperation in helping us gather and disseminate information about how states, 

districts, and schools are developing, implementing, and using KEAs in order to strengthen these efforts 

across the country. We look forward to learning more about the interesting work that you are doing.

Sincerely,

[[Name], Director [Name], Deputy Assistant Secretary for 

Policy and Program Studies Service Policy and Early Learning

Office of Planning, Evaluation and Policy Development Office of Elementary and Secondary Education

cc: [SEA representative]

 [District representative]
Appendix D Recruitment Materials 7

KEA Case Studies Sample (Interview) Recruitment Email

District	Version	for	Key	Informants

Dear [District representative],

The U.S. Department of Education is sponsoring a study to learn about the development, 

implementation, and use of Kindergarten Entry Assessments (KEAs). [Insert name of state] is one of four 

included in the study, and [Insert name of district] is one of the three districts in [Insert name of state] 

selected to participate. [District contact name], [role of district contact], identified you as a key 

informant in your district to talk with about your experiences and lessons learned about the use of a 

KEA. We are asking for your participation in a phone interview. This email tells you about the purpose of 

the interview and how the responses will be used.

The study is being carried out by SRI International and Decision Information Resources, Inc. (DIR). Our 

study team is inviting district administrators as well as local educators to share with us what they have 

learned about the processes, accomplishments, and challenges of implementing KEAs. We believe 

information about your experiences will be valuable for other districts interested in implementing KEAs 

and for federal and state policy makers.

The interview will take approximately 60 minutes of your time. We can schedule it at your convenience 

during [proposed time period]. The interviews are intended to provide information about the overall 

KEA system and not to evaluate individuals’ capabilities or performance. Questions cover topics such as 

the administration of the KEA, the uses of KEA results, and lessons learned. 

Our study team will summarize findings from interviews conducted across your district and then 

integrate and consolidate results across your state with the findings from the three other states in the 

study. We will present the results to the U.S. Department of Education and other federal agencies, and 

we will produce a public final report planned to be available in February 2016. Your responses to the 

interview questions will be kept confidential, as names and other identifying information will not be 

included in any reports.

Please let me know when you are available for an interview during [propose time period]. If you have 

questions about the interview, please email me or contact me at the number below. We appreciate your 

cooperation in helping us gather and disseminate information about how states, districts, and schools 

are developing, implementing, and using KEAs in order to strengthen these efforts across the country. 

We look forward to learning more about the interesting work that you are doing.

Respectfully,

[SRI Interviewer name and contact information]
Appendix D Recruitment Materials 8

Follow-up email reminder about scheduled interview

Dear [District representative],

This is a friendly reminder about our 1-hour phone interview about the implementation and use of your 

kindergarten entry assessment (KEA). We are scheduled to talk on [Day], [Date], at [Time]. I will call you 

at this number [Phone Number] unless there is a more convenient place to reach you. 

In preparation for the interview, I am sending a few sample questions and a list of documents that might 

be helpful to have on hand during the call. The sample questions are: 

 How is the KEA administered in your district?

 How are KEA results shared and used to inform policy and practice (at the state, district, and 

school levels)?

 What are the most important lessons learned about KEA selection, implementation, and use of 

results?

These are the documents that may be helpful to have on hand:

 General information provided to parents, teachers, and administrators about the KEA

 Example reports of KEA results.

If these documents are available electronically, please send them to me before or after the interview.

If you have any questions about the interview, please don’t hesitate to contact me. I look forward to our 

discussion.

Sincerely,

[SRI Interviewer name and contact information]
Appendix D Recruitment Materials 9

KEA Case Studies Sample (Interview) Recruitment Email

School	Recruiting	Letter	

Dear [School Principal]:

The U.S. Department of Education is sponsoring a study to learn about the development, 

implementation, and use of Kindergarten Entry Assessments (KEAs). [Insert name of state] is one of 

only four in the United States selected to participate in case studies sponsored by the U.S. Department 

of Education to learn about the implementation and use of kindergarten entry assessments (KEAs) and 

your district was identified as a good one to learn from about how KEA implementation works on the 

ground. [District contact] identified your school as a good one to learn from about how KEA 

implementation works on the ground. 

The research study is being carried out by SRI International and Decision Information Resources, Inc. 

(DIR). In each district, the study team will visit two schools per district to conduct in-person interviews. 

At each school, the will interview the principal, two kindergarten teachers, two PreK educators (e.g., 

teachers or director working with 4-year-olds), and a test administrator or other staff member involved 

with KEA implementation (six staff members total). The study team will also collect and review state and 

local documents and possibly observe one professional development session on how to administer the 

KEA or how to use the resulting data. The result of this effort will be a report that identifies cross-site 

lessons, revealing how practices work on the ground and implementation challenges and solutions.

We will contact you soon to discuss the study and logistics of the site visit. If you have questions, you 

may email me or Erica Lee at the U.S. Department of Education at erica.lee@ed.gov.

We look forward to talking with you.

Respectfully,

[SRI Interviewer name and contact information]

Cc: District rep
Appendix E

Document Review Protocol
Appendix E Document Review Protocol 1

KEA Case Studies

Document Review Protocol

Purpose

The purpose of this protocol is to review documents in a systematic way to provide background 

information about a state’s KEA for interviews and more technical or detailed information about some 

aspects of the processes used for selection, implementation, and reporting and use of results. The data 

will be used to

 Provide background on a state’s KEA for state interviews and site visits on a state’s KEA 

development/adoption, implementation, and use for state and final report.

 Gather more technical and detailed information about the KEA tool and process.

 Abstract relevant information from documents obtained at interviews and observations.

Resources

Documents may be obtained from the following sources:

a. Applications to Race-to-the-Top Early-Learning Challenge

b. State Department of Education or Department of Early Learning website, which contains the 

following types of resources:

 KEA strategic plans or planning documentation

 Pilot or field test report

 KEA information for various stakeholders

 KEA training materials

 Assessment manuals

 KEA reports with findings for various stakeholders.

c. State, district, or school interview respondents who share documents with the interviewers.

General Instructions and Tips

a. Populate header information first so you do not forget.

b. This is a cumulative document that will first be populated with information from publically 

available documents and later updated with documents provided to the study team by interview 

respondents. 

c. There will be one document review per state. All sources of information should be listed in the 

table on page 2, “Document Source for Information.” Each document source should be 

numbered sequentially and referenced in each section as the review is being conducted. 

d. Make notes of any questions you have when reviewing the documents. When unsure of the 

information, flag it and note the document name and page number, if available, and ask another 

team member to review. This will allow the interview team to identify areas in which 

clarification are needed.

e. You are not limited to the format or the table presented. Add any notes as needed.
Appendix E Document Review Protocol 2

Document Source for Information

Source #

Title Date

Where document came 

from

If website, list URL Author

1.

2.

3.

4.

5.

6.

Add rows as needed

List sources of information used (by source number) next to each item below. 

Section I: Background on KEA Development or Adoption

1. What year did the state begin the initiative to implement the KEA? (Q1)

2. What agency or department was the key driver of the KEA initiative? List any collaborations with 

another agency or department during the early stages of the KEA initiative. (Q1)

3. What other stakeholders (such as district representatives, Kindergarten teachers, PreK teachers, 

parents, foundation representatives, outside contractors/expert consultants) were engaged in the 

KEA selection process and how? (Q1a)

4. For what key purposes was the KEA selected or developed? [Check all that apply.] (Q1b)

Check Key purposes Source #

Screening: To identify children who may need additional supports or 

further assessment

Data-informed instruction: To identify specific skills to teach to 

individual students or groups of students

Family engagement: To inform parents about children’s strengths and 

needs and ways to support learning

Professional development: To inform professional development for 

teachers about a common student need

Formative assessment: To benchmark learning in order to track 

progress throughout the year

EC program improvement: To inform improvement of early learning 

programs/policies

Other (specify):
Appendix E Document Review Protocol 3

5. What criteria were used in the selection of the KEA? [Check all that apply.] (Q1c)

Check Criteria Source #

Cost

Match to early learning standards

Burden

Use with certain populations

Connection to a PreK assessment

Connection to a K-3 assessment

Other (specify):

6. Which of the following KEA measures was selected? [Check all that apply.] (Q1d)

Check KEA measures Source #

Teaching Strategies-Gold

Desire Results Developmental Profile-School Readiness 

Work Sampling

BRIGANCE

DIBELS

Other (specify):

Other (specify):

7. How was the above measure modified, if at all, to better fit the state’s needs? (Q1d)

8. If state developed its own KEA measure, what was the main reason for doing so? (Q1d)

9. What agency or department is currently overseeing the implementation of KEA? (Q1)

10. Who was involved in selection of KEA measure? [Check all that apply.] (Q1a)

Check People/agencies involved Source #

State department leaders

District representatives

Teachers

Parents

Outside contractors (specify):

Experts (specify):

Foundation representatives (specify):

Other (specify):
Appendix E Document Review Protocol 4

11. What are the domains assessed by the KEA measure? [Check all that apply.] (Q2)

Check Domains Source #

Language and literacy

Cognition and general knowledge, including math and early science

Approaches to learning

Physical well-being, including motor development

Social and emotional development

Other (specify):

Other (specify):

Other (specify):

Other (specify):

Section II: KEA Implementation

12. What information on the administration of the KEA is provided to various stakeholders? (Q5)

Stakeholders Topics covered Source #

District 

administrators

Principals

K teachers

PreK teachers

Parents

Other (specify):

13. What training or support do teachers and administrators receive on the administration of the KEA? 

[Check all that apply.] (Q5a)

Is the training/support offered the same for all schools and districts statewide or is it school or 

district specific? (Q6g)

Check

Type and source of 

training/support

How often 

it is offered

How long 

the 

training is

Specific to 

district or 

same

statewide Source #

In-person training by:

1-on-1 coaching by:

TA by:

Webinars by:

Online self-paced courses 

or modules

List URL:

Other
Appendix E Document Review Protocol 5

14. When is the KEA administered (e.g., within what week of school)? (Q6a)

First assessment: ____________________________________________________

Second assessment (if applicable): ______________________________________

Third assessment (if applicable): ________________________________________

Any other administrations: ____________________________________________

15. Who administers or provides information used in the KEA? Also indicate what evidence or types of 

information are gathered. [Check all that apply.] (Q6b) (Q2,2b)

Are the procedures the same for all schools and districts statewide or is it school or district specific? 

(Q6g)

Use multiple rows for each skill/domain, depending on who collects the data and what 

information is collected.

Skill/

Domain

Who 

collects? 

(e.g., 

teachers, 

classroom 

aids, 

families, 

others)

What information is collected? (e.g., 

checklists, 

direct assessments, 

observation ratings)

Specific to 

district or 

same

statewide Source #

16. How is technology used, if at all, to support KEA administration and scoring? [Check all that apply.] 

(Q6e)

Check 

if used

Source #

Collect documentation for portfolios

Provide direct assessment tasks online

Enter data on ratings and student information

Score data collected by teachers or others

Generate reports on results

Other (specify):
Appendix E Document Review Protocol 6

17. What strategies are used to help ensure that children with disabilities and English learners are 

accurately assessed? [Check all that apply.] (Q6f)

Strategies used

Children 

with 

disabilities

English 

Learners Source #

State offers special training and 

support to those administering 

the KEA with CWD and/or ELs

Offer alternate assessments for 

CWD

Offer additional 

accommodations during 

administration of KEA for CWD

Offer support to conduct 

assessment in the child’s native 

language

Provide modified or adapted 

results

Other (specify):

18. How do schools and districts submit KEA data/results to the state? [Check all that apply.] (Q7)

Check if 

used

Specific to 

district or 

same

statewide Source #

Paper forms

Exports

Data entry into special KEA data system

Data entry into SLDS

Other (specify):

19. List how the state or districts monitor and evaluate implementation of the KEA. (Q8)

 Xx

 Xx
Appendix E Document Review Protocol 7

Section III: Use of KEA

20. Who receives or has access to the results of the KEAs? [Check all that apply.] (Q9,9a,9b)

Check if 

used

Who has access 

to the results?

In what format 

(e.g., printed 

reports, web data 

system)? When?

What type of 

information?

Source 

#

Families

PreK educators

K teachers

Principals

District 

administrators

SEA 

administrators

Other (specify)

21. What supporting information or training do teachers, principals, district staff, and others receive 

regarding interpreting or using KEA results? [Check all that apply.] (Q10)

Check if 

used Staff

Type of 

training/support 

(coaching, webinars, 

online modules, 

manuals or 

guidebooks, other)

Dosage of 

the training 

(frequency 

and length)

Source of training 

(training provider 

or URL)

Source 

#

PreK educators

K teachers

Principals

District administrators

SEA administrators

Other (specify)

22. When, if at all, was the KEA pilot-tested? When, if at all, was the KEA field-tested? What were the 

results?

 Pilot test:

 Field test:
Appendix F

Observation Protocol
Appendix F Observation Protocol 1

KEA Case Studies

Observation Protocol

Note: Please stay for the whole event. 

1. Name of KEA training state: __________________________________________________________

2. Name of district and school(s) attending: ________________________________________________

3. Name(s) of study team member(s): _____________________________________________________

4. Date of observation: _______________________

5. Type of training (Q5):

 In-person training (specify city) 

 Webinar (provide URL if archived)

 Online self-paced module (provide URL)

 Other (specify): _______________________________________________

6. Where does this session fit into other state and district training and supports about the KEA? (What 

came before? What is planned to come after?) (Q5)

7. Name(s) of trainer and affiliation (Q5): __________________________________________________

 Describe whether conducted by a statewide trainer or a local trainer who is part of a train-thetrainer or another model.

8. Scheduled training start and end time (Q5): ______________________________________________

Participants

9. Who is at the training? (Q5)

Number

K teachers

Other teachers

School administrators

Trainers

District staff

Other (specify):
Appendix F Observation Protocol 2

Activities

10. Enter the agenda below and describe major activities, questions, and responses under each 

agenda item. (Q5)

 [Agenda item]

o Topic or learning objective

o Format (lecture, small group work, role play, practice sessions)

o Materials used (videos, handouts)

o Participant engagement (level of engagement)

o Questions asked by participants and responses provided

o Amount of time

 [Agenda item]

o Topic or learning objective

o Format (lecture, small group work, role play, practice sessions)

o Materials used (videos, handouts)

o Participant engagement (level of engagement)

o Questions asked by participants and responses provided

o Amount of time

 [Agenda item]

o Topic or learning objective

o Format (lecture, small group work, role play, practice sessions, etc.)

o Materials used (videos, handouts, 

o Participant engagement (level of engagement)

o Questions asked by participants and responses provided

o Amount of time

[Add more agenda items as needed]

Using your notes, complete the checklists below soon after the observation.

Specify the topic(s) that were addressed. Mark all that apply.

11. Purpose of KEA (Q1b,5)

 Screening: To identify children who may need additional supports or further assessment

 Data-informed instruction: To identify specific skills to teach to individual students or groups of 

students

 Family engagement: To inform parents about children’s strengths and needs and ways to 

support learning

 Professional development: To inform professional development for teachers about a common 

student need

 Formative assessment: To benchmark learning in order to track progress throughout the year 

 EC program improvement: To inform improvement of early learning programs/policies

 Other
Appendix F Observation Protocol 3

12. Information and background on KEA implementation (Q1,5)

 KEA selection process—Summary of how district/state KEA instrument was selected or 

developed

 Stakeholder engagement—The KEA selection/modification/development was informed by 

stakeholders

 Evidence-based, including reliability and validity

 Pilot-/field-tested—The KEA was field-tested and key findings presented

 Cultural competence—Meets needs of diverse study body of state/districts (e.g., poverty, 

race/ethnicity)

 Mode of assessment—Description of mode of assessment: checklist, observational, or direct 

assessment

 Alignment of KEA with standards, initiative, policy, or other assessments—early learning 

standards, Common Core Standards, other K-12 standards and assessments

 Other (specify): ________________________________________________________________

13. Overview of essential domains of school readiness skills (Q2,5)

 Language and literacy development

 Cognition and general knowledge (including mathematics and early scientific development) 

 Approaches to learning 

 Social and emotional development

 Physical well-being and motor development (including adaptive skills)

 Other (specify): ________________________________________________________________

14. KEA for special populations: English learners (EL) and children with disabilities and special needs 

(Q5,6,6f)

 Instrument appropriate for ELs—Information provided on appropriateness of KEA for this 

population

 Administration accommodations provided ELs—Specific instructions or accommodations for 

assessing ELs, including having instrument in multiple languages or conducting the assessment 

in the child’s language spoken in the home

 Instrument appropriate for children with disabilities and special needs—Information provided 

on appropriateness of KEA for this population

 Administration accommodations for children with disabilities and special needs—Specific 

instructions or accommodations for assessing children with disabilities and special needs

 Population not appropriate for the KEA assessment—Whether the KEA administration excludes 

assessment of certain groups of students (e.g., instrument not appropriate for children in 

certain disability categories, age year, or characteristics)
Appendix F Observation Protocol 4

15. Logistics of administration (Q5,6,6a,b,d,e,g,7)

 Administration timeline—Indicated when KEA assessment should happen (e.g., first 2 weeks of 

school/enrollment, first 60 days of school/enrollment, 6 months after first administrations, end 

of school year)

 Administration frequency—Described the number of required administrations (e.g., beginning of 

school year, middle of school year, and end of school year, or at teacher’s or district’s discretion)

 Optional administrations—In addition to the required administrations, teachers are encouraged 

to use the KEA more frequently for progress monitoring

 Personnel administering the KEA—Description (or implied) of appropriate personnel to conduct 

the KEA (e.g., any staff who attended the appropriate trainings, teacher aids, parents)

 Consent requirements—Discussed school or district policy on consent and/or process to obtain 

consent, if needed

 Family data—Described how to gather data from families if applicable

 Data recording—Process of providing/submitting data to school/district/state (e.g., via paper 

submission through schools; enter or upload data online)

 Special circumstances—Procedure for assessing children who enrolled later in the school year, 

children who are chronically absent, late to school or sick, etc.

 Optional administration procedures—List any mentioned:

16. Support for reliability of KEA (Q6, 8, 10)

 Requirements to complete the assessment—Required ratings on specified items to obtain a 

valid score; required documents or documentation

 Strategies to improve rating or assessment reliability

 Use of technology tools to collect, enter or score assessments/report on results

 Demonstration of how to administer the measure

 Practice ratings (role-play, vignette, video exercises)

 Reliability check (via role-play, vignette, or in person)

 Planning for classroom implementation—Allowed teachers to discuss strategies to incorporate 

KEA assessment as part of routine classroom practices

 Certification exam

 Procedures that state or district will use to monitor implementation

 Other

17. Data use and interpretation (Q10, 12)

 Understanding scores/results from KEA assessment

 Use of scores or results from KEA for lesson planning

 Use of scores or results from KEA for individual instruction

 Sharing individual child’s data with parents or guardians

 Sharing class-, grade-, or school-level data with parents

 Demonstration of data use for lesson planning, sharing data with parents

 Designing tasks/assignments—teacher(s) focus on development of specific learning 

tasks/assignments based on KEA data in small groups, with coach, or individually

 Policies regarding how KEA data should NOT be used
Appendix F Observation Protocol 5

18. Sources for continued training and support (Q6b, 10)

 Online support

 TA support provided by district or school 

 TA support provided by outside consultants

 In-person support provided by coach

 Online collaboration with other teachers using the KEA

 Follow-up meetings

Summary 

 What type of training is this?

 Where does this session fit into other state and district training and supports about the KEA? 

(What came before? What is planned to come after?) 

 Write a brief narrative of the session, including the purpose and focus (stated or implied) and 

the sequence of topics and activities. Describe how the trainer/facilitator and teachers 

interacted. 

 Assess whether teachers’ questions were answered and suggested KEA processes were clearly 

explained. Where, if at all, did there appear to be confusion or unknowns?

 What stood out: The topics? The strategies? The overall training? Teacher participation?

 What strengths or key features may suggest this district as an illustrative case study?

 Any questions or notes for special follow-up? 

 Other?

Materials Collected

Gather materials (as available)

 Presentation and handout materials (MUST GET)

 Attendance/sign-in or registration list

 Evaluation form

 Evaluation results

 Online materials referenced in training (or before training)

 CEU paperwork

List documents and materials collected:

Title