1

APR for Performance Period 2012

(PP12)

I. MSP Project Information

In this section, you will be asked for basic information about your MSP project and partner 

organization(s). Please include partnership title, contact information for project director, and 

information about your partner organizations. 

Please click next to start.

----------------------------------------------------------------------------------------------------------

I. MSP Project Information

A. Project 

Indicate the following information about your MSP project.

* 1. Partnership title:

[______________]

* 2. MSP project director:

[______________]

* 3. Project director phone number:

[______________]

* 4. Project director email address:

[______________]

5. Sources of Funding for this MSP project for the 12-month reporting period. (DO NOT 

include dollar values of in-kind contributions.) 

* MSP Grant Funded through Title II, Part B ($): [______________]

 Other State Funds ($): [______________]

 LEA Funds ($): [______________]

 Other (Please specify): [______________]

Amount ($): [______________]

{ Save Report }

I. MSP Project Information
2

B. Lead Organization

Please report Lead Organization information on this tab, and also the total number of partner 

organizations. Please report the remaining partners on the next "C. Partner Organizations".

* 1. Number of partner organizations/institutions (including the lead organization):

[______________]

* 2. Name of lead organization/institution:

[______________]

* 3. Type of lead organization/institution: 

[______________]

- K-12 Institution (LEA)

- Institution of Higher Education (IHE)

- Other

Other (Please Specify):

[______________]

*Note: The term K-12 institution includes include local education agencies (LEAs), public 

charters, consortia of schools, etc.

{ Save Report }
3

I. MSP Project Information

C. Partner Organizations

In this section of the report, you will include information about all partner organizations, including 

the lead organization, for your MSP project. For each participating organization, you will enter 

descriptive information. When you are ready to begin entering information about your partner 

organization(s), click "Add Project Partner". A blank form requesting descriptive information about 

your partner organization will open. When you have entered the information about your partner 

organization, click the "Save Report" button at the bottom of the page. You will then see that your 

partner organization is listed on this page, and you will receive a prompt to notify you that the 

report was successfully updated. The prompt will also contain two links. Choose “next” to proceed 

to the following section of the report or choose “more partners” to enter information about an 

additional partner. You can also add information about a new partner by clicking "Add Project 

Partner" in the box below. To edit or delete information about the project partner, click on the 

"Edit" or "Delete" button next to each organization listed on this page.

Be sure to click "Save Report" button after entering information for each project partner. 

When all of your partner organizations appear in the list on this page, and all information has been 

included about each partner, you have completed this section of the report.

List of project partner forms:

There are a total of __ project partner forms.

1. Edit Delete Sample Project Partner #1

2. Edit Delete Sample Project Partner #2

To report an additional project partner form, please use this link: Add Project Partner.

------------------------------------------------------------------------------------------------------

Please enter information about your partner organization, and when done click the "Save Report" 

button at the bottom of the page.

Report was successfully updated. Please click next to fill out "II. Abstract", or click more 

partners to add more partners.

* 1. Name of participating organization/institution: 

[______________]

2. Type of participating organization/institution: 

[______________]

- Local educational agency (LEA)

- Institution of Higher Education (IHE)
4

- Other

Other (Please Specify): 

[______________]

*Note: The term K-12 institution includes include local education agencies (LEAs), public 

charters, consortia of schools, etc.

If Partner is a local education agency:

Does this partner meet your state’s definition of a high-need LEA? (This definition may be 

included in Section X of this APR. Otherwise, it is the definition that was listed in the RFP)

 Yes

 No

3. Partner's Roles on MSP Project (Check all roles in which the partner has engaged):

 Lead organization

 Project management and administration 

 Design professional development

 Identify and recruit teachers for professional development and/or comparison group

 Provide professional development

 Participate in/receive professional development

 Provide mentors/coaches/teacher leaders

 Evaluate the MSP

 Collect and/or provide data

 Analyze data

 Provide technical assistance to teachers and/or project

 Provide teacher support (e.g., substitute teachers, release time, planning time, teacher 

leaders)

 Advise project

 Other (Please specify): [______________]

{ Save Report }
5

II. MSP Project Abstract

In this section of the form, you are asked to provide a brief project abstract and summary of your 

MSP project. Please note that this project abstract will be used to describe your project in 

publications and on the ED-MSP website. Also, please note that this is a summary only (limited to 

1000 words) and you will be able to more fully describe your project in other sections of the APR. 

Please click next to start.

----------------------------------------------------------------------------------------------------------

II. MSP Project Abstract

A. Project Abstract

Please summarize your project’s goals and objectives, participants, and professional 

development. This abstract will be used to describe your project in publications and on the public 

ED-MSP website. It should provide a general overview for those who are not familiar with your 

work. A strong abstract typically includes the following elements:

 Who: Name the partners, participants, and/or students affected by your MSP. 

 Where: Specify the schools/districts, regions, counties, etc., that your project affects. 

 When: Briefly describe your project’s background, such as when the work began, what 

time period it covers, whether it builds on prior efforts, etc.

 What: Describe the professional development model implemented. Did you conduct 

summer institutes, online/distance learning, university courses, follow-up activities, and/or 

other PD interventions? Are there unique aspects of your PD that would be particularly 

interesting to others? 

 Why: What are the primary goals and objectives of your MSP? What needs does this 

partnership serve? 

You will be able to describe more technical details of your project in other sections of the APR. 

Please copy and paste or type text directly into the space below (Max. 1000 words). 

{ Save Report } { Spell Check }
6

III. Responsibilities

In this section, you will identify the areas of responsibility among your MSP partners. Various 

functions are listed (including "Other") that might be performed by partners in your MSP project. 

For each of the functions or activities, indicate the APPROXIMATE percentage of effort 

undertaken by each partner group. The partner groups listed are K-12 Institutions; IHE faculty; 

and Others (specify). 

For each activity, list the percentage of time spent on that activity by the partner group. For 

example, if the IHE provides almost all of the program administration, such as 90%, with a partner 

LEA providing a small amount of program administration, such as 10%, then under "A. Administer 

Overall Program", you will enter 90 next to IHE and 10 next to LEA.

Please click next to start.

-------------------------------------------------------------------------------

III. Responsibilities

A. Administer Overall Program

Indicate the APPROXIMATE percentage of effort undertaken by each partner group for 

Administering overall program. This includes, for example, budgeting and planning, recruiting 

professional development providers and teachers, organizing meeting space, etc. Total 

percentages should add up to 100%. 

1. % Provided by K-12 Institutions:

[______________]%

2. % Provided by IHE faculty (Institutions of Higher Education):

[______________]%

3. % Provided by Others (Please specify): [______________]:

[______________]%

{ Save Report }
7

III. Responsibilities

B. Design Professional Development

Indicate the APPROXIMATE percentage of effort undertaken by each partner group for 

Designing professional development (PD). This includes, for example, developing the 

professional development curriculum, designing learning tools for use in the professional

development, etc. Total percentages should add up to 100%. 

1. % Provided by K-12 Institutions:

[______________]%

2. % Provided by IHE faculty (Institutions of Higher Education):

[______________]%

3. % Provided by Others (Please specify): [______________]:

[______________]%

{ Save Report } 
8

III. Responsibilities

C. Deliver Professional Development

Indicate the APPROXIMATE percentage of effort undertaken by each partner group for 

Delivering the PD. Total percentages should add up to 100%. 

 N/A, during the past performance period, our project primarily planned and designed MSP 

activities, and we did not deliver any professional development (PD). (If you check this box, you 

will not fill out later sections pertaining to the teachers participating in PD, PD models, PD content 

and processes, assessment measures, or teacher or student gains).

1. % Provided by K-12 Institutions:

[______________]%

2. % Provided by IHE faculty (Institutions of Higher Education):

[______________]%

3. % Provided by Others (Please specify): [______________]:

[______________]%

{ Save Report } 
9

III. Responsibilities

D. Evaluate MSP

Indicate the APPROXIMATE percentage of effort undertaken by each partner group for 

Evaluating the MSP. This includes, for example, collecting data on teacher or student outcomes, 

observing and assessing the effectiveness of the professional development or teaching methods.

Total percentages should add up to 100%. 

1. % Provided by K-12 Institutions:

[______________]%

2. % Provided by IHE faculty (Institutions of Higher Education):

[______________]%

3. % Provided by Others (Please specify): [______________]:

[______________]%

{ Save Report }
10

IV. Professional Development 

In this section, you will be asked to provide information on participants in professional 

development activities. Select and complete the sub-sections for every category of teacher, 

administrator and participant student involved in your MSP program. 

Please click next to start.

IV. Professional Development

A. Number of Higher Ed Faculty Involved in MSP Project

Please indicate the number of Higher Ed faculty involved in the MSP project. (Count the faculty 

members with regular, substantive involvement in MSP. Count each person ONCE.)

1. Number of Mathematics faculty:

[______________]

2. Number of Science faculty:

[______________]

3. Number of Engineering faculty:

[______________]

4. Number of Education faculty:

[______________]

5. Number of Technology/Computer Science faculty:

[______________]

6. Number of other faculty involved:

[______________]

(Please specify discipline):

[______________]

{ Save Report }
11

IV. Professional Development

B. Indicate the Primary Goal and Target for the Intervention

MSP projects are designed to increase student achievement by improving teachers' content 

knowledge. MSP projects accomplish this by directly providing professional development to 

individual teachers to increase their content knowledge or by training teacher leaders who 

ultimately provide training to individual teachers. In the section below, you will be asked to 

indicate the primary goal of your project – directly improving teachers’ content knowledge, training 

teacher leaders, or another goal.

You will then be asked to indicate the primary target of your project. Please indicate whether the 

primary focus of your project is to affect individual teachers in one or more schools or all teachers 

within a school or set of schools. 

When you have selected the target of your MSP project, you will then be asked a series of 

questions about the level of participants you have selected. For example, if you select "Individual 

teacher", then a new sub-section "i. Teacher Selection Criteria" will appear and the instructions in 

the gray bordered box below will indicate that you should proceed to complete the sub-section. 

Please be sure to complete any new section of the form asking follow up information about the 

target for your intervention.

Report was successfully updated. Please click next to fill out "B(i). Teacher Selection Criteria".

1. Please select the main goal of the MSP project:

(Indicate whether the project’s main focus is on improving teachers' content knowledge, training 

teacher leaders, or another goal)

 Improving teachers’ content knowledge and teaching methods 

 

 Please note: In Section VIII of this report, projects whose goal is to increase teacher content 

 knowledge must submit information on teacher gains for all teachers that participated in 

 professional development and student achievement data for only the students of teachers 

 participating in the MSP program. 

 Training teacher leaders 

 Please note: In Section VIII of this report, projects whose goal is to train teacher leaders 

 must submit information on teacher gains for all teachers that directly participated in 

 professional development and school-level student achievement data. 
12

 Both – Improving individual teachers’ content knowledge and training teacher leaders 

are equally important aspects of our program

Please note: In Section VIII of this report:

For teachers that received training to improve their content knowledge, please submit 

information on teacher gains for all teachers that participated in professional development 

and student achievement data for only the students of teachers participating in the MSP 

program. 

For teacher leaders that were trained, please submit information on teacher gains for all 

teacher leaders that directly participated in professional development and school-level

student achievement data in schools in which teacher leaders worked.

 Other - Specify (Max. 100 words): 
13

IV. Professional Development 

C. Total Number of Participating Educators 

{ Save Report } 

1. Total number of teachers and/or administrators receiving MSP professional development in 

Math or Science: 

(Do not double-count teachers for this figure).

[______________] 
14

IV. Professional Development 

D. Elementary School Teachers.

Please provide information on the type of teachers who participated in the MSP professional 

development during this 12-month reporting period. For each participant group, indicate the 

number (or approximate number) of individuals who participated in professional development.

 Include full-time and part-time teachers. Count each person only once in their primary area 

of responsibility. ridiculousness

 Use locally applicable definitions of elementary, middle, and high schools. 

1. Total number of elementary school teachers:

[______________] 

A. Regular core content teachers: Elementary school:

[______________]

B. Gifted and talented teachers: Elementary school:

[______________]

C. Special education teachers: Elementary school:

[______________]

D. Teachers of English language learners: Elementary school:

[______________]

E. Non-teaching math teacher coaches (full or part time):Elementary school:

[______________]

F. Non-teaching science teacher coaches (full or part time):Elementary school:

[______________]

G. Paraprofessionals: Elementary school:

[______________]

2. Total number of elementary school teachers who primarily work in a high-need school, as 

defined by your state:

[______________] 

{ Save Report } 
15

IV. Professional Development 

E. Middle School Teachers.

Please provide information on the type of teachers who participated in the MSP professional 

development during this 12-month reporting period. For each participant group, indicate the 

number (or approximate number) of individuals who participated in professional development. If 

you do not know if the teachers fall into the categories listed below, then include them in the first 

block.

 Include full-time and part-time teachers. Count each person only once in their primary area 

of responsibility.

 Use locally applicable definitions of elementary, middle, and high schools. 

1. Total number of middle school teachers:

[______________] 

A. Regular core content teachers: Middle school:

[______________]

B. Gifted and talented teachers: Middle school:

[______________]

C. Special education teachers: Middle school:

[______________]

D. Teachers of English language learners: Middle school:

[______________]

E. Non-teaching math teacher coaches (full or part time): Middle school:

[______________]

F. Non-teaching science teacher coaches (full or part time): Middle school:

[______________]

G. Paraprofessionals: Middle school:

[______________]

2. Total number middle school teachers who primarily work in a high-need school, as defined by 

your state:

[______________] 

{ Save Report }
16

IV. Professional Development 

F. High School Teachers.

Please provide information on the type of teachers who participated in the MSP professional 

development during this 12-month reporting period. For each participant group, indicate the 

number (or approximate number) of individuals who participated in professional development.

 Include full-time and part-time teachers. Count each person only once in their primary area 

of responsibility (area/duty/position that involved more than 50 percent of the person's 

time). 

 Use locally applicable definitions of elementary, middle, and high schools. 

1. Total number of high school teachers:

[______________] 

A. Regular core content teachers: High school:

[______________]

B. AP/IB: High school:

[______________]

C. Special education teachers: High school:

[______________]

D. Teachers of English language learners: High school:

[______________]

E. Non-teaching math teacher coaches: High school:

[______________]

F. Non-teaching science teacher coaches: High school:

[______________]

G. Paraprofessionals: High school:

[______________]

2. Total number high school teachers who primarily work in a high-need school, as defined by 

your state:

[______________] 

{ Save Report }
17

IV. Professional Development 

G. Administrators.

Please provide information on the type of administrators who participated in the MSP professional 

development during this 12-month reporting period. For each participant group, indicate the 

number (or approximate number) of individuals who participated in professional development.

 Include full-time and part-time administrators. Count each person only once in their 

primary area of responsibility.

 Use locally applicable definitions of elementary, middle, and high schools. 

1. Total number of administrators:

[______________]

A. Administrators: Elementary school:

[______________]

B. Administrators: Middle school:

[______________]

C. Administrators: High school:

[______________]

D. Other:

[______________]

(please describe):

[______________]

{ Save Report }
18

IV. Professional Development 

H. Participant Students.

Indicate the number of students who were taught during the 12-month reporting period by the 

teachers who participated in MSP professional development activities. Approximate numbers of 

students are acceptable. (For teacher leader models, count all students who were affected by 

Teacher Leaders).

1. Number of elementary school students taught by participating teachers:

[______________]

2. Number of middle school students in math and/or science classes taught by participating 

teachers:

[______________]

3. Number of high school students in math and/or science classes taught by participating 

teachers:

[______________]

{ Save Report }
19

V. Professional Development Models

In this section, you will be asked for information about the type of professional development 

models used in your MSP program. 

Please click next to start.

----------------------------------------------------------------------------------------------------------

A. Contact Hours

1. 

How many professional development contact hours were offered in the average participant 

course load during the 12-month reporting period? (hours per participant).

For example: 

 If your project offered one 60-hour summer institute, report 60 hours even if participants 

on average attended only 55. 

 If your project offered ten 8-hour professional development courses during the academic 

year, and the average teacher participated in three, report 24 hours for this component 

of the professional development (not 80).

 Include all components of professional development that the average teacher 

participated in.

[______________] (number only please)

{ Save Report }
20

V. Professional Development Models

B. Type of Professional Development Activities.

Indicate the type of professional development activities used in your project during the 12-month 

reporting period. 

 Summer Institutes only:

The term summer workshop or institute means a workshop or institute, conducted during 

the summer, that...” 

1. is conducted for a period of not less than 2 full-time work weeks or the equivalent 

number of hours. 

2. includes, as a component, a program that provides direct interaction between students 

and faculty.”

 Summer Institutes with additional or follow up activities:

In addition to offering a full Summer Institute, as defined above, these projects offer 

additional or follow-up activities that build on material presented at the Summer Institute.

 Activities other than Summer Institutes only or Summer Institutes with follow up activities (This 

includes summer activities that total fewer than 60 hours):

All projects that do not provide a Summer Institute, as defined by the MSP program, or a 

summer institute with follow up. For example, training during the summer for a period less 

than 2 full time weeks and other school year activities should be categorized here.

{ Save Report }

V. Professional Development Models

B(i). Summer Institutes

Indicate the average duration per participant for the summer institute.

* 1. Total duration in HOURS per participant (on average):

[______________]

{ Save Report }
21

V. Professional Development Models

B(ii). Follow-Up

Indicate the average duration in hours per participant for all professional development activities 

provided by the project as follow-up to the Summer Institute. Then indicate the primary focus of 

the follow-up activities provided by your project during the 12-month reporting period. 

 A. On-site professional development during academic year

 B. Study groups, learning communities (e.g., lesson studies, action research) 

 C. On-line course work and distance learning networks:

 D. Course work at university (Please specify): [______________]

 E. Other (Please specify): [______________]

{ Save Report }

1. Duration on average of HOURS per participant for all follow-up professional development 

activities. (Do not include hours for the summer institute): 

[______________]
22

V. Professional Development Models

B(iii). Others

1. Please indicate the type of activities used in your project during the 12-month reporting 

period. 

 A. Professional development during the summer totaling at least 1 week in duration plus

other school year activities

 B. Professional development during the summer totaling less than 1 week in duration plus 

other school year activities

 C. Focus on school year activities only

2. Please indicate the primary focus of the professional development activities provided by 

your project during the 12-month reporting period. 

 A. On-site professional development during academic year

 B. Study groups, learning communities (e.g., lesson studies, action research) 

 C. On-line course work and distance learning networks

 D. Course work at university (Please specify): [______________]

 E. Other (Please specify): [______________]

{ Save Report }
23

V. Professional Development Models

C. Description of Professional Development Model

Please describe the model for professional development you used during the 12-month reporting 

period. (Max. 200 words)

In this description, please include the following types of information: 

 Who presented the PD? (IHE, Master Teacher, Other) 

 Explain how the professional development was structured. 

 Describe the specific activities that participants engaged in. 

 Why were those activities chosen? 

 What mechanisms or techniques did the facilitators (or the online resources) use to teach 

the participants? 

Please note that this is a text only view. If you have charts or tables, you can upload them as 

attachments by following this link: Attach Files.

{ Save Report }
24

VI. Professional Development Content and Practices

In this section, you will provide information about mathematics and/or science content and 

processes taught to teachers in MSP activities. 

Please click next to start.

----------------------------------------------------------------------------------------------------------

A. Mathematics Content and Practices.

1. Did your MSP project provide training in math content or processes in the MSP professional 

development during this 12-month reporting period?

 Yes  No 

2. Please indicate the major content, topics, or practices of mathematics taught to teachers in 

the MSP activities during this 12-month period. Select all that apply and indicate the GRADE 

LEVELS OF TEACHERS to whom each topic was taught.

2.1 Mathematical Practices

[______________]

 Elementary School Teachers

 Middle School Teachers

 High School Teachers

2.2 Number and Operations:

 Elementary School Teachers

 Middle School Teachers

 High School Teachers 

2.3 Ratios and Proportional Relationships:

 Elementary School Teachers

 Middle School Teachers

 High School Teachers 

2.4 Algebra:

 Elementary School Teachers

 Middle School Teachers

 High School Teachers 

2.5 Geometry:

 Elementary School Teachers

 Middle School Teachers

 High School Teachers 

2.6 Measurement and Data:

 Elementary School Teachers

 Middle School Teachers

2.8 Problem Solving:

 Elementary School Teachers

 Middle School Teachers

 High School Teachers 

2.9 Reasoning and Proof:

 Elementary School Teachers

 Middle School Teachers

 High School Teachers 

2.10 Modeling and Functions:

 Elementary School Teachers

 Middle School Teachers

 High School Teachers 

2.11 Calculus:

 Elementary School Teachers

 Middle School Teachers

 High School Teachers 

2.12 Other (Please Specify): 

[______________]

 Elementary School Teachers

 Middle School Teachers

 High School Teachers
25

 High School Teachers 

2.7 Probability and Statistics:

 Elementary School Teachers

 Middle School Teachers

 High School Teachers 

{ Save Report }
26

VI. Professional Development Content and Practices

B. Science Content and Practices

1. Did your MSP project provide training in science content or processes in the MSP 

professional development during this 12-month reporting period?

 Yes  No

2. Please indicate the major content, topics, or practices of science taught to teachers in the 

MSP activities during this 12-month period. Select all that apply and indicate the GRADE 

LEVELS OF TEACHERS to whom each topic was taught.

2.1 Scientific Practices:

 Elementary School Teachers

 Middle School Teachers

 High School Teachers 

2.2 Physical Science/Physics:

 Elementary School Teachers

 Middle School Teachers

 High School Teachers 

2.3 Chemistry:

 Elementary School Teachers

 Middle School Teachers

 High School Teachers 

2.4 Life Science/Biology:

 Elementary School Teachers

 Middle School Teachers

 High School Teachers 

2.5 Earth and Space Science:

 Elementary School Teachers

 Middle School Teachers

 High School Teachers 

2.6 Technology:

 Elementary School Teachers

 Middle School Teachers

 High School Teachers 

2.7 Engineering:

 Elementary School Teachers

 Middle School Teachers

 High School Teachers 

2.8 Other (Please Specify): 

[______________]

 Elementary School Teachers

 Middle School Teachers

 High School Teachers 

{ Save Report }
27

VII. Program Evaluation

In this section, you will be asked to describe your local evaluation design and methods, data 

collection/analysis methods, and assessment measures. You will also be asked to briefly describe 

the MSP impact on teacher content knowledge and student achievement.

Please note that you will be able to upload an electronic copy of an evaluation report (if you have 

one) when submitting your completed APR. See the tab above called "Upload Report" for 

information and instructions.

Please click next to start.

----------------------------------------------------------------------------------------------------------

A. Type of Evaluator

Please select from the list below the best description of your project's evaluator. (Select all that 

apply.)

 1. Hired external evaluator

Name of organization & contact information: 

[______________]

 2. MSP partnership organization staff (regardless of whether the staff conducting evaluation 

are also involved in the implementation of MSP activities, include staff from the partnership IHE)

 3. Statewide evaluation

 4. Other

Please specify:

[______________]

{ Save Report }
28

VII. Program Evaluation

B. Evaluation Design

Indicate the primary evaluation design used for evaluating this partnership. If you use a 

combination of designs, indicate the most rigorous design used in your project. In the following 

sections, you will be asked to describe this as well as any additional evaluation designs used. 

(You may consult your evaluator to answer the following questions.)

 Random assignment design

 Matched comparison group design

 Non-matched comparison group design

 One-group design

 Qualitative/descriptive design

 Other (Please Specify): [________________]

{ Save Report }
29

VII. Program Evaluation

B(i). Random Assignment Design

Provide a succinct overview of the process to randomly assign participants. Please include a 

description of the evaluation design, sample size, data collection methods, and the types of 

analyses to be performed. If any additional evaluation designs were used to measure project 

outcomes, please describe all evaluations in the space below. (Max. 1000 words) 

Please note that this is a text only view. If you have charts or tables, you can optionally upload 

them as attachments by following this link: Attach Files.

{ Save Report } { Spell Check }
30

VII. Program Evaluation

B(ii). Matched Comparison Group Design

Provide a succinct overview of the evaluation design, criteria for matching, sample size, data 

collection methods, and the types of analyses to be performed. If any additional evaluation 

designs were used to measure project outcomes, please describe all evaluations in the space 

below. (Max. 1000 words) 

Please note that this is a text only view. If you have charts or tables, you can optionally upload 

them as attachments by following this link: Attach Files.

{ Save Report } { Spell Check }
31

VII. Program Evaluation

B(iii). Non-Matched Comparison Group Design

Provide a succinct overview of the evaluation design, a description of how comparison groups 

were created, sample size, data collection methods, and the types of analyses to be performed. If 

any additional evaluation designs were used to measure project outcomes, please describe all 

evaluations in the space below. (Max. 1000 words) 

Please note that this is a text only view. If you have charts or tables, you can optionally upload 

them as attachments by following this link: Attach Files.

{ Save Report } { Spell Check }
32

VII. Program Evaluation

B(iv). One-Group, Qualitative/Descriptive, and Other Designs

Provide a succinct overview of the evaluation design, sample size, data collection methods, and 

the types of analyses to be performed. If any additional evaluation designs were used to measure 

project outcomes, please describe all evaluations in the space below. (Max. 1000 words) 

Please note that this is a text only view. If you have charts or tables, you can optionally upload 

them as attachments by following this link: Attach Files.

{ Save Report } { Spell Check }
33

VII. Program Evaluation

C. Phase of Implementation

1. Indicate your MSP project's stage of implementation.

 New: Conducting start-up tasks such as formalizing partnerships and implementing the 

professional development model for the first time. This project is not based on a prior model, 

nor does it build off of an earlier grant’s work. 

 Developing: Revising, enhancing, or further developing the professional development model. 

May be building on a prior model, an earlier grant, or a planning year. 

 Fully Developed: All components of the MSP model are fully operational. 

2. Current year of implementation:

 1st year of this grant cycle

 2nd year of this grant cycle

 3rd year of this grant cycle

3. Is this the final report that you will submit for this grant (i.e., this is the last APR you will 

complete to report on the funds you received for this MSP)?

 Yes

 No

{ Save Report }
34

VII. Program Evaluation

D. Teacher Assessment Measures

In this section, you will indicate which assessment measures you have used to assess teacher 

achievement.

Enter information for each assessment using a separate form. To add information about an 

assessment, click on the "Add Assessment Measure" link below to begin a three step process. In 

step one, select the type of assessment to add and click on the “Save Report” button to continue. 

In step two, a list of possible assessments will open. Select only ONE assessment from the list. 

When you have selected the assessment click the "Save Report" button at the bottom of the 

page. In step three, you will be asked to enter additional information about the test or instrument 

used. 

To edit or delete information about assessments listed below, click on the "Edit" or "Delete" button 

next to each assessment on this page.

Be sure to click "Save Report" button after entering information for each assessment. 

When all of your assessment measures appear in the list on this page, and all information has 

been included about each assessment, you have completed this section of the report.

List of assessment measure forms

There are a total of __ form(s).

1. Edit Delete Sample Assessment Measure #1 

2. Edit Delete Sample Assessment Measure #2

To add information on an additional assessment measure, please use this link: 

Add Assessment Measure.

------------------------------------------------------------------------------------------------------------------------

Step 1 of 3

Select the type of assessment measure you wish to add:

  Assessment of Teacher Content Knowledge – Math

  Assessment of Teacher Content Knowledge – Science

  Other Teacher Evaluation Instrument

{ Save Report }

VII. Program Evaluation

D. Teacher Assessment Measures (continued)
35

Assessment of Teacher Content Knowledge - Mathematics

Step 2 of 3

Select ONE assessment from the list below and click the "Save Report" button at the bottom of 

the screen when done, which will take you to step three of this form.

In step three you will be asked additional questions to describe the test or instrument used. 

Click on the blue question mark buttons to see additional information about each measure. 

  Diagnostic Mathematics Assessments for Middle School Teachers (Bush) 

(Bill Bush, University of Louisville: Multiple choice and open-ended assessments in four 

content areas address four knowledge domains: memorized knowledge, conceptual 

understanding, problem solving/reasoning, and pedagogical content knowledge)

  Pre-calculus Concept Assessment 

(CRESMET, Arizona State University: multiple-choice instrument that assesses students' 

understanding of the concept of function)

  Learning Mathematics for Teaching (LMT) 

(Hill and Ball, University of Michigan: multiple-choice assessment of three content areas 

set in the context of real classroom teaching)

  Knowledge of Algebra for Teaching 

(Ferrini-Mundy, Michigan State University: multiple choice and open-response items that 

assess algebra content or the knowledge needed for teaching algebra)

 PRAXIS I 

(Educational Testing Service: multiple choice and essay assessments measure basic 

skills in reading, writing, and mathematics)

 PRAXIS II 

(Educational Testing Service: multiple choice and constructed response assessments 

measure subject areas that K-12 educators teach and general and subject-specific 

teaching skills and knowledge)

 State Teacher Assessment 

(State-authorized teacher examinations of math content including teacher certification 

examinations designed to address state standards.)

 Other (Please specify): [______________]

{ Save Report }
36

Step 3 of 3

Please provide information for the instrument below.

Please describe the specific test or instrument you used for the following type of measure. You 

may describe one or more tests/instruments under each type of measure.

1. Description of the assessment measure/test: 

 Nationally normed and/or standardized test 

 Locally developed test with evidence of validity and reliability 

 Locally developed test, not tested for validity and reliability 

 Self-report survey/rating by teachers, students, or other MSP participants 

 Other (Please Specify): [______________]

2. Were the results of this measure used in the reporting of GPRA indicators for teachers in 

section VIII (Government Performance & Results Act Reporting) of this APR? 

  Yes

  No

{ Save Report }

Report was successfully updated. Please click next to fill out "E. Classroom Assessment", or 

click more assessment measures to add more assessment measures.
37

VII. Program Evaluation

D. Teacher Assessment Measures (continued)

Assessment of Teacher Content Knowledge - Science

Step 2 of 3

Select ONE assessment from the list below and click the "Save Report" button at the bottom of 

the screen when done, which will take you to step three of this form.

In step three you will be asked additional questions to describe the test or instrument used. 

Click on the blue question mark buttons to see additional information about each measure. 

  Assessing Teacher Learning about Science Teaching (ATLAST) 

(ATLAST, Horizon Research, Inc.: instruments assess teacher opportunities to learn and 

measure changes in teacher science content knowledge, teacher pedagogical content 

knowledge, and classroom practice in three content areas)

 Flow of Matter and Energy in Living Systems 

 Force and Motion 

 Plate Tectonics 

 Force Concept Inventory 

(Halloun, Hake, Mosca, and Hestenes, Arizona State University: instrument assesses 

basic knowledge state of students taking a first course in physics)

  Diagnostic Teacher Assessments in Mathematics and Science (DTAMS) 

(Bill Bush: web-based diagnostic teacher assessments in mathematics and science. A 

table lists the kinds of assessments according to grade level (elementary, middle, high) 

and field (math or science). All assessments are designed to be used as pre/post 

measures of content knowledge.)

 MOSART: Misconception Oriented Standards-Based Assessment 

(MOSART, Harvard-Smithsonian Center for Astrophysics: multiple choice instrument 

linked to K-12 physical science and earth science content in National Research Council's 

"National Science Education Standards" as well as to research literature documenting 

science misconceptions)

 PRAXIS II 

(Educational Testing Service: multiple choice and constructed response assessments 

measure subject areas that K-12 educators teach and general and subject-specific 

teaching skills and knowledge)

 State Teacher Assessment

(State-authorized teacher examinations of math content including teacher certification 

examinations designed to address state standards.)

 Other (Please specify): [______________]

{ Save Report }
38

Step 3 of 3

Please provide information for the instrument below.

Please describe the specific test or instrument you used for the following type of measure. You 

may describe one or more tests/instruments under each type of measure.

1. Description of the assessment measure/test: 

 Nationally normed and/or standardized test 

 Locally developed test with evidence of validity and reliability 

 Locally developed test, not tested for validity and reliability 

 Self-report survey/rating by teachers, students, or other MSP participants 

 Other (Please Specify): [______________]

2. Were the results of this measure used in the reporting of GPRA indicators for teachers in 

section VIII (Government Performance & Results Act Reporting) of this APR? 

  Yes

  No

{ Save Report }

Report was successfully updated. Please click next to fill out "E. Classroom Assessment", or 

click more assessment measures to add more assessment measures.
39

VII. Program Evaluation

D. Teacher Assessment Measures (continued)

Other Teacher Evaluation Instrument

Step 2 of 3 

Select ONE assessment from the list below and click the "Save Report" button at the bottom of 

the screen when done, which will take you to step three of this form.

In step three you will be asked additional questions to describe the test or instrument used. 

Click on the blue question mark buttons to see additional information about each measure. 

 PRAXIS III 

(Educational Testing Service: classroom performance assessments that measure the 

skills of beginning teachers within classroom settings)

 Inside the Classroom Observation and Analytic Protocol 

(Horizon Research, Inc: instrument measures the quality of an observed K-12 science or 

mathematics classroom lesson attending to the lesson's design, implementation, 

mathematics/science content, and culture)

 OMLI Classroom Observation Protocol 

(RMC Research Corporation: protocol documents the quantity and quality of 

mathematical discourse among K-12 students during classroom observations)

 Reformed Teaching Observation Protocol (RTOP) 

(CRESMET, Arizona State University: observational instrument designed to measure 

"reformed" teaching as defined by research in mathematics and science and national 

standards)

 Surveys of Enacted Curriculum 

(Council of Chief State School Officers: instrument for teachers in Mathematics, Science 

and English Language Arts (K-12) to report data on their instructional practices and 

content being taught in classrooms)

 Teacher Efficacy Belief Instrument 

(Self-report instruments designed to provide insight into how confident participants feel 

about their ability to teach Math or Science. 

Examples include: 

- Gibson and Dembo, 1984 Teacher Efficacy Scale (TES)

- Enochs & Riggs, 1988, 1990. Science Teaching Efficacy Beliefs (STEBI)

- Enochs, Smith, and Huinker, 2000. Mathematics Teaching Efficacy Beliefs (MTEBI)))

 Survey of Teacher Attitudes and Beliefs 

(Any self-report measure that gauges teacher attitudes and beliefs regarding classroom 

practices or their knowledge of math or science.)

 Other (Please specify): [______________]

{ Save Report }
40

Step 3 of 3

Please provide information for the instrument below.

Please describe the specific test or instrument you used for the following type of measure. You 

may describe one or more tests/instruments under each type of measure.

1. Description of the assessment measure/test: 

 Nationally normed and/or standardized test 

 Locally developed test with evidence of validity and reliability 

 Locally developed test, not tested for validity and reliability 

 Self-report survey/rating by teachers, students, or other MSP participants 

 Other (Please Specify): [______________]

{ Save Report }

Report was successfully updated. Please click next to fill out "E. Classroom Assessment", or 

click more assessment measures to add more assessment measures.
41

VII. Program Evaluation

E. Analysis of Changes in Teacher Practice

How are you measuring the extent to which teachers are applying lessons from the MSP PD to 

their classroom instruction? (Select all that apply.)

 Classroom Observation 

 Video taping 

 Questionnaire/Self-report 

 Journals 

 Blogs

 Lesson Plan Analysis

 Interviews/Focus Groups

 Other (Please specify): [______________]

{ Save Report }
42

VII. Program Evaluation

F. Teacher Findings

Describe the major findings from your MSP evaluation to date. Please provide specific evidence 

to support each of your findings and indicate the project stage of development.

Please summarize major findings about MSP impact on teacher content knowledge or practices 

and supporting evidence. (Max. 1000 words)

Please also note that this is a text only view. If you have charts or tables, you can optionally 

upload them as attachments by following this link: Attach Files.

{ Save Report } { Spell Check }
43

VII. Program Evaluation

G. Student Findings

In this section, please provide a narrative that describes and summarizes major findings about the 

effect of you MSP on student math and/or science achievement and supporting evidence. Also, 

please describe how you are measuring impact, including when assessments are given and what 

measures are being used. (Max. 1000 words)

Please note that this is a text only view. If you have charts or tables, you can optionally upload 

them as attachments by following this link: Attach Files.

{ Save Report } { Spell Check }
44

VII. Program Evaluation

H. Impact on the Partnership

In this section, please provide a narrative that describes and summarizes the impact of the MSP 

project on the Partnership. Provide evidence or indicators of this impact. (Max. 1000 words)

Please note that this is a text only view. If you have charts, tables, or pre-formatted paragraphs, 

you can optionally upload them as attachments by following this link: Attach Files.

{ Save Report } { Spell Check }
45

VII. Program Evaluation

I. Other Impacts (Optional)

In this section, please provide a narrative that describes and summarizes any additional findings 

about MSP impact. For example, some partnerships have reported impacts beyond teachers, 

students and partnerships (e.g., classroom, school, or behavior outcomes). If your project has 

experienced these impacts – please describe here. (Max. 1000 words)

Please also note that this is a text only view. If you have charts or tables, you can optionally 

upload them as attachments by following this link: Attach Files.

{ Save Report } { Spell Check }
46

VII. Program Evaluation

J. Upload Report.

Please upload any evaluation report prepared on this project that includes information aggregated 

across the entire life of the project. Indicate whether the attached report is the final report of the 

grant cycle. Click on "Browse" and select the document you will upload. (Limit file size of 10 MB)

 This is a Final Evaluation Report and includes data representing the entire life of the grant.

Attachment:

[______________] { Browse }

{ Save Report }
47

VIII. Government Performance & Results Act Reporting

Under the Government Performance and Results Act (GPRA), all federal agencies are required to 

develop indicators in order to report to the U.S. Congress on federal program impacts and 

outcomes. For the MSP Program, the following indicators have been developed: 

Teacher Knowledge 

1. The percentage of MSP teachers who significantly increase their content knowledge, 

 as reflected in project-level pre- and post-assessments. 

Student Achievement 

2. The percentage of students in classrooms of MSP teachers who score at the basic 

 level or below in State assessments of mathematics or science. 

3. The percentage of students in classrooms of MSP teachers who score at the 

 proficient level or above in State assessments of mathematics or science. 

Evaluation Design 

4. The percentage of MSP projects that report using an experimental or quasi-

 experimental design for their evaluations. 

5. The percentage of MSP projects that use an experimental or quasi-experimental 

 design for their evaluations that are conducted successfully and that yield scientifically 

 valid results. 

Efficiency 

6. The percentage of SEAs that submit complete and accurate data on MSP 

 performance measures in a timely manner. 

Under this section of the APR, you are asked to provide information about teachers and students 

participating in your MSP program to inform indicators 1, 2 and 3. 

Please click next to start.

------------------------------------------------------------------------------------------------------------------------

A. Teachers.

The MSP GPRA indicator for Teacher Content Knowledge is "the percentage of MSP teachers 

who significantly increase their content knowledge, as reflected in project-level pre- and postassessments." To inform GPRA, report the following information for the teachers participating in 

the MSP project during the 12-month reporting period. Note that although the GPRA indicators

are about percentages, we request that you provide raw numbers (not percentages) below. 

Please click here for instructions about providing data for the Government Performance & Results 

Act. 

Please include and submit information on teacher gains for all teachers that participated in MSP 

professional development. Projects whose goal indicated in section IV.B. of this report is to train 

teacher leaders should only include teachers that directly participated in professional 

development. 

To calculate the number of teachers that achieved significant gains in content knowledge through 

an MSP project, use the version of the MSPTCK spreadsheet that is compatible with your 

operating system: 
48

MSPTCK Spreadsheet for PC users

MSPTCK Spreadsheet for MAC users

Data entered on these spreadsheets are not stored in a central location. Please ensure that 

you SAVE the MSPTCK files to your own computer or print out a copy. 

Please note: The Macintosh version of the spreadsheet does not have a Go button. Instead, hit 

the Ctrl and Q at the same time to run the spreadsheet. 

For instruction on how to use the spreadsheet, read the MSPTCK User's Guide. 

Please enter "0" if you do not have any teachers or students in a particular category. 

Total number of teachers receiving MSP professional development in math:

 (If a teacher receives PD in more than one discipline within Math, count that teacher only 

once.) 

 [______________]

Total number of teachers receiving MSP professional development science:

 (If a teacher receives PD in more than one discipline within Science, count that teacher only 

once.) 

 [______________]

All teachers that participate in the MSP project should be counted in the questions that follow.

Mathematics

1. Number of participants receiving MSP professional development in all math PD courses (If a 

teacher receives PD in more than one course, count that teacher separately for each course 

they participate in.):

[______________]

2. Number of participants with both pretest and posttest scores in math content knowledge:

[______________]

3. Number of participants who showed significant gains in math content knowledge:

[______________]

Science

4. Number of participants receiving MSP professional development in all science PD courses (if 

a teacher receives PD in more than one course, count that teacher separately for each course 

they participate in):

[______________]
49

5. Number of participants with both pretest and posttest scores in science content knowledge:

[______________]

6. Number of participants who showed significant gains in science content knowledge:

[______________]

{ Save Report }

----------------------------------------------------------------------
50

Instructions for Section VIII A

Which teachers to include

If the primary target of your project from your response to Question IV B is individual teachers, 

include just those individual teachers.

Once you run a spreadsheet, be sure to save it on your computer and clearly specify the name 

of the subject (mathematics or science), test, and group of teachers (treatment or 

control/comparison) in the filename. Some projects may require the use of more than one 

spreadsheet, so it is important to name your files well.

Report results separately for mathematics and science. Use the guidelines below to generate 

results for mathematics teachers (items 2 and 3); then apply the same steps to generate results 

for science teachers (items 5 and 6).

How to determine significant gains

To calculate your responses for items 2, 3, 5, and 6 in section VIIIA, use the electronic 

spreadsheet supplied by the MSP federal program office. 

 The spreadsheet uses a statistical test called a dependent t-test (for 30 or more 

respondents) or the Wilcoxon signed ranks test (for less than 30 but at least 6 

respondents) to calculate, with 85 percent certainty, the number of teachers who showed 

significant gains on content knowledge tests. You will need to enter each teacher’s pretest and post-test scores on a test of content knowledge into the spreadsheet, and the 

spreadsheet will produce a report that can be used to respond to items 2, 3, 5, and 6. 

Please note: due to statistical constraints, you cannot report results from professional 

development courses for which fewer than 6 teachers have pre-test and post-test data. In 

this case, enter “0” for item 2 and leave item 3 blank.

 If you did not administer both a pre-test and post-test of content knowledge to any 

teachers who were the primary target of your project, complete items 1 and 4, enter "0" for 

items 2 and 5, and leave items 3 and 6 blank. 

 Grantees who administer more than one test for a single professional development course 

should select the most relevant or important test and report the results for that one.

 Grantees who provide multiple professional development courses within the same 

subject (math or science) should do the following: 

1) Report results for all professional development courses with at least 6 participants, 

regardless of whether there is an overlap in the samples of teachers. Teachers that 

receive PD in more than one course should be counted separately for each course 

they participate in. 

2) Complete a separate spreadsheet for each test administered. Do not complete a 

spreadsheet for a test in which there were five or fewer participants tested. Do not 

include teachers from control or comparison groups in the spreadsheet. 

3) Sum the results to respond to items 2, 3, 5, and 6. 

For example, if an MSP project used Test A for 30 teachers who participated in 

algebra and Test B for 30 teachers who participated in geometry, they should 

enter pre- and post-test scores for each teacher who took Test A in one 

spreadsheet, and then do the same for Test B in a separate spreadsheet. If 

the report for Test A showed that of the 30 teachers with both pretest and 

posttest scores, 25 had significant gains, and the report for Test B showed that 
51

of the 30 teachers with both pretest and posttest scores, 15 had significant 

gains, they should sum all mathematics teachers with both pretest and posttest 

scores (30 + 30) to report "60" for item 2, and then summed all participating 

mathematics teachers who had significant gains (25 + 15) to report a "40" for 

item 3. 

 If your evaluation design includes a control or comparison group, you may complete a 

separate spreadsheet for the teachers in this group; however, these results will not be 

entered into the APR. Save each spreadsheet on your computer and specify whether the 

teachers listed belong to the treatment or control group in the filename. 

To go back to Section VIII A, please click here

----------------------------------------------------------------------
52

VIII. Government Performance & Results Act Reporting

B. Students.

The MSP GPRA indicators for Student Achievement are “1) The percentage of students in 

classrooms of MSP teachers who score at the basic level or below in State assessments of 

mathematics or science, and 2) The percentage of students in classrooms of MSP teachers who 

score at the proficient level or above in State assessments of mathematics or science." Note that 

although the GPRA indicators are about percentages, we request that you provide raw numbers 

(not percentages) below. 

To inform GPRA, report the following information. Please click here for instructions. 

Please note that the information required to complete this section will differ, depending on 

whether the project’s main goal as indicated in section IV.B. of this report, is to train teacher 

leaders or increase teacher content knowledge: 

Projects whose goal is to train teacher leaders must report school-level student 

achievement data.

Projects whose goal is to increase teacher content knowledge must provide student 

achievement data for the students in classrooms of teachers participating in the MSP 

program. 

Please enter "0" if you do not have any teachers or students in a particular category.

Mathematics

1. Number of students taught math by MSP teachers:

[______________]

2. Number of students from question 1 with state assessment data in math:

:

[______________]

3. Number of students from question 2 who scored at basic or below in math:

[______________]

4. Number of students from question 2 who scored at proficient or above in math:

[______________]
53

Science

5. Number of students taught science by MSP teachers :

[______________]

6. Number of students from question 5 with state assessment data in science:

[______________]

7. Number of students from question 6 who scored at basic or below in science:

[______________]

8. Number of students from question 6 who scored at proficient or above in science:

[______________]

{ Save Report }

----------------------------------------------------------------------
54

Instructions for Section VIII B

Which students to include

Projects whose goal is to train teacher leaders must report school-level student achievement data.

Projects whose goal is to increase teacher content knowledge must provide student achievement 

data for students of teachers directly participating in the MSP professional development, including 

students in classrooms of all teachers from Section VIII A who were taught in the teacher's main 

subject (mathematics or science). Do not include students taught by the MSP teacher in other 

subjects (not mathematics or science). Sum the students across all MSP teachers. For example, 

an MSP project has 10 mathematics teachers. The numbers of students they teach in their 

mathematics classes are: 48, 43, 57, 52, 49, 47, 53, 45, 51, and 46. The project evaluator would 

sum these enrollments and report 491 students in question 1. 

Which assessments to include

Include assessments that were administered to students in the same academic year that the MSP 

teachers received their professional development. Count the summer before school starts as part 

of the academic year to come. For example, an MSP project conducted summer professional 

development institutes in mathematics in August 2012, with follow-ups in September, November, 

and January of the 2012-2013 academic year. The 2012-2013 state mathematics assessment will 

be administered in April 2008. The project evaluator will report the assessment results from April 

2013 for the students of those teachers whose MSP participation began in August 2012.

Assessment reporting levels

For questions 4 and 8, "proficient or above" refers to the assessment levels used to determine 

Adequate Yearly Progress (AYP). For questions 3 and 7, “basic or below” refers to the 

assessment levels below those that meet AYP. 

To go back to Question VIII B, please click here

-----------------------------------------------------------------------------------------------
55

IX. Lessons Learned

In this section you will be asked to describe both the successes and challenges in MSP 

implementation and evaluation. 

Please click next to start.

------------------------------------------------------------------------------------------------------------------------

A. MSP Implementation

What were the major successes and challenges in MSP implementation? Please provide specific 

stories, anecdotes, or exemplars in this text box. Information should be specific enough to share 

with the general audience, but please exclude personal identifiable information. (Max. 1000 

words)

Please note that this is a text only view. If you have charts or tables, you can optionally upload 

them as attachments by following this link: Attach Files.

{ Save Report } { Spell Check }
56

IX. Lessons Learned

B. MSP Evaluation

What were the major successes and challenges in MSP evaluation? (Max. 1000 words)

Please note that this is a text only view. If you have charts or tables, you can optionally upload 

them as attachments by following this link: Attach Files.

{ Save Report } { Spell Check }
57

X. State Review

This section is for State Coordinator use only. In this section, you will provide the definition of a 

high-need LEA, as defined in your RFP. You will also be able to submit this report to the 

Department of Education in this section. 

Please click next to start.

------------------------------------------------------------------------------------------------------------------------

 Please copy your state’s definition of a high-need LEA directly into the space below 

(Max. 1000 words). 

{ Save Report } { Spell Check }
58

XI. Attached Supplementary Documents 

Click "Browse" to select a file. You can attach files up to a total size of 10.0MB. You may attach 

zipped files to include information from more than 5 sources. Click here for instructions for how to 

create zipped files. 

File 1:

[__________] { Browse }

Download Attachment: filename1.doc

To delete: Delete

File 2:

[__________] { Browse }

Download Attachment: filename2.doc

To delete: Delete

File 3:

[__________] { Browse }

Download Attachment: filename3.doc

To delete: Delete

File 4:

[__________] { Browse }

Download Attachment: filename4.doc

To delete: Delete

File 5:

[__________] { Browse }

Download Attachment: filename5.doc

To delete: Delete

{ Attach Files }
59

Definitions

 Activities other than Summer Institutes only or Summer Institutes with follow up activities

o All projects that do not provide a Summer Institute, as defined by the MSP program, or a 

summer institute with follow up. For example, training during the summer for a period less 

than 2 full time weeks and other school year activities should be categorized here.

 Core Content Teacher

o Core content teachers teach one or more of the core subjects in a general education setting. 

The term ‘core academic subjects' means English, reading or language arts, mathematics, 

science, foreign languages, civics and government, economics, arts, history, and geography.

 Departmentalized 

o School organization in which subjects are taught by different teachers within each grade 

level. 

 Distance learning networks 

o Professional development model occurring over large geographical distances using 

electronic communication (e.g., e-mail, web-based discussion groups).

 External evaluator 

o Evaluator completely outside of partnership institutions. 

 GPRA Indicators

o The MSP Government Performance & Results Act (GPRA) indicators for Student 

Achievement are: 1) The percentage of students in classrooms of MSP teachers who score 

at the basic level or below in state assessments of mathematics or science, and 2) The 

percentage of students in classrooms of MSP teachers who score at the proficient level or 

above in state assessments of mathematics or science.

 High-Need Local Education Agency (LEA)

o It is up to each state to define what is considered a “high-need” LEA for the MSP program.

The state’s definition of a high-need LEA should be included in Section X or in the 

Request for Proposal (RFP). 

 High-Need School

o It is up to each state to define what is considered a “high-need” school for the MSP 

program.

 K-12 Institution

o K-12 institutions include local education agencies (LEAs), public charters, consortia of 

schools, etc.

 IHE 

o Institution of higher education, such as a university or college. 

 Impact 

o Effectiveness attributable to the intervention (your professional development model). 

 LEA 

o Local educational agency. 

 Lead organization 

o Lead organization is defined as the Fiscal Agent. Private K-12 schools may not serve as the 

lead organization

 Length of MSP Award

o The amount of time a project is funded before it must compete in a new competition.

 Matched comparison groups 

o Units that are assigned to conditions, but by some method other than random assignment. 

Matched comparison group studies attempt to show equivalence between groups and/or 

adjust for any baseline differences to show causality. 

 Multi-grade 

o Schools that serve a combination of elementary, middle school, and/or high school 

students. 

 Non-matched comparison groups
60

o Units are assigned to conditions, but by some method other than random assignment. 

Unmatched comparison groups do not attempt to show equivalence between groups and 

may or may not adjust for baseline differences.

 On-line course work 

o Professional development model of college courses delivered over the Internet. 

 On-site professional development activities

o These can include courses, workshops, coaching or any combination of these activities that 

take place in or near the schools of participating teachers.

 One-group design

o A one-group design only evaluates the effects on participants of the intervention. No 

comparison group is used.

 Partner Organization/Participating Organization 

o A partner of an MSP project including an institution of higher education (IHE), a local 

educational agency (LEA), a public or private school, a business, or a non-profit or forprofit organization working with the MSP to improve the quality of mathematics and 

science teachers. 

 Qualitative/Descriptive evaluation design

o Qualitative designs describe activities and components, but do not examine relationships 

among components or look for effects.

 Random assignment design

o Units are randomly assigned to the experimental and control conditions. Units can include 

students, teachers, schools, etc.

 Regular, substantive involvement in MSP 

o Involvement in MSP that constitutes important, major contribution to the design, 

development, and/or implementation of key aspects of your MSP program activities. The 

involvement need not be ongoing; the involvement can be for program administrative 

support as well as technical, instructive input. The involvement may also include persons 

who made short-term but significant contributions to your MSP activities. 

 Total Contact Hours

o The number of total contact hours should reflect the total hours of professional 

development offered, for those courses that the average participant participated in.. Thus, 

even if a participant missed a few hours of a course, we would count the full hours. But if 

20 courses were offered, and an average participant only participated in 2 courses, projects 

would list the total number of hours for the 2 courses.

 Summer workshop or institute 

o The term summer workshop or institute means a workshop or institute, conducted during 

the summer, that -

1. is conducted for a period of not less than 2 full-time work weeks or the equivalent 

number of hours. 

2. includes, as a component, a program that provides direct interaction between 

students and faculty.

 Summer Institutes with additional or follow up activities

o In addition to offering a full Summer Institute, as defined above, these projects offer 

additional or follow-up activities that build on material presented at the Summer Institute.