Peer Evaluation Resource Guide

**Prepared for the**

**Data Quality Initiative Meetings**

**Reno, Nevada**

**May 13-16, 2001**

1.  

# 

######## Table of Contents

## 

## 

**Introduction and Overview [1](#introduction-and-overview)**

Core Indicator Framework Overview
[2](#core-indicator-framework-overview)

Purpose of Core Indicator Framework: Defining Sufficient Rigor
[2](#purpose-of-core-indicator-framework-defining-sufficient-rigor)

Components of Core Indicator Framework
[2](#components-of-core-indicator-framework)

Student Populations and Definition of Threshold Level of Vocational
Education 3

1S1: Secondary Academic Attainment 5

1S2: Secondary Vocational and Technical Skill Attainment 10

1P1: Postsecondary Academic Attainment 15

1P2: Postsecondary Vocational and Technical Skill Attainment 20

2S1: Secondary Completion 25

2S2: Proficiency Credential with Secondary Diploma 28

2P1: Postsecondary Degree or Diploma 33

3S1: Secondary Placement 36

3P1: Postsecondary Placement 42

3P2: Postsecondary Retention 48

4S1: Participation in Secondary Non-Traditional Programs 54

4S2: Completion of Secondary Non-Traditional Programs 56

4P1: Participation in Postsecondary Non-Traditional Programs
[58](#__RefHeading___Toc504805106)

4P2: Completion of Postsecondary Non-Traditional Programs
[60](#p2-completion-of-postsecondary-non-traditional-programs)

## Introduction and Overview

This Peer Evaluation Resource Guide provides resource materials and
background information for state teams and peer reviewers in evaluating
state measures, measurement approaches, and the overall quality of state
data. The guide is based on the Perkins Core Indicator Framework and
includes updated information since the publication of the January, 2000
framework.

State teams should use this guide in preparing their state system
overviews and evaluations of their major data quality issues. They also
should use the guide to prepare their self-evaluations and in evaluating
other states participating in the team session on non-traditional
participation and completion.

This guide starts with an overview of the Perkins Core Indicator
Framework and definitions for key terms and components. This overview
draws from materials contained in the January 2000 Core Indicator
Framework. This first section is designed for state participants who are
new to Perkins accountability. This guide has separate sections for each
indicator and subindicator that will be addressed at the Reno meeting.
Each of these sections contains three parts:

-   [Performance Measures]{.underline}. This section presents an updated
    version of the performance measures in the January, 2000, Perkins
    Core Indicator Framework and provides definitions of some key terms.
    The Framework defines Performance Measures as "the definition of the
    performance measure for each subindicator including the definition
    of the numerator and denominator of each performance measure."

-   [Performance Measurement Approaches]{.underline}. This section
    presents the definitions of the allowable measurement approaches for
    each subindicator. The Framework defines these approaches as "the
    major state approaches for performance measurement for each
    performance measure. These approaches include assessment and data
    collection strategies."

-   [Quality Criteria and Scoring Rubrics]{.underline}. This section
    presents the definition of the quality criteria for each
    subindicator. The Framework defines quality criteria as "the quality
    criteria for performance measurement to ensure sufficient rigor and
    comparability of state performance measurement and reporting." The
    scoring rubrics have three levels:

> [Standard]{.underline}. State measurement approaches have met the
> desired levels of quality that all states should reach.
>
> [Satisfactory Progress]{.underline}. State measurement approaches are
> progressing to the desired levels of quality that all states should
> reach.
>
> [Does Not Meet]{.underline}. State measurement approaches are not
> progressing to the desired levels of quality that all states should
> reach.

The standards levels in the scoring rubrics are the
[proposed]{.underline} levels for use [only]{.underline} in state peer
reviews in the Data Quality Initiative. States will be asked to review
and suggest revisions after using them in the first peer reviews in
Reno. [These levels are not federal requirements]{.underline}.

## Core Indicator Framework Overview

### Purpose of Core Indicator Framework: Defining Sufficient Rigor 

Perkins III defines major roles for OVAE and states in establishing
performance accountability systems for vocational and technical
education. States are given the responsibility for developing
performance measures and data collection systems for the four required
core indicators as well as additional indicators. OVAE is responsible
for ensuring that state performance measures are sufficiently rigorous
(sec. 122(e)(1)(B)) and comparable to ensure consistency and fairness in
setting and adjusting performance levels (sec. 113(b)(3)(A)(vi)) and
reporting state comparisons (sec 113(c)(3)(B)).

The major purpose of the framework is to establish and define state
measures, measurement approaches, and quality criteria for each core
indicator and subindicator to ensure sufficient rigor and comparability
among state accountability systems.

### Components of Core Indicator Framework

The core indicator framework addresses four core indicators and fourteen
sub-indicators. These indicators and sub-indicators are shown below.

**Core Indicators and Subindicators**

Core Indicator 1. Student Attainment

1S1: Secondary Academic Attainment

1S2: Secondary Vocational and Technical Skill Attainment

1P1: Postsecondary Academic Attainment

1P2: Postsecondary Vocational and Technical Skill Attainment

Core Indicator 2. Credential Attainment

2S1: Secondary Completion

2S2: Proficiency Credential with Secondary Diploma (Not Required
1999-2000, REQUIRED 2000-2001)

2P1: Postsecondary Degree or Credential

Core Indicator 3. Placement and Retention

3S1: Secondary Placement

3P1: Postsecondary Placement

3P2: Postsecondary Retention (Not Required 1999-2000, REQUIRED
2000-2001)

Core Indicator 4. Participation in and Completion of Non-Traditional
Programs

4S1: Participation in Secondary Non-Traditional Programs

4S2: Completion of Secondary Non-Traditional Programs

4P1: Participation in Postsecondary Non-Traditional Programs

4P2: Completion of Postsecondary Non-Traditional Programs

After working with this framework in the first year, OVAE agreed to
[not]{.underline} require states to measure and report baseline data for
2S2 and 3P2 for 1999-2000, but **will require** states to report these
measures for 2000-2001.

As described below, the core indicator framework defines the performance
measures and state measurement approaches for each of the fourteen
subindicators. It also defines the quality criteria for assessing state
measurement approaches.

-   [Performance Measures]{.underline}. The definition of the
    performance measures for each subindicator including the definition
    of the numerator and denominator of each performance measure.

-   [Performance Measurement Approaches]{.underline}. The major state
    approaches for performance measurement for each performance measure.
    These approaches include assessment and data collection strategies.

-   [Quality Criteria for Performance Measurement]{.underline}. The
    quality criteria for performance measurement to ensure sufficient
    rigor and comparability of state performance measurement and
    reporting.

### Student Populations and Definition of Threshold Level of Vocational Education

Although Perkins III requires that states report student information on
all students participating in vocational education, the OVAE core
indicator framework applies only to those students who reach (i.e.,
enroll in) a state-defined threshold level of vocational education.
***Throughout the framework, these students are defined as vocational
concentrators. The only exceptions are the two subindicators for
participation in non-traditional programs---4S1---Participation in
Secondary Non-Traditional Programs and 4P1---Participation in
Postsecondary Non-Traditional Programs. These two subindicators address
all vocational participants---that is, students who enrolled in at least
one vocational-technical education course***. In addition, the placement
and retention subindicators only address vocational completers who
graduate. Vocational completers are those vocational concentrators who
have [attained]{.underline} the academic and technical
knowledge/skills/proficiencies in their programs/sequences of courses.

OVAE has worked cooperatively with states to develop the definition of
state-defined threshold level of vocational education for both secondary
and postsecondary vocational education. States have the flexibility to
develop their own definitions of threshold level of vocational education
within the context of their own unique vocational education system as
long as these definitions provide the foundation for the development of
high-quality performance measures for all four core indicators.

######## Student Population Definitions

######## 

######## Threshold Level of Vocational Education

A threshold level of vocational education is defined as a
program/sequence of courses or instructional units that provides an
individual with the academic and technical
knowledge/skills/proficiencies to prepare the individual for employment
and/or further/advanced education (Section 3 (29) Definitions).

+-----------------------------------------------------------------------+
| ###                                                                   |
|                                                                       |
| ######## Vocational Participant                                       |
|                                                                       |
| Student who enrolled in at least one vocational-technical education   |
| course.                                                               |
+-----------------------------------------------------------------------+
| ######## Vocational Concentrator                                      |
|                                                                       |
| Student who enrolled in a threshold level of vocational education.    |
+-----------------------------------------------------------------------+
| ######## Vocational Completer                                         |
|                                                                       |
| Student who attained the academic and technical                       |
| knowledge/skills/proficiencies within a program/sequence of courses   |
| or instructional units that provides an individual with the academic  |
| and technical knowledge/skills/proficiencies to prepare the           |
| individual for employment and/or further/advanced education.          |
+-----------------------------------------------------------------------+

#### 1S1: Secondary Academic Attainment

**Measure**

[Numerator]{.underline}: Number of students reaching a state-defined
threshold level of vocational education (concentrators) who have met
state academic standards and have left secondary education in the
reporting year.

[Denominator]{.underline}: Number of students reaching a state-defined
threshold level of vocational education (concentrators) who have left
secondary education in the reporting year.

**Measurement Approaches**

1.  [State Academic Assessment System]{.underline} -- State-defined
    performance benchmarks on state-developed academic assessment
    systems used in state educational accountability systems including
    high school graduation qualification examinations.

2.  [National Academic Assessment System]{.underline} -- State-defined
    performance benchmarks on national standardized assessment system.

3.  [Academic Course Completion]{.underline} -- State-defined
    performance benchmark (e.g., grade) in designated academic courses
    including integrated academic/vocational courses.

4.  [Vocational Course Completion]{.underline} -- Performance benchmark
    on course or assessments in vocational courses with academic
    foundations integrated.

5.  [Academic Grade Point Average]{.underline} -- State-defined grade
    point average for designated academic course including integrated
    academic/vocational courses.

6.  [Overall Grade Point Average]{.underline} -- State-defined grade
    point average for all courses in the school or program.

7.  [High School Graduation]{.underline} - Graduating from high school
    (when graduation is the same as attaining state or program-defined
    academic standards).

**Quality Criteria and Scoring Rubric**

The standards levels in the scoring rubrics are the
[proposed]{.underline} levels for use [only]{.underline} in state peer
reviews in the Data Quality Initiative. States will be asked to review
and suggest revisions after using them in the first peer review process.
[They are not federal requirements.]{.underline}

+-----------------+-------------------------+--------------------------+
| **Quality       | **National and/or State | **Approaches with Local  |
| Criteria**      | Academic Assessment     | Assessment Systems**     |
|                 | Systems**               |                          |
|                 |                         | **Academic Course        |
|                 |                         | Completion**             |
|                 |                         |                          |
|                 |                         | **Vocational Course      |
|                 |                         | Completion**             |
|                 |                         |                          |
|                 |                         | **Overall Grade Point    |
|                 |                         | Average**                |
|                 |                         |                          |
|                 |                         | **High School            |
|                 |                         | Graduation**             |
+-----------------+-------------------------+--------------------------+
| A.  **Alignment | **3-                    | **3-                     |
|     to State    | [                       | [Standard]{.underline}** |
|     Academic    | Standard]{.underline}** |                          |
|     Standards** |                         | -   All assessment       |
|                 | -   All assessment      |     systems used by all  |
| Attainment      |     systems are         |     schools are directly |
| measures and    |     directly aligned to |     aligned to state     |
| assessment      |     state academic      |     academic content     |
| systems are     |     content             |     [and]{.underline}    |
| aligned to      |     [and]{.underline}   |     performance          |
| (validity)      |     performance         |     standards for all    |
| state academic  |     standards for all   |     content areas (e.g., |
| content         |     content areas       |     language arts)       |
| [a              |     (e.g., language     |     addressed by the     |
| nd]{.underline} |     arts) addressed by  |     measures.            |
| performance     |     the measures.       |                          |
| standards       |                         | -   All assessment       |
|                 | -   All assessment      |     systems by all       |
|                 |     systems are         |     schools are          |
|                 |     regularly evaluated |     regularly evaluated  |
|                 |     on their alignment  |     on their alignment   |
|                 |     to these standards. |     to these standards.  |
|                 |                         |                          |
|                 | **2- [Satisfactory      | **2- [Satisfactory       |
|                 | Progress]{.underline}** | Progress]{.underline}**  |
|                 |                         |                          |
|                 | -   Statewide policies  | -   Statewide policies   |
|                 |     and systems have    |     and systems have     |
|                 |     been established to |     been established to  |
|                 |     ensure that all     |     ensure that all      |
|                 |     assessment systems  |     assessment systems   |
|                 |     are directly        |     used by all schools  |
|                 |     aligned to state    |     are directly aligned |
|                 |     academic content    |     to state academic    |
|                 |     standards for all   |     content standards    |
|                 |     content areas       |     for all content      |
|                 |     (e.g., language     |     areas (e.g.,         |
|                 |     arts) addressed by  |     language arts)       |
|                 |     the measures.       |     addressed by the     |
|                 |                         |     measures.            |
|                 | **1- [Does Not Meet     |                          |
|                 | ]{.underline}**         | **1- [Does Not           |
|                 |                         | Meet]{.underline}**      |
|                 | -   Statewide policies  |                          |
|                 |     and systems have    | -   Statewide policies   |
|                 |     [not]{.underline}   |     and systems have     |
|                 |     been established to |     [not]{.underline}    |
|                 |     ensure that all     |     been established to  |
|                 |     assessment systems  |     ensure that all      |
|                 |     are directly        |     assessment systems   |
|                 |     aligned to state    |     used by schools are  |
|                 |     academic content    |     directly aligned to  |
|                 |     standards for all   |     state academic       |
|                 |     content areas       |     content standards    |
|                 |     (e.g., language     |     for all content      |
|                 |     arts) addressed by  |     areas (e.g.,         |
|                 |     the measures.       |     language arts)       |
|                 |                         |     addressed by the     |
|                 | .                       |     measures.            |
+-----------------+-------------------------+--------------------------+
| B. Scope of     | **3-                    | **3-                     |
| Attainment      | [                       | [Standard]{.underline}** |
| Measurement     | Standard]{.underline}** |                          |
|                 |                         | -   All assessment       |
| Attainment      | -   All assessment      |     systems used by all  |
| measures        |     systems provide a   |     schools provide a    |
| provide a       |     representative      |     representative       |
| representative  |     coverage of all     |     coverage of all      |
| coverage        |     components of at    |     components of at     |
| (validity) for  |     least four academic |     least four academic  |
| all of four     |     content             |     content              |
| academic        |     areas---language    |     areas---language     |
| content         |     arts, mathematics,  |     arts, mathematics,   |
| a               |     social studies, and |     social studies, and  |
| reas---language |     science---addressed |     science---addressed  |
| arts,           |     by state academic   |     by state academic    |
| mathematics,    |     standards and       |     standards and        |
| social studies, |     assessment systems. |     assessment systems.  |
| and             |                         |                          |
| science---in    | -   All assessment      | -   All assessment       |
| state academic  |     systems are         |     systems used by all  |
| standards and   |     regularly evaluated |     schools are          |
| assessment      |     on their coverage   |     regularly evaluated  |
| systems, which  |     of these standards. |     on their coverage of |
| have been       |                         |     these standards.     |
| developed for   | **2- [Satisfactory      |                          |
| all students in | Progress]{.underline}** | **2- [Satisfactory       |
| the state.      |                         | Progress]{.underline}**  |
|                 | -   Statewide policies  |                          |
|                 |     and systems have    | -   Statewide policies   |
|                 |     been established to |     and systems have     |
|                 |     ensure that all     |     been established to  |
|                 |     assessment systems  |     ensure that all      |
|                 |     provide a           |     assessment systems   |
|                 |     representative      |     used by all schools  |
|                 |     coverage of all     |     provide a            |
|                 |     major components of |     representative       |
|                 |     the language arts   |     coverage of all      |
|                 |     (e.g., reading,     |     major components of  |
|                 |     writing) and        |     the language arts    |
|                 |     mathematics content |     (e.g., reading,      |
|                 |     areas addressed by  |     writing) and         |
|                 |     state academic      |     mathematics content  |
|                 |     standards and       |     areas addressed by   |
|                 |     assessment systems. |     state academic       |
|                 |                         |     standards and        |
|                 | **1- [Does Not          |     assessment systems.  |
|                 | Meet]{.underline}**.    |                          |
|                 |                         | **1- [Does Not           |
|                 | -   Statewide policies  | Meet]{.underline}**      |
|                 |     and systems have    |                          |
|                 |     [not]{.underline}   | -   Statewide policies   |
|                 |     been established to |     and systems have     |
|                 |     ensure that all     |     [not]{.underline}    |
|                 |     assessment systems  |     been established to  |
|                 |     provide a           |     ensure that all      |
|                 |     representative      |     assessment systems   |
|                 |     coverage of all     |     used by schools      |
|                 |     major components of |     provide a            |
|                 |     the language arts   |     representative       |
|                 |     (e.g., reading,     |     coverage of all      |
|                 |     writing) and        |     major components of  |
|                 |     mathematics content |     the language arts    |
|                 |     areas addressed by  |     (e.g., reading,      |
|                 |     state academic      |     writing) and         |
|                 |     standards and       |     mathematics content  |
|                 |     assessment systems. |     areas addressed by   |
|                 |                         |     state academic       |
|                 |                         |     standards and        |
|                 |                         |     assessment systems.  |
+-----------------+-------------------------+--------------------------+
| C.  **Timing of | **3-                    | **3-                     |
|     Attainment  | [                       | [Standard]{.underline}** |
|                 | Standard]{.underline}** |                          |
|   Measurement** |                         | -   Attainment is        |
|                 | -   Attainment is       |     measured in all      |
| Attainment is   |     measured            |     schools              |
| measured        |     [after]{.underline} |     [after]{.underline}  |
| concurrent with |     concentrated        |     concentrated         |
| or after        |     participation.      |     participation.       |
| concentrated    |                         |                          |
| participation   | **2- [Satisfactory      | **2- [Satisfactory       |
| in vocational   | Progress]{.underline}** | Progress]{.underline}**  |
| education.      |                         |                          |
|                 | -   Attainment is       | -   Attainment is        |
|                 |     measured            |     measured in all      |
|                 |     [concurrent with or |     schools [concurrent  |
|                 |     after]{.underline}  |     with or              |
|                 |     concentrated        |     after]{.underline}   |
|                 |     participation.      |     concentrated         |
|                 |                         |     participation.       |
|                 | **1- [Does Not Meet     |                          |
|                 | ]{.underline}**         | **1- [Does Not Meet      |
|                 |                         | ]{.underline}**          |
|                 | -   Attainment is       |                          |
|                 |     measured in         | -   Attainment is        |
|                 |     some/all systems    |     measured in some/all |
|                 |                         |     systems or schools   |
|                 |    [before]{.underline} |     [before]{.underline} |
|                 |     concentrated        |     concentrated         |
|                 |     participation.      |     participation.       |
+-----------------+-------------------------+--------------------------+
| D.              | **3-                    | **3-                     |
|   **Reliability | [                       | [Standard]{.underline}** |
|     of          | Standard]{.underline}** |                          |
|     Assessment  |                         | -   Attainment is        |
|                 | -   Attainment is       |     measured by all      |
|   Instruments** |     measured by all     |     assessment systems   |
|                 |     assessment systems  |     in all schools using |
| Attainment is   |     using assessment    |     assessment           |
| measured using  |     instruments that    |     instruments that     |
| reliable        |     meet national or    |     meet national or     |
| assessment      |     state reliability   |     state reliability    |
| instruments.    |     standards.          |     standards.           |
|                 |                         |                          |
|                 | -   All assessment      | -   All assessment       |
|                 |     systems and         |     systems/instruments  |
|                 |     instruments are     |     in all schools are   |
|                 |     regularly evaluated |     regularly evaluated  |
|                 |     based on these      |     based on these       |
|                 |     standards.          |     standards.           |
|                 |                         |                          |
|                 | **2- [Satisfactory      | **2- [Satisfactory       |
|                 | Progress]{.underline}** | Progress]{.underline}**  |
|                 |                         |                          |
|                 | -   Statewide policies  | -   Statewide policies   |
|                 |     and systems have    |     and systems have     |
|                 |     been established to |     been established to  |
|                 |     ensure that         |     ensure that          |
|                 |     attainment is       |     attainment is        |
|                 |     measured by all     |     measured by all      |
|                 |     assessment systems  |     assessment systems   |
|                 |     using assessment    |     in all schools using |
|                 |     instruments that    |     assessment           |
|                 |     meet national or    |     instruments that     |
|                 |     state reliability   |     meet national or     |
|                 |     standards.          |     state reliability    |
|                 |                         |     standards.           |
|                 | **1- [Does Not          |                          |
|                 | Meet]{.underline}**     | **1- [Does Not Meet      |
|                 |                         | ]{.underline}**          |
|                 | -   Statewide policies  |                          |
|                 |     have                | -   Statewide policies   |
|                 |     [not]{.underline}   |     and systems have     |
|                 |     been established to |     [not]{.underline}    |
|                 |     ensure that         |     been established to  |
|                 |     attainment is       |     ensure that          |
|                 |     measured by all     |     attainment is        |
|                 |     assessment systems  |     measured by all      |
|                 |     using assessment    |     assessment systems   |
|                 |     instruments that    |     in all schools using |
|                 |     meet national or    |     assessment           |
|                 |     state reliability   |     instruments that     |
|                 |     standards.          |     meet national or     |
|                 |                         |     state reliability    |
|                 |                         |     standards.           |
+-----------------+-------------------------+--------------------------+
| E.              | **3-                    | **3-                     |
|   **Reliability | [                       | [Standard]{.underline}** |
|     of          | Standard]{.underline}** |                          |
|     Assessment  |                         | -   Attainment is        |
|     A           | -   Attainment is       |     measured in all      |
| dministration** |     measured by all     |     schools using        |
|                 |     assessment systems  |     assessment           |
| Attainment is   |     using assessment    |     instruments that are |
| measured using  |     instruments that    |     administered         |
| assessment      |     are administered    |     consistently         |
| instruments     |     consistently        |     according to         |
| that are        |     according to        |     national or state    |
| administered    |     national or state   |     reliability          |
| consistently    |     reliability         |     standards (e.g.,     |
| within          |     standards (e.g.,    |     inter-rater          |
| assessment      |     inter-rater         |     reliability for      |
| systems.        |     reliability for     |     performance          |
|                 |     performance         |     assessment).         |
|                 |     assessment).        |                          |
|                 |                         | -   The administration   |
|                 | -   The statewide       |     of all assessments   |
|                 |     administration of   |     by all schools is    |
|                 |     all assessments is  |     regularly evaluated  |
|                 |     regularly evaluated |     based on these       |
|                 |     based on these      |     standards.           |
|                 |     standards.          |                          |
|                 |                         | **2- [Satisfactory       |
|                 | **2- [Satisfactory      | Progress]{.underline}**  |
|                 | Progress]{.underline}** |                          |
|                 |                         | -   Statewide policies   |
|                 | -   Statewide policies  |     and systems have     |
|                 |     and systems have    |     been established to  |
|                 |     been established to |     ensure that          |
|                 |     ensure that         |     attainment is        |
|                 |     attainment is       |     measured in all      |
|                 |     measured by all     |     schools using        |
|                 |     assessment systems  |     assessment           |
|                 |     using assessment    |     instruments that are |
|                 |     instruments that    |     administered         |
|                 |     are administered    |     consistently         |
|                 |     consistently        |     according to         |
|                 |     according to        |     national or state    |
|                 |     national or state   |     reliability          |
|                 |     reliability         |     standards (e.g.,     |
|                 |     standards (e.g.,    |     inter-rater          |
|                 |     inter-rater         |     reliability for      |
|                 |     reliability for     |     performance          |
|                 |     performance         |     assessment).         |
|                 |     assessment).        |                          |
|                 |                         | **1- [Does Not Meet      |
|                 | **1- [Does Not          | ]{.underline}**          |
|                 | Meet]{.underline}**     |                          |
|                 |                         | -   Statewide policies   |
|                 | -   Statewide policies  |     and systems have     |
|                 |     and systems have    |     [not]{.underline}    |
|                 |     [not]{.underline}   |     been established to  |
|                 |     been established to |     ensure that          |
|                 |     ensure that         |     attainment is        |
|                 |     attainment is       |     measured in all      |
|                 |     measured by all     |     schools using        |
|                 |     assessment systems  |     assessment           |
|                 |     using assessment    |     instruments that are |
|                 |     instruments that    |     administered         |
|                 |     are administered    |     consistently         |
|                 |     consistently        |     according to         |
|                 |     according to        |     national or state    |
|                 |     national or state   |     reliability          |
|                 |     reliability         |     standards.           |
|                 |     standards.          |                          |
+-----------------+-------------------------+--------------------------+
| F. Student      | **3-                    | **3-                     |
| Coverage in     | [                       | [Standard]{.underline}** |
| Attainment      | Standard]{.underline}** |                          |
| Measurement     |                         | -   Attainment outcomes  |
|                 | -   Attainment outcomes |     are reported by all  |
| ###             |     are reported by all |     schools for at least |
|                 |     schools for at      |     90% of their         |
| Performance     |     least 90% of their  |     students reaching    |
| measurement     |     students reaching   |     threshold levels and |
| reports         |     threshold levels    |     who are expected to  |
| attainment data |     who are expected to |     take local           |
| for all         |     take national or    |     assessments or their |
| students        |     state academic      |     equivalents (e.g.,   |
| reaching        |     assessments or      |     alternative          |
| state-defined   |     their equivalents   |     assessments)         |
| threshold       |     (e.g., alternative  |     according to state   |
| levels in the   |     assessments)        |     or local policies.   |
| state.          |     according to state  |                          |
|                 |     or local policies.  | **2- [Satisfactory       |
| ###             |                         | Progress]{.underline}**  |
|                 | **2- [Satisfactory      |                          |
|                 | Progress]{.underline}** | -   Attainment outcomes  |
|                 |                         |     are reported by all  |
|                 | -   Attainment outcomes |     schools for at least |
|                 |     are reported for    |     80% of their         |
|                 |     all schools for at  |     students reaching    |
|                 |     least 80% of their  |     threshold levels and |
|                 |     students reaching   |     who are expected to  |
|                 |     threshold levels    |     take local           |
|                 |     and who are         |     assessments or their |
|                 |     expected to take    |     equivalents (e.g.,   |
|                 |     national or state   |     alternative          |
|                 |     academic            |     assessments)         |
|                 |     assessments or      |     according to state   |
|                 |     their equivalents   |     or local policies.   |
|                 |     (e.g., alternative  |                          |
|                 |     assessments)        | **1- [Does Not Meet      |
|                 |     according to state  | ]{.underline}**          |
|                 |     or local policies.  |                          |
|                 |                         | -   Attainment outcomes  |
|                 | **1- [Does Not Meet     |     are                  |
|                 | ]{.underline}**         |     [not]{.underline}    |
|                 |                         |     reported by all      |
|                 | -   Attainment outcomes |     schools or some      |
|                 |     are                 |     schools report less  |
|                 |     [not]{.underline}   |     than 80% of their    |
|                 |     reported for all    |     students reaching    |
|                 |     schools             |     threshold levels and |
|                 |     [or]{.underline}    |     who are expected to  |
|                 |     some schools report |     take local           |
|                 |     less than 80% of    |     assessments or their |
|                 |     their students who  |     equivalents (e.g.,   |
|                 |     reach threshold     |     alternative          |
|                 |     levels and who are  |     assessments)         |
|                 |     expected to take    |     according to state   |
|                 |     national or state   |     or local policies.   |
|                 |     academic            |                          |
|                 |     assessments or      |                          |
|                 |     their equivalents   |                          |
|                 |     (e.g., alternative  |                          |
|                 |     assessments)        |                          |
|                 |     according to state  |                          |
|                 |     or local policies.  |                          |
+-----------------+-------------------------+--------------------------+

#### 1S2: Vocational and Technical Skill Attainment

#### 

**Measure**

[Numerator:]{.underline} Number of students reaching a state-defined
threshold level of vocational education (concentrators) who have met
state-established, industry-validated career and technical skill
standards and who have left secondary education in the reporting year.

[Denominator]{.underline}: Number of students reaching a state-defined
threshold level of vocational education (concentrators) who have left
secondary education in the reporting year.

**Measurement Approaches**

1.  [National/State Standards and Assessment Systems]{.underline} --
    Performance benchmarks on national or state standards and assessment
    systems using national or state-developed and standardized
    assessment instruments and assessment procedures (includes
    licensing/certification examinations).

2.  [National/State Standards and Local Assessment Systems]{.underline}
    -- Performance benchmarks on local assessment systems that have been
    approved and recognized by national organizations and/or states
    based on national or state standards.

3.  [Local Standards and Assessment Systems]{.underline} -- Performance
    benchmarks on state-approved local assessment systems based on
    state-approved or recognized local standards.

4.  [Vocational/Technical Course Completion]{.underline} -- Performance
    benchmarks (e.g., grades, competency completion) in vocational
    education courses or programs.

5.  [Vocational/Technical Grade Point Average]{.underline} --
    State-defined grade point average for designated vocational courses
    including integrated academic/vocational courses.

6.  [Program Completion]{.underline} - Completion of
    vocational/technical education program when program completion
    represents attainment of career and technical skill standards.

**Quality Criteria and Scoring Rubric**

The standards levels in the scoring rubrics are the
[proposed]{.underline} levels for use [only]{.underline} in state peer
reviews in the Data Quality Initiative. States will be asked to review
and suggest revisions after using them in the first peer review process.
[They are not federal requirements.]{.underline}

+-----------------+-------------------------+--------------------------+
| **Quality       | **National and/or       | **Approaches with Local  |
| Criteria**      | State**                 | Assessment Systems**     |
|                 |                         |                          |
|                 | **Assessment Systems**  | **Local Standards and    |
|                 |                         | Assessment Systems**     |
|                 |                         |                          |
|                 |                         | **Vocational/Technical   |
|                 |                         | Course Completion**      |
|                 |                         |                          |
|                 |                         | **Vocational Grade Point |
|                 |                         | Average**                |
|                 |                         |                          |
|                 |                         | **Program Completion**   |
+-----------------+-------------------------+--------------------------+
| A.  **Alignment | **3-                    | **3-                     |
|     to Industry | [                       | [Standard]{.underline}** |
|     Standards** | Standard]{.underline}** |                          |
|                 |                         | -   All assessment       |
| Attainment      | -   All assessment      |     systems used by all  |
| measures are    |     systems are         |     schools are directly |
| aligned         |     directly aligned to |     aligned to           |
| (validity) to   |     state-established,  |     state-established,   |
| sta             |     industry-validated  |     industry-validated   |
| te-established, |     content             |     content              |
| ind             |     [and]{.underline}   |     [and]{.underline}    |
| ustry-validated |     performance         |     performance          |
| skill           |     standards.          |     standards.           |
| s               |                         |                          |
| tandards---both | -   All assessment      | -   All assessment       |
| content and     |     systems are         |     systems in all       |
| performance     |     regularly evaluated |     schools are          |
| standards       |     on their alignment  |     regularly evaluated  |
|                 |     to these standards  |     on their alignment   |
|                 |     in cooperation with |     to these standards   |
|                 |     industry and        |     in cooperation with  |
|                 |     postsecondary       |     industry and         |
|                 |     partners.           |     postsecondary        |
|                 |                         |     partners.            |
|                 | **2- [Satisfactory      |                          |
|                 | Progress]{.underline}** | **2- [Satisfactory       |
|                 |                         | Progress]{.underline}**  |
|                 | -   Statewide policies  |                          |
|                 |     and systems have    | -   Statewide policies   |
|                 |     been established to |     and systems have     |
|                 |     ensure that all     |     been established to  |
|                 |     assessment systems  |     ensure that all      |
|                 |     are directly        |     assessment systems   |
|                 |     aligned to          |     used by all schools  |
|                 |     state-established,  |     are directly aligned |
|                 |     industry-validated  |     to                   |
|                 |     content standards.  |     state-established,   |
|                 |                         |     industry-validated   |
|                 | -   These statewide     |     content standards.   |
|                 |     policies and        |                          |
|                 |     systems have been   | -   These statewide      |
|                 |     established in      |     policies and systems |
|                 |     cooperation with    |     have been            |
|                 |     industry and        |     established in       |
|                 |     postsecondary       |     cooperation with     |
|                 |     partners.           |     industry and         |
|                 |                         |     postsecondary        |
|                 | **1- [Does Not          |     partners.            |
|                 | Meet]{.underline}**     |                          |
|                 |                         | **1- [Does Not Meet      |
|                 | -   Statewide policies  | ]{.underline}**          |
|                 |     and systems have    |                          |
|                 |     [not]{.underline}   | -   Statewide policies   |
|                 |     been established to |     and systems have     |
|                 |     ensure that all     |     [not]{.underline}    |
|                 |     assessment systems  |     been established to  |
|                 |     are directly        |     ensure that all      |
|                 |     aligned to          |     assessment systems   |
|                 |     state-established,  |     used by all schools  |
|                 |     industry-validated  |     are directly aligned |
|                 |     content standards   |     to                   |
|                 |     in cooperation with |     state-established,   |
|                 |     industry and        |     industry-validated   |
|                 |     postsecondary       |     content standards in |
|                 |     partners.           |     cooperation with     |
|                 |                         |     industry and         |
|                 | .                       |     postsecondary        |
|                 |                         |     partners.            |
+-----------------+-------------------------+--------------------------+
| **B. Scope of   | **3-                    | **3-                     |
| Attainment**    | [                       | [Standard]{.underline}** |
|                 | Standard]{.underline}** |                          |
| **Measurement** |                         | -   All assessment       |
|                 | -   All assessment      |     systems used by all  |
| Attainment      |     systems provide a   |     schools provide a    |
| measures        |     representative      |     representative       |
| provide a       |     coverage of         |     coverage of          |
| representative  |     state-established,  |     state-established,   |
| coverage        |     industry-validated  |     industry-validated   |
| (validity) of   |     content             |     content              |
| sta             |     [and]{.underline}   |     [and]{.underline}    |
| te-established, |     performance         |     performance          |
| ind             |     standards.          |     standards.           |
| ustry-validated |                         |                          |
| skill standards | -   All assessment      | -   All assessment       |
|                 |     systems are         |     systems in all       |
|                 |     regularly evaluated |     schools are          |
|                 |     on their coverage   |     regularly evaluated  |
|                 |     of these standards  |     on their coverage of |
|                 |     in cooperation with |     these standards in   |
|                 |     industry and        |     cooperation with     |
|                 |     postsecondary       |     industry and         |
|                 |     partners.           |     postsecondary        |
|                 |                         |     partners.            |
|                 | **2- [Satisfactory      |                          |
|                 | Progress]{.underline}** | **2- [Satisfactory       |
|                 |                         | Progress ]{.underline}** |
|                 | -   Statewide policies  |                          |
|                 |     and systems have    | -   Statewide policies   |
|                 |     been established to |     and systems have     |
|                 |     ensure that all     |     been established to  |
|                 |     assessment systems  |     ensure that all      |
|                 |     provide a           |     assessment systems   |
|                 |     representative      |     used by all schools  |
|                 |     coverage of         |     provide a            |
|                 |     state-established,  |     representative       |
|                 |     industry-validated  |     coverage of          |
|                 |     content standards.  |     state-established,   |
|                 |                         |     industry-validated   |
|                 | -   These statewide     |     content standards.   |
|                 |     policies and        |                          |
|                 |     systems have been   | -   These statewide      |
|                 |     established in      |     policies and systems |
|                 |     cooperation with    |     have been            |
|                 |     industry and        |     established in       |
|                 |     postsecondary       |     cooperation with     |
|                 |     partners.           |     industry and         |
|                 |                         |     postsecondary        |
|                 | **1- [Does Not          |     partners.            |
|                 | Meet]{.underline}**     |                          |
|                 |                         | **1- [Does Not           |
|                 | -   Statewide policies  | Meet]{.underline}**      |
|                 |     and systems have    |                          |
|                 |     [not]{.underline}   | -   Statewide policies   |
|                 |     been established to |     and systems have     |
|                 |     ensure that all     |     [not]{.underline}    |
|                 |     assessment systems  |     been established to  |
|                 |     provide a           |     ensure that all      |
|                 |     representative      |     assessment systems   |
|                 |     coverage of         |     used by all schools  |
|                 |     state-established,  |     provide a            |
|                 |     industry-validated  |     representative       |
|                 |     content standards   |     coverage of          |
|                 |     in cooperation with |     state-established,   |
|                 |     industry and        |     industry-validated   |
|                 |     postsecondary       |     content standards in |
|                 |     partners            |     cooperation with     |
|                 |                         |     industry and         |
|                 | .                       |     postsecondary        |
|                 |                         |     partners.            |
+-----------------+-------------------------+--------------------------+
| C. Timing of    | **3-                    | **3-                     |
| Attainment      | [                       | [Standard]{.underline}** |
| Measurement     | Standard]{.underline}** |                          |
|                 |                         | -   Attainment is        |
| Attainment is   | -   Attainment is       |     measured in all      |
| measured        |     measured            |     schools              |
| concurrent with |     [after]{.underline} |     [after]{.underline}  |
| or after        |     concentrated        |     concentrated         |
| concentrated    |     participation.      |     participation.       |
| participation   |                         |                          |
| in vocational   | **2- [Satisfactory      | **2- [Satisfactory       |
| education.      | Progress]{.underline}** | Progress ]{.underline}** |
|                 |                         |                          |
|                 | -   Attainment is       | -   Attainment is        |
|                 |     measured            |     measured in all      |
|                 |     [concurrent with or |     schools [concurrent  |
|                 |     after]{.underline}  |     with or              |
|                 |     concentrated        |     after]{.underline}   |
|                 |     participation.      |     concentrated         |
|                 |                         |     participation.       |
|                 | **1- [Does Not          |                          |
|                 | Meet]{.underline}**     | **1- [Does Not Meet      |
|                 |                         | ]{.underline}**          |
|                 | -   Attainment is       |                          |
|                 |     measured in         | -   Attainment is        |
|                 |     some/all systems    |     measured in some/all |
|                 |                         |     systems or schools   |
|                 |    [before]{.underline} |     [before]{.underline} |
|                 |     concentrated        |     concentrated         |
|                 |     participation.      |     participation.       |
+-----------------+-------------------------+--------------------------+
| D. Reliability  | **3-                    | **3-                     |
| of Assessment   | [                       | [Standard]{.underline}** |
|                 | Standard]{.underline}** |                          |
| **Instruments** |                         | -   Attainment is        |
|                 | -   Attainment is       |     measured by all      |
| Attainment is   |     measured by all     |     assessment systems   |
| measured using  |     assessment systems  |     in all schools using |
| reliable        |     using assessment    |     assessment           |
| assessment      |     instruments that    |     instruments that     |
| instruments.    |     meet national or    |     meet national or     |
|                 |     state reliability   |     state reliability    |
|                 |     standards.          |     standards.           |
|                 |                         |                          |
|                 | -   All assessment      | -   All assessment       |
|                 |     systems and         |     systems and          |
|                 |     instruments are     |     instruments in all   |
|                 |     regularly evaluated |     schools are          |
|                 |     based on these      |     regularly evaluated  |
|                 |     standards.          |     based on these       |
|                 |                         |     standards.           |
|                 | **2- [Satisfactory      |                          |
|                 | Progress                | **2- [Satisfactory       |
|                 | ]{.underline}**         | Progress ]{.underline}** |
|                 |                         |                          |
|                 | -   Statewide policies  | -   Statewide policies   |
|                 |     and systems have    |     and systems have     |
|                 |     been established to |     been established to  |
|                 |     ensure that         |     ensure that          |
|                 |     attainment is       |     attainment is        |
|                 |     measured by all     |     measured by all      |
|                 |     assessment systems  |     assessment systems   |
|                 |     using assessment    |     in all schools using |
|                 |     instruments that    |     assessment           |
|                 |     meet national or    |     instruments that     |
|                 |     state reliability   |     meet national or     |
|                 |     standards.          |     state reliability    |
|                 |                         |     standards.           |
|                 | -   These statewide     |                          |
|                 |     policies and        | -   These statewide      |
|                 |     systems have been   |     policies and systems |
|                 |     established in      |     have been            |
|                 |     cooperation with    |     established in       |
|                 |     industry and        |     cooperation with     |
|                 |     postsecondary       |     industry and         |
|                 |     partners            |     postsecondary        |
|                 |                         |     partners.            |
|                 | **1- [Does Not          |                          |
|                 | Meet]{.underline}**     | **1- [Does Not           |
|                 |                         | Meet]{.underline}**      |
|                 | -   Statewide policies  |                          |
|                 |     have                | -   Statewide policies   |
|                 |     [not]{.underline}   |     and systems have     |
|                 |     been established to |     [not]{.underline}    |
|                 |     ensure that         |     been established to  |
|                 |     attainment is       |     ensure that          |
|                 |     measured by all     |     attainment is        |
|                 |     assessment systems  |     measured by all      |
|                 |     using assessment    |     assessment systems   |
|                 |     instruments that    |     in all schools using |
|                 |     meet national or    |     assessment           |
|                 |     state reliability   |     instruments that     |
|                 |     standards.          |     meet national or     |
|                 |                         |     state reliability    |
|                 |                         |     standards.           |
+-----------------+-------------------------+--------------------------+
| E.              | **3-                    | **3-                     |
|   **Reliability | [                       | [Standard]{.underline}** |
|     of          | Standard]{.underline}** |                          |
|     Assessment  |                         | -   Attainment is        |
|     A           | -   Attainment is       |     measured in all      |
| dministration** |     measured by all     |     schools using        |
|                 |     assessment systems  |     assessment           |
| Attainment is   |     using assessment    |     instruments that are |
| measured using  |     instruments that    |     administered         |
| assessment      |     are administered    |     consistently         |
| instruments     |     consistently        |     according to         |
| that are        |     according to        |     national or state    |
| administered    |     national or state   |     reliability          |
| consistently    |     reliability         |     standards (e.g.,     |
| within          |     standards (e.g.,    |     inter-rater          |
| assessment      |     inter-rater         |     reliability for      |
| systems.        |     reliability for     |     performance          |
|                 |     performance         |     assessment).         |
|                 |     assessment).        |                          |
|                 |                         | -   The administration   |
|                 | -   The statewide       |     of all assessments   |
|                 |     administration of   |     by all schools is    |
|                 |     all assessments is  |     regularly evaluated  |
|                 |     regularly evaluated |     based on these       |
|                 |     based on these      |     standards.           |
|                 |     standards.          |                          |
|                 |                         | **2- [Satisfactory       |
|                 | **2- [Satisfactory      | Progress ]{.underline}** |
|                 | Progress]{.underline}** |                          |
|                 |                         | -   Statewide policies   |
|                 | -   Statewide policies  |     and systems have     |
|                 |     and systems have    |     been established to  |
|                 |     been established to |     ensure that          |
|                 |     ensure that         |     attainment is        |
|                 |     attainment is       |     measured in all      |
|                 |     measured by all     |     schools using        |
|                 |     assessment systems  |     assessment           |
|                 |     using assessment    |     instruments that are |
|                 |     instruments that    |     administered         |
|                 |     are administered    |     consistently         |
|                 |     consistently to     |     according to         |
|                 |     national/state      |     national or state    |
|                 |     reliability         |     reliability          |
|                 |     standards (e.g.,    |     standards (e.g.,     |
|                 |     inter-rater         |     inter-rater          |
|                 |     reliability for     |     reliability for      |
|                 |     performance         |     performance          |
|                 |     assessment).        |     assessment).         |
|                 |                         |                          |
|                 | -   These statewide     | -   These statewide      |
|                 |     policies and        |     policies and systems |
|                 |     systems have been   |     have been            |
|                 |     established in      |     established in       |
|                 |     cooperation with    |     cooperation with     |
|                 |     industry and        |     industry and         |
|                 |     postsecondary       |     postsecondary        |
|                 |     partners            |     partners             |
|                 |                         |                          |
|                 | **1- [Does Not          | **1- [Does Not Meet      |
|                 | Meet]{.underline}**     | ]{.underline}**          |
|                 |                         |                          |
|                 | -   Statewide policies  | -   Statewide policies   |
|                 |     and systems have    |     and systems have     |
|                 |     [not]{.underline}   |     [not]{.underline}    |
|                 |     been established to |     been established to  |
|                 |     ensure that         |     ensure that          |
|                 |     attainment is       |     attainment is        |
|                 |     measured by all     |     measured in all      |
|                 |     assessment systems  |     schools using        |
|                 |     using assessment    |     assessment           |
|                 |     instruments that    |     instruments that are |
|                 |     are administered    |     administered         |
|                 |     consistently        |     consistently         |
|                 |     according to        |     according to         |
|                 |     national or state   |     national or state    |
|                 |     reliability         |     reliability          |
|                 |     standards.          |     standards.           |
+-----------------+-------------------------+--------------------------+
| F. Student      | **3-                    | **3-                     |
| Coverage in     | [                       | [Standard]{.underline}** |
| Attainment      | Standard]{.underline}** |                          |
| Measurement     |                         | -   Attainment outcomes  |
|                 | -   Attainment outcomes |     are reported by all  |
| Performance     |     are reported by all |     schools for at least |
| measurement     |     schools for at      |     90% of their         |
| reports         |     least 90% of their  |     students reaching    |
| attainment data |     students reaching   |     threshold levels and |
| for all         |     threshold levels    |     who are expected to  |
| students        |     who are expected to |     take local           |
| reaching        |     take national or    |     assessments or their |
| state-defined   |     state assessments   |     equivalents (e.g.,   |
| threshold       |     or their            |     alternative          |
| levels in the   |     equivalents (e.g.,  |     assessments)         |
| state.          |     alternative         |     according to state   |
|                 |     assessments)        |     or local policies.   |
| ###             |     according to state  |                          |
|                 |     or local policies.  | **2- [Satisfactory       |
|                 |                         | Progress ]{.underline}** |
|                 | **2- [Satisfactory      |                          |
|                 | Progress                | -   Attainment outcomes  |
|                 | ]{.underline}**         |     are reported by all  |
|                 |                         |     schools for at least |
|                 | -   Attainment outcomes |     80% of their         |
|                 |     are reported for    |     students reaching    |
|                 |     all schools for at  |     threshold levels and |
|                 |     least 80% of all of |     who are expected to  |
|                 |     their students      |     take local           |
|                 |     reaching threshold  |     assessments or their |
|                 |     levels and who are  |     equivalents (e.g.,   |
|                 |     expected to take    |     alternative          |
|                 |     national or state   |     assessments)         |
|                 |     assessments or      |     according to state   |
|                 |     their equivalents   |     or local policies.   |
|                 |     (e.g., alternative  |                          |
|                 |     assessments)        | **1- [Does Not           |
|                 |     according to state  | Meet]{.underline}**      |
|                 |     or local policies.  |                          |
|                 |                         | -   Attainment outcomes  |
|                 | **1- [Does Not          |     are not reported by  |
|                 | Meet]{.underline}**.    |     all schools or some  |
|                 |                         |     schools report less  |
|                 | -   Attainment outcomes |     than 80% of their    |
|                 |     are not reported    |     students reaching    |
|                 |     for all schools     |     threshold levels and |
|                 |     [or]{.underline}    |     who are expected to  |
|                 |     some schools report |     take local           |
|                 |     less than 80% of    |     assessments or their |
|                 |     their students who  |     equivalents (e.g.,   |
|                 |     reach threshold     |     alternative          |
|                 |     levels and who are  |     assessments)         |
|                 |     expected to take    |     according to state   |
|                 |     national or state   |     or local policies.   |
|                 |     assessments or      |                          |
|                 |     their equivalents   |                          |
|                 |     (e.g., alternative  |                          |
|                 |     assessments)        |                          |
|                 |     according to state  |                          |
|                 |     or local policies.  |                          |
+-----------------+-------------------------+--------------------------+

#### 1P1: Postsecondary Academic Attainment

**Measure**

[Numerator]{.underline}: Number of students reaching a state-defined
threshold level of vocational education to complete a postsecondary
program (concentrators) who have met program-defined academic standards
in the reporting year.

[Denominator]{.underline}: Number of students reaching a state-defined
threshold level of vocational education to complete a postsecondary
program (concentrator) in the reporting year.

**Measurement Approaches**

1.  [National/State Academic Assessment
    System]{.underline}---State-defined performance benchmarks on
    statewide academic assessment systems.

2.  [Academic Course Completion]{.underline}---State-defined performance
    benchmarks (e.g., grade, certificate) in designated academic courses
    including integrated academic/vocational courses.

3.  [Vocational Courses Completion]{.underline}---Performance benchmarks
    on course or program assessments in vocational courses with
    integrated academics.

4.  [Academic Grade Point Average]{.underline}---State-defined grade
    point average for all designated academic courses including
    integrated academic/vocational courses.

5.  [Overall Grade Point Average]{.underline}---State-defined grade
    point average for all courses in the school or program.

6.  [Program Completion]{.underline}---Completion of post-secondary
    programs (when graduation or completion is the same as attaining
    state-or program-defined academic standards).

**Quality Criteria and Scoring Rubric**

The standards levels in the scoring rubrics are the
[proposed]{.underline} levels for use [only]{.underline} in state peer
reviews in the Data Quality Initiative. States will be asked to review
and suggest revisions after using them in first peer review process.
They are not federal requirements.

+----------------+--------------------------+--------------------------+
| **Quality      | **National and/or State  | **Approaches with Local  |
| Criteria**     | Academic Assessment      | Assessment Systems**     |
|                | Systems**                |                          |
|                |                          | **Academic Course        |
|                |                          | Completion**             |
|                |                          |                          |
|                |                          | **Vocational Course      |
|                |                          | Completion**             |
|                |                          |                          |
|                |                          | **Academic Grade Point   |
|                |                          | Average**                |
|                |                          |                          |
|                |                          | **Overall Grade Point    |
|                |                          | Average**                |
|                |                          |                          |
|                |                          | **Program Completion**   |
+----------------+--------------------------+--------------------------+
| A              | **3-                     | **3-                     |
| .  **Alignment | [Standard]{.underline}** | [Standard]{.underline}** |
|     to         |                          |                          |
|     P          | -   All assessment       | -   All assessment       |
| rogram-Defined |     systems are directly |     systems used by all  |
|     Academic   |     aligned to program   |     institutions are     |
|                |     academic content     |     directly aligned to  |
|    Standards** |     [and]{.underline}    |     program academic     |
|                |     performance          |     content              |
| Attainment     |     standards for all    |     [and]{.underline}    |
| measures and   |     content areas (e.g., |     performance          |
| assessment     |     language arts)       |     standards for all    |
| systems are    |     addressed by the     |     content areas (e.g., |
| aligned to     |     measures.            |     language arts)       |
| (validity)     |                          |     addressed by the     |
| p              | -   All assessment       |     measures.            |
| rogram-defined |     systems are          |                          |
| academic       |     regularly evaluated  | -   All assessment       |
| content        |     on their alignment   |     systems in all       |
| [an            |     to these standards.  |     institutions are     |
| d]{.underline} |                          |     regularly evaluated  |
| performance    | **2- [Satisfactory       |     on their alignment   |
| standards      | Progress]{.underline}**  |     to these standards.  |
|                |                          |                          |
|                | -   Statewide policies   | **2- [Satisfactory       |
|                |     and systems have     | Progress]{.underline}**  |
|                |     been established to  |                          |
|                |     ensure that all      | -   Statewide policies   |
|                |     assessment systems   |     and systems have     |
|                |     are directly aligned |     been established to  |
|                |     to program academic  |     ensure that all      |
|                |     content standards    |     assessment systems   |
|                |     for all content      |     used by all          |
|                |     areas (e.g.,         |     institutions are     |
|                |     language arts)       |     directly aligned to  |
|                |     addressed by the     |     program academic     |
|                |     measures.            |     content standards    |
|                |                          |     for all content      |
|                | **1- [Does Not Meet      |     areas (e.g.,         |
|                | ]{.underline}**          |     language arts)       |
|                |                          |     addressed by the     |
|                | -   Statewide policies   |     measures.            |
|                |     and systems have     |                          |
|                |     [not]{.underline}    | **1- [Does Not           |
|                |     been established to  | Meet]{.underline}**      |
|                |     ensure that all      |                          |
|                |     assessment systems   | -   Statewide policies   |
|                |     are directly aligned |     and systems have     |
|                |     to program academic  |     [not]{.underline}    |
|                |     content standards    |     been established to  |
|                |     for all content      |     ensure that all      |
|                |     areas (e.g.,         |     assessment systems   |
|                |     language arts)       |     used by schools are  |
|                |     addressed by the     |     directly aligned to  |
|                |     measures.            |     program academic     |
|                |                          |     content standards    |
|                | .                        |     for all content      |
|                |                          |     areas (e.g.,         |
|                |                          |     language arts)       |
|                |                          |     addressed by the     |
|                |                          |     measures.            |
+----------------+--------------------------+--------------------------+
| **B. Scope of  | **3-                     | **3-                     |
| Attainment**   | [Standard]{.underline}** | [Standard]{.underline}** |
|                |                          |                          |
| *              | -   All assessment       | -   All assessment       |
| *Measurement** |     systems provide a    |     systems used by all  |
|                |     representative       |     schools provide a    |
| Attainment     |     coverage of all      |     representative       |
| measures       |     components of the    |     coverage of all      |
| provide a      |     academic content     |     components of the    |
| representative |     areas addressed by   |     academic content     |
| coverage       |     program academic     |     areas addressed by   |
| (validity) for |     standards.           |     program academic     |
| all major      |                          |     standards.           |
| components of  | -   All assessment       |                          |
| the content    |     systems are          | -   All assessment       |
| areas in       |     regularly evaluated  |     systems in all       |
| program        |     on their coverage of |     institutions are     |
| academic       |     these standards.     |     regularly evaluated  |
| standards.     |                          |     on their coverage of |
|                | **2- [Satisfactory       |     these standards.     |
|                | Progress ]{.underline}** |                          |
|                |                          | **2- [Satisfactory       |
|                | -   Statewide policies   | Progress ]{.underline}** |
|                |     and systems have     |                          |
|                |     been established to  | -   Statewide policies   |
|                |     ensure that all      |     and systems have     |
|                |     assessment systems   |     been established to  |
|                |     provide a            |     ensure that all      |
|                |     representative       |     assessment systems   |
|                |     coverage of all      |     used by all          |
|                |     major components of  |     institutions provide |
|                |     the content areas    |     a representative     |
|                |     addressed by program |     coverage of all      |
|                |     academic standards.  |     major components of  |
|                |                          |     content areas        |
|                | **1- [Does Not Meet      |     addressed by program |
|                | ]{.underline}**          |     academic standards.  |
|                |                          |                          |
|                | -   Statewide policies   | **1- [Does Not           |
|                |     and systems have     | Meet]{.underline}**      |
|                |     [not]{.underline}    |                          |
|                |     been established to  | -   Statewide policies   |
|                |     ensure that all      |     and systems have     |
|                |     assessment systems   |     [not]{.underline}    |
|                |     provide a            |     been established to  |
|                |     representative       |     ensure that all      |
|                |     coverage of all      |     assessment systems   |
|                |     major components of  |     used by institutions |
|                |     the content areas    |     provide a            |
|                |     addressed by program |     representative       |
|                |     academic standards.  |     coverage of all      |
|                |                          |     major components of  |
|                |                          |     the content areas    |
|                |                          |     addressed by program |
|                |                          |     academic standards.  |
+----------------+--------------------------+--------------------------+
| C. Timing of   | **3-                     | **3-                     |
| Attainment     | [Standard]{.underline}** | [Standard]{.underline}** |
| Measurement    |                          |                          |
|                | -   Attainment is        | -   Attainment is        |
| Attainment is  |     measured             |     measured in all      |
| measured       |     [after]{.underline}  |     schools              |
| concurrent     |     concentrated         |     [after]{.underline}  |
| with or after  |     participation.       |     concentrated         |
| concentrated   |                          |     participation.       |
| participation  | **2- [Satisfactory       |                          |
| in vocational  | Progress]{.underline}**  | **2- [Satisfactory       |
| education.     |                          | Progress ]{.underline}** |
|                | -   Attainment is        |                          |
|                |     measured [concurrent | -   Attainment is        |
|                |     with or              |     measured in all      |
|                |     after]{.underline}   |     schools [concurrent  |
|                |     concentrated         |     with or              |
|                |     participation.       |     after]{.underline}   |
|                |                          |     concentrated         |
|                | **1- [Does Not Meet      |     participation.       |
|                | ]{.underline}**          |                          |
|                |                          | **1- [Does Not           |
|                | -   Attainment is        | Meet]{.underline}**      |
|                |     measured in some/all |                          |
|                |     systems              | -   Attainment is        |
|                |     [before]{.underline} |     measured in some/all |
|                |     concentrated         |     systems or schools   |
|                |     participation.       |     [before]{.underline} |
|                |                          |     concentrated         |
|                |                          |     participation.       |
+----------------+--------------------------+--------------------------+
| D. Reliability | **3-                     | **3-                     |
| of Assessment  | [Standard]{.underline}** | [Standard]{.underline}** |
| Instruments    |                          |                          |
|                | -   Attainment is        | -   Attainment is        |
| Attainment is  |     measured by all      |     measured by all      |
| measured using |     assessment systems   |     assessment systems   |
| reliable       |     using assessment     |     in all institutions  |
| assessment     |     instruments that     |     using assessment     |
| instruments.   |     meet national or     |     instruments that     |
|                |     state reliability    |     meet national or     |
|                |     standards.           |     state reliability    |
|                |                          |     standards.           |
|                | -   All assessment       |                          |
|                |     systems and          | -   All assessment       |
|                |     instruments are      |     systems and          |
|                |     regularly evaluated  |     instruments in all   |
|                |     based on these       |     schools are          |
|                |     standards.           |     regularly evaluated  |
|                |                          |     based on these       |
|                | **2- [Satisfactory       |     standards.           |
|                | Progress ]{.underline}** |                          |
|                |                          | **2- [Satisfactory       |
|                | -   Statewide policies   | Progress ]{.underline}** |
|                |     and systems have     |                          |
|                |     been established to  | -   Statewide policies   |
|                |     ensure that          |     and systems have     |
|                |     attainment is        |     been established to  |
|                |     measured by all      |     ensure that          |
|                |     assessment systems   |     attainment is        |
|                |     using assessment     |     measured by all      |
|                |     instruments that     |     assessment systems   |
|                |     meet national or     |     in all institutions  |
|                |     state reliability    |     using assessment     |
|                |     standards.           |     instruments that     |
|                |                          |     meet national or     |
|                | **1- [Does Not           |     state reliability    |
|                | Meet]{.underline}**      |     standards.           |
|                |                          |                          |
|                | -   Statewide policies   | **1- [Does Not           |
|                |     have                 | Meet]{.underline}**      |
|                |     [not]{.underline}    |                          |
|                |     been established to  | -   Statewide policies   |
|                |     ensure that          |     and systems have     |
|                |     attainment is        |     [not]{.underline}    |
|                |     measured by all      |     been established to  |
|                |     assessment systems   |     ensure that          |
|                |     using assessment     |     attainment is        |
|                |     instruments that     |     measured by all      |
|                |     meet national or     |     assessment systems   |
|                |     state reliability    |     in all institutions  |
|                |     standards.           |     using assessment     |
|                |                          |     instruments that     |
|                |                          |     meet national or     |
|                |                          |     state reliability    |
|                |                          |     standards.           |
+----------------+--------------------------+--------------------------+
| E. Reliability | **3-                     | **3-                     |
| of Assessment  | [Standard]{.underline}** | [Standard]{.underline}** |
| Administration |                          |                          |
|                | -   Attainment is        | -   Attainment is        |
| Attainment is  |     measured by all      |     measured in all      |
| measured using |     assessment systems   |     institutions using   |
| assessment     |     using assessment     |     assessment           |
| instruments    |     instruments that are |     instruments that are |
| that are       |     administered         |     administered         |
| administered   |     consistently         |     consistently         |
| consistently   |     according to         |     according to         |
| within         |     national or state    |     national or state    |
| assessment     |     reliability          |     reliability          |
| systems.       |     standards (e.g.,     |     standards (e.g.,     |
|                |     inter-rater          |     inter-rater          |
|                |     reliability for      |     reliability for      |
|                |     performance          |     performance          |
|                |     assessment).         |     assessment).         |
|                |                          |                          |
|                | -   The statewide        | -   The administration   |
|                |     administration of    |     of all assessments   |
|                |     all assessments is   |     by all schools is    |
|                |     regularly evaluated  |     regularly evaluated  |
|                |     based on these       |     based on these       |
|                |     standards.           |     standards.           |
|                |                          |                          |
|                | **2- [Satisfactory       | **2- [Satisfactory       |
|                | Progress]{.underline}**  | Progress ]{.underline}** |
|                |                          |                          |
|                | -   Statewide policies   | -   Statewide policies   |
|                |     and systems have     |     and systems have     |
|                |     been established to  |     been established to  |
|                |     ensure that          |     ensure that          |
|                |     attainment is        |     attainment is        |
|                |     measured by all      |     measured in all      |
|                |     assessment systems   |     institutions using   |
|                |     using assessment     |     assessment           |
|                |     instruments that are |     instruments that are |
|                |     administered         |     administered         |
|                |     consistently         |     consistently         |
|                |     according to         |     according to         |
|                |     national or state    |     national or state    |
|                |     reliability          |     reliability          |
|                |     standards (e.g.,     |     standards (e.g.,     |
|                |     inter-rater          |     inter-rater          |
|                |     reliability for      |     reliability for      |
|                |     performance          |     performance          |
|                |     assessment).         |     assessment).         |
|                |                          |                          |
|                | **1- [Does Not           | **1- [Does Not Meet      |
|                | Meet]{.underline}**      | ]{.underline}**          |
|                |                          |                          |
|                | -   Statewide policies   | -   Statewide policies   |
|                |     and systems have     |     and systems have     |
|                |     [not]{.underline}    |     [not]{.underline}    |
|                |     been established to  |     been established to  |
|                |     ensure that          |     ensure that          |
|                |     attainment is        |     attainment is        |
|                |     measured by all or   |     measured in all      |
|                |     some assessment      |     institutions using   |
|                |     systems using        |     assessment           |
|                |     assessment           |     instruments that are |
|                |     instruments that are |     administered         |
|                |     not administered     |     consistently         |
|                |     consistently         |     according to         |
|                |     according to         |     national or state    |
|                |     national or state    |     reliability          |
|                |     reliability          |     standards.           |
|                |     standards.           |                          |
+----------------+--------------------------+--------------------------+
| F. Student     | **3-                     | **3-                     |
| Coverage in    | [Standard]{.underline}** | [Standard]{.underline}** |
| Attainment     |                          |                          |
| Measurement    | -   Attainment outcomes  | -   Attainment outcomes  |
|                |     are reported by all  |     are reported by all  |
| ###            |     schools for at least |     schools for at least |
|                |     90% of their         |     90% of their         |
| Performance    |     students reaching    |     students reaching    |
| measurement    |     threshold levels who |     threshold levels and |
| reports        |     are expected to take |     who are expected to  |
| attainment     |     national or state    |     take local           |
| data for all   |     academic assessments |     assessments or their |
| students       |     or their equivalents |     equivalents (e.g.,   |
| reaching       |     (e.g., alternative   |     alternative          |
| state-defined  |     assessments)         |     assessments)         |
| threshold      |     according to state   |     according to state   |
| levels in the  |     or local policies.   |     or local policies.   |
| state.         |                          |                          |
|                | **2- [Satisfactory       | **2- [Satisfactory       |
| ###            | Progress]{.underline}**. | Progress]{.underline}**  |
|                |                          |                          |
|                | -   Attainment outcomes  | -   Attainment outcomes  |
|                |     are reported for all |     are reported by all  |
|                |     schools for at least |     schools for at least |
|                |     80% of their         |     80% of their         |
|                |     students reaching    |     students reaching    |
|                |     threshold levels and |     threshold levels and |
|                |     who are expected to  |     who are expected to  |
|                |     take national or     |     take local           |
|                |     state academic       |     assessments or their |
|                |     assessments or their |     equivalents (e.g.,   |
|                |     equivalents (e.g.,   |     alternative          |
|                |     alternative          |     assessments)         |
|                |     assessments)         |     according to state   |
|                |     according to state   |     or local policies.   |
|                |     or local policies.   |                          |
|                |                          | **1- [Does Not           |
|                | **1- [Does Not Meet      | Meet]{.underline}**      |
|                | ]{.underline}**          |                          |
|                |                          | -   Attainment outcomes  |
|                | -   Attainment outcomes  |     are                  |
|                |     are                  |     [not]{.underline}    |
|                |     [not]{.underline}    |     reported by all      |
|                |     reported for all     |     schools or some      |
|                |     schools              |     schools report less  |
|                |     [or]{.underline}     |     than 80% of their    |
|                |     some schools report  |     students reaching    |
|                |     less than 80% of     |     threshold levels and |
|                |     their students who   |     who are expected to  |
|                |     reach threshold      |     take local           |
|                |     levels and who are   |     assessments or their |
|                |     expected to take     |     equivalents (e.g.,   |
|                |     national or state    |     alternative          |
|                |     academic assessments |     assessments)         |
|                |     or their equivalents |     according to state   |
|                |     (e.g., alternative   |     or local policies.   |
|                |     assessments)         |                          |
|                |     according to state   |                          |
|                |     or local policies.   |                          |
+----------------+--------------------------+--------------------------+

#### 1P2: Postsecondary Vocational and Technical Skill Attainment

**Measure**

[Numerator]{.underline}: Number of students reaching a state-defined
threshold level of vocational education to complete a postsecondary
program (concentrators) who have met program-defined, industry-validated
career and technical standards in the reporting year.

[Denominator]{.underline}: Number of students reaching a state-defined
threshold level of vocational education to complete a postsecondary
program (concentrator) in the reporting year.

**Measurement Approach**

1.  [National/State Standards and Assessment
    System]{.underline}---Performance benchmarks on national or state
    standards and assessment systems using national, or state-developed
    and standardized assessment instruments and assessment
    administration procedures (including licensing/certification
    examinations).

2.  [National/State Standards and Local Assessment
    System]{.underline}---Performance benchmarks on local assessment
    systems that have been approved by national or state organizations
    based on national or state standards.

3.  [Local Standards and Assessment Systems]{.underline}---Performance
    benchmarks on state-approved local assessment systems based on
    state-approved local standards.

4.  [Vocational/Technical Course]{.underline} ---State-defined
    performance benchmarks (e.g., grade, certificate) in designated
    career and technical education courses including integrated
    academic/vocational courses.

5.  [Vocational/Technical Grade Point Average]{.underline}---Grade point
    average for all designated vocational/technical courses including
    integrated academic/vocational courses.

6.  [Program Completion]{.underline}---Completion of
    vocational/technical education program when program completion
    represents attainment of career and technical skill standards.

**Quality Criteria and Scoring Rubric**

The standards levels in the scoring rubrics are the
[proposed]{.underline} levels for use [only]{.underline} in state peer
reviews in the Data Quality Initiative. States will be asked to review
and suggest revisions after using them in the first peer review process.
[They are not federal requirements.]{.underline}

+----------------+-------------------------+---------------------------+
| **Quality      | **National and/or       | **Approaches with Local   |
| Criteria**     | State**                 | Assessment Systems**      |
|                |                         |                           |
|                | **Assessment Systems**  | **Local Standards and     |
|                |                         | Assessment Systems**      |
|                |                         |                           |
|                |                         | **Vocational/Technical    |
|                |                         | Course Completion**       |
|                |                         |                           |
|                |                         | **Vocational Grade Point  |
|                |                         | Average**                 |
|                |                         |                           |
|                |                         | **Program Completion**    |
+----------------+-------------------------+---------------------------+
| A              | **3-                    | **3-                      |
| .  **Alignment | [                       | [Standard]{.underline}**  |
|     to         | Standard]{.underline}** |                           |
|     Industry   |                         | -   All assessment        |
|                | -   All assessment      |     systems used by all   |
|    Standards** |     systems are         |     institutions are      |
|                |     directly aligned to |     directly aligned to   |
| Attainment     |     program-defined,    |     program-defined,      |
| measures are   |     industry-validated  |     industry-validated    |
| aligned        |     content             |     content               |
| (validity) to  |     [and]{.underline}   |     [and]{.underline}     |
| pr             |     performance         |     performance           |
| ogram-defined, |     standards.          |     standards.            |
| indu           |                         |                           |
| stry-validated | -   All assessment      | -   All assessment        |
| skill          |     systems are         |     systems in all        |
| st             |     regularly evaluated |     institutions are      |
| andards---both |     on their alignment  |     regularly evaluated   |
| content and    |     to these standards  |     on their alignment to |
| performance    |     in cooperation with |     these standards in    |
| standards      |     industry and other  |     cooperation with      |
|                |     postsecondary       |     industry and other    |
|                |     partners.           |     postsecondary         |
|                |                         |     partners.             |
|                | **2- [Satisfactory      |                           |
|                | Progress]{.underline}** | **2- [Satisfactory        |
|                |                         | Progress]{.underline}**   |
|                | -   Statewide policies  |                           |
|                |     and systems have    | -   Statewide policies    |
|                |     been established to |     and systems have been |
|                |     ensure that all     |     established to ensure |
|                |     assessment systems  |     that all assessment   |
|                |     are directly        |     systems used by all   |
|                |     aligned to          |     institutions are      |
|                |     program-defined,    |     directly aligned to   |
|                |     industry-validated  |     program-defined,      |
|                |     content standards.  |     industry-validated    |
|                |                         |     content standards.    |
|                | -   These statewide     |                           |
|                |     policies and        | -   These statewide       |
|                |     systems have been   |     policies and systems  |
|                |     established in      |     have been established |
|                |     cooperation with    |     in cooperation with   |
|                |     industry and other  |     industry and other    |
|                |     postsecondary       |     postsecondary         |
|                |     partners.           |     partners.             |
|                |                         |                           |
|                | **1- [Does Not          | **1- [Does Not Meet       |
|                | Meet]{.underline}**     | ]{.underline}**           |
|                |                         |                           |
|                | -   Statewide policies  | -   Statewide policies    |
|                |     and systems have    |     and systems have      |
|                |     [not]{.underline}   |     [not]{.underline}     |
|                |     been established to |     been established to   |
|                |     ensure that all     |     ensure that all       |
|                |     assessment systems  |     assessment systems    |
|                |     are directly        |     used by all schools   |
|                |     aligned to          |     are directly aligned  |
|                |     program-defined,    |     to program-defined,   |
|                |     industry-validated  |     industry-validated    |
|                |     content standards   |     content standards in  |
|                |     in cooperation with |     cooperation with      |
|                |     industry and other  |     industry and other    |
|                |     postsecondary       |     postsecondary         |
|                |     partners            |     partners.             |
|                |                         |                           |
|                | .                       |                           |
+----------------+-------------------------+---------------------------+
| **B. Scope of  | **3-                    | **3-                      |
| Attainment**   | [                       | [Standard]{.underline}**  |
|                | Standard]{.underline}** |                           |
| *              |                         | -   All assessment        |
| *Measurement** | -   All assessment      |     systems used by all   |
|                |     systems provide a   |     institutions provide  |
| Attainment     |     representative      |     a representative      |
| measures       |     coverage of         |     coverage of           |
| provide a      |     program-defined,    |     program-defined,      |
| representative |     industry-validated  |     industry-validated    |
| coverage       |     content             |     content               |
| (validity) of  |     [and]{.underline}   |     [and]{.underline}     |
| pr             |     performance         |     performance           |
| ogram-defined, |     standards.          |     standards.            |
| indu           |                         |                           |
| stry-validated | -   All assessment      | -   All assessment        |
| skill          |     systems are         |     systems in all        |
| standards.     |     regularly evaluated |     institutions are      |
|                |     on their coverage   |     regularly evaluated   |
|                |     of these standards  |     on their coverage of  |
|                |     in cooperation with |     these standards in    |
|                |     industry and other  |     cooperation with      |
|                |     postsecondary       |     industry and other    |
|                |     partners.           |     postsecondary         |
|                |                         |     partners.             |
|                | **2- [Satisfactory      |                           |
|                | Progress]{.underline}** | **2- [Satisfactory        |
|                |                         | Progress]{.underline}**   |
|                | -   Statewide policies  |                           |
|                |     and systems have    | -   Statewide policies    |
|                |     been established to |     and systems have been |
|                |     ensure that all     |     established to ensure |
|                |     assessment systems  |     that all assessment   |
|                |     provide a           |     systems used by all   |
|                |     representative      |     institutions provide  |
|                |     coverage of         |     a representative      |
|                |     program-defined,    |     coverage of           |
|                |     industry-validated  |     program-defined,      |
|                |     content standards.  |     industry-validated    |
|                |                         |     content standards.    |
|                | -   These statewide     |                           |
|                |     policies and        | -   These statewide       |
|                |     systems have been   |     policies and systems  |
|                |     established in      |     have been established |
|                |     cooperation with    |     in cooperation with   |
|                |     industry and other  |     industry and other    |
|                |     postsecondary       |     postsecondary         |
|                |     partners.           |     partners.             |
|                |                         |                           |
|                | **1- [Does Not          | **1- [Does Not            |
|                | Meet]{.underline}**     | Meet]{.underline}**       |
|                |                         |                           |
|                | -   Statewide policies  | -   Statewide policies    |
|                |     and systems have    |     and systems have      |
|                |     [not]{.underline}   |     [not]{.underline}     |
|                |     been established to |     been established to   |
|                |     ensure that all     |     ensure that all       |
|                |     assessment systems  |     assessment systems    |
|                |     provide a           |     used by all           |
|                |     representative      |     institutions provide  |
|                |     coverage of         |     a representative      |
|                |     state-established,  |     coverage of           |
|                |     industry-validated  |     state-established,    |
|                |     content standards   |     industry-validated    |
|                |     in cooperation with |     content standards in  |
|                |     industry and other  |     cooperation with      |
|                |     postsecondary       |     industry and other    |
|                |     partners.           |     postsecondary         |
|                |                         |     partners.             |
+----------------+-------------------------+---------------------------+
| C. Timing of   | **3-                    | **3-                      |
| Attainment     | [                       | [Standard]{.underline}**  |
| Measurement    | Standard]{.underline}** |                           |
|                |                         | -   Attainment is         |
| Attainment is  | -   Attainment is       |     measured in all       |
| measured       |     measured            |     institutions          |
| concurrent     |     [after]{.underline} |     [after]{.underline}   |
| with or after  |     concentrated        |     concentrated          |
| concentrated   |     participation.      |     participation.        |
| participation  |                         |                           |
| in vocational  | **2- [Satisfactory      | **2- [Satisfactory        |
| education.     | Progress]{.underline}** | Progress]{.underline}**   |
|                |                         |                           |
|                | -   Attainment is       | -   Attainment is         |
|                |     measured            |     measured in all       |
|                |     [concurrent with or |     institutions          |
|                |     after]{.underline}  |     [concurrent with or   |
|                |     concentrated        |     after]{.underline}    |
|                |     participation.      |     concentrated          |
|                |                         |     participation.        |
|                | **1- [Does Not Meet     |                           |
|                | ]{.underline}**         | **1- [Does Not            |
|                |                         | Meet]{.underline}**       |
|                | -   Attainment is       |                           |
|                |     measured in         | -   Attainment is         |
|                |     some/all systems    |     measured in some/all  |
|                |                         |     systems or            |
|                |    [before]{.underline} |     institutions          |
|                |     concentrated        |     [before]{.underline}  |
|                |     participation.      |     concentrated          |
|                |                         |     participation.        |
|                |                         |                           |
|                |                         | ```{=html}                |
|                |                         | <!-- -->                  |
|                |                         | ```                       |
|                |                         | -                         |
+----------------+-------------------------+---------------------------+
| D. Reliability | **3-                    | **3-                      |
| of Assessment  | [                       | [Standard]{.underline}**  |
|                | Standard]{.underline}** |                           |
| *              |                         | -   Attainment is         |
| *Instruments** | -   Attainment is       |     measured by all       |
|                |     measured by all     |     assessment systems in |
| Attainment is  |     assessment systems  |     all institutions      |
| measured using |     using assessment    |     using assessment      |
| reliable       |     instruments that    |     instruments that meet |
| assessment     |     meet national or    |     national or state     |
| instruments.   |     state reliability   |     reliability           |
|                |     standards.          |     standards.            |
|                |                         |                           |
|                | -   All assessment      | -   All assessment        |
|                |     systems and         |     systems and           |
|                |     instruments are     |     instruments in all    |
|                |     regularly evaluated |     institu-tions are     |
|                |     based on these      |     regularly evaluated   |
|                |     standards.          |     based on these        |
|                |                         |     standards.            |
|                | **2- [Satisfactory      |                           |
|                | Progress]{.underline}** | **2- [Satisfactory        |
|                |                         | Progress]{.underline}**   |
|                | -   Statewide policies  |                           |
|                |     and systems have    | -   Statewide policies    |
|                |     been established to |     and systems have been |
|                |     ensure that         |     established to ensure |
|                |     attainment is       |     that attainment is    |
|                |     measured by all     |     measured by all       |
|                |     assessment systems  |     assessment systems in |
|                |     using assessment    |     all institutions      |
|                |     instruments that    |     using assessment      |
|                |     meet national or    |     instruments that meet |
|                |     state reliability   |     national or state     |
|                |     standards.          |     reliability           |
|                |                         |     standards.            |
|                | -   These statewide     |                           |
|                |     policies and        | -   These statewide       |
|                |     systems have been   |     policies and systems  |
|                |     established in      |     have been established |
|                |     cooperation with    |     in cooperation with   |
|                |     industry and other  |     industry and other    |
|                |     postsecondary       |     postsecondary         |
|                |     partners            |     partners.             |
|                |                         |                           |
|                | 1.  **[Does Not         | **1- [Does Not            |
|                |     Meet]{.underline}** | Meet]{.underline}**       |
|                |                         |                           |
|                | -   Statewide policies  | -   Statewide policies    |
|                |     have                |     and systems have      |
|                |     [not]{.underline}   |     [not]{.underline}     |
|                |     been established to |     been established to   |
|                |     ensure that         |     ensure that           |
|                |     attainment is       |     attainment is         |
|                |     measured by all     |     measured by all       |
|                |     assessment systems  |     assessment systems in |
|                |     using assessment    |     all institutions      |
|                |     instruments that    |     using assessment      |
|                |     meet national or    |     instruments that meet |
|                |     state reliability   |     national or state     |
|                |     standards.          |     reliability           |
|                |                         |     standards.            |
+----------------+-------------------------+---------------------------+
| E. Reliability | **3-                    | **3-                      |
| of Assessment  | [                       | [Standard]{.underline}**  |
|                | Standard]{.underline}** |                           |
| **Ad           |                         | -   Attainment is         |
| ministration** | -   Attainment is       |     measured in all       |
|                |     measured by all     |     institutions using    |
| Attainment is  |     assessment systems  |     assessment            |
| measured using |     using assessment    |     instruments that are  |
| assessment     |     instruments that    |     administered          |
| instruments    |     are administered    |     consistently          |
| that are       |     consistently        |     according to national |
| administered   |     according to        |     or state reliability  |
| consistently   |     national or state   |     standards (e.g.,      |
| within         |     reliability         |     inter-rater           |
| assessment     |     standards (e.g.,    |     reliability for       |
| systems.       |     inter-rater         |     performance           |
|                |     reliability for     |     assessment).          |
|                |     performance         |                           |
|                |     assessment).        | -   The administration of |
|                |                         |     all assessments by    |
|                | -   The statewide       |     all institu-tions is  |
|                |     administration of   |     regularly evaluated   |
|                |     all assessments is  |     based on these        |
|                |     regularly evaluated |     standards.            |
|                |     based on these      |                           |
|                |     standards.          | **2- [Satisfactory        |
|                |                         | Progress]{.underline}**   |
|                | **2- [Satisfactory      |                           |
|                | Progress]{.underline}** | -   Statewide policies    |
|                |                         |     and systems have been |
|                | -   Statewide policies  |     established to ensure |
|                |     and systems have    |     that attainment is    |
|                |     been established to |     measured in all       |
|                |     ensure that         |     institutions using    |
|                |     attainment is       |     assessment            |
|                |     measured by all     |     instruments that are  |
|                |     assessment systems  |     administered          |
|                |     using assessment    |     consistently          |
|                |     instruments that    |     according to national |
|                |     are administered    |     or state reliability  |
|                |     consistently        |     standards (e.g.,      |
|                |     according to        |     inter-rater           |
|                |     national or state   |     reliability for       |
|                |     reliability         |     performance           |
|                |     standards (e.g.,    |     assessment).          |
|                |     inter-rater         |                           |
|                |     reliability for     | -   These statewide       |
|                |     performance         |     policies and systems  |
|                |     assessment).        |     have been established |
|                |                         |     in cooperation with   |
|                | -   These statewide     |     industry and other    |
|                |     policies and        |     postsecondary         |
|                |     systems have been   |     partners.             |
|                |     established in      |                           |
|                |     cooperation with    | **1- [Does Not Meet       |
|                |     industry and other  | ]{.underline}**           |
|                |     postsecondary       |                           |
|                |     partners            | -   Statewide policies    |
|                |                         |     and systems have      |
|                | **1- [Does Not Meet     |     [not]{.underline}     |
|                | ]{.underline}**         |     been established to   |
|                |                         |     ensure that           |
|                | -   Statewide policies  |     attainment is         |
|                |     and systems have    |     measured in all       |
|                |     [not]{.underline}   |     institutions using    |
|                |     been established to |     assessment            |
|                |     ensure that         |     instruments that are  |
|                |     attainment is       |     administered          |
|                |     measured by all     |     consistently          |
|                |     assessment systems  |     according to national |
|                |     using assessment    |     or state reliability  |
|                |     instruments that    |     standards in          |
|                |     are administered    |     cooperation with      |
|                |     consistently        |     industry and other    |
|                |     according to        |     postsecondary         |
|                |     national or state   |     partners.             |
|                |     reliability         |                           |
|                |     standards in        |                           |
|                |     cooperation with    |                           |
|                |     industry and other  |                           |
|                |     postsecondary       |                           |
|                |     partners.           |                           |
+----------------+-------------------------+---------------------------+
| **F. Student   | **3-                    | **3-                      |
| Coverage in**  | [                       | [Standard]{.underline}**  |
|                | Standard]{.underline}** |                           |
| Attainment     |                         | -   Attainment outcomes   |
| Measurement    | -   Attainment outcomes |     are reported by all   |
|                |     are reported by all |     institutions for at   |
| ###            |     institutions for at |     least 90% of their    |
|                |     least 90% of their  |     students reaching     |
| Performance    |     students reaching   |     threshold levels and  |
| measurement    |     threshold levels    |     who are expected to   |
| reports        |     who are expected to |     take local            |
| attainment     |     take national or    |     assessments or their  |
| data for all   |     state assessments   |     equivalents (e.g.,    |
| students       |     or their            |     alternative           |
| reaching       |     equivalents (e.g.,  |     assessments)          |
| state-defined  |     alternative         |     according to state or |
| threshold      |     assessments)        |     local policies.       |
| levels in the  |     according to state  |                           |
| state.         |     or local policies.  | **2- [Satisfactory        |
|                |                         | Progress]{.underline}**   |
| ###            | **2- [Satisfactory      |                           |
|                | Progress]{.underline}** | -   Attainment outcomes   |
|                |                         |     are reported by all   |
|                | -   Attainment outcomes |     institutions for at   |
|                |     are reported for    |     least 80% of their    |
|                |     all schools for at  |     students reaching     |
|                |     least 80% of their  |     threshold levels and  |
|                |     students reaching   |     who are expected to   |
|                |     threshold levels    |     take local            |
|                |     and who are         |     assessments or their  |
|                |     expected to take    |     equivalents (e.g.,    |
|                |     national or state   |     alternative           |
|                |     assessments or      |     assessments)          |
|                |     their equivalents   |     according to state or |
|                |     (e.g., alternative  |     local policies.       |
|                |     assessments)        |                           |
|                |     according to state  | **1- [Does Not Meet       |
|                |     or local policies.  | ]{.underline}**           |
|                |                         |                           |
|                | **1- [Does Not          | -   Attainment outcomes   |
|                | Meet]{.underline}**     |     are not reported by   |
|                |                         |     all schools or some   |
|                | -   Attainment outcomes |     institutions report   |
|                |     are not reported    |     less than 80% of      |
|                |     for all             |     their students        |
|                |     institutions        |     reaching threshold    |
|                |     [or]{.underline}    |     levels and who are    |
|                |     some institutions   |     expected to take      |
|                |     report less than    |     local assessments or  |
|                |     80% of students who |     their equivalents     |
|                |     reach threshold     |     (e.g., alternative    |
|                |     levels and who are  |     assessments)          |
|                |     expected to take    |     according to state or |
|                |     national or state   |     local policies.       |
|                |     assessments or      |                           |
|                |     their equivalents   |                           |
|                |     (e.g., alternative  |                           |
|                |     assessments)        |                           |
|                |     according to state  |                           |
|                |     or local policies.  |                           |
+----------------+-------------------------+---------------------------+

#### 2S1: Secondary Completion

**Measure**

[Numerator:]{.underline} Number of students reaching a state-defined
threshold level of vocational education (concentrators) and who have
attained a high school diploma or its recognized state equivalent and
have left secondary education in the reporting year.

[Denominator]{.underline}: Number of students reaching a state-defined
threshold level of vocational education (concentrators) and who have
left

secondary education in the reporting year.

**Measurement Approaches**

1.  [State / Local Administered Data]{.underline} ---Reporting those
    students receiving diplomas, degrees and other types of credentials
    using information from local administrative data. These data could
    be maintained as individual student records and sent to the state as
    individual student records or aggregated data. The data also could
    be maintained at the local level in aggregated form and sent to the
    state as aggregated data. Although these represent distinct types of
    reporting systems, they do not represent different performance
    measurement approaches.

**Quality Criteria and Scoring Rubric**

The standards levels in the scoring rubrics are the
[proposed]{.underline} levels for use [only]{.underline} in state peer
reviews in the Data Quality Initiative. States will be asked to review
and suggest revisions after using them in the first peer review process.
[They are not federal requirements.]{.underline}

+-----------------------+----------------------------------------------+
| **Quality Criteria**  | **State/Local Administrative Data**          |
+-----------------------+----------------------------------------------+
| A.  **Alignment of    | **3- [Standard]{.underline}**                |
|     Completion        |                                              |
|     Measure to State  | -   Completion measure includes only those   |
|     Graduation        |     students that meet all state             |
|     Requirements**    |     requirements for high school graduation  |
|                       |     to receive a high school diploma or a    |
| Completion measure    |     recognized state equivalent (e.g., GED). |
| reports only those    |                                              |
| students meeting all  | 2.  **NOT APPLICABLE**                       |
| state requirements    |                                              |
| for a high school     | **1- [Does Not Meet ]{.underline}**          |
| diploma or a          |                                              |
| recognized state      | -   Completion measure does                  |
| equivalent.           |     [not]{.underline} include only those     |
|                       |     students that meet all state             |
|                       |     requirements for high school diploma or  |
|                       |     a recognized state equivalent (e.g.,     |
|                       |     GED).                                    |
+-----------------------+----------------------------------------------+
| B. Scope of           | **3- [Standard]{.underline}**                |
| Completion            |                                              |
| Measurement           | -   Completion measure includes all students |
|                       |     reaching the threshold, not just         |
| Completion measure    |     seniors.                                 |
| includes all students |                                              |
| reaching thresholds   | -   Completion measure reports whether or    |
| and whether or not    |     not students receive a high school       |
| they receive diplomas |     diploma [or]{.underline} a recognized    |
| or recognized state   |     state equivalent including the GED.      |
| equivalents.          |                                              |
|                       | **2- [Satisfactory Progress]{.underline}**   |
|                       |                                              |
|                       | -   Completion measure includes at least all |
|                       |     senior students who are eligible to      |
|                       |     graduate.                                |
|                       |                                              |
|                       | -   Completion measure minimally reports     |
|                       |     whether or not students receive a high   |
|                       |     school diploma.                          |
|                       |                                              |
|                       | **1- [Does Not Meet ]{.underline}**          |
|                       |                                              |
|                       | -   Completion measure does                  |
|                       |     [not]{.underline} include all senior     |
|                       |     students who are eligible to graduate.   |
|                       |                                              |
|                       | -   Completion measure does                  |
|                       |     [not]{.underline} include all students   |
|                       |     who receive a high school diploma.       |
+-----------------------+----------------------------------------------+
| C. Timing of          | **3- [Standard]{.underline}**                |
| Completion            |                                              |
| Measurement           | -   Completion is measured at the same time  |
|                       |     after the end of the school year by all  |
| Completion is         |     schools.                                 |
| measured at the same  |                                              |
| time for all students | 2.  **NOT APPLICABLE**                       |
| reaching threshold    |                                              |
| levels in the state.  | **1- [Does Not Meet ]{.underline}**          |
|                       |                                              |
|                       | -   Completion is [not]{.underline} measured |
|                       |     at the same time after the end of the    |
|                       |     school year by all schools.              |
+-----------------------+----------------------------------------------+
| D. Reliability of     | **3- [Standard]{.underline}**                |
| Completion            |                                              |
| Measurement           | -   Completion measure uses a consistent     |
|                       |     statewide definition of completion.      |
| Completion            |                                              |
| measurement is based  | -   Completion measure uses a consistent,    |
| on consistent         |     standardized method to calculate         |
| definitions of state  |     completion rates**.**                    |
| requirements and is   |                                              |
| reported using        | **2- NOT APPLICABLE**                        |
| standardized methods  |                                              |
| for calculating       | **1- [Does Not Meet ]{.underline}**          |
| rates.                |                                              |
|                       | -   Completion measure does                  |
|                       |     [not]{.underline} use a consistent       |
|                       |     statewide definition of completion       |
|                       |     and/or a consistent, standardized method |
|                       |     to calculate completion rates.           |
+-----------------------+----------------------------------------------+
| **E. Student Coverage | **3- [Standard]{.underline}**.               |
| in Completion         |                                              |
| Measurement**         | -   Completion data are reported by all      |
|                       |     schools for at least 90% of students     |
| Performance           |     reaching state thresholds.               |
| measurement reports   |                                              |
| completion data for   | **2- [Satisfactory Progress]{.underline}**   |
| all students reaching |                                              |
| state thresholds.     | -   Completion data are reported by all      |
|                       |     schools for at least 80% of students     |
|                       |     reaching state thresholds.               |
|                       |                                              |
|                       | **1- [Does Not Meet ]{.underline}**          |
|                       |                                              |
|                       | -   Completion data are [not]{.underline}    |
|                       |     reported by all schools                  |
|                       |     [and/or]{.underline} some or all schools |
|                       |     report completion data for less than 80% |
|                       |     of students reaching state thresholds.   |
+-----------------------+----------------------------------------------+

#### 2S2: Proficiency Credential with Secondary Diploma

**Measure**

[Numerator:]{.underline} Number of students reaching a state-defined
threshold level of vocational education and who have attained a
proficiency credential in conjunction with a secondary school diploma or
its state recognized equivalent and who have left secondary education in
the reporting year.

[Denominator:]{.underline} Number of students reaching a state-defined
threshold level of vocational education and who have received a
secondary school diploma or its state equivalent and who have left
secondary education during the reporting period.

**Measurement Approaches (Skill Credential Only---See 2S1 for Diploma)**

1.  [National/State Standards and Assessment Systems]{.underline} --
    Performance benchmarks on national or state standards and assessment
    systems using national or state-developed and standardized
    assessment instruments and assessment procedures (includes
    licensing/certification examinations).

2.  [National/State Standards and Local Assessment Systems]{.underline}
    -- Performance benchmarks on local assessment systems that have been
    approved and recognized by national organizations and/or states
    based on national or state standards.

3.  [Local Standards and Assessment Systems]{.underline} -- Performance
    benchmarks on state-approved local assessment systems based on
    state-approved or recognized local standards.

4.  [Vocational/Technical Course Completion]{.underline} -- Performance
    benchmarks (e.g., grades, competency completion) in vocational
    education courses or programs.

5.  [Vocational/Technical Grade Point Average]{.underline} --
    State-defined grade point average for designated vocational courses
    including integrated academic/vocational courses.

6.  [Program Completion]{.underline} - Completion of
    vocational/technical education program when program completion
    represents attainment of career and technical skill standards.

**Quality Criteria and Scoring Rubric**

The standards levels in the scoring rubrics are the
[proposed]{.underline} levels for use [only]{.underline} in state peer
reviews in the Data Quality Initiative. States will be asked to review
and suggest revisions after using them in the first peer review process.
[They are not federal requirements.]{.underline}

+-----------------+-------------------------+--------------------------+
| **Quality       | **National and/or       | **Approaches with Local  |
| Criteria**      | State**                 | Assessment Systems**     |
|                 |                         |                          |
| **(apply only   | **Assessment Systems**  | **Local Standards and    |
| to skill        |                         | Assessment Systems**     |
| credential)**   |                         |                          |
|                 |                         | **Vocational/Technical   |
|                 |                         | Course Completion**      |
|                 |                         |                          |
|                 |                         | **Vocational Grade Point |
|                 |                         | Average**                |
|                 |                         |                          |
|                 |                         | **Program Completion**   |
+-----------------+-------------------------+--------------------------+
| A.  **Alignment | **3-                    | **3-                     |
|     to Industry | [                       | [Standard]{.underline}** |
|     Standards** | Standard]{.underline}** |                          |
|                 |                         | -   All assessment       |
| Attainment      | -   All assessment      |     systems used by all  |
| measures are    |     systems are         |     schools are directly |
| aligned         |     directly aligned to |     aligned to           |
| (validity) to   |     state-established,  |     state-established,   |
| sta             |     industry-validated  |     industry-validated   |
| te-established, |     content             |     content              |
| ind             |     [and]{.underline}   |     [and]{.underline}    |
| ustry-validated |     performance         |     performance          |
| skill           |     standards.          |     standards.           |
| s               |                         |                          |
| tandards---both | -   All assessment      | -   All assessment       |
| content and     |     systems are         |     systems in all       |
| performance     |     regularly evaluated |     schools are          |
| standards       |     on their alignment  |     regularly evaluated  |
|                 |     to these standards  |     on their alignment   |
|                 |     in cooperation with |     to these standards   |
|                 |     industry and        |     in cooperation with  |
|                 |     postsecondary       |     industry and         |
|                 |     partners.           |     postsecondary        |
|                 |                         |     partners.            |
|                 | **2- [Satisfactory      |                          |
|                 | Progress]{.underline}** | **2- [Satisfactory       |
|                 |                         | Progress]{.underline}**  |
|                 | -   Statewide policies  |                          |
|                 |     and systems have    | -   Statewide policies   |
|                 |     been established to |     and systems have     |
|                 |     ensure that all     |     been established to  |
|                 |     assessment systems  |     ensure that all      |
|                 |     are directly        |     assessment systems   |
|                 |     aligned to          |     used by all schools  |
|                 |     state-established,  |     are directly aligned |
|                 |     industry-validated  |     to                   |
|                 |     content standards.  |     state-established,   |
|                 |                         |     industry-validated   |
|                 | -   These statewide     |     content standards.   |
|                 |     policies and        |                          |
|                 |     systems have been   | -   These statewide      |
|                 |     established in      |     policies and systems |
|                 |     cooperation with    |     have been            |
|                 |     industry and        |     established in       |
|                 |     postsecondary       |     cooperation with     |
|                 |     partners.           |     industry and         |
|                 |                         |     postsecondary        |
|                 | **1- [Does Not          |     partners.            |
|                 | Meet]{.underline}**     |                          |
|                 |                         | **1- [Does Not Meet      |
|                 | -   Statewide policies  | ]{.underline}**          |
|                 |     and systems have    |                          |
|                 |     [not]{.underline}   | -   Statewide policies   |
|                 |     been established to |     and systems have     |
|                 |     ensure that all     |     [not]{.underline}    |
|                 |     assessment systems  |     been established to  |
|                 |     are directly        |     ensure that all      |
|                 |     aligned to          |     assessment systems   |
|                 |     state-established,  |     used by all schools  |
|                 |     industry-validated  |     are directly aligned |
|                 |     content standards   |     to                   |
|                 |     in cooperation with |     state-established,   |
|                 |     industry and        |     industry-validated   |
|                 |     postsecondary       |     content standards in |
|                 |     partners.           |     cooperation with     |
|                 |                         |     industry and         |
|                 | .                       |     postsecondary        |
|                 |                         |     partners.            |
+-----------------+-------------------------+--------------------------+
| **B. Scope of   | **3-                    | **3-                     |
| Attainment**    | [                       | [Standard]{.underline}** |
|                 | Standard]{.underline}** |                          |
| **Measurement** |                         | -   All assessment       |
|                 | -   All assessment      |     systems used by all  |
| Attainment      |     systems provide a   |     schools provide a    |
| measures        |     representative      |     representative       |
| provide a       |     coverage of         |     coverage of          |
| representative  |     state-established,  |     state-established,   |
| coverage        |     industry-validated  |     industry-validated   |
| (validity) of   |     content             |     content              |
| sta             |     [and]{.underline}   |     [and]{.underline}    |
| te-established, |     performance         |     performance          |
| ind             |     standards.          |     standards.           |
| ustry-validated |                         |                          |
| skill standards | -   All assessment      | -   All assessment       |
|                 |     systems are         |     systems in all       |
|                 |     regularly evaluated |     schools are          |
|                 |     on their coverage   |     regularly evaluated  |
|                 |     of these standards  |     on their coverage of |
|                 |     in cooperation with |     these standards in   |
|                 |     industry and        |     cooperation with     |
|                 |     postsecondary       |     industry and         |
|                 |     partners.           |     postsecondary        |
|                 |                         |     partners.            |
|                 | **2- [Satisfactory      |                          |
|                 | Progress]{.underline}** | **2- [Satisfactory       |
|                 |                         | Progress ]{.underline}** |
|                 | -   Statewide policies  |                          |
|                 |     and systems have    | -   Statewide policies   |
|                 |     been established to |     and systems have     |
|                 |     ensure that all     |     been established to  |
|                 |     assessment systems  |     ensure that all      |
|                 |     provide a           |     assessment systems   |
|                 |     representative      |     used by all schools  |
|                 |     coverage of         |     provide a            |
|                 |     state-established,  |     representative       |
|                 |     industry-validated  |     coverage of          |
|                 |     content standards.  |     state-established,   |
|                 |                         |     industry-validated   |
|                 | -   These statewide     |     content standards.   |
|                 |     policies and        |                          |
|                 |     systems have been   | -   These statewide      |
|                 |     established in      |     policies and systems |
|                 |     cooperation with    |     have been            |
|                 |     industry and        |     established in       |
|                 |     postsecondary       |     cooperation with     |
|                 |     partners.           |     industry and         |
|                 |                         |     postsecondary        |
|                 | **1- [Does Not          |     partners.            |
|                 | Meet]{.underline}**     |                          |
|                 |                         | **1- [Does Not           |
|                 | -   Statewide policies  | Meet]{.underline}**      |
|                 |     and systems have    |                          |
|                 |     [not]{.underline}   | -   Statewide policies   |
|                 |     been established to |     and systems have     |
|                 |     ensure that all     |     [not]{.underline}    |
|                 |     assessment systems  |     been established to  |
|                 |     provide a           |     ensure that all      |
|                 |     representative      |     assessment systems   |
|                 |     coverage of         |     used by all schools  |
|                 |     state-established,  |     provide a            |
|                 |     industry-validated  |     representative       |
|                 |     content standards   |     coverage of          |
|                 |     in cooperation with |     state-established,   |
|                 |     industry and        |     industry-validated   |
|                 |     postsecondary       |     content standards in |
|                 |     partners            |     cooperation with     |
|                 |                         |     industry and         |
|                 | .                       |     postsecondary        |
|                 |                         |     partners.            |
+-----------------+-------------------------+--------------------------+
| C. Timing of    | **3-                    | **3-                     |
| Attainment      | [                       | [Standard]{.underline}** |
| Measurement     | Standard]{.underline}** |                          |
|                 |                         | -   Attainment is        |
| Attainment is   | -   Attainment is       |     measured in all      |
| measured        |     measured            |     schools              |
| concurrent with |     [after]{.underline} |     [after]{.underline}  |
| or after        |     concentrated        |     concentrated         |
| concentrated    |     participation.      |     participation.       |
| participation   |                         |                          |
| in vocational   | **2- [Satisfactory      | **2- [Satisfactory       |
| education.      | Progress]{.underline}** | Progress ]{.underline}** |
|                 |                         |                          |
|                 | -   Attainment is       | -   Attainment is        |
|                 |     measured            |     measured in all      |
|                 |     [concurrent with or |     schools [concurrent  |
|                 |     after]{.underline}  |     with or              |
|                 |     concentrated        |     after]{.underline}   |
|                 |     participation.      |     concentrated         |
|                 |                         |     participation.       |
|                 | **1- [Does Not          |                          |
|                 | Meet]{.underline}**     | **1- [Does Not Meet      |
|                 |                         | ]{.underline}**          |
|                 | -   Attainment is       |                          |
|                 |     measured in         | -   Attainment is        |
|                 |     some/all systems    |     measured in some/all |
|                 |                         |     systems or schools   |
|                 |    [before]{.underline} |     [before]{.underline} |
|                 |     concentrated        |     concentrated         |
|                 |     participation.      |     participation.       |
+-----------------+-------------------------+--------------------------+
| D. Reliability  | **3-                    | **3-                     |
| of Assessment   | [                       | [Standard]{.underline}** |
|                 | Standard]{.underline}** |                          |
| **Instruments** |                         | -   Attainment is        |
|                 | -   Attainment is       |     measured by all      |
| Attainment is   |     measured by all     |     assessment systems   |
| measured using  |     assessment systems  |     in all schools using |
| reliable        |     using assessment    |     assessment           |
| assessment      |     instruments that    |     instruments that     |
| instruments.    |     meet national or    |     meet national or     |
|                 |     state reliability   |     state reliability    |
|                 |     standards.          |     standards.           |
|                 |                         |                          |
|                 | -   All assessment      | -   All assessment       |
|                 |     systems and         |     systems and          |
|                 |     instruments are     |     instruments in all   |
|                 |     regularly evaluated |     schools are          |
|                 |     based on these      |     regularly evaluated  |
|                 |     standards.          |     based on these       |
|                 |                         |     standards.           |
|                 | **2- [Satisfactory      |                          |
|                 | Progress                | **2- [Satisfactory       |
|                 | ]{.underline}**         | Progress ]{.underline}** |
|                 |                         |                          |
|                 | -   Statewide policies  | -   Statewide policies   |
|                 |     and systems have    |     and systems have     |
|                 |     been established to |     been established to  |
|                 |     ensure that         |     ensure that          |
|                 |     attainment is       |     attainment is        |
|                 |     measured by all     |     measured by all      |
|                 |     assessment systems  |     assessment systems   |
|                 |     using assessment    |     in all schools using |
|                 |     instruments that    |     assessment           |
|                 |     meet national or    |     instruments that     |
|                 |     state reliability   |     meet national or     |
|                 |     standards.          |     state reliability    |
|                 |                         |     standards.           |
|                 | -   These statewide     |                          |
|                 |     policies and        | -   These statewide      |
|                 |     systems have been   |     policies and systems |
|                 |     established in      |     have been            |
|                 |     cooperation with    |     established in       |
|                 |     industry and        |     cooperation with     |
|                 |     postsecondary       |     industry and         |
|                 |     partners            |     postsecondary        |
|                 |                         |     partners.            |
|                 | **1- [Does Not          |                          |
|                 | Meet]{.underline}**     | **1- [Does Not           |
|                 |                         | Meet]{.underline}**      |
|                 | -   Statewide policies  |                          |
|                 |     have                | -   Statewide policies   |
|                 |     [not]{.underline}   |     and systems have     |
|                 |     been established to |     [not]{.underline}    |
|                 |     ensure that         |     been established to  |
|                 |     attainment is       |     ensure that          |
|                 |     measured by all     |     attainment is        |
|                 |     assessment systems  |     measured by all      |
|                 |     using assessment    |     assessment systems   |
|                 |     instruments that    |     in all schools using |
|                 |     meet national or    |     assessment           |
|                 |     state reliability   |     instruments that     |
|                 |     standards.          |     meet national or     |
|                 |                         |     state reliability    |
|                 |                         |     standards.           |
+-----------------+-------------------------+--------------------------+
| F.              | **3-                    | **3-                     |
|   **Reliability | [                       | [Standard]{.underline}** |
|     of          | Standard]{.underline}** |                          |
|     Assessment  |                         | -   Attainment is        |
|     A           | -   Attainment is       |     measured in all      |
| dministration** |     measured by all     |     schools using        |
|                 |     assessment systems  |     assessment           |
| Attainment is   |     using assessment    |     instruments that are |
| measured using  |     instruments that    |     administered         |
| assessment      |     are administered    |     consistently         |
| instruments     |     consistently        |     according to         |
| that are        |     according to        |     national or state    |
| administered    |     national or state   |     reliability          |
| consistently    |     reliability         |     standards (e.g.,     |
| within          |     standards (e.g.,    |     inter-rater          |
| assessment      |     inter-rater         |     reliability for      |
| systems.        |     reliability for     |     performance          |
|                 |     performance         |     assessment).         |
|                 |     assessment).        |                          |
|                 |                         | -   The administration   |
|                 | -   The statewide       |     of all assessments   |
|                 |     administration of   |     by all schools is    |
|                 |     all assessments is  |     regularly evaluated  |
|                 |     regularly evaluated |     based on these       |
|                 |     based on these      |     standards.           |
|                 |     standards.          |                          |
|                 |                         | **2- [Satisfactory       |
|                 | **2- [Satisfactory      | Progress ]{.underline}** |
|                 | Progress]{.underline}** |                          |
|                 |                         | -   Statewide policies   |
|                 | -   Statewide policies  |     and systems have     |
|                 |     and systems have    |     been established to  |
|                 |     been established to |     ensure that          |
|                 |     ensure that         |     attainment is        |
|                 |     attainment is       |     measured in all      |
|                 |     measured by all     |     schools using        |
|                 |     assessment systems  |     assessment           |
|                 |     using assessment    |     instruments that are |
|                 |     instruments that    |     administered         |
|                 |     are administered    |     consistently         |
|                 |     consistently to     |     according to         |
|                 |     national/state      |     national or state    |
|                 |     reliability         |     reliability          |
|                 |     standards (e.g.,    |     standards (e.g.,     |
|                 |     inter-rater         |     inter-rater          |
|                 |     reliability for     |     reliability for      |
|                 |     performance         |     performance          |
|                 |     assessment).        |     assessment).         |
|                 |                         |                          |
|                 | -   These statewide     | -   These statewide      |
|                 |     policies and        |     policies and systems |
|                 |     systems have been   |     have been            |
|                 |     established in      |     established in       |
|                 |     cooperation with    |     cooperation with     |
|                 |     industry and        |     industry and         |
|                 |     postsecondary       |     postsecondary        |
|                 |     partners            |     partners             |
|                 |                         |                          |
|                 | **1- [Does Not          | **1- [Does Not Meet      |
|                 | Meet]{.underline}**     | ]{.underline}**          |
|                 |                         |                          |
|                 | -   Statewide policies  | -   Statewide policies   |
|                 |     and systems have    |     and systems have     |
|                 |     [not]{.underline}   |     [not]{.underline}    |
|                 |     been established to |     been established to  |
|                 |     ensure that         |     ensure that          |
|                 |     attainment is       |     attainment is        |
|                 |     measured by all     |     measured in all      |
|                 |     assessment systems  |     schools using        |
|                 |     using assessment    |     assessment           |
|                 |     instruments that    |     instruments that are |
|                 |     are administered    |     administered         |
|                 |     consistently        |     consistently         |
|                 |     according to        |     according to         |
|                 |     national or state   |     national or state    |
|                 |     reliability         |     reliability          |
|                 |     standards.          |     standards.           |
+-----------------+-------------------------+--------------------------+
| F. Student      | **3-                    | **3-                     |
| Coverage in     | [                       | [Standard]{.underline}** |
| Attainment      | Standard]{.underline}** |                          |
| Measurement     |                         | -   Attainment outcomes  |
|                 | -   Attainment outcomes |     are reported by all  |
| Performance     |     are reported by all |     schools for at least |
| measurement     |     schools for at      |     90% of their         |
| reports         |     least 90% of their  |     students reaching    |
| attainment data |     students reaching   |     threshold levels and |
| for all         |     threshold levels    |     who are expected to  |
| students        |     who are expected to |     take local           |
| reaching        |     take national or    |     assessments or their |
| state-defined   |     state assessments   |     equivalents (e.g.,   |
| threshold       |     or their            |     alternative          |
| levels in the   |     equivalents (e.g.,  |     assessments)         |
| state.          |     alternative         |     according to state   |
|                 |     assessments)        |     or local policies.   |
| ###             |     according to state  |                          |
|                 |     or local policies.  | **2- [Satisfactory       |
|                 |                         | Progress ]{.underline}** |
|                 | **2- [Satisfactory      |                          |
|                 | Progress                | -   Attainment outcomes  |
|                 | ]{.underline}**         |     are reported by all  |
|                 |                         |     schools for at least |
|                 | -   Attainment outcomes |     80% of their         |
|                 |     are reported for    |     students reaching    |
|                 |     all schools for at  |     threshold levels and |
|                 |     least 80% of all of |     who are expected to  |
|                 |     their students      |     take local           |
|                 |     reaching threshold  |     assessments or their |
|                 |     levels and who are  |     equivalents (e.g.,   |
|                 |     expected to take    |     alternative          |
|                 |     national or state   |     assessments)         |
|                 |     assessments or      |     according to state   |
|                 |     their equivalents   |     or local policies.   |
|                 |     (e.g., alternative  |                          |
|                 |     assessments)        | **1- [Does Not           |
|                 |     according to state  | Meet]{.underline}**      |
|                 |     or local policies.  |                          |
|                 |                         | -   Attainment outcomes  |
|                 | **1- [Does Not          |     are not reported by  |
|                 | Meet]{.underline}**.    |     all schools or some  |
|                 |                         |     schools report less  |
|                 | -   Attainment outcomes |     than 80% of their    |
|                 |     are not reported    |     students reaching    |
|                 |     for all schools     |     threshold levels and |
|                 |     [or]{.underline}    |     who are expected to  |
|                 |     some schools report |     take local           |
|                 |     less than 80% of    |     assessments or their |
|                 |     their students who  |     equivalents (e.g.,   |
|                 |     reach threshold     |     alternative          |
|                 |     levels and who are  |     assessments)         |
|                 |     expected to take    |     according to state   |
|                 |     national or state   |     or local policies.   |
|                 |     assessments or      |                          |
|                 |     their equivalents   |                          |
|                 |     (e.g., alternative  |                          |
|                 |     assessments)        |                          |
|                 |     according to state  |                          |
|                 |     or local policies.  |                          |
+-----------------+-------------------------+--------------------------+

#### 2P1: Postsecondary Degree or Credential

**Measure**

[Numerator:]{.underline} Number of students reaching a state-defined
threshold level of vocational education (concentrators) and who received
or were eligible to receive a postsecondary degree, certificate, or
credential and who stopped program participation in the reporting year.

[Denominator]{.underline}: Number of students reaching a state-defined
threshold level of vocational education (concentrators) and who stopped
program participation in the reporting year.

**Measurement Approach**

1.  [State/Local Administrative Data]{.underline} --- Reporting those
    students receiving degrees and other types of credentials using
    information from local administrative data. These data could be
    maintained as individual student records and sent to the state as
    individual student records or aggregated data. The data also could
    be maintained at the local level in aggregated form and sent to the
    state as aggregated data. Although these represent distinct types of
    reporting systems, they do not represent different performance
    measurement approaches.

**Quality Criteria and Scoring Rubric**

The standards levels in the scoring rubrics are the
[proposed]{.underline} levels for use [only]{.underline} in state peer
reviews in the Data Quality Initiative. States will be asked to review
and suggest revisions after using them in the first peer review process.
[They are not federal requirements.]{.underline}

+-----------------------+----------------------------------------------+
| **Quality Criteria**  | **State/Local Administrative Data**          |
+-----------------------+----------------------------------------------+
| A.  **Alignment of    | **3- [Standard]{.underline}**                |
|     Completion        |                                              |
|     Measurement to    | -   Completion measure includes only those   |
|     Standards**       |     students that meet program-defined       |
|                       |     standards for receiving degrees and/or   |
| Completion measure is |     certificates.                            |
| aligned to            |                                              |
| program-defined       | **2- NOT APPLICABLE**                        |
| academic and skill    |                                              |
| standards and related | 1.  **[Does Not Meet]{.underline}**          |
| program standards for |                                              |
| receiving degrees     | -   Completion measure does                  |
| and/or certificates.  |     [not]{.underline} include only those     |
|                       |     students that meet program-defined       |
|                       |     standards for receiving degrees and/or   |
|                       |     certificates.                            |
+-----------------------+----------------------------------------------+
| B. Scope of           | **3- [Standard]{.underline}**                |
| Completion            |                                              |
| Measurement           | -   Completion measure includes all students |
|                       |     reaching thresholds and whether or not   |
| Completion measure    |     they receive diplomas or certificates.   |
| includes all students |                                              |
| reaching thresholds   | **2- NOT APPLICABLE**                        |
| and whether or not    |                                              |
| they receive diplomas | **1- [Does Not Meet]{.underline}**           |
| or certificates.      |                                              |
|                       | -   Completion measure doe [not]{.underline} |
|                       |     include all students reaching thresholds |
|                       |     and whether or not they receive diplomas |
|                       |     or certificates.                         |
+-----------------------+----------------------------------------------+
| C. Timing of          | **3- [Standard]{.underline}**                |
| Completion            |                                              |
| Measurement           | -   Completion is measured at the same time  |
|                       |     after the end of the school year by all  |
| Completion is         |     institutions.                            |
| measured at the same  |                                              |
| time for all students | **2- NOT APPLICABLE**                        |
| reaching threshold    |                                              |
| levels in the state.  | **1- [Does Not Meet ]{.underline}**          |
|                       |                                              |
|                       | -   Completion is [not]{.underline} measured |
|                       |     at the same time after the end of the    |
|                       |     school year by all institutions.         |
+-----------------------+----------------------------------------------+
| D. Reliability of     | **3- [Standard]{.underline}**                |
| Completion            |                                              |
| Measurement           | -   Completion measure uses a consistent     |
|                       |     statewide definition of completion.      |
| Completion            |                                              |
| measurement is based  | -   Completion measure uses a consistent,    |
| on consistent         |     standardized method to calculate         |
| definitions of state  |     completion rates**.**                    |
| requirements and is   |                                              |
| reported using        | **2- NOT APPLICABLE**                        |
| standardized methods  |                                              |
| for calculating       | **1- [Does Not Meet]{.underline}**           |
| rates.                |                                              |
|                       | -   Completion measure does                  |
|                       |     [not]{.underline} use a consistent       |
|                       |     statewide definition of completion       |
|                       |     and/or a consistent, standardized method |
|                       |     to calculate completion rates.           |
+-----------------------+----------------------------------------------+
| E. Student Coverage   | **3- [Standard]{.underline}**                |
| in Completion         |                                              |
| Measurement           | -   Completion data are reported by all      |
|                       |     institutions for at least 90% of         |
| Performance           |     students reaching state thresholds.      |
| measurement reports   |                                              |
| completion data for   | **2- [Satisfactory Progress]{.underline}**   |
| all students reaching |                                              |
| state thresholds.     | -   Completion data are reported by all      |
|                       |     institutions for at least 80% of         |
|                       |     students reaching state thresholds.      |
|                       |                                              |
|                       | **1- [Does Not Meet ]{.underline}**          |
|                       |                                              |
|                       | -   Completion data are [not]{.underline}    |
|                       |     reported by all institutions             |
|                       |     [and/or]{.underline} some or all schools |
|                       |     report completion data for less than 80% |
|                       |     of students reaching state thresholds.   |
+-----------------------+----------------------------------------------+

#### 3S1: Secondary Placement

**Measure**

[Numerator]{.underline}: Number of completers who graduate and are
placed in postsecondary education or advanced training, employment,
and/or military within a state-defined time period (after leaving
secondary education) that ends within the reporting year.

[Denominator]{.underline}: Number of completers who graduate and are
included in placement measurement ( i.e., students surveyed or students
with valid identifiers) within a state-defined time period (after
leaving secondary education) that ends within the reporting year.

**Measurement Approach**

1.  [State-Developed, School-Administered Surveys/Placement
    Records]{.underline}--- Mail and/or telephone surveys of students
    using state-developed surveys or placement forms administered by
    schools. Schools administer and conduct the surveys and maintain
    placement records under state guidelines.

2.  [State-Developed and Administered Surveys]{.underline}---Mail and/or
    telephone surveys of students using state-developed and administered
    surveys.

3.  [Administrative Record Exchanges/Matching of Administrative
    Records]{.underline}---Matching of student records (e.g.,
    postsecondary education records), UI wage records, U.S. Department
    of Defense records---based on student identifiers.

**Quality Criteria and Scoring Rubrics**

The standards levels in the scoring rubrics are the
[proposed]{.underline} levels for use [only]{.underline} in state peer
reviews in the Data Quality Initiative. States will be asked to review
and suggest revisions after using them in the first peer review process.
[They are not federal requirements.]{.underline}

+----------------+--------------------------+--------------------------+
| **Quality      | **Administrative Record  | **Surveys**              |
| Criteria**     | Exchange**               |                          |
|                |                          | **State Developed and    |
|                |                          | Administered Surveys**   |
|                |                          |                          |
|                |                          | **State Developed and    |
|                |                          | Locally Administered     |
|                |                          | Surveys/Placement        |
|                |                          | Forms**                  |
+----------------+--------------------------+--------------------------+
| A.  Alignment  | **3-                     | **3-                     |
|     to         | [Standard]{.underline}** | [Standard]{.underline}** |
|                |                          |                          |
|    Definitions | -   A standardized       | -   A standardized       |
|     of Three   |     definition for each  |     definition for each  |
|     Types of   |     of the three types   |     of the three types   |
|     Placement  |     of placement         |     of placement         |
|                |     --postsecondary      |     --postsecondary      |
| Surveys,       |     education,           |     education,           |
| placement      |     employment, and      |     employment, and      |
| forms, or      |     military---are       |     military---are       |
| administrative |     adopted and          |     adopted and          |
| record         |     implemented          |     implemented          |
| exchanges use  |     consistently         |     consistently         |
| consistent     |     throughout the       |     throughout the       |
| definitions of |     state.               |     state.               |
| the three      |                          |                          |
| types of       | **2- [Satisfactory       | **2- [Satisfactory       |
| placement--    | Progress]{.underline}**  | Progress ]{.underline}** |
| -postsecondary |                          |                          |
| education,     | -   A standardized       | -   A standardized       |
| employment,    |     definition for at    |     definition for at    |
| and military.  |     least two types of   |     least two types of   |
|                |     pla                  |     pla                  |
|                | cement\-\--postsecondary | cement\-\--postsecondary |
|                |     education and        |     education and        |
|                |     employment\-\--are   |     employment\-\--are   |
|                |     adopted and          |     adopted and          |
|                |     implemented          |     implemented          |
|                |     consistently         |     consistently         |
|                |     throughout the       |     throughout the       |
|                |     state.               |     state.               |
|                |                          |                          |
|                | **1- [Does Not           | **1- [Does Not Meet      |
|                | Meet]{.underline}**      | ]{.underline}**          |
|                |                          |                          |
|                | -   No standardized      | -   No standardized      |
|                |     definitions or       |     definitions or       |
|                |     standardized         |     standardized         |
|                |     definitions for only |     definitions for only |
|                |     employment or only   |     employment or only   |
|                |     postsecondary        |     postsecondary        |
|                |     education are used   |     education are used   |
|                |     consistently across  |     consistently across  |
|                |     the state.           |     the state.           |
+----------------+--------------------------+--------------------------+
| B.  **Timing   | **3-                     | **3-                     |
|     of         | [Standard]{.underline}** | [Standard]{.underline}** |
|     Placement  |                          |                          |
|                | -   Placement is defined | -   Placement is defined |
|  Measurement** |     as placement in      |     as placement in      |
|                |     postsecondary        |     postsecondary        |
| Placement is   |     education,           |     education,           |
| measured       |     employment, and/or   |     employment, and/or   |
| during the     |     military at any time |     military at any time |
| same           |     during a             |     during a             |
| measurement    |     standardized time    |     standardized time    |
| intervals for  |     period after         |     period after         |
| all completers |     students com-plete   |     students com-plete   |
| (e.g., 3       |     secondary education  |     secondary education  |
| months, 6      |     (e.g., 6 months      |     (e.g., 6 months      |
| months) after  |     after students       |     after stu-dents      |
| completers     |     complete secondary   |     complete secondary   |
| leave          |     education) even if   |     education) even if   |
| secondary      |     students complete at |     students complete at |
| education.     |     different times      |     different times      |
|                |     during the school    |     during the school    |
|                |     year.                |     year.                |
|                |                          |                          |
|                | -   [Note]{.underline}:  | -   [Note]{.underline}:  |
|                |     This would result in |     This would result in |
|                |     the state having one |     the state having one |
|                |     or more periods for  |     or more periods for  |
|                |     data collection if   |     data collection if   |
|                |     students complete    |     students complete    |
|                |     during different     |     during different     |
|                |     months during the    |     months during the    |
|                |     year.                |     year.                |
|                |                          |                          |
|                | **2- [Satisfactory       | **2- [Satisfactory       |
|                | Progress]{.underline}**  | Progress]{.underline}**  |
|                |                          |                          |
|                | -   Placement is defined | -   Placement is defined |
|                |     as placement in      |     as placement in      |
|                |     postsecondary        |     postsecondary        |
|                |     education,           |     education,           |
|                |     employment, and/or   |     employment, and/or   |
|                |     military at any time |     military at any time |
|                |     during a             |     during a             |
|                |     standardized time    |     standardized time    |
|                |     period after         |     period after         |
|                |     students complete    |     students complete    |
|                |     secondary education. |     secondary education. |
|                |                          |                          |
|                | -   All students are     | -   All students are     |
|                |     measured using a     |     measured using a     |
|                |     common measure-ment  |     common measure-ment  |
|                |     period regardless of |     period regardless of |
|                |     when they completed  |     when they completed  |
|                |     secondary education  |     secondary education  |
|                |     during the school    |     during the school    |
|                |     year. (e.g. July 1   |     year. (e.g. July 1   |
|                |     to December 31       |     to December 31       |
|                |     following the school |     following the school |
|                |     year).               |     year).               |
|                |                          |                          |
|                | **1- [Does Not Meet      | **1- [Does Not Meet      |
|                | ]{.underline}**          | ]{.underline}**          |
|                |                          |                          |
|                | -   Placement is defined | -   Placement is defined |
|                |     as placement in      |     as placement in      |
|                |     postsecondary        |     postsecondary        |
|                |     education,           |     education,           |
|                |     employment, and/or   |     employment, and/or   |
|                |     military at any time |     military at any time |
|                |     during a designated  |     during a designated  |
|                |     time period after    |     time period after    |
|                |     students complete    |     students complete    |
|                |     secondary education. |     secondary education. |
|                |                          |                          |
|                | ```{=html}               | ```{=html}               |
|                | <!-- -->                 | <!-- -->                 |
|                | ```                      | ```                      |
|                | -   There is no          | -   There is no          |
|                |     standardized or      |     standardized or      |
|                |     common time period   |     common time period   |
|                |     used for all         |     used for all         |
|                |     completers who       |     completers who       |
|                |     graduate.            |     graduate.            |
+----------------+--------------------------+--------------------------+
| C.             | **3-                     | **3-                     |
|  **Reliability | [Standard]{.underline}** | [Standard]{.underline}** |
|     of         |                          |                          |
|     Placement  | -   Standardized written | -   Standardized         |
|                |     procedures are       |     statewide survey     |
|  Measurement** |     followed to ensure   |     instruments and      |
|                |     the reliability of   |     survey procedures    |
| Placement      |     administrative       |     are followed to      |
| measurement    |     records matches by   |     ensure the           |
| reports        |     all participating    |     reliability of state |
| placement data |     state agencies       |     or locally           |
| based on       |                          |     administered         |
| consistent     | -   Regular independent  |     surveys.             |
| surveys,       |     audits are conducted |                          |
| forms, and     |     to verify and        | -   Regular independent  |
| administrative |     improve procedures.  |     audits are conducted |
| records and    |                          |     to verify and        |
| measurement    | **2- [Satisfactory       |     improve instruments  |
| procedures.    | Progress]{.underline}**  |     and survey           |
|                |                          |     procedures.          |
|                | -   Standardized written |                          |
|                |     procedures are       | **2- [Satisfactory       |
|                |     followed to ensure   | Progress]{.underline}**  |
|                |     the reliability of   |                          |
|                |     administrative       | -   Standardized survey  |
|                |     record matches by    |     instruments and      |
|                |     all participating    |     survey procedures    |
|                |     state agencies.      |     are followed to      |
|                |                          |     ensure the           |
|                | **1- [Does Not Meet      |     reliability of state |
|                | ]{.underline}**          |     or locally           |
|                |                          |     administered         |
|                | -   No standardized,     |     surveys.             |
|                |     written procedures   |                          |
|                |     are used to ensure   | **1- [Does Not Meet      |
|                |     the reliability of   | ]{.underline}**          |
|                |     administrative       |                          |
|                |     record matches.      | -   No standardized      |
|                |                          |     statewide survey     |
|                |                          |     instruments and      |
|                |                          |     survey procedures    |
|                |                          |     are followed to      |
|                |                          |     ensure the           |
|                |                          |     reliability of state |
|                |                          |     or locally           |
|                |                          |     administered         |
|                |                          |     surveys.             |
+----------------+--------------------------+--------------------------+
| D.  **Student  | **3-                     | **3-                     |
|     Coverage   | [Standard]{.underline}** | [Standard]{.underline}** |
|     in         |                          |                          |
|     Placement  | -   States attempt to    | -   States attempt to    |
|                |     obtain valid         |     obtain contact       |
|  Measurement** |     personal identifiers |     information and      |
|                |     and conduct          |     survey all           |
| Placement      |     administrative       |     completers who       |
| measurement    |     record exchanges     |     graduate from all    |
| attempts to    |     from all schools and |     schools.             |
| gather data    |     for all completers   |                          |
| for all        |     who graduate.        | -   States are           |
| program        |                          |     successful in        |
| completers who | -   States are           |     obtaining and using  |
| graduate for   |     successful in        |     verified student     |
| all programs   |     obtaining and using  |     contact information  |
| within the     |     valid personal       |     for mail and/or      |
| state.         |     identifiers for      |     phone surveys for at |
|                |     administrative       |     least 80% of program |
|                |     record matching for  |     completers who       |
|                |     at least 70% of      |     graduate.            |
|                |     program completers   |                          |
|                |     who graduate.        | **2- [Satisfactory       |
|                |                          | Progress]{.underline}**  |
|                | **2- [Satisfactory       |                          |
|                | Progress]{.underline}**  | -   States attempt to    |
|                |                          |     obtain contact       |
|                | -   States attempt to    |     information and      |
|                |     obtain valid         |     survey all           |
|                |     personal identifiers |     completers who       |
|                |     and conduct          |     graduate from all    |
|                |     administrative       |     schools.             |
|                |     record exchanges     |                          |
|                |     from all schools and | -   States are           |
|                |     for all completers   |     successful in        |
|                |     who graduate.        |     obtaining and using  |
|                |                          |     verified student     |
|                | -   States obtain and    |     contact information  |
|                |     use valid personal   |     for mail and/or      |
|                |     identifiers for      |     phone surveys from   |
|                |     administrative       |     all schools with at  |
|                |     record matching from |     least 70% of program |
|                |     all schools with all |     completers who       |
|                |     schools providing    |     graduate. States     |
|                |     identifiers for at   |     with schools between |
|                |     least 60% of their   |     60 and 70% can meet  |
|                |     program completers   |     the standard if they |
|                |     who graduate.        |     can provide evidence |
|                |     [Note]{.underline}:  |     that falling below   |
|                |     States with schools  |     the standard does    |
|                |     with between 50 and  |     not lead to          |
|                |     60% can meet the     |     significant          |
|                |     standard if they can |     under-representation |
|                |     provide evidence     |     of student           |
|                |     that falling below   |     populations within   |
|                |     the standard does    |     the school and/or    |
|                |     not lead to          |     within the state.    |
|                |     significant          |                          |
|                |     under-representation | **1- [Does Not Meet      |
|                |     of student           | ]{.underline}**          |
|                |     populations within   |                          |
|                |     the school and/or    | -   States do            |
|                |     within the state.    |     [not]{.underline}    |
|                |                          |     attempt or obtain    |
|                | **1- [Does Not Meet      |     contact information  |
|                | ]{.underline}**          |     and survey all       |
|                |                          |     completers who       |
|                | -   States do            |     graduate from all    |
|                |     [not]{.underline}    |     schools.             |
|                |     attempt to obtain    |                          |
|                |     valid personal       | -   States are           |
|                |     identifiers and      |     [not]{.underline}    |
|                |     conduct              |     successful in        |
|                |     administrative       |     obtaining and using  |
|                |     record exchanges     |     contact information  |
|                |     from all schools and |     for at least 60% of  |
|                |     for all completers   |     program completers   |
|                |     who graduate.        |     who graduate.        |
|                |                          |                          |
|                | -   States obtain and    | **[NOTE]{.underline}:    |
|                |     use valid personal   | States can improve       |
|                |     identifiers for less | student coverage and     |
|                |     than 50% of program  | meet standards through   |
|                |     completers who       | the use of supplemental  |
|                |     graduate.            | administrative record    |
|                |                          | exchanges to reduce      |
|                | **[NOTE]{.underline}:    | coverage problems.**     |
|                | States can improve       |                          |
|                | student coverage and     |                          |
|                | meet standards through   |                          |
|                | the use of supplementary |                          |
|                | surveys to reduce        |                          |
|                | coverage problems.**     |                          |
+----------------+--------------------------+--------------------------+
| E.             | **3-                     | **3-                     |
| Response/Match | [Standard]{.underline}** | [Standard]{.underline}** |
|                |                          |                          |
| Capacity       | -   States match to      | -   States obtain        |
|                |     state UI records     |     response rates       |
| Placement      |     systems that provide |     (percent of          |
| measurement    |     at least 70%         |     responses from       |
| achieves       |     in-state employer    |     completers who       |
| acceptable and |     coverage (statewide) |     graduate with        |
| consistent     |     as well as military  |     verified contact     |
| survey         |     and federal          |     information) of at   |
| response rates |     employment records   |     least 70%.           |
| or sufficient  |     [and]{.underline}    |                          |
| match          |     selected             | **2- [Satisfactory       |
| capacity.      |     out-of-state         | Progress]{.underline}**  |
|                |     unemployment         |                          |
|                |     insurance record     | -   States obtain        |
|                |     matching.            |     response rates       |
|                |                          |     (percent of          |
|                | -   States also match to |     responses from       |
|                |     postsecondary        |     completers who       |
|                |     records that include |     graduate with        |
|                |     all public           |     verified contact     |
|                |     community/technical  |     information) of at   |
|                |     colleges and         |     least 50%. States    |
|                |     universities within  |     with between 40 and  |
|                |     the state            |     50% can meet the     |
|                |     [and]{.underline}    |     standard if they can |
|                |     selected             |     provide evidence     |
|                |     out-of-state         |     that falling below   |
|                |     educational record   |     the standard does    |
|                |     matching.            |     not lead to          |
|                |                          |     significant          |
|                | **2- [Satisfactory       |     under-representation |
|                | Progress]{.underline}**  |     of student           |
|                |                          |     populations in       |
|                | -   States match to      |     schools and/or       |
|                |     state UI records     |     within the state.    |
|                |     systems that provide |                          |
|                |     at least 60%         | **1- [Does Not Meet      |
|                |     in-state employer    | ]{.underline}**          |
|                |     coverage             |                          |
|                |     (statewide). States  | -   States obtain        |
|                |     with between 50 and  |     response rates       |
|                |     60% can meet the     |     (percent of          |
|                |     standard if they can |     responses from       |
|                |     provide evidence     |     completers who       |
|                |     that falling below   |     graduate with        |
|                |     the standard does    |     verified contact     |
|                |     not lead to          |     information) of less |
|                |     significant          |     than 50%.            |
|                |     under-representation |                          |
|                |     of major industries  | **[NOTE]{.underline}:    |
|                |     relevant to          | States can meet          |
|                |     vocational programs. | standards through the    |
|                |                          | use of supplemental      |
|                | -   States also match to | administrative record    |
|                |     postsecondary        | exchanges.**             |
|                |     education records    |                          |
|                |     that include at      |                          |
|                |     least all public     |                          |
|                |     community/technical  |                          |
|                |     colleges and         |                          |
|                |     universities within  |                          |
|                |     the state.           |                          |
|                |                          |                          |
|                | **1- [Does Not Meet      |                          |
|                | ]{.underline}**          |                          |
|                |                          |                          |
|                | -   States match to      |                          |
|                |     state UI records     |                          |
|                |     systems that provide |                          |
|                |     less than 50%        |                          |
|                |     in-state employer    |                          |
|                |     coverage. States     |                          |
|                |     also match to        |                          |
|                |     postsecondary        |                          |
|                |     records that do      |                          |
|                |     [not]{.underline}    |                          |
|                |     include all public   |                          |
|                |     community/technical  |                          |
|                |     colleges and         |                          |
|                |     universities within  |                          |
|                |     the state.           |                          |
|                |                          |                          |
|                | **[NOTE]{.underline}:    |                          |
|                | States can improve match |                          |
|                | capacity and meet        |                          |
|                | standards through the    |                          |
|                | use of supplemental      |                          |
|                | surveys to reduce        |                          |
|                | coverage problems for    |                          |
|                | key types of             |                          |
|                | employers/industries.**  |                          |
+----------------+--------------------------+--------------------------+
| F.             | **3-                     | **3-                     |
| Non-Duplicated | [Standard]{.underline}** | [Standard]{.underline}** |
| Counts         |                          |                          |
|                | -   Placement            | -   Placement            |
| Placement      |     measurement collects |     measurement collects |
| measurement is |     and reports          |     and reports          |
| based on       |     placement            |     placement            |
| non-duplicated |     information for each |     information for each |
| counts for the |     type of placement    |     type of placement    |
| three types of |     but reports only     |     but reports only     |
| placement---   |     non-duplicated       |     non-duplicated       |
| postsecondary, |     counts in            |     counts in            |
| employment and |     calculating the      |     calculating the      |
| military       |     overall performance  |     overall performance  |
|                |     level.               |     level.               |
|                |                          |                          |
|                | **2- NOT APPLICABLE**    | **2- NOT APPLICABLE**    |
|                |                          |                          |
|                | **1- [Does Not Meet      | **1- [Does Not Meet      |
|                | ]{.underline}**          | ]{.underline}**          |
|                |                          |                          |
|                | -   Placement            | -   Placement            |
|                |     measurement collects |     measurement collects |
|                |     and reports          |     and reports          |
|                |     placement            |     placement            |
|                |     information for each |     information for each |
|                |     type of placement    |     type of placement    |
|                |     but reports only     |     but reports only     |
|                |                          |                          |
|                | [duplicated]{.underline} | [duplicated]{.underline} |
|                |     counts in            |     counts in            |
|                |     calculating the      |     calculating the      |
|                |     overall performance  |     overall performance  |
|                |     level.               |     level.               |
+----------------+--------------------------+--------------------------+

#### 3P1: Postsecondary Placement

**Measure:**

[Numerator]{.underline}: Number of students who completed a
postsecondary program and who were placed in further postsecondary
education or advanced training, employment, and/or military service
within a state-defined time period (after stopping participation in the
postsecondary program) that ends within the reporting year.

[Denominator]{.underline}: Number of students who completed a
postsecondary program (completers) who are included in placement
measurement (i.e., students surveyed or students with valid identifiers
for matching) within a state-defined time period (after completing) that
ends within the reporting year.

**Measurement Approaches:**

1.  [State-Developed, School-Administered Surveys/Placement
    Records]{.underline}--- Mail and/or telephone surveys of students
    using state-developed surveys or placement forms administered by
    schools. Schools administer and conduct the surveys and maintain
    placement records under state guidelines.

2.  [State-Developed and Administered Surveys]{.underline}---Mail and/or
    telephone surveys of students using state-developed and administered
    surveys.

3.  [Administrative Record Exchanges/Matching of Administrative
    Records]{.underline}---Matching of student records (e.g.,
    postsecondary education records), UI wage records, U.S. Department
    of Defense records---based on student identifiers.

**Quality Criteria and Scoring Rubrics**

The standards levels in the scoring rubrics are the
[proposed]{.underline} levels for use [only]{.underline} in state peer
reviews in the Data Quality Initiative. States will be asked to review
and suggest revisions after using them in the first peer review process.
[They are not federal requirements.]{.underline}

+----------------+------------------------+---------------------------+
| **Quality      | ######## Administ      | ######## Surveys          |
| Criteria**     | rative Record Exchange |                           |
|                |                        | **State Developed and     |
|                |                        | Administered Surveys**    |
|                |                        |                           |
|                |                        | **State Developed and     |
|                |                        | Locally Administered      |
|                |                        | Surveys/Placement Forms** |
+----------------+------------------------+---------------------------+
| A.  Alignment  | **3-                   | **3-                      |
|     to         | [S                     | [Standard]{.underline}**  |
|                | tandard]{.underline}** |                           |
|    Definitions |                        | -   A standardized        |
|     of Three   | -   A standardized     |     definition for each   |
|     Types of   |     definition for     |     of the three types of |
|     Placement  |     each of the three  |     placement             |
|                |     types of placement |     --postsecondary       |
| Surveys,       |     --postsecondary    |     education,            |
| placement      |     education,         |     employment, and       |
| forms, or      |     employment, and    |     military---are        |
| administrative |     military---are     |     adopted and           |
| record         |     adopted and        |     implemented           |
| exchanges use  |     implemented        |     consistently          |
| consistent     |     consistently       |     throughout the state. |
| definitions of |     throughout the     |                           |
| the three      |     state.             | **2- [Satisfactory        |
| types of       |                        | Progress]{.underline}**   |
| placement--    | **2- [Satisfactory     |                           |
| -postsecondary | P                      | -   A standardized        |
| education,     | rogress]{.underline}** |     definition for at     |
| employment,    |                        |     least two types of    |
| and military.  | -   A standardized     |     pl                    |
|                |     definition for at  | acement\-\--postsecondary |
|                |     least two types of |     education and         |
|                |     place              |     employment\-\--are    |
|                | ment\-\--postsecondary |     adopted and           |
|                |     education and      |     implemented           |
|                |     employment\-\--are |     consistently          |
|                |     adopted and        |     throughout the state. |
|                |     implemented        |                           |
|                |     consistently       | **1- [Does Not Meet       |
|                |     throughout the     | ]{.underline}**           |
|                |     state.             |                           |
|                |                        | -   No standardized       |
|                | **1- [Does Not Meet    |     definitions or        |
|                | ]{.underline}**        |     standardized          |
|                |                        |     definitions for only  |
|                | -   No standardized    |     employment or only    |
|                |     definitions or     |     postsecondary         |
|                |     standardized       |     education are used    |
|                |     definitions for    |     consistently across   |
|                |     only employment or |     the state.            |
|                |     only postsecondary |                           |
|                |     education are used |                           |
|                |     consistently       |                           |
|                |     across the state.  |                           |
+----------------+------------------------+---------------------------+
| B.  **Timing   | **3-                   | **3-                      |
|     of         | [S                     | [Standard]{.underline}**  |
|     Placement  | tandard]{.underline}** |                           |
|                |                        | -   Placement is defined  |
|  Measurement** | -   Placement is       |     as placement in       |
|                |     defined as         |     postsecondary         |
| Placement is   |     placement in       |     education,            |
| measured       |     postsec-ondary     |     employment, and/or    |
| during the     |     education,         |     military at any time  |
| same           |     employment, and/or |     during a standardized |
| measurement    |     military at any    |     time period after     |
| intervals for  |     time during a      |     students complete     |
| all completers |     standardized time  |     programs (e.g., 6     |
| (e.g., 3       |     period after       |     months after students |
| months, 6      |     students complete  |     complete secondary    |
| months) after  |     programs (e.g., 6  |     education) even if    |
| completers     |     months after       |     students complete at  |
| complete       |     students complete  |     different times       |
| programs.      |     programs) even if  |     during the school     |
|                |     students complete  |     year.                 |
|                |     at different times |                           |
|                |     during the school  | -   [Note]{.underline}:   |
|                |     year.              |     This would result in  |
|                |                        |     the state having one  |
|                | -                      |     or more periods for   |
|                |    [Note]{.underline}: |     data collection if    |
|                |     This would result  |     students complete     |
|                |     in the state       |     during different      |
|                |     having one or more |     months during the     |
|                |     periods for data   |     year.                 |
|                |     collection if      |                           |
|                |     students complete  | **2- [Satisfactory        |
|                |     during different   | Progress]{.underline}**   |
|                |     months during the  |                           |
|                |     year.              | -   Placement is defined  |
|                |                        |     as placement in       |
|                | **2- [Satisfactory     |     postsecondary         |
|                | P                      |     education,            |
|                | rogress]{.underline}** |     employment, and/or    |
|                |                        |     military at any time  |
|                | -   Placement is       |     during a standardized |
|                |     defined as         |     time period after     |
|                |     placement in       |     students complete     |
|                |     postsecondary      |     programs.             |
|                |     education,         |                           |
|                |     employment, and/or | -   All students are      |
|                |     military at any    |     measured using a      |
|                |     time during a      |     common measurement    |
|                |     standardized time  |     period regardless of  |
|                |     period after       |     when they completed   |
|                |     students complete  |     programs during the   |
|                |     programs.          |     school year. (e.g.    |
|                |                        |     July 1 to December 31 |
|                | -   All students are   |     following the school  |
|                |     measured using a   |     year).                |
|                |     common measurement |                           |
|                |     period regardless  | **1- [Does Not            |
|                |     of when they       | Meet]{.underline}**       |
|                |     completed programs |                           |
|                |     during the school  | -   Placement is defined  |
|                |     year. (e.g. July 1 |     as placement in       |
|                |     to December 31     |     postsecondary         |
|                |     following the      |     education,            |
|                |     school year).      |     employment, and/or    |
|                |                        |     military at any time  |
|                | **1- [Does Not Meet    |     during a designated   |
|                | ]{.underline}**        |     time period after     |
|                |                        |     students complete     |
|                | -   Placement is       |     programs.             |
|                |     defined as         |                           |
|                |     placement in       | ```{=html}                |
|                |     postsecondary      | <!-- -->                  |
|                |     education,         | ```                       |
|                |     employment, and/or | -   There is no           |
|                |     military at any    |     standardized or       |
|                |     time during a      |     common time period    |
|                |     designated time    |     used for all          |
|                |     period after       |     completers.           |
|                |     students complete  |                           |
|                |     programs.          |                           |
|                |                        |                           |
|                | ```{=html}             |                           |
|                | <!-- -->               |                           |
|                | ```                    |                           |
|                | -   There is no        |                           |
|                |     standardized or    |                           |
|                |     common time period |                           |
|                |     used for all       |                           |
|                |     completers.        |                           |
+----------------+------------------------+---------------------------+
| **C.           | **3-                   | **3-                      |
| Reliability    | [S                     | [Standard]{.underline}**  |
| of**           | tandard]{.underline}** |                           |
|                |                        | -   Standardized          |
| **Placement**  | -   Standardized       |     statewide survey      |
|                |     written procedures |     instruments and       |
| *              |     are followed to    |     survey procedures are |
| *Measurement** |     ensure the         |     followed to ensure    |
|                |     reliability of     |     the reliability of    |
| Placement      |     administrative     |     state or locally      |
| measurement    |     records matches by |     administered surveys. |
| reports        |     all participating  |                           |
| placement data |     state agencies     | -   Regular independent   |
| based on       |                        |     audits are conducted  |
| consistent     | -   Regular            |     to verify and improve |
| surveys,       |     independent audits |     instruments and       |
| forms, and     |     are conducted to   |     survey procedures.    |
| administrative |     verify and improve |                           |
| records and    |     procedures.        | **2- [Satisfactory        |
| measurement    |                        | Progress]{.underline}**   |
| procedures.    | **2- [Satisfactory     |                           |
|                | P                      | -   Standardized survey   |
|                | rogress]{.underline}** |     instruments and       |
|                |                        |     survey procedures are |
|                | -   Standardized       |     followed to ensure    |
|                |     written procedures |     the reliability of    |
|                |     are followed to    |     state or locally      |
|                |     ensure the         |     administered surveys. |
|                |     reliability of     |                           |
|                |     administrative     | **1- [Does Not Meet       |
|                |     record matches by  | ]{.underline}**           |
|                |     all participating  |                           |
|                |     state agencies.    | -   No standardized       |
|                |                        |     statewide survey      |
|                | **1- [Does Not Meet    |     instruments and       |
|                | ]{.underline}**        |     survey procedures are |
|                |                        |     followed to ensure    |
|                | -   No standardized,   |     the reliability of    |
|                |     written procedures |     state or locally      |
|                |     are used to ensure |     administered surveys. |
|                |     the reliability of |                           |
|                |     administrative     |                           |
|                |     record matches.    |                           |
+----------------+------------------------+---------------------------+
| **D. Student   | **3-                   | **3-                      |
| Coverage in    | [S                     | [Standard]{.underline}**  |
| Placement      | tandard]{.underline}** |                           |
| Measurement**  |                        | -   States attempt to     |
|                | -   States attempt to  |     obtain contact        |
| Placement      |     obtain valid       |     information and       |
| measurement    |     personal           |     survey all completers |
| attempts to    |     identifiers and    |     who graduate from all |
| gather data    |     conduct            |     institutions.         |
| for all        |     administrative     |                           |
| program        |     record exchanges   | -   States are successful |
| completers for |     from all           |     in obtaining and      |
| all programs   |     institutions and   |     using verified        |
| within the     |     for all            |     student contact       |
| state.         |     completers.        |     information for mail  |
|                |                        |     and/or phone surveys  |
|                | -   States are         |     for at least 80% of   |
|                |     successful in      |     program completers.   |
|                |     obtaining and      |                           |
|                |     using valid social | **2- [Satisfactory        |
|                |     security numbers   | Progress]{.underline}**   |
|                |     for administrative |                           |
|                |     record matching    | -   States attempt to     |
|                |     for at least 80%   |     obtain contact        |
|                |     of program         |     information and       |
|                |     completers who     |     survey all completers |
|                |     graduate.          |     who graduate from all |
|                |                        |     institutions.         |
|                | **2- [Satisfactory     |                           |
|                | P                      | -   States are successful |
|                | rogress]{.underline}** |     in obtaining and      |
|                |                        |     using verified        |
|                | -   States attempt to  |     student contact       |
|                |     obtain valid       |     information for mail  |
|                |     personal           |     and/or phone surveys  |
|                |     identifiers and    |     from all schools with |
|                |     conduct            |     at least 70% of       |
|                |     administrative     |     program completers.   |
|                |     record exchanges   |     States with schools   |
|                |     from all           |     between 60 and 70%    |
|                |     institutions and   |     can meet the standard |
|                |     for all            |     if they can provide   |
|                |     completers.        |     evidence that falling |
|                |                        |     below the standard    |
|                | -   States obtain and  |     does not lead to      |
|                |     use valid personal |     significant           |
|                |     identifiers for    |     under-representation  |
|                |     administrative     |     of student            |
|                |     record matching    |     populations within    |
|                |     from all           |     the institution       |
|                |     institutions with  |     and/or within the     |
|                |     all institutions   |     state.                |
|                |     providing          |                           |
|                |     identifiers for at | **1- [Does Not Meet       |
|                |     least 70% of their | ]{.underline}**           |
|                |     program completers |                           |
|                |     who graduate.      | -   States do             |
|                |                        |     [not]{.underline}     |
|                |    [Note]{.underline}: |     attempt or obtain     |
|                |     States with        |     contact information   |
|                |     schools with       |     and survey all        |
|                |     between 60% and    |     completers who        |
|                |     70% can meet the   |     graduate from all     |
|                |     standard if they   |     institutions.         |
|                |     can provide        |                           |
|                |     evidence that      | -   States are            |
|                |     falling below the  |     [not]{.underline}     |
|                |     standard does not  |     successful in         |
|                |     lead to            |     obtaining and using   |
|                |     significant        |     contact information   |
|                |                        |     for at least 60% of   |
|                |   under-representation |     program completers.   |
|                |     of student         |                           |
|                |     populations within | **[NOTE]{.underline}:     |
|                |     the institution    | States can improve        |
|                |     and/or within the  | student coverage and meet |
|                |     state.             | standards through the use |
|                |                        | of supplemental           |
|                | **1- [Does Not Meet    | administrative record     |
|                | ]{.underline}**        | exchanges to reduce       |
|                |                        | coverage problems.**      |
|                | -   States do          |                           |
|                |     [not]{.underline}  |                           |
|                |     attempt to obtain  |                           |
|                |     valid personal     |                           |
|                |     identifiers and    |                           |
|                |     conduct            |                           |
|                |     administrative     |                           |
|                |     record exchanges   |                           |
|                |     from all schools   |                           |
|                |     and for all        |                           |
|                |     completers who     |                           |
|                |     graduate.          |                           |
|                |                        |                           |
|                | -   States obtain and  |                           |
|                |     use valid personal |                           |
|                |     identifiers for    |                           |
|                |     less than 60% of   |                           |
|                |     program completers |                           |
|                |     who graduate.      |                           |
|                |                        |                           |
|                | **[NOTE]{.underline}:  |                           |
|                | States can improve     |                           |
|                | student coverage and   |                           |
|                | meet standards through |                           |
|                | the use of             |                           |
|                | supplementary surveys  |                           |
|                | to reduce coverage     |                           |
|                | problems.**            |                           |
+----------------+------------------------+---------------------------+
| E.             | **3-                   | **3-                      |
| Response/Match | [S                     | [Standard]{.underline}**  |
|                | tandard]{.underline}** |                           |
| Capacity       |                        | -   States obtain         |
|                | -   States match to    |     response rates        |
| Placement      |     state UI records   |     (percent of responses |
| measurement    |     systems that       |     from completers with  |
| achieves       |     provide at least   |     verified contact      |
| acceptable and |     70% in-state       |     information) of at    |
| consistent     |     employer coverage  |     least 70%.            |
| survey         |     (statewide) as     |                           |
| response rates |     well as military   | **2- [Satisfactory        |
| or sufficient  |     and federal        | Progress]{.underline}**   |
| match          |     employment records |                           |
| capacity.      |     [and]{.underline}  | -   States obtain         |
|                |     selected           |     response rates        |
|                |     out-of-state       |     (percent of responses |
|                |     unemployment       |     from completers who   |
|                |     insurance record   |     graduate with         |
|                |     matching.          |     verified contact      |
|                |                        |     information) of at    |
|                | -   States also match  |     least 50%. States     |
|                |     to postsecondary   |     with between 40 and   |
|                |     records that       |     50% can meet the      |
|                |     include all public |     standard if they can  |
|                |     colleges and       |     provide evidence that |
|                |     universities       |     falling below the     |
|                |     within the state   |     standard does not     |
|                |     [and]{.underline}  |     lead to significant   |
|                |     selected           |     under-representation  |
|                |     out-of-state       |     of student            |
|                |     educational record |     populations in        |
|                |     matching.          |     institutions and/or   |
|                |                        |     within the state.     |
|                | **2- [Satisfactory     |                           |
|                | P                      | **1- [Does Not Meet       |
|                | rogress]{.underline}** | ]{.underline}**           |
|                |                        |                           |
|                | -   States match to    | -   States obtain         |
|                |     state UI records   |     response rates        |
|                |     systems that       |     (percent of responses |
|                |     provide at least   |     from completers with  |
|                |     60% in-state       |     verified contact      |
|                |     employer coverage  |     information) of less  |
|                |     (statewide).       |     than 50%.             |
|                |     States with        |                           |
|                |     between 50 and 60% | **[NOTE]{.underline}:     |
|                |     can meet the       | States can meet standards |
|                |     standard if they   | through the use of        |
|                |     can provide        | supplemental              |
|                |     evidence that      | administrative record     |
|                |     falling below the  | exchanges.**              |
|                |     standard does not  |                           |
|                |     lead to            |                           |
|                |     significant        |                           |
|                |                        |                           |
|                |   under-representation |                           |
|                |     of major           |                           |
|                |     industries         |                           |
|                |     relevant to        |                           |
|                |     vocational         |                           |
|                |     programs.          |                           |
|                |                        |                           |
|                | -   States also match  |                           |
|                |     to postsecondary   |                           |
|                |     education records  |                           |
|                |     that include at    |                           |
|                |     least all public   |                           |
|                |     colleges and       |                           |
|                |     universities       |                           |
|                |     within the state.  |                           |
|                |                        |                           |
|                | **1- [Does Not Meet    |                           |
|                | ]{.underline}**        |                           |
|                |                        |                           |
|                | -   States match to    |                           |
|                |     state UI records   |                           |
|                |     systems that       |                           |
|                |     provide less than  |                           |
|                |     50% in-state       |                           |
|                |     employer coverage. |                           |
|                |     States also match  |                           |
|                |     to postsecondary   |                           |
|                |     records that do    |                           |
|                |     [not]{.underline}  |                           |
|                |     include all public |                           |
|                |     colleges and       |                           |
|                |     universities       |                           |
|                |     within the state.  |                           |
|                |                        |                           |
|                | **[NOTE]{.underline}:  |                           |
|                | States can improve     |                           |
|                | match capacity and     |                           |
|                | meet standards through |                           |
|                | the use of             |                           |
|                | supplemental surveys   |                           |
|                | to reduce coverage     |                           |
|                | problems for key types |                           |
|                | of                     |                           |
|                | e                      |                           |
|                | mployers/industries.** |                           |
+----------------+------------------------+---------------------------+
| F.             | **3-                   | **3-                      |
| Non-Duplicated | [S                     | [Standard]{.underline}**  |
| Counts         | tandard]{.underline}** |                           |
|                |                        | -   Placement measurement |
| Placement      | -   Placement          |     collects and reports  |
| measurement is |     measurement        |     placement information |
| based on       |     collects and       |     for each type of      |
| non-duplicated |     reports placement  |     placement but reports |
| counts for the |     information for    |     only non-duplicated   |
| three types of |     each type of       |     counts in calculating |
| placement---   |     placement but      |     the overall           |
| postsecondary, |     reports only       |     performance level.    |
| employment and |     non-duplicated     |                           |
| military       |     counts in          | **2- NOT APPLICABLE**     |
|                |     calculating the    |                           |
|                |     overall            | **1- [Does Not Meet       |
|                |     performance level. | ]{.underline}**           |
|                |                        |                           |
|                | **2- NOT APPLICABLE**  | -   Placement measurement |
|                |                        |     collects and reports  |
|                | **1- [Does Not Meet    |     placement information |
|                | ]{.underline}**        |     for each type of      |
|                |                        |     placement but reports |
|                | -   Placement          |     only                  |
|                |     measurement        |                           |
|                |     collects and       |  [duplicated]{.underline} |
|                |     reports placement  |     counts in calculating |
|                |     information for    |     the overall           |
|                |     each type of       |     performance level.    |
|                |     placement but      |                           |
|                |     reports only       |                           |
|                |     [d                 |                           |
|                | uplicated]{.underline} |                           |
|                |     counts in          |                           |
|                |     calculating the    |                           |
|                |     overall            |                           |
|                |     performance level. |                           |
+----------------+------------------------+---------------------------+

#### 3P1: Postsecondary Retention

**Measure:**

[Numerator:]{.underline} Number of students who completed a
postsecondary program and were placed in further postsecondary education
or advanced training, employment, and/or military service in the
reporting period and were retained in one or more of these types of
placement within a state-designated time period (expressed in months or
UI wage record quarters).

[Denominator:]{.underline} Number of students who completed a
postsecondary program and who were placed in further postsecondary
education or advanced training, employment, and/or military service in
the reporting year.

**Measurement Approaches:**

1.  [State-Developed, School-Administered Surveys/Placement
    Records]{.underline}--- Mail and/or telephone surveys of students
    using state-developed surveys or placement forms administered by
    schools. Schools administer and conduct the surveys and maintain
    placement records under state guidelines.

2.  [State-Developed and Administered Surveys]{.underline}---Mail and/or
    telephone surveys of students using state-developed and administered
    surveys.

3.  [Administrative Record Exchanges/Matching of Administrative
    Records]{.underline}---Matching of student records (e.g.,
    postsecondary education records), UI wage records, U.S. Department
    of Defense records---based on student identifiers.

**Quality Criteria and Scoring Rubrics**

The standards levels in the scoring rubrics are the
[proposed]{.underline} levels for use [only]{.underline} in state peer
reviews in the Data Quality Initiative. States will be asked to review
and suggest revisions after using them in the first peer review process.
[They are not federal requirements.]{.underline}

+----------------+------------------------+---------------------------+
| **Quality      | ######## Administ      | ######## Surveys          |
| Criteria**     | rative Record Exchange |                           |
|                |                        | **State Developed and     |
|                |                        | Administered Surveys**    |
|                |                        |                           |
|                |                        | **State Developed and     |
|                |                        | Locally Administered      |
|                |                        | Surveys/Placement Forms** |
+----------------+------------------------+---------------------------+
| B.  Alignment  | **3-                   | **3-                      |
|     to         | [S                     | [Standard]{.underline}**  |
|                | tandard]{.underline}** |                           |
|    Definitions |                        | -   A standardized        |
|     of Three   | -   A standardized     |     definition for each   |
|     Types of   |     definition for     |     of the three types of |
|     Retention  |     each of the three  |                           |
|                |     types of retention |  retention--postsecondary |
| Surveys,       |     --postsecondary    |     education,            |
| placement      |     education,         |     employment, and       |
| forms, or      |     employment, and    |     military---are        |
| administrative |     military---are     |     adopted and           |
| record         |     adopted and        |     implemented           |
| exchanges use  |     implemented        |     consistently          |
| consistent     |     consistently       |     throughout the state. |
| definitions of |     throughout the     |                           |
| the three      |     state.             | **2- [Satisfactory        |
| types of       |                        | Progress]{.underline}**   |
| retention--    | **2- [Satisfactory     |                           |
| -postsecondary | P                      | -   A standardized        |
| education,     | rogress]{.underline}** |     definition for at     |
| employment,    |                        |     least two types of    |
| and military.  | -   A standardized     |     re                    |
|                |     definition for at  | tention\-\--postsecondary |
|                |     least two types of |     education and         |
|                |     reten              |     employment\-\--are    |
|                | tion\-\--postsecondary |     adopted and           |
|                |     education and      |     implemented           |
|                |     employment\-\--are |     consistently          |
|                |     adopted and        |     throughout the state. |
|                |     implemented        |                           |
|                |     consistently       | **1- [Does Not Meet       |
|                |     throughout the     | ]{.underline}**           |
|                |     state.             |                           |
|                |                        | -   No standardized       |
|                | **1- [Does Not Meet    |     definitions or        |
|                | ]{.underline}**        |     standardized          |
|                |                        |     definitions for only  |
|                | -   No standardized    |     employment or only    |
|                |     definitions or     |     postsecondary         |
|                |     standardized       |     education are used    |
|                |     definitions for    |     consistently across   |
|                |     only employment or |     the state.            |
|                |     only postsecondary |                           |
|                |     education are used |                           |
|                |     consistently       |                           |
|                |     across the state.  |                           |
+----------------+------------------------+---------------------------+
| C.  **Timing   | **3-                   | **3-                      |
|     of         | [S                     | [Standard]{.underline}**  |
|     Retention  | tandard]{.underline}** |                           |
|                |                        | -   Retention is defined  |
|  Measurement** | -   Retention is       |     as retention in       |
|                |     defined as         |     postsecondary         |
| Retention is   |     retention in       |     education,            |
| measured       |     postsec-ondary     |     employment, and/or    |
| during the     |     education,         |     military at any time  |
| same           |     employment, and/or |     during a standardized |
| measurement    |     military at any    |     time period after     |
| intervals for  |     time during a      |     students are placed   |
| all completers |     standardized time  |     (e.g., 6 months after |
| (e.g., 3       |     period after       |     students are placed)  |
| months, 6      |     students are       |     even if students are  |
| months) after  |     placed (e.g., 6    |     placed at different   |
| completers are |     months after       |     times during the      |
| placed.        |     students re        |     school year.          |
|                |     placed) even if    |                           |
|                |     students are       | -   [Note]{.underline}:   |
|                |     placed at          |     This would result in  |
|                |     different times    |     the state having one  |
|                |     during the school  |     or more periods for   |
|                |     year.              |     data collection if    |
|                |                        |     students are placed   |
|                | -                      |     during different      |
|                |    [Note]{.underline}: |     months during the     |
|                |     This would result  |     year.                 |
|                |     in the state       |                           |
|                |     having one or more | **2- [Satisfactory        |
|                |     periods for data   | Progress]{.underline}**   |
|                |     collection if      |                           |
|                |     students are       | -   Retention is defined  |
|                |     placed during      |     as retention in       |
|                |     different months   |     postsecondary         |
|                |     during the year.   |     education,            |
|                |                        |     employment, and/or    |
|                | **2- [Satisfactory     |     military at any time  |
|                | P                      |     during a standardized |
|                | rogress]{.underline}** |     time period after     |
|                |                        |     students are placed.  |
|                | -   Retention is       |                           |
|                |     defined retention  | -   All students are      |
|                |     in postsecondary   |     measured using a      |
|                |     education,         |     common measurement    |
|                |     employment, and/or |     period regardless of  |
|                |     military at any    |     when they are placed  |
|                |     time during a      |     during the school     |
|                |     standardized time  |     year. (e.g. July 1 to |
|                |     period after       |     December 31 following |
|                |     students are       |     the school year).     |
|                |     placed.            |                           |
|                |                        | **1- [Does Not            |
|                | -   All students are   | Meet]{.underline}**       |
|                |     measured using a   |                           |
|                |     common measurement | -   Retention is defined  |
|                |     period regardless  |     as retention in       |
|                |     of when they are   |     postsecondary         |
|                |     placed during the  |     education,            |
|                |     school year. (e.g. |     employment, and/or    |
|                |     July 1 to December |     military at any time  |
|                |     31 following the   |     during a designated   |
|                |     school year).      |     time period after     |
|                |                        |     students are placed.  |
|                | **1- [Does Not Meet    |                           |
|                | ]{.underline}**        | ```{=html}                |
|                |                        | <!-- -->                  |
|                | -   Retention is       | ```                       |
|                |     defined as         | -   There is no           |
|                |     retention in       |     standardized or       |
|                |     postsecondary      |     common time period    |
|                |     education,         |     used for all          |
|                |     employment, and/or |     completers who are    |
|                |     military at any    |     placed.               |
|                |     time during a      |                           |
|                |     designated time    |                           |
|                |     period after       |                           |
|                |     students are       |                           |
|                |     placed.            |                           |
|                |                        |                           |
|                | ```{=html}             |                           |
|                | <!-- -->               |                           |
|                | ```                    |                           |
|                | -   There is no        |                           |
|                |     standardized or    |                           |
|                |     common time period |                           |
|                |     used for all       |                           |
|                |     completers who are |                           |
|                |     placed.            |                           |
+----------------+------------------------+---------------------------+
| **C.           | **3-                   | **3-                      |
| Reliability    | [S                     | [Standard]{.underline}**  |
| of**           | tandard]{.underline}** |                           |
|                |                        | -   Standardized          |
| **Retention**  | -   Standardized       |     statewide survey      |
|                |     written procedures |     instruments and       |
| *              |     are followed to    |     survey procedures are |
| *Measurement** |     ensure the         |     followed to ensure    |
|                |     reliability of     |     the reliability of    |
| Retention      |     administrative     |     state or locally      |
| measurement    |     records matches by |     administered surveys. |
| reports        |     all participating  |                           |
| placement data |     state agencies     | -   Regular independent   |
| based on       |                        |     audits are conducted  |
| consistent     | -   Regular            |     to verify and improve |
| surveys,       |     independent audits |     instruments and       |
| forms, and     |     are conducted to   |     survey procedures.    |
| administrative |     verify and improve |                           |
| records and    |     procedures.        | **2- [Satisfactory        |
| measurement    |                        | Progress]{.underline}**   |
| procedures.    | **2- [Satisfactory     |                           |
|                | P                      | -   Standardized survey   |
|                | rogress]{.underline}** |     instruments and       |
|                |                        |     survey procedures are |
|                | -   Standardized       |     followed to ensure    |
|                |     written procedures |     the reliability of    |
|                |     are followed to    |     state or locally      |
|                |     ensure the         |     administered surveys. |
|                |     reliability of     |                           |
|                |     administrative     | **1- [Does Not Meet       |
|                |     record matches by  | ]{.underline}**           |
|                |     all participating  |                           |
|                |     state agencies.    | -   No standardized       |
|                |                        |     statewide survey      |
|                | **1- [Does Not Meet    |     instruments and       |
|                | ]{.underline}**        |     survey procedures are |
|                |                        |     followed to ensure    |
|                | -   No standardized,   |     the reliability of    |
|                |     written procedures |     state or locally      |
|                |     are used to ensure |     administered surveys. |
|                |     the reliability of |                           |
|                |     administrative     |                           |
|                |     record matches.    |                           |
+----------------+------------------------+---------------------------+
| **D. Student   | **3-                   | **3-                      |
| Coverage in    | [S                     | [Standard]{.underline}**  |
| Retention      | tandard]{.underline}** |                           |
| Measurement**  |                        | -   States attempt to     |
|                | -   States attempt to  |     obtain contact        |
| Retention      |     obtain valid       |     information and       |
| measurement    |     personal           |     survey all completers |
| attempts to    |     identifiers and    |     who are placed from   |
| gather data    |     conduct            |     all institutions.     |
| for all        |     administrative     |                           |
| program        |     record exchanges   | -   States are successful |
| completers who |     from all           |     in obtaining and      |
| are placed for |     institutions and   |     using verified        |
| all programs   |     for all            |     student contact       |
| within the     |     completers.        |     information for mail  |
| state.         |                        |     and/or phone surveys  |
|                | -   States are         |     for at least 80% of   |
|                |     successful in      |     completers who are    |
|                |     obtaining and      |     placed.               |
|                |     using valid social |                           |
|                |     security numbers   | **2- [Satisfactory        |
|                |     for administrative | Progress]{.underline}**   |
|                |     record matching    |                           |
|                |     for at least 80%   | -   States attempt to     |
|                |     of program         |     obtain contact        |
|                |     completers who are |     information and       |
|                |     placed.            |     survey all completers |
|                |                        |     who are placed from   |
|                | **2- [Satisfactory     |     all institutions.     |
|                | P                      |                           |
|                | rogress]{.underline}** | -   States are successful |
|                |                        |     in obtaining and      |
|                | -   States attempt to  |     using verified        |
|                |     obtain valid       |     student contact       |
|                |     personal           |     information for mail  |
|                |     identifiers and    |     and/or phone surveys  |
|                |     conduct            |     from all schools with |
|                |     administrative     |     at least 70% of       |
|                |     record exchanges   |     completers who are    |
|                |     from all           |     placed. States with   |
|                |     institutions and   |     schools between 60    |
|                |     for all completers |     and 70% can meet the  |
|                |     who are placed.    |     standard if they can  |
|                |                        |     provide evidence that |
|                | -   States obtain and  |     falling below the     |
|                |     use valid personal |     standard does not     |
|                |     identifiers for    |     lead to significant   |
|                |     administrative     |     under-representation  |
|                |     record matching    |     of student            |
|                |     from all           |     populations within    |
|                |     institutions with  |     the institution       |
|                |     all institutions   |     and/or within the     |
|                |     providing          |     state.                |
|                |     identifiers for at |                           |
|                |     least 70% of their | **1- [Does Not Meet       |
|                |     program completers | ]{.underline}**           |
|                |     who are placed.    |                           |
|                |                        | -   States do             |
|                |    [Note]{.underline}: |     [not]{.underline}     |
|                |     States with        |     attempt or obtain     |
|                |     schools with       |     contact information   |
|                |     between 60% and    |     and survey all        |
|                |     70% can meet the   |     completers who are    |
|                |     standard if they   |     placed from all       |
|                |     can provide        |     institutions.         |
|                |     evidence that      |                           |
|                |     falling below the  | -   States are            |
|                |     standard does not  |     [not]{.underline}     |
|                |     lead to            |     successful in         |
|                |     significant        |     obtaining and using   |
|                |                        |     contact information   |
|                |   under-representation |     for at least 60% of   |
|                |     of student         |     program completers    |
|                |     populations within |     who are placed.       |
|                |     the institution    |                           |
|                |     and/or within the  | **[NOTE]{.underline}:     |
|                |     state.             | States can improve        |
|                |                        | student coverage and meet |
|                | **1- [Does Not Meet    | standards through the use |
|                | ]{.underline}**        | of supplemental           |
|                |                        | administrative record     |
|                | -   States do          | exchanges to reduce       |
|                |     [not]{.underline}  | coverage problems.**      |
|                |     attempt to obtain  |                           |
|                |     valid personal     |                           |
|                |     identifiers and    |                           |
|                |     conduct            |                           |
|                |     administrative     |                           |
|                |     record exchanges   |                           |
|                |     from all schools   |                           |
|                |     and for all        |                           |
|                |     completers who are |                           |
|                |     placed.            |                           |
|                |                        |                           |
|                | -   States obtain and  |                           |
|                |     use valid personal |                           |
|                |     identifiers for    |                           |
|                |     less than 60% of   |                           |
|                |     program completers |                           |
|                |     who are placed.    |                           |
|                |                        |                           |
|                | **[NOTE]{.underline}:  |                           |
|                | States can improve     |                           |
|                | student coverage and   |                           |
|                | meet standards through |                           |
|                | the use of             |                           |
|                | supplementary surveys  |                           |
|                | to reduce coverage     |                           |
|                | problems.**            |                           |
+----------------+------------------------+---------------------------+
| E.             | **3-                   | **3-                      |
| Response/Match | [S                     | [Standard]{.underline}**  |
|                | tandard]{.underline}** |                           |
| Capacity       |                        | -   States obtain         |
|                | -   States match to    |     response rates        |
| Retention      |     state UI records   |     (percent of responses |
| measurement    |     systems that       |     from completers who   |
| achieves       |     provide at least   |     are placed with       |
| acceptable and |     70% in-state       |     verified contact      |
| consistent     |     employer coverage  |     information) of at    |
| survey         |     (statewide) as     |     least 70%.            |
| response rates |     well as military   |                           |
| or sufficient  |     and federal        | **2- [Satisfactory        |
| match          |     employment records | Progress]{.underline}**   |
| capacity.      |     [and]{.underline}  |                           |
|                |     selected           | -   States obtain         |
|                |     out-of-state       |     response rates        |
|                |     unemployment       |     (percent of responses |
|                |     insurance record   |     from completers who   |
|                |     matching.          |     are placed with       |
|                |                        |     verified contact      |
|                | -   States also match  |     information) of at    |
|                |     to postsecondary   |     least 50%. States     |
|                |     records that       |     with between 40 and   |
|                |     include all public |     50% can meet the      |
|                |     colleges and       |     standard if they can  |
|                |     universities       |     provide evidence that |
|                |     within the state   |     falling below the     |
|                |     [and]{.underline}  |     standard does not     |
|                |     selected           |     lead to significant   |
|                |     out-of-state       |     under-representation  |
|                |     educational record |     of student            |
|                |     matching.          |     populations in        |
|                |                        |     institutions and/or   |
|                | **2- [Satisfactory     |     within the state.     |
|                | P                      |                           |
|                | rogress]{.underline}** | **1- [Does Not Meet       |
|                |                        | ]{.underline}**           |
|                | -   States match to    |                           |
|                |     state UI records   | -   States obtain         |
|                |     systems that       |     response rates        |
|                |     provide at least   |     (percent of responses |
|                |     60% in-state       |     from completers who   |
|                |     employer coverage  |     are placed with       |
|                |     (statewide).       |     verified contact      |
|                |     States with        |     information) of less  |
|                |     between 50 and 60% |     than 50%.             |
|                |     can meet the       |                           |
|                |     standard if they   | **[NOTE]{.underline}:     |
|                |     can provide        | States can meet standards |
|                |     evidence that      | through the use of        |
|                |     falling below the  | supplemental              |
|                |     standard does not  | administrative record     |
|                |     lead to            | exchanges.**              |
|                |     significant        |                           |
|                |                        |                           |
|                |   under-representation |                           |
|                |     of major           |                           |
|                |     industries         |                           |
|                |     relevant to        |                           |
|                |     vocational         |                           |
|                |     programs.          |                           |
|                |                        |                           |
|                | -   States also match  |                           |
|                |     to postsecondary   |                           |
|                |     education records  |                           |
|                |     that include at    |                           |
|                |     least all public   |                           |
|                |     colleges and       |                           |
|                |     universities       |                           |
|                |     within the state.  |                           |
|                |                        |                           |
|                | **1- [Does Not Meet    |                           |
|                | ]{.underline}**        |                           |
|                |                        |                           |
|                | -   States match to    |                           |
|                |     state UI records   |                           |
|                |     systems that       |                           |
|                |     provide less than  |                           |
|                |     50% in-state       |                           |
|                |     employer coverage. |                           |
|                |     States also match  |                           |
|                |     to postsecondary   |                           |
|                |     records that do    |                           |
|                |     [not]{.underline}  |                           |
|                |     include all public |                           |
|                |     colleges and       |                           |
|                |     universities       |                           |
|                |     within the state.  |                           |
|                |                        |                           |
|                | **[NOTE]{.underline}:  |                           |
|                | States can improve     |                           |
|                | match capacity and     |                           |
|                | meet standards through |                           |
|                | the use of             |                           |
|                | supplemental surveys   |                           |
|                | to reduce coverage     |                           |
|                | problems for key types |                           |
|                | of                     |                           |
|                | e                      |                           |
|                | mployers/industries.** |                           |
+----------------+------------------------+---------------------------+
| F.             | **3-                   | **3-                      |
| Non-Duplicated | [S                     | [Standard]{.underline}**  |
| Counts         | tandard]{.underline}** |                           |
|                |                        | -   Retention measurement |
| Retention      | -   Retention          |     collects and reports  |
| measurement is |     measurement        |     retention information |
| based on       |     collects and       |     for each type of      |
| non-duplicated |     reports retention  |     retention but reports |
| counts for the |     information for    |     only non-duplicated   |
| three types of |     each type of       |     counts in calculating |
| retention---   |     retention but      |     the overall           |
| postsecondary, |     reports only       |     performance level.    |
| employment and |     non-duplicated     |                           |
| military       |     counts in          | **2- NOT APPLICABLE**     |
|                |     calculating the    |                           |
|                |     overall            | **1- [Does Not Meet       |
|                |     performance level. | ]{.underline}**           |
|                |                        |                           |
|                | **2- NOT APPLICABLE**  | -   Retention measurement |
|                |                        |     collects and reports  |
|                | **1- [Does Not Meet    |     retention information |
|                | ]{.underline}**        |     for each type of      |
|                |                        |     retention but reports |
|                | -   Retention          |     only                  |
|                |     measurement        |                           |
|                |     collects and       |  [duplicated]{.underline} |
|                |     reports retention  |     counts in calculating |
|                |     information for    |     the overall           |
|                |     each type of       |     performance level.    |
|                |     retention but      |                           |
|                |     reports only       |                           |
|                |     [d                 |                           |
|                | uplicated]{.underline} |                           |
|                |     counts in          |                           |
|                |     calculating the    |                           |
|                |     overall            |                           |
|                |     performance level. |                           |
+----------------+------------------------+---------------------------+

#### 4S1: Participation in Secondary Non-Traditional Programs

**Measure**

**[Numerator]{.underline}**: Number of students in underrepresented
gender groups who participated in a non-traditional secondary program in
the reporting year.

**[Denominator]{.underline}**: Number of students who participated in a
non-traditional secondary education program in the reporting year.

[**Note**:]{.underline} Non-Traditional programs are those programs that
address occupational areas in which underrepresented gender groups
represent less than 25 percent of employment.

**Measurement Approach**

[**State/Local Administrative Data**]{.underline} --- Reporting those
students participating in non-traditional programs using information
from local administrative data. These data could be maintained as
individual student records and sent to the state as individual student
records or aggregated program or school-level data. The data also could
be maintained at the local level in aggregated form and sent to the
state as aggregated data. Although these represent distinct types of
reporting systems, they do not represent different performance
measurement approaches.

**Quality Criteria and Scoring Rubric**

The standards levels in the scoring rubrics are the
[proposed]{.underline} levels for use [only]{.underline} in state peer
reviews in the Data Quality Initiative. States will be asked to review
and suggest revisions after using them in the first peer review process.
[They are not federal requirements.]{.underline}

+--------------------+-------------------------------------------------+
| # QUALITY CRITERIA | **APPROACHES WITH STATE/LOCAL ADMINISTRATIVE    |
|                    | DATA SYSTEMS**                                  |
+--------------------+-------------------------------------------------+
| A.  **Accurate     | **3- [Standard]{.underline}**                   |
|     Classification |                                                 |
|     of Programs as | -   All programs in schools are directly        |
|                    |     aligned to the state crosswalk system.      |
|  Non-Traditional** |                                                 |
|                    | ```{=html}                                      |
| Non-Traditional    | <!-- -->                                        |
| programs are       | ```                                             |
| accurately defined | -   All programs in schools are regularly       |
| at the state and   |     evaluated on a statewide basis on their     |
| local levels based |     alignment to the state crosswalk system.    |
| on a state         |                                                 |
| crosswalk between  | -   Statewide policies and systems have been    |
| programs and       |     established to ensure that the              |
| national or state  |     classification system(s) used by all        |
| occupational data. |     schools are directly aligned to the state   |
|                    |     crosswalk system                            |
|                    |                                                 |
|                    | **2- [Satisfactory Progress]{.underline}**      |
|                    |                                                 |
|                    | -   Statewide policies and crosswalk systems    |
|                    |     have been established to ensure that the    |
|                    |     classification system(s) used by all        |
|                    |     schools are directly aligned to the state   |
|                    |     crosswalk system.                           |
|                    |                                                 |
|                    | **1- [Does Not Meet]{.underline}**              |
|                    |                                                 |
|                    | -   Statewide policies and crosswalk systems    |
|                    |     have [not]{.underline} been established to  |
|                    |     ensure that the classification system(s)    |
|                    |     used by schools are directly aligned to     |
|                    |     state crosswalk systems.                    |
+--------------------+-------------------------------------------------+
| B. Reliability of  | **3- [Standard]{.underline}**                   |
| Participation      |                                                 |
| Measurement        | -   All school classification and participation |
|                    |     reporting systems meet state policies and   |
| Student            |     system requirements.                        |
| participation of   |                                                 |
| non-traditional    | ```{=html}                                      |
| programs is        | <!-- -->                                        |
| measured based on  | ```                                             |
| consistent         | -   All classification and participation        |
| definitions of and |     reporting systems in all schools are        |
| criteria for       |     regularly audited based on state policies   |
| participation.     |     and system requirements.                    |
|                    |                                                 |
|                    | ```{=html}                                      |
|                    | <!-- -->                                        |
|                    | ```                                             |
|                    | -   Statewide policies and systems have been    |
|                    |     established to ensure that participation is |
|                    |     measured in all schools using standard      |
|                    |     definitions of participation and            |
|                    |     standardized procedures for reporting       |
|                    |     participation..                             |
|                    |                                                 |
|                    | **2- [Satisfactory Progress]{.underline}**      |
|                    |                                                 |
|                    | -   Statewide policies and systems have been    |
|                    |     established to ensure that participation is |
|                    |     measured in all schools using standard      |
|                    |     definitions of participation and            |
|                    |     standardized procedures for reporting       |
|                    |     participation..                             |
|                    |                                                 |
|                    | **1- [Does Not Meet]{.underline}**              |
|                    |                                                 |
|                    | -   Statewide policies and systems have         |
|                    |     [not]{.underline} been established to       |
|                    |     ensure that participation is measured in    |
|                    |     all schools using standard definitions of   |
|                    |     participation and standardized procedures   |
|                    |     for reporting participation                 |
+--------------------+-------------------------------------------------+
| **C. Student       | **3- [Standard]{.underline}**                   |
| Coverage in        |                                                 |
| Reporting          | -   Participation is reported by all schools    |
| Non-Traditional    |     for at least 90% of their vocational        |
| Programs**         |     participants in non-traditional programs.   |
|                    |                                                 |
| All secondary      | **2- [Satisfactory Progress]{.underline}**      |
| vocational         |                                                 |
| educational        | -   Participation is reported by all schools    |
| participants are   |     for at least 80% of their vocational        |
| measured and       |     participants in non-traditional programs.   |
| reported if they   |                                                 |
| participate in     | **1- [Does Not Meet]{.underline}**              |
| non-traditional    |                                                 |
| programs.          | -   Participation is reported by all schools or |
|                    |     some schools for less than 80% of their     |
|                    |     vocational participants in non-traditional  |
|                    |     programs.                                   |
+--------------------+-------------------------------------------------+

#### 4S2: Completion of Secondary Non-Traditional Programs

#### 

**Measure**

**[Numerator:]{.underline}** Number of students in underrepresented
gender groups who completed a non-traditional secondary program in the
reporting year.

**[Denominator]{.underline}:** Number of students who completed a
non-traditional secondary education program in the reporting year.

[**Not**e**:**]{.underline} Non-Traditional programs are those programs
that address occupation or occupational areas in which underrepresented
gender groups represent less than 25 percent of employment.

**Measurement Approaches**

**[State/Local Administrative Data]{.underline}** -- Reporting those
students completing non-traditional programs using information from
local administrative data. These data could be maintained as individual
student records and sent to the state as individual student records or
aggregated program or school-level data. The data also could be
maintained at the local level in aggregated form and sent to the state
as aggregated data. Although these represent distinct types of reporting
systems, they do not represent different performance measurement
approaches.

**Quality Criteria and Scoring Rubric**

The standards levels in the scoring rubrics are the
[proposed]{.underline} levels for use [only]{.underline} in state peer
reviews in the Data Quality Initiative. States will be asked to review
and suggest revisions after using them in the first peer review process.
[They are not federal requirements.]{.underline}

+--------------------+-------------------------------------------------+
| # QUALITY CRITERIA | **APPROACHES WITH STATE/LOCAL ADMINISTRATIVE    |
|                    | DATA SYSTEMS**                                  |
+--------------------+-------------------------------------------------+
| **A. Accurate      | **3- [Standard]{.underline}**                   |
| Classification of  |                                                 |
| Programs as        | -   All programs in schools are directly        |
| Non-Traditional**  |     aligned to the state crosswalk system.      |
|                    |                                                 |
| Non-Traditional    | ```{=html}                                      |
| programs are       | <!-- -->                                        |
| accurately defined | ```                                             |
| at the state and   | -   All programs in schools are regularly       |
| local levels based |     evaluated on a statewide basis on their     |
| on a state         |     alignment to the state crosswalk system.    |
| crosswalk between  |                                                 |
| programs and       | -   Statewide policies and systems have been    |
| national or state  |     established to ensure that the              |
| occupational data. |     classification system(s) used by all        |
|                    |     schools are directly aligned to the state   |
|                    |     crosswalk system                            |
|                    |                                                 |
|                    | **2- [Satisfactory Progress]{.underline}**      |
|                    |                                                 |
|                    | -   Statewide policies and crosswalk systems    |
|                    |     have been established to ensure that the    |
|                    |     classification system(s) used by all        |
|                    |     schools are directly aligned to the state   |
|                    |     crosswalk system.                           |
|                    |                                                 |
|                    | **1- [Does Not Meet]{.underline}**              |
|                    |                                                 |
|                    | -   Statewide policies and crosswalk systems    |
|                    |     have [not]{.underline} been established to  |
|                    |     ensure that the classification system(s)    |
|                    |     used by schools are directly aligned to     |
|                    |     state crosswalk systems.                    |
+--------------------+-------------------------------------------------+
| **B. Reliability   | **3- [Standard]{.underline}**                   |
| of Completion      |                                                 |
| Measurement**      | -   All school classification and completion    |
|                    |     reporting systems meet state policies and   |
| Student completion |     system requirements.                        |
| of non-traditional |                                                 |
| programs is        | ```{=html}                                      |
| measured based on  | <!-- -->                                        |
| consistent         | ```                                             |
| definitions of and | -   All classification and completion reporting |
| criteria for       |     systems in all schools are regularly        |
| completion of      |     audited based on state policies and system  |
| non-traditional    |     requirements.                               |
| programs.          |                                                 |
|                    | ```{=html}                                      |
|                    | <!-- -->                                        |
|                    | ```                                             |
|                    | -   Statewide policies and systems have been    |
|                    |     established to ensure that completion is    |
|                    |     measured in all schools using standard      |
|                    |     definitions of completion and standardized  |
|                    |     procedures for reporting completion.        |
|                    |                                                 |
|                    | **2- [Satisfactory Progress]{.underline}**      |
|                    |                                                 |
|                    | -   Statewide policies and systems have been    |
|                    |     established to ensure that completion is    |
|                    |     measured in all schools using standard      |
|                    |     definitions of completion and standardized  |
|                    |     procedures for reporting completion.        |
|                    |                                                 |
|                    | **1- [Does Not Meet]{.underline}**              |
|                    |                                                 |
|                    | -   Statewide policies and systems have         |
|                    |     [not]{.underline} been established to       |
|                    |     ensure that completion is measured in all   |
|                    |     schools using standard definitions of       |
|                    |     completion and standardized procedures for  |
|                    |     reporting completion.                       |
+--------------------+-------------------------------------------------+
| C. Student         | **3- [Standard]{.underline}**                   |
| Coverage in        |                                                 |
| Reporting          | -   Completion is reported by all schools for   |
| Non-Traditional    |     at least 90% of their vocational completers |
| Programs           |     in non-traditional programs.                |
|                    |                                                 |
| All students who   | **2- [Satisfactory Progress]{.underline}**      |
| complete           |                                                 |
| vocational         | -   Completion is reported by all schools for   |
| education programs |     at least 80% of their vocational completers |
| are measured and   |     in non-traditional programs.                |
| reported if they   |                                                 |
| are in             | **1- [Does Not Meet]{.underline}**              |
| non-traditional    |                                                 |
| programs.          | -   Completion is reported by all schools or    |
|                    |     some schools for less than 80% of their     |
|                    |     vocational completers in non-traditional    |
|                    |     programs.                                   |
+--------------------+-------------------------------------------------+

#### 4P1: Participation in Postsecondary Non-Traditional Programs

**Measure**

**[Numerator]{.underline}**: Number of students in underrepresented
gender groups who participated in a non-traditional postsecondary
program in the reporting year.

**[Denominator]{.underline}**: Number of students who participated in a
non-traditional postsecondary education program in the reporting year.

[**Note**:]{.underline} Non-Traditional programs are those programs that
address occupational areas in which underrepresented gender groups
represent less than 25 percent of employment.

**Measurement Approach**

[**State/Local Administrative Data**]{.underline} --- Reporting those
students participating in non-traditional programs using information
from local administrative data. These data could be maintained as
individual student records and sent to the state as individual student
records or aggregated program or school-level data. The data also could
be maintained at the local level in aggregated form and sent to the
state as aggregated data. Although these represent distinct types of
reporting systems, they do not represent different performance
measurement approaches.

[]{#__RefHeading___Toc504805106 .anchor}

**Quality Criteria and Scoring Rubric**

The standards levels in the scoring rubrics are the
[proposed]{.underline} levels for use [only]{.underline} in state peer
reviews in the Data Quality Initiative. States will be asked to review
and suggest revisions after using them in first peer review process.
They are not federal requirements.

+--------------------+-------------------------------------------------+
| # QUALITY CRITERIA | **APPROACHES WITH STATE/LOCAL ADMINISTRATIVE    |
|                    | DATA SYSTEMS**                                  |
+--------------------+-------------------------------------------------+
| A.  **Accurate     | **3- [Standard]{.underline}**                   |
|     Classification |                                                 |
|     of Programs as | -   All programs in colleges are directly       |
|                    |     aligned to the state crosswalk system.      |
|  Non-Traditional** |                                                 |
|                    | ```{=html}                                      |
| Non-Traditional    | <!-- -->                                        |
| programs are       | ```                                             |
| accurately defined | -   All programs in colleges are regularly      |
| at the state and   |     evaluated on a statewide basis on their     |
| local levels based |     alignment to the state crosswalk system.    |
| on a state         |                                                 |
| crosswalk between  | -   Statewide policies and systems have been    |
| programs and       |     established to ensure that the              |
| national or state  |     classification system(s) used by all        |
| occupational data. |     colleges are directly aligned to the state  |
|                    |     crosswalk system                            |
|                    |                                                 |
|                    | **2- [Satisfactory Progress]{.underline}**      |
|                    |                                                 |
|                    | -   Statewide policies and crosswalk systems    |
|                    |     have been established to ensure that the    |
|                    |     classification system(s) used by all        |
|                    |     colleges are directly aligned to the state  |
|                    |     crosswalk system.                           |
|                    |                                                 |
|                    | **1- [Does Not Meet]{.underline}**              |
|                    |                                                 |
|                    | -   Statewide policies and crosswalk systems    |
|                    |     have [not]{.underline} been established to  |
|                    |     ensure that the classification system(s)    |
|                    |     used by colleges are directly aligned to    |
|                    |     state crosswalk systems.                    |
+--------------------+-------------------------------------------------+
| B. Reliability of  | **3- [Standard]{.underline}**                   |
| Participation      |                                                 |
| Measurement        | -   All college classification and              |
|                    |     participation reporting systems meet state  |
| Student            |     policies and system requirements.           |
| participation of   |                                                 |
| non-traditional    | ```{=html}                                      |
| programs is        | <!-- -->                                        |
| measured based on  | ```                                             |
| consistent         | -   All classification and participation        |
| definitions of and |     reporting systems in all colleges are       |
| criteria for       |     regularly audited based on state policies   |
| participation.     |     and system requirements.                    |
|                    |                                                 |
|                    | ```{=html}                                      |
|                    | <!-- -->                                        |
|                    | ```                                             |
|                    | -   Statewide policies and systems have been    |
|                    |     established to ensure that participation is |
|                    |     measured in all colleges using standard     |
|                    |     definitions of participation and            |
|                    |     standardized procedures for reporting       |
|                    |     participation..                             |
|                    |                                                 |
|                    | **2- [Satisfactory Progress]{.underline}**      |
|                    |                                                 |
|                    | -   Statewide policies and systems have been    |
|                    |     established to ensure that participation is |
|                    |     measured in all colleges using standard     |
|                    |     definitions of participation and            |
|                    |     standardized procedures for reporting       |
|                    |     participation..                             |
|                    |                                                 |
|                    | **1- [Does Not Meet]{.underline}**              |
|                    |                                                 |
|                    | -   Statewide policies and systems have         |
|                    |     [not]{.underline} been established to       |
|                    |     ensure that participation is measured in    |
|                    |     all colleges using standard definitions of  |
|                    |     participation and standardized procedures   |
|                    |     for reporting participation                 |
+--------------------+-------------------------------------------------+
| **C. Student       | **3- [Standard]{.underline}**                   |
| Coverage in        |                                                 |
| Reporting          | -   Participation is reported by all colleges   |
| Non-Traditional    |     for at least 90% of their vocational        |
| Programs**         |     participants in non-traditional programs.   |
|                    |                                                 |
| All postsecondary  | **2- [Satisfactory Progress]{.underline}**      |
| vocational         |                                                 |
| educational        | -   Participation is reported by all colleges   |
| participants are   |     for at least 80% of their vocational        |
| measured and       |     participants in non-traditional programs.   |
| reported if they   |                                                 |
| participate in     | **1- [Does Not Meet]{.underline}**              |
| non-traditional    |                                                 |
| programs.          | -   Participation is reported by all college or |
|                    |     some colleges for less than 80% of their    |
|                    |     vocational participants in non-traditional  |
|                    |     programs.                                   |
+--------------------+-------------------------------------------------+

#### 4P2: Completion of Postsecondary Non-Traditional Programs

**Measure**

**[Numerator:]{.underline}** Number of students in underrepresented
gender groups who completed a non-traditional postsecondary program in
the reporting year.

**[Denominator]{.underline}:** Number of students who completed a
non-traditional postsecondary education program in the reporting year.

[**Not**e**:**]{.underline} Non-Traditional programs are those programs
that address occupation or occupational areas in which underrepresented
gender groups represent less than 25 percent of employment.

**Measurement Approach**

**[State/Local Administrative Data]{.underline}** -- Reporting those
students completing non-traditional programs using information from
local administrative data. These data could be maintained as individual
student records and sent to the state as individual student records or
aggregated program or school-level data. The data also could be
maintained at the local level in aggregated form and sent to the state
as aggregated data. Although these represent distinct types of reporting
systems, they do not represent different performance measurement
approaches.

**Quality Criteria and Scoring Rubric**

The standards levels in the scoring rubrics are the
[proposed]{.underline} levels for use [only]{.underline} in state peer
reviews in the Data Quality Initiative. States will be asked to review
and suggest revisions after using them in the first peer review process.
[They are not federal requirements.]{.underline}

+---------------------+------------------------------------------------+
| # QUALITY CRITERIA  | **APPROACHES WITH STATE/LOCAL ADMINISTRATIVE   |
|                     | DATA SYSTEMS**                                 |
+---------------------+------------------------------------------------+
| A.  **Accurate      | **3- [Standard]{.underline}**                  |
|     Classification  |                                                |
|     of Programs as  | -   All programs in colleges are directly      |
|                     |     aligned to the state crosswalk system.     |
|   Non-Traditional** |                                                |
|                     | ```{=html}                                     |
| Non-Traditional     | <!-- -->                                       |
| programs are        | ```                                            |
| accurately defined  | -   All programs in colleges are regularly     |
| at the state and    |     evaluated on a statewide basis on their    |
| local levels based  |     alignment to the state crosswalk system.   |
| on a state          |                                                |
| crosswalk between   | -   Statewide policies and systems have been   |
| programs and        |     established to ensure that the             |
| national or state   |     classification system(s) used by all       |
| occupational data.  |     colleges are directly aligned to the state |
|                     |     crosswalk system                           |
| .                   |                                                |
|                     | **2- [Satisfactory Progress]{.underline}**     |
|                     |                                                |
|                     | -   Statewide policies and crosswalk systems   |
|                     |     have been established to ensure that the   |
|                     |     classification system(s) used by all       |
|                     |     colleges are directly aligned to the state |
|                     |     crosswalk system.                          |
|                     |                                                |
|                     | **1- [Does Not Meet]{.underline}**             |
|                     |                                                |
|                     | -   Statewide policies and crosswalk systems   |
|                     |     have [not]{.underline} been established to |
|                     |     ensure that the classification system(s)   |
|                     |     used by colleges are directly aligned to   |
|                     |     state crosswalk systems.                   |
+---------------------+------------------------------------------------+
| B.  **Reliability   | **3- [Standard]{.underline}**                  |
|     of Completion   |                                                |
|     Measurement**   | -   All college classification and completion  |
|                     |     reporting systems meet state policies and  |
| Postsecondary       |     system requirements.                       |
| completion is       |                                                |
| measured based on   | ```{=html}                                     |
| consistent          | <!-- -->                                       |
| definitions of and  | ```                                            |
| criteria for        | -   All classification and completion          |
| completion of       |     reporting systems in all colleges are      |
| non-traditional     |     regularly audited based on state policies  |
| programs.           |     and system requirements.                   |
|                     |                                                |
|                     | ```{=html}                                     |
|                     | <!-- -->                                       |
|                     | ```                                            |
|                     | -   Statewide policies and systems have been   |
|                     |     established to ensure that completion is   |
|                     |     measured in all colleges using standard    |
|                     |     definitions of completion and standardized |
|                     |     procedures for reporting completion.       |
|                     |                                                |
|                     | **2- [Satisfactory Progress]{.underline}**     |
|                     |                                                |
|                     | -   Statewide policies and systems have been   |
|                     |     established to ensure that completion is   |
|                     |     measured in all colleges using standard    |
|                     |     definitions of completion and standardized |
|                     |     procedures for reporting completion.       |
|                     |                                                |
|                     | **1- [Does Not Meet]{.underline}**             |
|                     |                                                |
|                     | -   Statewide policies and systems have        |
|                     |     [not]{.underline} been established to      |
|                     |     ensure that completion is measured in all  |
|                     |     colleges using standard definitions of     |
|                     |     completion and standardized procedures for |
|                     |     reporting completion.                      |
+---------------------+------------------------------------------------+
| C. Student Coverage | **3- [Standard]{.underline}**                  |
| in Reporting        |                                                |
| Non-Traditional     | -   Completion is reported by all colleges for |
| Programs            |     at least 90% of their vocational           |
|                     |     completers in non-traditional programs.    |
| All students who    |                                                |
| complete vocational | **2- [Satisfactory Progress]{.underline}**     |
| education programs  |                                                |
| are measured and    | -   Completion is reported by all colleges for |
| reported if they    |     at least 80% of their vocational           |
| are in              |     completers in non-traditional programs.    |
| non-traditional     |                                                |
| programs.           | **1- [Does Not Meet]{.underline}**             |
|                     |                                                |
|                     | -   Completion is reported by all colleges or  |
|                     |     some colleges for less than 80% of their   |
|                     |     vocational completers in non-traditional   |
|                     |     programs.                                  |
+---------------------+------------------------------------------------+