Source: http://www.neglected-delinquent.org/title-i-part-d-data-context-and-methodology
Timestamp: 2016-10-01 20:36:05
Document Index: 749815925

Matched Legal Cases: ['art 3', 'art 1', 'art 1', 'art 2', 'art 2', 'art 1', 'art 1', 'art 2', 'art 2']

Title I, Part D Data Context and Methodology | NDTAC: Technical Assistance Center for the Education of Youth Who Are Neglected, Delinquent, or At-Risk
Title I, Part D Data Context and Methodology You are hereHome This Web page provides information regarding the Title I, Part D (Part D) data collected by States through the Consolidated State Performance Report (CSPR) and also details the methodology the National Evaluation and Technical Assistance Center for Children and Youth Who Are Neglected, Delinquent, or At-Risk (NDTAC) uses for calculating the Part D results presented in the Annual Performance Overview, National and State Fast Facts, and other NDTAC presentations of Part D CSPR data.
Methodology for NDTAC Data Analysis
Student Counts Used in SY 2012–13 Annual Performance Overview Calculations
The following section provides contextual information useful for understanding Part D data. Federal Data Collection Requirement and Access to Data
States are required to report data from juvenile justice, neglect, and at-risk programs and facilities that receive Title I, Part D (Part D) funding to the U.S. Department of Education (ED) annually through the Consolidated State Performance Report (CSPR) as required by Subpart 3 of the Part D statute (Section 1431(d) (1)). ED provides access to Part D data online through ED Data Express, and copies of each State’s CSPR reports are posted on ED’s Web site, when available. In addition, the National Evaluation and Technical Assistance Center for Children and Youth Who Are Neglected, Delinquent, or At-Risk (NDTAC) analyzes these data and presents findings in the Annual Performance Overview and online on the National and State Fast Facts pages.
Assumptions and Limitations of Part D Data
Part D provides supplemental education funds to States, which are intended to assist in improving educational and other outcomes for youth who are neglected or delinquent. Given the supplemental nature of the funds and the flexibility in the way that States may elect to target these funds across programs (e.g., tutoring, mentoring, supplemental materials, family engagement activities), the academic achievement and outcome results demonstrated by students are not solely attributable to Part D funding.
State data are not reported to ED at the student level and as a result, outcomes cannot be examined by subgroups such as age, race/ethnicity, gender, limited English proficiency (LEP), or disability status.
Academic and Vocational Outcomes
Academic and vocational outcomes are collected and reported for all students who benefit from Part D funding, regardless of their length of stay in the programs or facilities. There have been changes to the academic and vocational outcome indicators in recent years, which may impact interpretation of longitudinal data (additional detail is provided under Recent Changes to the Part D Indicators).
Pretest and Posttest Results in Reading and Mathematics
Students are not always enrolled in a Part D facility or program when State assessments are administered and often are not enrolled for a full school year, making traditional methods for tracking progress difficult. Therefore, for Federal reporting, Part D programs monitor student progress in reading and mathematics by administering pretests upon entry and posttests prior to their exit. Results from pre- and posttests are reported only for long-term students, defined as students who have a length of stay of 90 consecutive days or longer.
Part D programs measure progress in reading and mathematics using the pre-posttesting instruments of their choice. As a result, there are a variety of reading and mathematics assessments used across Part D programs.
Recent Changes to the Part D Indicators
There have been several changes to the Part D collection that may impact interpretation of longitudinal data, especially for comparing recent data to academic and vocational outcomes collected prior to SY 2011–12.
SY 2012–13
The following items were added to the CSPR collection for SY 2012–13:
The number of students who have limited English proficiency (LEP)
The number of students receiving transition services
Information on whether facilities/programs are able to provide data on students after exit (in a “yes/no” format)
The following items were modified for SY 2012–13:
Academic and vocational outcomes:
All previous academic and vocational outcomes were disaggregated and reported separately as (1) those achieved during the enrollment period and (2) those achieved up to 90 days after exit.*
In addition, the transition period was extended from up to 30 days after exit to up to 90 days after exit (as noted above).
With this change, for most outcomes, students can now be counted as achieving the same outcome once while in the facility and then again up to 90 days after exit. Exceptions include indicators that capture one-time events, such as enrolled in a local district school, earned a GED, and obtained a high school diploma.
*Data for outcomes achieved up to 90 days after exit may be incomplete, as States and districts had varying abilities to track students after they transitioned back to their communities.
Pre- and posttests in reading and math:
Two of the improvement levels for reading and mathematics—“up to one 1/2 grade level” and “1/2 to one full grade level”—were merged into a single level, “up to one full grade level.”
SY 2011–12
The following items were removed for SY 2011–12:
Total average length of stay for each Subpart
The number of programs in a multiple purpose facility
Academic offerings at programs/facilities
The following items were modified for SY 2011–12:
Two postsecondary outcomes, “accepted into postsecondary education” and “enrolled in postsecondary education,” were combined into a single outcome, “accepted and/or enrolled in postsecondary education.” Two vocational outcomes, “enrolled in elective job training” and “enrolled in external job training,” were combined into the single vocational training indicator, “enrolled in job-training courses/programs.” Reporting the following items became optional for at-risk programs:
Number of long-term students served
Academic achievement (reading and mathematics) data
Information on changes to the Part D CSPR collection for years prior to SY 2011–12 is available in NDTAC’s Annual Program Performance Reports (found on the National and State Reports & Summaries Web page).
The following section provides general information about the National Evaluation and Technical Assistance Center for Children and Youth Who Are Neglected, Delinquent, or At-Risk’s (NDTAC’s) analysis of Title I, Part D (Part D) Consolidated State Performance Report (CSPR) data and specific details about analyses for SYs 2011–12 and 2012–13. Information about the analyses for SY 2010–11 and prior are available in NDTAC’s Annual Program Performance Reports (found on the National and State Reports & Summaries Web page).
Policies Protecting Student Privacy
In order to protect personally identifiable information, the U.S. Department of Education (ED) implements data suppression on student outcomes submitted through the CSPR. These rules apply to the academic outcomes, vocational outcomes, and reading and mathematics data for Part D students. (For more information on the Federal suppression rules and process, please see the ED Data Express FAQ Web page.)
Given these Federal suppression standards, Part D outcomes by State (e.g., academic outcomes, vocational outcomes, and reading and mathematics data) for SYs 2011–12 and 2012–13 are not posted on NDTAC’s Fast Facts pages.
Academic and Vocational Outcome Calculations and Age Eligibility Estimations
Because student-level data are not available, the academic and vocational outcomes are examined in the context of student eligibility to attain each outcome based on age. For the purpose of these analyses, ED has determined age ranges for each outcome based on the ages at which a student could reasonably be expected to achieve each outcome. For example, elementary-school-aged students are not eligible to earn high school course credits. Therefore, the percentage of students who earned high school course credits is estimated by using the number of students who were reported between 13 and 21 years old.
The age-eligible ranges were developed to standardize analyses across States and are assumed to be the common age range across most States for which students can reasonably attain an outcome. However, age eligibility for each outcome may vary by State and program. Therefore, States are encouraged to provide information in the CSPR describing age-eligibility requirements for their programs. Table 1 presents the age range specific to each academic and vocational outcome.
Table 1: Age-Eligibility Ranges and Type of Calculation by Outcome
13–21 years old
Calculated for in facility and after exit in 2012–13
Enrolled in a GED Program
Enrolled in Local District School
6–21 years old
Accepted or Enrolled in Postsecondary Education
Enrolled in Job Training Courses/Programs
Combined into a single indicator and timeframe
In SY 2012–13, as detailed previously, the academic and vocational outcomes were disaggregated to be reported as those achieved while in the facility and those achieved up to 90 days after exiting the facility. For the majority of the indicators, students can be reported once in each of these two time periods, and as such, the data are reported for each time period separately, as noted in Table 1 above and described below.
Earning a GED or high school diploma: Earning a high school diploma and/or a GED are combined together into a single indicator in order to align with the Federal Government Performance and Results Act (GPRA) Modernization Act of 2010 (GPRAMA) reporting measure. Both reflect achievements that can occur only one time, whether they occur while in the facility or after exit. In order to more closely compare this measure longitudinally over time, the values have also been summed across the two time periods for the National calculations provided.
Enrolled in local school district: In general, the indicator for students returning to school is a one-time outcome. In SY 2012–13, a number of States interpreted the disaggregation of this indicator as representing different information and it was inappropriate to combine the data across the time periods. As a result, for this first year, the data are presented separately for in facility and after exit to reflect this interpretation.
Academic Achievement Calculations for Reading and Mathematics
The percentage of students who improved from pretest to posttest in reading and mathematics is based on the number of long-term students (those enrolled for 90 consecutive days or more) with complete pretest and posttest data. Complete pretest and posttest data is calculated as the sum of students reported in each of the four results categories (negative grade level change, no grade level change, improvement up to one full grade level, and improvement of more than one full grade level).
NDTAC Data Cleaning and Adjustments
The following outlines the process by which the data NDTAC prepares may differ from the raw files provided by States.
State Comments: Within the CSPR, States may enter comments with their submission to further explain their data (such as regarding age eligibility) or to provide information about a problem related to data entry that could not be resolved during submission and impacted the manner in which their data may be presented. NDTAC reviews these comments and updates the dataset to reflect these notes, if an adjustment is necessary.
Data Quality: NDTAC may modify data when a data quality problem has been clearly identified and can be resolved without confirmation from the State, for example, when a data point or set of data points has been accidentally entered under the wrong program type (i.e., a portion of the juvenile detention data was entered in the neglect data column when no such programs were funded). Racial/Ethnic Category: NDTAC expands the Racial/Ethnic categories to include an “other” group. This category includes any student whose race/ethnicity was unidentifiable at the time of data collection and not reported or who refused to disclose his or her race or ethnicity, as noted by the State in comments.
Tables 2 and 3 present State data adjusted for the SY 2012–13 and SY 2011–12 National outcomes. Table 2: Adjustments to State Data in the SY 2012–13 Collection*
Reason for Adjustment in SY 2012–13
Idaho, Kansas, Massachusetts
These States reported that all students in Subpart 1 earned high school course credits in facility, which included students who were outside the determined age-eligible range for that outcome. For the purpose of these calculations, the number of students who achieved the outcome was adjusted to the total number of students in the age-eligible range.
This State provided updated data on outcomes achieved up to 90 days after exit after the CSPR closed. The data were changed to reflect these new data. *Adjustments based on State comments are not included in this table.
Table 3: Adjustments to State Data in the SY 2011–12 Collection*
Connecticut, Idaho, Massachusetts, Nevada
These States reported that all students in Subpart 1 earned high school course credits, which included students who were outside the determined age-eligible range for that outcome. For the purpose of these calculations, the number of students who achieved the outcome was adjusted to the total number of students in the age-eligible range.
*Adjustments based on State comments are not included in this table.
Exclusions of Data
There are some circumstances in which State data may not be included in the calculation of a particular indicator, including when
State-reported data exhibits quality issues or inconsistencies, such as reporting a greater number of students as having achieved an outcome than the total number of students served.
An indicator is not applicable for a State because, in some instances, a State may not be able to collect certain data due to State laws, policies, or regulations or the practices of a State education agency, local education agency, or State agency. For example, this can occur in States where the law prohibits the collection of information on students after they leave a facility.
A State did not report average length-of-stay values or related student counts (applies only to length-of-stay calculations).
In these instances, the related data (if applicable) would be removed from the calculation. All data are reviewed to determine if there is a reason to exclude data from any State in any calculation.
Academic and Vocational Outcomes in SY 2012–13
SY 2012–13 marked the first year in which States were asked to report on outcomes up to 90 days after exit. Due to the complexities in tracking students after exit, in this first year, many States were unable to provide these data or were only able to provide partial data for some facilities or some program types. Some States were prohibited from collecting data due to policies at the State level. In order to prevent a bias by including denominators (student counts) from States that could not provide data, in this first collection the following exclusionary approach was taken, per indicator:
If a State reported zeros or blank/no data across all program types (such that the sum across programs was zero or null), the State was removed from the calculations for that indicator. NDTAC expects to revisit this methodology in future years as State data systems expand to collect more thorough transition data.
Tables 4 and 5 present State data that were excluded from National calculations for SY 2012–13. Table 5 presents data that were excluded from National calculations for SY 2011–12.
Table 4: State Data Excluded From SY 2012–13 National Calculations
New Mexico (neglect and juvenile correction programs)
The average length of stay for programs was unavailable. Earned High School Course Credits
New York’s Title I, Part D programs are not accredited to award high school course credit to students. Subpart 2
California (at-risk, neglect, and juvenile detention programs), New Mexico (other programs)
The average length of stay for programs was unavailable. Testing Below Grade Level Upon Entry in Reading
The number of students testing below grade level upon entry in reading was greater than the number of long-term students.
Table 5: State Data Excluded From SY 2012–13 National Calculations—Academic and Vocational Outcomes After Exit
AL, AZ, AR, CA, CO, CT, GA, ID, IL, IN, KS, MA, MS, MO, NV, NJ, NY, NC, OH, OK, PR, RI, SC, SD, TN, TX, VT, WI Enrolled in a GED Program
AL, AK, AZ, AR, CA, CO, CT, FL, GA, HI, ID, IL, IN, KS, ME, MA, MN, NE, NV, NH, NY, NC, OK, PR, RI, SC, TN, VT, WI Enrolled in Local District School
AZ, AR, CO, DE, GA, IL, MA, MO, NV, NM, NY, NC, OK, SC, SD, TN, VT, WI
AK, AZ, AR, CA, CO, DE, GA, HI, IL, IN, IA, KS, ME, MA, MI, MO, NV, NY, NC, OK, RI, SD, VT, WI
AZ, AR, CA, CO, CT, DE, GA, HI, ID, IL, IN, IA, KS, ME, MA, MI, MS, MT, NE, NV, NY, NC, OK, PR, RI, SC, TN, VA, WV, WI
AZ, AR, CA, CT, DE, GA, IL, IA, KS, ME, MA, MN, NV, NJ, NM, NY, NC, OK, RI, SC, TN, VT, WI
AL, AR, CA, CT, GA, MA, MS, NJ, NC, OR, SC, VT, WI
AL, AK, AZ, AR, CA, CT, GA, ME, MA, MS, NJ, NC, OK, OR, VT, WI
AR, CA, GA, MA, NC, SC, VT, WI
AR, CA, CT, GA, IL, ME, MA, MS, MT, NE, NV, NH, NJ, NC, SC, VT, WV, WI
AK, AR, CA, GA, IL, IN, IA, ME, MA, MT, NE, NJ, NM, NC, SC, SD, VT, WI
AR, CA, GA, IL, ME, MA, NE, NV, NJ, NC, SD, VT, WI
Table 6: State Data Excluded From SY 2011–12 National Calculations
California (at-risk, neglect, and juvenile detention programs)
The average length of stay for programs was unavailable. Subpart 2
New York’s Title I, Part D programs are not accredited to award high school course credit to students. Student Counts Used in SY 2012–13 Annual Performance Overview Calculations
The tables below provide the student counts (denominators) used in the analyses of Title I, Part D data in the SY 2012–13 Annual Performance Overview.
Table 7: Subpart 1 Student Count Denominators
Total Number of Students with Age Data
Total Number of Students with Racial/Ethnic Data
Total Number of Students with Gender Data
Pre-Post Assessments (Long-Term Students)
Total Number of Long-Term (LT) Students
Total Number of LT Students with Complete Pre-Posttest Data in Reading
Total Number of LT Students with Complete Pre-Posttest Data in Mathematics
Age-Eligible Groups—In Facility or One-Time Outcomes*
6–21 year olds
13–21 year olds**
14–21 year olds
16–21 year olds
*N’s apply to all academic and vocational outcome indicators reported in facility or one-time outcomes (e.g., HS diploma/GED). **Students in Adult Corrections are not included.
Table 8: Subpart 1 Student Count Denominators for Academic and Vocational Outcomes Achieved After Exit
Earned High School Course Credits (13–21 Years)**
Enrolled in a GED Program (14–21 Years)
Enrolled in Local District School (6–21 Years)
Accepted or Enrolled in Postsecondary Education (16–21 Years)
Enrolled in Job Training Courses/Programs (16–21 Years)
Obtained Employment (14–21 Years)
**Students in Adult Corrections are not included.
Table 9: Subpart 2 Student Count Denominators
13–21 year olds
*N’s apply to all academic and vocational outcome indicators reported in facility or one-time outcomes (e.g., HS diploma/GED).
Table 10: Subpart 2 Student Counts for Outcomes Achieved After Exit
Earned High School Course Credits (13–21 Years)
ED's CSPR Web page
ED's ED Data Express
NDTAC's What is Title I, Part D? Web page
NDTAC's Annual Performance Overviews
NDTAC's State and National Fast Facts
NDTAC’s National and State Reports & Summaries Web page
For further clarification on any of the information presented above, contact your State liaison. Last updated March 2015