# Presentation: 233016

## CFSR Technical Bulletin #1

- Implementation Training

## Topics

- Background and Considerations
- Evaluating Achievement of PIP Goals
- ACF’s Approach to the Second Round

## Background and Considerations

## Regulatory Requirements

- A State found not to be operating in substantial conformity during an initial or subsequent review will....Begin a full review two years after the approval of the program improvement plan.
- 	Source: 45 CFR 1355.32 (b)(2)

##  Evaluating PIP ImprovementAnd Scheduling Reviews

- The total PIP evaluation period will be the two-year implementation period of the approved PIP followed by a one-year evaluative period that does not overlap with the PIP implementation period.
- The one-year non-overlapping period will also form the baseline for the next round of CFSRs

## Evaluation Period Rationale

- ACF used a 12-month period to establish the national standards and will use a similar period to evaluate States’ improvement
- ACF used a 12-month period to make the initial determination of conformity and will use a similar period to evaluate progress on agreed-upon amount of improvement (AAI)
- Seasonality issues preclude using an evaluation period of less than 12 months
- ACF wants to assure that States have the full two-year PIP period to implement programmatic changes

## Data Considerations

- Availability of data from the two Federal data sources used in the CFSR
- AFCARS
- NCANDS

## AFCARS

- Data are reported twice a year--every 6 months on a _Federal fiscal year basis_
- October 1 - March 31 (report period A) submitted by May 15
- April 1 - September 30 (report period B) submitted by November 14
- One record for each child served by the foster care system during the 6-month reporting period
- Reporting by the States is mandatory
- AFCARS consists of 2 databases—foster care and adoption; CFSR uses the foster care database for measurement of statewide data indicators

## NCANDS

- Submission by States is voluntary
- Reported annually on a _Federal fiscal year basis_ (was calendar year before 2003)
- States may submit child abuse and neglect information at different levels of detail
- Child File (CF) (44 States in 2004) or
- Summary Data Component (SDC)
- Note: Only the child file can be used to calculate the safety data indicators for CFSR

## Permanency Evaluation Period

- Annual files based on rolling six month AFCARS submissions
- Starting with the next full AFCARS six-month period immediately following the end of the two-year PIP implementation period ending March 31 or September 30
- Annual Files constructed to parallel State quarterly reporting periods

## Evaluation Period Example

- Initial Onsite Review – 2/15/2004

- Final Report Issued – 4/30/2004

- PIP Approval – 7/3/2004

- PIP Completion – 7/2/2006

- End of Evaluation Period – 9/30/2007

- Start of Evaluation Period – 10/1/2006

- One-year         Non-overlapping Period

- PIP Implementation Period

- 6

## Alternative Permanency Data Sources

- Is there any provision for continued use of alternative permanency data sources?
- No.  There is no longer any provision for the use of alternative permanency data sources.  AFCARS data must be submitted for determination of progress on the permanency data indicators.

## File Construction

- Will annual files continue to be constructed based on three AFCARS files?
- No.  In order to avoid undue delay, data for the PIP evaluation period will be compiled into an annual file using _two_ six-month AFCARS files.

## Safety Evaluation Period

- NCANDS or approved alternative source with proper documentation
- NCANDS child files for 12 month period corresponding to when AAI was achieved
- For period ending September 30, submission due by January 1
- For period ending March 31, submission due by July 1

## Alternative Safety Data Sources

- May States continue to submit alternative safety data sources if they do not submit the Child File or cannot use their submissions to calculate one of the safety data indicators?
- Yes.  There is still a provision for States to propose alternative safety data sources.  Such sources must be approved in the PIP. States should ensure the quality and accuracy of the data and their calculations.  Documentation should include sample size issues, databases, and methodology used in the calculations.  Further, States should indicate database quality and complete ACF data templates to allow for independent validation of the calculations by ACF.

## What’s Different?

- Aren’t States submitting AFCARS and NCANDS just like they did last time?
- Data from AFCARS and NCANDS are being used like last time, however NCANDS data will have to be submitted on a one-time “out of schedule” basis to allow for one year of _synchronized_, non-overlapping data on both permanency and safety or submitted quarterly to allow for verification of AAI of safety data indicators.

## Who Receives the Submissions?

- When States submit an “out of schedule” NCANDS file, who should receive the submission?
- The State should continue to submit their files to WRMA, utilizing their regular contact.

## Additional Submission Status

- Does the submission of an “out of schedule” file remove the need to submit the next regularly scheduled NCANDS file?
- No.  States should continue to submit their regular NCANDS files according to the regular schedule.

## Evaluating Achievement of PIP Goals

## Attainment of PIP Goals

- Three Possible Scenarios
- During Implementation
- Goals are met prior to PIP closeout
- Goal due dates in the PIP prior to closeout
- PIP Conclusion
- Goals are met at the time that the two-year PIP period ends
- After PIP Through Non-overlapping Period
- Goals are met during the period between the conclusion of the PIP and the end of non-overlapping year

## Example: Measurement of PIP Progress

- The State X PIP includes action steps and measurement goals in the following areas:
- Maltreatment in Foster Care (safety data indicator)
- Placement Stability (permanency data indicator)
- Adequate Assessment of Child’s Needs (Outcome WB1, Item 17)
- Frequency and Quality of Worker Visits with Child (Outcome WB1, Item 19)

## Example Timelines

- PIP Commences – 4/1/2003

- Q1 Ends – 6/30/2003

- End of Evaluation Period – 3/31/2006

- One-year         Non-overlapping Period

- Q2 Ends – 9/30/2003

- Q3 Ends – 12/31/2003

- Q4 Ends – 3/31/2004

- Q5 Ends – 6/30/2004

- Q6 Ends – 9/30/2004

- Q7 Ends – 12/31/2004

-       Q8 Ends – 3/31/2005
-          Start of Evaluation Period – 4/1/2005

- PIP Implementation Period

- 22

## Example Timelines

- PIP Commences: April 1, 2003
- Q1 Ends: June 30, 2003
- Q2 Ends: September 30, 2003
- Q3 Ends: December 31, 2003
- Q4 Ends: March 31, 2004
- Q5 Ends: June 30, 2004
- Q6 Ends: September 30, 2004
- Q7 Ends: December 31, 2004
- Q8 Ends: March 31, 2005
- PIP Evaluation Period Ends: March 31, 2006

## Closure Prior to Completion of Two-year PIP Implementation Period

- Maltreatment in Foster Care (April 1, 2004) Data Indicator
- Quarterly data reporting occurs using 12 months of data on a rolling quarter basis.
- 90 days after the end of Q4, State X submits four quarters of NCANDS data ending on March 31, 2004 that show they have met the AAI for the safety data indicator.
- The indicator is considered to be met and closed after concurrence by ACF.

## Closure at the Conclusion of the Two-year PIP Implementation Period

- Frequency and quality of worker visits with child (March 31, 2005).
- At the end of Q8, State X has produced two consecutive quarters (as allowed in negotiated PIP) of qualitative case review data showing that they have met the AAI for Item 19.
- The item is considered met and closed after concurrence by ACF

## Closure After PIP Implementation During the Evaluation Period

- Placement Stability Data Indicator (September 30, 2005)
- State X has completed all action steps and strategies by the end of the PIP implementation period, but does not reach the AAI and continues to monitor its data throughout the PIP evaluation period.
- Six months into the PIP evaluation period, State X submits a six-month AFCARS file and when _combined_ by CB with the prior AFCARS file it indicates they have met the AAI for the Placement Stability data indicator.
- The data indicator is considered met and closed after concurrence by ACF.

## Closure at the End of the Twelve- month PIP Evaluation Period

- Item 17 – Needs and Services of Child, Parents, and Foster Parents (March 31, 2006)
- State X has completed all action steps by the end of the PIP implementation period, but continues to monitor progress with quarterly qualitative case reviews throughout the PIP evaluation period.
- State X submits four quarters of qualitative case review data at the end of the PIP evaluation period showing that the AAI was achieved for Item 17 for two consecutive quarters (as specified in PIP).
- The item is considered met and closed after concurrence by ACF.

## To Summarize: Evaluating AAI For Data Indicators

- 12 Month Implementation Period (Unless PIP specifies different)
- States may submit 2 consecutive AFCARS files and/or 4 quarters of NCANDS data (or approved alternate safety data source)
- ACF will construct rolling annual files, when requested, based on 2 AFCARS submissions or combination of quarters from 3 AFCARS submissions
- States may generate 12 month source files for evaluating AAI - data should be verified and/or reproduced by ACF
- 12-month non-overlapping period – AFCARS(AB) or (BA) and NCANDS data for same period

## Timely Submission of Data

- Why is there such an emphasis on timely submission of data?
- Since ACF has determined that we will wait for one year of non-overlapping data for final PIP closeout and commencement of the next CFSR, additional time spent resolving issues with the data would unduly delay the next review.

## Timely Data Cleanup

- Can States still resubmit data upon completion of cleanup activities?
- Yes, but ACF expects that any concerns regarding the quality and accuracy of the data will be identified and resolved prior to the end of the non-overlapping twelve-month period.
- Any resubmissions should be received by the time the submission for the non-overlapping period is due in order to not delay start of the second CFSR or closeout of the PIP

## Available Assistance

- What assistance is available to States to ensure timely data cleanup?
- ACF Regional Office staff, in conjunction with the staff of the National Resource Center for Child Welfare Data and Technology (NRC-CWDT) as appropriate, will work proactively with States to identify and resolve any data issues in a timely manner.

## ACF’s Approach to the Scheduling and Data Profiles for Second Round of CFSRs

## Scheduling

- Data profile based on one year of non-overlapping data beginning with next full AFCARS six-month period immediately following the end of the two-year PIP implementation period
- States asked to make additional NCANDS submission by January 1st and July 1st
- Statewide assessment at earliest possible date following non-overlapping year and time for submissions and preparation of data profile
- Coordinated with Regional Office in a manner that allows ACF to manage the review process effectively

## Timeframes For Data Profiles

| PIP Completion Dates | Oct. 1 to March 31 | April 1 to Sept. 30 |
| --- | --- | --- |
| 12-month non-overlapping period start | April 1 | October 1 |
| 12-month non-overlapping period end | March 31 of following year | September 30 of following year |
| AFCARS Annual File | B + A | A + B |
| NCANDS or Approved Alternative Due | July 1 | January 1 of following year |
| Start Date of Next Statewide Assessment | After July 1 | After January 1 of the following year |
| Start Date of Next Onsite CFSR | After January 1 of the following year | After July 1 of the following year |

## Example: State of Y

- Initial Onsite Review
- 5/14/03

- Final Report Issued
- 8/10/03

- PIP Approval
- 12/7/03

- PIP Completion
- 12/6/05

- Data Period for Final Evaluation and Next CFSR
- 4/1/06 – 3/31/07
- AFCARS 06B + 07A
- Safety Data Due 7/1/07

- Next Statewide Assessment
- July 1, 2007 or later

- Next Onsite CFSR
- January 1, 2008 or later

## Example: State of Y

- Initial Onsite Review – 5/14/2003

- Final Report Issued – 8/10/2003

- PIP Approval – 12/7/2003

- PIP Completion – 12/6/2005

- End of Evaluation Period – 3/31/2007

- Start of Evaluation Period – 4/1/2006

- One-year         Non-overlapping Period

- Next Statewide Assessment – 7/1/2007 or later

- Next Onsite CFSR – 1/1/2008 or later

- 37

## Additional Resources

- Your ACF Regional Office
- National Resource Center for Child Welfare Data and Technology (NRC-CWDT)
- Toll Free: (877) 672-4829
- Email: _[nrccwdt@cwla.org](mailto:nrccwdt@cwla.org)_
- Web: _[www.nrccwdt.org](http://www.nrccwdt.org/)_