1

California

Part C On-Site Visit

June 13-15, 2017

DMS Area: Results/SSIP DMS Designation: Intensive

Background: 

On October 31, 2016, OSEP issued the Department of Developmental Services (DDS) a Differentiated 

Monitoring and Support (DMS) document that specified engagement activities that OSEP would 

conduct with the State across five areas: results, compliance, State Systemic Improvement Plan 

(SSIP), child find, and fiscal. Under Results, OSEP gave CA a designation of intensive engagement 

for performance outcomes for infants and toddlers in the areas of positive social relationships, 

knowledge and skill and actions to meet needs based on the FFY 2014 data reported in the State’s 

Annual Performance Report (APR). A designation of intensive was given for the SSIP based on a late 

submission of the Phase II plan, missing FFY 2014 data for the State-identified Measurable Result 

(SiMR), as well as missing or incomplete information in the plan for the requirement components of 

evidence-based practices, evaluation and stakeholder engagement. In March 2017, OSEP reviewed a 

draft of the Phase III, Year 1 SSIP submission, due on April 3, 2017 and provided feedback to CA 

Part C staff on some of the content and level of detail included in the report. OSEP leadership 

determined that an on-site visit would be an effective engagement activity to support the efforts of the 

early intervention program to improve child outcomes for infants and toddlers with disabilities and 

their families. The final site visit agenda included a focus on the SSIP.

Visit Summary

OSEP engaged with representatives of CA Part C’s leadership to discuss the strategies and activities of

the Part C program to implement and evaluate the SSIP and agree on next steps specific to the three 

child outcomes for results. The CA officials that provided information and responded to questions from 

OSEP included staff from DDS. Also participating in the discussion were staff from two technical 

assistance (TA) centers: the National Center for Systemic Improvement (NCSI), and a TA provider that 

works closely with the State.

Data Reviewed

Child outcomes data submitted for FFY 2015 could not be discussed during the site visit because the 

SPP/APR was under review by OSEP staff. The Part C coordinator and data manager indicated that the 

data for child outcomes has improved since the FFY 2014 SPP/APR. The Part C data manager has been 

working to improve the State’s data collection and analysis process and indicated she would like to 

follow-up with OSEP once determination letters are issued to discuss this work. The State’s Phase III, 

Year 1 SSIP submitted on April 3, 2017 was the primary focus of the results and SSIP discussion. 

Topics Discussed:

SSIP- Phase III, Year 1 Report

The Phase III SSIP submission was reviewed by OSEP staff prior to the site visit and feedback provided 

during the monitoring and support activity. During Phase III, Year 1 CA developed resources such as 

training modules and provider checklists consistent with the Phase II plan and implemented most of the 

intended activities . The SSIP included a report of activities and corresponding outputs for each of the 

three strands that comprise the SSIP logic model and align with its theory of action. Although 

milestones identified in the logic model were not met, CA began implementation of planned activities in 
2

Cohort 1 and collected data to inform the implementation of activities in Cohort 2 in Phase III, Year 2. 

There were references to stakeholder groups and the roles and responsibilities of stakeholders during the 

first year of SSIP implementation. The data collected during Phase III, Year 1 was typically obtained via

staff and family surveys or program record review. CA described the formation of implementation 

teams and an intention to align the SSIP with other state initiatives to support the professional 

development activities specific to social-emotional outcomes which is the SiMR focus for the SSIP. 

Challenges and Barriers

CA’s intensive designation for the SSIP was the result of two factors. First, the SSIP, due on April 1, 

2016, was not submitted until April, 25, 2016. Second, the SSIP did not include FFY 2014 data for the 

SiMR, as required. The State reported that steps have been identified and implemented to ensure future 

submissions are submitted on-time. The Phase III submission was received on-time in April, 2017 and 

FFY 2015 data for the SiMR was included. The SSIP report for Phase III, Year 1 reflected delays 

implementing the timelines for SSIP activities that impacted the State’s collection and analysis of data 

for most activities. The SSIP included information on the trainings that were held throughout the State. 

However, the SSIP did not include data on how training participants used new knowledge or skills in 

early intervention programs. Nor did the SSIP include information specific to fidelity assessment for the 

selected evidence-based practices. It was unclear as to what data will be collected on provider and 

parent behaviors or practices that are intended to impact social-emotional outcomes. A concern is that 

data for parent and provider practices will not be collected until Phase III, Year 2 and not allow for 

adjustments or supports before Phase III, Year 4 when the final SiMR target will need to be met. The CA 

Part C office discussed challenges with data collection due to strong local control of regional centers and 

different procedures that occur at the local level. Additional barriers identified by the State included 

limited funding for providers to attend trainings, limited provider participation in trainings due to large 

or remote catchment areas, and challenges communicating SSIP expectations and requirements between 

the state office and the local implementation sites.

Outcome of Engagement Activity

The FFY 2015 SPP/APR determination letter was issued after OSEP’s on-site visit, allowing OSEP to 

discuss the most recent data. OSEP has begun discussions with CA around follow-up activities specific 

to improving child outcomes and performance for positive social relationships, knowledge and skills, 

and the ability to meet needs. The SSIP Phase III, Year 2 activities are underway. OSEP staff 

encouraged the Part C coordinator and TA center representatives to collect and report relevant data on 

the professional development activities and efforts to assess provider knowledge and use of the training 

content. Two requirements in the measurement language for the indicator are to report progress 

implementing the SSIP’s activities and coherent improvement strategies and progress toward meeting 

the SiMR. The TA providers working closely with CA reported that data is available for analysis and 

reporting for the professional development activities implemented by the State that would be indicative 

of progress implementing the SSIP that would lead to changes in the SiMR. They will work with the 

CA office as Phase III, Year 2 activities are implemented to identify appropriate data sources and 

communicate the results of data analysis in next year’s SSIP submission. OSEP staff including the state 

lead, Kate Moran, and performance accountability representative, Leslie Fox, agreed to have on-going 

conversations throughout the year and review drafts of the SSIP as it is developed for Phase III, Year 2.

Use of Technical Assistance and Professional Development Resources 

NCSI and State TA providers work regularly with the Part C program to support CA’s data quality 

improvement efforts, analyze and report data, and improve the Part C progress assessment process and 

procedures.
3

Next Steps

During FFY 2016, OSEP and CA will continue to use regular TA calls to discuss resources for 

improving child and family outcomes and support CA’s infrastructure improvement strategies and 

evidence-based practices implemented for the SSIP. In addition, OSEP will share TA resources and 

materials with CA that may be most helpful for addressing data quality and data completeness for the 

APR and evaluation of the system improvement efforts outlined in the SSIP.