ACF’s Children’s Bureau administers and oversees federal funding to states for child welfare services under Titles IV-B and IV-E of the Social Security Act, and states and counties provide these child welfare services, either directly or indirectly through contracts with private agencies. Among other activities, ACF staff are responsible for developing appropriate policies and procedures for states to follow to obtain and use federal child welfare funds, reviewing states’ planning documents required by Title IV-B, conducting states’ data system reviews, assessing states’ use of Title IV-E funds, and providing technical assistance to states through all phases of the CFSR process. In addition, ACF staff coordinate the work of the 10 resource centers to provide additional support and assistance to the states. Spurred by the passage of the 1997 Adoption and Safe Families Act (ASFA), ACF launched the CFSR in 2001 to improve its existing monitoring efforts, which had once been criticized for focusing exclusively on states’ compliance with regulations rather than on their performance over a full range of child welfare services. The CFSR process combines a statewide self-assessment, an on-site case file review that is coupled with stakeholder interviews, and the development and implementation of a 2- year PIP with performance benchmarks to measure progress in improving noted deficiencies. In assessing performance through the CFSR, ACF relies, in part, on its own data systems, known as NCANDS and AFCARS, which were designed prior to CFSR implementation to capture, report, and analyze the child welfare information collected by the states. Today, these systems provide the national data necessary for ACF to calculate national standards for key performance items against which all states are measured and to determine, in part, whether or not states are in substantial conformity on CFSR outcomes and systemic factors. Once ACF approves the PIP, states are required to submit quarterly progress reports. Pursuant to CFSR regulations, federal child welfare funds can be withheld if states do not show adequate PIP progress, but these penalties are suspended during the 2-year PIP implementation term. In preparation for the next round of CFSRs, ACF officials have formed a Consultation Work Group of ACF staff, child welfare administrators, data experts, and researchers who will propose recommendations on the CFSR measures and processes. The group’s resulting proposals for change, if any, are not yet available. ACF and many state officials perceive the CFSR as a valuable process— highlighting many areas needing improvement—and a substantial undertaking, but some state officials and child welfare experts told us that data enhancements could improve its reliability. ACF staff in 8 of the 10 regions considered the CFSR a helpful tool to improve outcomes for children. Further, 26 of the 36 states responding to a relevant question in our survey commented that they generally or completely agreed with the results of the final CFSR report, even though none of the 41 states with final CFSR reports released through 2003 has achieved substantial conformity on all 14 outcomes and systemic factors. In addition, both ACF and the states have dedicated substantial financial and staff resources to the process. However, several state officials and child welfare experts we interviewed questioned the accuracy of the data used to compile state profiles and establish the national standards. While ACF officials in the central office contend that stakeholder interviews and case reviews compliment the data profiles, many state officials and experts reported that additional data from the statewide assessment could bolster the evaluation of state performance. ACF and state officials support the objectives of the review, especially in focusing on children’s outcomes and strengthening relationships with stakeholders, and told us they perceive the process as valuable. For example, ACF officials from 8 regional offices noted that the CFSRs were more intensive and more comprehensive than the other types of reviews they had conducted in the past, creating a valuable tool for regional officials to monitor states’ performance. In addition, state officials from every state we visited told us that the CFSR process helped to improve collaboration with community stakeholders. Furthermore, state staff from 4 of the 5 states we visited told us the CFSR led to increased public and legislative attention to critical issues in child welfare. For example, caseworkers in Wyoming told us that without the CFSR they doubted whether their state agency’s administration would have focused on needed reforms. They added that the agency used the CFSR findings to request legislative support for the hiring of additional caseworkers. Along with the value associated with improved stakeholder relations, the ACF officials we talked to and many state officials reported that the process has been helpful in highlighting the outcomes and systemic factors, as well as other key performance items that need improvement. According to our survey, 26 of the 36 states that commented on the findings of the final CFSR report indicated that they generally or completely agreed with the findings, even though performance across the states was low in certain key outcomes and performance items. For example, not one of the 41 states with final reports released through 2003 was found to be in substantial conformity with either the outcome measure that assesses the permanency and stability of children’s living situations or with the outcome measure that assesses whether states had enhanced families’ capacity to provide for their children’s needs. Moreover, across all 14 outcomes and systemic factors, state performance ranged from achieving substantial conformity on as few as 2 outcomes and systemic factors to as many as 9. As figure 1 illustrates, the majority of states were determined to be in substantial conformity with half or fewer of the 14 outcomes and systemic factors assessed. States’ performance on the outcomes related to safety, permanency, and well-being—as well as the systemic factors—is determined by their performance on an array of items, such as establishing permanency goals, ensuring worker visits with parents and children, and providing accessible services to families. The CFSR showed that many states need improvement in the same areas. For example, across all 41 states reviewed through 2003, the 10 items most frequently rated as needing improvement included assessing the needs and services of children, parents, and foster parents (40 states); assessing the mental health of children (37 states); and establishing the most appropriate permanency goal for the child (36 states). Given the value that ACF and the states have assigned to the CFSR process, both have spent substantial financial resources and staff time to prepare for and implement the reviews. In fiscal years 2001-03, when most reviews were scheduled, ACF budgeted an additional $300,000 annually for CFSR-related travel. In fiscal year 2004, when fewer reviews were scheduled, ACF budgeted about $225,000. To further enhance its capacity to conduct the reviews, and to obtain additional logistical and technical assistance, ACF spent approximately $6.6 million annually to hire contractors. Specifically, ACF has let three contracts to assist with CFSR- related activities, including training reviewers to conduct the on-site reviews, tracking final reports and PIP documents, and, as of 2002, writing the CFSR final reports. Additionally, ACF hired 22 new staff to build central and regional office capacity and dedicated 4 full-time staff and 2 state government staff temporarily on assignment with ACF to assist with the CFSR process. To build a core group of staff with CFSR expertise, ACF created the National Review Team, composed of central and regional office staff with additional training in and experience with the review process. In addition, to provide more technical assistance to the states, ACF reordered the priorities of the national resource centers to focus their efforts primarily on helping states with the review process. Like ACF, states also spent financial resources on the review. While some states did not track CFSR expenses—such as staff salaries, training, or administrative costs—of the 25 states that reported such information in our survey, the median expense to date was $60,550, although states reported spending as little as $1,092 and as much as $1,000,000 on the CFSR process. Although ACF officials told us that states can use Title IV- E funds to pay for some of their CFSR expenses, only one state official addressed the use of these funds in our survey, commenting that it was not until after the on-site review occurred that the state learned these funds could have been used to offset states’ expenses. States also reported that they dedicated staff time to prepare for the statewide assessment and to conduct the on-site review, which sometimes had a negative impact on some staffs’ regular duties. According to our survey, 45 states reported dedicating up to 200 full-time staff equivalents (FTE), with an average of 47 FTEs, to the statewide assessment process. Similarly, 42 states responded that they dedicated between 3 and 130 FTEs, with an average of 45 FTEs, to the on-site review process. For some caseworkers, dedicating time to the CFSR meant that they were unable or limited in their ability to manage their typical workload. For example, Wyoming caseworkers whose case files were selected for the on-site review told us that they needed to be available to answer reviewers’ questions all day every day during the on-site review, which they said prevented them from conducting necessary child abuse investigations or home visits. Child welfare-related stakeholders—such as judges, lawyers, and foster parents—also contributed time to the CFSR. State officials in the 5 states we visited, as well as child welfare experts, reported on several data improvements that could enhance the reliability of CFSR findings. In particular, they highlighted inaccuracies with the AFCARS and NCANDS data that are used for establishing the national standards and creating the statewide data profiles, which are then used to determine if states are in substantial conformity. These concerns echoed the findings of a prior GAO study on the reliability of these data sources, which found that states are concerned that the national standards used in the CFSR are based on unreliable information and should not be used as a basis for comparison and potential financial penalty.