Testimony 

Before the Subcommittee on Human 

Resources, Committee on Ways and 

Means, House of Representatives 

United States General Accounting Office 

GAO

For Release on Delivery 

Expected at 2:00 p.m. EST 

Wednesday, November 19, 2003 CHILD WELFARE

States Face Challenges in 

Developing Information 

Systems and Reporting 

Reliable Child Welfare Data 

Statement of Cornelia M. Ashby, Director, 

Education, Workforce, and Income Security Issues 

GAO-04-267T 
www.gao.gov/cgi-bin/getrpt?GAO-04-267T.

To view the full product, including the scope 

and methodology, click on the link above. 

For more information, contact Cornelia M. 

Ashby at (202) 512-8403 or 

ashbyc@gao.gov. 

Highlights of GAO-04-267T, a testimony 

for the Subcommittee on Human 

Resources, Committee on Ways and 

Means, House of Representatives 

November 19, 2003 

CHILD WELFARE

States Face Challenges in Developing 

Information Systems and Reporting 

Reliable Child Welfare Data 

HHS reported that 47 states are developing or operating a SACWIS, but many 

states continue to face challenges developing their systems. Most state 

officials said they recognize the benefit their state will achieve by developing 

SACWIS, such as contributing to the timeliness of child abuse and neglect 

investigations; however, despite the availability of federal funds since 1994, 

states reported a median delay of 21⁄2 years beyond the time frames they set 

for completion. States reported that they encountered some difficulties 

during SACWIS development, such as challenges receiving state funding and 

creating a system that reflected their work processes. In response to some 

of these challenges, HHS has provided technical assistance to help states 

develop their systems and conducted on-site reviews of SACWIS to verify 

that the systems meet federal requirements. 

Despite efforts to implement comprehensive information systems, several 

factors affect the states’ ability to collect and report reliable adoption, foster 

care, and child abuse and neglect data. States responding to GAO’s survey 

and officials in the five states GAO visited reported that insufficient 

caseworker training and inaccurate and incomplete data entry affect the 

quality of the data reported to HHS. In addition, states reported technical 

challenges reporting data. Despite HHS assistance, many states report 

ongoing challenges, such as the lack of clear and documented guidance on 

how to report child welfare data. In addition, although states were 

mandated to begin reporting data to the Adoption and Foster Care Analysis 

Reporting System (AFCARS) in 1995, few reviews of states’ AFCARS 

reporting capabilities have been conducted to assist states in resolving some 

of their reporting challenges. 

Some states are using a variety of practices to address the challenges 

associated with developing SACWIS and improving data reliability. For 

example, 28 states reported using approaches to help caseworkers identify 

and better understand the data elements that are required for federal 

reporting. 

In a related report, we recommended that the Secretary of HHS consider 

ways to enhance the guidance and assistance offered to states to help them 

overcome the key challenges in collecting and reporting child welfare data. 

These efforts could include a stronger emphasis placed on conducting 

AFCARS reviews and timelier follow-up to help states implement their 

improvement plans or identifying a useful method to provide clear and 

consistent guidance. HHS generally agreed with our findings but, in 

response to our recommendation, said that we did not recognize the longterm efforts to provide AFCARS and National Child Abuse and Neglect Data 

System related guidance. HHS also noted that the data definitions need to be 

updated and revised and said it was in the process of revising regulations. 

HHS added that it is firmly committed to continue to support the states and 

to provide technical guidance and assistance as resources permit.

To better monitor children and 

families served by state child 

welfare agencies, Congress 

authorized matching funds for the 

development of statewide 

automated child welfare 

information systems (SACWIS) and 

required that the Department of 

Health and Human Services (HHS) 

compile information on the 

children served by state agencies. 

This testimony is based on our July 

2003 report and addresses the 

following: (1) states’ experiences in 

developing child welfare 

information systems and HHS’s 

role in assisting in their 

development, (2) factors that affect 

the reliability of data that states 

collect and report on children 

served by their child welfare 

agencies and HHS’s role in 

ensuring the reliability of those 

data, and (3) practices that child 

welfare agencies use to overcome 

challenges associated with SACWIS 

development and data reliability. 

For the July 2003 report, we 

surveyed all 50 states and the 

District of Columbia regarding their 

experiences developing and using 

information systems and their 

ability to report data to HHS. We 

also reviewed a variety of HHS 

documents and visited five states 

to obtain firsthand information. 

Finally, we interviewed HHS 

officials and child welfare and data 

experts and reviewed relevant 

literature. 
Page 1 GAO-04-267T 

Mr. Chairman and Members of the Subcommittee: 

Thank you for inviting me here today to discuss states’ development of 

automated child welfare information systems. As you are aware, the 

Congress required that the Department of Health and Human Services 

(HHS) compile information on the children served by state agencies and 

authorized federal funds to match those of states for use in the 

development of state child welfare information systems. Since 1994, 

designated federal matching funds have been available to states to develop 

and implement comprehensive case management systems—statewide 

automated child welfare information systems (SACWIS)—to manage their 

child welfare cases as well as to report child abuse and neglect, foster 

care, and adoption information to the federal government. States have the 

option to implement a SACWIS or develop different information systems 

without using SACWIS funds to support their child welfare agencies and 

collect information on their child welfare cases. Regardless of the type of 

system a state develops, child welfare caseworkers at the county or local 

level are the key personnel who collect and document information on 

children and families served by child welfare agencies, in addition to 

performing a wide range of services to protect children—such as 

investigating child abuse or neglect reports or providing support services 

to maintain the children in their homes. 

Currently, HHS compiles state-reported child welfare data in two 

databases: the Adoption and Foster Care Analysis and Reporting System 

(AFCARS) and the National Child Abuse and Neglect Data System 

(NCANDS). HHS relies on the information available in its databases to 

analyze and track children’s experiences in the child welfare system, to 

determine states’ performance on federal child welfare outcome measures, 

and to report to Congress on children’s well being and child welfare 

experiences. 

My testimony today will focus on three key issues: (1) states’ experiences 

in developing child welfare information systems and HHS’s role in 

assisting in their development; (2) factors that affect the reliability of data 

that states collect and report on children served by their child welfare 

agencies, and HHS’s role in ensuring the reliability of those data; and (3) 

practices that child welfare agencies use to overcome challenges 

associated with SACWIS development and data reliability. My comments 

are based on the findings from our July 2003 report, Child Welfare: Most 

States Are Developing Statewide Information Systems, but the 

Reliability of Child Welfare Data Could Be Improved (GAO-03-809, July 

31, 2003). Those findings were based on our survey of all 50 states and the 
Page 2 GAO-04-267T 

District of Columbia regarding their experiences in developing and using 

information systems and their ability to report data to HHS. We received 

responses from 49 states and the District of Columbia,1

 although some 

states did not respond to every question. We also reviewed a variety of 

HHS documents, including the protocol and reports for its reviews of 

SACWIS systems and states’ AFCARS reporting capabilities and visited 

five states—Colorado, Iowa, New York, North Carolina, and Oklahoma—

to obtain firsthand information on their experiences developing SACWIS 

and reporting data to HHS. We selected these states to represent 

geographic diversity and different stages of SACWIS implementation. 

Finally, we interviewed HHS officials and child welfare and data experts 

and reviewed relevant literature. 

In summary, HHS reported that 47 states were developing or operating a 

SACWIS, but many states continue to face challenges developing their 

systems. Most state officials said they recognize the benefit their state will 

achieve by developing SACWIS, but added that they have encountered 

difficulties in receiving state funding and in creating a system that 

reflected their work processes. Despite the availability of federal funds 

since 1994, states reported a median delay of 21⁄2 years beyond the time 

frames they set for completion. Several factors affect the states’ ability to 

collect and report reliable adoption, foster care, and child abuse and 

neglect data. For example, insufficient caseworker training and inaccurate 

and incomplete data entry affect the quality of data reported to HHS. 

States also reported technical challenges reporting data. Despite HHS’s 

assistance, many states reported ongoing challenges, such as the lack of 

clear and documented guidance from HHS on how to report child welfare 

data. In addition, although states were mandated to begin reporting data to 

AFCARS in 1995, few reviews of states’ AFCARS reporting capabilities 

have been conducted. Some states are using a variety of practices to 

address the challenges they face in developing SACWIS and improving 

data reliability. For example, 28 states reported using approaches to help 

caseworkers identify and better understand the data elements that are 

required for federal reporting. To improve the reliability of state-reported 

child welfare data, we recommended in our July 2003 report that the 

Secretary of HHS consider ways to enhance the guidance and assistance 

 1

Throughout this testimony, references to state survey responses include the District of 

Columbia. Forty-six of these states reported that they are developing or operating a 

SACWIS. Nevada, which HHS reported has an operational SACWIS, did not respond to our 

survey. 
Page 3 GAO-04-267T 

offered to states to help them overcome the key challenges in collecting 

and reporting child welfare data. 

ACF’s Children’s Bureau is responsible for the administration and 

oversight of federal funding to states for child welfare services under 

Titles IV-B and IV-E of the Social Security Act. However, the monitoring of 

children served by state child welfare agencies is the responsibility of the 

state agencies that provide the services to these children and their 

families. Child welfare caseworkers at the county or local level are the key 

personnel responsible for documenting the wide range of services offered 

to children and families, such as investigations of abuse and neglect, 

treatment services offered to keep families intact and prevent the need for 

foster care, and arrangements made for permanent or adoptive placements 

when children must be removed from their homes. Caseworkers are 

supported by supervisors, who typically assign new cases to workers and 

monitor caseworkers’ progress in achieving desired outcomes, analyzing 

and addressing problems and making decisions about cases. 

To qualify for federal funding for SACWIS, states must prepare and submit 

an advance planning document (APD) to the Children’s Bureau, in which 

they describe the state’s plan for managing the design, development, 

implementation, and operation of a SACWIS that meets federal 

requirements and state needs in an efficient, comprehensive, and costeffective manner. In addition, the state must establish SACWIS and 

program performance goals in terms of projected costs and benefits in the 

APD. States are required to submit separate APDs for the planning and 

development phases, in addition to periodic updates. 

Since the administration and structure of state child welfare agencies vary 

across the nation, states can design their SACWIS to meet their state 

needs, as long as states meet certain federal requirements. Federal funding 

is available to states for SACWIS that 

• meet the requirements for reporting AFCARS data to HHS; 

• to the extent practicable, are capable of linking with the state data 

collection system that collects information on child abuse and neglect; 

• to the extent practicable, are capable of linking with, and retrieving 

information from, the state data collection system that collects 

information on the eligibility of individuals under Title IV-A—Temporary 

Assistance for Needy Families; and 

Background 
Page 4 GAO-04-267T 

• provides for more efficient, economical, and effective administration of 

the programs carried out under a state’s plans approved under Titles IV-B 

and IV-E of the Social Security Act. 

A SACWIS must operate uniformly as a single system in each state and 

must encompass all entities that administer programs provided under 

Titles IV-B and IV-E. In some cases, HHS will allow the statewide system 

to link to another state system to perform required functions, such as 

linking to financial systems to issue and reconcile payments to child 

welfare service providers. The state’s APD must describe how its SACWIS 

will link to other systems to meet the requirements in the SACWIS 

regulations. 

In addition to monitoring the APDs of the states that are developing 

SACWIS, HHS reviews state information systems through formal SACWIS 

assessment reviews and the Child and Family Services Reviews (CFSR)—a 

federal review process to monitor states’ compliance with child welfare 

laws and federal outcome measures. The formal SACWIS reviews are 

conducted by the Children’s Bureau to determine if a state has developed 

and implemented all components detailed in the state’s APD and if the 

system adheres to federal requirements. The CFSR assesses statewide 

information systems, along with other systemic factors, to determine if the 

state is operating a system that can readily identify the status, 

demographic characteristics, location, and goals for placement of every 

child who is in foster care. This systemic factor is reviewed in all states, 

regardless of whether the state is developing a SACWIS or the stage of 

system development. For the 40 CFSR reports that are available, HHS 

found that four states were not in substantial conformity on the statewide 

information system indicator.2 These four states must address how they 

will come into conformity with this factor in a program improvement plan. 

HHS has also conducted SACWIS reviews in two of these states. 

 2

We are currently conducting an engagement on states’ and HHS’s experiences in 

conducting the CFSRs. 
Page 5 GAO-04-267T 

While 47 states are developing or operating a SACWIS, many challenges 

remain despite HHS’s oversight and technical assistance. Since 1994, 

states reported that they have spent approximately $2.4 billion in federal, 

state, and local funding on SACWIS. While most state officials we 

interviewed and those responding to our survey said that they recognize 

the benefits their state will achieve by developing a statewide system, 

many states reported that the development of their SACWIS is delayed 

between 2 months and 8 years beyond the time frames the states set for 

completion, with a median delay of 21⁄2 years. Most states responding to 

our survey faced challenges, such as obtaining state funding and 

developing a system that met the child welfare agency’s needs statewide. 

In response to some of these challenges, HHS has provided technical 

assistance to help states develop their systems and conducted on-site 

SACWIS reviews to verify that the systems meet all federal requirements. 

Currently, 47 states are developing or operating a SACWIS and are in 

various stages of development—ranging from planning to complete. The 

states responding to our survey reported using approximately $1.3 billion 

in federal funds3

 and approximately $1.1 billion in state and local funds4 for 

their SACWIS. However, HHS estimated that it allocated approximately 

$821 million between fiscal years 1994 and 2001 in SACWIS developmental 

funds5

 and $173 million between fiscal years 1999 and 2001 in SACWIS 

operational funds.6

 The total amount of federal funding provided to states 

for SACWIS is unknown because states claimed operational costs as a part 

 3

Forty-four states provided information on the total amount of federal funds they received 

to develop and operate SACWIS. Alaska, Hawaii, Missouri, North Carolina, Texas, and 

Vermont did not report federal funding information. Nevada did not respond to our survey. 

State-reported figures may include some funding allocated in fiscal year 2003, since the 

survey was issued in October 2002 and completed as late as December 2002. 

4

Forty-four states provided information on the total amount of state funds used to develop 

and operate SACWIS. Arkansas, Hawaii, Missouri, North Carolina, Texas, and Vermont did 

not report state funding information. Nevada did not respond to our survey. State-reported 

figures may include some funding allocated in fiscal year 2003 since the survey was issued 

in October 2002 and completed as late as December 2002. 

5

This figure includes developmental funds allocated by HHS to 49 states and the District of 

Columbia. Hawaii did not take any federal money for SACWIS development. 

6

This figure includes operational funds allocated to 35 states. States begin claiming 

operational costs when some or all components of their SACWIS are operating in local 

offices. Operational activities include routine maintenance, minor enhancements, and other 

changes that do not significantly increase or modify the functionality of the system. 

Most States Are 

Developing SACWIS, 

But Challenges 

Remain Despite HHS’s 

Oversight and 

Technical Assistance 

States Are Using Federal 

and State Funds and 

Various Participants to 

Develop Multicomponent 

SACWIS 
Page 6 GAO-04-267T 

of their Title IV-E administrative expenses prior to 1999.7 Although the 

federal government matched state funding at an enhanced rate of 

75 percent beginning in 1994, many states did not apply for federal funding 

or begin SACWIS development until 1996 or 1997, when more than 

$467 million—the bulk of federal funds—were allocated. Most states were 

still developing their SACWIS by the time enhanced funding expired in 

1997, after which states could receive a 50 percent federal financial 

participation for SACWIS development and operation. Although 47 states 

are currently developing or operating a SACWIS, all states except Hawaii 

received some federal SACWIS funds. For example, according to figures 

provided by HHS, North Carolina and North Dakota received some 

developmental funds but encountered difficulties that prevented them 

from completing their systems. 

In order to track states’ SACWIS development, HHS places them in six 

categories that identify their stage of development (see table 1). HHS 

sometimes recategorizes states into a lower stage of development when 

problems are encountered. In addition, while HHS may classify a state 

system as complete following an assessment of the state’s SACWIS, a state 

may make additional changes to the system since SACWIS, like other 

computer systems, continually evolve as technology and child welfare 

practices change. States can claim federal funding for these changes as 

operational expenses. An HHS official reported that such changes do not 

need prior approval unless they are in excess of $5 million. 

 7

According to HHS officials, prior to fiscal year 2000, states reported SACWIS operational 

expenses as part of their Title IV-E administrative expenses because the claims sheet states 

used for reporting did not have a separate column for SACWIS operational expenditures. In 

fiscal year 2000, states were required to use a claims sheet that was reformatted to provide 

space for SACWIS operational expenditures. In addition, an HHS official explained that the 

difference between the state-reported figures and the federal figures may be due to states 

claiming some SACWIS expenses under different programs, such as Title IV-E 

administrative funds, rather than separately as SACWIS expenses. 
Page 7 GAO-04-267T 

Table 1: Number of States in Various Stages of SACWIS Development 

Stage Number of states 

Completea5 

Operationalb 24 

Partially operationalc 9 

Implementationd 2 

Planninge7 

No SACWISf 4 

Source: HHS. 

Note: Status is as of October 13, 2003. 

a

The SACWIS assessment process is completed, and all functional requirements and specifications 

set forth in the APD are either included in the system or in an accepted corrective action plan. 

b

All functional requirements and specifications in the APD are included in the system, and the system 

is functional statewide, but state has not completed a SACWIS assessment or is working on other 

issues. 

c

The state is still rolling out a system to field sites or still adding functions to systems that are 

operational statewide. 

d

In active design and development, even if delayed while waiting to resolve problems such as funding. 

e

Working through options for a SACWIS. 

f

Have never pursued SACWIS funding or have abandoned plans to develop a system. 

States have considerable flexibility in the design of their SACWIS. 

According to HHS officials, a state should be using its SACWIS as a case 

management tool that uses automation to support the various aspects of 

state child welfare programs, such as recording child protection, out-ofhome care, and foster care and adoption services. To further assist child 

welfare practice, states have designed their systems to follow the natural 

flow of child welfare practice in their state and have added design features 

to help track key events during a case. For example, in Iowa child welfare 

work is divided between child abuse and neglect investigations and 

ongoing case management for children brought into the care of the child 

welfare agency. As a result, Iowa designed a SACWIS to reflect this work 

process by linking two databases—one to record child abuse and neglect 
Page 8 GAO-04-267T 

information and one to record ongoing case records—that share 

information with each other.8

Since many states are in different phases of SACWIS development, their 

systems currently support to varying degrees a variety of child welfare and 

administrative components (see table 2). According to HHS, while the 

components listed in table 2 are required for a state’s SACWIS to be 

considered compliant with federal guidance—either through an interface 

or built within the system—some of the subcomponents, such as a 

function that helps caseworkers manage their caseloads, are optional. 

HHS has encouraged states to automate as many functions as possible in 

the SACWIS in an effort to cut down on the additional paperwork or 

duplicative steps inherent in manual data collection. 

 8

Although the Iowa state officials described their SACWIS as including the child abuse and 

neglect system, HHS commented on a draft of the July 2003 report that it does not view the 

child abuse and neglect system as part of the state’s SACWIS. However, HHS said that the 

state has met the SACWIS requirement in this area by building an interface between the 

two systems. 
Page 9 GAO-04-267T 

Table 2: Selected SACWIS Child Welfare and Administrative Services 

Service 

Fully or partially 

operational in 

SACWIS 

Planned 

for SACWIS

Child welfare services 

Child protectiona 38 5

Out-of-home careb 35 8

Adoption 34 9

Independent living 27 14

Intensive home-based servicesc 27 13

Administrative services 

Workload management 32 8

IV-E eligibilityd 29 14

Foster care maintenance payments 28 14

Adoption assistance payments 25 17

Contract provider payment 24 15

Source: GAO survey. 

Note: This table is based on responses from 46 states developing or operating a SACWIS. The rows 

for the columns “fully or partially operational” and “planned” do not add to 46 because the 

respondents may have answered “not supported,” “don’t know,” or “no answer.” 

a

Child protection includes services such as intake and screening, investigation, and disposition. 

b

Out-of-home care includes things such as foster care, group homes, and residential placement. 

c

Intensive home-based services include efforts to avoid placing a child in foster care. 

d

IV-E funding is available for foster care, adoption, and independent living services. 

To assist with the design of their SACWIS, states relied on a number of 

different participants, including internal users, such as caseworkers and 

managers, information technology (IT) staff, and contractors. In 

Oklahoma, for example, 150 child welfare staff from the field worked 

closely with the contractor in intensive work group sessions to design and 

test the system. To complement the caseworkers’ knowledge of child 

welfare practice, 43 states relied on IT staff. Finally, 42 states reported that 

they hired private contractors to conduct a large part of SACWIS design 

and development. 

At the time of our review, HHS reported that four states were not pursuing 

SACWIS development, and most of these states reported various reasons 

in our survey for not developing a system. In Hawaii, for example, the 

child welfare agency chose not to pursue SACWIS because it already had a 
Page 10 GAO-04-267T 

statewide system in place that it believed was adequately meeting its 

needs and which was collecting and reporting federal child welfare data. 

While most state child welfare agency officials said they recognize the 

benefits the state will achieve by developing SACWIS, such as enhancing 

their ability to track the whereabouts of foster children, 31 state agencies 

lag behind the time frames they set for completion, with 26 states 

reporting delays ranging from 2 months to 8 years. According to survey 

results, automated systems provided easier access to data and allowed 

caseworkers to better monitor children in their care, a fact that may 

contribute to additional child welfare and administrative benefits, such as 

decreased incidences of child abuse and neglect, shortened length of time 

to achieve adoption, timeliness of payments to foster families, and 

timeliness of payments to foster facilities. New Jersey, which is in the 

planning stage, reported in our survey that its goal in developing a SACWIS 

is to integrate the more than 40 stand-alone systems that currently capture 

information on the children served by their child welfare agency.9

 By 

pulling all of these systems together into a uniform SACWIS, the state 

hopes to improve the recording of casework activities in a timely manner 

and to develop a tool to better target resources and services. Effectively 

integrating these systems will require the state to use a disciplined IT 

management approach that includes (1) detailed analyses of users’ needs 

and requirements, (2) a clearly defined strategy for addressing information 

needs, and (3) sufficient technical expertise and resources to support the 

effort. 

Despite the benefits that many states have accrued with SACWIS, 31 states 

reported in our survey that they have been delayed in system completion 

beyond their initial deadline and identified a number of challenges that 

have led to the delay (see table 3).10 Some of the common difficulties states 

 9

New Jersey reported in our survey that it had spent approximately $9 million in federal 

funds and $4 million in state and local funds on system development. According to HHS, 

New Jersey first received federal funds in 1996. 

10Twelve of the 46 states reporting that they are developing or operating a SACWIS 

reported that they have not experienced delays in developing their systems. In response to 

the length of the delays reported by 26 states in our survey, ACF commented on a draft of 

the July 2003 report that these states may be using different definitions in defining their 

delays. However, ACF did not provide further information on how the delays represented 

in that report differ from its perception of states’ experiences. In our survey, we asked 

states to report on the delays that exceeded the time line outlined in their initial APD. 

States Accrue Benefits 

from Using SACWIS, but 

Several Issues Create 

Delays in Completing 

States’ Systems 
Page 11 GAO-04-267T 

reported in developing SACWIS included receiving state funding approval, 

reaching internal agreement on system development, and creating a 

system that reflects child welfare work processes and is user-friendly. 

Vermont officials, for example, reported that the state legislature declined 

to provide the matching state funds needed to secure federal funding for 

SACWIS. As a result, the state could not pursue development. 

Table 3: Number of Months States Delayed in SACWIS Development 

State Length of delay in monthsa

Alabama 36

Arkansas 6

California 36

Colorado 26

Connecticut 96

District of Columbia 36

Georgia 25

Idaho 21

Illinois 79

Indiana 6

Kansas 72

Louisiana 12

Maryland 12

Michigan 26

Minnesota 12

Mississippi 12

New Jersey 42

New Mexico 3

Ohio 36

Oregon 70

Rhode Island 14

South Carolina 47

Tennessee 36

Utah 48

Virginia 2

Washington 36

Source: GAO survey. 

Note: While 31 states reported in the survey that they have experienced a delay in SACWIS 

development, only 26 states reported the length of their delay. The survey was issued in October 

2002 and completed by states as late as December 2002. 
Page 12 GAO-04-267T 

a

States were asked to report the number of months the delays exceeded the time line outlined in their 

APD. 

Despite user involvement in system design, some states still faced 

challenges trying to reach internal agreement among agency officials and 

caseworkers on the design of a system, resulting in a delay in 

development. In New York—a state where the counties are responsible for 

administering child welfare services—the development of SACWIS was 

stalled when significant frustration with the system’s design led 

commissioners from five large counties and New York City to request that 

the state stop SACWIS development until a reassessment of the design of 

and plans for the implementation of the system was completed. 

Similarly, despite states’ heavy reliance on contractors, many reported that 

securing contractors with knowledge of child welfare practice was a 

challenge for timely SACWIS development. Contractors are hired by the 

state for their system development knowledge but often are unfamiliar 

with child welfare policies and practices, especially since they vary from 

state to state. A contractor who has worked with seven states to develop 

their SACWIS reported that contractors are asked to learn the child 

welfare business practices of a state in a short amount of time and that 

states cannot devote many resources, such as caseworkers, to help in the 

design process because caseworkers need to devote their time to 

providing services to children and families. 

Many states reported that creating a system that reflects child welfare 

work processes and is user-friendly was a challenge in developing 

SACWIS. These issues were also identified in the federal reviews of states’ 

SACWIS. For example, one state explained in the SACWIS review that it 

had designed a system to meet the caseworkers’ needs and reflect the 

nature of the child welfare work processes by developing a system that 

required events to be documented as they occurred. However, this design 

limited the SACWIS’s functionality because it did not allow the 

caseworkers to go back and enter information after an event happened. 

The state explained that caseworkers do not use the system in real time, 

but provide services to the children and families and then record the 

information in the system. The state had to redesign the system to correct 

for this design flaw. 
Page 13 GAO-04-267T 

HHS has assisted states in a variety of ways in developing and completing 

their SACWIS.11 As a part of its regulatory responsibilities, HHS must 

review, assess, and inspect the planning, design, development, installation, 

and operation of SACWIS. In addition to reviewing and monitoring states’ 

APDs, HHS conducts on-site SACWIS reviews to comply with these 

responsibilities. HHS officials told us that these reviews are a detailed and 

thorough assessment of state systems to ensure the systems’ compliance 

with SACWIS requirements. In addition, officials reported that they 

provide technical assistance during the on-site review to help states that 

do not fully conform with the applicable regulations and policies. As of 

October 2003, HHS had reviewed 27 SACWIS—5 of which were 

determined as meeting all the requirements and classified as complete. 

HHS officials told us that since states have the flexibility to build a 

SACWIS that meets their needs, a large portion of the formal reviews 

concentrate on ensuring that the systems conform to state business 

practices. For example, while SACWIS regulations require that a state 

report all AFCARS data from their SACWIS, one state HHS reviewed relied 

on a separate state system to report data on the children served by the 

juvenile justice agency who are eligible for IV-E foster care funds. The 

state proved it had developed an automated process to merge data from 

both systems to compile a single AFCARS report that included children 

captured in both their SACWIS and juvenile justice systems. Therefore, 

HHS recognized that this process best met the state’s needs and 

determined the SACWIS to be complete and meeting all requirements. 

Few systems have been determined complete after an on-site review 

because of unresolved issues, such as not being able to build links to other 

state information systems or not implementing certain eligibility 

determination functions. To help states address some of these 

development challenges, the SACWIS review team provides the state with 

recommendations for complying with SACWIS requirements. In addition, 

HHS officials reported that once the draft report with the results of the 

SACWIS review is completed, federal staff schedule a conference call with 

 11With regard to the budget difficulties that states reported facing, since 1994 the federal 

government has made a commitment to help states develop and maintain their SACWIS by 

matching 75 percent of states’ development funds through 1997 and providing an ongoing 

match of 50 percent of state funding for the development and maintenance of their 

systems. However, since the states’ legislatures must make the initial commitment to fund 

SACWIS, the federal government cannot assist state child welfare agencies with this 

challenge. 

HHS Provides Some 

Assistance to Help States 

Meet SACWIS 

Requirements 
Page 14 GAO-04-267T 

the state officials to walk through the system’s deficiencies and offer 

guidance on how the state can move forward. 

HHS facilitates the sharing of information between states developing 

SACWIS through an automated system users’ group that allows state and 

federal officials to exchange information, ideas, and concerns. In addition 

to the users’ group, HHS officials also sponsor a Listserv—an electronic 

mailing list—that allows state officials to exchange information and a 

monthly conference call with state information technology directors. 12 

Technical assistance for SACWIS development is also available to states 

through the National Resource Center for Information Technology in Child 

Welfare (Resource Center), which opened in 1999. According to survey 

results, 9 states said they used the Resource Center for assistance in 

developing SACWIS and 14 states reported using it for help with SACWIS 

maintenance and improvements. According to Resource Center officials, 

they assist states with SACWIS development by helping states understand 

the technology that is available for use, providing information on the 

automation of child welfare work and converting data, and reviewing the 

APD documentation. 

 12In commenting on a draft of the July 2003 report, HHS indicated that a Web resource is 

available to states interested in learning about other states’ efforts to develop human 

services—child welfare, food stamps, Temporary Assistance to Needy Families, child care, 

and child support enforcement—information systems at http://www.acf.hhs.gov/nhsitrc. 
Page 15 GAO-04-267T 

Several factors affect states’ ability to collect and report reliable13 data on 

children served by state child welfare agencies, and some problems exist, 

such as a lack of clear and documented guidance, with HHS’s oversight 

and technical assistance. Almost all of the states responding to our survey 

reported that insufficient caseworker training and inaccurate and 

incomplete data entry affect the quality of the data reported to HHS.14 In 

addition, 36 of the 5015 states that responded to our survey reported that 

technical challenges, such as matching their state data element definitions 

to HHS’s data categories, affected the quality of the data that they report to 

the federal government. Despite the assistance that HHS offers to states, 

such as testing state data quality and providing the results to states to aid 

them in resubmitting data, states report ongoing challenges receiving clear 

and documented guidance and obtaining technical assistance. 

Almost every state responding to our survey and all the states we visited 

reported that insufficient training for caseworkers and inaccurate and 

incomplete data entry affect the quality of the data reported to AFCARS 

and NCANDS (see fig. 1). Although most states reported these as separate 

factors, HHS and the states we visited found that insufficient training and 

inaccurate and incomplete data entry are often linked. In official reviews 

of states’ information systems’ capability to capture data and report them 

to AFCARS, HHS advised states to offer additional training to caseworkers 

on several AFCARS data elements, such as recording the reasons for a 

child leaving foster care, to improve the accuracy of the data submitted. 

However, state officials told us that training is typically one of the first 

programs cut when states face tight budget restrictions. For example, 

Iowa officials told us that training has been significantly reduced in recent 

years because of budget cuts and new workers may wait 2 to 3 months 

before being trained how to enter data appropriately into their SACWIS. 

 13Data are reliable when they are complete and accurate. A subcategory of accuracy is 

consistency. Consistency refers to the need to obtain and use data that are clear and well 

defined enough to yield similar results in similar analysis. See U.S. General Accounting 

Office, Assessing the Reliability of Computer-Processed Data, GAO-02-15G (Washington, 

D.C.: Sept. 2002). 

14States were asked the extent to which certain problems may decrease the quality of the 

data submitted to AFCARS and NCANDS using the following scale: very great, great, 

moderate, some, and no affect. 

15The analysis of survey responses about reporting data to HHS is based on responses from 

49 states and the District of Columbia. All states, regardless of SACWIS development, were 

asked to complete these questions. 

Several Factors Affect 

the States’ Ability to 

Ensure Reliable Data 

on Children’s 

Experiences, and 

Some of HHS’s 

Oversight and 

Assistance Is 

Problematic 

Insufficient Caseworker 

Training and Inaccurate 

and Incomplete Data Entry 

Are the Most Common 

Factors That Affect Data 

Reliability 
Page 16 GAO-04-267T 

Figure 1: Most Common Caseworker Issues That Affect Data Quality 

Notes: Based on responses from 50 states. 

The results reported in the figure are a sum of the states that reported the issue had a very great 

affect, great affect, moderate affect, or some affect on the quality of state data submitted to HHS. 

Very great and great affect responses are represented in the top section of each bar. Moderate and 

some affect responses are represented in the bottom section of each bar. States not included 

answered “no affect,” “don’t know,” or “no answer.” 

Inaccurate and incomplete data entry can also result from a number of 

other factors, such as caseworkers’ hesitation to ask families for sensitive 

information. For example, caseworkers in Oklahoma reported that they 

did not feel comfortable asking if a child’s mother was married at the time 

of birth or if a child is of Hispanic origin—both of which are required 

AFCARS data elements. In commenting on a draft of this report, Oklahoma 

added that caseworkers did not understand why the data elements were 

required and how the federal government used the information. HHS noted 

States reporting at least some affect in data quality

0

10

20

30

40

50

Inaccurate data

entry by

caseworkers

Insufficient

training for

caseworkers

Incomplete data

entry by

caseworkers

AFCARS

NCANDS

Source: GAO survey.

49

45

49

42

46

43

18

31

12

33

15

34

11

31

17

29

12

31
Page 17 GAO-04-267T 

similar issues in five states that have had an AFCARS review.16

Caseworkers were inaccurately recording a child’s race as “unable to 

determine” even though this option should be selected only if the child’s 

parents or relatives cannot provide the information, such as when a child 

is abandoned.17

Caseworkers, supervisors, and managers in the 5 states we visited 

reported that additional factors, such as difficulties balancing data entry 

with the time that they spend with the families and children, contributed 

to inaccurate or incomplete data entry. Supervisors in Iowa explained that 

since caseworkers are responsible for ensuring that children and their 

families receive the services they need, the caseworkers tend to initially 

limit data entry to the information that is necessary to ensure timely 

payment to foster care providers and complete all other data elements 

when they have time. In addition, caseworkers in Colorado said that they 

are between 30 and 60 days behind in their data entry, so the information 

in the automated system may not accurately reflect the current 

circumstances of children in care. HHS’s Inspector General recently issued 

a report in which more than two-thirds of the states reported that 

caseworkers’ workloads, turnover, a lack of training, and untimely and 

incomplete data entry affected the reporting of AFCARS data.18

 16For the July 2003 report, we reviewed AFCARS reports from six of the eight states that 

had been assessed by HHS—Arkansas, Connecticut, New Mexico, Texas, Vermont, and 

Wyoming. HHS conducted reviews in Delaware and West Virginia after we completed our 

analysis. As of October 2003, HHS had completed three additional reviews for North 

Dakota, Rhode Island, and Washington. 

17In commenting on a draft of the July 2003 report, ACF said that the finding from the 

AFCARS reviews indicates that information is often defaulted to the response “unable to 

determine” in order for the element not to fail the missing data standard, not that workers 

are recording “unknown”; however, the report findings we used in this analysis instruct 

states to fix the defaults and address caseworker practice by enhancing training on the 

correct use of “unable to determine” when noting a child’s race. 

18Department of Health and Human Services, Office of Inspector General, Adoption and 

Foster Care Analysis and Reporting System (AFCARS): Challenges and Limitations, 

OEI-07-01-00660 (Washington, D.C.: Mar. 2003). 
Page 18 GAO-04-267T 

In addition to data quality being affected by caseworker issues, many 

states experienced technical challenges reporting their data to HHS. The 

problems reported by states are typically a result of challenges associated 

with data “mapping”—matching state data elements to the federal data 

elements. For example, 36 states reported in our survey that matching 

their state-defined data to HHS’s definitions affected the quality of the data 

reported to NCANDS and AFCARS. Similarly, 24 states reported that 

matching the more detailed data options available in their states’ 

information systems to the federal data elements affected the quality of 

the data reported to NCANDS. Twenty-nine states reported that this issue 

created challenges in reporting data to AFCARS. For example, following 

an AFCARS assessment, HHS instructed a state that collects detailed 

information on children’s disabilities, such as attention deficit disorder 

and eating disorders, to map the information to the more limited options in 

AFCARS, such as mental retardation and emotionally disturbed. 

In many cases, states have to balance state policy with federal 

requirements to ensure that they are reporting accurate data to AFCARS 

and NCANDS, but are not contradicting their state policies. For example, 

Texas officials reported that although the findings of their AFCARS review 

instructed them to modify their SACWIS to collect, map, and extract data 

on guardianship placements, the state does not support guardianship 

arrangements.19 In addition, a recent report from the Child Welfare League 

of America (CWLA) found that when reporting the number of times 

children move from one foster care placement to another, states varied in 

the type of placements included in that count.20 For example, 29 percent of 

the states responding to CWLA’s survey included respite,21 25 percent 

included runaways, and 16 percent included trial home visits when 

reporting the number of placements a child had during the AFCARS report 

period. According to federal guidance, the “number of placements” 

element is meant to gather information on the number of times the child 

welfare agency found it necessary to move a child while in foster care and 

that by including runaways or trial home visits, a state is inflating the 

number of moves a child experienced. 

 19Guardianship arrangements occur when permanent legal custody of a child is awarded to 

an individual, such as a relative, but the child is not legally adopted. 

20Child Welfare League of America. National Working Group Highlights, “Placement 

Stability Measure and Diverse Out-of-Home Care Populations” (Washington, D.C., Apr. 

2002). 

21Respite care provides temporary child care for children away from their caretakers. 

Technical Challenges, such 

as Matching State 

Definitions to Federal 

Definitions, Affect Data 

Reliability 
Page 19 GAO-04-267T 

HHS provides technical assistance for AFCARS and NCANDS reporting 

through a number of resources. HHS officials in the central office and 

NCANDS contractor staff serve as the points of contact for states to ask 

questions and seek guidance on reporting child welfare data. The officials 

in three of the five states that we visited said that the one-on-one focused 

technical assistance was useful when provided in a timely fashion. Most 

state officials found the NCANDS data easier to report, in part because 

more people were available for consultation and they were more 

accessible and responsive. For example, states have access to four 

NCANDS specialists and staff in the contractor’s central office when they 

need assistance reporting child abuse and neglect information. However, 

some of the states we visited reported that only one or two staff in HHS’s 

central office are available to assist with AFCARS reporting. 

In addition, the Resource Center offers states assistance with improving 

data quality. However, Resource Center staff reported that the assistance 

is geared more toward improving the limited data used in the federal 

review process to monitor states’ compliance with child welfare laws and 

federal outcome measures—CFSR—rather than all the data reported to 

HHS. The Resource Center also sponsors an annual information 

technology conference during which sessions covering all data-related 

issues are held, including practices for ensuring data quality and outcome 

evaluation in child welfare. In conjunction with this conference, the HHS 

officials and the contractors that operate NCANDS hold an annual 

technical assistance meeting for states to share ideas with one another, 

discuss data elements that pose difficulties, and explore ways to address 

these problems. In addition, an NCANDS state advisory group meets 

annually to talk with HHS officials about NCANDS data and their 

experiences reporting data. From these meetings, the state advisory group 

proposes changes or improvements to NCANDS. HHS and state officials 

reported that this partnership has helped ease some of the challenges in 

reporting child abuse and neglect data. 

HHS has also made available to states the software it uses to examine 

states’ AFCARS and NCANDS submissions for inconsistencies and invalid 

data. Officials in all the states we visited said that they regularly use this 

software, and an HHS official said that nearly every state has used the 

software at least once. When the data are submitted to HHS, they are run 

through the same software, and HHS notifies the states of areas where 

data are missing or inconsistent and allows the states to resubmit the data 

after errors are corrected. HHS officials reported that these tests help 

them to identify some data quality errors, such as missing data, and said 

that they believe that, in general, data have improved in recent years. 

Although HHS Has Taken 

Steps to Help States 

Improve Their Data, Some 

Problems with Its Efforts 

Exist 
Page 20 GAO-04-267T 

However, they indicated that the tests cannot pinpoint the underlying 

problems contributing to these errors. Furthermore, one official reported 

that no specific efforts have been conducted to track the individual data 

elements, and therefore HHS cannot report on how data quality has 

changed over time. 

In an attempt to help states comply with the reporting standards and 

address some of the factors that contribute to data quality problems, HHS 

performs comprehensive reviews of state information systems’ ability to 

capture AFCARS data to identify problems associated with data collection 

and reporting and to ensure that the information in the automated system 

correctly reflects children’s experiences in care. The assessments include 

a technical review of the states’ computer code, a comparison of the data 

from selected cases available in the information system to the case files, 

and an improvement plan to resolve any errors. In addition, HHS officials 

offer guidance to the states on improvements that can be made to the 

information system and changes to program code used to report the 

AFCARS data. HHS conducted pilot reviews in eight states between 1996 

and 2000. By October 2003, HHS had conducted 11 official reviews—even 

though states began reporting to AFCARS in 1995. According to results 

from 6 of the 11 official AFCARS assessments we reviewed, no state met 

the reporting requirements for all AFCARS data elements. The problems 

noted in the reviews are similar to those of states responding to our survey 

and those we visited. For example, most states received ratings of 2 or 3, 

indicating technical and/or data entry errors that affect the AFCARS data 

quality.22 For the current placement setting data element,23 for instance, 4 

states received a rating of 2, 1 state received a rating of 3, and 1 state 

received a rating of 4. In Connecticut, which received a rating of 2, HHS 

found that, among other things, workers were not consistently entering 

placement information in a timely way. It also found that workers entered 

 22HHS rates each data element using a four-point scale: (1) the AFCARS requirement(s) has 

not been implemented in the information system; (2) the technical system requirements for 

AFCARS reporting do not fully meet the standards; (3) the technical system requirements 

for AFCARS reporting are in place, but there are data entry problems affecting the quality 

of the data; (4) all of the AFCARS requirements have been met. According to an HHS 

official, data elements that have a combination of technical and data entry problems are 

rated as 2 until the technical issues are resolved. HHS will then rate the element as a 3 until 

the data entry practices are changed. 

23Current placement setting refers to a pre-adoptive home, foster family home-relative, 

foster family home-nonrelative, group home, institution, supervised independent living, 

runaway, or trial home visit. 
Page 21 GAO-04-267T 

placement data only into a narrative field, which resulted in placement 

history gaps and incomplete AFCARS reports. 

State officials in the six states for which we reviewed the HHS AFCARS 

assessments reported that they found the reviews useful for improving 

their AFCARS data submissions. In particular, they valued the thorough 

review by HHS officials of the computer code states use to report the data. 

Some of these officials reported that if all states were reviewed, the quality 

of data available in AFCARS would improve tremendously. However, HHS 

officials reported that they are not mandated to conduct the AFCARS 

reviews and that priority is placed on other reviews, such as the CFSR and 

SACWIS reviews. In addition, officials explained that the AFCARS reviews 

are not conducted in states developing SACWIS until the systems are 

operational. HHS expects to complete approximately four reviews each 

year, depending on available resources, and has scheduled states through 

2006. Similar to the SACWIS reviews, HHS officials offer recommendations 

and technical assistance to states during the review on how they can 

improve the quality of the data reported to AFCARS. 

Although the states we visited appreciated some of HHS’s efforts to assist 

with improving state data quality, they and most states responding to our 

survey agreed that the assistance is not always consistent or easily 

accessible (see fig. 2). States reported similar information to the Inspector 

General—AFCARS data elements were not clearly and consistently 

defined and technical assistance is effective but difficult to access. 
Page 22 GAO-04-267T 

Figure 2: Federal Practices That Affect Data Quality 

Notes: Based on responses from 50 states. 

The results reported in the figure are a sum of the states that reported the issue had a very great 

affect, great affect, moderate affect, or some affect on the quality of state data submitted to HHS. 

Very great and great affect responses are represented in the top section of each bar. Moderate and 

some affect responses are represented in the bottom section of each bar. States not included 

answered “no affect,” “don’t know,” or “no answer.” 

The primary concerns reported by the states we visited were delays in 

receiving clear written guidance on defining and reporting certain data 

elements and the lack of state input in suggesting changes to AFCARS. 

Despite the written guidance available to states in the form of regulations 

and an online policy manual, states reported that the variation in state 

policies and practices makes it difficult to interpret how to apply the 

general guidance. As a result, states consult with HHS to ensure they are 

applying the regulations appropriately. However, in commenting on a draft 

of this report, officials in Oklahoma told us that a common concern among 

the states is the lack of timely response from HHS when seeking guidance 

0

10

20

30

40

50

AFCARS

NCANDS

States reporting at least some affect in data quality

Source: GAO survey.

Imprecise

definitions for

required fields

Lack of clear,

documented

guidance

Difficulty

accessing

technical

assistance

and guidance

45

35

41

25

35

17

17

28

9

26

11

30

5

20

6

29 4

13
Page 23 GAO-04-267T 

on how to report data. In addition, officials in New York explained they 

have made it a practice to check the HHS Web site on a regular basis for 

current guidance but have not found it a useful tool, and may turn to other 

states for guidance on AFCARS reporting. In commenting on a draft of this 

report, HHS explained that it first refers states to its Web site for 

information and believes that the available guidance addresses states’ 

concerns in most instances. In addition, the states that have had an 

AFCARS review experienced delays in obtaining guidance on how to 

proceed following the on-site review. For example, Texas officials 

reported that the state sought clarification on its improvement plan and 

submitted additional questions to HHS following the review. However, 

when we spoke with the state officials, they said that they had been 

waiting 3 months for a response on how to proceed. An HHS official told 

us that since the review process is relatively new, the agency is still 

developing a process to respond to the states and recognizes that it has 

not been responsive to the states already reviewed. In addition, HHS is 

taking steps to gather feedback from states and other users of AFCARS 

data to determine how to improve the system to make the data more 

accurate and usable. As a part of these efforts, HHS has published a 

Federal Register notice soliciting comments and held focus group 

meetings at national conferences. The difficulties states face in receiving 

federal guidance and assistance, as well as the other challenges they face 

in reporting data, may negatively affect the reliability of the data available 

in AFCARS and NCANDS. 

Some states are using a variety of practices to address the challenges 

associated with developing SACWIS and improving data reliability, 

although no formal evaluations of their effectiveness are available. To 

address the challenge of developing a system to meet statewide needs, 

states relied on caseworkers and supervisors from local offices to assist in 

the design and testing of the system. Few states reported in our survey 

strategies to overcome the other key challenges, such as limited funding 

and the difficulty of securing knowledgeable contractors, but some states 

we visited have devised some useful approaches. To improve data 

reliability, the five states we visited routinely review their data to identify 

data entry errors so that managers can ensure that the missing data are 

entered appropriately. 

States Are Using 

Various Practices to 

Overcome System 

Development 

Challenges and 

Improve Data on 

Children’s 

Experiences 
Page 24 GAO-04-267T 

To overcome development challenges, survey respondents emphasized the 

importance of including system users in the various phases of completing 

SACWIS—planning, design, development, testing, and implementation. 

Past GAO work and other research efforts have determined similar 

approaches as best practices in building information systems.24 Forty-four 

of the 46 states responding to our survey that they are developing or 

operating a SACWIS indicated that they relied on internal users, such as 

caseworkers and supervisors, in the development of their systems and 34 

of these states said that they were extremely helpful participants. The 

extent to which the users were involved in development differed across 

the states. For example, in Texas, caseworkers from all of their child 

welfare regions were recruited to provide input on design and 

development, as well as during initial testing, pilot testing, and 

implementation of the system. Arkansas reported establishing a committee 

made up of users to review the work plan and sign off on recommended 

changes. 

Ten states noted that user input should not be limited to frontline workers, 

such as caseworkers, but should include representatives from other areas 

of the agency, such as the financial staff, and other agencies that serve 

children, such as child support enforcement.25 While not one of the most 

common challenges reported in our survey, New Hampshire reported that 

one of its challenges with meeting its SACWIS timeframe was not working 

collaboratively with other agencies, such as Temporary Assistance for 

Needy Families (TANF)26 and child support enforcement, to develop the 

payment component of SACWIS. To attempt to overcome this challenge, 

26 of the 46 states responding to our survey that they are developing or 

operating a SACWIS indicated that they included external public agency 

 24See U.S. General Accounting Office, Executive Guide: Improving Mission Performance 

Through Strategic Information Management and Technology, GAO/AIMD-94-115 

(Washington, D.C.: May 1, 1994); Center for Technology in Government, University of 

Albany, SUNY. Tying a Sensible Knot: A Practical Guide to State-Local Information 

Systems. Albany, N.Y., June 1997. 

25The Child Support Enforcement Program is a joint federal, state, and local partnership 

that was established in 1975 under Title IV-D of the Social Security Act. Each state runs a 

child support program, which provides four major services: locating noncustodial parents, 

establishing paternity, establishing child support obligations, and collecting child support 

for families. 

26In 1996, the Congress created the block grant Temporary Assistance for Needy Families 

program replacing the Aid to Families with Dependent Children (AFDC) and related 

welfare programs. States were given increased flexibility in designing the eligibility criteria 

and benefit rules, which require work in exchange for time-limited benefits. 

States Are Primarily 

Relying on SACWIS Users 

to Overcome Some of the 

Challenges to Completing 

Their Systems 
Page 25 GAO-04-267T 

users, and 23 reported using representatives from other state agencies that 

serve children in developing their SACWIS. 

In addition to seeking input from caseworkers and other system users 

while developing SACWIS, many states continue to include users as a part 

of the implementation teams, to serve as contacts in the field and provide 

ongoing assistance, and to provide input on system enhancements. 

Alabama responded in our survey that the state had “mentors” in each 

county to help caseworkers adjust to the new system. These mentors 

continue to provide ongoing support now that the system is implemented. 

Oklahoma recruits experienced child welfare field staff for its SACWIS 

help desk because of their knowledge of the system and child welfare 

policy and practice. 

Although states faced other challenges in completing their SACWIS, few 

reported implementing approaches to overcome the barriers. According to 

survey results, a common problem states faced in developing SACWIS was 

receiving insufficient state funding for development. States did not report 

in our survey, however, approaches for obtaining more funding for 

developing SACWIS, and few states reported developing strategies in an 

attempt to overcome the challenges associated with tight budgets for 

maintaining their systems. For example, Iowa officials engaged in careful 

planning with system users to ensure that they addressed the highest 

priorities when enhancing the system. In particular, the officials reported 

that maintaining tight control over the development and maintenance 

processes helps them avoid investing inordinate amounts of resources to 

make corrections to the system. Similarly, few states reported on 

approaches to overcome the challenge of finding contractors with 

knowledge of child welfare practice. However, Iowa officials explained 

that once the contract staff are hired, they are required to attend the same 

training as new caseworkers to ensure that they are familiar with the 

state’s child welfare policies and to familiarize themselves with casework 

practices. 

Twenty-eight states reported using approaches to help caseworkers 

identify the data elements that are required for federal reporting and to 

help them better understand the importance of entering timely and 

accurate data. Ten states responding to our survey reported reviewing the 

federal reporting requirements in training sessions as a way to improve 

data quality. For example, Tennessee reported that the state added a 

component about AFCARS to the initial and ongoing training workers 

receive about using SACWIS. The curriculum addresses the AFCARS 

States Use Strategies, such 

as Producing Reports That 

Identify Missing Data, in an 

Attempt to Improve the 

Reliability of the Data 

Reported to HHS 
Page 26 GAO-04-267T 

report in general and the individual data elements to help the caseworkers 

better understand the purpose of collecting the information. In Nebraska, 

a “desk aid” that explains the data elements and where and why to enter 

them in the system is available on the caseworkers’ computer desktops. In 

addition, New York has developed a step-by-step guide explaining to 

workers how NCANDS data should be entered, with references to the 

policy or statute requiring the information. 

To improve data reliability, some states have designed their information 

systems with special features to encourage caseworkers to enter the 

information. Four states responding to our survey and three states we 

visited designed their SACWIS with color-coded fields to draw attention to 

the data elements that caseworkers are required to enter. Colorado, Iowa, 

New York, and Oklahoma have built into their systems alerts—also known 

as “ticklers”—to remind caseworkers and supervisors of tasks that they 

need to complete. For example, in Oklahoma, a stoplight icon on the 

caseworker’s computer desktop reminds the worker when tasks are due. A 

green light indicates that nothing is due within 5 days; a yellow light means 

that something is due within 5 days; and a red light means that something 

is overdue. Caseworkers and supervisors in the states we visited had 

mixed responses about the usefulness and effectiveness of the alerts. 

Some caseworkers found them to be a nuisance, while other caseworkers 

and supervisors found them to be useful tools in managing workloads and 

prioritizing daily tasks. 

Six states reported that the best way to improve data quality was to use 

the data in published reports and hold the caseworkers and supervisors 

accountable for the outcomes of the children in their care. In addition, six 

states responding to our survey reported using the data available in their 

information systems to measure state outcomes similar to the CFSR. State 

officials reported that this approach is an effective way to get local offices 

invested in the quality of the data. For example, North Carolina publishes 

monthly reports for each county comparing their performance on state 

data indicators, such as the length of time children spend in care, to 

counties of similar size and the state as a whole. County officials reported 

that these reports encourage workers to improve the quality of the data 

collected and entered into the state system since their performance is 

being widely published and compared with that of other counties. 

In addition, all the states we visited reported that frequent review of their 

data, such as using software from HHS to test their AFCARS and NCANDS 

data to pinpoint data entry errors prior to submitting them to HHS, has 

helped improve data quality. When the states identify poor data, they alert 
Page 27 GAO-04-267T 

the caseworkers and supervisors of needed corrections and data entry 

improvements. For example, Colorado runs these reports about four to 

five times a year, with one run occurring approximately 6 weeks before 

each AFCARS submission. When the data specialists find errors, they 

notify the caseworker to clean up the data. 

While most states are developing statewide information systems, 

challenges with data reliability remain. Although SACWIS development is 

delayed in many states, state officials recognize the benefits of having a 

uniform system that enhances the states’ ability to monitor the services 

provided and the outcomes for children in their care. Although states 

began reporting to NCANDS in 1990 and were mandated to begin reporting 

to AFCARS in 1995, most states continue to face challenges providing 

complete, accurate, and consistent data to HHS. In addition, the results of 

more recent HHS efforts, such as conducting AFCARS-related focus 

groups, are unknown. Reliable data are essential to the federal 

government’s development of policies that address the needs of the 

children served by state child welfare agencies and its ability to assist 

states in improving child welfare system deficiencies. Without welldocumented, clearer guidance and the completion of more comprehensive 

reviews of states’ AFCARS reporting capabilities, states are limited in 

overcoming challenges that affect data reliability. Because these 

challenges still remain, HHS may be using some questionable data as the 

foundation for national reports and may not have a clear picture of how 

states meet the needs of children in their care. 

To improve the reliability of state-reported child welfare data, we 

recommended in our July 2003 report that the Secretary of HHS consider, 

in addition to HHS’s recent efforts to improve AFCARS data, ways to 

enhance the guidance and assistance offered to states to help them 

overcome the key challenges in collecting and reporting child welfare 

data. These efforts could include a stronger emphasis placed on 

conducting AFCARS reviews and more timely follow-up to help states 

implement their improvement plans or identifying a useful method to 

provide clear and consistent guidance on AFCARS and NCANDS 

reporting. ACF generally agreed with our findings and commented that the 

report provides a useful perspective of the problems states face in 

collecting data and of ACF’s effort to provide ongoing technical assistance 

to improve the quality of child welfare data. In response to our 

recommendation, ACF said that we categorized its efforts as “recent” and 

did not recognize the long-term efforts to provide AFCARS- and NCANDSrelated guidance to the states. Although we did not discuss each effort in 

Concluding 

Observations 
Page 28 GAO-04-267T 

depth, we did mention the agency’s ongoing efforts in our report. ACF also 

noted in its comments that the data definitions need to be updated and 

revised and said it is currently in the process of revising the AFCARS 

regulations to further standardize the information states are to report—

which we acknowledged in our report. ACF also commented that it is 

firmly committed to continue to support the states and to provide 

technical assistance and other guidance as its resources will permit. ACF 

commented that it provided increased funding to the National Resource 

Centers in fiscal year 2003, and it believed that this increase will improve 

ACF’s ability to provide assistance to the states. After receiving the draft 

report for comment, HHS separately provided information on an 

additional service the National Resource Center for Information 

Technology in Child Welfare provides to states. More recently, HHS said 

that it would be creating policy guidance that will delineate what will 

happen if a state fails to complete its SACWIS within a reasonable time 

frame. For example, funding may become contingent on successful 

completion of specific milestones. 

Mr. Chairman, this concludes my prepared statement. I would be pleased 

to respond to any questions that you or other members of the 

Subcommittee may have. 

For further contacts regarding this testimony, please call Cornelia M. 

Ashby at (202) 512-8403. Individuals making key contributions to this 

testimony include Diana Pietrowiak and Sara Schibanoff. 

GAO Contact and 

Acknowledgments 
Page 29 GAO-04-267T 

Child Welfare: Enhanced Federal Oversight of Title IV-B Could Provide 

States Additional Information to Improve Services. GAO-03-956. 

Washington, D.C.: September 12, 2003. 

Child Welfare: Most States Are Developing Statewide Information 

Systems, but the Reliability of Child Welfare Data Could Be Improved. 

GAO-03-809. Washington, D.C.: July 31, 2003. 

Child Welfare and Juvenile Justice: Several Factors Influence the 

Placement of Children Solely to Obtain Mental Health Services. 

GAO-03-865T. Washington, D.C.: July 17, 2003. 

Child Welfare and Juvenile Justice: Federal Agencies Could Play a 

Stronger Role in Helping States Reduce the Number of Children Placed 

Solely to Obtain Mental Health Services. GAO-03-397. Washington, D.C.: 

April 21, 2003. 

Child Welfare: HHS Could Play a Greater Role in Helping Child Welfare 

Agencies Recruit and Retain Staff. GAO-03-357. Washington, D.C.: March 

31, 2003. 

Human Services: Federal Approval and Funding Processes for States’ 

Information Systems. GAO-02-347T. Washington, D.C.: July 9, 2002. 

Foster Care: Recent Legislation Helps States Focus on Finding 

Permanent Homes for Children, but Long-Standing Barriers Remain. 

GAO-02-585. Washington, D.C.: June 28, 2002. 

Human Services Integration: Results of a GAO Cosponsored Conference 

on Modernizing Information Systems. GAO-02-121. Washington, D.C.: 

January 31, 2002. 

District of Columbia Child Welfare: Long-Term Challenges to Ensuring 

Children’s Well-Being. GAO-01-191. Washington, D.C.: December 29, 2000. 

Child Welfare: New Financing and Service Strategies Hold Promise, but 

Effects Unknown. GAO/T-HEHS-00-158. Washington, D.C.: July 20, 2000. 

Welfare Reform: Improving State Automated Systems Requires 

Coordinated Federal Effort. GAO/HEHS-00-48. Washington, D.C.: April 27, 

2000. 

Related GAO Products 
Page 30 GAO-04-267T 

Foster Care: States’ Early Experiences Implementing the Adoption and 

Safe Families Act. GAO/HEHS-00-1. Washington, D.C.: December 22, 1999. 

Foster Care: HHS Could Better Facilitate the Interjurisdictional 

Adoption Process. GAO/HEHS-00-12. Washington, D.C.: November 19, 

1999. 

Foster Care: Effectiveness of Independent Living Services Unknown. 

GAO/HEHS-00-13. Washington, D.C.: November 5, 1999. 

Foster Care: Kinship Care Quality and Permanency Issues. 

GAO/HEHS-99-32. Washington, D.C.: May 6, 1999. 

Juvenile Courts: Reforms Aim to Better Serve Maltreated Children. 

GAO/HEHS-99-13. Washington, D.C.: January 11, 1999. 

Child Welfare: Early Experiences Implementing a Managed Care 

Approach. GAO/HEHS-99-8. Washington, D.C.: October 21, 1998. 

Foster Care: Agencies Face Challenges Securing Stable Homes for 

Children of Substance Abusers. GAO/ HEHS-98-182. Washington, D.C.: 

September 30, 1998. 

Managing Technology: Best Practices Can Improve Performance and 

Produce Results. GAO/T-AIMD-97-38, January 31, 1997. 

Child Welfare: HHS Begins to Assume Leadership to Implement National 

and State Systems. GAO/AIMD-94-37. Washington, D.C.: June 8, 1994. 

Executive Guide: Improving Mission Performance Through Strategic 

Information Management and Technology. GAO/AIMD-94-115. 

Washington, D.C.: May 1, 1994. 

(130344) 
This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately. 
The General Accounting Office, the audit, evaluation and investigative arm of 

Congress, exists to support Congress in meeting its constitutional responsibilities 

and to help improve the performance and accountability of the federal 

government for the American people. GAO examines the use of public funds; 

evaluates federal programs and policies; and provides analyses, 

recommendations, and other assistance to help Congress make informed 

oversight, policy, and funding decisions. GAO’s commitment to good government 

is reflected in its core values of accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost is 

through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and fulltext files of current reports and testimony and an expanding archive of older 

products. The Web site features a search engine to help you locate documents 

using key words and phrases. You can print these documents in their entirety, 

including charts and other graphics. 

Each day, GAO issues a list of newly released reports, testimony, and 

correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site 

daily. The list contains links to the full-text document files. To have GAO e-mail 

this list to you every afternoon, go to www.gao.gov and select “Subscribe to e-mail 

alerts” under the “Order GAO Products” heading. 

The first copy of each printed report is free. Additional copies are $2 each. A 

check or money order should be made out to the Superintendent of Documents. 

GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a 

single address are discounted 25 percent. Orders should be sent to: 

U.S. General Accounting Office 

441 G Street NW, Room LM 

Washington, D.C. 20548 

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm 

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800 

U.S. General Accounting Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548 

GAO’s Mission 

Obtaining Copies of 

GAO Reports and 

Testimony 

Order by Mail or Phone 

To Report Fraud, 

Waste, and Abuse in 

Federal Programs 

Public Affairs