U.S. Department of Justice 

Office of Justice Programs 

Office of Juvenile Justice and Delinquency Prevention 

Court Performance Measures 

in Child Abuse and Neglect Cases 

User’s Guide 

U.S. Department of Justice 

Office of Justice Programs 

Office of Juvenile Justice and Delinquency Prevention 
U.S. Department of Justice

Office of Justice Programs

810 Seventh Street NW.

Washington, DC 20531

Eric H. Holder, Jr.

Attorney General

Laurie O. Robinson

Acting Assistant Attorney General

Jeff Slowikowski

Acting Administrator

Office of Juvenile Justice and Delinquency Prevention

Office of Justice Programs

Innovations • Partnerships • Safer Neighborhoods

www.ojp.usdoj.gov

Office of Juvenile Justice and Delinquency Prevention

www.ojp.usdoj.gov/ojjdp

The Office of Juvenile Justice and Delinquency Prevention is a component of the Office of Justice 

Programs, which also includes the Bureau of Justice Assistance; the Bureau of Justice Statistics; the 

Community Capacity Development Office; the National Institute of Justice; the Office for Victims 

of Crime; and the Office of Sex Offender Sentencing, Monitoring, Apprehending, Registering, and 

Tracking (SMART).
Court Performance Measures in Child 

Abuse and Neglect Cases: User’s Guide 

to Nonautomated Data Collection

Sophia I. Gatowski, Ph.D., Assistant Director

Permanency Planning for Children Department

National Council of Juvenile and Family Court Judges

Shirley Dobbin, Ph.D.

National Council of Juvenile and Family Court Judges

Original Printing December 2008

Second Printing April 2009

NCJ 223569
This publication was prepared under grant number 2003–JS–FX–K008 from the Office of Juvenile Justice and Delinquency Prevention, U.S. Department of 

Justice, and grant number 90XW0008 from the U.S. Department of Health and Human Services, Administration for Children, Youth and Families, Children’s 

Bureau. Points of view or opinions expressed in this publication are those of the authors and do not necessarily reflect the official position or policies of 

the funding agencies, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Department of Justice or 

the U.S. Department of Health and Human Services. This information is in the public domain. Readers are encouraged to copy portions of the text which 

are not the property of copyright holders and share them, but please credit the Office of Juvenile Justice and Delinquency Prevention, U.S. Department of 

Justice, and the Children’s Bureau, U.S. Department of Health and Human Services. 
Preface 

In ensuring the protection and welfare of children, the 

Federal Government has concentrated on three primary 

goals: safety, permanency, and well-being for abused 

and neglected children. The Government has led efforts 

to ensure that child welfare agencies, courts, and other 

stakeholders work together to achieve these worthy goals. 

In 1997, the Adoption and Safe Families Act (ASFA) further 

focused child welfare agencies and courts on system 

reforms organized around these goals. The ASFA also 

emphasized that courts play a crucial role in achieving 

positive outcomes for vulnerable children. 

The Federal Government recognizes that everyone involved 

in the protection of children is committed to the goals 

of safety, permanency, and well-being for every child. 

However, commitment to these goals is not enough. As 

stakeholders in whom the public has placed its trust, we 

must commit to a continuous process of improving and 

strengthening our dependency systems and cross-system 

supports. Performance measurement is only one step in 

that process, but it is a critical first step. To better serve 

and protect vulnerable children, we must first know how 

our current systems are doing. 

Two Federal agencies—the U.S. Department of Health 

and Human Services’ Children’s Bureau and the U.S. 

Department of Justice’s Office of Juvenile Justice and 

Delinquency Prevention (OJJDP)—are cosponsoring a 

broad-based effort to measure the progress of juvenile 

and family courts in addressing the needs of abused and 

neglected children. This effort models the Federal ideals 

of collaboration and cooperation. It blends information and 

experience from two key initiatives: the Children’s Bureau 

performance measurement project and OJJDP’s Strengthening Abuse and Neglect Courts Act (SANCA) project. Three 

of the Nation’s leading court reform organizations—the 

American Bar Association, the National Center for State 

Courts, and the National Council of Juvenile and Family 

Court Judges—have provided technical support. 

The Toolkit for Court Performance Measures in Child Abuse 

and Neglect Cases is the result of this collaborative effort. 

The Toolkit provides practical, comprehensive guidance on 

how to undertake performance measurement and move 

toward more efficient and effective dependency court 

operations. Pilot tested in 12 diverse sites, the Toolkit

reflects a breadth and richness of experience that will 

make it useful for any juvenile or family court.

The Toolkit could not have been produced without the 

combined expertise and leadership of the 3 court reform 

organizations and the cooperation of the 12 pilot sites. 

Working together, all of these contributors demonstrated 

that performance measurement can be done in any court 

and that it is essential to improving how we address the 

needs of abused and neglected children.

About the Toolkit 


Performance Measurement: 

A Critical Need 

Developing objective and qualitative measurements of 

practice is essential to a court’s capacity to improve the 

effectiveness and efficiency of its operations and to sustain 

those improvements. Like child welfare agencies, juvenile 

and family courts must focus not only on the timeliness 

of case processing and decisionmaking, but also on the 

quality of the process and the outcomes resulting from the 

court’s efforts. 

Courts must focus on child safety by assessing their safety 

performance data and developing plans for improving 

the safety of children under their jurisdiction. Courts also 

must focus on ensuring secure, permanent homes for 

children in foster care and must improve their effectiveness in achieving permanency. In addition, courts need to 

determine how well they are protecting the rights of the 

children and adults who come before them. Finally, courts 

need to set aspirational performance goals in each of these 

areas—goals designed to focus efforts, motivate staff, 

evaluate achievements, and lead to better outcomes for 

children and families. 

Few courts currently have the capacity to effectively 

measure their performance in child abuse and neglect 

cases. Whereas for-profit businesses have long taken 

for granted the need for performance measurement, it is 

still a relatively new concept for the Nation’s courts. Yet, 

without this essential information, courts with jurisdiction 

over abuse and neglect cases cannot know what types of 

improvements they need to make and whether their efforts 

to improve are working. 

Performance measurement makes it possible for courts 

to diagnose and assess areas in need of improvement 

and review progress in those areas. In this process, courts 

build improvements from a baseline of current practices 

and then conduct regular reassessments as reforms are 

implemented. 

The purpose of the measures in the Toolkit for Court Performance Measures in Child Abuse and Neglect Cases is 

to help courts establish their baseline practices; diagnose 

what they need to improve; and use that information to 

make improvements, track their efforts, and identify, document, and replicate positive results. 

By capturing data for the 30 measures in the Toolkit, courts 

will be able to evaluate four areas of operation: child safety, 

child permanency, due process or fairness, and timeliness. 

u	Safety (Measures 1A and 1B). The goal of these two 

measures is to ensure that children are protected from 

abuse and neglect while under court jurisdiction. The 

performance outcome promoted by these measures is 

based on the principle of “first, do no harm.” Children 

should be protected from abuse and neglect, no child 

should be subject to maltreatment while in placement, 

and children should be safely maintained in their homes 

whenever possible and appropriate. 

u Permanency (Measures 2A–2E). The goal of these five 

measures is to ensure that children have permanency 

and stability in their living situations. The permanency 

measures are closely related to timeliness measures 

but also include additional considerations. With this 

category, courts assess whether children change 

placements, whether cases achieve permanent legal 

status, and whether children reenter foster care 

(a possible safety issue as well). The permanency 

measures encourage courts to examine the “bigger 

picture” of the court experience for the abused or 

neglected child. In using the permanency measures, 

a court will need to obtain information from partner 

agencies such as the State child welfare system or 

private providers who track children placed in foster 

care. 

u	Due Process (Measures 3A–3J). The goal of these 10 

measures is for the court to decide cases impartially 

and thoroughly based on evidence brought before it. 

Due process measures address the extent to which 

individuals coming before the court are provided basic 

protections and are treated fairly. 

u	Timeliness (Measures 4A–4M). The goal of these 

13 measures is to minimize the time from the 

filing of the petition or emergency removal order to 

permanency. Courts generally are most familiar with 

timeliness measures. These measures help courts 

identify areas where they are doing well and areas 
User’s Guide to Nonautomated Data Collection
 

where improvement is needed. To ensure that courts 

can pinpoint specific stages of the hearing process 

in need of improvement, these measures must be 

comprehensive (applied to all stages of proceedings) 

and sufficiently detailed. 

None of the measures includes a standard or benchmark 

of performance. Rather, the measures suggest a base of 

experience from which to develop reasonable and achievable benchmarks. The measures are designed to help 

courts improve services to maltreated children and their 

families, and it is important for courts to measure their 

progress toward achieving that goal. The measures are 

intended to be part of a process of continuing improvement. They are also intended to be developmental; that is, 

the measures can be refined as more is learned about the 

factors associated with a model process for handling child 

abuse and neglect cases. 

The developers of the Toolkit expect courts to collaborate 

with child welfare agencies in applying these measures; 

for this reason, the court performance measures in the 

Toolkit are designed to be compatible with the Child 

and Family Services Review (CFSR) outcome measures 

developed for child welfare agencies. The Toolkit developers encourage courts to work with child welfare agencies 

to establish not only minimum acceptable standards of 

performance but also aspirational goals that challenge 

both stakeholders to improve even further. 

The national court performance measures also reinforce 

the goals of other current Federal reform programs and 

legislation, including the Court Improvement Program (CIP) 

and the Adoption and Safe Families Act (ASFA). These initiatives recognize that courts, as well as State child welfare 

agencies, are crucial stakeholders in achieving positive 

outcomes for maltreated children who become involved in 

the child welfare system. Court performance has an impact 

on overall system performance in achieving safety and 

permanence for these children in a fair and timely manner. 

History of the Performance 

Measures 

The history of court performance measurement for child 

abuse and neglect cases began with a miniconference 

held in Scottsdale, AZ, on May 5, 1998. The miniconference 

was cosponsored by the Court Improvement Conference 

and the Conference of State Court Administrators’ Court 

Statistics Project Advisory Committee. Participants worked 

with the following resource materials: 

u	Trial court performance standards and measurement 

system (prepared by the National Center for State Courts 

(NCSC) and funded by the Bureau of Justice Assistance 

(BJA)). These standards touched on five fundamental 

purposes of courts: access to justice; expediency and 

timeliness; equality, fairness, and integrity; independence 

and accountability; and public trust and confidence. 

Although general trial court standards could be applied 

to juvenile and family courts, miniconference participants 

perceived a need for measures and standards tailored 

specifically to child abuse and neglect cases. 

u	Draft sets of child abuse and neglect performance 

measures developed by the American Bar Association 

(ABA) Center on Children and the Law, NCSC, and 

Walter R. McDonald & Associates, with comments and 

suggestions from the National Council of Juvenile and 

Family Court Judges (NCJFCJ). 

u	A set of measurement goals from the National CourtAppointed Special Advocates Association. 

u	Best practice recommendations for handling child abuse 

and neglect cases, outlined by NCJFCJ in RESOURCE 

GUIDELINES: Improving Court Practice in Child Abuse 

and Neglect Cases. 

u	Technical assistance bulletins on information 

management in child abuse and neglect cases and 

judicial workload assessment in dependency cases, 

developed by NCJFCJ. 

Miniconference participants summarized key performance 

measures for dependency courts in a consensus statement, which was then presented in the following forums: 

u	To participants in the ABA Summit on Unified Family 

Courts, May 1998. 

u	To child welfare professionals at the Permanency 

Partnership Forum, June 1998. 

u	To managers of statewide automated child welfare 

information systems at the conference “Continuing To 

Build the Future: Using Automation for Children and 

Families,” September 1998. 

u	To juvenile and family court judges at the NCJFCJ 

Annual Conference, July 1998. 

u	To judges, court administrators, and child welfare 

workers at “Improving Outcomes for Abused and 

Neglected Children,” a symposium sponsored by the 

Bureau of Justice Assistance and the David and Lucille 

Packard Foundation, June 2000. 

vi 
About the Toolkit 

In addition, Dr. Ying-Ying Yuan prepared a critique of the 

performance measures in a September 1999 report for 

the ABA entitled “Feasibility of Implementing Court SelfAssessment Measures for Dependency Cases.” 

The measures were then revised to reflect input from 

these sources, and the revisions were summarized by Dr. 

Victor E. Flango in an article entitled “Measuring Progress 

in Improving Court Processing of Child Abuse and Neglect 

Cases” (Family Court Review, Volume 39, pp.158–169, 

April 2001). 

In their present form, the court performance measures 

in the Toolkit grew out of the Attaining Permanency for 

Abused and Neglected Children Project, conducted jointly 

by the ABA Center on Children and the Law, NCSC, and 

NCJFCJ, with funding from the David and Lucile Packard 

Foundation. Over a 3-year period, these measures were 

pilot tested to determine their applicability in different 

types of courts with different measurement needs and data 

collection capabilities. The measures were also examined 

for compatibility with the CFSR outcome measures for 

child welfare agencies. One result of this effort was the 

2004 publication Building a Better Court: Measuring and 

Improving Court Performance and Judicial Workload in 

Child Abuse and Neglect Cases. This publication described 

dependency court performance measures for safety, 

permanency, due process, and timeliness. It also outlined a 

process for assessing judicial workload that encompasses 

both on-the-bench and off-the-bench aspects of dependency work. 

The Children’s Bureau Project 

After publishing Building a Better Court, the ABA, NCSC, 

and NCJFCJ received funding from the Children’s Bureau 

of the U.S. Department of Health and Human Services to 

support efforts by courts to improve their handling of child 

abuse and neglect cases. The Children’s Bureau project 

provided targeted technical assistance to six sites: Charlotte, NC; Clackamas County, OR; Little Rock, AR; Minneapolis, MN; New Orleans, LA; and Omaha, NE. During this 

project, the partnering organizations were able to test and 

refine the court performance measures, and data collection 

instruments at these sites. 

The Children’s Bureau project helped the six sites do the 

following: 

u	Use the performance measures outlined in Building 

a Better Court—compatible with Adoption and Foster 

Care Analysis and Reporting System (AFCARS) and 

CFSR measures—to assess their performance in abuse 

and neglect cases. This included evaluating each site’s 

capacity to generate data for each of the performance 

measures. 

u	Examine judicial workloads to determine whether 

judges were able to spend enough time on child abuse 

and neglect cases to make timely and well-considered 

decisions in these cases. The partnering organizations 

disseminated information about and provided technical 

assistance in judicial workload assessment. 

u	Develop a court-specific strategic plan for using 

performance and workload data to achieve increased 

accountability and better court performance. 

A major goal of the Children’s Bureau project was to 

enhance the sites’ self-assessment capacity so they would 

be able to track and measure their own progress after their 

involvement in the project ended. This strengthened capacity also makes the sites better able to assess their ASFA 

compliance and CIP implementation. The project sought to 

enable project sites—and eventually all courts handling 

abuse and neglect cases—both to begin a process of 

continuing self-improvement and to help child welfare 

agencies determine the impact of court proceedings on 

achievement of CFSR outcomes. 

The Strengthening Abuse and Neglect 

Courts Act Project 

While the Children’s Bureau project was underway, the 

ABA, NCSC, and NCJFCJ received funding from the 

U.S. Department of Justice’s Office of Juvenile Justice 

and Delinquency Prevention (OJJDP) to help courts use 

automated management information systems to improve 

their performance in child abuse and neglect cases. The 

Strengthening Abuse and Neglect Courts Act (SANCA) 

project supported SANCA implementation in six States: 

Colorado, Florida, Georgia, Idaho, New Jersey, and Virginia. 

At each site, the SANCA project partners helped improve 

automated management information systems, implement performance measurement, develop case-tracking 

capabilities, and perform other management information 

system functions specifically for child abuse and neglect 

cases. The SANCA project provided this assistance through 

meetings of representatives from all SANCA sites, onsite 

training and technical assistance to each site, and offsite 

consultation. 

vii 
viii 

User’s Guide to Nonautomated Data Collection
 

The SANCA project has not focused on improving court 

information systems as an end in itself. Rather, the focus 

has been on improving these systems in ways that will 

have the greatest positive impact on efforts to improve 

quality and timeliness in courts’ handling of abuse and 

neglect cases, to target reforms for court improvement 

efforts, and, ultimately, to improve the lives of abused and 

neglected children. 

The Toolkit Volumes 

All the aforementioned work has culminated in the production of the Toolkit for Court Performance Measures in Child 

Abuse and Neglect Cases. The Toolkit content is informed 

by the experiences of the Children’s Bureau and SANCA 

project sites. 

In addition to providing detailed guidance about court performance measures for child abuse and neglect cases, the 

Toolkit offers a general approach—a way of thinking—that 

can help dependency courts successfully implement a 

performance measurement process. Using the Toolkit, 

dependency courts can: 

u	Establish a baseline of current practice, diagnose what 

they need to improve, and use that information to build 

and track improvement efforts. 

u	Measure their progress in achieving the goals of safety, 

permanency, and well-being for children. 

u	Identify and document practices that are achieving 

positive results and replicate those results. 

The Toolkit includes the five volumes described below. 

Although each volume focuses on a particular audience, 

the Toolkit developers encourage everyone involved in 

court performance measurement for abuse and neglect 

cases to consult all the volumes for instruction, guidance, 

and inspiration. 

Court Performance Measures in Child Abuse and 

Neglect Cases: Key Measures. This booklet outlines nine 

measures that the national partners have identified as 

key to determining court performance in child abuse and 

neglect cases. The booklet succinctly discusses the goal 

of each measure, data requirements, calculation and interpretation, and important related measures. It is an ideal 

tool for making the case for performance measurement to 

legislators, funders, and other high-level decisionmakers. 

Court Performance Measures in Child Abuse and 

Neglect Cases: Implementation Guide. This step-by-step 

guide provides practical advice on how to set up a performance measurement team, assess capacity (determine 

which measures the team can currently implement and 

which measures will require capacity building), prioritize 

among measurement needs, plan data collection activities, and use the data generated through the performance 

measurement process to plan reforms. The Implementation Guide uses examples from the Children’s Bureau and 

SANCA project sites to illustrate key points. It also highlights lessons learned from the sites about performance 

measurement approaches, as well as challenges and 

strategies for overcoming those challenges. Performance 

measurement teams and project managers will find the 

Implementation Guide helpful as they plan and implement 

a performance measurement program and use results to 

drive improvement efforts. 

Court Performance Measures in Child Abuse and 

Neglect Cases: Technical Guide. This comprehensive 

volume describes all 30 court performance measures for 

child abuse and neglect cases. The Technical Guide details 

the goals and purpose of each measure, discusses alternate or proxy measures, provides step-by-step specifications for calculating the measures, articulates what data 

elements need to be collected to produce each measure, 

suggests ways to present data effectively, and provides 

examples of how data obtained for each measure can be 

used in reform efforts. The Technical Guide also includes a 

detailed dictionary of technical terms and a flowchart outlining the typical child abuse and neglect hearing process. 

This volume is ideal for project managers and information 

technology (IT) staff tasked with obtaining performance 

measures. It will give them an indepth understanding of 

all the measures, what is needed to obtain data for the 

measures, and how to report findings in a way that is easily understood by various target audiences. 

Court Performance Measures in Child Abuse and 

Neglect Cases: User’s Guide to Nonautomated Data 

Collection. Some courts may lack automated systems for 

gathering performance measurement data on abuse and 

neglect cases. Even if a court has adequate automation resources, certain performance measures (such as those assessing due process) may not be captured via automated 

systems. Furthermore, qualitative information can help to 

explain quantitative outcomes. This volume explains how to 

use nonautomated data collection methods—such as file 

review, court observation, interviews, and focus groups— 

to complete the performance measurement picture. 
About the Toolkit 

Court Performance Measures in Child Abuse and 

Neglect Cases: Guide to Judicial Workload Assessment. To improve their handling of abuse and neglect 

cases, courts need to be able to measure workloads as 

well as performance. Measuring judicial workloads makes 

it possible for courts to track existing resources and 

argue persuasively for additional resources when they 

are needed. This volume presents a method for obtaining 

data on judicial workloads in abuse and neglect cases 

which includes an assessment of what is required for best 

practice in these cases. Drawing on work from the pilot 

project sites, this volume discusses different approaches 

to workload analysis and provides tools for conducting 

analyses. 

Toolkit DVD and Web Site 

All Toolkit publications and related materials, such 

as presentations and instruments, are available 

on DVD and at www.courtsandchildren.org. 

ix 

Acknowledgments 

The authors wish to thank each project site who participated in the field-testing and refinement of the data collection 

instruments described in this volume. 

Sophia I. Gatowski, Ph.D. 

Assistant Director 

Permanency Planning for Children Department 

National Council of Juvenile and Family Court Judges 

Shirley Dobbin, Ph.D. 

National Council of Juvenile and Family Court Judges 

Contents

preface .................................................................................................................................................................iii

About the toolkit............................................................................................................................................v

Acknowledgments.........................................................................................................................................xi

Chapter 1. Using the Guide........................................................................................................................ 1

Chapter 2. Case File Review...................................................................................................................... 3

Chapter 3. Court Observation ............................................................................................................. 11

Chapter 4. Focus Groups.......................................................................................................................... 15

Appendixes

Appendix A. Key Performance Measures for Case File Review and Court Observations ....................... 21

Appendix B. Managing Case File Review and Court Observation Data .................................................. 25

Appendix C. Displaying Case File Review Data in a Meaningful Way ..................................................... 27

Using the Guide 


Quantitative methods involve research methods that produce data appropriate for quantitative 

or statistical analysis. When a survey, for example, produces data that allows researchers to 

calculate the percentage of responses received, 

or calculate the average, mean, or degree to 

which a particular event occurs, then that survey 

is a quantitative method. 

Qualitative methods are research methods 

that yield information that is not easily reduced to 

numbers. Court observation designed to assess 

the quality of hearings, for example, is a qualitative method. Other qualitative methods include 

interviews, which can range from semistructured 

to open-ended conversations, and focus groups, 

which are essentially group interviews. 

When no automated, or computerized, measure of court 

performance is available, courts should pursue other 

means of obtaining information about performance in 

child abuse and neglect case processing. Computer-based 

systems (outlined in other volumes of this Toolkit) may not 

effectively measure certain types of performance—such 

as those assessing due process. Court staff should explore 

other means of gathering performance data. 

Even if automated data are available, qualitative information is critical to an understanding of why a particular 

quantitative outcome was achieved. In the context of court 

performance measurement, quantitative and qualitative 

methods should be used together in order to provide a 

complete picture of court performance. This volume of the 

Toolkit provides guidance on how to use nonautomated 

data collection methods, including qualitative approaches. 

Funding from the David and Lucile Packard Foundation 

allowed the American Bar Association (ABA) Center on 

Children and the Law, the National Center for State Courts 

(NCSC), and the National Council of Juvenile and Family 

Court Judges (NCJFCJ) to design a series of data collection instruments that can help dependency courts gather 

information about court performance. These instruments 

were extensively field-tested in a number of dependency 

courts around the country, and made available to jurisdictions embarking on performance measurement.1 

With funding from the Children’s Bureau at the U.S. Departthe instruments and made them specific to each of the 

project sites involved in creating the Toolkit. 

As a result, researchers developed a set of tools that 

assess court performance in child abuse and neglect 

cases, and can be used to collect data nationally. These 

extensively field-tested tools can be tailored by individual 

courts to make them compatible with a court’s individual 

measurement needs and ability to collect data. 

The tools discussed in this volume include: 

u	Case file review form. A case file review (CFR) is a 

data collection method used to discover preexisting, 

objective data from court records (and social services 

files, if relevant). The data are then used to assess court 

performance. 

u	Court observation hearing code sheets. A court 

observation hearing code sheet allows an observer to 

systematically record data about a court’s performance 

in child abuse and neglect hearings. The method works 

especially well for evaluating due process proceedings. 

u	Expert opinion focus group guide. Focus groups 

convene a panel of experts who share their knowledge 

and opinions on specific court performance issues. 

Topics 

Nonautomated Data Collection Methods 

u Interviews. 

u Questionnaires. 

u Focus Groups. 

u Case File Review. 

u Court Observation. 

To obtain copies of the instruments referred to 

in this volume, please visit: 

www.courtsandchildren.org. 

Chapter 1 

ment of Health and Human Services, researchers refined 
User’s Guide to Nonautomated Data Collection
 

u	Interview and questionnaire forms. Interviews and 

questionnaires gather survey information about court 

performance using a sample from a population of 

relevant individuals. 

This User’s Guide to Nonautomated Data Collection 

provides the reader with instructions about how to use 

these tools to measure court performance. Each chapter 

is devoted to a specific instrument and describes the 

purpose of the instrument, provides tips on how to use 

it, and explains how the data collected can assess court 

performance on a national level. Each instrument is flexible 

enough to meet the needs of individual courts, yet each 

can meet national standards. Jurisdictions are encouraged 

to adapt the instruments to reflect their own structures, 

practices, and additional measurement needs. 

Endnotes 

1. See American Bar Association Center on Children and 

the Law, National Center for State Courts, and National 

Council of Juvenile and Family Court Judges. Building a Better Court: Measuring and Improving Court 

Performance and Judicial Workload in Child Abuse and 

Neglect Cases (Reno: NV: NCJFCJ, 2004). The instruments described in this Guide are also available online 

at www.ncjfcj.org and at www.courtsandchildren.org. 

2 
A case file review is a data collection method that uses 

preexisting, objective data from court records (and social 

u	Should cohorts of cases be reviewed based on their 

entry into or exit from the system, or should staff review 

a single, point-in-time sample of cases, regardless of 

when the cases entered the system? 

u	Should cases involving termination of parental rights 

be treated separately from adoption cases because of 

the way the court is structured (e.g., different judges or 

courts hear these two types of cases)? 

u	Should courts review files from social service agencies 

in addition to court records? For what reason (i.e., what 

data are available in the social service agency files that 

would inform the court’s performance assessment)? 

u	Who should conduct the case file review and what kind 

of training does that person need? 

u	How will court staff ensure the reliability of the information collected from the case file review (i.e., what 

Chapter 2 

Why Conduct a Case File Review? 

Data from the case file review provides important 

qualitative and quantitative information about 

court performance. These data can supplement 

data from automated management information 

systems. Although it can be a time and resource 

intensive process, conducting a court case file 

review helps court performance assessment, 

especially when automated data may not be 

available or easily accessible. 

Case File Review 


service files if relevant) to assess court performance. 

With respect to child abuse and neglect cases, a case 

file review can be used to collect information on topics 

including hearing dates and timeframes, whether one or 

more judges were involved in the case, whether all parties 

appeared (including parents, children, foster parents, and 

counsel), whether one or more attorneys were involved in 

the case, the frequency that cases undergo judicial reexamination, and whether court orders specifically addressed 

issues raised by Adoption and Safe Families Act (ASFA) 

findings (reasonable efforts, contrary to welfare findings). 

Before beginning a case file review, a court should answer 

the following questions: 

u	Is there a clearly defined purpose for the case file 

review—what types of performance should be assessed 

and what will data from the court files contribute to the 

assessment? 

u	What specific information will be gathered? 

u	How will staff select cases for review and how many 

cases need to be reviewed? 

u	Should a review include only completed court cases 

to ensure all events in a dependency proceeding are 

involved in a measurement? Or, should staff include 

open cases to focus solely on the court’s current 

caseload? 

quality assurance processes will be employed)? 

u	How should the data be analyzed? 

u	What is the most effective way to present findings? 

u	How much time and how many resources are available 

for people who review case files? 

u	What resources are needed for training, implementation, 

analysis, and report writing? 

The Case File Review Instrument: 

General Categories of Information 

The Toolkit contains a case file review instrument for collecting information that can be used to measure national 

dependency court performance. This instrument can be 

adapted to account for an individual court’s structure, 

ability to collect data, and overall performance goals. 

The instrument collects answers to the following questions 

and related information: 

u	What jurisdiction is the case in (county, circuit, etc.)? 

u	What is the coder’s name (the name of the person 

completing the review)? 

u	What is the case number? 
User’s Guide to Nonautomated Data Collection
 

If multiple children are included in a single case 

file, a primary child should be selected for coding 

purposes. Events specific to that child should be 

coded. 

u	On what date was the case coded?
 

u	Who was the judge or hearing officer?
 

u	What is the demographic information for the primary
 

child in the case?
 

G	When was the child born?
 

G	What is the child’s gender?
 

G	What is the child’s ethnicity?
 

u	Did the case involve the Indian Child Welfare Act 

(ICWA)1

? 

u	Is there a prior history of court involvement in the case? 

u	When was the child removed from his or her home? 

u	When was the initial hearing (emergency removal or 

shelter care hearing) held? 

u	When was the petition filed? 

u	Was a pretrial hearing held? 

u	Was there an adjudicatory hearing? 

G	When was it held? 

G	Was it contested? 

u	Was there a disposition hearing? 

G	Was it held in conjunction with the adjudication 

hearing? 

G	When was it held? 

u	When was the first judicial review hearing held? 

G	How many judicial review hearings were held in the 

case? 

u	Was there a permanency hearing? 

G	When was it held? 

u	Was a termination of parental rights (TPR) petition filed? 

G	When was it filed? 

u	Did the case involve mediation? 

G	When was the mediation held? 

u	Was a TPR trial held? 

G	When was that trial held? 

G	What was the final TPR judgement? 

u	Was a TPR appeal filed? 

G	When was it filed? 

u	Was a TPR appeal heard? 

u	What was decided in a TPR appeal? 

u	Was an adoption petition filed? 

u	Was an adoption finalized? 

u	When was the case closed? 

u	How many continuances occurred with each court 

event? 

G	What were the reasons for those continuances? 

u	Which parties appeared at each hearing? 

u	What legal counsel were present at each hearing? 

u	How often did legal counsel change during the case? 

u	Was the judicial officer the same officer throughout the 

case? 

u	Were ASFA findings made in each hearing? 

G	Were “reasonable efforts” findings made? 

G	Were “contrary to welfare” findings made? 

u	Why was the case closed (i.e., what was the final 

outcome of the case)? 

Case File Review and Court 

Performance Measures 

The case file review instruments included in the Toolkit 

provide important data about a number of the court performance measures. This section includes suggestions on 

how to expand the focus of the case file review instrument 

to address all court performance measures. However, each 

assessment costs money and time. So courts must decide 

which court performance measures are most important. 

Some important facts about how to make decisions about 

which measures are most important to assess are outlined 

in appendix A (p. 21). 

4 
Chapter 2: Case File Review 

How To Implement a Case File 

Review Process 

To begin the case file review process, a court must 

establish a collaborative committee to identify and prioritize court performance measures that need addressing.2 

The committee should adapt the case file review instrument to reflect a local court’s structure and terminology, 

and data collection should be prioritized according to an 

individual jurisdiction’s critical needs. Once a court has 

determined the scope of the case file review and prioritized 

which court performance measures are most critical for 

the jurisdiction, staff should decide which cases must be 

coded and move on to the actual review process. This 

requires a number of steps: 

Define the sample of cases for coding. Court staff must 

first define the target population of cases they wish to 

evaluate. The case file review will then focus on a subset 

of cases that will represent the whole population. For 

example, a target population might include all children who 

entered foster care within a specific period of time. 

Closed versus open cases. If a court decides a case file 

review should calculate how long it takes to achieve final 

permanency (reunification, adoption, guardianship), then 

a sample of closed cases should be used. Coding closed 

cases ensures that all court events have occurred and 

that the case has been resolved (i.e., that court jurisdiction has been terminated). If the case file review focuses 

on case processing events that happen early on in a case 

(e.g., early appointment of counsel) and on current case 

practice, then open case files can be coded. 

Point-in-time versus cohort sampling. 

u	Point-in-time sampling. Point-in-time sampling 

involves sampling cases of children in care on a 

particular date (e.g., all children in care on September 

30, 2005), regardless of when the child entered care. 

This analysis has a bias toward children with the longest 

lengths of stay because they are most likely to be in 

care at any given time. It does not account for when a 

child entered care and would not include children who 

have left care. 

u	Entry cohort sampling. An entry cohort sample is 

based on the date of the child’s most recent entry into 

care. This date is typically tracked by year (e.g., children 

who entered foster care in 2004). Entry cohort sampling 

tracks how children who entered foster care in the same 

year perform throughout their foster care experience 

and thus allows for comparison across years. 

u	Exit cohort sampling. An exit cohort is based upon the 

date of case closure. This date is tracked by year. An 

exit cohort provides information on the group of children 

leaving care during a specific period. Within the same 

exit cohort, some children may have entered care in 

2000 and some in 2003—those two groups of children 

may have had very different experiences in foster care. 

Sampling timeframes. When deciding which cases to 

include in the case file review sample, consider the timeframe the court staff should examine. Timeframes depend 

on the purpose and goals of the case file review process. 

Examples include: 

u	If a court wants to examine closed cases, cases should 

be drawn from a timeframe that allows enough cases 

to have reached closure. 

u	If a court wants to examine the impact of a statutory 

change or change in the law or court rule on court 

performance, then a sample of cases should be drawn 

from the timeframe before that change was enacted and 

another sample of cases drawn from the timeframe after 

that change. The two samples can then be compared for 

differences. 

u	If a court wants to examine recent case practice, then 

the sample should be drawn from the most recent 

timeframe that allows a sufficient number of cases 

to have reached stages of interest (e.g., a sufficient 

number of cases within the last year that have involved 

a permanency hearing). 

Number of cases in sample. After deciding on a target 

population, a court should determine how many cases 

need coding to represent the larger population of cases. 

This small number of cases is referred to as the “sample 

population.” The sample population of cases is the actual 

cases that will be coded. 

A number of factors should determine sample size, 

including: 

u	The purpose of the case file review. 

u	The population size (e.g., total number of petitions filed, 

total number of cases under the court’s jurisdiction). 

u	The risk of selecting cases outside of the population of 

interest (e.g., cases that were dismissed or transferred 

to another jurisdiction). 

u	The allowed sampling error (i.e., how precisely a sample 

represents the greater population—the greater the 

sample size, the smaller the error). 

5 
User’s Guide to Nonautomated Data Collection
 

Court staff must calculate a sampling statistic to determine 

sample size. A sampling statistic is a formula that produces 

the necessary sample size, creating different levels of 

precision and different confidence intervals.3 The sampling 

statistic should account for the range of possible variations in cases. The performance measurement committee 

should seek assistance from information technology staff 

or research consultants who can calculate an appropriate 

sample size. 

If a court has limited time and resources, case samples do 

not have to be proportional to target populations to provide 

useful information. For example, a court may suggest that 

the files it has chosen to review constitute a kind of “snapshot” of a certain court practice. But this snapshot should 

not be used to represent the target population. Smaller jurisdictions, with small numbers of child abuse and neglect 

cases, may not need to use a population sample, because 

they can study the entire population of child abuse and 

neglect cases. 

Once a court targets which cases will be coded and how 

many cases will be coded, it can define the scope of the 

case file review. Performing a case file review uses a 

court’s time and resources, so any review will be limited 

in size and scope. A court should keep in mind that data 

are limited relative to the size of the sample population. 

Identify and train coders. Without skilled coders, 

a court cannot produce accurate and meaningful data. 

Coders can include a wide range of individuals, including 

court administrative staff, law clerks, and interns. Students 

at a local university or law school may also be a resource. 

All coders will require training in the following subjects: 

u	The importance of court performance measurement. 

u	The specific goals of the case file review process, 

including the performance measures that will be 

addressed and how they relate to case file review data 

(i.e., how the case file review will be used to generate 

court performance data, including timeliness and due 

process data). 

u	The target population of the case file review process and 

the parameters of the sample population of hearings 

that will be coded. 

u	The structure of court hearings and timeframes for 

hearings in dependency cases. 

u	An orientation to the case file (e.g., What documents are 

included in the case file review? Which documents are 

sources for the information that will be coded on the 

case file review instrument?) 

u	An element-by-element discussion of the case file 

review instrument, including where the court can obtain 

data needed to complete each measure, definitions and 

explanations of terminology, and any other issues. 

Pretest the case file review instrument and make 

necessary revisions. The case file review instrument 

included in the Toolkit has been extensively pretested in a 

number of courts around the country. Nevertheless, each 

jurisdiction may need to modify the instrument to best 

address its specific practices and unique measurement 

needs. Even without modification, the instrument should 

be pretested with several sample case files to determine 

if coders are coding cases appropriately. This pretest will 

help determine: 

u	Whether court files include all the data necessary for 

review and measurement. If a court does not have 

access to some data, this data point can be removed 

from the case file review instrument and save coding 

time. However, the court may wish to find ways to 

collect data it currently does not have access to. 

u	The best and most reliable source of data for coding 

purposes. For example, hearing dates may be documented in several places in the court file (e.g. in the 

court order, court report, and judicial notes), but coders 

may want to develop a general coding rule that, for 

instance, only the information in the court order should 

be coded. 

u	Whether coders are consistently interpreting and coding 

information in hearings. To ensure reliability, several 

coders should code the same hearing. Courts should 

determine the source of any inconsistencies. 

Conduct case file review. Court clerks and administrative 

staff should pull the case file review sample in advance. 

Coding should occur in a secure location that maintains 

case file confidentiality and ensures proper storage of the 

files (so that files do not become lost or misplaced). Coders 

should work in a common, quiet location so that they can 

consult with each other on any specific issues, not disturb 

other staff, and maintain research protocols concerning 

confidentiality. 

Check-coding and quality control. A small sample 

should be checked by a second, experienced coder and 

any discrepancies noted. All errors should be addressed 

6 
Chapter 2: Case File Review 

For information on developing a database to 

manage case file review data, see appendix B 

(p. 25). 

with the original coder. Errors can occur for a number of 

reasons (e.g., the file review form is unclear or fails to 

provide opportunities to record needed data, the coder 

misinterprets items in the file, or the coder lacks the skills 

and ability to record data accurately). If the check reveals 

a high rate of errors, those case file review code sheets 

should be discarded and not used in analysis. Coders 

should be informed of the errors and additional training should be provided to ensure accuracy of the 

coding process. 

Analyzing Case File Review Data 

Simple descriptive statistics can help summarize the data 

collected during case file review. These statistics involve 

frequency data (a simple tally or count of data), frequency 

distributions (putting data in groups, such as grouping all 

cases by the year they opened or by county), graphical 

representations of data, and summary statistics (statistics 

that describe the typical value or most representative 

value, such as the mean or average). Case file review data 

can also be analyzed using inferential statistics.4 

The next few paragraphs will focus on how to understand 

simple descriptive statistics. In order to analyze inferential 

statistics, consult a statistical expert on your committee.5 

Descriptive Statistics 

Frequency counts. Frequency counts compute how 

many cases are in a specific category (e.g., permanency 

outcomes). When the number of cases in each category is 

added together, the total number of cases should be equal 

to the total number of cases coded. Frequencies can be 

converted into percentages: Divide the number of cases in 

each category by the total number of cases and multiply by 

100. Present both the actual numbers and the percentage 

when reporting data. 

Cross-tabulations. A cross-tabulation describes data 

on the basis of more than one variable at a time. Crosstabulations take two variables (e.g., permanency outcome 

and the child’s age) and compare the results of one 

variable against the other variable (e.g., Do permanency 

outcomes for children of different age groups differ?). They 

describe how variables interrelate. 

Graphical representations of data. Presenting statistical information visually (in graphs, charts, or tables) often 

makes it easily understood. Graphs and tables should be 

presented in as simple a format as possible. See subsequent sections of this chapter for tips on graphically 

presenting data. 

Summary statistics can help make findings easier to 

understand. Examples of summary statistics include the 

median, mean, mode, and range. 

u	Median. The median is the middle score when a set 

of scores have been arranged in order from lowest to 

highest. The median is the 50th percentile, which means 

that half the scores fall below the median and half 

the scores fall above the median. Unlike the mean, or 

average, the median is least affected by extreme scores 

at either end of a range of scores. 

u	Mean. The mean, or average, is the most commonly 

used summary statistic. The mean is the arithmetic 

average of all the scores. The mean is computed by 

adding up all the scores and then dividing by the total 

number of scores. 

u	Mode. The mode is the score that occurs most often. 

u	Range. The range represents the span between 

the lowest and highest data measured (e.g., time to 

adjudication ranged from 15 to 30 days). 

The following section discusses how to analyze data 

collected on permanency, due process, and timeliness 

measures. 

Analyzing Permanency Outcomes 

Permanency assesses if children have stable living situations and continuous family relationships and connections. 

The key court performance measures for permanency 

include the percentage of children in the case file review 

sample that achieved legal permanency, the percentage of 

children in the case file review sample that did not achieve 

legal permanency, the average time it took for a child to 

achieve permanency, and how well a court complied with 

statutory and Federal timeframes in achieving permanency. 

Calculating permanency outcomes. Data elements 

needed from the case file review include: 

u	Case number or child identification number. 

7 
User’s Guide to Nonautomated Data Collection
 

u	The date the child was removed from the home. 

u	The date adoption was finalized. 

u	The date of case closure. 

u	Reason for case closure/type of permanency achieved. 

To calculate the types of permanency outcomes being 

achieved, sort all cases in the case file review sample 

according to the reason for the case closure and record the 

number of children in each category. Divide the number of 

children in each category by the total number of children 

in e sample includes 

ses in the sample 

ber of cases in the

open cases, then the Total 52 children 

n

fr

c

umber of open ca

 the sample. If th

must be subtracted 

*Another Planned Permanent Living Arrangement 

ases should be u

om the total num

sed as the 

 sample (only closed 

52 children ÷ 250 children = 20.8 percent 

For example, for a sample of 250 clo

denominator). 

sed cases (250 Thus, 20.8 percent of children in the sample did not 

children): achieve legal permanency. 

Reunification 142 children 56.8 percent To calculate the average time to achieve permanency, calAdoption

Guardianship

 43 children 

13 children

17.2 percent 

5.2 percent 

culate the number of days or months between the case’s 

initiation (e.g., removal of the child or filing of the original 

petition) to achieving final permanency and case closure 

APPLA* 37 children 14.8 percent for each case in the sample. Find the mean or average 

Age of Majority 12 children 4.8 percent number of days or months to permanency. 

Reached For example, for a sample of five closed cases of children 

Other 3 children 1.2 percent who entered care in January, 2004: 

Total 250 children 100 percent Removal Date Final Number 

*Another Planned Permanent Living Arrangement Permanency of Months 

Date 

because they found another permanent living arrangement, 

aged out of the system, or had other case outcomes. Sum 

the total, and divide by the total number of children in the 

sample. 

For example, for the same sample of 250 closed cases 

(250 children): 

APPLA* 37 children 

Age of Majority Reached 12 children 

Other 3 children 

To calculate the percentage of children in the CFR sample 

that achieved legal permanency, calculate the number of 

children who achieved legal permanency through reunification, adoption, or guardianship. Sum the total, and divide 

by the total number of children in the sample. 

For example, for the same sample of 250 closed cases 

(250 children): 

Reunification 142 children 

Adoption 43 children 

Guardianship 13 children 

Total 198 children 

01/01/2004 03/13/2005 14.4 

01/13/2004 09/04/2004 7.7 

01/21/2004 05/23/2005 16.0 

01/23/2004 02/01/2005 13.0 

01/30/2004 01/30/2005 12.0 

Total 63.1 months 

63.1 months ÷ 5 cases = 12.6 months to permanency. 

Thus, in this example, the average length of time from removal of the child to the achievement of final permanency 

was 12.6 months. 

198 children ÷ 250 children = 79.2 percent 

Thus, 79.2 percent of children in the sample achieved legal 

permanency. 

To calculate the percentage of children in the case file 

review sample that did not achieve permanency, calculate 

the number of children who failed to achieve permanency 

To calculate compliance rates with statutory and Federal 

timeframes, categorize cases as compliant or noncompliant based on whether they met time restrictions. For 

example, if State statute requires that adjudication be 

reached within 30 days of removal, then those cases that 

reached adjudication within that timeframe are categorized as “compliant.” Those cases that took more than 30 

8 
Chapter 2: Case File Review 

days are categorized as “noncompliant.” Frequency data, 

including percentages, can be calculated for each category 

of cases. 

It is also possible to further categorize noncompliant cases 

by “degree of noncompliance.” For example, some cases 

may reach adjudication within a week of statutory requirements and others may be noncompliant by several weeks. 

Analyzing Due Process Outcomes 

Courts should assess how well they use the information 

presented to deal with cases impartially and thoroughly. 

The key court performance measures for due process 

include the percentage of cases in which legal counsel for 

parents and children were present in each hearing, the 

percentage of cases in which legal counsel for parents and 

children changes between hearings, and the percentage of 

cases heard by a single judicial officer. 

Calculating due process outcomes. Data elements 

needed from the case file review include: 

u	Case number or child identification number. 

u	Representation present at a hearing by party and by 

hearing type. 

u	Whether there was a change of counsel. 

Calculate the percentage of cases in which legal counsel 

for parents and children were present at each hearing by 

assessing the presence or absence of each representative 

at each type of hearing. 

Party Present Percentage of Emergency Removal 

Hearings in Sample (n=45) in Which 

Party Is Present 

Mother’s Attorney 60 percent (n=27 of 45) 

Father’s Attorney 29 percent (n=13 of 45) 

Child’s Attorney 11 percent (n=5 of 45) 

To calculate the percentage of cases in which legal 

counsel for parents and children changed between hearings, categorize each case according to “change” or “no 

change” in specific representative. 

To calculate the percentage of cases in which the child’s 

case was heard by one judicial officer, cases can be 

categorized according to “change” or “no change” in 

judicial officer between hearing types. 

Analyzing Timeliness Outcomes 

Timeliness assesses whether courts minimize the time 

from the filing of the original petition to the achievement of 

case closure and permanency. 

The key performance measures for timeliness measure the 

average length of time between court hearings and events, 

as well as the overall time to achieve permanency. 

Calculating timeliness outcomes. Data elements needed 

from the case file review include: 

u	Case number or child identification number. 

u	Date of the original petition. 

u	Dates of each court hearing and court event for each 

case in the sample of cases. 

u	Date of case closure. 

Calculating average timeframes between court events and 

calculating the time from when a child is removed from 

his home to when he or she reaches permanency involve 

the same general format. Typically, the number of days per 

case is added together and then the total is divided by the 

total number of cases. 

Per Case: 

Number of days 

[Date of event 

2 (Date of 

permanency 

hearing)] 

– 

[Date of event 

1 (Date of 

petition filing)] 

= 

elapsed from 

petition filing to 

permanency 

hearing for that 

case (or months 

or years). 

For Entire Sample: 

Total number Average 

of days timeframe from 

elapsed from 

petition filing ÷ 

Total number 

of cases = petition filing 

to permanency 

to permanency for the case 

for each case sample. 

Timeliness can be measured several ways. For example, 

events in a case can be measured from a common start 

date (e.g., the date that a child is removed from her home 

or the date the original petition is filed) to a specific court 

hearing or court event (e.g., removal to adjudication, 

removal to disposition, removal to permanency hearing). 

Courts can measure the time between any specific events 

(e.g., adjudication to disposition) or the time for the overall 

9 
10 

User’s Guide to Nonautomated Data Collection
 

case processing (i.e., removal from home to final permanency placement). Timeframes can also be measured by 

looking at the percentage of cases that comply or do not 

comply with statutory and Federal requirements. Each type 

of time calculation produces different information. 

When measuring timeliness— 

u	Clearly define the start date—date the child is removed 

from home or the date the petition is filed. 

u	Clearly define the events that will be measured. Does 

“permanency” mean the permanency hearing or the 

final permanency outcome? 

u	Clearly define the unit of time (days, months, years). This 

must be appropriately reflected in computations. 

For information on how to display case file review 

data, please see appendix C (p. 27). 

Endnotes 

1. The Indian Child Welfare Act (ICWA) was created in 

1978 by the Federal Government to reestablish tribal 

authority over the adoption of American Indian children. 

The goal of the act was to strengthen and preserve 

American Indian families and culture. If a child is a 

member of a tribe or eligible for membership in a 

tribe, his or her family has the right to protection under 

ICWA. These rights apply to any child protective case, 

adoption, guardianship, termination of parental rights 

action, runaway/truancy matter, or voluntary placement 

of children (25 U.S.C. § 1901 et seq.). 

2. See the Toolkit Implementation Guide, chapter 2: “The 

Committee,” for how to establish a collaborative committee, and chapter 3: “Getting Started,” for strategies 

and tips on how to prioritize the performance measures 

according to an individual court’s needs. 

3. The confidence interval is the plus-or-minus figure 

usually reported in newspaper or television opinion poll 

results. For example, say a researcher uses a confidence interval of 4, and 47 percent of the sample picks 

an answer. If the entire relevant population had been 

asked the question, the researcher knows that between 

43 percent (47-4) and 51 percent (47+4) would have 

picked that answer. 

4. Unlike descriptive statistics, inferential statistics 

test the reliability of the study’s findings and let staff 

draw inferences about the target population of cases 

based on the sample population of cases. Descriptive 

statistics describe the data (e.g., percentages, averages), but inferential statistics say what the data mean 

and explore relationships among the data. Inferential 

statistics range in complexity and require that certain 

sampling conditions and population parameters are in 

place. 

5. For more guidance about stakeholders who should be 

included on a court’s performance measurement committee, please refer to the Implementation Guide of this 

Toolkit (chapter 2, “The Committee”). 
u	How many cases need to be observed? 

u	Who should conduct the court observations? 

u	What kind of training do observers need? 

u	How should the data be analyzed? 

u	How can findings be presented in a meaningful way? 

u	How much time and how many resources are available 

to conduct the court observations, including training, 

implementation, analysis, and report writing? 

Toolkit instruments at www.courtsandchildren.org include 

court observation hearing forms that can be used to assess 

the child abuse and neglect hearing process. The Toolkit 

includes a general hearing checklist that can be used 

in any child abuse and neglect hearing, and checklists 

specific to the type of hearing observed. 

Chapter 3 

Why Conduct Court Observation? 

Data obtained from court observation provides 

important qualitative information about court 

performance that can be used to supplement 

data obtained from automated management 

information systems. Conducting a court observation provides a snapshot of court practice that 

can enhance or clarify information obtained from 

other data sources such as interviews and focus 

groups. 

The process can be time and resource intensive, 

but may be important to ensure that stakeholders 

work to support a performance assessment 

process. 

Court Observation 


Observation can provide important qualitative information 

about court performance that may not be available using 

quantitative data collected from automated management 

information systems. Another source of qualitative 

information—focus groups—will be discussed in the next 

chapter. 

Observation allows court staff to understand court practice 

in a way that would not be possible using an interview, 

questionnaire, or focus group process. However, effective 

observation can be a highly labor-intensive and potentially 

expensive data collection strategy. 

Before beginning a hearing observation process, a court 

should answer the following questions: 

u	Is there a clearly defined purpose for the court 

observation process—what types of performance 

should be observed and what will data from court 

observations contribute to the overall court performance 

measurement? 

u	What specific information will be gathered? 

u	How will cases be selected for observation and coding? 

Each of the observation checklists or forms has been 

extensively field-tested in child abuse and neglect hearings 

around the country. They capture data that are relevant to 

best practices in child abuse and neglect cases, and can 

be an important qualitative tool in the overall performance 

measurement process. Types of data may include the 

presence of parties at hearings, level of judicial inquiry into 

substantive issues such as placement, service provision, 

educational issues, parent engagement in the hearing 

process, judicial findings of reasonable efforts, sources of 

delay, and time and calendar management (e.g., setting 

the next court date and time at the conclusion of each 

hearing). 

Court Observation Instruments: 

General Categories of Information 

The Toolkit’s related Web site, www.courtsandchildren.org, 

contains general and hearing-specific checklists relevant 

to best practice in child abuse and neglect hearings. A 

court’s performance measurement committee should 

determine whether a general or specific checklist meets 

court needs. This chapter provides an overview of the 

general hearing checklist instrument. It also provides a 

step-by-step guide on how to conduct a court hearing 

observation. 
User’s Guide to Nonautomated Data Collection
 

The court observation instrument collects the following 

data: 

u	What jurisdiction is the case in (county, circuit, etc.) 

and what court is being observed? 

u	What is the coder’s name? 

u	When was the hearing observed? 

u	What type of hearing was observed? 

u	Did any delays prevent the hearing from starting? 

u	When did the hearing start and end? 

u	Were there any interruptions to the hearing? 

u	Where there any continuances? 

u	Why were there continuances? 

u	Which parties appeared at each hearing? 

u	Was notice given to each party? 

u	Was explanation required if parties were not present? 

u	Were parties asked if they received notice or if court 

staff made other attempts to contact them? 

u	Who represented each party? 

u	Were nonrepresented parties advised of right to 

counsel? 

u	Was information provided to parties about how to obtain 

counsel? 

u	If coding a preliminary hearing (e.g., initial shelter 

hearing), what specific preliminary matters did the 

hearing address? 

u	Was the child’s paternity discussed? 

u	Was an Indian Child Welfare Act inquiry made? 

u	Who was assigned to financially support children? 

u	What was the level of discussion of child placement, 

services, and visitation rights? 

u	What was the level of discussion of permanency goals? 

u	What Adoption and Safe Families Act findings were 

made?
 

G	Were “reasonable efforts” findings made?
 

G	Were “contrary to welfare” findings made?
 

u	Were parents engaged in the discussion? 

u	Did the court provide an explanation of the hearing 

process? 

u	Did the court provide an explanation of the legal time 

constraints on the case? 

u	Was the parent questioned directly by the court? 

u	Was the parent given the opportunity to speak or ask 

questions? 

u	What other comments would you make about the 

hearing? 

Court Observation Forms and Court 

Performance Measures 

The court observation forms within the Toolkit provide 

the user with a method to collect qualitative data about a 

number of the court performance measures and serve as 

a template that jurisdictions can modify to reflect their own 

measurement needs, court structure, and unique practice. 

Coders should be knowledgeable, reliable, and well-trained 

on the coding instrument. Data from court observation 

on performance measures (e.g., due process measures) 

should be used to supplement information from other data 

sources. For further information on using observation forms 

to address specific measures, see appendix A (p. 21). 

How To Implement a Court 

Observation Process 

After establishing a collaborative committee, prioritizing 

court performance measures, and adapting instrumentation to suit the jurisdiction’s individual needs, courts must 

determine an appropriate sample of hearings that will be 

coded. This process involves a number of decisions: 

Define the sample of hearings that will be coded. 

Court staff must define a target population in order to understand what hearings to observe and code. For example, 

if a target population includes all the judges or hearing 

officers presiding over child abuse and neglect hearings in 

a jurisdiction, then staff must be sure to observe a sample 

of hearings from all judges and hearing officers. Depending 

on the purpose of the observation, a court may not have to 

include all types of hearings in their review. If, for example, 

a court wishes to focus on the review process, a sample of 

review and permanency planning review hearings should 

be observed for coding purposes. 

12 
Chapter 3: Court Observation 

Number of cases in the sample. Once the court has 

determined the population, staff must decide on a sample 

size. A sample population of hearings is the actual number 

of hearings that will be coded. The sample population 

can be used to draw (limited) inferences about the larger 

population. Once this population has been defined, a court 

should be able to understand the scope of its court observation process. 

Identify and train coders. Without skilled coders, a court 

cannot produce accurate and meaningful data. Coders can 

include a wide range of individuals, including court administrative staff, law clerks, and interns. A local university or 

law school may also be a resource. All coders will require 

proper training, to include the following subjects: 

u	The importance of court performance measurement and 

court observation. 

u	The specific goals of the court observation process, 

including performance measures that will be addressed 

and how these relate to court observation data. 

u	The target population of the court observation process 

and the parameters of the sample population of hearings 

that will be coded. 

u	The structure of court hearings in dependency cases. 

u	An element-by-element discussion of the court 

observation instrument. 

Pretest the court observation instrument and make 

necessary revisions. The court observation forms 

included in the Toolkit have been extensively pretested in 

a number of courts around the country. Nevertheless, each 

jurisdiction may need to modify the instrument to best 

address its specific practices and unique measurement 

needs. Even without modification, the instrument should be 

pretested with several sample court hearings to determine 

if coders are coding cases appropriately. This pretest 

should determine whether additional training is needed to 

code observations accurately and whether modifications 

need to be made to the court observation instrument for it 

to accurately capture items of interest to the performance 

measurement study. 

u	Pretesting can be used to determine whether coders 

are consistently interpreting and coding information in 

hearings. To ensure reliability, several coders should 

code the same hearing. Courts should determine the 

source of any inconsistencies. 

Conduct the observations. 

Check-coding and quality control. A small sample of 

hearings should be checked by a second, experienced 

coder and any discrepancies noted. All errors should be 

addressed with the original coder. Errors can occur for a 

number of reasons (e.g., the observation form is unclear 

or fails to provide opportunities to record needed data, 

the coder misinterprets events during the hearing, or the 

coder lacks the skills or ability to record data accurately). 

If the check reveals a high rate of errors, those hearing 

observations should be discarded and not used in analysis. 

Coders should be informed of the errors and additional 

training should be provided to ensure accuracy of the coding process. 

For more information on how to develop a database to manage court observation data, please 

see appendix B (p. 25). 

Analyzing Court Observation Data 

Simple descriptive statistics can help summarize the data 

collected during court observation. These statistics involve 

frequency data (a simple tally or count of data), frequency 

distributions (putting data in groups, such as grouping all 

cases by the year they opened or by county), graphical 

representations of data, and summary statistics (statistics 

that describe the typical value or most representative 

value, such as the mean or average). 

If coders have included relevant comments during court 

observation, that information can be reviewed for common 

themes and reported in a narrative that accompanies the 

frequency data. For more information about descriptive 

statistics, please see chapter two of this Guide. 

Analyzing Due Process Outcomes 

Courts should assess how well they use the evidence 

presented to deal with cases impartially and thoroughly. 

A performance measure involving due process that can 

be obtained from court observation is the percentage of 

hearings in which legal counsel for parents and children 

were present. 

To calculate the percentage of cases in which legal 

counsel for parents and children were present at each 

13 
User’s Guide to Nonautomated Data Collection
 

14 

hearing observed, categorize individual hearings according Endnotes 

to presence or absence of each representative. Then, count 1. For details on each of the performance measures, in- totals and calculate the percentage of hearings in which cluding definitions and data needed for measurement, parties are present. please refer to the Technical Guide of this Toolkit. 

Party Present Percentage of Emergency Removal 

Hearings Observed (n=25) in Which 

the Party Is Present 

Mother’s Attorney 80 percent (n=20 of 25 hearings) 

Father’s Attorney 12 percent (n=3 of 25 hearings) 

Child’s Attorney 100 percent (n=25 of 25 hearings) 

For information on how to display court observation data, please see appendix C (p. 27). 
A focus group brings together experts in a given area to 

discuss a specific topic or issue. Focus groups use guided 

discussions to solicit participants’ feelings and attitudes 

about a particular topic. The goal is to create an environment that encourages participants to share their perceptions and their points of view. Focus groups are used 

to create new ideas; develop new products or policies; 

When Should Focus Groups Be Used 

To Measure Court Performance? 

As with any data collection strategy, the appropriateness of 

the methodology and data collection instrument depends 

largely on the purpose and goals of data collection. 

A focus group can be used to gather information about 

what stakeholders see as the priority issues for measurement and the primary challenges and barriers to the 

process. Convening a focus group early in the performance measurement process can assist a court in setting 

priorities and finalizing how best to use data collection 

instruments. 

Focus groups also offer an excellent opportunity for system 

stakeholders to reflect on the findings, data produced by 

an automated management information system or file 

review process. They also help courts understand specific 

Chapter 4 

Why Conduct a Focus Group? 

Data obtained from a focus group provides 

important qualitative information about court 

performance that can be used to supplement 

data obtained from other sources. 

Focus Groups
 

identify needs, expectations, and issues; or generate theoretical constructs, models, and hypotheses. Participants 

are selected because they have certain characteristics in 

common related to the topic of discussion. Their responses 

are used to supplement and enhance data gained from 

other data sources. 

When planning a focus group, courts should consider the 

following questions: 

u	When should a focus group be used to collect 

information from stakeholders about court performance? 

u	What is the specific area of focused discussion? 

u	Who should participate in the focus group? 

u	How many people should participate in the focus group? 

u	Who should facilitate, or moderate, the focus group? 

u	How should the focus group be structured? 

u	How many focus groups should be held? 

In answering these questions, courts should take into 

account the specific purpose and goals of the focus group, 

the range of stakeholders involved, and any unique jurisdictional characteristics. This Chapter provides an overview 

of how to use focus groups to collect and analyze information that can inform a court performance or workload 

assessment. 

results. Some examples include: 

u	A court’s observations and case file review find that 

there is no substantive difference between the focus of 

case review hearings and permanency hearings and no 

substantive difference in the court orders resulting from 

these hearings. A focus group of judges might provide 

insight into why this is the case (e.g., a lack of judicial 

understanding of the specific purpose of the different 

types of hearings, unclear expectations for parties 

involved in the hearings, or a misunderstanding about 

the type of information that should be present at each 

hearing). 

u	A case file review reveals that different attorneys appear 

at different hearings. A focus group of attorneys might 

provide insight into the reasons for this occurrence (e.g., 

calendaring practice, law firm practice, and philosophy1

) 

and offer strategies to address these issues. 

u	An analysis of automated data and a review of case files 

show that a high number of hearings are continued. 

A focus group of judges or a focus group of attorneys 

might offer insight into the most common reason why 

continuances are requested and examine the court’s 

policy and practice on granting continuances. The focus 

group could generate procedural and policy-related 

strategies to reduce the number of continuances. 
User’s Guide to Nonautomated Data Collection
 

The following are circumstances when using a focus group 

could be beneficial: 

u	A court wants to examine the range of ideas or 

attitudes that people have about a specific area of court 

performance. 

u	A court wants to understand differences in perspectives 

between groups or categories of people. 

u	A court wants to uncover policies, practices, or 

philosophies that may impact performance outcomes. 

u	Expert opinion would help shed light on data obtained 

through other methods. 

u	A court wants ideas and strategies to address performance issues to emerge from the group discussion. 

Focus groups are likely to be ineffective and should not be 

considered in the following instances: 

u	A court wants to educate people. 

u	A court does not intend to use the results to inform court 

performance but only wants to give the appearance 

of listening to input—this is disrespectful and 

disingenuous. 

u	The issue or environment is emotionally charged and 

a group discussion is likely to intensify the conflict. 

This is likely to occur when people are polarized on an 

issue, trust has deteriorated, and the participants are 

confrontational. 

u	A court wants statistical projections. Typically, focus 

groups do not involve enough participants to make 

statistical projections or causal statements. 

u	The nature of the issue that will be discussed makes it 

unlikely that people will be open and honest in a group 

setting. 

Focus groups are a highly efficient method for collecting 

qualitative data. A large amount of information can be collected in a relatively short period of time through focused 

group discussion with key individuals. A well-structured 

focus group discussion ensures checks and balances 

among the participants and eliminates extreme views. 

Focus groups use flexible, but systematic, data collection 

techniques that can be effective for many purposes. They 

provide an opportunity for researchers to explore new 

issues and gather unique information. 

Determining Who and How Many 

People Should Participate in a Focus 

Group 

Once the court has determined that a focus group is an 

appropriate data collection strategy, the court should 

select the group’s participants. Participants should include 

stakeholders who have extensive experience with the issue 

and representatives from other groups with an interest in 

the issue. 

When considering timeliness, for example, a court may 

be interested getting judges’ opinions on the primary 

sources of court delay, or it may want to understand how 

different stakeholders perceive delay in the court process. 

In order to target a variety of stakeholders, a court may 

want to conduct a focus group of judges, a focus group 

of attorneys (perhaps even a separate focus group for 

each type of attorney), and a focus group of caseworkers. 

Alternatively, a court could conduct several focus groups 

that include a multidisciplinary group of participants. However, mixing groups of people who have different levels of 

status, power, and expertise may make some individuals 

feel uncomfortable and hesitant to share their ideas and 

perspectives. 

Depending on the extent of the problem or scope of 

the measurement issue, it may be beneficial to involve 

several groups. Two to four focus groups per issue, with 

10 to 12 people in each group, is generally sufficient, but 

courts should monitor progress to ensure that the focus 

groups are effective. Too many focus groups or too many 

members in a group can result in saturation, or exhausting the subjects and no longer being able to generate 

new ideas. On the other hand, if a court is still gathering 

new information and perspectives by the third or fourth 

focus group, then staff may want to consider conducting 

additional groups—depending on time and resources. 

Also, when addressing more complex issues, courts should 

create groups with only six to eight members for maximum 

productivity. 

When developing a discussion plan for a focus group, a 

court may wish to get the input of its working or advisory 

committees. For example, if a court wants to create a focus 

group of judges, staff should consider: 

u	Which specific judges should be included? 

u	How should the court invite these judges to participate? 

16 
Chapter 4: Focus Groups 

u	When is the most convenient time for these judges to 

attend a focus group (e.g., a particular day of the week 

or time of day)? 

u	What questions should be asked? 

u	Who should ask the questions and facilitate the focus 

group? 

Developing an Effective Focus Group 

Effective focus groups require careful planning with wellstructured and pertinent questions to ensure an efficient 

use of time. Groups should avoid asking questions that 

do not relate to the specific purpose of the focus group, 

phrasing questions in a way that does not generate group 

discussion (avoid questions with “yes” or “no” answers), or 

generating a long list of questions that cannot be properly 

discussed within the necessary time constraints. 

Typically, a focus group will address, at most, about a 

dozen questions in a 2-hour session. As participants answer questions, their responses will spark ideas from other 

participants, which leads to more discussion. Good focus 

group questions should: 

u	Sound conversational. A focus group is a social 

experience and conversational questions help create 

and maintain an informal, conversational environment. 

u	Use terminology that participants use regularly. 

u	Be easy to ask. Good questions are written so the 

facilitator will not stumble over them when speaking. 

u	Be clear and concise. Respondents should understand 

the questions they are asked. Questions should be short 

and to the point. Lengthy questions can be confusing. 

u	Be open ended. Asking open-ended questions 

facilitates more discussion and often elicits richer 

descriptions, explanations, and insights. 

The order or sequence of the questions is also important. 

Questions should flow from one another in a logical way. 

A group leader should begin with questions that are 

easy for all participants to answer. Tough or controversial 

questions should come later in the focus group. Questions 

with a positive focus should be asked before those with a 

negative or controversial aspect. 

Understanding the function and purpose of different types 

of questions can also lead to effective discussions: 

u	Opening questions should get people talking and to 

make them feel comfortable. The questions should 

be easy and require quick, 30-second answers. 

u	Introductory questions identify the topic of conversation. 

Participants start thinking about the issue, and begin to 

converse. 

u	Transition questions move the conversation to the 

key questions that drive the discussion of a court 

performance issue. 

u	Key questions are the most important aspect of the 

focus group discussion. Key questions address issues 

vital to the performance measurement process, bridging 

information gaps and solving analysis issues. In a 

2-hour focus group, there may be only two to five key 

questions. Although a few minutes may be allocated to 

answering earlier opening questions, groups will want 

to allow as much as 10 to 20 minutes to answer a key 

question. 

u	Ending questions bring closure to the discussion, 

enabling participants to reflect on previous comments. 

These questions are critical for analysis. Three 

common ending questions include a question that asks 

participants to reach a final conclusion on the issues 

addressed, a question that summarizes the discussion, 

and a question that asks if there is anything else that 

should have been talked about or anything that was 

missed. 

Several strategies can enhance participation in the focus 

group. Preparation is a key component of a good focus 

group. Introductory statements, questions (in order), 

time allocations for questions, and possible probes (see 

“Probes,” p. 18) should be written down in advance of the 

focus group session. The facilitator should not read the 

statements and questions and should be very knowledgeable about, and comfortable with, the structure and focus 

of the discussion and the overall flow of questions. 

Use a flip chart to engage participants in a focus group. 

Flip charts should be used to keep track of ideas and 

concepts. Use flip charts to: 

u	Make a list of key points. 

u	Ask the group to rank or rate items in a list. 

u	Ask the group to choose among alternatives on a list. 

17 
User’s Guide to Nonautomated Data Collection
 

18 

Probes 

An initial focus group question opens the door 

to an issue. However, a focus group leader may 

need to ask more questions to get a full answer. 

These questions, called probes, help a group get 

more information. Probes can also help manage 

the dynamics of the focus group. For example, 

probes can help— 

u Clarify. 

“Can you tell me more about what you mean 

by that?” 

“It sounds like you are saying, ______, is 

that a fair summary?” 

u Get more details. 

“Can you give me an example?” 

“If I was observing that, what would I see?” 

u Get feelings, thoughts, and rationale. 

“Why does that matter to you?” 

“How do you feel about that?” 

u Explore variations. 

“How has your approach changed over 

time?” 

“What motivated this change?” 

u Handle digressions. 

“How does that relate to the topic we started 

with?” 

“Can you recall the associations that led us 

from our original topic to this one?” 

u Accommodate emotions. 

“What aspects of this issue do you think 

prompts such strong emotions?” 

“Why do you think people feel so strongly 

about this issue?” 

u	Ask the group to provide opinions or “pilot test” ideas 

on a list. 

Selecting a Focus Group Facilitator or 

Moderator 

The success of a focus group is also dependent on a good 

facilitator or moderator. Courts should consider the best 

person to conduct a discussion with a specific group of 

people. For example, who is the most appropriate person 

to facilitate a focus group of judges—Another judge? A 

court administrator? An attorney? An independent facilitator? Selecting the right facilitator is a critical decision point. 

A good facilitator or moderator should: 

u	Be an active listener. 

u	Be an effective communicator. 

u	Have adequate background knowledge on the topic of 

discussion to place comments in perspective, probe 

effectively, and follow up on critical areas of concern. 

u	Have respect for all focus group participants. 

u	Have a friendly manner and a sense of humor. 

Sometimes, it is helpful to have a “facilitator team” that 

includes a lead facilitator and an assistant facilitator. Typically, the lead facilitator will direct the conversation and 

take a few notes (on a flip chart, if available). These notes 

should help the group remember issues or ideas to come 

back to in later conversation, or to jot down questions 

for followup. The assistant facilitator takes more comprehensive notes and may ask followup questions when 

appropriate. 

Analyzing and Reporting Your Focus 

Group Results 

The final and essential step when conducting a focus 

group is sharing the results of other court staff and interested stakeholders. It is important to take written notes 

to capture key discussion points and comments during 

a conversation. A facilitator team should be used when 

comprehensive notes are needed. The quality of any report 

results is directly related to the quality of the notes taken 

during the focus group. 

The report a focus group produces should effectively 

communicate results to the appropriate audience and report individual information that can assist decisionmakers. 

It should provide guidance and recommendations for future 
Chapter 4: Focus Groups 

research. Quotations and illustrations often help to clarify 

a point and engage a reader. The language in the report 

should be simple and understandable, avoiding 

terminology or jargon that might be unfamiliar to the 

intended audience. 

Focus group reports often fall into one of four general 

types: 

u	Narrative reports are between 10 to 30 pages and use 

quotations. The report should include a cover page, 

summary, purpose and procedures, results, conclusion, 

recommendations, and appendix. It should also discuss 

the group’s analysis of the key questions and ideas. 

u	A report memo consists of a summary of key findings 

and a response from the project’s sponsor, and is 

usually one to two pages long. The memo is typically 

written by the sponsor of the focus group (the convener) 

and the research team (the focus group facilitator 

and person who analyzes the results) and is targeted 

toward participants in the focus groups. It outlines 

recommendations and suggestions for actions, and 

should include thanks to the participants for taking time 

to share their thoughts and experiences. 

u	Bullet reports provide an outline of a narrative report, 

highlighting important information in bullet points. These 

reports can be prepared in a relatively short period of 

time. 

u	A top-line report combines narrative and bullet 

reports and conveys only the critical findings (they 

are approximately one to two pages or more in length 

depending on the purpose of the focus group). Top-line 

reports are prepared within a day or two of a focus 

group, are targeted to clients, and are often used 

in marketing research. They are often considered 

preliminary and are produced in order to provide 

immediate findings to the focus group participants. Topline reports can be followed by full narrative reports. 

Endnotes 

1. For example, a focus group of attorneys could provide 

information, suggesting that in order to cover the hearings scheduled for their juvenile delinquency and child 

protection caseloads (many of which are scheduled for 

the same time) they need to secure substitute counsel 

to attend hearings on their behalf. Additionally, the 

philosophy and practice of the prosecuting attorney’s 

office may suggest that a different attorney handle 

contested trials or termination cases. 

19 

Key Performance Measures for Case 


File Review and Court Observations 
 Appendix A 

This appendix highlights key performance measures that 

can be captured through a case file review (CFR) and court 

observation process. 

Safety. Courts must ensure that children under their jurisdiction do not face abuse or neglect.1 

Key Performance Measures: 

u	Measure 1A: Percentage of children who are abused or 

neglected while under court jurisdiction. 

u	Measure 1B: Percentage of children who are abused 

or neglected within 12 months after the case is closed 

following a permanent placement. 

Additional Performance Measures for Safety (for CFR): 

In addition to measuring the percentage of children who 

experience further abuse or neglect while under the court’s 

jurisdiction, court staff could modify the case file review 

instrument by adding a measure that collects information 

about whether additional petitions were filed for abuse 

and neglect before the case was closed. NOTE: The two 

child safety-related measures listed above are two of the 

nine key or essential performance measures. Data should 

be collected through a case file review or other clearly 

documented procedures.2 

Permanency. Children need permanent and stable living 

situations. Courts should ensure that children have stable 

family relationships and connections. 

Key Performance Measures: 

u	Measure 2A: Percentage of children who reach 

legal permanency by reunification, adoption, or legal 

guardianship. 

u	Measure 2B: Percentage of children in foster care 

who do not reach legal permanency by reunification, 

adoption, or legal guardianship. 

Additional Performance Measures for Permanency (for 

CFR): Court staff may wish to add performance measures 

on the extent to which a child’s placement in care is stable 

and permanent, and how often a child reenters foster care. 

A court may want to include these data if these issues 

are a priority or if these data are not reliably available 

elsewhere. 

Appendix A

SAFETY EXAMPLE 

Case File Review Instrument 

Examines—Was there a previous 

closed case of abuse/neglect 

involving this child? 

The “previous closed case of abuse or neglect 

involving this child” requires the coder to check 

“Yes” or “No” and provide the date and previous 

petition number if that number is available in the 

court file. 

Analyzing this data provides information about 

the percentage of children who return to court 

with a new petition of abuse and neglect after a 

previous case was closed. 

SAFETY EXAMPLE 

Court Observation Instrument 

Examines—Was there a discussion 

in the hearings about whether efforts 

had been made to reunify a family? 

The court observation forms require the coder to 

note whether at some point during the hearings, 

parties and legal representatives discuss whether 

efforts have been made to safely reunify the 

family, if their conclusions are contrary to welfare 

findings, and to rate the extent of discussion as 

minimally or greatly substantive. Coders are 

also required to record observations related to 

the current safety and well-being of the child 

(e.g., whether a substantive discussion was 

had during the hearing about the child’s safety 

and well-being). Analysis of these data provides 

information about a child’s safety while under 

court jurisdiction. 
User’s Guide to nonautomated data Collection
 

22 

PERMANENCY EXAMPLE 

Case File Review Instrument 

Examines—What was the case 

number or child identification 

number? When was the child 

removed from home? When was 

the child’s adoption finalized? 

When was the case closed? 

Why was the case closed? 

Dates of events in a court case help calculate 

overall timeframes until a child reaches a permanent living situation. Time calculations require 

staff to compute how much time has passed between key case events by type for each permanent living situation children find themselves in. 

Analyzing these data provides information about 

the average length of time it takes for a child 

to reach a permanent living situation, and more 

specifically, how long it takes a child to reach 

each type of permanent living situation. 

PERMANENCY EXAMPLE 

Court Observation Instrument 

Examines—Did hearings include 

a discussion about permanency 

goals for the child? 

Court observation forms require the coder to 

note whether parties and legal representatives 

discussed the permanency goal for the child(ren). 

The coder should note if there is a discussion of 

a concurrent permanency goal (e.g., a reunification goal and a concurrent goal of termination 

of parental rights and adoption), a timeframe 

for achieving permanency goals, and if barriers 

to achieving permanency exist. Coders rate the 

extent of the discussion from minimal (1) to a 

great deal (3) of discussion. 

Due Process. Courts must deal with cases impartially and 

thoroughly, based on the evidence presented. 

Key Performance Measures: 

u	Measure 3A: Percentage of child abuse and neglect 

cases in which the same judicial officer presides over all 

hearings. (Additionally, court staff should measure how 

many hearings are heard by two, three or more judicial 

officers if that information is available.) 

u	Measure 3B: Percentage of child abuse and neglect 

cases in which both parents receive written service of 

process of the original petition. 

u	Measure 3I: Percentage of child abuse and neglect 

cases in which the same legal advocate represents 

the child throughout the case. (Additionally, court staff 

should measure the percentage of cases in which legal 

counsel for children changes, and how many times this 

counsel changes.) 

u	Measure 3J: Percentage of child abuse and neglect 

cases in which the same legal counsel represents the 

parents throughout the case. (Additionally, court staff 

should measure the percentage of cases in which legal 

counsel for children changes, and how many times this 

counsel changes.) 

Additional Due Process Measures. The national court 

performance measures include how quickly cases move to 

hearings, whether parties receive notice of hearings, and 

whether counsel are appointed in a timely fashion. 

Courts may want to consider modifying the case file review 

instrument to document the date of the court service, the 

date all parties receive a notice of hearing, and the date 

the counsel was appointed. 

Court observation may help courts understand how well 

their performance meets the requirements of these measures. Court observation instruments allow coders to report 

if the court required an explanation as to why parties are 

missing from hearings and what efforts were made to 

locate missing parties. If foster parents are not present at 

hearings, coders can note if the court or parties inquired 

whether foster parents received notice of hearings. Analyzing these data can provide insight into the court’s notification procedures. 

Timeliness. Courts must assess their ability to place 

children in permanent care in a timely fashion. This 

involves determining the time from the petition filing in 

each court hearing to the time a permanent placement is 
Appendix A: Key Performance Measures 

Case File Review and Observation 

Instruments Examine— 

“Representation” 

In the case file review and observation instruments, “representation” includes several discrete 

areas of data collection. For each type of hearing, 

the coder is asked to code whether or not counsel is present for the mother, father, and primary 

child involved in the case. The coder is asked to 

further indicate whether the child’s representative is an appointed attorney, attorney guardian 

ad litem, nonattorney guardian ad litem, courtappointed special advocate, or private counsel. 

The coder assesses whether any of the parties’ 

counsel has changed since the last hearing or 

court event. 

Analyzing this data provides information about 

the percentage of cases in which parties have 

counsel present at each hearing type, how often 

counsel changes, and the type of counsel the 

child has. 

Possible data sources: Court orders, court reports. 

DUE PROCESS EXAMPLE 

Case File Review Instrument 

Examines: Did the Judge or Judicial 

Officer Change? 

The coder should indicate whether or not the 

judge or judicial officer has changed from the 

prior court hearing or court event. 

Analyzing these data provides information about 

the percentage of children for whom one judicial 

officer hears all hearings, and the number of 

judicial officers involved in one case if more than 

one officer is involved. 

Possible data sources: Court orders, court reports. 

achieved. The case file review instrument provides court 

performance information about the timeliness of each 

court event. Court observation forms cannot assess the 

timeliness of a court event, but can provide information 

about the reasons behind court delays. 

Key Performance Measures (these measures are key for 

case file review only): 

u	Measure 4A: Average (median) time from filing of the 

original petition to legal permanency. 

u	Measure 4B: Average (median) time from filing of the 

original petition to adjudication. 

u	Measure 4C: Percentage of cases that are adjudicated 

within 30, 60, or 90 days after the filing of the petition. 

u	Measure 4D: Average (median) time from filing of the 

original petition to the disposition hearing. 

u	Measure 4E: Percentage of cases in which the 

disposition hearing occurs within 10, 30, or 60 days 

after adjudication. 

u	Measure 4G: Average (median) time from filing of the 

original petition to first permanency hearing. 

u	Measure 4I: Average (median) time from filing of 

the original child abuse and neglect petition to the 

termination of parental rights (TPR). 

u	Measure 4K: Percentage of cases in which the 

termination of parental rights (TPR) petition is filed 

within 3, 6, 12, and 18 months after the disposition 

hearing. 

u	Measure 4L: Percentage of cases in which the adoption 

petition is filed within 3, 6, and 12 months after the 

termination of parental rights (TPR). 

u	Measure 4M: Percentage of adoption cases finalized 

within 3, 6, and 12 months after the filing of the 

adoption petition. 

Additional Timeliness Measures. In addition to the measures listed above, courts may wish to assess the average 

time between filing of the original TPR petition and adoption. Measuring the percentage of hearings not completed 

within timeframes set forth in statute or court rules may 

also help the court determine if delay is occurring at the 

TPR petition filing stage of the court process. 

23 
24 

User’s Guide to nonautomated data Collection
 

TIMELINESS EXAMPLE 

Case File Review Instrument Data 

Elements—What date did certain 

court events occur? 

Coders should collect the dates of all key court 

hearings and events. 

Analyzing this data provides information about 

the timeliness of events within hearings, the time 

between hearing dates, and overall time it takes 

to process a case. Timeframes for hearing events 

and case processing can be assessed against 

the standard timeframes required by statute and 

Federal law. 

Endnotes 

1. For details on each of the performance measures, including definitions and what data needs to be collected 

for measurement, please refer to the Technical Guide of 

this Toolkit. 

2. The nine key court performance measures listed in the 

Booklet are as follows: 

u	Safety 

u Measure 1A (Child Safety While Under Court 

Jurisdiction): Percentage of children who 

are abused or neglected while under court 

jurisdiction. 

u Measure 1B (Child Safety After Release From 

Court Jurisdiction): Percentage of children 

who are abused or neglected within 12 months 

after the case is closed following a permanent 

placement. 

u	Permanency 

u Measure 2A (Achievement of Child 

Permanency): Percentage of children who reach 

legal permanency by reunification, adoption, or 

legal guardianship. 

u	Due Process and Fairness 

u Measure 3A (Number of Judges Per Case): 

Percentage of child abuse and neglect cases in 

which the same judicial officer presides over all 

hearings. 

u Measure 3B (Service of Process to Parties): 

Percentage of child abuse and neglect cases in 

which both parents receive written service of 

process of the original petition. 

u	Timeliness 

u Measure 4A (Time to Permanent Placement): 

Average (median) time from filing of the original 

petition to legal permanency. 

u Measure 4B (Time to Adjudication): Average 

(median) time from filing of the original petition to 

adjudication. 

u Measure 4G (Time to First Permanency Hearing): Average (median) time from filing of the 

original petition to the first permanency hearing. 

u Measure 4I (Time to Termination of Parental 

Rights): Average (median) time from filing of the 

original petition to the termination of parental 

rights (TPR). 

The supplemental performance measures listed in this 

appendix are key court performance measures for the 

purposes of case file review and court observation only. 
Appendix B 

Developing a Database 

A database is designed to manage individual data elements in a logical and structured way. The data collected 

through a case file review or court observation process 

can generally be managed and analyzed through a simple 

database format using readily available programs such as 

Microsoft Excel® or Microsoft Access® . 

Using Excel, a similar database, or a statistical program 

(e.g., SPSS), develop a database that tracks each piece of 

data collected in the case file review or court observation 

process. A table or spreadsheet is usually sufficient and 

easy to manage. Data are usually presented in rows and 

columns. Specific data categories (e.g., date of adjudication) are usually assigned a column, and each column 

should represent one piece of data for each performance 

Each case should be a row of data. The case data should 

be entered with specific information placed in the appropriate column. Ideally, the columns of data in the 

spreadsheet should follow the same order as they appear 

on the case file review or court observation instrument. 

Using simple equations, a database program can calculate 

column totals. 

Entering Data. Each case file reviewed should generate a 

copy of a completed case file review form. To facilitate the 

accuracy of data entry, one person should enter the data 

for each completed case file review form. 

Tips for Quality and Accuracy Assurance. Accurate data 

entry is essential for meaningful results. It is recommended 

that a second data entry person check data entry for a 

small sample of entered cases and note any discrepancies 

Managing Case File Review and 

Court Observation Data 

measure. These categories should be placed in columns or errors. All errors should be addressed in consultation 

in the order they appear on the case file review or court with the original coder. If a high rate of error occurs in the 

observation instrument. sample that is checked, all data should be reentered. 

The sample table below illustrates what a database containing case file review data should look like. 

Coder 

Case 

Number 

Child’s Date 

of Birth Allegations 

Child 

Removed? 

Removal 

Date 

Date 

Petition 

Filed 

Date Shelter 

Hearing Held 

Date 

Adjudication 

Hearing Held 

1 J39299 9/15/2004 Neglect Yes 2/4/2005 2/8/2005 2/9/2005 4/8/2005 

1 J40172 1/1/1990 Physical Abuse Yes 7/17/2001 7/19/2001 7/20/2001 8/24/2001 

1 J40380 9/15/2000 Physical Abuse Yes 6/10/2003 6/12/2003 6/13/2003 8/1/2003 

1 J40392 9/12/1988 Physical Abuse Yes 3/5/2004 3/11/2004 3/12/2004 5/7/2004 

1 J39704 5/4/1998 Neglect Yes 4/18/2004 4/20/2004 4/21/2004 6/25/2004 

1 J34039 5/2/1988 Neglect Yes 4/21/1998 4/23/1998 4/24/1998 7/3/1998 

1 J35066 5/23/2004 Neglect Yes 3/21/2005 3/25/2005 3/25/2005 4/22/2005 

Appendix C 

Many different data presentations can be effective. 

Consider who will be listening to the presentation, their 

knowledge base, their interests, and the purpose of the 

presentation.1,2 

Tables. Presenting data in a table helps to organize and 

summarize the data in a more understandable way. Tables 

should be simple and straightforward. 

Charts. A chart is a visual explanation of data. The goal of 

a chart is to give the reader a quick message, but charts 

should not be the message—they should provide visual 

support for the report. Charts show relationships among 

variables and let a reader grasp complex data at a glance. 

There are three basic types of charts: 

u	Bar charts compare items at one point in time. They 

Some general tips to keep in mind when developing a 

chart or graph: 

u	Graphs should be easy to read. 

u	Graphs should only contain essential information. 

u	All parts of the graph should be clearly labeled. 

u	Graphs should use geometric figures (not colored lines) 

for data points. This makes copying the graphs easier. 

Endnotes 

1. For more general considerations about how to use 

charts and graphs to display data please refer to 

chapter five of the Implementation Guide; for detailed 

guidance on displaying results of specific performance 

measures, see the Technical Guide. 

Displaying Case File Review Data 

in a Meaningful Way 

compare a set of individual items or several sets of 

related items. 

u	Line charts show change over time. They show trends, 

increases, or declines of one variable over a period 

of time. They have an X-axis (horizontal) and a Y-axis 

(vertical). Usually, the X-axis has numbers for the time 

period, and the Y-axis has numbers for what is being 

measured. 

u	Pie charts show relative proportions. They show 

percentages of items. Pie charts read easily if they have 

no more than five slices. All slices should be easy to 

read. 

2. Reporting should be done to meet the jurisdiction’s 

needs. The needs assessment should answer what the 

individual court needs to know. Theories need support 

with numbers. 




*NCJ~223569*
 

National Council of Juvenile and 

Family Court Judges 

National Center for State Courts 

ABA Center on 

Children and the Law