*Final Report*

Study of the Implementation of the Performance-Based Incentive System

Prepared for:\
Office of Child Support Enforcement

Prepared by:\
The Lewin Group\
Karen N. Gardiner\
Michael E. Fishman\
Asaph Glosser

and

ECONorthwest\
John Tapogna

#  {#section .unnumbered}

Table of Contents

Executive Summary [1](#executive-summary)

A. Purpose of the Study [1](#purpose-of-the-study)

B. Key Findings [2](#key-findings)

I. Introduction [5](#introduction)

A. Background [5](#a.-background)

1\. Original Incentive System [5](#original-incentive-system)

2\. Performance-Based Incentive System
[6](#performance-based-incentive-system)

B. Methodology [7](#b.-methodology)

1\. Data Analysis [7](#data-analysis)

2\. In-depth Interviews [7](#in-depth-interviews)

3\. Conference Calls Facilitated by the National Council of Child
Support Directors
[9](#conference-calls-facilitated-by-the-national-council-of-child-support-directors)

C. Structure of Report [9](#structure-of-report)

II\. State performance [10](#state-performance)

A. Introduction [10](#introduction-1)

B. Trends [10](#trends)

C. Empirical Analysis [11](#c.-empirical-analysis)

D. Conclusion [14](#conclusion)

III\. State Interviews: Structure of the Incentive System
[16](#state-interviews-structure-of-the-incentive-system)

A. Assessment of the Performance Measures
[16](#assessment-of-the-performance-measures)

1\. Current Measures [16](#current-measures)

2\. Definitional Issues [17](#definitional-issues)

3\. Additional Measures [18](#additional-measures)

B. Assessment of Performance Standards
[19](#assessment-of-performance-standards)

1\. Performance Thresholds [20](#performance-thresholds)

2\. Penalties [20](#penalties)

C. Factors that Affect Performance
[21](#factors-that-affect-performance)

D. Assessment of the Cap [22](#assessment-of-the-cap)

E. Assessment of Phase-in Period [23](#assessment-of-phase-in-period)

IV\. State Interviews: Data Reliability Issues
[25](#state-interviews-data-reliability-issues)

A. Data Reliability Audit Parameters
[25](#data-reliability-audit-parameters)

1\. Sample Size [25](#sample-size)

2\. Efficiency Rate [26](#efficiency-rate)

3\. Timing [26](#timing)

4\. Suggested Audit Process Improvements
[27](#suggested-audit-process-improvements)

B. Quality Control Efforts [27](#quality-control-efforts)

1\. Revisions to Automated Systems [28](#revisions-to-automated-systems)

2\. Data Clean-up Projects [28](#data-clean-up-projects)

3\. Assessing Performance of Local Offices
[28](#assessing-local-offices)

V. State Interviews: Efforts to Improve Performance
[29](#state-interviews-efforts-to-improve-performance)

A. Staff Education and Assessment [29](#staff-education-and-assessment)

1\. Education and Training [29](#education-and-training)

2\. Assessment [30](#assessment)

B. Policy Changes [30](#policy-changes)

VI\. State Interviews: Budget Issues
[33](#state-interviews-budget-issues)

A. Forecasting Payments [33](#forecasting-payments)

B. Reinvestment of Incentives [34](#reinvestment-of-incentives)

VII\. Conclusions [36](#conclusions)

# Executive Summary {#executive-summary .unnumbered}

## Purpose of the Study

Since 1975, the federal government has paid incentives to state child
support enforcement (CSE) programs to encourage improved collections
through efficient establishment and enforcement techniques. These
incentive payments are a key source of funding for state programs. The
method for calculating incentive payments changed with the adoption of
the Child Support Performance and Incentive Act (CSPIA) in 1998. Prior
to CSPIA, the incentive amount was determined by the cost-effectiveness
of each state's program---that is, total collections divided by total
administrative expenditures. CSPIA changed the incentive process in
multiple ways. Key elements of the new system include:

-   Incentive payments are linked to performance in five areas:
    paternity establishment, order establishment, collections on current
    support due, cases paying toward past-due support, and
    cost-effectiveness.

-   Data must be reliable and complete, as determined by data
    reliability audits.

```{=html}
<!-- -->
```
-   Incentives are based on state child support collections and
    performance*.* Each state's collection base is determined by the sum
    of (1) collections for current and former assistance cases and
    Medicaid-only cases multiplied by two and (2) collections for cases
    never on public assistance.[^1] The state collection base is then
    multiplied by a specified percentage, depending on performance.

-   States are paid from a capped incentive pool. The pool increases
    from \$422 million in Fiscal Year 2000 to \$483 million in Fiscal
    Year 2008.

-   Penalties are associated with failure to meet or improve performance
    for three of the measures: paternity establishment, order
    establishment, and current collections.

-   Incentives must be reinvested into state CSE programs; they must
    supplement, and not supplant, other funds used by the states to
    carry out CSE activities.

The 1998 law requires the Department of Health and Human Services to
produce interim and final reports that detail the implementation of this
new system and offer recommendations for changes in the system that
would improve operation of the program. The *Interim Report*[^2]
provided background on the new incentive system, including its
structure; described the incentive calculation and payment processes,
including data collection and data reliability audits; and provided
information on audit results and incentive payments for Fiscal Years
(FYs) 1999 to 2001, including trends in incentive payments. This report
first updates trends in state performance. Next, it explores state CSE
programs' experiences with the implementation of the new system and
their perceptions of the system's problems, successes, advantages, and
disadvantages.

## Key Findings

**State and local CSE staff expressed near universal support for the new
incentive system**. While almost every state and local official
interviewed recommended refinements, nearly all agreed the
performance-based incentive system focused state and local energies on
the important core functions of the CSE program.

**New system measures the appropriate areas**. The majority of state and
local CSE respondents believed that appropriate performance measures
were selected and weighted correctly. Specifically, they expressed
strong support for the measures that are most directly associated with
the core CSE mission of getting support to families: paternity
establishment, order establishment, and collections on current support
due. Most respondents said the five measures were adequate and that
adding new ones would be premature.

**Performance thresholds are reasonable.** Most respondents agreed that
the performance thresholds associated with incentive payments were about
right (e.g., states that establish 80 percent of paternities receive 100
percent of the payment).

**Reporting system shows performance has improved in most areas.**
Analysis of OCSE administrative data shows state performance in most
incentive areas improved between FYs 2000 and 2002. Median scores for
states with reliable data increased for CSE paternity establishment
(from 70 percent to 87 percent), cases with orders (from 67 percent to
73 percent), cases paying toward past-due support (from 57 percent to 61
percent), percent of current support collected (55 percent to 59
percent), and cost-effectiveness (from \$4.13 to \$4.49). The median
state score for the statewide paternity establishment measure remained
over 90 percent.

**Data reliability improved during the three-year implementation
period.** In addition to improving performance, states also improved
data reliability. Between FYs 2001 and 2002, the number of states
failing a data reliability audit for at least one measure decreased by
half, from 25 to 12. Discussions with CSE officials indicated that
states have devoted considerable resources to improve data reliability,
such as revisions to their automated systems (e.g., automating key forms
used in the audit and creating safeguards to catch human errors);
extensive data clean-up projects, often in response to issues identified
by federal auditors; and assessment of performance of local offices. In
general, states suggested that the increased importance the new system
places on data reliability has had a positive effect on the overall
effectiveness of their programs.

**State CSE programs have taken steps to ensure that all staff is
working toward performance improvement.** States have gone to
significant lengths to disseminate the information on the federal
performance measures. States reported that child support staff is aware
of the performance measures, their importance to the program's mission,
and their fiscal implications. Activities include:

-   Providing education and training in multiple forums, including
    statewide and regional CSE conferences, monthly office meetings,
    newsletters, emails, and presentations.

-   Creating specialized training teams that review regional and
    office-level performance and tailor training activities to address
    observed deficiencies.

-   Developing automated systems to track performance at regional and
    office levels.

-   Using some or all federal measures in workers' performance
    evaluations.

**State CSE programs adopted a number of policy changes to affect
measures**. States report that performance improvement is factored into
almost every initiative that child support agencies undertake. Policy
changes include:

-   Reviewing cases that are appropriate for closure under federal
    guidelines. Under the previous incentive system, keeping inactive
    cases on the system did not affect state performance. Under the new
    incentive system, these cases are included in the universe of open
    cases and can negatively affect state performance on the paternity
    establishment and cases with orders measures.

-   Emphasizing the creation of "payable" child support orders. State
    policies include an attempt to lower the rate of default orders
    (i.e., those entered when a non-custodial parent does not respond to
    requests for financial information or requests to appear at an
    administrative forum or court) that are often based on imputed
    income, or by adding a self-sufficiency reserve to child support
    guidelines that allows the non-custodial parent to maintain a
    minimal subsistence level. The policies are not intended to absolve
    the non-custodial parents of their responsibilities to their
    children, but aim to increase the likelihood that they will provide
    consistent support to their children over the long term.

-   Increasing reliance on administrative processes to establish orders
    in an attempt to expedite establishment and improve performance on
    the order establishment measure.

-   Encouraging payment of current and past-due support orders by
    offering non-custodial parents amnesty from interest payments if
    they negotiate an arrearage payment plan or extending current
    support collections when a child reaches the age of majority if the
    non-custodial parent owes arrears.

**Change in incentive payments to individual states---relative to what
they would have received under the old system---is related to
performance and the nature of the state's caseload.** Empirical analysis
indicates that change in any given state's incentive payments---relative
to what it would have earned under the old system---could be largely
explained by the following: (1) state performance and reliable data are
strongly and positively associated with higher payments under the new
system, and (2) the proportion of a states' collections that come from
current assistance cases is strongly and negatively associated with the
change in incentive payments under the new system.

**State CSE officials do not want OCSE to adjust incentive payments for
state socio-economic conditions**. Child support officials were asked if
the new system favors states with certain profiles, such as affluent
populations, and, if so, whether states should be compensated for
factors beyond their control. Most respondents agreed that CSE programs
operate in different socio-economic environments. However, the majority
opposed adjusting incentive payments to account for those differences,
because such adjustments would add complexity and uncertainty to an
already complicated payment system and gaining consensus on an
appropriate list of adjustment factors would be extremely challenging.

**State CSE officials recommended refinements to the new incentive
system.**

-   *The capped incentive pool raised concerns*. State officials noted
    that the cap makes estimating incentive payments very difficult.
    Because states are competing for a fixed pool of incentive dollars,
    a state must anticipate not only its own performance and data
    reliability, but also that of every other state, in order to
    forecast its own incentive payments. Moreover, they expressed
    concern that the cap creates a dynamic in which better performance
    is not always matched by higher payments. Because incentive payments
    are a function of both performance and state collections, a state
    could improve performance but receive a lower payment if other
    states with larger collection bases also improved their performance
    or data reliability. For these reasons, officials from all study
    states voiced support for removing the cap. Although states strongly
    supported the cap's removal, the interview protocol did not explore
    how the federal government would pay for incentive payments that
    exceeded the amount currently authorized.

-   *The audit schedule does not allow for a full corrective action
    period*. A state faces a penalty if OCSE auditors find its data to
    be unreliable or incomplete and the state fails to correct the
    deficiencies. OCSE grants a one-year corrective action period before
    assessing the penalty. State officials reported that the current
    audit schedule provides insufficient time to correct errors. In the
    past two years, OCSE has released its audit findings between March
    and July, which has left states with as little as three months to
    correct any errors before they must submit data for the next fiscal
    year (October 30^th^, with corrections permissible through the end
    of December). States receive informal feedback from the auditors
    when they are on site---after the start of the calendar year.
    However, even with the early feedback, states still have less than a
    full corrective action year.

-   *Less frequent data reliability audits would be sufficient***.** The
    majority of state officials suggested that once a state has passed
    the audit for all measures for a number of years, OCSE should move
    to a less frequent audit process---perhaps every two or three years.

# Introduction

## A. Background {#a.-background .unnumbered}

States finance their child support enforcement (CSE) programs through a
variety of funding streams. The largest stream of program revenue is
Federal Financial Participation, in which the federal government
reimburses states for 66 percent of allowable child support costs.
States finance the remaining 34 percent (or the state share) of CSE
expenditures with state or local general fund appropriations and some
combination of the following revenue sources:

-   State share of retained Temporary Assistance for Needy Families
    (TANF) collections

-   User charges and fees

-   Federal incentive payments

Federal incentive payments are the subject of this study. An earlier
study found that in Fiscal Year (FY) 1997, incentives funded 25 percent
of state CSE programs.[^3] Directed by the Personal Responsibility and
Work Opportunity Reconciliation Act of 1996 (PRWORA), the Department of
Health and Human Services (DHHS), in consultation with state CSE
directors, developed a performance-based incentive funding system. In
1998, Congress enacted the Child Support Performance and Incentive Act
(CSPIA).

The 1998 law requires DHHS to produce interim and final reports that
detail the implementation of this new system and offer recommendations
for changes in the system that would improve operation of the child
support program. The *Interim Report*[^4] provided background on the new
incentive system, including its structure; described the incentive
calculation and payment processes, including data collection and data
reliability audits; and provided information on audit results and
incentive payments for FYs 1999 to 2001, including trends in incentive
payments. This report first updates trends in state performance. Next,
it explores state CSE programs' experiences with the implementation of
the new system and their perceptions of the system's problems,
successes, advantages, and disadvantages.

### Original Incentive System

The federal government has paid incentives to state programs since the
inception of the child support enforcement program (Title IV-D of the
Social Security Act) in 1975. The initial rationale was to encourage
improved collections. Originally, incentives were calculated by
determining each state's cost-effectiveness---defined as total
collections divided by total administrative expenditures. Depending upon
a state's cost effectiveness, the federal government paid an incentive
ranging from 6 to 10 percent of the state's collections. [^5] Incentive
payments were not contingent upon any measure of data reliability.[^6]

Federal and state policymakers believed this incentive payment structure
had two shortcomings. First, the difference between minimum and maximum
incentive payments was only 4 percentage points, and the system
guaranteed states a minimum of 6 percent of total collections,
regardless of their performance. To a degree, these conditions acted as
a disincentive for states to actively attempt to improve the quality of
their child support programs. Second, the system focused only on
cost-effectiveness, ignoring other important aspects of state CSE
programs, such as paternity establishment and order establishment.

### Performance-Based Incentive System

PRWORA required the DHHS Secretary to develop a performance-based,
revenue-neutral incentive system in collaboration with state CSE
directors. In response, the Office of Child Support Enforcement (OCSE)
convened the Incentive Funding Work Group (Work Group), which consisted
of 26 representatives from state and local IV-D programs, DHHS regional
offices, and the OCSE central office. The Work Group's final report
recommended to the Secretary of DHHS the structure and components of the
new incentive system.

In 1998, Congress enacted CSPIA, which adopted many---but not all---of
the Work Group's recommendations. CSPIA changed the incentive process in
multiple ways. Key elements of the new system include:

**Incentive payments are linked to state performance in five areas.**
States are assessed on the percent of paternities established, the
percent of orders established, the percent of current support collected,
the percent of cases paying toward past-due support, and
cost-effectiveness.

**Data must be reliable and complete**. OCSE conducts annual data
reliability audits for each state. If an audit finds that a state's data
is not complete and reliable for a given measure, the state receives
zero payments for that measure.

**Incentives are based on state collections and performance***.* The
incentive calculation process begins with the calculation of each
state's collection base, which is the sum of (1) collections for current
and former assistance cases and Medicaid-only cases multiplied by two
and (2) collections for cases never on public assistance. Collections
from current and former TANF, Foster Care, and Medicaid-only cases are
doubled to encourage states to continue to work what are often difficult
cases and to collect support for families working toward
self-sufficiency.

The collection base for the paternity establishment, cases with orders,
and current collections measures are weighted higher (at 100%) than the
other two measures (at 75%).

Once the collection bases are calculated, they are multiplied by a
specified incentive percentage, depending on state performance (e.g., a
state establishing an order for 80% of its cases is eligible for 100% of
the incentive payment). The Work Group recommended the percentages based
on established standards of performance.

**OCSE pays incentives from a capped incentive payment pool**. Congress
limited incentive payments to the levels projected by the Congressional
Budget Office (CBO) at the time Congress was considering the
legislation. By capping the amount of incentives paid, Congress ensured
total payments would not exceed those anticipated under the old system,
and thus would have no effect on the federal budget.

**States must reinvest incentives into state CSE programs**. Incentives
must supplement, and not supplant, other funds used by the state to
carry out child support enforcement activities.

**Penalties are associated with failure to meet or improve
performance**. Penalties are assessed for three measures: paternity
establishment, order establishment, and current collections.

## B. Methodology {#b.-methodology .unnumbered}

Project work for the final report included data analysis, in-depth
interviews with CSE officials in nine states, and conference calls
facilitated by the National Council of Child Support Directors.

### Data Analysis

The *Interim Report* included state performance trend analysis for FYs
1999 to 2001. This report updates the analysis. In addition, we
conducted empirical research to help us understand the factors that
affect state performance under the new incentive system and to inform
the state selection process. We developed a statistical model to explain
changes in state incentive payments during the transition to the new
system. Underlying this work is a multivariate regression analysis that
predicts the percentage difference in incentive payments received under
the new and old incentive systems based on a number of factors,
including:

-   State's share of the national collection base;

-   Share of the state's collections associated with current- and
    former-assistance cases;

-   State performance on the five measures; and

-   Other economic and demographic characteristics.

### In-depth Interviews 

The primary method of data collection was in-depth interviews with IV-D
staff from nine states. In selecting states, we took into account a
number of factors:

**Incentive funding under the new system**. We selected states that
received more funding under the new system as well as those that
received less funding.

**Experience with audits**. The data reliability audits are a key step
in the new incentive system and can have a large effect on incentive
payments. We selected states that had multiple audit failures as well as
those that passed all of their audits.

**Change in performance**. Some of the states selected performed poorly
during the initial year of the new incentive system and subsequently
improved. We wanted to learn what specific steps these states took to
improve their performance.

**Percent of collections for current and former public assistance
recipients**. Collections associated with current and former public
assistance cases and Medicaid-only cases are doubled when determining a
state's collection base. Interviews conducted for the *Interim Report*
suggested that the proportion of collections associated with the welfare
system would have an effect on incentive payments. We included states
with a high proportion of public assistance-related collections and
states with lower proportions of such collections.

**State characteristics**. We selected states that were diverse in terms
of geography, size, urbanicity, and structure of the CSE program (e.g.,
state versus county administered). States were selected in consultation
with the federal project officer. The nine study states were:

-   California

-   Maine

-   Maryland

-   New Jersey

-   North Carolina

-   South Dakota

-   Texas

-   Virginia

-   Washington

In each state, we interviewed a range of child support officials at both
the state and local level (see ***Exhibit I.1***).

Exhibit I.1: Staff Interviews

+---------------------+--------------------+---------------------------+
| **State-Level       | **Local-Level      | **County-Level Staff (if  |
| Staff**             | Staff**            | appropriate)**            |
+---------------------+--------------------+---------------------------+
| -   IV-D director   | -   Office-level   | -   County-level IV-D     |
|                     |     supervisors    |     director              |
| -   Policy director |     selected by    |                           |
|                     |     the central    |                           |
| -   Budget director |     office         |                           |
|                     |                    |                           |
| -   Systems or data | -   Office-level   |                           |
|     director        |     caseworkers    |                           |
+---------------------+--------------------+---------------------------+

We developed detailed interview guides that were divided into six
sections: structure of the new incentive system, the incentive
implementation process, data collection and audit processes, state
activities to improve performance, budget ramifications of the new
incentive system, and state assessments of the new system. The guides
were tailored to the staff person interviewed (e.g., state-level staff
guides contained more questions about data reliability audits than the
caseworker guides).

### Conference Calls Facilitated by the National Council of Child Support Directors

Finally, the National Council of Child Support Directors notified us
that a number of states---beyond the nine selected for the study---were
interested in providing input into the study. The chair of the Council
offered to coordinate voluntary conference calls with interested state
staff---typically IV-D directors. Topics discussed included the
structure of the new system (e.g., whether the correct areas are
measured and, if not, what should be included or excluded); whether the
new system favors states with certain profiles and, if so, whether
states should be compensated for factors beyond their control; and
general perceptions about the data reliability audit. These comments are
incorporated into the report where appropriate.

## Structure of Report

The remainder of this report is divided into six sections.

-   **Section II** details recent trends in state performance and the
    empirical research findings.

-   **Section III** describes state perceptions about the structure of
    the new incentive system, including the five performance measures,
    the performance standards, the factors that affect state
    performance, the cap, and the implementation time frame.

-   **Section IV** explores views about the data reliability audit and
    describes state data quality control efforts.

-   **Section V** describes state CSE efforts to improve performance on
    the incentive measures, including staff education, tracking the
    performance of line workers, and policy changes.

-   **Section VI** explores budget issues, including state CSE efforts
    to predict incentive payments.

-   **Section VII** provides conclusions.

# State performance

## Introduction

In FY 2002, state CSE programs had three years of experience with the
new incentive system.[^7] As states' familiarity with the new system
grows and they adopt policies and practices to improve their programs,
one would expect them to exhibit stronger performance in each of the
five measures and produce more reliable data. This section reviews trend
data for the five performance measures from FYs 2000 to 2002; describes
state performance on the data reliability audits; and explores factors
that are associated with strong performance under the new system.

## Trends

The trend analysis explored performance for FYs 2000 and 2002 in six
areas: the two paternity establishment measures (states have the option
of calculating paternity establishment for the IV-D caseload or
statewide), cases with orders, collections on current support due, cases
paying toward past-due support, and cost-effectiveness. As ***Exhibit
II.1*** indicates, for states that passed data reliability audits,
performance improved in most areas.

Exhibit II.1: Trends in State Performance

Median State Scores, FYs 2000 and 2002\*

  ----------------------- ----------------- ----------------- -----------------
  **Measure**             **FY 2000 Median  **FY 2002 Median  **Percent Change
                          Score**           Score**           FY 00-02**

  Paternity---IV-D        69.93             86.64             24%

  Paternity---Statewide   91.92             90.14             (2%)

  Cases with Orders       67.09             73.10             9%

  Current Support         54.60             58.50             7%
  Collected                                                   

  Cases Paying Arrears    57.41             61.03             6%

  Cost-Effectiveness      \$4.13            \$4.49            9%
  ----------------------- ----------------- ----------------- -----------------

\* States that did not pass data reliability audits in FYs 2000 or 2002
are excluded from the calculations of the measure for which they failed.

The median state score on each of the five performance measures
increased. Improvements were especially pronounced for states that use
the IV-D paternity establishment measure. The median state increased its
performance by 24 percent, from about 70 percent of paternities
established to over 86 percent. States increased the percent of cases
with orders by almost 10 percent, from 67 percent of cases to 73
percent. There was also improvement in the areas of current support
collected, cases paying toward past-due support (arrears), and
cost-effectiveness. Performance decreased slightly for states that used
the statewide paternity establishment measure, but remained above 90
percent.

These calculations do not include states that did not pass data
reliability audits. As will be discussed further in **Section IV**, each
performance measure is audited for completeness and accuracy. A state
that does not pass a data reliability audit for a measure does not
receive an incentive payment for that measure.

State data accuracy improved markedly between FYs 2000 and 2002. In FY
2000, 21 states failed an audit on at least one measure. In FY 2002,
only 12 states did so, despite the increased standard that needed to be
met to pass the audit. No states failed the audit for the current
collections and cost-effectiveness measures, and less than 5 percent of
all states failed the statewide paternity establishment, order
establishment, and arrears measures. However, states that use the IV-D
paternity establishment measure continued to experience data problems.
In FY 2000, over 40 percent of states using this measure failed their
data reliability audits; 35 percent did so in FY 2002.

## C. Empirical Analysis {#c.-empirical-analysis .unnumbered}

Our final data analysis task explored factors that affect state payments
under the new system relative to payments earned had the old incentive
system remained in place. State officials advanced many theories about
how the new incentive system is either an advantage or disadvantage to
states with certain characteristics. For example, some officials believe
that "universal states," those with large non-assistance caseloads, have
an advantage under the new system. Under prior rules, OCSE capped
incentives for non-assistance collections. The new system continues to
emphasize assistance-related collections by double-weighting current-
and former-assistance collections. However, CSPIA also removes the cap
on never-assistance collections. Observers also note that states with
large shares of never-assistance cases should---holding other factors
constant---exhibit better performance because the cases consist of
higher-income clients with more stable economic profiles.

Others counter that the mix of collections matters less than overall
size of the state's caseload and its collection base because incentive
payments for each performance measure are a function of a state's
collection base. Under the previous system, incentive payments were
based on collections, too. But under the new system, a state's share of
the incentive pool is its collections as a proportion of the national
incentive base, or the sum of all state collection bases. Although this
system is designed so that a state's share of the incentive pool is
proportional to the size of its collection base, some believe that
states with a very small share of the national collection base are
inherently disadvantaged under the new system.

Finally, other observers point to demographic and economic conditions
and suggest that states with traditionally high rates of unemployment
and poverty, and urban populations are inherently disadvantaged by
factors that are largely outside the control of the IV-D agency.

Given the variety of observations and theories advanced, we analyzed the
states that performed better under the new system and those that
received fewer incentive dollars in the transition between incentive
systems. To conduct such an analysis, we reviewed two components of
incentive calculations in FY 2001:

-   FY 2001 incentive payments if the new incentive system had been
    fully phased in; and

-   FY 2001 incentives payment if the old system were still fully in
    place.

In FY 2001, OCSE based actual incentive payments on the weighted average
of the two amounts, with two-thirds of the actual payment based on the
new system and one-third based on the old one. By focusing on the
constituent parts of this calculation, a clearer picture of states that
benefited and those that lost under the new system emerges.

The analysis found the average state would receive 13 percent more
incentive dollars under a fully phased-in new system than under the old
system. The average state payment would have been \$8.3 million under a
fully phased-in new system compared with \$7.4 million under the old
system. The change in incentive payments between the new and old systems
ranged from a 186 percent increase in payments in one state to an 88
percent decline in another. Data reliability played a key role in
payments. The 33 states that would have earned more incentives under a
fully phased-in new system combined for a total of 14 audit failures, or
an average of 0.42 failures per state. The 18 states that would have
earned fewer incentive dollars under a fully phased-in new system
combined for 19 audit failures, or an average of 1.05 failures per
state.

Although state performance and data accuracy and completeness are
critical elements of the new system, we noted examples of states that
had solid performance and few or zero audit failures that still lost
incentive funding relative to the old system. To gain a better
understanding of the factors that affect payment outcomes in the
transition from one incentive system to the other, we conducted
regression analyses. The data lend themselves to an application of
ordinary least squares regression analysis. This statistical technique
allows an exploration of the independent relationships between the
change in incentive payments and each of the factors that we hypothesize
affect that change.[^8] For example, we would like to know whether
collections associated with current- and former-assistance cases
continue to exhibit a relationship with the change in incentives
payments---after holding constant other important factors. In this
analysis, we used the following independent variables:

-   **State performance under new incentive system**. This variable is a
    weighted sum of state performance on each measure. It is possible,
    for instance, that a state passed all its audits but received low
    incentive payments because of poor performance (e.g., 45 percent of
    orders established). *The variable also takes into account audit
    failures* (i.e., a state receives zero payments for any measure in
    which it fails an audit). No state received 100 percent of its
    potential payment in FY 2001. The proportion of potential incentives
    earned ranged from 7 percent (in a state that failed data
    reliability audits for four measures) to 91 percent.

-   **State's share of the national collection base**. Several IV-D
    officials speculated that states with a larger share of the
    collection base are advantaged under the new system. State share of
    the national collection base ranged from 0.002 percent to 12
    percent.

-   **Percent of state collections for current and former
    public-assistance recipients.** Under the previous incentive system,
    states received a payment that was based largely on public
    assistance-related collections. Those for former-assistance cases
    and never-assistance cases were capped at 115 percent of
    public-assistance collections. The new incentive system removes the
    cap from collections for former- and never-assistance cases.
    However, to encourage states to make extra efforts to collect
    support for families that are receiving public assistance or
    Medicaid or those that left and are working toward self-sufficiency,
    collections for current- and former-assistance cases are doubled.
    State proportion of collections for current- and former-assistance
    cases ranged from 17 percent to 86 percent.

-   **Share of state population living in urban areas.** An earlier
    study found strong, negative associations between the percent of
    population living in urban areas and four performance
    indicators.[^9] Thus, states with large shares of their populations
    living in urban areas should fare worse---holding other factors
    constant---under the new system. The proportion of state population
    residing in urban areas ranged from 17 percent to 100 percent.

-   **Share of state's population that lived in the same house in 1999
    and 2000**. A forthcoming study for OCSE[^10] found a positive
    relationship between this variable and several performance measures.
    The share of the state's population residing in the same home ranged
    from 76 percent to 89 percent.

-   **Three year state poverty rate (2000)**. The same study found that
    the state poverty rate had a significant and negative relationship
    to state performance in three out of the five measures. The
    three-year state poverty rate ranged from 7 percent to 19 percent.

We used two models in our analysis. The only difference between the
models is the variables used to represent current- and former-assistance
collections as a share of total collections. CSPIA combines current- and
former-assistance collections in the determination of each state's
collection base. As such, the first regression model combined the
percent of collections for current- and former-assistance cases in one
variable. The second model separates the two variables. An analysis
indicated that current-assistance collections were strongly and
negatively associated with the change in incentive payments.[^11] That
is, an increase in the proportion of a state's collections that are from
current-assistance cases is associated with a decrease in incentive
payments. The state's share of collections from former-assistance cases
is not significantly associated with the change in payments. Separating
these variables allowed us to explore the individual effect of each on
incentive payments.

***Exhibit II.2*** depicts the results of the first model.[^12] Two
variables had statistically significant coefficients, both in the
expected direction. The variable measuring current- and
former-assistance collections is negative and statistically significant
at the 1 percent level, indicating that an increase in the proportion of
collections that are assistance-related is associated with lower
payments under the new system. The coefficient for the state performance
variable is positive and significant at the 1 percent level. This
suggests that as state performance increases, so, too, will incentive
payments.

Exhibit II.2: Model 1 Multivariate Regression to Predict Changes in FY
2001 Incentives

  ----------------------------------- -----------------------------------
  Factor                              Estimated Relationship with Change
                                      in Incentive Payment

  State Share of National Collection  Not significant
  Base                                

  Current and Former Assistance       Negative, significant at the 1
  Collections as a Share of Total     percent level

  State Performance (weighted)        Positive, significant at the 1
                                      percent level

  Percent Population in Urban Area    Not significant

  Population Stability                Not significant

  Three Year State Poverty Rate       Not significant
  ----------------------------------- -----------------------------------

> Source: The Lewin Group

***Exhibit II.3*** summarizes the second regression model, in which
current-assistance and former-assistance collections were separate
variables.[^13] Three variables show relationships that are
statistically different from zero. The current-assistance collections
variable is negative and statistically significant at the 1 percent
level. The former-assistance collection variable is not statistically
significant. State performance continues to have a strong positive
affect on the change in incentive payments. Finally, a stable
population---measured as the share of population that lived in the same
house in 1999 and 2000---also is positively associated with incentive
payments.

Exhibit II.3: Model 2 Multivariate Regression to Predict Changes in FY
2001 Incentives

  ----------------------------------- -----------------------------------
  Factor                              Estimated Relationship with Change
                                      in Incentive Payment

  State Share of National Collection  Not significant
  Base                                

  Current Assistance Collections as a Negative, significant at the 1
  Share of Total                      percent level

  Former Assistance Collections as a  Not significant
  Share of Total                      

  State Performance (weighted)        Positive, significant at the 1
                                      percent level

  Percent Population in Urban Area    Not significant

  Population Stability                Positive, significant at the 5
                                      percent level

  Three Year State Poverty Rate       Not significant
  ----------------------------------- -----------------------------------

> Source: The Lewin Group

Although it was mentioned as a possible factor in incentive payments by
a number of state officials, state share of the national collection base
was not statistically significant. In addition, the proportion of the
population residing in urban areas and state poverty rates were not
significant.

## Conclusion

Trend data from FYs 2000 and 2002 indicate that states continue to
improve performance in the majority of the five measured areas. In
addition, states' data reliability improved, even in light of the more
stringent audit requirements that went into effect in FY 2001.

The empirical analysis indicates that the change in incentive payments
in the transition to the new system can be largely explained by two
factors. First, state performance is associated with higher payments
under the new system. That is, states that earn a large share of their
maximum incentive for each measure and fail zero or few data reliability
audits receive larger incentive payments under the new system. This
finding supports the design of the new incentive system, which rewards
states for the performance of their programs and the accuracy of their
data. Second, the proportion of collections for current-assistance cases
is strongly and negatively associated with the change in incentive
payments under the new system. That is, states do not do as well if a
large share of their collections is assistance-related. The association
remains when former-assistance collections are included.

Under the previous incentive system, payments were linked to
current-assistance collections. CSPIA double-weights these collections,
along with those for former-assistance cases. The rationale for
double-weighting these collections was to ensure that states were not
penalized for pursuing collections on cases that may help reduce
custodial parents' dependence on public assistance. Although the
double-weighting of collections for current- and former-assistance cases
may lessen the burden on states with higher assistance-related
caseloads, the current formula may not fully compensate for the lifting
of the ceiling on never-assistance collections.

# State Interviews: Structure of the Incentive System

The remaining sections of this *Final Report* describe states'
perception of the new system. This section focuses on the structure of
the new system. Specifically, we asked CSE respondents from the nine
study states to assess the following issues:

-   The performance measures;

-   The performance standards;

-   Whether characteristics predisposed some states to perform better
    than others;

-   The capped incentive pool; and

-   The adequacy of the phase-in period.

State responses to questions about each of these issues are summarized
below.

## Assessment of the Performance Measures

The new incentive system assesses performance in five areas: paternity
establishment, order establishment, collections on current support due,
cases paying toward past-due support (arrears), and cost-effectiveness.
The first three are weighted more heavily when calculating incentive
payments.

We asked IV-D directors, other CSE state-level staff, and local office
staff to reflect on the five current performance measures and their
weighting. We explored whether states agreed that the correct areas were
measured and whether they are defined appropriately, whether any of the
current measures should be discarded, and whether new measures should be
added.

### Current Measures

The majority of the respondents in the nine study states said that the
new incentive system measures the correct performance areas.
Specifically, there was strong support for the first three
measures---paternity establishment, order establishment, and current
collections---because they are directly associated with the core child
support mission of getting support to families. As one state noted, "The
measures represent the type of work that CSE agencies do. It makes sense
to weigh those associated with collections higher." Due to the
importance of these functions, states agreed that they should be
weighted more heavily than either cases paying toward arrears or
cost-effectiveness. This sentiment was echoed during the calls
facilitated by the National Council of Child Support Directors.

Cost effectiveness was the only measure that received less than
unanimous support. Eight of the nine states provided a number of
justifications for de-emphasizing or dropping the measure: it is a
remnant of the old system; unlike the other indicators, it does not
directly measure CSE efforts to get support to families; and it is
short-sighted in that it penalizes investments that could improve the
program in the long run (e.g., purchasing new technology). Despite these
concerns, officials in only one state recommended dropping the measure
entirely. The others agreed that it is appropriate to weight
cost-effectiveness less than paternity establishment, order
establishment, and current collections.

Respondents in one state, however, disagreed with the prevailing
sentiment and argued that cost-effectiveness should remain in the
formula because it demonstrates to legislators and taxpayers that the
program is operating efficiently.

### Definitional Issues

Conversations with officials in the nine study states indicated general
support for the way in which the measures are defined. No issues were
raised for three of the measures---cases with orders, current
collections, and cost-effectiveness. Some state officials discussed
specific issues related to the definitions of paternity establishment
and cases paying toward arrears.

**Paternity Establishment**. States have the option of using one of two
paternity establishment definitions: IV-D or statewide. This is the only
measure that was legislated prior to CSPIA.

-   The IV-D definition measures the number of *children in the IV-D
    caseload* in the fiscal year (or at state option as of the end of
    the fiscal year) who were born out of wedlock who have paternity
    established divided by the total number of children in the IV-D
    caseload born out of wedlock in the prior fiscal year.

-   The statewide definition measures the number of *minor children in
    the state* born out of wedlock and have paternity established during
    the fiscal year by the total number of children in the state born
    out of wedlock in the prior fiscal year.

The majority of IV-D directors agreed that states should have a choice
of paternity definitions. As one director noted, states need to work
toward a 90 percent paternity establishment percentage to avoid
penalties.[^14] Thus, having multiple ways to measure paternity is
helpful.

Most of the comments about the paternity definition surrounded the
statewide measure. Some questioned the legitimacy of a statewide
measure, arguing that incentives should reward states for performance on
their cases, not for paternity established for children outside of the
IV-D program's domain. Proponents of the statewide measure, however,
noted that child support staff do paternity establishment work in the
hospitals irrespective of a child's IV-D status.

The treatment of interstate cases differs under the statewide measure.
Unlike the IV-D definition, the statewide definition does not give
states credit for establishing paternity for children born in another
state.[^15] Thus, a state could do the work to establish paternity for a
child and not be able to count the case. Directors and staff noted that
this can be especially problematic in big metropolitan areas that border
state lines (e.g., New York City).

**Arrears**. Officials also commented on the definition of the arrears
measure. The measure divides the *number of IV-D cases making at least
one payment* toward arrears by the number of cases with arrears due.

Some thought that a definition more similar to the current collections
measure---the *dollar amount* of arrears collected divided by the total
amount due---would be more appropriate. Under the current arrears
definition, a state could get the same credit for a case paying once
toward arrears as a case that pays multiple times, and for a case that
pays \$1 as a case that pays \$5,000. One staff person stated, "Counting
one payment on a case per year does not set the bar very high" and will
not result in the reduction of arrears. However, the majority of
officials opposed a change in definition, noting that treatment of
arrears varies considerably across states (e.g., states that cannot
"write off" arrears after a period of time will be greatly disadvantaged
by a definition that considers the total dollar amount of arrears due).

While there was disagreement among the states in terms of the arrears
unit of analysis (case versus dollar amount), CSE officials from all
states agreed that they should get credit for all cases that make
arrears payments via federal tax return intercepts, rather than only
those cases in which there was a payment to the former assistance
family. They noted that some non-custodial parents purposely do not make
payments because they know CSE agencies will intercept their tax
refunds. In many instances, CSE agencies work cases and get no
performance credit because of rules that dictate which collections can
and cannot be counted.[^16]

### Additional Measures

CSPIA requires the Secretary of DHHS, in consultation with state IV-D
directors, to develop a performance measure based on state effectiveness
in establishing and enforcing medical support obligations. The Secretary
and IV-D directors also will make recommendations for the incorporation
of a medical support measure into the incentive payment system in a
revenue neutral manner. Additionally, CSPIA requires that IV-D agencies
use the National Medical Support Notice to report on medical support
enforcement beginning in FY 2002.

We asked the IV-D officials in the nine study states if they thought
medical support, or any other measure, should be adopted. The majority
of state officials said the current five measures were sufficient and
that adding new measures would be premature.

All of the respondents addressed the issue of medical support. Officials
in three of the nine study states noted that CSE agencies are already
doing work in the medical support area and they should be compensated
for it. However, these states mentioned two caveats. First, if OCSE adds
a sixth measure, the national incentive pool should increase
proportionately. Second, developing a workable definition would be key
to the successful adoption of the measure. Officials from the other six
states had serious reservations about a medical support measure. Their
concerns generally fell into the following categories:

-   Officials questioned whether it is CSE's role to ensure that
    children receive medical coverage.

-   Non-custodial parents have limited incomes and would have difficulty
    providing both cash support and medical support.

-   Performance on medical support would vary widely across states. The
    measure would be difficult to define in a way that ensures a level
    playing field. A number of state officials indicated that, given the
    structure of their economies (e.g., seasonal workers, farming) it is
    hard to insure the general population, not to mention low-income,
    non-custodial parents. Moreover, the availability of health
    insurance for an employee's dependents varies based on employer
    practices and often state law. CSE programs have no control over
    whether a non-custodial parent's employer provides medical coverage.

-   No reliable data exits to formulate a viable measure.

Representatives from the nine states on the conference calls generally
agreed with these sentiments. The majority opposed a new medical support
measure but said that, if there was a new measure, Congress should add
funds to the incentive pool.

## Assessment of Performance Standards

State performance on each of the five incentive measures corresponds to
a specified percentage of the payment, which is defined in CSPIA. For
each measure, there is an upper threshold at which the state is eligible
for the maximum payment, and a lower threshold below which performance
is not rewarded unless the state can demonstrate substantial
improvement. For four of the measures---paternity establishment, cases
with orders, current collections, and cases paying toward
arrears---states must meet an 80 percent performance threshold to
receive 100 percent of the payment. States that establish less than 50
percent of paternities or orders receive no incentive payment; the
corresponding lower-bound threshold for current collections and cases
paying toward arrears is 40 percent. For the fifth measure, a
cost-effectiveness ratio of \$5.00 corresponds to 100 percent of the
payment. A ratio under \$2.00 receives no payment.

In addition to performance standards, states face possible *penalties*
for failure to reach specified thresholds for three measures: paternity
establishment, cases with orders, and current collections. The penalties
associated with paternity establishment performance were created by
PRWORA, thus prior to the adoption of CSPIA. Congress did not set the
penalty thresholds for the cases with orders and current collections
measures. Instead, the state and Federal Incentives Work Group used
historical data to determine penalty thresholds.[^17] The
recommendations were submitted to OCSE, which adopted them through a
standard rule-making process.[^18]

-   Paternity: States must achieve a score of 90 percent or higher or
    improve their performance by at least 2 to 6 percentage points over
    the previous FY, depending on their performance.[^19]

-   Cases with orders: States that fail to establish child support
    orders for at least 40 percent of open cases must improve their
    performance by 5 percentage points over the previous FY.

-   Current collections: States that fail to collect 35 percent of
    current support due must improve by 5 percentage points over the
    previous FY.

Penalties are assessed to the state's adjusted TANF block grant.[^20]
CSPIA grants states an automatic corrective action period of one fiscal
year immediately succeeding the performance year before imposing any
penalties.

### Performance Thresholds

The majority of respondents in the study states thought the performance
levels associated with incentive payments were about right. In a number
of instances, IV-D directors made reference to their participation in
the Work Group and the role it played in the development of the new
performance measures. Respondents from two states thought the
performance standards were too high. They pointed to current collections
and cases paying toward arrears as measures where very few states could
reach the top-performance threshold. During each conference call, at
least one or two IV-D representatives echoed this sentiment. They were
especially concerned that the recession would make it difficult to
perform at a high standard for current collections. Several directors
thought an upper threshold of 75 percent would be more realistic.
However, others cautioned against lowering the standard because it would
signal to lawmakers and the public that the program could not operate at
a high level.

### Penalties

Although the majority of respondents agreed with the established
performance standards, most had reservations about the penalties
associated with the paternity measure. First, states expressed concern
that a 90 percent penalty threshold for paternity establishment is too
high and that some states, particularly those with large urban centers,
will never reach it. Others noted that the 90 percent threshold, once
achieved, can be difficult to maintain.

A second issue was the incongruity between performance standards and
penalty thresholds. Officials from four states noted that it was
possible for a state to get 100 percent of incentives for the paternity
establishment measure and still face a penalty. A popular suggestion
during the conference calls was to lower the penalty threshold to 80
percent.

Finally, states commented on the penalty to the TANF block grant. Some
respondents thought it was unfair to penalize the TANF program for the
child support agency's performance.

## Factors that Affect Performance

We asked CSE staff from each of the nine study states two questions
relating to possible factors that affect performance under the new
incentive system. First, does the new system favor states with certain
profiles (e.g., large states)? Second, should OCSE alter the incentive
system to compensate states for economic or demographic factors beyond
their control?

With respect to the first question, officials in seven of the nine
states agreed that some states are predisposed to performing better
under the new incentive system. Some directors believe states that serve
a large share of cases that were never associated with the welfare
program are advantaged under the new system (e.g., non-custodial parents
associated with never-assistance cases are generally better off
economically, thus are better able to pay support). CSE staff,
particularly those from the smaller states, indicated that states with
larger populations do better under the new system because their
collection bases comprise a large proportion of the national incentive
base, thus the capped incentive pool.[^21] As such, improved performance
by smaller states does not necessarily translate into increased
incentives if the larger states also increase their collection bases.

The other two states, however, said that all states are disadvantaged in
some way. Because every state has poverty, unemployment, rural
populations, and urban pockets, the system does not predispose some
states to stronger performance.

In response to the second question, few officials supported the idea of
compensating states for factors beyond their control. Officials in six
of the nine states thought there should be no adjustments to the formula
to take into account economic, demographic, or other factors. Some
expressed concern that adding additional steps to the incentive
determination process would delay the payment of incentives and increase
the complexity of the new system. Others argued that there is no way to
accurately measure economic and demographic factors. For one, the data
used would not be audited. Further, there would be a lag between the
program year and the availability of statistics (e.g., final poverty
statistics for FY 2001 might not be available until FY 2002). Still
others contended that many economic and demographic characteristics are
not outside the state's control but rather are the outcomes of public
policy decisions. The majority of representatives on the National
Council of Child Support Directors calls agreed that every state faces
some challenges and expressed concern about the logistics of trying to
control for specific economic or demographic factors.

Officials in three states, however, thought adjustments were merited.
They argued that a number of "social ills" (e.g., unemployment rates,
non-marital birth rates) increase during economic downturns and that
states with large urban centers are particularly affected.

Finally, one IV-D Director believed OCSE should consider economic and
demographic characteristics in the incentive system but suggested any
related adjustments should be made during the penalty phase. That is,
OCSE would take into account circumstances beyond a state's control only
when determining a penalty (e.g., if a state's performance on paternity
establishment fell below 90 percent). This idea was shared by a number
of representatives during the conference calls, as well.

## Assessment of the Cap

Unlike the previous incentive system, CSPIA caps the incentive payment
pool for each fiscal year (see ***Exhibit III.1***).

**Exhibit I**II.1: Incentive Payment Pool

  -----------------------------------------------------------------------
  **Fiscal Year**        **Pool Amount**            **Change from
                                                    Previous FY**
  ---------------------- -------------------------- ---------------------
  2000                   \$422,000,000              n/a

  2001                   \$429,000,000              2%

  2002                   \$450,000,000              5%

  2003                   \$461,000,000              2%

  2004                   \$454,000,000              (2%)

  2005                   \$446,000,000              (2%)

  2006                   \$458,000,000              3%

  2007                   \$471,000,000              3%

  2008                   \$483,000,000              3%

  Succeeding FYs         Amount of pool from        
                         previous FY multiplied by  
                         percentage (if any) by     
                         which the Consumer Price   
                         Index[^22] for the         
                         preceding FY exceeds the   
                         CPI for the second         
                         preceding FY.              
  -----------------------------------------------------------------------

The capped system creates an interactive effect. That is, an increased
payment to one state must be matched by a decreased payment to another.
Congress based the levels of the incentive payment pool on CBO
projections of incentive payments at the time CSPIA was passed.[^23]

In addition to acting as a payment ceiling, the cap serves as a floor.
That is, OCSE distributes the full amount of the pool each year.

The interview protocol asked state-level staff about their perception of
the cap and its consequences. Many respondents noted that their states
experienced substantial increases in incentive payments under the new
system. In addition, the officials were unanimous in their opinion that
the cap should be removed. However, interview protocol did not ask
respondents how, in the absence of a cap, the federal government would
pay for incentives that exceeded the authorized amount. Respondents'
concerns generally fell into two areas:

-   **Cap as a disincentive.** The majority of officials suggested that
    the cap acts as a disincentive to improve performance because
    improvements in performance are not always rewarded. Under the new
    system, a state can improve its performance and because of the
    interactive nature of the cap, get a smaller incentive payment.
    Respondents seemed particularly worried that, if the states with the
    largest collection bases improve their performance, smaller states
    will face sizable reductions in their incentive amounts no matter
    how well they perform. One recommendation that received support
    during the conference calls was a "hold-harmless" provision that
    would prevent a reduction in a state's incentive payment over the
    previous year if the state maintained or improved its performance.
    Another official suggested OCSE establish a maximum incentive amount
    for which each state is eligible (i.e., instead of capping incentive
    payments to states as a whole, OCSE caps payments to individual
    states). If a state qualifies for the full payment, it would receive
    the total amount allotted to it, irrespective of other states'
    performance. If a particular state underperforms, its unearned
    incentives would be distributed to the higher performing states. The
    means by which the federal government would pay for any additional
    incentives, if needed, was not explored.

-   **Estimating incentive payments. *Section VI*** describes the
    difficulties state officials have encountered in estimating
    incentive payments. The majority of respondents said the cap was a
    main factor contributing to the difficulties. Because of the
    interactive nature of the cap, states need to estimate their own
    performance as well as the performance of others.

    As noted above, the interview protocol did not ask explicitly how
    the federal government would pay for incentive payments that exceed
    the amount currently authorized, should the cap be lifted. A number
    of states, however, did note that the cap was not necessary to
    preserve budget neutrality. They suggested that performance serves
    as a natural cap (i.e., not all states will receive 100 percent of
    the payment for all five incentive areas).

## Assessment of Phase-in Period 

OCSE implemented the new incentive formula over three years, from FYs
2000 to 2002. States reported information on the OCSE 157 in FY 1999,
but state performance did not factor into the incentive payment
calculation. In FY 2000, OCSE used the new incentive formula to
calculate one-third of states' incentive payment and the old incentive
system for the remaining two-thirds. In FY 2001, OCSE based two-thirds
of states' incentive payments on the new system. The new system became
the sole determinant of incentive payments in FY 2002.

Most officials said three years was sufficient to revise internal
management structures, familiarize staff with the new performance
measures and budgetary policies, update their computer systems, and
implement procedures to ensure data reliability. The phase-in period
allowed states to take corrective actions on performance and data
reliability before the new system became the sole determinant of
incentive payments. Similarly, the staff from states whose incentive
payments decreased under the new system said that the phase-in period
eased the impact of the new system on their budgets. Even officials in
states that benefited financially under the new system said that the
phase-in period was necessary in order for staff to make the necessary
adjustments and to identify any potential problems that might emerge.

# State Interviews: Data Reliability Issues

Annual data reliability audits (DRAs) determine if a state's data and
the system that produces it are accurate, complete, reliable, and
secure. The multi-step process begins with the state's submission of the
OCSE 157 report, which is used to calculate four of the five performance
measures.[^24] The report is due October 30^th^; states can make
corrections through December 31^st^.[^25]

The interview protocol included a number of questions about the DRA,
including the parameters of the audit (e.g., sample size) and data
quality control efforts.

## Data Reliability Audit Parameters

IV-D officials commented extensively on the DRA process. Specifically,
staff responded to questions about the number of cases audited, the
efficiency rate, the timing of the audit, and possible improvements to
the process.

### Sample Size

The OCSE auditors need at least 120 open cases to conduct the audit.
Thus, they select a minimum of 150 cases from the state's universe,
which consists of both open and closed cases. The size of the sample
depends on the differential between the size of each state's universe
and the number of open cases.[^26] If the universe and number of open
cases are equal, the sample size would be 150. If the universe was twice
as large as the number of open cases, the sample would be 300 (150 \*
2.0 differential).

Officials in six of the nine study states commented directly on the
sample size. The majority said that the sample size was adequate. One
respondent noted that the size is acceptable so long as it is
statistically valid. Another remarked that a small sample is good
because it reduces the time that staff need to spend pulling individual
cases for auditing. Officials in two other states, however, thought that
150 or 200 cases out of a universe of several hundred thousand were too
few, even if statistically valid. They noted that they routinely pull
several thousand cases for their annual self-assessments. They also
expressed concern that the actual number of cases audited for some
measures (e.g., cases paying toward arrears) would be so small that one
or two problem cases would cause the state to fail the audit for the
measure.

Representatives on the conference calls were divided, with a slight
majority arguing that the sample size was too small. One suggested that
OCSE use a "focus sample" method for the DRA. This process, used by
states for self-assessments, involves pulling a specific sample of cases
for each measure, thus ensuring an adequate sample size.[^27]

### Efficiency Rate

In order to pass the audit for each measure, states must achieve an
"efficiency rate" of 95 percent.[^28] The rate is determined by dividing
the number of sampled cases reported correctly by the number of sampled
cases reviewed for a particular line item. The upper and lower
confidence limits of the efficiency rate are calculated at the 95
percent confidence level.[^29] For example, if a state had 87 cases
reviewed for a measure, and 68 of the cases were reported correctly, the
efficiency rate for the measure would be 78 percent.[^30] Using the
confidence interval, OCSE would determine that the actual efficiency
rate was between 68 percent and 86 percent---below the 95 percent
threshold.

The majority of officials thought a 95 percent efficiency rate was too
high and left too little room for error. Interviewees from the nine
study states and conference call participants expressed support for some
type of sliding scale in which a state that failed to meet the 95
percent efficiency rate would lose some proportion of the incentive
payment but would not fail.

### Timing

The audit process begins when states submit finalized OCSE 157 forms at
the end of the calendar year. Audits begin in January and are conducted
by OCSE. The OCSE auditors begin with an entrance conference, at which
time the audit process is explained to the key state stakeholders. The
OCSE auditors present their results to state CSE staff at an exit
conference. If the auditors uncover problems while on site, they will
notify the state immediately. Following the audit, the state receives a
report summarizing the auditors' findings, including data that was
deemed incomplete or unreliable. Audit reports are completed in early
summer. Once all state data have been audited, OCSE calculates the
incentive payments. Incentive payments generally are finalized later in
the summer.

CSE officials in almost all of the study states thought the audit
process was too slow. As a result, it created difficult situations in
terms of estimating incentive payments and avoiding penalties. As noted
above, a state faces penalties if the DRA finds its data to be
unreliable or incomplete and it fails to correct the deficiencies. The
law grants states an automatic corrective action period of one fiscal
year immediately succeeding the performance year before OCSE imposes any
penalties. For example, a state that had problems with its paternity
data in FY 2002 would have until FY 2003 to correct the problem or face
a penalty. States reported that results from the audit are not received
in time to address issues during the corrective action year. They argue
that often audit results are not available until near the end of the
fiscal year, giving them very little time to correct any problems. Even
if auditors identify and relay problems while on site (generally in the
late winter or spring), states still would not have a full corrective
action year.

### Suggested Audit Process Improvements

CSE staff in the study states suggested a number of improvements OCSE
could make to the audit process.

**Conduct audits less frequently**. The majority of CSE officials said
that OCSE should eliminate annual audits once a state has passed all of
its measures for a specified number of years (e.g., three consecutive
years). At that point, audits should occur no more than every two or
three years. States that experience data problems with one or more
measures, however, should continue to be audited annually. IV-D
directors expressed similar sentiments during the conference calls.

A small number of directors on the calls, however, expressed concern
about reducing the frequency of audits, noting that there is a learning
curve associated with the audit process and that staff might need to be
re-educated every three years. Moreover, states might focus less
attention on data quality if they are not audited annually. One director
suggested that instead of tri-annual audits, a state that passes its
audits for a few consecutive years should be placed in a pool from which
OCSE would randomly select states to audit each year. Another official
noted that annual audits could be combined with the self-assessment
process. That is, an annual data reliability check-up could be included
in the self-assessment, and the more formal DRA could be conducted every
three years.[^31]

**Institute a delay period***.* One suggestion that received support was
the institution of a delay period in the incentive process so that IV-D
directors would have audit information available when estimating their
budgets. For example, the FY 2003 incentives would be paid in FY 2004.
When developing their FY 2004 budgets, directors would be able to refer
to the FY 2003 audit results. The audit would be pre-estimate, rather
than post-estimate.

**Increase audit staff***.* Officials in two states suggested increasing
the number of audit staff so that audits would be completed earlier in
the year.

**Establish a clear appeals process**. Some state officials noted that
the new system does not include a formal process for states to appeal
findings or decisions made by the auditors or OCSE. They suggested that
a formal appeal mechanism be inserted into the audit process prior to
the calculation of incentive payments. However, states can appeal an
audit finding to the Commissioner of OCSE.

## Quality Control Efforts

In addition to questions regarding the DRA process, we asked state and
local IV-D staff about data quality control efforts. We found that the
study states devoted considerable resources to improving data
reliability, including changes to automated systems, extensive case
cleanup exercises, and assessing the performance of local offices.

### Revisions to Automated Systems

Staff from most of the study states indicated that the OCSE 157 forms
are fully automated, making the audit process easier. They also changed
their systems to catch human errors. One created fire walls so that a
case cannot leave the establishment mode and move to enforcement without
a line worker completing a checklist (e.g., ensures the non-custodial
parent's Social Security Number (SSN) is in the system, paternity is
established). Another state's automated fact-checking programs look for
odd patterns in data and flag cases for attention. State officials note
that these automated fail-safes have forced workers to better evaluate
their cases.

One state added a new field to its system---state of birth. The state
failed the paternity establishment audit in part because the automated
system did not capture state of birth, but instead attributed a state
based on the first three digits of the SSN. New entrants into the
program are now asked about the child's state of birth.

### Data Clean-up Projects

A number of respondents said their states undertook data clean-up
projects in response to the new incentive system---often focusing on
paternity data. One state took a sample of paternity establishment cases
and went to the local offices to check that the staff had properly
documented the cases. Another state uncovered problems with its
paternity data when it switched to a new automated system. The state
found that about 30,000 children had incorrect paternity information in
the new system due to a combination of data entry errors and incorrect
data definitions. An 18-month data clean-up effort ensued. The state ran
automated paternity checks on a monthly basis to ensure that local
offices were working to correct inaccuracies. The state's automated
system generated case lists for each office. After the clean-up effort,
less than 500 cases had incorrect information.

State officials also use ad hoc reports to identify data problems. For
example, in one state, staff coded the order establishment measure
incorrectly by coding cases where a court's decision was still pending.
Cases are sent to local offices for corrective action.

### Assessing Local Offices

Several states created state-level or regional auditing teams that
randomly check data quality on elements specific to the OCSE 157. When
these auditing teams identify consistent errors across offices, they
typically offer worker-training sessions to expedite corrections. Office
supervisors also have been trained to conduct audits of a random sample
of cases and to take corrective actions.

# State Interviews: Efforts to Improve Performance

The goal of the new incentive system is to provide a financial reward to
states that find innovative, cost-effective means to improve child
support performance in the five measured areas. During our interviews
with state officials, we explored in detail the means by which they
attempted to affect performance during the first three years of the new
system. These include staff education and training, assessment of staff
performance, and major policy changes that were implemented in an effort
to improve performance. We found numerous examples of innovative efforts
to adapt to the requirements of the new incentive system.

## Staff Education and Assessment

IV-D officials in every state interviewed reported that they have made
substantial efforts to ensure that all staff are working toward the
common goal of improving performance in the five designated areas. As
indicated earlier, respondents agreed that the performance measures in
the new system largely align with the central goals of CSE programs. As
such, CSE programs were quick to integrate these standards into staff
assessment procedures and provide the necessary staff training.

### Education and Training

According to IV-D staff, education and training regarding the new
incentive system occurs during conferences and monthly meetings. Some
IV-D agencies have created special training teams to visit local offices
when problems arise. The new system also is described in revised policy
manuals. Each is discussed below.

The new incentive system, and its effect on CSE programs' business
models and fiscal positions, has been a prominent, standing topic of
statewide and regional IV-D conferences. Moreover, local staff indicated
that office- and worker-level performance on the indicators is an
on-going focus of office meetings (usually monthly) and training
sessions. Finally, IV-D officials reported that their states'
performance on the federal indicators and the quality of their data were
frequently covered in agency newsletters, emails, and presentations
(many states make this information available for staff on their
intranet). In short, officials from all nine study states reported that
CSE staff are aware of the performance measures, their importance to the
programs' missions, and their fiscal implications.

Officials in several states noted that CSE programs have developed
specialized training teams that review regional or office-level
performance and tailor training activities to address observed
deficiencies. Typically, the teams monitor office or regional
performance and notify relevant supervisors of downward trends in
performance or if staff are not adhering to particular procedures (e.g.,
automatically requesting a wage withholding when verifying an employer
address for a case with an order). If the poor performance persists
after notification, these roving teams develop a training workshop and
implement it in relevant offices.

Several states also indicated that they have updated and revised IV-D
policy manuals to explicitly highlight the importance of the performance
measures and emphasize tasks or processes deemed important to enhancing
performance on the five measures.

### Assessment

In addition to on-going education and training opportunities at
conferences and staff meetings, IV-D officials report that they assess
the performance by region, county, local office, and, in some cases,
staff member.

IV-D officials in all of the study states reported they follow trends on
the federal performance measures very closely and that automated systems
were enhanced to track performance at multiple levels. At the state
level, policy directors review performance trends at the regional,
county, or---depending on the size of the state---office levels.
Interviews with local office supervisors indicated that many of them
track the performance of individual workers on a monthly basis for all
relevant indicators. One state, for instance, uses a "traffic light"
system to rank offices by each incentive measure: Red indicates below
the minimum standard for payment; Yellow indicates below state goals but
above the minimum; and Green indicates above the state goal. The
officials hope this will create friendly competition among the local
offices.

Officials in a minority of states indicated that relevant performance
indicators are explicitly considered in annual performance reviews of
staff. While most officials indicated that performance did not directly
influence office-level funding or the salaries of individual staff, a
number of respondents indicated that their IV-D agency takes into
account performance on the federal indicators when selecting staff for
promotions. In most states, the IV-D agency rewards strong performance
on the federal indicators with award lunches, plaques, and certificates.
Budget enhancements (for offices) or merit bonuses (for individuals) are
less common, particularly given states' difficult fiscal positions.

Interviews indicated that the periodic performance reports often lead to
competition between jurisdictions (e.g., between counties across the
state or between offices within a county). On balance, IV-D officials
viewed the competition as positive. Moreover, officials indicated the
process did not damage inter-office relations or hamper the sharing of
best practices. However, some individual caseworkers interviewed said
that open reporting of caseworker-level performance was not productive.
As they put it, "nobody likes to be singled out."

In addition to office and staff tracking, several states---particularly
those with county-administered programs---have called on regional and
local offices to develop detailed business plans that establish
performance benchmarks, as well as step-by-step actions that offices
will take to achieve the benchmarks. Several officials reported that the
act of developing and updating the plans strengthened localities'
understanding of their current performance, as well as the key barriers
to improving it. Moreover, plan development actively engages local
supervisors and workers in the details of performance improvement.

## Policy Changes

As part of this study, OCSE was interested in learning about significant
policy changes enacted by states to affect one or more of the
performance measures. According to IV-D directors, performance
improvement is factored into almost every initiative their agencies
undertake. Rather than discuss the full range of policy changes
implemented in recent years, we asked respondents to highlight
significant changes that, in their views, had sizable, short-term
impacts on one or more of the measures. Those changes include:

**Closing appropriate cases**. Most states indicated that they carefully
reviewed their caseloads to find cases that are eligible for closure
according to published federal guidelines. [^32] The regulations do not
mandate that cases be closed. Specified closure criteria protect client
families by clarifying conditions under which cases can be closed. One
example is cases in which there is a failure to locate the non-custodial
parent despite legitimate and repeated attempts, due to inadequate
identifying or location information.[^33]

Under the prior incentive system, states could hold cases open for long
periods of time with little or no consequence. However, under the new
system, an inactive case, with no likelihood for a change in status,
weakens the state's performance---particularly on the cases with orders
measure---because the inactive cases remain in the universe of open
cases against which the number of orders established is assessed.

**Emphasizing payable orders.** All states use guidelines to determine
child support orders. They generally take into consideration the needs
of the child, other dependents, and the ability of the parents to pay.
States expressed concern that guidelines may set orders too high for
low-income parents, resulting in the failure to pay support and the
accumulation of arrearages. The officials suggested that orders more in
line with a parent's ability to pay would result in a higher percentage
of collections.

Examples of policies to create payable orders include addition of a
self-sufficiency reserve to state child support guidelines. A reserve
allows the non-custodial parent to maintain a minimum subsistence level
regardless of the number of children he or she is obligated to support.
State officials generally believe the reserves strengthen performance on
the current support and arrears measures. These policies are not
intended to absolve non-custodial parents of their responsibilities to
their children. Rather, state officials believe that these policies will
facilitate self-sufficiency for non-custodial parents and increase the
likelihood that they will be able to provide consistent support to their
children over the long term.

**Reducing the rate of default orders.** Default orders are entered when
a non-custodial parent does not respond to requests for financial
information, or does not respond to requests to appear in court or at a
IV-D agency meeting. Often, the order is based on imputed income levels.
States use a variety of methods to impute income, such as minimum wage
for a 40 hour week, the average wage for workers in the state, and
median income for non-custodial parents on the IV-D caseload. There is
concern that default orders are often unpayable. IV-D officials were
near unanimous in their opinions that default orders ultimately weaken
performance on the collection-related measures---current collections and
cases paying toward arrears.

State and local officials reported a number of initiatives designed to
reduce the number of default orders and instead create orders that have
a higher likelihood of resulting in regular payments. In some states,
IV-D agencies have developed special order establishment teams that
permit more upfront interactions with the non-custodial parent and his
or her employer. Another state offers group establishment interviews for
non-custodial parents, which are designed to disseminate information
about the child support enforcement process and the importance of
collaborating with requests for financial information.

**Expediting order establishment.** Respondents in two states pointed to
an increased reliance on administrative processes in order establishment
as the key to improving their performance on the cases with orders
measure. State officials credited the moves with reducing case backlogs
in the court system and improving establishment timeframes.

**Offering arrearage amnesty.** In an attempt to boost collections on
arrears, some officials noted that their states have offered
non-custodial parents amnesty from interest payments if the parent
negotiates an arrearage payment plan during a specified time period.

**Extending current support when child reaches majority age for cases
with arrears.** To boost performance in cases paying toward arrears, a
few respondents said their states now automatically extend the child
support order---at current support levels---beyond the age of majority
if arrears are outstanding. After a child reaches majority status and
child support technically ends, the non-custodial parent continues to
make payments at the current support level until the arrears are paid in
full. The policy saves caseworkers the step of making an upward
modification to the arrears-only order.

**Examining staff caseloads.** Staff from one surveyed state said that
the performance-based incentive system was the impetus to conduct a
staffing standard study; it is looking to other states for evidence of
the optimal number of cases per caseworker.

# State Interviews: Budget Issues

Incentives under the new system play a critical role in the development
of IV-D agency budgets. Unlike the system prior to CSPIA, the federal
government now requires states to reinvest incentives in the IV-D
program. Because the reinvested incentive dollars are eligible for
federal matching funds, each additional dollar in incentives supports
three dollars in total program funding. Similarly, the loss of an
incentive dollar implies a three-dollar loss in total program funding if
not replaced.

OCSE essentially pays incentives on an on-going basis using state
*forecasts* of what their incentives will be for a particular year. For
example, if a state expects \$8 million in incentives for fiscal year
2003, it would draw \$2 million each quarter during the federal fiscal
year.[^34] Once the audited performance data become available, OCSE
reconciles the amount actually earned with the amount previously
estimated and retained by states. For example, if OCSE determines actual
incentives for the fiscal year totaled \$10 million, the state would
draw an additional \$2 million. If, on the other hand, OCSE determines
the state actually earned only \$6 million, the state would have to pay
\$2 million.

The mechanics of the payment process puts a premium on an accurate
forecast of incentives. However, the complexity of the incentive
calculation, as well as the inherent payment interdependency created by
the cap, makes accurate incentives forecasts very challenging.

## Forecasting Payments

State officials unanimously agreed that the new incentive system makes
it very difficult to estimate incentive payments. Under the old system,
they had to estimate only their cost-effectiveness, public assistance
collections, and non-assistance collections. Under the new system, they
must predict cost-effectiveness and collections, as before, but
additionally must estimate their performance in three other areas.
Furthermore, even with perfect information about their own collections
and performance, state IV-D officials cannot accurately predict their
own payments because they are influenced by the collections,
performance, and data reliability of the other states. Respondents from
the nine study states offered a number of observations about forecasting
incentive payments under the new system.

First, state methods to predict incentive payments vary in their
sophistication. A number of officials said they simply have held their
quarterly incentive draws from the federal government unchanged for a
number of years and make no formal attempt to pinpoint a particular
year's incentives. Officials from two states calculate incentives using
a spreadsheet model that incorporates assumptions about own-state
performance, collections, and data reliability, the national collection
base, and trends in performance and data reliability for other states.
Individual IV-D agencies have developed and shared the spreadsheet
models with colleagues in other states.

IV-D officials in the nine study states indicated that there would be
severe consequences if they overestimate their incentive payments. Most
officials believed their programs would suffer layoffs and the loss of
special programs. Given incentive earmarking to IV-D programs, the loss
of each incentive dollar translates to a loss of three dollars in total
program funding. Budget officials noted that in the event of an
overestimate, the agency would unexpectedly have to pay back OCSE over a
period of time. The majority of states suggested that a required payback
would result in a temporary or permanent disruption to the program.[^35]

Given payment uncertainties and harmful potential consequences of
overestimating payments, state officials said they are conservative in
their forecasts; many intentionally underestimated incentives. [^36]
When OCSE announces the actual incentive award, the surplus (or amount
over the forecast) does not typically result in a windfall to the
agency. More often it is the case that legislative appropriators advance
monies to the IV-D agencies based on the directors' demonstrated need,
keeping the potential for incentive payments in mind. If the incentive
award is higher than assumed in the IV-D budget, the excess incentive
amount reimburses the state general fund. In short, most program
directors reported that---as long as the state has met its reinvestment
requirements---higher than expected incentives do not automatically
translate into a windfall for the agency.

State officials suggested several changes to aid predictability of
incentives**.** Some suggested OCSE could distribute the estimated
*national collection base* in advance of the audit findings on
performance.[^37] All recommended eliminating the cap on incentive
payments. Because the effect of the cap was raised within the context of
payment forecasting, respondents were not asked to comment on the issue
of maintaining budget neutrality.

## Reinvestment of Incentives

As noted above, states must reinvest incentive payments into their CSE
programs. OCSE enforces the reinvestment requirement as follows. OCSE
calculated a "base" child support expenditure for each state, which
equaled the state's total expenditures in FY 1998 *minus* the amount of
incentive funds that it received and reinvested during the same year.
Additionally, OCSE allows states to substitute an alternate three-year
average for the FY 1998 amount.[^38] Going forward, OCSE requires states
to maintain the base expenditure. Moreover, the incentive payments must
be in addition to, and not in lieu of, the base amount.

Six of the nine states interviewed earmarked some or all of their
federal incentives to the IV-D agency *prior* to CSPIA.[^39]
Consequently, the federal requirement to reinvest incentives in the IV-D
program affected only a minority of states. In states that required a
change in policy, budget officials created earmarked accounts and
implemented special regulations to allow the IV-D agency to accept and
spend the monies. In one state that did not previously earmark
incentives to the IV-D agency, the reinvestment rule eliminated the
agency's short-term need to request a general appropriation.

One area explored with regard to reinvestments was the use of an
alternate three-year average base amount. Most of the interview states
selected the method that resulted in the lowest base amount. However, in
one state, the executive branch of the government, representing the IV-D
agency, attempted to implement the highest base amount. Later a
legislative oversight committee discovered the lower options and
required the agency to adopt the lowest base amount. Officials from that
state believe the resulting tensions over the agency's budget could have
been avoided if OCSE had provided states more guidance on how to adopt
the base amount.

# Conclusions

The 1998 Child Support Performance and Incentive Act created a new
incentive system to reward states that operate effective child support
enforcement programs. The new system assesses state performance in five
areas---paternity establishment, order establishment, percent of current
support paid, cases paying toward past-due support, and
cost-effectiveness. The *Interim Report* provides background on the new
incentive system, including its structure; describes the incentive
calculation and payment processes; and provides information on initial
program results and incentive payments. This final report explored state
experiences implementing the new system, including perceptions of its
successes, problems, advantages, and disadvantages. Project work
included analysis of state performance trend data, research on factors
that affect performance, and discussions with child support officials in
nine states.

Key findings are:

**State and local CSE staff expressed near universal support for the new
incentive system**. While almost every state and local official
interviewed recommended refinements, nearly all agreed the
performance-based incentive system focused state and local energies on
the important core functions of the IV-D program.

**New system measures the appropriate areas**. The majority of
respondents believed that the appropriate performance measures were
selected and weighted correctly. Specifically, they expressed strong
support for the measures that are most directly associated with the core
IV-D mission of getting support to families: paternity establishment,
order establishment, and collections on current support due. Most
respondents said the five measures were adequate and that adding new
ones would be premature.

**Performance thresholds are reasonable.** Most respondents agreed that
the performance thresholds associated with incentive payments were about
right (e.g., states that establish 80 percent of paternities receive 100
percent of the payment).

**Reporting system shows performance has improved in most areas.**
Analysis of OCSE administrative data shows state performance in most
incentive areas improved between FYs 2000 and 2002. Median scores for
states with reliable data increased for IV-D paternity establishment
(from 70% to 87%), cases with orders (from 67% to 73%), cases paying
toward past-due support (from 57% to 61%), percent of current support
collected (55% to 59%), and cost-effectiveness (from \$4.13 to \$4.49).
The median state score for the statewide paternity establishment measure
remained over 90 percent.

**Data reliability improved during the three-year implementation
period.** In addition to improving performance, states also improved
data reliability. Between FYs 2001 and 2002, the number of states
failing a data reliability audit for at least one measure decreased by
half, from 25 to 12. Discussions with IV-D officials indicated that
states have devoted considerable resources to improve data reliability,
such as: revisions to their automated systems (e.g., automating key
forms used in the audit and creating safeguards to catch human errors);
extensive data clean-up projects, often in response to issues identified
by federal auditors; and assessment of performance of offices. In
general, states suggested that the increased importance the new system
places on data reliability has had a positive effect on the overall
effectiveness of their programs.

**State CSE programs have taken steps to ensure that all staff are
working toward performance improvement.** States have gone to
significant lengths to disseminate the information on the federal
performance measures. States reported that child support staff are aware
of the performance measures, their importance to the program's mission,
and their fiscal implications. Activities include:

-   Providing education and training in multiple forums, including
    statewide and regional IV-D conferences, monthly office meetings,
    newsletters, emails, and presentations;

-   Creating specialized training teams that review regional and
    office-level performance and tailor training activities to address
    observed deficiencies;

-   Developing automated systems to track performance at regional and
    office levels; and

-   Using some or all federal measures in workers' performance
    evaluations.

**State CSE programs adopted a number of policy changes to affect
measures**. States report that performance improvement is factored into
almost every initiative that child support agencies undertake. Policy
changes include:

-   Making a concerted effort to review cases that are appropriate for
    closure under federal guidelines. Under the previous incentive
    system, keeping inactive cases on the system did not affect state
    performance. Under the new incentive system, these cases are
    included in the universe of open cases and can negatively affect
    state performance on the paternity establishment and cases with
    orders measures.

-   Emphasizing the creation of "payable" child support orders. State
    policies include an attempt to lower the rate of default orders
    (i.e., orders that are entered when a non-custodial parent does not
    respond to requests for financial information or requests to appear
    at an administrative forum or court), which are often based on
    imputed income, or by adding a self-sufficiency reserve to child
    support guidelines that allows the non-custodial parent to maintain
    a minimal subsistence level. The policies are not intended to
    absolve the non-custodial parents of their responsibilities to their
    children, but aim to increase the likelihood that they will provide
    consistent support to their children over the long term.

-   Increasing reliance on administrative processes to establish orders
    in an attempt to expedite establishment and improve performance on
    the order establishment measure.

-   Encouraging payment of current and past-due support orders by
    offering non-custodial parents amnesty from interest payments if
    they negotiate an arrearage payment plan or extending current
    support collections when a child reaches the age of majority if the
    non-custodial parent owes arrears.

**Change in incentive payments to individual states---relative to what
they would have received under the old system ---is related to
performance and the nature of the state's caseload.** Empirical analysis
indicates that change in any given state's incentive payments---relative
to what it would have earned under the old system\--could be largely
explained by the following factors: (1) state performance and reliable
data are strongly and positively associated with higher payments under
the new system; (2) the proportion of a states' collections that come
from current assistance cases is strongly and negatively associated with
the change in incentive payments under the new system.

**State CSE officials do not want OCSE to adjust incentive payments for
state socio-economic conditions**. Child support officials were asked if
the new system favors states with certain profiles, such as stable or
affluent populations, and, if so, whether states should be compensated
for factors beyond their control. Most respondents agreed that IV-D
programs operate in different socio-economic environments. However, the
majority of state representatives opposed adjusting incentive payments
to account for those differences because such adjustments would add
complexity and uncertainty to an already complicated payment system and
that gaining consensus on an appropriate list of adjustment factors
would be extremely challenging.

**State CSE officials recommended refinements to the new incentive
system.**

-   *The capped incentive pool raised concerns*. State officials noted
    that the cap makes estimating incentive payments very difficult.
    Because states are competing for a fixed pool of incentive dollars,
    a state must anticipate not only its own performance and data
    reliability, but also that of every other state, in order to
    forecast its own incentive payments. Moreover, they expressed
    concern that the cap creates a dynamic in which better performance
    is not always matched by higher payments. Because incentive payments
    are a function of both performance and state collections, a state
    could improve performance but receive a lower payment if other
    states with larger collection bases also improved their performance
    or data reliability. For these reasons, officials from all study
    states voiced support for removing the cap. Although states strongly
    supported the cap's removal, the interview protocol did not explore
    how the federal government would pay for incentive payments that
    exceeded the amount currently authorized.

-   *The audit schedule does not allow for a full corrective action
    period*. A state faces a penalty if OCSE auditors find its data to
    be unreliable or incomplete and the state fails to correct the
    deficiencies. OCSE grants a one-year corrective action period before
    assessing the penalty. State officials reported that the current
    audit schedule provides insufficient time to correct errors. In the
    past two years, OCSE has released its audit findings between March
    and July, which has left states with as little as three months to
    correct any errors before they must submit data for the next fiscal
    year (October 30^th^, with corrections permissible through the end
    of December). States receive informal feedback from the auditors
    when they are on site---after the start of the calendar year.
    However, even with the early feedback, states still have less than a
    full corrective action year. OCSE is aware of the concerns related
    to the timing of the corrective action year and is working with
    Congress to address.

-   *Less frequent data reliability audits would be sufficient***.** The
    majority of state officials suggested that once a state has passed
    the audit for all measures for a number of years, OCSE should move
    to a less frequent audit process---perhaps every two or three years.

[^1]: Assistance cases are those receiving Temporary Assistance for
    Needy Families or Foster Care services.

[^2]: K. Gardiner, M. Fishman, A. Glosser, and J. Tapogna. *Study of the
    Implementation of the Performance-Based Incentive System: Interim
    Report.* Prepared for the Office of Child Support Enforcement, U.S.
    Department of Health and Human Services, Washington, D.C. October
    2003.

[^3]: M. Fishman, K. Dybdal, and J. Tapogna (1999). *State Financing of
    Child Support Enforcement Programs*. Prepared for the Assistant
    Secretary for Planning and Evaluation and the Office of Child
    Support Enforcement, U.S. Department of Health and Human Services,
    Washington, D.C.

[^4]: K. Gardiner, M. Fishman, A. Glosser, and J. Tapogna. *Study of the
    Implementation of the Performance-Based Incentive System: Interim
    Report.* Prepared for the Office of Child Support Enforcement, U.S.
    Department of Health and Human Services, Washington, D.C. October
    2003.

[^5]: Between 1975 and 1984, incentive payments applied only to
    collections for families on public assistance. Starting in 1984,
    incentive payments were extended to non-welfare collections.
    However, as an inducement to retaining a focus on welfare-related
    collections, Congress limited non-welfare collections to a
    percentage of welfare collections (115 percent).

[^6]: Audits of state IV-D programs were conducted at least once every
    three years to ensure compliance with federal IV-D requirements. The
    primary focus of the audits was on administrative procedures and
    processes as opposed to performance outcomes and results.

[^7]: Incentive payments were based, at least in part, on the new system
    starting in FY 2000. However, states were first exposed to the new
    system in FY 1999, the first year that data was audited. The audit
    results were not factored into the incentive payments that year.

[^8]: The relationship between the change in incentive payments Y and
    the explanatory variables X is assumed to be of the form Y~i\ =~
    ~0~ + ~1~X~1i~ + ~2~X~2i~ + \.... + ~n~X~ni~ + ~i~. Regression
    analysis provides estimates of the values of the ß terms.

[^9]: M. Fishman, J. Tapogna, K. Dybdal, and S. Laud. (2000).
    *Preliminary Assessment of the Associations between State Child
    Support Enforcement Performance and Financing Structure.* Prepared
    for the Assistant Secretary for Planning and Evaluation and the
    Office of Child Support Enforcement, U.S. Department of Health and
    Human Services, Washington, D.C.

[^10]: J. Tapogna, K. Gardiner, B. Barnow, M. Fishman, and P. Nikolov.
    August 2003. *Study of State Demographic and Economic Variables and
    their Impact on the Performance-Based Child Support Incentive
    System.* Prepared for the Office of Child Support Enforcement, U.S.
    Department of Health and Human Services, Washington, D.C.

[^11]: The variable correlation coefficient was -0.67 and was
    significant at the 1 percent level.

[^12]: The variables in the model explain 32 percent of the variation in
    the dependent variable.

[^13]: Overall, the variables explain 73 percent of the variation in the
    dependent variable.

[^14]: According to PRWORA, states that score below 90 percent on the
    paternity measure must have improved their performance from the
    previous year by a specified amount (dependent on their performance)
    to avoid a penalty to the TANF block grant. For instance, a state
    that had a paternity establishment percentage between 75 percent and
    89 percent would need to meet the 90 percent threshold or improve 2
    percentage points to avoid a 1-2 percent decrease in the TANF block
    grant. A state that had a score of 39 percent or less would need to
    improve 6 percentage points. See **Section III.B**.

[^15]: OCSE's instructions for the statewide definition specify that a
    state may only count children for whom paternity was established *if
    the child was born in the state*.

[^16]: The general rule regarding the counting of cases for the
    arrearage measure is as follows: Cases can be counted if (1) all of
    the past-due support was disbursed to the family because all support
    was owed to the family, as would be the situation with
    never-assistance cases; or (2) All of the past-due support was
    retained by the state because all support was assigned to the state,
    as would be the situation with current assistance cases. But for
    former assistance cases, states can only count the cases if some
    past-due support was owed to the family and was paid to the family.
    When collections for families no longer on welfare are made through
    tax intercepts, they are retained by the state and applied to
    arrearages that are owed to the state. Thus, these payments do not
    count toward the arrearage incentive measure.

[^17]: Under Section 409(a)(8)(A) of Title IV-A of the Social Security
    Act, the Secretary of DHHS may establish penalties for additional
    performance measures.

[^18]: OCSE AT-99-12, dated October 12, 1999.

[^19]: States must improve by a specified percentage each year until
    they reach 90 percent. For instance, a state that had a paternity
    establishment percentage between 75 percent and 89 percent would
    need to improve 2 percentage points each year. A state that
    established only 39 percent of paternities would need to improve 6
    percentage points.

[^20]: The first time a state fails the penalty threshold for a
    particular measure, it may be penalized between 1 and 2 percent of
    its TANF funds. For the second failure on a particular measure, the
    penalty increases to 2 to 3 percent of TANF funds, and so forth, up
    to a maximum of 5 percent of TANF funds for each measure.

[^21]: A step in the incentive calculation process involves totaling all
    of the state collection bases to create a national incentive base. A
    state's collection base divided by the national incentive base
    determines its share of the capped incentive pool.

[^22]: CPI for a FY is the average CPI for the 12-month period ending
    9/30. For example, for fiscal year 2009, if the CPI increases by 1
    percent between FYs 2007 and 2008, then the incentive pool for FY
    2009 will be a 1 percent increase over the \$483,000,000 incentive
    payment pool for FY 2008, or \$487,830,000.

[^23]: At that time, CBO predicted declines in public assistance
    caseloads and associated collections during FYs 2004-2005,
    explaining the pool's decline in those years.

[^24]: Paternity establishment, order establishment, current
    collections, and cases paying toward arrears.

[^25]: States use two other forms---the OCSE 34A and 396A---to report
    information used to calculate the cost-effectiveness ratio; states
    submit these quarterly.

[^26]: Reported on Line 1 of the OCSE 157.

[^27]: OCSE notes that auditors pull random statistically valid samples.

[^28]: In FY 2000, the first year of the new incentive system, the
    efficiency rate was set at 90 percent. Federal regulation set the
    efficiency rate to increase to its current level of 95 percent in FY
    2001 and all future years.

[^29]: States only fail audits when the upper bound of the 95 percent
    confidence interval for the percent of cases within the sample for
    which the data is unreliable exceeds the current data reliability
    standard---95 percent. The confidence interval is a statistical
    method employed to reduce the chance that a value is the result of
    random chance as opposed to being representative of the universe in
    question. The size of the confidence interval is dependent on the
    sample size. Because there is less room for error the larger the
    sample, the upper and lower bounds of the confidence interval are
    much smaller for large samples. The confidence interval counteracts
    any potential error that could otherwise result from a smaller
    sample.

[^30]: 68 cases divided by 87 cases.

[^31]: OCSE is currently considering policies to reduce the frequency of
    the audits.

[^32]: In recent years, OCSE has released a number of documents that
    provide guidance to states on proper case closure procedures.

    AT-99-04, "Case Closure Criteria Final Rule, 45 CFR Part 303"
    explains the legislation dictating case closure policy and responds
    to the comments of representatives from state and local IV-D
    programs and other interested parties. Available on-line:
    http://www.acf.hhs.gov/programs/cse/pol/AT/at-9904.htm.

    PIQ-00-02, "Interstate Case Closure When Custodial Parent Location
    Is Unknown" addresses two specific questions related to case
    closures raised by states. Available on-line:
    http://www.acf.dhhs.gov/programs/cse/pol/PIQ/piq-00-02.htm.

[^33]: According to the guidance, "Decisions to close cases are linked
    with notice to recipients of the intent to close the case and an
    opportunity to respond with information or a request that the case
    be kept open."

[^34]: In practice, the states receive their incentives by reducing the
    amount of retained public assistance collections they send to the
    federal government. For example, in a given quarter, a state may
    have retained \$20 million in public assistance collections---with
    one half of the total owed to the state and the other half owed to
    the federal government. Rather than sending the full \$10 million to
    the federal government, the state might send \$8 million and hold
    \$2 million in anticipated incentive payments.

[^35]: Among the states interviewed as part of this study, only one
    overestimated incentives---and did so only the first year the new
    system was implemented. In that case, the IV-D agency repaid the
    federal government over a four-quarter period and revised its budget
    to reflect the unanticipated shortfall.

[^36]: Some noted that there is a downside to being overly conservative
    with estimates. One state noted that conservative incentive
    estimates can weaken trust with appropriators. Legislative
    appropriators, who may be only vaguely aware of the mechanics of the
    new incentive system, may view conservative incentive forecasts as
    short-term attempts to secure larger general-fund appropriations.

[^37]: OCSE releases preliminary data early in the year in an effort to
    assist states with their estimates.

[^38]: OCSE allows states to use the three-year average because
    officials believe it may more closely approximate the amount a state
    has been spending on its IV-D program and will not give undue weight
    to any extraordinary or non-recurring expenditures the state may
    have made in FY 1998.

[^39]: Nationally, 42 of 51 states did so in FY 1997. See Fishman et al.
    (1999).