![](media/image1.wmf){width="2.4993055555555554in"
height="0.9368055555555556in"}

**Second Error Rate Pilot Report**

*Prepared for:*

Child Care Bureau

Office of Family Assistance

Administration for Children and Families

1250 Maryland Ave. SW, 8th Floor

Washington, DC 20024

Under Contract Number 233--02--0093

Task Order Number 08

*Prepared by:*

Walter R. McDonald & Associates, Inc.

12300 Twinbrook Parkway, Suite 310

Rockville, MD 20852

June, 2007

**TABLE OF CONTENTS**

[Executive Summary 1](#__RefHeading___Toc167079279)

[I. INTRODUCTION 8](#__RefHeading___Toc167079280)

[**A. Background** 8](#__RefHeading___Toc167079281)

[**B. Objective of the Second Pilot** 9](#__RefHeading___Toc167079282)

[II. METHODOLOGY 11](#__RefHeading___Toc167079283)

[**A. overview of the Process to Measure Improper Payments**
11](#__RefHeading___Toc167079284)

[**B. Fieldwork Preparation** 12](#__RefHeading___Toc167079285)

[**C. Generating the Sample** 12](#__RefHeading___Toc167079286)

[**D. Customizing Record Review Worksheets**
15](#__RefHeading___Toc167079287)

[**E. Site Visits** 16](#__RefHeading___Toc167079288)

[**F. Conducting Record Reviews** 17](#__RefHeading___Toc167079289)

[**G. Completing the Data Entry Form** 17](#__RefHeading___Toc167079290)

[**H. Computing Error Rates** 18](#__RefHeading___Toc167079291)

[**I. State Response Form** 19](#__RefHeading___Toc167079292)

[III. MEASURING IMPROPER PAYMENTS IN FIVE STATES
20](#__RefHeading___Toc167079293)

[**A. Improper Payments Process** 20](#__RefHeading___Toc167079294)

[**B. Standards, Processes, and Procedures**
21](#__RefHeading___Toc167079295)

[**C. Automation** 21](#__RefHeading___Toc167079296)

[IV. FINDINGS AND STATES\' RESPONSES 24](#__RefHeading___Toc167079297)

[**A. Florida** 24](#__RefHeading___Toc167079298)

[1. Results 25](#__RefHeading___Toc167079299)

[2. Florida Responses 25](#__RefHeading___Toc167079301)

[**B. Kansas** 26](#__RefHeading___Toc167079302)

[1. Results 26](#__RefHeading___Toc167079303)

[2. Kansas Responses 27](#__RefHeading___Toc167079305)

[**C. New Jersey** 27](#__RefHeading___Toc167079306)

[1. Results 28](#__RefHeading___Toc167079307)

[2. New Jersey Responses 28](#__RefHeading___Toc167079309)

[**D. Oregon** 29](#__RefHeading___Toc167079310)

[1. Results 29](#__RefHeading___Toc167079311)

[2. Oregon Responses 30](#__RefHeading___Toc167079313)

[**E. West Virginia** 30](#__RefHeading___Toc167079314)

[1. Results 30](#__RefHeading___Toc167079315)

[2. West Virginia Responses 31](#__RefHeading___Toc167079317)

[**F. Conclusion** 32](#__RefHeading___Toc167079318)

[1. Summary of Results 32](#__RefHeading___Toc167079319)

[2. Most Common Errors 34](#__RefHeading___Toc167079321)

[3. Interpretation of Missing Documentation Errors
35](#__RefHeading___Toc167079324)

[4. Coding of Missing Documentation Error Types
35](#__RefHeading___Toc167079325)

[5. Review Process and Missing Documentation Error
35](#__RefHeading___Toc167079326)

[6. Interpretation of Income Errors 36](#__RefHeading___Toc167079327)

[7. Coding of Income Error Types 36](#__RefHeading___Toc167079328)

[8. Interpretation of Hours of Care Errors
36](#__RefHeading___Toc167079329)

[9. Summary of State Responses 37](#__RefHeading___Toc167079330)

[V. COST ANALYSIS 39](#__RefHeading___Toc167079331)

[**A. Objectives of the Cost Analysis**
39](#__RefHeading___Toc167079332)

[**B. Conceptual Design** 39](#__RefHeading___Toc167079333)

[**C. Measuring the Costs to Conduct the Error Rate Pilot**
40](#__RefHeading___Toc167079334)

[**D. Cost Analysis Worksheet Development**
40](#__RefHeading___Toc167079335)

[**E. Preparation and Site Visit Costs**
41](#__RefHeading___Toc167079336)

[**F. Record Review Costs** 42](#__RefHeading___Toc167079338)

[1. Four States in the First Pilot 42](#__RefHeading___Toc167079339)

[2. Five States in the Second Pilot 43](#__RefHeading___Toc167079340)

[VI. RECOMMENDATIONS 48](#__RefHeading___Toc167079344)

[**A. Management of the Error Rate Methodology Process**
49](#__RefHeading___Toc167079345)

[1. State Project Team Formation and Responsibilities
49](#__RefHeading___Toc167079346)

[**B. Provision of Instructions and Managing the Review Process**
50](#__RefHeading___Toc167079347)

[1. State Review Team Selection and Training
50](#__RefHeading___Toc167079348)

[2. Instructions for Error Interpretation and Coding
51](#__RefHeading___Toc167079349)

[3. Record Review Site 52](#__RefHeading___Toc167079350)

[**C. Use of the Instruments** 52](#__RefHeading___Toc167079351)

[1. Record Review Worksheet Development and Use
52](#__RefHeading___Toc167079352)

[2. Data Entry Form 53](#__RefHeading___Toc167079353)

[3. Cost Analysis Form 54](#__RefHeading___Toc167079354)

[4. State Response Form 55](#__RefHeading___Toc167079355)

[**D. Modifications to Improve Reliability across Future Pilot States**
56](#__RefHeading___Toc167079356)

[**E. Implications for ongoing Technical Assistance**
57](#__RefHeading___Toc167079357)

APPENDICES

[Appendix A. Florida 59](#__RefHeading___Toc167079358)

[**A. Organizational Structure** 59](#__RefHeading___Toc167079359)

[**B. Overview of the Child Care Eligibility Process**
59](#b.-overview-of-the-child-care-eligibility-process)

[**C. Improper Payment Process** 59](#__RefHeading___Toc167079361)

[**D. Case Review Process** 61](#__RefHeading___Toc167079362)

[**E. Automation** 61](#__RefHeading___Toc167079363)

[**F. Next Steps** 62](#__RefHeading___Toc167079364)

[**Florida Site Visit Participants** 62](#__RefHeading___Toc167079365)

[Appendix B. Kansas 72](#__RefHeading___Toc167079367)

[**A. Organizational Structure** 72](#__RefHeading___Toc167079368)

[**B. Overview of the Child Care Eligibility Process**
72](#__RefHeading___Toc167079369)

[**C. Improper Payment Process** 72](#__RefHeading___Toc167079370)

[**D. Case Reviews** 73](#__RefHeading___Toc167079371)

[**E. Automation** 74](#__RefHeading___Toc167079372)

[**F. Next Steps** 75](#__RefHeading___Toc167079373)

[**Kansas Site Visit Participants** 75](#__RefHeading___Toc167079374)

[Appendix C. New Jersey 83](#__RefHeading___Toc167079376)

[**A. Organizational Structure** 83](#__RefHeading___Toc167079377)

[**B. Overview of the Child Care Eligibility Process**
83](#__RefHeading___Toc167079378)

[**C. Improper Payment Process** 84](#__RefHeading___Toc167079379)

[**D. Case Review Process** 85](#__RefHeading___Toc167079380)

[**E. Automation** 85](#__RefHeading___Toc167079381)

[**F. Next Steps** 86](#__RefHeading___Toc167079382)

[**New Jersey Site Visit Participants**
86](#__RefHeading___Toc167079383)

[Appendix D. Oregon 98](#__RefHeading___Toc167079385)

[**A. Organizational Structure** 98](#__RefHeading___Toc167079386)

[**B. Overview of the Child Care Eligibility Process**
98](#__RefHeading___Toc167079387)

[**C. Improper Payment Process** 99](#__RefHeading___Toc167079388)

[**D. Case Reviews** 100](#__RefHeading___Toc167079389)

[**E. Automation** 100](#__RefHeading___Toc167079390)

[**F. Next Steps** 101](#__RefHeading___Toc167079391)

[**Oregon Site Visit Participants** 101](#__RefHeading___Toc167079392)

[Appendix E. West Virginia 105](#__RefHeading___Toc167079394)

[**A. Organizational Structure** 105](#__RefHeading___Toc167079395)

[**B. Overview of the Child Care Eligibility Process**
105](#__RefHeading___Toc167079396)

[**C. Improper Payments** 105](#__RefHeading___Toc167079397)

[**D. Case Review Process** 106](#__RefHeading___Toc167079398)

[**E. Automation** 106](#__RefHeading___Toc167079399)

[**F. Next Steps** 107](#__RefHeading___Toc167079400)

[**West Virginia Site Visit Participants**
108](#__RefHeading___Toc167079401)

[APPENDIX F. RECORD REVIEW WORKSHEET TEMPLATE
115](#__RefHeading___Toc167079403)

[APPENDIX G. DATA ENTRY FORM 111](#__RefHeading___Toc167079404)

[Appendix H. ERROR RATE COST ANALYSIS 112](#__RefHeading___Toc167079405)

[APPENDIX I. STATE RESPONSE FORM 114](#__RefHeading___Toc167079406)

[APPENDIX J. SITE VISIT AGENDA 118](#__RefHeading___Toc167079407)

[APPENDIX K. Summary of Pilot Study States
119](#__RefHeading___Toc167079408)

**LIST OF EXHIBITS**

[Exhibit 1. Florida Findings
[25](#__RefHeading___Toc167078076)](#__RefHeading___Toc167078076)

Exhibit 2. Kansas Findings [26](#__RefHeading___Toc167078080)

Exhibit 3. New Jersey Findings [28](#__RefHeading___Toc167078084)

[Exhibit 4. Oregon Findings
[29](#__RefHeading___Toc167078088)](#__RefHeading___Toc167078088)

[Exhibit 5. West Virginia Findings
[31](#__RefHeading___Toc167078092)](#__RefHeading___Toc167078092)

[Exhibit 6. Summary of Findings
[32](#__RefHeading___Toc167078096)](#__RefHeading___Toc167078096)

[Exhibit 7. Frequency of Elements in Error
[34](#__RefHeading___Toc167078451)](#__RefHeading___Toc167078451)

[Exhibit 8. Dollar Amounts of Elements in Error
[35](#__RefHeading___Toc167078452)](#__RefHeading___Toc167078452)

[Exhibit 9. Preparation, Training and Site Visit Costs for Nine Pilot
States [41](#__RefHeading___Toc167078111)](#__RefHeading___Toc167078111)

[Exhibit 10. Cost of Record Review Process for Four States in the First
Pilot [44](#__RefHeading___Toc167078115)](#__RefHeading___Toc167078115)

[Exhibit 11. Cost of Record Review Process for California
[45](#__RefHeading___Toc167078116)](#__RefHeading___Toc167078116)

[Exhibit 12. Cost of Project Tasks for Five States in the Second Pilot
[46](#__RefHeading___Toc167078117)](#__RefHeading___Toc167078117)

[Exhibit A-1. Florida Record Review Worksheet
[64](#__RefHeading___Toc167078139)](#__RefHeading___Toc167078139)

[Exhibit B-1. Kansas Record Review Worksheet
[76](#__RefHeading___Toc167078148)](#__RefHeading___Toc167078148)

[Exhibit C-1. New Jersey Record Review Worksheet
[88](#__RefHeading___Toc167078156)](#__RefHeading___Toc167078156)

[Exhibit D-1. Oregon Record Review Worksheet
[102](#__RefHeading___Toc167078165)](#__RefHeading___Toc167078165)

[Exhibit E-1. West Virginia Record Review Worksheet
[109](#__RefHeading___Toc167078174)](#__RefHeading___Toc167078174)

[]{#__RefHeading___Toc167079279 .anchor}**Executive Summary**

Congress responded to the issue of improper payments in Federal programs
by enacting the Improper Payments Information Act (IPIA) of 2002. This
law requires Federal agencies to identify programs that are vulnerable
to improper payments and to estimate annually the amount of
underpayments and overpayments made by these programs.

The Executive Branch also has worked to address the improper payments
issue. A goal of the President's Management Agenda (PMA) is "eliminating
improper payments." To this end, agencies are to establish a baseline of
the extent of improper payments and to set goals to reduce improper
payments.

Because the PMA and IPIA provide for establishing a child care error
rate as one on-going measure of program efficiency, the Child Care
Bureau (CCB) launched the project: \"Measuring Improper Payments in the
Child Care Program\" to identify and describe methods that could help
States identify, measure, and prevent improper payments in the
administration of the Child Care and Development Fund (CCDF). A major
focus of the CCB project was to work in partnership with States to
determine the feasibility of computing a child care improper payments
error rate.

The CCDF is a block grant that allotted over \$ 5 billion to States,
Territories, and Tribes to support child care subsidies for low-income
working families during Fiscal Year (FY) 2006.[^1] The CCDF block grant
allows maximum flexibility for States, Territories, and Tribes to set
policies regarding eligibility and fiscal management approaches, as well
as define administrative structures that allow maximum choice for
parents. As a result, States\' eligibility criteria, rates, regulation
of child care providers, and payment mechanisms vary widely among
jurisdictions.

During FY 2005, the CCB contracted with Walter R. McDonald & Associates,
Inc. (WRMA) to develop and pilot test in four States, a common
methodology that States could use to compute an error rate. The CCB, in
consultation with an initial workgroup of "partner States," chose to
focus the analysis on administrative error related to eligibility in
order to measure an element that is common to every State and to
mitigate some of the variation among State policies and procedures.

The methodology of the first pilot had four main components:

-   The contractor assisted each pilot State to select a random sample
    of up to 150 cases (children), using a sampling frame of all
    children in the State authorized to receive a child care payment[^2]
    during October 2004. The sample size was designed to produce a
    statistically valid estimate of erroneous payments.

-   Pilot States customized a *Record Review Worksheet* template to
    reflect child care policies in the State. The States used this
    instrument to guide a record review of the sampled cases to identify
    administrative errors in eligibility determination. States collected
    data regarding the number of cases with errors and whether the
    errors led to an improperly authorized payment. Although all of the
    worksheets contained common elements, the definitions pertaining to
    those elements varied from State to State.

-   The contractor conducted site visits and provided technical
    assistance to the pilot States who then conducted record reviews and
    collected data. The contractor computed the error rates using the
    data submitted by the pilot States.

-   The contractor also conducted telephone interviews to gather
    additional information about improper payment activities in five
    States. These States were Arizona, California, Kansas, Nebraska, and
    New Hampshire.

The purpose of the FY 2006 pilot was to further test an error rate
methodology in five States. Similar to the first error rate pilot, the
methodology of the current pilot focuses on client eligibility and
employs a case record review process to identify cases with errors,
cases with errors that result in improperly authorized payments, and
percentages of improperly authorized payments. However, in the second
pilot the methodology provides pilot States with their results in order
to analyze the types and sources of error. Following an analysis of the
findings from the case record review process, the contractor forwarded
the results to the pilot States for their internal review. Pilot States
responded to a short survey providing an explanation for the causes of
the errors and a description of next steps or corrective actions to be
taken as a result of participation in the error rate pilot.

Definitions of the error rates calculated for this pilot and the
relevant findings are as follows:

-   **Percentage of cases with an error**---This percentage is based on
    the number of sampled cases with an error, regardless of whether it
    resulted in an improperly authorized payment or not, compared to the
    total number of cases in the sample. *In the second pilot, the
    percentage of sampled cases with errors ranged from 19 percent to 35
    percent. The four States in the first pilot had a slightly wider
    range, with the percentages of cases with errors ranging from 12
    percent to 44 percent.*

-   **Percentage of cases that have an improperly authorized
    payment---**This percentage is based on the number of sampled cases
    with errors that have an improperly authorized payment, compared to
    the total number of cases in the sample. *In the second pilot, the
    percentage of sampled cases with errors that had an improperly
    authorized payment ranged from 11 percent to 35 percent. The four
    States in the first pilot had a slightly lower range, with the
    percentages of cases with errors that resulted in an improperly
    authorized payment ranging from 6 percent to 32 percent.*

-   **Payment Error Rate (Percentage of improperly authorized payments
    for the review period)---**The payment error rate is the percentage
    of the gross amount of improperly authorized payments (overpayments
    plus underpayments) for the review period compared to the total
    amount of authorized payments in the sample. *The estimated
    percentage of improperly authorized payments in the five pilot
    States ranged from a low of 2 percent to a high of 18 percent. The
    results are very similar to the percentages of improperly authorized
    payments in the first pilot which ranged from 4 percent to 20
    percent.*

-   **Average amount of improperly authorized payment**---The average
    amount of improperly authorized payment is the average amount of
    money the State improperly authorized on a per child basis during
    the designated review month. *The average amount of the improperly
    authorized payment made in the five pilot States ranged from a low
    of \$89.42 to a high of \$215.82. The four States in the first pilot
    had a somewhat wider range, with the average amount of the
    improperly authorized payment ranging from \$88.99 to \$289.53.*

*The findings from both pilots include several sources of administrative
error, such as incomplete or missing documentation and inaccurate
calculation of income. The three predominant error types in the second
pilot were incomplete or missing documentation, the miscalculation of
income (both earned and unearned), and incorrect hours of care. All five
States in the second pilot documented these three error types in
significant numbers, which accounted for 79 percent of the total dollars
in error. Although the treatment of missing documentation varied between
the States, missing documentation was the primary error cause and
accounted for 28 percent of the total improperly authorized payments in
the second pilot and 57 percent of the total improperly authorized
payments in the first pilot.*

One goal of the error rate pilots was to test a methodology that could
provide States with useful information on areas for improvement in
administrative policies and procedures. All nine States acknowledged
learning additional useful information about the quality of their
eligibility processes. As a result of this pilot, each of the five
States has planned action steps or has implemented several new
systematic changes to improve monitoring and reduce improper payments.
The State strategies include:

-   Strengthen supervision of new eligibility workers;

```{=html}
<!-- -->
```
-   Clarify selected standards with eligibility workers;

-   Improve IT system elements to:

    -   Prevent or decrease calculation errors,

    -   Generate exception reports to highlight areas of potential
        problems or concern,

    -   Implement automatic income calculation, and

    -   Enhance the capability of extracting data from other data
        systems;

```{=html}
<!-- -->
```
-   Conduct extensive technical assistance in counties to address
    error-prone areas;

-   Institute changes in the monitoring process;

-   Introduce statutory changes to simplify access to other State
    databases; and

-   Examine State policies to determine what changes may be necessary to
    provide more consistent application of policies and procedures.

Some of the State specific actions are:

-   West Virginia Technical Assistance staff will continue to perform
    random quarterly case audits. Prior to the pilot, the FACTS
    electronic record was the object of the review. An expanded review
    will now include the physical case record.

-   Oregon continues to conduct a series of "Accuracy Summits" in all
    areas of the State that focus on techniques to reduce errors in the
    Food Stamp program. Beginning with the next series of summits in
    July 2007, workshops to improve payment accuracy in child care will
    be included in the summits.

-   Oregon plans to modify the Food Stamp targeted review process and
    database that contains review information to ensure that income
    errors identified in the Food Stamp case are corrected in the
    companion Employment Related Day Care (ERDC) case.

-   Kansas continues to complete monthly Child Care QC reviews and uses
    the results of these reviews to inform training needs. Kansas is
    updating the Child Care Personnel Trainer and Training Academy to
    emphasize case documentation and computation of hours of care needed
    and income. Kansas supervisors now complete monthly case reviews for
    the Child Care Assistance Program.

-   Kansas contracted with a firm to develop software to track all case
    review findings and provide aggregate review data. The web based
    system tracks aggregate case review data for the mandatory Food
    Stamp reviews and the child care reviews. The child care reviews
    began in July 2006. The software cost \$75,000 and there is an 18
    percent annual maintenance costs agreement.

-   Florida developed a desk reference tool for eligibility procedures,
    initiated focused monitoring and training on eligibility procedures
    on an annual basis and is providing each Early Learning Coalition
    (ELC) with its individual results of the pilot review process along
    with recommendations for corrective action.

```{=html}
<!-- -->
```
-   New Jersey is implementing a process to conduct electronic matching
    of the automated child care client database with records on other
    wage, SSI, and child support information systems to identify and
    reduce the number of improper payments.

```{=html}
<!-- -->
```
-   New Jersey plans to continue conducting child care record reviews
    utilizing the pilot monitoring tool, using a methodology that
    guarantees a statistically valid sample size of randomly selected
    cases, and will hire or allocate quality assurance staff needed.

```{=html}
<!-- -->
```
-   Recognizing the limitations of its legacy automated system, KIDS
    (Key Information Delivery System), Arkansas designed and developed a
    new automated eligibility system called Keying in Day Care
    Accurately, Reliably, and Efficiently (KIDCare), to be fully
    operational as of July 2005. Arkansas designed KIDCare to determine
    eligibility based on program specific guidelines and has
    incorporated numerous edits to prevent inaccuracies from occurring
    on the front end of eligibility determination.

-   Illinois developed a resource guide for workers to outline
    acceptable forms of documentation or verification needed to
    determine eligibility accurately. The guide provides clarification
    for workers to use with clients during the eligibility process when
    the client cannot produce the required documentation. A worker can
    place a case in a pending status while awaiting necessary
    documentation.

-   As a direct result of the first pilot, Ohio began to retool its
    monitoring and technical assistance processes with the county
    agencies, implemented a quality control process for the child care
    program, and examined policy for possible revisions to strengthen
    and clarify procedures.

-   In response to the first pilot, Colorado planned to implement an
    automatic income calculation into Colorado's automated Child Care
    Tracking System. State staff also conducted training and feedback
    with all participating counties to address deficiencies identified
    in the pilot. Staff members then shared the results of the error
    rate analysis at the State child care conference and changed
    statutory language to simplify access to other State databases such
    as those for new hires and unemployment compensation. Colorado also
    examined State policies to determine where changes may be necessary
    to provide more consistent application, particularly in the area of
    self-employment.

The other goal of the error rate pilot was to test the error rate
methodology in different States to produce useful lessons learned
regarding management of the process to reduce State burden and improve
the accuracy of the review findings. Based on the experience of nine
pilot States, recommendations for replication or expansion of the error
rate process based on the pilot findings[^3], are:

-   *Create a State Project Team (Project Team) to coordinate the error
    rate methodology process.* Leadership of the Project Team should be
    assigned to a high level management staff member. The Project Team
    leader's responsibilities involve managing the entire process,
    including: recruiting additional Project Team members; establishing
    and adhering to timelines; customizing the *Record Review
    Worksheet*; training the reviewers; arranging Project Team meetings;
    answering questions and consulting with Project Team members as
    needed.

-   *Allow enough time to adequately complete the error rate
    methodology*. The pilots provided illustrative data on the amount of
    time needed to complete the methodology, but it should be remembered
    that these statistics are related to (1) a 150 case sample from a
    single sampling frame and (2) a project completed with considerable
    technical support, particularly in the sample selection and data
    analysis.

-   *Establish a State Review Team (Review Team) under the leadership of
    the Project Team.* While sample size and technical support clearly
    play a role in time estimates, the length of time for States to
    complete the error rate methodology is largely dependent on
    selecting and providing adequate staff resources to the actual
    record review component. Those pilot States with Review Team members
    who were experienced in any local operations or compliance
    monitoring function (Quality Control (QC), audit, technical
    assistance) accomplished the record review efficiently.

-   *Provide thorough training to the reviewers in State rules,
    policies, and procedures.* Even with staff experienced in conducting
    case record reviews, it should not be assumed that all members of
    the Review Team agree on the interpretation of policy and, as a
    result, "what is" or "is not" an error. The training needs to
    include actual case reviews. *It is also very important for training
    to emphasize the need for sufficient detail in Column 3 of the
    Record Review Worksheet so case information and error findings can
    be understood.*

-   *Provide formal training on standardizing error interpretation,
    descriptions and coding in the Record Review Worksheet* *to achieve
    inter-State reliability of data among future pilot States.* The
    quality of the information produced by the case record review
    process depends on the consistency of error definition and coding.
    Inconsistencies across pilot States in error interpretation
    dramatically influenced the outcomes. States in the second pilot
    were not consistent in their error coding. There were even instances
    where States had cases with similar findings, but with very
    dissimilar error results. Consistency of error definitions and
    coding has a direct impact on the results.

-   *Involve some form of re-review or joint review to enhance
    inter-rater accuracy, regardless of any existing review processes
    and the composition of the Review Team.* It is recommended that the
    re-reviews be drawn from a sub-sample of **all** cases and not
    limited to just cases with an improperly authorized payment.
    Oregon's re-review changed the review findings in three cases from
    an error to a correct case.

-   *Make careful up-front decisions about the error information desired
    at the conclusion of the review, to reduce the time needed for data
    analysis.* The pilot findings indicated that States' choices in
    customizing the *Record Review Worksheet* impact the timeline and
    costs of that portion of the error rate process. Additional data
    fields can be incorporated into the customized *Record Review
    Worksheet,* reducing the analysis time.

```{=html}
<!-- -->
```
-   *Develop consistent standardized policy and procedures as part of
    ongoing or routine auditing processes**. ***The State responses to
    the error findings in both pilots include numerous initiatives to
    improve the quality of case work and the frequency of reviews*.*
    Training to increase staff awareness of error-prone areas, knowledge
    of policy, interviewing skills, and the quality of routine case
    reviews are the most effective strategies States can use to prevent
    or reduce procedural or policy errors.

-   Focus *State review strategies on **both** the most common error
    types and those error causes which produce errors of greater
    amounts, such as ineligibility.* Both pilots' findings suggest that
    there is little, if any, correlation between States' percentage of
    cases in error, the percentage of cases in error that had an
    improperly authorized payment, and the percentage of improperly
    authorized payments. In comparison to New Jersey, Kansas had
    relatively high numbers of cases with errors and cases with an
    improperly authorized payment. Yet the Kansas percentage of
    improperly authorized payments was the lower of the two States. For
    these reasons, State improper payment strategies need to focus on
    both the most common error types and those error causes which
    produce errors of greater amounts, such as ineligibility.

-   *Make accommodations to implement the methodology independent of a
    pilot*[^4]*, such as:*

    -   *Work with IT departments or a local university for assistance
        on generating the universe and a random sample of cases for
        review*. Most IT departments have access to programming for
        random number generators for sample selection. When planning the
        error rate methodology, States need to decide on the frequency
        of sample selection and the definition of the universe of cases
        from which to select the sample. For example, how many times per
        year will the sample be drawn and what will be the sample size?

    -   *Regarding sample size, consult with a statistician to achieve
        the desired sample size*. For example, California worked with a
        local university to determine a sample size for calculating an
        error rate that met the IPIA guidelines[^5].

    -   *For data consolidation, designate a research analyst or program
        specialist who can create a database or an excel spreadsheet
        that includes at a minimum, all of the variables listed in the
        Data Entry Form and several error categories determined to be
        most important, such as the review element, missing or
        insufficient information, data entry errors, or calculation
        errors*. An *Excel* spreadsheet provides all the formulas needed
        to compute the error measures described in the pilot methodology
        that if combined with an electronic *Record Review Worksheet,*
        can aggregate individual review data automatically.

Combined with those of the first pilot, the second pilot error findings
are promising as a baseline test of the methodology. The pilot
methodology was also useful for States to gain evaluative information on
potential areas for improvement in administrative policy for the CCDF
block grant program.

[]{#__RefHeading___Toc167079280 .anchor}**I. INTRODUCTION**

Congress responded to the issue of improper payments in Federal programs
by enacting the Improper Payments Information Act (IPIA) of 2002. This
law requires Federal agencies to identify programs that are vulnerable
to improper payments and to estimate annually the amount of
underpayments and overpayments made by these programs.

The Executive Branch also has worked to address the improper payments
issue. A goal of the President's Management Agenda (PMA) is "eliminating
improper payments." To this end, agencies are to establish a baseline of
the extent of improper payments and to set goals to reduce improper
payments.

Because the PMA and IPIA provide for establishing a child care error
rate as one on-going measure of program efficiency, the Child Care
Bureau (CCB) launched the project: \"Measuring Improper Payments in the
Child Care Program\" to identify and describe methods that could help
States identify, measure, and prevent improper payments in the
administration of the Child Care and Development Fund (CCDF). A major
focus of the CCB project was to work in partnership with the States to
determine the feasibility of computing a child care improper payments
error rate.

During Fiscal Year 2004 of the project, the CCB worked with 11 partner
States[^6] to gather information and recommend methods to identify,
define, measure, and prevent improper payments. During FY 2005 of the
project, the CCB contracted with Walter R. McDonald & Associates, Inc.
(WRMA) to develop and pilot test in four States---Arkansas, Colorado,
Illinois and Ohio---a common methodology that States could use to
compute an error rate.

The purpose of this report is to summarize the findings from the second
pilot to further refine and develop a methodology to compute an error
rate in five States---Florida, Kansas, New Jersey, Oregon and West
Virginia. Following a brief overview of the Child Care Program, this
chapter provides the background on the pilot project activities to date
and the objectives of the current pilot.

[]{#__RefHeading___Toc167079281 .anchor}**A. Background**

The CCDF is a block grant that allotted over \$5 billion to States,
Territories and Tribes to support child care subsidies for low-income
working families during FY 2006.[^7] The CCDF block grant allows maximum
flexibility for States to set policies regarding eligibility and fiscal
management approaches, as well as define administrative structures that
allow maximum choice for parents. As a result, States\' eligibility
criteria, rates, regulation of child care providers, and payment
mechanisms vary widely among jurisdictions. This flexibility makes it
difficult to develop common approaches for identifying and measuring
improper payments.

In FY 2004, the CCB visited six States (Arkansas, Connecticut, Indiana,
Ohio, Oklahoma, and Virginia) and worked extensively with five
additional partner States (Georgia, Maryland, Oregon, South Carolina,
and Wisconsin). Together, the CCB and State partners gathered
information on the range of strategies that States use to prevent and
identify errors and appropriate enforcement actions taken when improper
payments occur. Results revealed that efforts varied extensively among
States. Few States had implemented systematic methodologies to estimate
the amount of improper payments.

In FY 2005, the CCB contracted with Walter R. McDonald & Associates,
Inc. (WRMA) to develop and pilot test a common methodology that States
could use to compute an error rate and work with States to validate
existing protocols or develop new approaches to address improper
payments and fraud. Specific requirements, including statistically valid
sampling, guided the conceptual design of this pilot. The CCB, in
consultation with a number of "partner States," chose to focus the
analysis on *eligibility error* in order to measure an element that is
common to every State and to mitigate some of the variation among State
definitions encountered during the first error rate pilot. The
methodology---conducted in Arkansas, Colorado, Illinois, and Ohio---had
four main components:

-   The contractor assisted each pilot State to select a random sample
    of up to 150 cases (children), using a sampling frame of all
    children in the State authorized to receive a child care payment[^8]
    during October 2004. The contractor designed the sample size to
    produce a statistically valid estimate of erroneous payments.

-   Pilot States customized a *Record Review Worksheet* template to
    reflect child care policies in the State. The States used this
    instrument to guide a record review of the sampled cases to identify
    administrative errors in eligibility determination. States collected
    data regarding the number of cases with errors and whether the
    errors led to an improperly authorized payment. Although all of the
    worksheets contained common elements, the definitions pertaining to
    those elements varied from State to State.

-   The contractor conducted site visits and provided technical
    assistance to the pilot State representatives who conducted record
    reviews and collected data. The contractor computed the error rates
    using the data submitted by the pilot States.

-   The contractor also conducted telephone interviews, using a
    consistent protocol to gather additional information about improper
    payment activities in five States. These States were Arizona,
    California, Kansas, Nebraska, and New Hampshire.

    []{#__RefHeading___Toc167079282 .anchor}**B. Objective of the Second
    Pilot**

The purpose of the FY 2006-2007 pilot was to further refine and develop
a methodology to compute an error rate in five States---Florida, Kansas,
New Jersey, Oregon, and West Virginia. Similar to the first error rate
pilot, the methodology of the current pilot focuses on client
eligibility and employs a case record review process to identify cases
with errors, cases with errors that result in improperly authorized
payments[^9], and percentages of authorized payments in error. However,
in the second pilot the methodology provided pilot States with their
results so they could analyze the types and sources of error. Following
an analysis of the findings from the case record review process, the
contractor forwarded the results to the pilot States for their internal
review. Pilot States responded to a short survey providing an
explanation for the causes of the errors and a description of next steps
or corrective actions to be taken as a result of participation in the
error rate pilot.

[]{#__RefHeading___Toc167079283 .anchor}**II. METHODOLOGY**

This chapter describes the various components of the second pilot
methodology and the process that the States used to measure improperly
authorized payments. The chapter begins by providing an overview of the
key components of the error rate methodology, followed by a detailed
description of how each was implemented with the five pilot States. The
final section of this chapter describes an additional area of inquiry in
the second pilot. Following the data analysis, the contractor shared the
findings with the five pilot States and asked the States to respond to
three questions about the causes of the errors, the role of automation
in reducing errors, and any corrective actions to be taken as a result
of the error rate pilot.

[]{#__RefHeading___Toc167079284 .anchor}**A. overview of the Process to
Measure Improper Payments**

The second pilot methodology for measuring improper authorizations for
payment focused on client eligibility and employed a case record review
process to determine whether child care subsidies were properly
authorized. This methodology deviated from the first pilot methodology
only in the addition of one element to the *Record Review Worksheet* and
formalizing attention to findings and intended actions of States by
requesting response to a *State Response Form.*

The methodology focused on improper authorizations for payment during
the eligibility and eligibility redetermination processes. Eligibility
and authorization are the first steps in the child care subsidy process
and errors made at this stage in the process are likely to affect the
administration of the entire program.

The process to measure improper payments in five pilot States consisted
of several phases, discussed in more detail within the remainder of this
chapter. They include:

-   Fieldwork preparation by States, including creation of State Project
    Teams, formal planning conference calls and site visits with each
    pilot State by the Federal Project Team;

-   Generation of a sample of 150 cases;

-   Customizing *Record Review Worksheets* by States to reflect
    State-specific policies and procedures;

-   State training of case record reviewers, case record reviews using
    customized *Record Review Worksheets*, and selected re-reviews of
    cases;

-   Completion of *Data Entry Forms;*

-   Computation of error rates and findings; and

-   Completion of *State Response Forms* to include anticipated changes
    in response to the error rates and findings.

As with the first pilot, a contractor performed several key functions
throughout these phases:

-   Facilitating planning conference calls with States;

-   Planning and facilitating site visits;

-   Receiving State sampling frames and selecting the statewide samples;

-   Providing ongoing technical assistance regarding customization of
    the *Record Review Worksheet*, conducting record reviews, completing
    *Data Entry Forms* and *State Response Forms; and*

-   Receiving States\' *Record Review Worksheets, Data Entry Forms,* and
    using them to compute error rates and analyze the findings.

    []{#__RefHeading___Toc167079285 .anchor}**B. Fieldwork Preparation**

To prepare for the second pilot, each of the five pilot States formed a
State Project Team and identified a project coordinator. Each State
Project Team and project coordinator identified who would be responsible
for planning for a one-day site visit with the Federal Project Team,
generating a universe of cases to send to the contractor, customizing a
*Record Review Worksheet*, forming and training members of the Record
Review Team, and assuring that completed *Data Entry Forms* were
submitted for analysis. Members of State Project Teams also agreed to
consider findings and determine next steps.

State Project Team members participated in Error Rate Pilot Study
overview conference calls. Each planning conference call began with a
brief overview of the error rate pilot followed by specific discussions
to select the most appropriate sampling plan, consider approaches to
customize that State\'s *Record Review Worksheet*, and schedule the
one-day site visit. To accomplish the fieldwork preparation and site
visits within a six month timeframe, States worked quickly with the
Federal Project Team to choose their sampling method, generate a
universe of cases, and schedule site visits.

[]{#__RefHeading___Toc167079286 .anchor}**C. Generating the Sample**

> **1. Pilot Sample Size**
>
> Similar to the first pilot, the primary unit of measurement for this
> pilot was a child authorized to receive a child care payment[^10]
> during a single month (October 2005 for the second pilot). This did
> not include children who were denied or terminated, or were billed but
> not served during October 2005.[^11] States would review a sample of
> 150 cases, a sample size designed to produce a statistically valid
> estimate of erroneous payments. The Improper Payment Information Act
> of 2002 recommends using a formula to determine the sample size for
> calculating error rates in federally funded programs, leading to an
> error rate with a 90 percent confidence interval of +/-2.5
> percent.[^12] A sample of 150 cases per State yields an estimate with
> approximately +/-6 percent confidence intervals with 90 percent
> confidence for each State, adequate for a pilot intended to establish
> feasibility and estimate the cost of the methodology.
>
> **2. Sampling Options**
>
> States chose from two basic sampling techniques---a single-stage
> random statewide sample and a two-stage random statewide sample.[^13]
> States selected the sampling method most appropriate for their
> circumstances. Florida, Kansas, and West Virginia elected to use the
> statewide random sample, while Oregon and New Jersey used the
> two-stage random sampling approach. Oregon and New Jersey chose the
> two-stage approach to minimize the number of counties selected for the
> pilot, thereby decreasing the number of counties from which cases
> would be drawn and reducing the amount of travel necessary to conduct
> record reviews in the counties, while still ensuring an equal chance
> of selection for any child in the population.
>
> **a. Creating a Sampling Frame and Child Identification Number**
>
> Each State created a sampling frame consisting of the universe of
> children that met the criteria for a child authorized to receive a
> child care payment during the month of October 2005. Each case was
> identified by:

(1) Sequential number;

(2) Child identification (ID); and

(3) County of service.

> The sequential number refers to the numerical order that the case
> appears on the list of cases comprising the sampling frame. Each State
> determined its own parameters for child IDs. Specifications for the
> child ID were:

-   Each child in the sampling frame received a unique ID. If there were
    several children within one family, case record, or household, each
    child had a unique ID.

-   The ID could contain both letters and numbers.

-   The unique ID did not contain identifying information; however, it
    was linked to a county or State data system so that the county or
    State could pull the record if the case was selected for the pilot.

> The identification of cases by county made it easier for States to
> pull the randomly selected sample cases for review.
>
> The contractor randomly selected a sample of 150 cases for each State.
> In addition, a 20-case supplemental sample was randomly selected as
> record replacements in Florida, Kansas, and West Virginia. Because New
> Jersey and Oregon chose to use a random two-stage sample, 30 records
> were needed for its supplemental random sample. More replacement cases
> were selected for the two-stage samples to assure enough replacements
> were available within the county sample, while 20 replacement cases
> across the State was deemed more than sufficient under the
> single-stage approach. States used the replacement records only if the
> case selected did not meet the pilot criteria for valid reasons.
>
> **b. Single-Stage Sample**
>
> The steps of the single-stage sampling method, used to select a random
> statewide sample for Florida, Kansas, and West Virginia, were:

-   Calculation of a sampling interval based on the size of the sampling
    frame or universe of cases authorized to receive a payment during
    October 2005. To do this, the total number of cases within the
    sampling frame (i.e., all cases authorized to receive a child care
    payment during October 2005) was divided by the number of cases
    selected for the sample (i.e., 150). (For example, if the sampling
    frame included 15,000 cases, the sampling interval was 15,000/150 or
    100---every 100^th^ case was selected.)

-   Consultation of a random number table to determine a random starting
    point within the sampling frame. From that starting point, cases
    within each sampling interval were selected, as described above.
    (For example, if the random start was seven and the sampling
    interval was 100, then the contractor selected the case listed
    seventh, the case listed 107^th^, the case listed 207^th^, and so
    on.) After randomly selecting one case for each sampling interval,
    the sample included 150 cases.

-   Forwarding of a list of the 150 sample cases---including sequential
    number, child ID, and county of service---to the designated State
    representative.

```{=html}
<!-- -->
```
-   Removal of the sample of 150 cases from the overall sampling frame
    and use of the steps described above to select this second sample of
    20 replacement cases.

> **c. Two-Stage Sample**
>
> The sampling plan for New Jersey and Oregon followed a two-stage
> approach, as follows:

-   Provision by New Jersey and Oregon of a sampling frame that listed
    all cases authorized to receive a child care payment during October
    2005, grouped by county. The sampling frame was sorted by county
    size (defined as the number of cases the county served during
    October 2005)---from largest to smallest counties.

```{=html}
<!-- -->
```
-   Selection of 15 counties from which to draw samples of 10 cases (15
    counties X 10 cases = sample of 150 cases). A first-stage sampling
    interval was determined by dividing the total number of cases
    represented in the sampling frame by 15. A random starting point
    within the sampling frame was determined by consulting a random
    number table. From that starting point, counties were selected
    within each sampling interval. (For example, if the total number of
    cases listed in the sampling frame was 30,000, the sampling interval
    would be 30,000/15 or 2,000. If the random start was seven, the
    contractor selected the county in which the case listed seventh
    lived, the county in which the case listed 2,007^th^ lived, the
    county in which the case listed 4,007^th^ lived, and so on.) Through
    this process, counties were selected from which subsamples of 10
    cases would be drawn during the second stage of the sampling
    approach. By this process, some large counties would be selected for
    at least one subsample of 10 cases with certainty and, in some
    instances, a county with a relatively large number of cases could be
    selected for two or three subsamples of 10 cases.[^14]

```{=html}
<!-- -->
```
-   Selection of a series of county-level random samples, as the second
    stage of the sampling approach. For each of the counties selected in
    the first stage, the contractor determined a unique sampling
    interval based on the number of cases served by the county during
    October 2005 and the number of subsamples that would be selected
    from the county. (For example, a large county may have been selected
    two times in the first stage, meaning that two subsamples of 10
    cases would be selected from this county. If the population of the
    county was 3,000 and two subsamples of 10 cases (20) were selected,
    the sampling interval for that county would be 3,000/20 or 150. If
    the random start was seven, the contractor selected the case listed
    seventh, the case listed 157^th^, and so on.) This process was
    repeated for every county selected in the first stage.

    []{#__RefHeading___Toc167079287 .anchor}**D. Customizing *Record
    Review Worksheets***

The CCDF block grant allows maximum flexibility for States to set
critical policies and procedures. Eligibility criteria, rates,
regulation of child care providers, and payment mechanisms vary widely
among jurisdictions. Because each State has developed its own child care
and data collection policies, it is important that all reviewers within
a State and all States participating in this pilot define and collect
data elements in a similar fashion to increase the validity of the pilot
findings.

To determine errors based on standards contained in current statutes and
applicable regulations, the contractor developed a *Record Review
Worksheet* template, which was a composite of existing review sheets
used by several States to conduct Child Care, Food Stamps, and Temporary
Assistance for Needy Families (TANF) reviews. States participating in
the first pilot agreed that it was a workable template to develop
State-specific worksheets.. The *Record Review Worksheet* template
captures the detail for each element of eligibility, the calculation of
and amount of authorized subsidy for the review month, and any resulting
errors (See Appendix F for a template of the *Record Review Worksheet*.)
States were encouraged to customize the template to reflect
State-specific policies and procedures.

The template worksheet consists of four columns:

-   Column 1 lists the basic elements of eligibility and resulting
    computations;

-   Column 2 contains the findings of the record review and notes any
    pertinent facts, questions, or conflicts in information;

-   Column 3 provides a space for the reviewer to note the ultimate
    findings of the analysis and a summary of any error; and

-   Column 4 contains error coding---in the event that a review element
    contains more than one error, the largest dollar error is the one
    coded.

Each pilot State customized a *Record Review Worksheet* to capture
client information (name, address, case number, client ID, and household
members), provider information, payment data, and error findings. Each
State customized the data elements within the *Record Review Worksheet*
to capture the analysis, investigation, and results of the individual
eligibility and payment elements. The most common modifications were
made in Column 1 and were designed to capture the State's specific
eligibility policy and procedures.

In the first pilot, Colorado reordered elements in Column 1 of the
worksheet to follow its workflow more efficiently. Arkansas used the
changes made by Colorado and added some content modifications to the
elements in Column 1. Ohio reformatted the template by adding an
additional column indicating the citation authority for each element.
Illinois created a "check-off" rather than a written summary of the
record material in Column 2. Another common alteration was to include a
section addressing the review findings of the subsidy payment process.
(See Appendices A--D in the first Error

Rate Study Final Report for each State's Record Review Worksheet:
Available at
<http://www.acf.dhhs.gov/programs/ccb/ta/ipi/phase2/phase2.pdf>.)

In the second pilot, States included additional descriptive information
in the first two columns of the worksheet to assist reviewers in
reliably identifying the required documentation to support the
eligibility element. West Virginia specified all applicable forms or
data screens required for each element including form numbers and
requirements. Florida modeled its worksheet after West Virginia,
specifying each applicable form required under elements of eligibility.
Kansas included additional prompts or check-offs for reviewers to
indicate the presence of applicable evidence. New Jersey and Oregon made
changes to the narrative elements in Column 1 of the *Record Review
Worksheet* to reference relevant State policies. See Appendices A--1 to
E-1 for each State's *Record Review Worksheet*.)

[]{#__RefHeading___Toc167079288 .anchor}**E. Site Visits**

Following the fieldwork preparation stage of the pilot, and after the
States had selected their pilot samples and developed their *Record
Review Worksheets*, the Federal Project Team visited each of the five
States. The Federal Project Team consisted of Federal Central Office
staff members, Regional Office staff representatives, and contractor
staff members. The purpose of the one-day site visit was to explain and
review the pilot methodology for the case record reviews with the State
Project Team. The site visits followed the basic agenda presented in
Appendix J. (See Appendices A--E for information obtained during the
site visits.)

During their site visits, Florida and New Jersey presented and discussed
each element of the *Record Review Worksheet* explaining how they would
interpret potential errors according to State policy and procedure. In
both New Jersey and Oregon, the site visit occurred before their *Record
Review Worksheet* had been finalized. Although local staff had met
several times previously to customize the worksheet, the site visit
provided an additional opportunity to reinforce the review methodology.
During the site visit, Kansas decided to scale back the scope of its
review and to modify its *Record Review Worksheet* accordingly. West
Virginia completed its case record review process prior to the Federal
Project Team site visit. West Virginia's site visit presentation
included a review of several examples of eligibility errors discovered
and lessons learned during the case record review process. In all
States, review team members attended a portion of the site visit and
described their planned process for reviewing the cases. Most teams
conducted a training following the site visit that would involve group
discussion and/or re-review of cases to assure inter-rater reliability.
This process allowed for greater consistency within the State regarding
error determination.

[]{#__RefHeading___Toc167079289 .anchor}**F. Conducting Record Reviews**

The purpose of the record review process is to ensure that all elements
are addressed during the case review and to document the findings as the
review is conducted. Reviewers need to be experienced with child care
program regulations and the quality control techniques and methodology
common to the review of other Federal programs, such as Medicaid and
Food Stamps. Training and reliability checks need to be conducted to
avoid inconsistencies in verification techniques and/or the
identification and calculation of errors.

Using the *Record Review Worksheet* as a guide, States conducted record
reviews of the random sample of 150 cases authorized to receive a child
care payment during October 2005. Florida, Kansas and Oregon conducted
the training of the reviewers and the case record review process
centrally. All three States also conducted a re-review process,
involving a second team member reviewing each case to increase
inter-rater reliability and consistency. New Jersey and West Virginia
conducted training centrally; however, the reviewers conducted the
record review process regionally at the local CCR&R agencies.

[]{#__RefHeading___Toc167079290 .anchor}**G. Completing the *Data Entry
Form***

State reviewers recorded the findings from each *Record Review
Worksheet* onto a *Data Entry Form*. (See Appendix G)

The *Data Entry Form* summarizes the composite results of the record
review documented on the *Record Review Worksheet*. Below is a list of
the information included on the *Data Entry Form*:

-   State;

-   County of service;

-   Child ID;

-   Pilot period month (October 2005);

-   Date of data collection;

-   One or more administrative errors during the pilot period;

-   Cause of improperly authorized payment;

-   Total amount of improperly authorized payment during review month;
    and

-   Total amount of authorized payment during review month.

For both pilots, States calculated and recorded the authorized payment
amount based on the eligibility determination process as indicated
within the case record. In both pilots the term "payment" refers to the
amount authorized for payment.

In the second pilot, two States combined the *Data Entry Form* with *the
Record Review Worksheet.* West Virginia and Florida combined the *Data
Entry Form* at the end of the *Record Review Worksheet*. Kansas and New
Jersey kept the *Data Entry Form* as a separate attachment. (See
Appendices A--1 to E-1 for each State's *Record Review Worksheet*.)
Combining the *Data Entry Form* with the *Record Review Worksheet*
reduces the number of forms used in the review of a case and assists in
the reliable transfer of information from the *Record Review Worksheet*
to the *Data Entry Form.* However, once the *Data Entry Forms* have been
completed and the task becomes the calculation of the five error
measures the combined *Record Review Worksheet* and *Data Entry Form*
can be a cumbersome document.

[]{#__RefHeading___Toc167079291 .anchor}**H. Computing Error Rates**

As a final review, the contractor matched the information contained in
the *Data Entry Forms* with the companion *Record Review Worksheet.*
Project Teams sent copies of all *Data Entry Forms* and each *Record
Review Worksheet* to the contractor. Using these data, the contractor
computed four error rate measures---the percentage of cases with an
error, the percentage of cases that have an improperly authorized
payment[^15], payment error rate, and average amount of improperly
authorized payment. (See Appendix G for the *Data Entry Form*.)

-   **Percentage of cases with an error**---This percentage is based on
    the number of cases with an error, regardless of whether it resulted
    in an improperly authorized payment or not, compared to the total
    number of cases in the sample. This percentage is determined by
    dividing the number of sampled cases with an error by the total
    number of cases reviewed in the sample and then multiplying by 100.

-   **Percentage of cases that have an improperly authorized
    payment---**This percentage is based on the number of cases with an
    error that resulted in an improperly authorized payment compared to
    the total number of cases in the sample. The percentage is
    determined by dividing the number of cases with an error that
    resulted in an improperly authorized payment by the total number of
    cases in the sample and then multiplying by 100.

-   **Payment Error Rate (Percentage of authorized payments in error for
    the review period)---**The payment error rate is the percentage of
    the gross amount of improperly authorized payments (overpayments
    plus underpayments) for the review period compared to the total
    amount of authorized payments in the sample. This rate is determined
    by dividing the gross amount of improperly authorized payments by
    the total dollar amount of authorized payments for the sample cases
    and then multiplying by 100.

-   **Average amount of improperly authorized payment**---The average
    amount of improperly authorized payments is the average amount of
    money the State authorized for payment improperly on a per child
    basis during the designated review month. This rate is determined by
    dividing the gross amount (overpayments plus underpayments) of
    improperly authorized payments in the sample by the number of cases
    with errors that had an improperly authorized payment.

    []{#__RefHeading___Toc167079292 .anchor}**I. *State Response Form***

An additional area of inquiry in the second pilot included sharing the
findings with the five pilot States following the data analysis. After
reviewing its error rate findings and site visit summary description,
each pilot State submitted a written response to the following three
topic areas included in the *State Response Form:*

-   Causes of the improperly authorized payments identified in the
    record review process;

-   Use of information systems that assist the State in identifying and
    reducing improperly authorized payments; and

-   Actions to be taken in order to reduce errors in the future.

Appendix I contains the *State Response Form*. Chapter IV contains a
summary of each State's responses to the *State Response Form* within
the section entitled Next Steps. Appendices A--E also contain summaries
of the next steps States are taking as indicated in their responses.

[]{#__RefHeading___Toc167079293 .anchor}**III. MEASURING IMPROPER
PAYMENTS IN FIVE STATES**

The contractor visited five States---Florida, Kansas, New Jersey,
Oregon, and West Virginia---in the second pilot to gather information
about the States' policies and practices regarding improper payments.
This chapter presents a summary of the processes, standards and
procedures, improper payment strategies, and automation that
characterize the five States in the second pilot and contrasts
similarities and differences with the four States involved in the first
pilot.

[]{#__RefHeading___Toc167079294 .anchor}**A. Improper Payments Process**

Similar to the four States in the first pilot, the five States in the
second pilot use many similar processes to identify, investigate, and
recover improper payments.

All nine States in both pilots identify improper payments in four
primary ways:

-   Notification of the child care agency of suspected fraud by
    providers or the public;

-   Exception reports and matches, produced on a regular basis from the
    State's automated child care tracking system, identify areas of
    potential improper payments;

-   Regular audits or reviews identify improper payments; and

-   Ad hoc studies or reviews identify improper payments.

An investigation typically begins after the State identifies a payment
problem. The staff persons responsible for calculating the overpayment
differ among the five States in the second pilot. In four States, local
staffs calculate the overpayment. In New Jersey, Florida, and West
Virginia, Child Care Resource and Referral (CCR&R) agencies calculate
the overpayment. The local Social Rehabilitation Services (SRS) offices
complete the overpayment calculations in Kansas. In Oregon local staff
refer the overpayment information to a State level central unit. Oregon
has a central State level Overpayment Writing Unit in the Department of
Human Resources that calculates the overpayments.

All nine States in both pilots follow similar processes to recover funds
for overpayments. The process proceeds in one of the following ways,
listed in order from least to most intensive action:

-   The State sends the person receiving the overpayment a notice
    describing the overpayment and the person\'s responsibility to
    resolve the debt;

-   The State establishes a repayment schedule, if possible. In some
    States if the client is an active recipient, a reduction of current
    benefits may occur;

-   The State initiates judgments, liens, and garnishments if the
    participant does not voluntarily repay; and

-   The State employs tax or lottery intercepts. This is a process where
    the State will intercept any tax refund or lottery winnings that
    would normally be paid to an individual and apply those monies
    toward the overpayment amount.

Once an overpayment amount is established, a dollar threshold generally
determines whether the State pursues a fraud determination in addition
to taking actions for repayment of the overpayment. The threshold varies
greatly among the States. For example, West Virginia forwards
overpayments of \$1,000 or more to the Office of the Inspector General
for collection and the establishment of penalties. The Kansas threshold
for referral to the local Fraud Units is also \$1,000, while in Oregon
the dollar threshold is \$200. Florida refers cases to the Office of the
Inspector General or the State's Department of Law Enforcement but the
referral criteria do not specify a dollar threshold. These differences
in the dollar threshold were also present in the States participating in
the first pilot, ranging from a \$200 threshold in Arkansas to a \$2,000
threshold in Arizona.

[]{#__RefHeading___Toc167079295 .anchor}**B. Standards, Processes, and
Procedures**

As was seen in the first pilot, the five States in the second pilot
indicate a trend toward establishing more formalized standards,
processes, and procedures regarding reducing improper payments. The five
second pilot States made the following procedural changes:

-   While Florida's eligibility policy stipulates that a case has, at a
    minimum, an annual recertification, the State now randomly selects
    50 percent of all cases for recertification at six month intervals.
    This interim recertification also requires a face-to-face interview.

-   Kansas has established a Quality Assurance Unit. The unit conducted
    two child care quality assurance reviews during 2006. Kansas
    contracted with a firm to develop software that tracks all case
    review findings and provides aggregate review data. Kansas utilized
    the software in July 2006 for the mandatory Food Stamp reviews and
    plans to include the child care reviews in the database in 2007.

-   New Jersey established a Quality Assurance in Child Care committee
    to develop policies and procedures to decrease the potential for
    improper payments. The committee includes both program and systems
    staff members. A recently created Program Integrity Unit uses data
    matching and shared data warehouse technology to identify improper
    payments.

-   Oregon completes a monthly desk audit of 200 randomly selected
    billing forms. A part of the State's Food Stamp payment accuracy
    strategy is a monthly review of approximately 5000 cases. Since
    roughly 94 percent of these cases are linked to child care cases,
    the review now includes child care eligibility.

-   West Virginia technical assistance staffs conduct random quarterly
    child care case audits. Prior to the pilot, the audits only targeted
    the automated system (FACTS) case record. The child care case audits
    now include a review of the physical case record. CCR&R agencies
    must develop and implement Quality Assurance Plans. Those plans now
    require payment accuracy initiatives if a CCR&R agency has an error
    rate that exceeds the State average.

    []{#__RefHeading___Toc167079296 .anchor}**C. Automation**

This section provides a description of automated processes and systems
the five States in the current pilot use to reduce improper payments.

All of the States that participated in both pilots have a statewide
automated system. The automated systems in the five States from the
second pilot differ considerably in function, interfaces, and age. Even
though New Jersey and Oregon are locally-administered States, those
States mandate that providers use the statewide system to determine
eligibility for the child care voucher application.

The Florida Child Care Enhanced Field System (EFS) is a Windows based
system that has an eligibility determination module, generates payments,
manages waiting lists, and collects data for Federal reports and the
market rate survey. The system does not interface with other benefits
programs. EFS produces a selection of management reports to identify
improper payments. There is ad hoc report writing capability at the
local level. Sharing of the local report formats occurs throughout the
State through an e-mail Q & A system. An area of concern for the EFS and
the quality of its data is its decentralized structure. Each of the 31
Early Learning Coalitions has an independent section and differing
processes. The result is data that is inconsistent, incompatible, and
centrally unavailable. The proposed enhanced system Early Learning
Information System (ELIS) will address the weaknesses of this
decentralized system.

The Kansas KSCARES system, a transferred mainframe system, has been
operational since 1994. KSCARES determines eligibility, authorizes and
tracks payments for the Child Care Subsidy program and processes TANF
Work Program referrals and payments for employment related activities
and support services. KSCARES interfaces with a number of other State
systems including KAECSES (TANF and Food Stamp eligibility and Child
Support Enforcement), the Electronic Benefits Transfer (EBT) system, and
the Kansas Department of Health and Environment for access to child care
provider data. Planning and development are currently underway to build
a new integrated enterprise wide web based system that would incorporate
the functionality of KSCARES and several other agency systems. This
multi-system interface will strengthen eligibility processes by sharing
data across programs.

The New Jersey automated system occupies three different databases. Case
information for Work First New Jersey (WFNJ/TANF) resides on the OMEGA
(TANF) system. The Care for Kids (NJCK) case information resides in the
Child Care Automated Resources Eligibility System (CARES). The voucher
payments system and case information on clients receiving Center Based
Services (CBC) resides on the Contracts System (CTRX) and Contracts
Accounting Tracking System (CATS). None of the systems interface. CARES
includes intake, eligibility determination, agreement establishment,
subsidy and co-pay calculation, voucher generation, attendance input,
check write process, accounting procedures, letters, notices, forms, and
reports. New Jersey's Consolidated Assistance and Support System (CASS)
is a re-engineering effort designed to integrate all of the Division of
Family Development's support systems by utilizing an Enterprise
Architecture and Framework Approach. This future approach will increase
New Jersey's ability to share data across programs, eliminating the need
to do data matching.

The Oregon Client Maintenance System (CMS) is one of thirty-seven
components of a 28 year old legacy system. CMS bundles child care
eligibility with TANF and Medicaid and interfaces with the Service
Authorization System, the State's payment system. The child care
application process includes a match with other DHS systems as well as
child support, Employment Department wage and unemployment records and
the Social Security Benefit system. CMS calculates the provider payment
based on the family's co-pay, the provider's usual charge, hours
authorized and billed, age of the child(ren), type of care, and
provider's zip code. This capability remedies many of the payment
accuracy problems associated with fee schedule and payments.

West Virginia transferred the Family and Child Tracking System (FACTS)
from Oklahoma. FACTS is a customized statewide automated case management
system for all Child Welfare, Child Care, and Adult Service Programs. In
August 2000, the State modified FACTS to include case management and
payment functionality for the Child Care Program. This change fully
implemented the transition of the contracted CCR&R agencies to an
automated system for on-line and real time determination of eligibility
for child care services and an automated determination for child care
payments. The FACTS System is designed to take information entered in
the system by the CCR&R staff and use it to determine eligibility. This
eliminates many errors with the exception of those caused by faulty data
entry. The system has payment controls of eligible days and children
only.

FACTS also provides the capability of accessing the State's TANF, Food
Stamp and Medicaid system (RAPIDS). This allows additional verification
of information provided in the eligibility process.

Appendix K provides an extensive summary table for all nine States who
participated in the two phases of the error rate pilot. As was the case
in the first pilot, the five States in the second pilot were a mixture
of State administered (Florida, Kansas and West Virginia) and County
administered but State supervised (New Jersey, Oregon). Four States in
each pilot defined an improper payment in either State law,
administrative policy or both. Florida and Ohio were the only States
where fraud was not articulated in State policy. All nine States follow
similar processes to identify and recover overpayments funds. There is
considerable uniformity in the approaches all nine States take in
assessment of risk. Some form of audit, ad hoc review, compliance review
or site visit exists in all nine states. Case reviews, either
independent or in conjunction with other programs (Food Stamps), are
common. Two of the nine States included technical support and training
as major payment accuracy risk mitigators. Although all of the States
that participated in the pilots have a statewide automated system, the
systems differ considerably in function, interfaces, and age. All of the
States have incorporated in their payment accuracy strategies various
automated tools (edits, benchmark/exception/anomilie reports, data
mining).

[]{#__RefHeading___Toc167079297 .anchor}**IV. FINDINGS AND STATES\'
RESPONSES**

This chapter reviews findings from the record review process and error
rate analysis for the five pilot States---Florida, Kansas, New Jersey,
Oregon, and West Virginia. It also includes actions of States, or steps
they plan to take, in response to the causes of errors. The review of
findings for each State includes a brief description of the record
review process followed by a presentation of the error rate findings.
The findings include four measures computed for each State as follows:

-   **Percentage of cases with an error**---This percentage is based on
    the number of sampled cases with an error, regardless of whether it
    resulted in an improperly authorized payment or not, compared to the
    total number of cases in the sample. This percentage is determined
    by dividing the number of sampled cases with an error by the total
    number of cases reviewed in the sample and then multiplying by 100.

-   **Percentage of cases that have an improperly authorized
    payment**[^16]**---** This percentage is based on the number of
    sampled cases with errors that have an improperly authorized
    payment, compared to the total number of cases in the sample. The
    percentage is determined by dividing the number of cases with an
    error that resulted in an improperly authorized payment by the total
    number of cases in the sample and then multiplying by 100.

-   **Payment Error Rate (Percentage of authorized payments in error for
    the review period)---**The payment error rate is the percentage of
    the gross amount of improperly authorized payments (overpayments
    plus underpayments) for the review period compared to the total
    amount of authorized payments in the sample. This rate is determined
    by dividing the gross amount of improperly authorized payments by
    the total dollar amount of authorized payments in the sample cases
    and then multiplying by 100.

-   **Average amount of improperly authorized payment**---The average
    amount of improperly authorized payments is the average amount of
    money the State authorized for payment improperly on a per child
    basis during the designated review month. This rate is determined by
    dividing the gross amount (overpayments plus underpayments) of
    improperly authorized payments in the sample by the number of cases
    that had an improperly authorized payment.

Following the discussion of the error rate findings, the chapter
outlines State explanations of the error causes and summarizes the
corrective actions to be taken as a result of the error rate findings.
The source of this information is State responses to the State Response
Form included as Appendix I and discussed within the *Next Steps*
section of each State's findings.

[]{#__RefHeading___Toc167079298 .anchor}**A. Florida**

In Florida, the Review Team consisted of two Managers, two Performance
Analysts, and three Program Analysts. The Project Team made substantial
modifications to the *Record Review Worksheet* template. The review
process occurred centrally, with all records sent to Tallahassee. A
second member of the team re-reviewed all records for inter-rater
reliability to ensure that findings and coding in Columns 3 and 4 of the
*Record Review Worksheet* would be the same regardless of reviewer.

Following completion of the review, Florida learned from a finding in a
IV-E Federal audit that only the State's auditor general or Department
of Children and Family's (DCF's) Family Safety Unit could access case
information related to any protective services placement. The review had
found six cases in error due to missing protective services
documentation. Florida submitted amended review sheets on these six
cases, changing the finding from ineligible to correct.

[]{#__RefHeading___Toc167079299 .anchor}**1. Results**

> As shown in Exhibit 3, the data indicated that 53 records (35% of the
> sample) had one or more administrative errors and all 53 cases had
> administrative errors that resulted in an improperly authorized
> payment. The total of authorized payments to the 150 sampled cases for
> the review month was \$37,571 of which \$6,908 was the result of
> error. This error amount represents 18 percent of all authorized
> payments for the sample.

[]{#__RefHeading___Toc167078076 .anchor}**Exhibit 1**. Florida Findings
(N=150)

  ------------------------------------------------------- ---------------
  Percentage of cases with errors                         **35%**

  Percentage of cases that have an improperly authorized  **35%**
  payment                                                 

  Percentage of improperly authorized payments            **18%**

  Average amount of improperly authorized payment         **\$130.35**
  ------------------------------------------------------- ---------------

> Missing documentation as an error cause accounted for the largest
> percentage (36%) of the total dollars in error in the Florida sample.
> All of the errors attributed to missing documentation resulted in
> ineligibility. Missing documentation occurred in a wide range of
> eligibility elements, including Application Forms (Element 100),
> Income (earned and unearned), work activity, TANF, transitional TANF,
> and age.
>
> Twenty-two of the 53 total errors occurred in Payments (Element 420);
> however, these errors resulted in a much smaller percentage of total
> dollars in error (11%). Reasons for Payments errors included the
> incorrect use of the fee/rate schedule or the authorization of
> full-time payments for part-time attendance. The ten errors in
> Application Forms represented 36 percent of the total dollars in
> error. The five errors in Household Members accounted for 19 percent
> of the total dollars in error. An additional 9 errors occurred in
> Earned Income (Element 400), which accounted for 18 percent of the
> total dollars in error. The remaining 7 errors were dispersed among
> several review elements, such as Residency, Qualifying Child and
> Qualifying Care.

[]{#__RefHeading___Toc167079301 .anchor}**2. Florida Responses**

> This section identifies actions taken by the Project Team in response
> to the causes and corrective actions taken based on the findings of
> the pilot:

-   Developed a desk reference tool for eligibility procedures;

-   Initiated focused monitoring and training on eligibility procedures
    on an annual basis;

-   Provided each Early Learning Coalition (ELC) with its individual
    results of the pilot review process along with recommendations for
    corrective action;

-   Created mechanisms to track the results of the eligibility
    monitoring and other contract monitoring audits;

-   Implemented an annual eligibility review process of all 31 ELCs;

-   Implemented a Data Quality Initiative to standardize data collection
    and to identify data anomalies throughout the State; and

-   Implemented procedures to conduct post audits on reimbursement
    requests.

    []{#__RefHeading___Toc167079302 .anchor}**B. Kansas**

The Kansas Review Team consisted of three Quality Assurance Supervisors
(Topeka, Wichita, Kansas City) and three Quality Assurance Specialists.
The Review Team also conducted a re-review of all cases by having a
second team member conduct the review. An error summary was sent to the
related local Social Rehabilitation Services (SRS) office whenever an
error was found. The local SRS office corrected the case based on the
review information. The Review Team reviewed all records centrally in
Topeka. The Project Team modified the *Record Review Worksheet* to
include "Yes" or "No" questions to reduce the amount of written
narration for the reviewers.

[]{#__RefHeading___Toc167079303 .anchor}**1. Results**

> As shown in Exhibit 4, the data show that 52 (35%) of the 150 sample
> records had one or more errors. All of the 52 cases with errors
> resulted in an improperly authorized payment. The total authorized
> payments to the sampled cases for the review month was \$40,623, of
> which \$4,650 was the result of error. This error amount represents 11
> percent of all authorized payments for the sample.

[]{#__RefHeading___Toc167078080 .anchor}**Exhibit 2**. Kansas Findings
(N=150)

  --------------------------------------------------------- -------------
  Percentage of cases with errors                           **35%**

  Percentage of cases that have an improperly authorized    **35%**
  payment                                                   

  Percentage of improperly authorized payments              **11%**

  Average amount of authorized improper payment             **\$89.42**
  --------------------------------------------------------- -------------

> While 92 percent of the total dollars in error were in Hours of Care
> (Element 340), the causes varied. The most common error causes
> included:

-   Failure to allow or the miscalculation of time for meals and travel;

-   Failure to reconcile the hours of care needed with the hours
    verified in the employment documentation or the school schedule; and

-   Allowing, without proper documentation, hours that exceeded need to
    satisfy a provider minimum.

> There were several cases where the reviewer was unable to attribute an
> error to a specific cause because the case documentation was
> nonexistent.

[]{#__RefHeading___Toc167079305 .anchor}**2. Kansas Responses**

> This section identifies actions taken by Kansas in response to the
> causes and corrective actions taken based on the findings of the
> pilot:

-   Prior to the pilot, Kansas had completed child care quality control
    (QC) reviews for the month of October 2005, the second month of
    statewide payments using the new child care EBT payment system.
    Kansas was already aware of the error causes found in the pilot and
    had conducted statewide refresher training in June 2006 to address
    these errors. The training focused on hours of care, use of child
    care plan hour worksheets, computation of income, use of income
    worksheets to document computations, and general case documentation.

```{=html}
<!-- -->
```
-   Kansas continues to complete monthly Child Care QC reviews and uses
    the results of these reviews to inform training needs. Kansas is
    updating the Child Care Personnel Trainer and Training Academy to
    emphasize case documentation and computation of hours of care needed
    and income. Kansas requires this training for all new workers and
    for others as needed. Supervisors now complete monthly case reviews
    for the Child Care Assistance Program.

-   Kansas hired a child care trainer in July 2006 to revise training
    materials and conduct several training sessions for eligibility
    staff. The QC unit compiles results monthly for regional
    administrators to keep them informed about findings and areas
    needing improvement. Supervisory case reviews at the regional level
    now include child care cases.

-   Kansas contracted with a firm to develop software to track all case
    review findings and provide aggregate review data. The web based
    system tracks aggregate case review data for the mandatory Food
    Stamp reviews and the child care reviews. The child care reviews
    began in July 2006. The software costs \$75,000 with an 18 percent
    maintenance costs agreement.

    []{#__RefHeading___Toc167079306 .anchor}**C. New Jersey**

After receipt of technical assistance during the site visit, New
Jersey\'s Project Team substantially customized the *Record Review
Worksheet* template to include coding and specific boilerplate language
to guide the reviewer's written case narrative. The Project Team
submitted a universe to the contractor to select a statewide sample that
included only cases using a child care voucher and appearing in the
Child Care Automated Resources Eligibility System (CARES). This universe
did not include contracted child care slots because these cases did not
meet pilot requirements for the unit of analysis of an authorized
payment on a per child basis utilizing the voucher system. The Project
Team trained its Review Team centrally and then reviewers conducted the
reviews in their assigned regions at the local Child Care Resource and
Referrals (CCR&R) agencies. The Review Team consisted of three
supervisors and 13 Child Care Specialists.

[]{#__RefHeading___Toc167079307 .anchor}**1. Results**

> As shown in Exhibit 5, the data show that 28 (19%) of the 150 sample
> records had one or more errors. All of the 28 cases with errors had an
> improperly authorized payment. The total authorized payments for the
> sampled cases for the review month was \$45,807, of which \$6,043 was
> considered to be in error. This error amount represents 13 percent of
> all authorized payments for the sample.

[]{#__RefHeading___Toc167078084 .anchor}**Exhibit 3**. New Jersey
Findings (N=150)

  --------------------------------------------------------- --------------
  Percentage of cases with errors                           **19%**

  Percentage of cases that have an improperly authorized    **19%**
  payment                                                   

  Percentage of improperly authorized payments              **13%**

  Average amount of improperly authorized payment           **\$215.82**
  --------------------------------------------------------- --------------

> Calculation of Household Income (Element 400) accounted for the
> largest percentage (56%) of the total dollars in error. Errors in
> Application Forms (Element 100) resulted in an additional 27 percent
> of the total dollars paid in error. The most common errors included:

-   Authorizing a payment for hours that exceeded the documented need
    (half-time vs. full-time and full-time for school-aged children);

-   Using an incorrect household size;

-   Allowing income without verification;

-   Ignoring income from a second job;

-   Failing to react to a reported income change;

-   Applying the fee schedule incorrectly; and

-   Paying for days of non-attendance.

    []{#__RefHeading___Toc167079309 .anchor}**2. New Jersey Responses**

> This section identifies actions taken by New Jersey in response to the
> causes and corrective actions taken based on the findings of the
> pilot:

-   Allocate time during meetings with the Child Care Resource and
    Referral (CCR&R) Directors meeting to provide feedback and direction
    on implementing corrective measures.

-   Institute the Error Rate Pilot monitoring tool for use of all future
    file reviews, in addition to increasing the number of files for
    review.

-   Provide technical assistance to CCR&R agency staff in those areas
    where improperly authorized payments were detected in file reviews.

-   Improve lines of communication between Division of Youth and Family
    Services (DYFS), the lead State child protective service agency and
    the Division of Family Development, the lead State child care agency
    and the CCR&R agencies at State and local levels.

-   Look into implementing changes in the CARES system to better detect
    data entry errors or create error messages when data entry errors
    occur.

-   Implement a process to conduct electronic matching of the automated
    child care client database with records on other wage, SSI, and
    child support information systems to identify and reduce the number
    of improperly authorized payments.

```{=html}
<!-- -->
```
-   Hire quality assurance staff or allocate the FTE needed to enable
    the Division to conduct file reviews, using a methodology that
    guarantees a statistically valid sample size of randomly selected
    cases.

-   Link reduced improperly authorized payments to penalties in Child
    Care Resource and Referral contracts.

    []{#__RefHeading___Toc167079310 .anchor}**D. Oregon**

Oregon\'s Project Team made very few modifications to the *Record Review
Worksheet* template. The Review Team consisted of five Quality Control
(QC) Staff Persons who have responsibility for TANF and Food Stamp case
reviews. Following training on child care policy and procedures, the
Review Team reviewed the cases centrally. The Project Team Leader
reviewed all cases with errors a second time and discovered that the
Review Team went beyond the level of a desk audit, expecting greater
detail in documentation than what had been anticipated or required of
the child care review process. As a result, the Review Team reviewed a
subset of cases a third time and changed three review results.

[]{#__RefHeading___Toc167079311 .anchor}**1. Results**

> As shown in Exhibit 6, the data show that 40 (27%) of the 150 sample
> cases had one or more errors; however, only 16 of the cases with
> errors resulted in an improperly authorized payment. The total
> authorized payment for the sampled cases for the review month was
> \$81,757, of which \$1,925 was considered to be in error. This error
> amount represents 2 percent of all authorized payments for the sample.

[]{#__RefHeading___Toc167078088 .anchor}**Exhibit 4. Oregon Findings
(N=150)**

  --------------------------------------------------------- --------------
  Percentage of cases with errors                           **27%**

  Percentage of cases that have an improperly authorized    **11%**
  payment                                                   

  Percentage of improperly authorized payments              **2%**

  Average amount of authorized improper payment             **\$120.33**
  --------------------------------------------------------- --------------

> The 24 cases with errors with no improperly authorized payment
> involved discrepancies such as missing applications or income
> verification, miscalculation of hours, and failure to document need
> for care. Fifty-six percent of the total dollars in error occurred in
> the area of calculation of household Income (Element 400). A general
> summary of the error causes includes:

-   Use of unearned income that had ended;

-   Failure to react when the client reported a job change;

-   Use of outdated pay stubs;

-   Failure to include regular overtime pay in the income calculations;
    and

-   Improper treatment of biweekly vs. bimonthly income.

> The remainder of the dollars in error occurred in Qualifying Care
> (Element 340). The causes of these errors were the miscalculation of
> hours of care and either an understatement or overstatement of the
> hours needed.

[]{#__RefHeading___Toc167079313 .anchor}**2. Oregon Responses**

> This section identifies actions taken by Oregon in response to the
> causes and corrective actions taken based on the findings of the
> pilot:

-   Oregon plans to modify the Food Stamp targeted review process and
    database that contains review information to ensure that income
    errors identified in any Food Stamp case are corrected in the
    companion ERDC case.

-   Each year, the Oregon Department of Human Services (DHS) conducts in
    all areas of the State a series of "Accuracy Summits" that focus on
    techniques to reduce errors in the Food Stamp program. DHS plans to
    include workshops to improve ERDC accuracy in the next series of
    summits that begins in July 2007.

-   DHS produces a monthly newsletter that focuses on techniques to
    reduce Food Stamp errors. The newsletter will now include
    information about other programs, including ERDC. A recent
    newsletter included an article about correctly determining co-pay
    amounts in ERDC cases. Future newsletters will contain information
    about ERDC payment accuracy, including an article about the results
    of the pilot review.

-   The Governor's recommended budget for the next biennium would allow
    DHS to extend Targeted Review outcome measures to the Child Care
    programs in local offices by FY2008. This comprehensive Program
    Integrity infrastructure provides a method for gathering timely,
    local performance data and will enable the Department to develop
    corrective action measures.

    []{#__RefHeading___Toc167079314 .anchor}**E. West Virginia**

West Virginia\'s Project Team made substantial modifications to the
*Record Review Worksheet*, using check-off boxes, "Yes" and "No" fields,
and coding to eliminate the need for extensive narrative recording. The
Review Team consisted of two Field Consultants and a Policy Specialist.
All of the reviews occurred on site at the regional CCR&R agencies. The
Review Team had already read 10 cases during one of their regular CCR&R
agency audits. To improve the review process, the three reviewers
completed several reviews jointly and consulted by telephone frequently.
Because the Review Team reviewed the cases in regions, travel time was a
considerable factor with several hours of driving time from one CCR&R to
another.

[]{#__RefHeading___Toc167079315 .anchor}**1. Results**

> As can be seen in Exhibit 5, West Virginia found 33 (22%) of the 150
> sample records to have one or more errors; however, only 20 (14%) of
> the cases with errors resulted in an improperly authorized payment.
> The total authorized payment to the sample cases for the review month
> was \$34,506, of which \$2,522 was found to be in error. This error
> amount represents 7 percent of all authorized payments for the sample.

[]{#__RefHeading___Toc167078092 .anchor}**Exhibit 5**. West Virginia
Findings (N=150)

+-----------------------------------------------------+----------------+
| Percentage of cases with errors                     | > **22%**      |
+-----------------------------------------------------+----------------+
| Percentage of cases that have an improperly         | > **14%**      |
| authorized payment                                  |                |
+-----------------------------------------------------+----------------+
| Percentage of improperly authorized payments        | > **7%**       |
+-----------------------------------------------------+----------------+
| Average amount of improperly authorized payment     | > **\$126.14** |
+-----------------------------------------------------+----------------+

> Thirteen of the cases with errors did not result in an improperly
> authorized payment. These case errors involved discrepancies such as
> failure to enter mandatory information into the Family and Child
> Tracking System (FACTS), computation errors that did not result in a
> fee change, and missing documentation. Errors in Application Forms
> (Element 100) represented 35 percent of total dollars in error.
> Household Members (Element 320) accounted for 28 percent of the total
> dollars in error. Earned Income (Element 400) and Income Eligibility
> (Element 410) represented a combined 19 percent of the total dollars
> in error.
>
> A general summary of the error causes includes:

-   Signatures on attendance sheets;

-   Missing parent agreements;

```{=html}
<!-- -->
```
-   Failure to enter the work schedule into the FACTS system.

```{=html}
<!-- -->
```
-   Failure to include documented earned and unearned income;

-   Use of an incorrect conversion factor (4 vs. 4.3, weekly vs.
    biweekly, etc.);

-   Use of net rather than gross pay; and

-   Failure to reconcile the hours of care allowed with the hours
    verified in the employment documentation or a school schedule.

    []{#__RefHeading___Toc167079317 .anchor}**2. West Virginia
    Responses**

> This section includes a summary of the West Virginia response to the
> error causes and corrective actions taken based on the findings of the
> pilot:

-   West Virginia will conduct refresher policy training with the CCR&R
    directors. The directors will then provide the training to their
    staff. West Virginia feels that, by training the directors rather
    than the staff, the directors will become more involved in
    eligibility activities.

```{=html}
<!-- -->
```
-   Technical Assistance staff will continue to perform random quarterly
    case audits. Prior to the pilot, the FACTS electronic record was the
    object of the review. An expanded review will now also include the
    physical case record.

-   CCR&R agencies must now develop and implement Quality Assurance
    Plans. The plans will now require inclusion of payment accuracy
    initiatives if a CCR&R agency has an error rate that is above the
    State mean.

-   Family child care providers must submit sign-in and sign-out sheets
    to verify attendance along with their billing forms. CCR&R agencies
    must audit billing forms and compare work and school schedules to
    times shown on the sheets to verify that the child care usage
    complies with time approved.

-   Child care providers who submit incorrect or improper billing forms
    must attend a retraining session on proper billing procedures.

-   Two State level child care consultants will continue to sample CCR&R
    agency cases to ensure compliance with appropriate policies and
    provide training and technical assistance on policy, procedures, and
    the use of FACTS. The consultants also will provide follow-up to
    ensure the CCR&R agency corrective action plans are completed.

    []{#__RefHeading___Toc167079318 .anchor}**F. Conclusion**

This section provides a summary of the findings and conclusions based on
States' responses to the questions about causes of errors and corrective
actions. The summary also includes a discussion of the similarities and
differences between findings from the four States in the initial pilot
and the five States in the second pilot.

This was the second pilot in an exploratory effort to develop and pilot
a methodology that States could use to compute an error rate. Similar to
the first pilot conducted in Arkansas, Colorado, Illinois, and Ohio,
this pilot yielded extremely useful findings.

[]{#__RefHeading___Toc167079319 .anchor}**1. Summary of Results**

> Exhibit 6 presents a summary of the findings for the five States
> involved in the second pilot and for the four States involved in the
> first pilot. The results from the second pilot demonstrate a very
> similar range to the findings in the first pilot. As can be seen in
> the summary table below, the estimated percentage of improperly
> authorized payments in the five pilot States ranged from a low of 2
> percent to a high of 18 percent. These results are very similar to the
> percentage of improperly authorized payments for the four States in
> the first pilot, which ranged from 4 percent to 20 percent.

[]{#__RefHeading___Toc167078096 .anchor}**Exhibit 6. Summary of
Findings**

Second Pilot

  ------------------ ----------- ----------- ------------ ----------- -----------
  **Measure**        **Florida   **Kansas    **New Jersey **Oregon    **West VA
                     (N=150)**   (N=150)**   (N=150)**    (N=150)**   (N=150)**

                                                                      

  Percentage of      35%         35%         19%          27%         22%
  cases with errors                                                   

  Percentage of      35%         35%         19%          11%         14%
  cases that have an                                                  
  improperly                                                          
  authorized payment                                                  

  Percentage of      18%         11%         13%          2%          7%
  improperly                                                          
  authorized                                                          
  payments                                                            

  Average amount of  \$130.35    \$89.42     \$215.82     \$120.33    \$126.14
  improperly                                                          
  authorized payment                                                  
  ------------------ ----------- ----------- ------------ ----------- -----------

First Pilot

+-------------------------------+--------+---------+-------+---------+
| **Measure**                   | **Ar   | **I     | *     | **Col   |
|                               | kansas | llinois | *Ohio | orado** |
|                               | (N=    | (N      | (N=1  |         |
|                               | 150)** | =150)** | 23)** | **(N    |
|                               |        |         |       | =150)** |
+-------------------------------+--------+---------+-------+---------+
| Percentage of cases with      | 12%    | 24%     | 44%   | 35%     |
| errors                        |        |         |       |         |
+-------------------------------+--------+---------+-------+---------+
| Percentage of cases that have | 12%    | 6%      | 32%   | 25%     |
| an improperly authorized      |        |         |       |         |
| payment                       |        |         |       |         |
+-------------------------------+--------+---------+-------+---------+
| Percentage of improperly      | 14%    | 4%      | 20%   | 8%      |
| authorized payments           |        |         |       |         |
+-------------------------------+--------+---------+-------+---------+
| Average amount of improperly  | \$     | \       | \$1   | \$88.99 |
| authorized payment            | 289.53 | $150.64 | 94.28 |         |
+-------------------------------+--------+---------+-------+---------+

> For the estimated percentage of sampled cases determined to contain
> administrative errors related to eligibility, the five pilot States\'
> percentages ranged from 19 percent to 35 percent. The four States in
> the first pilot had a wider range, with the percentages of cases with
> errors ranging from 12 percent to 44 percent.
>
> For the percentage of sampled cases with administrative errors that
> resulted in an improperly authorized payment, the five pilot States
> percentages ranged from 11 percent to 35 percent. The four States in
> the first pilot had a slightly lower range with the percentage of
> cases with an improperly authorized payment ranging from 6 percent to
> 32 percent.
>
> The average amount of an improperly authorized payment in the five
> pilot States ranged from a low of \$89.42 to a high of \$215.82. The
> four States in the first pilot had a slightly higher range with the
> average amount of an improperly authorized payment ranging from
> \$88.99 to \$289.53.
>
> Both pilots' findings suggest that there is little, if any
> correlation, between States' percentage of cases in error, the
> percentage of cases in error that had an improperly authorized
> payment, and the percentage of improperly authorized payments. In
> comparison to New Jersey, Kansas had relatively high numbers of cases
> with errors and cases with an improperly authorized payment. Yet the
> Kansas percentage of improperly authorized payments was the lower of
> the two States. In the first pilot, Arkansas had a percentage of
> improperly authorized payments that exceeded both its percentage of
> cases with errors and percentage of cases with errors that had
> improperly authorized payments. State improper payment strategies need
> to focus on both the most common error types and those error causes
> which produce errors of greater amounts, such as ineligibility.
>
> A major difference in the findings of the two pilots is the ratio of
> the percentage of cases with errors and the percentage of cases with
> errors that resulted in an improperly authorized payment. Of the four
> States in the first pilot, only Arkansas found all cases with errors
> to have an improperly authorized payment, whereas in the second pilot,
> three of the five States--Florida, Kansas, and New Jersey--found all
> cases with errors to have an improperly authorized payment. This
> finding should not be interpreted as an indication of the quality of
> the case work in a specific State, but rather as a product of the
> variances in the State eligibility policies. More importantly, it has
> implications for the methodology of conducting a consistent review
> process across States.

[]{#__RefHeading___Toc167079321 .anchor}**2.** **Most Common Errors**

> Exhibit 7 presents the most common errors within various eligibility
> elements, ordered from highest to lowest, for the five pilot States.
> Qualifying Care (Element 340), Income (Element 400), Payments (Element
> 420), Application Forms (Element 100) and Household Members (Element
> 320) were the five most common errors representing 91 percent of all
> errors. The remaining 9 percent (16) of the errors occurred in Other
> review elements, such as: Residency, Qualifying Child and
> Computations. All five pilot States experienced errors in Qualifying
> Care and Income. Three of the four States in the first pilot also had
> Income errors. Kansas had the highest frequency of errors in
> Qualifying Care, 43 of the 54 total errors. Florida had the highest
> frequency of errors in Payments, 22 of the 24 total errors. In
> comparing the frequency of the error elements, Kansas and Florida had
> the greatest frequency of Qualifying Care and Payment errors. Only
> Florida and New Jersey had errors in all five of the most common
> elements.

[]{#__RefHeading___Toc167078451 .anchor}**Exhibit 7.** Frequency of
Elements in Error

  --------------- ------------- ------------ ---------- ------------ ------------ -----------
                  **Florida**   **Kansas**   **New\     **Oregon**   **West\      **Total**
                                             Jersey**                Virginia**   

  **Qualifying    1             43           4          5            1            54 (32%)
  Care\                                                                           
  (Element 340)**                                                                 

  **Income\       9             8            13         11           4            45 (27%)
  (Element 400)**                                                                 

  **Payments\     22                         2                                    24 (14%)
  (Element 420)**                                                                 

  **Application   10                         3                       7            20 (12%)
  Forms\                                                                          
  (Element 100)**                                                                 

  **Household     5                          3                       2            10 (6%)
  Members\                                                                        
  (Element 320)**                                                                 

  **Other**       6             1            3                       6            16 (9%)

  **Total**       53            52           28         16           20           169
  --------------- ------------- ------------ ---------- ------------ ------------ -----------

> Missing documentation was the primary cause of the most common errors
> displayed in Exhibit 7. The States, in both pilots, differed in their
> coding or attribution of error types. Some States assigned all errors
> of missing documentation to Applications (Element 100). Others
> assigned the error to the element pertinent to the missing
> documentation, for example, assigning missing pay stubs to Income
> (Element 400). This is an important difference when comparing data
> between States.
>
> Exhibit 8 presents the most costly errors in total dollars from
> highest to lowest amount. As with error frequency (Exhibit 7), Income
> and Qualifying Care were the two most costly errors representing 28
> and 26 percent respectively of all improper authorized payments.
> Application Forms represent an additional 23 percent of the improper
> authorized payments. Household Members and Payments errors were 11
> percent and 4 percent of the total respectively. The remaining 8
> percent (\$1,791) of the total improper payments occurred in other
> review elements such as Residency, Qualifying Child and Computations.

[]{#__RefHeading___Toc167078452 .anchor}**Exhibit 8.** Dollar Amounts of
Elements in Error

  --------------- ------------- ------------ ---------- ------------ ------------ --------------
                  **Florida**   **Kansas**   **New\     **Oregon**   **West\      **Total**
                                             Jersey**                Virginia**   

  **Income\       \$1,271       \$342        \$3,391    \$1,082      \$176        \$6,262 (28%)
  (Element 400)**                                                                 

  **Qualifying    \$30          \$4,278      \$470      \$843        \$63         \$5,684 (26%)
  Care\                                                                           
  (Element 340)**                                                                 

  **Application   \$2,499                    \$1,638                 \$889        \$5,026 (22%)
  Forms\                                                                          
  (Element 100)**                                                                 

  **Household     \$1,331                    \$434                   \$708        \$2,473 (11%)
  Members\                                                                        
  (Element 320)**                                                                 

  **Payments\     \$733         0            \$79                    0            \$812 (3%)
  (Element 420)**                                                                 

  **Other**       \$1,044       \$30         \$31                    \$686        \$1,791 (8%)

  **Total**       \$6,908       \$4,650      \$6,043    \$1,925      \$2,522      \$22,048
  --------------- ------------- ------------ ---------- ------------ ------------ --------------

[]{#__RefHeading___Toc167079324 .anchor}**3. Interpretation of Missing
Documentation Errors**

> A detailed review of the error findings in the area of missing
> documentation shows that States had similar findings with very
> dissimilar error results. Great latitude was given to the pilot States
> in defining errors. Some States interpreted missing or out-of-date
> verification as causing improperly authorized payments, while other
> States ruled the missing or out-of-date verification as a procedural
> error or no error. This inconsistency between reviewer's
> interpretations also occurred within a State's findings, despite
> States\' efforts to achieve inter-rater reliability.

[]{#__RefHeading___Toc167079325 .anchor}**4. Coding of Missing
Documentation Error Types**

> In addition to differing definitions of errors, the States in both
> pilots differed in their coding or attribution of error types. Some
> States assigned all errors of missing documentation to Applications
> (Element 100). Others assigned the error to the element pertinent to
> the missing documentation, for example, assigning missing pay stubs to
> Income (Element 400). This is an important difference when comparing
> data between States. Although the treatment of missing documentation
> varied between the States, missing documentation was the primary error
> cause and accounted for 28 percent of the total improperly authorized
> payments in the second pilot and 57 percent of the total improperly
> authorized payments in the first pilot.

[]{#__RefHeading___Toc167079326 .anchor}**5. Review Process and Missing
Documentation Error**

> States determined whether to review records centrally or in a local
> office. A concern specific to centralized reviews is the shipment of
> case records. It is not uncommon for record material to be missed when
> a record is shipped, for example, failure to obtain retired material
> or recently received but un-filed documentation. However, the findings
> from the two pilots do not indicate that the decision to review the
> records centrally, rather than at the local office, was related to the
> frequency of missing documentation as a cause for errors. New Jersey
> reviewed records at the local CCR&R agencies and had as high a
> frequency of missing documentation errors as Florida, which reviewed
> records centrally. Ohio reviewed some records centrally and those from
> the larger jurisdictions, such as Cuyahoga County, on site. Cuyahoga
> County accounted for 30 of the 42 missing documentation errors.
>
> As an error type, missing documentation is not affected by some
> corrective action strategies, such as exception reports, system edits,
> and data matches. Training to increase staff awareness of the problem,
> knowledge of policy, interviewing skills, and quality of routine case
> reviews are the most effective strategies States can use to prevent or
> reduce procedural or policy errors. The State responses to the error
> findings in both pilots include numerous initiatives to improve the
> quality of case work and the frequency of reviews.

[]{#__RefHeading___Toc167079327 .anchor}**6. Interpretation of Income
Errors**

> Income errors were the only error type observed in all nine States.
> Income errors accounted for 28% of improperly authorized payments in
> the second pilot and was a major error prone area in all five pilot
> States. Similar percentages existed in the first pilot, where income
> accounted for 23 percent of the improperly authorized payments. Income
> error causes included: conversion, averaging, deductions, failure to
> include documented income, and inclusion of income, following job
> termination. The State responses to the error findings included
> initiatives targeting income verification and calculation policies.

[]{#__RefHeading___Toc167079328 .anchor}**7. Coding of Income Error
Types**

> States in both pilots differed in their coding of income computation
> errors. Some reviewers assigned the error to Income (Element 400),
> while others assigned it to Payments/Computations (Element 430).
> Another inconsistency involved the connection between an Income error
> (Element 400) and the effect it has on the Rate/Fee schedule (Element
> 340). Some reviewers coded an income error and others assigned it to
> both review elements. Oregon's responses to the error findings in the
> second pilot included an initiative regarding training of reviewers on
> error interpretation.

[]{#__RefHeading___Toc167079329 .anchor}***8. Interpretation of Hours of
Care Errors***

> The calculation of hours of care was the third most costly error in
> the second pilot, accounting for 26 percent of the improper authorized
> payments. Unlike missing documentation and income, the significance of
> errors in the calculation of hours of care varied widely across the
> five States. West Virginia had one hours of care error that accounted
> for 2 percent (\$63) of the improper authorized payments within the
> sampled cases. New Jersey also had a relatively small number of hours
> of care errors that accounted for only seven percent (\$470) of the
> improper authorized payments. These findings contrast with Kansas
> which had 43 case errors accounting for 92 percent (\$4,277) of the
> improper authorized payments. The hours of care error causes were most
> often the failure to allow travel/meals, hours that were inconsistent
> with work or school schedules, and the incorrect use of half time vs.
> full time. Based on previous child care case reviews, Kansas
> anticipated having cases with errors based on the fact that the sample
> month of October 2005 was the second month following conversion to the
> EBT system. Florida had the additional issue of the closure of centers
> due to hurricanes. As with missing documentation, the State responses
> to these error findings include training and case reviews.

[]{#__RefHeading___Toc167079330 .anchor}**9. Summary of State
Responses**

> As a result of this pilot, each of the five States has planned action
> steps or has implemented several new systematic changes to improve
> monitoring and reduce improper payments. The State strategies include:

-   Strengthen supervision of new eligibility workers;

```{=html}
<!-- -->
```
-   Clarify selected standards with eligibility workers;

-   Improve IT system elements to:

    -   Prevent or decrease calculation errors,

    -   Generate exception reports to highlight areas of potential
        problems or concern,

    -   Implement automatic income calculation, and

    -   Enhance the capability of extracting data from other data
        systems;

```{=html}
<!-- -->
```
-   Conduct extensive technical assistance in counties to address
    error-prone areas;

-   Institute changes in the monitoring process;

-   Introduce statutory changes to simplify access to other State
    databases; and

-   Examine State policies to determine what changes may be necessary to
    provide more consistent application of policies and procedures.

> Some of the State specific actions are:

-   West Virginia Technical Assistance staff will continue to perform
    random quarterly case audits. Prior to the pilot, the FACTS
    electronic record was the object of the review. An expanded review
    will now include the physical case record.

-   Oregon continues to conduct a series of "Accuracy Summits" in all
    areas of the State that focus on techniques to reduce errors in the
    Food Stamp program. Beginning with the next series of summits in
    July 2007, workshops to improve payment accuracy in child care will
    be included in the summits.

-   Oregon plans to modify the Food Stamp targeted review process and
    database that contains review information to ensure that income
    errors identified in the Food Stamp case are corrected in the
    companion Employment Related Day Care (ERDC) case.

-   Kansas continues to complete monthly Child Care QC reviews and uses
    the results of these reviews to inform training needs. Kansas is
    updating the Child Care Personnel Trainer and Training Academy to
    emphasize case documentation and computation of hours of care needed
    and income. Kansas supervisors now complete monthly case reviews for
    the Child Care Assistance Program.

-   Kansas contracted with a firm to develop software to track all case
    review findings and provide aggregate review data. The web based
    system tracks aggregate case review data for the mandatory Food
    Stamp reviews and the child care reviews. The child care reviews
    began in July 2006. The software cost was \$75,000 and there is an
    18 percent annual maintenance costs agreement.

-   Florida developed a desk reference tool for eligibility procedures,
    initiated focused monitoring and training on eligibility procedures
    on an annual basis and is providing each Early Learning Coalition
    (ELC) with its individual results of the pilot review process along
    with recommendations for corrective action.

```{=html}
<!-- -->
```
-   New Jersey is implementing a process to conduct electronic matching
    of the automated child care client database with records on other
    wage, SSI, and child support information systems to identify and
    reduce the number of improper payments.

```{=html}
<!-- -->
```
-   New Jersey plans to continue conducting child care record reviews
    utilizing the pilot monitoring tool, using a methodology that
    guarantees a statistically valid sample size of randomly selected
    cases, and will hire or allocate quality assurance staff needed.

```{=html}
<!-- -->
```
-   Recognizing the limitations of its legacy automated system, KIDS
    (Key Information Delivery System), Arkansas designed and developed a
    new automated eligibility system called Keying in Day Care
    Accurately, Reliably, and Efficiently (KIDCare), to be fully
    operational as of July 2005. Arkansas designed KIDCare to determine
    eligibility based on program specific guidelines and has
    incorporated numerous edits to prevent inaccuracies from occurring
    on the front end of eligibility determination.

-   Illinois developed a resource guide for workers to outline
    acceptable forms of documentation or verification needed to
    determine eligibility accurately. The guide provides clarification
    for workers to use with clients during the eligibility process when
    the client cannot produce the required documentation. A worker can
    place a case in a pending status while awaiting necessary
    documentation.

-   As a direct result of the first pilot, Ohio began to retool its
    monitoring and technical assistance processes with the county
    agencies, implemented a quality control process for the child care
    program, and examined policy for possible revisions to strengthen
    and clarify procedures.

-   In response to the first pilot, Colorado planned to implement an
    automatic income calculation into Colorado's automated Child Care
    Tracking System. State staff also conducted training and feedback
    with all participating counties to address deficiencies identified
    in the pilot. Staff members then shared the results of the error
    rate analysis at the State child care conference and changed
    statutory language to simplify access to other State databases such
    as those for new hires and unemployment compensation. Colorado also
    examined State policies to determine where changes may be necessary
    to provide more consistent application, particularly in the area of
    self-employment.

    []{#__RefHeading___Toc167079331 .anchor}**V. COST ANALYSIS**

In order to estimate the burden to States to implement the pilot error
rate methodology, each of the nine pilot States (including the four
States from the first pilot and five States from the second pilot)
agreed to work with the Child Care Bureau (CCB) to determine the costs
associated with the methodology implementation. This chapter reviews the
costs incurred by all nine pilot States to implement the methodology
with a sample size of 150 selected from the universe of cases for one
month.

[]{#__RefHeading___Toc167079332 .anchor}**A. Objectives of the Cost
Analysis**

The CCB asked the contractor to expand upon the cost analysis findings
from the first pilot of the error rate methodology to include the
additional five States from the second pilot. In addition to the data
from the two pilots, this cost analysis also draws upon data from
California, where staff computed cost information and provided it to the
CCB. The inclusion of the California data is useful because the
California legislature mandated implementation of an on-going child care
error rate methodology. The California methodology closely mirrors the
methodology used in the two CCB pilots and, therefore, California's
results lend additional substance to the analysis.

During any pilot, two important questions are how much effort did the
State expend and was that effort worth the results that the State
achieved. The first pilot provided an initial baseline analysis of costs
incurred by the four States. The first pilot findings also underscored
the importance of regularly conducted compliance reviews by States to
improve administrative practice. In order to further that analysis, this
chapter examines the estimated costs for the initial four pilot States,
the costs for the five States that participated in the second pilot, and
the additional cost information from California.

[]{#__RefHeading___Toc167079333 .anchor}**B. Conceptual Design**

The conceptual design for the cost analysis involved two general types
of cost variables---fixed and variable (usually per capita). For
example, costs associated with training the Review Team are essentially
fixed, one-time costs for a specific year's cost analysis, provided all
of the Review Team members can be trained during a single session. If
multiple sessions are required for review of a one-month sample of
cases, which was not the case for the nine pilot States, additional
sessions would involve only the costs associated with presenter and
participant time, since the preparation time will have already been
incurred (e.g., development of a training agenda and training
materials).

Site visit costs to the pilot States are relatively fixed, with the
major exception that there were two-day site visits to the four States
in the first pilot and only one-day site visits to the five States in
the second pilot. In the first pilot, the Federal Project Team visited
one or more county locations in some of the States. Therefore, the first
pilot includes county costs for the additional day(s), travel, and
county staff burden. The second pilot did not include a second day to
visit a county or local child care office/Child Care Resources and
Referral (CCR&R) agency, so no county costs are reflected in the
preparation, site visit, and training costs below.

In both pilots, costs also were broken out by general levels of pay
(hourly) rates, plus benefits, to achieve better precision. The
principal cost variable is the average per capita cost to review a
record. The review costs would also include the staff costs for the
re-review of records in some states. Additional per capita costs would
be the cost of follow-up verification, as one example, but this only
occurred for a subset of records in the State of Colorado in the first
pilot.

Finally, the contractor calculated the costs for the nine pilot sites
based on the sample size of 150 cases reviewed; however, when viewing
the California costs it is important to note that California used a
sample of 1,744 cases due to State legislative requirements.

[]{#__RefHeading___Toc167079334 .anchor}**C. Measuring the Costs to
Conduct the Error Rate Pilot**

To begin the cost analysis in the first pilot, the contractor contacted
each of the pilot States and California to explain the expanded[^17]
cost analysis. Each of the States was eager to continue the work begun
during the pilot and pledged to provide the data on costs incurred
during the process. The next step was to develop a tool to collect data
regarding the costs associated with measuring improper payments in the
child care program. For the initial four pilot States, the contractor
also captured the costs associated with conducting a site visit in
association with measuring improper payments, as well as the estimated
costs associated with conducting the record review.

[]{#__RefHeading___Toc167079335 .anchor}**D. Cost Analysis Worksheet
Development**

The contractor developed a draft worksheet template to capture the
relevant cost information and shared it with the initial four pilot
States. After the States commented on the draft worksheet, the
contractor modified the worksheet and sent it to the participating
States. After discussions with the pilot States, a consensus set of cost
categories emerged. The cost categories were staff time, materials,
copying, postage and site visit costs. For the first pilot, the
worksheet included :

-   Costs incurred by different State, county, CCR&R agency and
    contractor staff for conducting the record review process;

-   Costs related to participating in the site visits; and

-   Federal participants estimated costs for the site visit only.

The contractor used a slightly modified cost analysis worksheet in the
second pilot in order to capture greater detail on the costs incurred by
different staff to complete each of the error rate pilot tasks:

-   Preparation and training of the Review Team (including the Federal
    Project Team site visit);

-   Drawing the sample;

-   Record review process;

-   Consolidating the data on the *Data Entry Forms*;

-   Reviewing the findings and responding to the three questions; and

-   Discussion of causes/strategies and report preparation.

The second pilot did not ask States to gather county, local agency, and
contractor costs, since local staff did not participate in the second
pilot.

Staff time was divided into categories according to job
classification---Review Team, Computer Support, and Clerical
Support---to allow for differing salary ranges. Within these job
categories, costs were estimated for the above activities.

[]{#__RefHeading___Toc167079336 .anchor}**E. Preparation and Site Visit
Costs**

For estimating the costs involved in preparation and training for the
error rate project, the second pilot States estimated the costs
associated with participating in the technical assistance planning calls
with the contractor, customizing the *Record Review Worksheet* template,
planning for and participating in a one-day site visit, selecting and
training the record review team, conducting the case record review, and
completing the *Data Entry Form*.

The first pilot focused on the costs associated specifically with the
site visits. In the first pilot, the Federal Project Team consisted of
two contractor staff members, the Federal Regional Office
representatives, and several Federal Central Office staff members.
During the second pilot, the Federal Project Team consisted of two
contractor staff members, Federal Regional Office representatives, and
Federal Central Office staff. Only one Federal Central Office staff
member was present in Kansas and New Jersey, two Federal Central Office
staff members were present in Florida, and three were present in West
Virginia.

In both pilots, the focus of the site visit was to review the tools that
States would use to conduct the record review process, discuss the
universe of cases (sampling frame) and the selected sample, and provide
technical assistance that might be necessary for a successful record
review. In some of the States, the site visit team also conducted a
review of several of the records. This process allowed for questions and
clarifications concerning the tools and definitions prior to beginning
the actual review. This training process increased inter-rater
reliability and ensured consistency across States regarding error
interpretation.

Exhibit 9 highlights the aggregate State and County costs for the
two-day site visit activities in the first pilot and the State only
costs for preparation, training, and the one-day site visit in the
second pilot.

[]{#__RefHeading___Toc167078111 .anchor}**Exhibit 9**. Preparation,
Training and Site Visit Costs for Nine Pilot States

  ------------------------------------------ ----------------------------
  **Initial Pilot States**                   

  Arkansas                                   \$7,352

  Colorado                                   \$7,140

  Illinois                                   \$5,000

  Ohio                                       \$10,295

  **Sub Total**                              **\$29,787**

  **Second Phase Pilot States**              

  Florida                                    \$10,061

  Kansas                                     \$3,814

  New Jersey[^18]                            

  Oregon                                     \$3,173

  West Virginia                              \$2,154

  **Sub Total**                              **\$19,202**

  **TOTAL COSTS**                            **\$48,989**
  ------------------------------------------ ----------------------------

In the first pilot, the consensus among all participants was that the
time spent discussing the *Record Review Worksheets* and reviewing the
definition of each element was essential to establish consistency, both
within and across States. In the second pilot, the site visit had a dual
focus of reviewing State eligibility practices regarding improper
payments in addition to providing technical assistance around sampling
and the review methodology.

Costs were higher in the first pilot because the site visit occurred
over two days and involved visiting one or two local child care offices.
County staff time accounted for a substantial number of hours and costs
in the first pilot. The second pilot focused on gauging State costs only
involved with preparation, training, and the one-day site visit. Costs
were lowest for the States, such as Kansas, Oregon, and West Virginia,
that used existing quality assurance and monitoring staff to conduct the
child care reviews. While Florida recorded approximately the same number
of hours as the other States for preparation and training, Florida used
a higher average hourly (salary + benefits) rate to estimate the cost of
staff time.

[]{#__RefHeading___Toc167079338 .anchor}**F. Record Review Costs**

Although each of the participating pilot States engaged in the same
basic process to conduct the record reviews, there was some variation in
implementation.

[]{#__RefHeading___Toc167079339 .anchor}**1. Four States** **in the
First Pilot**

-   Ten staff members from county offices in Arkansas comprised the
    Review Team. The reviews occurred in Little Rock. The Review Team
    members received an overview and training on the process used to
    review the records. The Review Team conducted the record reviews in
    a large room and questions about individual records were resolved by
    asking questions within the group.

-   In Colorado, staff from the counties first reviewed the records
    before they submitted them to the State. The Review Team then
    conducted an in-depth review of all records. For the record review
    process, two staff members from two different State offices---one
    from the Audit Division of the Office of Performance Improvement and
    one from the Child Care Division---participated in the pilot.

-   In Illinois, staff from the Program Integrity and Quality Assurance
    Unit comprised the Review Team. This review was an extension of the
    regular duties and functions of this unit. Little modification or
    variation from the normal processes was necessary to meet the record
    review requirements. Because of the volume of cases in the Cook
    County CCR&R agency, the Cook County CCR&R---Action for
    Children---reviewed the cases on site. The other CCR&R and
    site-administered programs sent their records to the Department of
    Human Services for review.

-   Ohio did not have an existing process in place to conduct the case
    record reviews. Prior to the pilot, the Research, Assessment, and
    Accountability Division, Quality Control Unit did not audit child
    care, believing that child care program requirements were too vague
    to lend themselves to a quality assurance review process. This unit,
    along with staff from the Office for Children and Families, Bureau
    of Child Care and Development, comprised the State Review Team that
    conducted the record reviews.

    []{#__RefHeading___Toc167079340 .anchor}**2. Five States** **in
    the** **Second Pilot**

```{=html}
<!-- -->
```
-   In Florida, the Review Team consisted of two Managers, two
    Performance Analysts, and three Program Analysts. Florida made
    substantial modifications to the *Record Review Worksheet* template.
    The review process occurred centrally, with all records sent to
    Tallahassee. A second member of the team re-reviewed all records for
    inter-rater reliability.

```{=html}
<!-- -->
```
-   The Kansas Review Team consisted of three Quality Assurance
    Supervisors (Topeka, Wichita, Kansas City) and three Quality
    Assurance Specialists. The team also conducted a re-review of all
    cases by having a second team member conduct the review. When both
    team members agreed on an error, they sent a notice to the local
    Social Rehabilitation Services (SRS) office for corrective action.
    The Review Team reviewed all records centrally in Topeka. Kansas
    modified the *Record Review Worksheet* to include "Yes" or "No"
    questions to reduce the amount of written narration for the
    reviewers.

-   New Jersey substantially amended the *Record Review Worksheet*
    template to substitute coding and specific boilerplate language to
    guide the reviewer's written case narrative. New Jersey trained the
    Review Team members centrally and then reviewers conducted the
    reviews in their assigned regions at the local CCR&R agencies. The
    Review Team consisted of three Supervisors and 13 Child Care
    Specialists.

-   Oregon made very few modifications to the *Record Review Worksheet*
    template. The Review Team consisted of five Quality Control (QC)
    Staff Persons who have responsibility for TANF and FS case reviews.
    Following training on child care policy and procedures, the Review
    Team reviewed the cases centrally. The State Team Leader reviewed
    all cases with errors a second time and discovered that the Review
    Team went beyond the level of a desk audit, expecting greater detail
    in documentation than what had been anticipated or required of the
    child care review process. As a result, the Review Team reviewed a
    subset of cases a third time and changed some review results
    accordingly.

-   West Virginia made substantial modifications to the *Record Review
    Worksheet* template, using check-off boxes, "Yes" and "No" fields
    and coding to eliminate narrative recording. West Virginia completed
    the record review process and compiled the results prior to the site
    visit. The Review Team consisted of two Field Consultants and a
    Policy Specialist. The reviews occurred on site at the regional
    CCR&R agencies. The reviewers had already read 10 cases during one
    of their regular CCR&R audits. To improve the review process, the
    three reviewers completed several reviews jointly and consulted by
    telephone frequently. Because the team reviewed the cases
    regionally, travel time was a considerable factor with several hours
    of driving time from one CCR&R agency to another.

Each of the four States in the first pilot submitted information
concerning the costs for conducting the record reviews. Exhibit 10
presents the State and local/county costs for the record review process
and site visits for the four States in the first pilot.

[]{#__RefHeading___Toc167078115 .anchor}**Exhibit 10**. Cost of Record
Review Process for Four States in the First Pilot

+------------+------+------+------+------+------+------+------+------+
| **Task**   | **Ar |      | **Co |      | **Il |      | **Oh |      |
|            | kans |      | lora |      | lino |      | io** |      |
|            | as** |      | do** |      | is** |      |      |      |
+------------+------+------+------+------+------+------+------+------+
|            | *    | **   | *    | **   | *    | **   | *    | **   |
|            | *Sta | Coun | *Sta | Coun | *Sta | Coun | *Sta | Coun |
|            | te** | ty** | te** | ty** | te** | ty** | te** | ty** |
+------------+------+------+------+------+------+------+------+------+
| Staff Time | \$12 | \$5  | \$22 | \$10 | \$16 | \$6  | \$26 | \$9  |
|            | ,030 | ,742 | ,920 | ,465 | ,760 | ,475 | ,571 | ,173 |
| (Review    |      |      |      |      |      |      |      |      |
| Team)      |      |      |      |      |      |      |      |      |
+------------+------+------+------+------+------+------+------+------+
| Staff Time | \    | \    | \    | \    | \$1  |      | \$2  | \$4  |
|            | $150 | $110 | $560 | $175 | ,338 |      | ,568 | ,275 |
| (Computer  |      |      |      |      |      |      |      |      |
| Support)   |      |      |      |      |      |      |      |      |
+------------+------+------+------+------+------+------+------+------+
| Staff Time |      |      | \    | \$1  | \$2  | \$1  | \$1  | \$3  |
|            |      |      | $170 | ,780 | ,094 | ,906 | ,545 | ,947 |
| (Clerical  |      |      |      |      |      |      |      |      |
| and Other  |      |      |      |      |      |      |      |      |
| Support)   |      |      |      |      |      |      |      |      |
+------------+------+------+------+------+------+------+------+------+
| Site Visit | \$1  | \$5  | \$5  | \$1  | \$3  | \$1  | \$9  | \    |
|            | ,920 | ,432 | ,970 | ,170 | ,500 | ,500 | ,341 | $954 |
+------------+------+------+------+------+------+------+------+------+
| Materials  | \$50 |      | \$50 | \$50 | \$10 | \    | \$1  | \    |
|            |      |      |      |      |      | $115 | ,378 | $726 |
+------------+------+------+------+------+------+------+------+------+
| Copying    | \$25 |      | \$1  |      | \$20 | \    | \$42 | \    |
|            |      |      | ,378 |      |      | $138 |      | $865 |
+------------+------+------+------+------+------+------+------+------+
| Postage    |      |      | \    | \    | \$6  | \    | \    | \    |
|            |      |      | $260 | $260 |      | $212 | $183 | $412 |
+------------+------+------+------+------+------+------+------+------+
| Travel     |      |      | \    |      | \$4  |      | \    |      |
|            |      |      | $100 |      |      |      | $855 |      |
+------------+------+------+------+------+------+------+------+------+
| Other      |      |      | \    |      |      | \    | \$4  | \$2  |
| Costs      |      |      | $500 |      |      | $385 | ,298 | ,003 |
+------------+------+------+------+------+------+------+------+------+
| **         | **\$ | **\$ | **\$ | **\$ | **\$ | **\$ | **\$ | **\$ |
| Subtotal** | 14,1 | 11,2 | 31,9 | 13,2 | 23,7 | 10,7 | 46,7 | 22,3 |
|            | 75** | 84** | 08** | 90** | 32** | 31** | 81** | 55** |
+------------+------+------+------+------+------+------+------+------+
| **COMBINED | **\$ |      | **\$ |      | **\$ |      | **\$ |      |
| TOTAL      | 25,4 |      | 45,1 |      | 34,4 |      | 69,1 |      |
| COSTS**    | 59** |      | 98** |      | 63** |      | 36** |      |
+------------+------+------+------+------+------+------+------+------+

In the first pilot, several factors contributed to differing totals
among States. Arkansas reduced many of its costs by requesting that
field staff members transport the records to Little Rock---a central
location---for the record review. Illinois had an existing process in
place for conducting record reviews; therefore, the costs in Illinois
were less than in Colorado and Ohio, where a new process was developed
and implemented. The two county-administered States incurred higher
costs than the State-administered States, suggesting that the cost
burden for county-administered States (Colorado and Ohio) may be
substantially greater than for State-administered States (Arkansas and
Illinois). Exhibit 11 shows that California, a county-administered
State, corroborated this finding, because it had 11 times as large a
sample size, compared with the pilot States, but it had a total cost of
more than 20-30 times the costs for Colorado and Ohio and 50-60 times
the costs for Illinois and Arkansas.

Exhibit 11 presents California's total costs for the record review
process. California's total costs are substantially higher due to the
larger sample size of 1,744 cases.

[]{#__RefHeading___Toc167078116 .anchor}**Exhibit 11. Cost of Record
Review Process for California**

+---------------------------------------+--------------+---------------+
| **Task**                              | **Time**     | **Cost**      |
+---------------------------------------+--------------+---------------+
| **Staff Time (Review Team)**          |              |               |
+---------------------------------------+--------------+---------------+
| > Preparation                         |              | \$142,333     |
+---------------------------------------+--------------+---------------+
| > Training                            |              | \$71,167      |
+---------------------------------------+--------------+---------------+
| > Record Review                       |              | \$640,500     |
+---------------------------------------+--------------+---------------+
| > **Subtotal**                        |              | **\$854,000** |
+---------------------------------------+--------------+---------------+
| **Staff Time (Computer Support)**     |              |               |
+---------------------------------------+--------------+---------------+
| > Preparation                         |              | \$500,000     |
+---------------------------------------+--------------+---------------+
| > Training                            |              | \$0           |
+---------------------------------------+--------------+---------------+
| > Record Review                       |              | \$50,000      |
+---------------------------------------+--------------+---------------+
| > **Subtotal**                        |              | **\$550,000** |
+---------------------------------------+--------------+---------------+
| **Staff Time (Clerical and Other      |              |               |
| Support)**                            |              |               |
+---------------------------------------+--------------+---------------+
| > Preparation                         |              | \$10,833      |
+---------------------------------------+--------------+---------------+
| > Training                            |              | \$5,417       |
+---------------------------------------+--------------+---------------+
| > Record Review                       |              | \$48,750      |
+---------------------------------------+--------------+---------------+
| > **Subtotal**                        |              | **\$65,000**  |
+---------------------------------------+--------------+---------------+
| **Materials**                         |              | **\$0**       |
+---------------------------------------+--------------+---------------+
| **Copying**                           |              | **\$500**     |
+---------------------------------------+--------------+---------------+
| **Postage**                           |              | **\$0**       |
+---------------------------------------+--------------+---------------+
| **Travel**                            |              | **\$0**       |
+---------------------------------------+--------------+---------------+
| **Other Costs**                       |              | **\$100,000** |
+---------------------------------------+--------------+---------------+
| **TOTAL COSTS**                       |              | **            |
|                                       |              | \$1,569,500** |
+---------------------------------------+--------------+---------------+

Because of the modified cost analysis worksheet in the second pilot,
States tracked the time utilized by different staff to complete each of
the error rate pilot tasks:

-   Preparation and training of Review Team (including the Federal
    Project Team site visit);

-   Drawing the sample;

-   Record review process;

-   Consolidating the data on the *Data Entry Forms*;

-   Reviewing the findings and responding to the three questions; and

-   Discussion of causes/strategies and report preparation.

The second pilot did not ask States to gather county, local agency, and
contractor costs, since local staff members did not participate in the
second pilot.

Exhibit 12 displays the aggregate costs summarized by tasks for four of
the five States in the second pilot. New Jersey's cost data were
unavailable at the time of report submission.

[]{#__RefHeading___Toc167078117 .anchor}**Exhibit 12**. Cost of Project
Tasks for Five States in the Second Pilot

  ------------------- --------------- -------------- ---------------- -------------- -------------
  ***Tasks***         ***Florida***   ***Kansas***   ***New           ***Oregon***   ***West
                                                     Jersey***[^19]                  Virginia***

  **Preparation,      \$10,061        \$3,814                         \$3,173        \$2,154
  Training, and Site                                                                 
  Visits**                                                                           

  **Record Review     \$14,550        \$8198                          \$6,981        \$2,260
  Process**                                                                          

  **Drawing the       \$2,225         \$388                                          \$655
  Sample**                                                                           

  **Consolidation of  \$1299          \$561                                          \$310
  The Data**                                                                         

  **Reviewing the     \$3533          \$157                           \$1454         \$717
  Findings**                                                                         

  **Discussion of     \$1073          \$179                                          \$388
  Causes/Strategies                                                                  
  and Report                                                                         
  Preparation**                                                                      

  **Materials**       \$279                                                          \$107

  **Copying**         \$225           \$771                                          \$19

  **Postage**         \$620           \$128                                          

  **Travel**          \$2,477                                                        \$690

  **Other Costs**                                                                    

  **Total             **\$36,671**    **\$13,603**                    **\$11,609**   **\$7,686**
  Costs**[^20]                                                                       
  ------------------- --------------- -------------- ---------------- -------------- -------------

Final analysis of the cost data does not include the data from New
Jersey. For the remaining States, the costs incurred during the second
pilot are less than those incurred by States which participated in the
first pilot. Reasons for this include:

-   States in the second pilot had access to information and
    instrumentation published in the final report from the first pilot.

-   Four of the five States in the second pilot made significant
    modifications to the *Record Review Worksheet* to reduce the time
    needed for its completion.

-   Costs were lowest for the States, such as Kansas, Oregon, and West
    Virginia, that used existing quality assurance and monitoring staff
    members to conduct the child care reviews.

-   While Florida recorded approximately the same number of hours as the
    other States for preparation and training, Florida used a higher
    average hourly (salary + benefits) rate to estimate the cost of
    staff time.

-   The site visits in the first pilot lasted two days to allow time for
    local office visits (Ohio had two local office visits). The States
    estimated a cost of \$9,000 for county staff for these visits. In
    the second pilot site visits were reduced to one day and the local
    office visits were eliminated.

-   The States made different choices in the staff assigned to the
    Review Team. Those States that assigned the task to central office
    administrators, auditors and policy specialists had significantly
    higher review team costs than those States where existing review
    staff completed the review. The three States in the second pilot
    that used existing Quality Control and monitoring staff members for
    the review had the lowest record review team costs among the nine
    States.

-   Total costs in the second pilot included a process to re-review
    cases to increase inter-rater reliability and to verify the accuracy
    and consistency of error determination. Some States included these
    costs as part of the record review process, while Oregon recorded
    time and costs for the re-review process as a separate activity. The
    re-review process did not add substantially to the total per case
    review time.

-   County costs were not estimated in the second pilot.

    []{#__RefHeading___Toc167079344 .anchor}**VI. RECOMMENDATIONS**

A goal of the error rate pilots was to test a methodology that could
provide States with useful information on areas for improvement in
administrative policies and procedures. All nine States acknowledged
learning additional useful information about the quality of their
eligibility processes.

The two most frequent errors in the Second Pilot occurred in the
calculation of hours of care and treatment of income. The three most
costly errors occurred in the treatment of income, calculation of hours
of care and application forms. Although the error coding of missing
documentation varied between the States, it was the primary error cause
in the second pilot.

As a result of this pilot, each of the five States has planned action
steps or has implemented several new systematic changes to improve
monitoring and reduce improper payments. Highlights of some of the State
strategies include:

-   Strengthen supervision of new eligibility workers;

```{=html}
<!-- -->
```
-   Clarify selected standards with eligibility workers;

-   Improve IT system elements to:

    -   prevent or decrease calculation errors;

    -   generate exception reports to highlight areas of potential
        problems or concern;

    -   implement automatic income calculation; and

    -   enhance the capability of extracting data from other data
        systems.

```{=html}
<!-- -->
```
-   Conduct extensive technical assistance in counties to address
    error-prone areas;

-   Institute changes in the monitoring process;

-   Introduce statutory changes to simplify access to other State
    databases; and

-   Examine State policies to determine what changes may be necessary to
    provide more consistent application of policies and procedures.

The other goal of the error rate pilot was to test the error rate
methodology in different States to produce useful lessons learned
regarding management of the methodology process to result in reduction
of State burden and improvement in the accuracy of the review findings.
Based on the experience of nine pilot States, this chapter provides the
implications for replication or expansion of the error rate process
including recommendations regarding:

-   Management of the error rate methodology process;

-   Provision of instructions and managing the review process;

-   Use of the instruments;

-   Modifications to improve reliability across States; and

-   Implications for ongoing technical assistance.

    []{#__RefHeading___Toc167079345 .anchor}**A. Management of the Error
    Rate Methodology Process**

    []{#__RefHeading___Toc167079346 .anchor}**1. State Project Team
    Formation and Responsibilities**

> *To coordinate the error rate methodology process, a State needs to
> create a State Project Team (Project Team).* A critical factor in the
> quality of the review process in the second pilot was the leadership
> role played by the Project Team. The more successful teams had similar
> characteristics. Leadership of the Project Team was assigned to a high
> level management staff member. The Project Team leader's
> responsibilities included establishing and adhering to timelines;
> customizing the *Record Review Worksheet*; training the reviewers; and
> arranging Project Team meetings.
>
> In addition to the Project Team leader, the Project Team membership
> should include the State Lead Agency administrator, staff
> representatives from program, policy, training, information
> technology, and the Review Team leader.
>
> The inclusion of the State Lead Agency administrator provides the
> appropriate level of importance to the Project Team's assignment.
> Program and policy team members provide critical input into the
> customization of the *Record Review Worksheet*, logistics of the
> record review process, record review process training, coordination of
> the re-review process, consolidation of the data analysis and error
> findings, and completion of the *State Response Form*. IT staff
> provide critical logistical support to the sample selection process,
> as evidenced in the sampling issues New Jersey confronted (pg. 27),
> and automation of the *Record Review Worksheet.*
>
> **2. Provision of Time to Complete All Steps of the Process**
>
> *In managing the review process a State must allow enough time to
> adequately complete the error rate methodology*. The pilots provided
> illustrative data on the amount of time needed to complete the
> methodology, but it should be remembered that these statistics are
> related to (1) a 150 case sample for a single sampling frame and (2) a
> project completed with considerable technical support, particularly in
> the sample selection and data analysis.
>
> While sample size and technical support clearly play a role in time
> estimates the length of time for States to complete the error rate
> methodology is largely dependent on selecting and providing sufficient
> staff resources to the actual record review component. Those pilot
> States with Review Team members who were experienced in any local
> operations or compliance monitoring function (Quality Control (QC),
> audit, technical support) accomplished the record review efficiently
> and accurately.
>
> Setting timelines for completion of the error rate methodology process
> requires consideration of several other factors. If the State decides
> to expand the pilot methodology and conduct reviews more frequently
> than once a year, arrangements must be made in advance with IT for
> additional sampling and with county agencies or local contractors to
> obtain additional case records for the review process. Likewise time
> will be a factor if a State decides to include in the review
> additional client information sources (social security and wage data
> bases, motor vehicles, etc.) and attendance and payment files.
> Arrangements would need to be made in advance to coordinate with other
> parts of the agency to obtain records or payment files.
>
> Three States captured review completion information (date
> assigned/date completed) in their *Record Review Worksheet*. Two of
> these had large State Review Teams and conducted the review centrally
> and the other was one of the smallest State Review Teams and read
> records on site. Based on that information, the review of the 150 case
> sample with a single review month took one month for all three States.
> All States agreed with the estimate given at the site visit. *It is
> recommended that the methodology timeframe allow for one hour for each
> case record review.*
>
> The decision of whether to conduct the case record review process
> locally or centrally will have a time and workload effect at the
> county agency or local contractor level. The State timeline needs to
> provide the time necessary for each reviewer's travel or the shipping
> of case records to a central location for reviews. *It is important
> for States to estimate adequate time for the re-review process, an
> important reliability check used by the pilot States*. Because the
> re-review component of a centralized record review has fewer practical
> problems than if the records are read at the local site, it is
> especially important for the State to include a step in the timeline
> for a re-review method if its decision is to review records locally.
> West Virginia addressed the re-review issue through the use of
> conference calls on error cases.
>
> The time needed for sample selection is dependent on a State's IT
> resources and whether or not the sample selection can occur centrally.
> For some pilot States, sample selection was dependent on cooperation
> with county agencies or local contractors who may have separate data
> systems or none at all.

[]{#__RefHeading___Toc167079347 .anchor}**B. Provision of Instructions
and Managing the Review Process**

*Under the leadership of the Project Team, a State needs to establish a
State Review Team (Review Team).* The State will need to consider
existing review processes and staff assignments when making decisions
about the actual composition and size of the Review Team.

[]{#__RefHeading___Toc167079348 .anchor}**1. State Review Team Selection
and Training**

> The States made different choices in the type of staff assigned to the
> record review. Those States that assigned the task to central office
> administrators, auditors and policy specialists had significantly
> higher review team costs than those States where existing review staff
> completed the review. The two States in the second pilot that used
> existing Quality Control staff for the review (Kansas and Oregon) had
> two of the lowest record review team costs of all nine States.
>
> Kansas assigned the task to three of its existing quality assurance
> reviewers. These staff members had extensive experience conducting
> reviews in the TANF and Food Stamps programs and had conducted an
> earlier review of child care cases. Oregon also chose existing Food
> Stamp review staff members. The Florida review staff members included
> Performance and Program Analysts, who normally conduct audits and
> reviews (grants, financials, contracts, etc.) of the Early Learning
> Coalitions (ELCs). West Virginia assigned the task to two Field
> Consultants and a central office Policy Specialist. Staff members from
> the State's Central office audit and policy divisions completed the
> Colorado review. Arkansas chose local office staff members as
> reviewers.
>
> The staff resources assigned to the actual record review varied among
> the States in the pilots. The West Virginia and Colorado reviews were
> completed by three staff members and the Ohio review by four staff
> members. New Jersey and Arkansas were at the high end of the range
> with 13 and 10 staff members respectively. A determinant of the team
> size is the composition of the team. To the assigned staff members the
> record review is best described as a "special project." Some
> consideration had to be given to their normal responsibilities as
> program manager, policy specialist, auditor or Quality Control
> reviewer. Only West Virginia, Ohio and Illinois managed to combine
> record reviews with the staff members' normal field visits.
>
> *Regardless of the existing review processes and the composition of
> the Review Team, formal training is necessary to achieve inter-rater
> reliability*. The training needs to include actual case reviews to
> increase the consistency of error definition, attribution, and
> interpretation.
>
> *States need to establish a process to conduct re-reviews of a subset
> of all cases to enhance inter-rater reliability of error
> interpretation.* The findings from the second pilot underscored the
> need to incorporate some form of re-review or joint review to enhance
> inter-rater accuracy. It is recommended that the re-reviews be drawn
> from a sub-sample of **all** cases and not limited to just cases with
> an improperly authorized payment.

[]{#__RefHeading___Toc167079349 .anchor}**2. Instructions for Error
Interpretation and Coding**

> *Feedback from the pilot States and review findings underscore the
> critical importance of training the reviewers in State rules,
> policies, and procedures.*
>
> Even with staff experienced in conducting case record reviews, it
> should not be assumed that all members of the Review Team agree on the
> interpretation of policy and, as a result, "what is" or "is not" an
> error. Many State eligibility policies are replete with "mays" and
> "shoulds." In contrast, the quality of the information produced by the
> case record review process depends on the consistency of error
> definition and coding.
>
> *It is also very important for training to emphasize the need for
> sufficient detail in Column 3 of the Record Review Worksheet so case
> information and error findings can be understood.* Without training,
> reviewers will rely on their own experiences and perspectives in
> deciding errors and causes. For example, several site visits included
> case reviews as an agenda item in the review process overview. During
> the discussion, it was common for the reviewers to express very
> different decisions about the case situations. This was particularly
> true if the Review Team consisted of Quality Control (QC) staff
> members who may be accustomed to very strict or narrow interpretations
> regarding failing the case based on missing documentation.
>
> *A major emphasis for any future pilot is to provide formal training
> on standardizing error interpretation, descriptions and coding in the
> Record Review Worksheet* *to achieve inter-State reliability of data.*
> The quality of the information produced by the case record review
> process depends on the consistency of error definition and coding.
> Inconsistencies across pilot States in error interpretation
> dramatically influenced the outcomes. States in the second pilot were
> not consistent in their error coding. There were instances within a
> State where cases with similar findings had very different error
> results, in terms of improper payment vs. a correct case. Consistency
> of error definitions and coding has a direct impact on the accuracy of
> the data and as a result the development of strategies to address
> improper payments.

[]{#__RefHeading___Toc167079350 .anchor}**3. Record Review Site**

> States determined in their review processes whether to review records
> centrally or in the local office. The choice of whether to conduct the
> record review at the local office or centrally does not appear to have
> a major role in deciding the number of staff members assigned to the
> project. Both West Virginia (three State Review Team members) and New
> Jersey (thirteen State Review Team members) reviewed the records in
> the local office. Arkansas, with the second largest State Review Team,
> read the records centrally.
>
> A concern specific to centralized reviews is the shipment of case
> records. It is not uncommon for record material to be missed when a
> record is shipped, for example, failure to obtain retired material or
> recently received but un-filed documentation. The findings from the
> two pilots do not indicate that the decision to review the records
> centrally, rather than at the local office, was related to the
> frequency of missing documentation as a cause for errors. *A
> recommendation is to include a response time within the review cycle
> to allow local agencies the opportunity to provide the missing
> documentation.* The Quality Control procedures in TANF, Food Stamps,
> and Medicaid all include local response requirements. Because the
> re-review component of a centralized record review has fewer practical
> problems than if the records are read at the local site, the State
> needs to include a re-review method if its decision is to review
> records locally.

[]{#__RefHeading___Toc167079351 .anchor}**C. Use of the Instruments**

[]{#__RefHeading___Toc167079352 .anchor}**1. *Record Review Worksheet*
Development and Use**

> The pilot States customized several versions of the *Record Review
> Worksheet* that can be a reference for other States (Exhibits A-1 --
> E-1). The different versions were practical in terms of use by the
> reviewers, but still included the level of detail needed for data
> analysis. Some observations of the various versions are:

-   The *Record Review* *Worksheet* template contains a section
    requiring Client ID, State, County, and Date. Four of the nine
    States (Arkansas, Colorado, Kansas, and New Jersey) developed a face
    sheet for their *Record Review Worksheet*. The face sheets included
    summary information of the case being reviewed. Examples of the
    information captured are case name, address, case number, the date,
    and type of the most recent case action (application or
    recertification), and particulars of the other household members
    (name, date of birth, relationship). These face sheets also
    contained information related to the review processing such as
    reviewer name, date assigned, and completion date. Face sheets are a
    useful addition to the *Record Review Worksheet*, since they provide
    a valuable overview of the case and its review. Each of the nine
    States modified the *Record Review Worksheet* template. *It is
    recommended that States add a face sheet to the Record Review
    Worksheet in order to summarize key case review information.* Within
    the review elements, most States substituted predefined codes,
    check-off blocks, "Yes" and "No" fields, and specific questions for
    written narrative summary. States may choose to make changes such as
    these to simplify and standardize staff worksheet completion.

```{=html}
<!-- -->
```
-   A quality summary of the error finding is critical for data
    analysis. Pilot States differed in the quality of the review
    narratives in error cases. In some States, the narratives lacked the
    detail necessary to understand the case information or the error
    finding. In contrast, the narrative in other reviews provided a
    thorough synopsis of the case and the review results. Reviewers must
    provide detail in their error findings in Column 3 of the worksheet.
    Boilerplate language, "Yes" or "No" fields, and check-offs alone in
    Column 2 of the worksheet provide sufficient detail for non-errors,
    but too little detail for error findings. Review Team staff training
    must stress the importance of the error narrative.

```{=html}
<!-- -->
```
-   Three of the States (Florida, New Jersey, and West Virginia) created
    electronic versions of the *Record Review Worksheet*. Compared to
    hard copies, the obvious advantages of an electronic version are
    standardization, ease of analysis, ease of report submission, and
    ease of archiving. State reviewers access to laptops, portable
    memory, and shared network files is necessary to eliminate duplicate
    data entry or manually filling out worksheets and later entering the
    data electronically. *All States need to make every effort to commit
    their Record Review Worksheet to an electronic format.* (See
    Appendices A--1 to E-1 for each State's *Record Review Worksheet*.)

    []{#__RefHeading___Toc167079353 .anchor}**2. *Data Entry Form***

> The Data Entry Form summarizes the findings contained in the *Record
> Review Worksheet.* Because the information from the *Data Entry Form*
> must be standardized if used for inter-State pilot purposes, there are
> no recommendations pertaining to this form.

-   Most States in the second pilot combined the *Data Entry Form* with
    the *Record Review Worksheet* to assist reviewers in reliably
    recording the required documentation resulting from the case review
    findings. West Virginia, New Jersey, and Florida combined the *Data
    Entry Form* at the end of the *Record Review Worksheet*. Kansas kept
    the *Data Entry Form* as a separate attachment. (See Appendices A--1
    to E-1 for each State's *Record Review Worksheet*.)

-   The *Data Entry Form* is designed to capture the minimum amount of
    review information necessary to calculate error measures. Since the
    information in the form may not be modified, a State may wish to
    consider methods for collecting a greater level of detail when
    developing its *Record Review Worksheet.* Each State can decide its
    information needs in the development of targeted corrective action
    strategies after consolidation of the data and calculation of the
    error measures.

    []{#__RefHeading___Toc167079354 .anchor}**3. Cost Analysis Form**

> To determine the feasibility of expanding the review methodology, it
> was necessary to establish the costs associated with both the site
> visits and the record review process. The States had the flexibility
> to modify the costs worksheet, if modifications helped them to collect
> the necessary cost information. In the first pilot, the contractor
> instructed the States to gather all costs incurred by the State,
> counties, and CCR&R agencies and contractors that participated in the
> pilot. The contractor provided a slightly modified worksheet for the
> cost analysis in the second pilot. The second pilot did not ask States
> to gather separate county, CCR&R agency, and contractor costs, since
> local staff members did not participate in the second pilot. Several
> States in the second pilot stated that the costs were a "best guess"
> estimate.
>
> The Federal Project Team provided a substantial amount of flexibility
> to States during both pilots in gathering their costs. The results of
> the two pilots demonstrated that most States, given existing workloads
> and the additional work from the pilot, were unable to collect the
> time and costs data while, at the same time, conducting the
> methodology. Instructions for States in the first pilot were to
> estimate time and cost data retrospectively. This retrospective
> estimation resulted in less precision regarding accounting of time
> spent and actual costs. State review procedures must include a clearly
> defined method for collecting cost data that places emphasis on
> reviewer's time recording starting at project outset. It is
> recommended that a State use a standardized time/cost worksheet for
> all time and costs. It is also important that the leadership reinforce
> the importance of collecting all time and cost.
>
> With sufficient data, it would be possible to develop and use a
> time/cost standard and thereby reduce the State burden of tracking and
> compiling actual task data.
>
> Instructions for States in the second pilot were to keep track of the
> length of time to review each case during the actual case record
> review process. Only three States in the second pilot captured review
> completion information (date assigned/date completed) in their *Record
> Review Worksheet*. Two had large review teams and conducted the review
> centrally and the other was one of the smallest teams and read records
> on site. Based on that information, the review of the 150 sample cases
> took one month for all three. All States agreed with the estimate
> given at the site visit that the case review required 45 minutes to
> one hour to complete, but only three States systematically collected
> this information.
>
> As the States move forward to address improper payments, an accurate
> cost accounting of both fixed and variable costs is important to the
> error rate process. State consistency in data collection of time and
> costs from the beginning of the process is critical to accurately
> establish the true cost of this effort. *Recommendations for future
> time and cost accounting include use of a standardized time/cost
> worksheet for all State/county participants and administrative
> leadership to reinforce the importance of collecting all time and
> costs from the outset of the error rate process.*

[]{#__RefHeading___Toc167079355 .anchor}**4. State Response Form**

> New for the second pilot, the contractor shared the findings with each
> of the five pilot States following the calculation of error measures
> and data analysis. After reviewing its error rate findings and site
> visit summary description, each pilot State submitted a written
> response to the following three topic areas included in the *State
> Response Form*:

-   What are the causes of the improper payments that have been
    identified? What are the actions that will be taken to correct those
    causes in order to reduce errors in the future?

-   Describe the information systems and other infrastructure that
    assist the State in identifying and reducing improper payments. If
    the Lead Agency does not have these tools, describe actions to be
    taken to acquire the necessary information systems and other
    infrastructure, as required in the Improper Payments Information Act
    of 2002; and

-   What steps will the State take to ensure that the Lead Agency and
    eligibility workers will be accountable for reducing improper
    payments?

> Recommendations for States to consider for planning responses to the
> three topic areas include:

-   *For the first topic area, analyze the error data to determine the
    causes and how to reduce the errors going forward*. West Virginia
    completed an analysis that included error elements, error cause and
    CCR&R specific results.

-   *For the second topic area,* *analyze ways that automation could
    have prevented errors detected in the review process and steps that
    the State is taking to address needed changes to the automated
    system*. In response to the second topic area, only two States in
    the second pilot listed that changes were needed to their automated
    systems to reduce improper payments. There are several reasons for
    this limited response. Most of the States in the pilots have
    statewide systems with multiple payment accuracy features. The
    exceptions were New Jersey and Florida. New Jersey plans to improve
    its system's links with other data sources to verify information and
    plans to request additional programming to increase up-front edits
    to reduce data entry errors. Florida is designing and implementing a
    new automated system that will improve the accuracy of the
    eligibility process.

-   *A recommendation related to the third topic area is to describe how
    the State intends to monitor staff performance over time to reduce
    payment errors*. In response to he third topic area, pilot States
    offered several new initiatives including:

    -   New Jersey plans to link error rate findings to contractor
        performance.

    -   West Virginia CCR&R agency Quality Assurance Plans will now
        require inclusion of payment accuracy initiatives if a CCR&R
        agency has an error rate that is above the State mean.

    -   West Virginia child care providers with incorrect or improper
        billing forms must attend a retraining session on proper billing
        procedures.

    -   Florida provided each Early Learning Coalition (ELC) with its
        individual results of the pilot review process along with
        recommendations for corrective action.

> The *State Response Form* questions provide States a starting point in
> the summation of the review data and the resulting corrective action
> strategies. The contractor provided the causes for the errors as an
> example of the types of analyses States may want to complete with the
> data they collect from the record review process. See Chapter IV (pg.
> 36-37) for a summary description of the error causes and reasons
> provided to each State.

[]{#__RefHeading___Toc167079356 .anchor}**D. Modifications to Improve
Reliability across Future Pilot States**

An initial error rate measure provides both the States and the CCB with
a baseline against which to compare future findings. More importantly,
the findings from the two pilots suggest the following recommendations
for revising the pilot methodology in order to achieve a more consistent
review process across States.

-   *Provide for all pilot States formal training on standardizing error
    interpretation and coding.* A detailed review of the error findings
    in the area of missing documentation shows that States had similar
    findings with very dissimilar error results. Great latitude was
    given to the pilot States in defining errors. Some States
    interpreted missing or out-of-date verification as causing
    authorized payment errors, while other States ruled the missing or
    out-of-date verification as a procedural error or no error. This
    inconsistency between reviewer's interpretations also occurred
    within a State's findings.

-   *Establish within each State a process to conduct re-reviews of a
    subset of **all** cases to enhance inter-rater reliability of error
    interpretation*. State training sessions must emphasize reviews of
    sample cases as part of the training process.

-   *Develop consistent standardized policy and procedures as part of
    ongoing or routine auditing processes**. ***The State responses to
    the error findings in both pilots include numerous initiatives to
    improve the quality of case work and the frequency of reviews*.*
    Training to increase staff awareness of error-prone areas, knowledge
    of policy, interviewing skills, and the quality of routine case
    reviews are the most effective strategies States can use to prevent
    or reduce procedural or policy errors.

-   *Focus State review strategies on **both** the most common error
    types and those error causes which produce errors of greater
    amounts, such as ineligibility.* Both pilots' findings suggest that
    there is little, if any, correlation between States' percentage of
    cases in error, the percentage of cases in error that had an
    improperly authorized payment, and the percentage of improperly
    authorized payments. In comparison to New Jersey, Kansas had
    relatively high numbers of cases with errors and cases with an
    improperly authorized payment. Yet the Kansas percentage of
    improperly authorized payments was the lower of the two States. In
    the first pilot, Arkansas had a percentage of improperly authorized
    payments that exceeded both its percentage of cases with errors and
    percentage of cases with errors that had improperly authorized
    payments. For these reasons, State improper payment strategies need
    to focus on both the most common error types and those error causes
    which produce errors of greater amounts, such as ineligibility.

> Although the treatment of missing documentation varied between the
> States, missing documentation was the primary error cause in the
> second pilot and accounted for the largest percentage (57%) of the
> total improperly authorized payments in the first pilot. As an error
> type, missing documentation is not affected by some corrective action
> strategies such as exception reports, system edits, and data matches.
> Training to increase staff knowledge of appropriate documentation and
> policy, coupled with routine case reviews targeted on problem areas
> can assist States in reducing missing documentation errors.

[]{#__RefHeading___Toc167079357 .anchor}**E. Implications for ongoing
Technical Assistance**

> If future pilots do not include consultant assistance, States will
> need to consider accommodations to reflect additional State functions.
> States may choose to incorporate the following:

-   *Work with IT departments or a local university for assistance on
    generating the universe and a random sample of cases for review*.
    Most IT departments have access to programming for random number
    generators for sample selection. When planning the error rate
    methodology, States need to decide on the frequency of sample
    selection and the definition of the universe of cases from which to
    select the sample. For example, how many times per year will the
    sample be drawn and what will be the sample size?

-   *Regarding sample size, consult with a statistician to achieve the
    desired sample size*. For example, California worked with a local
    university to determine a sample size for calculating an error rate
    that met the IPIA guidelines[^21].

-   *For data consolidation, designate a research analyst or program
    specialist who can create a database or an excel spreadsheet that
    includes at a minimum, all of the variables listed in the Data Entry
    Form and several error categories determined to be most important,
    such as the review element, missing or insufficient information,
    data entry errors, or calculation errors*. An *Excel* spreadsheet
    provides all the formulas needed to compute the error measures
    described in the pilot methodology that if combined with an
    electronic *Record Review Worksheet,* can aggregate individual
    review data automatically.

This second pilot demonstrated that improvements to administrative
practice to reduce administrative errors can result in substantially
preventing or deterring improperly authorized payments. Combined with
those of the first pilot, these error findings are promising as a
baseline test of the methodology. The pilot methodology also was useful
for States to gain evaluative information on potential areas for
improvement in administrative policy in the CCDF block grant program.

[]{#__RefHeading___Toc167079358 .anchor}**Appendix A. Florida**

[]{#__RefHeading___Toc167079359 .anchor}**A. Organizational Structure**

In January 2005, legislation established the Office of Early Learning
(OEL) within the Agency for Workforce Innovation (AWI). OEL administers
the child care program in Florida's 67 counties through grant awards
with 31 local Early Learning Coalitions (ELC) and one statewide agency
serving the unique population of migrant, seasonal, and Farm worker
families. The ELCs are quasi-governmental community agencies
incorporated as private, not-for-profit organizations. OEL is
responsible for monitoring the grant awards. In FFY 2005, OEL through
its grant awards provided childcare assistance to over 256,601 children
in 170,497 families through 13,879 providers.

## B. Overview of the Child Care Eligibility Process

The ELCs determine eligibility priorities based on unique populations
within their communities. To be eligible for assistance, a family must
have an income of less than 150 percent of the Federal Poverty Level
(FPL), be a resident of the State, have children under the age of 13 (or
age 19 for special care needs), and be employed, or in an approved
education or training activity. Children may remain eligible provided
income does not exceed 200 percent of FPL. Once eligible, parents may
choose a certificate or a purchase of service subcontract. Families may
use their home, licensed centers or family child care homes.

The child care payment is based on the hours of need determined by the
eligibility process and attendance policy. Each family is assigned a
co-payment determined by income, family size, and a sliding fee schedule
that must be approved by OEL. Co-payments may be waived for protective
services clients on a case by case basis. Benefits are determined prior
to services being rendered. Reauthorization of eligibility is
re-determined on an annual basis. However, as one payment accuracy
strategy, Florida requires the ELCs to reauthorize a random selection of
an additional 50 percent of their caseloads after six months.
Reauthorizations include a re-verification of income, need for care, and
all other eligibility requirements. All clients are responsible for
reporting changes in their income within 10 days of the change.

[]{#__RefHeading___Toc167079361 .anchor}**C. Improper Payment Process**

Florida currently defines an improper payment as the misapplication of
funds resulting from error or intentional acts, or fraud. Categories of
fraud applicable to the child care program include: 1) misrepresentation
of material facts; 2) concealment of material facts; 3) conflicts of
interest; 4) theft of money or property; and 5) statutory offenses.
Recipients of improperly authorized payments may include program clients
(families), program contractors (child care providers/vendors), and
program employees.

Florida identifies improper payments through a variety of methods,
including:

-   ELCs and their contractors are responsible for prevention,
    reduction, identification, reporting, and recuperation of improper
    overpayments. When suspected fraud has occurred, a Suspected Fraud
    Referral is completed and forwarded to the AWI Inspector General,
    OEL and the Florida Department of Law Enforcement. OEL tracks the
    disposition of fraud cases, restitution orders, and receipt of
    restitution payments.

-   OEL fulfills its local coalition monitoring responsibilities through
    several mechanisms. Each ELC must submit a plan for service delivery
    for approval by OEL. Subject to biannual review and revision, ELC
    plans must provide objectives, activities, and measurements for each
    statutory requirement in the Florida School Readiness Act, and the
    Voluntary Pre-kindergarten Education Program and applicable Federal
    requirements. There is also a plan amendment process for submitting
    and approving plan changes. Effective 2007-08, each ELC will
    complete an internal control questionnaire, reviewed annually with
    expanded on-site review procedures on a rotational basis.

-   OEL assigns a coalition analyst to monitor ELC grant award
    agreements and plan adherence, provide technical assistance,
    regularly attend Board meetings, and conduct site visits. In
    conjunction with the coalition analysts, OEL fiscal analysts process
    reimbursement requests on a monthly basis and conduct post audits of
    these requests on a sample basis. Starting in March 2007, OEL
    Services and Support staff will conduct eligibility reviews of
    client files in all 31 ELCs annually to examine the eligibility
    documentation, procedures used and the accuracy of child care
    service payments.

-   OEL's Office of Program Effectiveness and Analysis will conduct a
    six month performance review for each ELC every three years that
    addresses the following areas:

    -   Governance and Operations;

    -   Grant Award and Contract Management;

    -   Community Partnerships;

    -   Educational Service Delivery;

    -   Health & Developmental Screening; and

    -   Child Care Resource & Referral.

```{=html}
<!-- -->
```
-   AWI OEL has contracted with qualified CPA firms licensed to practice
    in the State of Florida to conduct annual financial compliance
    monitoring reviews for the ELCs and selected sub-recipients. AWI OEL
    has developed financial monitoring tools for the ELCs that include,
    but are not limited to, the following compliance categories (as
    applicable):

    -   Financial;

    -   Property;

    -   Procurement;

    -   Matching Requirements;

    -   Prior Approval;

    -   General Conditions; and

    -   Merger Activity.

```{=html}
<!-- -->
```
-   AWI has developed separate monitoring tools for the selected sub
    recipients (statewide contracts, universities, etc.). The CPA
    contractors will conduct an onsite visit to perform the specified
    financial monitoring procedures and will prepare a monitoring report
    for each ELC to identify the procedures performed and any findings
    noted. Financial Management Systems Assurance Section staff will
    review and approve these reports before release to the monitored
    entities.

```{=html}
<!-- -->
```
-   Families applying for financial assistance are required to sign a
    Rights and Responsibilities form that notifies them of their
    responsibility for reporting changes in income or employment. When a
    family fails to report a change and the result is an overpayment,
    the eligibility worker prepares a Suspected Fraud Referral and
    forwards it to AWI's Office of the Inspector General and the Florida
    Department of Law Enforcement.

```{=html}
<!-- -->
```
-   ELCs conduct the following activities to monitor payment accuracy:

```{=html}
<!-- -->
```
-   Complete post attendance monitoring monthly;

-   Audit sign-in and sign-out sheets and the companion attendance
    sheet;

-   Randomly select provider payments and compare them to attendance
    sheets and payment reports;

-   Match the names of divorced or separated parents with the Child
    Support database;

-   Conduct peer review of every completed eligibility file;

-   Conduct supervisory review of 10 percent of the completed cases; and

-   Conduct monthly review of payment overrides.

    []{#__RefHeading___Toc167079362 .anchor}**D. Case Review Process**

In Florida, the Review Team consisted of two Managers, two Performance
Analysts, and three Program Analysts. Florida made substantial
modifications to the *Record Review Worksheet* template. The review
process occurred centrally, with all records sent to Tallahassee. A
second member of the team re-reviewed all records for inter-rater
reliability.

Following completion of the review, Florida learned from a finding in a
IV E Federal audit that only the State's auditor general or DCF's Family
Safety Unit could access case information related to a protective
services placement. The payment accuracy review had found six cases in
error due to missing protective services documentation. Florida
submitted amended review sheets on these six cases that changed the
finding from ineligible to correct.

[]{#__RefHeading___Toc167079363 .anchor}**E. Automation**

There are various automated systems used by OEL. The Child Care Enhanced
Field System (EFS) generates payments, based on days attended and
approved rate schedule. This system also has an eligibility
determination module. The allowance of off-line eligibility calculations
compromises the systems payment accuracy edits.

The EFS processes payments, manages waiting lists, and collects data for
Federal reports and the market rate survey. Local child care staff have
access to other State automated systems, particularly the TANF/Food
Stamp system: One Stop Service Tracking (OSST) varies by local
coalition. In many cases, it depends on co-location of services such as
in a "One Stop" location.

The EFS system generates a selection of management reports to identify
improper payments, including special reports available by request. The
EFS system has an ad hoc report writing capability at the local level.
The ELC network shares local report formats through a centralized e-mail
Q & A system. Examples of existing exception batch reports include:

-   The CCMS Care Level Report identifies children whose age does not
    match the care level.

-   The Duplicate Parent, Child and Provider Report identifies:

    -   Individuals with the same name but different numeric IDs;

    -   Children enrolled, but not paid; and

    -   Clients who have providers in another county.

The EFS has numerous edits and functions that prevent improper payments,
including:

-   Setting and changing of care level based on child's age;

-   Change from pre-school to school care level based on age;

-   Assignment of funding sources;

-   System Edits preventing changes to closed payment periods to
    preserve the integrity of the audit trail;

-   Automated Fee Assessment based on Coalition rules;

-   Edits to ensure rates do not exceed Coalition-defined maximums; and

-   Interface to local accounting systems to reduce data entry errors.

An area of concern for the EFS and the quality of its data is its
decentralized structure. Each of the 31 local coalitions has an
independent section and differing processes. The result is data that is
inconsistent, incompatible, and centrally unavailable. The proposed
enhanced system Early Learning Information System (ELIS) will address
the weaknesses of this decentralized system.

[]{#__RefHeading___Toc167079364 .anchor}**F. Next Steps**

After reviewing the improper payment data Florida implemented the
following strategies to improve payment accuracy:

-   Developed a desk reference tool for eligibility procedures;

-   Initiated focused monitoring and training on eligibility procedures
    on an annual basis;

-   Provided each ELC with its individual results of the pilot review
    process along with recommendations for corrective action;

-   Created mechanisms to track the results of the eligibility
    monitoring and other contract monitoring audits;

-   Implemented an annual eligibility review process of all 31 ELCs;

-   Implemented a Data Quality Initiative to standardize data collection
    and to identify data anomalies throughout the State; and

-   Implemented procedures to conduct post audits on reimbursement
    requests.

    []{#__RefHeading___Toc167079365 .anchor}**Florida Site Visit
    Participants**

Gladys Wilson -- Deputy Director, OEL

Sooni Raymaker -- Interim Executive Administrator of Operations, OEL

Anne Cahn -- Program Services Manager, OEL

Tim Elwell - Program Effectiveness & Analysis Manager, OEL

Cheryl Blanton - Performance Analyst, OEL

Betty Wallace - Performance Analyst, OEL

Courtnie Wheeless -- Performance Analyst, OEL

Ed Hachenberger - Performance Analyst, OEL

John Hughes -- Research and Analysis Supervisor, OEL

Maxine McConnell -- Fiscal Support Unit Manager, OEL

Anne Mulbach -- Government Analyst, OEL

Stephanie Gehres -- Senior Management Analyst, OEL

Tom Imholt -- Coalition Services & Support Manager, OEL

Marlin Seay -- Fiscal/Grant Analyst Lead, OEL

Stephanie Savestanan -- Program Analyst Lead, OEL

Doreen Moskowitz - Program Analyst, OEL

Sam Sweazy - Program Analyst, OEL

Greg Allerellie - Program Analyst, OEL

Barbara Griffin -- Assistant Director, AWI

Phillip Wilcox -- Investigations Manager, Office of the Inspector
General

Diane Parcel -- UC Program Manager, Office of Benefit Payment Control

Bob Pope -- Director, Saber Corp

Abdel Gonzalez -- Senior Manager, Saber Corp

[]{#__RefHeading___Toc167078139 .anchor}**Exhibit A-1. Florida Record
Review Worksheet**

**Child Study ID#** [\_\_\_\_\_\_\_\_\_]{.underline} **ELC:
\_\_\_\_\_\_\_\_\_**

**Child Name/ID# \_\_\_\_\_\_\_\_\_ County: \_\_\_\_\_\_\_\_\_**

**Reviewer \_\_\_\_\_\_\_\_\_\_\_ Date Reviewed: \_\_\_\_\_\_ / \_\_\_ /
\_\_\_\_\_**

+----------------------------------------------+----+---+---+---+---+--------+
| ##                                           |    |   |   |   |   |        |
| ## SECTION I. STATE CHILD CARE PROGRAM FORMS |    |   |   |   |   |        |
+----------------------------------------------+----+---+---+---+---+--------+
| **100 APPLICATION FORMS\                     | AN |   |   |   |   |        |
| **ELEMENTS OF ELIGIBILITY & PAYMENT          | AL |   |   |   |   |        |
| DETERMINATION (1) Determine presence, date,  | YS |   |   |   |   |        |
| and completeness of required eligibility     | IS |   |   |   |   |        |
| forms, may include:                          | OF |   |   |   |   |        |
|                                              | CA |   |   |   |   |        |
|                                              | SE |   |   |   |   |        |
|                                              | RE |   |   |   |   |        |
|                                              | CO |   |   |   |   |        |
|                                              | RD |   |   |   |   |        |
|                                              | (  |   |   |   |   |        |
|                                              | 2) |   |   |   |   |        |
+----------------------------------------------+----+---+---+---+---+--------+
|                                              | Y  | N |   | C |   |        |
|                                              | es | o |   | o |   |        |
|                                              |    |   |   | m |   |        |
|                                              |    |   |   | m |   |        |
|                                              |    |   |   | e |   |        |
|                                              |    |   |   | n |   |        |
|                                              |    |   |   | t |   |        |
|                                              |    |   |   | s |   |        |
+----------------------------------------------+----+---+---+---+---+--------+
| \(1\) Signed and dated application           |    |   |   |   |   |        |
+----------------------------------------------+----+---+---+---+---+--------+
| \(2\) Parental choice form is signed, dated  |    |   |   |   |   |        |
| and current                                  |    |   |   |   |   |        |
+----------------------------------------------+----+---+---+---+---+--------+
| \(3\) Rights and responsibilities form is    |    |   |   |   |   |        |
| signed, dated and current                    |    |   |   |   |   |        |
+----------------------------------------------+----+---+---+---+---+--------+
| \(4\) If applicable, notice of               |    |   |   |   |   |        |
| re-determination/termination form is         |    |   |   |   |   |        |
| complete and current                         |    |   |   |   |   |        |
+----------------------------------------------+----+---+---+---+---+--------+
| \(5\) Referral Form (if applicable)          |    |   |   |   |   |        |
+----------------------------------------------+----+---+---+---+---+--------+
| > Is form complete and current? (including   |    |   |   |   |   |        |
| > applicable authorized service period)      |    |   |   |   |   |        |
+----------------------------------------------+----+---+---+---+---+--------+
| Referring entity and worker are authorized   |    |   |   |   |   |        |
| to submit referrals                          |    |   |   |   |   |        |
+----------------------------------------------+----+---+---+---+---+--------+
| > Reason for referral is valid (purpose for  |    |   |   |   |   |        |
| > care) and documented (Acceptable           |    |   |   |   |   |        |
| > Documentation is listed below)             |    |   |   |   |   |        |
|                                              |    |   |   |   |   |        |
| -   For at-risk/protective service/foster    |    |   |   |   |   |        |
|     care/relative placement cases:           |    |   |   |   |   |        |
|     INFORMATION IS CONFIDENTIAL REFERRAL IS  |    |   |   |   |   |        |
|     SUFFICIENT or;                           |    |   |   |   |   |        |
|                                              |    |   |   |   |   |        |
| -   For Relative Caregiver (RCG) cases:      |    |   |   |   |   |        |
|     Florida system "IQCH" screen documents   |    |   |   |   |   |        |
|     that relative is in receipt of the RCG   |    |   |   |   |   |        |
|     payment. Ensure that payment is RCG      |    |   |   |   |   |        |
|     (usually in the amount of \$242 per      |    |   |   |   |   |        |
|     child) and not regular cash assistance   |    |   |   |   |   |        |
|     payment amount or;                       |    |   |   |   |   |        |
|                                              |    |   |   |   |   |        |
| -   For cash assistance families that are    |    |   |   |   |   |        |
|     subject to work participation            |    |   |   |   |   |        |
|     requirements: FLORIDA system "IQAA"      |    |   |   |   |   |        |
|     screen shows cash assistance case is     |    |   |   |   |   |        |
|     open and "IQEL" screen show individuals  |    |   |   |   |   |        |
|     are eligible and documentation of        |    |   |   |   |   |        |
|     activity is present or;                  |    |   |   |   |   |        |
|                                              |    |   |   |   |   |        |
| -   For transitional child care (TCC) cases: |    |   |   |   |   |        |
|     FLORIDA system "IQCH" screen shows       |    |   |   |   |   |        |
|     recent closure of cash assistance and    |    |   |   |   |   |        |
|     case was closed due to earned income, or |    |   |   |   |   |        |
|     other qualifying factor or;              |    |   |   |   |   |        |
|                                              |    |   |   |   |   |        |
| -   All paperwork is valid and present for   |    |   |   |   |   |        |
|     "diversion" cases. Diversion forms are   |    |   |   |   |   |        |
|     signed and appropriately signed by       |    |   |   |   |   |        |
|     designated workforce individual.         |    |   |   |   |   |        |
+----------------------------------------------+----+---+---+---+---+--------+
| \(6\) If any of the above conditions lead to |    |   |   |   |   |        |
| ineligibility please specify conditions of   |    |   |   |   |   |        |
| the dollar error, including                  |    |   |   |   |   |        |
+----------------------------------------------+----+---+---+---+---+--------+
| \(1\) form expired                           |    |   |   |   |   |        |
+----------------------------------------------+----+---+---+---+---+--------+
| \(2\) No application form                    |    |   |   |   |   |        |
+----------------------------------------------+----+---+---+---+---+--------+
| \(3\) Referral not valid                     |    |   |   |   |   |        |
+----------------------------------------------+----+---+---+---+---+--------+
| FINDINGS (3)                                 |    |   |   |   |   |        |
|                                              |    |   |   |   |   |        |
|                                              |    |   |   |   |   |        |
+----------------------------------------------+----+---+---+---+---+--------+
| Results (4)                                  | No |   | C |   |   | Agency |
|                                              | E  |   | l |   |   | Error  |
|                                              | rr |   | i |   |   | \_     |
|                                              | or |   | e |   |   | \_\_\_ |
|                                              | \_ |   | n |   |   | 2      |
|                                              | \_ |   | t |   |   |        |
|                                              | \_ |   | E |   |   |        |
|                                              | 0  |   | r |   |   |        |
|                                              |    |   | r |   |   |        |
|                                              |    |   | o |   |   |        |
|                                              |    |   | r |   |   |        |
|                                              |    |   | \ |   |   |        |
|                                              |    |   | _ |   |   |        |
|                                              |    |   | \ |   |   |        |
|                                              |    |   | _ |   |   |        |
|                                              |    |   | \ |   |   |        |
|                                              |    |   | _ |   |   |        |
|                                              |    |   | 1 |   |   |        |
+----------------------------------------------+----+---+---+---+---+--------+
| #### SECTION II. PRIORITY GROUP PLACEMENT    |    |   |   |   |   |        |
+----------------------------------------------+----+---+---+---+---+--------+
| **200 PRIORITY GROUP PLACEMENT\              | AN |   |   |   |   |        |
| **ELEMENTS OF ELIGIBILITY & PAYMENT          | AL |   |   |   |   |        |
| DETERMINATION (1)\                           | YS |   |   |   |   |        |
| Determine if client meets criteria of        | IS |   |   |   |   |        |
| State-designated priority groups, e.g.,      | OF |   |   |   |   |        |
| (Florida has determined that placement based | CA |   |   |   |   |        |
| on priority group does not result in an      | SE |   |   |   |   |        |
| improper payment.) NOT APPLICABLE            | RE |   |   |   |   |        |
|                                              | CO |   |   |   |   |        |
|                                              | RD |   |   |   |   |        |
|                                              | (  |   |   |   |   |        |
|                                              | 2) |   |   |   |   |        |
+----------------------------------------------+----+---+---+---+---+--------+
|                                              | Y  | N |   | C |   |        |
|                                              | es | o |   | o |   |        |
|                                              |    |   |   | m |   |        |
|                                              |    |   |   | m |   |        |
|                                              |    |   |   | e |   |        |
|                                              |    |   |   | n |   |        |
|                                              |    |   |   | t |   |        |
|                                              |    |   |   | s |   |        |
+----------------------------------------------+----+---+---+---+---+--------+
| FINDINGS (3)                                 |    |   |   |   |   |        |
|                                              |    |   |   |   |   |        |
|                                              |    |   |   |   |   |        |
+----------------------------------------------+----+---+---+---+---+--------+
| Results (4)                                  | No |   | C |   | A |        |
|                                              | E  |   | l |   | g |        |
|                                              | rr |   | i |   | e |        |
|                                              | or |   | e |   | n |        |
|                                              | \_ |   | n |   | c |        |
|                                              | \_ |   | t |   | y |        |
|                                              | \_ |   | E |   | E |        |
|                                              | 0  |   | r |   | r |        |
|                                              |    |   | r |   | r |        |
|                                              |    |   | o |   | o |        |
|                                              |    |   | r |   | r |        |
|                                              |    |   | \ |   | \ |        |
|                                              |    |   | _ |   | _ |        |
|                                              |    |   | \ |   | \ |        |
|                                              |    |   | _ |   | _ |        |
|                                              |    |   | \ |   | \ |        |
|                                              |    |   | _ |   | _ |        |
|                                              |    |   | 1 |   | \ |        |
|                                              |    |   |   |   | _ |        |
|                                              |    |   |   |   | 2 |        |
+----------------------------------------------+----+---+---+---+---+--------+

+-----------------------------------------------+----+---+---+---+--------+
| #                                             |    |   |   |   |        |
| ### SECTION III. General Program Requirements |    |   |   |   |        |
+-----------------------------------------------+----+---+---+---+--------+
| **300 QUALIFYING HEAD OF HOUSEHOLD\           | AN |   |   |   |        |
| **ELEMENTS OF ELIGIBILITY & PAYMENT           | AL |   |   |   |        |
| DETERMINATION (1)\                            | YS |   |   |   |        |
| Determine if client meets parent definition   | IS |   |   |   |        |
| (parent means a parent by blood, marriage or  | OF |   |   |   |        |
| adoption and also means a legal guardian, or  | CA |   |   |   |        |
| other person standing in loco parentis),      | SE |   |   |   |        |
| e.g., (Note: Identification has been          | RE |   |   |   |        |
| classified as a best practice or good         | CO |   |   |   |        |
| internal control)                             | RD |   |   |   |        |
|                                               | (  |   |   |   |        |
|                                               | 2) |   |   |   |        |
+-----------------------------------------------+----+---+---+---+--------+
|                                               | Y  | N |   | C |        |
|                                               | es | o |   | o |        |
|                                               |    |   |   | m |        |
|                                               |    |   |   | m |        |
|                                               |    |   |   | e |        |
|                                               |    |   |   | n |        |
|                                               |    |   |   | t |        |
|                                               |    |   |   | s |        |
+-----------------------------------------------+----+---+---+---+--------+
| Does the client meet the definition of parent |    |   |   |   |        |
| as defined by Section 98.20, CFR?             |    |   |   |   |        |
+-----------------------------------------------+----+---+---+---+--------+
| > \(1\) parent (identification provided)      |    |   |   |   |        |
+-----------------------------------------------+----+---+---+---+--------+
| > \(2\) step-parent (identification provided) |    |   |   |   |        |
+-----------------------------------------------+----+---+---+---+--------+
| > \(3\) legal guardian (identification        |    |   |   |   |        |
| > provided)                                   |    |   |   |   |        |
+-----------------------------------------------+----+---+---+---+--------+
| > \(4\) relative caretaker (identification    |    |   |   |   |        |
| > provided)                                   |    |   |   |   |        |
+-----------------------------------------------+----+---+---+---+--------+
| > \(5\) spouse of same (identification        |    |   |   |   |        |
| > provided)                                   |    |   |   |   |        |
+-----------------------------------------------+----+---+---+---+--------+
| > \(6\) Other person standing in loco         |    |   |   |   |        |
| > parentis (identification provided) (Note:   |    |   |   |   |        |
| > Defined in Rule 60BB-4.100 as temporary     |    |   |   |   |        |
| > guardian of child)                          |    |   |   |   |        |
+-----------------------------------------------+----+---+---+---+--------+
| FINDINGS (3)                                  |    |   |   |   |        |
|                                               |    |   |   |   |        |
|                                               |    |   |   |   |        |
+-----------------------------------------------+----+---+---+---+--------+
| Results (4)                                   | No |   | C |   | Agency |
|                                               | E  |   | l |   | Error  |
|                                               | rr |   | i |   | \_     |
|                                               | or |   | e |   | \_\_\_ |
|                                               | \_ |   | n |   | 2      |
|                                               | \_ |   | t |   |        |
|                                               | \_ |   | E |   |        |
|                                               | 0  |   | r |   |        |
|                                               |    |   | r |   |        |
|                                               |    |   | o |   |        |
|                                               |    |   | r |   |        |
|                                               |    |   | \ |   |        |
|                                               |    |   | _ |   |        |
|                                               |    |   | \ |   |        |
|                                               |    |   | _ |   |        |
|                                               |    |   | \ |   |        |
|                                               |    |   | _ |   |        |
|                                               |    |   | 1 |   |        |
+-----------------------------------------------+----+---+---+---+--------+

  ----------------------------------------------- ---------- ---- -------- ---------- ----------
  **310 RESIDENCY\                                ANALYSIS                            
  **ELEMENTS OF ELIGIBILITY & PAYMENT             OF CASE                             
  DETERMINATION (1)\                              RECORD (2)                          
  Determine if client is a resident of the State                                      
  and if client is a resident of the county.                                          

                                                  Yes        No            Comments   

  (1)Is the Client is a resident of the State? (a                                     
  client statement indicating residency and                                           
  intent to stay is sufficient for non TANF                                           
  clients) (Section 414.095(2)(a)2, FS                                                
  requirement for TCA clients -- clients are not                                      
  eligible for TCC unless previously served under                                     
  TCA)                                                                                

  \(2\) Is child a US citizen or a qualified                                          
  eligible alien? (Section 414.095(2)(a)1, FS                                         
  requirement for TCA clients -- clients are not                                      
  eligible for TCC unless previously served under                                     
  TCA)                                                                                

  (3)Child care authorized by county in which the                                     
  parent resides? (School readiness funding and                                       
  services are provided based on the county where                                     
  the parent resides)                                                                 

  FINDINGS (3)                                                                        

  RESULTS (4)                                     No Error        Client              Agency
                                                  \_\_\_ 0        Error               Error
                                                                  \_\_\_ 1            \_\_\_\_ 2
  ----------------------------------------------- ---------- ---- -------- ---------- ----------

+-----------------------------------------------+----+---+---+----+-------+
| **320 HOUSEHOLD MEMBERS\                      | AN |   |   |    |       |
| **ELEMENTS OF ELIGIBILITY & PAYMENT           | AL |   |   |    |       |
| DETERMINATION (1)\                            | YS |   |   |    |       |
| To receive services a child\'s parent or      | IS |   |   |    |       |
| parents must be working or attending a job    | OF |   |   |    |       |
| training or educational program.              | CA |   |   |    |       |
|                                               | SE |   |   |    |       |
|                                               | RE |   |   |    |       |
|                                               | CO |   |   |    |       |
|                                               | RD |   |   |    |       |
|                                               | (  |   |   |    |       |
|                                               | 2) |   |   |    |       |
+-----------------------------------------------+----+---+---+----+-------+
|                                               | Y  | N |   | Co |       |
|                                               | es | o |   | mm |       |
|                                               |    |   |   | en |       |
|                                               |    |   |   | ts |       |
|                                               |    |   |   |    |       |
|                                               |    |   |   | \[ |       |
|                                               |    |   |   | Co |       |
|                                               |    |   |   | mm |       |
|                                               |    |   |   | en |       |
|                                               |    |   |   | ts |       |
|                                               |    |   |   | sh |       |
|                                               |    |   |   | ou |       |
|                                               |    |   |   | ld |       |
|                                               |    |   |   | i  |       |
|                                               |    |   |   | nc |       |
|                                               |    |   |   | lu |       |
|                                               |    |   |   | de |       |
|                                               |    |   |   | b  |       |
|                                               |    |   |   | il |       |
|                                               |    |   |   | li |       |
|                                               |    |   |   | ng |       |
|                                               |    |   |   | g  |       |
|                                               |    |   |   | ro |       |
|                                               |    |   |   | up |       |
|                                               |    |   |   | a  |       |
|                                               |    |   |   | nd |       |
|                                               |    |   |   | e  |       |
|                                               |    |   |   | li |       |
|                                               |    |   |   | gi |       |
|                                               |    |   |   | bi |       |
|                                               |    |   |   | li |       |
|                                               |    |   |   | ty |       |
|                                               |    |   |   | co |       |
|                                               |    |   |   | de |       |
|                                               |    |   |   | r  |       |
|                                               |    |   |   | ei |       |
|                                               |    |   |   | mb |       |
|                                               |    |   |   | ur |       |
|                                               |    |   |   | se |       |
|                                               |    |   |   | me |       |
|                                               |    |   |   | nt |       |
|                                               |    |   |   | w  |       |
|                                               |    |   |   | as |       |
|                                               |    |   |   | ma |       |
|                                               |    |   |   | de |       |
|                                               |    |   |   | u  |       |
|                                               |    |   |   | nd |       |
|                                               |    |   |   | er |       |
|                                               |    |   |   | \] |       |
+-----------------------------------------------+----+---+---+----+-------+
| (1)Does the client meet one of the following  |    |   |   |    |       |
| criteria?                                     |    |   |   |    |       |
+-----------------------------------------------+----+---+---+----+-------+
| > Employment (working a minimum of 20 hours   |    |   |   |    |       |
| > per week. Purpose of Care established)      |    |   |   |    |       |
+-----------------------------------------------+----+---+---+----+-------+
| > Attending a job training or educational     |    |   |   |    |       |
| > program. (Minimum of 20 hours per week.     |    |   |   |    |       |
| > Purpose of Care established)                |    |   |   |    |       |
+-----------------------------------------------+----+---+---+----+-------+
| > Seeking employment (documentation supported |    |   |   |    |       |
| > and time limited)                           |    |   |   |    |       |
+-----------------------------------------------+----+---+---+----+-------+
| > Child only under the age of 16 (TANF)       |    |   |   |    |       |
| > "family of one" -- adult must meet purpose  |    |   |   |    |       |
| > for care                                    |    |   |   |    |       |
+-----------------------------------------------+----+---+---+----+-------+
| > Disability: An individual who receives      |    |   |   |    |       |
| > benefits under the SSI or the SSDI program. |    |   |   |    |       |
+-----------------------------------------------+----+---+---+----+-------+
| > Protective Services Case (supporting        |    |   |   |    |       |
| > documentation is present)                   |    |   |   |    |       |
+-----------------------------------------------+----+---+---+----+-------+
| > TANF Respite Child Care                     |    |   |   |    |       |
+-----------------------------------------------+----+---+---+----+-------+
| > Child within Natural Disaster Area, and is  |    |   |   |    |       |
| > in need of protection to ensure child's     |    |   |   |    |       |
| > safety                                      |    |   |   |    |       |
+-----------------------------------------------+----+---+---+----+-------+
| > Other : State GR Funded Only                |    |   |   |    |       |
| > \_\_\_\_\_\_\                               |    |   |   |    |       |
| _\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ |    |   |   |    |       |
+-----------------------------------------------+----+---+---+----+-------+
| (2)Is the child classified in the correct     |    |   |   |    |       |
| billing group and eligibility code? (See      |    |   |   |    |       |
| chart for applicable billing group and        |    |   |   |    |       |
| eligibility codes. Note: this will not result |    |   |   |    |       |
| in an improper payment unless the billing     |    |   |   |    |       |
| group should have been one that is funding by |    |   |   |    |       |
| State GR Only)                                |    |   |   |    |       |
+-----------------------------------------------+----+---+---+----+-------+
| FINDINGS (3)                                  |    |   |   |    |       |
|                                               |    |   |   |    |       |
|                                               |    |   |   |    |       |
+-----------------------------------------------+----+---+---+----+-------+
| > Results (4)                                 | No |   | C |    | A     |
|                                               | E  |   | l |    | gency |
|                                               | rr |   | i |    | Error |
|                                               | or |   | e |    | \_\   |
|                                               | \_ |   | n |    | _\_\_ |
|                                               | \_ |   | t |    | 2     |
|                                               | \_ |   | E |    |       |
|                                               | 0  |   | r |    |       |
|                                               |    |   | r |    |       |
|                                               |    |   | o |    |       |
|                                               |    |   | r |    |       |
|                                               |    |   | \ |    |       |
|                                               |    |   | _ |    |       |
|                                               |    |   | \ |    |       |
|                                               |    |   | _ |    |       |
|                                               |    |   | \ |    |       |
|                                               |    |   | _ |    |       |
|                                               |    |   | 1 |    |       |
+-----------------------------------------------+----+---+---+----+-------+

+-----------------------------------------------+---+---+---+---+----+---+-------+
| **330 QUALIFYING CHILD\                       | A |   |   |   |    |   |       |
| **ELEMENTS OF ELIGIBILITY & PAYMENT           | N |   |   |   |    |   |       |
| DETERMINATION (1)\                            | A |   |   |   |    |   |       |
| Determine if child (ren) is eligible for      | L |   |   |   |    |   |       |
| services, including                           | Y |   |   |   |    |   |       |
|                                               | S |   |   |   |    |   |       |
|                                               | I |   |   |   |    |   |       |
|                                               | S |   |   |   |    |   |       |
|                                               | O |   |   |   |    |   |       |
|                                               | F |   |   |   |    |   |       |
|                                               | C |   |   |   |    |   |       |
|                                               | A |   |   |   |    |   |       |
|                                               | S |   |   |   |    |   |       |
|                                               | E |   |   |   |    |   |       |
|                                               | R |   |   |   |    |   |       |
|                                               | E |   |   |   |    |   |       |
|                                               | C |   |   |   |    |   |       |
|                                               | O |   |   |   |    |   |       |
|                                               | R |   |   |   |    |   |       |
|                                               | D |   |   |   |    |   |       |
|                                               | ( |   |   |   |    |   |       |
|                                               | 2 |   |   |   |    |   |       |
|                                               | ) |   |   |   |    |   |       |
+-----------------------------------------------+---+---+---+---+----+---+-------+
|                                               | Y |   | N |   | Co |   |       |
|                                               | e |   | o |   | mm |   |       |
|                                               | s |   |   |   | en |   |       |
|                                               |   |   |   |   | ts |   |       |
+-----------------------------------------------+---+---+---+---+----+---+-------+
| Is the child                                  |   |   |   |   |    |   |       |
|                                               |   |   |   |   |    |   |       |
| > \(1\) younger than 13 years (Proof of Age   |   |   |   |   |    |   |       |
| > Included -- see FPSR-PI-PPA-04-08) or       |   |   |   |   |    |   |       |
| >                                             |   |   |   |   |    |   |       |
| > \(2\) younger than 19 years and physically  |   |   |   |   |    |   |       |
| > or mentally incapable of caring for himself |   |   |   |   |    |   |       |
| > or herself, or under court supervision      |   |   |   |   |    |   |       |
| > (proof of age on file -- see                |   |   |   |   |    |   |       |
| > FPSR-PI-PPA-04-08; verification of physical |   |   |   |   |    |   |       |
| > or mental incapacity or court supervision   |   |   |   |   |    |   |       |
| > is on file)                                 |   |   |   |   |    |   |       |
+-----------------------------------------------+---+---+---+---+----+---+-------+
| FINDINGS (3)                                  |   |   |   |   |    |   |       |
|                                               |   |   |   |   |    |   |       |
|                                               |   |   |   |   |    |   |       |
+-----------------------------------------------+---+---+---+---+----+---+-------+
| > Results (4)                                 |   | N |   | C |    |   | A     |
|                                               |   | o |   | l |    |   | gency |
|                                               |   | E |   | i |    |   | Error |
|                                               |   | r |   | e |    |   | \_\   |
|                                               |   | r |   | n |    |   | _\_\_ |
|                                               |   | o |   | t |    |   | 2     |
|                                               |   | r |   | E |    |   |       |
|                                               |   | \ |   | r |    |   |       |
|                                               |   | _ |   | r |    |   |       |
|                                               |   | \ |   | o |    |   |       |
|                                               |   | _ |   | r |    |   |       |
|                                               |   | \ |   | \ |    |   |       |
|                                               |   | _ |   | _ |    |   |       |
|                                               |   | 0 |   | \ |    |   |       |
|                                               |   |   |   | _ |    |   |       |
|                                               |   |   |   | \ |    |   |       |
|                                               |   |   |   | _ |    |   |       |
|                                               |   |   |   | 1 |    |   |       |
+-----------------------------------------------+---+---+---+---+----+---+-------+
| **340 QUALIFYING CARE\                        | A |   |   |   |    |   |       |
| **ELEMENTS OF ELIGIBILITY & PAYMENT           | N |   |   |   |    |   |       |
| DETERMINATION (1)                             | A |   |   |   |    |   |       |
|                                               | L |   |   |   |    |   |       |
|                                               | Y |   |   |   |    |   |       |
|                                               | S |   |   |   |    |   |       |
|                                               | I |   |   |   |    |   |       |
|                                               | S |   |   |   |    |   |       |
|                                               | O |   |   |   |    |   |       |
|                                               | F |   |   |   |    |   |       |
|                                               | C |   |   |   |    |   |       |
|                                               | A |   |   |   |    |   |       |
|                                               | S |   |   |   |    |   |       |
|                                               | E |   |   |   |    |   |       |
|                                               | R |   |   |   |    |   |       |
|                                               | E |   |   |   |    |   |       |
|                                               | C |   |   |   |    |   |       |
|                                               | O |   |   |   |    |   |       |
|                                               | R |   |   |   |    |   |       |
|                                               | D |   |   |   |    |   |       |
|                                               | ( |   |   |   |    |   |       |
|                                               | 2 |   |   |   |    |   |       |
|                                               | ) |   |   |   |    |   |       |
+-----------------------------------------------+---+---+---+---+----+---+-------+
|                                               | Y |   | N |   | Co |   |       |
|                                               | e |   | o |   | mm |   |       |
|                                               | s |   |   |   | en |   |       |
|                                               |   |   |   |   | ts |   |       |
+-----------------------------------------------+---+---+---+---+----+---+-------+
| Does the required number of hours of care     |   |   |   |   |    |   |       |
| included in the authorized schedule meet the  |   |   |   |   |    |   |       |
| definition of the unit of care (i.e., amount  |   |   |   |   |    |   |       |
| of care authorized is related to activity     |   |   |   |   |    |   |       |
| plus reasonable travel time)? (Care may be    |   |   |   |   |    |   |       |
| authorized as part-time, full-time or based   |   |   |   |   |    |   |       |
| on other factors that the coalition has put   |   |   |   |   |    |   |       |
| in place (e.g., part-time "light" for before  |   |   |   |   |    |   |       |
| and/or after school care))                    |   |   |   |   |    |   |       |
+-----------------------------------------------+---+---+---+---+----+---+-------+
| FINDINGS (3)                                  |   |   |   |   |    |   |       |
|                                               |   |   |   |   |    |   |       |
|                                               |   |   |   |   |    |   |       |
+-----------------------------------------------+---+---+---+---+----+---+-------+
| > Results (4)                                 |   | N |   | C |    | A |       |
|                                               |   | o |   | l |    | g |       |
|                                               |   | E |   | i |    | e |       |
|                                               |   | r |   | e |    | n |       |
|                                               |   | r |   | n |    | c |       |
|                                               |   | o |   | t |    | y |       |
|                                               |   | r |   | E |    | E |       |
|                                               |   | \ |   | r |    | r |       |
|                                               |   | _ |   | r |    | r |       |
|                                               |   | \ |   | o |    | o |       |
|                                               |   | _ |   | r |    | r |       |
|                                               |   | \ |   | \ |    | \ |       |
|                                               |   | _ |   | _ |    | _ |       |
|                                               |   | 0 |   | \ |    | \ |       |
|                                               |   |   |   | _ |    | _ |       |
|                                               |   |   |   | \ |    | \ |       |
|                                               |   |   |   | _ |    | _ |       |
|                                               |   |   |   | 1 |    | \ |       |
|                                               |   |   |   |   |    | _ |       |
|                                               |   |   |   |   |    | 2 |       |
+-----------------------------------------------+---+---+---+---+----+---+-------+

+-----------------------------------------------+---+---+---+---+----+-------+
| **350 QUALIFYING PROVIDER ARRANGEMENT\        | A |   |   |   |    |       |
| **ELEMENTS OF ELIGIBILITY & PAYMENT           | N |   |   |   |    |       |
| DETERMINATION (1)\                            | A |   |   |   |    |       |
| Determine if services are provided within a   | L |   |   |   |    |       |
| legally operating provider type. (Provider of | Y |   |   |   |    |       |
| child care services must be licensed, license | S |   |   |   |    |       |
| exempt, or registered under applicable State  | I |   |   |   |    |       |
| of local law )                                | S |   |   |   |    |       |
|                                               | O |   |   |   |    |       |
|                                               | F |   |   |   |    |       |
|                                               | C |   |   |   |    |       |
|                                               | A |   |   |   |    |       |
|                                               | S |   |   |   |    |       |
|                                               | E |   |   |   |    |       |
|                                               | R |   |   |   |    |       |
|                                               | E |   |   |   |    |       |
|                                               | C |   |   |   |    |       |
|                                               | O |   |   |   |    |       |
|                                               | R |   |   |   |    |       |
|                                               | D |   |   |   |    |       |
|                                               | ( |   |   |   |    |       |
|                                               | 2 |   |   |   |    |       |
|                                               | ) |   |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+
|                                               | Y |   | N |   | Co |       |
|                                               | e |   | o |   | mm |       |
|                                               | s |   |   |   | en |       |
|                                               |   |   |   |   | ts |       |
+-----------------------------------------------+---+---+---+---+----+-------+
| Is the Provider a legally operating provider  |   |   |   |   |    |       |
| type? (Types allowable under Section 98, CFR  |   |   |   |   |    |       |
| listed below, Karate Schools and Ballet       |   |   |   |   |    |       |
| Schools would be examples of unallowable      |   |   |   |   |    |       |
| providers)                                    |   |   |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+
| > \(1\) Center-based child care provider      |   |   |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+
| > \(2\) family child care provider            |   |   |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+
| > \(3\) in-home child care provider           |   |   |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+
| > \(4\) informal provider                     |   |   |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+
| FINDINGS (3)                                  |   |   |   |   |    |       |
|                                               |   |   |   |   |    |       |
|                                               |   |   |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+
| > Results (4)                                 |   | N |   | C |    | A     |
|                                               |   | o |   | l |    | gency |
|                                               |   | E |   | i |    | Error |
|                                               |   | r |   | e |    | \_\   |
|                                               |   | r |   | n |    | _\_\_ |
|                                               |   | o |   | t |    | 2     |
|                                               |   | r |   | E |    |       |
|                                               |   | [ |   | r |    |       |
|                                               |   | \ |   | r |    |       |
|                                               |   | _ |   | o |    |       |
|                                               |   | ] |   | r |    |       |
|                                               |   | { |   | \ |    |       |
|                                               |   | . |   | _ |    |       |
|                                               |   | u |   | \ |    |       |
|                                               |   | n |   | _ |    |       |
|                                               |   | d |   | \ |    |       |
|                                               |   | e |   | _ |    |       |
|                                               |   | r |   | 1 |    |       |
|                                               |   | l |   |   |    |       |
|                                               |   | i |   |   |    |       |
|                                               |   | n |   |   |    |       |
|                                               |   | e |   |   |    |       |
|                                               |   | } |   |   |    |       |
|                                               |   | \ |   |   |    |       |
|                                               |   | _ |   |   |    |       |
|                                               |   | \ |   |   |    |       |
|                                               |   | _ |   |   |    |       |
|                                               |   | 0 |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+

+-----------------------------------------------+---+---+---+---+----+-------+
| **360 PROVIDER REQUIREMENTS\                  | A |   |   |   |    |       |
| **ELEMENTS OF ELIGIBILITY & PAYMENT           | N |   |   |   |    |       |
| DETERMINATION (1)                             | A |   |   |   |    |       |
|                                               | L |   |   |   |    |       |
|                                               | Y |   |   |   |    |       |
|                                               | S |   |   |   |    |       |
|                                               | I |   |   |   |    |       |
|                                               | S |   |   |   |    |       |
|                                               | O |   |   |   |    |       |
|                                               | F |   |   |   |    |       |
|                                               | C |   |   |   |    |       |
|                                               | A |   |   |   |    |       |
|                                               | S |   |   |   |    |       |
|                                               | E |   |   |   |    |       |
|                                               | R |   |   |   |    |       |
|                                               | E |   |   |   |    |       |
|                                               | C |   |   |   |    |       |
|                                               | O |   |   |   |    |       |
|                                               | R |   |   |   |    |       |
|                                               | D |   |   |   |    |       |
|                                               | ( |   |   |   |    |       |
|                                               | 2 |   |   |   |    |       |
|                                               | ) |   |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+
|                                               | Y |   | N |   | Co |       |
|                                               | e |   | o |   | mm |       |
|                                               | s |   |   |   | en |       |
|                                               |   |   |   |   | ts |       |
+-----------------------------------------------+---+---+---+---+----+-------+
| (1)Determine if regulatory requirements are   |   |   |   |   |    |       |
| met. Regulatory requirements means            |   |   |   |   |    |       |
| requirements necessary for a provider to      |   |   |   |   |    |       |
| legally provide child care services in a      |   |   |   |   |    |       |
| State or locality, including registration     |   |   |   |   |    |       |
| requirements established under State, local,  |   |   |   |   |    |       |
| or tribal law. (Each Coalition should         |   |   |   |   |    |       |
| document in file that the provider meets the  |   |   |   |   |    |       |
| regulatory requirements; an example would be  |   |   |   |   |    |       |
| DCF Form 6920)                                |   |   |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+
| (2)Verify if the provider license is current? |   |   |   |   |    |       |
| (Review CCIS for applicable dates)            |   |   |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+
| (3)Determine if the provider is a gold seal   |   |   |   |   |    |       |
| provider? (Review CCIS)                       |   |   |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+
| FINDINGS (3)                                  |   |   |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+
| > Results (4)                                 |   | N |   | C |    | A     |
|                                               |   | o |   | l |    | gency |
|                                               |   | E |   | i |    | Error |
|                                               |   | r |   | e |    | \_\_2 |
|                                               |   | r |   | n |    |       |
|                                               |   | o |   | t |    |       |
|                                               |   | r |   | E |    |       |
|                                               |   | \ |   | r |    |       |
|                                               |   | _ |   | r |    |       |
|                                               |   | [ |   | o |    |       |
|                                               |   | \ |   | r |    |       |
|                                               |   | _ |   | \ |    |       |
|                                               |   | ] |   | _ |    |       |
|                                               |   | { |   | \ |    |       |
|                                               |   | . |   | _ |    |       |
|                                               |   | u |   | \ |    |       |
|                                               |   | n |   | _ |    |       |
|                                               |   | d |   | 1 |    |       |
|                                               |   | e |   |   |    |       |
|                                               |   | r |   |   |    |       |
|                                               |   | l |   |   |    |       |
|                                               |   | i |   |   |    |       |
|                                               |   | n |   |   |    |       |
|                                               |   | e |   |   |    |       |
|                                               |   | } |   |   |    |       |
|                                               |   | \ |   |   |    |       |
|                                               |   | _ |   |   |    |       |
|                                               |   | 0 |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+

+-----------------------------------------------+---+---+---+---+----+-------+
| #### SECTION IV. INCOME AND PAYMENTS          |   |   |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+
| **400 INCOME\                                 | A |   |   |   |    |       |
| **ELEMENTS OF ELIGIBILITY & PAYMENT           | N |   |   |   |    |       |
| DETERMINATION (1) Describe income             | A |   |   |   |    |       |
| documentation verification for each member of | L |   |   |   |    |       |
| the household. Calculation of income is to be | Y |   |   |   |    |       |
| based on six weeks of gross income for period | S |   |   |   |    |       |
| prior to application/redetermination: Collect | I |   |   |   |    |       |
| the following data:                           | S |   |   |   |    |       |
|                                               | O |   |   |   |    |       |
|                                               | F |   |   |   |    |       |
|                                               | C |   |   |   |    |       |
|                                               | A |   |   |   |    |       |
|                                               | S |   |   |   |    |       |
|                                               | E |   |   |   |    |       |
|                                               | R |   |   |   |    |       |
|                                               | E |   |   |   |    |       |
|                                               | C |   |   |   |    |       |
|                                               | O |   |   |   |    |       |
|                                               | R |   |   |   |    |       |
|                                               | D |   |   |   |    |       |
|                                               | ( |   |   |   |    |       |
|                                               | 2 |   |   |   |    |       |
|                                               | ) |   |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+
|                                               | Y |   | N |   | Co |       |
|                                               | e |   | o |   | mm |       |
|                                               | s |   |   |   | en |       |
|                                               |   |   |   |   | ts |       |
+-----------------------------------------------+---+---+---+---+----+-------+
| \(1\) Head of household employment income     |   |   |   |   |    |       |
| and/or spouse employment income (Per          |   |   |   |   |    |       |
| 60BB-4.208 shall include, at a minimum,       |   |   |   |   |    |       |
| documentation by hours of employment and rate |   |   |   |   |    |       |
| of pay based on: Six weekly, three biweekly,  |   |   |   |   |    |       |
| or two monthly pay stubs that are current and |   |   |   |   |    |       |
| consecutive, or a signed statement by the     |   |   |   |   |    |       |
| employer, or a signed contract for            |   |   |   |   |    |       |
| employment.)                                  |   |   |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+
| \(2\) Self employment (Per 60BB-4.208         |   |   |   |   |    |       |
| provides that documentation be sufficient to  |   |   |   |   |    |       |
| determine hours worked and income, such as    |   |   |   |   |    |       |
| business account ledgers, written             |   |   |   |   |    |       |
| documentation from customers, contractors, or |   |   |   |   |    |       |
| federal tax returns)                          |   |   |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+
| \(3\) any changes in income reported (clients |   |   |   |   |    |       |
| are required to report changes in income)     |   |   |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+
| \(4\) income during job training for          |   |   |   |   |    |       |
| parent/caregiver                              |   |   |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+
| \(5\) child support received/alimony received |   |   |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+
| \(6\) loss of income during eligibility       |   |   |   |   |    |       |
| period.                                       |   |   |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+
| \(7\) Cash assistance benefits (TANF)         |   |   |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+
| \(8\) Other countable income documented (SSI, |   |   |   |   |    |       |
| TANF, Spousal Support, gifts, etc.)           |   |   |   |   |    |       |
|                                               |   |   |   |   |    |       |
| -   Income is defined as "family income," and |   |   |   |   |    |       |
|     means the combined gross income, from all |   |   |   |   |    |       |
|     sources, of all members of the family     |   |   |   |   |    |       |
|     unit who are eighteen (18) years of age   |   |   |   |   |    |       |
|     or older, including earned and unearned   |   |   |   |   |    |       |
|     income, and excluding the following:      |   |   |   |   |    |       |
|                                               |   |   |   |   |    |       |
| ```{=html}                                    |   |   |   |   |    |       |
| <!-- -->                                      |   |   |   |   |    |       |
| ```                                           |   |   |   |   |    |       |
| -   Food stamp benefits;                      |   |   |   |   |    |       |
|                                               |   |   |   |   |    |       |
| -   Documented child support payments paid to |   |   |   |   |    |       |
|     an individual outside of the home;        |   |   |   |   |    |       |
|                                               |   |   |   |   |    |       |
| -   Documented alimony payments being paid to |   |   |   |   |    |       |
|     an individual outside the home; and       |   |   |   |   |    |       |
|                                               |   |   |   |   |    |       |
| -   Housing assistance payments from HUD      |   |   |   |   |    |       |
|     issued directly to a landlord and         |   |   |   |   |    |       |
|     associated utilities expenses.            |   |   |   |   |    |       |
|                                               |   |   |   |   |    |       |
|     -   *As foster parents, shelter parents,  |   |   |   |   |    |       |
|         and court ordered relative and        |   |   |   |   |    |       |
|         non-relative caregivers are not       |   |   |   |   |    |       |
|         considered a part of the child's      |   |   |   |   |    |       |
|         family unit, their income is not      |   |   |   |   |    |       |
|         considered for purposes of            |   |   |   |   |    |       |
|         eligibility.*                         |   |   |   |   |    |       |
|                                               |   |   |   |   |    |       |
| Any exceptions or variations should be        |   |   |   |   |    |       |
| documented and explained in the case notes    |   |   |   |   |    |       |
| section.                                      |   |   |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+
| FINDINGS (3)                                  |   |   |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+
| > Results (4)                                 |   | N |   | C |    | A     |
|                                               |   | o |   | l |    | gency |
|                                               |   | E |   | i |    | Error |
|                                               |   | r |   | e |    | \_\   |
|                                               |   | r |   | n |    | _\_\_ |
|                                               |   | o |   | t |    | 2     |
|                                               |   | r |   | E |    |       |
|                                               |   | \ |   | r |    |       |
|                                               |   | _ |   | r |    |       |
|                                               |   | \ |   | o |    |       |
|                                               |   | _ |   | r |    |       |
|                                               |   | 0 |   | \ |    |       |
|                                               |   |   |   | _ |    |       |
|                                               |   |   |   | \ |    |       |
|                                               |   |   |   | _ |    |       |
|                                               |   |   |   | \ |    |       |
|                                               |   |   |   | _ |    |       |
|                                               |   |   |   | 1 |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+

+-----------------------------------------------+---+---+---+---+----+-------+
| **410 INCOME ELIGIBILITY\                     | A |   |   |   |    |       |
| **ELEMENTS OF ELIGIBILITY & PAYMENT           | N |   |   |   |    |       |
| DETERMINATION (1)                             | A |   |   |   |    |       |
|                                               | L |   |   |   |    |       |
|                                               | Y |   |   |   |    |       |
|                                               | S |   |   |   |    |       |
|                                               | I |   |   |   |    |       |
|                                               | S |   |   |   |    |       |
|                                               | O |   |   |   |    |       |
|                                               | F |   |   |   |    |       |
|                                               | C |   |   |   |    |       |
|                                               | A |   |   |   |    |       |
|                                               | S |   |   |   |    |       |
|                                               | E |   |   |   |    |       |
|                                               | R |   |   |   |    |       |
|                                               | E |   |   |   |    |       |
|                                               | C |   |   |   |    |       |
|                                               | O |   |   |   |    |       |
|                                               | R |   |   |   |    |       |
|                                               | D |   |   |   |    |       |
|                                               | ( |   |   |   |    |       |
|                                               | 2 |   |   |   |    |       |
|                                               | ) |   |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+
|                                               | Y |   | N |   | Co |       |
|                                               | e |   | o |   | mm |       |
|                                               | s |   |   |   | en |       |
|                                               |   |   |   |   | ts |       |
+-----------------------------------------------+---+---+---+---+----+-------+
| Determine if household income meets State     |   |   |   |   |    |       |
| requirements (e.g., family gross income must  |   |   |   |   |    |       |
| be at or below 150% of the FPL for families   |   |   |   |   |    |       |
| to enter and 200% to remain eligible). NOTE:  |   |   |   |   |    |       |
| For coalitions participating in the Child     |   |   |   |   |    |       |
| Care Executive Partnership (CCEP), families   |   |   |   |   |    |       |
| may enter with income at or below 200% of the |   |   |   |   |    |       |
| FPL. (Use Spreadsheet provided for            |   |   |   |   |    |       |
| calculation)                                  |   |   |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+
| FINDINGS (3)                                  |   |   |   |   |    |       |
|                                               |   |   |   |   |    |       |
|                                               |   |   |   |   |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+
| > Results (4)                                 |   | N |   | C |    | A     |
|                                               |   | o |   | l |    | gency |
|                                               |   | E |   | i |    | Error |
|                                               |   | r |   | e |    | \_\   |
|                                               |   | r |   | n |    | _\_\_ |
|                                               |   | o |   | t |    | 2     |
|                                               |   | r |   | E |    |       |
|                                               |   | \ |   | r |    |       |
|                                               |   | _ |   | r |    |       |
|                                               |   | \ |   | o |    |       |
|                                               |   | _ |   | r |    |       |
|                                               |   | 0 |   | \ |    |       |
|                                               |   |   |   | _ |    |       |
|                                               |   |   |   | \ |    |       |
|                                               |   |   |   | _ |    |       |
|                                               |   |   |   | \ |    |       |
|                                               |   |   |   | _ |    |       |
|                                               |   |   |   | 1 |    |       |
+-----------------------------------------------+---+---+---+---+----+-------+

+-----------------------------------------------+---+---+---+---+---+--------+
| **420 PAYMENTS, GENERAL\                      | A |   |   |   |   |        |
| **ELEMENTS OF ELIGIBILITY & PAYMENT           | N |   |   |   |   |        |
| DETERMINATION (1)                             | A |   |   |   |   |        |
|                                               | L |   |   |   |   |        |
| Determine if payments were made. Parent fees  | Y |   |   |   |   |        |
| are established on a sliding fee scale based  | S |   |   |   |   |        |
| on income and the size of the family and may  | I |   |   |   |   |        |
| be based on other factors as appropriate.     | S |   |   |   |   |        |
|                                               | O |   |   |   |   |        |
|                                               | F |   |   |   |   |        |
|                                               | C |   |   |   |   |        |
|                                               | A |   |   |   |   |        |
|                                               | S |   |   |   |   |        |
|                                               | E |   |   |   |   |        |
|                                               | R |   |   |   |   |        |
|                                               | E |   |   |   |   |        |
|                                               | C |   |   |   |   |        |
|                                               | O |   |   |   |   |        |
|                                               | R |   |   |   |   |        |
|                                               | D |   |   |   |   |        |
|                                               | ( |   |   |   |   |        |
|                                               | 2 |   |   |   |   |        |
|                                               | ) |   |   |   |   |        |
+-----------------------------------------------+---+---+---+---+---+--------+
|                                               | Y |   | N |   | C |        |
|                                               | e |   | o |   | o |        |
|                                               | s |   |   |   | m |        |
|                                               |   |   |   |   | m |        |
|                                               |   |   |   |   | e |        |
|                                               |   |   |   |   | n |        |
|                                               |   |   |   |   | t |        |
|                                               |   |   |   |   | s |        |
+-----------------------------------------------+---+---+---+---+---+--------+
| \(1\) Calculate amount of parent fee based on |   |   |   |   |   |        |
| household income, family size and the ELC's   |   |   |   |   |   |        |
| sliding fee scale. Does this agree with       |   |   |   |   |   |        |
| amount included in reimbursement records?     |   |   |   |   |   |        |
| (Each coalition's sliding fee scale provided) |   |   |   |   |   |        |
+-----------------------------------------------+---+---+---+---+---+--------+
| \(2\) Determine provider payment rate based   |   |   |   |   |   |        |
| on provider type, age of child, care schedule |   |   |   |   |   |        |
| and the ELC's approved provider rate          |   |   |   |   |   |        |
| schedule. Is the provider rate less than or   |   |   |   |   |   |        |
| equal to the approved reimbursement rate and  |   |   |   |   |   |        |
| the private pay rate? (Each Coalition's       |   |   |   |   |   |        |
| Reimbursement Rates provided, review file for |   |   |   |   |   |        |
| private pay rate; the private pay rate may    |   |   |   |   |   |        |
| need to be requested from the Coalition or    |   |   |   |   |   |        |
| CCRR)                                         |   |   |   |   |   |        |
+-----------------------------------------------+---+---+---+---+---+--------+
| \(3\) Does the Final Provider Reimbursement   |   |   |   |   |   |        |
| Report agree with the Provider's Attendance   |   |   |   |   |   |        |
| Sheet? (Note: if there are any holidays       |   |   |   |   |   |        |
| confirm with coalition\'s approved schedule)  |   |   |   |   |   |        |
+-----------------------------------------------+---+---+---+---+---+--------+
| \(4\) Is the Providers Attendance Sheet       |   |   |   |   |   |        |
| signed?                                       |   |   |   |   |   |        |
+-----------------------------------------------+---+---+---+---+---+--------+
| FINDINGS (3)                                  |   |   |   |   |   |        |
+-----------------------------------------------+---+---+---+---+---+--------+
| > Results (4)                                 |   | N |   | C |   | Agency |
|                                               |   | o |   | l |   | Error  |
|                                               |   | E |   | i |   | \_     |
|                                               |   | r |   | e |   | \_\_\_ |
|                                               |   | r |   | n |   | 2      |
|                                               |   | o |   | t |   |        |
|                                               |   | r |   | E |   |        |
|                                               |   | \ |   | r |   |        |
|                                               |   | _ |   | r |   |        |
|                                               |   | \ |   | o |   |        |
|                                               |   | _ |   | r |   |        |
|                                               |   | \ |   | \ |   |        |
|                                               |   | _ |   | _ |   |        |
|                                               |   | 0 |   | \ |   |        |
|                                               |   |   |   | _ |   |        |
|                                               |   |   |   | \ |   |        |
|                                               |   |   |   | _ |   |        |
|                                               |   |   |   | 1 |   |        |
+-----------------------------------------------+---+---+---+---+---+--------+

+-----------------------------------------------+----+---+---+---+--------+
| **430 PAYMENTS, COMPUTATIONS\                 | AN |   |   |   |        |
| **ELEMENTS OF ELIGIBILITY & PAYMENT           | AL |   |   |   |        |
| DETERMINATION (1)                             | YS |   |   |   |        |
|                                               | IS |   |   |   |        |
| Determine difference in dollar amount of      | OF |   |   |   |        |
| child care benefits authorized versus the     | CA |   |   |   |        |
| amount that should have been                  | SE |   |   |   |        |
| authorized---indicate if it is an overpayment | RE |   |   |   |        |
| or underpayment.                              | CO |   |   |   |        |
|                                               | RD |   |   |   |        |
|                                               | (  |   |   |   |        |
|                                               | 2) |   |   |   |        |
+-----------------------------------------------+----+---+---+---+--------+
|                                               | Y  | N |   | C |        |
|                                               | es | o |   | o |        |
|                                               |    |   |   | m |        |
|                                               |    |   |   | m |        |
|                                               |    |   |   | e |        |
|                                               |    |   |   | n |        |
|                                               |    |   |   | t |        |
|                                               |    |   |   | s |        |
+-----------------------------------------------+----+---+---+---+--------+
| Calculate amount of payment that child was    |    |   |   |   |        |
| eligible to receive (daily rate -- parent fee |    |   |   |   |        |
| x number of days = monthly payment).          |    |   |   |   |        |
|                                               |    |   |   |   |        |
| Does this agree with the reimbursement? If    |    |   |   |   |        |
| not determine difference in dollar amount of  |    |   |   |   |        |
| child care benefits authorized versus the     |    |   |   |   |        |
| amount that should have been                  |    |   |   |   |        |
| authorized\--indicate if it is an overpayment |    |   |   |   |        |
| or underpayment. (Use spreadsheet provided)   |    |   |   |   |        |
+-----------------------------------------------+----+---+---+---+--------+
| FINDINGS (3)                                  |    |   |   |   |        |
|                                               |    |   |   |   |        |
|                                               |    |   |   |   |        |
+-----------------------------------------------+----+---+---+---+--------+
| Results (4)                                   | No |   | C |   | Agency |
|                                               | E  |   | l |   | Error  |
|                                               | rr |   | i |   | \_\_\_ |
|                                               | or |   | e |   | 2      |
|                                               | \_ |   | n |   |        |
|                                               | \_ |   | t |   |        |
|                                               | \_ |   | E |   |        |
|                                               | 0  |   | r |   |        |
|                                               |    |   | r |   |        |
|                                               |    |   | o |   |        |
|                                               |    |   | r |   |        |
|                                               |    |   | \ |   |        |
|                                               |    |   | _ |   |        |
|                                               |    |   | \ |   |        |
|                                               |    |   | _ |   |        |
|                                               |    |   | \ |   |        |
|                                               |    |   | _ |   |        |
|                                               |    |   | 1 |   |        |
+-----------------------------------------------+----+---+---+---+--------+

+-------------------------------------------------+--------------------+
| **Data Summary**                                |                    |
+-------------------------------------------------+--------------------+
| State                                           | Florida            |
+-------------------------------------------------+--------------------+
| County                                          |                    |
+-------------------------------------------------+--------------------+
| Child ID                                        |                    |
+-------------------------------------------------+--------------------+
| Study Period Month                              |                    |
+-------------------------------------------------+--------------------+
| One or more eligibility errors during pilot     |                    |
| period                                          |                    |
|                                                 |                    |
| 0= no errors, 1= one or more errors             |                    |
+-------------------------------------------------+--------------------+
| Total amount of improper payment during review  |                    |
| month                                           |                    |
+-------------------------------------------------+--------------------+
| Total Payment Amount during review month (shown |                    |
| in Client Payment Information screen)           |                    |
+-------------------------------------------------+--------------------+

[]{#__RefHeading___Toc167079367 .anchor}**Appendix B. Kansas**

[]{#__RefHeading___Toc167079368 .anchor}**A. Organizational Structure**

The Division of Economic and Employment Support within the Kansas
Department of Social and Rehabilitation Services (SRS) administers the
Child Care Subsidy Program. In six regional offices and 39 county
offices, SRS provides eligibility services to the State's 105 counties.
In FY 2005, average per month assistance was provided through 6,450
providers to over 18,800 children in 10,200 families.

Kansas has processes in place at both the Central and Regional level to
handle improper payments. At the Central Office level, the Economic and
Employment Support section provides supervision and oversight of Quality
Assurance and Performance Improvement functions. At the Regional Office
level, both the Economic and Employment Support and the Performance
Improvement divisions are tasked with the responsibility to identify and
correct improper payments.

[]{#__RefHeading___Toc167079369 .anchor}**B. Overview of the Child Care
Eligibility Process**

The Kansas Child Care Subsidy Program allows families the choice to
select the child care provider that best meets their needs. Parents may
use licensed centers, licensed family childcare homes, and group homes
as well as license exempt centers and homes.

To be eligible for child care assistance, families must have an income
less than 185 percent of the Federal Poverty Level (FPL), be a resident
of the State, have children under the age of 13 (or age 19 for special
care needs), be employed or in an approved education or training
activity, and, when applicable, cooperate with child support
enforcement. A monthly family payment is assessed to income eligible
clients with income at or over 70 percent of the FPL. Kansas does not
have a resource limit as a condition of eligibility.

The basis of the child care payment is the hours of need. Payment rates
vary from county to county (excluding the In-Home care rate) and for
special needs. Benefits are determined prospectively, i.e. for the
upcoming month. Reauthorization of eligibility is on an annual basis.
Clients are responsible for reporting specific types of changes
(household composition, income, residence, and provider changes) within
10 days of the change. With some exceptions for changes requiring
verification, the change takes effect the following month.

[]{#__RefHeading___Toc167079370 .anchor}**C. Improper Payment Process**

Kansas defines an underpayment as the amount a provider was due but did
not receive and an overpayment as either assistance to a client or a
payment to a provider over the entitled amount. Overpayments are also
classified according to cause: agency, client, or provider. Client and
provider overpayments may be further categorized as fraudulent if a
court of appropriate jurisdiction determines that the actions or
omissions have been willful.

Kansas identifies improper payments through a variety of methods listed
below:

-   The automated eligibility system, KSCARES, contains a number of
    edits that identify anomalies dealing with income, hours, duplicate
    eligibility, and multiple providers.

-   The system generates reports that identify cases reporting excessive
    hours or providers whose capacity exceeds their licensing.

-   Kansas staff members currently complete the following audits and
    quality assurance reviews designed to improve payment accuracy:

```{=html}
<!-- -->
```
-   Targeted reviews on specific populations and/or portions of
    eligibility and payment such as attendance records, in-home child
    care cases, special needs, and child welfare cases;

-   General statewide quality assurance reviews initiated in October
    2005 on a randomly selected sample of 80 cases per month; and

-   Provider audits involving reviews of 10 randomly selected providers
    per month, initiated with EBT in June 2005, to audit hours of care
    charges against attendance records and contracts.

```{=html}
<!-- -->
```
-   Kansas contracted with a firm to develop software, which would track
    all case review findings and provide aggregate review data. The
    system, which is web based, provides detailed tracking information
    on all reviews, including the child care reviews. The software costs
    \$75,000 with an 18 percent maintenance costs agreement.

    []{#__RefHeading___Toc167079371 .anchor}**D. Case Reviews**

Kansas Performance Improvement staff completed two targeted child care
quality assurance reviews over the past two years. One focused on
in-home care as it was believed to be more error prone than other types
of care. The review findings did not support this belief. The second
review focused on field staff adhering to State policy and procedures in
the area of social services child care. The review tool was a version of
the Food Stamp quality control review sheet. The review began with a
reading of the case record. All aspects of eligibility were reviewed.
Each element was re-verified with collateral contacts (customer,
employers, providers, etc.) All relevant data bases were queried (CSE --
Child Support, EBT, FACTS -- Child Welfare, BARI/BASI -- wages and
unemployment, KDHE -- vital records, EATTs -- licensing, KAECSES --
TANF, Food Stamps, General Assistance, Medicaid). The information was
compared to the customer's June 2006 child care payment.

Kansas chose to participate in the pilot in order to learn the error
rate review methodology undergoing a pilot test. From the Kansas
perspective, because the pilot error rate review model was limited to a
desk audit it was less involved than the most recent State child care
review, which involved contacting employers and verifying income.

For the pilot, the Kansas Review Team consisted of three Quality
Assurance supervisors (Topeka, Wichita, Kansas City) and three Quality
Assurance Specialists. The team also conducted a re-review of all cases
by having a second team member conduct the review. When both team
members agreed on an error, they sent a notice to the local Social
Rehabilitation Services (SRS) office for corrective action. The Review
Team reviewed all records centrally in Topeka. Kansas modified the
*Record Review Worksheet* to include "Yes" or "No" questions to reduce
the amount of written narration for the reviewers.

Prior the pilot, the Quality Assurance Specialists had conducted two
reviews of child care cases in 2006. These reviews were more extensive,
involving re-verification of all elements of eligibility, queries of
relevant State databases, and follow-up contacts with clients,
employers, and providers.

[]{#__RefHeading___Toc167079372 .anchor}**E. Automation**

The Kansas System for Child Care and Realizing Economic Self Sufficiency
(KsCARES) has played a significant role in preventing and identifying
improper payments in the subsidized child care program. Kansas obtained
the system from Wyoming and then modified it extensively to fit its
policy requirements.

KsCARES has extensive edits that help enforce policies and perform the
calculations used for eligibility determination, including:

-   Indicating whether income is countable based on income type;

-   Enforcing maximum hours for child care service plans during a month;

-   Preventing duplicate benefit issuances; and

-   Determining the reimbursement rates automatically, based on the
    child\'s age, provider\'s individual rate, and the State's maximum
    rates.

KsCARES is a mainframe system that Kansas has modified extensively since
the 1990's. Some of the modifications enhance the sharing of information
with the systems supporting Temporary Assistance for Needy Families,
Food Stamps (KAECSES) and child care licensing (KDHE/CLARIS). KsCARES
processes all eligibility determinations automatically, but workers may
perform some eligibility calculations off-line and enter the results
into the system. However, KsCARES computes rates to ensure accuracy. The
system will pay the lesser rate between the provider maximum, State, and
private pay rates.

KsCARES has many system edits that improve payment accuracy including:
case record search when entering application information, "215" hour
block, duplicate plans, automatic income eligibility test, the number of
placements and maximums for licensed home, child on more than one case,
automation of provider rates, and provider rates automatically pay
lesser of provider/SRS rate. Also, Kansas has recently changed the child
care payment system to automatically deduct a percentage of the
overpayment from each monthly payment.

Kansas installed a new Electronic Benefits Transfer (EBT) system in
June, 2005 that handles all aspects of billing. Providers no longer send
in attendance sheets but use a point-of-service (POS) system or a 1-800
number to submit attendance records electronically. Kansas plans to
establish a system of random audits of providers.

[]{#__RefHeading___Toc167079373 .anchor}**F. Next Steps**

Based on the experience with the error rate pilot Kansas took the
following actions:

-   Prior to the pilot, Kansas had completed child care quality control
    (QC) reviews for the month of October 2005, the second month of
    statewide payments using the new child care EBT payment system.
    Kansas was already aware of the error causes found in the pilot and
    had conducted statewide refresher training in June 2006 to address
    these errors. The training focused on hours of care, use of child
    care plan hour worksheets, computation of income, use of income
    worksheets to document computations, and general case documentation.

```{=html}
<!-- -->
```
-   Kansas continues to complete monthly Child Care QC reviews and uses
    the results of these reviews to inform training needs. Kansas is
    updating the Child Care Personnel Trainer and Training Academy to
    emphasize case documentation and computation of hours of care needed
    and income. Kansas requires this training for all new workers and
    for others as needed. Supervisors now complete monthly case reviews
    for the Child Care Assistance Program.

-   Kansas hired a child care trainer in July 2006 to revise training
    materials and conduct several training sessions for eligibility
    staff. The QC unit compiles results monthly for regional
    administrators to keep them informed about findings and areas
    needing improvement. Supervisory case reviews at the regional level
    now include child care cases.

-   Kansas contracted with a firm to develop software to track all case
    review findings and provide aggregate review data. The web based
    system tracks aggregate case review data for the mandatory Food
    Stamp reviews and the child care reviews. The child care reviews
    began in July 2006. The software costs \$75,000 with an 18 percent
    maintenance costs agreement.

    []{#__RefHeading___Toc167079374 .anchor}**Kansas Site Visit
    Participants**

Alice Womack -- Program Administrator SRS

Sally Hargis -- Kansas SRS, Family and Children Programs, Policy

Darlene Kinion -- Kansas SRS, Quality Assurance Specialist

Dennis Priest -- Kansas SRS, Program Administrator

Diane Dykstra -- Kansas SRS Performance Improvement Manager

Tonie Dugan -- Kansas Quality Assurance Supervisor (Topeka Unit)

Diane Britton -- Kansas Quality Assurance Supervisor (Wichita Unit)

Cindy Barshesky -- Kansas Quality Assurance Supervisor (Kansas City
Unit)

Kelly Ross - Kansas Quality Assurance Specialist

Cletus Linenberger -- Kansas Quality Assurance Specialist

John Gibson -- Kansas Quality Assurance Specialist

Allen Mossman -- Kansas SRS, Training

Rachel Katuin -- Kansas SRS

Jean Morgan -- Kansas SRS, Policy

Nancy Caudle -- Kansas EES, Management Evaluation

[]{#__RefHeading___Toc167078148 .anchor}**Exhibit B-1. Kansas Record
Review Worksheet**

<table style="width:100%;">
<colgroup>
<col style="width: 27%" />
<col style="width: 3%" />
<col style="width: 3%" />
<col style="width: 12%" />
<col style="width: 0%" />
<col style="width: 4%" />
<col style="width: 5%" />
<col style="width: 3%" />
<col style="width: 7%" />
<col style="width: 0%" />
<col style="width: 1%" />
<col style="width: 8%" />
<col style="width: 3%" />
<col style="width: 1%" />
<col style="width: 0%" />
<col style="width: 1%" />
<col style="width: 6%" />
<col style="width: 7%" />
<col style="width: 0%" />
</colgroup>
<tbody>
<tr class="odd">
<td colspan="19">7-31-06</td>
</tr>
<tr class="even">
<td colspan="19"><strong>WORKSHEET FOR Child Care QUALITY CONTROL
REVIEWS</strong></td>
</tr>
<tr class="odd">
<td colspan="3"><strong>A. IDENTIFYING INFORMATION</strong></td>
<td colspan="16"><strong>B. PERSONS LIVING IN THE HOME</strong></td>
</tr>
<tr class="even">
<td><strong>1. Region</strong></td>
<td></td>
<td colspan="4"><strong>NAME</strong></td>
<td><strong>BIRTH DATE</strong></td>
<td><strong>AGE</strong></td>
<td colspan="3"><p><strong>RELATIONSHIP OR</strong></p>
<p><strong>SIGNIFICANCE</strong></p></td>
<td><strong>SOCIAL SECURITY NUMBER</strong></td>
<td colspan="3"><strong>Sample child</strong></td>
<td></td>
<td colspan="3"></td>
</tr>
<tr class="odd">
<td><strong>2. CASE NAME</strong></td>
<td><strong>1</strong></td>
<td colspan="4"></td>
<td></td>
<td></td>
<td colspan="3"></td>
<td><strong>| |</strong></td>
<td colspan="3"></td>
<td></td>
<td colspan="3"></td>
</tr>
<tr class="even">
<td><strong>3. ADDRESS</strong></td>
<td><strong>2</strong></td>
<td colspan="4"></td>
<td></td>
<td></td>
<td colspan="3"></td>
<td><strong>| |</strong></td>
<td colspan="3"></td>
<td></td>
<td colspan="3"></td>
</tr>
<tr class="odd">
<td></td>
<td><strong>3</strong></td>
<td colspan="4"></td>
<td></td>
<td></td>
<td colspan="3"></td>
<td><strong>| |</strong></td>
<td colspan="3"></td>
<td></td>
<td colspan="3"></td>
</tr>
<tr class="even">
<td><strong>4. PHONE NUMBER</strong></td>
<td><strong>4</strong></td>
<td colspan="4"></td>
<td></td>
<td></td>
<td colspan="3"></td>
<td><strong>| |</strong></td>
<td colspan="3"></td>
<td></td>
<td colspan="3"></td>
</tr>
<tr class="odd">
<td><strong>5. Child’s name</strong></td>
<td><strong>5</strong></td>
<td colspan="4"></td>
<td></td>
<td></td>
<td colspan="3"></td>
<td><strong>| |</strong></td>
<td colspan="3"></td>
<td></td>
<td colspan="3"></td>
</tr>
<tr class="even">
<td><strong>6. Child’s ID number</strong></td>
<td><strong>6</strong></td>
<td colspan="4"></td>
<td></td>
<td></td>
<td colspan="3"></td>
<td><strong>| |</strong></td>
<td colspan="3"></td>
<td></td>
<td colspan="3"></td>
</tr>
<tr class="odd">
<td><strong>7. Case Number</strong></td>
<td><strong>7</strong></td>
<td colspan="4"></td>
<td></td>
<td></td>
<td colspan="3"></td>
<td><strong>| |</strong></td>
<td colspan="3"></td>
<td></td>
<td colspan="3"></td>
</tr>
<tr class="even">
<td><strong>8. Review Number</strong></td>
<td><strong>8</strong></td>
<td colspan="4"></td>
<td></td>
<td></td>
<td colspan="3"></td>
<td><strong>| |</strong></td>
<td colspan="3"></td>
<td></td>
<td colspan="3"></td>
</tr>
<tr class="odd">
<td><strong>9. REVIEW DATE</strong></td>
<td><strong>9</strong></td>
<td colspan="4"></td>
<td></td>
<td></td>
<td colspan="3"></td>
<td><strong>| |</strong></td>
<td colspan="3"></td>
<td></td>
<td colspan="3"></td>
</tr>
<tr class="even">
<td><strong>10. Date of Most Recent child care plan</strong></td>
<td><strong>10</strong></td>
<td colspan="4"></td>
<td></td>
<td></td>
<td colspan="3"></td>
<td><strong>| |</strong></td>
<td colspan="3"></td>
<td></td>
<td colspan="3"></td>
</tr>
<tr class="odd">
<td><strong>11. Most Recent Action</strong></td>
<td colspan="17"><strong>C. SIGNIFICANT PERSONS NOT LIVING IN THE
HOME</strong></td>
<td></td>
</tr>
<tr class="even">
<td><strong>A. DATE</strong></td>
<td colspan="3" rowspan="2"><strong>NAME</strong></td>
<td colspan="3" rowspan="2"><strong>RELATIONSHIP OR
SIGNIFICANCE</strong></td>
<td colspan="3" rowspan="2"><strong>SOCIAL SECURITY NUMBER</strong></td>
<td colspan="3" rowspan="2"><strong>ADDRESS</strong></td>
<td colspan="4" rowspan="2"><strong>PHONE NUMBER</strong></td>
<td rowspan="2"><strong>FINANCIAL SUPPORT</strong></td>
<td></td>
</tr>
<tr class="odd">
<td><strong>B. TYPE</strong></td>
<td></td>
</tr>
<tr class="even">
<td><strong>12. Reviewer</strong></td>
<td><strong>11</strong></td>
<td colspan="2"></td>
<td colspan="3"></td>
<td colspan="3"><strong>| |</strong></td>
<td colspan="3"></td>
<td colspan="4"></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td><strong>13. Date Assigned</strong></td>
<td><strong>12</strong></td>
<td colspan="3"></td>
<td colspan="2"></td>
<td colspan="2"><strong>| |</strong></td>
<td colspan="4"></td>
<td></td>
<td colspan="5"></td>
</tr>
<tr class="even">
<td></td>
<td><strong>13</strong></td>
<td colspan="3"></td>
<td colspan="2"></td>
<td colspan="2"><strong>| |</strong></td>
<td colspan="4"></td>
<td></td>
<td colspan="5"></td>
</tr>
<tr class="odd">
<td></td>
<td colspan="17"><strong>D. REVIEW FINDINGS</strong></td>
<td></td>
</tr>
<tr class="even">
<td><strong>14. Date of Case reading</strong></td>
<td colspan="17"><strong>Child Care</strong></td>
<td></td>
</tr>
<tr class="odd">
<td></td>
<td colspan="8">CASE Eligibility</td>
<td colspan="9">PAYMENT FINDING</td>
<td></td>
</tr>
<tr class="even">
<td></td>
<td colspan="8"><strong>c</strong> Correct <strong>c</strong> One
element in error <strong>c</strong> More than one element in error</td>
<td colspan="9">SRS Payment under Review ____________</td>
<td></td>
</tr>
<tr class="odd">
<td></td>
<td colspan="8"></td>
<td colspan="9"><strong>c</strong> Correct <strong>c</strong> Improper
Payment $ ________</td>
<td></td>
</tr>
<tr class="even">
<td><strong>18. DATE COMPLETED</strong></td>
<td colspan="8"><strong>c</strong> Drop</td>
<td colspan="9"><strong>c</strong> Underpayment $ ______</td>
<td></td>
</tr>
<tr class="odd">
<td><strong>19. SUPERVISOR</strong></td>
<td colspan="8">Drop Reason</td>
<td colspan="9"><strong>c</strong> Overpayment $ ______</td>
<td></td>
</tr>
<tr class="even">
<td><strong>20. DATE CLEARED</strong></td>
<td colspan="8"></td>
<td colspan="9"><strong>c</strong> Ineligible $ ______</td>
<td></td>
</tr>
</tbody>
</table>

Coding on worksheet

1 = Correct C= Case error

2= Agency caused error P= Payment error

3= Customer caused error

+---------+------------------------+--------------------+------+-----+
| **E     |                        |                    | **Re |     |
| LEMENTS |                        |                    | view |     |
| OF      |                        |                    | No** |     |
| ELIG    |                        |                    |      |     |
| IBILITY |                        |                    |      |     |
| AND     |                        |                    |      |     |
| PAYMENT |                        |                    |      |     |
| D       |                        |                    |      |     |
| ETERMIN |                        |                    |      |     |
| ATION** |                        |                    |      |     |
+---------+------------------------+--------------------+------+-----+
| **E     | **QC ANALYSIS OF CASE  | **FINDINGS OF      |      | **C |
| LEMENTS | RECORD**               | FIELD              |      | C** |
| OF      |                        | INVESTIGATION**    |      |     |
| ELIG    | ***(Pertinent facts,   |                    |      |     |
| IBILITY | sources of             | ***(Facts          |      |     |
| AND**   | verification,          | obtained,          |      |     |
|         | reliability, gaps or   | verification and   |      |     |
| **      | deficiencies)***       | substantiation,    |      |     |
| PAYMENT |                        | nature or          |      |     |
| D       |                        | errors)***         |      |     |
| ETERMIN |                        |                    |      |     |
| ATION** |                        |                    |      |     |
+---------+------------------------+--------------------+------+-----+
| **(1)** | **(2)**                | **(3)**            |      | *   |
|         |                        |                    |      | *(4 |
|         |                        |                    |      | )** |
+---------+------------------------+--------------------+------+-----+
|         | **SECTION I. STATE     |                    |      |     |
|         | CHILD CARE PROGRAM     |                    |      |     |
|         | FORMS**                |                    |      |     |
+---------+------------------------+--------------------+------+-----+
| **100   | **Date of Application: | **Case reviewed in |      | C   |
| APPL    | \_\_\_\_\_             | last 12 month:     |      |     |
| ICATION | \_\_\_\_\_\_\_\_\_\_** | \_\_\_ Yes \_\_\_  |      | 1   |
| FORMS** |                        | No \_\_\_N/A**     |      |     |
|         | **Date Child Care Plan |                    |      | 2   |
|         | approved:              | **Most recent      |      |     |
|         | \_\_\_\_\_             | application        |      | 3   |
|         | \_\_\_\_\_\_\_\_\_\_** | approved within 30 |      |     |
|         |                        | days:**            |      |     |
|         | **Child Care Plan from |                    |      |     |
|         | \_                     | **\_\_\_ Yes       |      |     |
|         | \_\_\_\_\_\_\_\_\_\_\_ | \_\_\_ No \_\_\_   |      |     |
|         | to                     | Unable to          |      |     |
|         | \_\_                   | determine**        |      |     |
|         | \_\_\_\_\_\_\_\_\_\_** |                    |      |     |
|         |                        | **Approval Notice  |      |     |
|         | **First Month of Plan: | sent to consumer:  |      |     |
|         | \_\_\_\_\_             | \_\_\_ Yes \_\_\_  |      |     |
|         | \_\_\_\_\_\_\_\_\_\_** | No**               |      |     |
|         |                        |                    |      |     |
|         | **Income worksheet:    | **Reporting        |      |     |
|         | \_\_\_ Yes \_\_\_ No** | requirements       |      |     |
|         |                        | notice sent to     |      |     |
|         | **Hours of Care        | consumer: \_\_\_   |      |     |
|         | worksheet: \_\_\_ Yes  | Yes \_\_\_ No**    |      |     |
|         | \_\_\_ No**            |                    |      |     |
|         |                        | **Copy of case     |      |     |
|         |                        | plan sent to       |      |     |
|         |                        | consumer: \_\_\_   |      |     |
|         |                        | Yes \_\_\_ No**    |      |     |
|         |                        |                    |      |     |
|         |                        | **Verification of  |      |     |
|         |                        | income in record:  |      |     |
|         |                        | \_\_\_ Yes \_\_\_  |      |     |
|         |                        | No**               |      |     |
|         |                        |                    |      |     |
|         |                        | **Verification of  |      |     |
|         |                        | work hours in      |      |     |
|         |                        | record: \_\_\_ Yes |      |     |
|         |                        | \_\_\_ No**        |      |     |
+---------+------------------------+--------------------+------+-----+
|         | **SECTION II. PRIORITY |                    |      |     |
|         | GROUP PLACEMENT**      |                    |      |     |
+---------+------------------------+--------------------+------+-----+
| **200   | **Case was set up      | **Child care type  |      | C   |
| P       | under what subtype.**  | correct: \_\_\_    |      |     |
| RIORITY |                        | Yes \_\_\_ No**    |      | P   |
| GROUP   | **\_\_\_ JO \_\_\_ MO  |                    |      |     |
| PLAC    | \_\_\_SS**             | **If no,           |      | 1   |
| EMENT** |                        | explain:**         |      |     |
|         | **\_\_ N/A see element |                    |      | 2   |
|         | 320**                  | **\_\_ N/A see     |      |     |
|         |                        | element 320**      |      | 3   |
+---------+------------------------+--------------------+------+-----+
|         | **SECTION III. GENERAL |                    |      |     |
|         | PROGRAM REQUIREMENTS** |                    |      |     |
+---------+------------------------+--------------------+------+-----+
| **300   | **\_\_\_ Parent**      | **Were all family  |      | C   |
| QUA     |                        | members properly   |      |     |
| LIFYING | **\_\_\_ Other         | included in the    |      | P   |
| HEAD OF | (list)**               | household and      |      |     |
| HOUS    |                        | listed on CAMA?    |      | 1   |
| EHOLD** | **\_\_\_ Step-parent** | \_\_\_ Yes \_\_\_  |      |     |
|         |                        | No explain**       |      | 2   |
|         | **\_\_\_ Legal         |                    |      |     |
|         | Guardian**             | **Was a CSE        |      | 3   |
|         |                        | referral found in  |      |     |
|         | **\_\_\_ Needy         | KsCARES, KAECSES,  |      |     |
|         | Caretaker Relative**   | or by paper in the |      |     |
|         |                        | case record? \_\_  |      |     |
|         | **\_\_\_ Spouse of     | Yes \_\_ No        |      |     |
|         | Needy Caretaker        | explain**          |      |     |
|         | Relative**             |                    |      |     |
|         |                        |                    |      |     |
|         | **Cooperating with     |                    |      |     |
|         | CSE: \_\_\_ Yes \_\_\_ |                    |      |     |
|         | No**                   |                    |      |     |
|         |                        |                    |      |     |
|         | **If not cooperating   |                    |      |     |
|         | with CSE, is there     |                    |      |     |
|         | good cause: \_\_\_ Yes |                    |      |     |
|         | \_\_\_ No**            |                    |      |     |
+---------+------------------------+--------------------+------+-----+

+---------+------------------------+---+--------------------+-----+------+
| **E     |                        |   |                    | **  |      |
| LEMENTS |                        |   |                    | Rev |      |
| OF      |                        |   |                    | iew |      |
| ELIG    |                        |   |                    | N   |      |
| IBILITY |                        |   |                    | o** |      |
| AND     |                        |   |                    |     |      |
| PAYMENT |                        |   |                    |     |      |
| D       |                        |   |                    |     |      |
| ETERMIN |                        |   |                    |     |      |
| ATION** |                        |   |                    |     |      |
+---------+------------------------+---+--------------------+-----+------+
| **E     | **QC ANALYSIS OF CASE  | * |                    |     | **   |
| LEMENTS | RECORD**               | * |                    |     | CC** |
| OF      |                        | F |                    |     |      |
| ELIG    | ***(Pertinent facts,   | I |                    |     |      |
| IBILITY | sources of             | N |                    |     |      |
| AND**   | verification,          | D |                    |     |      |
|         | reliability, gaps or   | I |                    |     |      |
| **      | deficiencies)***       | N |                    |     |      |
| PAYMENT |                        | G |                    |     |      |
| D       |                        | S |                    |     |      |
| ETERMIN |                        | O |                    |     |      |
| ATION** |                        | F |                    |     |      |
|         |                        | F |                    |     |      |
|         |                        | I |                    |     |      |
|         |                        | E |                    |     |      |
|         |                        | L |                    |     |      |
|         |                        | D |                    |     |      |
|         |                        | I |                    |     |      |
|         |                        | N |                    |     |      |
|         |                        | V |                    |     |      |
|         |                        | E |                    |     |      |
|         |                        | S |                    |     |      |
|         |                        | T |                    |     |      |
|         |                        | I |                    |     |      |
|         |                        | G |                    |     |      |
|         |                        | A |                    |     |      |
|         |                        | T |                    |     |      |
|         |                        | I |                    |     |      |
|         |                        | O |                    |     |      |
|         |                        | N |                    |     |      |
|         |                        | * |                    |     |      |
|         |                        | * |                    |     |      |
|         |                        |   |                    |     |      |
|         |                        | * |                    |     |      |
|         |                        | * |                    |     |      |
|         |                        | * |                    |     |      |
|         |                        | ( |                    |     |      |
|         |                        | F |                    |     |      |
|         |                        | a |                    |     |      |
|         |                        | c |                    |     |      |
|         |                        | t |                    |     |      |
|         |                        | s |                    |     |      |
|         |                        | o |                    |     |      |
|         |                        | b |                    |     |      |
|         |                        | t |                    |     |      |
|         |                        | a |                    |     |      |
|         |                        | i |                    |     |      |
|         |                        | n |                    |     |      |
|         |                        | e |                    |     |      |
|         |                        | d |                    |     |      |
|         |                        | , |                    |     |      |
|         |                        | v |                    |     |      |
|         |                        | e |                    |     |      |
|         |                        | r |                    |     |      |
|         |                        | i |                    |     |      |
|         |                        | f |                    |     |      |
|         |                        | i |                    |     |      |
|         |                        | c |                    |     |      |
|         |                        | a |                    |     |      |
|         |                        | t |                    |     |      |
|         |                        | i |                    |     |      |
|         |                        | o |                    |     |      |
|         |                        | n |                    |     |      |
|         |                        | a |                    |     |      |
|         |                        | n |                    |     |      |
|         |                        | d |                    |     |      |
|         |                        | s |                    |     |      |
|         |                        | u |                    |     |      |
|         |                        | b |                    |     |      |
|         |                        | s |                    |     |      |
|         |                        | t |                    |     |      |
|         |                        | a |                    |     |      |
|         |                        | n |                    |     |      |
|         |                        | t |                    |     |      |
|         |                        | i |                    |     |      |
|         |                        | a |                    |     |      |
|         |                        | t |                    |     |      |
|         |                        | i |                    |     |      |
|         |                        | o |                    |     |      |
|         |                        | n |                    |     |      |
|         |                        | , |                    |     |      |
|         |                        | n |                    |     |      |
|         |                        | a |                    |     |      |
|         |                        | t |                    |     |      |
|         |                        | u |                    |     |      |
|         |                        | r |                    |     |      |
|         |                        | e |                    |     |      |
|         |                        | o |                    |     |      |
|         |                        | r |                    |     |      |
|         |                        | e |                    |     |      |
|         |                        | r |                    |     |      |
|         |                        | r |                    |     |      |
|         |                        | o |                    |     |      |
|         |                        | r |                    |     |      |
|         |                        | s |                    |     |      |
|         |                        | ) |                    |     |      |
|         |                        | * |                    |     |      |
|         |                        | * |                    |     |      |
|         |                        | * |                    |     |      |
+---------+------------------------+---+--------------------+-----+------+
| **310   | **Residents of Kansas: |   | **Child met        |     | C 1  |
| RESI    | \_\_\_ Yes \_\_\_ No** |   | residency          |     |      |
| DENCY** |                        |   | requirement:       |     | 2    |
|         |                        |   | \_\_\_ Yes \_\_\_  |     |      |
|         |                        |   | No explain**       |     | P 3  |
+---------+------------------------+---+--------------------+-----+------+
| **320   | **N/A \_\_\_\_\_\_ See |   | **N/A \_\_\_\_\_\_ |     | 1    |
| HO      | element 200**          |   | See element 200**  |     |      |
| USEHOLD |                        |   |                    |     | 2 C  |
| ME      | **The customer is      |   | **Child care type  |     |      |
| MBERS** | employed by (the       |   | correct: \_\_\_    |     | 3 P  |
|         | employer of record) or |   | Yes \_\_\_ No**    |     |      |
|         | in school (record      |   |                    |     |      |
|         | States)                |   | **If no,           |     |      |
|         | \_\_\_                 |   | explain:**         |     |      |
|         | \_\_\_\_\_\_\_\_\_\_** |   |                    |     |      |
|         |                        |   |                    |     |      |
|         | **Case was set up      |   |                    |     |      |
|         | under what subtype**   |   |                    |     |      |
|         |                        |   |                    |     |      |
|         | **\_\_\_ EM \_\_\_     |   |                    |     |      |
|         | ET**                   |   |                    |     |      |
+---------+------------------------+---+--------------------+-----+------+
| **330   | **Age of child:        |   | **Child eligible   |     | 1    |
| QUA     | \_\_\_\_\_\_\_\_**     |   | to received child  |     |      |
| LIFYING |                        |   | care assistance:** |     | 2 C  |
| CHILD** | **Under 13 age \_\_\_  |   |                    |     |      |
|         | Yes \_\_\_ No go to    |   | **\_\_\_ Yes       |     | 3 P  |
|         | next question**        |   | \_\_\_ No**        |     |      |
|         |                        |   |                    |     |      |
|         | **Between 13 & 18      |   | **If no,           |     |      |
|         | (under 19) is the      |   | explain:**         |     |      |
|         | child physically or    |   |                    |     |      |
|         | mentally incapable of  |   | **All              |     |      |
|         | self care or under     |   | documentation for  |     |      |
|         | court supervision:     |   | special needs in   |     |      |
|         | \_\_\_ Yes \_\_\_ No ( |   | case record.**     |     |      |
|         | [Inel                  |   |                    |     |      |
|         | igible]{.underline})** |   | **\_\_ Yes \_\_    |     |      |
|         |                        |   | No**               |     |      |
|         | **Child is a special   |   |                    |     |      |
|         | needs child: \_\_\_    |   | **Child is citizen |     |      |
|         | Yes \_\_\_ No**        |   | of the United      |     |      |
|         |                        |   | States: \_\_\_ Yes |     |      |
|         | **There is a need for  |   | \_\_\_ No          |     |      |
|         | child care: \_\_\_ Yes |   | ([Ineli            |     |      |
|         | \_\_\_\_ No            |   | gible]{.underline} |     |      |
|         | ([Inel                 |   | if alien not       |     |      |
|         | igible]{.underline})** |   | met)**             |     |      |
|         |                        |   |                    |     |      |
|         |                        |   | **Child is         |     |      |
|         |                        |   | resident Alien:    |     |      |
|         |                        |   | \_\_\_ Yes \_\_\_  |     |      |
|         |                        |   | No**               |     |      |
|         |                        |   |                    |     |      |
|         |                        |   | **If not citizen   |     |      |
|         |                        |   | or resident alien, |     |      |
|         |                        |   | Social Services    |     |      |
|         |                        |   | involvement:**     |     |      |
|         |                        |   |                    |     |      |
|         |                        |   | **\_\_\_ Yes       |     |      |
|         |                        |   | \_\_\_ No**        |     |      |
+---------+------------------------+---+--------------------+-----+------+
| **340   | **\_\_\_\_\_\_\_\_     |   | **CASE RECORD      |     | 1    |
| QUA     | Hours of Care          |   | INCLUDING TAF AND  |     |      |
| LIFYING | Authorized**           |   | FS CASES IF ANY.** |     | 2 C  |
| CARE**  |                        |   |                    |     |      |
|         | **Calculation of       |   |                    |     | 3 P  |
|         | hours:**               |   |                    |     |      |
+---------+------------------------+---+--------------------+-----+------+

+---------+------------------------+---+--------------------+------+---+-----+
| **E     |                        |   |                    | **Re |   |     |
| LEMENTS |                        |   |                    | view |   |     |
| OF      |                        |   |                    | No** |   |     |
| ELIG    |                        |   |                    |      |   |     |
| IBILITY |                        |   |                    |      |   |     |
| AND     |                        |   |                    |      |   |     |
| PAYMENT |                        |   |                    |      |   |     |
| D       |                        |   |                    |      |   |     |
| ETERMIN |                        |   |                    |      |   |     |
| ATION** |                        |   |                    |      |   |     |
+---------+------------------------+---+--------------------+------+---+-----+
| **E     | **QC ANALYSIS OF CASE  | * |                    |      | * |     |
| LEMENTS | RECORD**               | * |                    |      | * |     |
| OF      |                        | F |                    |      | C |     |
| ELIG    | ***(Pertinent facts,   | I |                    |      | C |     |
| IBILITY | sources of             | N |                    |      | * |     |
| AND**   | verification,          | D |                    |      | * |     |
|         | reliability, gaps or   | I |                    |      |   |     |
| **      | deficiencies)***       | N |                    |      |   |     |
| PAYMENT |                        | G |                    |      |   |     |
| D       |                        | S |                    |      |   |     |
| ETERMIN |                        | O |                    |      |   |     |
| ATION** |                        | F |                    |      |   |     |
|         |                        | F |                    |      |   |     |
|         |                        | I |                    |      |   |     |
|         |                        | E |                    |      |   |     |
|         |                        | L |                    |      |   |     |
|         |                        | D |                    |      |   |     |
|         |                        | I |                    |      |   |     |
|         |                        | N |                    |      |   |     |
|         |                        | V |                    |      |   |     |
|         |                        | E |                    |      |   |     |
|         |                        | S |                    |      |   |     |
|         |                        | T |                    |      |   |     |
|         |                        | I |                    |      |   |     |
|         |                        | G |                    |      |   |     |
|         |                        | A |                    |      |   |     |
|         |                        | T |                    |      |   |     |
|         |                        | I |                    |      |   |     |
|         |                        | O |                    |      |   |     |
|         |                        | N |                    |      |   |     |
|         |                        | * |                    |      |   |     |
|         |                        | * |                    |      |   |     |
|         |                        |   |                    |      |   |     |
|         |                        | * |                    |      |   |     |
|         |                        | * |                    |      |   |     |
|         |                        | * |                    |      |   |     |
|         |                        | ( |                    |      |   |     |
|         |                        | F |                    |      |   |     |
|         |                        | a |                    |      |   |     |
|         |                        | c |                    |      |   |     |
|         |                        | t |                    |      |   |     |
|         |                        | s |                    |      |   |     |
|         |                        | o |                    |      |   |     |
|         |                        | b |                    |      |   |     |
|         |                        | t |                    |      |   |     |
|         |                        | a |                    |      |   |     |
|         |                        | i |                    |      |   |     |
|         |                        | n |                    |      |   |     |
|         |                        | e |                    |      |   |     |
|         |                        | d |                    |      |   |     |
|         |                        | , |                    |      |   |     |
|         |                        | v |                    |      |   |     |
|         |                        | e |                    |      |   |     |
|         |                        | r |                    |      |   |     |
|         |                        | i |                    |      |   |     |
|         |                        | f |                    |      |   |     |
|         |                        | i |                    |      |   |     |
|         |                        | c |                    |      |   |     |
|         |                        | a |                    |      |   |     |
|         |                        | t |                    |      |   |     |
|         |                        | i |                    |      |   |     |
|         |                        | o |                    |      |   |     |
|         |                        | n |                    |      |   |     |
|         |                        | a |                    |      |   |     |
|         |                        | n |                    |      |   |     |
|         |                        | d |                    |      |   |     |
|         |                        | s |                    |      |   |     |
|         |                        | u |                    |      |   |     |
|         |                        | b |                    |      |   |     |
|         |                        | s |                    |      |   |     |
|         |                        | t |                    |      |   |     |
|         |                        | a |                    |      |   |     |
|         |                        | n |                    |      |   |     |
|         |                        | t |                    |      |   |     |
|         |                        | i |                    |      |   |     |
|         |                        | a |                    |      |   |     |
|         |                        | t |                    |      |   |     |
|         |                        | i |                    |      |   |     |
|         |                        | o |                    |      |   |     |
|         |                        | n |                    |      |   |     |
|         |                        | , |                    |      |   |     |
|         |                        | n |                    |      |   |     |
|         |                        | a |                    |      |   |     |
|         |                        | t |                    |      |   |     |
|         |                        | u |                    |      |   |     |
|         |                        | r |                    |      |   |     |
|         |                        | e |                    |      |   |     |
|         |                        | o |                    |      |   |     |
|         |                        | r |                    |      |   |     |
|         |                        | e |                    |      |   |     |
|         |                        | r |                    |      |   |     |
|         |                        | r |                    |      |   |     |
|         |                        | o |                    |      |   |     |
|         |                        | r |                    |      |   |     |
|         |                        | s |                    |      |   |     |
|         |                        | ) |                    |      |   |     |
|         |                        | * |                    |      |   |     |
|         |                        | * |                    |      |   |     |
|         |                        | * |                    |      |   |     |
+---------+------------------------+---+--------------------+------+---+-----+
| **350   | **1^st^ Provider Name: |   | **Was the provider |      |   | 1   |
| QUA     | \_\_\_                 |   | listed on KSCARES  |      |   |     |
| LIFYING | \_\_\_\_\_\_\_\_\_\_\_ |   | also the provider  |      |   | 2 C |
| P       | \_\_\_\_\_\_\_\_\_\_\_ |   | paid (provider on  |      |   |     |
| ROVIDER | \_\_\_\_\_\_\_\_\_\_** |   | sample assignment  |      |   | 3 P |
| ARRANG  |                        |   | list) by the       |      |   |     |
| EMENT** | **Provider Address:    |   | client:**          |      |   |     |
|         | \_\_\_                 |   |                    |      |   |     |
|         | \_\_\_\_\_\_\_\_\_\_\_ |   | **\_\_\_ Yes       |      |   |     |
|         | \_\_\_\_\_\_\_\_\_\_\_ |   | \_\_\_ No          |      |   |     |
|         | \_\_\_\_\_\_\_\_\_\_** |   | Explain:**         |      |   |     |
|         |                        |   |                    |      |   |     |
|         | **\_\_\_               |   |                    |      |   |     |
|         | \_\_\_\_\_\_\_\_\_\_\_ |   |                    |      |   |     |
|         | \_\_\_\_\_\_\_\_\_\_\_ |   |                    |      |   |     |
|         | \_\_\_\_\_\_\_\_\_\_** |   |                    |      |   |     |
|         |                        |   |                    |      |   |     |
|         | **PLHC/CHCP**          |   |                    |      |   |     |
|         |                        |   |                    |      |   |     |
|         | **Provider Type:       |   |                    |      |   |     |
|         | \_\_\_ LIC \_\_\_ REG  |   |                    |      |   |     |
|         | \_\_\_ REL \_\_\_ INH  |   |                    |      |   |     |
|         | \_\_\_ CCC**           |   |                    |      |   |     |
|         |                        |   |                    |      |   |     |
|         | **2^nd^ Provider Name: |   |                    |      |   |     |
|         | \_\_\_                 |   |                    |      |   |     |
|         | \_\_\_\_\_\_\_\_\_\_\_ |   |                    |      |   |     |
|         | \_\_\_\_\_\_\_\_\_\_\_ |   |                    |      |   |     |
|         | \_\_\_\_\_\_\_\_\_\_** |   |                    |      |   |     |
|         |                        |   |                    |      |   |     |
|         | **Provider Address:    |   |                    |      |   |     |
|         | \_\_\_                 |   |                    |      |   |     |
|         | \_\_\_\_\_\_\_\_\_\_\_ |   |                    |      |   |     |
|         | \_\_\_\_\_\_\_\_\_\_\_ |   |                    |      |   |     |
|         | \_\_\_\_\_\_\_\_\_\_** |   |                    |      |   |     |
|         |                        |   |                    |      |   |     |
|         | **\_\_\_               |   |                    |      |   |     |
|         | \_\_\_\_\_\_\_\_\_\_\_ |   |                    |      |   |     |
|         | \_\_\_\_\_\_\_\_\_\_\_ |   |                    |      |   |     |
|         | \_\_\_\_\_\_\_\_\_\_** |   |                    |      |   |     |
|         |                        |   |                    |      |   |     |
|         | **Provider Type:       |   |                    |      |   |     |
|         | \_\_\_ LIC \_\_\_ REG  |   |                    |      |   |     |
|         | \_\_\_ REL \_\_\_ INH  |   |                    |      |   |     |
|         | \_\_\_ CCC**           |   |                    |      |   |     |
+---------+------------------------+---+--------------------+------+---+-----+

+---------+---+------------------------+---+--------------------+-----+---+-----+
| **E     |   |                        |   |                    | **  |   |     |
| LEMENTS |   |                        |   |                    | Rev |   |     |
| OF      |   |                        |   |                    | iew |   |     |
| ELIG    |   |                        |   |                    | N   |   |     |
| IBILITY |   |                        |   |                    | o** |   |     |
| AND     |   |                        |   |                    |     |   |     |
| PAYMENT |   |                        |   |                    |     |   |     |
| D       |   |                        |   |                    |     |   |     |
| ETERMIN |   |                        |   |                    |     |   |     |
| ATION** |   |                        |   |                    |     |   |     |
+---------+---+------------------------+---+--------------------+-----+---+-----+
| **E     |   | **QC ANALYSIS OF CASE  | * |                    |     |   | **C |
| LEMENTS |   | RECORD**               | * |                    |     |   | C** |
| OF      |   |                        | F |                    |     |   |     |
| ELIG    |   | ***(Pertinent facts,   | I |                    |     |   |     |
| IBILITY |   | sources of             | N |                    |     |   |     |
| AND**   |   | verification,          | D |                    |     |   |     |
|         |   | reliability, gaps or   | I |                    |     |   |     |
| **      |   | deficiencies)***       | N |                    |     |   |     |
| PAYMENT |   |                        | G |                    |     |   |     |
| D       |   |                        | S |                    |     |   |     |
| ETERMIN |   |                        | O |                    |     |   |     |
| ATION** |   |                        | F |                    |     |   |     |
|         |   |                        | F |                    |     |   |     |
|         |   |                        | I |                    |     |   |     |
|         |   |                        | E |                    |     |   |     |
|         |   |                        | L |                    |     |   |     |
|         |   |                        | D |                    |     |   |     |
|         |   |                        | I |                    |     |   |     |
|         |   |                        | N |                    |     |   |     |
|         |   |                        | V |                    |     |   |     |
|         |   |                        | E |                    |     |   |     |
|         |   |                        | S |                    |     |   |     |
|         |   |                        | T |                    |     |   |     |
|         |   |                        | I |                    |     |   |     |
|         |   |                        | G |                    |     |   |     |
|         |   |                        | A |                    |     |   |     |
|         |   |                        | T |                    |     |   |     |
|         |   |                        | I |                    |     |   |     |
|         |   |                        | O |                    |     |   |     |
|         |   |                        | N |                    |     |   |     |
|         |   |                        | * |                    |     |   |     |
|         |   |                        | * |                    |     |   |     |
|         |   |                        |   |                    |     |   |     |
|         |   |                        | * |                    |     |   |     |
|         |   |                        | * |                    |     |   |     |
|         |   |                        | * |                    |     |   |     |
|         |   |                        | ( |                    |     |   |     |
|         |   |                        | F |                    |     |   |     |
|         |   |                        | a |                    |     |   |     |
|         |   |                        | c |                    |     |   |     |
|         |   |                        | t |                    |     |   |     |
|         |   |                        | s |                    |     |   |     |
|         |   |                        | o |                    |     |   |     |
|         |   |                        | b |                    |     |   |     |
|         |   |                        | t |                    |     |   |     |
|         |   |                        | a |                    |     |   |     |
|         |   |                        | i |                    |     |   |     |
|         |   |                        | n |                    |     |   |     |
|         |   |                        | e |                    |     |   |     |
|         |   |                        | d |                    |     |   |     |
|         |   |                        | , |                    |     |   |     |
|         |   |                        | v |                    |     |   |     |
|         |   |                        | e |                    |     |   |     |
|         |   |                        | r |                    |     |   |     |
|         |   |                        | i |                    |     |   |     |
|         |   |                        | f |                    |     |   |     |
|         |   |                        | i |                    |     |   |     |
|         |   |                        | c |                    |     |   |     |
|         |   |                        | a |                    |     |   |     |
|         |   |                        | t |                    |     |   |     |
|         |   |                        | i |                    |     |   |     |
|         |   |                        | o |                    |     |   |     |
|         |   |                        | n |                    |     |   |     |
|         |   |                        | a |                    |     |   |     |
|         |   |                        | n |                    |     |   |     |
|         |   |                        | d |                    |     |   |     |
|         |   |                        | s |                    |     |   |     |
|         |   |                        | u |                    |     |   |     |
|         |   |                        | b |                    |     |   |     |
|         |   |                        | s |                    |     |   |     |
|         |   |                        | t |                    |     |   |     |
|         |   |                        | a |                    |     |   |     |
|         |   |                        | n |                    |     |   |     |
|         |   |                        | t |                    |     |   |     |
|         |   |                        | i |                    |     |   |     |
|         |   |                        | a |                    |     |   |     |
|         |   |                        | t |                    |     |   |     |
|         |   |                        | i |                    |     |   |     |
|         |   |                        | o |                    |     |   |     |
|         |   |                        | n |                    |     |   |     |
|         |   |                        | , |                    |     |   |     |
|         |   |                        | n |                    |     |   |     |
|         |   |                        | a |                    |     |   |     |
|         |   |                        | t |                    |     |   |     |
|         |   |                        | u |                    |     |   |     |
|         |   |                        | r |                    |     |   |     |
|         |   |                        | e |                    |     |   |     |
|         |   |                        | o |                    |     |   |     |
|         |   |                        | r |                    |     |   |     |
|         |   |                        | e |                    |     |   |     |
|         |   |                        | r |                    |     |   |     |
|         |   |                        | r |                    |     |   |     |
|         |   |                        | o |                    |     |   |     |
|         |   |                        | r |                    |     |   |     |
|         |   |                        | s |                    |     |   |     |
|         |   |                        | ) |                    |     |   |     |
|         |   |                        | * |                    |     |   |     |
|         |   |                        | * |                    |     |   |     |
|         |   |                        | * |                    |     |   |     |
+---------+---+------------------------+---+--------------------+-----+---+-----+
| **360   | * |                        |   | **Provider         |     | 1 |     |
| P       | * |                        |   | requirements were  |     |   |     |
| ROVIDER | P |                        |   | met: \_\_\_ Yes    |     | 2 |     |
| REQUIRE | r |                        |   | \_\_\_ No**        |     | C |     |
| MENTS** | o |                        |   |                    |     |   |     |
|         | v |                        |   | **If no,           |     | 3 |     |
|         | i |                        |   | explain:**         |     | P |     |
|         | d |                        |   |                    |     |   |     |
|         | e |                        |   | **Child with in    |     |   |     |
|         | r |                        |   | home provider and  |     |   |     |
|         | o |                        |   | provider was       |     |   |     |
|         | n |                        |   | enrolled for that  |     |   |     |
|         | P |                        |   | child (see memo    |     |   |     |
|         | R |                        |   | from Regional      |     |   |     |
|         | L |                        |   | provider           |     |   |     |
|         | I |                        |   | manager)**         |     |   |     |
|         | ( |                        |   |                    |     |   |     |
|         | e |                        |   | **\_ Yes \_ No \_  |     |   |     |
|         | n |                        |   | NA**               |     |   |     |
|         | r |                        |   |                    |     |   |     |
|         | o |                        |   | **Child in the     |     |   |     |
|         | l |                        |   | home of a relative |     |   |     |
|         | l |                        |   | provider and is    |     |   |     |
|         | e |                        |   | related to that    |     |   |     |
|         | d |                        |   | provider (verified |     |   |     |
|         | a |                        |   | ) \_\_\_\_ Yes     |     |   |     |
|         | s |                        |   | \_\_\_\_ No \_\_\_ |     |   |     |
|         | q |                        |   | NA**               |     |   |     |
|         | u |                        |   |                    |     |   |     |
|         | a |                        |   |                    |     |   |     |
|         | l |                        |   |                    |     |   |     |
|         | i |                        |   |                    |     |   |     |
|         | f |                        |   |                    |     |   |     |
|         | i |                        |   |                    |     |   |     |
|         | e |                        |   |                    |     |   |     |
|         | d |                        |   |                    |     |   |     |
|         | p |                        |   |                    |     |   |     |
|         | r |                        |   |                    |     |   |     |
|         | o |                        |   |                    |     |   |     |
|         | v |                        |   |                    |     |   |     |
|         | i |                        |   |                    |     |   |     |
|         | d |                        |   |                    |     |   |     |
|         | e |                        |   |                    |     |   |     |
|         | r |                        |   |                    |     |   |     |
|         | ) |                        |   |                    |     |   |     |
|         | . |                        |   |                    |     |   |     |
|         | \ |                        |   |                    |     |   |     |
|         | _ |                        |   |                    |     |   |     |
|         | \ |                        |   |                    |     |   |     |
|         | _ |                        |   |                    |     |   |     |
|         | Y |                        |   |                    |     |   |     |
|         | e |                        |   |                    |     |   |     |
|         | s |                        |   |                    |     |   |     |
|         | \ |                        |   |                    |     |   |     |
|         | _ |                        |   |                    |     |   |     |
|         | \ |                        |   |                    |     |   |     |
|         | _ |                        |   |                    |     |   |     |
|         | \ |                        |   |                    |     |   |     |
|         | _ |                        |   |                    |     |   |     |
|         | N |                        |   |                    |     |   |     |
|         | O |                        |   |                    |     |   |     |
|         | * |                        |   |                    |     |   |     |
|         | * |                        |   |                    |     |   |     |
|         |   |                        |   |                    |     |   |     |
|         | * |                        |   |                    |     |   |     |
|         | * |                        |   |                    |     |   |     |
|         | P |                        |   |                    |     |   |     |
|         | R |                        |   |                    |     |   |     |
|         | R |                        |   |                    |     |   |     |
|         | 2 |                        |   |                    |     |   |     |
|         | i |                        |   |                    |     |   |     |
|         | s |                        |   |                    |     |   |     |
|         | a |                        |   |                    |     |   |     |
|         | n |                        |   |                    |     |   |     |
|         | o |                        |   |                    |     |   |     |
|         | t |                        |   |                    |     |   |     |
|         | h |                        |   |                    |     |   |     |
|         | e |                        |   |                    |     |   |     |
|         | r |                        |   |                    |     |   |     |
|         | h |                        |   |                    |     |   |     |
|         | e |                        |   |                    |     |   |     |
|         | l |                        |   |                    |     |   |     |
|         | p |                        |   |                    |     |   |     |
|         | f |                        |   |                    |     |   |     |
|         | u |                        |   |                    |     |   |     |
|         | l |                        |   |                    |     |   |     |
|         | s |                        |   |                    |     |   |     |
|         | c |                        |   |                    |     |   |     |
|         | r |                        |   |                    |     |   |     |
|         | e |                        |   |                    |     |   |     |
|         | e |                        |   |                    |     |   |     |
|         | n |                        |   |                    |     |   |     |
|         | * |                        |   |                    |     |   |     |
|         | * |                        |   |                    |     |   |     |
+---------+---+------------------------+---+--------------------+-----+---+-----+

+---------+------------------------+----------------------+----+------+
| **E     |                        |                      | ** |      |
| LEMENTS |                        |                      | Re |      |
| OF      |                        |                      | vi |      |
| ELIG    |                        |                      | ew |      |
| IBILITY |                        |                      | No |      |
| AND     |                        |                      | ** |      |
| PAYMENT |                        |                      |    |      |
| D       |                        |                      |    |      |
| ETERMIN |                        |                      |    |      |
| ATION** |                        |                      |    |      |
+---------+------------------------+----------------------+----+------+
| **E     | **QC ANALYSIS OF CASE  | **FINDINGS OF FIELD  |    | **   |
| LEMENTS | RECORD**               | INVESTIGATION**      |    | CC** |
| OF      |                        |                      |    |      |
| ELIG    | ***(Pertinent facts,   | ***(Facts obtained,  |    |      |
| IBILITY | sources of             | verification and     |    |      |
| AND**   | verification,          | substantiation,      |    |      |
|         | reliability, gaps or   | nature or errors)*** |    |      |
| **      | deficiencies)***       |                      |    |      |
| PAYMENT |                        |                      |    |      |
| D       |                        |                      |    |      |
| ETERMIN |                        |                      |    |      |
| ATION** |                        |                      |    |      |
+---------+------------------------+----------------------+----+------+
|         | **SECTION IV. INCOME   |                      |    |      |
|         | AND PAYMENTS**         |                      |    |      |
+---------+------------------------+----------------------+----+------+
| **400   | **EMPLOYMENT**         | **Case record**      |    | 1    |
| I       |                        |                      |    |      |
| NCOME** | **#\_\_\_ Employer of  |                      |    | 2 C  |
|         | Record:                |                      |    |      |
|         | \_\_\_\_\_\_\_\_\_\_\_ |                      |    | 3 P  |
|         | \_\_\_\_\_\_\_\_\_\_\_ |                      |    |      |
|         | \_\_\_\_\_\_\_\_\_\_** |                      |    |      |
|         |                        |                      |    |      |
|         | **#\_\_\_ Employer of  |                      |    |      |
|         | Record:                |                      |    |      |
|         | \_\_\_\_\_\_\_\_\_\_\_ |                      |    |      |
|         | \_\_\_\_\_\_\_\_\_\_\_ |                      |    |      |
|         | \_\_\_\_\_\_\_\_\_\_** |                      |    |      |
|         |                        |                      |    |      |
|         | **Calculation:**       |                      |    |      |
|         |                        |                      |    |      |
|         | **CHILD SUPPORT**      |                      |    |      |
|         |                        |                      |    |      |
|         | **Budgeted:            |                      |    |      |
|         | \_\_\_\_\_\_\_**       |                      |    |      |
|         |                        |                      |    |      |
|         | **Calculation:**       |                      |    |      |
|         |                        |                      |    |      |
|         | **\_\_\_ No changes    |                      |    |      |
|         | AORD**                 |                      |    |      |
|         |                        |                      |    |      |
|         | **\_\_\_ Change        |                      |    |      |
|         | reported on            |                      |    |      |
|         | \_                     |                      |    |      |
|         | \_\_\_\_\_\_\_\_\_\_\_ |                      |    |      |
|         | acted on               |                      |    |      |
|         | \_\_                   |                      |    |      |
|         | \_\_\_\_\_\_\_\_\_\_** |                      |    |      |
+---------+------------------------+----------------------+----+------+
| **410   | **Household            | **Financial          |    | 1    |
| INCOME  | financially eligible:  | eligibility & family |    |      |
| ELIGIB  | \_\_\_ Yes \_\_\_ No** | fee were determined  |    | 2 C  |
| ILITY** |                        | correctly:**         |    |      |
|         | **Family Fee:          |                      |    | 3 P  |
|         | \_\_\_\_\_\_\_**       | **\_\_\_ Yes \_\_\_  |    |      |
|         |                        | No**                 |    |      |
|         |                        |                      |    |      |
|         |                        | **Corrected family   |    |      |
|         |                        | fee:                 |    |      |
|         |                        | \_\_\_\_\_\_\_**     |    |      |
|         |                        |                      |    |      |
|         |                        | **Explain:**         |    |      |
+---------+------------------------+----------------------+----+------+
| **420   | **Payments made on     | **Payments were made |    | 1    |
| PA      | CHCP:                  | correctly: \_\_\_    |    |      |
| YMENTS, | \_\_\_\_\_\_\_\_\_\_** | Yes \_\_\_ No**      |    | 2 C  |
| GE      |                        |                      |    |      |
| NERAL** |                        | **If no, explain:**  |    | 3 P  |
+---------+------------------------+----------------------+----+------+
| **430   | **Case manager used    | **\_\_\_ Correct**   |    | 1    |
| P       | conversion 4.3, 2.15,  |                      |    |      |
| AYMENTS | ?, : Comments:**       | **\_\_\_ Overpayment |    | 2 C  |
| /       |                        | of:                  |    |      |
| COMPUTA |                        | \_\_\_               |    | 3 P  |
| TIONS** |                        | \_\_\_\_\_\_\_\_\_** |    |      |
|         |                        |                      |    |      |
|         |                        | **\_\_\_             |    |      |
|         |                        | Underpayment of:     |    |      |
|         |                        | \_\_\_               |    |      |
|         |                        | \_\_\_\_\_\_\_\_\_** |    |      |
+---------+------------------------+----------------------+----+------+

[]{#__RefHeading___Toc167079376 .anchor}**Appendix C. New Jersey**

[]{#__RefHeading___Toc167079377 .anchor}**A. Organizational Structure**

The New Jersey Department of Human Services (DHS), Division of Family
Development (DFD), is the State agency responsible for the overall
administration of the Child Care Development Fund. DFD works in
collaboration with the Division of Youth and Family Services (DYFS) and
the DHS Office of Licensing in the administration and operation of the
program. In addition, DFD supervises supportive services including
(child care) for Work First New Jersey (WFNJ), New Jersey's Temporary
Assistance for Needy Families (TANF), and post--TANF participants under
New Jersey's Transitional Child Care (TCC) program.

DFD currently contracts with 16 Child Care Resource and Referral (CCR&R)
agencies, 14 of which are non-profit community based agencies, and two
units of local government to administer and coordinate the CCDF subsidy
programs and other child care initiatives in each of New Jersey's 21
counties. DFD also contracts directly with approximately 480 contracted
child care provider agencies. These agencies provide over 36,000
subsidized child care slots on a statewide basis annually.

Program staff in DFD periodically review and set the standards for
operation of all aspects of child care services and related programs,
including all contracted child care agencies and providers. The CCR&R
agencies and approximately 180 center based contract child care
providers (CBCs) determine the eligibility of families and the
availability of child care service funds for all DHS administered child
care service programs.

Staffs of the 21 New Jersey County Welfare Agencies (CWAs) and Boards of
Social Services (BSSs) are responsible for determining eligibility for
child care services for TANF families. Referrals of eligible
participants with the appropriate income and work activities are made to
one of the CCR&R agencies to obtain the child care services.

[]{#__RefHeading___Toc167079378 .anchor}**B. Overview of the Child Care
Eligibility Process**

To be eligible for child care assistance, families must have an income
less than 200 percent of the Federal Poverty Level (FPL) for new cases,
and up to 250 percent of the FPL for on-going cases, re-determined
annually, be employed or in an approved education or training activity,
and cooperate with child support enforcement when applicable.

A re-determination of the subsidy amount occurs when there is a change
in the family size, provider type, income, hours of
employment/training/education, and hours of care needed or the number of
children receiving services. If such a re-determination is warranted,
the CCR&R agency must notify the family that all changes become
effective with the beginning of the next full month of service. In these
instances a new Parent/Applicant Provider Agreement (PAPA) must be
completed and signed.

Through case monitoring or based on reported changes, the CCR&R agency
must assess changes in family circumstances, employment, or training and
determine whether a reassessment of the program eligibility, subsidy
amount, and/or co-pay amount is necessary. The parent is required to
provide the required verification of the change in circumstance.

When the family reports a change, the contracted agency inputs the new
information into the Child Care Resources Eligibility System (CARES)
automated system. The CCR&R agency uses this information and all
required supporting documentation to verify client eligibility. All
CCR&R agency actions are also documented and maintained in the family
case record.

When the CCR&R agency is notified of a change in family circumstance
involving a layoff, termination from employment, or a break in
participation in an education or training program, the family may
continue to receive services for 30 days.

[]{#__RefHeading___Toc167079379 .anchor}**C. Improper Payment Process**

New Jersey defines a client overpayment as funds that have been
erroneously paid on behalf of a family to a child care provider,
whenever the family or child was ineligible to receive the issued
benefit amount. Examples include:

-   Non-reporting or underreporting of income;

-   Receiving payments in more than one jurisdiction;

-   Incorrect reporting of household size;

-   Incorrect information on compliance with program; and

-   Other program requirements, such as participating in required
    activity.

Provider overpayments are funds that have been erroneously paid but the
eligibility of the family or child(ren) is not affected. These
overpayments may result from intentional or unintentional errors in the
completion of the voucher made by the provider(s) or the CCR&R agency.
Examples include:

-   Overstating level of service (e.g., child attendance); and

-   Claims for services not rendered (e.g., child enrollment).

Whenever an overpayment occurs, the CCR&R agency recovers the entire
amount of the overpayment to the best of its ability and whenever cost
effective. The overpayment amount is a portion of the total subsidy paid
to the child care provider. After determining the overpayment, the CCR&R
agency establishes a repayment agreement with the applicant or the
provider following issuance of a written notice advising them of the
overpayment. Repayment may occur through either a lump sum payment or
monthly payments. If the family or provider indicates that repayment
within the current eligibility period will create a hardship, the CCR&R
agency is authorized to negotiate a longer repayment period. DFD must
approve all repayment agreements that exceed one year.

> **1. Strategies to prevent or identify improper payments**
>
> New Jersey uses several strategies to prevent or identify improper
> payments, including:

-   Twice a year monitoring of the CCR&R agencies and CBCs to review
    randomly selected case records for completeness and accuracy;

-   Establishing a Quality Assurance in Child Care committee that
    includes systems and program staff to develop policies and
    procedures to decrease the potential for improper payments;

-   Establishing a new Program Integrity Unit to oversee work on using
    technology such as, data matching and shared data warehouse
    technology to identify improper payments;

```{=html}
<!-- -->
```
-   Strengthening the voucher system through upgraded security measures
    through enhancement of internal controls and by CCR&R supervisory
    control of the voucher approval process;

-   Data matching on a quarterly basis of client information from the
    CARES and CTRX systems with the Income Eligibility Verification
    System (IEVS);

-   Data matching on a quarterly basis of client information from the
    CARES and CTRX systems with the OMEGA (TANF) system; and

-   Using data sources to better ensure accurate payments for WFNJ/TANF
    and TCC child care services, such as:

```{=html}
<!-- -->
```
-   Income Eligibility Verification System (IEVS);

-   Public Assistance Reporting Information System (PARIS);

-   Social Security Administration (SSA) form W-2 (wage statements);

-   SSA Social Security number verification;

-   SSA Supplemental Security Income (SSI) data; and

-   SSI death information.

    []{#__RefHeading___Toc167079380 .anchor}**D. Case Review Process**

After receipt of technical assistance during the site visit, New Jersey
substantially customized the *Record Review Worksheet* template to
include coding and specific boilerplate language to guide the reviewer's
written case narrative. After consulting with the contractor during the
site visit, New Jersey determined that the statewide sample could not
include the cases utilizing contracted child care slots. These child
care cases did not meet pilot requirements for the unit of analysis of
an authorized payment on a per child basis utilizing the voucher system.
New Jersey submitted a second universe to the contractor to select a new
statewide sample that included only cases using a child care voucher and
appearing in the Child Care Automated Resources Eligibility System
(CARES). New Jersey trained its Review Team centrally and then reviewers
conducted the reviews in their assigned regions at the local Child Care
Resource and Referrals (CCR&R) agencies. The Review Team consisted of
three supervisors and 13 Child Care Specialists.

[]{#__RefHeading___Toc167079381 .anchor}**E. Automation**

A major challenge for New Jersey is that information on child care
clients resides in three different databases. Work First New Jersey
(WFNJ/TANF) case information resides on the OMEGA (TANF) system, New
Jersey Care for Kids (NJCK) CCR&R case information resides in CARES, the
child care automated voucher payments system, and the information on
clients receiving Center Based Contract (CBC) resides on the Contracts
System (CTRX) and Contracts Accounting Tracking System (CATS). None of
these systems "talk" to one another, requiring the periodic generation
of matching reports to verify client household and income information
with third party or outside income databases.

CARES is a system custom designed for the State. The system is a single
Oracle platform using a UNIX operating system. The application includes
client application intake, eligibility determination, agreement
establishment, subsidy and co-pay calculation, voucher generation,
attendance input, check writing process, accounting procedures, letters,
notices, forms, and reports.

[]{#__RefHeading___Toc167079382 .anchor}**F. Next Steps**

For correction and reduction of future improper payments, New Jersey
will:

-   Allocate time during meetings with the Child Care Resource and
    Referral (CCR&R) Directors meeting to provide feedback and direction
    on implementing corrective measures.

-   Institute the Error Rate Pilot monitoring tool for use of all future
    file reviews, in addition to increasing the number of files for
    review.

-   Provide technical assistance to CCR&R agency staff in those areas
    where improper payments were detected in file reviews.

-   Improve lines of communication between Division of Youth and Family
    Services (DYFS), the lead State child protective service agency and
    the Division of Family Development, the lead State child care
    agency, and the CCR&R agencies at State and local levels.

-   Look into implementing changes in the CARES system to better detect
    data entry errors or create error messages when data entry errors
    occur.

-   Implement a process to conduct electronic matching of the automated
    child care client database with records on other wage, SSI, and
    child support information systems to identify and reduce the number
    of improper payments.

```{=html}
<!-- -->
```
-   Hire quality assurance staff or allocate the FTE needed to enable
    the Division to conduct file reviews, using a methodology that
    guarantees a statistically valid sample size of randomly selected
    cases.

-   Link reduced improper payments to penalties in Child Care Resource
    and Referral contracts.

    []{#__RefHeading___Toc167079383 .anchor}**New Jersey Site Visit
    Participants**

Beverly Wellons, Assistant Director for Child Care Operations and State
Child Care Administrator, DFD

George Kobil, Program & Policy Analyst, Child Care Operations, DFD

Elaine Mullen, Supervisor, Child Care Operations, DFD

Maribel Steath, Supervisor, Child Care Operations, DFD

Shonda Laurel, Supervisor, Child Care Operations, DFD

Cammelia Bacon-Cassagnol, Child Care Specialist, DFD

Sharrelle Blue, Child Care Specialist, DFD

LaTasha Brown, Child Care Specialist, DFD

Shaaron Crossland, Child Care Specialist, DFD

Debra Freeman, Child Care Specialist, DFD

Betty Griffin, Child Care Specialist, DFD

Sharon McBride, Child Care Specialist, DFD

Traci McMillan, Child Care Specialist, DFD

Andrea Masella, Child Care Specialist, DFD

Lisa Ojibway, Child Care Specialist, DFD

Reyland Malvern, Child Care Specialist, DFD

Andrea Petranto, Child Care Specialist, DFD

Randy Singer, Child Care Specialist, DFD

[]{#__RefHeading___Toc167078156 .anchor}**Exhibit C-1. New Jersey Record
Review Worksheet**

+-----+---+----------------+---+-------------+-----+---+--------+------+
| *   |   |                | * |             |     |   |        |      |
| *A. |   |                | * |             |     |   |        |      |
| ID  |   |                | B |             |     |   |        |      |
| ENT |   |                | . |             |     |   |        |      |
| IFY |   |                | P |             |     |   |        |      |
| ING |   |                | E |             |     |   |        |      |
| I   |   |                | R |             |     |   |        |      |
| NFO |   |                | S |             |     |   |        |      |
| RMA |   |                | O |             |     |   |        |      |
| TIO |   |                | N |             |     |   |        |      |
| N** |   |                | S |             |     |   |        |      |
|     |   |                | I |             |     |   |        |      |
|     |   |                | N |             |     |   |        |      |
|     |   |                | T |             |     |   |        |      |
|     |   |                | H |             |     |   |        |      |
|     |   |                | E |             |     |   |        |      |
|     |   |                | F |             |     |   |        |      |
|     |   |                | A |             |     |   |        |      |
|     |   |                | M |             |     |   |        |      |
|     |   |                | I |             |     |   |        |      |
|     |   |                | L |             |     |   |        |      |
|     |   |                | Y |             |     |   |        |      |
|     |   |                | * |             |     |   |        |      |
|     |   |                | * |             |     |   |        |      |
+-----+---+----------------+---+-------------+-----+---+--------+------+
| 1\. |   |                | N |             | DOB | A | RELATI | SO   |
| CCR |   |                | A |             |     | G | ONSHIP | CIAL |
| &R: |   |                | M |             |     | E |        | SECU |
|     |   |                | E |             |     |   | SIGNIF | RITY |
|     |   |                |   |             |     |   | ICANCE | NU   |
|     |   |                |   |             |     |   |        | MBER |
+-----+---+----------------+---+-------------+-----+---+--------+------+
| 2\. |   |                | 1 |             |     |   |        |      |
| C   |   |                | \ |             |     |   |        |      |
| ase |   |                | . |             |     |   |        |      |
| Na  |   |                |   |             |     |   |        |      |
| me: |   |                |   |             |     |   |        |      |
+-----+---+----------------+---+-------------+-----+---+--------+------+
| 3\. |   |                | 2 |             |     |   |        |      |
| A   |   |                | \ |             |     |   |        |      |
| ddr |   |                | . |             |     |   |        |      |
| ess |   |                |   |             |     |   |        |      |
+-----+---+----------------+---+-------------+-----+---+--------+------+
|     |   |                | 3 |             |     |   |        |      |
|     |   |                | \ |             |     |   |        |      |
|     |   |                | . |             |     |   |        |      |
+-----+---+----------------+---+-------------+-----+---+--------+------+
|     |   |                | 4 |             |     |   |        |      |
|     |   |                | \ |             |     |   |        |      |
|     |   |                | . |             |     |   |        |      |
+-----+---+----------------+---+-------------+-----+---+--------+------+
| 4\. |   |                | 5 |             |     |   |        |      |
| Fam |   |                | \ |             |     |   |        |      |
| ily |   |                | . |             |     |   |        |      |
| ID  |   |                |   |             |     |   |        |      |
| N   |   |                |   |             |     |   |        |      |
| umb |   |                |   |             |     |   |        |      |
| er: |   |                |   |             |     |   |        |      |
+-----+---+----------------+---+-------------+-----+---+--------+------+
| 5\. |   |                | 6 |             |     |   |        |      |
| Rev |   |                | \ |             |     |   |        |      |
| iew |   |                | . |             |     |   |        |      |
| D   |   |                |   |             |     |   |        |      |
| ate |   |                |   |             |     |   |        |      |
+-----+---+----------------+---+-------------+-----+---+--------+------+
| 6\. |   |                | 7 |             |     |   |        |      |
| D   |   |                | \ |             |     |   |        |      |
| ate |   |                | . |             |     |   |        |      |
| of  |   |                |   |             |     |   |        |      |
| Ap  |   |                |   |             |     |   |        |      |
| pli |   |                |   |             |     |   |        |      |
| cat |   |                |   |             |     |   |        |      |
| ion |   |                |   |             |     |   |        |      |
+-----+---+----------------+---+-------------+-----+---+--------+------+
| 7\. |   |                | 8 |             |     |   |        |      |
| S   |   |                | \ |             |     |   |        |      |
| pec |   |                | . |             |     |   |        |      |
| ial |   |                |   |             |     |   |        |      |
| ist |   |                |   |             |     |   |        |      |
+-----+---+----------------+---+-------------+-----+---+--------+------+
| 8\. |   |                | 9 |             |     |   |        |      |
| D   |   |                | \ |             |     |   |        |      |
| ate |   |                | . |             |     |   |        |      |
| Ass |   |                |   |             |     |   |        |      |
| ign |   |                |   |             |     |   |        |      |
| ed: |   |                |   |             |     |   |        |      |
+-----+---+----------------+---+-------------+-----+---+--------+------+
| 9\. |   |                | 1 |             |     |   |        |      |
| D   |   |                | 0 |             |     |   |        |      |
| ate |   |                |   |             |     |   |        |      |
| of  |   |                |   |             |     |   |        |      |
| F   |   |                |   |             |     |   |        |      |
| ile |   |                |   |             |     |   |        |      |
| R   |   |                |   |             |     |   |        |      |
| evi |   |                |   |             |     |   |        |      |
| ew: |   |                |   |             |     |   |        |      |
+-----+---+----------------+---+-------------+-----+---+--------+------+
| 1   |   |                | 1 |             |     |   |        |      |
| 0\. |   |                | 1 |             |     |   |        |      |
| D   |   |                |   |             |     |   |        |      |
| ate |   |                |   |             |     |   |        |      |
| C   |   |                |   |             |     |   |        |      |
| omp |   |                |   |             |     |   |        |      |
| let |   |                |   |             |     |   |        |      |
| ed: |   |                |   |             |     |   |        |      |
+-----+---+----------------+---+-------------+-----+---+--------+------+
| 1   |   |                | 1 |             |     |   |        |      |
| 1\. |   |                | 2 |             |     |   |        |      |
| S   |   |                |   |             |     |   |        |      |
| upe |   |                |   |             |     |   |        |      |
| rvi |   |                |   |             |     |   |        |      |
| sor |   |                |   |             |     |   |        |      |
| S   |   |                |   |             |     |   |        |      |
| Ign |   |                |   |             |     |   |        |      |
| atu |   |                |   |             |     |   |        |      |
| re: |   |                |   |             |     |   |        |      |
+-----+---+----------------+---+-------------+-----+---+--------+------+
| 1   |   |                | 1 |             |     |   |        |      |
| 2\. |   |                | 3 |             |     |   |        |      |
| D   |   |                |   |             |     |   |        |      |
| ATE |   |                |   |             |     |   |        |      |
| APP |   |                |   |             |     |   |        |      |
| ROV |   |                |   |             |     |   |        |      |
| ED: |   |                |   |             |     |   |        |      |
+-----+---+----------------+---+-------------+-----+---+--------+------+

**c.** Review Findings:

+-----------------------------------------------------------------------+
| CO-PAYMENT AMOUNT: \$[\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_]{.underline} |
|                                                                       |
| (CIRCLE ONE)                                                          |
|                                                                       |
| AMOUNT CORRECT:                                                       |
|                                                                       |
| OVERPAMENT:                                                           |
|                                                                       |
| CASE INELIGIBLE:                                                      |
|                                                                       |
| AMOUNT IN ERROR: \$[\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_]{.underline}   |
+-----------------------------------------------------------------------+

  ----------------- ----------------- ----------------------- -------------
  **Child ID**      **STATE:**        **COUNTY:**             **DATE:**

                    **New Jersey**                            
  ----------------- ----------------- ----------------------- -------------

**SECTION I Child Care Eligibility Application**

+-------------------+--------------------+-------------------+---+------+
| **APPLICATION     | ANALYSIS OF CASE   | FINDINGS          | R |      |
| FORM**            | RECORD             |                   | E |      |
|                   |                    |                   | S |      |
|                   |                    |                   | U |      |
|                   |                    |                   | L |      |
|                   |                    |                   | T |      |
|                   |                    |                   | S |      |
+-------------------+--------------------+-------------------+---+------+
| **Is application  |                    |                   | 0 | NO   |
| present?**        |                    |                   |   | E    |
|                   |                    |                   |   | RROR |
| **Is it dated,    |                    |                   |   |      |
| signed and        |                    |                   |   |      |
| completed?**      |                    |                   |   |      |
|                   |                    |                   |   |      |
| **If the date of  |                    |                   |   |      |
| application is 12 |                    |                   |   |      |
| months or older,  |                    |                   |   |      |
| has the case been |                    |                   |   |      |
| re-determined     |                    |                   |   |      |
| timely?**         |                    |                   |   |      |
|                   |                    |                   |   |      |
| **Date of         |                    |                   |   |      |
| re-determination  |                    |                   |   |      |
| application.**    |                    |                   |   |      |
|                   |                    |                   |   |      |
| **Is the          |                    |                   |   |      |
| re-determination  |                    |                   |   |      |
| signed and        |                    |                   |   |      |
| completed?**      |                    |                   |   |      |
+-------------------+--------------------+-------------------+---+------+
|                   |                    |                   | 1 | CL   |
|                   |                    |                   |   | IENT |
|                   |                    |                   |   | E    |
|                   |                    |                   |   | RROR |
+-------------------+--------------------+-------------------+---+------+
|                   |                    |                   | 2 | AG   |
|                   |                    |                   |   | ENCY |
|                   |                    |                   |   | E    |
|                   |                    |                   |   | RROR |
+-------------------+--------------------+-------------------+---+------+

**SECTION II** GENERAL PROGRAM REQUIRMENTS FOR CASE RECORDS

  -------------------- -------------------- -------------------- --------- --------
  **QUALIFYING HEAD OF ANALYSIS OF CASE     FINDINGS             RESULTS   
  HOUSEHOLD**          RECORD                                              

  **Establish                                                    0         NO ERROR
  relationship.**                                                          

                                                                 1         CLIENT
                                                                           ERROR

                                                                 2         AGENCY
                                                                           ERROR
  -------------------- -------------------- -------------------- --------- --------

  -------------------- -------------------- -------------------- --------- --------
  **FAMILY SIZE**      ANALYSIS OF CASE     FINDINGS             RESULTS   
                       RECORD                                              

  **Determine family                                             0         NO ERROR
  size. Does family                                                        
  size information                                                         
  agree?**                                                                 

                                                                 1         CLIENT
                                                                           ERROR

                                                                 2         AGENCY
                                                                           ERROR
  -------------------- -------------------- -------------------- --------- --------

+-------------------+--------------------+-------------------+---+------+
| **FAMILY INCOME** | ANALYSIS OF CASE   | FINDINGS          | R |      |
|                   | RECORD             |                   | E |      |
|                   |                    |                   | S |      |
|                   |                    |                   | U |      |
|                   |                    |                   | L |      |
|                   |                    |                   | T |      |
|                   |                    |                   | S |      |
+-------------------+--------------------+-------------------+---+------+
| **Verify          |                    |                   | 0 | NO   |
| income.**         |                    |                   |   | E    |
|                   |                    |                   |   | RROR |
| **Include         |                    |                   |   |      |
| Co-payment        |                    |                   |   |      |
| Calculations.**   |                    |                   |   |      |
+-------------------+--------------------+-------------------+---+------+
|                   |                    |                   | 1 | CL   |
|                   |                    |                   |   | IENT |
|                   |                    |                   |   | E    |
|                   |                    |                   |   | RROR |
+-------------------+--------------------+-------------------+---+------+
|                   |                    |                   | 2 | AG   |
|                   |                    |                   |   | ENCY |
|                   |                    |                   |   | E    |
|                   |                    |                   |   | RROR |
+-------------------+--------------------+-------------------+---+------+

  -------------------- -------------------- -------------------- --------- --------
  **INCOME             ANALYSIS OF CASE     FINDINGS             RESULTS   
  ELIGIBLITY**         RECORD                                              

  **Document income                                              0         NO ERROR
  eligibility                                                              
  criteria.**                                                              

                                                                 1         CLIENT
                                                                           ERROR

                                                                 2         AGENCY
                                                                           ERROR
  -------------------- -------------------- -------------------- --------- --------

  ----------------------- -------------------- -------------------- --------- --------
  **ELIGIBLE              ANALYSIS OF CASE     FINDINGS             RESULTS   
  PARENT/APPLICANTS**     RECORD                                              

  **Document hours of                                               0         NO ERROR
  work and/or                                                                 
  education/training.**                                                       

                                                                    1         CLIENT
                                                                              ERROR

                                                                    2         AGENCY
                                                                              ERROR
  ----------------------- -------------------- -------------------- --------- --------

**SECTION III PRIORITY PLACEMENT QUESTIONS AND AGENCY OFFICAL USE**

  -------------------- -------------------- -------------------- --------- --------
  **PRIORITY           ANALYSIS OF CASE     FINDINGS             RESULTS   
  PLACEMENT**          RECORD                                              

  **Is Section "D" of                                            0         NO ERROR
  the application                                                          
  completed                                                                
  properly?**                                                              

                                                                 1         CLIENT
                                                                           ERROR

                                                                 2         AGENCY
                                                                           ERROR
  -------------------- -------------------- -------------------- --------- --------

**SECTION IV CHILD RECORD(S)**

  -------------------- -------------------- -------------------- --------- --------
  **ELIGIBLE           ANALYSIS OF CASE     FINDINGS             RESULTS   
  CHILD(REN)**         RECORD                                              

  **Document                                                     0         NO ERROR
  child(ren)'s age                                                         
  with services.**                                                         

                                                                 1         CLIENT
                                                                           ERROR

                                                                 2         AGENCY
                                                                           ERROR
  -------------------- -------------------- -------------------- --------- --------

**[New Jersey Child Care Error Rate Pilot]{.smallcaps}**

**APPENDIX C-1**

**DATA ENTRY FORM**

+------------------------------------------------+---------------------+
| **VARIABLE**                                   | **VALUE**           |
+------------------------------------------------+---------------------+
| **State**                                      | **New Jersey**      |
+------------------------------------------------+---------------------+
| **County**                                     |                     |
+------------------------------------------------+---------------------+
| **Child ID \[ID\]**                            |                     |
+------------------------------------------------+---------------------+
| **Study Period Month (October 2005)            |                     |
| \[PERIOD\]**                                   |                     |
+------------------------------------------------+---------------------+
| **Date of data collection \[DATE\]**           |                     |
+------------------------------------------------+---------------------+
| **One or more eligibility errors during pilot  |                     |
| period \[ERROR\]**                             |                     |
|                                                |                     |
| **0 =no errors, 1 = one or more errors**       |                     |
+------------------------------------------------+---------------------+
| **Cause of improper payment \[CAUSE\]**        |                     |
|                                                |                     |
| **0 =no errors, 1 = client, 2 = agency**       |                     |
+------------------------------------------------+---------------------+
| **Total amount of improper payments during     |                     |
| review month \[TIMP\]**                        |                     |
+------------------------------------------------+---------------------+
| **Total amount of payments during review month |                     |
| \[AMPAY\]**                                    |                     |
+------------------------------------------------+---------------------+
|                                                |                     |
+------------------------------------------------+---------------------+
| Note: The terms in brackets represent the      |                     |
| variables names in the THE                     |                     |
| CONTRACTOR-maintained database.                |                     |
+------------------------------------------------+---------------------+

**COMPUTING Potential Error Rates**

**This form contains nine fields:**

-   **State;**

-   **County of service;**

-   **Child ID;**

-   **Study period month (October 2005);**

-   **Date of data collection;**

-   **One or more administrative errors during the pilot period;**

-   **Cause of improper payment;**

-   **Total amount of improper payments during review month; and**

-   **Total amount of payments during review month.**

***Copies of all Data Entry Forms and each Record Review Worksheet will
be sent to the contractor. Using these data, the contractor will compute
three types of error rates---case error rate, payment error rate, and
average dollars spent in error per ineligible child.***

[]{#__RefHeading___Toc167079385 .anchor}**Appendix D. Oregon**

[]{#__RefHeading___Toc167079386 .anchor}**A. Organizational Structure**

The Child Care Division (CCD), which administers the child care program,
is within the Oregon Employment Department. CCD, in partnership with
Oregon's child care community, sets the State level priorities for
spending on child care services. It contracts with other agencies and
organizations to provide specific services. One of those agencies is the
Oregon Department of Human Services (DHS). DHS manages the integrated
child care subsidy program for children of low-income working parents
and post secondary student parents.

There are three types of care in the DHS child care subsidy program.
Through Certified Care the CCD certifies approximately 1000 child care
centers and 250 certified family homes. Centers care for more than 13
children in a non-residential facility. Certified family homes can care
for up to 16 children, including the provider's own children in a
residential setting. Registered Care is provided through approximately
5000 family child care businesses. These providers are registered for
two years at a time and can care for up to 10 children, including their
own, in their own home. Lastly, some child care is exempt from
regulation, including providers caring for three or fewer children,
providers caring for children from only one family, school district
programs, care provided in the child's home or by a relative of the
child, and limited duration programs such as summer camps.\
 \
In addition, CCD provides child care funding to over 200 community-based
organizations that serve the following high-risk families, including:

-   Teen parents participating in approved high school or GED completion
    programs;

```{=html}
<!-- -->
```
-   Parents participating in State-approved substance abuse programs;

```{=html}
<!-- -->
```
-   Migrant and seasonal farm workers; and

```{=html}
<!-- -->
```
-   Parents who have children with special needs.

In FY2005, average per month assistance was provided through 13,826
providers to over 21,300 children in 10,700 families. Approximately 78
percent of the children were served in family home settings.

[]{#__RefHeading___Toc167079387 .anchor}**B. Overview of the Child Care
Eligibility Process**

The CCD child care subsidy programs help low-income families achieve
self-sufficiency by providing the child care assistance necessary for
employment or participation in other activities that promote
self-sufficiency. Several types of child care assistance are combined to
form the Integrated Child Care Program (ICC). They are:

-   Employment Related Day Care (ERDC) assists low-income working
    parents. Parents in this program pay a share of the child care;

-   Temporary Assistance to Needy Families (TANF) child care assistance
    assists low-income working parents who are receiving TANF cash
    benefits; and

-   JOBS child care assists parents receiving TANF or Food Stamps pay
    for child care so they can participate in DHS-approved
    self-sufficiency activities.

To be eligible for ERDC families must be State residents and have an
income below 150 percent of the Federal Poverty Level (FPL). Children
must either be younger than 12 or under 18 and circumstances are such
that the child is unable to safely care for him or herself. Children
must be younger than 12 for ERDC and younger than 13 for the other
programs. Children up to age 17 can receive assistance if it is
determined that the child should not be unsupervised during the hours
the caretaker is working or participating in self-sufficiency
activities. At least one caretaker must receive income from employment,
including employment through a work study program. For clients who are
in the start-up phase of self-employment, working on commission, or
participating in job-related training that is a condition of employment,
the requirement to have earned income may be waived for three months.
The client's co-pay is calculated prospectively. Payment for services is
determined by the age of the child, geographic area, type of facility
(family or center), special needs, and the level of training of the
provider. The co-pay is assigned to the family rather than the child and
all of the family's co-pays are assigned to one provider.

Reauthorization of eligibility (Anticipating with Periodic Review/APR)
varies in length. The primary determinant of the length of the APR
period is the projected stability of the client's income. The most
common period is six months. Clients must report within 10 days those
changes that are expected to continue in either the source of income or
rate of pay. The month the change is effective depends on when during
the previous month the report is made and whether the change results in
a significant increase or decrease in the co-pay.

[]{#__RefHeading___Toc167079388 .anchor}**C. Improper Payment Process**

Oregon defines an improper payment as one made for care that was not
provided or one made to an ineligible provider or on behalf of a client
who was not eligible for the total amount paid. Oregon identifies
improper payments through a variety of methods, including:

-   A monthly desk audit of 200 randomly selected billing forms;

-   A monthly match with TANF, Food Stamps, and Medicaid to identify
    ineligible providers and questionable child care payments;

-   Reports via the hot line number included in notices sent to clients
    and providers; and

-   Monthly record reviews of 5000 Food Stamp cases. Approximately 95
    percent of child care recipients receive Food Stamps. In these cases
    both the Food Stamp and child care subsidy eligibility is subject to
    review. Oregon plans on expanding a database that currently contains
    the Food Stamp review information to capture Child Care review
    results.

> **1. Suspected Fraud**
>
> Within the DHS Office of Payment Accuracy, three staff members in the
> Overpayment Writing Unit are responsible for completing the
> calculation of child care overpayments. Legal action may result in
> addition to collection of the overpayment. The Overpayment Writing
> Unit may refer cases to the Investigations Unit to establish intent
> based on a pattern of behavior, clear falsification, or behavior
> contrary to reasonable expectation.
>
> Client fraud occurs when a client attempts to establish eligibility,
> increase the amount of assistance, or prevent a reduction by:

-   Intentionally making a false or misleading statement;

-   Misrepresenting, concealing, or withholding facts; or

-   Intentionally taking or failing to take an action, such as leaving a
    child in care when not participating in activities authorized by
    DHS.

> Provider fraud occurs when the provider intentionally:

-   Bills for more child care than actually provided, such as duplicate
    billing;

-   Collects payment directly from a client after the DHS\'s payment for
    its services has been garnished or after overpayment recovery
    actions have reduced the amount paid;

-   Fails to list required persons on the provider application to
    participate in the subsidy program; and

-   Applies for participation on behalf of another provider who would
    not be eligible to participate.

> Once calculated, DHS recovers the overpayments through withholding a
> portion of the authorized monthly payment or through direct repayment
> by the customer. In the case of provider fraud, the payment is
> recouped at the 100 percent level.
>
> In situations of a closed case, the Overpayment Writing Unit sends the
> customer a series of recovery notices. In instances where there is
> either no response to the notices or no payment, the case is referred
> to the Department of Revenue which uses tax intercepts and private
> collection action to recover overpayments.

[]{#__RefHeading___Toc167079389 .anchor}**D. Case Reviews**

Oregon made very few modifications to the *Record Review Worksheet*
template. The Review Team consisted of five Quality Control (QC) Staff
Persons who have responsibility for TANF and FS case reviews. Following
training on child care policy and procedures, the Review Team reviewed
the cases centrally. The Project Team Leader reviewed all cases with
errors a second time and discovered that the Review Team went beyond the
level of a desk audit, expecting greater detail in documentation than
what had been anticipated or required of the child care review process.
As a result, the Review Team reviewed a subset of cases a third time and
changed some review results accordingly.

[]{#__RefHeading___Toc167079390 .anchor}**E. Automation**

DHS uses several automated systems in support of the child care program.
The Client Maintenance System (CMS) is used in determining eligibility
for TANF, Medicaid, Food Stamps, and ERDC. The Service Authorization
System is the subsidy payment system for ERDC cases. Both systems
contain various edits, tables, and charts to support payment accuracy.
The ERDC application process includes a match with other DHS systems as
well as child support, Employment Department wage and unemployment
records, and the Social Security Benefit system. CMS calculates the
provider payment based on the family's co-pay, the provider's usual
charge, hours authorized and billed, age of the child(ren), type of
care, and provider's zip code.

[]{#__RefHeading___Toc167079391 .anchor}**F. Next Steps**

After reviewing the improper payment data Oregon listed the following
corrective action strategies:

-   Currently, there is no process for ensuring that income errors
    identified in Food Stamp cases are dealt with in the companion
    Employment Related Day Care (ERDC) case. Oregon plans to modify the
    Food Stamp targeted review process and database that contains review
    information to ensure that income errors identified in any Food
    Stamp case are corrected in companion ERDC case.

-   Each year, the Oregon Department of Human Services (DHS) conducts in
    all areas of the State a series of "Accuracy Summits" that focus on
    techniques to reduce errors in the Food Stamp program. DHS plans to
    include workshops to improve ERDC accuracy in the next series of
    summits that begins in July 2007.

-   DHS produces a monthly newsletter that focuses on techniques to
    reduce Food Stamp errors. The newsletter will now include
    information about other programs, including ERDC. A recent
    newsletter included an article about correctly determining co-pay
    amounts in ERDC cases. Future newsletters will contain information
    about ERDC payment accuracy, including an article about the results
    of the pilot review.

-   The Governor's recommended budget for the next biennium includes
    increased funding to build on the efforts that have worked to
    improve accuracy in the Food Stamp program. This funding would allow
    DHS to extend Targeted Review outcome measures to the Child Care
    programs in local offices by FY2008. This comprehensive Program
    Integrity infrastructure provides a method for gathering timely,
    local performance data and will enable the Department to develop
    corrective action measures.

    []{#__RefHeading___Toc167079392 .anchor}**Oregon Site Visit
    Participants**

Mark Anderson -- Manager, Child Care and Refugee Programs

Cassie Day -- Program Analyst/Trainer CAF Child Care Unit

Rhonda Prodzinski -- Program Analyst CAF Child Care Unit

Carol Bennett -- Overpayment Writer CAF

Lisa Stegman -- Business Analyst CAF Program Systems Support

Peggy Cain -- Program Trainer

Kathy Van-Brocklin -- Operations Manager Direct Pay Unit

Janet Dornhecker -- Program Manager CAF Food Stamp Reinvestment Unit

Robert O'Shea -- Manager, Overpayment Writing Unit

Chris Bravo -- Investigator

[]{#__RefHeading___Toc167078165 .anchor}**Exhibit D-1. Oregon Record
Review Worksheet**

+-------------------------+------------------+-----------------+---+----+
| > **CLIENT ID#**        | > **STATE:       | > **COUNTY:**   | * |    |
|                         | > Oregon**       |                 | * |    |
|                         |                  |                 | D |    |
|                         |                  |                 | A |    |
|                         |                  |                 | T |    |
|                         |                  |                 | E |    |
|                         |                  |                 | : |    |
|                         |                  |                 | * |    |
|                         |                  |                 | * |    |
+=========================+==================+=================+===+====+
|                         |                  |                 |   |    |
+-------------------------+------------------+-----------------+---+----+
| > **ELEMENTS OF         | > **ANALYSIS OF  | > **FINDINGS    | * |    |
| > ELIGIBILITY & PAYMENT | > CASE RECORD    | > (3)**         | * |    |
| > DETERMINATION (1)**   | > (2)**          |                 | R |    |
|                         |                  |                 | E |    |
|                         |                  |                 | S |    |
|                         |                  |                 | U |    |
|                         |                  |                 | L |    |
|                         |                  |                 | T |    |
|                         |                  |                 | S |    |
|                         |                  |                 | ( |    |
|                         |                  |                 | 4 |    |
|                         |                  |                 | ) |    |
|                         |                  |                 | * |    |
|                         |                  |                 | * |    |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 |   |    |
+-------------------------+------------------+-----------------+---+----+
| > **SECTION I. STATE    |                  |                 |   |    |
| > CHILD CARE PROGRAM    |                  |                 |   |    |
| > FORMS**               |                  |                 |   |    |
+-------------------------+------------------+-----------------+---+----+
| > **100                 |                  |                 | 1 |    |
| > APPLI                 |                  |                 | 0 |    |
| CATION/RE-DETERMINATION |                  |                 | 0 |    |
| > FORMS**               |                  |                 | R |    |
|                         |                  |                 | E |    |
|                         |                  |                 | S |    |
|                         |                  |                 | U |    |
|                         |                  |                 | L |    |
|                         |                  |                 | T |    |
|                         |                  |                 | S |    |
+-------------------------+------------------+-----------------+---+----+
| > Determine presence,   |                  |                 | 0 | No |
| > date, and             |                  |                 |   | E  |
| > completeness of       |                  |                 |   | rr |
| > required eligibility  |                  |                 |   | or |
| > forms, may include    |                  |                 |   |    |
| > (1) signed and dated  |                  |                 |   |    |
| > application form or   |                  |                 |   |    |
| > (2) narration from    |                  |                 |   |    |
| > case manager for      |                  |                 |   |    |
| > case(s) transitioning |                  |                 |   |    |
| > from TANF to          |                  |                 |   |    |
| > Employment related    |                  |                 |   |    |
| > Day Care. Specify     |                  |                 |   |    |
| > conditions of dollar  |                  |                 |   |    |
| > error, including (1)  |                  |                 |   |    |
| > incomplete            |                  |                 |   |    |
| > application after 45  |                  |                 |   |    |
| > day processing time   |                  |                 |   |    |
| > frame, (2) no         |                  |                 |   |    |
| > application form, and |                  |                 |   |    |
| > (3) no narration      |                  |                 |   |    |
| > regarding transition  |                  |                 |   |    |
| > or extension of       |                  |                 |   |    |
| > application           |                  |                 |   |    |
| > processing time       |                  |                 |   |    |
| > frame.                |                  |                 |   |    |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 | 1 | Cl |
|                         |                  |                 |   | ie |
|                         |                  |                 |   | nt |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 | 2 | Ag |
|                         |                  |                 |   | en |
|                         |                  |                 |   | cy |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 |   |    |
+-------------------------+------------------+-----------------+---+----+
| > **SECTION II.         |                  |                 |   |    |
| > PRIORITY GROUP        |                  |                 |   |    |
| > PLACEMENT**           |                  |                 |   |    |
+-------------------------+------------------+-----------------+---+----+
| > **200 PRIORITY GROUP  |                  |                 | 2 |    |
| > PLACEMENT**           |                  |                 | 0 |    |
|                         |                  |                 | 0 |    |
|                         |                  |                 | R |    |
|                         |                  |                 | E |    |
|                         |                  |                 | S |    |
|                         |                  |                 | U |    |
|                         |                  |                 | L |    |
|                         |                  |                 | T |    |
|                         |                  |                 | S |    |
+-------------------------+------------------+-----------------+---+----+
| > N/A                   |                  |                 | 0 | No |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 | 1 | Cl |
|                         |                  |                 |   | ie |
|                         |                  |                 |   | nt |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 | 2 | Ag |
|                         |                  |                 |   | en |
|                         |                  |                 |   | cy |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 |   |    |
+-------------------------+------------------+-----------------+---+----+
| > **SECTION III.        |                  |                 |   |    |
| > GENERAL PROGRAM       |                  |                 |   |    |
| > REQUIREMENTS**        |                  |                 |   |    |
+-------------------------+------------------+-----------------+---+----+
| > **300 QUALIFYING      |                  |                 | 3 |    |
| > CARETAKER**           |                  |                 | 0 |    |
|                         |                  |                 | 0 |    |
|                         |                  |                 | R |    |
|                         |                  |                 | E |    |
|                         |                  |                 | S |    |
|                         |                  |                 | U |    |
|                         |                  |                 | L |    |
|                         |                  |                 | T |    |
|                         |                  |                 | S |    |
+-------------------------+------------------+-----------------+---+----+
| > Determine if client   |                  |                 | 0 | No |
| > meets definition of   |                  |                 |   | E  |
| > caretaker. A          |                  |                 |   | rr |
| > caretaker is the      |                  |                 |   | or |
| > person, regardless of |                  |                 |   |    |
| > age, who is           |                  |                 |   |    |
| > responsible for the   |                  |                 |   |    |
| > care, control and     |                  |                 |   |    |
| > supervision of the    |                  |                 |   |    |
| > dependent child. The  |                  |                 |   |    |
| > caretaker does not    |                  |                 |   |    |
| > have to be related to |                  |                 |   |    |
| > the child. Caretaker  |                  |                 |   |    |
| > status ends when the  |                  |                 |   |    |
| > responsibility for    |                  |                 |   |    |
| > care, control and     |                  |                 |   |    |
| > supervision is given  |                  |                 |   |    |
| > to another person for |                  |                 |   |    |
| > 30 days or more.      |                  |                 |   |    |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 | 1 | Cl |
|                         |                  |                 |   | ie |
|                         |                  |                 |   | nt |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 | 2 | Ag |
|                         |                  |                 |   | en |
|                         |                  |                 |   | cy |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 |   |    |
+-------------------------+------------------+-----------------+---+----+
| > **310 RESIDENCY**     |                  |                 | 3 |    |
|                         |                  |                 | 1 |    |
|                         |                  |                 | 0 |    |
|                         |                  |                 | R |    |
|                         |                  |                 | E |    |
|                         |                  |                 | S |    |
|                         |                  |                 | U |    |
|                         |                  |                 | L |    |
|                         |                  |                 | T |    |
|                         |                  |                 | S |    |
+-------------------------+------------------+-----------------+---+----+
| > Determine if client   |                  |                 | 0 | No |
| > is a resident of the  |                  |                 |   | E  |
| > State of Oregon.      |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 | 1 | Cl |
|                         |                  |                 |   | ie |
|                         |                  |                 |   | nt |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 | 2 | Ag |
|                         |                  |                 |   | en |
|                         |                  |                 |   | cy |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 |   |    |
+-------------------------+------------------+-----------------+---+----+
| > **320 ELIGIBILITY     |                  |                 | 3 |    |
| > GROUPS**              |                  |                 | 2 |    |
|                         |                  |                 | 0 |    |
|                         |                  |                 | R |    |
|                         |                  |                 | E |    |
|                         |                  |                 | S |    |
|                         |                  |                 | U |    |
|                         |                  |                 | L |    |
|                         |                  |                 | T |    |
|                         |                  |                 | S |    |
+-------------------------+------------------+-----------------+---+----+
| > Requirement for       |                  |                 | 0 | No |
| > caretaker to be       |                  |                 |   | E  |
| > *employed* or         |                  |                 |   | rr |
| > participating in      |                  |                 |   | or |
| > DHS-approved          |                  |                 |   |    |
| > *self-sufficiency     |                  |                 |   |    |
| > activities*. The ERDC |                  |                 |   |    |
| > program is used only  |                  |                 |   |    |
| > for clients not       |                  |                 |   |    |
| > receiving TANF,       |                  |                 |   |    |
| > unless the adult is   |                  |                 |   |    |
| > not included in the   |                  |                 |   |    |
| > TANF benefit group    |                  |                 |   |    |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 | 1 | Cl |
|                         |                  |                 |   | ie |
|                         |                  |                 |   | nt |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 | 2 | Ag |
|                         |                  |                 |   | en |
|                         |                  |                 |   | cy |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 |   |    |
+-------------------------+------------------+-----------------+---+----+
| > **330 QUALIFYING      |                  |                 | 3 |    |
| > CHILD**               |                  |                 | 3 |    |
|                         |                  |                 | 0 |    |
|                         |                  |                 | R |    |
|                         |                  |                 | E |    |
|                         |                  |                 | S |    |
|                         |                  |                 | U |    |
|                         |                  |                 | L |    |
|                         |                  |                 | T |    |
|                         |                  |                 | S |    |
+-------------------------+------------------+-----------------+---+----+
| > Determine if          |                  |                 | 0 | No |
| > child(ren) is         |                  |                 |   | E  |
| > eligible for          |                  |                 |   | rr |
| > services, including   |                  |                 |   | or |
| > (1) younger than 12   |                  |                 |   |    |
| > years, (2) younger    |                  |                 |   |    |
| > than 18 years and     |                  |                 |   |    |
| > physically or         |                  |                 |   |    |
| > mentally incapable of |                  |                 |   |    |
| > caring for himself or |                  |                 |   |    |
| > herself. Need for     |                  |                 |   |    |
| > care of older         |                  |                 |   |    |
| > children must be      |                  |                 |   |    |
| > documented.           |                  |                 |   |    |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 | 1 | Cl |
|                         |                  |                 |   | ie |
|                         |                  |                 |   | nt |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 | 2 | Ag |
|                         |                  |                 |   | en |
|                         |                  |                 |   | cy |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 |   |    |
+-------------------------+------------------+-----------------+---+----+
| > **340 QUALIFYING      |                  |                 | 3 |    |
| > CHILD CARE            |                  |                 | 4 |    |
| > NEED/HOURS**          |                  |                 | 0 |    |
|                         |                  |                 | R |    |
|                         |                  |                 | E |    |
|                         |                  |                 | S |    |
|                         |                  |                 | U |    |
|                         |                  |                 | L |    |
|                         |                  |                 | T |    |
|                         |                  |                 | S |    |
+-------------------------+------------------+-----------------+---+----+
| > Determine if there is |                  |                 | 0 | No |
| > a child care need and |                  |                 |   | E  |
| > authorized hours.     |                  |                 |   | rr |
| > Determine if the      |                  |                 |   | or |
| > co-pay is less than   |                  |                 |   |    |
| > the child care        |                  |                 |   |    |
| > subsidy amount.       |                  |                 |   |    |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 | 1 | Cl |
|                         |                  |                 |   | ie |
|                         |                  |                 |   | nt |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 | 2 | Ag |
|                         |                  |                 |   | en |
|                         |                  |                 |   | cy |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 |   |    |
+-------------------------+------------------+-----------------+---+----+
| > **350 LISTABLE        |                  |                 | 3 |    |
| > PROVIDER**            |                  |                 | 5 |    |
|                         |                  |                 | 0 |    |
|                         |                  |                 | R |    |
|                         |                  |                 | E |    |
|                         |                  |                 | S |    |
|                         |                  |                 | U |    |
|                         |                  |                 | L |    |
|                         |                  |                 | T |    |
|                         |                  |                 | S |    |
+-------------------------+------------------+-----------------+---+----+
| > Determine if provider |                  |                 | 0 | No |
| > of child care         |                  |                 |   | E  |
| > service(s) is a       |                  |                 |   | rr |
| > listed or in the      |                  |                 |   | or |
| > process of becoming   |                  |                 |   |    |
| > listed through the    |                  |                 |   |    |
| > Department of Human   |                  |                 |   |    |
| > Services              |                  |                 |   |    |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 | 1 | Cl |
|                         |                  |                 |   | ie |
|                         |                  |                 |   | nt |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 | 2 | Ag |
|                         |                  |                 |   | en |
|                         |                  |                 |   | cy |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 |   |    |
+-------------------------+------------------+-----------------+---+----+
| > **360 PROVIDER        |                  |                 | 3 |    |
| > REQUIREMENTS**        |                  |                 | 6 |    |
|                         |                  |                 | 0 |    |
|                         |                  |                 | R |    |
|                         |                  |                 | E |    |
|                         |                  |                 | S |    |
|                         |                  |                 | U |    |
|                         |                  |                 | L |    |
|                         |                  |                 | T |    |
|                         |                  |                 | S |    |
+-------------------------+------------------+-----------------+---+----+
| > N/A                   |                  |                 | 0 | No |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 | 1 | Cl |
|                         |                  |                 |   | ie |
|                         |                  |                 |   | nt |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 | 2 | Ag |
|                         |                  |                 |   | en |
|                         |                  |                 |   | cy |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
| > **SECTION IV. INCOME  |                  |                 |   |    |
| > AND PAYMENTS**        |                  |                 |   |    |
+-------------------------+------------------+-----------------+---+----+
| > **400 INCOME\         |                  |                 | 4 |    |
| > **                    |                  |                 | 0 |    |
|                         |                  |                 | 0 |    |
|                         |                  |                 | R |    |
|                         |                  |                 | E |    |
|                         |                  |                 | S |    |
|                         |                  |                 | U |    |
|                         |                  |                 | L |    |
|                         |                  |                 | T |    |
|                         |                  |                 | S |    |
+-------------------------+------------------+-----------------+---+----+
| > Describe income       |                  |                 | 0 | No |
| > verification for each |                  |                 |   | E  |
| > member of the filing  |                  |                 |   | rr |
| > group. Specify time   |                  |                 |   | or |
| > period and all        |                  |                 |   |    |
| > income, earned or     |                  |                 |   |    |
| > unearned, to be       |                  |                 |   |    |
| > considered. If        |                  |                 |   |    |
| > historical income is  |                  |                 |   |    |
| > not representative of |                  |                 |   |    |
| > future income review  |                  |                 |   |    |
| > explanation of how    |                  |                 |   |    |
| > anticipated income    |                  |                 |   |    |
| > was calculated.       |                  |                 |   |    |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 | 1 | Cl |
|                         |                  |                 |   | ie |
|                         |                  |                 |   | nt |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 | 2 | Ag |
|                         |                  |                 |   | en |
|                         |                  |                 |   | cy |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 |   |    |
+-------------------------+------------------+-----------------+---+----+
| > **410 INCOME          |                  |                 | 4 |    |
| > ELIGIBILITY**         |                  |                 | 1 |    |
|                         |                  |                 | 0 |    |
|                         |                  |                 | R |    |
|                         |                  |                 | E |    |
|                         |                  |                 | S |    |
|                         |                  |                 | U |    |
|                         |                  |                 | L |    |
|                         |                  |                 | T |    |
|                         |                  |                 | S |    |
+-------------------------+------------------+-----------------+---+----+
| > Determine if income   |                  |                 | 0 | No |
| > meets State           |                  |                 |   | E  |
| > requirements (e.g.,   |                  |                 |   | rr |
| > gross income must be  |                  |                 |   | or |
| > within 150% of the    |                  |                 |   |    |
| > Federal Poverty       |                  |                 |   |    |
| > Level).               |                  |                 |   |    |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 | 1 | Cl |
|                         |                  |                 |   | ie |
|                         |                  |                 |   | nt |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 | 2 | Ag |
|                         |                  |                 |   | en |
|                         |                  |                 |   | cy |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 |   |    |
+-------------------------+------------------+-----------------+---+----+
| > **420 PAYMENTS,       |                  |                 | 4 |    |
| > GENERAL**             |                  |                 | 2 |    |
|                         |                  |                 | 0 |    |
|                         |                  |                 | R |    |
|                         |                  |                 | E |    |
|                         |                  |                 | S |    |
|                         |                  |                 | U |    |
|                         |                  |                 | L |    |
|                         |                  |                 | T |    |
|                         |                  |                 | S |    |
+-------------------------+------------------+-----------------+---+----+
| > Determine if the      |                  |                 | 0 | No |
| > co-pay requirement    |                  |                 |   | E  |
| > was met.              |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 | 1 | Cl |
|                         |                  |                 |   | ie |
|                         |                  |                 |   | nt |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 | 2 | Ag |
|                         |                  |                 |   | en |
|                         |                  |                 |   | cy |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 |   |    |
+-------------------------+------------------+-----------------+---+----+
| > **430                 |                  |                 | 4 |    |
| >                       |                  |                 | 3 |    |
| PAYMENTS/COMPUTATIONS** |                  |                 | 0 |    |
|                         |                  |                 | R |    |
|                         |                  |                 | E |    |
|                         |                  |                 | S |    |
|                         |                  |                 | U |    |
|                         |                  |                 | L |    |
|                         |                  |                 | T |    |
|                         |                  |                 | S |    |
+-------------------------+------------------+-----------------+---+----+
| Determine difference in |                  |                 | 0 | No |
| dollar amount of child  |                  |                 |   | E  |
| care benefits           |                  |                 |   | rr |
| authorized versus the   |                  |                 |   | or |
| amount that should have |                  |                 |   |    |
| been                    |                  |                 |   |    |
| authorized\--indicate   |                  |                 |   |    |
| if it is an overpayment |                  |                 |   |    |
| or underpayment.        |                  |                 |   |    |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 | 1 | Cl |
|                         |                  |                 |   | ie |
|                         |                  |                 |   | nt |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+
|                         |                  |                 | 2 | Ag |
|                         |                  |                 |   | en |
|                         |                  |                 |   | cy |
|                         |                  |                 |   | E  |
|                         |                  |                 |   | rr |
|                         |                  |                 |   | or |
+-------------------------+------------------+-----------------+---+----+

[]{#__RefHeading___Toc167079394 .anchor}**Appendix E. West Virginia**

[]{#__RefHeading___Toc167079395 .anchor}**A. Organizational Structure**

The Office of Children and Family Policy of the West Virginia Department
of Health and Human Resources administer the child care subsidy program,
Adventures in Better Child Care (ABCC). The State contracts with six
regional Child Care Resource and Referral agencies (CCR&R) to provide
eligibility services to the State's 55 counties. The CCR&R agencies also
educate consumers on selecting appropriate child care and refer parents
to providers in their area. In FY 2005, average per month assistance was
provided through 3,370 providers to over 10,000 children in 5,900
families.

To allow families the choice to select the child care provider that best
meets their needs, parents may use these care settings: the child's
home, registered family providers, registered informal family providers,
registered relative family providers, certified family facilities,
licensed child care centers, and unlicensed school-age child care
programs.

[]{#__RefHeading___Toc167079396 .anchor}**B. Overview of the Child Care
Eligibility Process**

To be eligible for assistance, families must have an income below 150
percent of the Federal Poverty Level (FPL), be a resident of the State,
a US citizen or a "qualified alien," have children under the age of 13
(or age 19 for special care needs), be employed or in an approved
education or training activity, and when applicable, cooperate with
child support enforcement. Parents with earnings of less than 40 percent
of Federal Poverty Level are exempt from fees. State foster children in
approved foster homes are also exempt.

West Virginia's subsidized child care program is a certificate system.
Provider payments are made monthly and retrospectively. Payment for
services includes a base rate for each type of care as well as incentive
rates. Rates are based on the type of care, age of child, special needs
of individual children, hours of care, nontraditional hours, and the
accreditation status of the provider. Benefits are determined
prospectively, i.e., for the upcoming month. Reauthorization of
eligibility is every six months. A new certificate is issued at the six
month mark following the completion of a "status check" (re-verification
of need and income). Clients must report specific types of changes
(identifying information, household composition, employment, need for
job search, and provider changes) within five days of the change. With
some exceptions for changes requiring verification, the change takes
effect the following month.

[]{#__RefHeading___Toc167079397 .anchor}**C. Improper Payments**

West Virginia defines an improper payment as a payment that should not
have been made, or that was made in an incorrect amount due to worker
error in determining and verifying eligibility and/or calculation and
input of information in the Family and Children's Tracking System
(FACTS). Incorrect amounts include overpayments, underpayments, and
inappropriate denials of payment. West Virginia identifies improper
payments through a variety of methods detailed below.

-   Use of information technology**:** The CCR&R agencies must use the
    State's information system, FACTS, to determine financial
    eligibility and calculate payment amounts due. FACTS contains a
    number of edits that identify anomalies dealing with income, hours,
    child's age, shift differential, duplicate eligibility, assessment
    periods, and multiple providers. All payments, while entered and
    approved by a supervisor at the local level, are approved by
    management at the Lead Agency level before issuance. This system of
    checks and balances also prevents improper payments.

-   Documentation of Need and Eligibility Training**:** A Child Care
    Policy Specialist works with CCR&R agencies to identify problem
    policies, procedures and forms. Solutions are developed by committee
    and implemented in the CCR&R agencies through training and technical
    assistance provided quarterly.

-   Quality Assurance Activities**:** Two Child Care Consultants provide
    support to the local CCR&R agencies through a quarterly sampling of
    cases to ensure compliance with policies and procedures. They also
    provide training in child care policy and use of the FACTS system

-   Suspected Fraud Activities:

```{=html}
<!-- -->
```
-   If the CCR&R agency becomes aware that the client or provider is
    attempting to or has received payments to which they are not
    entitled, the worker must take corrective action to prevent further
    payments from occurring. The CCR&R agencies are required to
    cooperate with one another and share information, including payment
    forms, sign-in and sign-out sheets, documents, and records to reduce
    and prevent improper payments. When information is requested by
    other CCR&R agencies for this purpose, the CCR&R agencies shall
    respond within five days, unless extenuating circumstances exist.

-   If over payment is due to error by the CCR&R agency or error on the
    part of the provider and the amount is less than \$1,000, the CCR&R
    agency is responsible for negotiating the repayment. If the
    overpayment is \$1,000 or greater and is due to misrepresentation by
    the client or provider, a referral is to be made to the Director of
    Investigation and Fraud Management.

    []{#__RefHeading___Toc167079398 .anchor}**D. Case Review Process**

West Virginia made substantial modifications to the *Record Review
Worksheet*, using check-off boxes, "Yes" and "No" fields, and coding to
eliminate the need for extensive narrative recording. West Virginia
completed the record review process and compiled the results prior to
the site visit. The Review Team consisted of two Field Consultants and a
Policy Specialist. All of the reviews occurred on site at the regional
CCR&R agencies. The Review Team had already read 10 cases during one of
its regular CCR&R audits. To improve the review process, the three
reviewers completed several reviews jointly and consulted by telephone
frequently. Because the Review Team reviewed the cases in regions,
travel time was a considerable factor with several hours of driving time
from one CCR&R agency to another.

[]{#__RefHeading___Toc167079399 .anchor}**E. Automation**

The Families and Children Tracking System (FACTS) is a customized
statewide automated case management system for all Child Welfare, Child
Care, and Adult Service Programs. In August 2000, the State modified
FACTS to include case management and payment functionality for the Child
Care Program. This change fully implemented the transition of the
contracted CCR&R agencies to an automated system for on-line and real
time determination of eligibility for child care services and an
automated determination for child care payments. The FACTS System
utilizes information entered by the CCR&R staff to determine
eligibility. This eliminates many errors, with the exception of those
caused by faulty data entry. The system has payment controls for
eligible days and for eligible children.

The FACTS system also provides staff with the capability of accessing
the State's TANF, Food Stamp and Medicaid system - RAPIDS. This allows
additional verification of information provided in the eligibility
process.

The major goals of the system are to improve the organization's capacity
for:

-   Program compliance and planning;

-   Accurate data collection for State and Federal reporting;

-   Program quality assurance efforts;

-   Case record management; and

-   Information collection and analysis.

    []{#__RefHeading___Toc167079400 .anchor}**F. Next Steps**

This section includes a summary of the West Virginia response to the
error causes and corrective actions taken based on the finding of the
pilot.

-   West Virginia will conduct refresher policy training with the CCR&R
    directors. The directors will then provide the training to their
    staff. It is felt that by training the directors rather than the
    staff that the directors will become more involved in eligibility
    activities.

```{=html}
<!-- -->
```
-   Technical assistance staff will continue to perform random quarterly
    case audits. Prior to the pilot, the FACTS electronic record was the
    object of the review. An expanded review will now include the
    physical case record.

-   CCR&R agencies must develop and implement Quality Assurance Plans.
    The plans will now require inclusion of payment accuracy initiatives
    if a CCR&R agency has an error rate that is above the State mean.

-   Family child care providers must submit sign-in and sign-out sheets
    to verify attendance along with their billing forms. CCR&R agencies
    must audit billing forms and compare work and school schedules to
    times shown on the sheets to verify that the child care usage
    complies with time approved.

-   Child care providers who submit incorrect or improper billing forms
    must attend a retraining session on proper billing procedures.

-   Two State level Child Care Consultants will continue to sample CCR&R
    cases to ensure compliance with appropriate policies and provide
    training and technical assistance on policy, procedures, and the use
    of FACTS. The consultants also provide follow up to ensure the CCR&R
    corrective action plans are completed.

    []{#__RefHeading___Toc167079401 .anchor}**West Virginia Site Visit
    Participants**

Charlie Young -- Director, Bureau for Children and Families Policy

Kay Tilton, Director, Division of Early Care and Education

Deidre Craythorne, Policy Specialist, Division of Early Care and
Education

Mary Hayden, Field Consultant, Division of Early Care and Education

Shannon Richards, HHR Specialist, FACTS Project

Raechelle Miller, HHR Specialist, FACTS Project

Judy Currry, Child Care Program Manager

Brenda Howell, Director, FACTS

Susan Richards, Director, Training

[]{#__RefHeading___Toc167078174 .anchor}**Exhibit E-1. West Virginia
Record Review Worksheet**

**Client Study ID# \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ County:
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_**

**Client FACTS ID# \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ Date
Reviewed: \_\_\_ / \_\_\_ / \_\_\_**

+--------------------------------------------------+------+-----+------+
| #### SECTION I. STATE CHILD CARE PROGRAM FORMS   |      |     |      |
+--------------------------------------------------+------+-----+------+
| **100 APPLICATION FORMS\                         | ANAL |     |      |
| **ELEMENTS OF ELIGIBILITY & PAYMENT              | YSIS |     |      |
| DETERMINATION (1)\                               | OF   |     |      |
| Determine presence, date, and completeness of    | CASE |     |      |
| required eligibility forms, may include:         | RE   |     |      |
|                                                  | CORD |     |      |
|                                                  | (2)  |     |      |
+--------------------------------------------------+------+-----+------+
|                                                  | No   | Cli | Age  |
|                                                  | E    | ent | ncy\ |
|                                                  | rror | Er  | E    |
|                                                  |      | ror | rror |
+--------------------------------------------------+------+-----+------+
| \(1\) Signed and dated application (ECE-CC-1) or | \_   | \_\ | \_   |
| review form (DAY-0614) or signature page         | \_\_ | _\_ | \_\_ |
| (DAY-0617) effective for 10-05                   | 0    | 1   | 2    |
+--------------------------------------------------+------+-----+------+
| \(2\) Child care parent services agreement       | \_   | \_\ | \_   |
| (signed DAY 0162)                                | \_\_ | _\_ | \_\_ |
|                                                  | 0    | 1   | 2    |
+--------------------------------------------------+------+-----+------+
| \(3\) Certificate (completed DAY-0176)           | \_   | \_\ | \_   |
|                                                  | \_\_ | _\_ | \_\_ |
|                                                  | 0    | 1   | 2    |
+--------------------------------------------------+------+-----+------+
| (4.a.) Provider invoice. (ECE-CC-10 A)           | \_   | \_\ | \_   |
|                                                  | \_\_ | _\_ | \_\_ |
|                                                  | 0    | 1   | 2    |
+--------------------------------------------------+------+-----+------+
| Is form completely filled out?                   | \_   | \_\ | \_   |
|                                                  | \_\_ | _\_ | \_\_ |
|                                                  | 0    | 1   | 2    |
+--------------------------------------------------+------+-----+------+
| Are Sign In/Out Sheets attached? (if required)   | \_   | \_\ | \_   |
|                                                  | \_\_ | _\_ | \_\_ |
|                                                  | 0    | 1   | 2    |
+--------------------------------------------------+------+-----+------+
| Does Provider Signature match signature in       | \_   | \_\ | \_   |
| provider file?                                   | \_\_ | _\_ | \_\_ |
|                                                  | 0    | 1   | 2    |
+--------------------------------------------------+------+-----+------+
| Do parent signatures match signatures in         | \_   | \_\ | \_   |
| parent's paper file?                             | \_\_ | _\_ | \_\_ |
|                                                  | 0    | 1   | 2    |
+--------------------------------------------------+------+-----+------+
| Do the number of days billed match sign in/out   | \_   | \_\ | \_   |
| sheet?                                           | \_\_ | _\_ | \_\_ |
|                                                  | 0    | 1   | 2    |
+--------------------------------------------------+------+-----+------+
| Does sign in/out sheet match parent's            | \_   | \_\ | \_   |
| work/school schedule in FACTS?                   | \_\_ | _\_ | \_\_ |
|                                                  | 0    | 1   | 2    |
+--------------------------------------------------+------+-----+------+
| Was payment entered correctly?                   | \_   | \_\ | \_   |
|                                                  | \_\_ | _\_ | \_\_ |
|                                                  | 0    | 1   | 2    |
+--------------------------------------------------+------+-----+------+
| Was payment form signed by worker?               | \_   | \_\ | \_   |
|                                                  | \_\_ | _\_ | \_\_ |
|                                                  | 0    | 1   | 2    |
+--------------------------------------------------+------+-----+------+
| Was form signed by approving supervisor?         | \_   | \_\ | \_   |
|                                                  | \_\_ | _\_ | \_\_ |
|                                                  | 0    | 1   | 2    |
+--------------------------------------------------+------+-----+------+
| (4.b.) Specify conditions of dollar error,       | \_   | \_\ | \_   |
| including                                        | \_\_ | _\_ | \_\_ |
|                                                  | 0    | 1   | 2    |
+--------------------------------------------------+------+-----+------+
| \(1\) form expired- (expired                     | \_   | \_\ | \_   |
| certificate-DAY-0176)                            | \_\_ | _\_ | \_\_ |
|                                                  | 0    | 1   | 2    |
+--------------------------------------------------+------+-----+------+
| \(2\) No application form (signed ECE-CC-1       | \_   | \_\ | \_   |
| /DAY-0614 or DAY-0617)                           | \_\_ | _\_ | \_\_ |
|                                                  | 0    | 1   | 2    |
+--------------------------------------------------+------+-----+------+
| \(3\) No documentation of income and work hours. | \_   | \_\ | \_   |
| ( copies of pay stubs or ECE-CC-1B or 1C, if     | \_\_ | _\_ | \_\_ |
| self-employed-check application form for work    | 0    | 1   | 2    |
| hours and check the employment screen for all    |      |     |      |
| adults in FACTS)                                 |      |     |      |
+--------------------------------------------------+------+-----+------+
| FINDINGS (3)                                     |      |     |      |
+--------------------------------------------------+------+-----+------+

+-------------------------------------------------+------+------+------+
| #### SECTION II. PRIORITY GROUP PLACEMENT       |      |      |      |
+-------------------------------------------------+------+------+------+
| **200 PRIORITY GROUP PLACEMENT\                 | ANAL |      |      |
| **ELEMENTS OF ELIGIBILITY & PAYMENT             | YSIS |      |      |
| DETERMINATION (1)\                              | OF   |      |      |
| Determine if client meets criteria of           | CASE |      |      |
| State-designated priority groups, e.g., (West   | RE   |      |      |
| Virginia's State plan only addresses priorities | CORD |      |      |
| for foster and adoptive parents, since we       | (2)  |      |      |
| operate without a waiting list, so we changed   |      |      |      |
| the priorities list.)                           |      |      |      |
+-------------------------------------------------+------+------+------+
|                                                 | No   | Cl   | Age  |
|                                                 | E    | ient | ncy\ |
|                                                 | rror | E    | E    |
|                                                 |      | rror | rror |
+-------------------------------------------------+------+------+------+
| \(1\) foster parents (check general information | \_   | \_   | \_   |
| screen to see if role is a foster parent and if | \_\_ | \_\_ | \_\_ |
| child is checked as a foster child. If so,      | 0    | 1    | 2    |
| check finance screen to make sure foster parent |      |      |      |
| income was disregarded)                         |      |      |      |
+-------------------------------------------------+------+------+------+
| \(2\) adoptive parent (check general            | \_   | \_   | \_   |
| information screen to see if role in case is    | \_\_ | \_\_ | \_\_ |
| adoptive parent. If so, check finance screen to | 0    | 1    | 2    |
| make sure adoption subsidy amounts were         |      |      |      |
| disregarded)                                    |      |      |      |
+-------------------------------------------------+------+------+------+
| FINDINGS (3)                                    |      |      |      |
|                                                 |      |      |      |
|                                                 |      |      |      |
+-------------------------------------------------+------+------+------+

+-------------------------------------------------+------+------+------+
| #### SECTION III. General Program Requirements  |      |      |      |
+-------------------------------------------------+------+------+------+
| **300 QUALIFYING HEAD OF HOUSEHOLD\             | ANAL |      |      |
| **ELEMENTS OF ELIGIBILITY & PAYMENT             | YSIS |      |      |
| DETERMINATION (1)\                              | OF   |      |      |
| Determine if client meets parent definition     | CASE |      |      |
| (parent means a parent by blood, marriage or    | RE   |      |      |
| adoption and also means a legal guardian, or    | CORD |      |      |
| other person standing in loco parentis), e.g.,  | (2)  |      |      |
| (check general information screen to see if the |      |      |      |
| client is one of the following:)                |      |      |      |
+-------------------------------------------------+------+------+------+
|                                                 | No   | Cl   | Age  |
|                                                 | E    | ient | ncy\ |
|                                                 | rror | E    | E    |
|                                                 |      | rror | rror |
+-------------------------------------------------+------+------+------+
| \(1\) parent                                    | \_   | \_   | \_   |
|                                                 | \_\_ | \_\_ | \_\_ |
|                                                 | 0    | 1    | 2    |
+-------------------------------------------------+------+------+------+
| \(2\) step-parent                               | \_   | \_   | \_   |
|                                                 | \_\_ | \_\_ | \_\_ |
|                                                 | 0    | 1    | 2    |
+-------------------------------------------------+------+------+------+
| \(3\) legal guardian                            | \_   | \_   | \_   |
|                                                 | \_\_ | \_\_ | \_\_ |
|                                                 | 0    | 1    | 2    |
+-------------------------------------------------+------+------+------+
| \(4\) caretaker                                 | \_   | \_   | \_   |
|                                                 | \_\_ | \_\_ | \_\_ |
|                                                 | 0    | 1    | 2    |
+-------------------------------------------------+------+------+------+
| \(5\) relative                                  | \_   | \_   | \_   |
|                                                 | \_\_ | \_\_ | \_\_ |
|                                                 | 0    | 1    | 2    |
+-------------------------------------------------+------+------+------+
|                                                 |      |      |      |
+-------------------------------------------------+------+------+------+
| FINDINGS (3)                                    |      |      |      |
|                                                 |      |      |      |
|                                                 |      |      |      |
+-------------------------------------------------+------+------+------+

+------------------------------------------------+------+-----+------+
| **310 RESIDENCY\                               | ANAL |     |      |
| **ELEMENTS OF ELIGIBILITY & PAYMENT            | YSIS |     |      |
| DETERMINATION (1)\                             | OF   |     |      |
| Determine if (WV has a statewide system, so we | CASE |     |      |
| are checking only if the client is a resident  | RE   |     |      |
| of the State. We also added " is the child a   | CORD |     |      |
| US citizen or a qualified eligible alien)      | (2)  |     |      |
+------------------------------------------------+------+-----+------+
|                                                | No   | Cli | Age  |
|                                                | E    | ent | ncy\ |
|                                                | rror | Er  | E    |
|                                                |      | ror | rror |
+------------------------------------------------+------+-----+------+
| Client is a resident of the State (check       | \_   | \_\ | \_   |
| address on address screen)                     | \_\_ | _\_ | \_\_ |
|                                                | 0    | 1   | 2    |
+------------------------------------------------+------+-----+------+
| Is child a US citizen? ( Check race/ethnicity  | \_   | \_\ | \_   |
| screen)                                        | \_\_ | _\_ | \_\_ |
|                                                | 0    | 1   | 2    |
+------------------------------------------------+------+-----+------+
| Is Child a qualified eligible alien? ( Check   | \_   | \_\ | \_   |
| race/ethnicity screen)                         | \_\_ | _\_ | \_\_ |
|                                                | 0    | 1   | 2    |
+------------------------------------------------+------+-----+------+
| Client is a resident of the county and for     |      |     |      |
| what duration.                                 |      |     |      |
|                                                |      |     |      |
| There is an agreement regarding eligibility    |      |     |      |
| among counties.                                |      |     |      |
+------------------------------------------------+------+-----+------+
| FINDINGS (3)                                   |      |     |      |
+------------------------------------------------+------+-----+------+

  ------------------------------------------------- ---------- -------- ---------
  **320 HOUSEHOLD MEMBERS\                          ANALYSIS            
  **ELEMENTS OF ELIGIBILITY & PAYMENT DETERMINATION OF CASE             
  (1)\                                              RECORD (2)          
  To receive services a child\'s parent or parents                      
  must be                                                               

                                                    No Error   Client   Agency\
                                                               Error    Error

  Working (check employment screen and finances     \_\_\_ 0   \_\_\_ 1 \_\_\_ 2
  screen to see if work has been verified or check                      
  pay stubs in record)                                                  

  Attending a job training or educational program.  \_\_\_ 0   \_\_\_ 1 \_\_\_ 2
  ( check education screen and see if and how                           
  education was verified- check paper file to see                       
  if verification is in file)                                           

  FINDINGS (3)                                                          
  ------------------------------------------------- ---------- -------- ---------

+-------------------------------------------------+------+----+------+
| **330 QUALIFYING CHILD\                         | ANAL |    |      |
| **ELEMENTS OF ELIGIBILITY & PAYMENT             | YSIS |    |      |
| DETERMINATION (1)\                              | OF   |    |      |
| Determine if child (ren) is eligible for        | CASE |    |      |
| services, including (WV does not pay for        | RE   |    |      |
| children in foster care unless the foster       | CORD |    |      |
| parents are working/attending school, so we     | (2)  |    |      |
| changed this to "for protective services        |      |    |      |
| reasons)                                        |      |    |      |
+-------------------------------------------------+------+----+------+
|                                                 | No   | Cl | Age  |
|                                                 | E    | ie | ncy\ |
|                                                 | rror | nt | E    |
|                                                 |      | E  | rror |
|                                                 |      | rr |      |
|                                                 |      | or |      |
+-------------------------------------------------+------+----+------+
| \(1\) younger than 13 years (check date of      | \_   | \_ | \_   |
| birth on client general information screen --if | \_\_ | \_ | \_\_ |
| over 13, see #2)                                | 0    | \_ | 2    |
|                                                 |      | 1  |      |
+-------------------------------------------------+------+----+------+
| \(2\) younger than 18 years and physically or   | \_   | \_ | \_   |
| mentally incapable of caring for himself or     | \_\_ | \_ | \_\_ |
| herself, or under court supervision ( see if    | 0    | \_ | 2    |
| verification of physical or mental incapacity   |      | 1  |      |
| or court supervision is on file in record)      |      |    |      |
+-------------------------------------------------+------+----+------+
| \(3\) in child care for protective services     | \_   | \_ | \_   |
| reasons ( Check in Child Care assessment to see | \_\_ | \_ | \_\_ |
| if CPS safety or treatment plan is checked. Is  | 0    | \_ | 2    |
| a copy of safety or treatment plan or a Safety  |      | 1  |      |
| First Plan on file?                             |      |    |      |
+-------------------------------------------------+------+----+------+
| FINDINGS (3)                                    |      |    |      |
|                                                 |      |    |      |
|                                                 |      |    |      |
+-------------------------------------------------+------+----+------+

+-------------------------------------------------+------+----+------+
| **340 QUALIFYING CARE\                          | ANAL |    |      |
| **ELEMENTS OF ELIGIBILITY & PAYMENT             | YSIS |    |      |
| DETERMINATION (1)                               | OF   |    |      |
|                                                 | CASE |    |      |
|                                                 | RE   |    |      |
|                                                 | CORD |    |      |
|                                                 | (2)  |    |      |
+-------------------------------------------------+------+----+------+
|                                                 | No   | Cl | Age  |
|                                                 | E    | ie | ncy\ |
|                                                 | rror | nt | E    |
|                                                 |      | E  | rror |
|                                                 |      | rr |      |
|                                                 |      | or |      |
+-------------------------------------------------+------+----+------+
| Determine hours authorized. (check              | \_   | \_ | \_   |
| employment/education screens and services to    | \_\_ | \_ | \_\_ |
| see if hours of care are entered and            | 0    | \_ | 2    |
| consistent- If not, exceptions are noted on     |      | 1  |      |
| contacts or paper record)                       |      |    |      |
+-------------------------------------------------+------+----+------+
| Determine required number of hours of care      | \_   | \_ | \_   |
| during authorized schedule. (Check to see if    | \_\_ | \_ | \_\_ |
| number of hours authorized matches pay stubs,   | 0    | \_ | 2    |
| school schedules or other documentation)        |      | 1  |      |
+-------------------------------------------------+------+----+------+
| FINDINGS (3)                                    |      |    |      |
|                                                 |      |    |      |
|                                                 |      |    |      |
+-------------------------------------------------+------+----+------+
| **350 QUALIFYING PROVIDER ARRANGEMENT\          | ANAL |    |      |
| **ELEMENTS OF ELIGIBILITY & PAYMENT             | YSIS |    |      |
| DETERMINATION (1)\                              | OF   |    |      |
| Determine if services are provided within a     | CASE |    |      |
| legally operating provider type. (Check the     | RE   |    |      |
| provider record status screen to see if a valid | CORD |    |      |
| license or certificate was in effect as of      | (2)  |    |      |
| October 2005 )                                  |      |    |      |
+-------------------------------------------------+------+----+------+
|                                                 | No   | Cl | Age  |
|                                                 | E    | ie | ncy\ |
|                                                 | rror | nt | E    |
|                                                 |      | E  | rror |
|                                                 |      | rr |      |
|                                                 |      | or |      |
+-------------------------------------------------+------+----+------+
| Center-based child care provider                | \_   | \_ | \_   |
|                                                 | \_\_ | \_ | \_\_ |
|                                                 | 0    | \_ | 2    |
|                                                 |      | 1  |      |
+-------------------------------------------------+------+----+------+
| group home child care provider                  | \_   | \_ | \_   |
|                                                 | \_\_ | \_ | \_\_ |
|                                                 | 0    | \_ | 2    |
|                                                 |      | 1  |      |
+-------------------------------------------------+------+----+------+
| family child care provider                      | \_   | \_ | \_   |
|                                                 | \_\_ | \_ | \_\_ |
|                                                 | 0    | \_ | 2    |
|                                                 |      | 1  |      |
+-------------------------------------------------+------+----+------+
| an in-home child care provider                  | \_   | \_ | \_   |
|                                                 | \_\_ | \_ | \_\_ |
|                                                 | 0    | \_ | 2    |
|                                                 |      | 1  |      |
+-------------------------------------------------+------+----+------+
| other provider of child care services for       | \_   | \_ | \_   |
| compensation that is licensed, regulated, or    | \_\_ | \_ | \_\_ |
| registered under applicable State or local law, | 0    | \_ | 2    |
| including informal, relative or legally         |      | 1  |      |
| operating school-age child care?                |      |    |      |
+-------------------------------------------------+------+----+------+
| FINDINGS (3)                                    |      |    |      |
|                                                 |      |    |      |
|                                                 |      |    |      |
+-------------------------------------------------+------+----+------+

  ------------------------------------------------- ---------- -------- ---------
  **360 PROVIDER REQUIREMENTS\                      ANALYSIS            
  **ELEMENTS OF ELIGIBILITY & PAYMENT DETERMINATION OF CASE             
  (1) If provider status screen shows a valid       RECORD (2)          
  license, regulatory requirements are met, so this                     
  is a duplicate question for WV.                                       

                                                    No Error   Client   Agency\
                                                               Error    Error

  Determine if regulatory requirements are met.                         
  Regulatory requirements means requirements                            
  necessary for a provider to legally provide child                     
  care services in a State or locality, including                       
  registration requirements established under                           
  State, local, or tribal law.                                          

  Findings (3)                                                          
  ------------------------------------------------- ---------- -------- ---------

+-------------------------------------------------+------+------+------+
| #### SECTION IV. INCOME AND PAYMENTS            |      |      |      |
+-------------------------------------------------+------+------+------+
| **400 INCOME\                                   | ANAL |      |      |
| **ELEMENTS OF ELIGIBILITY & PAYMENT             | YSIS |      |      |
| DETERMINATION (1) Describe income documentation | OF   |      |      |
| verification for each member of the household.  | CASE |      |      |
| Specify time period and all income to be        | RE   |      |      |
| considered, e.g., based on 4 weeks prior to     | CORD |      |      |
| application: Collect the following data: (WV    | (2)  |      |      |
| does not require reporting of changes in income |      |      |      |
| during the six month period; b. food stamps are |      |      |      |
| not considered income; and c. loss of income    |      |      |      |
| does not need to be reported. )                 |      |      |      |
+-------------------------------------------------+------+------+------+
|                                                 | No   | Cl   | Age  |
|                                                 | E    | ient | ncy\ |
|                                                 | rror | E    | E    |
|                                                 |      | rror | rror |
+-------------------------------------------------+------+------+------+
| (1.) head of household employment income (Check | \_   | \_   | \_   |
| record to see if pay stubs or other acceptable  | \_\_ | \_\_ | \_\_ |
| income verification is on file that document    | 0    | 1    | 2    |
| the 1-mo. period prior to review or             |      |      |      |
| application. Verification can be any of the     |      |      |      |
| below)                                          |      |      |      |
+-------------------------------------------------+------+------+------+
| (1.a.) 2 pay stubs if paid bi-weekly or twice   | \_   | \_   | \_   |
| monthly, 4 pay stubs if paid weekly             | \_\_ | \_\_ | \_\_ |
|                                                 | 0    | 1    | 2    |
+-------------------------------------------------+------+------+------+
| (1.b.) Three months of pay stubs if income      | \_   | \_   | \_   |
| fluctuates                                      | \_\_ | \_\_ | \_\_ |
|                                                 | 0    | 1    | 2    |
+-------------------------------------------------+------+------+------+
| (1.c.) New Employment Verification Letter       | \_   | \_   | \_   |
|                                                 | \_\_ | \_\_ | \_\_ |
|                                                 | 0    | 1    | 2    |
+-------------------------------------------------+------+------+------+
| (1.d.) Self Employment income verification      | \_   | \_   | \_   |
| (must also have copy of applicable business     | \_\_ | \_\_ | \_\_ |
| licenses, Registration and Tax Return- if       | 0    | 1    | 2    |
| available)                                      |      |      |      |
+-------------------------------------------------+------+------+------+
| \(2\) spouse employment income (Check record to | \_   | \_   | \_   |
| see if pay stubs or other acceptable income     | \_\_ | \_\_ | \_\_ |
| verification is on file that document a 1-mo.   | 0    | 1    | 2    |
| Period)                                         |      |      |      |
+-------------------------------------------------+------+------+------+
| (2.a.) 2 pay stubs if paid bi-weekly or twice   | \_   | \_   | \_   |
| monthly, 4 pay stubs if paid weekly             | \_\_ | \_\_ | \_\_ |
|                                                 | 0    | 1    | 2    |
+-------------------------------------------------+------+------+------+
| (2.b.) Three months of pay stubs if income      | \_   | \_   | \_   |
| fluctuates                                      | \_\_ | \_\_ | \_\_ |
|                                                 | 0    | 1    | 2    |
+-------------------------------------------------+------+------+------+
| (2.c.) New Employment Verification Letter       | \_   | \_   | \_   |
|                                                 | \_\_ | \_\_ | \_\_ |
|                                                 | 0    | 1    | 2    |
+-------------------------------------------------+------+------+------+
| (2.d.) Self Employment income verification      | \_   | \_   | \_   |
| (must also have Copy of business                | \_\_ | \_\_ | \_\_ |
| licenses/Registration and Tax Return- if        | 0    | 1    | 2    |
| available)                                      |      |      |      |
+-------------------------------------------------+------+------+------+
| \(3\) any changes in income reported West       |      |      |      |
| Virginia does not require reporting of income   |      |      |      |
| changes within the eligibility period,          |      |      |      |
| therefore there would be no error within this   |      |      |      |
| data element.                                   |      |      |      |
+-------------------------------------------------+------+------+------+
| \(4\) income during job training for            | \_   | \_   | \_   |
| parent/caregiver                                | \_\_ | \_\_ | \_\_ |
|                                                 | 0    | 1    | 2    |
+-------------------------------------------------+------+------+------+
| \(5\) child support, if included as part of     | \_   | \_   | \_   |
| income (check FACTS or application to see if    | \_\_ | \_\_ | \_\_ |
| child support is verified)                      | 0    | 1    | 2    |
+-------------------------------------------------+------+------+------+
| \(6\) Food Stamps, if included as part of       |      |      |      |
| income West Virginia does not count food stamps |      |      |      |
| as income, therefore there would be no error    |      |      |      |
| within this data element.                       |      |      |      |
+-------------------------------------------------+------+------+------+
| \(7\) loss of income during eligibility period. |      |      |      |
| West Virginia does not require reporting of     |      |      |      |
| income changes within the eligibility period    |      |      |      |
| therefore there would be no error within this   |      |      |      |
| data element.                                   |      |      |      |
+-------------------------------------------------+------+------+------+
| \(8\) Other countable income documented (SSI,   | \_   | \_   | \_   |
| TANF, Spousal Support, gifts, etc.)             | \_\_ | \_\_ | \_\_ |
|                                                 | 0    | 1    | 2    |
+-------------------------------------------------+------+------+------+
| FINDINGS (3)                                    |      |      |      |
+-------------------------------------------------+------+------+------+

  ------------------------------------------------- ---------- -------- ---------
  **410 INCOME ELIGIBILITY\                         ANALYSIS            
  **ELEMENTS OF ELIGIBILITY & PAYMENT DETERMINATION OF CASE             
  (1) (WV's automated system calculates income and  RECORD (2)          
  compares it with the sliding fee scale, so we                         
  opted to check for mathematical errors in                             
  averaging income or data entry.)                                      

                                                    No Error   Client   Agency\
                                                               Error    Error

  Determine if household income meets State         \_\_\_ 0   \_\_\_ 1 \_\_\_ 2
  requirements ( Check income verification in paper                     
  record against that in FACTS to see if any                            
  mathematical or data entry errors were made by                        
  the case worker?)                                                     

  FINDINGS (3)                                                          
  ------------------------------------------------- ---------- -------- ---------

+-------------------------------------------------+-----+------+------+
| **420 PAYMENTS, GENERAL\                        | AN  |      |      |
| **ELEMENTS OF ELIGIBILITY & PAYMENT             | ALY |      |      |
| DETERMINATION (1)                               | SIS |      |      |
|                                                 | OF  |      |      |
|                                                 | C   |      |      |
|                                                 | ASE |      |      |
|                                                 | REC |      |      |
|                                                 | ORD |      |      |
|                                                 | (2) |      |      |
+-------------------------------------------------+-----+------+------+
|                                                 | No  | Cl   | Age  |
|                                                 | Er  | ient | ncy\ |
|                                                 | ror | E    | E    |
|                                                 |     | rror | rror |
+-------------------------------------------------+-----+------+------+
| Determine if payments were made. A sliding fee  |     |      |      |
| scale based on income and the size of the       |     |      |      |
| family and may be based on other factors as     |     |      |      |
| appropriate.                                    |     |      |      |
|                                                 |     |      |      |
| West Virginia's pilot sample consists only of   |     |      |      |
| clients for whom a payment was made for the     |     |      |      |
| service month of October 2005, therefore there  |     |      |      |
| are no errors possible for this data element.   |     |      |      |
+-------------------------------------------------+-----+------+------+
| Findings (3)                                    |     |      |      |
+-------------------------------------------------+-----+------+------+

+-------------------------------------------------+------+----+-------+
| **430 PAYMENTS, COMPUTATIONS\                   | ANAL |    |       |
| **ELEMENTS OF ELIGIBILITY & PAYMENT             | YSIS |    |       |
| DETERMINATION (1)                               | OF   |    |       |
|                                                 | CASE |    |       |
|                                                 | RE   |    |       |
|                                                 | CORD |    |       |
|                                                 | (2)  |    |       |
+-------------------------------------------------+------+----+-------+
|                                                 | No   | Cl | Ag    |
|                                                 | E    | ie | ency\ |
|                                                 | rror | nt | Error |
|                                                 |      | E  |       |
|                                                 |      | rr |       |
|                                                 |      | or |       |
+-------------------------------------------------+------+----+-------+
| Determine difference in dollar amount of child  | \_   | \_ | \     |
| care benefits authorized versus the amount that | \_\_ | \_ | _\_\_ |
| should have been authorized---indicate if it is | 0    | \_ | 2     |
| an overpayment or underpayment.                 |      | 1  |       |
+-------------------------------------------------+------+----+-------+
| FINDINGS (3)                                    |      |    |       |
+-------------------------------------------------+------+----+-------+
| **Data Summary**                                |      |    |       |
+-------------------------------------------------+------+----+-------+
| State                                           | West |    |       |
|                                                 | Virg |    |       |
|                                                 | inia |    |       |
+-------------------------------------------------+------+----+-------+
| County                                          |      |    |       |
+-------------------------------------------------+------+----+-------+
| Child ID                                        |      |    |       |
+-------------------------------------------------+------+----+-------+
| Study Period Month                              |      |    |       |
+-------------------------------------------------+------+----+-------+
| One or more eligibility errors during pilot     |      |    |       |
| period                                          |      |    |       |
|                                                 |      |    |       |
| 0= no errors, 1= one or more errors             |      |    |       |
+-------------------------------------------------+------+----+-------+
| Total amount of improper payment during review  |      |    |       |
| month                                           |      |    |       |
+-------------------------------------------------+------+----+-------+
| Total Payment Amount during review month (shown |      |    |       |
| in FACTS Client Payment Information screen)     |      |    |       |
+-------------------------------------------------+------+----+-------+

[]{#__RefHeading___Toc167079403 .anchor}**APPENDIX F. RECORD REVIEW
WORKSHEET TEMPLATE**

  ------------------------------------------------------------------------------------------
  **CLIENT ID#**                 **STATE:**          **COUNTY:**        **DATE:**   
  ------------------------------ ------------------- ------------------ ----------- --------
                                                                                    

  **ELEMENTS OF ELIGIBILITY &    **ANALYSIS OF CASE  **FINDINGS (3)**   **RESULTS   
  PAYMENT DETERMINATION (1)**    RECORD (2)**                           (4)**       

                                                                                    

  **SECTION I. STATE CHILD CARE                                                     
  PROGRAM FORMS**                                                                   

  **100                                                                 100 RESULTS 
  APPLICATION/RE-DETERMINATION                                                      
  FORMS**                                                                           

  Determine presence, date, and                                         0           No Error
  completeness of required                                                          
  eligibility forms, may include                                                    
  (1) signed and dated                                                              
  application form, (2) child                                                       
  care agreement, (3) voucher or                                                    
  certificate, and (4) provider                                                     
  invoice. Specify conditions of                                                    
  dollar error, including (1)                                                       
  form expired, (2) no                                                              
  application form, and (3) no                                                      
  documentation of income and                                                       
  work hours.                                                                       

                                                                        1           Client
                                                                                    Error

                                                                        2           Agency
                                                                                    Error

                                                                                    

  **SECTION II. PRIORITY GROUP                                                      
  PLACEMENT**                                                                       

  **200 PRIORITY GROUP                                                  200 RESULTS 
  PLACEMENT**                                                                       

  Determine if client meets                                             0           No Error
  criteria of State-designated                                                      
  priority groups, e.g., (1)                                                        
  teen parent in high school,                                                       
  (2) TANF recipients in                                                            
  eligible work activities, (3)                                                     
  working parents on TANF, and                                                      
  (4) foster parents etc.                                                           

                                                                        1           Client
                                                                                    Error

                                                                        2           Agency
                                                                                    Error

                                                                                    

  **SECTION III. GENERAL PROGRAM                                                    
  REQUIREMENTS**                                                                    

  **300 QUALIFYING HEAD OF                                              300 RESULTS 
  HOUSEHOLD**                                                                       

  Determine if client meets                                             0           No Error
  parent definition (parent                                                         
  means a parent by blood,                                                          
  marriage or adoption and also                                                     
  means a legal guardian, or                                                        
  other person standing in loco                                                     
  parentis), e.g., (1) parent,                                                      
  (2) step-parent, (3) legal                                                        
  guardian, (4) needy caretaker                                                     
  relative, or (5) spouse of                                                        
  same. Child(ren) must be                                                          
  citizen(s).                                                                       

                                                                        1           Client
                                                                                    Error

                                                                        2           Agency
                                                                                    Error

                                                                                    

  **310 RESIDENCY**                                                     310 RESULTS 

  Determine if client is a                                              0           No Error
  resident of the State and for                                                     
  what duration, if client is a                                                     
  resident of the county and for                                                    
  what duration, and whether                                                        
  there is an agreement                                                             
  regarding eligibility among                                                       
  counties.                                                                         

                                                                        1           Client
                                                                                    Error

                                                                        2           Agency
                                                                                    Error

                                                                                    

  **320 HOUSEHOLD MEMBERS**                                             320 RESULTS 

  To receive services a child's                                         0           No Error
  parent or parents must be                                                         
  working or attending a job                                                        
  training or educational                                                           
  program.                                                                          

                                                                        1           Client
                                                                                    Error

                                                                        2           Agency
                                                                                    Error

                                                                                    

  **330 QUALIFYING CHILD**                                              330 RESULTS 

  Determine if child(ren) is                                            0           No Error
  eligible for services,                                                            
  including (1) younger than 13                                                     
  years, (2) younger than 19                                                        
  years and physically or                                                           
  mentally incapable of caring                                                      
  for himself or herself, or                                                        
  under court supervision, and                                                      
  (3) in foster care when                                                           
  defined in the State Plan.                                                        

                                                                        1           Client
                                                                                    Error

                                                                        2           Agency
                                                                                    Error

                                                                                    

  **340 QUALIFYING CARE**                                               340 RESULTS 

  Determine hours and type of                                           0           No Error
  care authorized. Determine                                                        
  required number of hours of                                                       
  care during authorized                                                            
  schedule.                                                                         

                                                                        1           Client
                                                                                    Error

                                                                        2           Agency
                                                                                    Error

                                                                                    

  **350 QUALIFYING PROVIDER                                             350 RESULTS 
  ARRANGEMENT**                                                                     

  Determine if services are                                             0           No Error
  provided within a center-based                                                    
  child care provider, a group                                                      
  home child care provider, a                                                       
  family child care provider, an                                                    
  in-home child care provider,                                                      
  or other provider of child                                                        
  care services for compensation                                                    
  that Is licensed, regulated,                                                      
  or registered under applicable                                                    
  State or local law. Is                                                            
  informal care included?                                                           

                                                                        1           Client
                                                                                    Error

                                                                        2           Agency
                                                                                    Error

                                                                                    

  **360 PROVIDER REQUIREMENTS**                                         360 RESULTS 

  Determine if regulatory                                               0           No Error
  requirements are met.                                                             
  Regulatory requirements means                                                     
  requirements necessary for a                                                      
  provider to legally provide                                                       
  child care services in a State                                                    
  or locality, including                                                            
  registration requirements                                                         
  established under State,                                                          
  local, or tribal law.                                                             

                                                                        1           Client
                                                                                    Error

                                                                        2           Agency
                                                                                    Error

  **SECTION IV. INCOME AND                                                          
  PAYMENTS**                                                                        

  **400 INCOME\                                                         400 RESULTS 
  **                                                                                

  Describe income documentation                                         0           No Error
  verification for each member                                                      
  of the household. Specify time                                                    
  period and all income to be                                                       
  considered, e.g., based on 4                                                      
  weeks prior to application:                                                       
  Collect the following data:                                                       
  (1) head of household                                                             
  employment income, (2) spouse                                                     
  employment income, (3) any                                                        
  changes in income reported,                                                       
  (4) income during job training                                                    
  for parent/caregiver, (5)                                                         
  child support, if included as                                                     
  part of income, (6) Food                                                          
  Stamps, if included as part of                                                    
  income, and (7) loss of income                                                    
  during eligibility period.                                                        

                                                                        1           Client
                                                                                    Error

                                                                        2           Agency
                                                                                    Error

                                                                                    

  **410 INCOME ELIGIBILITY**                                            410 RESULTS 

  Determine if household income                                         0           No Error
  meets State requirements                                                          
  (e.g., family gross income                                                        
  must be within 50% of State's                                                     
  median income).                                                                   

                                                                        1           Client
                                                                                    Error

                                                                        2           Agency
                                                                                    Error

                                                                                    

  **420 PAYMENTS, GENERAL**                                             420 RESULTS 

  Determine if payments were                                            0           No Error
  made. A sliding fee scale                                                         
  based on income and the size                                                      
  of the family and may be based                                                    
  on other factors as                                                               
  appropriate.                                                                      

                                                                        1           Client
                                                                                    Error

                                                                        2           Agency
                                                                                    Error

                                                                                    

  **430 PAYMENTS/COMPUTATIONS**                                         430 RESULTS 

  Determine difference in dollar                                        0           No Error
  amount of child care benefits                                                     
  authorized versus the amount                                                      
  that should have been                                                             
  authorized---indicate if it is                                                    
  an overpayment or                                                                 
  underpayment.                                                                     

                                                                        1           Client
                                                                                    Error

                                                                        2           Agency
                                                                                    Error
  ------------------------------------------------------------------------------------------

[]{#__RefHeading___Toc167079404 .anchor}**APPENDIX G. DATA ENTRY FORM**

+------------------------------------------------+---------------------+
| **VARIABLE**                                   | **VALUE**           |
+------------------------------------------------+---------------------+
| **State**                                      |                     |
+------------------------------------------------+---------------------+
| **County**                                     |                     |
+------------------------------------------------+---------------------+
| **Child ID**                                   |                     |
+------------------------------------------------+---------------------+
| **Study Period Month (October 2004)\]**        |                     |
+------------------------------------------------+---------------------+
| **Date of data collection**                    |                     |
+------------------------------------------------+---------------------+
| **One or more eligibility errors during pilot  |                     |
| period\]**                                     |                     |
|                                                |                     |
| **0 =no errors, 1 = one or more errors**       |                     |
+------------------------------------------------+---------------------+
| **Cause of improper payment**                  |                     |
|                                                |                     |
| **0 =no errors, 1 = client, 2 = agency**       |                     |
+------------------------------------------------+---------------------+
| **Total amount of improper payments during     |                     |
| review month**                                 |                     |
+------------------------------------------------+---------------------+
| **Total amount of payments during review       |                     |
| month\]**                                      |                     |
+------------------------------------------------+---------------------+

[]{#__RefHeading___Toc167079405 .anchor}**Appendix H. ERROR RATE COST
ANALYSIS**

Pilot States are asked to track the time utilized by different staff to
complete each of the error rate pilot tasks. These tasks include:

-   Preparation and training (including the CCB site visit);

-   Drawing the sample;

-   Record review process;

-   Consolidating the data on the Data Report Forms;

-   Reviewing the findings and responding to the three questions; and

-   Discussion of causes/strategies and report preparation.

Costs per task are estimated by multiplying the length of time to
conduct the task by the average hourly rate (salary + benefits) of the
relevant staff person's job classification. Each task may require time
from different staff persons, e.g., staff time should be tracked
according to job classification---Administrative Staff/Review Team,
Computer Support, and Clerical Support---to allow for differing salary
ranges. Within these job categories, costs will be estimated to conduct
each of the activities listed above. Please note that States should
include the time involved in the site visits as part of the first
preparation and training activity.

In addition, States are asked to keep track of other indirect costs,
e.g., materials, copying, postage, travel and other costs. The following
template lists the categories of tasks, job classifications, time in
hours, hours (salary + benefits) and total costs. A spreadsheet with
each of the tasks is attached for States to use to enter their hours and
cost information.

  -------------------- ---------------- ---------- ------------ -----------
  **Task**             **Job category** **Time**   **Cost Per   **Cost**
                                                   Unit**       

  **Preparation and                                              
  training**                                                    

                       Administrative                           

                       Computer support                         

                       Clerical support                         

  **Record review                                                
  process**                                                     

                       Administrative                           

                       Computer support                         

                       Clerical support                         

  **Drawing the                                                  
  sample**                                                      

                       Administrative                           

                       Computer support                         

                       Clerical support                         

  **Consolidation of                                             
  the data and other                                            
  support activities**                                          

                       Administrative                           

                       Computer support                         

                       Clerical support                         

  **Reviewing the                                                
  findings and                                                  
  responding to the                                             
  three questions**                                             

                       Administrative                           

                       Computer support                         

                       Clerical support                         

  **Discussion of                                                
  causes/strategies                                             
  and report                                                    
  preparation**                                                 

                       Administrative                           

                       Computer support                         

                       Clerical support                         

  **Other indirect                                               
  costs**                                                       

                       Materials                                

                       Copying                                  

                       Postage                                  

                       Travel                                   

                       Other                                    

  **Total**                                                      
  -------------------- ---------------- ---------- ------------ -----------

[]{#__RefHeading___Toc167079406 .anchor}**APPENDIX I. STATE RESPONSE
FORM**

**[Walter R. McDonald & Associates, Inc.]{.underline}**

**Child Care Improper Payments**

**Error Rate Pilot Study**

Following receipt of the contractor statistical analysis of the pilot
findings, the State will prepare---and return to the contractor---a
written response to *each* of the three questions presented in this
document.

**Directions**

Please identify the respondent to each question. If one respondent
provides a response for all of the questions, it is necessary to provide
the contact information only once. The information will be used by the
Error Rate Pilot Study Team to contact the respondent should the need
arise for further clarification of the response.

The respondent(s) should be as thorough as possible in the responses to
the question(s). All responses will be shared with the Child Care
Bureau.

Please return this document in its entirety via e-mail to Carol Pearson,
Project Director of the Child Care Improper Payments Pilot Study, at
<cpearson@wrma.com>. Should you have questions, please contact Carol at
the above e-mail or by phone at (302) 226-1542.

**Please return the completed document by**

***Thank you!***

**Question 1.**

**Date**:

**Name**:

**Title**:

**State Agency**:

**(Area Code) Phone Number**:

**E-mail**:

**What are the causes of the improper payments that have been
identified? What are the actions that will be taken to correct those
causes in order to reduce errors in the future?**

**Question 2.**

**Date**:

**Name**:

**Title**:

**State Agency**:

**(Area Code) Phone Number**:

**E-mail**:

**Describe the information systems and other infrastructure that assist
the State in identifying and reducing improper payments. If the Lead
Agency does not have these tools, describe actions to be taken to
acquire the necessary information systems and other infrastructure, *as
required in the Improper Payments Information Act of 2002*.**

**Question 3.**

**Date**:

**Name**:

**Title**:

**State Agency**:

**(Area Code) Phone Number**:

**E-mail**:

**What steps will the State take to ensure that the Lead Agency and
eligibility** **workers will be accountable for reducing improper
payments?**

[]{#__RefHeading___Toc167079407 .anchor}**APPENDIX J. SITE VISIT
AGENDA**

+---+---------------------------------------------------------------------+
| I |                                                                     |
| n |                                                                     |
| t |                                                                     |
| r |                                                                     |
| o |                                                                     |
| d |                                                                     |
| u |                                                                     |
| c |                                                                     |
| t |                                                                     |
| i |                                                                     |
| o |                                                                     |
| n |                                                                     |
| s |                                                                     |
+---+---------------------------------------------------------------------+
| O |                                                                     |
| v |                                                                     |
| e |                                                                     |
| r |                                                                     |
| v |                                                                     |
| i |                                                                     |
| e |                                                                     |
| w |                                                                     |
| o |                                                                     |
| f |                                                                     |
| P |                                                                     |
| r |                                                                     |
| o |                                                                     |
| j |                                                                     |
| e |                                                                     |
| c |                                                                     |
| t |                                                                     |
| G |                                                                     |
| o |                                                                     |
| a |                                                                     |
| l |                                                                     |
| s |                                                                     |
| a |                                                                     |
| n |                                                                     |
| d |                                                                     |
| O |                                                                     |
| b |                                                                     |
| j |                                                                     |
| e |                                                                     |
| c |                                                                     |
| t |                                                                     |
| i |                                                                     |
| v |                                                                     |
| e |                                                                     |
| s |                                                                     |
+---+---------------------------------------------------------------------+
| O |                                                                     |
| v |                                                                     |
| e |                                                                     |
| r |                                                                     |
| v |                                                                     |
| i |                                                                     |
| e |                                                                     |
| w |                                                                     |
| o |                                                                     |
| f |                                                                     |
| I |                                                                     |
| m |                                                                     |
| p |                                                                     |
| r |                                                                     |
| o |                                                                     |
| p |                                                                     |
| e |                                                                     |
| r |                                                                     |
| P |                                                                     |
| a |                                                                     |
| y |                                                                     |
| m |                                                                     |
| e |                                                                     |
| n |                                                                     |
| t |                                                                     |
| s |                                                                     |
| i |                                                                     |
| n |                                                                     |
| C |                                                                     |
| h |                                                                     |
| i |                                                                     |
| l |                                                                     |
| d |                                                                     |
| C |                                                                     |
| a |                                                                     |
| r |                                                                     |
| e |                                                                     |
| i |                                                                     |
| n |                                                                     |
| y |                                                                     |
| o |                                                                     |
| u |                                                                     |
| r |                                                                     |
| S |                                                                     |
| t |                                                                     |
| a |                                                                     |
| t |                                                                     |
| e |                                                                     |
| ( |                                                                     |
| P |                                                                     |
| l |                                                                     |
| e |                                                                     |
| a |                                                                     |
| s |                                                                     |
| e |                                                                     |
| c |                                                                     |
| o |                                                                     |
| v |                                                                     |
| e |                                                                     |
| r |                                                                     |
| t |                                                                     |
| h |                                                                     |
| e |                                                                     |
| f |                                                                     |
| o |                                                                     |
| l |                                                                     |
| l |                                                                     |
| o |                                                                     |
| w |                                                                     |
| i |                                                                     |
| n |                                                                     |
| g |                                                                     |
| t |                                                                     |
| o |                                                                     |
| p |                                                                     |
| i |                                                                     |
| c |                                                                     |
| s |                                                                     |
| i |                                                                     |
| n |                                                                     |
| y |                                                                     |
| o |                                                                     |
| u |                                                                     |
| r |                                                                     |
| p |                                                                     |
| r |                                                                     |
| e |                                                                     |
| s |                                                                     |
| e |                                                                     |
| n |                                                                     |
| t |                                                                     |
| a |                                                                     |
| t |                                                                     |
| i |                                                                     |
| o |                                                                     |
| n |                                                                     |
| ) |                                                                     |
+---+---------------------------------------------------------------------+
|   | How are improper payments defined in legislation or policy in your  |
|   | State?                                                              |
+---+---------------------------------------------------------------------+
|   | Describe any activities for which your State has policies or        |
|   | regulations in place for the program, such as:                      |
+---+---------------------------------------------------------------------+
|   | -   Definition of an improper payment                               |
+---+---------------------------------------------------------------------+
|   | -   Steps involved in identifying a potential improper payment      |
+---+---------------------------------------------------------------------+
|   | -   Steps involved in verifying an improper payment                 |
+---+---------------------------------------------------------------------+
|   | -   Establishing claims for underpayments                           |
+---+---------------------------------------------------------------------+
|   | -   Collecting overpayments                                         |
+---+---------------------------------------------------------------------+
|   | -   Sources of funding for addressing improper payments             |
+---+---------------------------------------------------------------------+
|   | What data sources does your State use to ensure accurate payments?  |
+---+---------------------------------------------------------------------+
|   | What steps does your State take to prevent and detect improper      |
|   | payments?                                                           |
+---+---------------------------------------------------------------------+
|   | How will your State utilize an error rate to improve payment        |
|   | accuracy?                                                           |
+---+---------------------------------------------------------------------+
|   | What automated systems are in place in your State?                  |
+---+---------------------------------------------------------------------+
|   | Do you use automation to determine eligibility?                     |
+---+---------------------------------------------------------------------+
|   | Are family data, provider data, licensing, attendance or            |
|   | payments/authorizations data automated?                             |
+---+---------------------------------------------------------------------+
|   | Do child care staff have access to other State systems, such as     |
|   | employment/wage data, TANF, Child Support, etc.?                    |
+---+---------------------------------------------------------------------+
|   | Describe any automated systems features such as edits, exception    |
|   | reports or red flags used to identify potential errors.             |
+---+---------------------------------------------------------------------+
| B |                                                                     |
| r |                                                                     |
| e |                                                                     |
| a |                                                                     |
| k |                                                                     |
+---+---------------------------------------------------------------------+
| O |                                                                     |
| v |                                                                     |
| e |                                                                     |
| r |                                                                     |
| v |                                                                     |
| i |                                                                     |
| e |                                                                     |
| w |                                                                     |
| o |                                                                     |
| f |                                                                     |
| t |                                                                     |
| h |                                                                     |
| e |                                                                     |
| C |                                                                     |
| l |                                                                     |
| i |                                                                     |
| e |                                                                     |
| n |                                                                     |
| t |                                                                     |
| E |                                                                     |
| l |                                                                     |
| i |                                                                     |
| g |                                                                     |
| i |                                                                     |
| b |                                                                     |
| i |                                                                     |
| l |                                                                     |
| i |                                                                     |
| t |                                                                     |
| y |                                                                     |
| P |                                                                     |
| r |                                                                     |
| o |                                                                     |
| c |                                                                     |
| e |                                                                     |
| s |                                                                     |
| s |                                                                     |
| i |                                                                     |
| n |                                                                     |
| y |                                                                     |
| o |                                                                     |
| u |                                                                     |
| r |                                                                     |
| S |                                                                     |
| t |                                                                     |
| a |                                                                     |
| t |                                                                     |
| e |                                                                     |
+---+---------------------------------------------------------------------+
| L |                                                                     |
| u |                                                                     |
| n |                                                                     |
| c |                                                                     |
| h |                                                                     |
+---+---------------------------------------------------------------------+
| O |                                                                     |
| v |                                                                     |
| e |                                                                     |
| r |                                                                     |
| v |                                                                     |
| i |                                                                     |
| e |                                                                     |
| w |                                                                     |
| o |                                                                     |
| f |                                                                     |
| t |                                                                     |
| h |                                                                     |
| e |                                                                     |
| R |                                                                     |
| e |                                                                     |
| c |                                                                     |
| o |                                                                     |
| r |                                                                     |
| d |                                                                     |
| R |                                                                     |
| e |                                                                     |
| v |                                                                     |
| i |                                                                     |
| e |                                                                     |
| w |                                                                     |
| P |                                                                     |
| r |                                                                     |
| o |                                                                     |
| c |                                                                     |
| e |                                                                     |
| s |                                                                     |
| s |                                                                     |
+---+---------------------------------------------------------------------+
|   | *Record Review Worksheet* criteria                                  |
+---+---------------------------------------------------------------------+
|   | Data sources for the *Data Entry Form*                              |
+---+---------------------------------------------------------------------+
| D |                                                                     |
| e |                                                                     |
| m |                                                                     |
| o |                                                                     |
| n |                                                                     |
| s |                                                                     |
| t |                                                                     |
| r |                                                                     |
| a |                                                                     |
| t |                                                                     |
| i |                                                                     |
| o |                                                                     |
| n |                                                                     |
| o |                                                                     |
| f |                                                                     |
| t |                                                                     |
| h |                                                                     |
| e |                                                                     |
| R |                                                                     |
| e |                                                                     |
| c |                                                                     |
| o |                                                                     |
| r |                                                                     |
| d |                                                                     |
| R |                                                                     |
| e |                                                                     |
| v |                                                                     |
| i |                                                                     |
| e |                                                                     |
| w |                                                                     |
| P |                                                                     |
| r |                                                                     |
| o |                                                                     |
| c |                                                                     |
| e |                                                                     |
| s |                                                                     |
| s |                                                                     |
| w |                                                                     |
| i |                                                                     |
| t |                                                                     |
| h |                                                                     |
| t |                                                                     |
| h |                                                                     |
| e |                                                                     |
| * |                                                                     |
| R |                                                                     |
| e |                                                                     |
| c |                                                                     |
| o |                                                                     |
| r |                                                                     |
| d |                                                                     |
| R |                                                                     |
| e |                                                                     |
| v |                                                                     |
| i |                                                                     |
| e |                                                                     |
| w |                                                                     |
| W |                                                                     |
| o |                                                                     |
| r |                                                                     |
| k |                                                                     |
| s |                                                                     |
| h |                                                                     |
| e |                                                                     |
| e |                                                                     |
| t |                                                                     |
| * |                                                                     |
| a |                                                                     |
| n |                                                                     |
| d |                                                                     |
| * |                                                                     |
| D |                                                                     |
| a |                                                                     |
| t |                                                                     |
| a |                                                                     |
| E |                                                                     |
| n |                                                                     |
| t |                                                                     |
| r |                                                                     |
| y |                                                                     |
| F |                                                                     |
| o |                                                                     |
| r |                                                                     |
| m |                                                                     |
| * |                                                                     |
+---+---------------------------------------------------------------------+
| B |                                                                     |
| r |                                                                     |
| e |                                                                     |
| a |                                                                     |
| k |                                                                     |
+---+---------------------------------------------------------------------+
| O |                                                                     |
| v |                                                                     |
| e |                                                                     |
| r |                                                                     |
| v |                                                                     |
| i |                                                                     |
| e |                                                                     |
| w |                                                                     |
| o |                                                                     |
| f |                                                                     |
| S |                                                                     |
| t |                                                                     |
| a |                                                                     |
| t |                                                                     |
| e |                                                                     |
| R |                                                                     |
| e |                                                                     |
| v |                                                                     |
| i |                                                                     |
| e |                                                                     |
| w |                                                                     |
| P |                                                                     |
| r |                                                                     |
| o |                                                                     |
| c |                                                                     |
| e |                                                                     |
| s |                                                                     |
| s |                                                                     |
+---+---------------------------------------------------------------------+
|   | Sampling frame and number per county sampled                        |
+---+---------------------------------------------------------------------+
|   | Logistics of the review process                                     |
+---+---------------------------------------------------------------------+
|   | Timeframe                                                           |
+---+---------------------------------------------------------------------+
|   | Next steps                                                          |
+---+---------------------------------------------------------------------+
| M |                                                                     |
| e |                                                                     |
| e |                                                                     |
| t |                                                                     |
| w |                                                                     |
| i |                                                                     |
| t |                                                                     |
| h |                                                                     |
| A |                                                                     |
| d |                                                                     |
| d |                                                                     |
| i |                                                                     |
| t |                                                                     |
| i |                                                                     |
| o |                                                                     |
| n |                                                                     |
| a |                                                                     |
| l |                                                                     |
| S |                                                                     |
| t |                                                                     |
| a |                                                                     |
| f |                                                                     |
| f |                                                                     |
| ( |                                                                     |
| T |                                                                     |
| h |                                                                     |
| e |                                                                     |
| S |                                                                     |
| t |                                                                     |
| a |                                                                     |
| t |                                                                     |
| e |                                                                     |
| s |                                                                     |
| i |                                                                     |
| d |                                                                     |
| e |                                                                     |
| n |                                                                     |
| t |                                                                     |
| i |                                                                     |
| f |                                                                     |
| i |                                                                     |
| e |                                                                     |
| d |                                                                     |
| a |                                                                     |
| d |                                                                     |
| d |                                                                     |
| i |                                                                     |
| t |                                                                     |
| i |                                                                     |
| o |                                                                     |
| n |                                                                     |
| a |                                                                     |
| l |                                                                     |
| s |                                                                     |
| t |                                                                     |
| a |                                                                     |
| f |                                                                     |
| f |                                                                     |
| w |                                                                     |
| h |                                                                     |
| o |                                                                     |
| w |                                                                     |
| o |                                                                     |
| u |                                                                     |
| l |                                                                     |
| d |                                                                     |
| a |                                                                     |
| d |                                                                     |
| d |                                                                     |
| t |                                                                     |
| o |                                                                     |
| t |                                                                     |
| h |                                                                     |
| e |                                                                     |
| u |                                                                     |
| n |                                                                     |
| d |                                                                     |
| e |                                                                     |
| r |                                                                     |
| s |                                                                     |
| t |                                                                     |
| a |                                                                     |
| n |                                                                     |
| d |                                                                     |
| i |                                                                     |
| n |                                                                     |
| g |                                                                     |
| o |                                                                     |
| f |                                                                     |
| t |                                                                     |
| h |                                                                     |
| e |                                                                     |
| C |                                                                     |
| h |                                                                     |
| i |                                                                     |
| l |                                                                     |
| d |                                                                     |
| C |                                                                     |
| a |                                                                     |
| r |                                                                     |
| e |                                                                     |
| s |                                                                     |
| y |                                                                     |
| s |                                                                     |
| t |                                                                     |
| e |                                                                     |
| m |                                                                     |
| - |                                                                     |
| - |                                                                     |
| - |                                                                     |
| c |                                                                     |
| h |                                                                     |
| i |                                                                     |
| l |                                                                     |
| d |                                                                     |
| c |                                                                     |
| a |                                                                     |
| r |                                                                     |
| e |                                                                     |
| p |                                                                     |
| a |                                                                     |
| y |                                                                     |
| m |                                                                     |
| e |                                                                     |
| n |                                                                     |
| t |                                                                     |
| o |                                                                     |
| r |                                                                     |
| f |                                                                     |
| i |                                                                     |
| s |                                                                     |
| c |                                                                     |
| a |                                                                     |
| l |                                                                     |
| s |                                                                     |
| t |                                                                     |
| a |                                                                     |
| f |                                                                     |
| f |                                                                     |
| o |                                                                     |
| r |                                                                     |
| o |                                                                     |
| t |                                                                     |
| h |                                                                     |
| e |                                                                     |
| r |                                                                     |
| q |                                                                     |
| u |                                                                     |
| a |                                                                     |
| l |                                                                     |
| i |                                                                     |
| t |                                                                     |
| y |                                                                     |
| c |                                                                     |
| o |                                                                     |
| n |                                                                     |
| t |                                                                     |
| r |                                                                     |
| o |                                                                     |
| l |                                                                     |
| s |                                                                     |
| t |                                                                     |
| a |                                                                     |
| f |                                                                     |
| f |                                                                     |
| f |                                                                     |
| r |                                                                     |
| o |                                                                     |
| m |                                                                     |
| a |                                                                     |
| s |                                                                     |
| s |                                                                     |
| o |                                                                     |
| c |                                                                     |
| i |                                                                     |
| a |                                                                     |
| t |                                                                     |
| e |                                                                     |
| d |                                                                     |
| p |                                                                     |
| r |                                                                     |
| o |                                                                     |
| g |                                                                     |
| r |                                                                     |
| a |                                                                     |
| m |                                                                     |
| s |                                                                     |
| s |                                                                     |
| u |                                                                     |
| c |                                                                     |
| h |                                                                     |
| a |                                                                     |
| s |                                                                     |
| T |                                                                     |
| A |                                                                     |
| N |                                                                     |
| F |                                                                     |
| o |                                                                     |
| r |                                                                     |
| F |                                                                     |
| o |                                                                     |
| o |                                                                     |
| d |                                                                     |
| S |                                                                     |
| t |                                                                     |
| a |                                                                     |
| m |                                                                     |
| p |                                                                     |
| s |                                                                     |
| . |                                                                     |
| ) |                                                                     |
+---+---------------------------------------------------------------------+

[]{#__RefHeading___Toc167079408 .anchor}**APPENDIX K. Summary of Pilot
Study States**

<table>
<colgroup>
<col style="width: 9%" />
<col style="width: 11%" />
<col style="width: 10%" />
<col style="width: 9%" />
<col style="width: 10%" />
<col style="width: 9%" />
<col style="width: 9%" />
<col style="width: 10%" />
<col style="width: 10%" />
<col style="width: 9%" />
</colgroup>
<thead>
<tr class="header">
<th rowspan="2"><strong>Summary Topic</strong></th>
<th colspan="4"><strong>States Participating in First
Pilot</strong></th>
<th colspan="5"><strong>States Participating in Second
Pilot</strong></th>
</tr>
<tr class="odd">
<th><strong>Arkansas</strong></th>
<th><strong>Colorado</strong></th>
<th><strong>Illinois</strong></th>
<th><strong>Ohio</strong></th>
<th><strong>Florida</strong></th>
<th><strong>Kansas</strong></th>
<th><strong>New Jersey</strong></th>
<th><strong>Oregon</strong></th>
<th><strong>West Virginia</strong></th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td><strong>State Structure</strong></td>
<td>State administered</td>
<td>County administered, State supervised</td>
<td>State administered</td>
<td>County administered, State supervised</td>
<td>State administered</td>
<td>State administered</td>
<td>County administered, State TANF supervised</td>
<td>County administered, State supervised</td>
<td>State administered</td>
</tr>
<tr class="even">
<td><strong>Improper Payment Defined</strong></td>
<td>Administrative Policy (Only misrepresentation in statute)</td>
<td>Administrative Policy</td>
<td>Administrative Policy</td>
<td>Policy (Only loosely defined)</td>
<td>General, not established in state policy</td>
<td>Administrative Policy</td>
<td>Administrative Policy</td>
<td>Administrative Policy</td>
<td>Administrative Policy</td>
</tr>
<tr class="odd">
<td><strong>Improper Payments Processes</strong></td>
<td><ul>
<li><p>Complaints or calls to the agency</p></li>
<li><p>Worker reviews</p></li>
<li><p>Supervisory reviews</p></li>
<li><p>Random provider reviews</p></li>
<li><p>Unannounced visits</p></li>
<li><p>Annual audits</p></li>
</ul></td>
<td><ul>
<li><p>Complaints or calls to the agency</p></li>
<li><p>Counties investigate first</p></li>
<li><p>Office of Program Improvement involved if necessary</p></li>
</ul></td>
<td><ul>
<li><p>Complaints or calls to the agency</p></li>
<li><p>Program Integrity and Quality Assurance Unit conducts
regularly-scheduled audits</p></li>
</ul></td>
<td><ul>
<li><p>Complaints or calls to the agency</p></li>
<li><p>Some counties have regular supervisor reviews</p></li>
<li><p>Counties conduct monitoring reviews, State follows-up</p></li>
</ul></td>
<td><ul>
<li><p>Technical assistance provided local staff</p></li>
<li><p>Annual monitoring plan of all ELCs</p></li>
<li><p>Monthly invoice reviews</p></li>
</ul></td>
<td><ul>
<li><p>Complaints from the public</p></li>
<li><p>Review attendance sheets in the field</p></li>
<li><p>Field staff review</p></li>
<li><p>Inclusion of child care in QC review process</p></li>
</ul></td>
<td><ul>
<li><p>Case review of a sampling of cases</p></li>
<li><p>Internal controls for monitoring vouchers</p></li>
</ul></td>
<td><ul>
<li><p>Hotline number</p></li>
<li><p>Improper Payments staff</p></li>
<li><p>Monthly desk audit of 200 randomly selected billing
forms</p></li>
</ul></td>
<td><ul>
<li><p>Policy Specialist provides technical assistance to
CCR&amp;Rs</p></li>
<li><p>Case review of a samp-ling of cases</p></li>
<li><p>Two staff provide technical assistance to CCR&amp;Rs</p></li>
</ul></td>
</tr>
<tr class="even">
<td><strong>Assessment of Risk</strong></td>
<td><ul>
<li><p>Compliance Review Team</p></li>
<li><p>Data mining</p></li>
<li><p>Benchmark reporting</p></li>
<li><p>Exception reporting</p></li>
</ul></td>
<td><ul>
<li><p>Office of Program Improvement</p></li>
<li><p>Ad hoc studies</p></li>
<li><p>Counties monitor improper payments</p></li>
<li><p>Special studies</p></li>
</ul></td>
<td><ul>
<li><p>Reliance on exception reporting</p></li>
<li><p>Ongoing training</p></li>
</ul></td>
<td><ul>
<li><p>State visits each county twice during a 15-month period</p></li>
<li><p>Incident Tracking system of injury to a child in care</p></li>
</ul></td>
<td><ul>
<li><p>Field staff review local financial &amp; program
operations</p></li>
<li><p>Exception reports</p></li>
<li><p>Local Ad hoc report writing capability</p></li>
</ul></td>
<td><ul>
<li><p>Exception reports generated from automated systems</p></li>
<li><p>Audits based on department request</p></li>
</ul></td>
<td><ul>
<li><p>Data Matching</p></li>
<li><p>Exception reports generated from CARES and OMEGA systems</p></li>
<li><p>Quality Assurance committee</p></li>
</ul></td>
<td><ul>
<li><p>Exception reports</p></li>
<li><p>Monthly match with, Food Stamps and Medicaid files</p></li>
<li><p>Child care reviews with Food Stamp reviews</p></li>
</ul></td>
<td><ul>
<li><p>FACTS edits identify anomalies</p></li>
<li><p>Mandatory CCR&amp;R Quality Assurance Plans</p></li>
</ul></td>
</tr>
<tr class="odd">
<td><strong>Automation</strong></td>
<td><ul>
<li><p>New system</p></li>
<li><p>Up-front edits</p></li>
<li><p>Interface with TANF, Food Stamps, child support</p></li>
</ul></td>
<td><ul>
<li><p>Mainframe statewide system that must be used</p></li>
<li><p>Most counties have own systems</p></li>
</ul></td>
<td><ul>
<li><p>Mainframe system</p></li>
<li><p>Interface with TANF, Food Stamps, child support</p></li>
<li><p>Up-front edits</p></li>
<li><p>Billing process is automated for CCR&amp;Rs and will soon be for
contract providers</p></li>
</ul></td>
<td><ul>
<li><p>Mainframe system</p></li>
<li><p>Interface with TANF, Food Stamps, child support</p></li>
<li><p>Up-front edits, but eligibility determin-ation is not
automated</p></li>
</ul></td>
<td><ul>
<li><p>Enhanced Field System (EFS) makes payments</p></li>
<li><p>Eligibility determin-ation module</p></li>
</ul>
<ul>
<li><p>EFS has edits for age, atten-dance,and payment rate.</p></li>
</ul></td>
<td><ul>
<li><p>Mainframe system</p></li>
<li><p>Interface with TANF, Food Stamps, child support</p></li>
<li><p>Up-front edits, eligibility determin-ation is not
automated</p></li>
<li><p>EBT for billing</p></li>
</ul></td>
<td><ul>
<li><p>Mainframe system</p></li>
<li><p>Eligibility determine-ation, subsidy and co-pay calculation,
voucher generation, attendance, check write process </p></li>
</ul></td>
<td><ul>
<li><p>Mainframe legacy system</p></li>
<li><p>Eligibility determin-ation</p></li>
<li><p>Interface with SAS payment system</p></li>
</ul></td>
<td><ul>
<li><p>Web based SACWIS system.</p></li>
<li><p>Eligibility processing with eligibility and payment
edits.</p></li>
<li><p>Connectivity with the public assistance eligibility system
(RAPIDS)</p></li>
<li><p>Payment Processing</p></li>
</ul></td>
</tr>
</tbody>
</table>

+-----------+---------+----------+-----------+-----------------------+
| **Summary | *       |          |           |                       |
| Topic**   | *States |          |           |                       |
|           | Partic  |          |           |                       |
|           | ipating |          |           |                       |
|           | in      |          |           |                       |
|           | Disc    |          |           |                       |
|           | ussions |          |           |                       |
|           | **[^22] |          |           |                       |
+===========+=========+==========+===========+=======================+
|           | **Ar    | **Cali   | **N       | **New Hampshire**     |
|           | izona** | fornia** | ebraska** |                       |
+-----------+---------+----------+-----------+-----------------------+
| **State   | Stat    | Coun     | State-adm | State-supervised      |
| St        | e-admin | ty-admin | inistered |                       |
| ructure** | istered | istered, |           |                       |
|           |         | State-su |           |                       |
|           |         | pervised |           |                       |
+-----------+---------+----------+-----------+-----------------------+
| *         | Statute | Statute  | Policy    | State Plan            |
| *Improper |         |          | (Only IPV |                       |
| Payment   |         |          | in        |                       |
| Defined** |         |          | Statute)  |                       |
+-----------+---------+----------+-----------+-----------------------+
| *         | -   Com | -   Co   | -   C     | -   Complaints or     |
| *Improper | plaints | mplaints | omplaints |     calls to the      |
| Payments  |     or  |     or   |     or    |     agency            |
| Pr        |         |          |     calls |                       |
| ocesses** |   calls |    calls |     to    | -   Task Force for    |
|           |     to  |     to   |     the   |     Improper Payments |
|           |     the |     the  |           |     established       |
|           |         |          |    agency |     across program    |
|           |  agency |   agency |           |     areas             |
|           |         |          | -         |                       |
|           | -       | -        |  Issuance |                       |
|           | Monthly | Improper |     and   |                       |
|           |     sup |          |     C     |                       |
|           | ervisor |  Payment | ollection |                       |
|           | y-level |     Unit |           |                       |
|           |         |     est  |    Center |                       |
|           |    case | ablished |     Unit  |                       |
|           |         |          |           |                       |
|           | reviews | -        |    audits |                       |
|           |         |   Annual |     child |                       |
|           | -       |          |     care  |                       |
|           |  Annual |   audits |           |                       |
|           |         |          |           |                       |
|           |    case |          |           |                       |
|           |         |          |           |                       |
|           | reviews |          |           |                       |
|           |         |          |           |                       |
|           | -       |          |           |                       |
|           |  Audits |          |           |                       |
|           |     of  |          |           |                       |
|           |     p   |          |           |                       |
|           | rovider |          |           |                       |
|           |         |          |           |                       |
|           | records |          |           |                       |
+-----------+---------+----------+-----------+-----------------------+
| **A       | Ex      | Improper | -         | -   Developing        |
| ssessment | ception | Payments | Exception |     queries for       |
| of Risk** | reports | Unit     |           |     error-prone       |
|           |         |          |   reports |     profiling         |
|           |         |          |           |                       |
|           |         |          | -         | -   Educating         |
|           |         |          |  Attorney |     providers to      |
|           |         |          |     is    |     encourage         |
|           |         |          |           |     referrals         |
|           |         |          |  assigned |                       |
|           |         |          |     to    | -   Establishing      |
|           |         |          |     child |     Internet billing  |
|           |         |          |     care  |                       |
|           |         |          |           | -   Developing        |
|           |         |          |           |     automated         |
|           |         |          |           |     exception reports |
+-----------+---------+----------+-----------+-----------------------+
| **Aut     | -   Ma  | -   No   | -         | -   New system        |
| omation** | inframe |          | Mainframe |                       |
|           |         |    State |           | -   Up-front edits    |
|           |  system |     a    |    system |                       |
|           |         | utomated |           | -   System integrated |
|           | -   In  |          | -   I     |     with TANF, Food   |
|           | terface |  system, | ntegrated |     Stamps, child     |
|           |         |     only |     with  |     support           |
|           |    with |     a    |     TANF, |                       |
|           |         | ggregate |     Food  | -   Interfaces to     |
|           |   TANF, |     data |           |     most other        |
|           |         |     c    |   Stamps, |     systems           |
|           |    Food | ollected |     child |                       |
|           |         |     to   |           |                       |
|           | Stamps, |     meet |   support |                       |
|           |         |          |           |                       |
|           |   child |  Federal | -   El    |                       |
|           |         |     requ | igibility |                       |
|           | support | irements |     c     |                       |
|           |         |          | alculated |                       |
|           |         | -   Many |           |                       |
|           |         |          |  off-line |                       |
|           |         | counties |           |                       |
|           |         |     have |           |                       |
|           |         |          |           |                       |
|           |         |    their |           |                       |
|           |         |     own  |           |                       |
|           |         |     a    |           |                       |
|           |         | utomated |           |                       |
|           |         |          |           |                       |
|           |         |  systems |           |                       |
+-----------+---------+----------+-----------+-----------------------+

[^1]: Child Care Development Fund Fact Sheet (October 2006) available on
    the Child Care Bureau website:
    http://www.acf.hhs.gov/programs/ccb/ccdf/factsheet.htm.

[^2]: For both pilots, States calculated and recorded the authorized
    payment amount based on the eligibility determination process as
    indicated within the case record. In both pilots the term "payment"
    refers to the amount authorized for payment.

    .

[^3]: See Chapter IV for the specific findings about administrative
    policies and procedures upon which these findings are based.

[^4]: This would apply if a State wishes to apply the methodology in a
    review program independent of a CCB pilot.

[^5]: The Improper Payment Information Act recommends using a formula to
    determine the sample size for calculating error rates in Federally
    funded programs, leading to an error rate with a 90 percent
    confidence interval of +/- 2.5 percent.

[^6]: In FFY 2004, the CCB visited six States (Arkansas, Connecticut,
    Indiana, Ohio, Oklahoma, and Virginia) and worked extensively with
    five additional partner States (Georgia, Maryland, Oregon, South
    Carolina, and Wisconsin).

[^7]: Child Care Development Fund Fact Sheet (October 2006) available on
    the Child Care Bureau website:
    http://www.acf.hhs.gov/programs/ccb/ccdf/factsheet.htm.

[^8]: For both pilots, States calculated and recorded the authorized
    payment amount based on the eligibility determination process as
    indicated within the case record. In both pilots the term "payment"
    refers to the amount authorized for payment.

    .

[^9]: For both pilots, States calculated and recorded the authorized
    payment amount based on the eligibility determination process as
    indicated within the case record. In both pilots the term "payment"
    refers to the amount authorized for payment.

    []{.mark}

    []{.mark}

[^10]: For both pilots, States calculated and recorded the authorized
    payment amount based on the eligibility determination process as
    indicated within the case record. In both pilots the term "payment"
    refers to the amount authorized for payment.

[^11]: For the purposes of this pilot a child randomly selected for
    review is referred to as a case.

[^12]: The Improper Payment Information Act, Public Law No: 107-300, §
    2, 116 Stat. 2350.

[^13]: The single-stage random sample selects cases directly from the
    statewide universe of cases authorized to receive payments during
    the sample month. The two-stage random statewide sample selects a
    set of counties first (first stage), from which subsamples (second
    stage) of cases are drawn such that the total cases from all of the
    subsamples equals the total sample size of a single-stage sample for
    the sample month.

[^14]: Some counties served 10 or fewer cases during October 2005. The
    pilot team grouped into clusters counties that served fewer than 25
    cases, such that at least 25 cases were in the cluster. (For
    example, if county A served 6 cases in October 2005 and county B
    served 19 cases, together they served 25 cases in that month. If
    this cluster were selected via the random process, a subsample of 10
    cases would be selected randomly from among the 25 cases in the
    cluster. This process ensured that an entire county would not be
    included in a subsample due to its small size.)

[^15]: For both pilots, States calculated and recorded the authorized
    payment amount based on the eligibility determination process as
    indicated within the case record. In both pilots the term "payment"
    refers to the amount authorized for payment.

[^16]: For both pilots, the payment refers to the amount authorized for
    payment.

[^17]: After the first pilot, the CCB asked the contractor to expand
    upon the findings and analyze the costs incurred by the State and
    Federal participants to conduct the site visits and record reviews.

[^18]: New Jersey data were not available at the time of final report
    submission.

[^19]: As of the submission of this report, New Jersey cost data were
    not available.

[^20]: These are State costs. County costs were not estimated in this
    pilot

[^21]: The Improper Payment Information Act recommends using a formula
    to determine the sample size for calculating error rates in
    Federally funded programs, leading to an error rate with a 90
    percent confidence interval of +/- 2.5 percent.

[^22]: In the first pilot, the team conducted telephone discussions with
    five additional States---Arizona, California, Kansas, Nebraska, and
    New Hampshire---to gather information about the States' policies and
    practices regarding child care improper payments.