Department of Health— --- and -.——_ -

Human se~”rPc - —------ . .Ww” 

OFFICE OF 

INSPECTOR GENERAL 

JOBS MONITORING AND 

TECHNICAL ASSISTANCE: 

HEADQUARTERS’ ROLE 

‘@ SERVICE$., 

AUGUST 1993

OFFICE OF INSPECTOR GENERAL 

The mission of the Office of Inspector General (OIG), as mandated by Public Law 95-

452, as amended, is to protect the integrity of the Department of Health and Human 

Services’ (HHS) programs as well as the health and welfare of beneficiaries served 

by those programs. This statutory mission is carried out through a nationwide 

network of audits, investigations, and inspections conducted by three OIG operating 

components: the Office of Audit Services, the Office of Investigations, and the 

Office of Evaluation and Inspections. The OIG also informs the Secretary of HHS of 

program and management problems and recommends courses to correct them. 

OFFICE OF AUDIT SERVICES 

The OIG’S Office of Audit Semites (OAS) provides all auditing services for HHS, 

either by conducting audits with its own audit resources or by overseeing audit work 

done by others. Audits examine the performance of HHS programs and/or its grantees 

and contractors in carrying out their respective responsibilities and are intended 

to provide independent assessments of HHS programs and operations in order to reduce 

waste, abuse, and mismanagement and to promote economy and efficiency throughout the 

Department. 

OFFICE OF INVESTIGATIONS 

The OIG’S Office of Investigations (01) conducts criminal, civil, and


administrative investigations of allegations of wrongdoing in HHS programs or to HHS


beneficiaries and of unjust enrichment by providers. The investigative efforts of


01 lead to criminal convictions, administrative sanctions, or civil money


penalties. The 01 also oversees State Medicaid fraud control units which


investigate and prosecute fraud and patient abuse in the Medicaid program.


OFFICE OF EVALUATION AND INSPECI’IONS 

The OIG’S Office of Evaluation and Inspections (OEI) conducts short-term management


and program evaluations (called inspections) that focus on issues of concern to the


Department, the Congress, and the public. The findings and recommendations


contained in these inspection reports generate rapid, accurate, and up-to-date


information on the efficiency, vulnerability, and effectiveness of departmental


programs. This report was prepared in the Kansas City regional office under the


direction of James H. Wolf, Regional Inspector General. Project staffi


REGION


Hugh Owens, Project Leader


Perry A. Seaton


Hope Hartshorn


For additional copies of this repo~


at (816) 426-5959.


HEADQUARTERS 

Ruth Folchman 

Mark Krushat 

please contact the Kansas City Re@onal Office 
Department of Health and Human Services 

OFFICE OF 

NSPECTOR GENERAL 

JOBS MONITORING AND 

TECHNICAL ASSISTANCE: 

HEADQUARTERS’ ROLE 

AUGUST 1993 OEI-07-92-00380 
EXECUTIVE SUMMARY 

PURPOSE 

To evaluate the policies and procedures established by the Administration for 

Children and Families (ACF) Headquarters for monitoring and providing technical 

assistance to States for -the Job Opp&tunities and Basic S~lls T;aining (JOBS) 

program. 

BACKGROUND 

The Family Support Act of1988, Public Law 100-485, created the JOBS program for 

recipients of Aid to Families with Dependent Children (AFDC). It establishes a . . 

comprehensive framework and progr~mmatic structure for Federal and State efforts to


help welfare recipients become self sufficient. Included are provisions related to job


skills training, education, job readiness activities, job placement/job development, child


care, transportation, and other supportive services.


The ACF monitors States’ JOBS programs and provides them with policy and


procedural guidance. The ACF Headquarters develops guidance and provides some


technical assistance for regions and States. It also directs and oversees the technical


assistance contract. The regional offices serve as the principal point of contact for the


States. In this role, they monitor JOBS program operations, review and approve State


Plans, and provide technical assistance to States.


METHODOLOGY


In recognition of the differing roles and responsibilities of ACF Headquarters and


regional office components, we are reviewing ACF oversight of the JOBS program in


two phases. Phase one, which is completed, involved a review of ACF Headquarters.


Phase two, currently in progress, will review ACF regional offices and State agencies.


In phase one, we researched the legislative history of the JOBS program by reviewing


the Federal law and regulations and relevant background material. In addition, we


reviewed the written policies, processes, and products developed by ACF


Headquarters for monitoring and providing technical assistance in the JOBS program.


We conducted an on-site visit at ACF Headquarters and interviewed members of the


Agency’s staff responsible for the following key areas: field reviews, on-site visits, State


Plans, data collection, and the technical assistance contract. We also interviewed


members of Washington, D.C.-based advocacy organizations that have an active


interest in the JOBS program.


i 
The ACF and the States faced a number of challenges in implementing the JOBS 

program.This program requires active coordination betieen the Department of 

Health and Human Services (HHS), the Department of Education, and the 

Department of Labor. The States had varying levels of experience in administering a 

job training program for AFDC recipients. In addition, ACF and the States had 

limited resources and funding available. Furthermore, the program went into effect 

less than one year after enactment, leaving ACF and the States very little lead time 

for implementation. 

Despite these challenges, ACF released the final regulations within the one year time 

period mandated by Congress. In addltlon, ~“ it provided guidance and technical

assistance to the States. 

We recognize these efforts and the extraordinarily difficult circumstances under which 

ACFlabored. Wealsorealize thatwedo nothave acomplete picture of ACF 

activities since we have not completed phase two of the study. Nevertheless, we 

identified some weaknesses which we believe merit immediate attention. We offer 

findings and recommendations regarding improvements that need to be made now 

that the flurry of initial implementation is over and the JOBS program is beginning to 

stabilize and mature. These are in the areas of monitoring, technical assistance, 

program guidance, and management information. 

MONRORING 

The ACF Headquarters’ guidance for field reviews and data collection has been 

inadequate. 

The ACF conducted field reviews to obtain information on States’ activities during the 

first two years of program implementation. However, we found that review guides 

were not released timely to the regions and field review reports were not released 

timely to the States. The ACF Headquarters has not issued guidance to the regional 

offices to provide the States with a comment period prior to the release of final field 

review reports; to require States to submit corrective action plans; or to require 

regional offices to follow-up with States on field review findings and recommendations. 

Reviews in FY’s 1990 and 1991 examined supportive services, child care, data 

collection and systems as well as JOBS. However, ACF has not used the field review 

process to monitor the States for compliance with their State Plans. It felt that it was 

inappropriate to do so during the early stages of implementation when it was more 

important to simply get a feel for what was happening at the State level and ensure 

that instructions and guidance were understood. Now that this stage is over, however, 

we believe that a more systematic method of compliance monitoring is needed. 

ii 

.. 
With respect to data collection, we found that ACF has not established a closure date 

for the submission of States’ data and is not receiving State data timely. 

TECHNICAL ASSIST~CE 

The ACF did not reap the full benefits that could have been obtained from the 

technical assistance contractors. 

The ACF chose to use contractors to provide technical assistance to the States. While 

these contractors did provide valuable training and assistance, they did not address all 

of the subjects requested by the States, and some of the materials were not 

disseminated timely or to all the appropriate audiences. 

PROGRAM GULZMNCE 

Some action transmittals and information memoranda were not disseminated in a 

timely fashion. 

JW4.N.AGEMENTINFORhL4TION 

The ACF Headquarters has not systematically validated or analyzed information from


field reviews, State Plans, and data collection systems; nor has it routinely


disseminated management information to Congress, Federal and State managers,


policy makers, and advocacy groups.


RECOMMENDATIONS


Specifically, to address the problems identified above, ACF should:


develop adequate processes to monitor States for compliance with their State 

Plans and JOBS program requiremen~, 

review and clar@ the purpose of existing monitoring initiatives and strengthen 

the policies and procedures guiding their administratio~ 

In partiadq with respect to field reviews: 

.� provide field review guides to the regional offices by the second quarter 

of the fiscal year to allow sufficient lead time to schedule, conduct, and 

complete these reviews; 

�	 ensure that field review reports to the States are prepared and released 

in draft for their comment within 90 days of the field review visit; 

... 

111

� 

.� provide guidance to the regional offices on requiring States to submit 

corrective action plans to address problem areas identified during field 

reviews; and 

provide guidance to the regional offices to ensure that problem areas 

identified in the field reviews are being corrected. 

Wtih respect to data collection: 

�	 continue to work closely with regional offices and the States to ensure 

the accurate, timely, and complete submission of data; and 

� establish deadlines for the correction and finalization of States’ data. 

systematically reassess the specific needs of States for technical assistance and 

provide a timely and appropriate response; 

ensure that action transmittals and information memoranda conform with 

Federal law and regulations and Agency policie~ 

allow 60 to 90 days lead time, whenever possl%le, for the effective dates of 

action transmittals and information memorand~, 

develop a systematic analysis plan for JOBS program information available 

from field reviews, State Plans, and data collection system and 

disseminate program information and data analysis through timely reports on 

varied program elements to Federal and State managers and policy makers. 

AGENCY COMMENTS AND OIG RESPONSE 

The ACF has agreed with most of our recommendations and has outlined specific 

actions it has planned or already taken to implement them. Their one major 

resonation concerns monitoring. They believe that it would be prudent to delay a 

final decision on these findings and recommendations until the second phase of the 

inspection--Regional Offices and States’ Perspectives--has been completed. 

We continue to believe that our findings and recommendations represent concerns 

that should be addressed in Headquarter’s operation of the JOBS program. 

Nevertheless, we accept ACF’S suggestion to delay final action on monitoring until the 

second phase of our study is done. 

The ACF also expressed concerns about some of our findings and wished to 

emphasize numerous positive actions they have taken on matters addressed in our 

rep”ort. We wish to emphasize that we agree that ACF has taken aggressive action to 

implement the JOBS program. We highlighted this in the Executive Summary and in 

iv 
the introduction to our findings. We have provided some additional examples of their 

actions in this final version of the report. Space and presentation style simply do not 

allow us to enumerate everything which ACF has done. For more examples, we invite 

the reader to read ACF comments, which are provided verbatim in Appendix D. 

However, it should be noted that the purpose of this report was to point out areas 

needing further attention. Such areas for improvement inevitably arise no matter how 

intensively management attempts to deal with complex problems. 

We made other modifications to the report in response to ACF’S comments. Again, 

the full text of their comments is contained in Appendix D. 

We understand that ACF has recently established several task forces, one of which is 

to address monitoring and technical assistance. We believe that this is a positive step 

and hope that the information in this report will be useful to the task force addressing 

these functional areas. 

v 
� 

TABLE OF CONTENTS 

PAGE 

EXECUTIVESUMMARY


INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1


FINDINGS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...4


� Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...4


� Technical Assistance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...9


� Program Guidance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11


Management Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12


RECOMMENDATIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14


APPENDICES


&Fiscal Years 1990 -1991 Field Reviews . . . . . . . . . . . . . . . . . . . . . . . . . . ..A-l 

B:	 Submission of Quarterly Participation Rate Statistical Report 

(FSA-103) For FY’s 1991 - 1992 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. B-l 

CDelayed Action Transmittals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ..C-1


D: Agency Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ..D-l 
INTRODUCTION 

PURPOSE 

To evaluate the policies and procedures established by the Administration for 

Children and Families (ACF) Headquarters for monitoring and providing technical 

assistance to States for the Job Opportunities and Basic Skills Training (JOBS) 

program, 

BACKGROUND 

l%eJOBSRqyam 

The Family Support Act of1988, Public Law 100-485, created the JOBS program for


recipients of Aid to Families with Dependent Children (AFDC). Its purpose is to


promote the self sufficiency of welfare recipients through a broad range of


employment, trainin”g, education, and support services. The States must offer job skills


training, education, job readiness activities, job placement/job development, child care,


transportation, and other supportive services. They must also offer at least two of the


following optional services: job search, on-the-job training, work supplementation, or


community work experience. Educational setices include high school or equivalent


education, basic and remedial education to achieve a basic literacy level, and


education for individuals with limited English proficiency.


The Challqp of Program Implementation 

The JOBS program is markedly different from its predecessors. The ACF and the 

States faced a number of challenges in implementing it. 

The program, administered by ACF, requires active coordination among the 

Department of Health and Human Services (HHS), the Department of Education, 

and the Department of Labor. 

The States had varying levels of experience in administering a job training program for 

AFDC recipients. Some of the States had existing job training programs which they 

had to convert to meet the JOBS program requirements, which included education as 

a component. Others had little or no recent experience in administering an education 

and job training program for welfare recipients. Some States were required to create 

new systems or to re-design their existing systems during the period when systems and 

data collection rules and policies were being formulated. Overall, ACF and the States 

had limited resources and funding available. 

Under the legislation, States could implement their program as early as July 1, 1989 or 

as late as October 1, 1990. Fifteen States elected to start on July 1, 1989, prior to the 

issuance of the final regulations. Even those who chose to wait had at most two years 

1 
after enactment of the law to implement their programs. Thus, ACF and the States 

had very little lead time. Despite these challenges, ACF released the final JOBS 

regulations within the one year time period mandated by Congress. 

ACF Head@atiem and Re@nal Ojjice OvernightReqxmsibilitie.s 

The ACF monitors States’ JOBS programs and provides them with policy and


procedural guidance. The ACF Headquarters develops guidance for and directs the


nationwide administration of the JOBS program. It provides some direct technical


assistance to regions and States and oversees the technical assistance contract,


The ACF regional offices serve as the principal point of contact for the States. They


monitor JOBS program operations and review and approve State Plans. They also are


responsible for providing some technical assistance. The roles and responsibilities of


ACF Headquarters and regional offices are explained in greater detail in the


August 27, 1991 edition of the Federal Register (56 FR 42332 No. 166) under ACF


Functions and Delegations of Authority.


METHODOLOGY


In recognition of the differing roles and responsibilities of ACF Headquarters and


regional office components, we decided to review the JOBS program in two phases.


We did this to more accurately reflect the overall performance of each component in


providing monitoring and technical assistance for the JOBS program,


Phase one, which is completed, involved a review of ACF Headquarters. Phase two,


currently in progress, will review ACF regional offices and State agencies involved in


the JOBS program. Our review of ACF regional offices will focus on their roles and


responsibilities in monitoring the JOBS program and providing technical assistance to


States. Our review of the States will focus on their perspectives of the monitoring and


technical assistance received from ACF.


In phase one, we researched the legislative history of the JOBS program by reviewing


the Federal law, regulations, and relevant background material. In addition, we


reviewed the policies, processes, and products developed by ACF Headquarters for


monitoring and providing technical assistance in the JOBS program. This included a


review ofi manuals, transmittals, and memoranda; field review documentation;


statistical data reported by the States; analyzed data; and contract/performance


indicators for the technical assistance contractors.


We examined ACF Headquarters’ role in monitoring and providing technical


assistance to regional offices and States for the JOBS program. We conducted an


on-site visit at ACF Headquarters and interviewed members of the Agency’s staff


responsible for the following key areas: field reviews, on-site visits, State Plans, data


collection, and the technical assistance contract. We interviewed staff from the office


of the Assistant Secretary for Planning and Evaluation. We also intemiewed members


2 
-of Washington, D. C.-based advocacy organizations that have an active interest in the , 

JOBS program. These included the National Governors’ Association, the American 

Public Welfare Association, and the Center for Law and Social Policy. 

Our review was conducted in accordance with the QuuMy Stan&rd3 for In.specfions 

issued by the President’s Council on Integrity and Efficiency. 

3

FINDINGS


The complexity of the JOBS program, the traditional difficulties that accompany 

implementation of a new program, and the budget constraints that are a mainstay in 

today’s economy make administering the JOBS program a constant challenge. We 

recognize the extraordinarily difficult circumstances under which ACF Headquarters 

labored and are well aware of its success in getting the complex regulations issued 

timely and of the various forms of technical assistance and guidance which it provided 

to the States, either directly or through its regional offices. We realize that we do not 

have a complete picture of ACF activities since we have not completed phase two. 

Nevertheless, we identified weaknesses in national guidance and oversight which we 

believe merit immediate attention. We offer findings and recommendations regarding 

improvements that need to be made now that the flurry of initial implementation is 

over and the JOBS program is beginning to stabilize and mature. These weaknesses 

are in the areas of monitoring, technical assistance, program guidance, and 

management information. 

MONITORING 

The ACF Headquarters’ guidance for field reviews and data collection has been 

inadequate. 

The ACF performs the monitoring function of the States primarily through on-site 

field reviews. These are conducted by teams composed of ACF Headquarters and 

regional office personnel. Monitoring is also performed through analysis of data that 

States submit to ACF Headquarters and regional offices. 

Field reviews 

The ACF conducted field reviews of 13 States in Fiscal Year (FY) 1990 and of the 

remaining States in FY 1991. These reviews covered child care, supportive services, 

systems, and data collection as well as JOBS. In FY 1992, ACF conducted field 

reviews covering the program operations of 18 States. The principal focus of these 

reviews was to describe and evaluate significant changes in program operations and to 

obtain a sense of client flow through the JOBS program since the previous review. 

According to ACF staff responsible for field reviews, “interesting practices” identified 

in the field review process are shared with the regions and States. However, we did 

find some problems with the reviews. 

� Field review guides were not released timely to the regions. 

The final field review guides for both FY 1991 and FY 1992 were released in 

late spring, which limited the time available for the regions to conduct and 

complete required field review activities. 

4

� 

� 

An ACF work group convened in February 1992 for the purpose of designing 

the field review guide for FY 1992. A draft version of this guide was provided 

to the regions in March 1992. The final field review guide was released in 

June 1992. The ACF Headquarters stated that regions had the option of using 

the draft guide to conduct their field reviews. However, many regions deferred 

their field reviews until after the issuance of the final field review guide. 

The field review guides for FY 1993 were released in draft to the regions in 

December 1992. As of February 2, 1993, these had not been released in final. 

Field review reports were not rekz.wd time~ to the States. 

The ACF Headquarters has not established uniform national time limits for the 

preparation and release of field review reports. In FY 1991, the average 

elapsed time from the field review visit to the release of the field review reports 

was over seven months. Two field review reports were released after 17 months. 

At the time of our review in September 1992, 6 States had not received their FY 

1991 field review reports, a time lapse of more than 12 months. By February 

1993, ACF had released four of these FY 1991 reports. The two outstanding FY 

1991 field review reports were for Kansas and Nebraska. This delay adversely 

affects the usefulness of these reports to States. (Appendix A has a complete 

listing of field reviews conducted and the report status for FY’s 1990- 1991). 

At the time of our review in September 1992, no final FY 1992 field review 

reports had been released. 

The ACF Headquarters does not have readily available the dates that regional 

offices release final field review reports. The ACF Headquarters’ staff stated 

that, under the Agency’s decentralized structure, Regional Administrators are 

responsible for the final approval and release of these reports. 

Regional ofices were not directed to provh!e the States with a comment period 

prior to the release of final field review repr&. 

The ACF Headquarters has not provided guidance to the regional offices on 

releasing draft reports to the States for their review and comment. Under 

ACF’S decentralized structure, final field review reports are released by ACF 

Regional Administrators upon completion of ACF Headquarter’s internal 

review of the report content for “accuracy and sensitivity.” The ACF staff 

informed us that States have the opportunity to comment on final reports. 

However, ACF does not solicit these comments. 

5

�	 Re@mal oj@e.s were not directed to reqtie States to submit comective action 

plans. 

The ACF Headquarters has not provided guidance to the regional offices on 

State submittal of corrective action plans. The field review process has 

identified problem areas which merit States’ attention. Currently, there are no 

legal, regulatory, or Agency policies which require the States to respond to 

these problem areas. 

�	 Regional ofices were not directed to follow-up m“th States on jiekl review @dings 

and recommendah”ons. 

The ACF Headquarters does not require the regional offices to follow-up with 

States to determine if they have addressed the problems or weaknesses 

identified in the field review process. Some regional offices may conduct 

follow-up as part of their overall JOBS program oversight, but there is no 

assurance that this will be done timely or even done at all. 

� l%e regions were not directed to use the fieki review process to monitor the States 

for compliance with their State Pkms. 

Field review guides, which are the primary means of monitoring JOBS, do not 

specifically address the review of the States’ and political subdivisions’ 

compliance with State Plan provisions. Field reviews are for “fact finding” 

purposes and are not used as a monitoring tool for compliance. However, 

when the review team identified problem areas while conducting past field 

reviews, the issues were often included in the field review report. 

The ACF staff felt it was inappropriate to monitor for compliance during the 

early stages of implementation when it was more important to understand what 

was happening at the State level and ensure that instructions and guidance 

were understood. Now that this stage is over, however, we believe a more 

systematic method of compliance monitoring is needed. 

Data collection 

States are required by law, regulation, or administrative guidance (in the form of 

“action transmittals”) to submit data to ACF on various aspects of the JOBS program. 

However, there are no penalty mechanisms to enforce JOBS program reporting 

requirements. The ACF Headquarters’ staff stated that the Agency is applying “gentle 

pressure” on the States to get them to comply with all reporting requirements. The 

regional offices contact those States that are having difficulty meeting the deadlines for 

data submission. Once these States have been contacted, the regional offices offer 

assistance and guidance, if necessary, to ensure timely submission of data. The ACF 

Headquarters data branch is responsible for providing technical assistance in this area. 

6

� l’he ACF k not receiving State data bzely. 

Quarter& Pa~”cipation Rate Stathtical Repoti (FSA-103) 

Action Transmittal 90-11 provided conflicting instructions concerning the dates 

that the Quarterly Participation Rate Statistical Reports (FSA-103) are due in 

ACF Headquarters and regional offices. To compensate for conflicting due 

dates, we allowed a 30-day grace period from the specific dates mentioned in 

the action transmittal when determining the timeliness of these reports. Even 

with this allowance, over one third of the States and political subdivisions have 

not consistently submitted Quarterly Participation Rate Statistical Reports 

(FSA-103) timely in FY 1991 and FY 1992 (Appendix B). 

JOBS l?rogram Participation Data Collection Information (FSA-108J 

All States and political subdivisions are required to electronically submit the 

monthly JOBS Program Participation Data Collection Information (FSA-108). 

The first submission was due by December 15, 1991 for the sample month of 

October 1991. The Code of Federal Regulations [45 CFR 250.80(a)(l)], 

requires States and political subdivisions to electronically submit data no later 

than 45 days after the end of the sample month. Action Transmittal 91-17 

improperly extended the submission due date for the JOBS Program 

Participation Data Collection Information (FSA-108) report to 60 days. 

However, some States are not meeting either time frame for the submission of 

JOBS Program Participation Data Collection Information (FSA-108) reports 

(Chart 1). 

Chart 1 

The Number of States Not Reporting IWA-108 Data 

for October 1991 - August 1992 

Status as of November 1992 

30, 

28


26


24


22


20


18


16


14


12


10


8


6


4


10)91 ‘ “-” 12)91 “ “-- 02)92 “u’‘- 04)92 ‘-’“- 06)92 u ‘‘‘- 08}92


ReportMonth 

7

� 

November 1992 data furnished by ACF Headquarters established that of the 

50 States and the District of Columbia: 

�	 Six (Delaware, Minnesota, Missouri, Nebraska, New Hampshire, and 

Pennsylvania) had not electronically submitted any of the JOBS Program 

Participation Data Collection Information (FSA-108) to ACF 

Headquarters; 

�	 Seven (Alabama, Connecticut, Indiana, Ohio, Rhode Island, Tennessee, 

and West Virginia) were 7 to 11 months late submitting some or all of 

their reports; 

�	 Three (District of Columbia, Montana, and South Dakota) were four to 

six months late submitting some or all of their reports; 

�	 Eight (California, Georgia, Kansas, Maryland, Michigan, New Jersey, 

New York, and Oklahoma) were up to three months late submitting 

some or all of their reports; 

�	 Three (Illinois, Massachusetts, and Mississippi) had submitted data for 

all periods except the most recent report month (August 1992); and 

� The remaining 24 were current with their data reports. 

Implementation of (?uarterly Report of Ztle IV-F Bvenditures Unifonm Repoti”ng 

Requirements (ACF-332) 

According to one ACF report, 24 States were late submitting Implementation 

of Quarterly Report of Title IV-F Expenditures, Uniform Reporting 

Requirements (ACF-332) reports for the first and/or second quarter of 

FY 1992. 

ChiL.iCare Reporting Requirements (A CF-115} 

Based upon information contained in an ACF report, 33 States were late 

submitting Implementation of Child Care Reporting Requirements 

(ACF-115) reports for the first and/or second quarter of FY 1992. 

T%eACF Headqumten has not established a closure date for the submiwion of 

States’ data. 

Data for the implementation of Quarterly Participation Rate Statistical Report 

(FSA-103) is due on a quarterly basis. The data that was due for the first 

quarter of 1991 (in February 1991) was still being revised by some States in 

April 1992. The data that was due for the first quarter of 1992 (in February 

1992) was still being revised by some States in August of 1992 (Appendix B). 

8 
Data for the JOBS Program Participation Data Collection Information 

(FSA-108) is due 60 days after the sample month. Four States submitted 

revisions to the October 1991 data in August 1992, eight months after the 

report due date. 

Additional discussion of matters related to data collection are contained in the


Program Guidance and Management Information sections that appear later in this


report.


TECHNICAL ASSISTANCE


The ACF provided technical assistance and training to regional office staff through


conferences held in FY’s 1989, 1991, and 1992. Shortly after the publication of the


final regulations in the Fall of 1989, ACF held a major conference for over 700


Federal, State, and Tribal officials to provide operational and policy guidance for the


JOBS program. In addition, ACF Headquarters’ staff visited every region in the


summer of 1990 to train staff on reviewing State Plans, provided pre-printed guides of


the State Plan to the States, and issued numerous action transmittals and information


memoranda. Further, ACF Headquarters retained technical assistance contractors


specifically for the purpose of providing technical assistance and guidance to the


States.


The field reviews and States’ data were used to identify technical assistance needs.


The HHS, in conjunction with the Departments of Labor and Education, provide


technical assistance primarily through the use of contractors. In 1990, the three


Departments entered into a three year, six million dollar technical assistance contract


with the National Alliance of Business, Maximus, and Pelavin Associates to help States


and localities operate effective and efficient JOBS programs. Contract funding is


apportioned among the three. All deliverables developed by the technical assistance


contractors had to be reviewed and cleared by the three Departments before being


distributed to States.


The ACF awarded the technical assistance contract in August 1990. The contract was


written to be an “evolving contract to address States’ needs.” A November 1990


memo to ACF Regional Administrators spelled out the three tasks assigned to the


technical assistance contractors. These tasks were: assistance to program managers,


assistance to providers, and systems assessment and information dissemination. After


consultation with the Departments of Labor and Education, the first two tasks were


modified to reflect the following priority areas: program design, coordination,


marketing, and agency cultural/structural change.


9

� 

The ACF did not reap the full benefits that could have been obtained from the 

technical assistance contractors. 

As indicated above, ACF and the technical assistance contractors initiated important 

and useful training and guidance. However, the full potential of available resources 

was not realized. 

The techrkal assistance contraction did not address the fowpriody areas timely. 

The contractors were to focus on the four priority areas listed above for the 

first six to nine months of their contract. Additional areas were to be covered 

in subsequent years. As shown below, the contractors did not cover all four of 

these priority areas during the first year of their contract. The only additional 

area that has been addressed is literacy. 

One of the four priority areas was addressed in FY 1991, when the technical 

assistance contractors conducted six regional workshops on program design. 

The remaining priority areas that were to be covered in the first six to nine 

months of the contract were not addressed until FY 1992. This was 

accomplished through five regional workshops covering coordination, 

management, and agency cultural change; three regional workshops on 

marketing; and a national training conference on agency cultural change, 

�	 7%e technical &tance contraction dii not address all of the subjecti the Sates 

requested 

As noted above, the contractors were to provide technical assistance in addition 

to the four priority areas. Therefore, they queried the States in FY 1990 and 

FY 1991 to determine the States’ training needs. The States responded that 

they needed technical assistance in the areas of child care, teen parents, 

systems, and data. 

The ACF Headquarters’ Project Officer responsible for the technical assistance 

contract stated that the technical assistance contractors did act as facilitators for 

a session on child care for Federal staff. However, the contractors provided no 

training for the States on the subject. The ACF purposely elected to address 

child care rather than have the contractors do it. One means of addressing this 

was through a child care conference held in December 1992. 

With respect to teen parents, the contractors devoted one session to this topic 

during the Program and Component Design workshops held in FY 1991. 

Beyond this, ACF indicates it addressed teen parents through the field review 

process, specific policy guidance, and other means. 

10

� 

The Project Officer considered systems and data training to be a responsibility 

of ACF Systems and Data branches since Agency expertise was available. 

Therefore, the contractors did not conduct this training. 

The contractors had very little involvement in providing technical assistance for 

these areas as ACF chose to address three of the four on their own. However, 

the contractors have scheduled additional workshops and on-site forums for 

FY 1993 that relate to literacy. 

7he technical assktance materials were not diwrniiuzted t$ectively. l%is limited 

their u@hlneM to States. 

The technical assistance contractors sent a memorandum to State agencies in 

December of 1990 stating that they were working on a General Systems Design 

for the JOBS Automated System and a Data Element Dictionary. At that time, 

while all States had implemented their JOBS program, several States were 

continuing to design and implement their JOBS reporting and data systems. 

The General Systems Design and Data Element Dictionary were not published 

and distributed to all States until August 1991. 

In May of 1991, the contractors promised that a Coordination Handbook would 

be completed by the end of the year. This handbook was designed to facilitate 

the coordination between Federal, State, and local entities involved in the 

JOBS program. The Coordination Handbook was not distributed until 

November 1992. 

The technical assistance contractors have concentrated most of their efforts on 

the development of technical assistance workshops and workshop materials. 

The handouts and workbooks developed for the workshops were given to all 

attenders. Personnel from non-attending States could request copies of the 

workshop materials. However, ACF did not routinely send this material to 

non-attenders, due to the expense to the agency and because non-attenders 

may not have been able to fully understand and utilize the information without 

the workshop presentations. 

PROGRAM GUIDANCE 

Given the magnitude of the implementation requirements and the fact that States had 

the option to begin JOBS programs within the first year, ACF faced a formidable task 

in issuing the necessary guidelines to them. It did issue the implementing regulations 

within one year. The ACF also used action transmittals and information memoranda 

to communicate Agency policies, procedures, and requirements to regions and States. 

To its credit, 42 of the 53 action transmittals and information memoranda we reviewed 

were disseminated in a timely fashion. However, we did identify some weaknesses in 

this area. 

11

Some action transmittals and information memoranda were not disseminated in a


timely fashion.


Appendix C lists eleven action transmittals which were issued close to or after their


effective date. Most of these relate to systems and data reporting, critical aspects of


program implementation. The delays in issuing the transmittals gave the States and


political subdivisions very little lead time to undertake the complex task of revising


their computer systems and begin capturing data.


In addition, as we noted in the earlier section on data collection, an Action


Transmittal (AT-91-17), improperly extended the submission due date for the JOBS


Program Participation Data Collection Information (FSA-108) to 60 days. This is


contrary to the Federal Regulations which require States and political subdivisions to


electronically submit data no later than 45 days after the end of the sample month


[45 CFR 250.80(a)(l)].


MANAGEMENT INFORMATION


The ACF Headquarters has not systematically validated or analyzed information from


field reviews, State Plans, and data collection System$ nor has it routinely


disseminated management information to Congress, Federal and State managers,


policy makers, and advocacy groups.


The ACF Headquarters has worked with the States to develop several information


initiatives. However, we found weaknesses both with the data itself and with the use


being made of it.


�	 % ACF Head@atiem has not adequately validated and ana@xi information 

submitted by the States 

The ACF Headquarters utilizes several means of analyzing information 

submitted by the States. Computer edits point out coding and transmission 

errors in State data. Consistency checks are used to compare total fields with 

information contained in the data elements to ensure uniformity. Acceptable 

ranges for data are established based upon historical data parameters that the 

States previously reported. The FY 1992 field review process involved 

reviewing a sample of case files against JOBS Program Participation Data 

Collection Information (FSA-108). However, while the field review process 

does validate case file information from one or two County offices within a 

State against the data submitted by that office, the review is too limited in 

scope (less than 100 cases total at two sites) to adequately determine if States’ 

data is accurate. It should also be noted that the FY 1992 field reviews were 

conducted in less than half of the States. 

12

� % ACF Headquarter has not routidy dkwrnika ted management informatiim in 

a usejhl foimat to those who need it 

Management information has not been made routinely available to Federal and 

State managers, policy makers, the Congress, and advocacy groups. Agency 

products, such as summary statistical reports on participation, outcomes, 

characteristics of State Plans, demographic information about participants, and 

“interesting practices” are useful to Congress, HHS components, ACF 

Headquarters/regional offices, States, and advocacy groups in their research and 

planning activities. This information has not been widely disseminated within 

and outside the Department. 

13

� 

RECOMMENDATIONS


As indicated in the background and findings sections, we recognize the difficulties 

faced by ACF and its success in issuing the implementing regulations and providing 

the initial structure for monitoring the program. Nevertheless, we believe that certain 

improvements, which are the responsibility of ACF Headquarters, require immediate 

attention. 

Specifically, to address the problems identified above, ACF should 

develop adequate processes to monitor States for compliance with their State Plans 

and JOBS program requiremen~, 

review and clari.tj the purpose of existing monitoring initiatives and strengthen the 

policies and procedures guiding their administratiory 

In particulq with respect to jield reviews: 

�	 provide field review guides to the regional offices by the second quarter 

of the fiscal year to allow sufficient lead time to schedule, conduct, and 

complete these reviews; 

�	 ensure that field review reports to the States are prepared and released 

in draft for their comment within 90 days of the field review visit; 

provide guidance to the regional offices on requiring States to submit 

corrective action plans to address problem areas identified during field 

reviews; and 

�	 provide guidance to the regional offices to ensure that problem areas 

identified in the field reviews are being corrected. 

Wtih respect to data collection 

�	 continue to work closely with regional offices and States to ensure the 

accurate, timely, and complete submission of data; and 

� establish deadlines for the correction and finalization of States’ data. 

systematically reassess the specific needs of the States for technical assistance and 

provide a timely and appropriate response; 

ensure that action transmittals and information memoranda conform with Federal 

law and regulations and Ageney policieq 

14

allow 60 to 90 days lead time, whenever posslkde, for the effective dates of action 

transmittals and information memorand~ 

develop a systematic analysis plan for JOBS program information available from 

field reviews, State Plans, and data collection Systew, and 

disseminate program information and data analysis through timely reports on 

varied program elements to Federal and State managers and policy makers. 

AGENCY COMMENTS AND OIG RESPONSE 

The ACF has agreed with most of our recommendations and has outlined specific 

actions it has planned or already taken to implement them. Their one major 

reservation concerns monitoring. They believe that it would be prudent to delay a 

final decision on these findings and recommendations until the second phase of the 

inspection--Regional Offices and States’ Perspectives--has been completed. 

We continue to believe that our findings and recommendations represent concerns 

that should be addressed in Headquarter’s operation of the JOBS program. 

Nevertheless, we accept ACF’S suggestion to delay final action on monitoring until the 

second phase of our study is done. 

The ACF also expressed concerns about some of our findings and wished to 

emphasize numerous positive actions they have taken on matters addressed in our 

report. We wish to emphasize that we agree that ACF has taken aggressive action to 

implement the JOBS program. We highlighted this in the Executive Summary and in 

the introduction to our findings. We have provided some additional examples of their 

actions in this final version of the report. Space and presentation style simply do not 

allow us to enumerate everything which ACF has done. For more examples, we invite 

the reader to read ACF comments, which are provided verbatim in Appendix D. 

However, it should be noted that the purpose of this report was to point out areas 

needing further attention. Such areas for improvement inevitably arise no matter how 

intensively management attempts to deal with complex problems. 

We made other modifications to the report in response to ACF’S comments. Again, 

the full text of their comments is contained in Appendix D. 

We understand that ACF has recently established several task forces, one of which is 

to address monitoring and technical assistance. We believe that this is a positive step 

and hope that the information in this report will be useful to the task force addressing 

these functional areas. 

15

Connecticut 

Maine 

Massachusetts 

New Hampshire 

Rhode Island 

Vermont 

Delaware 

APPENDIX A


I 05/91 I 09/91 I 4 months 

I 04/91 I 08/91 I 4 months 

I 03/91 I 06/91 I 3 months 

I 03/91 I 06/91 I 3 months 

I 07/90 I 12/90 I 5 months 

I 06/91 I 10/91 I 4 months 

I 03/91 I 10/91 I 7 months 

District of Columbia I 02/91 I 10/91 I 8 months 

Maryland


Pennsylvania


Virginia


West Virginia


Alabama


Florida


Georgia


Kentucky


Mississippi


North Carolina


South Carolina


04/91 02/92 10 months 

I 09/91 I 07/92 I 10 months 

I 08/91 ! 04/92 I 8 months 

I 09/91 I 06/92 I 9 months 

04/91 07/91 3 months 

02/91 05/91 3 months 

10/90 01/91 3 months 

09/91 01/92 4 months 

06/91 09/91 3 months 

I 09/91 I 03/92 I 6 months 

04/91 07/91 3 months 

A-1 
II Tennessee I 

II Illinois 

Indiana 

Michigan 

Minnesota 

Ohio 

II 

II 

II Wisconsin I 

Arkansas 

Louisiana 

New Mexico 

Oklahoma 

Texas 

II Iowa I 

II Kansas I


Missouri


II Nebraska I


II 

Colorado


Montana


North Dakota


South Dakota


II 

II 

Utah


Wyoming


05/91 I 10/91 I 5 months 

07/91 09/92 14 months 

09/91 10/92 13 months 

09/91 09/92 12 months 

08/90 11/90 3 months 

06/91 09/92 15 months 

08/90 I 12/90 I 4 months 

03/91 05/91 2 months 

08/91 01/92 5 months 

06/91 09/91 3 months 

09/90 03/91 6 months 

09/91 12/91 3 months 

09/90 \ 04/91 I 7 months 

06/91 I Not Released* I 19 + monthsl 

08/91 01/93 17 months 

09/91 I Not Releasedl I 16 + monthsl 

08/90 01/91 5 months 

07/91 02/92 7 months 

07/91 12/91 5 months 

04/91 10/91 6 months 

03/91 10/91 7 months 

09/91 05/92 8 months 

A-2 
Arizona I 03/91 I 05/91 I 2 months II


II California I 08/90 I 11/90 I 3 months II


Hawaii 07/91 12/91 5 months


Nevada 02/91 02/91 O months


Alaska 06/91 09/92 15 months


Idaho 04/91 01/92 9 months


II Oregon I 07/91 I 07/92 I 12 months II


A-3

APPENDIX B 

1st 

Quarter 

FY 1991 25 11 15 0 4 11 6 

2nd 

Quarter 

FY 1991 37 8 6 1 1 5 1 

3rd 

Quarter 

FY 1991 30 9 12 1 2 4 0 

4th 

Quarter 

FY 1991 34 10 7 0 1 2 0 

1st 

Quarter 

FY 1992 32 9 104 0 1 4 0 

2nd 

Quarter 

FY 1992 37 5 P o 0 2 0 

3rd 

Quarter 

B-1 
APPENDIX C


AT-90-1 

AT-90-2 

AT-90-8 

AT-90-11 

AT-90-14 

AT-91-22 

AT-91-23 

AT-92-1 

AT-92-3 

CC-AT-92-1 

01/31/90 10/01/89 

02/06/90 10/01/89 

05/11/90 10/01/89 

09/07/90 10/01/s9 

10/10/90 07/01/89 

12/06/91 10/01/91 

12/26/91 10/01/91 

01/14/92 01/01/92 

02/28/92 02/28/92 

04/06/92 10/01/91 

JOBS Systems Information


Data Reporting-Implementation of


Quarterly Information Collection of JOBS


Program Participation (Form FSA-104)


Data Reporting-Implementation of Annual


Report of Target Group Expenditures Under


the JOBS Promam fForm FSA-302)


Data Reporting-Implementation of Statistical


Report on Recipients Under Public Assistance


Programs (Form FSA-3637) and Data


Reporting-Implementation of Monthly Flash


Report of Selected Program Data


(Form FSA-36451


Data Reporting-Implementation of Quarterly


Participation Rate Statistical Report


(Form FSA-103)


JOBS Training Program Systems Information


Data Reporting-Coding of Missing Data on


the JOBS Program Participation Data


Collection Form (Form FSA-108)


Data Reporting-Implementation of Quarterly


Report of Title IV-F Expenditures, Uniform


Reporting Requirements (Form ACF-332)


Data Reporting-Implementation of Uniform


Data Collection and Reporting Requirements


for Tribal Grantees, Tribal Jobs Program


Quarterly Report (Form ACF-114)


Pre-printed Guide for the JOBS State Plans:


FY 1992 Biennial Update


Data Reporting-Implementation of Child Care


Reporting Requirements (Form ACF-115)


c-1 
APPENDIX D 

ADMINISTRATION FOR CHILDREN AND 

FAMILIESCOMMENTS 

D-1 
.

——. .=29- - .. .._._....— -—

TO: 

FROM :


SUBJECT :


Thank YOU


We.believe 

ADMINISTRATION FOR CHILDREN AND FAtilLII 

Offlceof the Assistant Secreta~, Suite 600 

370 L’Enfant Promenade, S.W. 

June 28, 1993 Washington, D.C. 20447 

Bryan B. Mitcheu 

Pr~nci,pal D pu Insne r General 

Laurence . e 

Acting Ass<st nt Se retary for 

Child’ #+ n and Families 

Response to OIG Draft Report: ‘JOBS Monitoring and


Technical Assistance: Headquartersi Role,”


OE1-07-92-00380


for the opportunity to”coxmnenton this draft renort. 

the repo~t is an ‘tiprovementover the working ;ersion . 

“in providing a more balanced description of the Administration 

for Children and Families’ (ACF) monitoring tid technical 

assistance efforts to states for the Job Opportunities and Basic ..+ Skills Training (JOBS) program. 

However, we still have a major concern with the OIG’S methodology


of separately reviewing Central Office (CO) and Regional Office


...(RO)
administration of JOBS. AS th”ereport itself acknowledges, 

.,~.~eOIG does not have a complete picture of,ACl? activities s.i.nce 

:..::;t”has riot Completed phasetwo of thestudy. Yet many:of the 

:reCo~endationS--Darticularly those relative to monitoring--if 

‘adopted would sigfiificantly affect ItO operations. We believe 

that the OIG would be in a better position to dete~ine if such 

‘recorrunendations (and/or others) are necessary once it completes 

phase two of the study. 

Our response is organized by the sections of the OIG report.


‘MOliITORING ,


As stated above, we believe that the section on monitoring-­


specifically on the subject of the field review process--is the 

most troublesome for two reasons which we elaborate on below. 

First, the report does not give sufficient recognition to ACF 

Policy on the respective roles of CO and RO’S in conducting 

program reviews. Second, it seems to assume that if CO did not 

issUe a written directive on how to carry out a review, RO’S may 

not have conducted reviews in a professional manner. 

The specific approach to monitoring JOBS and the respective roleS 

of CO and the ROlS were established by the Assistant Secretary as 

follows: 

D-2

. 

Page 2 - Mr. Bryan B. Mitchell 

all States would be reviewed during FY 1990 and FY 

1991; 

field reviews would be a broad look at management,


client flow, financial operations, systems, and child


care;


field reviews at this early point in implementation


would be geared to gaining information and identifying


potential problems;


ACF would be explicit with States that the reviews were


not compliance reviews; and


Regional Offices would”have lead responsibility for


conducting the reviews, but that CO would review draft


reports narrowly for “accuracy and sensitivity” given


the high profile of the JOBS program.


Lead responsibility for conducting reviews means that the RO


establishes the schedule for the review, makes all arrangements,


writes the report, issues it (after CO review for accuracy and


sensitivity) , receives comments, and follows up as appropriate.


...As a result it i.s highly likely that the OIG will. find a range of 

..,procedures in the Regional offices when itconducts.that.part of 

the review. Such variations may reflect the experiences of the 

..Regional Administrator, the resources available to the Region, or 

the nature of the revi,ew that was conducted. That i.s different 

from the implication in the draft report that AC??’S lack of a 

national policy and written procedures means that no opportunity 

for comment i,s provided or that Regions do not follow-up on the 

results of field reviews. We believe that such findings and 

recommendations should only be made after the IG conducts its 

review of RO operations. 

we do agree with the OIG that there were delays in i,ssuing some 

of the FY 1991 reports. We believe much of the delay was 

attributable to the comprehensive nature of the reviews that were ~ 

conducted and that narrower reviews conducted in subsequent years 

addressed the problem of timeliness. Nevertheless, this is an 

area that”we will look at more carefully. 

TE~IcAL ASSISTM?CE 

In general, the report suggests that the Technical Assistance 

(TA) contract was the only vehicle by which ACF provided TA to 

State JOBS programs. The JOBS TA contract has been the primary

vehicle by which co has provided TA directly to states, but nOt 

the only one. For example, shortly after the publication of the 

final regulations, the Office of Family Assistance held a major 

D-3

Page 3 - Mr. Bryan B. Mitchell 

conference for over 700 Federal, State and Tribal officials to


provide operation and policy guidance for the JOBS program.


However, the RO’S also have major responsibility for providing


ongoing TA to States. They have conducted meetings with their


respective States and provided operational guidance to States on


a regular basis.


The report states on page nine that in less than one month the


scope of work of the contract changed. This is misleading. The


November 1990 memo to the Regional Administrators indicated that


HHS was working with the Departments of Labor and Education,to


establish priority areas under Tasks 1 and 2 (assistance to


program managers, assistance to providers). Once the priority


areas were identified and amounted in the December memo to the


Regional Administrators, we no,longer used the terms Tasks 1


and 2. We focussed our assistance under the priority areas and


targeted first year efforts to program managers.


The report indicates on page 10 that only one priority area was


addressed in FY 1991 through the conduct of six regional


workshops on program design. This is inaccurate. The regional


prograxndesign workshops (attended by 49 States and 4


jurisdictions) also emphasized interagency coordination


strategies including State discussion forums on coordination. In


~..addition,the contractor assisted the Department of Education in


..designing and facilitating various sessions at a national


conference for all so States to promote coordination among adult


and vocational education, JOBS and JTPA programs. In.addition,


the contractor completed Task 3--prepared and distributed a


General Systems Design and Data Element Dictionary to States and


conducted various on-site State visits.


In addition to the FY 1992 and FY 1993 deliverables listed in the


report, the contractor also developed an agency


cul,tural/structural change training curriculum for IV-A/IV-D/


IV-F workers, provided on-site assistance to four States, and


developed and distributed to 43 States and jurisdictions copies


(in final draft) of Volumes I and II of a coordination handbook
 . . ‘Meeting the Challenges: Coordination to Promote SelfSufficiency. n -... 

The report is correct in saying that the contractor did not


address all of the subjects requested by States. We believe that


it is entirely appropriate for ACF (in conjunction with its


partners- -the-De&t~ents of Education and-Labor) to consider


requests for technical assistance and determine the best way to “


meet those needs. That might be the contract or it might be


internal technical assistance. We believe that as to the 

subjects identified by the States--child care, teen parents, 

systems, and data--such decisions were made. 

D-4

. 

Page 4 - Mr. Bryan B. Mitchell


For example, ACF determined that technical assistance relative to


data collection for JOBS could best be delivered by ACF.staff who


had developed the reporting requiraents. Therefore, CO staff


res~onsible for JOBS data collection made presentations at the


national JOBS conference and numerous regional conferences and


has provided individual assistance to Regional Offices and States


on an on-going basis.


In the area of systems, the contractor did develop a General


Systems Design and Data Element Dictionary under Task 3.


As to teen parents, it is inaccurate to say that the contractor


did not provide assistance. A session specifically devoted to


teen parent programs was included at all the first year’s RO


program design workshops. In addition, ACF has continued to


address teen parent issues through the field review process,


specific policy guidance and identification of promising practice


models.


As to child care, we purposely elected to have child care


addressed by ACF staff rather than the contractor. This was due


to the fact that two new major child care programs were passed by


Congress in November 1990- -At Risk Child Care and the Child Care 

:,Development Block Grant-- and we believed that policy issues 

relative to child care were the predominant concern of States and 

that ACF staff were in a better position to address those 

concerns. ACF published final regulations for these.progr- in 

August 1992 and held a child care conference in December 1992.


On the distribution of products developed for workshops, we 

believe that there has been broad distribution of products 

although we acknowledge that some of the products should have 

been released more timely. Kll States and territories (except 1) 

attended the program design workshops in FY 1991 and received


many products that the contractor had developed; forty-three


States/territories attended regional workshops beginning in -Y


1992 where they received the final draft of Volumes I and II Of


the Coordination Handbook. The final version of the Coordination -


Handbook was then distributed to all 54 States/territories in ..


November 1992. The General Systems Design and Data Element


Dictionary was distributed to all States in August 1991.


PROGWM GUIDANCE 

The ACF acknowledges that some action transmittals and


information memoranda were not issued prior to their effective


date or were issued without much lead time. However, we believe


that in most instances the timing was a result of the short lead


time for implementation of the JOBS program which the OIG


acknowledges in its Executive S~rY.


D-5

Page 5 - Mr. Bryan B. Mitchell 

For example, the provisions governing enhanced “FFP were effective


October 1, 1989 (prior to publication of the final rules) so ACF


had to collect information back to that date even though it would


have been impossible to issue data collection reporting forms


prior to the publication of the final rules. ~us although the


forms were at OMB before October 13, 1989, they were not actually


approved and issued until early in the second quarter of


FY 1990--less than four months after the final rules were


published. It should be noted that while the forms were late for


the States that elected to start early, for the rest of the


States the reporting forms were issued prior to their


implementation of the JOBS program.


As to the eleven specific Action”TransKd-ttals (AT) and


Information Memoranda (IM) cited by the OIG, we believe that it


is misleading to characterize some of them as ‘delayed


instmctionson AT’s 89-41 and 90-14 do not contain instructions.


They provide information about systems development and provide 

guidance on FFP rates for systeMS. The effective dates reflect 

the earliest date for which a State could claim FFP for Systems 

development, but do not impose any requirement on States to 

retroactively report data. In the case of AT-92-3 on the FY 1992 

Biennial update, while the effective date was also the date it 

was issued (2/28/92), States were not required to submit their 

plans until July 1, 1992. 

ACF RESPONSE To SPECIFIC RECOMMENDATIONS 

OIG Recommendation


ACF should develop adequate processes to monitor States for


compliance with their State Plans and JOBS program requirements.


ACF Res~onse


Each year ACF determines its strategy for conducting JOBS reviews


considering the priorities of the Assistant Secretary and the


resources available to Central Office and the Regional Offices. “-
 AS ACF prepares its FY 1994 strategy, we will carefully consider. .


the OIG’S recommendation.


OIG Recommendation


ACF should review and clarify the purpose of existing monitoring


initiatives and strengthen the policies and procedures guiding “


their administration.


D-6

. 

Page 6 - Mr. Bqan B. Mitchell 

ACF Resuonse


We agree with the importance to provide guides early enough to


plan reviews in quarters 2, 3 and 4. We provided the ~ 1993


guide to the regions on February 2, 1993 for that reason.


As to the specific recommendations about issuing written 

directives to ROIS on how to conduct field reviews, we believe 

that it would be prudent to wait for the findings of the second 

phase of this review to determine whether such guidance is 

required. 

OIG Recormnendation


ACF should work closely with regional offices and the States to


ensure the accurate, timely, and complete submission of data.


ACF Resnonse


ACF cannot find anything in the findings section of the OIG


report to suggest that it has not worked closely with RO’S and


States to improve data collection. We believe that the OIG will


find when it conducts phase two of this investigation that CO has


worked very closely with ROIS and States. We will continue to do


so.


We note that one of our efforts to work closely with the States


to ensure better data--extending the deadline for submission of


the FSA-108 from 4S days to 60 days--is criticized by the OIG in


this report. Discussions with States had indicated that 45 days


was not enough time to submit the required data. Therefore, we


believed that it was a reasonable accommodation to give the


States an additional 15 days to submit even if this was


inconsistent with the Federal regulations.


OIG Recommendation


ACF should egta.bli.sh deadlines for the correction and . . finalization of States’ data. 

-. 

ACF Res~onse


ACF agrees that JOBS data collection has been in place 10ng


enough now to establish such deadlines. We will consult with


RO’S and States on establishing reasonable deadlines and will


give States sufficient notice of the deadlines.


OIG Recommendation


ACF should systematically reassess the specific needs of the 

States for technical assistance and provide a timely and 

appropriate response. 

D-7

. 

. 

Page 7 - Mr. Bryan B. Mitchell”


ACF Resnonse


ACF will continue to assess the specific needs of States for


technical assistance and provide as timely response as feasible.


ACF will use various vehicles to identify State needs including


field reviews, State surveys by contractor and ROIS, TA request


sheets at conferences and specific State requests either to ACF


or the contractor. AS a result of our ongoing efforts, we have a


number of initiatives underway and planned for FY 1993 (in and


outside of the TA contract) including a National JOBS Directors


Conference, various on-site individual State visits, special


topic informational packets and workshops and a databank


directory of proxni.si,ng iJOBS and child care State/local practices.


OIG Recommendation


ACF should ensure that action transmittals and information 

memoranda conform with Federal law and regulations and Agency 

policies. 

ACF Resnonse


ACF agrees. We believe that our decision to give States an 

additional 15 days beyond what is specified in the regulations 

to submit data was a reasonable response-to the problems States 

were having meeting the 45-day deadline. 

PIG Recommendation


ACF should allow 60 to 90 days lead time, whenever possible, for 

the effective dates of action transmittals and information 

memoranda. 

ACF agrees. whenever possible, we will allow 60 to 90 days lead 

time for the effective dates of action transmittals and . . information memoranda. 

-.. 

OIG Recommendation 

ACF should develop a systematic analysis plan for JOBS program


information available from field reviews, State plans, — and data


collection systems.


ACF Resnonse


ACF agrees. We believe that JOBS has reached a level of maturity


where such analysis will be useful.


D-8

Page 8 - Mr. Bryan B. Mitchell 

OIG Recommendation


ACF Should disseminate program information and data analysis 

through timely reports on varied program elements to Federal and 

State managers and policy makers. 

ACF ResDonse 

ACF has provided large amounts of information to a wide audience.


However, we agree that management information has not been


disseminated in a systematic manner. With the maturity of the


JOBS program, we agree that it is now appropriate to do so and we


are developing some reports that would accomplish that goal.


Zechnical comments


Page ii:	 The second sentence in the third paragraph under


MONITORING minimizes the dimensions of the early


reviews. It also incorrectly makes staff rather than


management responsible for defining the ACF strategy


(the same point applies to the second paragraph from


the bottom of page 6). We offer this as a replacement


paragraph:


‘ACF believed it was premature to conduct


compliance reviews during the early stages of


implementation when it was more important to


conduct comprehensive reviews to provide for a


baseline understanding of how the program wer@


operating. Reviews in FYs 1990 and 1991 examined


supportive semices, child care, data collection


and systems as well as JOBS.”


Page 4:	 In the first paragraph under Field Reviews, we suggest


that the second sentence be revised in this way:


In FYs 1990 and 1991, reviews were comprehensive


in nature, covering child care, supportive
 ..
 senices, systems, and data collection as well as


JOBS . -.


Page 5:	 The fourth sentence in the first paragraph hypothesizes


that RO’S may have been reluctant to use the draft


guide and in the next sentence says that regions


‘deferred” their field reviews until issuance of the


final guide. However, it is not clear how the OIG “


knows this since it has not interviewed RO’S yet.


Indeed in the letter issuing the guidance to ROIS on


March 30, 1992, the Director of the Office of Family


Assistance specifically tells ROIS to use the guide and


asks for a schedule of planned reviews. We suggest


that the fourth and fifth sentences be deleted.


D-9 
,. 

. 

page 9 - m. Bqan B. Mitchell 

page 6: The next to last paragraph minind.zes the importance and 

the level of detail of the n 1991/1992 ‘eviewsf~~ what


to simply get a feel


saying that they were “...


was happening.. .” We recomend referring to the


breadth and depth of the reviews. Further, wh~l~ the


reviews were not designed specifically for compliance


purposes, instmctions were that whenever coqliance


issues were identified, they were to be addressed.


. . 

-.. . 

D-10