Department  of Health
and  Human  se~rPc
  -._  - - ------ --- . 
.Ww 
OFFICE  OF 
INSPECTOR  GENERAL 
JOBS  MONITORING  AND 
TECHNICAL  ASSISTANCE: 
HEADQUARTERS  ROLE 
@  SERVICE$., 
AUGUST  1993
 OFFICE OF INSPECTOR GENERAL 
The mission of the Office of Inspector General (OIG), as mandated by Public Law 95-
452, as amended, is to protect the integrity of the Department  of Health  and Human 
Services (HHS) programs as well as the health and welfare of beneficiaries served 
by those programs.  This statutory mission is carried out through a nationwide 
network of audits, investigations, and inspections conducted by three OIG operating 
components:  the Office of Audit Services, the Office of Investigations, and the 
Office of Evaluation and Inspections.  The OIG also informs the Secretary of HHS of 
program and management problems and recommends courses to correct them. 
OFFICE OF AUDIT  SERVICES 
The OIGS Office of Audit Semites (OAS) provides all auditing services for HHS, 
either by conducting audits with its own audit resources or by overseeing audit work 
done by others.  Audits examine the performance of HHS programs and/or its grantees 
and contractors in carrying out their respective responsibilities and are intended 
to provide independent  assessments of HHS programs and operations  in order to reduce 
waste, abuse, and mismanagement and to promote economy and efficiency throughout the 
Department. 
OFFICE OF INVESTIGATIONS 
The OIGS Office of Investigations (01) conducts criminal, civil, and

administrative investigations of allegations of wrongdoing in HHS programs or to HHS

beneficiaries and of unjust enrichment by providers.  The investigative efforts of

01 lead to criminal convictions, administrative sanctions, or civil money

penalties.  The 01 also oversees State Medicaid fraud control units which

investigate and prosecute fraud and patient abuse in the Medicaid program.

OFFICE OF EVALUATION  AND  INSPECIIONS 
The OIGS Office of Evaluation and Inspections (OEI) conducts short-term management

and program evaluations (called inspections) that focus on issues of concern to the

Department,  the Congress, and the public.  The findings and recommendations

contained in these inspection reports generate rapid, accurate, and up-to-date

information on the efficiency, vulnerability, and effectiveness of departmental

programs.  This report was prepared in the Kansas City regional office under the

direction of James H. Wolf, Regional Inspector General.  Project staffi

REGION
 HEADQUARTERS 
Hugh Owens, Project Leader
 Ruth Folchman 
Perry A. Seaton
 Mark Krushat 
Hope Hartshorn

For additional copies of this repo~
 please contact the Kansas City Re@onal Office 
at (816) 426-5959.
 Department  of Health  and Human  Services 
OFFICE  OF 
NSPECTOR  GENERAL 
JOBS  MONITORING  AND 
TECHNICAL  ASSISTANCE: 
HEADQUARTERS  ROLE 
AUGUST  1993  OEI-07-92-00380  EXECUTIVE  SUMMARY 
PURPOSE 
To  evaluate  the  policies  and  procedures  established  by the  Administration  for 
Children  and  Families  (ACF)  Headquarters  for monitoring  and  providing  technical 
assistance  to States  for -the Job  Opp&tunities  and  Basic  S~lls  T;aining  (JOBS) 
program. 
BACKGROUND 
The  Family  Support  Act  of1988,  Public  Law  100-485, created  the  JOBS  program  for 
recipients  of Aid  to Families  with Dependent  Children  (AFDC).  It establishes  a 
. . 
comprehensive  framework  and  progr~mmatic  structure  for Federal  and  State  efforts  to

help  welfare  recipients  become  self sufficient.  Included  are  provisions  related  to job

skills training,  education,  job  readiness  activities,  job  placement/job  development,  child

care,  transportation,  and  other  supportive  services.

The  ACF  monitors  States  JOBS  programs  and  provides  them  with  policy and

procedural  guidance.  The  ACF  Headquarters  develops  guidance  and  provides  some

technical  assistance  for regions  and  States.  It also directs  and  oversees  the  technical

assistance  contract.  The  regional  offices serve  as the  principal  point  of contact  for the

States.  In this role,  they  monitor  JOBS  program  operations,  review  and  approve  State

Plans,  and  provide  technical  assistance  to States.

METHODOLOGY

In recognition  of the  differing  roles  and  responsibilities  of ACF  Headquarters  and

regional  office  components,  we are  reviewing  ACF  oversight  of the  JOBS  program  in

two phases.  Phase  one,  which is completed,  involved  a review  of ACF  Headquarters.

Phase  two, currently  in progress,  will review  ACF  regional  offices  and  State  agencies.

In phase  one,  we researched  the  legislative  history  of the  JOBS  program  by reviewing

the  Federal  law and  regulations  and  relevant  background  material.  In addition,  we

reviewed  the  written  policies,  processes,  and  products  developed  by ACF

Headquarters  for monitoring  and  providing  technical  assistance  in the  JOBS  program.

We  conducted  an  on-site  visit at ACF  Headquarters  and  interviewed  members  of the

Agencys staff  responsible  for the  following  key areas:  field  reviews,  on-site  visits, State

Plans,  data  collection,  and  the  technical  assistance  contract.  We  also interviewed

members  of Washington,  D.C.-based  advocacy  organizations  that  have  an active

interest  in the  JOBS  program.

i  The  ACF  and  the  States  faced  a number  of challenges  in implementing  the  JOBS 
program.This  program  requires  active  coordination  betieen  the  Department  of 
Health  and  Human  Services  (HHS),  the  Department  of Education,  and  the 
Department  of Labor.  The  States  had  varying  levels  of experience  in administering  a 
job  training  program  for AFDC  recipients.  In addition,  ACF  and  the  States  had 
limited  resources  and  funding  available.  Furthermore,  the  program  went  into  effect 
less than  one  year  after  enactment,  leaving  ACF  and  the  States  very  little  lead  time 
for implementation. 
Despite  these  challenges,  ACF  released  the  final regulations  within  the  one  year  time 
period  mandated  by Congress.  In addl~tlon,  it provided  guidance  and  technical
assistance  to the  States. 
We  recognize  these  efforts  and  the  extraordinarily  difficult  circumstances  under  which 
ACFlabored.  Wealsorealize  thatwedo  nothave  acomplete  picture  of ACF 
activities  since we have  not  completed  phase  two of the  study.  Nevertheless,  we 
identified  some  weaknesses  which we believe  merit  immediate  attention.  We  offer 
findings  and  recommendations  regarding  improvements  that  need  to be  made  now 
that  the  flurry  of initial  implementation  is over  and  the  JOBS  program  is beginning  to 
stabilize  and  mature.  These  are  in the  areas  of monitoring,  technical  assistance, 
program  guidance,  and  management  information. 
MONRORING 
The  ACF  Headquarters  guidance  for field  reviews  and  data  collection  has  been 
inadequate. 
The  ACF  conducted  field  reviews  to obtain  information  on States  activities  during  the 
first  two years  of program  implementation.  However,  we found  that  review  guides 
were  not  released  timely  to the  regions  and  field  review  reports  were  not  released 
timely  to the  States.  The  ACF  Headquarters  has  not  issued  guidance  to the  regional 
offices  to provide  the  States  with a comment  period  prior  to the  release  of final  field 
review  reports;  to require  States  to submit  corrective  action  plans;  or to require 
regional  offices  to follow-up  with States  on field  review  findings  and  recommendations. 
Reviews  in FYs  1990 and  1991 examined  supportive  services,  child  care,  data 
collection  and  systems  as well as JOBS.  However,  ACF  has not  used  the  field  review 
process  to monitor  the  States  for compliance  with  their  State  Plans.  It felt  that  it was 
inappropriate  to do so during  the  early  stages  of implementation  when  it was more 
important  to simply get  a feel  for what  was happening  at the  State  level  and  ensure 
that  instructions  and  guidance  were  understood.  Now that  this stage  is over,  however, 
we believe  that  a more  systematic  method  of compliance  monitoring  is needed. 
ii 
..  With  respect  to data  collection,  we found  that  ACF  has  not  established  a closure  date 
for the  submission  of States  data  and  is not  receiving  State  data  timely. 
TECHNICAL  ASSIST~CE 
The  ACF  did not  reap  the  full benefits  that  could  have  been  obtained  from  the 
technical  assistance  contractors. 
The  ACF  chose  to use  contractors  to provide  technical  assistance  to the  States.  While 
these  contractors  did provide  valuable  training  and  assistance,  they  did not  address  all 
of the  subjects  requested  by the  States,  and  some  of the  materials  were  not 
disseminated  timely  or to all the  appropriate  audiences. 
PROGRAM  GULZMNCE 
Some  action  transmittals  and  information  memoranda  were  not  disseminated  in a 
timely  fashion. 
JW4.N.AGEMENTINFORhL4TION 
The  ACF  Headquarters  has not  systematically  validated  or analyzed  information  from

field  reviews,  State  Plans,  and  data  collection  systems;  nor  has it routinely

disseminated  management  information  to Congress,  Federal  and  State  managers,

policy makers,  and  advocacy  groups.

RECOMMENDATIONS

Specifically,  to address  the  problems  identified  above,  ACF  should:

develop  adequate  processes  to monitor  States  for compliance  with  their  State 
Plans  and  JOBS  program  requiremen~, 
review  and  clar@  the  purpose  of existing  monitoring  initiatives  and  strengthen 
the  policies  and  procedures  guiding  their  administratio~ 
In partiadq  with respect to field reviews: 
. provide  field  review  guides  to the  regional  offices by the  second  quarter 
of the  fiscal year  to allow sufficient  lead  time  to schedule,  conduct,  and 
complete  these  reviews; 
ensure  that  field  review  reports  to the  States  are  prepared  and  released 
	
in draft  for their  comment  within  90 days of the  field  review  visit; 
... 
111
 . provide  guidance  to the  regional  offices  on requiring  States  to submit 
corrective  action  plans  to address  problem  areas  identified  during  field 
reviews;  and 
provide  guidance  to the  regional  offices  to ensure  that  problem  areas 
 
identified  in the  field  reviews  are  being  corrected. 
Wtih respect to data collection: 
continue  to work  closely with  regional  offices  and  the  States  to ensure 
	
the  accurate,  timely,  and  complete  submission  of data;  and 
establish  deadlines  for  the  correction  and  finalization  of States  data. 
 
systematically  reassess  the  specific  needs  of States  for technical  assistance  and 
provide  a timely  and  appropriate  response; 
ensure  that  action  transmittals  and  information  memoranda  conform  with 
Federal  law and  regulations  and  Agency  policie~ 
allow 60 to  90 days lead  time,  whenever  possl%le, for the  effective  dates  of 
action  transmittals  and  information  memorand~, 
develop  a systematic  analysis  plan  for JOBS  program  information  available 
from  field  reviews,  State  Plans,  and  data  collection  system  and 
disseminate  program  information  and  data  analysis  through  timely  reports  on 
varied  program  elements  to  Federal  and  State  managers  and  policy makers. 
AGENCY  COMMENTS  AND  OIG  RESPONSE 
The  ACF  has  agreed  with  most  of our  recommendations  and  has  outlined  specific 
actions  it has  planned  or already  taken  to implement  them.  Their  one  major 
resonation  concerns  monitoring.  They  believe  that  it would  be  prudent  to delay  a 
final  decision  on these  findings  and  recommendations  until  the  second  phase  of the 
inspection--Regional  Offices  and  States  Perspectives--has  been  completed. 
We  continue  to believe  that  our  findings  and  recommendations  represent  concerns 
that  should  be  addressed  in Headquarters  operation  of the  JOBS  program. 
Nevertheless,  we accept  ACFS suggestion  to delay  final  action  on monitoring  until  the 
second  phase  of our  study is done. 
The  ACF  also expressed  concerns  about  some  of our  findings  and  wished  to 
emphasize  numerous  positive  actions  they have  taken  on matters  addressed  in our 
report.  We wish to emphasize  that  we agree  that  ACF  has taken  aggressive  action  to 
implement  the  JOBS  program.  We  highlighted  this in the  Executive  Summary  and  in 
iv  the  introduction  to  our  findings.  We  have  provided  some  additional  examples  of their 
actions  in this  final version  of the  report.  Space  and  presentation  style simply  do not 
allow us to enumerate  everything  which  ACF  has done.  For  more  examples,  we invite 
the  reader  to read  ACF  comments,  which  are  provided  verbatim  in Appendix  D. 
However,  it should  be  noted  that  the  purpose  of this report  was to point  out  areas 
needing  further  attention.  Such  areas  for improvement  inevitably  arise  no matter  how 
intensively  management  attempts  to deal  with  complex  problems. 
We  made  other  modifications  to the  report  in response  to ACFS comments.  Again, 
the  full text  of their  comments  is contained  in Appendix  D. 
We  understand  that  ACF  has recently  established  several  task  forces,  one  of which  is 
to address  monitoring  and  technical  assistance.  We believe  that  this is a positive  step 
and  hope  that  the  information  in this report  will be  useful  to the  task  force  addressing 
these  functional  areas. 
v  TABLE  OF  CONTENTS 
PAGE 
EXECUTIVESUMMARY

INTRODUCTION  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  1

FINDINGS  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...4

Monitoring  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...4

 
Technical  Assistance  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...9

 
Program  Guidance  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  11

 
Management  Information  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  12

 
RECOMMENDATIONS  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  14

APPENDICES

&Fiscal  Years  1990 -1991  Field  Reviews  . . . . . . . . . . . . . . . . . . . . . . . . . . ..A-l 
B:	 Submission  of Quarterly  Participation  Rate  Statistical  Report 
(FSA-103)  For FYs 1991 - 1992 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ..  B-l 
CDelayed  Action  Transmittals  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ..C-1

D:  Agency  Comments  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ..D-l  INTRODUCTION 
PURPOSE 
To  evaluate  the  policies  and  procedures  established  by the  Administration  for 
Children  and  Families  (ACF)  Headquarters  for monitoring  and  providing  technical 
assistance  to States  for the  Job  Opportunities  and  Basic Skills Training  (JOBS) 
program, 
BACKGROUND 
l%eJOBSRqyam 
The  Family  Support  Act  of1988,  Public  Law  100-485, created  the  JOBS  program  for

recipients  of Aid  to Families  with  Dependent  Children  (AFDC).  Its purpose  is to

promote  the  self sufficiency  of welfare  recipients  through  a broad  range  of

employment,  training, education,  and  support  services.  The  States  must  offer job  skills

training,  education,  job  readiness  activities,  job  placement/job  development,  child  care,

transportation,  and  other  supportive  services.  They  must  also offer  at least  two of the

following  optional  services:  job  search,  on-the-job  training,  work  supplementation,  or

community  work  experience.  Educational  setices  include  high  school  or equivalent

education,  basic  and  remedial  education  to achieve  a basic  literacy  level,  and

education  for individuals  with  limited  English  proficiency.

The Challqp  of Program Implementation 
The  JOBS  program  is markedly  different  from  its predecessors.  The  ACF  and  the 
States  faced  a number  of challenges  in implementing  it. 
The  program,  administered  by ACF,  requires  active  coordination  among  the 
Department  of Health  and  Human  Services  (HHS),  the  Department  of Education, 
and  the  Department  of Labor. 
The  States  had  varying  levels  of experience  in administering  a job  training  program  for 
AFDC  recipients.  Some  of the  States  had  existing job  training  programs  which  they 
had  to convert  to meet  the  JOBS  program  requirements,  which  included  education  as 
a component.  Others  had  little  or no recent  experience  in administering  an  education 
and job  training  program  for welfare  recipients.  Some  States  were  required  to create 
new systems  or to re-design  their  existing  systems  during  the  period  when  systems  and 
data  collection  rules  and  policies  were  being  formulated.  Overall,  ACF  and  the  States 
had  limited  resources  and  funding  available. 
Under  the  legislation,  States  could  implement  their  program  as early  as July  1, 1989 or 
as late  as October  1, 1990.  Fifteen  States  elected  to start  on July  1, 1989, prior  to the 
issuance  of the  final  regulations.  Even  those  who chose  to wait  had  at most  two years 
1  after  enactment  of the  law to implement  their  programs.  Thus,  ACF  and  the  States 
had  very little  lead  time.  Despite  these  challenges,  ACF  released  the  final  JOBS 
regulations  within  the  one  year  time  period  mandated  by Congress. 
ACF  Head@atiem  and Re@nal  Ojjice  OvernightReqxmsibilitie.s 
The  ACF  monitors  States  JOBS  programs  and  provides  them  with  policy and

procedural  guidance.  The  ACF  Headquarters  develops  guidance  for and  directs  the

nationwide  administration  of the  JOBS  program.  It provides  some  direct  technical

assistance  to regions  and  States  and  oversees  the  technical  assistance  contract,

The  ACF  regional  offices  serve  as the  principal  point  of contact  for  the  States.  They

monitor  JOBS  program  operations  and  review  and  approve  State  Plans.  They  also  are

responsible  for providing  some  technical  assistance.  The  roles  and  responsibilities  of

ACF  Headquarters  and  regional  offices  are  explained  in greater  detail  in the

August  27, 1991 edition  of the  Federal  Register  (56 FR  42332 No.  166) under  ACF

Functions  and  Delegations  of Authority.

METHODOLOGY

In recognition  of the  differing  roles  and  responsibilities  of ACF  Headquarters  and

regional  office  components,  we decided  to review  the  JOBS  program  in two phases.

We  did this to more  accurately  reflect  the  overall  performance  of each  component  in

providing  monitoring  and  technical  assistance  for the  JOBS  program,

Phase  one, which  is completed,  involved  a review  of ACF  Headquarters.  Phase  two,

currently  in progress,  will review  ACF  regional  offices  and  State  agencies  involved  in

the  JOBS  program.  Our  review  of ACF  regional  offices will focus  on their  roles  and

responsibilities  in monitoring  the  JOBS  program  and  providing  technical  assistance  to

States.  Our  review  of the  States  will focus  on their  perspectives  of the  monitoring  and

technical  assistance  received  from  ACF.

In phase  one,  we researched  the  legislative  history  of the  JOBS  program  by reviewing

the  Federal  law, regulations,  and  relevant  background  material.  In addition,  we

reviewed  the  policies,  processes,  and  products  developed  by ACF  Headquarters  for

monitoring  and  providing  technical  assistance  in the  JOBS  program.  This included  a

review  ofi manuals,  transmittals,  and  memoranda;  field review  documentation;

statistical  data  reported  by the  States;  analyzed  data;  and  contract/performance

indicators  for the  technical  assistance  contractors.

We  examined  ACF  Headquarters  role  in monitoring  and  providing  technical

assistance  to regional  offices  and  States  for the  JOBS  program.  We  conducted  an

on-site  visit at ACF  Headquarters  and  interviewed  members  of the  Agencys  staff

responsible  for the  following  key areas:  field  reviews,  on-site  visits, State  Plans,  data

collection,  and  the  technical  assistance  contract.  We interviewed  staff  from  the  office

of the  Assistant  Secretary  for Planning  and  Evaluation.  We  also intemiewed  members

2  -of Washington,  D. C.-based  advocacy  organizations  that  have  an  active  interest  in the  , 
JOBS  program.  These  included  the  National  Governors  Association,  the  American 
Public  Welfare  Association,  and  the  Center  for Law and  Social  Policy. 
Our  review  was conducted  in accordance  with  the  QuuMy Stan&rd3 for In.specfions 
issued  by the  Presidents  Council  on Integrity  and  Efficiency. 
3
 FINDINGS

The  complexity  of the  JOBS  program,  the  traditional  difficulties  that  accompany 
implementation  of a new program,  and  the  budget  constraints  that  are  a mainstay  in 
todays  economy  make  administering  the  JOBS  program  a constant  challenge.  We 
recognize  the  extraordinarily  difficult  circumstances  under  which ACF  Headquarters 
labored  and  are  well aware  of its success  in getting  the  complex  regulations  issued 
timely  and  of the  various  forms  of technical  assistance  and  guidance  which  it provided 
to the  States,  either  directly  or through  its regional  offices.  We  realize  that  we do not 
have  a complete  picture  of ACF  activities  since we have  not  completed  phase  two. 
Nevertheless,  we identified  weaknesses  in national  guidance  and  oversight  which  we 
believe  merit  immediate  attention.  We  offer  findings  and  recommendations  regarding 
improvements  that  need  to be  made  now that  the  flurry  of initial  implementation  is 
over  and  the  JOBS  program  is beginning  to stabilize  and  mature.  These  weaknesses 
are  in the  areas  of monitoring,  technical  assistance,  program  guidance,  and 
management  information. 
MONITORING 
The  ACF  Headquarters  guidance  for field  reviews  and  data  collection  has been 
inadequate. 
The  ACF  performs  the  monitoring  function  of the  States  primarily  through  on-site 
field reviews.  These  are  conducted  by teams  composed  of ACF  Headquarters  and 
regional  office  personnel.  Monitoring  is also performed  through  analysis  of data  that 
States  submit  to ACF  Headquarters  and  regional  offices. 
Field reviews 
The  ACF  conducted  field reviews  of 13 States  in Fiscal  Year  (FY)  1990 and  of the 
remaining  States  in FY  1991.  These  reviews  covered  child care,  supportive  services, 
systems,  and  data  collection  as well as JOBS.  In FY  1992, ACF  conducted  field 
reviews  covering  the  program  operations  of  18 States.  The  principal  focus  of these 
reviews  was to describe  and  evaluate  significant  changes  in program  operations  and  to 
obtain  a sense  of client  flow through  the  JOBS  program  since  the  previous  review. 
According  to ACF  staff responsible  for field  reviews,  interesting  practices  identified 
in the  field  review  process  are  shared  with  the  regions  and  States.  However,  we did 
find some  problems  with  the  reviews. 
Field review guides were not released timely to the regions. 
 
The  final  field review  guides  for both  FY  1991 and  FY  1992 were  released  in 
late  spring,  which  limited  the  time  available  for the  regions  to conduct  and 
complete  required  field review  activities. 
4
 An ACF  work  group  convened  in February  1992 for the  purpose  of designing 
the  field  review  guide  for FY  1992.  A draft  version  of this guide  was provided 
to the  regions  in March  1992.  The  final field review  guide  was released  in 
June  1992.  The  ACF  Headquarters  stated  that  regions  had  the  option  of using 
the  draft  guide  to conduct  their  field  reviews.  However,  many  regions  deferred 
their  field  reviews  until  after  the  issuance  of the  final field  review  guide. 
The  field  review  guides  for FY  1993 were  released  in draft  to the  regions  in 
December  1992.  As of February  2, 1993, these  had  not been  released  in final. 
  Field review reports were not rekz.wd  time~  to the States. 
The  ACF  Headquarters  has  not  established  uniform  national  time  limits  for the 
preparation  and  release  of field  review  reports.  In FY  1991, the  average 
elapsed  time  from  the  field  review  visit to the  release  of the  field  review  reports 
was over  seven  months.  Two  field  review  reports  were  released  after  17 months. 
At  the  time  of our  review  in September  1992, 6 States  had  not  received  their  FY 
1991 field  review  reports,  a time  lapse  of more  than  12 months.  By February 
1993, ACF  had  released  four  of these  FY  1991 reports.  The  two outstanding  FY 
1991 field  review  reports  were  for Kansas  and  Nebraska.  This  delay  adversely 
affects  the  usefulness  of these  reports  to States.  (Appendix  A has  a complete 
listing  of field  reviews  conducted  and  the  report  status  for  FYs  1990- 1991). 
At  the  time  of our  review  in September  1992, no final FY  1992 field  review 
reports  had  been  released. 
The  ACF  Headquarters  does  not  have  readily  available  the  dates  that  regional 
offices  release  final field  review  reports.  The  ACF  Headquarters  staff  stated 
that,  under  the  Agencys decentralized  structure,  Regional  Administrators  are 
responsible  for the  final  approval  and  release  of these  reports. 
  Regional ofices  were not directed to provh!e the States with a comment period 
prior to the release of final field review repr&. 
The  ACF  Headquarters  has  not  provided  guidance  to the  regional  offices  on 
releasing  draft  reports  to the  States  for their  review  and  comment.  Under 
ACFS decentralized  structure,  final  field  review  reports  are  released  by ACF 
Regional  Administrators  upon  completion  of ACF  Headquarters  internal 
review  of the  report  content  for accuracy  and  sensitivity.  The  ACF  staff 
informed  us that  States  have  the  opportunity  to  comment  on final  reports. 
However,  ACF  does  not  solicit  these  comments. 
5
 	 Re@mal  oj@e.s  were not directed to reqtie  States to submit comective action 
plans. 
The  ACF  Headquarters  has  not  provided  guidance  to the  regional  offices  on 
State  submittal  of corrective  action  plans.  The  field  review  process  has 
identified  problem  areas  which  merit  States  attention.  Currently,  there  are  no 
legal,  regulatory,  or Agency  policies  which require  the  States  to respond  to 
these  problem  areas. 
Regional ofices  were not directed to follow-up  mth States on jiekl  review @dings 
	
and recommendahons. 
The  ACF  Headquarters  does  not  require  the  regional  offices  to follow-up  with 
States  to  determine  if they  have  addressed  the  problems  or weaknesses 
identified  in the  field  review  process.  Some  regional  offices  may conduct 
follow-up  as part  of their  overall  JOBS  program  oversight,  but  there  is no 
assurance  that  this will be  done  timely  or even  done  at all. 
  l%e regions were not directed to use the fieki review process  to monitor the States 
for compliance  with their State Pkms. 
Field  review  guides,  which  are  the  primary  means  of monitoring  JOBS,  do not 
specifically  address  the  review  of the  States  and  political  subdivisions 
compliance  with State  Plan  provisions.  Field  reviews  are  for  fact finding 
purposes  and  are  not  used  as a monitoring  tool  for compliance.  However, 
when  the  review  team  identified  problem  areas  while  conducting  past  field 
reviews,  the  issues were  often  included  in the  field  review  report. 
The  ACF  staff felt  it was inappropriate  to monitor  for compliance  during  the 
early  stages  of implementation  when  it was more  important  to understand  what 
was happening  at the  State  level  and  ensure  that  instructions  and  guidance 
were  understood.  Now that  this stage  is over, however,  we believe  a more 
systematic  method  of compliance  monitoring  is needed. 
Data collection 
States  are  required  by law, regulation,  or administrative  guidance  (in the  form  of 
action  transmittals)  to submit  data  to ACF  on various  aspects  of the  JOBS  program. 
However,  there  are  no penalty  mechanisms  to enforce  JOBS  program  reporting 
requirements.  The  ACF  Headquarters  staff  stated  that  the  Agency  is applying  gentle 
pressure  on the  States  to get  them  to comply  with all reporting  requirements.  The 
regional  offices  contact  those  States  that  are  having  difficulty  meeting  the  deadlines  for 
data  submission.  Once  these  States  have  been  contacted,  the  regional  offices  offer 
assistance  and  guidance,  if necessary,  to ensure  timely submission  of data.  The  ACF 
Headquarters  data  branch  is responsible  for providing  technical  assistance  in this area. 
6
   lhe ACF  k not receiving State data bzely. 
Quarter& Pa~cipation Rate Stathtical Repoti  (FSA-103) 
Action  Transmittal  90-11 provided  conflicting  instructions  concerning  the  dates 
that  the  Quarterly  Participation  Rate  Statistical  Reports  (FSA-103)  are  due  in 
ACF  Headquarters  and  regional  offices.  To  compensate  for  conflicting  due 
dates,  we allowed  a 30-day grace  period  from  the  specific  dates  mentioned  in 
the  action  transmittal  when  determining  the  timeliness  of these  reports.  Even 
with  this  allowance,  over  one  third  of the  States  and  political  subdivisions  have 
not  consistently  submitted  Quarterly  Participation  Rate  Statistical  Reports 
(FSA-103)  timely  in FY  1991 and  FY  1992 (Appendix  B). 
JOBS l?rogram Participation Data  Collection Information  (FSA-108J 
All States  and  political  subdivisions  are  required  to electronically  submit  the 
monthly  JOBS  Program  Participation  Data  Collection  Information  (FSA-108). 
The  first  submission  was due  by December  15, 1991 for  the  sample  month  of 
October  1991.  The  Code of Federal Regulations [45  CFR 250.80(a)(l)], 
requires  States  and  political  subdivisions  to electronically  submit  data  no later 
than  45 days after  the  end  of the  sample  month.  Action  Transmittal  91-17 
improperly  extended  the  submission  due  date  for  the  JOBS  Program 
Participation  Data  Collection  Information  (FSA-108)  report  to 60 days. 
However,  some  States  are  not  meeting  either  time  frame  for  the  submission  of 
JOBS  Program  Participation  Data  Collection  Information  (FSA-108)  reports 
(Chart  1). 
Chart  1 
The  Number  of  States  Not  Reporting  IWA-108  Data 
for  October  1991  - August  1992 
Status  as  of  November  1992 
30, 
28

26

24

22

20

18

16

14

12

10

8

6

4

10)91  -  12)91   -- 02)92  u- 04)92  -- 06)92  u- 08}92

ReportMonth 
7
 November  1992 data  furnished  by ACF  Headquarters  established  that  of the 
50 States  and  the  District  of Columbia: 
Six (Delaware,  Minnesota,  Missouri,  Nebraska,  New Hampshire,  and 
	
Pennsylvania)  had  not  electronically  submitted  any of the  JOBS  Program 
Participation  Data  Collection  Information  (FSA-108)  to ACF 
Headquarters; 
Seven  (Alabama,  Connecticut,  Indiana,  Ohio,  Rhode  Island,  Tennessee, 
	
and  West  Virginia)  were  7 to  11 months  late  submitting  some  or all of 
their  reports; 
Three  (District  of Columbia,  Montana,  and  South  Dakota)  were  four  to 
	
six months  late  submitting  some  or all of their  reports; 
Eight  (California,  Georgia,  Kansas,  Maryland,  Michigan,  New  Jersey, 
	
New  York,  and  Oklahoma)  were  up to three  months  late  submitting 
some  or all of their  reports; 
Three  (Illinois,  Massachusetts,  and  Mississippi)  had  submitted  data  for 
	
all periods  except  the  most  recent  report  month  (August  1992); and 
The  remaining  24 were  current  with their  data  reports. 
 
Implementation  of (?uarterly Report of Ztle  IV-F Bvenditures  Unifonm Repoting 
Requirements  (ACF-332) 
According  to one  ACF  report,  24 States  were  late  submitting  Implementation 
of Quarterly  Report  of Title  IV-F  Expenditures,  Uniform  Reporting 
Requirements  (ACF-332)  reports  for the  first and/or  second  quarter  of 
FY  1992. 
ChiL.iCare Reporting Requirements (ACF-115} 
Based  upon  information  contained  in an ACF  report,  33 States  were  late 
submitting  Implementation  of Child  Care  Reporting  Requirements 
(ACF-115)  reports  for the  first and/or  second  quarter  of FY  1992. 
  T%eACF  Headqumten  has not established a closure date for  the submiwion  of 
States data. 
Data  for the  implementation  of Quarterly  Participation  Rate  Statistical  Report 
(FSA-103)  is due  on a quarterly  basis.  The  data  that  was due  for the  first 
quarter  of  1991 (in February  1991) was still being  revised  by some  States  in 
April  1992.  The  data  that  was due  for  the  first  quarter  of 1992 (in February 
1992) was still being  revised  by some  States  in August  of 1992 (Appendix  B). 
8  Data  for the  JOBS  Program  Participation  Data  Collection  Information 
(FSA-108)  is due  60 days after  the  sample  month.  Four  States  submitted 
revisions  to the  October  1991 data  in August  1992, eight  months  after  the 
report  due  date. 
Additional  discussion  of matters  related  to data  collection  are  contained  in the

Program  Guidance  and  Management  Information  sections  that  appear  later  in this

report.

TECHNICAL  ASSISTANCE

The  ACF  provided  technical  assistance  and  training  to regional  office  staff  through

conferences  held  in FYs  1989, 1991, and  1992.  Shortly  after  the  publication  of the

final regulations  in the  Fall  of 1989, ACF  held  a major  conference  for  over  700

Federal,  State,  and  Tribal  officials  to provide  operational  and  policy  guidance  for the

JOBS  program.  In addition,  ACF  Headquarters  staff visited  every  region  in the

summer  of 1990 to train  staff  on reviewing  State  Plans,  provided  pre-printed  guides  of

the  State  Plan  to the  States,  and  issued  numerous  action  transmittals  and  information

memoranda.  Further,  ACF  Headquarters  retained  technical  assistance  contractors

specifically  for the  purpose  of providing  technical  assistance  and  guidance  to the

States.

The  field  reviews  and  States  data  were  used  to identify  technical  assistance  needs.

The  HHS,  in conjunction  with  the  Departments  of Labor  and  Education,  provide

technical  assistance  primarily  through  the  use  of contractors.  In  1990, the  three

Departments  entered  into  a three  year,  six million  dollar  technical  assistance  contract

with the  National  Alliance  of Business,  Maximus,  and  Pelavin  Associates  to help  States

and  localities  operate  effective  and  efficient  JOBS  programs.  Contract  funding  is

apportioned  among  the  three.  All deliverables  developed  by the  technical  assistance

contractors  had  to be  reviewed  and  cleared  by the  three  Departments  before  being

distributed  to States.

The  ACF  awarded  the  technical  assistance  contract  in August  1990.  The  contract  was

written  to be  an  evolving contract  to address  States  needs.  A November  1990

memo  to ACF  Regional  Administrators  spelled  out  the  three  tasks  assigned  to the

technical  assistance  contractors.  These  tasks  were:  assistance  to program  managers,

assistance  to providers,  and  systems  assessment  and  information  dissemination.  After

consultation  with  the  Departments  of Labor  and  Education,  the  first  two tasks  were

modified  to reflect  the  following  priority  areas:  program  design,  coordination,

marketing,  and  agency  cultural/structural  change.

9
 The  ACF  did not  reap  the  full benefits  that  could  have  been  obtained  from  the 
technical  assistance  contractors. 
As indicated  above,  ACF  and  the  technical  assistance  contractors  initiated  important 
and  useful  training  and  guidance.  However,  the  full potential  of available  resources 
was not  realized. 
  The techrkal  assistance contraction did not address the fowpriody  areas timely. 
The  contractors  were  to focus  on the  four  priority  areas  listed  above  for the 
first six to nine  months  of their  contract.  Additional  areas  were  to be  covered 
in subsequent  years.  As shown below,  the  contractors  did not  cover  all four  of 
these  priority  areas  during  the  first year  of their  contract.  The  only additional 
area  that  has been  addressed  is literacy. 
One  of the  four  priority  areas  was addressed  in FY  1991, when  the  technical 
assistance  contractors  conducted  six regional  workshops  on program  design. 
The  remaining  priority  areas  that  were  to be  covered  in the  first six to nine 
months  of the  contract  were  not  addressed  until  FY  1992.  This was 
accomplished  through  five regional  workshops  covering  coordination, 
management,  and  agency  cultural  change;  three  regional  workshops  on 
marketing;  and  a national  training  conference  on agency  cultural  change, 
	 7%e technical &tance  contraction dii  not address all of the subjecti  the Sates 
requested 
As noted  above,  the  contractors  were  to provide  technical  assistance  in addition 
to the  four  priority  areas.  Therefore,  they  queried  the  States  in FY  1990 and 
FY  1991 to determine  the  States  training  needs.  The  States  responded  that 
they needed  technical  assistance  in the  areas  of child  care,  teen  parents, 
systems,  and  data. 
The  ACF  Headquarters  Project  Officer  responsible  for the  technical  assistance 
contract  stated  that  the  technical  assistance  contractors  did  act  as facilitators  for 
a session  on child  care  for Federal  staff.  However,  the  contractors  provided  no 
training  for the  States  on the  subject.  The  ACF  purposely  elected  to address 
child care  rather  than  have  the  contractors  do it.  One  means  of addressing  this 
was through  a child  care  conference  held  in December  1992. 
With  respect  to teen  parents,  the  contractors  devoted  one  session  to this topic 
during  the  Program  and  Component  Design  workshops  held  in FY  1991. 
Beyond  this, ACF  indicates  it addressed  teen  parents  through  the  field  review 
process,  specific  policy guidance,  and  other  means. 
10
 The  Project  Officer  considered  systems  and  data  training  to be  a responsibility 
of ACF  Systems  and  Data  branches  since Agency  expertise  was available. 
Therefore,  the  contractors  did not  conduct  this training. 
The  contractors  had  very little  involvement  in providing  technical  assistance  for 
these  areas  as ACF  chose  to address  three  of the  four  on their  own.  However, 
the  contractors  have  scheduled  additional  workshops  and  on-site  forums  for 
FY  1993 that  relate  to literacy. 
  7he technical assktance  materials were not diwrniiuzted t$ectively.  l%is limited 
their u@hlneM  to States. 
The  technical  assistance  contractors  sent  a memorandum  to State  agencies  in 
December  of 1990 stating  that  they were  working  on a General  Systems  Design 
for the  JOBS  Automated  System  and  a Data  Element  Dictionary.  At  that  time, 
while  all States  had  implemented  their  JOBS  program,  several  States  were 
continuing  to design  and  implement  their  JOBS  reporting  and  data  systems. 
The  General  Systems  Design  and  Data  Element  Dictionary  were  not  published 
and  distributed  to all States  until August  1991. 
In May of  1991, the  contractors  promised  that  a Coordination  Handbook  would 
be  completed  by the  end  of the year.  This  handbook  was designed  to facilitate 
the  coordination  between  Federal,  State,  and  local  entities  involved  in the 
JOBS  program.  The  Coordination  Handbook  was not  distributed  until 
November  1992. 
The  technical  assistance  contractors  have  concentrated  most  of their  efforts  on 
the  development  of technical  assistance  workshops  and workshop  materials. 
The  handouts  and  workbooks  developed  for the  workshops  were  given  to all 
attenders.  Personnel  from  non-attending  States  could  request  copies  of the 
workshop  materials.  However,  ACF  did not  routinely  send  this material  to 
non-attenders,  due  to the  expense  to the  agency  and  because  non-attenders 
may not  have  been  able  to fully understand  and  utilize  the  information  without 
the  workshop  presentations. 
PROGRAM  GUIDANCE 
Given  the  magnitude  of the  implementation  requirements  and  the  fact  that  States  had 
the  option  to begin  JOBS  programs  within  the  first year,  ACF  faced  a formidable  task 
in issuing the  necessary  guidelines  to them.  It did issue  the  implementing  regulations 
within  one year.  The  ACF  also used  action  transmittals  and  information  memoranda 
to communicate  Agency  policies,  procedures,  and  requirements  to regions  and  States. 
To its credit,  42 of the  53 action  transmittals  and  information  memoranda  we reviewed 
were  disseminated  in a timely  fashion.  However,  we did identify  some  weaknesses  in 
this area. 
11
 Some  action  transmittals  and  information  memoranda  were  not  disseminated  in a

timely  fashion.

Appendix  C lists eleven  action  transmittals  which  were  issued  close  to  or after  their

effective  date.  Most  of these  relate  to systems  and  data  reporting,  critical  aspects  of

program  implementation.  The  delays  in issuing  the  transmittals  gave  the  States  and

political  subdivisions  very little  lead  time  to undertake  the  complex  task  of revising

their  computer  systems  and  begin  capturing  data.

In addition,  as we noted  in the  earlier  section  on data  collection,  an Action

Transmittal  (AT-91-17),  improperly  extended  the  submission  due  date  for the  JOBS

Program  Participation  Data  Collection  Information  (FSA-108)  to  60 days.  This  is

contrary  to the  Federal  Regulations  which require  States  and  political  subdivisions  to

electronically  submit  data  no later  than  45 days after  the  end  of the  sample  month

[45 CFR  250.80(a)(l)].

MANAGEMENT  INFORMATION

The  ACF  Headquarters  has  not  systematically  validated  or analyzed  information  from

field  reviews,  State  Plans,  and  data  collection  System$  nor  has it routinely

disseminated  management  information  to Congress,  Federal  and  State  managers,

policy makers,  and  advocacy  groups.

The  ACF  Headquarters  has worked  with the  States  to develop  several  information

initiatives.  However,  we found  weaknesses  both  with  the  data  itself  and  with  the  use

being  made  of it.

	 %  ACF  Head@atiem  has not adequately validated and ana@xi  information 
submitted by the States 
The  ACF  Headquarters  utilizes  several  means  of analyzing  information 
submitted  by the  States.  Computer  edits  point  out  coding  and  transmission 
errors  in State  data.  Consistency  checks  are  used  to compare  total  fields with 
information  contained  in the  data  elements  to ensure  uniformity.  Acceptable 
ranges  for  data  are  established  based  upon  historical  data  parameters  that  the 
States  previously  reported.  The  FY  1992 field  review  process  involved 
reviewing  a sample  of case  files against  JOBS  Program  Participation  Data 
Collection  Information  (FSA-108).  However,  while  the  field  review  process 
does  validate  case  file information  from  one  or two County  offices  within  a 
State  against  the  data  submitted  by that  office,  the  review  is too  limited  in 
scope  (less  than  100 cases  total  at two sites)  to adequately  determine  if States 
data  is accurate.  It should  also be noted  that  the  FY  1992 field  reviews  were 
conducted  in less than  half  of the  States. 
12
   %  ACF  Headquarter  has not routidy  dkwrnika ted management  informatiim in 
a usejhl foimat  to those who need it 
Management  information  has not  been  made  routinely  available  to Federal  and 
State  managers,  policy makers,  the  Congress,  and  advocacy  groups.  Agency 
products,  such as summary  statistical  reports  on participation,  outcomes, 
characteristics  of State  Plans,  demographic  information  about  participants,  and 
interesting  practices  are  useful  to Congress,  HHS  components,  ACF 
Headquarters/regional  offices, States,  and  advocacy  groups  in their  research  and 
planning  activities.  This information  has not been  widely disseminated  within 
and  outside  the  Department. 
13
 RECOMMENDATIONS

As indicated  in the  background  and  findings  sections,  we recognize  the  difficulties 
faced  by ACF  and  its success  in issuing the  implementing  regulations  and  providing 
the  initial  structure  for monitoring  the  program.  Nevertheless,  we believe  that  certain 
improvements,  which  are  the  responsibility  of ACF  Headquarters,  require  immediate 
attention. 
Specifically,  to address  the  problems  identified  above,  ACF  should 
develop  adequate  processes  to monitor  States  for compliance  with  their  State  Plans 
and JOBS program  requiremen~, 
review and clari.tj the purpose  of existing monitoring  initiatives and strengthen the 
policies  and procedures  guiding their administratiory 
In particulq  with respect to jield reviews: 
provide  field  review  guides  to the  regional  offices  by the  second  quarter 
	
of the  fiscal year  to  allow sufficient  lead  time  to schedule,  conduct,  and 
complete  these  reviews; 
ensure  that  field  review  reports  to the  States  are  prepared  and  released 
	
in draft  for their  comment  within  90 days of the  field  review  visit; 
provide  guidance  to the  regional  offices  on requiring  States  to  submit 
 
corrective  action  plans  to address  problem  areas  identified  during  field 
reviews;  and 
provide  guidance  to the  regional  offices to ensure  that  problem  areas 
	
identified  in the  field  reviews  are  being  corrected. 
Wtih respect to data collection 
continue  to work  closely with  regional  offices  and  States  to ensure  the 
	
accurate,  timely,  and  complete  submission  of data;  and 
establish  deadlines  for the  correction  and  finalization  of States  data. 
 
systematically reassess the specific  needs  of the States for technical  assistance and 
provide  a timely and appropriate  response; 
ensure that action transmittals and information  memoranda  conform  with Federal 
law and regulations and Ageney  policieq 
14
 allow 60 to 90 days lead time, whenever posslkde, for the effective  dates of action 
transmittals and information  memorand~ 
develop  a systematic analysis plan for JOBS program  information  available  from 
field  reviews,  State  Plans,  and  data  collection  Systew,  and 
disseminate  program  information  and data analysis through timely reports  on 
varied program  elements  to Federal  and State managers  and policy  makers. 
AGENCY  COMMENTS  AND  OIG  RESPONSE 
The  ACF  has  agreed  with  most  of our recommendations  and  has  outlined  specific 
actions  it has planned  or  already  taken  to implement  them.  Their  one  major 
reservation  concerns  monitoring.  They  believe  that  it would  be  prudent  to delay  a 
final  decision  on these  findings  and  recommendations  until  the  second  phase  of the 
inspection--Regional  Offices  and  States  Perspectives--has  been  completed. 
We  continue  to believe  that  our  findings  and  recommendations  represent  concerns 
that  should  be  addressed  in Headquarters  operation  of the  JOBS  program. 
Nevertheless,  we accept  ACFS suggestion  to delay  final  action  on monitoring  until  the 
second  phase  of our  study  is done. 
The  ACF  also  expressed  concerns  about  some  of our  findings  and  wished  to 
emphasize  numerous  positive  actions  they have  taken  on matters  addressed  in our 
report.  We wish to  emphasize  that  we agree  that  ACF  has  taken  aggressive  action  to 
implement  the  JOBS  program.  We  highlighted  this in the  Executive  Summary  and  in 
the  introduction  to our  findings.  We have  provided  some  additional  examples  of their 
actions  in this final version  of the  report.  Space  and  presentation  style simply  do not 
allow us to enumerate  everything  which ACF  has done.  For  more  examples,  we invite 
the  reader  to read  ACF  comments,  which  are  provided  verbatim  in Appendix  D. 
However,  it should  be  noted  that  the  purpose  of this report  was to  point  out  areas 
needing  further  attention.  Such  areas  for improvement  inevitably  arise  no matter  how 
intensively  management  attempts  to deal  with complex  problems. 
We  made  other  modifications  to the  report  in response  to ACFS comments.  Again, 
the  full text  of their  comments  is contained  in Appendix  D. 
We  understand  that  ACF  has  recently  established  several  task  forces,  one  of which  is 
to address  monitoring  and  technical  assistance.  We believe  that  this  is a positive  step 
and  hope  that  the  information  in this report  will be  useful  to the  task  force  addressing 
these  functional  areas. 
15
 APPENDIX  A

I 
Connecticut  I  05/91  I  09/91  4 months 
I  I  I 
Maine  04/91  08/91  4 months 
I  I  I 
Massachusetts  03/91  06/91  3 months 
I  I  I 
New Hampshire  03/91  06/91  3 months 
I  I  I 
Rhode  Island  07/90  12/90  5 months 
I  I  I 
Vermont  06/91  10/91  4 months 
I  I  I 
Delaware  03/91  10/91  7 months 
I 
District  of Columbia  I  02/91  I  10/91  8 months 
Maryland
 04/91  02/92  10 months 
I 
Pennsylvania
 I  09/91  I  07/92  10 months 
I  I 
Virginia
 08/91  !  04/92  8 months 
I  I 
West  Virginia
 09/91  I  06/92  9 months 
Alabama
 04/91  07/91  3 months 
Florida
 02/91  05/91  3 months 
Georgia
 10/90  01/91  3 months 
Kentucky
 09/91  01/92  4 months 
Mississippi
 06/91  09/91  3 months 
I 
North  Carolina
 I  09/91  I  03/92  6 months 
South  Carolina
 04/91  07/91  3 months 
A-1  I 
II  Tennessee  I  05/91  I  10/91  5 months 
II  Illinois  07/91  09/92  14 months 
II  Indiana  09/91  10/92  13 months 
Michigan  09/91  09/92  12 months 
Minnesota  08/90  11/90  3 months 
II  Ohio  06/91  09/92  15 months 
II  I  I  I 
Wisconsin  08/90  12/90  4 months 
Arkansas  03/91  05/91  2 months 
Louisiana  08/91  01/92  5 months 
New Mexico  06/91  09/91  3 months 
Oklahoma  09/90  03/91  6 months 
Texas  09/91  12/91  3 months 
II  I 
Iowa  I  09/90 \  04/91  7 months 
II  I
 I  I 
Kansas  06/91  Not  Released*  19 +  monthsl 
II  Missouri
 08/91  01/93  17 months 
II  I
 I  I 
Nebraska  09/91  Not  Releasedl  16 +  monthsl 
Colorado
 08/90  01/91  5 months 
Montana
 07/91  02/92  7 months 
II  North  Dakota
 07/91  12/91  5 months 
II  South  Dakota
 04/91  10/91  6 months 
Utah
 03/91  10/91  7 months 
Wyoming
 09/91  05/92  8 months 
A-2  I  I  II

Arizona  03/91  I  05/91  2 months 
II

II  California  I  08/90  I  11/90  I  3 months 
Hawaii  07/91  12/91  5 months

Nevada  02/91  02/91  O months

Alaska  06/91  09/92  15 months

Idaho  04/91  01/92  9 months

II  I  I  I  II

Oregon  07/91  07/92  12 months 
A-3
 APPENDIX  B 
1st 
Quarter 
FY  1991  25  11  15  0  4  11  6 
2nd 
Quarter 
FY  1991  37  8  6  1  1  5  1 
3rd 
Quarter 
FY  1991  30  9  12  1  2  4  0 
4th 
Quarter 
FY  1991  34  10  7  0  1  2  0 
1st 
Quarter 
FY  1992  32  9  104  0  1  4  0 
2nd 
Quarter 
FY  1992  37  5  P  o  0  2  0 
3rd 
Quarter 
B-1  APPENDIX  C

JOBS Systems Information

Data Reporting-Implementation  of

Quarterly Information  Collection of JOBS

AT-90-1  01/31/90  10/01/89  Program Participation  (Form FSA-104)

Data Reporting-Implementation  of Annual

Report  of Target Group Expenditures  Under

AT-90-2  02/06/90  10/01/89  the JOBS Promam fForm FSA-302)

Data Reporting-Implementation  of Statistical

Report on Recipients Under  Public Assistance

Programs (Form FSA-3637) and Data

Reporting-Implementation  of Monthly Flash

Report of Selected Program Data

AT-90-8  05/11/90  10/01/89  (Form FSA-36451

Data Reporting-Implementation  of Quarterly

Participation Rate Statistical Report

AT-90-11  09/07/90  10/01/s9  (Form FSA-103)

AT-90-14  10/10/90  07/01/89  JOBS Training Program Systems Information

Data Reporting-Coding of Missing Data on

the JOBS Program Participation  Data

AT-91-22  12/06/91  10/01/91  Collection Form (Form FSA-108)

Data Reporting-Implementation  of Quarterly

Report  of Title IV-F Expenditures,  Uniform

AT-91-23  12/26/91  10/01/91  Reporting Requirements  (Form ACF-332)

Data Reporting-Implementation  of Uniform

Data Collection and Reporting  Requirements

for Tribal Grantees, Tribal Jobs Program

AT-92-1  01/14/92  01/01/92  Quarterly Report  (Form ACF-114)

Pre-printed  Guide for the JOBS State Plans:

AT-92-3  02/28/92  02/28/92  FY 1992 Biennial Update

Data Reporting-Implementation  of Child Care

CC-AT-92-1  04/06/92  10/01/91  Reporting Requirements  (Form ACF-115)

c-1  APPENDIX  D 
ADMINISTRATION FOR CHILDREN AND 
FAMILIESCOMMENTS 
D-1  ..  .._._....  -.
.=29- -
ADMINISTRATION  FOR CHILDREN AND FAtilLII 
Offlceof  the Assistant  Secreta~,  Suite 600 
370 LEnfant  Promenade,  S.W. 
Washington,  D.C.  20447 
June  28,  1993 
TO:  Bryan  B. Mitcheu 
Pr~nci,pal  D  pu  Insne  r  General 
FROM:
 Laurence  .  e 
Acting  Ass<st  nt  Se  retary  for 
Child  #+ n  and  Families 
SUBJECT :
 Response to OIG Draft Report: JOBS Monitoring  and

Technical Assistance:  Headquartersi Role,

OE1-07-92-00380

Thank YOU
 for the opportunity tocoxmnenton this draft renort. 
We.believe  the  repo~t  is an tiprovementover the working ;ersion  . 
in providing  a more balanced description of the Administration 
for Children  and Families (ACF)monitoring tid technical 
assistance  efforts  to  states  for  the  Job  Opportunities  and  Basic 
...+
Skills Training  (JOBS) program. 
However, we still have a major concern with the OIGS methodology

of separately  reviewing Central Office  (CO) and Regional Office

...(RO)a
dministration  of JOBS.  AS thereport itself acknowledges, 
.,~.~eOIG does not have a complete picture of,ACl?  activities  s.i.nce 
:..::;thas  riot  Completed  phasetwo  of thestudy.  Yet many:of the 
:reCo~endationS--Darticularly  those  relative  to  monitoring--if 
adopted  would  sigfiificantly  affect  ItO  operations.  We  believe 
that  the  OIG  would  be  in  a  better  position  to  dete~ine  if  such 
recorrunendations  (and/or  others)  are  necessary  once  it  completes 
phase  two  of  the  study. 
Our response  is organized by the sections of the OIG report.

MOliITORING  ,

As stated above, we believe that the section on monitoring-

specifically  on the subject  of the field review  process--is the 
most troublesome  for two reasons which we elaborate  on below. 
First, the report does not give  sufficient  recognition  to ACF 
Policy on the respective roles of CO and ROS in conducting 
program reviews.  Second, it seems to assume that  if CO did not 
issUe a written  directive on how  to  carry  out  a  review,  ROS may 
not have conducted reviews in a professional manner. 
The specific  approach to monitoring JOBS and the respective roleS 
of CO and the ROlS were  established  by  the  Assistant  Secretary  as 
follows: 
D-2
 . 
Page 2 - Mr. Bryan B. Mitchell 
all  States would be reviewed during FY 1990 and FY 
1991; 
field reviews would be a broad look at management,

client flow, financial operations, systems, and child

care;

field reviews at this early point in implementation

would be geared to gaining information and identifying

potential problems;

ACF would be explicit with States that the reviews were

not compliance reviews; and

Regional Offices wouldhave lead responsibility for

conducting the reviews, but that CO would review draft

reports narrowly for accuracy and sensitivity given

the high profile of the JOBS program.

Lead responsibility for conducting reviews means that the RO

establishes the schedule for the review, makes all arrangements,

writes the report, issues it (after CO review for accuracy and

sensitivity), receives comments, and follows up as appropriate.

...As  a  result  it  i.s  highly  likely  that  the  OIG  will.  find  a  range  of 
..,procedures  in  the  Regional  offices  when  itconducts.that.part  of 
the  review.  Such  variations  may  reflect  the  experiences  of  the 
..Regional  Administrator,  the  resources  available  to  the  Region,  or 
the  nature  of  the  revi,ew  that  was  conducted.  That  i.s  different 
from  the  implication  in  the  draft  report  that  AC??S lack  of  a 
national  policy  and  written  procedures  means  that  no  opportunity 
for  comment  i,s  provided  or  that  Regions  do  not follow-up  on  the 
results  of  field  reviews.  We  believe  that  such  findings  and 
recommendations  should  only  be  made  after  the  IG  conducts  its 
review  of  RO  operations. 
we  do  agree  with  the  OIG  that  there  were  delays  in  i,ssuing  some 
of  the  FY  1991  reports.  We  believe  much  of  the  delay  was 
attributable  to  the  comprehensive  nature  of  the  reviews  that  were  ~ 
conducted  and  that  narrower  reviews  conducted  in  subsequent  years 
addressed  the  problem  of  timeliness.  Nevertheless,  this  is  an 
area  thatwe  will  look  at  more  carefully. 
TE~IcAL  ASSISTM?CE 
In general, the report suggests that the Technical Assistance 
(TA) contract was the only vehicle by which ACF provided TA to 
State JOBS programs.  The JOBS TA contract has been the primary
vehicle by  which  co  has  provided  TA  directly  to  states,  but  nOt 
the  only  one.  For example, shortly after the publication of the 
final regulations, the Office of Family Assistance held a major 
D-3
 Page 3  - Mr.  Bryan B. Mitchell 
conference for over 700 Federal, State and Tribal officials to

provide operation and policy guidance for the JOBS program.

However, the ROS also have major responsibility for providing

ongoing TA to States.  They have conducted meetings with their

respective States and provided operational guidance to States on

a regular basis.

The report states on page nine that in less than one month the

scope of work of the contract changed.  This is misleading.  The

November 1990 memo to the Regional Administrators indicated that

HHS was working with the Departments of Labor and Education,to

establish priority areas under Tasks 1 and 2 (assistance to

program managers, assistance to providers).  Once the priority

areas were identified and amounted  in the December memo to the

Regional Administrators, we no,longer used the terms Tasks 1

and 2.  We focussed our assistance under the priority areas and

targeted first year efforts to program managers.

The report indicates on page 10 that only one priority area was

addressed in FY 1991 through the conduct of six regional

workshops on program design.  This is inaccurate.  The regional

prograxndesign workshops  (attended by 49 States and 4

jurisdictions) also emphasized interagency coordination

strategies  including State discussion forums on coordination.  In

~..addition,the contractor assisted the Department of Education in

..designing  and facilitating various sessions at a national

conference  for all so States to promote coordination among adult

and vocational education, JOBS and JTPA programs.  In.addition,

the contractor completed Task 3--prepared and distributed a

General Systems Design and Data Element Dictionary to States and

conducted various on-site State visits.

In addition to the FY 1992 and FY 1993 deliverables listed in the

report, the contractor also developed an agency

cul,tural/structuralchange training curriculum for IV-A/IV-D/

IV-F workers, provided on-site assistance to four States, and

developed and distributed to 43 States and jurisdictions copies

(in final draft) of Volumes I and II of a coordination handbook

Meeting the Challenges: Coordination to Promote Self- . .
Sufficiency. n 
-... 
The report is correct in saying that the contractor did not

address all of the subjects requested by States.  We believe that

it is entirely appropriate for ACF  (in conjunction with its

partners- -the-De&t~ents  of Education and-Labor) to consider

requests for technical assistance and determine the best way to  

meet those needs.  That might be the contract or it might be

internal  technical assistance.  We believe that as to the 
subjects  identified by the States--child care, teen parents, 
systems,  and data--such decisions were made. 
D-4
 . 
Page 4  - Mr. Bryan B. Mitchell

For example, ACF determined that technical assistance relative to

data collection for JOBS could best be delivered by ACF.staff who

Therefore, CO staff

had developed the reporting requiraents. 
res~onsible for JOBS data collection made presentations at the

national JOBS conference and numerous regional conferences and

has provided individual assistance to Regional Offices and States

on an on-going basis.

In the area of systems, the contractor did develop a General

Systems Design and Data Element Dictionary under Task 3.

As to teen parents, it is inaccurate to say that the contractor

did not provide assistance.  A session specifically devoted to

teen parent programs was included at all the first years RO

program design workshops.  In addition, ACF has continued to

address teen parent issues through the field review process,

specific policy guidance and identification of promising practice

models.

As to child care, we purposely elected to have child care

This was due

addressed by ACF staff rather than the contractor. 
to the fact that two new major child care programs were passed by

Congress in November 1990- -At  Risk  Child  Care  and  the  Child  Care 
:,Development  Block  Grant-- and  we  believed  that  policy  issues 
relative  to  child  care  were  the  predominant  concern  of  States  and 
that  ACF staff were in a better position to address those 
concerns.  ACF published final regulations for these.progr- in 
August 1992 and held a child care conference in December 1992.

On  the distribution of products developed for workshops, we 
believe that there has been broad distribution of products 
although we acknowledge that some of the products should have 
been released more timely.  Kll States and territories  (except 1) 
attended the program design workshops in FY 1991 and received

many products that the contractor had developed; forty-three

States/territories attended regional workshops beginning in -Y

1992 where they received the final draft of Volumes I and II Of

the Coordination Handbook.  The final version of the Coordination -

Handbook was then distributed to all 54 States/territories in  ..

November 1992.  The General Systems Design and Data Element

Dictionary was distributed to all States in August  1991.

PROGWM  GUIDANCE 
The ACF acknowledges that some action transmittals and

information memoranda were not issued prior to their effective

date or were issued without much lead time.  However, we believe

that in most instances the timing was a result of the short lead

time for implementation of the JOBS program which the OIG

acknowledges in its Executive S~rY.

D-5
 Page 5  - Mr.  Bryan B. Mitchell 
For example, the provisions governing enhanced FFP were effective

October 1, 1989  (prior to publication of the final rules) so ACF

had to collect information back to that date even though it would

have been impossible to issue data collection reporting forms

prior to the publication of the final rules.  ~us  although the

forms were at OMB before October 13, 1989, they were not actually

approved and issued until early in the second quarter of

FY 1990--less than four months after the final rules were

published.  It should be noted that while the forms were late for

the States that elected to start early, for the rest of the

States the reporting forms were issued prior to their

implementation of the JOBS program.

As to the eleven specific ActionTransKd-ttals (AT) and

Information Memoranda  (IM) cited by the OIG, we believe that it

is misleading to characterize some of them as delayed

instmctionson  ATs 89-41 and 90-14 do not contain instructions.

They provide information about systems development and provide 
guidance on FFP rates for systeMS.  The effective dates reflect 
the earliest date for which a State could claim FFP for Systems 
development, but do not impose  any  requirement  on  States  to 
retroactively  report  data.  In  the  case  of  AT-92-3  on  the  FY  1992 
Biennial  update,  while  the  effective  date  was  also  the  date  it 
was  issued  (2/28/92),  States  were  not  required  to  submit  their 
plans  until  July  1,  1992. 
ACF  RESPONSE  To  SPECIFIC  RECOMMENDATIONS 
OIG Recommendation

ACF should develop adequate processes to monitor States for

compliance with their State Plans and JOBS program requirements.

ACF Res~onse

Each year ACF determines its strategy for conducting JOBS reviews

considering the priorities of the Assistant Secretary and the

resources available to Central Office and the Regional Offices.  -

AS ACF prepares its FY 1994 strategy, we will carefully consider.  .

the OIGS recommendation.

OIG Recommendation

ACF should review and clarify the purpose of existing monitoring

initiatives and strengthen the policies and procedures guiding  

their administration.

D-6
 . 
Page 6  - Mr.  Bqan  B. Mitchell 
ACF Resuonse

We agree with the importance to provide guides early enough to

plan reviews in quarters 2, 3 and 4.  We provided the ~  1993

guide to the regions on February 2, 1993 for that reason.

As  to the specific  recommendations  about  issuing  written 
directives  to  ROIS  on  how  to  conduct  field  reviews,  we  believe 
that  it  would  be  prudent  to  wait  for  the  findings  of  the  second 
phase  of  this  review  to  determine  whether  such  guidance  is 
required. 
OIG Recormnendation

ACF should work closely with regional offices and the States to

ensure the accurate, timely, and complete submission of data.

ACF Resnonse

ACF cannot find anything in the findings section of the OIG

report to suggest that it has not worked closely with ROS and

States to improve data collection.  We believe that the OIG will

find when it conducts phase two of this investigation that CO has

worked very closely with ROIS and States.  We will continue to do

so.

We note that one of our efforts to work closely with the States

to ensure better data--extending the deadline for submission of

the FSA-108 from 4S days to 60 days--is criticized by the OIG in

this report.  Discussions with States had indicated that 45 days

was not enough time to submit the required data.  Therefore, we

believed that it was a reasonable accommodation to give the

States an additional 15 days to submit even if this was

inconsistent with the Federal regulations.

OIG Recommendation

ACF  should  egta.bli.sh  deadlines  for  the  correction  and 
. .
finalization  of  States  data. 
-. 
ACF Res~onse

ACF agrees that JOBS data collection has been in place 10ng

enough now to establish such deadlines.  We will consult with

ROS and States on establishing reasonable deadlines and will

give States sufficient notice of the deadlines.

OIG Recommendation

ACF  should  systematically  reassess  the  specific  needs  of  the 
States  for  technical  assistance  and  provide  a  timely  and 
appropriate  response. 
D-7
 . 
. 
Page 7 - Mr. Bryan B. Mitchell

ACF Resnonse

ACF  will  continue  to  assess  the  specific  needs  of  States  for

technical  assistance  and  provide  as  timely  response  as  feasible.

ACF  will  use  various  vehicles  to  identify  State  needs  including

field  reviews,  State  surveys  by  contractor  and  ROIS,  TA  request

sheets  at  conferences  and  specific  State  requests  either  to  ACF

or  the  contractor.  AS  a  result  of  our  ongoing  efforts,  we  have  a

number  of  initiatives  underway  and  planned  for  FY  1993  (in  and

outside  of  the  TA  contract)  including  a  National  JOBS  Directors

Conference,  various  on-site  individual  State  visits,  special

topic  informational  packets  and  workshops  and  a  databank

directory  of  proxni.si,ng  iJOBS  and  child  care  State/local  practices.

OIG Recommendation

ACF  should  ensure  that  action  transmittals  and  information 
memoranda  conform  with  Federal  law  and  regulations  and  Agency 
policies. 
ACF Resnonse

ACF  agrees.  We believe that our decision to give States an 
additional 15 days beyond what is specified in the regulations 
to  submit data was a reasonable response-to the problems States 
were having meeting the  45-day  deadline. 
PIG Recommendation

ACF should allow 60 to  90  days  lead  time,  whenever  possible,  for 
the effective dates of  action  transmittals  and  information 
memoranda. 
ACF  agrees.  whenever  possible,  we  will  allow  60  to  90  days  lead 
time  for  the  effective  dates  of  action  transmittals  and 
. .
information  memoranda. 
-.. 
OIG  Recommendation 
ACF should develop a systematic analysis plan for JOBS program

information available from field reviews, State plans, and data

 
collection systems.

ACF Resnonse

ACF agrees.  We believe that JOBS has reached a level of maturity

where such analysis will be useful.

D-8
 Page  8  - Mr.  Bryan B. Mitchell 
OIG Recommendation

ACF  Should  disseminate  program  information  and  data  analysis 
through  timely  reports  on  varied  program  elements  to  Federal  and 
State  managers  and  policy  makers. 
ACF  ResDonse 
ACF has provided large amounts of information to a wide audience.

However, we agree that management information has not been

disseminated in a systematic manner.  With the maturity of the

JOBS program, we agree that it is now appropriate to do so and we

are developing some reports that would accomplish that goal.

Zechnical comments

Page ii:	 The second sentence in the third paragraph under

MONITORING minimizes the dimensions of the early

reviews.  It also incorrectly makes staff rather than

management responsible for defining the ACF strategy

(the same point applies to the second paragraph from

the bottom of page 6).  We offer this as a replacement

paragraph:

ACF believed it was premature to conduct

compliance reviews during the early stages of

implementation when it was more important to

conduct comprehensive reviews to provide for a

baseline understanding of how the program  wer@

operating.  Reviews in FYs 1990 and 1991 examined

supportive semices,  child care, data collection

and systems as well as JOBS.

Page 4:	 In the first paragraph under Field Reviews, we suggest

that the second sentence be revised in this way:

In FYs 1990 and 1991, reviews were comprehensive

in nature, covering child care, supportive

..

senices,  systems, and data collection as well as

JOBS.  -.

Page 5:	 The fourth sentence in the first paragraph hypothesizes

that ROS may have been reluctant to use the draft

guide and in the next sentence says that regions

deferred their field reviews until issuance of the

final guide.  However, it is not clear how the OIG  

knows this since it has not interviewed ROS yet.

Indeed in the letter issuing the guidance to ROIS on

March 30, 1992, the Director of the Office of Family

Assistance specifically tells ROIS to use the guide and

asks for a schedule of planned reviews.  We suggest

that the fourth and fifth sentences be deleted.

D-9  ,. 
. 
page  9  - m.  Bqan  B. Mitchell 
page  6:  The  next  to  last  paragraph  minind.zes  the  importance and 
the level of detail of the n  1991/1992 eviewsf~~ what

to simply get a feel

saying that they were ...

We recomend  referring to the

was happening... 
breadth and depth of the reviews.  Further, wh~l~ the

reviews were not designed specifically for compliance

purposes, instmctions  were that whenever coqliance

they were to be addressed.

issues were identified, 
. . 
-..  . 
D-10