Department  of  Health  and  Human  Services 
OFFICE  OF 
INSPECTOR  GENERAL 
JOBS  MONITORING  AND 
TECHNICAL  ASSISTANCE: 
States  and  ACF  Regional  Offices 
Perspectives  OFFICE OF INSPECTOR  GENERAL 
The mission of the Office of Inspector General (OIG), as mandated by Public Law 95-452,  as 
amended,  is to protect the integrity of the Department of Health and Human Services (HHS) 
programs  as well as the health and welfare of beneficiaries  served by those programs.  This 
statutory mission  is carried out through a nationwide network of audits,  investigations,  and 
inspections conducted by three OIG operating components:  the Office of Audit Services,  the 
Office of Investigations,  and the Office of Evaluation and Inspections.  The OIG also informs the 
Secretary of HHS of program  and management problems  and recommends  courses to correct them. 
OFFICE  OF  AUDIT  SERVICES 
The OIGS Office of Audit Services (OAS) provides all auditing services for HHS,  either by 
conducting  audits with its own audit resources or by overseeing audit work done by others.  Audits 
examine the performance  of HHS programs  and/or  its grantees and contractors  in carrying  out 
their respective  responsibilities  and are intended to provide independent assessments  of HHS 
programs  and operations  in order to reduce waste, abuse, and mismanagement  and to promote 
economy and efficiency throughout  the Department. 
OFFICE  OF  INVESTIGATIONS 
The OIGS Office of Investigations  (01) conducts criminal,  civil, and administrative  investigations

of allegations  of wrongdoing  in HHS programs  or to HHS beneficiaries  and of unjust enrichment

by providers.  The investigative efforts of 01 lead to criminal convictions,  administrative

sanctions,  or civil money penalties.  The 01 also oversees State Medicaid fraud control units which

investigate and prosecute fraud and patient abuse in the Medicaid program.

OFFICE  OF  EVALUATION  AND  INSPECTIONS 
The OIGS Office of Evaluation and Inspections (OEI) conducts short-term  management  and 
program  evaluations  (called inspections) that focus on issues of concern to the Department,  the 
Congress,  and the public.  The findings and recommendations  contained in these inspection reports 
generate rapid,  accurate,  and up-to-date information on the efficiency, vulnerability,  and 
effectiveness of departmental  programs.  This report was prepared in the Kansas City Regional 
Office under the direction of James H. Wolf, Regional Inspector General.  Project staff 
REGION  HEADQUARTERS 
Hugh Owens,  Project Leader  Ruth Folchman

Perry A. Seaton  W. Mark Krushat,  Ph.D

Hope H. Anderson

For  additional  copies  of this report,  please contact  the Kansas  City  Regional  Office 
at (816)  426-5959.  Department  of  Health  and  Human  Services 
OFFICE  OF 
INSPECTOR  GENERAL 
JOBS  MONITORING  AND 
TECHNICAL  ASSISTANCE: 
States  and  ACF  Regional  Offices 
Perspectives 
J# SERVICE$ 
#+  ~nq 
GIBBS  BROWN
JUNE 
&w 
~  Inspector  General

s

%

%+--# APRIL 1994 
*>dIla  0EI-07-92-O0381  EXECUTIVE  SUMMARY 
PURPOSE 
To provide  an assessment  of the  Administration  for  Children  and  Families  (ACF) 
monitoring  and  technical  assistance  activities  for the  Job  Opportunities  and  Basic  Skills 
Training  (JOBS)  program  based  on the perspectives  of States  and  ACF  regional  offices. 
BACKGROUND 
The  Family  Support  Act  of  1988 established  the JOBS  program  for recipients  of Aid to 
Families  with Dependent  Children  (AFDC) payments.  This program  is designed  to assist 
welfare  recipients  to become  self-sufficient  through  training,  education,  work experience, 
and support  services.  The ACF administers  the JOBS program  through  its Headquarters 
and 10 regional  offices.  In this role,  they monitor  and provide  technical  assistance  to 
States. 
We reviewed  ACFS monitoring  and technical  assistance  activities  for the JOBS program 
in three phases.  First,  we examined  ACF Headquarters  activities.  Then we contacted 
ACF regional  office staff and representatives  of 10 States to obtain their perspectives  and 
other information  about their operations.  The report  on Headquarters  activities  JOBS 
Monitoring  and Technical  Assistance:  Headquarters  Role,  (OEI-07-92-00380)  was issued 
August  1993.  This report  contains a synthesis of regional  office and State perspectives. 
FINDINGS 
Since completing  our Headquarters  report  we note substantial progress  in some areas such 
as timely  release  of field review  guides.  States were also complimentary  about 
contributions  made by ACF regional  office specialists.  However,  additional  improvements 
are needed,  particularly  in the areas of technical  assistance  and management  information. 
We hope this report  will be useful to ACF in preparing  their response  as well as their 
implementation  plan for all of the recommendations  previously  made.  Following  is a 
summary  of our findings. 
Monitoring  - Release  of field review  guides to regions  has been more timely,  which has 
assisted them in their work planning activities.  However,  the regions  diverse  methods 
for sharing  field review  information  has had an adverse  impact on its usefulness  to States. 
States consider  on-site visits to be usefid,  although they prefer  that the visits be better 
structured.  They consider  the regional  office State specialist position  beneficial.  Both 
States and the regional  offices feel these specialists need to visit States more frequently  to 
keep abreast  of what is going on in their JOBS programs. 
Technical Assistance  - Many States believe ACF is not meeting  their technical  assistance 
needs.  Conversely,  Headquarters  and the regions believe they are attempting  to meet 
i  most of them.  The  States  prefer  establishment  of  technology  transfer  over  the  use  of 
technical  assistance  contractors.  Neither the regions  nor the States considered  the 
technical  assistance  provided  by the contractors  to be very useful. 
Program  Guidance  - States believe the regions are generally  responsive  to their policy 
questions,  while Headquarters  takes too long to respond.  States need policy  and 
operational  instructions  much earlier  so they can implement  their own policy  and computer 
system changes  in a timely manner. 
Management  Inforrntion - States question the usefulness  of the data being collected  since 
they receive  little feedback  from ACF on the analysis of this information.  Neither  the 
States nor the regions  believe ACF is adequately  validating  the data. 
ii  TABLE  OF  CONTENTS

EXECUTIVE  SUMMARY

INTRODUCTION  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1

FINDINGS  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3

 
MONITORING  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...3

  Field Review Guides  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...3

 
Field Review Reports  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...3

  On-site Visits  . . . . . . . . . . . . . . . . . . . . . . . . . ..OO.  . . . ..0<.03

 
Regional  Office State Specialist  . . . . . . . . . . . . . . . . . . . . . . . . . ...4

 
TECHNICAL  ASSISTANCE  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...4

  Technical Assistance Needs.  . . . . . . . . . . . . . . . . . . . . . . . . . . . ...4

  Technical Assistance Contractor  . . . . . . . . . . . . . . . . . . . . . . . . . ...4

  PROGRAMGUIDANCE.  . . . . . . . . . . . . . . . . . . . .  . . . . . . . . . . . ...5

 
MANAGEMENT  INFORMATION  . . . . . . . . . . . . . . . . . . . . . . . . . . ...5

CONCLUSIONS  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...-...  6

APPENDICES

A:  PERSPECTIVES  OF STATE OFFICIALS  . . . . . . . . . . . . . . . . . . . . . .  A-1

B:  PERSPECTIVES  OF ACF REGIONAL OFFICES  . . . . . . . . . . . . . . . . . . B-1
 INTRODUCTION 
PURPOSE 
To provide  an assessment  of the Administration  for Children  and Families  (ACF) 
monitoring  and technical  assistance  activities  for the Job Opportunities  and Basic Skills 
Training  (JOBS) program  based on the perspectives  of States and ACF regioml  offices. 
BACKGROUND 
The Family  Support Act of 1988 established  the JOBS program  for recipients  of Aid to 
Families  with Dependent  Children  (AFDC) payments.  This program  is designed to assist 
welfare  recipients  to become  self-sufficient  through training,  education,  work experience, 
and support  services.  The ACF administers  the JOBS program  through  its Headquarters 
and 10 regional  offices.  In this role,  they monitor  and provide  technical  assistance  to 
States. 
METHODOLOGY 
In recognition  of the differing  roles and responsibilities  of ACF Headquarters  and regional 
office components,  we reviewed  the JOBS program  in three phases.  We did this to more 
accurately  reflect  the overall performance  of each component  in providing  monitoring  and 
technical  assistance  for the JOBS program  and to obtain the States observations  and 
perspectives  concerning  this performance. 
In phase  one,  we reviewed  the statute,  written policies,  processes,  and products  developed 
by ACF Headquarters  for monitoring  and providing  technical  assistance  in the JOBS 
program.  We conducted  an on-site visit at ACF Headquarters  and interviewed  members 
of the agencys  staff responsible  for the following key areas:  field reviews,  on-site visits, 
State plans,  data collection,  and the technical  assistance contract.  We also interviewed 
members  of Washington,  D. C. -based advocacy organizations  that have an active interest 
in the JOBS program.  The results are presented  in our report  entitled  JOBS Monitoring 
and Technical  Assistance:  Headquarters  Role,  (OEI-07-92-O0380) issued August  1993. 
In phase two,  (Regional  Offices Perspectives),  we examined  ACFs program  instructions 
and regional  offices documentation,  including policies,  directives,  memoranda,  telephone 
contacts,  on-site visit reports,  States report  summaries,  and other instructions  used in the 
region  or disseminated  to States.  We interviewed  management  and operational  staff in the 
10 regional  offices that administer  the JOBS program. 
In phase three,  (States Perspectives),  we examined  States program  documentation, 
including  regional  policies,  directives,  memoranda,  telephone  contacts,  on-site visit 
reports,  program  report  summaries,  and other materials  used by them or created  to 
summarize  their JOBS program  activities.  We interviewed  program  staff in 10 States, 
9 of which were selected because they are the State in which the ACF regional  office is 
1  located.  In Region  VII, Kansas City,  we selected Nebraska,  since we had previously 
contacted  Missouri  as part of our preinspection  research. 
Upon completion  of the Headquarters  review,  ACF requested  that we delay action on our 
report  findings and recommendations  related  to monitoring  activities until we had 
completed  our review  of the regional  offices and States.  We agreed.  We have now 
completed  the remaining  phases of the study.  This report  contains the synthesis of what 
we found,  as well as the Regional  offices and States perspectives  of ACFs monitoring 
and technical  assistance  efforts. 
We conducted  our review  in accordance  with the Quality Standards for  Inspections  issued 
by the Presidents  Council  on Integrity  and Efficiency. 
2  FINDINGS 
We are providing  here  a synthesis of our findings covering  the regional  offices and 
States perspectives  conceding  ACF'smonitoring,  tectical  assistance,  program  guidance 
and management  information  activities.  More  in-depth information  is contained  in the 
Appendices  A and B of this report. 
MONITORING 
Field Review  Guides 
The ACF regional  offices  noted great improvement  in the release of the 1993 and 
1994 field review guides as compared to earlier years.  However,  they prefer that 
Headquarters  release the guides even earlier. 
Seven offices would like to receive these guides in the first quarter  of the fiscal year.  The 
remainder  would prefer  to receive the guides before the start of the fiscal year,  preferably 
in July or August.  All regions believe that releasing  the guides earlier  would assist them 
in their work planning  and strategic planning process. 
Field Review  Repotis 
For many States,  ACF regional office diversity in methods for sharing field review 
information  has adversely impacted on the timeliness  and usefulness  of this material. 
States report  that regional  offices use diverse methods for releasing  field review  reports, 
for soliciting  their comments,  and for following up on their corrective  actions to report 
findings and recommendations.  Our review  of ACF Headquarters  supports that it has not 
established  national  guidelines  for the dissemination  of reports,  for soliciting  States 
comments,  and for conducting  follow-up.  The review  of regional  offices also confirmed 
that they have diverse  methodologies  for carrying  out these fimctions. 
On-site  Visits 
Most States found the regional office on-site visits to be beneficial.  However,  they 
prefer that these visits be more systematically  planned and be conducted  more 
frequently. 
Both States and regional  offices consider  on-site visits a useful tool for monitoring,  for 
offering  assistance  to States,  for follow-up,  and for educating  their staff.  States also see 
these visits as valuable  in establishing  good working relationships  with the regional  office 
staff. 
The frequency  of these visits varies,  due mainly to limited funding and the distance  from 
the regional  offices to the States capitals.  States prefer  more regular  and frequent  visits. 
3  Regional  Office State Specialist 
Many regions  and States consider the regional office  State specialist  position as vital 
to ensure  good relations. 
All regional  offices have assigned State specialists who are the primary  contact  with the 
States.  They consider  the specialists to be field-oriented  positions.  Further,  they believe 
that the specialists  need to be out in the States to know what is going on in the JOBS 
programs. 
TECHNICAL  ASSISTANCE 
Technical Assistance  Needs 
Some States believe that ACF is not consistently meeting their technical  assistance 
needs. 
Six States believe  that their technical  assistance needs are not being met or are only 
somewhat being met by ACF Headquarters  and regional  offices.  The remaining  four 
States believed  that ACF was meeting their technical  assistance  needs.  ACF Headquarters 
believes  they are attempting  to meet most of the States technical  assistance  needs, 
primarily  through  the training  and materials  provided  by the technical  assistance 
contractors.  Most regional  offices also believe they are attempting  to satisfy most of the 
States needs and consider  themselves  to be the primary  source for providing  this 
assistance. 
Technical Assistance  Contractor 
Regional  offices  and States believe that the technical  assistance contractors  training 
was not useful. 
All regional  offices and seven States stated that the technical  assistance  contractors 
training  was not useful.  Nine States favor the use of technology transfer  to meet their 
training  needs.  This method provides  partial  Federal  funding to facilitate  cross-State 
training  with States that are using innovative and successful program  approaches.  We 
found in the Headquarters  review that the full potential  of the technical  assistance 
contractors  was not realized.  All regional  offices commented  that the contractors  training 
was not timely or appropriate  for States needs. 
4  PROGRAM  GUIDANCE 
Most States believe that the regional offices  are responsive to policy questions. 
However,  they believe that problems remain with the timeliness  of Headquarters 
responses  and in the release and timing for regional office interim responses. 
All regions  believe  that Headquarters  responsiveness  to policy  issues is problematic, 
particularly  as related  to providing  interim  status updates for these issues.  While most 
regions  provide  timely responses  to States questions,  they vary in the method  and timing 
for release  of interim  responses,  especially  for issues submitted to Headquarters  for their 
review. 
Most States believe  that ACF has not issued action transmittals  timely.  As a result,  States 
do not have adequate time to implement the needed policy  and computer  system changes. 
In the Headquarters  review,  we found that ACF did not allow sufficient  lead time for 
States implementation  of action transmittals,  particularly  those related  to computer 
systems and data reporting.  In some cases,  States were required  to retroactively  capture 
and report  data. 
Regional  offices noted that Headquarters  program  instructions  are improving.  However, 
they believe  that instructions  related to data collection  and some policy areas need to be 
released  earlier  to allow States time to plan and implement  the changes. 
MANAGEMENT  INFORMATION 
All regional  offices  and States commented  that they receive little feedback  from ACF 
Headquarters  on the analysis of States data.  Therefore,  they question the usefulness 
of the data being collected. 
Most regions  believe  they are not adequately  validating  States data and computer  systems 
for accuracy  and consistency.  Although validation  of data is required  as part of the field 
review  process,  most regions believe that efforts to do so are not adequate.  In addition, 
both regions  and States commented  that ACF has not routinely  disseminated  management 
information  in a usefid format to those who need it. 
Most regions  are adopting varied and innovative  approaches  for data analysis  to enhance 
their monitoring,  technical  assistance,  and consultative  functions  with States.  However, 
they believe  that lack of Headquarters  guidance,  access to data,  and standardization  of 
analysis techniques  is impeding their efforts. 
5  CONCLUSIONS 
As noted earlier,  ACFasked  iftieycould  delay responding  totiemotitoring  section of 
our Headquarters  report until we completed  our review of regional  offices and States. 
We agreed. 
Now that our activities  are concluded,  we are pleased to note that real progress  has been 
made in some areas,  particularly  in the more timely release  of field review  guides and 
protocols.  Nevertheless,  our State and regional  respondents  confirm  the need for 
additional  improvements.  We therefore  look forward  to ACFS response  to the earlier 
findings and recommendations  on monitoring,  which were published  in our report  on 
Headquarters  activities.  We hope these reports  will be useful to ACF in preparing  their 
response  as well as their implementation  plan for all of the recommendations  previously 
made. 
Our respondents  have also raised concerns  about additional matters,  particularly  related  to 
technical  assistance  and management  information.  We hope ACF will address these 
concerns  as well in their response. 
6  APPENDIX  A 
PERSPECT~S  OF STAZl!lOFFICIALS 
The following  is a catalogue  of the perspectives  and observations  of staff from  10 States 
staffs who are involved  in the operation  and administration  of the JOBS program.  We 
summarized  the States comments  to reflect the general  tone of the issues voiced during 
our review. 
MONITORING 
FIELD REVIEWS 
States believe that field report information  is of maximum  usefulness  when it is 
disseminated  no later than 90 days following the review.  They experience difficulty 
in developing  and implementing  corrective  action plans when report recommendations 
are not issued in a timely manner. 
States described  the varied processes  ACF uses to share the results of field review 
activity.  These methods  are shown in Table  1. 
Table 1 
Method of sharing draft report findings 
and recommendations  with States 
Method  used  Number  of States 
Informal  sharing  of findings and

recommendations--no  draft report  disseminated  2

Draft  report  disseminated  5

No informal  sharing--no  draft report

disseminated  3

All States confirmed  that regional  offices are routinely  conducting  exit conferences  at the 
conclusion  of the field review.  Nevertheless,  they believe that delays in disseminating 
field review  reports  adversely  affects their ability to utilize the information  to improve  the 
operation  of the JOBS program.  This is particularly  true where expenditures  of States 
funds are involved and where they disagree  with the information  provided  during the exit 
conference.  The elapsed times for States receipt  of these reports  are shown in Table 2 
for Fiscal  Years  (FYs)  1990-1992. 
A-1
 Table 2

Elapsed time from the field review to

States receipt of the report

for FYs 1990-1992

Elapsed  time from the field review  Number of States 
3 months or less  5 
6- 12 months  2 
Over  1 year  3 
To be  of maximum  usefulness,  all States  favor  report  findings  and  recommendations  be 
shared as part of an exit conference  which is followed by a written report  that is 
distributed  no later than 90 days following the review.  States preference  for receipt  of 
these reports  are shown in Table 3. 
Table 3 
States preference  for receipt 
of the field review report 
Elapsed time from the field review  Number of States 
Within  30 days  3 
Within 60 days  3 
Within  90 days  4 
States report  that regional  offices use varied methods to disseminate  field review  reports. 
In addition,  States note that regional  offices differ  in the methods used to obtain their 
comments  to draft and final reports  (see Table 4). 
A-2
 Table 4 
Methods for obtaining States comments to 
field review reports 
Draft report methods used  Number of States 
Informal  sharing  of findings

and recommendations  through  a visit

or a telephone  call--no formal

comment  period.  1  2

Comment  period provided  2  5

No informal  sharing--no  comment period

provided.  3

Final report methods used  Number of States 
Comment  period provided  12  9

No informal  sharing--no  comment period

provided.  1

1	 Tworegional  offices donotformally  solicit S@tes comments  to the 
draft report,  butdoprovide  aformal  comment period  fortiefinal  repoti. 
2	 Five regional  offices provide  comment periods  for both draft and 
final reports. 
States report  a variance  in regional  offices notification  of issues needing corrective  action 
and their follow-up  activity.  While eight States believe  that regional  offices responses  to 
their corrective  action plans are good, the remaining  two States are not notified  of items 
needing  corrective  action and have not been involved in regional  office follow-up 
activities. 
ON-SITE  VISITS 
Most States believe that on-site visits by regional office staff are beneficial.  While all 
States receive these visits,  the frequency varies. 
States reported  a variance  in the frequency  of regional  office on-site visits as shown in 
Table 5.  All States receive  at least one annual visit, while some receive  more than 
four visits per year,  in addition to scheduled field reviews. 
A-3
 Table 5 
Frequency  of regional office on-site visits 
Frequency  Number of States 
4 or more visits per year  1  6 
2 visits per year  1 
1 visit per year  3 
1	 These States are located in proximity  to the regional  office.  However, 
we found that regions travel to remote portions  of these States and to 
other States less frequently  due to limited travel funds. 
Eight States consider  regional  on-site visits to be beneficial.  All of these receive  either 
formal  or informal  feedback  summarizing  on-site visit issues. 
REGIONAL  OFFICE STATE SPECIALIST 
Many States consider the regional office State specialist position as vital to ensure 
good relations. 
Six States rated the State specialist as effective,  while three States characterized  the 
specialist  as improving.  Only one State considered  the specialist to be ineffective. 
States that rated their  specialist  as effective  or improving  listed the following  traits  as 
desirable  for a good State specialist: 
  openness

  helpfulness

  accessibility

  responsiveness

  knowledgeable  about the States program

  serves as an information  resource

  sometimes  acts as the States advocate

Only one State rated the State specialist as ineffective.  This State received  only one 
on-site visit per year  (Table 5).  They commented  that the specialists  visits were not well 
planned  and he did not provide  visit summaries. 
A-4
 TECHNICAL  ASSISTANCE 
TECHNICAL  ASSISTANCE  NEEDS 
Some States perceive that ACF is not consistently meeting their technical  assistance 
needs. 
Six States believed  that their technical  assistance  needs are not being met or are only 
somewhat  being met by ACF Headquarters  and regional  offices.  The remaining 
four States believed  that ACF was meeting  their technical  assistance  needs. 
TECHNICAL  ASSISTANCE  CONTMCTOR 
Many States believe that the technical  assistance contractor training  was not useful. 
Seven States asserted  that the contractor  training  was not useful or practical  for their 
program.  Several commented  that the major benefit from the technical  assistance 
conferences  (workshops)  was the ability to informally  discuss issues (network)  with staff 
from other  States. 
Most States favor use of technology transfer rather than a technical  assistance 
contract to meet their training  needs. 
Nine States commented  that they would like ACF to establish  and fund a technology 
transfer  among States.  This concept,  which previously  existed under AFDC,  enabled 
State staffs to travel to other States having the expertise  to meet their  specific training 
needs. 
The one State not supporting  this concept expressed  concern  about limited  State funds 
available  to pay for interstate  training.  In addition,  they were concerned  about the loss of 
their employees  services during the training  period. 
Most States believe that regional offices  are responsive to their policy questions. 
However,  they believe that problems remain with the timeliness  of Headquarters 
responses  and in the release  and timing for regional office interim responses. 
Nine States commented  that regional  offices are responsive  to their policy  needs. 
However,  six of these States believe  that Headquarters  has not provided  timely responses 
to their  questions,  many of which had far reaching  policy or budget implications. 
Nine States commented  that the ideal regional  offices response time for a policy  question 
is 30 days or less.  The remaining  State felt that a response within 60 days would meet its 
needs.  Seven commented  that the ideal Headquarters  response time for a policy  question 
was 60 days or less.  Four  of these would prefer  a response within 30 days or less. 
A-5
 Five Swtesprovided  the following examples  ofdelayed  Headquatiers'  responsiveness  to 
policy  issues: 
	 A question regarding  work supplementation.  Submitted  inthe  Fall of 1992, they 
received  an answer in June 1993 (no interim response  issued by the regional 
office). 
	 A question concerning  the use of Housing and Urban Development  funding for 
Community  Development  Block Grants as Federal  matching  funds for JOBS--the 
response  took seven months. 
	 A question  on whether unemployed  parents  and 16-17  year old AFDC  youth can 
be included  in the definition  and calculation  of target groups.  The State submitted 
the question  in March  1992, no response received  (as of July  1993). 
	 A request  to change the definition  of a target group from being on assistance 
36 out of the last 60 months to being on assistance  24 out of the last 60 months. 
The request  was submitted  in September  1990.  They received  the response  in 
July  1992.  The State expressed  some displeasure  with the lateness of an answer, 
since the issue had an impact on State budget and policy. 
	 A question  concerning_whether  the JOBS program  can pay for child care  in 
facilities  that provide -&ligious instruction-was  submitted  in 1988.  The State has 
not received  a formal  written response from Headquarters.  While they did get an 
informal  (verbal)  response  from the regional  office,  the State does not consider  this 
to be an official answer to their question. 
PROGRAM  GUIDANCE 
Most States believe that ACF has not issued action transmittals  in a timely manner to 
allow them adequate time to implement the needed policy and computer  system 
changes. 
Nine States commented  that action transmittals,  particularly  in the area of data reporting, 
have not been released  with enough lead time to allow States to implement  the needed 
changes  to policy  and computer  systems.  They believe that problems  remain,  particularly 
as related  to the issuance of policy on the changes to the unemployed  parent program 
which was effective  on October  1, 1993.  As of November  1993, States had not received 
final policies  and reporting  instructions  for these changes. 
States vary widely  in the amount of advance notice needed to implement  policy  changes. 
They commented  that they need adequate  advance notification  to plan and implement  these 
changes,  to train  staff,  and to issue their policy and regulations.  States preferences  for 
this notification  are shown in Table 6. 
A-6
 Table 6 
States preference  for advance notification 
of policy changes 
Time frame  Number of States 
1 
2 months 
1 
3 months 
4 
6 months 
1 
9 months 
2 
Over  1 year 
1 
Depends  on the nature  and extent of the change 
States also vary widely in the amount of advance notice needed to implement  computer 
system changes.  They commented  that they need adequate advance notification  to plan, 
implement  these changes,  and to train staff. Their preferences  for this notification  are 
shown in Table 7. 
Table 7 
States preferences  for advance notification 
of computer changes 
Time frame  Number of States 
1 
2 months  1

3 
3 months

1 
6 months 2

1 
9 months

2 
12 months  12

3 
Over  1 year

1 
Depends on the nature and extent of the change

1	 This State believed  that two months was adequate for minor  changes 
but that  12 months would be needed for major changes.  The State is 
listed for both time frames. 
2	 This State believed  that six months was adequate for minor  changes 
but that  12 months would be needed for major changes.  The State is 
listed for both time frames. 
A-7
 MANAGEMENT  INFORMATION 
All States commented  that they receive little feedback  from ACF on the analysis of 
States data.  They question the usefulness of the data being collected. 
None of the States are receiving  analysis of data beyond the recipient  participation  and 
target  group rates.  They have not received  an ACF analysis of the JOBS Program 
Participant  Data Collection  Form FSA-108 data, 
A-8
 APPENDIX  B 
PERSPECT~S  OF ACF REGIONAL Ol?FICES 
The following  is a catalogue  of the perspectives  and observations  of staff from  10 ACF 
regional  offices who are involved in the administration  and oversight  of the JOBS 
program.  We summarized  the regional  office comments to reflect the general  tone of the 
issues voiced during  our review. 
MONITORING 
FIELD REVIEW  GUIDES 
The regional  offices  noted great improvement  in the release of the 1993 and 1994 field 
review  guides as compared to earlier years.  However,  they prefer that Headquarters 
release the guides even earlier.  Most offices would like to receive these guides in the 
first quarter of the fiscal year. 
The regions  received  the final review guides for FYs  1992,  1993, and 1994 in June  1992, 
February  1993, and October  1993, respectively.  The regions noted that the  1994 guides 
were essentially  a reissuance  of the 1993 documents.  Most believe that the timing  for the 
guide release  has improved  considerably  over this three year period.  Table  1 summarizes 
regional  offices responses  concerning  the impact of the 1992 and 1993 guides on their 
office operations. 
Table 1 
Impact of the release of the 1992 and 1993 guides 
on regional office operations 
Number of regions 
Response  FY 1992  FY 1993 
Adverse  impact
 7 2 
No adverse  impact
 2 8 
New staff not involved in FY  1992 review

1 
Regions reporting  an adverse  impact commented  that the  late  release  of these  guides 
resulted  in delaying  field reviews until the last three months of the fiscal year.  This 
created  a burden  for them in terms of workload  and a time  crunch for completing  the 
reviews  during  the year.  Several regions noted that guides should be released  before  the 
fiscal year or in the first quarter  of the fiscal year as shown in Table 2. 
B-1
 Table 2 
Regional offices ideal time frames for 
receipt of field review guides 
Time frame  Number of regions 
Before  start of the fiscal year 1  3 
In the first quarter  of the fiscal year 2  7 
1	 Three  regions prefer  to receive the guides before the start of the fiscal 
year,  preferably  in July or August. 
2	 Three  of these regions would like to have a working  draft of the guide 
by August or September 
All regions  believe that releasing  the guides earlier  would assist them in their work 
planning  as well as their  strategic planning process. 
FIELD REVIEW  REPORTS 
The regions  use different  methods to share report information  with States and to 
obtain comments  on draft and final reports. 
Table  3 summarizes  the various methods used to disseminate  this information. 
Table 3 
Methods of sharing report findings 
and recommendations  with States 
Methods  used  Number of regions 
Informal  sharing of findings 
and recommendations--no  draft report  disseminated  2 
Draft  reports  disseminated  5 
No informal  sharing--no  draft report 
disseminated  3 
Regions  also report  using diverse methods to obtain States comments  on draft  and final 
reports.  This is shown in Table 4. 
B-2
 Table 4 
Methods for obtaining States commentson 
field review reports 
Draft report methods used  Number of regions 
Informal  sharing  of findings and 
recommendations  through  a visit 
or a telephone  call--no formal 
comment  period  1  2 
Comment  period provided  2  5 
No comment  period provided  3 
Final report methods used  Number of regions 
Comment  period provided  12  9 
No comment  period provided  1 
1	 Two regional  offices do not formally  solicit States comments  to the 
draft report,  but do provide  a formal comment period  for the final 
report. 
2	 Five regional  offices provide  comment periods  for both the draft and 
the final reports. 
ON-SITE  VISITS 
All regions  consider on-site visits to be a useful tool for monitoring,  offering 
assistance  to States,  for follow-up,  and for educating their staff. 
The frequency  of regional  office on-site visits varies,  due mainly to limited funding  and 
distances  from the regional  offices to the States capitals.  Table 5 provides  a summary  of 
the frequency  of these visits. 
B-3
 Table 5 
Frequency  of annual on-site visits 
Number  of  Frequency  of visits  Frequency  of visits 
regional  offices  to close States 1  to distant States 2 
1  Weekly to Monthly  Less than annual

1  Bimonthly  NIA

1  Bimonthly  Less than annual

2  Quarterly  Quarterly

1  Quarterly  Semiannually

1  Quarterly  None performed

1  Three to five  Three to five

2  Annually  Less than annual

1	 Close States are defined as being within one days travel, 
no overnight  stay required  to conduct a visit. 
2	 Distant States are defined as requiring  an overnight  stay 
by regional  office staff to conduct a visit.  An N/A designates 
that the region(s)  have no States where an overnight  stay would 
be required. 
REGIONAL  OFFICE STATE SPECIALIST 
All regional  offices  have assigned State specialists who are the primary contact with 
the States for the JOBS program.  They consider the specialists to be field-oriented 
positions. 
The regional  offices believe that the specialists have to be out in their  States to know what 
is going on in the JOBS programs.  Regions commented  that limited travel  fi.mds are 
impeding  specialists  from making periodic  visits during the year. 
TECHNICAL  ASSISTANCE 
TECHNICAL  ASSISTANCE  NEEDS 
Regional  offices  commented  that they are the primary source for addressing  States 
technical  assistance  needs.  Eight out of ten believe they are addressing these needs. 
The remaining  regions are not providing technical  assistance. 
Regional  offices believe  that technical  assistance  is an important  activity.  The primary 
sources of information  for six of the regions are field reviews  and staff contacts.  The 
other four regions  found that surveys of States and staff contacts are the most usefd  for 
them in identifying  technical  assistance needs. 
B-4
 Despite  regional  offices efforts,  they believe that States continue to have significant 
technical  assistance  needs (see Table 6). 
Table 6 
Regional office views regarding 
States technical  assistance needs 
Technical  assistance needs  Number of regions  1 
Understanding  legislation,  regulations  and policy  6 
Data collectionheporting  4 
Goal setting and component  effectiveness  3 
Unemployed  parents  program  3 
Computer  systems development  2 
Facilitating  the exchange  of information  with other States  2 
Participation  rates,  target groups,  and teen parents  2 
Simplifying  funding streams/budget  issues  2 
Better coordination  among Federal  agencies  1 
Child care issues  1 
Market  rate survey  1 
Orientation  of new States JOBS program  managers  1 
Writing  a good contract  1 
1 All regional  offices listed multiple technical  assistance  needs 
Examples  of how regions are providing  or sponsoring technical  assistance  to States 
One region assisted a State whose legislature  wanted information  on the 
effectiveness  of jobs  search as a component  of the JOBS program.  The region 
obtained  a study that showed job  search was cost effective over the short term. 
However,  it was not as effective as education  and training  programs  over the long 
term. 
Through  its monitoring  effort,  one region brought  State JOBS directors  and ACF 
Headquarters  staff together.  The ACF staff provided  definitions  for terms used in 
data collection. 
One regions planning process will identify five items the individual  States need in 
the coming year.  Some of these are common to other States in the region.  Others 
will be specific to individual  States. 
B-5
 TECHNICAL  ASSISTANCE  CONTRACTOR 
All regions  reported that the contractors  assistance to States during program 
implementation  was not timely or appropriate.  Only three judged  the technical 
assistance  contractors  training materials  and workshops as being useful to States. 
All regions are concerned  with Headquarters  response times to States policy  questions. 
Regions report  a wide variance  in the timeliness  of Headquarters  responses  to States 
policy  questions as shown in Table 7. 
Table 7 
Regions reporting of Headquarters  response times 
for States policy questions 
Response time ranges  Number of regions 
30 days to several months
 3 
30 to 45 days
 1 
60 days or less
 1 
90 days
 1 
90 to 120 days
 1 
4 to 6 weeks
 1 
4 to 6 months
 1 
4 to 8 months
 1 
A majority  of the regions  generally  prefer  a 30 day time frame as the outside limit for a 
response  from Headquarters  (see Table 8). 
Table 8 
Regions  preference  for Headquarters  response times 
to States policy questions 
Response times  Number of regions 
Not to exceed  30 days 1  7 
30 to 60 days  2 
45 to 60 days (closer to 45 days)  1 
1	 One region commented  that a continuing  dialogue  (no time expressed) 
is needed between  the region and Headquarters  if the response  will 
take a relatively  long time. 
B-6
 Regions  report use of diverse methods and time frames for releasing  interim 
responses  to questions from States. 
The regions  vary in the methods and timing for providing  interim responses  to States (see 
Table 9). 
Table 9 
Methods  and time frames for issuing interim responses 
Method  Number of regions 
Interim  response  if final  reply  will  take  a

long  time  (period  not  defined)  4

Interim  response  by  telephone  when  referred

to Headquarters,  no  subsequent  reply  will  be issued  1

Interim  responses  every  30 days  1

Respond  verbally  or  in writing  (period  not  defined)  1

Periodic  telephone  follow-up  with  Headquarters

and  furnish  update  to State  (period  not  defined)  2

30 day  telephone  follow-up  with  Headquarters  and

Ii,umish update  to  State  1

PROGRAM  GUIDANCE 
All regions believe that Headquarters  program instructions  are improving. 
However,  material  related to data collection  and some policy areas could be more 
timely. 
Five  regions  commented  that  the  lack  of timeliness  of instructions  pertaining  to data 
reporting  and  systems  resulted  in States  having  to collect  and  report  the  information 
retroactively.  States  agreed  that  these  instructional  delays  made  it difficult,  if not 
impossible,  to collect  and  report  the  needed  information  since  their  information  systems 
were  not  set up  to capture  this  data. 
B-7
 MANAGEMENT  INFORMATION 
Most regions believe they are not adequately validating  States data and computer 
systems for accuracy  and consistency,  despite the required validation  performed  as 
part of the field review process. 
All regions  are validating  tie  data on the JOBS Program  Participant  Data Collection  Form 
FSA-108,  which is a requirement  under the field review process.  However,  seven regions 
believed  that this validation  is not adequate to assure accurate  State data. 
Most regions report use of varied and innovative approaches  for data analysis to 
enhance their monitoring,  technical  assistance and consultative  functions with States. 
However,  they believe that lack of Headquarters  guidance,  access to data,  and 
standardization  of analysis techniques  is impeding their efforts. 
While eight regions  are performing  limited analysis of States data,  they believe  this role 
is not adequately  supported  by ACF Headquarters.  Nine regions  believe that 
Headquarters  should provide  greater  access to States data,  and assist them in its analysis. 
B-8