Department of Health and Human Services 

OFFICE OF 

INSPECTOR GENEW 

EVALUATING HEAD START EXPANSION 

THROUGH PERFORMANCE INDICATORS 

MAY 1993 OE.I-B91-(X)762 .. 
EXECUTIVE SUMMARY


PURPOSE


This inspection used selected performance indicators to assess the impact of Head


Start expansion on grantees and delegates.


BACKGROUND


Head Start operates on the premise that children are best prepared for success in


school when they and their parents participate in a comprehensive program that


addresses their educational, economic, social, physical, and emotional needs. In


addition to providing children with classes and health services, Head Start addresses


the needs of the entire family.


Both Congress and the Administration are committed to expanding Head Start. Since


Fiscal Year (FY) 1990, the total funding for Head Start has increased more than


$1 billion, and the number of children served has increased by almost 300,000.


Within the Department of Health and Human Semites (HHS), the Administration for


Children and Families (ACF) is responsible for administering Head Start. The ACF,


the Assistant Secretary for Management and Budget (ASMB), and the Assistant


Secretary for Planning and Evaluation (ASPE) requested that the Office of Inspector


General (OIG) review the implementation and status of Head Start expansion because


they were concerned that rapid expansion might jeopardize the quality of services.


This report is one in a series prepared by the OIG on this subject.


From May through July 1992, we conducted either on-site visits or telephone


interviews with a random sample of 80 Head Start grantees and delegates using


structured discussion guides. In a companion report entitled “Head Start Expansion:


Grantee Experiences” (OEI-09-91-O0760), we reported the results of these interviews.


According to the grantees, expansion posed problems for them in such areas as child


enrollment, facility acquisition, staffing, transportation, and social services.


Nevertheless, grantees told us that they were still able to meet all of Head Start’s


performance standards. This is consistent with the data that grantees report to ACF


in their annual Program Information Reports (PIRs). Based on the following findings,


however, we question the accuracy of some of these self-reported data and


assessments.


In addition to the intemiews, we reviewed health, education, and social semice files for


more than 3100 children as well as other performance data which were maintained by


the 80 sampled grantees. We assessed grantee performance by applying 18 specific


performance indicators that we developed with input from ACF’S Head Start Bureau,


ASPE, ASMB, Head Start grantees and delegates, program experts, and researchers.


In most cases, the performance indicators mirror ACF’S performance standards. We


primarily used ACF’S policy guidance to grantees on the performance standards to


i 
assess grantees’ ability to provide services to children and families and to measure the 

quality of sewices provided in general. The results of this assessment are contained in 

this report. 

FINDINGS 

The original purpose of this inspection was to assess the impact of expansion on Head 

Start grantee performance. We did not find any statistically significant difference in 

grantee performance as a result of expansion. We did, however, find that the level of 

grantee performance as measured by our indicators was considerably lower than the 

level of performance reported by grantees and published by ACF. Because of 

(1) inadequate grantee record keeping, (2) the lack of specificity in the Head Start 

performance standards, and (3) the fact that many grantees disregard ACF policy 

guidance, we are unable to determine if the program and performance data 

weaknesses that we found reflect serious deficiencies in the quality of services 

provided by Head Start. Our specific findings are: 

l%e ACF’S l%ogram Information Report (PIR) data do not accurate~ reflect information 

iia childrenk health recomk 

During our review of grantees’ files, we found that grantees are not providing medical 

and dental services and follow-up to the extent that they report in their PIRs. 

Children may not be jWy immunized before they leave the Head Start program 

Our review of health files and all other available information provided by grantees 

revealed that only 43.5 percent of Head Start children were fully immunized as 

defined by ACF guidelines at the end of the 1991-92 school year. 

Gmntees fiequent~ do not identifi or addkxs fam”litx’ socidl sena”ceneeh 

Grantees frequently did not administer family needs assessments (FNAs), did not 

identi~ family needs or goals when conducting an FNA, rarely used Family Assistance 

Plans, and did not help families meet all or most of their needs. 

Gmntees’ jk and recortk jiequendy are incomplete, inconszken~ and dificult to rew”ew 

Some grantees have developed exemplary internal record keeping requirements, but 

others prematurely disposed of files and maintained children’s files in a haphazard 

manner. 

ii 
RECOMMENDATIONS 

T7w Secretary shouId convene a task foce to conduct a formal and thorough review of 

the management of the Head S~atiprogram 

The task force should analyze the respective roles and functions of the Head Start 

Bureau, ACF regional offices, grantees, and delegates. It should review how the Head 

Start program is currently organized and managed and how it can best be structured 

to administer the current and anticipated expansions. The task force should examine 

the planning, distribution of funds, grants management, oversight, technical assistance, 

management information systems, and performance standards that are currently used 

to manage the program. 

The Secretary has initiated a comprehensive examination of Head Start with a 

particular emphasis on quality and accountability. The Secretary’s initiative includes 

(1) immediately identi~ng poorly performing grantees, designing corrective actions, 

providing technical assistance, and ensuring that grantees that provide high quality 

services receive program funds and (2) conducting an in-depth review of the program 

with the help of an intellectually diverse expert study panel and using the results to 

design the Head Start program of the future. This plan addresses our concerns and is 

consistent with our recommendation. We anticipate that the examination will consider 

all of the issues that we have identified as well as the following recommendations: 

l%e ACF should create, dkn”bute, and mandate specific record keeping and children’s file 

maintenance ihstrucO”onsfor grantees 

Standardized record keeping and file maintenance would assure easy access to 

children’s files. Grantees could identify missing semices or needed follow-up more 

readily, and on-site reviews could be completed more expeditiously. 

% ACF shouki strengthen its monitoring of grantetx to better identifi and address 

pmblkm areas 

We believe that ACF should consider restructuring and condensing its on-site program 

reviews so that more time and resources can be devoted to monitoring and providing 

technical assistance to the most needy grantees. Furthermore, ACF should randomly 

sample PIR data to assure its validity. 

Ihe ACF should develop a modified FNA form that should be requtied of all Head Start 

-~ 

The Model FNA should be less cumbersome and be more conducive to an open 

dialogue with the family. The ACF should require grantees to use the revised FNA 

throughout the school year to track and follow up on family needs. 

111

AGENCY COMMENTS 

As discussed in the first recommendation, the Secretary has initiated a comprehensive 

review of the management of the Head Start program. 

We received written comments on the draft report from ACF, ASPE, and ASMB. 

The ACF generally concurred with our recommendations but was concerned about the 

presentation of some of the data and conclusions in our findings. They were 

particularly concerned about the immunization and social setices findings. We found 

their comments helpful. As a result, we have revised the immunization section and 

presented additional data to explain and clari~ our finding. In addition, we have 

provided a detailed response to all of ACF’S comments, including immunizations and 

social semices, in appendix C. The ASPE and ASMB provided editorial and technical 

comments and suggestions which have been incorporated in the final report. The 

complete text of the ASPE and ASMB comments can be found in appendix D. 

iv 
TABLE OF CONTENTS 

EXECUTIVE suMMARY 

INTRODLKX’ION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 

OIG PERFORMANCE INDICATORS 

FINDINGS . . . . . . . . . . . . . . . . . . . . . 

� Health services . . . . . . . . . . 

� 

� 

Immunizations 

Social services 

.** . . . . ..*. 

. . . . . . . . . . . 

� Grantee record keeping . . . . 

. ..0.. 

. . . . . . 

� *..** 

. . . . . . 

. . . . . . 

.*.*.. 

.0 . ...0 � . . . 

. ..**** � *.* 

� .0...0 ..0. 

. ..0... � ..* 

.*.*... .*.. 

.*...*. . . . . 

RECOMMENDATIONS AND AGENCY COMMENTS . . . . 

APPENDICES 

. . . . . . . � � ...0. . 

. ..0..0 .000 .00 � 

.0 .0.., � ..000. � 

� ..**.. . . ...*. . 

. . . . . . . .0 .0..0 � 

. . . . . . . � *...*. . 

. . . . . . . . . ...0. . 

k Performance Indicator Methodology 

B: How Head Start Program Performance Changed 

According to the OIG Performance Indicators 

c ACF Comments on the Draft Report and Detailed OIG Response 

. 

. 

� 

� 

6 

7 

7 

9 

11 

14 

16 

D: ASPE and ASMB Comments on the Draft Report 

— 
INTRODUCTION 

PURPOSE 

This inspection used selected performance indicators to assess the impact of Head 

Start expansion on grantees and delegates. 

BACKGROUND 

Head Start operates on the premise that children are best prepared for success in 

school when they and their parents participate in a comprehensive program that 

addresses their educational, economic, social, physical, and emotional needs. In 

addition to providing children with classes and health services, Head Start addresses 

the needs of the entire family. 

The Administration for Children and Families (ACF) is responsible for administering 

Head Start. The ACF, the Assistant Secretary for Management and Budget (ASMB), 

and the Assistant Secretary for Planning and Evaluation (ASPE) requested that the 

office of Inspector Genera] (OIG) review the implementation and status of Head 

Start expansion because they were concerned that rapid expansion might jeopardize 

the quality of semices. This report is one in a series prepared by the OIG on this 

subject. 

71ie Head Start R-ogram 

Head Start is a child development program funded primarily by the Federal 

government. At the headquarters level, ACF’S Head Start Bureau provides leadership 

and develops legislative and budgetary proposals for Head Start management and 

operations. In each regional office, ACF’S Head Start and Youth Branch monitors all 

Head Start programs, except Native American and migrant programs which are 

monitored by branches within headquarters. According to ACF, Head Start served 

approximately 622,000 children with Federal support of $2.2 billion in 

Fiscal Year (FY) 1992. 

Head Start programs are community-based, so agencies can respond to local needs 

and coordinate with other community organizations. As a result, program options, 

locations, and hours vary. The Federal government awards grants to a public or 

private nonprofit agency (called a grantee) to operate a Head Start program. A 

grantee may contract with one or more other public or private nonprofit organizations 

in the community (called delegates) to run all or part of its Head Start program. For 

the purposes of this report, we will refer to both grantees and delegates as “grantees” 

unless specific differences need to be noted. 

Head Start programs consist of four major components: health, education, social 

services, and parent involvement. Specific performance standards, which have been 

1 
published in the Code of 

grantees: 

� develop children’s 

Federal Regulations, require, among other things, that 

intellectual skills by encouraging them to solve problems, 

� provide children medical and dental screenings and 

� offer children nutritious meals and snacks, and 

� identi~ families’ social semice needs and work with 

to meet those needs. 

All grantees must comply with the performance standards. 

performance standards, ACF has issued guidance material 

intent of the performance standards and provides methods 

examinations, 

other community agencies 

In addition to the 

which elaborates on the 

and procedures for their 

implementation. The guidance material, however, is not mandated. 

Head Start Expanst”on 

Both the Administration and Congress are committed to expanding Head Start. The 

Head Start Supplemental Authorization Act of 1989 and the Dire Emergency 

Supplemental Appropriation of 1990 marked the beginning of expansion for all 

grantees and provided funding for the first two expansions. Since FY 1990, the total 

funding for Head Start has increased more than $1 billion, to an FY 1993 total of 

$2.779 billion. The Human Senices Reauthorization Act of 1990 continues the Head 

Start program through FY 1994 with the goal to serve all eligible children by 1994. 

The following table summarizes the portions of the funding increases that have been 

allocated for expanding enrollment: 

HEAD START EXPANSIONS, 1990-1993 

F-l Announcement Proposed Number Funds to Additional 

Year Date of Additional Expand Enrollment Expansion 

Children Funding 

Expansion I 1990 Februmy 6, 1990 37,500 $99,980,000 $51,335,000 

Expansion 11 1990 June 12, 1990 60,000 $165,315,000 None 

Expansion III 1991 undated 51,000 $159,447,000 $240,363,000 

Expansion IV 1992 Februaty 19, 1992 38,500 $131,513,000 $118,487,000 

Expansion V 1993 December 17, 1992 100,000 $372,706,000 $201,779,600 

‘IOTAL 287,000 $92&9’61,000 $611,964,600 

2 
The additional expansion funding, identified in the chart on the preceding page, was


set aside for, among other things, quality improvement, salary enhancement, cost-of-


living increases, and training and technical assistance improvement. Grantees


generally use quality improvement funding to increase salaries and benefits, strengthen


.	 the social semice, parent involvement, and/or health components, improve semices to 

disabled children, initiate or improve family literacy programs, and/or otherwise 

enhance services to children and families. 

To receive the allocated funds, grantees submitted expansion proposals. These 

proposals specified planned objectives, such as the number of additional children they 

would enroll and the staff they would require to serve these children. The Head Start 

Bureau advised grantees to prepare proposals that would result in high quality services 

that fully comply with the Head Start performance standards. 

The ACF monitors compliance with Head Start performance standards primarily by 

conducting on-site reviews. During site visits, review teams assess compliance with the 

performance standards using the On-Site Performance Review Instrument (OSPRI). 

The Head Start Act requires that ACF review each grantee or delegate at least once 

every 3 years to measure compliance with the Head Start performance standards 

[42 U.S.C. 9836]. The Head Start Improvement Act requires that ACF review each 

new grantee or delegate after its first year and conduct follow-up reviews of all 

grantees when appropriate. 

In addition to conducting on-site reviews, ACF requires each grantee to report 

performance data annually using the Program Information Report (PIR). The PIR 

contains data that ACF can use to assess individual grantee and overall program 

performance. 

Concerns about Ezpansion 

This report is one in a series prepared by the OIG concerning Head Start expansion. 

A companion report, “Head Start Expansion: Grantee Experiences” 

(OEI-09-91-O0760), describes the problems that Head Start grantees and ACF staff 

experienced during the 1990 and 1991 expansions as well as concerns about their 

ability to handle future expansions. 

METHODOLOGY 

We selected a random sample of 80 regional and Native American Head Start 

grantees and delegates. We excluded migrant grantees because of the unique nature 

of their programs, and we conducted a separate study, “Migrant Head Start Grantees: 

Perspectives and Challenges” (OEI-09-91-00761), to assess their experiences with 

expansion. From a universe of the 50 States plus the District of Columbia, Puerto 

Rico, and the Trust Territories (hereafter referred to as “localities”), we randomly 

selected 8 localities with probability proportional to size with replacement. f% a 

result, California was selected twice. The total amount of Federal funding received in 

3 
~1991 determined thesize ofeachof the localities. The following table presents 

the localities selected, their probability ofselection based upon their total budget, and 

the number of grantees, original and adjusted, in each location: 

I 

State 

California (One)


California (Two)


Florida


Maine


Michigan


Puerto Rico


Texas


Washington


Sampled


All States


Total Budget, 

All Grantees 

$184,762,665 

$184,762,665 

$57,466,594 

$7,601,468 

$71,904,040 

$70,947,229 

$95,413,705 

$24,586,682 

$697,445,048 

$1,700,448,467 

I Probability I Number of Grantees 

of 

Selection Original Adjusted 

10.87% 133 95 

10.87% 133 95 

3.38910 49 49 

0.45% 13 13 

4.23% 80 80 

4.17% 34 28 

5.61% 85 60 

1.45% 29 26 

41.03% 

I 

Because not all grantees and delegates received expansion funds, not all were eligible 

for selection at the second stage of sampling. At this stage, we selected grantees and 

delegates using simple random sampling until we obtained 10 grantees or delegates 

within each locality who received expansion funds in FYs 1990 or 1991. 

The adjusted number of grantees in the above table represents the estimated number 

of grantees and delegates that received expansion funds based upon our sampling 

results. Projections used in this report are based upon this adjusted number of 

grantees. 

From May through July 1992, we conducted either on-site visits or telephone 

interviews with each grantee and delegate using structured discussion guides and 

reviewed children’s files and other performance data. For the children’s file review, 

we selected a random sample of 20 children enrolled during the 1988-89 school year 

and 20 children enrolled during the 1991-92 school year for each program. We 

reviewed health, education, and social sewice files for more than 3100 children as well 

as other performance data which were maintained by the 80 sampled grantees. 

P~onnance Indicaton 

we assessed grantees’ experiences with expansion based on specific performance 

indicators. The Head Start Bureau developed the initial list of indicators. We 

modified and expanded the list with input from ASPE, ASMB, Head Start grantees 

and delegates, program experts, and researchers. In most cases, the performance 

indicators mirror ACF’S performance standards. We primarily used ACF’S policy 

guidance to grantees on the performance standards to assess grantees’ ability to 

4

provide sexvices to children and families and to measure the quality of semices 

provided in general. See page 6 for a list of the indicators. 

At the completion of the fieldwork, we weighted all data to project to the universe of 

Head Start grantees and delegates that received expansion funds. We determined 

statistical significance by assessing overlapping confidence internals. After we 

completed our analysis, we conducted follow-up interviews with several grantees to see 

if we could ascertain the reasons for the discrepancies between our data and theirs (as 

reported in the PIRs). 

Appendix A contains a description of how we collected and reviewed grantee 

management and performance data. Appendix B compares grantee performance in 

1988-89 to 1991-92 using the OIG performance indicators. 

We conducted this inspection in accordance with the QuaZity Sfandard for Impectiorw 

issued by the President’s Council on Integrity and Efficiency. 

-5 
1

2

3

4

5

6

7

OIG PERFORMANCE INDICATORS 

.In@ator ~ Description Data Source ,. 

Percent of chil(irenmedicallyscreened Children’shealth file review 

percent of children receivingthe needed medical Children’shealth file review treatment 

Percent of children receivingdental exams Chiltlren’shealth file review 

Percent of children receivingthe neededdental Children’shealth file review treatment 

Percent of children fullyimmunized Children’shealth file review 

Programprovidesnutritious meals and snacks Grantee menu analysis 

percent of children for whomthe teacher has Children’seducationfile review, 

entered in the child’sfolderobservational teacher/educationcomponent 

commentsat least once a month files 

8 Averageclasssize Enrollment data, attendance 

record sample 

9 Percent of familiesfor whoma familyneetls Children’ssocialservicesfile 

assessmenthas been completed review 

Percent of families with identified needs for Children’s social services file 

10 whom the Head Start program has attempted to review, social services 

meet these needs component log and file review 

11 Percent of parents volunteering at least 50 hours Grantee interviews, volunteer 

per year and in-kind file review 

12 Percent of full enrollmen t Enrollment data, attendance 

data review 

13 Average vacancy time Grantee interviews, attendance 

data review 

14 Program is not at fiscal or program high risk ACF regional office data 

15 Child-to-staff (teaching, sacial service, health, Program-provided staff tally, 

parent involvement, total ) ratio enrollment data 

16 Number of volunteer trai,ningsand parent Grantee intewiews, volunteer, 

workshops training, and in-kind file review 

Children’s social semices file 

17 Average number of conta :ts with each parent review, social se~ices 

component log and file review 

18 Average daily attendance Attendance data review 

6 
FINDINGS 

The original purpose of this inspection was to assess the impact of expansion on Head 

Start grantee performance. Using the 18 specific performance indicators listed on the 

previous page, we reviewed records for pre- and post-expansion school years. We 

found no statistically significant differences in these indicators before and after the first 

three expansions. Appendix B contains tables comparing grantees’ performance in the 

1988-89 and 1991-92 school years. 

Although grantee performance was relatively stable in both school years, we found 

that the level of performance as measured by the OIG performance indicators was 

considerably lower than that reported in ACF databases and published reports. 

Because of (1) inadequate grantee record keeping, (2) the lack of specificity in the 

Head Start performance standards, and (3) the fact that many grantees disregard ACF 

policy guidance, we are unable to determine if the program and performance data 

weaknesses that we found reflect serious deficiencies in the quality of services 

provided by Head Start. Since the level of performance in both school years was 

lower than the performance indicated in ACF’S databases, we focused our analysis on 

these discrepancies. We present the results in this report. 

This report is based primarily on the data related to the OIG performance indicators. 

A companion report, “Head Start Expansion: Grantee Experiences,” describes 

problems that grantees reported enrolling children, obtaining facilities, hiring qualified 

staff, providing transportation, providing social se~ices, and planning for past and 

future expansions. Despite these problems, grantees indicated that they were able to 

meet ACF’S performance standards during expansion. This is consistent with the data 

reported in their PIRs. Our data, however, question the accuracy of some of these 

self-reported data and assessments. 

THE ACF’S PIR DATA DO NOT ACCURATELY REFLECT 

CHILDREN’S HEALTH RECORDS 

INFORMATION IN 

Grantees are not completing the minimum health requirements to the extent reported 

in ACF’S “Project Head Start Statistical Fact Sheet” from January 1992. The 

Statistical Fact Sheet reports aggregate data submitted by grantees in their PIRs. The 

PIR data are self-reported. The ACF does not veri~ or confirm the accuracy of PIR 

data. The following table illustrates the contrast between ACF’S statistics and ours: 

7 
ACCORDING TO THE OIG PERFORMANCE INDICATORS, 

PIR DATA ARE INACCURATE 

ACFS Statistical Fact 

Indicator Sheet, Janua~ 1992 OIG’S Review of Children’s Head Start 

(1990-91 Program Year) Health Files for 1991-92 Program Year 

Percent of “97percent of children 91.7 percent of children received a medical screen, 

children enrolled 90 days or more but only 53.8 percent received all required health 

receiving medical completed medical screen assessments.* Overall, 70.2 percent of the 

screens screening including al! of children received a medical screen within 90 days 

the appropn”ate tests. u of enrollment, 11.4 percent received one after 

90 days, and 10.1 percent received one during the 

previous year. The Head Start performance 

standards simply require grantees to ensure that 

children receive health screens. The ACFS 

guidance to grantees urges that the screenings be 

completed within 90 days of enrollment. The 

guidance also informs grantees that children who 

have received screens in the past 12 months do not 

need to be reevaluated. 

Percent of “97percent of those Of the children requiring follow-up medical 

children with tientified as needing treatment, 76.2 percent had all of their medical 

medical needs [medical] treatment needs met and 5.5 percent had only some of their 

who received the received treatment. ” needs met. Approximately 18.2 percent received 

needed treatment no follow-up treatment. 

Percent of “94.5percent of the 84.7 percent of children received a dental screen. 

children chi~dren enrol[ed 90 days While 67.6 percent of children received a dental 

receiving dental or more completed dental screen 90 days before or after enrollment, an 

screens exanukations. n additional 17.1 percent received the screen after 

90 days. In most cases, we found no 

documentation explaining why a child did not 

receive a dental screen. The performance 

standards require that each child receive a dental 

screen. The ACF’Sguidance to grantees states that 

the screen should be performed within 90 days of 

enrollment. 

Percent of “94.5percent of those Of the children requiring follow-up dental 

children with needing dental treatment treatment, 66.6 percent had aI1of their dental 

dental needs who received treatment. ” needs met and 7.4 percent had only some needs 

received the met. Approximately 26 percent received no followneeded treatment up treatment. 

* According to the Head Start guidelines, the required eiements include growth, speech, and 

immunization assessments and special needs identification annual~ and vkion testing heatig 

testing and hemoglobin or hematocrit determination evety 2 years. Because of the different 

methodr that grantees use to identifi special needs and lack of documentation about whether a 

child was in his second year in the program, we determined that a child received ‘alI appropn”ate 

tests” if he received the growth, speech, and immunuation assessments. 

8 
CHILDREN MAY NOT BE FULLY IMMUNIZED BEFORE THEY LEAVE THE


HEAD START PROGRAM


The Head Start performance standards require that grantees assess each child’s


immunization status and assure that each child receives all recommended


immunizations, as appropriate. To evaluate grantee compliance with this standard, we


used ACF’S published guidance to grantees which was developed by the Centers for


Disease Control and complies with the recommendations of the American Academy of


Pediatrics. Our review of health files and all other available information provided by


grantees revealed that only 43.5 percent of Head Start children were fully immunized


at the end of the 1991-92 school year, according to the following age-specific


guidelines:


\

,.., 

:. 

:... ACF’S IMMUNIZATION GUIDANCE TO GRANTEES 

~~lhe requtied immunizations include DPT (diphtheria, pertussis, tetanus), polio, MUR (measles, 

‘ mumps,rubella), and HIB (haemophilus injluenzae type b). The minimum number of doses by 

4-year-olds:	 5* DPT 

4* POL1O 

1 MMR 

1 HIB 

*~ a child h not immunized as an infaruand 

tvceives the last dose afkr /t& 4th birthday, he 

child tnay receive one less dose and be considered 

Jidly immunized 

Some children required several immunizations to be fully immunized. The following 

chart shows how many immunizations Head Start children required at the end of the 

1991-92 school year: 

HOW MANY IMMUNIZATIONS DID CHILDREN NEED 

TO BE FULLY IMMUNIZED? 

NumberofNeededImmunizations 

o 

1 

2 

3 or More 

PercentofChildren 

43.5 

19.8 

29.5 

7.2 

—. 9 
Obtaining the serial immunizations (DPT and polio) posed the most problems for 

grantees. Grantees had not provided ~ DPT immunizations during the 

1991-92 school year for 61 percent of the 378,000 children who were not fully 

immunized for DPT. In addition, grantees had not provided any polio immunizations 

during the school year to almost 66 percent of the 244,000 children who were not fully 

immunized for polio. 

In general, children had received the required MMR and HIB immunizations, which 

they are supposed to receive before entering Head Start. Approximately 97 percent 

of the children had received the required MMR immunization and 91 percent had 

received the HIB immunization. The table below illustrates how many DPT, polio, 

MMR, and HIB immunizations children required to be fully immunized at the end of 

the 1991-1992 school year: 

HOW MANY OF EACH IMMUNIZATION DID CHILDREN NEED 

TO BE FULLY IMMUNIZED? 

Numberof Nccdcd 

~ of Immunization Immunizations Pcrccntageof Chikircn 

o 47.6 

1 47.9 

2 3.3 

3 0.9 

4 0.3 

5 0.0 

DPT 

POLIO 

o 66.3 

1 32.4 

2 1.0 

3 0.3 

4 0.0 1 , 

II I I 

o 97.4 

MMR 

1 2.6 

0 

HIB 

91.0 

1 9.0 

In contrast to our finding that 43.5 percent of Head Start children are fully 

immunized, ACF’S PIR data indicate that 88 percent of children are fully immunized, 

and an additional 8 percent, although not fully immunized, are up-to-date in their 

immunizations. The major reason for this discrepancy is conflicting definitions of what 

constitutes complete immunization for 4-year-olds. The ACF’S guidance to grantees, 

10 
which we used to measure full immunization, contlicts with the definition that grantees 

use to report complete immunization in the PIR. The PIR instructions allow grantees 

to consider 4-year-olds fully immunized if they have received 4 DPT and 3 polio 

immunizations. This is one less of each immunization than recommended by the 

Centers for Disease Control, the American Academy of Pediatrics, and ACF’S own 

published guidance. If we apply the PIR instruction to our data, we find that 

84 percent of children are fully immunized. This corresponds much more closely to 

ACF’S published figure of 88 percent. 

Several other factors, which were validated in our follow-up conversations with 

grantees, may contribute to the low immunization rates that we found: 

. Some grantees are ignoring ACF’S guidance and are applying less-stringent 

State, local, or other immunization standards to Head Start children; 

� Some grantees do not keep children’s health files up-to-date; 

. A few grantees are having difficulty finding providers who are willing to provide 

immunizations throughout the school year; and 

� Not all parents, who have been referred to providers for immunizations, 

schedule or keep appointments. 

G&WVTEES FREQUENTLY DO NOT IDENTIFY OR ADDRESS FAMILIES’ 

SOCIAL SERVICE NEEDS 

During both the 1988-89 and 1991-92 school years, grantees frequently did not 

administer family needs assessments (FNAs), did not identi~ family needs or goals 

when conducting an FNA, rarely used Family Assistance Plans, and did not help 

families meet all or most of their needs. 

Gmntea ofien did not conduct FNh or iiient~y familj neek and goak 

Families who participate in Head Start are low-income and often must struggle to 

meet their basic needs. The ACF’S guidance to grantees states that: 

In order to accomplish the comprehensive objectives of the Social 

Services component, the Head Start program should use some form of 

FNA with every family having a child enrolled in the program .... The 

FNA will identify the interests, desires, goals, needs, and strengths of the 

family, and will help the Social Services staff determine how Head Start 

can best work with the family to maximize and maintain its strengths, 

while strengthening areas of need and/or concern. 

Despite this, our file review revealed that grantees frequently did not complete family 

needs assessments. In other cases, grantees (1) conducted FNAs but did not identi~ 

11 
any family needs, (2) only partially completed FNAs, (3) used scaled-down forms to 

accelerate the needs assessment process, or (4) substituted intake or application forms 

for a formal FNA. 

Approximately 140,000--or 26.3 percent--families participating in Head Start nationally 

did not receive an FNA. This represents an improvement over 1988-89 when more 

than one in three families received no FNA. 

Grantees that use the Model FNA developed by the Head Start Bureau were more 

likely to conduct a needs assessment than grantees that developed their own form. 

Fewer than 40 percent of grantees use the Model FN~ however. Grantees that use 

the Model FNA completed it for 87.7 percent of the families they served during the 

1991-92 school year. In comparison, grantees that do not use the Model FNA 

completed a needs assessment for only 63.5 percent of their families. The chart on 

page 13 describes the key components of the Model FNA. 

Grantees that do not use the Model FNA believe that it is too cumbersome, requires 

too much staff time to administer, and is not conducive to an open dialogue with the 

family. Five percent of programs claim that they have never received or seen the 

Model FNA. 

Even when grantees administered an FNA, they frequently did not identi~ family 

needs or goals. In addition to the approximately 140,000 families who received no 

FN~ an additional 70,000 families who received an FNA in 1991-92 had no needs or 

goals identified. This represents approximately 20 percent of all families who received 

FNAs during the 1991-92 school year. This percentage is virtually the same as in 

1988-89. 

Grantees rarely utiltied comprehensive Family Assktance Plans as a blueprint to meet 

fami& neeak and goaik 

Although ACF does not require 

FAP is an important part of the 

the specific measures that Head 

grantees to use Family Assistance Plans (FAPs), the 

needs assessment process. It outlines and describes 

Start staff and family members will take to meet the 

family’s needs. Head Start grantees developed comprehensive FAPs for only one-third 

of the families who received a needs assessment and had needs identified. 

Even though the FAP is part of the Model FNA, grantees that administer the Model 

FNA were only slightly more likely to complete full FAPs (35.9 percent) than grantees 

that use their own needs assessment (29.5 percent). 

Head Stati attempted to meet all or most familj neeh fork than half of the familitx .-

with identified neeh 

By reviewing children’s files and social service files and logs, we 

meet family needs identified in the FNAs. Grantees attempted 

assessed efforts to 

to meet all or most 

12 
.

:,

.

,,.

....

WHAT ELEMEIW3 DOES A COMPREHENSIVE FAMILY NEEDS


ASSESSMENT CONTAIN?


A LOOK AT THE MODEL FNA


: The Model FNA includes several discrete forms designed to obtain a full Picture of a 

: familv’ssocial service needs and to develop a stratew to meet these needs. The Model 

FNA contains: 

U51?	 Intake/Family Profile: This includes information about the family’s composition, 

health, income, education, employment, social semices historv, and resources. 

,::’= 

..:.::: Family Needs Form: This form prompts social sexvice staff to describe anv needs 

.:.:,.,, ,, 

,... for financial assistance, employment, education or training, housinq 

.:.,., 

.,.. transportation, health and nutrition, mental health, family inter-relationships, and

:,,.

:. parenting. The form also has a space for staff to list any Head Start classes that ,. 

,.

:...:, might help the familv meet some of these needs. . 

““I@? Family Goals Sheet: This sheet lists family needs and goals and sets target dates 

for the family to meet them. The interviewer probes the family to identifj the 

strengths it Possesses to meet these goals and identities communitv resources they 

mav need to achieve the goal. 

.

,;

.,

,.

,.

,. .,,.

,,. 

:,:;’;,.@ Family &istance Plan: The intemiewer uses this form to work with the family 

on the s~ecific actions that will be necessarv to meet family needs, who will take 

these actions, and when. 

R3F Family Contact Notes: Social semice staff may use this form throughout the year 

to document contacts with family members and any referrals made on behalf of 

the familv. 

. 

(Jla? Family Goals Attainment Checklist: Social services staff use this checklist
 ..,, ,.
 ‘.., throughout the Vear to track a family’sprogress in achieving its ~oafs.


::


.,,.,


needs for approximately 46.8 percent of families with identified needs during the


1991-92 school year. Overall, this represents 28.0 percent of all families who


participated in Head Start.


13

---------------

---------------

The following chart illustrates Head Start’s attempts to meet identified needs: 

Grantees Had Difficulty Meeting Families' Needs During the 1991-92 School Year 

Families with 

Percent of families for whom Head Start: identified All families 

needs 

attempted to meet all or most family needs 46.8 28.0 

attempted to meet. some family needs 22.7 13.6 

attempted to meet few or none of family needs 26.9 16.1 

identified family needs, but deemed no action necessary 3.7 2.2 

conducted an FNA but did not klenti~ family needs 13.9 

did not conduct an FNA 26.3 

Despite these findings, three-quarters of all Head Start directors believe that their 

social sexvices components have improved or have not changed because of expansion. 

Furthermore, based on the results of ACF on-site reviews, less than 13 percent of the 

grantee directors indicated that they are not able to meet the performance standards 

for social services. 

Nationwide, the Head Start child-to-social-service-staff ratio increased, in contrast to 

the education, health, and parent involvement components, where the ratios 

decreased. The number of children per social service staff rose from 91.2 in 

1988-89 to 94.5 in 1991-92. This is well above the 35-to-1 ratio recommended by the 

National Head Start Social Services Task Force in 1988. 

The OIG companion report, “Head Start Expansion: Grantee Experiences,” describes 

grantees’ difficulties serving children and families with severe problems. Grantees are 

concerned about providing quality setvices with the diminishing availability of 

community semices and decreasing numbers of professionals w-ho are willing to donate 

services. 

GRANTEES’ FILES AND RECORDS FREQUENTLY ARE INCOMPLE~ 

INCONSISTENT, AND DIFFICULT TO REVIEW 

Some grantees have developed exemplary internal record keeping requirements and 

procedures, while others need technical assistance and guidance in this area. The 

ACF’S Grants Management Branch has issued instructions that require grantees to 

maintain children’s records for 3 years, but has issued no other record keeping 

guidelines. As a result, it frequently was difficult to review and verify performance 

indicator data. In summary: 

14 
.	 Several grantees hadalready disposed of 1988-89 files despite being informed 

that they should keep these records for at least 3 years. 

.	 Certain record keeping was haphazard. Grantees generally did not document 

important events--such as family contacts and semices provided to the family--in 

a manner that is conducive to follow-up. Grantees commonly used logs to 

document contacts rather than inserting specific notes and documentation into 

the child’s file. 

�	 It was impossible to track individual parents’ participation in the program 

during the school year. The only source of this information was cumbersome 

in-kind report documentation. 

.	 Education, health, social services, and special needs files remained separated, 

even after the school year ended. A grantee might have three or four different 

files for one child, all in different locations. 

15

RECOMMENDATIONS 

THE SECRETARY SHOULD CONVENE A TASK FORCE TO CONDU~ A 

FORMAL AND THOROUGH REVIEW OF THE MANAGEMENT OF THE 

HEAD START PROGRAM 

Because of (1) inadequate grantee record keeping, (2) the lack of specificity in the 

Head Start performance standards, and (3) the fact that many grantees disregard ACF 

policy guidance, we are unable to determine if the program and performance data 

weaknesses that we found reflect serious deficiencies in the quality of services 

provided by Head Start. This, when considered with the organizational and 

programmatic concerns expressed by grantees in the companion report entitled “Head 

Start Expansion: Grantee Experiences” and anticipated major future expansions, leads 

us to recommend that the Secretary convene a task force to conduct a formal and 

thorough review of the management of the Head Start program. The task force 

should analyze the respective roles and functions of the Head Start Bureau, ACF 

regional offices, grantees, and delegates. It should review how the Head Start 

program is currently organized and managed and how it can best be structured to 

administer the current and anticipated expansions. The task force should examine the 

planning, distribution of funds, grants management, oversight, technical assistance, 

management information systems, and performance standards that are currently used 

to manage the program. 

The Secretary has initiated a comprehensive examination of Head Start with a 

particular emphasis on quality and accountability. The Secretary’s initiative includes 

(1) immediately identifying poorly performing grantees, designing corrective actions, 

providing technical assistance, and ensuring that grantees that provide high quality 

services receive program funds and (2) conducting an in-depth review of the program 

with the help of an intellectually diverse expert study panel and using the results to 

design the Head Start program of the future. This plan addresses our concerns and is 

consistent with our recommendation. We anticipate that the examination will consider 

all of the issues that we have identified as well as the following recommendations: 

THE ACF SHOULD CREA~ DISTRIBU~ AND MANDATE SPECIFIC 

RECORD KEEPING AND CHILDREN’S FILE MAINTENANCE INSTRUCIIONS 

FOR GRANTEES 

Standardized record keeping and file maintenance would assure easy access to 

children’s files. Grantees could identifj missing semices or needed follow-up more 

readily, and on-site reviews could be completed more expeditiously. 

16 
THE ACF SHOULD STRENGTHEN ITS MONITORING OF GRANTEES TO 

BETTER IDENTIFY AND ADDRESS PROBLEM AREAS 

Although Head Start directors and PIR data indicate that grantees are in compliance 

With Ifit prti”mmmce stzmbwti, we identified on-going deficiencies in t’ne ‘nea’tin and 

social services components. Although we did not conduct a review of ACF’S on-site 

monitoring process, we believe that ACF should consider restructuring and condensing 

its on-site program reviews so that more time and resources can be devoted to 

monitoring and providing technical assistance to the most needy grantees. By 

conducting a condensed OSPRI, ACF could reduce the time normally required to 

complete an on-site review. The time and resources saved could be used to conduct 

follow-up activities and to provide technical assistance to grantees that are having 

difficulty providing quality services. Furthermore, ACF should randomly sample PIR 

data to assure its validity. 

THE ACF SHOULD DEVELOP A MODIFIED FNA FORM THAT SHOULD BE 

REQUIRED OF ALL HEAD START GRANTEES 

Fewer than 40 percent of grantees use the Model FNA. Grantees do not use the 

Model FNA because it is cumbersome, requires too much staff time to administer, and 

is not conducive to an open dialogue with the family. The Model FNA should be 

revised to reflect these comments. The ACF should require grantees to use the 

revised FNA throughout the school year to track and follow up on family needs. 

AGENCY COMMEN’IX 

AS discussed in the first recommendation, the Secretary has initiated a comprehensive 

review of the management of the Head Start program. 

We received written comments on the draft report from ACF, ASPE, and ASMB. 

The ACF generally concurred with our recommendations but was concerned about the 

presentation of some of the data and conclusions in our findings. They were 

particularly concerned about the immunization and social services findings. We found 

their comments helpful. As a result, we have revised the immunization section and 

presented additional data to explain and clarify our finding. In addition, we have 

provided a detailed response to all of ACF’S comments, including immunizations and 

social services, in appendix C. The ASPE and ASMB provided editorial and technical 

comments and suggestions which have been incorporated in the final report. The 

complete text of the ASPE and ASMB comments can be found in appendix D. 
APPENDIX A


PERFORMANCE DATA COLLECTED 

To collect information on some of the performance indicators, we reviewed files for 

children who had participated in the program during the 1988-89 (pre-expansion) and 

1991-92 (post-expansion) school years. We requested that each of the 80 sampled 

grantees send us rosters of children enrolled during February 1989 and February 

1992, in all program options so that we could select a random sample of 20 students 

per grantee for each school year. To replace children who were in the program less 

than 120 days or whose files were incomplete, we actually selected 30 children for 

each school year. 

We reviewed each child’s medical, social services, education, special needs, and other 

files. overall, we reviewed files for 1,598 children for the 1991-92 school year and 

1,573 children for the 1988-89 school year. 

We also conducted a review of attendance records to evaluate average daily 

attendance, actual enrollment, and percent of full enrollment. To accomplish this, we 

requested that grantees send us a list of their classes. We produced a set of tables 

that allowed the reviewer to randomly select 20 class-days for each year. For each 

selected class-day, we recorded the funded enrollment, actual enrollment, and the 

number of children who attended class that day. 

To collect information on the other performance indicators, we interviewed directors 

of all 80 grantees and obtained management and performance data. These data 

included menus for February 1989 and February 1992, staffing tallies, budget and 

expansion figures, parent training and workshop documentation, and Federal matching 

(in-kind) records. 

At the completion of the fieldwork, we weighted all data to project to the universe of 

Head Start grantees and delegates that expanded. We determined statistical 

significance by assessing overlapping confidence internals. 
APPENDIX B


HOW HEAD START PROGRAM PERFORMANCE CHANGED 

ACCORDING TO THE OIG PERFORMANCE INDICATORS 

INDICATOR #1: Percent of children medically screened 

SOURCE: Children’s health file review 

‘Percent of ~ children who: 1988-89 1991-92 Difference 

received a medical screen within 90 days of 74.4 70.2 -4.2 

enrollment 

received a medical screen after 90 days of 8.6 11.4 +2.8 

enrollment 

did not a receive a medical screen during 17.0 1s.4 +1.4 

the current school year 

‘percent of fl children who did not 1988-89 1991-92 Difference


receive a medical screen because:
 :.,


.,...


they received one during previous 8.4 10.1 +1.7 

12 months 

parent refused treatment 1.8 0.7 -1.1 

screening was not required because child 0.1 0.6 +0.5 

was currently receiving treatment 

No Documentation 5.7 5.2 -0.5 

Other 1.1 1.8 +0.7 

B-1

Out of children who received a medical 1988-89 1991-92 Difference 

screen, percent who received each of 

following assessments as part of the 

screen 

charted height/weight assessment 82.0 76.4 -5.6 

immunization assessment 97.6 98.4 +0.8 

vision screening 92.1 93.2 +1.1 

hearing screening 91.0 93.3 +2.3 

hemoglobin/hematocrit screen 86.9 88.5 +1.6 

tuberculosis test 57.6 60.3 +2.7 

speech assessment 77.9 84.5 +6.6 

special needs identification 46.2 48.5 +2.3 

:. 

‘percent of ~ children who received a 1988-89 1991-92 Difference 

~ctimplete medical screen: 

Annual requirements** 55.0 53.8 -1.2 

Annual and biennial requirements** 43.8 45.7 +1.9 

““According to the Head Start Performance Standardsy the required elements include 

(a) growth, speech, and immunization assessments and special needs identification 

annually and (b) vkion testing, hearing testing, and hemoglobin or hematocrit 

determination eveiy 2 years. Because of the different methods that grantees use to 

identifi special neeak, we omitted this from our M of required elements. 

B-2 
INDICATOR #2: Percent of children receiving the needed medical treatment 

SOURCE: Children’s health file review 

.:& 

ent of need for medical follow-up: 1988-89 1991-92 Difference 

Out of children who received a medical 

screen, percent who had medical needs 24.4 24.2 -0.2 

identified 

.: 

‘.out of children who had medical 1988-89 1991-92 Difference 

“needs, percent for whom: 

i 

grantees met all medical needs I 74.9 I 76.2 I +1.3 

grantees met some medical needs 9.4 5.5 -3.9 

grantees met no medical needs 15.7 18.2 +2.5 

B-3 
.

INDICATOR #3: Percent of children receiving dental exams 

SOURCE: Children’s health file review 

Percent of ~ children who: 1988-89 1991-92 Difference ,. 

received a dental screen within 90 days of 63.6 67.6 +4.0 

enrollment 

received a dental screen after 90 days of 17.6 17.1 -0.5 

enrollment 

did not receive a dental screen 18.8 15.3 -3.5 

,,. 

“Ptxcent of&l children who did not 1988-89 1991-92 Difference 

~receive a dental screen because: . . . 

parent refused treatment 1.4 0.5 -0.9 

parent missed appointments 1.0 0.7 -0.3 

appointment scheduled, not yet completed 0.2 0.6 +0.4 

dental needs were checked only during 0.8 0.3 -0.5 

medical screen 

provider was unable to complete screen 1.6 0.3 -1.3 

No Documentation, but child received 3.7 3.7 0.0 

screen during previous year in Head Start 

No Documentation 10.1 9.3 -0.8 

B-4

,:

..

INDICATOR #4: Percent of children receiving the needed dental treatment 

SOURCE: Children’s health file review . 

Ektent of need for dental follow-up: 1988-89 1991-92 Difference :,..:, 

Out of children who received a dental 42.9 37.6 -5.3 

screen, percent who had dental needs 

,. 

Out of children who had dental needs, 1988-89 1991-92 Difference


‘:percent for whom:
 ::


:,,


grantees met all dental needs 83.3 66.6 -16.7


grantees met some dental needs I 5.3 I 7.4 I +2.1 

grantees met no dental needs I 11.3 I 26.0 I + 14.7 

INDICATOR #5: Percent of children fully immunized 

SOURCE: Children’s health file review 

Ektent :.. , of immunizations: 1988-89 1991-92 Difference . 

.,. 

Percent of ~ children fully immunized I 39.9 I 43.5 I +3.6 

B-5

..

INDICATOR #6: Grantee provides nutritious meals and snacks 

SOURCE: Grantee menu analysis 

“Percent of grantees that meet Head 19s8-89 1991-92 Difference 

,,Start nutrition requirements, by 

[“program option: . 

.. 

Overall, all grantees 89.1 88.8 -0.3 

Part-day grantees 100.0 100.0 0.0 

Full-day grantees 60.9 59.8 -1.1 

,Percent of grantees that meet Head 1988-89 1991-92 Difference 

$tart nutrition requirements, by meal: ;,.:. . ,,:.:

,, 

Breakfast, all grantees 100.0 100.0 0.0 

Snack, all grantees 72.9 79.2 +6.3 

Lunch, all grantees 100.0 100.0 0.0 

B-6 — 
.... .:

.

,,.:.,.:

. .

INDICATOR #7: Percent of children for whom the teacher has entered in the 

child’s folder observational comments at least once a month 

(center-based only) 

SOURCE: Children’s education file review, teacher/education component files 

,,..,.: ::..., :::.. 

~went of observational comments: 1988-89 1991-92 Difference

:,.,,.,,. : 

Percent of all center-based children for


whom teacher made observational 28.3 34.3 +6.0


comments in child’s file at least once a


month


INDICATOR #8: Average class size 

SOURCE: Enrollment data, attendance record sample 

:,:.: . .. 

‘Average ,.., class size: 1988-89 1991-92 Difference

,.:,:,

. . 

Funded enrollment (number of children) 17.9 17.8 -0.1 

per class 

Actual enrollment (number of children) 17.5 17.6 +0.1 

per class 

B-7

INDICATOR #9:	 Percent of families for whom a family needs assessment has been 

completed 

SOURCE: Children’s social service file 

:, 

Percent of alI families for whom 1988-89 1991-92 Difference 

!grantees: — 

conducted a Family Needs Assessment 62.7 73.7 +11.0 

completed a family profile 92.1 89.5 -2.6 

identified family needs and/or goals 51.3 61.9 +10.6 

:. 

Percent of all families for whom 1988-89 1991-92 Difference


‘gtiantees: —
 :...
 :,


developed a full Family Assistance Plan 12.6 20.7 +8.1 

developed a partial Family Asistance Plan 14.4 18.3 +3.9 

developed no Family Assistance Plan 24.1 23.0 -1.1 

identified no needs, therefore did not 48.9 38.0 -10.9 

develop a Family Assistance Plan 

B-8

INDICATOR #10: Percent of families with identified needs for whom the Head Start 

program has attempted to meet these needs 

SOURCE: Children’s social services file review, social services component log and 

file review 

Out of families with identified needs, 1988-89 1991-92 Difference 

‘percent for whom grantees: 

attempted to meet all or most needs 39.4 46.8 +7.4 

attempted to meet some needs 27.8 22.7 -5.1 

attempted to meet few or none of the 24.0 26.9 +2.9 

needs 

deemed no action necessary to meet needs 8.8 3.7 -5.1 

:,.

. 

!%)ut of families with identified needs, 1988-89 1991-92 Different 

~percent for whom grantees: :::....... .:.,,. : :.:. 

furnished information about available 52.7 60.6 +7.9 

community services 

gave referrals to community agencies 43.6 46.1 +2.5 

provided direct services 20.8 19.2 -1.6 

followed-up 25.2 30.7 +5.5 

B-9 
INDICATOR #11: Percent of parents volunteering at least 50 hours per year 

SOURCE: Grantee interviews, volunteer and in-kind file review 

* Although we reviewed parent volunteer files, they were not organized to track 

individual parents ) volunteer participation. We asked program directo~ whether 

the percent of parents volun~een’ngat least 50 houns per year (or 6 houns per 

month) had increased, decreased, or s~ayed the same. 

:,. 

“.Percent of grantees reporting: Percent

:..: >: ,.., 

an increase in parent volunteering since 1988-89 42.6 

no change in parent volunteering since 1988-89 34.8 

IIa decrease in parent volunteering since 1988-89 I 22.6 

INDICATOR #12: Percent of full enrollment 

SOURCE: Enrollment data, attendance record sample review 

IIPercent of full enrollment 

1988-89 1991-92 Difference 

98.0 I 99.1* I +1.1 II 

� The OIG report, “Head Start Expansion: Grantee Experiences” found that 13 

percent of grantees were unable to enroll all additional children planned under 

expansion. The funded enrollment jigures used to compute percent of full 

enrollment in the above table do not include children who could not be enrolled 

because of problems witit expansion. 

B-10 
INDICATOR #13: Average vacancy time 

SOURCE: Grantee inte~iews, attendance data review 

,,, , 

Percent of grantees reporting that their 

average vacancy time is: 

one week or less 

8 to 14 days 

15 to 30 days 

Don’t know 

1988-89 

61.2 

13.6 

14.0 

11.2 

1991-92 

72.3 

12.4 

13.7 

1.7 

INDICATOR #14: Grantee is not at fiscal or program high risk 

SOURCE: ACF regional office data 

None of the grantees that we vkited were at f~cal or program high rzkk. 

INDICATOR #15: Child-to-staff ratio (based on actual enrollment) 

SOURCE: Grantee-provided staff tally, enrollment data 

+11.1 

-1.2 

-0.3 

-9.5 

;;:,,, .,.,.. 

f’ildren :,.,, per: 1988-89 1991-92 Difference ... ., ,..

,.,. 

teacher, not including teaching assistants 19.5 18.8 -0.7 

teacher, including teaching assistants 9.8 9.4 -0.4 

social service staff person 91.2 94.5 +3.3 

health staff person 202.2 186.8 -15.4 

parent involvement staff person 165.1 150.5 -14.6 

B-II 
INDICATOR #16: Number of volunteer trainings and parent workshops 

SOURCE: Grantee interviews, volunteer, training, and in-kind file review 

UNABLE TO DETERMINE. Grantee files were not conducive to analysis. 

INDICATOR #17: Average number of contacts with each parent 

SOURCE: Children’s social services file review, social semices component log and 

file review 

UNABLE TO DETERMINE. Grantees record parent contacts differently. Our 

reviewen had limited success attempting to quantifi parent contacts. 

INDICATOR #18: Average daily attendance 

SOURCE: Attendance data review 

,: 

‘Children present divided by funded 1988-89 1991-92 Difference ... 

~enrollment 

r 

Average daily attendance I 84.3 I 86.3 I +2.0 

B-12 
APPENDIX C 

ACF COMMENTS ON THE DIL%171’REPORT AND 

DETAILED OIG RESPONSE 

c-l 
DEPARTMENT OF HEALTH & HUJMAN SERVICES 

Apti 7, 1993 

TO: 

FROM : 

Officeof the 

370 L’Enfant 

Washington, 

ADMINISTRATION FOR CHILDREN AND FAMILIE! 

Assistant Secreta~, Suite 600 

Promenade, 

D.C. 20447 

m 

S.w. 

. 

Inspector General 

Acting As~istant Secretary for 

Children and Families M!rls am 

/

SUBJECT : Office of Inspector General Draft Report, “Evaluating 

Head Start E@ansion Through Performance Indicatorstg 

(OEI-09-91-O0762) 

The Administration for Children and Families (14CF) appreciates 

the opportunity to review and conumrk on the Office of Inspector 

Generalts (OIG) draft Report. While there are clearly sexvice 

~allty problems in some programs, ACF believes that the large majority ‘of Head Sta~ p~~ams are ~roviclin~ th#r qlled 

children- and famili&-win* good quallty servlce~ and ‘arestriving 

to be responsive to tieir needs, to improve program management 

and to meet Head Start’s Performance Standards. The problem is 

not that service quality is low in any general sense but, rather, 

that it is uneven. While Head Start programs at the high end of 

the quality spectrum are excellent, it is evident, based on ACWS 

own experience managinq the program, that some local programs do 

not fully comply with important elements of the Head Start 

Performance Standards. 

ACF program officials are aware of the variability in quality 

among local ?lead start programs and a Variety of steps have been 

and are being taken to correct deficiencies. These include the 

strengthening of ACFQS program monitoring capacity, modifying 

Head Startts training and technical assistance delivery system, 

providing additional funding to local programs for service 

~ality mprovement, providln~ training for Head Start directors, 

zmprovlng management informatlcm systems and promulgating needed 

regulations and guidelines. 

TO the extent that the OIGCS study helps to identify areas 

requiring further attention it is a useful contribution to our 

own efforts to upgrade Head Start service quality. However, our 

overall reaction to the draft Report is mixed. On the one hand, 

we believe that the Report correctly points to ~portztrlt issues 

related to specific program components and to local Head Start 

recordkeeping and reporting tha~ 

examination and action. The two 

clearly require further 

most prominent of these program 
Page 2 - Bryan B. Mitchell 

We also believe that 

components are health and social services. which broadly 

a full-scale management reviewthat of may associated with the the bepro~={ 

examines all of the factors 

unevenness in service quality found among Head Start progrms, is


an appropriate step toward addressing the problem and zs


welcomed.


On the other hand, however, we believe that the Report overstates


the magnitude of the problems identified and that it is


misleading --certainly with respect In 

to ~unlzatlon

despite 

and to some


extent in other areas as well. ?dd+tlon, the Reportts


cautionary lan~a~e regarding the dlst+nctlon between.


recor-eepin? deficits and actual service quallty deficits, the


manner in which the OIG study data are discussed and interpreted


betray at least a partial disposition toward concluding,tiat what


may be recor~eeping problems are, in fact, servxce and dellverY,


problems. This gives the Report a more negatxve conclusive


tone than may be warranted or intended. 

and . Outlined b~-low~M greater detail are ACF~s ~specif~c concerns 

co-ents on thelRep~rtts findings and reco~endatlons. : 

OIG Report Findinqs 

1) Children may not be fully immunized before they leave


he Head Start Program 

The ACF believes that the data and conclusions Head presented,inchildren the 

OIG Report relative to the immunization of Start 

are misleading. The Report states that ‘only 43.2 percent of 

children were fully immunized as defined,by Head Start 

idelines.t* While 97 percent of all children were fully 

?% unized for MKR and 90 percent had received the HIB 

immunization, the Report states that chil~en fre~ently did not 

receive needed DPT and polio immunizations. 

To be considered ‘fully immunized,“ Head Start guidelines four-


recommend 5 doses of DPT and 4 doses of polio vaccine for 

year-olds and 4 doses of DPT and 3 doses of pollo vacczne for 

three-year-olds. However, the guidelines make allowances for . 

children who were not immunized as infants and ellmlnate me 

fifth DPT and fourth polio vaccine doses for children whose 

previous doses of these vaccines were given after the clnldts 

fourti birthday. The Head Start guidelines also provide a 

schedule which indicates the timing and spacing of the 

vaccinations. 

To the extent that there may be a shoticoming in the ~unization 

of children, it is largely related to children not recelvln9 tie 

last in the series of doses for DPT and polio and not to a 

general disregard of immunization requlr-ents. On the contra~~ 

c-3 . 
. 

. . . 

Page 3 - Bryan B. Mitchell 

according to the OIG% findings, 98.4 percent of all Head Start 

children were assessed for current immunization status (not 80 

percent as stated on page 9; see table on page B-2), 97 percent 

received the MM? immunization and 90 percent received the HIB 

immunization. These data strongly suggest that Head Start 

programs recognize the importance of immunization and argue for 

the exercise of caution before concluding that Head Start is 

grossly deficient in immunizing children. 

Part of the problem associated with the OIG Reportts analysis may 

lie in the differing interpretations of what is required in the 

serial immunizations for DPT and polio. Head Start guidelines 

notwithstanding, some States, including the study States of 

Michigan and Washington for example, do not require that children 

receive a fifth dose of OPT vaccine andjor a fourth dose of polio 

vaccine, and Head Start programs and their health service 

providers typically do not provide children with these final 

doses. 

Another source= ?f confusion is the reporting specifications of 

the Head ~tart’$&ogr&m In~ormation Report (PIR); tie annual 

survey instrument on which Head Start programs provide 

information about their operations. As the OIG Report itself 

notes, the PIR does not reference a fifth dose of DPT vaccine or 

a fourth dose of polio vaccine as a requirement for reporting 

four-year-olds as having completed their immunizations. Rather, 

for reporting purposes, Head Start programs are instructed to use 

4 doses of DPT and 3 doses of polio vaccine as the threshold for 

a child having satisfied immunization requirements, irrespective 

of whether the child is three or four years of age. 

Using the 4 doses of DPT/3 doses of polio vaccine threshold 

specified in the PIR instructions, Head Start programs reported 

that 88 percent of their enrollees were fully immunized. Based 

on verbal communications with OIG study team staff, we were 

informed that using this service standard the OIGCS study data 

are very close to the PIR data: 88 percent immunized according 

to the PIR and 84 percent immunized according to the OIG study 

team@s data. 

BY using Head Startgs 5 DPT/4 polio vaccine dose guideline as the 

standard to be met by all four-year-olds in Head Start, the 

Report exacerbated the apparent discreparwy which the Study found 

between the data rep~rtecl by Head start pro~arns on the plR and 

the data collected on site by the OIG study team. Since 

approximately 63 percent of the Head Start children are fouryear-olds, the manner in which they are dealt with is critical in 

any assessment of humanizations in Head Start. 

c-4 
. 

Page 4 - Bryan B. Mitchell 

ACF will need to examine the issue of what threshold should be 

used for reporting children’s immunization status on the PIR, 

recognizing that programs in some States may not meet the 5 DPT/4 

polio vaccination guideline for four-year-olds and also that some 

four-year-olds should not receive these final boosters. It 

should also be noted that using the more st~ingent 5 DPT/4 polio 

standard for reporting four-year-old children as fully immunized 

on the PIR is not likely to alter immunization practices. The 

principal result is likely to be that more children are reported 

as Up-to-date as opposed to having coxnpleted immunizations. 

Finally, we wish to point out that, apart from the issue of what


constitutes “fully immunized,‘1the Report gives the impression


that large numbers of Head Start children are not up-to-date in


their immunizations. It asserts that ‘Approximately 69.4 percent 

of the children who required polio immunizations during the 

program year did not receive all they needed, 58.6 percent of the 

children who requhed DPT immunizations during the program year 

did not receive all they needed” (page 9). 

Acc&~*ing ;&;th&’Head ,Startguideline”&~”=ch~ic@en{should
 begin 

immunizations @u~ingzZnfancy and should ha~?recei”ved 4 doses of 

DPT and 3 doses of polio vaccine by their Isth month and another 

dose of each vaccine between the ages of 4 and 6. HoWeV=,oSO~e 

children, especially in low-income communities, do not receive 

immunizations on time and must be immunized according to a 

revised schedule which is also specified in the Head Start 

guidelines. For example, a three-year-old entering Head Start 

without any prior immunizations receives his first shots in 

August . The child should receive DPT-1, polio-l, MMR and HIB. 

Two months later, in October, the child should receive DPT-2 and 

polio-2. Then, in December, the child should receive DPT-3. The 

child should not receive another booster until the period between 

June and December of the following year. Thus, the child should 

not have more than 3 doses of DPT and 2 doses of polio vaccine 

during the Head Start year. Additional doses of vaccine during 

that time period would be of no benefit. The same schedule would 

apply to a four-year-old with no prior immunizations. 

NO data are presented either in the body of the OIG Report or in 

Appendix B concerning how OIG’S analysis took into account the 

child’s age of first vaccination, the child~s age and vaccination 

status upon entry into Head Start and the child~s length of stay 

in the program before concluding that children did not receive 

‘all they needed “ by way of immunizations. 

For the reasons cited above, we recommend a full and careful 

reanalysis of the OIG study data. The reanalysis should present 

immunization rates for four-year-olds using the immunization 

c-5 
8 . 

Page 5 - Bryan B. Mitchell 

thresholds in both the Head Start guidelines and the PIR. In 

determining whether the children are up-to-date in their 

immunizations, the reanalysis should make appropriate ad]ustrnents 

for children’s ages at first immunization and other relevant 

factors. We also recommend that all of the data used in the 

analysis be presented in the Report along with such explanations 

of methodology as are necessary to understand how the data were 

compiled and analyzed. Absent this, we recommend that the 

finding that only 43.2 percent of Head Start children were fully 

immunized be dropped from the Report and that the immunization 

section be recast to reflect only what can be asserted with 

confidence. 

2) ~ ect ormation in 

childrenls health records. 

The ACF has several suggestions related to this finding. First, 

the tables on page B-1, which provide some of the information 

used to illustrate this conclusion, should be modified~ The 

first table-is misleading in representing 18.4 percent of Head 

Start ~ildren as not having receive-da medical screen. The 

information in the second table indicates that 10.1 percent did 

not require screening because they had been screened during the 

previous 12 months and an additional 0.6 percent did not require 

screening because they were under treatment. The data on these 

two categories of children should be removed from the second 

table and included in the first table so as to clearly show that 

only 7.7 percent of the children who required a medical screening 

did not receive it. The fact that 92.3 percent of all Head Start 

children received a medical screen and that only 7.7 percent did 

not should be highlighted on page 8 along with the other 

information presented. 

Second, it would be helpful if the Report provided additional 

information regarding how it was determined that only 53.8 

percent of Head Start children received growth, speech and 

immunization assessments (pages 8 and B-2). More information 

about this aspect of Head Start program performance might shed 

some light on why this figure is so much lower than, and in 

apparent conflict with, the data on assessments presented in the 

table on the top of page B-2. The data on page B-2 show that 

76.4 percent of children received height/weight assessments, 84*5 

percent received speech assessments and 98.4 received 

immunization assessments. We also suggest the need for 

clarification of the note at the bottom of page B-2 which seems 

at variance with the notation on page 8 regarding the threshold 

for determining whether a child had received ‘all appropriate 

tests.w 

- .. . 

C-6 

. 
Page 6 - Bryan B. Mitchell 

3) Grantees freuuentlv do not identifv or address 

families’ social service needs. 

The OIG Report indicates that 26.3 percent of families 

participating in Head Start did not receive a family needs 

assessment (FNA). It also states that, in some cases, Head Start 

programs only partially completed FNAs, used “scaled do~” fo~s 

to accelerate the needs assessment process or substituted intake 

or application forms for a formal FNA. TWO points related to 

these observations are unclear: First what criteria did the OIG 

study team use to determine what constitutes an acceptable FNA? 

Might not at least some Head Start programs believe that the 

‘family profile 1’forms completed for 89.5 percent of enrolled 

families (see table on page B-8) contain sufficient information 

to serve as family needs assessments? Second, how detrimental is 

the use of locally developed forms and protocols which may be 

less comprehensive and elaborate than the model FNA developed by 

the Federal government? It should be noted that the use of the 

model FNA is not required. Head Start guidance merely indicates 

that programs should use ‘“some form of FNA.19 7 

W*& re~~ct to lieaci Start programs’ efforts to-actually address 

family needs, we suggest that the tables on the top of page B-g 

and on page 13 be modified and made consistent. The table on 

page B-9 indicates that the percent of families with identified 

needs for whoxa Head Start prograxns “attempted to meet few or none 

of the needs’t was 26.9 percent. No definition is given for what 

is meant by “few . . .of the needs” or how these families were 

distinguished from the preceding category on the table, f-ilies 

on whose behalf programs attempted to meet ‘some of the need~en 

Unless satisfactorily distinguishable, we suggest that farnilles 

in the few-needs-met category be iriclucied with the families in 

the some-needs-xnet category and that the 26.9 percent figure be 

adjusted downward to reflect only those families who apparently 

received no assistance. 

A corresponding modification should be made to the table on page 

13 � Please note in this connection that, as it stands, the table 

on page 13 is inconsistent with the table on page B-g in that it 

acknowledges no Head Start program efforts whatsoever on behalf 

of the 26.9 percent of families with identified needs. 

4) Grantees ~ files and records freuuentlv are incomplete. 

inconsistent and difficult to review. 

This finding provides an important contextual consideration 

within which the entire OIG Report Should be viewed and has Z1 

material bearing on each of the other three major findings. As 

noted in the Report itself, and as apparent in our own review of 

it, the OIG study team found it difficult in many instances to 

determine whether a problem identified during the review was an 

c-7 . . 
� 

Page 7 - Bryan B. Mitchell 

actual service delivery weakness or 

record keeping. Since the issue of 

permeates so much of the Report, we 

presented first. 

the result of inadequate 

records and record keeping 

suggest that this finding be 

As indicated earlier in this memorandum, we also suggest 

modifying some of the language used to present findings and 

conclusions which is inconsistent with the Report’s own 

cautionary note regarding the study teamts inability ‘to 

determine if the program and performance data 

weaknesses. ..reflect serious deficiencies in the quality of 

senices provided by Head Start’g (pages ii and 7). A few 

examples of language which appear to clis~egarci this caveat are: 

‘Grantees are not completing the minimum health requirements to 

the extent reported...” (page 7) ; “apparently... percent received 

no follow-up treatment” (page 8); “Approximately 140@0--or 26=3 

percent --families participating in Head start nationally did not 

receive an FNA’~ (page 10); and “made no attempt to meet family 

needs” (page 13). In these and other instances, it would be more 

appropriate to use language such as ‘the study team could find no 

documentation that...tt or “local Head Start program records were 

inadequate/incomplete with respect to...,” etc. 

Finally, we also recommend that the Report include information on 

the results of the OIG study tearn~s follow-up interviews with 

selected programs which are mentioned on page 5. The purpose of 

these interviews was to ascertain the reasons for the apparent 

discrepancies between the data collected by the study teams on 

site and the PIR data. Since the reliability of the PIR data is 

presented as an issue in the Report, it would be helpful to knOW 

the results of this follow-up, including the perceptions of the 

programs whose operations were the subject of the OIG review. 

Our own informal discussions about immunizations with 18 of the 

programs reviewed by the OIG study team indicate that these Head 

Start programs do not consider their records to be inadequate and 

that they believe that the PIR information which they submit at 

the end of the program year is accurate. However, several 

programs indicated that, although services may have been 

provided, an individual childts health record might be incomplete 

because of several factors: the lag-time associated with several 

centers reporting to central recardkeeping locations; the 

difficulty of obtaining actual copies of medical records from 

physicians not reimbursed by the Head Start program itself; and 

the fact that all of a child~s records may not be in the master 

file for each child prior to the actual Close-out of tie program 

year. 

C-8 
� 

� 

Page 8 - Bryan B. Mitchell


OIG Report Recommendations


1) The Secretaq of HHS should convene a task force to 

conduct a tormal and t borough review of the management 

of t he Head Start program. 

The ACF believes that a management review of Head Start would be 

useful in helping to identify and address issues bearing upon


pro~am quality. In a March 31, 1993 letter to Senator Kennedy,


Chazrman of the Senate Labor and Human Resources Commttee, the


Secretary indicated that she has asked for a ‘top-to-bottom looktt


at the program. The review would be undertaken with the help of


an expert study panel and the results used to design an improved


Head Start program.


2)	 The ACF should create, distribute and mandate specific 

recordkeeplnq and children Bs tzles maintenance 

r	 *,”+ “ lnstructzons for grantees. . - 1.8 El 

‘ The ACF believes this recommendation has merit. The agency has 

already begun work on the implementation of new recordkeepxng and 

reportzng protocols for Head Start, entitled the Head Start 

Family Information System (HSFIS). The new system WZU xnclude 

tiformation on the services provided to individual Head Start


children as well as on family needs and how they are being 

addressed. As envisioned, HSFIS would meet the recordkeeping 

needs of local Head Start programs and also facilitate their 

reporting of information to ACYF. A request for clearance to 

field test HSFIS in 22 Head Start programs is currently at the 

Office of Management and Budget for approval.


3)	 The ACF should strengthen its monitoring of grantees to 

better ~dentlfy and address Problem areas. 

The ACF agrees with the suggestion under this recommendation that 

the PIR data should be checked for validity on a sample ~asiq. 

However, other aspects of this recommendat~on, as ~pl~fled m 

the text on page 15 which follows it, are problematic. 

First, although we a~ee that we need to continue to stren~en 

our monitoring capacity, it is not clear what the Report means 

when it states that ACF should consider ‘restructuring and 

condensing its on-site pro~am reviews.” The OIG% study did not 

e%amzne how monitoring viszts are conducted, and the Report 

itself presents no findings or analyses concerning on-site 

monitoring. 

c-9 
. . 

I 

Page 9 - Bryan B. Mitchell


Second, while we agree with the intent of the suggestion that ACF


should devote more time and resources to monitoring and providing


technical assistance to the most problematic ~antees~ tiere ~e


severe constraints related to its- implementat~on.” The Head Start 

legislation requires that ACF cenduct an on-site monitoring visit 

to one-third of the Head Start grantees each year (approximately 

460 programs), that all newly established programs be visited 

early in their second year of operation and that follow-up visits 

to Head Start a~encies be conducted when appropriate. Currently


available staffng and travel resources are barely adequate to


meet these statutory requirements, particularly the follow-up


visit requirement. In so far as existing resources permit


follow-up visits, they are already being targeted on programs


identified as havinq significant Problems. Any expansion of on-


site monitoring to ~ddi~ional pro~ams would requfie more 

resources than are currently available. 

We suggest that this recommendation focus on validating the PIR 

data and that the topic of ACPS monitoring processes and 

priorities be examined as part of the larger management review of 

Head Start envisioned by the Report”s first recommendation. 

4)	 The ACF should develop a modified FNA form that should 

be re qulred ot all Head tart grantees. 

We believe that the HSFIS will satisfy the intent of this


recommendation.


If you have any questions about our comments or if we can be of 

further assistance, please do not hesitate to call. 

c-lo 
OIG RESPONSE TO ACF’S COMMENT’S 

ACF CO MMENT (Page 1): “While there are clearly service quality problems in some 

programs, ACF believes that the laqp majority of Head Start programs are providing their 

enrolled children and families with good quality services and are stn”vingto be responsive 

to their needs., to improve program management and to meet Head Start’s Performance 

Standards. The problem h not that service quality is low in any general sense but, rathe~ 

that it is uneven. While Head Start programs at the high end of the quality spectrum are 

excellent, it k evident, based on A CFk own expen”encemanaging the program, that some 

local programs do not fully comply wi~h important elemenls of the Head Start 

Performance Standards. ” 

“ACF program oficials are aware of the variability in quality among local Head Start 

programs and a variety of steps have been and are being ~aken [o corect deficiencies. 

These include the strengthening of ACFk program monitoring capacity, modi~ing Head 

Start’s training and technical assistance deiivery system, providing additional funding to 

local programs for service quality improvement, providing training for Head Start directo~, 

improving management information ~stems and promulgating needed regulations and 

guidelines. ” 

OIG RESPONSE: Based on our experiences while conducting on-site visits, we 

generally agree with ACF’S assessment that some grantees provide better and more 

complete services than others. Given the variation in quality, we believe that certain 

grantees require greater technical assistance and oversight than others. This was the 

reason that we recommended that ACF concentrate its resources on the most needy 

grantees, rather than on the ones that provide high quality services already (who could 

receive a condensed on-site review). Although we did not conduct a tull study of 

ACF’S on-site review process and we recognize that there may be legislative 

constraints, we believe that ACF should consider this optiom 

ACF COMMENT (Page 2): ‘tOn tileo{iwr hand, however, we believe iha~ the Repoti 

overstates the magnitude of lhe problems identified and dlat it is misleading--certainly 

with respect to immunization and to some extent in other areas as well. In addition, 

despite the Report’s cautionary language regarding [he dhtinction between record keeping 

deficits and actual service quality deficits, the manner in which the OIG study data are 

discussed and interpreted betray at least a partial deposition ~oward concluding that what 

may be record keeping problems are, in fact, service delivery problems. Thk gives the 

Report a more negative and conchisive tone than may be warranted or intended.” 

OIG RESPONSE: While the report includes many references to the poor file 

maintenance, we cannot reject the probability that many grantees are not providing 

the services required by the performance standards. The immunization problem, in 

particular, should not be dismissed as merely a record keeping problem, since we 

found conflicting ACF immunization @delines and learned that many grantees 

disregard ACF/CDC immunization guidance, anyway. 

C-11 
Given the inconsistency and poor condition of tiles maintained by many grantees, it is 

impossible for any reviewer, including ACF staff, to ascertain whether grantees are 

adequately providing services to children and families. The OIG reviewers conducted 

extensive file reviews of all available information, including logs that were maintained 

by the grantees. They also gave grantees the benefit of the doubt, whenever possible. 

ACF COMMENT (Page 2): “The ACF believes that the data and conclusions presented 

in the OIG Report relative to the immunization of Head Start cilildren are mideading. 

17te Report states that “only 43.2 percent of c}lildren were fully immunized as defined by 

Head Start guidelines.” W}lile 97 percent of all children were fully immunized for MJ4R 

and 90 percent had received the HIB immunization, the Report sta[es tha~ children 

frequently did not receive needed DPT and polio immunizations.” 

“To be considered ~uh’yimmunized,’ Head Start guidelines recommend 5doses of DPT 

and 4 doses of polio vaccine for four-year-olds and 4 doses of DPT and 3 doses of polio 

vaccine for three-year-olds. However, t}le guidelines make allowances for children who 

were not immunized as infants and eliminate the fif(h DPT and fourth polio vaccine 

doses for children whose previous doses of these vaccines were given after the childh 

fourth birthday. Tile Head Start guidelines also provide a schedule which indicates the 

h“mingand spacing of the vaccinations. ” 

OIG RESPONSE: Our analysis took all of these factors into consideration. 

ACF COMMENT (Page 2): “To the extent that [here may be a shortcoming in the 

immunization of children, it is la~ely related to children not receiving the last in the sen”es 

of doses for DPT and polio and not [o a general disregard of immunization requirements. ” 

OIG RESPONSE: We agree that most children who are not fully immunized require 

only DPT andior polio immunizations. We tound, however, that children sometimes 

required more than simply the last in the series of immunizations. We have added 

additional narrative and tables in the report to clarify this and provide additional 

information for ACF. 

ACF COMMENT (Page 3): “On t}le contrary, according to the OIG’S findings, 

98.4 percent of all Head Start children were assessed for current immunization status 

(not 80 percent as stated on page 9; see table on page B-2)...” 

OIG ItESPONSE: The ACF did not interpret the tables correctly. The narrative on 

page 9 of the draft report referred to the percentage of all children who received 

immunization assessments. Therefore, children who did not receive medical screens 

were included in this computation. The table on page B-2 refers to the percentage of 

the children who received medical screens and received an immunization assessment 

as part of the screen. This excluded children who did not receive medical screens. 

C-12 
ACF COMMENT (Page 3): “These data strongly suggest that Head Start programs 

recognize the importance of immunization and a~ue for lhe exercise of caution before 

concluding that Head S~art is grossly deficient in immunizing children. ” 

OIG RESPONSE: In no section of the report did we conclude or imply that Head 

Start is grossly deficient in immunizing children. At the same time, the data do not 

lead us to conclude that all Head Start grantees are vigilant in ensuring that children 

obtain all immunizations as required by Head Start’s published guidance. We have 

added additional narrative and tables to the report to clearly document the extent to 

which children have not received all required immunizations. 

ACF COMMENT (Page 3): “Part of tile problem associated wi[h the OIG Report k 

analysis may lie in lhe differing interpretations of whu~ is required in ~he serial 

immunizations for DPT and polio. Head S[art guidelines notwithstanding some States, 

including ~he study Sta[es of Michigan and Washington for example, do not require that 

children receive a fifth dose of DPT vaccine and/or a fourtil dose of polio vaccine, and 

Head Start programs and their heal~h service providers typically do not provide children 

wi~h these final doses. ” 

OIG RESPONSE: We agree that grantees use different standards to determine how 

many DPT and polio immunizations are required. However, ACF’S reference to 

Michigan and Washington using lower standards is somewhat inappropriate. In fact, 

almost half of the Michigan and Washington grantees were above the mean in terms 

of children fully immunized according to Head Start’s published guidance. In addition, 

senior-level ACF staff commented during meetings that the presence of less-stringent 

State standards would not be an appropriate rationale for grantees to obtain fewer 

immunizations than those published in ACF’S guidance. The ACF should clarify what 

is expected of grantees and assess compliance with the performance standards based 

on clear and consistent guidelines. 

ACF COMMENT (Page 3): “Ano[hcr source of con.h.sion is [he reporting specifications 

of the Head Start Program Information Report (PIR), the annual survey instrument on 

which Head Start programs provide information about (}~eiroperations. As [he OIG 

Repofl itself notes, the PIR does ]lot refere~lce u Jfth dose of DPT vaccine or a fourth 

dose of polio vaccine as a requirement for reporting four-year-okis as having completed 

their immunizations. Rather, for reporting purposes, Head Start programs are instructed to 

use 4 doses of DPT and 3 doses of polio vaccine as dle threshold for a child having 

satisjied immunization requirements, irrespective of whether the child h three or four yearn 

of age.” 

“Usingthe 4 doses of DPT/3 doses of polio vaccine threshold specified in the PIR 

inshuctions, Head Start programs reported (hat 88 percent of their enrollees were fully 

immunized. Based on verbal communicatio]ls widl OIG study (earn staff, we were 

informed that using this service s[andard (}w OIG’s study data are very close to dle PIR 

data: 88 percent immunized according [o the PIR and 84 percent immunized according 

to the OIG study team h da~a.” 

C-13 
OIG RESPONSE: We agree with ACF’S assessment. This also indicates that 

grantees’ immunization records are fairly accurate. From this, we can conclude that 

grantees are, to a large extent, disregarding the ACF’S immunization guidance, and our 

statement that 43.5 percent are fully immunized m measured by ACF’S guidance is 

accurate. 

ACF COMMENT (Page 3): “By using Head Start’s 5 DPT/4 polio vaccine dose 

guideline as the standard to be met by all four-year-olds in Head Start, the Report 

exacerbated the apparent dhcrepancy which the study found between the data reported by 

Head Start programs on the PIR and the data collected on site by the OIG study team. 

Since approximately 63 percent of the Head Start children are four-year-olds, the manner 

in which they are dealt with is cn”[icalin any assessment of immunizations in Head Start.” 

OIG RESPONSE: We agree that the percent of children fully immunized as 

tabulated by ACF and OIG are not comparable. This discrepancy was caused by 

conflicting ACF criteria for determining full immunization. While its guidance 

suggests that 5 DPT and 4 polio are necessary, ACF’S published data--the 88 percent 

stated in ACF’S Statistical Fact Sheet--uses the less stringent PIR criteria. In addition, 

ACF does not validate PIR data. Our random sample of children is a clear, accurate 

measure of the extent of immunizations that Head Start grantees provide. 

ACF COMMENT (page 4): “ACF will need lo examine [he issue of what threshold 

should be used for reporting c}iildrcnk immunization status on the PIR, recognizing that 

programs in some States may no[ meet the S DPT/# polio vaccination guideline for fouryear-olds and ako that some four-year-olds should not receive these Jinal boostem It 

should ako be noted that using the more s[ringenl 5 DPT14 polio standard for repofling 

four-year-old children as jidly immunized on [he PIR is not likely to alter immunization 

practices. The principal result is likely to be that more children are reported as up-to-date 

as opposed to having completed immunizations.” 

OIG RESPONSE: We agree that ACF should examine its criteria for reporting full 

immunization on the PIR. This is far less important, however, than clarifying its 

guidance to grantees on how many immunizations each child should receive. If ACF 

believes that grantees should have the option of ignoring the guidelines created by the 

Centers for Disease Control and distributed by ACF in favor of State or other 

standards, then it should issue formal guidance to grantees that states this policy. 

ACF ~wNT (page 4): “Finah’y,we wikh lo poinl OLUlhat, apart from the ksue of 

what constitutes ‘@llyimmunized, ) the Repor~gives !Ile impression t}lat la~e nurnben of 

Head Start children are not up-to-dale in llleir immunizations. 11 asserts that 

‘Approximately 69.4 percent of the chi[dren who required polio immunizations during the 

program year did not receive all ihey needed, 58.6 percent of the children who required 

DPT immunizations during [he progr~imyear did not receive all [hey needed’ (page 9).” 

“According 10 the Head Star{ guiddines, chi[dren shou!d begin immunizations during 

infancy and should have received 4 doses of DPT and 3 doses of polio vaccine by lheir 

C-14 
18th month and another dose of each vaccine between the ages of 4 and 6. Howeve4 

some children, especially in low-income communities, do not receive immunizations on 

time and must be immunized according [o a revised schedule which is also specified in 

the Head Start guidelines. For example, a three-yeur-old en~eringHead Start wit}~outany 

ptior immunizations receives his first sho!s in August. Tile c}lild should receive DPT-1, 

polio-l, MMR and HIB. Two months late~ in Oc[obe~ the child should receive DPT-2 

and polio-2. T/len, in Decembe~ the c}lild should receive DPT-3. T}le child should not 

receive anot}ler booster until the period between June and December of the following year. 

Z4us, the child should not have more than 3 doses of DPT and 2 doses of polio vaccine 

during the Head Start year. Additional doses of vaccine during that time period would be 

of no benefit. Tile same schedule would apply to a four-year-old with no pn”or 

immunizah’ons. ” 

OIG RESPONSE: As we commented during meetings with ACF, we were unable to 

determine whether underimmunized children were indeed up-to-date in their 

immunizations. The up-to-date measure would represent a very small percentage of 

Head Start children who were severely underimmunized when entering Head Start 

and could not receive all required immunizations during the school year. Clearly, very 

few children come into Head Start as underimmunized as the one used by ACF as an 

example. In fact, OIG reviewers were unable to recall a single instance in which a 

child had no immunizations when entering Head Start. In addition, ACF data indicate 

that only 8 percent of all Head Start children are “up-to-date” rather than fully 

immunized or not fully immunized. Even assuming that 8 percent of our sample was 

up-to-date (rather than not fully immunized), approximately half of all Head Start 

children still are not fully immunized or up-to-date as defined by Head Start’s 

published guidance. 

We have revised the report to exclude references to children “receiving all 

immunizations they need.” The final report focuses on whether children who are not 

fully immunized received ~ of their needed immunizations during the school year. 

We believe that this gives grantees the benefit of the doubt, given that it is possible for 

a child to receive as many as 3 DPTs and 2 polios during the school year. 

ACF COMMENT (Page 4): “No data am presented either in the body of the OIG 

Report or in appendix B concerning how OIG b analysis took into account lhe child’s age 

of jkst vaccination, the cilildh age and vaccination ~[utus upon entry into Head Start and 

the childh length of stay in the program before concluding tha~ children did not receive 

‘all they needed’ by way of immuniza~ions.” 

OIG RESPONSE: We have revised the final report to include more information 

about how we assessed immunizations. We ascertained whether each child received 

his first DPT and polio immunizations by the time she was 4-months-old. In addition, 

we used this and the child’s age to determine how many immunizations were 

necessary. All children were in the program for at least 120 days. 

C-15 
It should be noted that only 39 percent of children who required DPT immunizations 

received at least one and only 34 percent of children who required polio 

immunizations received at least one during the school year. The final report 

incorporates this information. 

AC.F COMMENT (Pages 4-5): “For[he reasons cited above, we recommend a full and 

careful reanalysti of the OIG study data. The reanalysis s}lould present immunization 

rates for four-year-olds wing the immunization thresholds in both the Head Start 

guidelines and the PIR. In determining whelher the children are up-to-date in their 

immunizati”om, the reanalysh s}lould make appropria~e adjustments for children b ages at 

fint immunization and other relevant factors. We also recommend that ali of the data 

used in the analysis be presented in ~}leReport along with such explanations of 

methodology as are necessary lo understand how the data were compiled and analyzed. 

Absent t}u$ we recommend that the finding [hat onb 4.?.2percent of Head Start c}lildren 

were fully immunized be dropped from t}w Report and lhat the immunization section be 

recast to rejlect only what can be asserted wi(h confide rice.” 

OIG RESPONSE: The immunization finding, as written, presents clear, accurate 

information about the extent of immunization among Head Start children. We have 

revised the report to address ACF’S concerns, but our main concerns remain: 

1. 43.5 percent of children are fully immunized as defined by Head Start 

guidelines (the draft report figure was 43.2 percent). These guidelines are the 

same as those published by CDC and many other organizations. Using the PIR 

reporting guidelines, 84.0 percent are fully immunized. We are unaware, 

however, of any policy that states that grantees should use PIR reporting as the 

standard requirement for immunizations. 

2. Grantees do not have clear instructions about how many immunization they 

should obtain for children. The PIR and Head Start published guidance are in 

conflict. 

3. Many children participating in Head Start are missing key immunizations, and 

some need several immunizations to be fully immunized. 

4. Many children did not receive immunizations during the program year, although 

they needed them, according to ACF/CDC guidelines. Less than 40 percent of 

children who required DPT and/or polio immunizations received ~ during the 

1991-92 school year. 

ACF COMMENT (Page 5): “T/le A CF’S PIR da[a do not accurately reflect information 

in children k health records. Tile A (3F has several suggestions related to this finding. 

Fimt, the tables on page B-1, wilic}l provide some of the information used to illustrate thk 

conclusion, should be modi~ed. Tile Jirst table is mtileading in representing 18.4 percent 

of Head Start c}lildren as not ho ving received a medical screen. The information in the 

second table indicates that 10.1 percent did not require screening because they had been 

C-16 
screened during the previous 12 months and an addi~ional 0.6 percent did not require 

screening because they were under treatment. Tlw data on t}lese two categories of 

children should be removed from the second table and included in the first table so as to 

clear~ show t}lat only 7.7 percent o! the children who required a medical screening did 

not receive it. Tlw fact that 92.3 percent of ull Head Start c}lildren received a medical 

screen and that only 7.7 percent did nol should be highligh~ed on page 8 along with the 

other information presented. ” 

OIG RESPONSE: We have modified the table on page B-1 to indicate that 

18.4 percent of Head Start children did not receive a medical screen during the 

current Vear. While we understand that ACF does not require grantees to obtain 

screens for second year participants, we found no documentation or guidance that 

would allow grantees to do this. The performance standard guidance states that 

“screening tests should be carried out for all the Head Start children” (page 17). In 

this regard, we believe the distinction in the table on Page B-1 is appropriate. We 

found that some model grantees obtained health screens for all children, regardless of 

whether they had one the previous year. 

A broader concern that ACF does not address is the lack of complete health screens. 

Only about half of the children in Head Start received all three required annual 

assessments during their screens (growth, speech, and immunization--we gave grantees 

the benefit of the doubt for the special needs assessment). This does not include the 

biennial assessments (vision testing, hearing testing, and hemoglobin or hematocrit 

determination). If we used “complete health screens” as the measurement for whether 

children received a health screen, grantee performance would be much lower. 

ACF COMMENT (Page 5): “Secotul, i~ would be hclpfid if the Report provided 

additional information regarding how it was determined tilat only S3.8 percent of Head 

Start children received growth, speech and immunization assessments (pages 8 and B-2). 

More information about this aspect of Head Start program performance might shed some 

light on why d~ti figure h so much lower ~han, and in apparent con.ict with, the data on 

assessments presented in ~he table on (he [op or page B-2. Tile data on page B-2 show 

that 76.4 percent of children received height/weig}lt assessmen~s, 84.S percent received 

speech assessments and 98.4 received immunization assessments. We ako suggest the 

need for clarification of [he note at the bottom oJ’page B-2 which seems at van”ance with 

the notation on page 8 regarding {he ihreshold for determining whether a child had 

received ‘all appropriate tests. ‘“ 

OIG RESPONSE: We clarified the wording of the tables in the final report. The 

53.8 percent refers to the percent of ~ Head Start children who received a full health 

screen. This includes children who did not receive any screen. The table on page 

B-2 refers to the assessments received by children who received a health screen. We 

also have revised the report to ensure that the footnotes--which describe the criteria 

for determining whether a child received a “complete” health screen--are consistent. 

C-17 
AC!F COMMENT (Page 6): Grantees frequently do not z’dentifi or address families’ 

social service needs. Tile (2IG Report indicates tilat 26.3 percent of families participating 

in Head Start did not receive a family needs assessment (FNA). It also states that, in 

some cases, Head Start programs only partially completed FNAs, used healed down) 

forms to accelerate the needs assessment process or substituted intake or application forms 

for a formal FNA. Two points related [o [hese observations are unclear: Fint what 

criteria did the OIG study team use to determine wha[ constitutes an acceptable FNA ? 

Might not at least some Head Start programs believe ~ha~the ~amily profile’ forms 

completed for 89.5 percent of enrolled families (see table on page B-8) contain su.cient 

information to serve as family needs assessments?” 

OIG RESPONSE: We used ACF’S performance indicator guidance to determine the 

intent of the FNA process. The guidance states: “The FNA will identi~ the interests, 

desires, goaks, needs and strengths of the family, and will help the Social Sewices staff 

determine how Head Start can best work with the family to maximize and maintain its 

strengths, while strengthening areas of need and/or concern.” ln addition, we 

examined each component of ACF’S Model FNA, which is described on page 12 of 

the report. While we did not assess grantees’ forms against the model FNA, we 

looked for an attempt to identi~ needs and goals at a minimum. 

The family profiles we saw merely list family members and ages. We frequently found 

that this information was part of the enrollment application, rather than part of a 

social sexvices file. If a family profile contained an assessment of family needs, we 

considered it to be an FNA (as well as a family profile). However, we did not deem a 

listing of family members and ages as a suitable substitute for an assessment of the 

reasons the family is in poverty and the services it requires. 

We would consider it a severe program weakness if ACF found grantees in 

compliance with social service requirements if they systematically failed to ascertain 

parents needs for income supplements, education, job assistance, medical and mental 

health services, and other assistance. 

In addition, we found little evidence that grantees used and updated FNAs throughout 

the year. This is critical because families may be hesitant to share their needs and 

problems during intake. Overall, grantee efforts to identify needs are clearly a 

problem. 

ACF COMMENT (Page 6): “..,Iww dc[rimental is [he use of locally developed fomns 

and protocols which may be less comprehensive and elaborate than the model FNA 

developed by the Federal government? It should be noted that the use of the model FNA 

b not required. Head Start guidance merely indicates that programs should use ‘some 

form of FNA. ‘“ 

OIG RESPONSE: The report does not state that grantees’ development of forms is 

detrimental. However, ACF should be aware that some grantees have developed 

forms that clearly make no attempt to identi$ family needs or goals or develop action 

C-18 
plans. Inaddition,
 OIG reviewers often found that enrollment forms were used as 

FNAs and contained few or none of the elements in the Model FNA. Some model 

grantees have developed scaled-down forms that comprehensively identi& family needs 

and help develop a framework to address these needs. 

our fourth recommendation addresses theweaknesses we foundinthesocial
 services


component.Achieving social service objectives iscritical tomeeting
the


Adminstration’s goalof“empowerment rather thanentitlement.” Grantees’ failure
 to

identi~ andaddress family needsseverelyimits theDepartment’s ability
 toachieve


thk.


ACF COMMENT (Page 6): “Wi~h respect to Head Start programs’ efforts to actually 

address family needk, we suggest that the tables on the top of page B-9 and on page 13 be 

modified and made consistent. The table on page B-9 indicates that the percent of 

families with identified needs for whom Head S[art programs ‘at~empted [o meet few or 

none of the needs’ was 26.9 percent. No deflni[ion is given for what is meant by ~ew...of 

the needs’ or how [Ilesefamilies were distinguislw.i from tile preceding category on the 

table, families on whose bella~prograrns cit[empted to meet ‘some of the needs. ) Unless 

sat~actotily dtitinguishable} we sugges~that f~~miliesin [he few-needs-met catego~ be 

included with the families in the some-needs-met categoy and that the 26.9 percent figure 

be adjusted downward to rejlect only [hose families who apparently received no 

assistance. ” 

OIG RESPONSE: The data in the two tables are consistent. We have modified the 

category names to ensure that they are not misunderstood. 

The OIG reviewers examined the extent to which grantees attempted to meet each 

family’s needs. We excluded families who had no identified needs. If a family had 

needs, we considered any documented effort by the grantee to address the need-­

including providing direct semices, making a referral, or even simply making a 

telephone call--to be an attempt to meet the need. 

The ACF’S comments indicate that it is unaware that grantees are not addressing 

family needs to a large extent. We found clear evidence that many grantees do not or 

are unable to address ~ family needs due to excessive social service staff caseloads, 

general staff overload, lack of available community services, and other reasons. The 

“few or none” category was one category. lf we found absolutely no attempt to meet 

the family’s needs, we judged the effort to be an attempt to meet few or no needs. If 

a family had many serious needs and the grantee made minimal efforts to address only 

one or perhaps two--with no follow-up--we determined that they had met “few or 

none.” 

During training, we instructed reviewers that moderate attempts to meet a portion of 

family needs should be judged as an attempt to meet “some” needs. The ACF should 

note that we also combined “most” and “all” into one category, which makes this 

category’s data somewhat higher than if we had split them. 
ACI? COMMENT (Page 6): “A corresponding modi~cation should be made to the table 

on page 13. Please note in ~hk connection that, as it stands, the table on page 13 h 

inconsistent with the table on page B-9 in [hat i~ acknowledges no Head Start program 

efforts whatsoever on behalf of t}w 26.9 percent of families with identified needs” 

OIG RESPONSE: We have revised the table on page 13 so that “made no attempt to 

meet family needs” is replaced by “attempted to meet few or no family needs.” 

ACF COMMENT (Page 6): “Gran[ees’files and records frequently are incomplete 

inconshtent and dificult to review. T}u3finding provides an important contextual 

consideration wit}lin which the entire OIG Report s}tould be viewed and has a material 

bearing on each of the other three major jindings. As noted in tile Report itse~ and as 

apparent in our own review of it, the OIG study team found it diflcult in many instances 

to determine whether a problem identified during the review was an actual service delivery 

weakness or the result of inadequate record keeping. Since the issue of records and record 

keeping permeates so muc}l of [he Report, wc suggest tilat ibis finding be presented ftnt. ” 

OIG RESPONSE: We believe that the lack of documented evidence that children 

received the required health and social services is the most important finding. The 

ACF should examine the issue of inadequate record keeping under the context that 

serious service delivery problems may exist. Each finding refers to record keeping 

problems, if these indeed exist and influence the findings. 

ACF COMMENT (Page 7): “As indicated eorlier in this memorandum, we also suggest 

modifiing some of the language used to present findings and conclusions which is 

inconsistent with the Report’s own cautionav note regarding the study team k inability ‘to 

determine if the program and performance do[u weaknesses... re~ect serious deficiencies in 

the quality of services provided by Head S[urt’ (pages ii and 7). A few examples of 

language which appear to disregard (his caveat [ire: ‘Grantees are not completing the 

minimum health requirements 10 the ex[en( reported...’ (page 7); ‘apparently...percenl 

received no follow-up treatment’ (page 8); ‘Approximately 140,000--or 26.3 percent-­

families participating in Head Sturt nationally did not receive an FNA’ (page IO); and 

‘made no attempt to meet family needs’ (page 13). In these and other instances, it would 

be more appropriate to use language such as ‘the study team couid find no 

documentation t}lat...’ or ‘local Head S[art program records were inadequate fincomplete 

with respect to...,’ etc. ” 

OIG RESPONSE: As it is written, the report emphasizes that some grantees were 

unable to provide documentation or data to prove that certain services had been 

provided. In other cases, OIG reviewers concluded that selvice delivery problems 

exist, and the problems are not solely related to record keeping problems. 

we areunable to conclude that the absence of documentation is related to file 

incompleteness. The absence of documentation could also indicate that the services 

were not provided. The finding states that “it frequently was difficult to review and 

C-20 
verify performance indicator data.” Other grantees were exemplary in their internal 

recordkeeping. 

The OIG reviewers frequently found blank FNAs in children’s files. To a large extent, 

we found underimmunized children who did not receive any immunizations during the 

year, according to the charts in their files. We found health screen documentation 

that clearly showed that certain assessments were not completed. The ACF should 

understand that these service delivery weaknesses clearly exist. We are unable to 

ascertain the extent of these weaknesses, however, because of some grantees’ poor 

record keeping. 

ACF COMMENT (Page 7): “Finally, we also recommend tilat the Report include 

information on the results of the OIG study team’s follow-up interviews with selected 

programs which are mentioned on page 5. T}le purpose of these interviews was to 

ascertain the reasons for tile apparent discrepancies between the data collected by the 

study teams on si[e and the PIR data. Since the reliabilip of the PIR data k presented as 

an zksue in the Repor~, i~ would be helpJhl to know lhe results of this follow-up, including 

the perceptions of ~heprograms whose opero[ions were the subject of the OIG review. 

Our own informal discussions aboul immunizations with 18 of the programs reviewed by 

the (21G study team indicate that [hew Head Start programs do not consider their records 

to be inadequate and that they believe !ha[ [he PIR information which they submit at the 

end of the program year h accurate. ” 

OIG RESPONSE: We have incorporated information obtained during follow-up calls 

in the final report. We also found that grantees believe that their files are adequate. 

This clearly indicates the need for ACF to take leadership to provide file maintenance 

guidelines. 

We would expect grantees to assert that their PIR information is accurate. Without a 

process to audit or otherwise validate PIR data, neither ACF nor OIG could verify its 

accuracy. 

ACF COMMENT (Page 7): “...severai programs indicated that, al~hough services may 

have been provided, an individual child? heaith record might be incomplete because of 

several factotx the lag-time associated with several centers reporting to cen[ral record 

keeping locations; the dificulty of obtaining actual copies of medical records from 

physicians not reimbursed by the Head Start program itself and the fact that all of a 

child’s records may not be in the master file for each child prior to the actual close-out of 

the program year. ” 

OIG RESPONSE: We contacted the grantees well before our visit and provided them 

with the names of the children whose files we wished to review. We clearly instructed 

them to obtain all ap ProPriate files including health. social semices, enrollment, s~ecial 

needs, education, and anv other files related to the child or familv. When we found a 

pattern of missing information, we coded it as “missing” in our analysis and removed 

these children from the statistical calculations. In addition, all grantees should have 

C-21 
been able to provide complete 198&89 files. We found that grantees performed 

similarly in 1988-89 compared to 1991-92, and the issues of concern--medical and 

dental follow-up, immunizations, and social services among others--have not changed 

since 1988-89. 

RECOMMENDATION #1: The SecretarvofHHS shouldconvene a task force to 

conduct a formal and thoroughreviewofthemana~ementoftheHead startProaam. 

ACF CO MMENT: “The ACF believes that u management review of Head Start would 

be useful in helping to identifi and address issues bearing upon program quality. In a 

March 31, 1993 letter to Senator Kennedy, Chairman of t}le Senate Labor and Human 

Resources Committee, the Secretary indicaled that she has asked for a “top-~o-bottom 

look” at t}~eprogram. The review would be undertaken with the help of an expert study 

panel and the results used to design an improved Head S~artprogram. ” 

OIG RESPONSE: No response is necessary. 

RECOMMENDATION #2: The ACF should create. distribute and mandate specific 

record keeping and children’s files maintenance instructions for aantees. 

ACF COMMENT: “The A CF believes (Ilis recommendation has merit. The agency has 

already begun work on the implementation OJnew record keeping and reporting protoco~ 

for Head Start, entitled die Head Start Family Information System (HSFIS). The new 

system will include information on the services provided to individual Head S~art children 

as well as on family needs and how t}ley are being addressed. As envisioned, HSFIS 

would meet the record keeping needs of local Head Start programs and also facilitate 

their reporting of information to A CYF. A request for clearance to field test HSFIS in 

22 Head Start programs k currently al [he OJJke of Management and Budget for 

approval. ” 

OIG RESPONSE: We have not reviewed the HSFIS and do not know if it should be 

a mandated record keeping system for all grantees. 

RECOMMENDATION #3: The ACF should strengthen its monitorin~ of grantees to 

better identifv and address problem areas. 

ACF COMMENT: “The ACF agrees with [he suggestion under this recommendation 

that the PIR data should be checked f~x- validity on a sample basis. However, other 

aspects of this recommendation, as amplified in [he {CX[on page 15 which follows it, are 

problematic. ” 

“First,although we agree that we need [o continue to strengthen our monitoring capacity, 

it k not clear what the Report means when it states lhat A CF should consider 

“restructuring and condensing its on-site program reviews.” T}le OIG h study did not 

examine 

analyses 

how rnonitoting visits are conducted, and the Report itself presents no findings or 

concerning on-site rnoni[oring.” 

c-22 
“Second, while we agree wi[h [he intent of the suggestion that ACF should devote more 

time and resources to monitoring and providing technical assistance to ~hemost 

problematic grantees, there are severe constraints re!a~ed to its implementation. Tile Head 

Start legti[ation requires that ACF conduct an on-site monitoring visit to one-t}lird of the 

Head Start grantees each year (approximately 460 programs), that all newly established 

programs be vbited early in their second year of operation and that follow-up vkits to 

Head Start agencies be conducted when appropriate. Cuvently available stajjing and 

travel resources are barely adequate to meet these statutory requirements, particularly the 

follow-up visit requirement. In so far as exbting resources permit follow-up vikits, they are 

already being targeted on programs identified as having significant problems. Any 

expansion of on-sile moniton’ng to additional programs would require more resources than 

are currently available. ” 

“We suggest that this recommendation focus on validating the PIR data and that the topic 

of ACF? monitoring processes and priorities be examined as part of the laiger 

management review of Head Start envisioned by the Report k first recommendation. ” 

OIG RESPONSE: We do not believe that a review of the on-site review process is 

necessary in order to suggest that ACF focus its resources to target and provide 

technical assistance and oversight to the grantees that are having the most problems. 

Clearly, some grantees require more assistance than others. We suggest that ACF 

consider conducting a scaled-down on-site review to assess a grantee’s general 

performance. Based on ACF’S findings, further review and technical assistance could 

be provided, depending on the grantee’s need. This would also avoid wasting time 

andresourcesconductinga thoroughon-sitereviewofa granteewithexcellent 

performanceandcleandocumentation. 

We are not suggesting that ACF expand its on-site review program. Conducting a 

scaled-down initial on-site review would allow ACF to continue to meet its 

requirement to visit all new grantees and all existing grantees every 3 years. Initial 

visits could be completed in a fraction of the time that ACF currently spends at each 

grantee. This would allow more time for follow-up visits. 

We agree with ACF that this issue should be addressed during the review of the 

management of Head Start made in Recommendation #1. 

RECOMMENDATION #4: The ACF should develop a modified FNA form that 

should be required of all Head Start ~rantees. 

ACF COMMENT: “We believe that HSFIS will satisfy the intent of this 

recommendation. ” 

OIG RESPONSE: As mentioned above, we have not reviewed the HSFIS. If ACF 

mandates it and it includes a modified FNA, it would satisfj the intent of the 

recommendation. 

C-23 
APPENDIX D 

ASPE AND AS~ CO-NTS ON THE Dm REPORT 

D-1 
--

. 

. . . 

EB19@3 

To:	 Bryan B. Hitchell 

PrincipalDeputyInspectorGeneral 

PROM:	 Acting Assistant Secretq 

for Plannhg and Evaluation 

s~JECT :	 OIG Draft Report “Evalua&g Head Start Expansion 

Through Perfo~ce lndicat~rs,” OE1-09-91-00762 

CONCURRENCE 

Thank you for the opportunity to review this draft repoti. AS my Staff and 1 have noted in previous discussions, this repotiand 

its companionreporton granteeexperiences highlighta numberof

issues which will become central to pltig forfuture 

expansions of the HeadStartProgr-. I commendtheworkyour staffhave done in prepaing both repo~. 

s


Ow


Shl


wox~g closely with your staff on upc~g 

I believe the prmess of dismssfig a working clearance document was beneficial and, from 

least, clears the way forquickapproval of 

. We look forward to 

xepoxts as well. 

draft prior to this 

our stand~intat 

this version. 

I r.
 ,.-


-4 ..
 *
 = 

-

k­ . 

4 

D-2

paragraph

� 

To: Bryan B. Mitchell . Principal Deputy Inspetior1 Geneml 

% � 

=x: Elizabeth M. ~ames 

Acting Assistmt Secre ry fOr Man ent and Budget 

Tp uating He ~ ~ansion 

Smm: OIG Draft Repoti: n 

Through Perfo~nce Indicators, 1-09-91-00762
 n


Thti you for the oppo~ity . to review the draft re caters. . aluatlna Head Start ~on ~rauqh Perfo~nc e 

This repofi as well as OIGtS two other cmapartlon repo~ on Head 

Start expansion will be very valuable to the pro- at this 

point in its histoq. I have seve=l specific comments 

concemtig this repofi. 

Page ii and Page 9 - Finding “Chiltien may not be fully immunized 

before they leave the Head Stati Progr~.t$ Children who,mter 

Head Stan without any previous immunizations or are seriously 

behind with their serial immunizations cannot besuggested ‘ftily 

immunized’ by the time m~~~enleave myHead Start. fully ~unized~ or they not be replac=ent sentence: 

up-to-ate with their imniunizatio=, before they leave the Eead 

sia* Program.” 

p.2- We reco=end replacing the COL- ‘Additional Exp-lon 

Rxnding~ with ‘CO~/Quali~ Improv-ent.’c The currentcalshows the COLA and QualitYImprov-at along wi~ T=in~g ~~ Technical Assis~ce. We believe that the Training and Technical 

Assist=ce number should not be included because these ** are


not as directly related to imprmtig a grantee qualityas tie


COLA or Quality Improv-at fun~ are. Also, please verifY final “


colm numbers with the Head St- Bureau.
 .


Suggested replac-at column: 

n 

Expansion I


mansion 11 

Expansion III 

Expansion IV


Expansion V 

COLA/Qualit’Y


Impravemmt .


$49,000,000 

. ~—— . ..* 

. ‘$195,000,000 

$108,535,000 

$189,828,000 

P.3- Please revise the first sent=ce in the first suggested
 to


reflect the change in the table on the previo= page.


replacmmts “The COLA/QualitY Improv=-t funds, identified on


the cha~ on the preceding page~ were set-aside funds given to


D-3 
---

,+ 

-*:* 

Page 2 - Bryan B. Mitchell 

. .- — 

If you have any questionsor con~ 

please con~­ Karen S&afar (690-6238) 

. 

. 

.. 

5 

. .

., 

.0 

,. . . 1 

. : .’, 

D-4 
? 

Department of Health and Human Sefices 

OFFICE OF 

INSPECTOR GENERAL 

EVALUATING HEAD START EXPANSION 

THROUGH PERFORMANCE INDICATORS 

MAY 1993 
OFFICE OF IIWPE~OR GENERAL . 

The mission of the Office of Inspector General (OIG), as mandated by Public Law 95-452, as amended, 

is to protect the integrity of the Department of Health and Human Semites’ (HHS) programs as well 

as the health and welfare of beneficiaries served by those programs. This statuto~ mission is carried 

out through a nationwide network of audits, investigations, and inspections conducted by three OIG 

operating components: the Office of Audit Semites, the Office of Investigations, and the Office of 

Evaluation and Inspections. The OIG also informs the Secretaxy of HHS of program, and management 

problems, and recommends courses to correct them. 

OFFICE OF AUDIT SERVICES 

The OIG’S Office of Audit Services (OAS) provides all auditing services for HHS, either by conducting 

audits with its own audit resources or by overseeing audit work done by others. Audits examine the 

performance of HHS programs and/or its grantees and contractors in carrying out their respective 

responsibilities and are intended to provide independent assessments of HHS programs and operations 

in order to reduce waste, abuse, and mismanagement and to promote economy and efficiency 

throughout the Department. 

OFFICE OF INVESTIGATIONS 

The OIG’S Officx of Investigations (01) conducts criminal, civil, and administrative investigations of 

allegations of wrongdoing in HHS programs or to HHS beneficiaries and of unjust enrichment by 

providers. The investigative efforts of 01 lead to criminal convictions, administrative sanctions, or civil 

money penalties. The 01 also oversees State Medicaid fraud control units which investigate and 

prosecute fraud and patient abuse in the Medicaid program. 

OFFICE OF EVALUATION AND INSPECI’IONS 

The OIG’S Office of Evaluation and Inspections (OEI) conducts short-term management and program 

evaluations (called inspections) that focus on issues of concern to the Department, the Congress, and 

the public. The findings and recommendations contained in these inspection reports generate rapid, 

accurate, and up-to-date information on the efficienq, vulnerability, and effectiveness of departmental 

programs. This report was prepared in the San Francisco regional office under the direction of Kaye 

D. Kidwell, Regional Inspector General, and Paul A. Gottlober, Deputy Regional Inspector General. 

Project staff included: 

SAN FRANCISCO HEADQUARTERS 

Jemifer Mallen, Co-Project Mder Wm. Mark Krushat, SC.D.,Mathematical Statistician 

Brad Rollin, Co-Project Leader Alan Levine, Program Specialist 

Elizabeth Bell 

Kathy Dezotte ATLANTA 

Robert Gibbons 

Deborah Harvey Betty Apt 

Ellen Kotler Ron Kalil 

Donald Loeb Chris Koehler 

Brian Pattison 

CHICAGO 

Suzanne Johnson 

Alicia Simon OFFICE OF INVESTIGATIONS 

BOSTON Baldur Krahl Jose Munoz-Caro 

Timothy Corbett AID lXl FAMILIES WITH DEPENDENT CHILDREN 

Lori Rutter 

Agustin Quiles-Ocasio 

Irelys Ramirez-Ramos 

For additional copies of this report, please contact the San Franciscoofficeat (415)556-6830.