The 2004 Annual Report to the Office of Management and Budget:
Implementing the VFC-AFIX Project:
A National Strategy to Improve the Quality of
Immunization Practices among VFC Providers
May 2005
Background/Introduction:
In 1995, Congress directed the Centers for Disease Control and Prevention (CDC) to set guidelines for assessing coverage levels in all public clinics as part of the federal funding for immunization programs.  A continuous quality improvement strategy was developed to assess coverage levels and provide insight on how to improve coverage levels.  This strategy is known by the acronym AFIX.  The four components of AFIX are Assessment of immunization coverage levels, Feedback of the assessment findings to providers, Incentives to motivate and acknowledge change, and eXchange of information on best practices.  This strategy has been documented in the CDC Guide to Community Preventive Services as an effective method to improve and sustain immunization coverage levels.
Private provider participation in the VFC program has created a shift in the provision of vaccine services over the last decade from public health clinics to private provider offices.  Recognizing this shift, an objective related to the assessment of immunization coverage levels was included in the Healthy People 2010 goals.  The objective is to increase the proportion of providers who have measured the vaccination coverage levels among children in their practice population within the last two years.  The Healthy People 2010 report included a 1997 baseline measurement for providers that had assessed their coverage levels in the last two years as 6% for private providers and 66% for public providers.  
In 2000, CDC/National Immunization Program (NIP) launched the VFC-AFIX initiative which linked AFIX, the continuous quality improvement strategy, with the VFC program. The year 2004 represents the fourth year that the 61 eligible VFC grantees requested and received funding for the VFC-AFIX project.  The purpose of this project is to assess and improve immunization delivery practices at the provider level to assure that VFC-eligible children are receiving quality services.
2004 NIP Programmatic Activities
The activities in 2004 enhanced the VFC-AFIX project at the federal and grantee level. The completed and on-going priority activities for NIP/AFIX staff are described below. 
* The Core Elements of AFIX Training and Implementation was revised and published as a 2nd Edition.  The Core Elements of AFIX Training and Implementation was first released in 2002 for use by grantees in training their staff and implementing the AFIX process at the provider level. This guide was revised and released in 2004 at the National Immunization Conference. The 2nd edition of this document included more information about how to implement AFIX for adult populations and how to use immunization registries for the assessment component of AFIX. Hard copies of the Core Elements were distributed to the grantees in 2004.  The Core Elements document 2nd edition is available electronically on the AFIX website http://www.cdc.gov/vaccines/programs/afix/ . A hard copy of the Core Elements document is attached for review.
* In 2004, NIP/AFIX staff and grantee Clinic Provider Assessment Workgroup (CPAWG) members developed AFIX Standards for implementation by all grantees. The purpose of AFIX Standards is to provide grantee AFIX staff information and guidance on the process for developing fundamentally sound AFIX programs that meet CDC grant requirements. The Standards include specific components that address the development, implementation and evaluation of an AFIX program at the grantee level. The AFIX Standards also provide NIP staff with the ability to evaluate the grantees performances in effectively implementing AFIX programs. The AFIX Standards were officially released at the 2005 National Immunization Conference, and hard copies of the AFIX Standards were mailed to grantees in April 2005.  The AFIX Standards also are posted on the AFIX website under Major Links, Whats New? and Publications.  The website link to the AFIX Standards document is http://www.cdc.gov/vaccines/programs/afix/stds-guide.htm.   A hard copy of the AFIX Standards is attached for review.
* VFC-AFIX project staff coordinated, moderated or presented the following workshop sessions at the 2004 National Immunization Conference: 
* Core Elements of AFIX Two: Whats New? 
* Preliminary Results of the VFC/AFIX Activities Survey of Program Managers
* The Immunization Grant Process: What Does it Really Mean? 
* Making the Transition: From Chart-Based to Registry-Based Assessment
* VFC-AFIX project staff coordinated and facilitated discussions during quarterly conference calls between grantees and NIP staff regarding VFC-AFIX activities.
* NIP/VFC-AFIX staff developed an AFIX grant review tool to create a standard process for reviewing the AFIX information provided by the grantees in the 2005 grant applications submitted in the fall of 2004.
* Progress slowed on the Comprehensive Clinic Assessment Software Application (CoCASA) project. This interruption was due in part to contract issues that delayed hiring the majority of software programming staff that work on this project.  Once completed, CoCASA will combine and improve the functionality of the existing Clinic Assessment Software Application (CASA), the Adult Clinic Assessment Software Application (ACASA), and the VFC-AFIX Evaluation Software.  Work conducted on this project in 2004 included advancement of the CoCASA functionality requirements with specific focus on the VFC-AFIX Evaluation software component since this component would be released for use by all grantees in January 2005. When the software is completely finished, it will also include a tutorial that provides instruction on the AFIX process, using CoCASA as a tool for implementing AFIX and using CoCASA to document VFC-AFIX activities.  NIP staff worked on the content development for the tutorial and recorded narration for the completed sections of this tutorial. The updated timeline for release of the full software product is late 2005.  
2004 Grantee Programmatic Activities
Grantees actively worked to improve VFC/AFIX operations in 2004.  This section of the report illustrates some of the new or refined VFC/AFIX activities conducted by a sample of grantees in 2004. 
* Chicago 
The Chicago VFC-AFIX staff recorded VFC provider site visits in the CDC VFC-AFIX Evaluation software to assist with organizing the VFC/AFIX site visits and producing aggregate results.  The site visits were entered monthly and reviewed to develop a follow-up plan for each individual site.   
* Indiana
A provider satisfaction survey was conducted to improve interaction with providers by making the forms easier to understand, reviewing and improving CASA/AFIX practices, increasing the educational material distributed to providers, and evaluating the usefulness of the Vaccine E-letter.  
The Immunization Program routinely promotes reminder/recall systems at all AFIX site visits. Field representatives discuss reminder/recall during site visits with public providers.
* Massachusetts
Follow-up activities by partner organizations continued with a particular focus on ensuring implementation of reminder/recall systems.
In order to streamline the site visit process and reduce the number of forms to be completed, a comprehensive assessment tool was created, combining the multiple forms previously used into one comprehensive assessment questionnaire.
Training was provided in May for all immunization program staff conducting assessments. This training included review of proper vaccine storage and handling, documentation of immunizations, compliance with VFC Program requirements, LQA methodology, and feedback/exchange of information with providers.  
The Massachusetts immunization program will update and distribute Immunization Tool Kits to all newly assessed primary care provider sites.
* Missouri
During 2004, Missouris Immunization Quality Improvement Workgroup continued to develop and expand the VFC Site Visit and AFIX Policies and Procedures manual.  A standardized action plan was developed and incorporated into VFC site and AFIX visits to ensure that visits are conducted in a uniform manner.  
* Oregon
Oregon staff held community provider meetings across the state in cooperation with LHDs. Several of these meetings were titled AFIX Exchange and focused on sharing information about AFIX.   
* San Antonio
San Antonio employed AFIX methodologies utilizing CASA to evaluate WIC staff participation and assessment accuracy. These methods helped to identify deficiencies in the WIC staffs immunization assessment processes and provided valuable feedback to the WIC Director for future improvement efforts.  
* Tennessee
VFC/AFIX trainings were conducted in April and August to an audience of 52 participants.  Instruction was given on VFC/AFIX program overview and standards, vaccine ordering, 24 month-old survey, record de-duplication, Sensaphone use, and Tennessee Web Immunization System (TWIS).  
* Texas
The Immunization Branch implemented a review process; every reviewer hired by the contractor will be observed conducting a minimum of one site per year.  A review tool has been developed, and feedback is provided to the contractors administrative office for coaching/training purposes.   
* Vermont
Development of the AFIX program included creation of a program specific AFIX manual incorporating the CDC AFIX Core Elements.
VFC AFIX FY 2004 Awards to Grantees
In 2004, a total of $ 16.4 million in VFC AFIX funds was awarded to grantees.  See attached table.  
Summary of Program Findings
The AFIX Standards were developed to provide a framework for grantees to standardize their VFC/AFIX programs within their geographic location to improve the quality of the VFC/AFIX provider interaction.  A wide variety of challenges continue to face the grantees implementing the VFC/AFIX initiative including grantee imposed travel restrictions, limited staffing and staff turnover.  All these factors impact the quality and effectiveness of grantee provider site visits.  
Analysis of 2004 Grantee Submitted Data 
CDC/NIP created the VFC/AFIX Evaluation Software as a tool for grantees to monitor their VFC/AFIX activities. This software is a Microsoft ACCESS database that can be used to store the VFC/AFIX site visit data in accordance with the report requirements due at the end of each calendar year. If this software is used by the grantees, then the aggregate data required for submittal to CDC/NIP can be automatically generated by the report options built into the software. The VFC/AFIX Evaluation Software was initially distributed to the grantees in 2002, and a revised copy has been provided annually through 2004. 
Grantees are not required to use the VFC/AFIX Evaluation Software; however, they are advised to develop their own tracking system that would capture the same data fields if they choose not to use the software. Grantees were required to submit documentation of their 2004 VFC/AFIX activities as part of the Annual VFC Management Survey due March 1, 2005. The Annual VFC Management Survey data were collected using a web-based reporting method. Of the 61 eligible grantees, 59 submitted data for the Annual VFC Management Survey.  
VFC/AFIX Staff
The number of full-time employees (FTEs) working on VFC/AFIX activities is tabulated in Table 1 below. In total, 306 FTEs are employed at the state level, and 158 FTEs are employed at the local level. Together, 464 FTEs at the state or local level are currently working on VFC/AFIX related activities across the country.  Additionally, there are 77 contract staff members working on VFC/AFIX related activities at the state or local level.
As shown in Table 1, these numbers can be categorized by new and existing positions for calendar year 2004.  The grantee VFC/AFIX projects can be carried out at the state level or the local level by immunization program staff or by staff hired through contracts with outside agencies. 
Table 1. Number of FTEs Working on VFC/AFIX Project CY2004
Personnel
Program Staff*
Contract Staff**
Total
State
Local
State
Local
State
Local
New Positions
15
0
7
4
22
4
Existing Positions 
291
158
52
14
343
172
Total 
306
158
59
18
365
176
*State Program Staff: state-employed staff working on VFC/AFIX at the state immunization program level. Local Program Staff: local health department staff funded with federal funds who work on VFC/AFIX.
**State Contract Staff: VFC/AFIX staff hired by the state immunization programs through third party contracts with federal funds. Local Contract Staff: VFC/AFIX staff at local health departments hired through contracts with outside agencies using federal funds. 
Provider Information
Table 2 below includes the number of provider sites that received at least one visit during 2004 for each category of provider. Public providers are divided into three categories. The Public category includes local health departments and Indian Health Service clinics.  Community or Migrant Health Centers (C/MHC) includes Federally Qualified Health Care Centers, and the Other Public category captures all other facilities that are not included in the other two categories. Private represents all private providers that received at least one visit in 2004. The providers are categorized into VFC-Enrolled sites and Non-VFC Enrolled sites. Including both types of provider sites, a total of 24,662 provider sites received at least one visit in calendar year 2004. Of the 24,662 provider sites, 99 % were enrolled in the VFC program. 
The 24,400 VFC-Enrolled provider sites that were visited constitute 55% of the total      active VFC-Enrolled provider sites, an increase from 52% (22,732 provider sites) in 2003. Table 3 illustrates the percentage of visits in the public sector and the private sector.  The percentage of visits in the private sector increased from 47% in 2003 to 53% in 2004.
Table 2. Number of Provider Sites Receiving at Least One visit, CY2004
Provider Information
Public
C/MHC*
Other Public
Subtotal All Public
Private
Total
VFC-Enrolled Provider Sites 
3,435
2,672
1,244
7,351
17,049
24,400
Non-VFC Provider Sites 
7
13
1
21
241
262
Total 
3,442
2,685
1,245
7,372
17,290
24,662
*Community or Migrant Health Center
Table 3. Number and Percent of VFC-enrolled Provider Sites Receiving at Least One Visit, CY2004
Provider Information
All Public
Private
Total
Number of VFC-Enrolled Provider Sites
11,809
32,292
44,101
Number (Percent) of VFC-Enrolled Providers Who Received at Least One Visit, CY2004
7,351
(62%)
17,049
(53%)
24,400
(55%)
Site Visit Information
While 24,400 VFC-enrolled provider sites were visited in 2004, the actual number of VFC Only, AFIX Only, VFC/AFIX combined and educational visits to a VFC-enrolled provider site totaled 27,525. Table 4 below details the number of visits to VFC and Non-VFC provider sites by visit type and provider type. As the table shows, 8,864 visits were conducted in public VFC-enrolled sites and 18,661 visits were conducted in private VFC-enrolled sites. In addition, 211 visits (AFIX Only and Educational) were conducted in Non-VFC enrolled provider sites. 
Table 4.  Total Number of Visits by Provider Type, CY2004
Type of Visit
Public
VFC Enrolled
Public Non-VFC
C/MHC VFC Enrolled
C/MHC Non-VFC
Other Public VFC Enrolled
Other Public Non-VFC
Private VFC Enrolled
Private Non-VFC
VFC Only*
1,148
606
511
4,455
AFIX Only**
794
1
257
0
127
2
1,356
172
VFC/AFIX Combined***
1,906
1,731
538
9,370
Educational
573
0
444
0
229
2
3,480
34
* VFC Only is defined as a visit to a VFC enrolled provider to ensure compliance with VFC program requirements. 
** AFIX Only is defined as a quality improvement strategy utilizing assessment of immunization records, feedback, incentive, and exchange of information through performance measurement, diagnosis of service delivery problems, and data feedback during a visit to a medical practice.  One AFIX visit should contain an assessment and a feedback component even though more than one physical visit to the provider site may be required to complete the assessment and the feedback sessions. 
*** A VFC/AFIX Combined site visit is defined as a visit to a VFC-enrolled provider site which integrates the review to ensure compliance with VFC program requirements and immunization record assessment and feedback activities. 
Table 5 includes additional documentation regarding site visits in CY 2004. Repeat AFIX visits are a subset of the AFIX visits documented in Table 4, and they are included to illustrate the number of visits that occurred at a site that had previously received an AFIX visit between January 1, 2000, and December 31, 2003. The nature of AFIX as a continuous quality improvement strategy requires that provider sites are visited on more than one occasion to evaluate incremental progress. The number of repeat AFIX visits allows CDC/NIP to track the grantees progress in implementing this ongoing quality improvement strategy. The VFC Follow-Up visits describe the number of visits that occurred as a result of a problem and/or concern found at the initial VFC visit. The information in Tables 4 and 5 reveals that grantees are actively visiting provider sites, following up on problems identified in previous visits, providing education as well as service, and ultimately building relationships with staff.  
Table 5. Additional Visits, CY2004
Type of Visit
Public
VFC Enrolled
Public Non-VFC
C/MHC VFC Enrolled
C/MHC Non-VFC
Other Public VFC Enrolled
Other Public Non-VFC
Private VFC Enrolled
Private Non-VFC
Repeat AFIX*
1,386
0
697
0
119
0
3,098
50
Follow-up VFC**
1,510
613
70
3,006
*Repeat AFIX: the number of AFIX visits (includes AFIX only and Combined VFC/AFIX) from Table 4 that are repeat assessments (e.g. the provider received an assessment during a previous year).
**Follow-up VFC: the number of visits completed to evaluate provider response to previously identified problems found during the initial VFC site visit.  
As part of the annual grant application process, grantees are required to specify the proposed number of site visits to be conducted in the upcoming calendar year. For the 2004 grant applications, the planned number of site visits included three categories: VFC Only, AFIX Only and VFC/AFIX Combined. In an effort to examine the accuracy with which a grantee can estimate VFC/AFIX activities, the number of proposed site visits from the grant applications were compared with the actual number of site visits (for the categories of VFC Only, AFIX Only and VFC/AFIX Combined). Grantees were not able to conduct the total number of proposed site visits in CY 2004 (see Table 6); however, the grantees did exceed the number of visits completed in CY2003 by 762.
Table 6.  Proposed and Actual Number of Site Visits, CY2004 
Type of Site Visit
Public Provider*
Private Provider
Total Visits
Proposed
Actual
Proposed
Actual
Proposed
Actual
VFC only
2,682
2,265
4,972
4,455
7,654
6,720
AFIX only
1,265
1,181
1,579
1,528
2,844
2,709
VFC/AFIX combined
5,700
4,175
9,496
9,370
15,196
13,545
Total visits
9,647
7,621
16,047
15,353
25,694
22,974
*Public provider includes Public, C/MHC, and Other Public.
Assessment Outcome Measures for Public Providers 
Of the 61 grantees that receive funds for VFC-AFIX activities, 55 submitted public provider data for the annual report (see Appendix A, Table 1). Of the 55 that submitted data, 47 use the CASA, Mini-CASA or equivalent method (designated as CASA). Three grantees use the Hybrid method, and five grantees use a combination of CASA and Hybrid (designated as Both). Information on difference between the CASA and Hybrid methods is found in Appendix B of the results section. Grantees that use CASA as the assessment method will receive an estimated immunization coverage level for each provider assessed. Grantees that use the Hybrid assessment method will receive a result indicating the provider is above or below a pre-determined threshold immunization coverage level. (See Appendix B for more in-depth explanation.) 
The majority of the grantees (28/55) assess children 24-35 months of age. Other age groups reported include 19-35 months (10 grantees) and 12-23 months (2 grantees); five grantees reported that they assess both ages 12-23 and 24-35 months. Ten grantees responded to this question with other. 
For those grantees using CASA for the assessment, nine grantees reported provider vaccination coverage levels averaging 80% or higher. Seventeen grantees reported provider vaccination coverage levels averaging 70-79 %.  Of the remaining results, 16 grantees indicated vaccination coverage levels averaging 60-69%, and ten indicated vaccination coverage levels below 60%. In comparison to the same results in 2003, these outcomes show improvement. In 2003, 14 grantees indicated vaccination coverage levels below 60%. 
For those grantees using the Hybrid Method, threshold levels of 65, 70, 75, 80 and 90 percent were used. The most common threshold levels were 75, 80 and 90 percent.  For those grantees that used the Hybrid methodology, two grantees reported the number of providers above the selected threshold level as greater than 80% of the providers assessed. 
Note: The above numbers are not mutually exclusive  meaning one grantee could have done some AFIX visits using the Hybrid method and some visits using the CASA, Mini-CASA, or Equivalent method. 
Note: Coverage levels were rounded to the nearest whole percentage point for decimals ?0.5
Assessment Outcome Measures for Private Providers
Of the 61 grantees that receive funds for VFC-AFIX activities, 55 submitted private provider data for the annual report (see Appendix A, Table 2). Of the 55 that submitted data, 47 use the CASA, Mini-CASA or Equivalent method (designated as CASA). Three grantees use the Hybrid method, and five grantees use a combination of CASA and Hybrid (designated as Both).   
A large number of the grantees (28/55) assess children 24-35 months of age. Other age groups reported include 19-35 months (11 grantees) and 12-23 months (2 grantees); and five grantees reported that they assess children ages 12-23 and 24-35 months of age. Nine grantees responded to this question with other. 
For those grantees using CASA for the assessment, ten grantees reported provider vaccination coverage levels averaging 80% or higher. Sixteen grantees reported provider vaccination coverage levels averaging 70-79%.  For the remainder, 15 grantees indicated vaccination coverage levels averaging 60-69%, and 11 indicated vaccination coverage levels below 60%.  
For those providers using the Hybrid method, threshold levels of 65, 70, 75, 80 and 90 percent were used. The most common threshold levels were 75 and 80 percent.  Three grantees using the Hybrid Method reported 80% of the providers assessed as above the designated threshold level; one of whom reported 100% of the providers assessed as above the 90% designated threshold level. 
Note: The above numbers are not mutually exclusive  meaning one grantee could have done some AFIX visits using the Hybrid method and some visits using the CASA, Mini-CASA, or Equivalent method. 
Note: Coverage levels were rounded to the nearest whole percentage point for decimals ?0.5
Change in coverage when using CASA to assess public providers
Forty-two grantees used the CASA method to assess coverage during previous and CY2004 assessments in the public sector (see Appendix A, Table 3). The majority (32 of 42 ) assessed for 4 diphtheria, tetanus, and pertussis (DTaP) doses; 3 polio doses; 1 measles, mumps and rubella dose (MMR), 3 haemophilus influenzae type b doses (Hib) and 3 hepatitis B doses (HepB) for previous and CY 2004 assessments. This series of vaccinations is documented as 4:3:1:3:3.  In addition, six grantees reported change in coverage levels for an immunization series that included the varicella vaccine. (The 4:3:1:3:3:1 series includes 4 DTaP doses; 3 polio doses; 1 MMR, 3 Hib, 3 HepB, and 1 dose of varicella.) Three grantees assessed for 4 doses of DTaP, 3 doses of polio and 1 dose of MMR for previous and CY 2004 assessments; commonly referred to as the 4:3:1 series. One grantee assessed for the 3:2:2:2 series for previous and CY 2004 assessments, which is 3 doses of DTaP, 2 doses of polio, 2 doses of Hib and 2 doses of HepB at 12 months of age. 
Twelve grantees documented an increase in average public provider vaccination coverage levels to be 10% or greater compared to eight grantees in 2003. Twenty grantees found the average public provider vaccination levels improved 1-9% as compared to 12 grantees in 2003. The average public provider vaccination coverage levels did not change for one grantee.  Nine grantees reported the change in average public provider vaccination as a negative value in 2004 as compared to 17 grantees in 2003. The negative values for the change in coverage only ranged from -1% to -8% for seven of the nine grantees, and -13% and -16% for each of the other two. This is an improvement when compared to the 2003 results which included a negative value change in coverage ranging from -1% to -27.3%.
One grantee also documented a change in coverage levels among public providers using the Hybrid method. This grantee reported an increasing percentage of providers who met the designated 80% threshold level from 43% to 51%.
Note: Coverage levels were rounded to the nearest whole percentage point for decimals ?0.5. Values between -0.4 and 0.4 were defined as 0 values and interpreted as sites with no change.
Change in coverage when using CASA to assess private providers
Thirty-nine grantees used the CASA method to assess private provider coverage levels during previous and CY2004 assessments in the private sector (see Appendix A, Table 3). The majority (31 of 39) assessed for the 4:3:1:3:3 series for previous and CY 2004 assessments. Five assessed for the 4:3:1:3:3:1 series and three grantees assessed for the 4:3:1 series. 
Eleven grantees documented change in the average private provider vaccination level to be 10% or greater as compared to five grantees in 2003. Nineteen grantees found the average public provider vaccination level to change between 1-9%.  Overall, more than half of the grantees that submitted 2004 data (n=30) reported an increase in the average private provider vaccination levels compared to only 21 grantees reporting an increase in 2003. The average private provider vaccination coverage levels did not change for three grantees. The change was negative for six grantees in comparison to the 16 grantees that reported negative change in 2003. This negative change ranged from -1 % to -7%.
Two grantees documented a change in coverage levels among private providers using the Hybrid method. Both grantees reported an increasing number of providers who met the designated 80% threshold level. One grantee reported an increase in the percentage of providers that met the threshold from 64% to 67% and the other grantee reported an increase from 56% to 87% of providers. 
Note: Coverage levels were rounded to the nearest whole percentage point for decimals ?0.5. Values between -0.4 and 0.4 were defined as 0 values and interpreted as sites with no change.
2004 NIP Training and Education Activities
In 2004, NIP staff working on the VFC/AFIX project limited the number of trainings in 2004 to focus on the development of a training curriculum for a series of National AFIX trainings in the summer of 2005 that would incorporate training on the new CoCASA software.  Due to delays beyond the control of AFIX/NIP staff, the completion date of the CoCASA was extended to late 2005.  This delay necessitated that the National AFIX trainings be postponed.  During 2004, NIP staff conducted on-site AFIX and/or CASA trainings to the following grantees:
* Mississippi
* Kansas
* Missouri
* Regional Training for Oregon, Washington and Alaska
In addition conference calls were held with the following grantees to provide guidance on revising, restructuring and/or improving their respective AFIX programs: 
* Georgia
* Michigan
* Tennessee
APPENDIX A:
Tables
Table A-1: Assessment Outcome Measures for Public Providers by Grantee, CY2004
Grantee
Age Group Assessed
(Months) (1)
Assessment Method (2)
Vaccination Series Measured with CASA (3)
Number Assessed with CASA (4)
Minimum Coverage Level (5)
Maximum Coverage Level (6)
Crude Average Coverage Level (7)
Vaccination Series Assessed with Hybrid (8)
Number Assessed with Hybrid (9)
Hybrid Threshold Level (10)
Number of Providers At or Above Threshold Level (11) 
Alabama
19-35
casa      
4:3:1:3:3
32

100
68.2




Alaska
Other
casa      
4:3:1:3:3
27
46
100
76




Arizona
Other
casa      
4:3:1:3:3
24
0
97
68.5




Arkansas
24-35
casa      
4:3:1:3:3
93
44
100
77




California
Other
both      
4:3:1:3:3
331
0
100
67.6
4:3:1:3:3
2
80
1
Chicago
24-35
both      
4:3:1:3:3
40
21
96
52.6
4:3:1:3:3
4
75
4
Colorado
19-35
both      
4:3:1:3:3
22
22.2
99.9
73.2
4:3:1:3:3
36
65
21
Connecticut
24-35
casa      
4:3:1:3:3:1
28
36.5
100
81.3




District of Columbia
19-35
casa      
4:3:1:3:3
20
38
88
69




Florida
24-35
casa      
4:3:1:3:3
121
33
100
78.7




Georgia
24-35
casa      
4:3:1:3:3:1
191
0
100
76




Guam
24-35
casa      
4:3:1:3:3
1
0
0
76




Hawaii
24-35
casa      
4:3:1:3:3
9
27
89
61.2




Idaho
19-35
casa      
4:3:1:3:3
27
40
100
83.3




Illinois
12 & 24
casa      
4:3:1:3:3:1
57
0
88.9
44




Indiana
19-35
casa      
4:3:1:3:3
169
0
100
82.5




Iowa
Other
casa      
4:3:1:3:3
144
27
100
88.7




Kansas
12 & 24
casa      
4:3:1:3:3
127
15
100
79.3




Kentucky
Other
casa      
4:3:1:3:3:1
143
0
97
75.8




Louisiana
Other
both      
4:3:1
116
19.2
100
63.3
4:3:1
30
90
11
Maine
24-35
casa      
4:3:1:3:3:1
63

100
45.6




Maryland
24-35
casa      
4:3:1:3:3:1
29
54.5
100
78.8




Massachusetts
24-35
hybrid    




4:3:1:3:3
55
80
28
Michigan
19-35
hybrid     




4:3:1:3:3:1
218
70
105
Minnesota
24-35
casa      
4:3:1:3:3
18
17
100
66




Mississippi
Other
casa      
4:3:1
282
0
100
78.3




Missouri
12 & 24
casa      
4:3:1:3:3:1
176
6.7
95
53.3




Grantee
Age Group Assessed
(Months) (1)
Assessment Method (2)
Vaccination Series Measured with CASA (3)
Number Assessed with CASA (4)
Minimum Coverage Level (5)
Maximum Coverage Level (6)
Crude Average Coverage Level (7)
Vaccination Series Assessed with Hybrid (8)
Number Assessed with Hybrid (9)
Hybrid Threshold Level (10)
Number of Providers At or Above Threshold Level (11) 
Montana
24-35
hybrid    




4:3:1:3:3
65
90
65
Nebraska
19-35
both      
4:3:1:3:3
54
36.4
100
77.3
4:3:1:3:3
5
75
0
Nevada
24-35
casa      
4:3:1:3:3
21
35.2
80
58.2




New Hampshire
24-35
casa      
4:3:1:3:3
43
0
100
93




New Jersey
24-35
casa      
4:3:1
23
40.4
100
73.2




New Mexico
12 & 24
casa      
4:3:1:3:3:1
29
33
100
75




New York State
24-35
casa   
4:3:1:3:3
77
0
100
67.2




New York City
24-35
casa      
4:3:1:3:3
17
43
100
69




North Carolina
24-35
casa      
4:3:1:3:3
100
12
96
74




North Dakota
24-35
casa      
4:3:1:3:3
20
77.8
100
91.9




Ohio
24-35
casa      
4:3:1:3:3
77
22.4
94.4
64.5




Oklahoma
24-35
casa      
4:3:1:3:3:1
47
0
85
53.1




Oregon
12 & 24
casa      
4:3:1:3:3
53
33.3
76.5
58.6




Pennsylvania
24-35
casa      
4:3:1:3:3
65
81.2
84.2
82.7




Philadelphia
19-35
casa      
4:3:1:3:3
18
0
100
76




Puerto_Rico
Other
casa      
4:3:1:3:3:1
216
0
100
46




Rhode Island
12-23
casa      
3:2:2:2
2
86
89
87.5




San Antonio
Other
casa      
4:3:1:3:3
18
0
100
83.5




South Carolina
24-35
casa      
4:3:1:3:3
30
19.3
98
62.5




Tennessee
24-35
casa      
4:3:1:3:3:1
65
125
100
57.5




Texas
Other
casa      
4:3:1
673
0
100
67




Utah
24-35
casa      
4:3:1:3:3
51
9.4
96.6
60.7




Vermont
19-35
casa      
4:3:1:3:3
20
25
100
68.5




Virginia
12-23
casa      
4:3:1:3:3:1
25
13
90
53.8




Washington
19-35
casa      
4:3:1:3:3
30
20
100
61.2




West Virginia
24-35
casa      
4:3:1:3:3
31
29
100
69.1




Wisconsin
24-35
casa      
4:3:1:3:3
5
56.3
93.7
76.2




Wyoming
24-35
casa      
4:3:1:3:3:1
546


75




Although Delaware, South Dakota, the Mariana Islands and the Virgin Islands conducted site visits, they did not report any measured assessment outcomes prior to the deadline for this report. Houston and American Samoa did not report any data for this survey.
1. Age Group Assessed: the age range of the patients included in the assessment
2. Assessment Method: method used to evaluate vaccine coverage level (CASA or Hybrid)
3. Vaccination Series Measured: vaccine series (type and number of doses of vaccine) used to evaluate up-to-date status during the assessment visit. All coverage levels will refer to the completion of this series. 
4. Number Assessed with CASA: number of providers that received an assessment using the CASA method.
5. Minimum Coverage Level: among all providers assessed, the lowest determined vaccination coverage for the vaccination series measured
6. Maximum Coverage Level: among all providers assessed, the highest determined vaccination coverage for the vaccination series measured
7. Crude Average Coverage Level: an unweighted average vaccination coverage level among all provider sites
8. Vaccination Series Assessed with Hybrid: vaccine series (type and number of doses of vaccine) used to evaluate up-to-date status during the assessment visit when using the Hybrid Method.  
9. Number Assessed with Hybrid: number of providers that received an assessment using the CASA method.
10. Hybrid Threshold Level: vaccination coverage level at which providers are expected to be performing at or above
11. Number of Providers At or Above Threshold Level: number of providers at or above threshold level 
Table A-2: Assessment Outcome Measures for Private Providers by Grantee, CY2004
Grantee
Age Group Assessed
(Months) (1)
Assessment Method (2)
Vaccination Series Measured with CASA (3)
Number Assessed with CASA (4)
Minimum Coverage Level (5)
Maximum Coverage Level (6)
Crude Average Coverage Level (7)
Vaccination Series Assessed with Hybrid (8)
Number Assessed with Hybrid (9)
Hybrid Threshold Level (10)
Number of Providers At or Above Threshold Level (11) 
Alabama
19-35
casa      
4:3:1:3:3
148

100
70.4




Alaska
Other
casa      
4:3:1:3:3
11
0
82
64




Arizona
Other
casa      
4:3:1:3:3
90
0
94
60.6




Arkansas
24-35
casa      
4:3:1:3:3
127
0
100
50




California
Other
both      
4:3:1:3:3
35
20
100
70.1
4:3:1:3:3
198
80
138
Chicago
24-35
hybrid   
4:3:1:3:3
95
11
100
58.9
4:3:1:3:3
57
75
32
Colorado
19-35
both      




4:3:1:3:3
37
65
30
Connecticut
24-35
casa      
4:3:1:3:3:1
101
0
100
80.8




Delaware
19-35
casa      
4:3:1:3:3
99
8
100
66




District of Columbia
19-35
casa      
4:3:1:3:3
39
60
100
67




Florida
24-35
casa      
4:3:1:3:3
466
0
100
76.7




Georgia
24-35
casa      
4:3:1:3:3:1
194
0
100
83.9




Guam
24-35
casa      
4:3:1:3:3
7
30
64
47




Hawaii
24-35
casa      
4:3:1:3:3
54
6
100
65.9




Idaho
19-35
casa      
4:3:1:3:3
93
0
100
72.2




Illinois
12 & 24
casa      
4:3:1:3:3:1
24
0
79
38




Indiana
19-35
casa      
4:3:1:3:3
519
0
100
82.6




Iowa
Other
casa      
4:3:1:3:3
85
50
100
81.8




Kansas
12 & 24
casa      
4:3:1:3:3
52
0
93
54.4




Kentucky
Other
casa      
4:3:1:3:3:1
37
10
94
73.3




Louisiana
Other
both      
4:3:1
153
0
100
70.8
4:3:1
45
90
16
Maine
24-35
casa      
4:3:1:3:3:1
162

100
47.4




Maryland
24-35
casa      
4:3:1:3:3:1
550
15.4
100
87.8




Grantee
Age Group Assessed
(Months) (1)
Assessment Method (2)
Vaccination Series Measured with CASA (3)
Number Assessed with CASA (4)
Minimum Coverage Level (5)
Maximum Coverage Level (6)
Crude Average Coverage Level (7)
Vaccination Series Assessed with Hybrid (8)
Number Assessed with Hybrid (9)
Hybrid Threshold Level (10)
Number of Providers At or Above Threshold Level (11) 
Massachusetts
24-35
hybrid    




4:3:1:3:3
254
80
169
Michigan
19-35
both      
4:3:1:3:3:1
187
22
99
64
4:3:1:3:3:1
535
70
317
Minnesota
24-35
casa      
4:3:1:3:3
93
14
98
65




Mississippi
Other
casa      
3:2:2:2
140
23
100
76.1




Missouri
12 & 24
casa      
4:3:1:3:3:1
180
0
100
59.7




Montana
24-35
hybrid    




4:3:1:3:3
58
90
58
Nebraska
19-35
both      
4:3:1:3:3
61
0
100
82.2
4:3:1:3:3
54
75
45
Nevada
24-35
casa      
4:3:1:3:3
29
0
100
56.2




New Hampshire
24-35
casa      
4:3:1:3:3
83
50
100
85




New Jersey
24-35
casa      
4:3:1
3
52
62
57.3




New Mexico
12 & 24
casa      
4:3:1:3:3:1
117
0
100
70




New York State
24-35
casa  
4:3:1:3:3
184
0
100
75.7




New York City
24-35
casa      
4:3:1:3:3
135
0
100
59




North Carolina
24-35
casa      
4:3:1:3:3
256
0
100
69




North Dakota
24-35
casa      
4:3:1:3:3
26
45
100
89.1




Ohio
24-35
casa      
4:3:1:3:3
313


63.62




Oklahoma
24-35
casa      
4:3:1:3:3:1
55
0
100
39.1




Oregon
12 & 24
casa      
4:3:1:3:3
147
0
100
57.26




Pennsylvania
24-35
casa      
4:3:1:3:3
346
83.5
85.3
84.4




Philadelphia
19-35
casa      
4:3:1:3:3
105
0
100
72




Rhode Island
12-23
casa      
3:2:2:2
25
0
100
84.4




San Antonio
Other
casa      
4:3:1:3:3
98
0
100
73.3




South Carolina
24-35
casa      
4:3:1:3:3
32
16.7
100
70.8




Tennessee
24-35
casa      
4:3:1:3:3:1
126
0
100
68.3




Texas
Other
casa      
4:3:1
1795
0
100
70




Utah
24-35
casa      
4:3:1:3:3
160
0
100
58.6




Vermont
19-35
casa      
4:3:1:3:3
83
0
100
68.98




Virginia
12-23
casa      
4:3:1:3:3:1
118
0
100
65.64




Washington
19-35
casa      
4:3:1:3:3
95
17
100
65.64




West Virginia
24-35
casa      
4:3:1:3:3
38
14
100
76.4




Wisconsin
24-35
casa      
4:3:1:3:3
55
0
100
69.7




Wyoming
24-35
casa      
4:3:1:3:3:1
400


77




Although South Dakota, the Mariana Islands, Puerto Rico, and the Virgin Islands conducted site visits, they did report any measured assessment outcomes prior to the deadline for this report. Houston and American Samoa did not report any data for this survey.
1. Age Group Assessed: the age range of the patients included in the assessment
2. Assessment Method: method used to evaluate vaccine coverage level (CASA or Hybrid)
3. Vaccination Series Measured with CASA: vaccine series (type and number of doses of vaccine) used to evaluate up-to-date status during the assessment visit. All coverage levels will refer to the completion of this series. 
4. Number Assessed with CASA: number of providers that received an assessment using the CASA method.
5. Minimum Coverage Level: among all providers assessed, the lowest determined vaccination coverage for the vaccination series measured
6. Maximum Coverage Level: among all providers assessed, the highest determined vaccination coverage for the vaccination series measured
7. Crude Average Coverage Level: an unweighted average vaccination coverage level among all provider sites
8. Vaccination Series Assessed with Hybrid: vaccine series (type and number of doses of vaccine) used to evaluate up-to-date status during the assessment visit when using the Hybrid Method.  
9. Number Assessed with Hybrid: number of providers that received an assessment using the CASA method.
10. Hybrid Threshold Level: vaccination coverage level at which providers are expected to be performing at or above
11. Number of Providers At or Above Threshold Level: number of providers at or above threshold level 
Table A-3:  Change in Vaccination Coverage Levels, Grantees That Used CASA Method for Previous and CY2004 Assessments 
Grantee
Total Number of Providers Assessed (1)
Vaccination Series Measured (2)
Crude Average Coverage Level (Previous) (3)
Crude Average Coverage Level (CY2004) (4)
Average Percentage Point Change in Coverage (5)
Crude Average Coverage Level (Previous) (3)
Crude Average Coverage Level (CY2004) (4)
Average Percentage Point Change in Coverage (5)
Alaska
38
4:3:1:3:3
77
76
-1
60
64
4
Arizona
22
4:3:1:3:3
72
75.4
3.4
58.9
61.6
2.7
Arkansas
94
4:3:1:3:3
94
93
-1



California*
284
4:3:1:3:3
64.5
68.5
4



Chicago
269
4:3:1:3:3


40
66.7
26.7
Connecticut
52
4:3:1:3:3:1
71
79.8
8.8
75.2
78.2
3
District of Columbia
54
4:3:1:3:3
54
69
15
58
67
9
Florida

4:3:1:3:3
75
80.4
5.4
73.2
77.7
4.5
Georgia
202
4:3:1:3:3:1
82.7
74.4
-8.2
86.6
87
0.4
Hawaii
10
4:3:1:3:3
58.6
61
2.4
69
70.7
1.7
Idaho
124
4:3:1:3:3
82.5
80.3
-2.2
70
72
2
Indiana
680
4:3:1:3:3
77.5
82.5
5
78.4
82.6
4.2
Iowa
14
4:3:1:3:3
60.3
79.7
19.4
65.6
75.5
9.9
Kansas
117
4:3:1:3:3
65
81.1
16.1
59.6
59.9
0.3
Kentucky
246
4:3:1:3:3
71.2
83.4
12.2
71.8
79.4
7.6
Louisiana
175
4:3:1
56.4
62.8
6.4
67.1
77
9.9
Maine
151
4:3:1:3:3
76.2
60
-16.2
70.8
63.4
-7.4
Massachusetts*

4:3:1:3:3






Michigan
84
4:3:1:3:3



60
69
9
Minnesota
53
4:3:1:3:3
62
63
1
70
67
-3
Mississippi
422
4:3:1
77.7
78.6
0.9
76
71.4
-4.6
Missouri
37
3:2:2:2
56.5
77
20.5

Grantee
Total Number of Providers Assessed (1)
Vaccination Series Measured (2)
Crude Average Coverage Level (Previous) (3)
Crude Average Coverage Level (CY2004) (4)
Average Percentage Point Change in Coverage (5)
Crude Average Coverage Level (Previous) (3)
Crude Average Coverage Level (CY2004) (4)
Average Percentage Point Change in Coverage (5)
Nebraska
73
4:3:1:3:3
72
76.5
4
72
80.3
8.3
Nevada
32
4:3:1:3:3
60.2
58.4
-1.9
66
64.4
-1.6
New Hampshire
116
4:3:1:3:3
82
85
3
83
84
1
New Mexico
95
4:3:1:3:3:1
58
75
17
58
73
15
New York State
195
4:3:1:3:3
65.4
68.4
3
76.1
74.4
-1.7
New York City
60
4:3:1:3:3
68
70
2
46
58
12
North Carolina
222
4:3:1:3:3
72
74
2
72
74
2
North Dakota
5
4:3:1:3:3
86.7
83
-3.7
35.7
90.7
55
Ohio
75
4:3:1:3:3
57.4
64.5
7.1



Oklahoma
103
4:3:1:3:3:1
47.3
62.6
15.3
42
60.2
18.2
Oregon
200
4:3:1:3:3
56.8
58.6
1.8
58.8
57.3
-1.5
Pennsylvania

4:3:1:3:3
83
82.7
-0.3
82.1
84.4
2.3
Philadelphia
109
4:3:1:3:3
72
76
4
70
72
2
Puerto_Rico
196
4:3:1:3:3:1
53
49
-4
San Antonio
116
4:3:1:3:3
66.4
86.9
20.5
61.3
73.3
12
South Carolina
25
4:3:1:3:3
29.7
61.4
31.7
55.6
69.2
13.6
Tennessee
39
4:3:1:3:3:1
57.4
66.7
9.3
63.9
72.2
8.3
Texas
1949
4:3:1
57
67
10
59.1
70
11
Utah
156
4:3:1:3:3
78
64.5
-13.4
54.3
62.9
8.6
Virginia
260
4:3:1:3:3
57.2
61.9
4.7
73.1
75.5
2.4
Washington
193
4:3:1:3:3
49.5
61.2
11.7
54.5
65.6
11.1
West Virginia
55
4:3:1:3:3
63.1
70
6.9
66.3
74
7.7
Wisconsin
91
4:3:1:3:3
38.1
93.7
55.6
100
100
0
* Grantee provided change in coverage levels for public and/or private providers using hybrid assessment methodology. See text for assessment results.								
1Total Number Assessed: total number of providers that received an AFIX visit
2 Vaccination Series Measured: vaccine series (type and number of doses of vaccine) used to evaluate up-to-date status during the assessment visit. All coverage levels will refer to the completion of this series. 
3 Crude Average Coverage Level (Previous): an unweighted average vaccination coverage level among all provider sites based on vaccination coverage levels from assessments performed between 1/01/01 and 12/31/02
4 Average Coverage Level (CY 2002): an unweighted average vaccination coverage level among all provider sites based on vaccination coverage levels from assessments performed during CY 2003
5 Average Percentage Point Change in Coverage: increase or decrease between previous coverage level and coverage level CY 2002
APPENDIX B: 
Assessment Methods
Appendix B: Assessment Methodology Options
Method
Description
Advantages
Disadvantages
Diagnostic Tools
CASA
Sample selection: random sample
Sample size: approximately 100 per cohort
Inputs:	childs demographic information; date of each immunization; other information related to diagnostic analysis
Outputs:	diagnostic information on late starts, drop-offs and missed opportunities
baseline measure of coverage
Precise estimates of immunization coverage levels; evaluation of missed opportunities; evaluation of late starts, etc.
Large sample size
Resource burden (staff, time)
Registry based
Sample selection: census of all  eligible records
Sample size: all eligible records
Inputs:	 data from registry downloaded to CASA 
Outputs:    diagnostic information on late starts, drop-outs and        missed opportunities
Minimal time and effort for data collection
no sampling error since estimates based on census of records. 
Potential biases:
- Registry may not contain 100% of providers records
- Reliability of registry data
Mini-CASA
Sample selection:  consecutive, convenience or random sample
Sample size: 40-60
Inputs: 	  same as CASA
Outputs:     diagnostic information on late starts, drop-off and missed opportunities
Smaller sample sizes
Coverage estimates have less precision; sample may not be randomly selected
Hybrid
Hybrid
(LQA/CASA) 
Sample selection: random sample
Sample size: 30
Inputs:	  same as CASA
Outputs:	determines if a provider has immunization coverage       above or below a specified threshold
                    vaccine histories of not up-to-date clients as examples to discuss with provider and staff
Smallest sample size
Rapid assessment
More feedback information than LQA alone
Identifies providers who may benefit from a diagnostic assessment
Computer needed (not paper and pencil as LQA)
Does not give point estimate of coverage
Smaller basis for diagnostic feedback
1