# Presentation: 142696

## Counting the Shots:

- Abigail Shefer, M.D.
- Lance Rodewald, M.D.
- National Immunization Program
- Immunization Services Division
- August, 2002

**A Model for Immunization Assessment and**
**Referral in Non-Medical Settings**

**Counting the Shots:**

## “The task of the public health agency has been not only to define objectives for the health care system ... but also to find means to implement health care goals within a social structure.”

- Institute of Medicine, *The Future of Public Health*

## Special Challenges ...

-    State Medicaid agencies have increasingly relied on private managed care organizations to provide services to indigent children
-   The service delivery role of local public health agencies has been weakened
-   WIC’s direct link to the health care system has been weakened

## Special Challenges ...

-   public health officials are turning to WIC to help address the needs of low income children
-   WIC has become the virtual gateway to health services for low-income preschool children such as those related to overweight and obesity, anemia, HIV, and blood lead level screening in addition to immunization

## Special Challenges ...

- In addition, WIC Programs
-    Are seeing significant health and demographic changes in the low-income populations that it serves
-    Are facing difficulties in recruiting and keeping skilled staff
-   Are having difficulties in funding information technology to manage program operations while simultaneously enhancing service delivery

## The White House Memorandum, December, 2000

-   	Addressed socioeconomic disparities in pediatric immunization coverage levels
-    Directed Secretaries of DHHS and Agriculture to work together to
-   make IZ screening and referral a standard part of WIC certification
-   develop a national strategic plan to improve IZ coverage rates in WIC children
-   report back on progress

**Notes:**
The measles outbreak in 1989-91 alerted health officials  and the public that the 11,000 children born each day in our country need to be protected with immunizations. Since most immunizations are given in the first 18 months of age, a focus on the WIC population revealed that many of the WIC-eligible children were underimmunized.  In the 1990’s efforts intensified to reach all children .In  1995, the Senate Report for the Labor, Health and Human Services appropriations bill identified improving immunization coverage among the WIC participants as a strategy for improving overall immunization coverage rates.  The children in the WIC population represent a significant percentage of our birth cohort.  As was pointed out in the AAP WIC Policy Statement, approximately 47 % of infants born in 1998 were receiving benefits through the WIC program.

	In  October, 1996, a coalition group formed  to facilitate a national strategy to improve immunization rates in children served by WIC while protecting the WIC program’s core mission of providing nutritious foods, nutrition counseling and referrals to health and other social services to WIC participants at no charge. The AAP accepted the responsibility of hosting and facilitating the first meeting  in our Washington office.  The working group was  composed of the following partners: the Food and Nutrition Service of the U. S. Department of Agriculture, which administers the WIC program, (FNS/USDA); the National Immunization Program of the Center for Disease Control and Prevention, (NIP/CDC); the American Academy of Pediatrics, (AAP); the Association of State and Territorial Health Officials, (ASTHO); and the National Association of WIC Directors, (NAWD).  This original group was subsequently joined by the Association of Immunization Managers, (AIM) and Every Child By Two, (ECBT), an advocacy group spearheaded by Betty Bumpers and Roselyn Carter.   This public/private partnership continues to be successful in pooling their resources to ensure that children most at risk for contracting vaccine preventable diseases are protected.  

	This working group has met several times over the past  5 years in an attempt to develop a WIC-Immunization action plan.  On December 11, 2000, President Clinton issued an Executive Memorandum directing the Secretaries of Agriculture and Health and Human Services to “focus efforts to increase immunization levels among the children at risk in a place where we clearly can find them: the Supplemental Nutrition Program for Women, Infants and Children (WIC).”  The WIC-Immunization Action Plan developed by this group was ready to implement the directives of the Executive Memorandum.

	The WIC-Immunization Action Plan has many goals and objectives.  Here are some examples:
		-Develop and disseminate consensus about good practices of WIC -Immunization linkages
		-Obtain adequate funding for WIC-Immunization activities		
		-Every child should have a medical home as defined by the AAP that facilitates access to WIC,
	                       immunizations and other health services
	                -Ensure that children participating  in WIC are screened and referred for immunizations using 
	                       a documented immunization history

	The implementation of this last goal is already underway in FNS/USDA.  When scheduling WIC certification appointments for children under the age of two, the parents or caretakers of the infant/child WIC applicants are requested to bring the immunization records. At initial certification and all subsequent certification visits for children under two, the child’s immunization status is screened using a documented record.  If the child is underimunized or does not have a documented immunization record, the parent/caretaker is provided information on the recommended immunization schedule and is provided with a referral for immunization services, ideally with the child’s usual source of medical care.  This may also be an opportunity to refer a child to a medical home if the family does not already have one.

Page 2 
	An integral and parallel process of improving childhood immunizations began on October 1, 1994 when the Vaccines For Children (VFC) program was created to provide federally purchased vaccines to children 0 to 18 years of age who are American Indian, Alaska Native, enrolled in Medicaid, or without health insurance covering immunizations.  Private providers may enroll in the VFC program and administer vaccines to eligible children they serve.  Participation in the VFC program by private providers reduces the practice of referring children from the private sector to the public sector for vaccination  by keeping children in their medical home for comprehensive health care.

	Out of the combined effort of the WIC-Immunization partnership and the VFC program, there has developed solid support for the Medical Home as defined by the AAP.  The thoughtful recommendations of the AAP Policy Statement on WIC program are timely and pertinent to the practice of Pediatrics now and in the future.  The children in the WIC  program represent a  significant segment of our childhood population and they deserve the care and expertise of pediatricians and  they deserve the cooperative and supportive effort that the Medical Home can offer. Pediatricians on the other hand will need to learn how to work collaboratively with WIC and VFC. 

	Here are some examples of collaboration and support that pediatricians can provide:
		-provide the medical home for WIC-eligible children
		-encourage and facilitate transmission of relevant health information (e.g., immunization status,
		     height, weight, Hgb/Hct etc. between  pediatricians, WIC clinics and immunization 
		     registries.
		-advise WIC on appropriate referrals for immunizations
		-participate in the VFC program
		-encourage the AAP to develop closer coordination between AAP  State Chapters and WIC
		     organizations at the state and local level

	Working together with WIC and VFC, pediatricians can take the lead in  this effort to ensure that children most at risk for contracting vaccine preventable diseases are protected.

Richard M. Narkewicz, M.D., FAAP
Former President , American Academy of Pediatrics
Former Chairperson, National Advisory Council of Maternal, Infant and Fetal Nutrition

(editor’s note:  Dr. Narkewicz and  AAP Washington staff member, Janis Guerney have been the AAP’s representatives to the WIC-Immunization working group)

## The Final WIC Policy Memorandum, August, 2001

-   	Sets **minimum** guidelines for IZ screening and referral
-    Requires documented immunization history
-     Applies to infants and children under age 2 years
-     Screening and referral to be done at certification visits
-     WIC clinic staff will count DTaP immunizations only
-    1 dose of DTaP by age 3 months
-    2 doses of DTaP by age 5 months
-    3 doses of DTaP by age 7 months
-    4 doses of DTaP by age 19 months

## Diagram of the process described in the WIC policy memorandum

## Rationale

- In studies using NIS data, the provider’s portion of the record is considered the more accurate index of the child’s true immunization status.  The household portion of the record is assumed to be a less accurate representation.  In most WIC clinic settings, only the household-based record will be available.  For this reason, our aim was to see how closely the household-based record approximates the provider-based record.

## Purpose of this analysis ...

**Primary objective:**
-   To estimate the percentage of WIC children who, by the new WIC minimum assessment criteria, will be correctly classified as either up-to-date (UTD) or not for the universal 4:3:1:3 pediatric vaccination series
-   To see how this compares with the percentage that would be correctly classified if all 4 antigens were assessed

## Method

- We used approximately 6,000 household and provider records from the  2000 National Immunization Survey (NIS) to  calculate the sensitivity, specificity, and test efficiency of using DtaP shots as a predictor of UTD status for the universal 4:3:1:3 pediatric immunization series.

**Data Source and Statistics**

## Method ...

**Total NIS 2000 Survey Records = 34,087**

**Ever enrolled in WIC = 17,451 (51% of NIS 2000)**

**Shot card used for household reporting = 8,617 (25% of NIS 2000)**

**Adequate provider data for verification = 6,277 (18% of NIS 2000)**

## Method:  Definitions

**3** doses of Polio at 2, 4, and 6 - 18 months
**1** dose of MMR or MCV at 12 - 15 months
**3 **doses of Hib at 2, 4, 6, and 12 - 15 months

- Up-to-Date by the 4:3:1:3 ACIP schedule is defined in terms of the age-specific vaccination schedule for the following 4 vaccines:

## Method:  Variable Specification

- NIS Household Survey
-   child’s age at each vaccination
- NIS Provider survey
-   child’s UTD status for each vaccine 	at 3, 5, 7, 13, 19, and 24 months
- We assigned bivariate UTD status variables
-   Household DTaP
-   Household 4:3:1:3
-   Provider 4:3:1:3

**Notes:**
The household interview section of the NIS contains information about the child’s age at the time each vaccination was received.  The provider section contains information about the child’s UTD status for each vaccine at ages 3,5,7,13,19, and 24 months.  We used these data to incorporate age criteria in assigning a bivariate UTD status variable from the household report and from the provider report at each of the four target ages specified in the WIC policy memorandum.

## Method:  Definitions

- Let **“test**” refer to any measure used to make a decision about the “**true status**” when information about the true status is incomplete.
**  ****test 1**:  the household-reported DTaP count
-   **test 2:**  the household-reported 4:3:1:3 count
- We compared test 1 and test 2 to see if they differed in their ability to predict the child’s true 4:3:1:3 UTD **true** **status** as shown in the provider record.

## Method:  Definitions

**Specificity** of a test measures the percentage of truly UTD children who are correctly identified as such by the test.
**Test efficiency** measures the total percentage of times the test gives the correct answer, relative to all times the test is given.

**Notes:**
Test efficiency is the sum of the predictive value positive and the predictive value negative of a test.

## SAS

**	****To develop the enhanced database**
- SUDAAN
**	****To conduct the weighted crosstabulation ****	****procedures for the sensitivity, specificity ****	****and test efficiency statistics.**

- Method:  Statistical Analyses

**Notes:**
Test efficiency is the sum of the predictive value positive and the predictive value negative of a test.

## Research Questions ...

- What is the estimated nationwide difference in  immunization coverage rates between WIC vs non-WIC children under age 2?

- Socioeconomic Disparities

## Differences in Immunization Completion

**Differences in Immunization Completion**
**Rates, WIC vs Non-WIC Children, 2000**

## How good is the household-based record of UTD status for DTaP at identifying children who, by the provider-based record, are underimmunized?  How does this compare with the household-based count of all 4 antigens?

- Research Question:

- How good is the **household-based** record of UTD status for **DTaP** at identifying children who, by the **provider-based** record, are underimmunized?  How does this compare with the household-based count of all 4 antigens?

- Comparative Sensitivity

**Notes:**
Sensitivity as here defined is the percentage of all children who are not UTD who are correctly classified as not UTD by the tests.

## Results:  Comparative Sensitivity

**Notes:**
The ages that are specifically targeted in the WIC policy are outlined in bold.  In those 4 age groups, the greatest disparity in sensitivity occurs at 3 months, the least at 7 months.  Weighted averages are based on the 4 target age groups.

## How good is the household-based record of UTD status for DTaP at identifying children who, by the provider-based record, are truly UTD for the complete 4:3:1:3 series?  How does this compare with the household-based count of all 4 antigens?

- How good is the **household-based** record of UTD status for **DTaP** at identifying children who, by the **provider-based** record, are truly UTD for the complete **4:3:1:3 **series?  How does this compare with the household-based count of all 4 antigens?

- Research Question:

- Comparative Specificity

**Notes:**
Specificity is the percentage of all children who are truly UTD and who are correctly classified as UTD.

## Results:  Comparative Specificity

**Notes:**
DTaP is more specific a predictor of true UTD status at every age.  

## Research Question ...

- Comparative Test Efficiency

## Comparative Test Efficiency

**Notes:**
Overall test efficiency (the percentage of times that the test gives the correct answer relative to the total number of tests given) is higher for the DTaP count than for the entire count, although the difference is slight.

## Summary

-   **The NIS 2000 data confirm that children under age 2 who have ever been enrolled in WIC are significantly more likely to be underimmunized than those who have never been enrolled.  The overall disparity is 7%.**

## Summary

- The household DTaP count and the household 4:3:1:3 count are both imperfect predictors of true 4:3:1:3 UTD status.
-   If the DTaP count alone is used, 70% of underimmunized children, on average, will be identified.
-   If all 4 antigens documented in the household record are counted, 77% of underimmunized children, on average, will be identified
-   Therefore the count of all 4 antigens is a more sensitive predictor in that it is slightly better (7%) at identifying truly underimmunized children.  However...

## Summary

-   If the DTaP count is used, 86% of truly UTD children, on average, will be identified as such.
-   If all 4 antigens from the household record are counted, 82% of truly UTD children, on average will be identified as such.
-   Therefore the DTaP count is a more specific predictor of true UTD status in that it is slightly better (4%) at identifying truly UTD children.  Moreover,
-   Overall test efficiency is slightly higher (1% on average) for the DTaP count, making it a more efficient predictor of true immunization status.

## Conclusions

- Training non-medical staff to assess completion status for  1 vaccination is simpler than training them to assess it for 4 or 5
- The immunization assessment process should go more quickly and is therefore more efficient in terms the demands on staff time
- The likelihood that non-medical staff will make an error is smaller when only one vaccine is assessed.

**Use of the DTaP count as a predictor of true 4:3:1:3 UTD Status has specific advantages:**

**Notes:**
Bullet #3.  A more complex iz assessment has an inherently higher probability of error.

## Conclusions

- The higher specificity of the DTaP count means that children are less likely to be inappropriately referred to an immunization provider
- If the gains in simplicity and efficiency of assessment result in more clinics being able/willing to evaluate children’s immunization status, this may offset the loss  of sensitivity, and ...

**Advantages ... (continued)**

## Conclusions

- Success in this initiative will likely strengthen the partnership, which will
- Set the stage for assisting WIC in developing achievable objectives toward improving immunization coverage in WIC children, as well as
- Serve as a model for developing immunization linking initiatives with other federal/state programs, such as Housing and Urban Development, Medicaid and TANF

**Advantages ... (continued)**

## Contact Information:

- e-mail:  djr7@cdc.gov
- phone:  (404) 639-8827