Abstract
In 1996 we launched a community–provider partnership to raise immunization coverage for children aged younger than 3 years in Northern Manhattan, New York City. The partnership was aimed at fostering provider knowledge and accountability, practice improvements, and community outreach. By 1999 the partnership included 26 practices and 20 community groups. Between 1996 and 1999, immunization coverage rates increased in Northern Manhattan 5 times faster than in New York City and 8 times faster than in the United States (respectively, 3.4% vs 0.4% [t = 6.05, p < 0.001] and vs 0.6% [t = 5.65, p < 0.001]). The coverage rate for Northern Manhattan stayed constant through 2000, although it declined during this period for the United States and New York City. We attribute the success at reducing the gap to the effectiveness of our partnership.
WE CONDUCTED A STUDY TO assess the effectiveness of our community–provider partnership in reducing immunization disparities among the disadvantaged communities of Harlem and Washington Heights in Northern Manhattan, New York City. Previous studies showed that interventions by either health providers or community organizations alone had limited impact on immunization coverage.1–4 Our partnership combined the efforts of health providers and community organizations.
Starting in 1997, the Northern Manhattan Immunization Partnership targeted 3 broad programmatic areas: provider knowledge and accountability, practice improvements, and community outreach (Table 1). This partnership has involved up to 20 community programs and 26 private and public practices serving virtually all children aged younger than 3 years in Northern Manhattan (n = 19 800). Half of the children (52%) are Latino, 32% are African American, and 44% receive government assistance.5 This study reports on the 8 practices, both public and private, consistently involved in the program between 1996 and 2001, serving annually 11 712 children aged younger than 3 years (range = 167–4682). Four practices served the African American community of Harlem and 4 served the Latino community of Washington Heights. These 8 practices also had more complete integration of provider and community-based initiatives.
From the start, provider knowledge and accountability were facilitated through practice assessments and feedback, using Centers for Disease Control and Prevention methodology.6 Practice-specific report cards integrated coverage, documentation, and disengagement from care. Beginning in 1998, we initiated system-wide improvements in immunization delivery: flow sheets summarizing information about all immunizations ever given to a child, reminder/recall, standing orders, centralized immunization policies, and parent vaccination cards. We also organized educational programs for providers, nurses, and office staff. In 2000, we launched a Web-based immunization registry at the hospital-affiliated practices, and in 2001 we extended the registry to community provider offices.
Simultaneously, we collaborated with community organizations to expand outreach. We developed bilingual immunization outreach materials, expanded our linkages with the Special Supplemental Nutrition Program for Women, Infants, and Children, and contracted with selected community organizations for targeted outreach. In 1999, the Northern Manhattan Start Right Coalition was launched to promote immunization at ongoing service programs. Regular process evaluation with feedback from community partners and providers has helped to improve these community-based programs (Table 2).
We conducted semiannual chart audits of randomly sampled children aged 6 to 35 months with at least one visit to the practice. These were compared with the National Immunization Survey (NIS) coverage rates for US and New York City children. We limited our analyses to children aged 19 to 35 months so that our data would be comparable to those of the NIS. After excluding our 1998 data, which only included children aged 6 to 24 months, we reassembled our semiannual audit data into birth cohorts comparable to the NIS cohorts (n = 852, 1503, 2111, and 2177 for 1996, 1997, 1999, and 2000, respectively; mean age = 27.7 months).
The NIS immunization coverage rates are population-based rates obtained through a random-digit-dialed telephone survey.7 The NIS US sample included 22 521 to 33 305 children per year (mean age = 27.0 months8–11), and the New York City subsample ranged from 211 to 274 children per year (mean age = 27.1 months) (L. Barker, written communication, March 25, 2002).
Up-to-date immunization coverage was defined as 4 injections for diphtheria-tetanus-pertussis, 3 for polio, 1 for measles, and 3 for Haemophilus influenzae type b—the 4:3:1:3 series. We calculated annual coverage, coverage differences for Northern Manhattan versus New York City and the United States, and rates of change in coverage. One-sample t tests and χ2 tests were used to assess significant differences.
In 1996, before the Northern Manhattan Immunization Partnership was launched, immunization coverage in Northern Manhattan was 45.8%, 30.7 percentage points lower than in the United States as a whole (t = −18.0, P < .001) and 29.2 percentage points lower than in New York City (t = −17.11, P < .001) (Figure 1). By 2000, the disparities had been significantly reduced—immunization rates in Northern Manhattan were 18.8 percentage points lower than in the United States (t = −17.6, P < .001) and 10.7 percentage points lower than in New York City (t = −10.8, P < .001). The annual rate of increase in coverage between 1996 and 1999 was greater in Northern Manhattan than in the United States or New York City: 3.4% versus 0.4% and 0.6%, respectively (t = 6.05, P < .001 for Northern Manhattan vs United States; t = 5.65, P < .001 for Northern Manhattan vs New York City). Coverage in Northern Manhattan declined from 1999 to 2000 by 0.2%, significantly less than the 2.2% decline in the United States (χ2 = 16.87, P < .001) and the 10.2% decline in New York City (χ2 = 101.43, P < .001).
Between 1996 and 1999, immunization coverage in Northern Manhattan increased 8 times faster than in the United States and 5 times faster than in New York City, enabling a significant narrowing of disparities. While rates declined from 1999 to 2000 in the United States and New York City, Northern Manhattan maintained its coverage, further narrowing the gap. While the changes may not seem large, these gains are robust and compare quite favorably to the achievements of other community immunization programs.1,3,12
While we cannot attribute direct causality, we associate this sustained increase in coverage to the effectiveness of our partnership at mobilizing both providers and communities. Specifically, the increase in coverage observed in 1997 was probably attributable to improvements in immunization delivery, especially documentation. Although the practice improvements continued, their impact from 1997 to 2000 was offset by increased complexity of the immunization schedule, which resulted in a doubling of the vaccines required. The community interventions appear to have sustained the coverage rate improvements, but these were not fully expanded until 1999, when the Start Right Coalition launched its program of immunization outreach and promotion.
Despite these improvements, the Northern Manhattan practice coverage still lags behind the US and New York City rates. Recently released 2001 NIS estimates show an increase of 7% for New York City and 1% for the United States.13 We stopped chart audits in 2001, but the trial registry–based audits for 2002 show an additional increase for the Washington Heights practices, to 64%. Thus, it appears that we are continuing to make improvements in coverage.
The difference between our coverage rates and those of the NIS may reflect different assessment methodologies.14 Practice-based assessments tend to underestimate coverage because immunizations given by other providers may not be documented in the record.15 The difference may also reflect the high rates of disengagement from care at these practices.16 Ongoing efforts by community outreach workers of the Start Right Coalition, provider reminder/recall efforts, and registry uptake are expected to further reduce disparities in immunization coverage.
Note. WIC = Special Supplemental Nutrition Program for Women, Infants, and Children; SCHIP = State Child Health Insurance Program; X = intervention implemented. “The NMIP chart reviews let us recognize missed opportunities as an important thing. We have had repeated discussions that it is better to give 4 shots in one day than miss giving them. It is so much easier for us to find local solutions to the problem.” “Initially, we were afraid that if we gave vaccinations on walk-ins we wouldn’t have the patients coming back for well-child visits, and would miss the opportunity for anticipatory guidance and routine things like plotting the growth. What we found was that many of our patients were going unimmunized because they would come in for what was bothering them and would miss their appointments for well-child care. So then we changed our policy. We switched it so that vaccinations were given at any type of visit.” “Before, it was always, ‘It’s not my patient.’ Now the last doctor to see the patient is the patient’s doctor. As far as I’m concerned, if you’re the doctor and a patient comes in as a walk-in and you did not choose to give immunizations, you’re at fault. The immunization project has helped change that.” “When parents have a newborn, they get a yellow card and are told to make an appointment. But they don’t know why or understand the importance of immunizations. Now our parents can ask why. Before they would get a bunch of papers from the doctor and they didn’t ask questions. Now they observe, and they ask questions.” “Learning how to read the immunization card was one of the best trainings I’ve ever gotten, because now I can tell parents if their children are up to date or not.” “This is my passion. I love being a child health educator. It gives me a shot of adrenaline, I’m so excited.” “When I show parents our books, they ask questions and really realize the importance of getting their children immunized. They actually do take them to the doctor to get their shots.” “This [immunizations] isn’t a separate program. It is really about everything we do, helping families have better homes and better lives.” “We want to move towards empowering our parents to be immunization advocates with other families in the community.” “We are all so excited about this work. It links many threads of our work.” “It is good that you worry about the community. We didn’t even know that all children here weren’t immunized.” “I was glad that she gave me the form to take to my doctor. Then they realized my child was missing shots.” “I know what shot my child will get and I can tell the doctor what shot is needed. ” “She [the outreach worker] made me understand my child’s immunization card, the facts about vaccines and how it is important to have our children’s vaccines up to date. The program made me more aware about my children’s immunizations, and how important it is to have them done on time.” FIGURE 1— Immunization coverage rates (4:3:1:3 series) for children aged 19 to 35 months in Northern Manhattan, New York City, and the United States, 1996 to 2000. Note. The 4:3:1:3 series is 4 injections for diphtheria-tetanus-pertussis, 3 for polio, 1 for measles, and 3 for Haemophilus influenzae type b. Excludes data for 1998.
Program Areas Interventions 1997 1998 1999 2000 Interventions targeting provider knowledge and accountability Semiannual practice immunization coverage assessments X X X X Basic practice immunization report cards X X X X Centralized immunization policies X X X X Annual immunization grand rounds X X X X Provider round tables X X X X Immunization coverage feedback workshop X Immunization workshops for nurses and office staff X X X X Annual childhood immunization update conference X X Detailed practice immunization report cards X X Immunization documentation report cards X X Practice and system improvements Practice survey of immunization, scheduling, reminder/recall policies X X X Nursery–primary care linkages survey X Standing orders trial X Immunization flow sheet for records X X X Protocol for child vaccination cards X X X Appointment reminder trial X NMIP Web-based immunization registry pilot X X Telephone reminder pilot X X Wait time/patient flow study X NMIP registry deployed at practices and school-based clinics X X Linkages between NMIP and New York City Department of Health registries X Registry reminder/recall trial X NMIP registry deployed at community provider offices X Community mobilization Direct outreach using NMIP community-oriented vaccination flyers X Parent feedback surveys on immunization barriers X Collaborative WIC parent surveys and outreach X X X Active membership and support of community child vaccination coalitions X X X X NMIP–community organization outreach partnerships X X X NMIP immunization outreach worker curriculum X NMIP Community Resource Directory (2 editions) X X Northern Manhattan Start Right Coalition community action plan X Child Health Insurance facilitated enrollment X X Survey of children disengaged from care X Start Right launches outreach worker training program X X Start Right mobilizes parents through day cares, tenants’ associations, parenting classes, churches, WIC, and facilitated SCHIP enrollment. X Feedback From Providers • • • Feedback From Community Organizations Outreach workers • • • • Organization leaders • • • Feedback From Parents • • • • 
This study was supported by grant U66/CCU212961 from the National Immunization Program, Centers for Disease Control and Prevention, and grant U50/CCU217475 from the Reach 2010 program, Office of Minority Health, Centers for Disease Control and Prevention.
