Objectives. We compared the effectiveness and cost-effectiveness of population-based recall (Pop-recall) versus practice-based recall (PCP-recall) at increasing immunizations among preschool children.

Methods. This cluster-randomized trial involved children aged 19 to 35 months needing immunizations in 8 rural and 6 urban Colorado counties. In Pop-recall counties, recall was conducted centrally using the Colorado Immunization Information System (CIIS). In PCP-recall counties, practices were invited to attend webinar training using CIIS and offered financial support for mailings. The percentage of up-to-date (UTD) and vaccine documentation were compared 6 months after recall. A mixed-effects model assessed the association between intervention and whether a child became UTD.

Results. Ten of 195 practices (5%) implemented recall in PCP-recall counties. Among children needing immunizations, 18.7% became UTD in Pop-recall versus 12.8% in PCP-recall counties (P < .001); 31.8% had documented receipt of 1 or more vaccines in Pop-recall versus 22.6% in PCP-recall counties (P < .001). Relative risk estimates from multivariable modeling were 1.23 (95% confidence interval [CI] = 1.10, 1.37) for becoming UTD and 1.26 (95% CI = 1.15, 1.38) for receipt of any vaccine. Costs for Pop-recall versus PCP-recall were $215 versus $1981 per practice and $17 versus $62 per child brought UTD.

Conclusions. Population-based recall conducted centrally was more effective and cost-effective at increasing immunization rates in preschool children.

Vaccination is recognized as one of the greatest public health achievements of the 20th century.1 Childhood vaccines developed in the previous century were associated with declines in the incidence of major childhood infectious diseases by 98% or more compared with baseline 20th century annual morbidity rates.2,3 Despite this, only 44.3% of children aged 19 to 35 months received all recommended vaccines in 2009.4 Because of the importance of timely vaccination in young children, one of the nation’s top health goals as outlined in Healthy People 2020, is to increase the proportion of children aged 19 to 35 months who receive all recommended doses of childhood vaccines to 80%.4

Based on strong evidence of effectiveness, the Community Preventive Services Task Force5,6 recommends the use of reminder/recall for increasing immunization rates, including notification for upcoming immunizations (reminders) or recall notices for overdue immunizations (recall). The use of regional or state immunization information systems (IISs) can greatly facilitate reminder/recall because such systems cannot only identify children who need immunizations but often can also generate reminder postcards or electronic data that can be used to produce autodialer messages. Current national data suggest that despite strong national recommendations, few providers are doing any type of reminder/recall for immunizations.7 Because of this, there has been interest in determining whether reminder/recall efforts might be more feasible and less costly to conduct centrally by health departments using a regional or state IIS.

We conducted a population-based, cluster-randomized pragmatic trial comparing the effectiveness of practice-based recall versus population-based recall conducted by the state health department using the Colorado Immunization Information System (CIIS) in increasing immunization rates among children aged 19 to 35 months as well as the cost-effectiveness of each method.

This study was a stratified cluster-randomized pragmatic trial in 14 counties (8 rural, 6 urban) in Colorado with randomization at the county level within rural and urban strata. We followed the criteria established for National Institutes of Health–funded pragmatic trials.8,9

Settings and Study Participants
Colorado Immunization Information System (CIIS).

CIIS receives client and vaccine event data through live data entry into the Web-enabled application and through electronic transfers from data sources maintained by providers and insurers throughout the state. CIIS also includes historical data about immunizations given outside of the state if entered by a Colorado provider. The percentage of children younger than 6 years in Colorado with at least 2 immunization records in CIIS was 82% in 2009. Because it is populated with names and demographic information from the Colorado Electronic Birth Certificate database each week, CIIS has the capacity to assess population-based immunization rates based on county of residence.

Selection of participating counties and randomization procedure.

We selected 14 of the 64 counties in Colorado to participate because they were relatively uniform with respect to prespecified criteria (Table 1) and did not have characteristics that could confound the trial. Exclusion criteria, the rationale for the criteria, and the numbers of counties excluded on the basis of each included (1) ongoing, existing county-wide reminder/recall efforts because we could not determine if patients responded to our recall or to existing recall efforts (6 rural and 4 urban counties excluded); (2) low CIIS saturation levels in the county, defined as less than 80% of 2-year-old children in the county with at least 2 immunizations in CIIS, because we were not able to adequately assess the results of the intervention (10 rural and 4 urban counties excluded); and (3) frontier counties with populations of less than 10 000 because the health care delivery systems in these counties were different from those in larger counties (23 counties excluded). An additional 2 urban counties were excluded because their populations were substantially smaller than the other urban counties and 1 other county was excluded because its vaccine refusal rate was much higher than all other Colorado counties.


TABLE 1— Baseline County Demographics Population-Based Counties vs Practice-Based Counties: Colorado, 2009

TABLE 1— Baseline County Demographics Population-Based Counties vs Practice-Based Counties: Colorado, 2009

CIIS Information
Counties% CIISaChildren Aged 19–35 Mo,b No.Up-to-Date Rate,c %White, %Non-White and ≥ 2 Races,d %Hispanic/ Latino, %Median Income,e $Ratio of Pediatric to Family Medicine PracticesfCommunity Health ClinicsgTotal Providers
 Urban 19310 0085088151466 05918/53677
 Urban 28453433685181156 2817/38449
 Urban 38231433879254139 0166/22129
 Rural 1933663775294635 9601/113
 Rural 28712743983193074 2201/1213
 Rural 3846144290121239 7141/337
 Rural 49317203982212862 7162/6210
 Overallh8832104083202653 42436:135i17188
 Urban 48811 8075173311856 09921/47674
 Urban 57012 3542980251555 62114/53572
 Urban 68260564383202854 5782/20426
 Rural 58324231948858 2091/56
 Rural 6793502789131639 953–/336
 Rural 7974014979254030 373–/538
 Rural 81002343778254237 350–/213
 Overallh8644923882212447 45538:135i22195

Note. CIIS = Colorado Immunization Information System.

aNumerator based on number of children aged 0–3 years, 11 months with ≥ 2 immunization encounters in CIIS; denominator based on 2010 Population Forecasts Source of Population Data: State Demography Office—numbers as before start of intervention.

bNumber of children aged 19–35 months per county as of June 2010 according to CIIS.

cPercent up-to-date rate is based on CIIS data for children aged 19–35 months who were up-to-date according to Advisory Committee on Immunization Practices guidelines.

dAll other races and those reporting ≥ 2 races (total is > 100%)

eData taken from the 2010 US Census.

fData compiled via telephone survey conducted by CIIS in 2009.

gCommunity Health Centers data compiled and updated by Colorado Department of Public Health and the Environment.

hThese values reflect the mean of each category mean.

iRatio of pediatric practices to family medicine practices. This is the sum of all providers.

jP values are between the overall values for population-based counties and practice-based counties.

We identified existing practices in all counties based on a state survey that had been conducted by CIIS in 2009; sites were included if they offered primary care visits and immunizations to children aged 19 to 35 months. Within rural and urban strata, counties were randomized to practice-based or population-based recall using a random number generator.

Denominating the study populations.

The target population included children who were aged 19 to 35 months as of June 2010 with a CIIS address in one of the selected counties. This included children born in Colorado and those who moved into a study county before the study if they were entered into CIIS. Because CIIS is populated from vital statistics, it also included children who were born in the county but whose provider did not enter information into CIIS, thereby providing the best population-based estimate of children within each county. The study population was fixed from this point in time, and children who either moved into or out of the counties after this time were not added to or subtracted from the study population. Although this meant that children who moved might have been inappropriately assigned during the study period, the potential bias was likely balanced between the 2 study arms.


The population-based intervention included an initial letter from CIIS and the local county health department in June 2010, notifying parents of children who were not up-to-date (UTD) and suggesting they have their child vaccinated at their primary care site or, if they had no usual source of care, at the county health department. Six weeks later, a postcard was mailed to those still not UTD, and 4 weeks after that, a final postcard was sent to those still not UTD. If any mailings were returned without a forwarding address, researchers contacted the provider of last service in CIIS to obtain new addresses. Children who moved out of the state were removed from the mailing lists, although not from the study population denominator. All practices in the population-based counties were sent a letter from their local Public Health Department, informing them about the centralized recall intervention.

In the practice-based intervention counties, all primary care practice sites were invited 5 times to attend 1 of 6 training webinars by CIIS on how to conduct practice-based recall. In the trainings, it was recommended that the practices use the mail method for recall with the same number of recalls as in the population-based counties. Practices were offered financial support for mailings if they did recall for 19- to 35-month-old children throughout the summer of 2010.

Outcome Measures

The definition of UTD was based on the national Advisory Committee on Immunization Practices recommended series of antigens (4:3:1:3:3:1:3) to be received by the age of 19 to 20 months, including 4 DtaP (diphtheria, tetanus, and acellular pertussis), 3 polio, 1 MMR (measles, mumps, and rubella), 3 Hib (Hemophilus influenza B), 3 Hepatitis B, 1 varicella (chickenpox), and 3 pneumococcal conjugate vaccines.10 Children could be brought UTD in CIIS by receiving needed vaccines or updating immunization records, as both might be attributed to the recall effort. The “reach” of each intervention was estimated based on the Re-Aim framework11; in the population-based counties, we estimated that children were reached by the intervention if no mailed notices were returned after attempts to update addresses. In the practice-based recall counties, we did not have information about how many notices were returned to practices; therefore, we conservatively estimated that all children sent a recall notice were reached.


The primary outcome measures were (1) UTD status, (2) documentation of any new vaccine 6 months after the intervention among 19- to 35-month-old children who were not UTD, and (3) cost–benefit comparison of the 2 interventions.

Data Analysis

The study was powered to detect a 4% difference in immunization rates among children who were not UTD at baseline between the 2 arms. The final eligible cohort was established by matching child identification numbers from the follow-up CIIS database obtained in December 2010 to the baseline cohort database from June 2010; 98.3% of children initially identified had a record in the follow-up database. Site of last service was defined as the site where the most recent immunization was given, or if that information was missing, the site where the most recent immunization data were entered.

The major analyses were intent-to-treat. We first compared percentages and then modeled the 2 immunization outcomes, using generalized linear mixed-effects models (PROC GLIMMIX) with a logit link function and unstructured covariance matrix. Mixed-effects models are needed when the assumption of independence is violated, such as in the case where patients are nested within practices.12,13 Because practices tend to be relatively homogeneous with respect to their patients, and because patients sometimes cross county lines for services, site of last service was included as a random effect to account for the clustering of patients within practices. County baseline UTD rates and rural or urban location were included as fixed effects in all models.

A secondary analytic cohort was created for the practice-based intervention, focusing only on the primary care practice sites that conducted recall to determine the impact of recall on rates for only those practices that conducted recall. All comparisons were set at a priori level of significance at P < .05. All analyses were performed using SAS version 9.2 (SAS Institute Inc, Cary, NC).

Cost Methods

Because the population-based recall was a centralized effort, all practices and their representative children who were not UTD were included in the cost-effectiveness analysis. In the practice-based recall intervention counties, only practices that conducted recall and their representative children were included. The costs for conducting recall included (1) staff education and training in conducting recall, (2) staff time related to conducting recall, (3) mailing and printing costs associated with recall, and (4) time spent correcting mailing addresses. Time spent by personnel was reported by staff involved in recall at the state health department and by practice staff identified as having conducted recall through reports generated by CIIS on a weekly basis. Nonpersonnel costs related to mailings and postage were identified through contractual invoices. Personnel costs were estimated using the position title linked to median salaries listed by the Bureau of Labor Statistics for the State of Colorado. Benefits were estimated at 25% of median salary.

Of the 55 173 children identified as residing in the 14 study counties as of June 2010, 58.2% (32 125) needed 1 or more immunizations according to CIIS records. Figure 1 shows a consort diagram for the trial. Table 1 compares the characteristics of the patient populations, providers, and CIIS penetration in counties randomized to the 2 intervention arms, demonstrating no significant differences in any of the prespecified criteria at baseline.


The intervention in the population-based counties involved all 188 practice sites and 12 832 children (Figure 2). Approximately 40% of initial mailings were returned initially, and after attempts to update these addresses, the final “bad address” rate was 15%. Thus, we estimated that approximately 85% of the children in the population-based counties (n = 10 907) were reached. Of the 195 practices in the practice-based counties, 26 practices participated in a webinar and 10 practice sites conducted recall for 19- to 35-month-old children. Five practices chose to mail a single recall postcard and 5 chose to make a single personal phone call to families. This resulted in 5% of patients (n = 887) who might have been reached by the intervention, assuming no bad addresses or telephone numbers.

Comparative Effectiveness

As shown in Figure 3, a significantly higher percentage of children needing immunizations became UTD in population-based counties compared with practice-based counties (F[1,31007] = 13.48; P < .001). The findings were similar for the outcome of receipt of at least 1 vaccine (F[1,31006] = 23.36; P < .001). Estimates of relative risk from multivariable modeling were 1.23 (95% confidence interval [CI] = 1.10, 1.37) for becoming UTD and 1.26 (95% CI = 1.15, 1.38) for documented receipt of any vaccine in population-based versus practice-based counties, respectively. All models were adjusted for baseline county UTD rates and rural location (all P > .05).

Among the 10 practices that actually conducted recall, 23.9% of children needing vaccines became UTD and 39.2% were documented to have received 1 or more vaccines. These were significantly higher than the rates achieved in the population-based counties for these 2 outcomes (both P < .001). Of note, children in the 10 practices that chose to do recall were significantly more likely to be UTD (59%) compared with children in the practices that chose not to do recall in the practice-based counties (39%; P < .001). None of the practices that conducted recall chose to be compensated by the study for their costs.


As shown in Table 2, the cost per practice for the practice-based recall method was more than 6 times the cost of the population-based method. For those practices that conducted recall, the cost per UTD child was $62 compared with $17 for the population-based method.


TABLE 2— Cost Analysis, Population-based versus Practice-based Recall for Childhood Immunizations, 2009

TABLE 2— Cost Analysis, Population-based versus Practice-based Recall for Childhood Immunizations, 2009

VariablePopulation-Based CountiesPractice-Based Counties
Total no. of children not UTD at baseline (eligible)12 83218 735
No. of practice sites conducing recall efforts18810
No. of eligible children in recall practices12 832887
Total costs (staff time, supplies), $40 36713 153
Cost per practice, $2151315
Cost per eligible child (total cost/total eligible), $315
No. of children who became UTD2394212
Cost per child who became UTD, $1762
No. of children with ≥ 1 documented immunization4083348
Cost per ≥ 1 documented immunization, $1038

Note. UTD = up-to-date.

Although strong evidence and national recommendations support the use of reminder/recall for increasing immunization rates in young children, only a small minority of primary care private practices are currently conducting reminders/recalls.7 The present study demonstrated that recall conducted centrally using data from the state IIS populated by birth vital statistics data with direct mailings to families from state and county public health departments was significantly more effective at increasing vaccination rates than were intensive efforts to support recall efforts by primary care practices. The increased effectiveness of the population-based approach was because of the fact that few practices did recall, even when provided with technical and financial assistance, thereby severely limiting the reach of practice-based recall efforts at the population level. When practices did conduct recall, they were slightly more effective at raising rates than was the population-based approach, although at much greater expense. Overall, recall conducted centrally was much more cost-effective.

Numerous previous studies demonstrated the effectiveness of reminder/recall methods when implemented within practice settings.14–17 However, review of the 47 trials included in a recent Cochrane review of the effectiveness of reminder/recall demonstrated that at least 35 were done with the assistance of an outside research study team, and none directly measured the sustainability of reminder/recall once the trial was completed.17 Current national data suggest that only 16% of providers are doing any type of reminder/recall,7 and it is unknown how many are using an IIS to do so. Barriers cited by providers included insufficient staff time and competing demands of primary care, staff turnover, costs related to mailings or telephone call reminders, and the lack of computerized systems to readily identify patients who needed immunizations.7,18 However, a recent study demonstrated that even when practices that are currently using a state IIS are offered training and technical assistance with reminder/recall using the IIS, few practices actually followed through with conducting reminder/recall.18 The present study not only offered training and technical assistance, but offered to pay for mailings, eliminating many previously identified barriers. Yet, despite these efforts, only 13% of practices attended the training offered, and 5% of practices conducted a recall during the study period. The practices that chose to do recall were not typical, in that they had higher baseline immunization rates before the trial. Improving technological interfaces between electronic medical records or billing systems and state IISs might increase provider involvement, but given available evidence in practices that already have the technical capabilities to conduct reminder/recall using an IIS and the many competing demands of primary care practice, advances in information technology might not make a substantial difference.

Given the findings of the present study, centralized recall by public health departments should be considered as a more effective and much less expensive alternative, if it is found to be acceptable to providers and parents. Population-based recall efforts such as the one described will need to rely on statewide IISs. According to the Centers for Disease Control and Prevention annual survey of IIS programs, as of December 31, 2009, 49 states plus Washington, DC, were operating a state IIS.19 Overall, in the United States, approximately 77% of children younger than 6 years were estimated to have participated in an IIS in 2009.19 In addition, in 2009, 42 of 47 (89%) states reported that their IIS was populated with birth data, usually from vital statistics but less commonly from birth hospital records.20 Based on these data, centralized, population-based recall of young children based on birth records is a realistic alternative at the present time in most states.

One challenge of using IIS data that has been populated with birth data is the amount of incorrect addresses in these data. Although we were able to decrease the number of incorrect addresses from 40% to 15% by contacting the site of last service in CIIS, these efforts increased the cost substantially. Contact information could be improved and cost reduced by at least 2 methods. First, if providers routinely updated demographic information in the state IIS, this could substantially increase the quality of contact data and permit a more accurate denominator in the IIS. This has been difficult for practices to do when it required data entry on a case-by-case basis, but could be greatly facilitated by increased use of Health Level Seven (HL7) messaging that permits automatic uploads from electronic medical records to IIS databases.21 In addition, further iterations of population-based recall could include a collaborative approach, whereby recall is conducted centrally by the public health department, but with the active involvement of primary care providers. Given the increased effectiveness of practice-based recall among those who conducted recall in our study, the optimal method might be one in which providers assist in uploading new addresses and the public health department generates notices that appear to come from both the health department and the individual practice. In this way, costs are reduced by sending to more valid addresses, the practice is credited by families for caring for their children, and the recall is more effective and less intrusive because it is endorsed by the child’s provider. A more collaborative approach could also be helpful in improving the accuracy of CIIS data, and therefore, in decreasing the amount of unnecessary recall.

There were important strengths and limitations to the data presented here. As much as feasible, this trial was population-based and, therefore, not subject to the types of bias introduced by studying subpopulations only. However, we were unable to examine differences in outcomes within socioeconomic or racial/ethnic subgroups that might have been of interest. We were also able to measure both effectiveness and cost-effectiveness for entire counties. The estimates of the denominator for the counties were not exact, although the degree of uncertainty should be similar between the 2 study arms. We were not able to control for other efforts in the area of immunization delivery in the counties of which we were unaware. Because this was a comparative effectiveness trial of 2 county-based interventions, there was no true control group in which to assess the rate of becoming UTD without any interventions. In addition, the number of practices that chose to participate in recall activities could have varied by geographic location, although national data suggested that our results about provider willingness to conduct recall were not unusual. The cost-effectiveness analysis was conducted from the perspective of the parties that might be responsible for conducting recall (either the health department or the practice) and did not take into account the costs and benefits from the perspective of the family or the much broader, theoretical perspective of society as a whole. The social benefits and costs to increasing immunization rates are expected to be the same in both types of interventions, and only vary by the rate itself. For the purpose of this study, it was important to focus on the costs to the potential implementers if reminder/recall methods are to be more widely adopted. Finally, because practice participation in CIIS is voluntary, incomplete entry of data into the CIIS undoubtedly resulted in artificially low immunization rates, both at baseline and after the interventions in all counties, although this would not be expected to affect the comparative trial.

This trial supports consideration of a new paradigm in the way reminder/recall for immunizations should be conducted in this country. Before this approach can be embraced, however, several issues need to be addressed. First, more data are needed about the acceptability of centralized recall to families and providers. Preliminary data from a survey conducted in the counties involved in this trial suggested only 15% of physicians would not want the public health department to conduct recalls, and 51% reported they thought population-based recall conducted by the health department was the optimal approach.22 Data from parents are lacking at present. Second, although more cost-effective than the practice-based approach, the implementation of population-based reminder/recall would still require additional resources either from state or federal funds or shared resources between practices and the public sector. Overwhelming evidence demonstrated that at a societal level, raising immunization UTD rates before entry into preschool would not only be cost-effective but cost saving,23–40 but we, as a society, would need to be willing to make the initial investment. If a population-based approach were used to remind parents about the need for preventive care before they were overdue for immunizations rather than recalling them after the fact, it could be more cost-effective than we demonstrated in this study because it could increase preventive care without inappropriately recalling children who were already immunized. Data from this trial supported a national dialogue to address options for more effective, less costly reminder/recall approaches to increase immunization levels in young children.


This study was supported by the National Library of Medicine of the National Institutes of Health (Award RC1LM010513). Clinical Trials Registration Number: NCT01296906 (https://register.clinicaltrials.gov/prs/app/action/SelectProtocol?sid=S0002V4A&selectaction=View&uid=U0000872&ts=5&cx=-ie3log).

Note. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Library of Medicine or the National Institutes of Health.

Human Participant Protection

The Colorado institutional review board (COMIRB) first approved the study (#09-0725) as an Exempt protocol September 2009. After some amendments to the protocol, it was found to fall under the Expedited category in August 2010. The current COMIRB protocol number is #10-0823.


1. Centers for Disease Control and Prevention. Ten great public health achievements–United States, 1900-1999. MMWR Morb Mortal Wkly Rep. 1999;48(12):241243. MedlineGoogle Scholar
2. Centers for Disease Control and Prevention. Impact of vaccines universally recommended for children–United States, 1990-1998. MMWR Morb Mortal Wkly Rep. 1999;48(12):243248. MedlineGoogle Scholar
3. McNabb SJ, Jajosky RA, Hall-Baker PA, et al. Summary of notifiable diseases—United States, 2005. MMWR Morb Mortal Wkly Rep. 2007;54(53):192. MedlineGoogle Scholar
4. US Department of Health and Human Services, Office of Disease Prevention and Health Promotion. Healthy People 2020. Available at: http://www.healthypeople.gov/2020/topicsobjectives2020. Accessed February 15, 2012. Google Scholar
5. Briss PA, Rodewald LE, Hinman AR, et al. Reviews of evidence regarding interventions to improve vaccination coverage in children, adolescents, and adults. The Task Force on Community Preventive Services. Am J Prev Med. 2000;18(suppl):97140. Crossref, MedlineGoogle Scholar
6. Task Force on Community Preventive Services. Recommendations regarding interventions to improve vaccination coverage in children, adolescents, and adults. Am J Prev Med. 2000;18(suppl):9296. MedlineGoogle Scholar
7. Tierney CD, Yusuf H, McMahon SR, et al. Adoption of reminder and recall messages for immunizations by pediatricians and public health clinics. Pediatrics. 2003;112(5):10761082. Crossref, MedlineGoogle Scholar
8. Thorpe KE, Zwarenstein M, Oxman AD, et al. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. CMAJ. 2009;180(10):E47E57. Crossref, MedlineGoogle Scholar
9. US Department of Health and Human Services. NIH health care systems research collaboratory - pragmatic clinical trials demonstration projects (UH2/UH3). Available at: http://grants.nih.gov/grants/guide/rfa-files/RFA-RM-12-002.html. Accessed July 31, 2012. Google Scholar
10. Centers for Disease Control and Protection (CDC). Recommended immunization schedule for persons aged 0 through 6 years–United States, 2012. Available at: http://www.cdc.gov/vaccines/recs/schedules/downloads/child/0-6yrs-schedule-bw.pdf. Accessed February 6, 2012. Google Scholar
11. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):13221327. LinkGoogle Scholar
12. Littel RC, Milliken GA, Stroup WW, Wolfinger RD. SAS System for Mixed Models. Cary, NC: SAS Institute, Inc.; 1996. Google Scholar
13. Raudenbush SW, Bryk AS. Hierarchical Linear Models: Applications and Data Analysis Methods (Advanced Quantitative Techniques in the Social Sciences). 2nd ed. Newbury Park, CA: Sage Publications, Inc.; 2001. Google Scholar
14. Szilagyi PG, Bordley C, Vann JC, et al. Effect of patient reminder/recall interventions on immunization rates: a review. JAMA. 2000;284(14):18201827. Crossref, MedlineGoogle Scholar
15. Szilagyi P, Vann J, Bordley C, et al. Interventions aimed at improving immunization rates. Cochrane Database Syst Rev. 2002;4:CD003941. Crossref, MedlineGoogle Scholar
16. Jacobson Vann JC, Szilagyi P. Patient reminder and patient recall systems to improve immunization rates. Cochrane Database Syst Rev. 2005;3:CD003941. MedlineGoogle Scholar
17. Jacobson Vann JC, Szilagyi P. Patient reminder and patient recall systems to improve immunization rates: edited 2009. Cochrane Database Syst Rev. 2009;CD003941. Google Scholar
18. Saville AW, Albright K, Nowels C, et al. Getting under the hood: exploring issues that affect provider-based recall using an immunization information system. Acad Pediatr. 2011;11(1):4449. Crossref, MedlineGoogle Scholar
19. Setse RW, Euler GL, Gonzalez-Feliciano AG, et al. Influenza vaccination coverage - United States, 2000-2010. MMWR Surveill Summ. 2011;60(suppl):3841. MedlineGoogle Scholar
20. Centers for Disease Control and Protection (CDC). IIS interactive data query tool. Available at: http://www2a.cdc.gov/nip/registry/IISAR/IISAR_QUERY.asp. Accessed February 6, 2012. Google Scholar
21. US Department of Health and Human Services and American Immunization Registry Association. HL7 Version 2.5.1: Implementation guide for immunization messaging. Available at: http://www.cdc.gov/vaccines/programs/iis/stds/downloads/hl7guide-08-2011.pdf. Accessed February 6, 2012. Google Scholar
22. Kempe A, Saville AW, Dickinson LM, et al. Population-based versus practice-based reminder/recall: what do providers think? Pediatric Academic Societies and Asian Society for Pediatric Research Annual Meeting; Denver, CO, April 30-May 3, 2011. Google Scholar
23. Zhou F, Santoli J, Messonnier ML, et al. Economic evaluation of the 7-vaccine routine childhood immunization schedule in the United States, 2001. Arch Pediatr Adolesc Med. 2005;159(12):11361144. Crossref, MedlineGoogle Scholar
24. Zhou F, Bisgard KM, Yusuf HR, Deuson RR, Bath SK, Murphy TV. Impact of universal Haemophilus influenzae type b vaccination starting at 2 months of age in the United States: an economic analysis. Pediatrics. 2002;110(4):653661. Crossref, MedlineGoogle Scholar
25. Lieu TA, Ray GT, Black SB, et al. Projected cost-effectiveness of pneumococcal conjugate vaccination of healthy infants and young children. JAMA. 2000;283(11):14601468. Crossref, MedlineGoogle Scholar
26. Ekwueme DU, Strebel PM, Hadler SC, Meltzer MI, Allen JW, Livengood JR. Economic evaluation of use of diphtheria, tetanus, and acellular pertussis vaccine or diphtheria, tetanus, and whole-cell pertussis vaccine in the United States, 1997. Arch Pediatr Adolesc Med. 2000;154(8):797803. Crossref, MedlineGoogle Scholar
27. Fendrick AM, Lee JH, LaBarge C, Glick HA. Clinical and economic impact of a combination Haemophilus influenzae and Hepatitis B vaccine: estimating cost-effectiveness using decision analysis. Arch Pediatr Adolesc Med. 1999;153(2):126136. Crossref, MedlineGoogle Scholar
28. Miller MA, Sutter RW, Strebel PM, Hadler SC. Cost-effectiveness of incorporating inactivated poliovirus vaccine into the routine childhood immunization schedule. JAMA. 1996;276(12):967971. Crossref, MedlineGoogle Scholar
29. Margolis HS, Coleman PJ, Brown RE, Mast EE, Sheingold SH, Arevalo JA. Prevention of hepatitis B virus transmission by immunization. An economic analysis of current recommendations. JAMA. 1995;274(15):12011208. Crossref, MedlineGoogle Scholar
30. Lieu TA, Cochi SL, Black SB, et al. Cost-effectiveness of a routine varicella vaccination program for US children. JAMA. 1994;271(5):375381. Crossref, MedlineGoogle Scholar
31. Huse DM, Meissner HC, Lacey MJ, Oster G. Childhood vaccination against chickenpox: an analysis of benefits and costs. J Pediatr. 1994;124(6):869874. Crossref, MedlineGoogle Scholar
32. Bloom BS, Hillman AL, Fendrick AM, Schwartz JS. A reappraisal of hepatitis B virus vaccination strategies using cost-effectiveness analysis. Ann Intern Med. 1993;118(4):298306. Crossref, MedlineGoogle Scholar
33. Hay JW, Daum RS. Cost-benefit analysis of two strategies for prevention of Haemophilus influenzae type b infection. Pediatrics. 1987;80(3):319329. MedlineGoogle Scholar
34. Cochi SL, Broome CV, Hightower AW. Immunization of US children with Hemophilus influenzae type b polysaccharide vaccine. A cost-effectiveness model of strategy assessment. JAMA. 1985;253(4):521529. Crossref, MedlineGoogle Scholar
35. White CC, Koplan JP, Orenstein WA. Benefits, risks and costs of immunization for measles, mumps and rubella. Am J Public Health. 1985;75(7):739744. LinkGoogle Scholar
36. Hinman AR, Koplan JP. Pertussis and pertussis vaccine. Reanalysis of benefits, risks, and costs. JAMA. 1984;251(23):31093113. Crossref, MedlineGoogle Scholar
37. Mulley AG, Silverstein MD, Dienstag JL. Indications for use of hepatitis B vaccine, based on cost-effectiveness analysis. N Engl J Med. 1982;307(11):644652. Crossref, MedlineGoogle Scholar
38. Koplan JP, Preblud SR. A benefit-cost analysis of mumps vaccine. Am J Dis Child. 1982;136(4):362364. MedlineGoogle Scholar
39. Albritton RB. Cost-benefits of measles eradication: effects of a federal intervention. Policy Anal. 1978;4(1):121. MedlineGoogle Scholar
40. Schoenbaum SC, Hyde JN Jr, Crampton K. Benefit-cost analysis of rubella vaccination policy. N Engl J Med. 1976;294(6):306310. Crossref, MedlineGoogle Scholar


No related items




Allison Kempe, MD, MPH, Alison Saville, MSPH, MSW, L. Miriam Dickinson, PhD, Sheri Eisert, PhD, Joni Reynolds, RN, MSN, Diana Herrero, MS, Brenda Beaty, MSPH, Karen Albright, PhD, Eva Dibert, MHA, Vicky Koehler, MPH, Steven Lockhart, BA, and Ned Calonge, MDAllison Kempe, Alison Saville, L. Miriam Dickinson, Brenda Beaty, Karen Albright, Eva Dibert, Vicky Koehler, and Steven Lockhart are with Children’s Outcomes Research Program, The Children’s Hospital, Denver, CO. Allison Kempe is with the Department of Pediatrics, University of Colorado Denver. L. Miriam Dicksinson and Brenda Beaty are with the Colorado Health Outcomes Program, University of Colorado Denver. Sheri Eisert is with the Department of Family Medicine, University of Colorado Denver. Joni Reynolds, Diana Herrero, and Ned Calonge are with the Colorado Department of Public Health and Environment, Denver. Karen Albright is with the Department of Community and Behavioral Health, Colorado School of Public Health, University of Colorado Denver. “Population-Based Versus Practice-Based Recall for Childhood Immunizations: A Randomized Controlled Comparative Effectiveness Trial”, American Journal of Public Health 103, no. 6 (June 1, 2013): pp. 1116-1123.


PMID: 23237154