Objectives. We implemented active surveillance for Guillain–Barré syndrome (GBS) following seasonal or H1N1 influenza vaccination among the Medicare population during the 2009-2010 influenza season.
Methods. We used weekly Medicare claims data to monitor vaccinations and subsequent hospitalizations with principal diagnosis code for GBS within 42 days. Group sequential testing assessed whether the observed GBS rate exceeded a critical limit based on the expected rate from 5 previous years adjusted for claims delay. We evaluated the lag between date of service and date of claims availability and used it for adjustment.
Results. By July 30, 2010 (after 26 interim surveillance tests), 14.0 million seasonal and 3.3 million H1N1 vaccinations had accrued. Taking into account claims delay appropriately lowered the critical limit during early monitoring. The observed GBS rate was below the critical limit throughout the surveillance.
Conclusions. Medicare data contributed rapid safety monitoring among millions of 2009–2010 influenza vaccine recipients. Adjustment for claims delay facilitates early detection of potential safety issues. Although limited by lack of medical record review to confirm cases, this claims-based surveillance did not indicate a statistically significant elevated GBS rate following seasonal or H1N1 influenza vaccination.
In 2009, public health efforts in the United States to address the pandemic of a novel influenza A (H1N1) virus included a federal effort to facilitate development and distribution of vaccines.1,2 Five monovalent vaccines against the novel strain (A/California/7/09-like virus) were approved as strain change formulations to each manufacturer’s licensed seasonal influenza vaccine.3–9
Safety monitoring of licensed vaccines has traditionally relied largely on passive surveillance,10 which is national and timely but subject to multiple limitations (e.g., underreporting, reporting biases, and lack of denominator data on vaccinated persons11), and epidemiological studies,12 which may require years to complete. Early detection of safety issues is especially important for influenza vaccine monitoring because most vaccinations are administered within a short period of a few months. Rapid active surveillance using data from managed care networks has emerged more recently.13–15 However, these databases may underrepresent the elderly and may not have sufficient statistical power to evaluate very infrequent adverse events.
To address these issues and prepare for a potential influenza pandemic, a pilot project was initiated in 2006 by the Food and Drug Administration (FDA) and Centers for Medicare and Medicaid Services (CMS) that aimed to monitor influenza vaccine safety as rapidly as possible (i.e., near real time) among the Medicare population. Medicare insures approximately 38.8 million elderly (aged ≥ 65 years) and 7.8 million younger persons with disability or end-stage renal disease.16 The pilot phase (2006–2009) aimed to develop the technical and methodological capability to use Medicare administrative data for safety monitoring as soon as data accrued each week. It focused on surveillance for specific conditions after vaccination such as Guillain–Barré syndrome (GBS), an autoimmune disorder that produces weakness or paralysis, which was associated with the 1976–1977 swine influenza (A/New Jersey/1976/H1N1) vaccine.17–20 Some studies of the 1976–1977 vaccine indicated relative risks of 4 to 9, corresponding to 5 to 10 excess cases per million persons vaccinated, with most or all of the elevated risk within 6 weeks after vaccination. Higher relative risks (approximately 11–18 in some analyses) were found for onset in weeks 2 and 3.18–20
A challenge in using claims and potentially other types of health care databases for active surveillance is the lag between date of service and date when information is available in the system. In the Medicare program, a vaccination or hospitalization is observed after a health care provider seeks payment for services by submitting a claim to Medicare administrative contractors that process it and then submit it to CMS. Not accounting for this delay can potentially lead to biased results during early monitoring. We implemented methods to account for claims delay in sequential monitoring and describe results of the first prospective application of these methods, specifically to monitor for GBS after seasonal or H1N1 influenza vaccination among the Medicare population during the 2009-2010 influenza season.
We used Medicare monthly enrollment and weekly claims data (carrier, outpatient, inpatient files) under data use agreement with CMS as part of the “SafeRx” project, an interagency collaboration focused on medical product safety. Vaccination was identified using first occurrence (starting mid-August) of one of the following Healthcare Common Procedure Coding System codes in the carrier or outpatient data files: seasonal influenza vaccination, G0008, 90655, 90656, 90657, 90658, 90659, 90660, 90724; H1N1 pandemic influenza vaccination, G9141, G9142.21,22 We defined hospitalization for possible GBS by using the inpatient files and International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) code 357.0 as a principal discharge diagnosis.23 Persons were included if enrolled in fee-for-service Medicare Parts A and B throughout the analysis window (or until they died).
We assessed timeliness of influenza vaccine claims, GBS claims, and GBS claims after vaccination with data from 5 influenza seasons (2004–2005, 2005–2006, 2006–2007, 2007–2008, and 2008–2009) accrued through May 21, 2010. We used data from the 2006–2007 season to evaluate the effect of claims delay on the observed number of cumulative vaccinations, and observed rate of GBS after vaccination. To evaluate the effect of adjusting for claims delay, we compared the GBS rate in the 2006–2007 influenza season with the weighted mean rate (i.e., weighted by vaccinations contributed) from the 2 previous influenza seasons. These retrospective analyses re-created how the data would appear if viewed in real time.
The comparison group was persons who received seasonal influenza vaccine in previous years (i.e., previous vaccine formulations). Advantages of the historical vaccinated group included the fact that data had already accrued so background rates of illnesses (e.g., GBS) could be assessed, and multiple years could be combined to define an expected rate. A contemporary unvaccinated group was not used as a comparator because of concerns about potential misclassification of some vaccinated persons as unvaccinated, either because the vaccination was not billed to Medicare24 or because of claims delay during surveillance. For each historical vaccinated cohort in the comparison group, we performed a screening analysis to confirm that the GBS rate in the period immediately after vaccination was not elevated based on claims data; to do this, we compared the GBS rate in weeks 0 through 6 to later periods for each cohort (weeks 7–12, 13–18, 19–24, 25–30, and 31–36 after vaccination) because later periods likely exhibit the background rate of GBS unrelated to vaccination. Persons who died were censored. We used data accrued through August 27, 2010.
We performed active surveillance by using group sequential testing methods, and adjusted for claims delay. Group sequential methods statistically account for assessment at multiple time points25,26 as the size of the vaccinated group increases because of batches of new claims. We defined a signal as an observed GBS rate higher than a critical limit. The critical limit is the threshold beyond which the observed GBS rate is statistically significantly higher than the expected rate. Specifically, we monitored the number of vaccinated persons and, of those, the number with GBS. We assessed the GBS rate within selected risk windows (0–42, 0–21, and 7–21 days) after vaccination. For the 2009–2010 influenza season, the comparison group was seasonal influenza vaccine recipients in 5 previous years; the expected GBS rate was defined as the weighted mean rate over those 5 years for the corresponding windows. We performed sequential tests of whether the observed GBS rate was higher than the expected rate.
To create the critical limits, we developed methods employing simulation (with Stata version 11 [StataCorp LP, College Station, TX] and Visual Basic .NET 3.5 [Microsoft, Redmond, WA]), and used a binomial distribution for the occurrence of GBS after vaccination. We used the observed cumulative number of vaccinations at each time point during surveillance, and characteristics of each vaccination claim (e.g., date of vaccination, date claim first appeared in the data), to simulate multiple samples with this size and structure. New claims were available each Friday night, and, generally, surveillance results were available by the following Tuesday. Observed vaccination claims for any particular Friday processing date were stratified by the interval since vaccination. The interval was a time-varying factor, incrementing by 1 for each sequential week that claims accrued. For each stratum (whole weeks between vaccination service date and claims processing date), the period represents the amount of time available for a corresponding GBS claim to appear in the data. The probability that a GBS claim would appear within that period (for the accrued vaccination claims) depends on 2 factors: the expected GBS rate, and the expected distribution of delay times between vaccination date and GBS claim processing date. Thus, in the simulation, GBS was assigned to some persons in each vaccinated sample according to the expected GBS rate (binomial probability) defined from the 5 previous influenza seasons. Also, a delay time for each simulated GBS case was assigned according to the distribution of delay times in the 5 previous seasons (probability of 0, 1, 2, 3, etc. weeks delay between vaccination date and GBS claim processing date). To calculate the stratum-specific GBS rate, a GBS event was only counted if its assigned delay time was less than or equal to the number of weeks elapsed after the vaccination date, which is the period available for the event to be observed. The stratum-specific rates (weighted identically, to reflect the observed data) were summed to yield the GBS rate adjusted for claims delay for the simulated sample. Repeated simulated samples (iterations) yielded a distribution of the GBS rates, and a selected percentile of the ranked rates defined the critical limit.
For multiple tests (e.g., weekly), the overall type-1 error (α) was fixed and a portion was assigned to each for group sequential testing.25 If the null hypothesis is true, multiplicatively combining the conditional probability of not signaling at each test yields the overall probability of not signaling (1 minus α). At each test date, we used the simulated distributions of GBS rates to select the percentile that defined the critical limit. Moving to the next test date, all simulated rates exceeding any of the previous critical values were not carried forward; these rates already indicate rejection of the null and were counted in the α spending up to that date. For the 2009–2010 surveillance, we specified an overall 1-sided α of 0.05 for each category of influenza vaccines (i.e., seasonal or H1N1 vaccines) and apportioned it among 25 consecutive weekly tests and a later 26th test (at the end of the influenza season), with a constant amount of α at each test. For seasonal influenza vaccines, testing began in late September 2009; at that time, approximately 1.7 million vaccinations had accrued. For H1N1 vaccinations, testing began in November 2009 after 100 000 vaccinations had accrued to increase the sensitivity for early detection of a safety concern.
We assessed timeliness on more than 72 million influenza vaccination claims and 352 GBS claims after vaccination. For vaccination claims, the median (25th, 75th percentile) whole weeks delay between vaccination date and claims processing date was 3 (1, 10). The distribution was right-skewed and the 90th, 95th, and 99th percentiles were 21, 30, and 55 weeks, respectively. For GBS claims, the median (25th, 75th percentile) whole weeks between hospital discharge date and claims processing date was 2 (1, 3), and the 90th, 95th, and 99th percentiles were 5, 7, and 12, respectively. Because surveillance uses hospital admission date, not discharge date, the duration of the hospital stay is another component of GBS claim delay (range = 0 to 56 days; median [25th, 75th percentile] = 9 [5, 14] days). Choice of risk interval contributes to the delay between vaccination date and GBS claim processing date, which is used in the simulation. We identified no secular trend in this delay from 2004–2005 through 2008–2009. For the 0-to-21–day interval, the median (25th, 75th percentile) whole weeks between vaccination date and GBS claim processing date was 5 (4, 6) and 90th, 95th, and 99th percentile was 9, 10, and 15, respectively. Analogous results for the 0-to-42–day interval were 7 (5, 9), and 11, 13, and 17, respectively. To check that GBS hospitalizations were likely incident events, we performed a search for previous GBS hospitalizations during the preceding 2 years. Because the occurrence was low (0%–4% per influenza season among persons enrolled for at least 2 years), a previous GBS code was not an exclusion criterion.
Because of claims delay, the number of vaccinations observed initially for a particular vaccination week underestimates the number that had truly occurred and would ultimately be observed once all data had accrued (Appendix, Figure A, available as a supplement to the online version of this article at http://www.ajph.org). Also, the GBS rate observed initially may be only a fraction of the true GBS rate that would ultimately be observed once all data have accrued (Appendix, Figure B, available as a supplement to the online version of this article at http://www.ajph.org). Adjusting for claims delay yields critical limits that are more appropriate and facilitates early signal detection (Appendix, Figure C, available as a supplement to the online version of this article at http://www.ajph.org).
For each of 5 historical influenza seasons, the GBS rate in weeks 0 to 6 after vaccination was either lower or not statistically significantly elevated compared with the rate in multiple later periods (Figure 1). The variability in GBS rates between years, and between 6-week periods after vaccination of each cohort, is illustrated.
For the 2009–2010 influenza season, 14.0 million seasonal and 3.3 million H1N1 influenza vaccinations were available in the Medicare data as of July 30, 2010. Seasonal influenza vaccinations were administered earlier in the season than the H1N1 vaccinations (Figure 2), consistent with recommendations and availability of vaccines for this predominantly elderly population.27
The observed GBS rate was lower than the critical limit throughout the surveillance. This was true for both seasonal vaccines (Figure 3a, Table 1) and H1N1 vaccines (Figure 3b, Table 1) for all of the risk windows evaluated (0–42, 0–21, and 7–21 days). Similar results were found for all persons and the elderly (aged ≥ 65 years). The GBS rate in the 0 to 42 days after vaccination was 5.6 per million persons vaccinated with seasonal influenza vaccine and 5.5 per million persons vaccinated with H1N1 influenza vaccine. These rates were not significantly different from the expected rate of 4.8 per million persons vaccinated as indicated by lack of a signal. The relative risks were 1.2 and 1.1. Slightly higher relative risks (not statistically significant) were found for the 7-to-21–day window among persons aged 65 years and older who received seasonal influenza vaccine, and the 0-to-21–day and 7-to-21–day windows among persons who received H1N1 influenza vaccine (Table 1).
Cumulative Number and Rate of Events After 2009–2010 Seasonal and H1N1 Influenza Vaccination, Compared With Expected Rate Based on Historical Data: US Medicare Population
|Influenza Vaccine||Risk Window, Days||Age Group, Years||Vaccine Doses Observed, No.||No. of GBSa Events in Vaccine-Exposed Window||Rate in Vaccine-Exposed Window (per Million Vaccinees)||Expected No. of Events Based on Historical Ratesb||Expected Rateb (per Million Vaccinees)||Relative Risk, Current Versus Historicalb||Signal? Is Observed Rate Greater Than Critical Limit?b|
|Seasonal||0–42||Allc||14 015 101||78||5.57||67.23||4.80||1.16||No|
|Seasonal||0–21||Allc||14 035 587||41||2.92||35.00||2.49||1.17||No|
|Seasonal||7–21||Allc||14 035 587||33||2.35||26.95||1.92||1.22||No|
|Seasonal||0–42||≥ 65||12 535 923||69||5.53||60.14||4.80||1.15||No|
|Seasonal||0–21||≥ 65||12 552 307||36||2.90||30.64||2.44||1.17||No|
|Seasonal||7–21||≥ 65||12 552 307||30||2.42||23.72||1.89||1.26||No|
|H1N1||0–42||Allc||3 283 576||18||5.48||15.74||4.79||1.14||No|
|H1N1||0–21||Allc||3 295 435||11||3.34||8.22||2.49||1.34||No|
|H1N1||7–21||Allc||3 295 435||9||2.73||6.33||1.92||1.42||No|
|H1N1||0–42||≥ 65||2 829 522||15||5.25||13.57||4.80||1.11||No|
|H1N1||0–21||≥ 65||2 838 756||9||3.12||6.93||2.44||1.30||No|
|H1N1||7–21||≥ 65||2 838 756||8||2.77||5.36||1.89||1.49||No|
Note. GBS = Guillain–Barré syndrome. Data are as of July 30, 2010.
aBased on International Classification of Diseases, Ninth Revision, Clinical Modification code.23
bAccounts for delay in claims accrual.
cPersons younger than 65 years have disability or end-stage renal disease.
This claims-based surveillance detected no elevation in the rate of GBS following seasonal or H1N1 influenza vaccination among the Medicare population. These results are based on approximately 14 million seasonal and 3.3 million H1N1 influenza vaccinations among this predominantly elderly population.
This surveillance represents the first prospective implementation of new methods to address the claims delay when using health care databases for rapid safety assessment. Pilot analyses demonstrated that claims delay leads to an artifactually low vaccination count, and GBS rate after vaccination, until all claims had accrued. If no adjustment for claims delay was incorporated, false assurance about safety could occur during early monitoring because of inappropriate comparison of rates based on delayed claims to completely accrued historical claims. Our method differs from the maximized sequential probability ratio test28 by defining the critical limit based on a distribution of rates rather than a likelihood ratio, and expanding the capability for early detection of a safety signal by not waiting for the risk window to close and by adjusting for claims delay.
We used the rate of GBS after seasonal influenza vaccination in 5 previous years as the comparator. The screening analysis did not provide evidence of an elevated GBS risk (defined by principal ICD-9-CM diagnosis code) in association with the previous vaccines (Figure 1). Had the rate in the period immediately after vaccination been statistically significantly higher than in later periods, it would have provided preliminary evidence of a potential association between vaccination and GBS in those years (e.g., a risk similar to the 1976–1977 vaccine would have appeared as bars 4 to 9 times higher in weeks 0 through 6 than in later periods). Although diagnosis codes have imperfect positive predictive value for true GBS, this analysis was useful in checking that the comparator was similar to the background rate of GBS-coded hospitalizations. An evaluation of GBS after the 2000–2001 and 2001–2002 seasonal influenza vaccines that additionally included medical record review to confirm GBS cases, did not support an association with vaccination among the elderly Medicare population.29 The current surveillance assessed whether the 2009–2010 seasonal and H1N1 influenza vaccines were less safe than the seasonal vaccines used in the 5 previous years, and the screening analysis provides evidence that the previous years’ vaccines did not have an elevated rate of GBS-coded hospitalizations. The use of multiple rather than single historical years of data to define the expected GBS rate makes results less subject to random variability.
CMS created new Healthcare Common Procedure Coding System codes for billing H1N1 vaccinations,21,22 which were necessary for distinguishing H1N1 from seasonal vaccinations for safety monitoring. Medicare allows vaccination billing from traditional and nontraditional providers (e.g., community pharmacies, public health clinics).30 Despite this, vaccine coverage rates from claims data have been lower than surveys23,31,32; the true vaccination coverage is unknown as limitations of surveys include potential recall or other response bias. We avoided this issue through our design by using only vaccinated cohorts, but it is unknown if the 2009–2010 vaccination recommendations or distribution system resulted in differences in characteristics of vaccinees compared with historical cohorts regarding unmeasured predisposing factors for GBS.
Although surveillance based on ICD-9-CM codes allows for rapid assessment of vaccine safety, a limitation is that not all GBS-coded events meet a standardized case definition.29 Also, admission date is an imperfect surrogate for onset date. Although such misclassification is likely nondifferential with respect to the current versus previous influenza seasons, it possibly could lead to a reduced relative risk (i.e., “bias toward the null”) and missing a signal even though one is present. Data from 2000 to 2002 indicated that the proportion of hospitalizations with principal diagnosis code 357.0 that met a standardized case definition for GBS after medical record review was 82% (21% definite, 36% probable, 25% possible).29 This was substantially higher than the positive predictive value of 20% found for secondary diagnoses (diagnoses in positions 2–10), and, therefore, we used principal diagnosis for surveillance. Also, in the 2000 to 2002 data, the median (25th, 75th percentile) interval between onset and admission date for definite, probable, and possible GBS cases with principal diagnosis code 357.0 was 4 (2, 11) days.29 Use of admission date (as a surrogate) may lead to some misclassification of GBS as being within the risk window, when true onset may be outside the window (e.g., before vaccination). In addition, persons with admission date after the risk window, but true onset within the risk window, are not counted. Use of the 7-to-21–day risk window potentially increased specificity (by excluding days 0–6), both because of increased likelihood of sufficient time for hypothetical development of an autoimmune process, and decreased likelihood of counting GBS events that actually had onset before vaccination.
Although we detected no signal of elevated risk in this surveillance, if a signal had occurred, it would not be sufficient to demonstrate a causal association. Further evaluation would be needed (e.g., data quality checks, assessment of potential confounding, hypothesis confirmation study using validated cases).
An overall assessment of the 2009–2010 H1N1 pandemic vaccine safety is ongoing. Another surveillance project (Emerging Infections Program sponsored by the Centers for Disease Control and Prevention) has identified a weak signal for GBS, but results are preliminary and further evaluation is continuing.33,34 The surveillance methods were different, as was the age distribution. The Emerging Infections Program project used medical record review to confirm cases, but ascertainment of potential cases and vaccinated and unvaccinated person-time relied on multiple unlinked data sources. In contrast, strengths of surveillance with Medicare data include that it is cohort-based with ascertainment of vaccination status and potential GBS all within the same individually linked database system. Available information from the Emerging Infections Program project suggests that, if the potential risk is confirmed, the excess risk may be approximately 1 to 2 excess cases per million persons vaccinated, similar to the risk found in a study of the 1992–1993 and 1993–1994 seasonal influenza vaccines.35 Data from the Medicare program are contributing to an ongoing effort coordinated by the US Department of Health and Human Services National Vaccine Program Office and Federal Immunization Safety Task Force H1N1 Data Coordination Working Group to synthesize data from federally sponsored projects evaluating the safety of the H1N1 pandemic influenza vaccine. In follow-up to the initial surveillance efforts, an additional evaluation has been initiated that will use supplementary data and will involve only cases that meet a standardized case definition. Data from the projects are also reviewed by a working group of the National Vaccine Advisory Committee, which comprises nongovernment personnel and advises the Department of Health and Human Services’ Assistant Secretary for Health.36
The reason(s) why the 1976–1977 swine influenza vaccine was associated with GBS remains unknown. Although contamination during manufacturing by Campylobacter jejuni antigens that mimic human gangliosides has been hypothesized, laboratory results did not support this.37 Also, antiganglioside antibodies, associated with GBS development in humans, were induced in mice by administration of the vaccine, but the effect was not unique to the 1976–1977 vaccine.37 Both the 1976 and 2009 H1N1 viruses descended from classical swine influenza,38–40 but differences have been noted. The number and percentage of nucleotide and amino acid differences in hemagglutinin, and in neuraminidase, were similar or greater than differences between H3N2 viruses from 1975 and 2007.41 Several of the 2009 virus gene segments likely originated in avian or human viruses that entered swine after 1976.39,40 Also, when ferret postinfection antisera was raised against the 2009 virus, no cross-reactivity was detected via hemagglutination inhibition assay with the 1976 virus.39
The current surveillance included the largest number of vaccinations ever reported to be monitored via prospective active adverse event surveillance, and provided timely rate-based comparisons among millions of vaccinees. The FDA Amendments Act of 2007 enhanced the authority of the FDA to develop a system of active postmarket risk identification and analysis.42 Active surveillance with Medicare data provides a potentially powerful tool for monitoring safety of influenza vaccines and other medical products among the elderly. For the 2009–2010 seasonal influenza vaccines, and the H1N1 vaccines distributed in the context of the public health emergency, these data provided a new and important surveillance resource.
This work was funded by the FDA, CMS, and National Vaccine Program Office, US Department of Health and Human Services.
The authors constituting the Safety Surveillance Working Group were Christopher M. Worrall, the Center for Medicare, Centers for Medicare and Medicaid Services, Baltimore, MD; Damien Marston and Kimberly Elmo, Office of Information Services, Centers for Medicare and Medicaid Services; Rebecca Kliman, Office of Clinical Standards and Quality, Centers for Medicare and Medicaid Services; and Yehuda Donde, Acumen LLC, Burlingame, CA. C. M. Worrall and K. Elmo contributed to aspects of project management pertaining to conception and design, and reviewed drafts of the article. D. Marston contributed to aspects of project and database management pertaining to conception and design, and reviewed drafts of the article. R. Kliman contributed to the project conception and article revision. Y. Donde contributed to data interpretation and article revision.
We are grateful to the following persons and indicate their affiliations at the time the work was performed. We wish to acknowledge Robyn Thomas, Office of Information Services, Centers for Medicare and Medicaid Services; M. Miles Braun, Office of Biostatistics and Epidemiology, Center for Biologics Evaluation and Research, FDA; and Emil Rusev, Acumen LLC, for contributions to the pilot phase. We also thank Rosalind Gullet, Acumen LLC, and Sanjaya Dhakal, Office of Biostatistics and Epidemiology, Center for Biologics Evaluation and Research, FDA, for technical assistance.
Aspects of this work were presented as abstracts at the 26th International Conference on Pharmacoepidemiology and Therapeutic Risk Management, Brighton, England, August 19 to 22, 2010, and the 45th Annual Meeting of the Infectious Diseases Society of America, San Diego, CA, October 4 to 7, 2007, and were included in other presentations (e.g., to federal government advisory committees) related to the safety surveillance activities.
Human Participant Protection
This work was approved by the institutional review board at the FDA and met regulatory criteria for a waiver of written informed consent. Data use was approved by the CMS Privacy Board.