Abstract
Objectives. We tested the effectiveness of offering home fecal immunochemical tests (FITs) during influenza vaccination clinics to increase colorectal cancer screening (CRCS).
Methods. In a clinical trial at Kaiser Permanente Northern California influenza clinics in Redwood City, Richmond, South San Francisco, Union City, and Fresno, we randomly assigned influenza clinic dates to intervention (FIT offered) or control (FIT not offered) and compared subsequent CRCS activity.
Results. Clinic staff provided FITs to 53.9% (1805/3351) of intervention patients aged 50 to 75 years. In the intent-to-treat analysis, 26.9% (900/3351) and 11.7% (336/2884) of intervention and control patients completed an FIT, respectively, within 90 days of vaccination (P ≤ .001). The adjusted odds ratio for completing FIT in the intervention versus the control arm was 2.75 (95% confidence interval = 2.40, 3.16). In the per protocol analysis, 35.4% (648/1830) of patients given FIT and 13.3% (588/4405) of patients not given FIT completed FIT within 90 days of vaccination (P ≤ .001).
Conclusions. This intervention may increase CRCS among those not reached by other forms of CRCS outreach. Future research should include the extent to which these programs can be disseminated and implemented nationally.
More than 143 000 new cases of colorectal cancer and more than 51 000 colorectal cancer deaths are expected in the US 2012 Census, making it the fourth leading cause of nonskin cancers diagnosed and the second leading cause of all cancer deaths among Americans.1 Colorectal cancer mortality can be reduced with screening.1,2 The US Preventive Services Task Force recommends colorectal cancer screening (CRCS) for average-risk adults aged 50 to 75 years using annual high-sensitivity fecal occult blood tests such as fecal immunochemical tests (FITs), flexible sigmoidoscopy every 5 years with interval fecal occult blood testing, or colonoscopy every 10 years.3 Only 58.6% of US adults aged 50 to 75 years were up to date with guideline-recommended screening in 2010.4
Achieving high CRCS rates requires evidence-based approaches, such as removal of barriers to obtaining and completing recommended tests, 1-to-1 health care team member interventions, and organized patient reminders.5 At Kaiser Permanente Northern California (KPNC), a combination of traditional primary care strategies and an organized system of FIT kit mailings has led to a CRCS rate of more than 75.0% for patients aged 50 to 75 years, exceeding the Healthy People 2020 target of 70.5%.6–8 Nonetheless, even at KPNC, many age-eligible patients remain unscreened. A strategy that may add to these efforts is the FLU-FIT Program, which is designed to allow non–physician-led health care teams to offer FIT kits to eligible patients when they seek annual influenza vaccinations. The program has been tested in safety net settings9–11 and was pilot tested at KPNC’s Santa Clara facility in 2008.12 In this new study, we provided a comprehensive test of the effectiveness of the FLU-FIT Program for KPNC influenza vaccination clinic attendees not reached with or responding to other CRCS opportunities.
KPNC is an integrated health care delivery system with more than 3 million members. KPNC provides financial incentives to its facilities to achieve annual CRCS targets and supports these goals with annual mailings of FIT kits (single-sample Polymedco OC-FIT-Chek brand, 100 ng cutoff; Polymedco, Cortlandt Manor, NY) to most health plan members who are due for CRCS. KPNC mails the kits between January and October to maximize year-end CRCS rates. Each October and November, KPNC also organizes drop-in influenza vaccination clinics at multiple facility locations, providing influenza vaccinations to hundreds of thousands of members of all ages. These clinics are coordinated centrally and overseen by local nursing administrators.
On the recommendation of KPNC’s CRCS and influenza vaccination team leaders, we approached the managers of 8 influenza vaccination clinics to participate. One site declined to participate because of lack of accurate local capture of colonoscopy status by the electronic health record, and another declined because they wished to pursue their own FLU-FIT Program outside the context of our research program. The remaining 6 clinics agreed to be enrolled. We later discovered that information systems at 1 enrolled clinic did not allow separation of patient data into intervention and control groups, so we excluded this clinic from our analyses. The remaining 5 participating KPNC clinics were located in Redwood City, Richmond, South San Francisco, Union City, and Fresno.
The study included patients aged 50 to 75 years who received an influenza vaccination at 1 of the 5 participating clinics during designated study dates. We conducted the trial at 2 sites in October–November 2009 and at the remaining 3 sites in October–November 2010.
This was a randomized clinical trial. Each facility provided us a list of influenza vaccination clinic dates for us to randomly assign to the intervention or control arms. Using blocks of 2 or 4 days, we randomly assigned each date in each site to either the intervention arm (FLU-FIT arm, providing FIT kits to eligible patients along with influenza vaccinations) or to the control arm (FLU-only arm, providing influenza vaccinations only). We selected this within-clinics design to allow each clinical site to serve as its own control and to ensure comparability between the control and intervention populations enrolled in each arm of the study at the different sites during successive influenza vaccination seasons. We asked the influenza vaccination clinic staff to provide FIT kits to eligible patients on FLU-FIT days either immediately before or when they registered for their influenza vaccination. They used KPNC member cards to identify patients aged 50 to 75 years and checked CRCS eligibility using the preventive health screen in the electronic health record.
The clinic staff provided FIT kits to eligible members they identified, with any combination of brief verbal messages, such as “Just like a flu shot, you need to complete a colon test every year,” “This test is free and could save your life,” and “You can do FIT today and mail it in tomorrow.” Patients with questions were directed to read the kit instructions (available in multiple languages with an added message about the importance of CRCS) or to contact their primary care clinician. Patients were free to decline FIT kits that were offered. For research purposes, we also asked the clinic staff to enter into a separate database the medical record number of each patient given a kit.
In the month before study initiation at each site, we met with the local clinic team to explain procedures, review staffing and computer requirements, and arrange for delivery of FIT kits and other study materials to each location. In a separate 1-hour session, we trained clinic staff to provide FIT kits to patients. Finally, we conducted a walkthrough of the clinic to help establish patient flow procedures.
During the study, the local clinic managers received weekly reminders about control and intervention dates, and a research associate visited each site at unannounced times to observe and assess fidelity to the research protocol and to answer implementation questions arising during these visits.
We designed the study to determine the effectiveness of the FLU-FIT intervention under circumstances representative of routine clinical care. For example, we permitted clinics to not offer the FLU-FIT Program at assigned times when they were understaffed or lacked resources or materials to provide FIT effectively. We asked each site to come up with its own procedures for clinic line management and gave them latitude in which messages to give patients and how much time to spend with each patient.
Sites that wanted to incorporate other activities besides the FLU-FIT Program into their influenza vaccination clinics were free to do so. Each site decided how many staff members to train in FLU-FIT procedures, how many computer stations to set up, and how to divide up the tasks of FLU-FIT implementation.
We created a data set from the electronic health record and patient registration data for patients aged 50 to 75 years who received influenza vaccinations at each participating site on study dates, including information clinic staff collected about which patients were given a FIT kit on intervention dates. Other data included patient age, gender, race and ethnicity, preferred language, number of primary care visits in the previous year, location of the influenza vaccination clinic attended, and dates of influenza vaccines and CRCS tests (FIT, flexible sigmoidoscopy, and colonoscopy).
At the time of our study, KPNC had a program to mail FIT kits to most patients who were overdue for CRCS. To measure the additive effect of the FLU-FIT Program to these mailings, and to assess the impact of timing of such mailings relative to influenza vaccination clinics, we obtained the dates on which KPNC had mailed study participants a FIT kit in the year before their influenza vaccination and 3 months after.
Our primary analyses focused on patients who were due for CRCS when they received their influenza vaccine. Unless otherwise noted, all analyses used the intent-to-treat method, that is, we included all people who were in the intervention group and were due for CRCS in the intervention group analyses regardless of whether they received a FIT kit. We compared demographic and clinical data between the intervention and control groups as well as between those in the intervention group who were and were not given a FIT kit, using the 2-sample t-test for continuous variables and the χ2 test for categorical variables. Next, using the χ2 test, we compared the completion in the 90 days after the immunization for each type of CRCS individually and in combination between those in the intervention and control groups.
We fit a set of logistic regression models that had up-to-date CRCS status in the 90 days after the vaccine as the dependent variable and 1 of the demographic or clinical variables as the sole independent variable. We included those variables that were statistically significant in a final multivariable logistic regression model. In an additional analysis, we included all study participants regardless of CRCS up-to-date status at the time of their influenza vaccination. Using generalized estimating equations, we fit models with CRCS up-to-date status as the dependent variable and the following independent variables: study group, time (at vaccination and 90 days after), and interaction between group and time, all while adjusting for the within-patient correlation. A statistically significant interaction term would indicate that the within-patient changes over time in up-to-date rates for the 2 groups were significantly different and, therefore, that the intervention had a significant effect.
Finally, we did a per protocol analysis in which we divided people due for CRCS at the time of their influenza vaccination into 2 groups based solely on whether they were given a FIT kit. Then, we compared the up-to-date CRCS status of the people in each group 3 months after their vaccinations between the 2 groups using the χ2 test. We conducted data analyses using SAS, version 9.1.3 (SAS Institute, Cary, NC).
Figure 1 depicts our study flow diagram. A total of 28 436 adults aged 50 to 75 years received an influenza vaccination on study dates. In the intervention arm, 3351 of 15 090 patients (22.2%) were due for CRCS; 2884 of 13 346 (21.6%) were due in the control arm. In the intervention arm, influenza clinic staff recorded providing FIT kits to 1805 of the 3351 patients (53.9%) due for CRCS. In the control arm, 1 site erroneously provided FIT kits to 25 CRCS-eligible patients on a single date. FIT kits were also provided erroneously to 417 of 11 322 patients (3.7%) who were already up to date with CRCS, including 172 who had colonoscopy in the prior 10 years.
FIGURE 1— Study flow diagram: FLU-FIT Program, Kaiser Permanente Northern California, 2009–2010.
Note. CRCS = colorectal cancer screening; FIT = fecal immunochemical test; FLU = influenza vaccination.
Table 1 displays the characteristics of individuals due for CRCS at the time of influenza vaccination, compared by study arm. There was a small but statistically significant difference in the average age between the 2 groups. There were more women than men overall but a similar gender distribution in both study arms. KPNC possesses race and ethnicity data for most of its population. Among those in each study arm for whom these data were available, there were similar levels of racial and ethnic diversity. Approximately 10% of the patient population identified a language preference other than English. Most patients had 1 or 0 primary care visits in the year before influenza vaccination, with more of these patients in the control arm (66.7% vs 62.4% in the intervention arm).
TABLE 1— Characteristics of Intervention and Control Group Participants: FLU-FIT Program, Kaiser Permanente Northern California, 2009–2010
| Variablea | Intervention Group (n = 3351), No. (%) or Mean ±SD | Control Group (n = 2884), No. (%) or Mean ±SD | P |
| Age, y | 61.9 ±7.7 | 61.1 ±7.6 | ≤ .001 |
| Male gender | 1444 (43.1) | 1211 (42.0) | |
| Race | .49 | ||
| Asian/Pacific Islander | 543 (16.2) | 500 (17.3) | |
| Black | 188 (5.6) | 170 (5.9) | |
| White | 1584 (47.3) | 1324 (45.9) | |
| Multiracial | 179 (5.3) | 136 (4.7) | |
| Unknown or other | 857 (25.6) | 754 (26.1) | |
| Ethnicity | .91 | ||
| Hispanic | 566 (21.7) | 465 (21.6) | |
| No. missing | 744 | 729 | |
| Preferred language spokenb | .45 | ||
| Spanish | 234 (7.1) | 192 (6.8) | |
| Various Asian | 115 (3.5) | 84 (3.0) | |
| English | 2921 (88.9) | 2533 (89.9) | |
| Other | 16 (0.5) | 9 (0.3) | |
| No. missing | 65 | 66 | |
| Outpatient primary care visits in previous y | .006 | ||
| 0 | 1493 (44.6) | 1365 (47.3) | |
| 1 | 597 (17.8) | 559 (19.4) | |
| 2–3 | 672 (20.1) | 535 (18.6) | |
| 4–10 | 494 (14.7) | 362 (12.6) | |
| > 10 | 95 (2.8) | 63 (2.2) | |
| Facility | ≤ .001 | ||
| 1 | 563 (16.8) | 443 (15.4) | |
| 2 | 562 (16.8) | 640 (22.2) | |
| 3 | 380 (11.3) | 412 (14.2) | |
| 4 | 978 (29.2) | 607 (21.1) | |
| 5 | 868 (25.9) | 782 (27.1) | |
| FIT kit mailed | .49 | ||
| 0–3 mo before vaccination | 385 (11.5) | 303 (10.5) | |
| 3–6 mo before vaccination | 643 (19.2) | 586 (20.3) | |
| 6–12 mo before vaccination | 547 (16.3) | 474 (16.4) | |
| Not in the y before vaccination | 1776 (53.0) | 1521 (52.7) | |
| 3 mo after vaccination | 67 (2.0) | 85 (3.0) | .016 |
Note. FIT = fecal immunochemical test. Percentages may not add up to 100% because of rounding.
a Characteristics of participants who were due for colorectal cancer screening and received influenza vaccination at 1 of the study facilities on a study date, divided by intervention and control arms.
b Asians include Arabic, Burmese, Cantonese, Farsi, Gujarati, Hindi, Hmong, Japanese, Khmer, Korean, Laotian, Mandarin, Mien, Punjabi, Samoan, Tagalog, Thai, Tongan, Vietnamese (on the basis of geographical definition of Asia). Other languages include Amharic, Dutch, French, Italian, Other, Polish, Portuguese, Russian, Turkish. English includes American Sign Language.
Table 2 shows the number and proportion of eligible patients who completed CRCS tests within 90 days of their influenza vaccination, compared by study group. Whereas 11.7% in the control arm completed FITs, 26.9% in the intervention arm completed FITs, representing a 15.2% point differential (P ≤ .001). Completion of sigmoidoscopy and colonoscopy in each arm was low. Overall, 15.2% in the control arm and 29.7% in the intervention arm completed any CRCS test in the 90 days after getting vaccinated, representing a 14.5% point differential (P ≤ .001).
TABLE 2— Proportion of Participants Completing Colorectal Cancer Screening Within 90 Days of Receiving Influenza Vaccination: FLU-FIT Program, Kaiser Permanente Northern California, 2009–2010
| Test | Intervention (n = 3351), No. (%) | Control (n = 2884), No. (%) | P |
| FIT | 900 (26.9) | 336 (11.7) | ≤ .001 |
| Sigmoidoscopy | 62 (1.9) | 68 (2.4) | .16 |
| Colonoscopy | 86 (2.6) | 61 (2.1) | .24 |
| FIT, sigmoidoscopy, or colonoscopy | 996 (29.7) | 438 (15.2) | ≤ .001 |
Note. FIT = fecal immunochemical test.
We repeated these analyses for different levels of each variable in Table 1. The relationships were consistent and statistically significant across gender, age, race (except for the small group of multiracial patients), and language groups. The intervention resulted in similarly increased rates of FIT and CRCS completion for patients regardless of whether they had been seen for primary care in the prior year and regardless of which of the 5 facilities was their primary site of care.
Using a per protocol analysis focusing only on the individuals due for CRCS at the time of their vaccine, we compared the rate of FIT completion in the 90 days after the vaccine for those who were given a FIT kit at the time of their shot and those who were not, ignoring the original treatment level to which their influenza shot day was assigned. In the intervention arm, influenza vaccination clinic staff gave only 1805 of 3351 patients due for CRCS (53.9%) a FIT kit. In the control arm, 25 of 2884 eligible patients (0.9%) received a FIT kit erroneously. Among those given a FIT kit (n = 1830), 35.4% completed a FIT in the 90 days after influenza vaccination; and among those not given a FIT kit (n = 4405), 13.4% completed a FIT in the 90 days after influenza vaccination, resulting in a percentage point differential of 22.0% (P ≤ .001).
We assessed the extent to which the FLU-FIT Program added benefit to KPNC’s system of mailing FIT kits to eligible patients. To do this, we examined the differences in FIT completion rates in each study arm for participants who were not independently mailed a FIT kit in the 12 months before influenza vaccination and for those who were mailed a FIT kit in 0 to 3 months or 3 to 12 months before influenza vaccination. This provided a nonrandomized test of the advantage of the FLU-FIT Program strategy for expanding the reach of colorectal screening over and above sending the kits by mail. Among intervention patients not mailed a kit in the prior year, 526 (29.6%) completed a FIT in the next 90 days compared with 179 (11.8%) of the 1521 control patients (P ≤ .001). In the group that was mailed a FIT kit in the past 3 months, 264 (31.4%) completed FITs in the next 90 days compared with 68 (22.4%) in the control arm (P = .009). In the group mailed FIT kits between 3 and 12 months before their influenza vaccination, 251 (21.1%) completed FITs in the intervention arm compared with 89 (8.4%) in the control arm (P ≤ .001). Only 67 (2.0%) patients in the intervention arm and 85 (3.0%) patients in the control arm were mailed FIT kits during the 90 days after receiving an influenza vaccination, indicating that kits mailed after the intervention had little influence on study outcomes.
The proportion of eligible intervention patients given FIT kits at the 5 sites ranged from 47.1% to 60.0%. Overall, the demographics, frequency of primary care visits, and proportion of intervention patients who had received FIT kit mailings were similar between those given versus those not given FIT kits with their influenza vaccinations. Among intervention patients given a FIT kit, 641 of 1805 patients (35.5%) completed it within 90 days. Among those not given a FIT kit with their influenza vaccination, 259 of 1546 patients (16.8%) completed a FIT within 90 days. This latter rate was greater than was the 11.8% completion rate in the control arm, suggesting that some patients in the intervention arm may have been given FIT kits without having it recorded or perhaps received encouragement to complete the FIT in some other way.
A total of 28 436 patients aged 50 to 75 years attended the influenza vaccination clinics at the 5 study sites during study dates. Because of extensive outreach to KPNC patients through organized FIT kit mailings and other methods, 78.1% of these influenza vaccination clinic attendees were already up to date with at least 1 CRCS test at the time of their influenza vaccination. A total of 33.1% had a FIT in the past year, 27.2% had a flexible sigmoidoscopy in the past 5 years, and 30.5% had a colonoscopy in the past 10 years. In the intervention arm, rates increased from a baseline of 77.8% to 82.0% at 90-day follow-up (4.2% point increase), and in the control arm, this increase was from 78.4% to 79.0% (0.6% point increase), with a P value for the difference in the percentage point change between these 2 groups of ≤ .001.
We created an unadjusted model for the target population of eligible patients’ completion of the FIT in the 90-day follow-up period, including variables from Table 1, excluding ethnicity (Hispanic vs non-Hispanic) because ethnicity data were unavailable for 1473 patients. Gender and language preference were not significant in the unadjusted model. In Table 3, we have presented an adjusted model with all significant variables. The adjusted odds ratio (AOR) for completion of FIT in the FLU-FIT group was 2.75 (95% confidence interval [CI] = 2.40, 3.16) and was similar to the AOR of 2.77 (95% CI = 2.42, 3.18) obtained in the unadjusted model, indicating little influence of possible confounding variables on the comparison of outcomes between these 2 groups.
TABLE 3— Multivariate Analysis of Predictors for Completing Fecal Immunochemical Test Within 90 Days of Influenza Vaccination: FLU-FIT Program, Kaiser Permanente Northern California, 2009–2010
| Variable | AOR (95% CI) |
| Treatment group | |
| Intervention | 2.75 (2.40, 3.16) |
| Control (Ref) | 1.00 |
| Age group, y | |
| 50–65 | 0.79 (0.69, 0.91) |
| 66–75 (Ref) | 1.00 |
| Race | |
| Asian/Pacific Islander | 1.23 (1.02, 1.47) |
| Black | 0.99 (0.74, 1.33) |
| Multiethnic | 1.04 (0.78, 1.39) |
| White (Ref) | 1.00 |
| Other/unknown | 0.93 (0.79, 1.09) |
| No. of primary care visits in the previous y | |
| 0 (Ref) | 1.00 |
| 1 | 1.36 (1.14, 1.62) |
| 2–3 | 1.25 (1.03, 1.51) |
| 4–10 | 1.28 (1.02, 1.61) |
| > 10 | 1.21 (0.79, 1.85) |
| Facility identification number | |
| 1 | 0.99 (0.80, 1.22) |
| 2 | 1.31 (1.08, 1.59) |
| 3 | 0.84 (0.65, 1.09) |
| 4 | 1.22 (1.00, 1.49) |
| 5 (Ref) | 1.00 |
| FIT kit mailed | |
| 0–3 mo before vaccination | 1.43 (1.18, 1.74) |
| 3–6 mo before vaccination | 0.75 (0.62, 0.89) |
| 6–12 mo before vaccination | 0.65 (0.53, 0.79) |
| Not in y before vaccination (Ref) | 1.00 |
Note. AOR = adjusted odds ratio; CI = confidence interval; FIT = fecal immunochemical test. The sample size was n = 6235.
Other independent predictors of completing the FIT within 90 days of vaccination were being aged 66 to 75 years, being of Asian American race, having at least 1 but no more than 10 primary care visits in the previous year, and having been sent a FIT kit by mail in the prior 3 months. Compared with patients not sent a FIT kit by mail in the past 12 months, the patients who were sent a kit between 3 and 12 months before receiving an influenza vaccination were less likely to complete the FIT within the next 90 days. Finally, the facility where patients received their influenza vaccine also influenced the likelihood of completing a FIT within the next 90 days. The AORs for each of these other independent predictors were all lower than were the AORs for the intervention versus the control arm.
The FLU-FIT intervention succeeded in reaching many patients who were due for CRCS, even in an environment in which baseline screening rates are already well above national norms. Most of the patients reached by the intervention had 1 or 0 primary care visits in the past year and had not been sent a FIT kit by mail in the past year. The FLU-FIT Program may therefore be a particularly effective way to reach patients who are not gaining access to CRCS through other methods. The FLU-FIT Program intervention was also effective at increasing screening activity among the patients reached. We saw this at all 5 sites analyzed and in all major patient subgroups, including those that typically experience disparities in screening.
The FLU-FIT intervention added to the already substantial benefits of KPNC’s FIT kit mailings. Regardless of whether or when FIT kits were previously mailed to patients, the offering of FIT kits with influenza vaccinations led to a clinically and statistically significant increase in CRCS rates. This shows the value of multilevel interventions to reach and follow up with eligible patients in multiple ways, as has been advocated for other aspects of the cancer care continuum.13 If the intervention were further supported with telephone or postcard reminders after FIT kits were dispensed, as done in some of our other studies and as currently done after KPNC’s FIT kit mailings, we expect that the results would be even better.6,7,9,12
Although simple in design, the implementation of the FLU-FIT Program presents some challenges. Our research protocol minimized the role of the research team in daily implementation of the intervention to maximize the external validity of our results.14,15 Under these conditions, none of the sites implemented the FLU-FIT intervention exactly as planned, and only a little more than half of eligible patients were provided a FIT kit, with a range of 47% to 60% across the 5 sites. Interestingly, intervention patients not given a FIT kit were somewhat more likely to complete the FIT than were control patients, suggesting that some patients in the intervention arm may have been given a FIT kit without having it recorded or perhaps received encouragement to complete the FIT in some other way. Nonetheless, we suspect that if all eligible patients were given a FIT kit, the impact of the intervention would increase. A few patients were given FIT kits erroneously, indicating the need for training and quality control in the FIT kit distribution process. We received few reports of patients refusing to accept FIT kits when offered, suggesting that the most important implementation barriers may pertain to characteristics of the implementation team and process rather than patient characteristics.
A potential limitation of this study pertains to generalizability beyond KPNC. However, the KPNC patient population is large and demographically diverse, and the experience of KPNC should apply to most integrated health care systems placing a high institutional priority on both annual influenza vaccination and CRCS. Combined with our research findings in limited resource clinical settings serving patient populations with low baseline CRCS rates,9–11 our new findings strongly support the generalizability of our approach. A second potential limitation is that there were some statistically significant baseline differences between the intervention and control groups, as reported in Table 1. However, the absolute magnitude of these differences was small and not likely to be clinically significant, and we controlled for these differences in our multivariate analyses.
In summary, the FLU-FIT Program holds promise as a method to increase CRCS among eligible individuals who have not been reached by or responded to other forms of CRCS outreach. Areas for future research include the extent to which FLU-FIT Programs can be disseminated and implemented nationally and best practices for applying lessons learned with an appropriate balance of fidelity and adaptation.
Acknowledgments
This study was funded by the American Cancer Society (research scholars grant RSGPB-09-010-01-CPPB) and the University of California, San Francisco Clinical and Translational Science Institute (grant UL1 RR02413 to M. B. P.).
The authors wish to acknowledge Lisa Brill, MPH, MPP, for her assistance with study logistics and Joan Schwalbe, MS, for her assistance extracting study data from Kaiser Permanente databases. Finally, the authors warmly thank the clinical, information technology, and management teams at all the participating Kaiser Permanente sites for collaborating on this research.
Human Participant Protection
The Kaiser Permanente Northern California institutional review board and the University of California, San Francisco Committee on Human Research approved the study with a waiver of informed consent.
