Concerns about administrative data on race/ethnicity have led some researchers to consider self-reported race/ethnicity as superior.1–5 However, few studies have examined the differential impact of the source of race/ethnicity data, that is, observed or selfreported, on study outcomes. We investigated whether differences in reporting of race/ethnicity led to different results with regard to the use of one therapeutic dental procedure, root canal therapy.
From a retrospective secondary data study of Department of Veterans Affairs (VA) dental outpatients who underwent either root canal or tooth extraction between October 1, 1997, and September 30, 1998 (Jones et al., unpublished data), we selected the first treatment of the year for 15 137 patients, on whom we had both self-reported race/ethnicity from the 1999 Large Health Survey of Veteran Enrollees6 and administrative race/ethnicity data from the VA outpatient clinic files. Clinical information on the dental procedure performed as well as the severity of dental disease and the medical comorbidities was obtained from the VA administrative data files (Jones et al., unpublished data).
In the administrative data each patient was assigned a single race/ethnicity from among 6 categories: (1) Hispanic, (2) American Indian, (3) Black, (4) Asian, (5) White, (6) unknown or missing. Survey respondents were asked to describe their race/ethnicity by selecting all that applied from among 6 categories: (1) American Indian or Alaska Native, (2) Asian, (3) Black or African American, (4) Spanish, Hispanic, or Latino, (5) Native Hawaiian or Pacific Islander, (6) White. Those who did not answer were coded as “missing.” We eliminated 35 who self-reported Native Hawaiian or Pacific Islander as their single race, since there was no comparable category in the administrative database. This reduced our sample to 15 102 patients with a single visit during which either a root canal or a tooth extraction was performed.
Using self-reported race/ethnicity as the gold standard, we calculated the proportion of each racial/ethnic category correctly recorded in the administrative database, once for those who self-reported a single race/ethnicity and once allowing for those who chose multiple responses. Using logistic regression, we estimated the probability of obtaining root canal therapy vs tooth extraction for patients of different race/ethnicity, calculating 3 models. The first model used administrative race/ethnicity data, the second self-reported single race/ethnicity, and the third used selfreported race/ethnicity by weighing multiple race/ethnicities by the number of categories.
We determined the amount of agreement between self-reported race/ethnicity and the administrative data, once using only the 82.4% of patients who reported a single race/ethnicity and once including the 4.9% who reported multiple (between 2 and 6) race/ethnicity categories. Following one of the US Census Bureau’s suggestions about the compilation of multiple race/ethnicity responses, we counted patients who reported combinations such as “White and Asian and African American” 3 times: (1) “White alone or in combination,” (2) “Asian alone or in combination,” and (3) “African American alone or in combination.”7 Thus the multiple counting increases the sample size from 15 102 patients to 15 906 race/ethnicity responses.
Table 1 summarizes the amount of agreement calculated both ways. Between 76.4% and 77.1% of self-reported Whites, between 68.4% and 68.9% of self-reported African Americans, between 57.1% and 61% of self-reported Hispanics, between 33.3% and 54% of Asians, and between 1.4% and 4.6% of self-reported American Indians were classified as such in the observer-based administrative data. Self-reported Whites had the fewest “unknown” classifications and the fewest incidences of being classified as something other than White in the administrative database, whereas self-reported Asians had the most administrative classifications of “unknown” and self-reported American Indians the most occurrences of being falsely classified in the administrative data.
Table 2 presents odds ratios for obtaining root canal therapy for different groups, using different sources of race/ethnicity data, both unadjusted and adjusted for dental and medical disease. Regardless of the source of race/ethnicity data, African Americans were significantly less likely and Asians significantly more likely than Whites to obtain root canal therapy. Source of race/ethnicity data significantly changed the adjusted odds ratios for Hispanics and persons of unknown race/ethnicity. Parameter estimates for the model using selfreported data and the model using administrative data showed no significant differences for any known race/ethnicity category. However, when administrative data were used, the effect of unknown race/ethnicity on the odds of obtaining root canal therapy was similar to the effect of White race/ethnicity, whereas when self-reported data were used, the effect of unknown race/ethnicity was statistically different from the effect of White race/ethnicity.
Race/ethnicity in administrative data were more frequently incorrect for patients whose race/ethnicity was other than White, as indicated by other studies.8,9 Source of race/ethnicity data influenced the assessment of the outcome, root canal therapy, in that results for patients with unknown race/ethnicity differed significantly by data source. The level of disagreement and the differences in assessment suggest that estimates of racial/ethnic differences are dependent on the source of race/ethnicity data. Our results suggest that future studies of racial/ethnic variations should be sensitive to the source of race/ethnicity data. Studies that depend on administrative race/ethnicity should note the limitations of this approach.
Note. NA = not applicable. aThese patients were not necessarily falsely classified. This number indicates the number of patients who provided no self-reported race/ethnicity information but had race/ethnicity recorded in the administrative data. Note. Odds ratios are shown both unadjusted and adjusted for dental and medical disease. aTo keep the sample size consistent, we included in the analysis 2 variables that reflected the 4.9% of patients who reported more than one race/ethnicity. bRace/ethnicity is weighted by number of responses, in that “Asian and white” is weighted as 0.5 Asian and 0.5 White. cOdds significantly different from those of the reference group of White patients.
Administrative Data Race/ethnicity category Self-Report Correctly Classified, No. (%) Unknown, No. (%) Falsely Classified, No. (%) White 8863 6832 (77.1) 1898 (21.4) 133 (1.5) White or in combination 9473 7235 (76.4) 2018 (21.3) 220 (2.3) Black or African American 2555 1760 (68.9) 668 (26.1) 127 (5.0) Black or African American or in combination 2721 1862 (68.4) 704 (25.9) 155 (5.7) Spanish, Hispanic, or Latino 766 467 (61.0) 191 (24.9) 108 (14.1) Spanish, Hispanic, or Latino or in combination 924 528 (57.1) 216 (23.4) 180 (19.5) Asian 87 47 (54.0) 28 (32.2) 12 (13.8) Asian or in combination 150 50 (33.3) 43 (28.7) 57 (38.0) American Indian 173 8 (4.6) 43 (24.9) 122 (70.5) American Indian or in combination 724 10 (1.4) 148 (20.4) 566 (78.2) Missing 1914 NA 531 (27.8) 1383 (72.3)a Differences between parameter estimates (P values), t test order Administrative Single self-reported race/ethnicitya Multiple self-reported race/ethnicityb 1 vs 2 1 vs 3 2 vs 3 Black Unadjusted 0.53c (0.46, 0.61) 0.49c (0.43, 0.56) 0.51c (0.45, 0.58) 0.48 0.66 0.74 Adjusted 0.57c (0.49, 0.66) 0.53c (0.46, 0.61) 0.54c (0.47, 0.63) 0.44 0.67 0.75 Hispanic Unadjusted 0.74c (0.59, 0.92) 0.68c (0.56, 0.84) 0.72c (0.59, 0.87) 0.65 0.85 0.80 Adjusted 0.83 (0.64, 1.07) 0.74c (0.58, 0.94) 0.76c (0.61, 0.94) 0.44 0.60 0.87 Asian Unadjusted 2.45c (1.50, 4.02) 1.98c (1.26, 3.10) 1.80c (1.24, 2.61) 0.55 0.38 0.77 Adjusted 1.99c (1.11, 3.57) 1.83c (1.06, 3.15) 1.81c (1.16, 2.81) 0.84 0.81 0.58 American Indian Unadjusted 0.19 (0.03, 1.39) 0.76 (0.51, 1.15) 0.78c (0.63, 0.96) 0.33 0.32 0.93 Adjusted 0.18 (0.02, 1.47) 0.93 (0.59, 1.49) 0.82 (0.65, 1.03) 0.13 0.18 0.50 Unknown Unadjusted 0.96 (0.87, 1.07) 0.59c (0.52, 0.69) 0.60c (0.52, 0.69) 0.00 0.00 0.53 Adjusted 0.95 (0.84, 1.07) 0.60c (0.51, 0.70) 0.60c (0.51, 0.71) 0.00 0.00 0.52 Multiple race/ethnicity including White Unadjusted — 0.79c (0.63, 0.98) — — — — Adjusted — 0.80 (0.62, 1.03) — — — — Multiple non-White race/ethnicity Unadjusted — 0.65 (0.40, 1.06) — — — — Adjusted — 0.62 (0.36, 1.08) — — — —
This research was supported by funding from the Department of Veterans Affairs Health Services Research and Development Service (HSR&D IIR 98.161) and by the Veterans Health Administration, Office of Quality and Performance.
The authors gratefully acknowledge A. Pitman for providing programming support.
Human Participant Protection The study was approved by the institutional review board of the Edith Nourse Rogers Memorial Veterans Hospital, Bedford, Mass.
