To assess the experienced or perceived barriers and facilitators to health research participation for major US racial/ethnic minority populations, we conducted a systematic review of qualitative and quantitative studies from a search on PubMed and Web of Science from January 2000 to December 2011.
With 44 articles included in the review, we found distinct and shared barriers and facilitators. Despite different expressions of mistrust, all groups represented in these studies were willing to participate for altruistic reasons embedded in cultural and community priorities.
Greater comparative understanding of barriers and facilitators to racial/ethnic minorities’ research participation can improve population-specific recruitment and retention strategies and could better inform future large-scale prospective quantitative and in-depth ethnographic studies.
THE IMPORTANCE OF RACIAL and ethnic minority participation in clinical research has been well established including, but not limited to, generalizability of research findings,1,2 equity in provision of health care,3,4 and accuracy of ethnicity-specific subgroup analyses.5,6 Despite a series of national-level initiatives in the past 2 decades from the National Institutes of Health,7 the Federal Drug Administration (FDA),8 and the Centers for Medicare and Medicaid Services,9 racial and ethnic minorities remain underrepresented in clinical research.10,11 Racial/ethnic minorities constitute more than 30% of the US population, but enrollment by race/ethnicity of National Cancer Institute publicly funded cancer clinical trials (phase I–III treatment studies, January 1, 2003, through June 30, 2005) revealed that they represented less than 18% of clinical trial participants.12 Evelyn et al. reported that racial/ethnic minorities constituted only 17% of FDA clinical trial participants in 185 studies of new molecular entities over a 5-year period.13
Several barriers to participation of racial and ethnic minorities in clinical research have been identified for both researchers and participants. For researchers, lack of knowledge about the cultural differences among ethnic minorities can result in ineffective communication strategies about health research at all stages, including recruitment, enrollment, and retention. Because research has historically been conducted by White researchers and has targeted mostly White research participants, the “gold standards” with regard to research processes have tended to include incorrect assumptions about effectiveness when unquestioningly transferred to ethnic minority populations. For example, researchers’ inappropriate use of recruitment strategies among racial/ethnic minority groups that were developed for White participants and lack of knowledge about how to culturally and linguistically adapt recruitment materials have been noted concerns.1,14,15
Given the paucity of evidence-based strategies and practices in the literature regarding non-White populations, recruitment of minorities can require additional investments of time and resources to learn what methods may work in distinct communities to improve community acceptance of clinical research and thus improve participation.14,16 Cultural and linguistic adaptation of recruitment strategies may include not only the selection of appropriate venues, methods, and topics of focus when addressing the gaps in knowledge about research among a particular minority group14,17,18 but also the translation of materials into appropriate languages and the implementation of such strategies by culturally and linguistically competent research staff.19,20
In addition, many researchers fail to facilitate culturally sensitive and meaningful discussions about informed consent to ensure truly informed choices in the enrollment process21,22 For example, although obtaining consent from a research participant is often practiced as a 1-time occurrence, research indicates that consent should be considered an ongoing process—a dialogue—rather than a discrete act of choice that takes place in a singular moment in time, thus supporting participants in making informed decisions throughout the trial.23 Moreover, among many culturally diverse and immigrant populations, it may be important to include families and communities in a dialogue around research participation decisions because individual decisions to participate in research are frequently not independent of family and community involvement, benefits, and costs.2,24,25
Furthermore, once participants have been recruited and enrolled in studies, research success is more likely if culturally informed retention strategies are used to engage such populations in research.2,26,27 Some suggested strategies focus on partnering with community organizations, including investigators and staff from the same targeted communities as participants and retaining the same staff and interviewers over time to ensure continuity.20,28,29 Such efforts can increase likelihood of greater rapport and trust building between study staff and participants and improved adherence to study protocol by participants.
For racial- and ethnic-minority participants, the concerns have ranged from psychosocial issues such as mistrust, fear, and lack of confidence to logistical concerns including childcare, schedule conflicts, lack of transportation, and appropriate support to research-related factors such as lengthy consent documents and lack of adequate information about clinical research.1,2,10,30 Several studies raise questions about both overt and subtle forms of racism and discrimination at multiple levels that may lead to barriers and the underrepresentation of ethnic minority populations in health research.31–34 Barriers to minority participation in health research resulting from such racism can occur at different levels, from institutional to interpersonal to internalized levels of racism.35
The continuing effects of slavery and colonization at a systemic institutionalized level have manifested in ongoing health inequalities through differential access to health care and poor health outcomes for racial/ethnic minorities in the United States.35–37 The US Public Health Services Syphilis Study at Tuskegee (Tuskegee Study) among African Americans and efforts to sterilize American Indians are some egregious examples of a dark history of systematic abuse and mistreatment both in health care and medical research for racial and ethnic minorities in the United States.38–40 As a consequence, mistrust of the medical establishment and of medical research has been well documented among minority groups and continues to be a formidable barrier to research participation among these populations.1,31,32,41–43
Racism at the interpersonal level is often manifested in prejudicial and discriminatory acts resulting in poor health care experiences and health outcomes for racial/ethnic minorities.42,44–47 Poor mental and physical health outcomes, delay or failure in seeking preventive services and treatment, and poor adherence behaviors have all been associated with reported experiences of discrimination among racial/ethnic minorities.46,48 Patients with such experiences are also less likely to participate in health research, contributing to the overall lower numbers of racial/ethnic minorities in clinical research.27,49,50
Finally, at the level of internalized racism, stigmatized populations accept negative messages and stereotypes about their own abilities and worth, resulting in lower psychological health and lower self-esteem, which can then have a negative effect on health practices and outcomes. Internalized racism has been associated with psychological and physiological negative effects, ranging from emotional distress and alcoholism to increased risk of obesity, high blood pressure, and high fasting blood sugar.51–54 More relevant to the current topic of minority participation in research, internalized racism can also adversely affect the provider–patient relationship to potentially impede communication abilities of the patient, leading to discounting of information from the provider, delays, or failure to obtain needed medical care and lower levels of adherence.55
The same stereotypes and negative messages internalized by minorities may also shape providers’ and health care organizations’ interactions with minorities. For example, in the case of African American women, a range of socially constructed stereotypical images of “mammy, strong matriarch, welfare mother, female overachiever, etc.” can influence diagnoses and treatment choices made for African American female patients.56(p32) Given that strong provider–patient relationships have been shown to be key to minority participation in research,57,58 these deleterious effects of internalized racism on the provider–patient relationships may ultimately negatively affect decisions of minorities to participate in health research. Notwithstanding these barriers, some studies have suggested that minorities are not necessarily less willing than Whites to participate in clinical research, especially in cancer research studies.59,60
The literature on racial- and ethnic-minority participation in clinical research has burgeoned over the past decade, gauging by several systematic reviews on the topic.1,10,15,30 However, most of this scholarship has focused on African Americans,10,41 in light of a prevailing mistrust of clinical research in the Black community stemming from the historical abuses associated with the Tuskegee Study and related concerns of ethical misconduct.61–63 There are fewer studies that have included a range of racial and ethnic populations, resulting in less information about the barriers and facilitators to participation in clinical research for a variety of groups.10,41,64 A recent systematic review of the literature on recruitment interventions showed that African Americans were the most targeted group (82% of the studies) and Latinos were also likely to be targeted (46%), but specific information on other minority groups was not included.15 Given the growing racial and ethnic diversity in the US population and as Latinos are the largest minority group and Asian and Pacific Islander populations are growing at a rate greater than any other group, such multiethnic analyses will become increasingly important.65
The existing literature reflects a trend of including 1 or 2 minority groups, but few studies have compared across several groups. An exception is a study by Brown and Moyer who used a nationally representative sample to compare predictors of awareness of clinical trials and feelings about the use of medical information for research across African American, Asian American, White, and Latino populations.66 The authors found that, relative to the White population, all other racial/ethnic minority participants were less aware of clinical trials and less positive about the use of medical information for research. Although this study identifies who is likely to participate in research, it does not identify specific barriers and facilitators for these different groups.
In a similar way, a collaborative research initiative entitled Project MICRO (Minority Involvement in Clinical Research Opportunities) is a multi-institutional (University of Hawaii, Charles Drew University, Morehouse College, Meharry Medical College, University of Puerto Rico), multicultural (West Coast and Southern African American, Mexican, Puerto Rican, Filipino, Chinese, Pacific Islander, Somali, White), multilingual (English, Spanish, Chinese, Samoan, Tagalog, Ilocano, Hawaiian, Somali), and multigeographic National Institutes of Health (NIH)–funded study that sought to identify predictors of research participation by gaining a better understanding of the factors that impede or enhance such participation among diverse racial/ethnic groups.41 This study was a first effort to address the need for comparative research about attitudes, beliefs, barriers, and facilitators to minority research participation across racial/ethnic groups in multiple geographic regions.41,67 The qualitative findings from Project MICRO showed both distinct barriers and facilitators based on historical and cultural factors specific to each of the groups and shared barriers and facilitators based on socioeconomic and environmental factors that were shared among the 4 distinct racial/ethnic groups (African Americans, Latinos, Native Hawaiians, and Filipinos). There were several distinct barriers, but the only barrier that was shared by all the groups was lack of information about clinical research.67
With the growing rates of racial/ethnic multicultural populations come growing rates of health disparities and disease burden among them and, consequently, the increasing importance of their participation in clinical research. Without assuming that all such minority groups have the same barriers and facilitators, it is important to identify context-specific culturally shared and distinct factors that deter or enhance their participation in clinical research. When factors that are relevant across multiple racial/ethnic groups are identified, interventions that address common issues can then be developed on a broad platform and adapted to meet the particular specificities of targeted racial/ethnic groups. When such recruitment materials resonate with racial/ethnic minority communities, they are more likely to participate effectively in clinical research. Furthermore, interventions can be developed more efficiently and in a cost-effective manner by leveraging recruitment efforts across multiple groups and their shared barriers and facilitators.
The ability of medical science advances to reach all Americans is predicated upon the participation of diverse study participants in an array of clinical trials and is echoed in the NIH’s call for translational research over the past decade. Successful translational research requires not only innovative strategies for the recruitment and retention of diverse populations into research but also increased investments into community education and the dissemination of results. However, there is little understanding of what are the key barriers and facilitators to address for which populations or what the driving issues around recruitment and retention and general community education are as they relate to clinical research.
We present a systematic review of the existing literature of both qualitative and quantitative studies that include multicultural racial/ethnic participant voices to identify the range of themes and papers and that take a comparative perspective in their assessment of barriers and facilitators to participation in health research. To the best of our knowledge, this is a first effort to do so. We have broadly defined “health research” as health-related research involving human participants in clinical trials, clinical research, and behavioral health interventions to be as inclusive as possible.
The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement consists of a 27-item checklist and a 4-phase flow diagram to help authors improve the complete reporting and transparency of systematic reviews and meta-analyses.68 As necessary, some of the items on the PRISMA checklist may be modified.69 We conducted a systematic review in conformance with modified items of the PRISMA Statement to answer the following question: among major US racial/ethnic minority populations, what are the experienced or perceived barriers and facilitators to participating in health research?
We chose the PubMed and Web of Science databases for their complementary content to conduct the literature search. PubMed provides biomedical and health content related to the life sciences, behavioral sciences, chemical sciences, and biomedical engineering, whereas Web of Science provides multidisciplinary content on the sciences, social sciences, arts, and humanities. PubMed and Web of Science are proprietary databases for peer-reviewed journal content that provide the ability to conduct transparent, controlled, and powerful searches.70,71 The decision to not conduct a search on Google Scholar limits the content of this review to peer-reviewed articles and inherently excludes gray literature and subsequently the results of recent studies presented at conference proceedings or institutional publications. Furthermore, although Google Scholar may have allowed for greater inclusion of online and open-access journals in our search process,72 we excluded this search engine because replicability of searches, an extremely important factor for systematic reviews, cannot be ensured because of the “constantly-changing content, algorithms and database structures” of Google Scholar.73(p214)
We selected titles, abstracts, and articles on the basis of the following eligibility criteria. We reviewed only English-language articles published in the United States between January 2000 and December 2011. We generally adhered to the norm of a maximum time frame of 5 to 10 years in selecting the time frame for this review to ensure the most current and relevant articles.74 The target population of interest in the article had to include at least 1 of the following adult racial/ethnic minority populations: African American, Latino, Asian American, or Asian–Pacific Islander. We limited the search to the United States to account for the health care context that is unique to US residents, such as its particular history of racism with regard to health care provision and research, most infamously exemplified in the Tuskegee Study, and the lack of universal health care in the United States unlike most other developed nations. The literature search related to the barriers (and facilitators) to health research participation perceived by adult African American, Latino, Asian American, and Asian and Pacific Islander populations. We did not include American Indians and Alaska Natives in this review because Indian Health Services presents a unique context for health care and research that is not available to other racial/ethnic minorities.
Key terms used for the literature search included racial/ethnic minority (e.g., African American, Latino, Pacific Islander, Asian American, or their derivative); health research study, clinical trial, and clinical research (or its derivative); and participation, access, recruitment, barriers, and facilitators (to health research). Examples of search strings included “research participation barriers among Latinos” and “African American AND research AND participation” as described in the search plan in Appendix 1 (available as a supplement to this article at http://www.ajph.org). In addition, the study’s methodology had to be clear and at least 1 of the study’s aims had to use primary data to evaluate or assess the barriers or facilitators to participation in health research. We limited quantitative studies to those that surveyed self-reported factors that contributed to barriers and facilitators, and individuals’ attitudes, beliefs, and values related to health research participation. We excluded studies with target populations younger than 18 years. We also excluded articles that did not provide the barriers or facilitators to health research participation from the perspective of the racial/ethnic–minority participant. We focused on articles that provided a voice to racial/ethnic–minority populations and, therefore, we excluded studies that reported the perspectives of research staff, physicians, or institutions.
We initially retrieved a total of 164 articles from the systematic literature search by using the search terms in Appendix 1 (available as a supplement to this article at http://www.ajph.org); 21 articles were duplicates (Figure 1). Duplicate articles occurred because articles appeared in both the PubMed and Web of Sciences databases, or when the articles with more than 1 target racial/ethnic minority population would appear in searches for each individual racial/ethnic minority population. We screened the remaining 143 titles for their relevance to the research question of this review and we excluded 27 literature reviews, commentaries, and editorials. We reviewed the remaining unique full-text articles (n = 116) from the combined searches on PubMed and Web of Science and we excluded 72 for not meeting the inclusion criteria; thus, we included 44 articles in the review.
We abstracted data in a standardized format and organized the data into tables that included the author’s names, title of the article, year of publication, target population, age of target population, methodology including whose perspective was being reported, and key findings related to the barriers and facilitators to participation in health research. We coded the studies for racial/ethnic minority population, methodology (i.e., qualitative, quantitative, or mixed methods), distinct barriers and facilitators to participation, and shared barriers and facilitators to participation, and grouping of themes across shared barriers and facilitators. Two authors (S. G. and N. D.) coded the findings of the barriers and facilitators in an iterative process. The consistency between the coders was more than 90% and they resolved conflicts by using an iterative process and discussion to reach consensus.
The articles presented in this review provide the perspectives of racial/ethnic minorities across age and socioeconomic status regarding participation in health research. Of the 44 articles included, 23 used qualitative methods such as focus groups and individual interviews, 16 articles used quantitative methods that statistically analyzed data collected through questionnaires, and the remaining 5 articles used mixed methods (Table 1). Sixteen studies were exclusively conducted among African Americans, 6 among Asian Americans, 3 among Latinos, and 1 among Pacific Islanders; the remaining 18 studies included a combination of at least 2 racial/ethnic minority groups including White populations that were often used as the reference group. Asian Americans were primarily represented by peoples of East Asia and Southeast Asia, and Pacific Islanders were represented by Native Hawaiians, Samoans, and Tahitians. Latinos were overwhelming represented by Mexican Americans and immigrants from Mexico; also represented were peoples of Puerto Rico and Central America. A number of studies did not differentiate the country of origin of Latinos and Hispanics.
Distribution of the Reviewed Articles by Method and Target Population Regarding Barriers and Facilitators to Health Research Participation for US Racial/Ethnic Minority Populations From a Search on PubMed and Web of Science From January 2000 to December 2011
|Article Type||Total, No. (%)||African American, No. (%)||Asian American, No. (%)||Latino, No. (%)||Pacific Islander, No. (%)||≥ 2 Racial/Ethnic Groups, No. (%)|
|Qualitative||23 (52.3)||10 (22.7)||3 (6.8)||2 (4.5)||1 (2.3)||7 (15.9)|
|Quantitative||16 (36.4)||3 (6.8)||3 (6.8)||0||0||10 (22.7)|
|Mixed methods||5 (11.4)||3 (6.8)||0||1 (2.3)||0||1 (2.3)|
Note. Total number of articles in the review was 44.
Many of the studies focused on women (14 of 44) or had an overrepresentation of women (66.5%; n = 5353), whereas only 1 study focused exclusively on men (Appendix 2, available as a supplement to this article at http://www.ajph.org). A few studies (n = 11) utilized bilingual staff or translated materials for non–English-speaking participants who immigrated from Latin America or Asia. Nonspecific health research studies (n = 14) dominated the literature, followed by cancer (n = 11) and HIV/AIDS (n = 11), and other health topics (n = 8; Table 2). The nonspecific health research was often referred to as medical research or hypothetical research. Other disease-specific studies included in this review focused on Alzheimer’s disease, kidney disease, and hypertension, along with pregnancy and genetic-related conditions. The results of the coded findings are summarized in the next section.
Distribution of the Reviewed Articles by Health Condition Regarding Barriers and Facilitators to Health Research Participation for US Racial/Ethnic Minority Populations From a Search on PubMed and Web of Science From January 2000 to December 2011
|Article Type||Nonspecific Health Condition, No. (%)||Cancer, No. (%)||HIV/AIDS, No. (%)||Other, No. (%)|
|Qualitative||7 (15.9)||8 (18.2)||5 (11.4)||3 (6.8)|
|Quantitative||5 (11.4)||3 (6.8)||5 (11.4)||3 (6.8)|
|Mixed methods||2 (4.5)||0||1 (2.3)||2 (4.5)|
|Total||14 (31.8)||11 (25.0)||11 (25.0)||8 (18.2)|
Note. Total number of articles in the review was 44.
The terms shared and distinct represent the relatively commonly and uncommonly reported barriers and facilitators reported in the articles on the 4 racial/ethnic communities included in this review. The concordance of messages across studies suggests that certain themes may be shared or distinct as a greater or lesser priority for a particular community but they are not necessarily distinct in the sense that those barriers or facilitators are exclusive to a particular community. It is important to note that the shared and distinct barriers and facilitators presented were drawn from a comparison of the limited numbers and types of studies included in this review and not from a comparison of the underlying populations. As described in Appendix 2 (available as a supplement to this article at http://www.ajph.org), the studies included in this review vary considerably in the extent to which they represent various populations. As a consequence, on the basis of the available number and types of studies of a specific population, we can only present examples of meanings of a barrier or facilitator for a given population expressed in the set of studies included in this review but cannot say very much about the distribution or importance of this barrier or facilitator in the defined population. Thus, observations are limited to the articles for this review and are not representative general statements about the populations under study and may allow us to make only explorative generalizations.
Both quantitative and qualitative research findings included in this review provided observations related to the shared barriers and facilitators, whereas the distinct barriers and facilitators were exclusively observed in the qualitative research findings. Furthermore, the examples of shared and distinct barriers and facilitators presented in the tables are not exclusive to a group but rather illustrative of the observations made from the studies reviewed.
The shared barriers to health research participation that were reported across all 4 racial/ethnic groups included mistrust and lack of access to information (Table 3). Lack of access to information as reported in the studies reviewed among Asian Americans, Latinos, and Pacific Islanders pointed to language barriers as a key to perceived lack of access to information among these 3 groups. Other barriers included competing demands that conflicted with ability to participate in research, fear of unintended outcomes, stigma, and issues related to health insurance coverage and legal status, which were all reported in 2 or more groups.
Examples of Shared Barriers to Health Research Participation as Observed in the Studies Reviewed From a Search on PubMed and Web of Science From January 2000 to December 2011
|Racial/Ethnic Group Observed Examples|
|Barriers||Articles, No. (%)||African American||Asian American||Latino||Pacific Islander|
|Mistrust||34 (77.3)||Perceive research will benefit Whites or the research institution and not people of color76||Concerns related to signing the informed consent77||Believe medical experimentation occurs when accessing health care67||Negative feelings about the purpose and intent of research41|
|Competing demands||20 (45.4)||Inconvenience62; cost of participation34||Lack of time78 and financial resources79||Time conflicts and lack of childcare67|
|Unintended outcomes||14 (31.8)||Concerns about future long- and short-term side effects80||Uncertainty of risks, side effects, and effectiveness of clinical trials81||Fear of vaccine-induced HIV infection82|
|Lack of access to information||14 (31.8)||Misconceptions about research83||Limited knowledge about clinical trials84; lack of translated materials, including key words or terms; feeling intimidated by English78||Unavailable health information in Spanish and lack of access to Spanish-speaking staff67; low perceived risk of diseasea,85,86||Information about accessing research87; language barrier between hospital staff, researchers, and patients40|
|Stigma||12 (27.3)||Related to genetic or mental illness research88||Related to judgment from husband or family for participation in health study79||Related to HIV-positive status89|
|Health insurance coverage||3 (6.8)||Have basic health care or no specific health care needs90; fear of discrimination88||Lack of information about insurance coverage for clinical trials77|
|Legal status in United States||2 (4.5)||Concerned immigration status will be affected among immigrants79||Fear of deportation among immigrants67|
aLow perceived risk of disease is a shared barrier for African Americans and Latinos.
Mistrust was a reported barrier across all 4 racial/ethnic minority groups and appeared in 77.3% (n = 34) of all articles included in this review (Table 3). Among the studies with African Americans, mistrust was frequently associated with the perception that research will benefit Whites or the research institution and not people of color.62,76,90–96 In a similar way, Native Hawaiians have reported a mistrust related to the researcher’s agenda not serving the community.97 Mistrust related to the fear of purposeful mistreatment and experimentation was often characterized as being treated like a “lab rat” or “guinea pig.”32,67,81,83,84,97–99 Mistrust with signing the informed consent was related to the perception that individuals are relinquishing their rights93 and providing the researcher with legal protection against any harm that may be inflicted onto the participants.91
Time98 and financial constraints related to the competing demands of working multiple jobs and needing to work, being the primary caretaker of children and or relatives, being the single head of household,90,93 and justifying the cost of participation with the perceived high risk for a disease.85
Unintended outcomes, such as the uncertainty of short- and long-term side effects or the uncertain effectiveness of the intervention under study may provoke participants to consider the benefits and risks to participation. The possible interference with current treatments92 or the lack of access to health care should injury100 or a disease diagnosis90 arise were examples of unintended outcomes that weigh into the decision of participating in health research. More specific to the study of HIV vaccines was the fear of vaccine-induced HIV infection.82
A lack of informational access about research opportunities was a barrier that was reported across all 4 racial/ethnic minority groups and represented in 31.8% (n = 14) of the articles. The absence of bilingual research staff and informational material were reported to be barriers among non– or limited–English-speaking racial/ethnic populations that have demonstrated an interest in participating in health research.41,67,78 Although research participation is viewed as unnecessary among those who perceive their risk for disease as low, researchers warn that individuals may be misperceiving their actual risk85 or delaying access to health care until there is an urgent need.90 The misperceived risk may also result from a lack of information regarding their health and risk of disease.
Stigma is understood to occur when labeling, stereotyping, separation, status loss, and discrimination co-occur in the context of an unequal power situation that permits stigma to occur.101 Stigma was often related to the health condition of interest in the research study such as HIV infection or mental illness.82,88,89 In HIV-related research, there exists the fear of the social repercussions of disclosing HIV status.82 Stigma was also coupled with privacy and confidentiality concerns related to the participant’s medical condition, personal health history, and genetics.34,77,79,93,102 The lack of acceptance and support from family members that may manifest itself in judgment for participating in a study also contributed to stigma.79,81,94,103
African Americans reported a fear of discrimination from health insurance companies that may result from participating in health research that discloses their genetic health status,88 and shared with Latinos concern about health insurance coverage for participation in clinical trials.77 Among individuals who had no specific health care needs, there was a perception that participation in research is unnecessary.90
The most commonly articulated facilitators to health research participation that were reported in the studies reviewed across all 4 racial/ethnic groups included culturally congruent study designs and a range of benefits to participation such as receiving adequate remuneration and access to health care resources. Altruism expressed as helping their family or community was the third most commonly reported facilitator across the groups. Convenience of participation and low risk in participation were shared facilitators reported in 3 of the 4 groups (Table 4).
Examples of Shared Facilitators to Health Research Participation as Observed in the Studies Reviewed From a Search on PubMed and Web of Science From January 2000 to December 2011
|Racial/Ethnic Group Observed Examples|
|Facilitators||Articles, No. (%)||African American||Asian American||Latino||Pacific Islander|
|Cultural congruence||27 (61.4)||Community education at fraternities and sororities104; use of culturally diverse staff95||Culturally matched research personnel and information in appropriate language41||Research staff that speak Spanish and can relate to participants103||Community involvement prerequisite to research for research staff or use of interpreters and translated material41|
|Benefits to participation||27 (61.4)||Money or free medical services104; access to new, better, or free medicines100; learn about their own health91; receive adequate information about the purpose of study105||No out-of-pocket costs84; no other effective treatment available84; more information about clinical trials81||Monetary compensation82; access to medical services89; sufficient or appropriate study information provided82||Personal need and all medical bills paid41; improved medical facilities for the community41; complete disclosure of risks and safeguards41|
|Altruism—helping family or community||24 (54.5)||Contribution to future generations and community91,106; increase scientific knowledge104; personal or family history of the disease80,106||Want to help family member or Asian American community in general78; care about the purpose of the research107||Help others89; advance medical knowledge103; burden of disease on family or community67||Clear contribution to future generations and community41|
|Convenience of participation||8 (18.2)||Having workplace support90||Transportation compensated107||Childcare provided67|
|Low risk in participation||5 (11.4)||Natural treatment or noninvasive95||Take medications with known efficacy84||Low risk of toxicitya,85|
aPreference to participate in research with a low risk of toxicity is shared among African Americans and Latinos.
Having research staff representative of the research participants’ racial/ethnic group was an important facilitator. Racial/ethnic participants preferred research staff that they can relate to and communicate with in their own language and rhythm of expression67,96; among African Americans this act would give research the “personal touch” needed to encourage participation.93 African Americans and immigrant Filipino women were more likely to participate when invited by a recruiter or researcher that was personally known to them.79,94 Although familiarity and comfort with the clinical setting,108 recruiter, or researcher is helpful in enabling participation in health research, Brugge et al. caution that attention to the risk of exploitation is necessary, as in their study of elderly Chinese immigrants.109 The availability of language-appropriate materials and research staff was reported to be important to facilitating research participation among Asian Americans, Latinos, and Pacific Islanders.77,84,89,97,103
Mild monetary incentive,106,108 free lunch,107 or free health examination85 may positively influence participation in health research. Research that allows people to help themselves, such as those wanting to lose weight41 and access health care, facilitates participation as well.67,89,91 In addition, receiving information about individual health and greater details about the study or clinical trial, such as risks and safeguards, were also observed as benefits to participation and potential facilatators.41,81,82,91
Altruism, articulated as helping family members and the community in the present and future, was an observed facilitator in the studies reviewed. Among studies that included African Americans and Latinos, advancing medical knowledge was a form of altruism and motivation for participation.
Addressing the logistics that make participating in health research convenient includes employer support to take time off to attend appointments,90 childcare,67 and transportation provisions.107 Efforts that make participation convenient may help address the aforementioned barriers related to competing demands.
Lastly, a preference for studies that are perceived to have the least risk of discomfort or invasiveness, such as completing a survey or an education intervention,34 may also enable participation.110 Examples of studies with the least risk that were articulated in the articles reviewed include interventions with natural treatment or taking medications with known efficacy and low risk of toxicity.
Distinct barriers represent those barriers that are not necessarily unique to each of the groups but rather more commonly voiced and emphasized in the articles for this review. This understanding is also applied to the distinct facilitators. Again, these “distinct” barriers and facilitators are meant to be examples of the those factors that were uncommonly reported in other groups relative to the group where the barrier or facilitator was commonly reported and not meant to suggest that these barriers or facilitators were exclusively reported in only 1 group.
Five themes appeared as distinct barriers among African Americans (Table 5). The legacy of the Tuskegee Study and lack of research integrity were each represented in 13.6% (n = 6) of the articles, followed by the legacy of racism and discrimination. Additional themes contributing to distinct barriers in participation included mistrust of the health care system and concerns related to the research process. African Americans’ negative personal experiences with the health care system, such as racism, differential treatment, and feelings of mistrust, may create distinct barriers to research participation.67,93,106
Examples of Distinct Barriers to Health Research Participation as Observed in the Studies Reviewed From a Search on PubMed and Web of Science From January 2000 to December 2011
|Racial/Ethnic Group, Barrier||Articles, No. (%)||Observed Examples|
|African American: legacy of mistrust|
|Legacy of the Tuskegee Study||6 (13.6)||Belief in the perceived conspiracy in the United States to harm Black people34; knowledge of the Tuskegee Study has negative impact on willingness111|
|Lack of research integrity||6 (13.6)||Concerns related to data use34; improper treatment of research participants32,88; questionable ethical practices related to cloning, reputation of researcher or institution,74,88 consent forms,91 and disclosure100|
|Legacy of racism and discrimination||3 (6.8)||Perception that researchers do not value Blacks62|
|Mistrust of health care system||2 (4.5)||Direct or indirect experience of the disregard for cultural norms among health care or research staff74|
|Concerns with the research process||2 (4.5)||Lack of choice over treatment in clinical trials95|
|Asian American: social context|
|Lack of social support||2 (4.5)||Lack of family’s support of the decision to participate in health research84,107|
|Acculturation||1 (2.3)||Acculturation in elderly Chinese reduced the likelihood of participation109|
|Pacific Islander: misrepresentation of community||1 (2.3)||Concerns related to the use of data to benefit the researchers and not the community, and the extrapolation of data that may overgeneralize Native Hawaiians41|
Among Asian American groups, 2 distinct themes emerged in the limited literature available—namely, lack of social support that would encourage participation in health research84 and acculturation among elderly Chinese.109
Native Hawaiians reported their concern over misrepresentation of their community particularly in Hawaii where other Pacific Islanders and racial/ethnic groups also reside, and these latter groups may be inadvertently included in the findings for Native Hawaiians.97
The lack of distinct barriers and facilitators for Latinos can be attributed to the limited articles exclusively focused on Latinos and the greater reporting of their shared experiences with other immigrants. Also, less detailed information on Latinos may have resulted from their inclusion in 61% (11 of 18) of the articles with multiple racial/ethnic groups that may have limited the attention to the nuances distinct to Latinos.
Four themes observed in the studies reviewed that may facilitate African Americans’ participation in health research are having safety assurances, trust in the researcher and reputation of the research institution, having treatment options, and the inclusion of diverse racial and ethnic groups (Table 6). In the studies reviewed, African American participants wanted not only reputable researchers, but also the assurance that researchers will put African Americans’ best interests ahead of the study’s objectives.91 Although having treatment options is contrary to the validity of randomized clinical trials, leveraging the importance of choice in participation may enable African Americans’ participation in research. Finally, wanting to see White people also enrolled in a study along with African Americans was motivated by the perception that African Americans are not valued by researchers and the belief that if White people are enrolled, researchers are less likely to deliberately harm the participants.34
Examples of Distinct Facilitators to Health Research Participation as Observed in the Studies Reviewed From a Search on PubMed and Web of Science From January 2000 to December 2011
|Racial/Ethnic Group, Facilitator||Articles, No. (%)||Observed Examples|
|African American: design and logistics|
|Having safety assurances||3 (6.8)||Guarantee compensation if participant is disabled or killed as a result of participating in the study76; communicating study process112 and results with participants88; having control in decision to participate or decline100|
|Trust in researcher or reputation||2 (4.5)||Trust researchers to put the participants’ best interest first91; trust in researcher and reputation76|
|Having treatment options||1 (2.3)||Having choice of treatment in randomized clinical trials92|
|Inclusion of diverse racial/ethnic groups||1 (2.3)||Seeing White people also recruited in the study34|
|Asian American: endorsement from family||3 (6.8)||Family involved in decision-making process among Chinese and Vietnamese81; if asked by son or daughter among elderly Chinese109 or spouse among Chinese and Vietnamese women78|
|Pacific Islander: Community mediation||2 (4.5)||Community has input over how research findings are used and reported; findings are reported back to the community41; study information is culturally tailored and directed to families87|
In the studies reviewed, among Asian Americans, having family members, such as a spouse or sibling, support the decision to participate in health research or being asked by a son or daughter to engage in health research have been reported to facilitate participation.78,81,109
Likewise, in the studies reviewed, among Pacific Islanders, particularly Native Hawaiians, the inclusion of family members was most relevant to elders who rely on youths to interpret information and thus increases access to information about research opportunities.87 The community’s involvement in being informed of the results and subsequently aiding in the interpretation of the results to a larger audience were key mediation strategies to increase trust and facilitate participation among Native Hawaiians in Hawaii.41
Our review of the literature between January 2000 and December 2011 revealed several key experienced or perceived barriers and facilitators to participating in health research across studies that focused on 1 or more of the major US racial/ethnic minority populations, namely African Americans, Latinos, Asian Americans, and Pacific Islanders. In our review, we found trends in the similarities and differences in perceptions and experiences across the 4 groups, presented in tables. In this section, we consider both the limitations of our findings and the implications of the shared and distinct (commonly and uncommonly reported) barriers and facilitators for the recruitment and retention of these 4 racial/ethnic groups in the studies reviewed for health research and for future large-scale prospective quantitative studies and in-depth qualitative ethnographic studies.
Key shared (commonly reported across groups) barriers included (1) mistrust and consequent fear of participation, (2) stigma related to research participation, and (3) competing demands. The distinct (uncommonly reported across groups) barriers for these groups were sometimes variations on the shared barriers, but expressed with greater intensity or with reference to a specific context by the particular racial/ethnic group.
Mistrust was the most common barrier appearing in 73% of the studies across the qualitative and quantitative research articles we reviewed. The expressions of mistrust regarding health research varied across all 4 groups, partly on the basis of context and experiences. Some groups expressed their mistrust of researchers and fear of participation in research in terms of community and communal experiences. For example, both African Americans and Native Hawaiians consistently emphasized the importance of community and shared a mistrust of research related to the belief that research may not benefit their communities. Such mistrust seemed to be rooted in historical communal experiences including slavery and colonialism and their legacy of racism and discrimination and from specific experiences such as the Tuskegee Study for African Americans, a finding that is consistent with previous reviews on barriers to participation in clinical research.1,113 Although 1 article reported the potential for similar Latino mistrust associated with negative associations with research connected to oral contraceptives that had been conducted in the 1960s among Hispanics,114 we did not find any other references to mistrust connected to this experience among Latino research participants in contrast with African Americans, among whom there were repeated references to the infamous Tuskegee Study. It was interesting that belief in the AIDS Origin Conspiracy Theory may contribute to mistrust but was not found to decrease participation in biomedical research.115
Perceptions of mistrust regarding 3 specific aspects of research participation included (1) fear of purposeful mistreatment, (2) fear of unknown research procedures, and (3) fear of unintended consequences. One example of an unfamiliar research process that led to some fear and mistrust was the informed consent process, about which participants expressed such fears as “Am I signing away my free will?” In addition to fear and mistrust related to unintended consequences, participants feared finding out that they had a disease that they did not know about. Other groups also talked about unintended consequences but did not express similar fears concerning participation. A distinct barrier reported among Native Hawaiians, which may have stemmed from mistrust connected to their experiences of colonization, was expressed specifically as a fear of their community being misrepresented in the outcomes of research.
A second key shared barrier across all groups was competing demands, which included both time and financial challenges associated with participation in research. The participants in many of these studies consistently appeared to be concerned about issues such as maintaining current jobs or needing to find work or holding multiple jobs to make ends meet. They also expressed concerns about being responsible for the care of children and other relatives, particularly when they were single parents. For such participants, a long-term commitment to a research study could be a very difficult barrier to overcome if appropriate retention strategies were not implemented.
Although stigma, another key shared barrier to participation in research studies, may have been related to the mistrust and fear of participation, for most of the participants of the studies we reviewed, stigma seemed to be associated more with the clinical condition or topic being researched rather than with participation in research itself. For example, participation in both mental illness– and genetic research–related studies was cause for stigma among African American participants in the studies reviewed. HIV-related studies were cause for stigma among Latino study participants. By contrast, in the 1 Asian American study where stigma emerged as a barrier, the cause of stigma was judgment from family members for participating in research. Lack of social support, particularly from family members, was a distinct barrier to research participation for Asian Americans, underlining the likely importance of family involvement and likely reflecting stigma in this group.
Despite the presence of these barriers, Katz et al. reported no difference in self-reported willingness to participate in biomedical research among African Americans, Latinos, and Whites.99 This finding suggests that, although barriers to participation can be significant across these populations, there are also facilitators that result in a willingness to participate among these groups, especially for select medical conditions. Key shared facilitators included culturally congruent research processes, benefits of participation, and altruism toward and involvement of family or community.
A key facilitator for these study participants seemed to be the use of community-based, linguistically appropriate, and personalized recruitment and retention practices in the research process. Participants preferred having the recruitment processes based in community settings, with culturally matched research personnel running the studies and study materials in the appropriate languages. Furthermore, they preferred being recruited in a more personal way, “face-to-face” such as by their physicians or others they knew. In fact, there were several interrelated facilitators related to community and culture, which all underscored the interest expressed by multicultural populations to learn about research and their willingness to participate when research processes were contextualized in community priorities and using culturally congruent practices. For example, a theme that resonated very strongly in the reviewed literature was the role played by altruism—construed as helping family and community—in facilitating participation among these populations. As mentioned earlier, in the studies reviewed, African Americans and Pacific Islanders, and specifically Native Hawaiians, were each articulate about their mistrust of research related to the belief that such efforts would not benefit their communities; however, at the same time, participants from both communities expressed a committed desire to help their future generations or community through research.
Participants from all groups expressed a desire for community contextualization and cultural congruence, but there were some distinct variations voiced among the groups. For example, Pacific Islanders, specifically Native Hawaiians, additionally emphasized community mediation as a facilitator to research participation, where they desired trusted community liaisons to take a more active role and for both facilitators and barriers to research to be mediated at the community level as opposed to the individual level. Therefore, whereas African Americans wanted follow-up and study results reported to individual participants, Native Hawaiians wanted results reported to families and communities. Studies on Asian Americans from multiple subgroups reported that endorsement of research from a trusted and known individual, such as a family member, was a distinct facilitator for research participation for members of this group.
Adequate remuneration for research participation was another key shared facilitator that was raised in almost half the studies reviewed. Study participants wanted to not have any “out of pocket” expenses and wanted all the medical care associated with the research to not be an additional expense for them. In addition to monetary remuneration, some study participants, across racial/ethnic groups, expressed preference for payment in direct health care, such as health screenings and other clinical services, which maybe otherwise unavailable to them. This may be particularly true for the poor, the underserved, and recent immigrants who may not have access to regular sources of health care.
A major limitation to our review was the lack of equivalent amounts of data across the 4 racial/ethnic groups. Our findings are consistent with previous scholarship in this area that relative to the studies available on the research participation of African Americans and Whites, there is little comparable research on Latinos, Asian Americans, and Pacific Islanders.10,40,116 With regard to this review, the availability of relatively more information on African Americans allowed for a better understanding and a more nuanced grasp of the barriers and facilitators to research participation present for this group. However, the absence of a similar range of scholarship for the other 3 groups has limited our ability to identify and compare barriers and facilitators with the same level of specificity across all 4 groups.
Furthermore, we found that there were other social categorical distinctions across the groups that were rarely represented in the literature. For example, the majority of the research articles that we reviewed included mostly female participants and few male participants. We found only 1 study that focused on African American men. In addition, distinctions such as immigrant versus nonimmigrant, English-speaking versus non–English-speaking, documented versus undocumented, and generational differences were seldom reflected in the literature. The exclusion of research participants younger than 18 years limits the ability of our findings to be generalized to younger persons where a unique interaction of child or adolescent and parent will likely introduce new considerations for participation that may also be disease-specific. Because of the limited number of articles in the review, our analysis did not distinguish between the internal diversity among Latino, Asian American, and Pacific Islander subgroups, which could limit generalizations to select ethnic communities.
The decision to exclude searching on Google Scholar limited the content of this review to peer-reviewed articles and inherently excluded gray literature such as conference proceedings or institutional publications.
Finally, this article presents a systematic review of the existing qualitative and quantitative studies that focus on participants’ voices to identify the range of themes and papers but cannot provide a truly quantitative comparative perspective. As already discussed in the Results section, although we have made comparative statements about the studies we have reviewed, we cannot make any definitive statements about the distribution or importance of barriers or facilitators in the defined populations represented in the studies. However, we believe it is the rigorous compilation of participant voices and expressions that lays the foundation to move future studies forward in a guided manner.
Given our findings, enhancing research participation might be viewed through 2 interdependent lenses: (1) addressing barriers that hinge to a large degree on a history of exclusion and vulnerability, which can be leveraged to appeal to most people’s desire to access trials as treatment of themselves or their community, and (2) addressing facilitators through the implementation of community-based participatory research (CBPR) strategies to ensure that individuals and communities are more fully engaged in health research projects from conceptualization to design and implementation.117–119
Because social and structural factors such as mistrust, stigma, and lack of adequate health knowledge impede participation, it is clear that social or structural interventions such as sensitivity training for community and academic researchers and research staff will be among important strategies to enhance participation of a diverse constituency. The overwhelming presence of mistrust of research in all 4 racial/ethnic groups is such a shared barrier that might be mitigated by increasing the health and health research literacy120 of these groups in a cost-effective manner to be leveraged across multiple racial/ethnic groups, given their similar range of mistrust issues. However, it will also be necessary to tailor such efforts to address the distinct barriers and leverage the distinct facilitators related to mistrust for particular groups. For example, educating African American and Native Hawaiian populations may include acknowledging the historical context of their mistrust and providing a venue where causes of mistrust can be discussed openly. Furthermore, providing reassurances such as (1) opportunities to learn and ask questions about the research process, (2) verifiable assurances of human participant protection measures to address fears of being experimented on, and (3) explicitly addressing concerns about unintended consequences may be helpful for all multicultural groups with little previous exposure to research.
Such efforts may be more acceptable to many of these study populations if they were inculcated in a CBPR approach using a culturally congruent manner, in community settings, with culturally matched research personnel, linguistically appropriate materials, and a personalized recruitment process with endorsement from trusted community figures.118,120–122 A more robust approach to the consent process should be developed to better address participant concerns.123 Whereas many CBPR practices involve community members from the research question development to design and interpretation of results, few include participants or community in the conceptualization, development, and implementation of the consent form.124 Having representative study participants or community members involved in the consent process should be considered as a new standard for clinical trials. Furthermore, with Native Hawaiian populations, additional efforts could be made to understand and address concerns about community misrepresentation resulting from research participation and to leverage the role of community liaisons as mediators of research participation.
An exception to community-based recruitment may be in the case of clinical studies involving conditions that are considered stigmatizing. For studies related to these conditions, recruitment of multicultural populations may become a challenge if they are based in the community because participants may be concerned about becoming stigmatized if their participation becomes common knowledge. But effective community engagement can overcome even these concerns as demonstrated in depression research.125 We found stigma to be a consistent concern across different cultural groups, but the genesis and impact of stigma differed across groups. Especially when conducting studies involving conditions that are considered stigmatizing in particular communities, researchers may need to use stigma-reduction strategies and interventions, which range from the intrapersonal to governmental levels, with the most effective strategies being multitargeted and multilevel, using counseling, education, and personal contact, particularly targeting individual and community levels.126
Finally, when a study is recruiting multicultural populations, there may be a need to pay special attention to competing demands, particularly with regard to time and money, for the poorer and underserved segments of these populations. Providing adequate remuneration, particularly in the form of clinical services, may be a facilitator that would be attractive to potential research participants from these groups. To the extent that such clinical services are otherwise unavailable to participating members of these populations, researchers must be vigilant to reduce the possibility that participants may agree to onerous research requirements to obtain the necessary clinical services. There are several other such social justice–related issues that arise in the context of CBPR given the potentially inherent differences in power, expectations, and priorities between research and historically disadvantaged community partners. These can result in ethical challenges all along the research spectrum from research processes such as informed consent and participant selection to addressing the risk–benefit ratio of doing research to the ownership, sharing, and decision-making around findings.123,127
The findings from this review point to the urgent need to conduct more research focusing on some groups such as Latinos, Asian Americans, and Pacific Islanders about whom there is a dearth of information regarding their experiences and perceptions of health research. Likewise, some groups, such as Latinos, and Asian Americans, Pacific Islanders, have a great deal of internal diversity among them, which is not represented in the sparse literature that is currently available. Again, inclusion of a greater representation of subgroups within larger groups is an important factor for future studies to consider when they are recruiting multicultural populations for health research. Such differences may be important in obtaining a more accurate reflection of barriers and facilitators to research participation, particularly among racial/ethnic groups with immigrant and nonimmigrant contingents with different language abilities and documentation statuses. It may be helpful to conduct a similar review exploring the literature on the health research experiences and perspectives of American Indians within the context of the Native American Research Centers for Health supported by the Indian Health Services and NIH to help tribes and tribal organizations expand research infrastructure in culturally congruent ways. Future studies that explore the factors influencing research participation also need to consider whether having a greater gender balance among study participants may be important to study outcomes and result in the identification of a distinct set of barriers and facilitators to research participation for men versus women among these groups.
Data on the perspectives of patients with diverse health conditions and their preferences with regard to type of clinical research are limited. Given that the majority of studies are focused on cancer and HIV, there are limited data available on other health conditions that allow for the examination of the nuances that influence the participants’ decision and willingness to participate in varying types of clinical research, especially chronic diseases not typically associated with a high risk of immediate mortality. Understanding the preferences for observational studies compared with intervention studies, and within interventions studies the preference for noninvasive compared with invasive interventions, may facilitate the better design of effective recruitment and retention methods. Data on US minority participation in clinical trials suggests an overrepresentation of minorities in phase I clinical trials and a continued underrepresentation in phase III clinical trials.128 Thus, although participants’ perspectives are integral in designing recruitment and retention strategies, additional studies are needed to fully understand the breadth of factors that facilitate and impede minority participation in all phases of clinical trials.
Furthermore, it is imperative to conduct further study on contexts of and approaches to recruitment and retention of minority populations. For example, when participation in clinical trials is the only avenue to receiving clinical care, there are likely to be higher rates of recruitment from across diverse populations, particularly in phase 1 trials.129,130 This is particularly true for certain diseases, such as cancer, which have relatively higher rates of minority participation in clinical trials. Such differences in the context of participation raise questions about how recruitment and retention under dire circumstances compares with other recruitment contexts, where participants may have multiple treatment and care options. Furthermore, approaches to engaging diverse populations in research vary considerably in terms of the level of community participation. In a recent systematic review examining the effectiveness of CBPR to enhance clinical trials in racial/ethnic minority groups, the authors found that trials using CBPR had “very high success rates in recruiting and retaining minority participants and achieving significant intervention effects.”131(p1363) With the expansion of CBPR approaches, community–research partnerships have an opportunity to overcome many of the key barriers and leverage key facilitators to increase racial/ethnic minority participation in research. Future studies need to address a range of approaches to community engagement, including projects identified and driven by communities and the varying impact on the recruitment and retention of multicultural populations in research.
Finally, researchers may be able to use the findings of this study to inform and motivate future large-scale prospective quantitative studies to make truly quantitative comparisons of barriers and facilitators across groups. We also need additional in-depth, ethnographic research in these communities to assess more accurately the shared and relatively distinct nature of barriers and facilitators, because quantitative surveys, qualitative focus group, and individual interview studies are limited by what participants are able or willing to share and articulate. In many instances, when barriers and facilitators are based in cultural practices, they occur at a subconscious level and may not be articulated by participants in responses to direct questions. Ethnographic studies, which involve observation of practices in community settings, are essential to identifying such not so easily articulated cultural variations in barriers and facilitators among these groups.
Our review of the literature points to the need to learn more about and refine our understanding of barriers and facilitators to research participation among African American, Latino, Asian American, and Pacific Islander groups. Mistrust and competing demands, as well as stigma and consequent fear of participation, were notable challenges to research participation among the racial and ethnic minorities in the studies reviewed. Furthermore, our review also identified several facilitators that resonated across groups including the importance of contextualizing recruitment and retention strategies among these populations within specific community priorities and using culturally congruent practices, where research is seen as altruism toward family and community, and adequate benefits for participation. Community-based participatory research approaches including involvement in the consent process may hold particular promise. Leveraging and integrating information about such shared barriers and facilitators into the development of recruitment and retention materials and practices are likely to result in more effective and ethical strategies to increase numbers of multicultural participants in health research and meet the translational research challenge of moving scientific discovery to practice for all Americans.
This work was supported by the National Center for Research Resources, National Institutes of Health (NIH; grant U54RR022762), with co-funding from the National Institute for Minority Health Disparities, NIH (grant U54MD007598, formerly U54RR026138), the National Center for Advancing Translational Sciences, NIH (grant UL1TR000124), and by the National Cancer Institute, NIH (grant 3U54CA153499-04S1).
The authors also wish to thank the reviewers and responsible editor for their insightful comments and suggestions, which have greatly improved this paper.
Note. The findings and conclusions in this report are those of the authors and do not necessarily represent the views of the NIH.
Human Participant Protection
Institutional review board approval was not obtained because human participants were not involved and only published data were reported.