December 2001, Vol 91, No. 12 | American Journal of Public Health 1926-1929
© 2001 American Public Health Association
COMMUNITY-BASED PARTICIPATORY RESEARCH |
Can Public Health Researchers and Agencies Reconcile the Push From Funding Bodies and the Pull From Communities?
Lawrence W. Green, DrPH and
Shawna L. Mercer, PhD
At the time of writing, Lawrence W. Green and Shawna L. Mercer were with the Office on Smoking and Health, National Center for Chronic Disease Prevention and Health Promotion, Centers for Disease Control and Prevention, Atlanta, Ga. Shawna L. Mercer is also a postdoctoral fellow of the Canadian Institutes for Health Research and the Canadian Health Services Research Foundation.
Correspondence: Requests for reprints should be sent to Lawrence W. Green, DrPH, Office of Extramural Prevention Research, Public Health Practice Program Office, CDC, 4770 Buford Hwy NE, Mail Stop K-56, Atlanta, GA 30341-3724 (e-mail: lgreen{at}cdc.gov).
 |
ABSTRACT
|
|---|
Responding to growing impatience with the limited application of research findings to health practices and policies, both funding bodies and communities are demanding that research show greater sensitivity to communities' perceptions, needs, and unique circumstances. One way to assure this is to employ participatory researchto engage communities at least in formulating research questions and interpreting and applying research findings and possibly also in selecting methods and analyzing data.
"Community" should be interpreted broadly as all who will be affected by the research results, including lay residents of a local area, practitioners, service agencies, and policymakers. Participatory research should not be required of every project, but when results are to be used for, in, and by communities, those communities should collaborate not only in applying findings but also in determining the ways in which the findings are produced and interpreted.
 |
INTRODUCTION
|
|---|
FOR MANY YEARS, MUCH HAS been made of the need to make better use of research, apply research results more assiduously, disseminate research findings more effectively, and synthesize research into evidence-based guidelines and "best practices" for more immediate application by practitioners. Recently, it has become increasingly clear that there is a need to increase the public's science and health literacy, so that people will consume research more intelligently and school-aged children will be able to integrate scientific methods and facts more fully into their educational experience.1,2
Many of these appeals have a patronizing tone to them, which suggests that the researcher is in the best position to decide what practitioners and the public need and then give it to them. The appeal for increased science literacy at least acknowledges a role for the public in the interpretation of science, but even here the assumption is that science will be delivered and consumed. Yet the field of public health yields ample evidence that disseminating the results of studies and telling people how they should incorporate this information into their lives produces minimal behavior change. In contrast, participatory researchresearch that is generated collaboratively in a partnership between scientists and othershas reemerged in recent decades as an alternative to top-down technical assistance from experts to practitioners or community residents that may help to ensure that research results address real needs and will actually be used.3
 |
THE CONVERGENCE OF RESEARCH AND PARTICIPATION
|
|---|
Participatory research has a rich and honored tradition in community development, especially in developing countries, where it has enabled social change and community development projects among populations who were suspicious of the motives of Western researchers.4 Much of the renaissance of participatory research since the 1960s has taken place in social, educational, and health services development and delivery, some in public health. In health services, nursing has led the way, with collaborative studies between academic nurses and nursing administrators and staff aimed at improving nursing roles and difficult working conditions that have resulted from changes in health care systems.5
In public health, the revival of participatory research (also referred to as participatory action research) has been most notable in minority health.68 Native American and Canadian First Nations communities, for example, after decades of serving as subjects for anthropologic and epidemiologic studies, behavioral surveys, and health education program evaluations, have put the brakes on external researchers' exploiting their circumstances while providing very little benefit to their communities. Similarly, African Americans living in inner cities have noticed that their lives have been described publicly by researchers in unflatteringif sympatheticways, but they have seen little come of it besides embarrassment and shame.
The recognition by these and other communities that they needed new information about their circumstances that only original research could render converged with a growing recognition by academic and other public health researchers that they could no longer get the data they needed without more active cooperation of the communities. This convergence led to a re-striking of the power balance between the observers and the observed. Research subjects became more than research objects. They gave more than informed consent; they gave their knowledge and experience to the formulation of research questions and methods to be applied in their communities. They became more than the victims described in studies of their health problems and living conditions; they became active partners in identifying key problems and in using the research findings to advocate policies and programs and in program development, monitoring, and evaluation.
 |
WHOSE PARTICIPATION, IN WHAT RESEARCH?
|
|---|
The origins of participatory research sketched above might give the impression that it is a research method designed solely for researcherpublic interaction. On the contrary, it is not a research method, nor is it limited to the direct relationship between academic researchers and the public. Participatory research is an approach that entails involving all potential users of the research and other stakeholders in the formulation as well as the application of the research. A wide range of research methodsepidemiologic, experimental, survey, focus-group, qualitative interviewcan be applied in the service of participatory research. The choice depends on the methods called for by the research questions and the feasibility of the methods in the particular circumstances.
The issue of whose participation needs to be solicited and incorporated in participatory research hinges on who is to be most directly affected by the research results. Because much of the discussion about participatory research in this issue of the Journal and elsewhere is in the context of community development and community programs, the assumption tends to be that participatory research necessarily engages the lay community. Typically, "community" is understood as a local geopolitical entity, as in the term "community-based participatory research." If, however, the notion of community includes other groupings of people sharing common characteristics or interests, or if the purpose under consideration is something other than community development, there emerge both the need and the opportunity for undertaking participatory research with groups other than community residents.
We therefore urge a broader application of participatory research, one in which participatory research is seen as systematic inquiry, with the collaboration of those affected by the issue being studied, for the purposes of education and taking action or effecting social change.9 With this more generic definition, "participants" can be public health practitioners and agencies as well as their constituents and clients or community residents.
What are the upper and lower bounds of participation for determining whether research is participatory? Maximum participation occurs when stakeholders remain active partners throughout the studyin the formulation of research questions, selection of methods, and analysis, interpretation, and application of findings. Minimally, stakeholders should be involved at least at the front end of the study, in formulating research questions, and at the back end, in interpreting and applying the findings. This demarcation of the range of participatory research clarifies the distinction between participatory research and basic research, which typically involves only researchers, and action research, which necessarily includes those involved in the action situation (usually practitioners) as subjects of, rather than as participants, in the research. The types of research and various stakeholders are shown in Figure 1 .

View larger version (21K):
[in this window]
[in a new window]
|
FIGURE 1— Levels of participation and collaboration of different groups in participatory research (PR).
|
|
How does one determine the extent to which participatory research should be incorporated into a given research project? One rule of thumb lies in determining the complexity of the research methods and analyses. Typically, there is no need (and no justification) to drag volunteer participants through a highly technical and labor-intensive research process as long as they have the opportunity to help shape the research questions and interpret the findings. Figure 2 illustrates the varying levels of participation by different actors and stakeholders in different types of research and community development.
 |
CURRENT USES OF PARTICIPATORY RESEARCH AND THE VALUE OF EXTENDING ITS PURVIEW
|
|---|
The happy confluence of mutualinterests and action that can be achieved through participatory research remains the exception rather than the rule. Indeed, Congress funded the Prevention Research Centers (PRCs) program of the Centers for Disease Control and Prevention (CDC) to survey, focus-group, develop innovative ways to conduct research and demonstration activities that would result in improved public health practice. Yet a 1996 Institute of Medicine a 1996 Institute of Medicinepanel reviewing the PRCs concluded that the partnership between universities, public health agencies, and communities was falling short of legislative intent.10
David McQueen, Patricia Riley, Marshall Kreuter, and Lynda Doll, former national Directors and supervisors of the PRCs program, have led the effort to ensure genuine communityinput into all 26 PRCs as well as at the national level.11,12 In this issue of the Journal, Dr. Kenneth Olden, the Director of the National Institute of Environmental Health Sciences (NIEHS) and a strong advocate of participatory research, describes how the NIEHS is responding to this demand.13 Thus, certain funding bodies are providing the push on academic researchers to engage their subjects more actively; communities are providing the pull.
How can participatory research contribute to better use of science, more assiduous application of research results, more extensive dissemination of research findings, and synthesis of evidence-based guidelines and "best practices" from previous research for more immediate application by practitioners? First, we encourage the expansion of participatory research, both in the number of studies undertaken and in application of a broader definition of participatory research to involve other stakeholdersincluding practitioners, other service providers, public health departments, and policymakersin addition to lay community members.
Second, it strikes us that what is most confining about "best practices," and what makes them subject to suspicion from local public health departments, practitioners, and other potential users, is their origins in distant places and under special circumstances. The research that is synthesized into best practices comes from carefully controlled trials conducted in other placesthat is, not locallyunder the direction and with the resources of "foreign" scientific groups.
Local practitioners and policymakers have every reason to suspect that their circumstances are different from those represented in the studies that went into formulating best practices.14 Participatory research offers them an opportunity to examine their own circumstances, to pilot-test the best practices within their own context, and to adapt these practices to their own needs. Such local adaptations, in turn, provide valuable feedback to the research community as it seeks to expand the arsenal of evidence-based guidelines and best practices into other areas of public health and as it seeks to explore, account for, and overcome disappointingly low levels of behavior change among lay individuals, practitioners, organizations, and populations.
Public health agencies can provide a bridge between university-based researchers and community-based projects, using participatory research at the agency level to adapt best practices and at the community level to ensure relevance of the research to the community's needs and actions. Engaging communities and practitioners will not invariably improve the internal validity of research, but almost certainly will improve its external validitythat is, its applicability and usability in the settings in which the research occurs.
 |
Footnotes
|
|---|
Note. The views expressed in this commentary do not necessarily represent the views or policy of the CDC.
Both authors contributed to the conception, writing, and editing of this paper.
Peer Reviewed
Accepted for publication September 18, 2001.
 |
References
|
|---|
1. Nutbeam D. Health literacy as a public health goal: a challenge for contemporary health education and communication strategies in the 21st century. Health Prom Int. 2000;15;259267.2. Ratzan SC. Health literacy: communication for the public good. Health Prom Int. 2001;16;209214. 3. Park P, Brydon-Miller M, Hall B, Jackson T, eds. Voices of Change: Participatory Research in the United States and Canada. Toronto, Ontario: OISE Press; 1993. 4. Kassam Y, Mustapha K. Participatory Research: An Emerging Alternative Methodology in Social Science Research. New Delhi, India: Participatory Research Network; 1982. 5. Ramudu L, Bellet B, Higgs J, Latimer C, Smith R. How effectively do we use double staff time? Aust J Adv Nurs. 1994;11(3):510.[Medline] 6. Frankish CJ, George A, Daniel M, Doyle-Waters M, Walker M. Participatory Health Promotion Research in Canada: A Community Guidebook. Ottawa, Ontario: Minister of Public Works and Government Services Canada; 1997. Catalogue no. H39-418/1-1997E. 7. Langton PA, ed. The Challenge of Participatory Research: Preventing Alcohol-Related Problems in Ethnic Communities. Washington, DC: US Dept of Health and Human Services; 1995. Publication SMA 95-3042. 8. Macaulay AC, Delormier T, McComber AM, et al. Participatory research with native community of Kahnawake creates innovative Code of Research Ethics. Can J Public Health. 1998;89:105108.[Medline] 9. Green LW, George MA, Daniel M, et al. Study of Participatory Research in Health Promotion. Ottawa, Ontario: Royal Society of Canada; 1995. 10. Stoto MA, Green LW, Bailey LA, eds. Linking Research and Public Health Practice: A Review of CDC's Program of Centers for Research and Demonstration of Health Promotion and Disease Prevention. Washington, DC: National Academy Press; 1997. 11. Caburnay CA, Kreuter MW, Donlin MJ. Disseminating effective health promotion programs from prevention research to community organizations. J Public Health Manage Prac. 2001;7;8189.[Medline] 12. Doll L, Berkelman R, Rosenfield A, Baker E. Extramural prevention research at the Centers for Disease Control and Prevention. Public Health Rep. 2001;116 (suppl 1):1019.[Medline] 13. Olden K, Guthrie J, Newton S. A bold new direction for environmental health research. Am J Public Health. 2001;91:19641967.[Abstract/Free Full Text] 14. Green LW. From research to "best practices" in other settings and populations. Am J Health Behav. 2001;25:165178.[Medline]
This article has been cited by other articles:

|
 |

|
 |
 
T. Blair and M. Minkler
Participatory Action Research With Older Adults: Key Principles in Practice
Gerontologist,
October 1, 2009;
49(5):
651 - 662.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
R. L. Williams, B. M. Shelley, A. L. Sussman, and on behalf of RIOS Net clinicians
The Marriage of Community-based Participatory Research and Practice-based Research Networks: Can It Work? -A Research Involving Outpatient Settings Network (RIOS Net) Study
J Am Board Fam Med,
July 1, 2009;
22(4):
428 - 435.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
C. R. Horowitz, M. Robinson, and S. Seifer
Community-Based Participatory Research From the Margin to the Mainstream: Are Researchers Prepared?
Circulation,
May 19, 2009;
119(19):
2633 - 2642.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
N. Abell and S. E. Rutledge
Awareness, Acceptance and Action: Developing Mindful Collaborations in International HIV/AIDS Research and Service
Br. J. Soc. Work,
May 2, 2009;
(2009)
bcp047v1.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
S. T. Leatherdale, S. Manske, S. L. Wong, and R. Cameron
Integrating Research, Policy, and Practice in School-Based Physical Activity Prevention Programming: The School Health Action, Planning, and Evaluation System (SHAPES) Physical Activity Module
Health Promot Pract,
April 1, 2009;
10(2):
254 - 261.
[Abstract]
[PDF]
|
 |
|

|
 |

|
 |
 
R. L. Smith-Ray, F. A. Almeida, J. Bajaj, S. Foland, M. Gilson, S. Heikkinen, H. Seagle, and P. A. Estabrooks
Translating Efficacious Behavioral Principles for Diabetes Prevention Into Practice
Health Promot Pract,
January 1, 2009;
10(1):
58 - 66.
[Abstract]
[PDF]
|
 |
|

|
 |

|
 |
 
K. J. Williams, P. Gail Bray, C. K. Shapiro-Mendoza, I. Reisz, and J. Peranteau
Modeling the Principles of Community-Based Participatory Research in a Community Health Assessment Conducted by a Health Foundation
Health Promot Pract,
January 1, 2009;
10(1):
67 - 75.
[Abstract]
[PDF]
|
 |
|

|
 |

|
 |
 
R. M. Goodman and R. K. Noonan
Empowerment Evaluation for Violence Prevention Public Health Programs
Health Promot Pract,
January 1, 2009;
10(1_suppl):
11S - 18S.
[Abstract]
[PDF]
|
 |
|

|
 |

|
 |
 
M. Cargo, T. Delormier, L. Levesque, K. Horn-Miller, A. McComber, and A. C. Macaulay
Can the democratic ideal of participatory research be achieved? An inside look at an academic-indigenous community partnership
Health Educ. Res.,
October 1, 2008;
23(5):
904 - 914.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
W K Cook
Integrating research and action: a systematic review of community-based participatory research to address health disparities in environmental and occupational health in the USA
J Epidemiol Community Health,
August 1, 2008;
62(8):
668 - 676.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
M. J. Dutta and I. Basnyat
The Radio Communication Project in Nepal: A Culture-Centered Approach to Participation
Health Educ Behav,
August 1, 2008;
35(4):
442 - 454.
[Abstract]
[PDF]
|
 |
|

|
 |

|
 |
 
S. Christopher, V. Watts, A. K. H. G. McCormick, and S. Young
Building and Maintaining Trust in a Community-Based Participatory Research Partnership
Am J Public Health,
August 1, 2008;
98(8):
1398 - 1406.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
S. F Morin, S. Morfit, A. Maiorana, A. Aramrattana, P. Goicochea, J. M. Mutsambi, J. L. Robbins, and T A. Richards
Building community partnerships: case studies of Community Advisory Boards at research sites in Peru, Zimbabwe, and Thailand
Clinical Trials,
April 1, 2008;
5(2):
147 - 156.
[Abstract]
[PDF]
|
 |
|

|
 |

|
 |
 
S. Flicker, B. Savan, M. McGrath, B. Kolenda, and M. Mildenberger
'If you could change one thing...' What community-based researchers wish they could have done differently
Community Dev. J.,
April 1, 2008;
43(2):
239 - 253.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
K. Horn, L. McCracken, G. Dino, and M. Brayboy
Applying Community-Based Participatory Research Principles to the Development of a Smoking-Cessation Program for American Indian Teens: "Telling Our Story"
Health Educ Behav,
February 1, 2008;
35(1):
44 - 69.
[Abstract]
[PDF]
|
 |
|

|
 |

|
 |
 
S. Flicker
Who Benefits From Community-Based Participatory Research? A Case Study of the Positive Youth Project
Health Educ Behav,
February 1, 2008;
35(1):
70 - 86.
[Abstract]
[PDF]
|
 |
|

|
 |

|
 |
 
C. J. Osborn, D. L. Thombs, and R. S. Olds
Reconceptualizing Research on Undergraduate Alcohol Use: The Need for Student Engagement
Eval Health Prof,
June 1, 2007;
30(2):
118 - 137.
[Abstract]
[PDF]
|
 |
|

|
 |

|
 |
 
R. E. Malone, V. B. Yerger, C. McGruder, and E. Froelicher
"It's Like Tuskegee in Reverse": A Case Study of Ethical Tensions in Institutional Review Board Review of Community-Based Participatory Research
Am J Public Health,
November 1, 2006;
96(11):
1914 - 1919.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
B. A. Carlson, D. Neal, G. Magwood, C. Jenkins, M. G. King, and C. L. Hossler
A Community-Based Participatory Health Information Needs Assessment to Help Eliminate Diabetes Information Disparities
Health Promot Pract,
July 1, 2006;
7(3_suppl):
213S - 222S.
[Abstract]
[PDF]
|
 |
|

|
 |

|
 |
 
K. K. Edgren, E. A. Parker, B. A. Israel, T. C. Lewis, M. A. Salinas, T. G. Robins, and Y. R. Hill
Community Involvement in the Conduct of a Health Education Intervention and Research Project: Community Action Against Asthma
Health Promot Pract,
July 1, 2005;
6(3):
263 - 269.
[Abstract]
[PDF]
|
 |
|

|
 |

|
 |
 
A. A. Gandelman, S. A. Vogan, and M. M. Dolcini
Assessing HIV Prevention Provider Knowledge of Behavior Science Theory: Building on Existing Intuitive Experience
Health Promot Pract,
July 1, 2005;
6(3):
299 - 307.
[Abstract]
[PDF]
|
 |
|

|
 |

|
 |
 
C. Woodsong and Q. A. Karim
A Model Designed to Enhance Informed Consent: Experiences From the HIV Prevention Trials Network
Am J Public Health,
March 1, 2005;
95(3):
412 - 419.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
M. Minkler
Ethical Challenges for the "Outside" Researcher in Community-Based Participatory Research
Health Educ Behav,
December 1, 2004;
31(6):
684 - 697.
[Abstract]
[PDF]
|
 |
|

|
 |

|
 |
 
L. W. Green
Ethics and Community-Based Participatory Research: Commentary on Minkler
Health Educ Behav,
December 1, 2004;
31(6):
698 - 701.
[PDF]
|
 |
|

|
 |

|
 |
 
L. H. Zayas, M. D. McKee, and K. R. B. Jankowski
Adapting Psychosocial Intervention Research to Urban Primary Care Environments: A Case Example
Ann. Fam. Med,
September 1, 2004;
2(5):
504 - 508.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
S. L. Mercer, G. MacDonald, and L. W. Green
Participatory Research and Evaluation: From Best Practices for All States to Achievable Practices within Each State in the Context of the Master Settlement Agreement
Health Promot Pract,
July 1, 2004;
5(3_suppl):
167S - 178S.
[Abstract]
[PDF]
|
 |
|

|
 |

|
 |
 
K. Wells, J. Miranda, M. L. Bruce, M. Alegria, and N. Wallerstein
Bridging Community Intervention and Mental Health Services Research
Am J Psychiatry,
June 1, 2004;
161(6):
955 - 963.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
M. W Leung, I. H Yen, and M. Minkler
Community based participatory research: a promising approach for increasing epidemiology's relevance in the 21st century
Int. J. Epidemiol.,
June 1, 2004;
33(3):
499 - 506.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
B. A. Laraia, J. Dodds, and E. Eng
A Framework for Assessing the Effectiveness of Antihunger Advocacy Organizations
Health Educ Behav,
December 1, 2003;
30(6):
756 - 770.
[Abstract]
[PDF]
|
 |
|

|
 |

|
 |
 
S. C. Saegert, S. Klitzman, N. Freudenberg, J. Cooperman-Mroczek, and S. Nassar
Healthy Housing: A Structured Review of Published Evaluations of US Interventions to Improve Health by Modifying Housing in the United States, 1990-2001
Am J Public Health,
September 1, 2003;
93(9):
1471 - 1477.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
M. Minkler, A. G. Blackwell, M. Thompson, and H. Tamir
Community-Based Participatory Research: Implications for Public Health Funding
Am J Public Health,
August 1, 2003;
93(8):
1210 - 1213.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
A. C. Gielen and D. Sleet
Application of Behavior-Change Theories and Methods to Injury Prevention
Epidemiol. Rev.,
August 1, 2003;
25(1):
65 - 76.
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
J. D. Crist and S. Escandon-Dominguez
Identifying and Recruiting Mexican American Partners and Sustaining Community Partnerships
J Transcult Nurs,
July 1, 2003;
14(3):
266 - 271.
[Abstract]
[PDF]
|
 |
|

|
 |

|
 |
 
M. M. Metzler, D. L. Higgins, C. G. Beeker, N. Freudenberg, P. M. Lantz, K. D. Senturia, A. A. Eisinger, E. A. Viruell-Fuentes, B. Gheisar, A.-G. Palermo, et al.
Addressing Urban Health in Detroit, New York City, and Seattle Through Community-Based Participatory Research Partnerships
Am J Public Health,
May 1, 2003;
93(5):
803 - 811.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
S. L Mercer, L. W Green, A. C Rosenthal, C. G Husten, L. K. Khan, and W. H Dietz
Possible lessons from the tobacco experience for obesity control
Am. J. Clinical Nutrition,
April 1, 2003;
77(4):
1073S - 1082.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
N. S. Sung, W. F. Crowley Jr, M. Genel, P. Salber, L. Sandy, L. M. Sherwood, S. B. Johnson, V. Catanese, H. Tilson, K. Getz, et al.
Central Challenges Facing the National Clinical Research Enterprise
JAMA,
March 12, 2003;
289(10):
1278 - 1287.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
E. Parker, L. H. Margolis, E. Eng, and C. Henriquez-Roldan
Assessing the Capacity of Health Departments to Engage in Community-Based Participatory Public Health
Am J Public Health,
March 1, 2003;
93(3):
472 - 476.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
J. J. Schensul
Democratizing Science Through Social Science Research Partnerships
Bulletin of Science Technology Society,
June 1, 2002;
22(3):
190 - 202.
[Abstract]
[PDF]
|
 |
|
Copyright © 2001 by the American Public Health Association
|