© 2004 American Public Health Association
Karin K. Coyle and Douglas B. Kirby are with ETR Associates, Scotts Valley, Calif. Barbara V. Marín, Cynthia A. Gómez, and Steven E. Gregorich are with The Center for AIDS Prevention Studies, University of California, San Francisco. Correspondence: Requests for reprints should be sent to Karin K. Coyle, PhD, ETR Associates, 4 Carbonero Way, Scotts Valley, CA 950662400 (e-mail: karinc{at}etr.org).
Objectives. This study evaluated the long-term effectiveness of Draw the Line/Respect the Line, a theoretically based curriculum designed to reduce sexual risk behaviors among middle school adolescents. Methods. The randomized controlled trial involved 19 schools in northern California. A cohort of 2829 sixth graders was tracked for 36 months. Results. The intervention delayed sexual initiation among boys, but not girls. Boys in the intervention condition also exhibited significantly greater knowledge than control students, perceived fewer peer norms supporting sexual intercourse, had more positive attitudes toward not having sex, had stronger sexual limits, and were less likely to be in situations that could lead to sexual behaviors. Psychosocial effects for girls were limited. Conclusions. The program was effective for boys, but not for girls.
Sexually transmitted diseases (STDs) (including HIV) and pregnancy remain serious problems for youths in the United States. According to the Office of National AIDS Policy,1 young people in the United States between the ages of 13 and 24 years are estimated to be contracting HIV at a rate of 2 per hour, and half of all new HIV infections occur among persons younger than 25 years. Further, roughly 1 in 4 sexually experienced adolescents aged 13 to 19 years acquires an STD each year.1,2 Finally, even with recent declines, the United States still has the highest teen pregnancy and birth rates among comparable industrialized nations.3,4 Unfortunately, many consequences of unprotected sexual intercourse are more common among some subgroups of youths (e.g., African American and Latino) than others. For example, African American and Latino youths are disproportionately affected by HIV and AIDS,1 and Latino adolescents aged 15 to 19 years currently have the highest birth rate of all racial/ethnic groups.5 Several prevention programs have been shown to reduce adolescent sexual risk behaviors.6 Many of these studies have been conducted in high school settings611 or with high school youths in nonschool settings.12,13 It is important to target younger youths with prevention messages before they begin having risky sexual intercourse so that programs can help individuals delay sexual intercourse or avoid unprotected sexual intercourse.14 Fortunately, most young people attend school,15 which provides an efficient mechanism to reach them with effective prevention programs. To date, however, there are no published randomized trials of school-based HIV, other STD, and pregnancy prevention programs in middle schools with significant positive effects on behavior.1619 Studies with quasi-experimental designs have found some positive effects.20,21 Additionally, randomized trials of nonschool-based programs have found effects.22,23 Given the significant consequences of unprotected sexual intercourse for adolescents, and the need for more program options at the middle school level, we developed Draw the Line/Respect the Line, an innovative theory-based program for middle school youths. Draw the Line/Respect the Line is a 3-year, school-based HIV, other STD, and pregnancy prevention program for youths in sixth, seventh, and eighth grades. The primary aim of Draw the Line/Respect the Line is to reduce the number of students who initiate or have sexual intercourse and to increase condom use among those students who do have sexual intercourse. In addition, the program was designed to affect mediating variables (e.g., attitudes, perceived norms) drawn from our theoretical models. This article presents the results of a randomized controlled trial undertaken to assess the impact of Draw the Line/Respect the Line. It also examines the relation between psychosocial variables and selected behavioral outcomes. The reported data are from a cohort of predominantly Latino sixth-grade students who were followed for 36 months.
Study Design The Draw the Line/Respect the Line intervention was implemented from spring 1997 to spring 1999. The study featured a randomized controlled trial involving 19 ethnically diverse public middle schools (grades 6 to 8) drawn from 3 small- to midsized school districts (6 to 8 schools per district) in an urban area of northern California. The 3 districts were selected because they served diverse populations (ethnically and in terms of socioeconomic status): at least 30% or more of the students in the schools within these districts were Hispanic. The districts also were in close proximity to the investigators. All schools within each of the 3 districts agreed to take part. A fourth district (representing 5 schools) was approached for study participation but declined because of concerns over surveying young students regarding their sexual behaviors. Ten schools were randomly assigned to receive the Draw the Line program; the remaining schools continued with usual classroom activities regarding HIV, other STD, and pregnancy prevention, and these activities were dictated by the schools. We used a restricted randomization process involving multiple steps. Schools within each district were initially partitioned into matched sets. Then 2 matched groups were formed, each consisting of 1 school set from each district. Finally, these 2 matched groups were randomized to the intervention or control condition. To facilitate the matching process, we used a principal component analysis of 9 school-level variables (e.g., ethnic composition, standardized test scores, socioeconomic status); this produced 4 principal component scores, and these were summed. This composite score, a school district identifier, and sixth-grade enrollment guided the creation of the school sets and groups. The equivalence of the final school sets and study groups was assessed by comparing profiles of the ethnic composition, test scores, socioeconomic status, and enrollment.
Intervention The curriculum was developed and pilot tested over a period of several years. Students in focus groups provided information about how youth think and feel about various topics related to sexuality and also provided feedback about lesson ideas. Each lesson activity was tested initially in schools that were not formally part of the study. Once the various activities were revised, lessons were created, and additional piloting and revision occurred. Finally, the full set of lessons for each grade was given to 10 or more classrooms of students in another school district and final revisions were made. Student feedback was used throughout the process to improve the lessons and make them more enjoyable. During the study, experienced health educators were hired to teach the lessons. Project staff trained these health educators before implementation. The health educators also practiced implementing the curriculum by teaching it at a school not involved in the study. The curriculum was taught during a specified period at each school; the period was scheduled in cooperation with the schools. According to the school site coordinators, the curriculum served as the main source of HIV, other STD, and pregnancy prevention education in the intervention schools, although some schools implemented other minimal activities (e.g., limited instruction in science classes, brief presentations/assemblies).
Data Collection Procedures
Participants Students average age at baseline was 11.5 years. The gender mix was nearly equal (50.1% female). The racial/ethnic composition of the sample was as follows: 5.2% African American, 15.9% Asian, 59.3% Latino, 16.5% White, and 3.1% other. Approximately 4% of the students reported having had sexual intercourse at baseline. The intervention and control groups were equivalent on demographic variables assessed at baseline.
Instrument
The sexual behavior outcome variables included binary measures of sexual intercourse (ever had sex), sexual intercourse in the past 12 months, and condom use at last intercourse. Additional items measured the numbers of sexual episodes and sexual partners during the past 12 months. The phrase having sex was defined as "a mans penis in a womans vagina." We also assessed coercive behaviors in the past 12 months (coercing others to kiss, touch, or have sexual intercourse when they did not want to), and unwanted sexual advances (being coerced by others to kiss, touch, or have sexual intercourse when it was unwanted). These 2 coercion variables were scaled and are summarized in Table 1
Statistical Analyses Primary Regression Analyses. Repeated measures logistic and linear regression models estimated the treatment group effects, separately for boys and girls, on the study outcomes from baseline to the end of the ninth grade. Model estimates were based on generalized estimating equations26 with an unstructured correlation matrix specified to account for the correlation of repeated measurements within respondents. Explanatory variables included an intervention group indicator, categorical time indicator, group-by-time interaction, student ethnicity, and the peer norm scale measured at baseline. The variation of schools within treatment groups was modeled by a fixed nested effect. Custom contrasts were used to estimate group differences at each grade level as well as the groups-by-time and groups-by-square-root-time interactions. Generally, outcome levels were a function of the square root of time for boys and a linear function of time for girls; exceptions are noted. R2 values for models of behavioral outcomes also are reported.27 Preliminary analyses revealed baseline differences between intervention and control groups in reports of ever having had sex. Percentages for boys in the intervention and control groups were 6.34 and 4.33 (P < .07). For girls, percentages were 3.99 and 2.78 (P < .20) for intervention and control groups, respectively. To control for these differences, we included the baseline peer norm covariate because it was significantly related to baseline reports of ever having sexual intercourse. After control for baseline peer norms, the baseline differences between the intervention and control groups on the variable ever had sex were greatly diminished (both P values < .60). Mediation Analyses. Additional analyses determined whether group membership affected each measured psychosocial construct assessed at the end of eighth grade. These analyses were conducted separately for boys and girls and also included ethnicity and baseline peer norms as explanatory variables. Next, the ninth-grade indicator of sexual intercourse in the past 12 months was regressed on the treatment group indicator, ethnicity, baseline peer norms, and all psychosocial constructs that were at least weakly related to group membership (P < .10). When primary regression analyses found treatment effects on sexual intercourse in the past 12 months, these subsequent analyses tested whether the psychosocial constructs mediated that effect.28,29 When primary analyses found no treatment effects, these analyses provided additional insight into the effect of the treatment group assignment on the psychosocial variables. We used sexual intercourse in the past 12 months rather than ever had sex as the outcome for these analyses because some youth initiated intercourse before the eighth-grade assessment. These models were fit within the logistic and ordinal logistic regression framework. Associated direct effects were reported as odds ratios with 95% confidence intervals. When the psychosocial constructs were modeled as explanatory variables, standardized odds ratios of their effects were reported. When the psychosocial constructs were modeled as outcomes, they were first coarsened to have 5 ordered categories, and ordinal logistic regression models were fit. Multiple Imputation. Each model was fit to 20 multiply-imputed data sets created with SAS PROC MI.30 The imputation model included all outcome and explanatory variables described in this article. Because PROC MI assumes a multivariate normal distribution, imputed values for binary and ordinal variables were subsequently rounded to the nearest applicable integer.31 All parameter estimates and significance tests were calculated by combining results across the imputed data sets.32,33
Attrition Overall retention rates were 91%, 88%, and 64% in seventh, eighth, and ninth grades, respectively. Generally, differential rates of attrition across treatment groups did not appear to be a problem. A longitudinal analysis regressed student retention indicators measured in seventh, eighth, and ninth grades on intervention group assignment, gender, time, ethnicity, and all interactions, as well as the fixed effect of schools nested within treatment groups. All 3- and 4-way interactions were insignificant and were dropped from the model. Four effects were significant: (1) time, P < .0001; (2) gender by time (retention declined more rapidly for boys [90%, 87%, 56%] than for girls [92%, 90%, 71%], P < .01; (3) race/ethnicity by time (retention declined more rapidly for some racial/ethnic groups [ninth-grade retention rates equaled 55%, 59%, 66%, 72%, and 78% for African Americans, Latinos, Asians, Others, and Whites, respectively]), P < .05; and (4) intervention group by race/ethnicity (within each racial/ethnic group there were no significant differences in retention across treatment groups. However, among Whites, retention was highest in the intervention group, whereas among African Americans, Latinos, and Others, retention was highest in the control group, P < .01.
Behavioral Outcomes
Sexual IntercoursePast 12 Months. There were no statistically significant group-by-time effects for boys or girls on whether or not they reported sexual activity in the 12 months before each survey administration (model R2 = 0.105 and 0.140, respectively). Nonetheless, for boys, there was a statistically significant effect at each follow-up assessment periodboys receiving Draw the Line/Respect the Line were less likely than boys in the control group to report having had sex in the 12 months before the survey in seventh, eighth, and ninth grades (Table 2 We also examined the effects of the intervention on the number of times students had sexual intercourse in the past 12 months and the number of sexual partners in that same time period (data not shown). We found no statistically significant group-by-time effects for boys or girls on either variable; however, among boys, there were statistically significant effects on both variables at the eighth grade follow-up favoring the intervention group boys (P = .01 and .02, respectively). These effects diminished somewhat by ninth grade (P = .09 and .13, respectively). No such treatment effects were found for girls. Condom Use. There were no statistically significant treatment group-by-time effects among sexually active boys or girls for condom use at last intercourse (data not shown). Further, none of the pairwise comparisons at each time point were statistically significant.
Psychosocial and Behavioral Determinants
Mediation-Type Analyses A final set of analyses determined whether intervention group assignment was related to each of the psychosocial constructs measured in eighth grade and whether the affected psychosocial constructs were associated with reports of sexual intercourse in the past 12 months as measured during ninth grade. These analyses tested whether the psychosocial constructs mediated the intervention effect for boys. For girls, no intervention effect was noted, and the analyses helped to document any limited impact of the intervention.
For boys, intervention group assignment was found to affect 6 of the 12 psychosocial constructs measured in eighth grade (Figure 1
For girls, the intervention group influenced 4 of the psychosocial constructs measured in eighth grade (Figure 2
Our findings suggest that the Draw the Line/Respect the Line curriculum produced several positive and programmatically important behavioral effects among boys in the cohort, but not among girls. Gender-specific effects on sexual behavior are not unique. Other studies have reported similar resultssome favoring males34,35 and some favoring females.14,36 Those studies finding no effects for females suggest that girls may need more intense skill-building opportunities, more support for handling coercion, and a more supportive social environment (e.g., one that addresses gender role and peer influences). Our findings support these conclusions and yield new ones. Our data reported elsewhere37 suggest that the influence of older boyfriends may have contributed to the lack of effects for girls. Almost 30% of eighth-grade girls in this study had a boyfriend 2 or more years older, and these girls were much more likely to report having had sex.37 Our intervention did not address this risk factor nor did it prepare girls on how to counter pressure from an older partner. Coercion also may have been a factor. Other studies support this as a potentially important factor for girls.35,38,39 As we found, however, small changes in this factor were not sufficient to produce a treatment effect for girls. Clearly, additional research is needed to better understand factors influencing girls decisions to engage in sexual intercourse so that more effective interventions can be developed. Our study also supports the long-held finding that increasing knowledge alone is not enough to yield changes in sexual behavior.6 In contrast, for the boys, this intervention may have created a new norm within the school environmentone that made boys more comfortable with the idea of refraining from sex. Many studies support the importance of the influence of peer norms on sexual behavior.6,40,41 The Draw the Line/Respect the Line intervention also provided boys with a crucial skillrecognizing and avoiding situations that might lead to sexual intercourse. Our mediation analyses suggest that this factor is the most important behavioral mediator of those we studied. Not many other published studies have reported data on this variable as we have constructed it, although Murphy et al.42 found that males had lower perceived ability to plan ahead and avoid risky situations than did females. The mediation analyses also suggest that having strong sexual limits and believing that girls should not pressure boys for sex are important factors that can be influenced by intervention. Contrary to expectations, we found no effects on condom use at last intercourse for boys or girls. Because so few students in the sample were engaging in recent sexual activity (i.e., within 3 months before the survey), our statistical power to detect changes for condom use at last intercourse was limited.
Study Limitations We also experienced a relatively high level of nonparticipation in the survey at baseline (24% of students who returned consent forms were denied participation in the survey). Although we do not have demographic data on these students, we do know that the majority of parents who refused participation did so because they felt their children were too young to complete a survey on sexual behaviors. Students lost to follow-up differed from students retained in the cohort. Across time, attrition rates of boys, African Americans, and Latinos increased more rapidly than those of girls and the other racial/ethnic groups. Further, a significant interaction suggested that attrition among White students was highest in the control group, whereas attrition among the Latino, African American, and other groups was highest within the intervention group. To the extent that the data were not missing at random,45 or at least approximately so, attrition could potentially bias the study results. However, our use of a rich imputation model limited this threat.
Conclusions
The study was supported by Grant MH51515 from the National Institute of Mental Health to Barbara Marín, with Douglas Kirby and Karin Coyle as co-investigators. The authors gratefully acknowledge the contributions to this research project by Romy Bernard, Nancy Calvin, Tiffany Chinn, Jennifer Cummings, Cherri Gardner, Leticia Gonzales, Deborah Ivie, Slyvia Ortiz, and Benny Vasquez. Additionally, the project could not have been a success without the support of the district representatives, principals, teachers, school staff, and students who welcomed us at their schools.
Human Participant Protection
Contributors B. Marín, K. Coyle, C. Gómez, and D. Kirby conceived the study, developed the curriculum, and supervised all aspects of the study. S. Gregorich developed the analysis plan and conducted the data analyses. K. Coyle and S. Gregorich led the writing of the article. All authors took part in interpreting the study findings and reviewing drafts of the article. Accepted for publication June 19, 2003.
1. Thurman S. Youth and HIV/AIDS 2000: a new American agenda. [The body: an AIDS and HIV information resource Web site]. October 2000. Available at: http://www.thebody.com/whitehouse/youthreport/director.html. Accessed February 6, 2004. 2. The Alan Guttmacher Institute. Sex and Americas Teenagers. New York: Alan Guttmacher Institute; 1994. 3. Flanigan C. Whats Behind the Good News: The Decline in Teen Pregnancy Rates During the 1990s. Washington, DC: National Campaign to Prevent Teen Pregnancy; 2001. 4. Singh S, Darroch J. Adolescent pregnancy and childbearing: levels and trends in developed countries. Fam Plann Perspect. 2000;32:1423.[Web of Science][Medline] 5. Martin JA, Park MM, Sutton PD. Births: preliminary data for 2001. Natl Vital Stat Rep. June 6, 2002;50(10):120.[Medline] 6. Kirby D. Emerging Answers: Research Findings on Programs to Reduce Teen Pregnancy. Washington, DC: National Campaign to Prevent Teen Pregnancy; 2001. 7. Coyle K, Basen-Engquist K, Kirby D, et al. Safer choices: reducing teen pregnancy, HIV, and STDs. Public Health Rep. 2001;116(suppl 1):8293.[Web of Science][Medline] 8. Hubbard BM, Giese ML, Rainey J. A replication of Reducing the Risk, a theory-based sexuality curriculum for adolescents. J Sch Health. 1998;68:243247.[Web of Science][Medline] 9. Kirby D, Barth R, Leland N, Fetro J. Reducing the Risk: a new curriculum to prevent sexual risk-taking. Fam Plann Perspect. 1991;23:253263.[Web of Science][Medline] 10. Main DS, Iverson DC, McGloin J, et al. Preventing HIV infection among adolescents: evaluation of a school-based education program. Prev Med. 1994;23:409417.[Web of Science][Medline]
11. Walter HJ, Vaughn RD. AIDS risk reduction among a multi-ethnic sample of urban high school students. JAMA. 1993;270:725730.
12. Jemmott JBIII, Jemmott LS, Fong GT. Reductions in HIV riskassociated sexual behaviors among black male adolescents: effects of an AIDS prevention intervention. Am J Public Health. 1992;82:372377. 13. St Lawrence JS, Jefferson KW, Alleyne E, Brasfield TL. Control of education versus behavioral skills training interventions in lowering sexual HIV risk behavior of substance dependent adolescents. J Consult Clin Psychol. 1995;63:221237.[Web of Science][Medline]
14. Siegel DM, Aten MJ, Enaharo M. Long-term effects of a middle school and high school based human immunodeficiency virus sexual risk prevention intervention. Arch Pediatr Adolesc Med. 2001;155:11171126. 15. Jamieson A, Curry A, Martinez G. School Enrollment in the United StatesSocial and Economic Characteristics of Students. US Census Bureau. Available at: http://www.census.gov/prod/2001pubs/p20-533.pdf. Accessed February 11, 2004 16. Kirby D, Korpi M, Adivi C, Weissman J. An impact evaluation of Project SNAPP: an AIDS and pregnancy prevention middle school program. AIDS Educ Prev. 1997;9(suppl A):4461.[Web of Science][Medline] 17. Kirby D, Korpi M, Barth RP, Cagampang HH. The impact of the Postponing Sexual Involvement curriculum among youths in California. Fam Plann Perspect. 1997;29:100108.[Web of Science][Medline]
18. Mitchell-DiCenso A, Thomas BH, Devlin MC, et al. Evaluation of an educational program to prevent adolescent pregnancy. Health Educ Behav. 1997;24:300312. 19. Moberg DP, Piper DL. The Healthy for Life Project: sexual risk behavior outcomes. AIDS Educ Prev. 1998;10:128148.[Web of Science][Medline] 20. Howard M, McCabe J. Helping teenagers postpone sexual involvement. Fam Plann Perspect. 1990;22:2126.[Web of Science][Medline] 21. Ekstrand M, Siegel D, Nido V, et al. Peer-led AIDS prevention delays onset of sexual activity and changes peer norms among urban junior high school students. Paper presented at: XI International Conference on AIDS; July 712, 1996; Vancouver, British Colombia.
22. Jemmott J, Jemmott L, Fong G. Abstinence and safer sex: a randomized trial of HIV sexual risk-reduction interventions for young African-American adolescents. JAMA. 1998;279:15291536.
23. Stanton BF, Xiaoming L, Ricardo I, Galbraith J, Feigelman S, Kaljee L. A randomized, controlled effectiveness trial of an AIDS prevention program for low-income African-American youths. Arch Pediatr Adolesc Med. 1996;150:363372. 24. Bandura A. Social Foundations of Thought and Action: A Social Cognitive Theory. Englewood Cliffs, NJ: Prentice Hall; 1986. 25. McGuire W. Inducing resistance to persuasion. In: Berkowitz L, ed. Advances in Experimental and Social Psychology. Vol 1. New York, NY: Academic Press; 1964:191229.
26. Liang KY, Zeger SL. Longitudinal data analysis using generalized linear models. Biometrika. 1986;73:1322. 27. Zheng B. Summarizing the goodness of fit of generalized linear models for longitudinal data. Stat Med. 2000;19:12651275.[Web of Science][Medline] 28. Baron RM, Kenny DA. The moderator-mediator variable distinction in social psychological research: conceptual, strategic and statistical considerations. J Pers Soc Psychol. 1986;51:11731182.[Web of Science][Medline]
29. Judd CM, Kenny DA. Process analysis: estimating mediation in treatment evaluations. Eval Rev. 1981;5:602619. 30. SAS OnlineDoc, Version 8 [software program]. Cary, NC: SAS Institute Inc; 1999. 31. Schafer JL. Analysis of incomplete multivariate data. London, England: Chapman & Hall; 1997.
32. Meng XL, Rubin DB. Performing likelihood ratio tests with multiply-imputed data sets. Biometrika. 1992;79:103111. 33. Rubin DB. Multiple Imputation for Nonresponse in Surveys. New York, NY: John Wiley & Sons; 1987. 34. OLeary A, Jemmott LS, Goodhart F, Gebelt J. Effects of an institutional AIDS prevention intervention: moderation by gender. AIDS Educ Prev. 1996;8:516528.[Web of Science][Medline] 35. Metzler C, Biglan A, Noell J, Ary D, Ochs L. A randomized controlled trial of a behavioral intervention to reduce high-risk sexual behavior among adolescents in STD clinics. Behav Ther. 2000;31:2754. 36. Aarons SJ, Jenkins RR, Raine TR, et al. Postponing sexual intercourse among urban junior high school studentsa randomized controlled evaluation. J Adolesc Health. 2000;27:236247.[Web of Science][Medline] 37. Marin BV, Kirby DB, Hudes ES, Gomez CA, Coyle KK. Youth with older boyfriends and girlfriends: associations with sexual risk. In: Albert B, Brown S, Flanigan C, eds. Too Much, Too Soon: The Sex Lives of Young Teens. Washington, DC: The National Campaign to Prevent Teen Pregnancy; 2003. 38. Leitenberg H, Saltzman H. A statewide survey of age at first intercourse for adolescent females and age of their male partners: relation to other risk behaviors and statutory rape implications. Arch Sex Behav. 2000;29:203215.[Web of Science][Medline] 39. Marin BV, Coyle KK, Gomez CA, Carvajal SC, Kirby DB. Older boyfriends and girlfriends increase risk of sexual initiation in young adolescents. J Adolesc Health. 2000;27:409418.[Web of Science][Medline] 40. Bearman PS, Bruckner H, Bradford B, Theobald W, Philliber S. Peer Potential: Making the Most of How Teens Influence Each Other. Washington, DC: The National Campaign to Prevent Teen Pregnancy; 1999. 41. Carvajal S, Parcel G, Basen-Engquist K, Banspach S, Coyle K, Kirby D. Psychosocial predictors of the delay of sexual intercourse by adolescents. Health Psychol. 1999;18:443452.[Web of Science][Medline] 42. Murphy DA, Rotheram-Borus MJ, Reid HM. Adolescent gender differences in HIV-related sexual risk acts, social-cognitive factors and behavioral skills. J Adolesc. 1998;21:197208.[Web of Science][Medline]
43. Brener ND, Collins JL, Kann L, Warren CW, Williams BI. Reliability of the youth risk behavior survey questionnaire. Am J Epidemiol. 1995;141:575580. 44. Sonenstein F, Ku L, Pleck J. Measuring sexual behavior among teenage males in the US. Paper presented at: Researching Sexual Behavior: Methodological Issues, Kinsey Institute for Research in Sex, Gender, and Reproduction, University of Indiana; April 2628, 1996; Bloomington, Ind.
45. Rubin,DB. Inference and missing data. Biometrika. 1976;63:581592. This article has been cited by other articles:
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||