© 2006 American Public Health Association DOI: 10.2105/AJPH.2004.047399
George Patton is with the Centre for Adolescent Health and Murdoch Childrens Research Institute, Melbourne, Australia. Lyndal Bond is with the Centre for Adolescent Health and Department of Pediatrics, University of Melbourne, Melbourne. John Carlin is with the Clinical Epidemiology and Biostatistics Unit, Department of Pediatrics, University of Melbourne. At the time of the study, Lyndal Thomas and Sara Glover were with the Centre for Adolescent Health and Department of Pediatrics, University of Melbourne. Helen Butler is with the Centre for Adolescent Health and Murdoch Childrens Research Institute. Richard Catalano is with the Seattle Social Development Research Group, University of Washington, Seattle, Washington. Glenn Bowes is with the Department of Pediatrics, University of Melbourne. Correspondence: Requests for reprints should be sent to George C Patton, William Buckland House, 2 Gatehouse St, Parkville, Victoria 3052, Australia. (e-mail: george.patton{at}rch.org.au; lyndal.bond{at}rch.org.au).
Objectives. We sought to test the efficacy of an intervention that was designed to promote social inclusion and commitment to education, in reducing among students health risk behaviors and improving emotional well-being. Methods. The design was a cluster-randomized trial in 25 secondary schools in Victoria, Australia. The subjects were 8th-grade students (aged 13 to 14 y) in 1997 (n=2545) and subsequent 8th-grade students in 1999 (n=2586) and 2001 (n=2463). The main outcomes were recent substance use, antisocial behavior, initiation of sexual intercourse, and depressive symptoms. Results. At 4-year follow-up, the prevalence of marked health risk behaviors was approximately 20% in schools in the comparison group and 15% in schools in the intervention group, an overall reduction of 25%. In ordinal logistic regression models a protective effect of intervention was found for a composite measure of health risk behaviors in unadjusted models (odds ratio [OR]= 0.69; 95% confidence interval [CI]= 0.50, 0.95) and adjusted models (OR= 0.71; CI =0.52, 0.97) for potential confounders. There was no evidence of a reduction in depressive symptoms. Conclusion. The study provides support for prevention strategies in schools that move beyond health education to promoting positive social environments.
Young people spend well over one third of their waking hours in school. For this reason, schools have often been a setting for preventive intervention. The use of school curricula to promote protective attitudes and skills has met with some success in reducing health risk behaviors.13 However, there have been some spectacular failures, which have led to calls for a reexamination of current approaches to prevention in schools.4 One potential but currently neglected focus for prevention is the schools social milieu.5,6 There is much evidence that the schools social atmosphere affects patterns of substance use, antisocial and disruptive behaviors, as well as how well students learn. Advocates of health promotion have argued that addressing organizational processes and social relationships are likely to be effective in bringing about behavioral change.79 Despite the attractiveness of such a health promotional approach, relatively few strategies have been tested that use this approach.1012 The Gatehouse Project intervention was designed as a structured process to promote a sense of social inclusion and connection in secondary schools, building on the principles of the Health Promoting Schools Project.7,13,14 The process involved use of information about a schools social climate to assist in setting priorities for action. Intervention elements ranged from establishing an inclusive classroom environment to creating opportunities for student participation in school life beyond the classroom, and included implementing a student curriculum that teaches interpersonal communication and emotional management.15,16 We examine the school-level effects of this intervention on indexes of health and behavior in lower secondary schools. measured at 2-year and 4-year follow-up.
Intervention The Gatehouse Project intervention strategies have previously been described elsewhere.1517 The intervention occurred during a 2-year period beginning in term 2 (April 19 to June 18), 1997. The structured process had 4 elements: (1) feedback from a student survey about security (a students personal sense of safety), communication with teachers, and broader participation in school life; (2) within each school, recruitment of staff involved in administration, student welfare, curriculum, or all 3 to a coordinating action team with a focus on school policies and professional practice of teachers; (3) consultation and training regarding specific intervention strategies; and (4) a curriculum element that focused on problem-solving in situations in which young people commonly experience emotional difficulties. This strategy typically included the following sequence: (1) conduct a survey of the students, (2) provide feedback from the school social profile, based on the student survey, to the school-based action team, (3) consult with the school team regarding intervention priorities, and (4) train teachers in the selected strategies. The feedback process included an examination of the school profile of students school environment experiences, which was used to set priorities, facilitate discussion, and examine specific areas in need of action. Thus, for example, a school with students reporting a high level of deliberate social exclusion by peers might choose this particular area as a priority focus. Strategies varied between schools according to students perceptions of need; but, the implementation of school policy and curriculum elements that focused on social and emotional skills and strategies to promote inclusive relationships within the classroom were always addressed. The curriculum element of the intervention was designed to be taught in the 8th grade during a 10-week period in English, health, or personal development classes. The median number of lessons using the Gatehouse Project Intervention curriculum in the first year, as reported by the school liaison team, was 20 (approximately 15 hours of instruction). An average of 40 hours of professional development were provided by the intervention team to each school during the first and second years, respectively, with the time split between a focus on curriculum and whole school strategies.18
Design and Participants From the 10 nonmetropolitan school districts outside of Melbourne, 2 regional districts were selected and randomly allocated to intervention and control status. Four schools (2 government, 2 independent /Catholic) were then randomly selected from each pool of schools in these districts. From the total group (metropolitan and nonmetropolitan), 2 schools1 in the intervention group and 1 in the controldeclined to participate, citing involvement in other projects. Four further schools were unable to participate because of threatened closure or amalgamation.
Three cross-sectional surveys of 8th grade students (aged 1314 y) in the participating schools were conducted at 2-year intervals. The initial survey took place in the school classroom between February 17 and March 28, 1997 (term 1), before the intervention. The survey was a self-administered questionnaire conducted on laptop computers provided by the research team. Absent students were surveyed at school at a later date or by telephone. New cohorts of 8th grade students were surveyed between April 19 and June 18, 1999 (term 2) and again between April 23 and June 22, 2001 (term 2); the items from the initial survey were identical but were administered in a pencil and paper format. Student participation on each occasion was voluntary and required written parental consent. School participation rates are shown in Figure 1
Measures Substance use was measured by the self-reported frequency of tobacco, alcohol, and cannabis use. Report of tobacco and alcohol use was limited to use within the past month. Participants who smoked cigarettes or consumed alcohol during this time kept a retrospective 7-day diary.19 Tobacco use was defined on 3 levels: no use in the past month, having smoked at least 1 cigarette in the past month, and having smoked on 3 or more days in the previous week (to distinguish weekend smokers from more consistent smokers).20 Alcohol use was defined on 3 levels as well: no recent use, use in the previous week, and binge drinking (consuming 5 or more drinks in a row on at least 2 occasions in the past 2 weeks). Report of cannabis use was limited to use within the past 6 months. Use was again defined on 3 levels: no use in the past 6 months, use at least once in the previous 6 months, and current use at least weekly. Dichotomous variables were defined for estimating prevalence of use. Any substance use was defined as either having used alcohol in the previous week, tobacco in the previous month, or cannabis in the previous 6 months. Heavy substance use was defined as binge drinking, tobacco use on at least 3 days in the previous week, or cannabis use at least weekly. A composite outcome on 3 levels was defined for the purpose of multivariate analysis: no recent use of any of the 3 substances, recent use of at least 1, and recent heavy use of 1 or more. Antisocial behavior was assessed with items from the Self-Reported Early Delinquency Scale covering property damage, interpersonal violence, and theft in the previous 6 months.21 Any antisocial behavior referred to at least 1 instance in the previous 6 months and frequent antisocial behavior referred to 2 or more instances. A composite antisocial behavior outcome on 3 levels was defined for each respondent for multivariate analysis: no reported behavior, 1 instance in the past 6 months, and more than 1 instance in the past 6 months. Early initiation of sexual intercourse was assessed with a single item about ever having had sexual intercourse (sexual intercourse, "gone all the way"). In order to assess the overall effect of the intervention on health risk behaviors, 2 further composite 3-level outcomes were defined, incorporating the substance use indicators, antisocial behavior, and early initiation of sexual intercourse. Any risky behavior was defined on 3 levels as either none, 1 behavior (any substance use, any antisocial behavior or early initiation of sexual intercourse), or 2 or more behaviors at this level. Marked risky behavior was defined as either none, 1 behavior at the highest level (heavy substance use, report of multiple antisocial behaviors, or early initiation of sexual intercourse), or 2 or more behaviors at this level. Emotional problems were assessed at baseline (1997) using a computerized revised Clinical Interview Schedule (CIS-R).22 The total scores were dichotomized at a cut-off point of 11/12.23 In later surveys (1999 and 2001) the short Mood and Feelings Questionnaire was used to assess depressive symptoms with a cut-off point of 11/12, which was used to delineate high symptom levels.24 School commitment was assessed with a questionnaire comprising 23 items and 5 subscales reflecting school attachment, studentteacher communication, perceived opportunities for participation, disincentives, and rewards for participation.25
Data Analysis
Comparison of demographic characteristics between intervention and control groups in 1997, 1999, and 2001 revealed small between-group differences (Table 1
Antisocial and Health Risk Behaviors Figure 2a
There was little difference at baseline in key outcome measures between schools in the intervention group and those in the control group. In 1999, trends were apparent for lower rates of substance use among students in schools in the intervention group. For example, the estimated difference in the prevalence of any health risk behavior was 2.8% (95% confidence intervals [CI] = 4.6, 10) and for marked health risk behavior 4.3%, (CI = 3.7, 12.2). In 2001, prevalence estimates were again generally lower for the intervention group. The absolute difference in prevalence of any risky behaviors among students in schools in the intervention group compared with those in the control group was 4.9% (CI = 3.1,12.9). The between-group difference for marked risky behaviors was also 4.9% (CI = 0.5, 10.4), which represents a reduction of 25% in schools in the intervention group compared with those in the control group.
Intervention effects were formally tested in a series of binary and ordinal logistic regression models, depending on whether the outcome was defined on 2 or 3 levels (Table 2
Emotional Problems Emotional problems at baseline, measured with the CIS-R, were 20.2% (CI=16.4, 24.0) in the control group and 17.2% (CI=14.2, 20.2) in the intervention group. At the 1999 follow-up survey, in which the Mood and Feelings Questionnaire measured emotional problems, rates were 16.9% (CI=13.9, 19.6) in the control group and 17.5% (CI=14.0, 21.0) in the intervention group. In 2001, rates of emotional problems were 14.2% (CI=11, 17.5) in the control group and 12.7% (CI=9.4, 15.8) in the intervention group. No differences were found in adjusted ORs for emotional problems at baseline (OR= 0.85; CI= 0.67, 1.1), in 1999 (OR=1.1; CI= 0.85, 1.4) or in 2001 (OR=0.90; CI=0.65, 1.3).
Student Commitment
Four years after beginning an intervention to promote social inclusion within schools, patterns of health risk behaviors among students in intervention schools differed from those in schools in the control group. Marked health risk behaviors were reported by approximately 15% of students in the intervention school group after the intervention, compared with 20% of those in the control group, an overall reduction of one quarter. This difference arose from lower rates of substance use, antisocial behavior, and early initiation of sexual intercourse by students in the intervention schools. This group difference was not explained by changes in student emotional problems because these did not differ between the intervention and control groups before or after intervention. These reductions in risky behaviors are greater than those found in most recent studies of health education.9,10,28,29 These findings stand out in 2 further respects. First, the effect was apparent across a range of behaviors. Second, the changes were apparent in subsequent cohorts of students, a finding that is consistent with the intent of maximizing sustainability by nesting the intervention within normal school processes and curricula. Given the effects of adolescent-initiated health risk behaviors on health later in life, reductions of this kind could have major public health benefits if the approach were adopted broadly. There were some study limitations. Cluster randomization took place at a school district level, but the intervention and analyses were at the individual school level. This design was necessary to reduce the risks of contamination but is a weaker randomization design. The number of schools that were available after randomization was diminished by nonparticipation of 6 selected schools, raising questions about the effectiveness of randomization, as well as study power. Similarities in the demographic profile of intervention and control groups across surveys are consistent with the integrity of randomization, but a possibility of unmeasured confounding cannot be totally excluded. Although overall response rates were moderately high, some differential response between intervention and control groups in the first 2 surveys might account for our findings. Nonresponse tended to be greater in the control group, and because nonresponders have higher levels of health risk behaviors, differential nonresponse seems more likely to have led to an underestimation of intervention effect. Social desirability, a potential source of measurement bias in nonblinded trials, was limited by integrating the intervention within usual school curriculum and administrative processes.30 The health education component was delivered by teachers through the usual curriculum, minimizing the chances that students were aware of being part of an intervention. It is possible that the first cohortthe 1997 baseline groupwere aware of being a special group, but this is unlikely to have been the case in subsequent surveys. Finally, we cannot exclude chance variation as an explanation for our findings, although changes in health risk behaviors after intervention consistently moved in a favorable direction at follow-up surveys. Our findings carry implications for concepts of how schools may influence student behavior. Measures of school engagement were strongly associated with health risk behaviors, but behavioral change in the intervention schools was not matched by equivalent changes on these measures. This raises a question of whether the constructs of school social inclusion and student connection to education were adequately operationalized to capture change in this study. The intervention also had no clear effect on emotional problems. It is possible that the intervention was not sufficiently specific or sustained to produce a detectable effect, given that study power was marginal. Alternatively, key determinants of depressive symptoms may differ from those for health risk behaviors, perhaps operating outside the secondary school setting or at a developmentally earlier point. Our study is a rare example of the implementation and evaluation of a complex systems intervention to improve health.30 The elements of the intervention processfeedback on the social context, creation of a coordinating structure (i.e., the action team), and ongoing consultationmay be useful in other settings. The process coordinated health promotional work with varied foci, as well as provided a means for nesting a health agenda within a schools policy and practice framework. The result was the development of interventions tailored to the needs of particular schools but differing between schools. Although there remains much to learn about the wider application of this particular approach, our findings support strategies to promote the social milieu of schools as a way of achieving better health and learning outcomes.9,10
Peer Reviewed
Contributors
Human Participant Protection Accepted for publication September 23, 2005.
1. Botvin GJ, Baker E, Dusenbury L, Botvin EM, Diaz T. Long-term follow-up results of a randomized drug abuse prevention trial in a white middle-class population. JAMA. 1995;273:11061112. 2. Ellickson PL, McCaffrey DF, Ghosh-Dastidar B, Longshore DL. New inroads in preventing adolescent drug use: results from a large-scale trial of project ALERT in middle schools. Am J Public Health. 2003; 93:18301836. 3. Ellickson PL, Bell RM. Drug prevention in junior high: a multisite longitudinal test. Science. 1990; 247:12991305. 4. Peterson AV, Kealey KA, Mann SL, Marek PM, Sarason IG. Hutchinson smoking prevention project: long-term randomized trial in school-based tobacco use preventionresults on smoking. J Natl Cancer Institute. 2000;92:19791991. 5. Rutter M, Maughan B, Mortimore P, Ouston J, Smith A. Fifteen Thousand Hours: Secondary Schools and their Effects on Children. London: Open Books, 1979. 6. Resnick MD, Bearman PS, Blum RW, Bearinger L, Harris KM, Jones J et al. Protecting adolescents from harm: findings from the National Longitudinal Study on Adolescent Health. JAMA. 1997;278:823832. 7. World Health Organization/UNESCO/UNICEF. Comprehensive School Health Education: Suggested Guidelines for Action. Geneva: World Health Organization; 1992. 8. Moon AM, Mullee MA, Rogers L, Thompson RL, Speller V, Roderick P. Helping schools become health-promoting environments: an evaluation of the Wessex Healthy Schools Award. Health Promotion Int. 1999; 14:111122. 9. Flay BR. Approaches to substance use prevention utilizing school curriculum plus environmental social change. Addict Behav. 2000;25:861885.[CrossRef][Web of Science][Medline] 10. Lynagh M, Schofield MJ, Sanson-Fisher RW. School health promotion programs over the past decade: a review of the smoking, alcohol, and solar protection literature. Health Promotion Int. 1997;12: 4360. 11. Hawkins JD, Catalano RF, Kosterman R, Abbott R, Hill KG. Preventing adolescent health-risk behaviors by strengthening protection during childhood. Arch Pediatrics Adol Med. 1999;153:226234. 12. Olweus D. Bully/victim problems among schoolchildren. In: Rubin K, Pepler D, eds. The Development and Treatment of Childhood Aggression. Hillsdale, NJ: Lawrence Erlbaum Associates, 1991. 13. Effective School Health PromotionTowards Health Promoting Schools. Canberra: National Health and Medical Research Council; 1996. 14. McNeely CA, Nonnemaker JM, Blum RW. Promoting school connectedness: evidence from the National Longitudinal Study of Adolescent Health. J School Health. 2002;72:138146.[Web of Science][Medline] 15. Patton GC, Bond L, Butler H, Glover S. Changing schools, changing health? The design and implementation of the Gatehouse Project. J Adol Health. 2003; 33:231239.[CrossRef][Web of Science][Medline] 16. Patton GC, Glover S, Butler H, Bond L, Bowes G. The Gatehouse Project: Building the capacity for mental health promotion in secondary schools. Aust NZ J Psych. 2000;34:586593.[CrossRef] 17. Bond L, Glover S, Godfrey C, Butler H, Patton GC. Measuring system-level change in schools: lessons from the Gatehouse Project. Health Educ Res. 2001;28: 368383. 18. Bond L, Patton GC, Glover S, Carlin JB, Butler H, Thomas L et al. The Gatehouse Project: can a multilevel school intervention affect emotional well being and health risk behaviours? J Epi Commun Health. 2004;58: 9971003.[CrossRef] 19. Hilton ME. A comparison of a prospective diary and two summary recall techniques for recording alcohol consumption. Br J Addict. 1989;84:10851092.[CrossRef][Web of Science][Medline] 20. Escobedo LG, Marcus SE, Holtsman D, Giovino GA. Sports participation, age at smoking initiation and the risk of smoking among US high-school students. JAMA. 1993;269:13911395. 21. Moffitt TE, Silva PA. Self-reported delinquency: Results from an instrument for New Zealand. Aust NZ J Criminol. 1988;21:227240. 22. Lewis G, Pelosi AJ. The manual of CIS-R. London: Institute of Psychiatry; 1992. 23. Lewis G, Pelosi AJ, Araya R, Dunn G. Measuring psychiatric disorder in the community: a standardized assessment for use by lay interviewers. Psychol Med. 1992;22:465486.[Web of Science][Medline] 24. Angold A, Costello EJ, Messer EC, Pickles A, Winder F, Silver D. Development of a short questionnaire for use in epidemiological studies of depression in children and adolescents. Int J Methods Psych Res. 1995;5:237249. 25. Arthur MW, Hawkins JA, Pollard RF, Catalano RF, Baglioni, AJ. Measuring risk and protective factors for substance use, delinquency, and other adolescent problem behaviours: the communities that care youth survey. Evaluation Review. 2002;26:575601. 26. Carlin JB, Wolfe R, Coffey C, Patton GC. Analysis of binary outcomes in longitudinal studies using weighted estimating equations and discrete-time survival methods: prevalence and incidence of smoking in an adolescent cohort. Statistics in Medicine. 1999;18: 26552679.[CrossRef][Web of Science][Medline] 27. Long JS, Freese J. Regression models for categorical variables using stata. Texas: Stata Press Publications; 2001. 28. Tobler NS, Stratton HH. Effectiveness of school-based drug prevention programs: a meta-analysis of the research. J Primary Prev. 1997;18:71128.[CrossRef] 29. Thomas R, Bushby K. School based programmes for preventing smoking (protocol for a Cochrane Review). The Cochrane Library, ed. 1999. Oxford. 30. Campbell M, Fitzpatrick R, Haines A, Kinmouth AL, Sandercock P, Spiegalhalter D, et al. Framework for design and evaluation of complex interventions to improve health. BMJ. 2000;321:694696. This article has been cited by other articles:
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||