© 2007 American Public Health Association DOI: 10.2105/AJPH.2006.089532
Linda Neuhauser, Norman A. Constantine, Karen Sokal-Gutierrez, and S. Leonard Syme are with Community Health and Human Development, School of Public Health, University of California, Berkeley. Wendy L. Constantine is with Research and Evaluation Systems, Layfayette, Calif. Susan K. Obarski is with Obarski Consulting, Los Angeles, Calif. At the time of the study, Lacy Clayton and Mona Desai were with Community Health and Human Development, School of Public Health, University of California, Berkeley, and Gerald Sumner was with the Institute for Social Science Research, University of California, Los Angeles. Correspondence: Requests for reprints may be sent to Linda Neuhauser, DrPH, Community Health and Human Development Division, School of Public Health, 50 University Hall, MC7360, University of California, Berkeley, CA 94720-7360 (e-mail: lindan{at}berkeley.edu).
Objectives. There is a critical need for effective, large-scale health communication programs to support parents of children aged 0–5 years. We evaluated the effectiveness of the Kit for New Parents, a multimedia health and parenting resource now distributed annually to 500000 parents in California. Methods. In this quasi-experimental study, 462 mothers in the intervention group and 1011 mothers in the comparison group, recruited from prenatal and postnatal programs, completed a baseline interview about health-relevant parenting knowledge, and mothers in the intervention group received the kit. Both groups were reinterviewed 2 months later. At 14-months postbaseline, 350 mothers in the intervention group and a sample of 414 mothers who had equivalent demographic characteristics (comparison group) were interviewed about parenting knowledge and practices. Results. Of the mothers in the intervention group, 87% reported using the kit within 2 months after receiving it, and 53% had shared it with their partner. At both follow-ups, mothers in the intervention group showed greater gains in knowledge and reported better practices at 14 months than did mothers in the comparison group. Gains were greater for prenatal recipients and for Spanish speakers. Providers considered the kit a valuable resource for their parenting programs. Conclusions. The kit is an effective, low-cost, statewide health intervention for parents.
A critical public health challenge is to translate research findings into effective, large-scale, and sustainable interventions.1–3 Interventions during pregnancy and early childhood are especially important because they have shown some of the greatest impacts on childrens long-term health and development.4–8 Parents knowledge and practices related to health care, bonding with their baby, nutrition, smoking, safety, and other factors can greatly affect childrens healthy development.9–13 Although parenting education should be a priority public health investment, this has not been achieved at the population level.6 A national survey found that first-time, adolescent, single, and low-income parents cited parenting information as one of their greatest needs.14 In a California survey, the majority of parents (including 70% of Spanish-speaking parents) believed they did not have adequate knowledge to care for their young children.15 Most of the effective interventions reported to date have involved relatively intensive, expensive, person-to-person programs.7,16,17 Affordable mass coverage will require greater reliance on less expensive, materials-based approaches. These interventions must address the traditional concern that such approaches have weak effects, especially for underserved and low-literate groups who may show relatively low levels of knowledge both before and after interventions, i.e., "knowledge gaps."18–20 We found 16 published experimental or quasi-experimental studies that tested materials-based parenting education interventions. Although most of these studies were small and focused on limited topics in fairly homogeneous populations, they generally showed positive outcomes on parents knowledge, attitudes, or practices.21 Research on communication, adult learning, and motivation indicates that educational interventions are more effective when the intended beneficiaries participate in their design and when the approaches are interactive, adaptable to different learning styles, build on peoples self-identified needs, fit within their social contexts, and connect them with local resources.21–25
In 2001, the First 5 California Children and Families Commission ("First 5") took up the challenge to provide health and parenting education to all new parents in the state by developing the Kit for New Parents; the kit was primarily funded by a tobacco tax. Using guidance from past research, the commission worked with experts and providers to develop and test content in the following priority areas: prenatal care; child safety, nutrition, and health care; early childhood learning and development; childcare; and access to services. Examples of content include ways to stop smoking, childproof homes, feed infants healthy foods, and encourage early learning. Each kit contains 6 videos (featuring celebrities, experts, and diverse parents and children), a 60-page resource guidebook, topical brochures, and a book to read to babies (Figure 1
To determine whether to launch the kit program statewide, First 5 commissioned an evaluation to answer the following questions: 1) Do parents use the kit and find it helpful? 2) Do mothers who use the kit show greater knowledge gains in the short and long term than mothers who do not receive a kit? 3) Do mothers make positive changes in parenting practices because of the kit? 4) Do Spanish-speaking mothers benefit from the kit as much as English-speaking mothers? And 5) What changes could improve the kit and its distribution?
Outcome Evaluation Study design and sample. We used a quasi-experimental longitudinal design (Figure 2
Upon recruitment, baseline interviews were conducted with 542 women who comprised the intervention group. After the interview, mothers received a kit and most were briefly oriented on its use. Two months later, 462 of these mothers (85%) completed a follow-up telephone interview. To better assess the kits effectiveness before statewide distribution, First 5 subsequently funded the addition of a comparison group of mothers who did not receive a kit. Six months after the mothers in the intervention group completed follow-up interviews, we used the same quotas to recruit a comparison group from pre- and postnatal programs that were carefully selected to be similar to those from which the mothers in the intervention group were recruited. Of the 1236 mothers in the comparison group who were recruited and interviewed at baseline, 1011 (82%) completed the 2-month follow-up telephone interview. First 5 later commissioned a 14-month follow-up interview. Of the original 542 women in the intervention group who completed the baseline interview, 350 (65%) completed this follow-up. First 5 released the kit for statewide distribution just before the scheduled 14-month intervention group follow-up. Because the original comparison group was recruited 6 months after the intervention group was, we were concerned that mothers in the comparison group might receive the kit during the 6-month wait for their 14-month interview, thereby contaminating the findings. Therefore, we recruited a new comparison sample of mothers who could be interviewed during the same time period that the intervention mothers had their 14-month follow-up interviews. The new comparison group was recruited through a snowball sampling method26–28; that is, the mothers in the original comparison group provided names of other mothers who had babies approximately 6 months older than theirs. This sample of 414 (82% of those referred and eligible) mothers in the new comparison group completed the 14-month interview. Overall, mothers in the baseline intervention and comparison groups had similar background characteristics. In both samples, approximately two thirds of the mothers were enrolled in Medicaid, half chose Spanish as the language for their interview, half were pregnant, half had a newborn, and 16% were adolescents. However, relative to the comparison group, more mothers in the intervention group were aged 29 years or younger (76% vs 70%) and were first-time mothers (49% vs 41%). Mothers in the intervention group and those in the snowball-sampled comparison group who were interviewed at 14 months also had similar background characteristics. There were no significant differences related to age, income, or enrollment in Medicaid and the Special Supplemental Nurtitional Program for Women, Infants, and Children (WIC). However, fewer mothers in the intervention group spoke Spanish compared with the comparison group (46% vs 59%), and more mothers in the intervention group were first-time mothers compared with the comparison group (48% vs 37%). Assessment instruments. The baseline questionnaire included demographic items (age, race, Hispanic/Latino origin, mothers Medicaid enrollment status, and information about pregnancy or birth status); an item regarding the infants health insurance coverage; and 8 knowledge items about priority issues addressed in the kit materials. These 8 knowledge items were developed by an expert panel to cover key areas of health and parenting knowledge cited in the literature21,29 and included in the kit. Five items, in multiple-choice format, related to sleep safety, infant feeding and nutrition, and early learning. The remaining 3 items, in yes or no format, were scenario questions on accessing resources for childcare, low-cost medical care for babies, and smoking cessation. The 2- and 14-month follow-up interviews included the 8 knowledge items, additional demographic and health insurance items, and for kit recipients only, multiple-choice and open-ended questions about kit use, helpfulness, and satisfaction since the previous interview. The 14-month interview contained additional multiple-choice items about mothers practices in caring for older babies related to child development, reading to infants, feeding, home safety, childcare, and discipline. Two of these items were intended to explore practices for which there was little information in the kit: co-sleeping and infant bedtime routines. Two Spanish translators reviewed initial instrument translations, and pretests were conducted in English and Spanish. The 8 knowledge items were each scored 1 for correct answers (otherwise 0), then summed, divided by 8, and multiplied by 100, yielding a scale score with a range of 0 to 100. Cronbachs coefficient alpha, calculated to estimate internal consistency reliability of the knowledge scale at baseline, was .49. A test–retest study of the stability of the knowledge scale was conducted with a separate sample. Pregnant women (N = 191; half Spanish speakers and half English speakers) were recruited at prenatal programs (similar to those in the outcome study) and interviewed by telephone. One week later, 180 (94%) were reinterviewed. The test–retest correlation coefficient was .83.
Data analyses.
Two independent coders grouped open-ended responses into categories. Interrater agreement was 90%. We used cross tabulations to display categorical variables by intervention versus comparison group and other variables, and the
Process Evaluation
Kit Use and Helpfulness At the 2-month interview, 87% of mothers reported using the kit—defined as having watched any of the kit videos or having read any of the written resources (except for the baby book). Kit usage among Spanish-speaking mothers was 95%, compared with 82% among English-speaking mothers. Overall, 53% of mothers reported that their partner also used the kit—61% among Spanish speakers and 43% among English speakers. Mothers who were briefly oriented to the kit by a provider (78%) used more of the 4 kit components (means of 2.1 vs 1.8 components, P<.05) than mothers who were not oriented. Mothers consulted the kit for both general knowledge and specific issues. For example, mothers commented: I wanted to know how to feed the baby. How they grow. My cousin gets the baby and throws her up in the air and catches her. I looked in the brochures, and it showed me it was dangerous. So I told him that it can cause convulsions or even death. He never did it again. At the 2-month interview, 94% of mothers said that the kit was helpful to them. At the 14-month follow-up, 60% of mothers reported using the kit since their 2-month interview. Mothers found the kit helpful for specific issues: child safety practices (76%), infant learning (72%), feeding solid food (71%), breastfeeding (58%), smoking (43%), health care (35%), and childcare (21%). Spanish-speaking mothers, mothers recruited during pregnancy, and mothers who shared the kit with their partners found it helpful for more issues (P < .01) than did English-speaking mothers, mothers who received the kit postnatally, or mothers who did not share it with a partner. There were no significant differences in kit helpfulness between first-time and experienced mothers, or between adolescent and older mothers.
Parenting Knowledge
Figure 3
Knowledge gains were greater with higher family use of the kit (P < .001, Figure 4
Parenting Practices At the 2-month follow-up, 48% of mothers who received the kit reported having changed their thinking or having done something differently because of the kit. Verbatim responses included: I learned that I need to talk to him, spend time with him, read to him, because hes not there just to lie in his crib. Now I see him respond to me when I talk to him like he wants to talk, and he smiles. At the 2-month follow-up, mothers who received the kit prenatally were much more likely to report the subsequent enrollment of their infants in health insurance than mothers who did not receive the kit (adjusted odds ratio of 2.37, P < .001). There was no significant difference in enrollment between mothers who received the kit postnatally and mothers in the comparison group. At the 14-month interview, there were statistically significant differences in parenting practices, with mothers who received the kit reporting a greater (1) number of childproofing steps taken (P < .01), (2) frequency of reading to infants (P < .01), (3) frequency of consistent medical care for infants (P < .01), and (4) frequency of safer bottle-feeding practices (P < .05). One mother reported: It gave us ideas, so we bought covers for the plugs, a gate so he wont get out, and a lock on the bathroom toilet so he wont fall inside. When you just hear about it, you dont really pay attention to it. But once you see it [in the kit] you pay more attention to it. There were no significant differences for breastfeeding initiation or duration, age of first solid foods, co-sleeping practices, talking aloud to the baby, or having a bedtime routine for the baby.
Process Evaluation Providers offered important recommendations to improve the kit program, including organizing the content around the childs age, rather than by topic; adding more information about breastfeeding, disabilities, co-parenting for shared custody, and other topics; offering the videos in a DVD format; reducing the size of the kit box; producing kits in more languages (especially Asian languages); and distributing the kit to parents of older children (aged 1–5 years).
There is increasing evidence that what occurs during pregnancy and early childhood profoundly affects childrens long-term health. Although research findings strongly suggest that interventions to support parents and their young children can be powerful investments, it has been challenging to translate these findings into successful, large-scale, affordable, and sustainable programs. Evidence of efficacy under highly controlled conditions often is not sufficient for policymakers who need confidence that programs will work in diverse, real-world settings. Green and Glasgow argued that public health research tends to neglect such assessment of external validity.30 They recommended that studies examine representative participants, realistic settings, and differential effects among subgroups over at least 1 year; dose–response effects; comparisons with results of other studies; diffusion; customization; realistic costs; and the interventions institutionalization and sustainability. These were also factors of interest to California policymakers, and each received favorable assessment in our study.
Kit Effectiveness for Diverse Groups Knowledge gains in this study compared favorably with results from other studies. The effect sizes for the kit recipients short- and long-term knowledge gains were approximately twice as large as those found in a national meta-analysis of mostly more intensive and more expensive interventions.16 The kit also affected important parenting practices. Mothers who received the kit reported higher rates of early health insurance enrollment for their infants and more positive practices related to infant safety, feeding, health care, and early reading than did mothers in the comparison group. Nonsignificant differences for breastfeeding and feeding solid food may reflect "ceiling effects"—more than 90% of both groups showed the desired practices. Nonsignificant differences related to co-sleeping safety and infant bedtime routines were expected because these items were included to explore practice areas about which there was little advice in the kit. These results challenge traditional thinking about the weaknesses of materials-based health promotion. By combining multiple media and low-literacy formats with a mix of didactic content, emotional support, and behavioral modeling, the kit potentially leverages the findings of newer communication models that emphasize interactivity, flexibility for different styles of learning, and participatory design.22,31,32 The kit is an "active" communication approach that engages parents to define their own issues and preferred times and ways of using the materials. The kit model is guided by public healths socioecological framework, which posits that people learn and make changes within the context of family, community, and culture.23,33 Providers use the kit as a complement to their existing programs. Current research indicates that such interpersonal connections should enhance the impact of this mass communication strategy.34 As a home-based resource, the kit promoted interaction among family members, and it was more effective when shared by the mother and her partner.
Refining and Extending the Kit Model There are several limitations to this study. First 5s iterative decisions to fund the studys components precluded a randomized design and required recruitment of the comparison group 6 months after the intervention group. However, the 2 groups were equivalent on key demographic variables and yielded identical average baseline knowledge scores. For the long-term follow-up, a separate comparison group had to be recruited through snowball referrals from the original comparison group. The snowball comparison group was also equivalent to the intervention group on key background variables. Unlike the intervention group, the snowball comparison group had no previous experience with the assessment. Because the study did not involve a representative sample of parents in California, the results cannot be statistically generalized statewide. However, the study involved diverse mothers from different state regions, from typical pre- and postnatal venues in which the kit is currently distributed. The range of domains and items represented in the knowledge scale was limited by the constraints of conducting baseline interviews that fit into the routine of busy providers in natural settings. It would be useful to explore additional knowledge areas with parents in future studies. Finally, although there were important self-reported practice results that favored the mothers in the intervention group in the cross-sectional analysis at 14 months, more study is needed to longitudinally evaluate kit-related practice changes and effects on child health outcomes. Overall, the study has addressed key threats to internal validity, including potential selection, history, maturation, and attrition effects.35 The patterns of results found across groups, times, and dosages provide converging evidence that supports the overall construct and discriminant validity of the measures employed and of the evaluation design. The external validity of the studys findings is further supported by favorable assessments of the 10 criteria proposed by Green and Glasgow.30 The kit program in California–and those adapted in other states—is an innovative, large-scale, health communication initiative for new parents. The evaluation results suggest that the kit has value as an effective and low-cost statewide resource to educate parents of children aged 0 to 5 years.
The First 5 California Children and Families Commission funded this study. We appreciated the technical assistance we received from Patricia Skelton and Oshi Ruelas, who were formerly staff at First 5 California. We also thank Victoria A. Albright, who was with Field Research Corporation, for her expert management of study surveys. Finally, we thank the thousands of parents and providers who participated in the development and evaluation of the kit.
Human Participant Protection
Peer Reviewed
Contributors Accepted for publication January 5, 2007.
1. Simpson L. Lost in Translation? Reflections on the role of research in improving health care for children. Health Aff.2004;23:125–130. 2. Green LW. From research to best practices in other settings and populations. Am J Health Behav.2001;25:165–178.[Web of Science][Medline] 3. Zervignon-Hakes AM. Translating research findings into large-scale public programs and policy. Future Child.1995;5: 175–191.[CrossRef][Web of Science][Medline] 4. Heckman J. Policies to Foster Human Capital. Chicago, Ill: Irving B. Harris Graduate School of Public Policy Studies, University of Chicago; 2000. 5. McCain MN, Mustard JF. The Early Years Study. Three Years Later. Toronto, Ontario: Publications Ontario; 2002. 6. Karoly LA, Greenwood PW, Everingham SS, et al. Investing in Our Children: What We Know and Dont Know about the Costs and Benefits of Early Childhood Interventions. Santa Monica, Calif: RAND Corp; 1998. Report No: MR-898-TCWF. 7. Gomby DS, Larner MB, Stevenson CS, Lewit EM, Behrman RE. Long-term outcomes of early childhood programs: analysis and recommendations. Future Child.1995;5:6–24.[CrossRef][Web of Science][Medline] 8. Case A, Fertig A, Paxson C. The lasting impact of childhood health and circumstance. J Health Econ.2005;24:365–389.[CrossRef][Web of Science][Medline] 9. License K. Promoting and protecting the health of children and young people. Child Care Health Dev.2004;30:623–635.[CrossRef][Web of Science][Medline] 10. Shonkoff JP, Phillips DA, eds. From Neurons to Neighborhoods: The Science of Early Child Development. Washington, DC: National Academy Press; 2000. 11. National Institute of Child Health and Human Development Early Child Care Research Network. The relation of child care to cognitive and language development. Child Develop.2000;71:960–980.[CrossRef][Medline] 12. Shore R. Rethinking the Brain: New Insights into Early Development. New York, NY: Families and Work Institute; 1997. 13. Brazelton TB. Touchpoints: Your Childs Emotional and Behavioral Development. Reading, Mass: Addison Wesley; 1992. 14. Zero to Three. Key Findings from a Nationwide Survey Among Parents of Zero-to-Three-Year-Olds. Washington, DC: Zero to Three: National Center for Infants, Toddlers and Families; April 1997. 15. Peter D. Hart Research Associates, Inc. Report on Survey Findings Conducted for the California Children and Families Commission and Team GILD. Washington DC: Peter D. Hart Research Associates, Inc; June 2001. 16. Layzer JI, Goodman BD, Bernstein L, Price C. National Evaluation of Family Support Programs Volume A: The Meta-Analysis Final Report. Cambridge, Mass: ABT Associates; 2001. 17. Brooks-Gunn J, Berlin LJ, Fuligni AS. Early childhood intervention programs: what about the family? In: Shonkoff JP, Meisels SJ, eds. Handbook of Early Childhood Intervention. 2nd ed. Cambridge, England: Cambridge University Press; 2000: 549–558. 18. Cassel MM, Jackson C, Cheuvront B. Health communication on the Internet: an effective channel for health behavior change? J Health Commun; 1988;3: 71–79. 19. Baum A. Behavioral and psychosocial interventions to modify pathophysiology and disease course. In: Smedley B, Syme SL, eds. Promoting Health: Intervention Strategies from Social and Behavioral Research. Washington, DC: National Academy Press; 2000: 450–488. 20. Freimuth VS. The chronically uninformed: closing the knowledge gap in health. In: Ray EB, Donohew L, eds. Communication in Health Care Contexts: A Systems Perspective. Hillsdale NJ: Lawrence Erlbaum Associates; 1989:168–183. 21. Sokal-Gutierrez K, Neuhauser L, Desai M, Gupte S, Constantine NA, Constantine WL. Literature review for the evaluation of the Kit for New Parents. Technical report submitted to First 5 California Children and Families Commission. Berkeley: University of California, Berkeley; 2003. Available at: http://www.ccfc.ca.gov/kit.htm. Accessed July 16, 2007. 22. Neuhauser L, Kreps GL. Rethinking communication in the e-health era. J Health Psychol. 2003;8:7–23. 23. Stokals D. The social ecological paradigm of wellness promotion. In: Jamner MS, Stokals D, eds. Promoting Human Wellness: New Frontiers for Research, Practice, and Policy. Berkeley and Los Angeles: University of California Press; 2000:21–37. 24. Merriam S, ed. The New Update on Adult Learning Theory. San Francisco, Calif: Jossey-Bass; 2001. 25. Deci EL. Why We Do What We Do: Understanding Self Motivation. New York, NY: Penguin Books; 1995. 26. Rothbart GS, Fine M, Sudman S. On finding and interviewing the needles in the haystack: the use of multiplicity sampling. Public Opin Q.1982;46: 408–421. 27. Frankel M, Frankel L. Fifty years of survey sampling in the US. Public Opin Q.1987;51:127–138.[CrossRef] 28. Faugier J, Sargeant M. Sampling hard to reach populations. J Adv Nurs.1996;26:790–797.[CrossRef][Web of Science] 29. Green M, ed. Bright Futures: Guidelines for Health Supervision for Infants, Children and Adolescents. 2nd ed. Arlington, Va: National Center for Education in Maternal Child Health, Georgetown University; 2000. 30. Green LW, Glasgow RE. Evaluating the relevance, generalization and applicability of research: issues in external validation and translation methodology. Eval Health Prof.2006;29:1–28.[Medline] 31. Bero LA, Grill R, Grimshaw JM, Harvey E, Oxman AD, Thomson MA Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. Br Med J.1988;317:465–468. 32. Becker MH. The tyranny of health promotion. Public Health Rev.1986;14:15–25.[Medline] 33. Smedley B, Syme SL, eds. Promoting Health: Intervention Strategies from Social and Behavioral Research. Washington, DC: National Academy Press; 2000. 34. Hornik RC. Public Health Communication: Evidence for Behavior Change. Mahwah, NJ: Lawrence Erlbaum Associates; 2002. 35. Shadish WR Jr, Cook TD, Campbell DT. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston, Mass: Houghton-Mifflin; 2002.
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||