© 2004 American Public Health Association
Ross C. Brownson, Jen Jen Chang, and Amy A. Eyler are with the Department of Community Health and Prevention Research Center, Saint Louis University School of Public Health, St Louis, Mo. Barbara E. Ainsworth and Karen A. Kirtland are with the Prevention Research Center, University of South Carolina School of Public Health, Columbia. Brian E. Saelens is with the Department of Pediatrics, Division of Psychology, Childrens Hospital Medical Center, Cincinnati, Ohio. James F. Sallis is with the Department of Psychology, San Diego State University, San Diego, Calif. Correspondence: Requests for reprints should be sent to Ross C. Brownson, Dept of Community Health and Prevention Research Center, Saint Louis University School of Public Health, Salus Center Room 469, 3545 Lafayette Ave, St Louis, MO 63104 (e-mail: brownson{at}slu.edu).
Objectives. We tested the reliability of 3 instruments that assessed social and physical environments. Methods. We conducted a testretest study among US adults (n = 289). We used telephone survey methods to measure suitableness of the perceived (vs objective) environment for recreational physical activity and nonmotorized transportation. Results. Most questions in our surveys that attempted to measure specific characteristics of the built environment showed moderate to high reliability. Questions about the social environment showed lower reliability than those that assessed the physical environment. Certain blocks of questions appeared to be selectively more reliable for urban or rural respondents. Conclusions. Despite differences in content and in response formats, all 3 surveys showed evidence of reliability, and most items are now ready for use in research and in public health surveillance.
An estimated 200 000 to 300 000 premature deaths occur each year in the United States because of physical inactivity.14 Accordingly, the goal of increasing physical activity is one of 10 "leading indicator" areas within the national health objectives of Healthy People 2010.5 Even with the known health benefits of physical activity, more than one quarter of the American population remains completely inactive, and US trends in activity showed little improvement from 1990 to 1998.6 More than 60% of the worlds population is not physically active enough to achieve health benefits.7 The physical, or built, environment is important in providing cues and opportunities for activity,8 and it is associated with rates of physical activity in intervention studies and in large population-based surveys.9 Support for the importance of the environment is derived from 2 distinct literatures. A review of 19 studies in the physical activity and health literature showed consistent associations of accessibility of recreational facilities, opportunities to be active, and certain aesthetic qualities with physical activity in adults.10 Researchers in the transportation and urban planning fields have examined the relationship between community design variables and walking or cycling for transportation. Fourteen studies have consistently shown that people walk and cycle more when their neighborhoods have higher residential density, a mixture of land uses (e.g., shops are within walking distance of homes), and connected streets (e.g., gridlike pattern instead of many cul-de-sacs).11 Other community design characteristics, such as the condition of sidewalks, the presence of bike paths, street design, traffic volume and speed, and crime, are hypothesized to be related to physical activity12,13 but have not been systematically examined. In addition, rural areas have important differences from urban areas in their activity-related design features1416 and are generally understudied.11 Multiple questionnaires have been developed to assess physical activitymeasurement properties (i.e., reliability/validity) are documented for many of these. For example, Ainsworth et al.17 reported on the measurement properties of 39 questionnaires, and Kriska and Caspersen18 described the validity, reliability, and comprehensiveness of 32 instruments. In contrast, considering the apparent importance of the built environment, there is limited information in the literature on how best to measure various aspects, such as the presence of well-maintained sidewalks or whether shopping venues are within walking distance.19 One method of measuring the perceived physical environment is through population-based surveys and surveillance systems.20 Individual responses from these surveys can be aggregated to identify patterns in important design/neighborhood features (e.g., lack of access to sidewalks in rural areas) and to determine associations between these design features and behavior.2123 As yet, it is unclear whether the objective environment (e.g., actual counts of traffic) or the perceived environment (e.g., an individuals self-reported perception of crime in his/her neighborhood) is more important in explaining physical activity.10,11 As measures of perceived environments are developed, it is important to ensure that they can be administered by multiple modes (e.g., self and interviewer administered) and are reliable for broad populations. Our study reports the results of reliability testing of 3 instruments among urban and rural residents across the United States. A major focus of the instruments tested was the assessment of environmental characteristics that are believed to be related to recreational physical activity and nonmotorized transportation, although some instruments assessed other related variables.
Sampling Plan Data were collected through telephone surveys of people aged 18 years and older who lived in the continental United States. We used a modified version of the Behavioral Risk Factor Surveillance System (BRFSS) sampling plan24,25 in which a random-digit sample was purchased from a database company; 50% of the telephone numbers were from rural areas and 50% were from urban areas. Rural or urban residences were defined by US Census Bureau categories. The Census Bureau classifies as urban all territory, population, and housing units located within an urbanized area or an urban cluster. It delineates urban area and urban cluster boundaries to encompass densely settled territory, which consists of census blocks (e.g., a block bounded by city streets) that have a population density of at least 1000 people per square mile and have surrounding census blocks with an overall density of at least 500 people per square mile. The Census Bureaus classification of rural includes all territory, population, and housing units located outside of urban areas and urban clusters.26 Because the purpose of our study was to determine testretest reliability, respondents who completed the survey were asked if they would be willing to complete the survey again in 7 to 21 days, and they were asked for the most convenient time to call for the resurvey. The second calls were made within the 7-to-21-day time frame, and the survey was readministered. This time frame is often used in testretest studies because it is a long enough period so that respondents are unlikely to remember their answers to the original survey, yet the time frame is short enough so that changes in behavior (e.g., seasonal changes in physical activity) are unlikely to have occurred. Each survey participant was assigned randomly to 1 of the 3 questionnaires.
Questionnaires The San Diego instrument (also called the Neighborhood Environment Walkability Survey). This 98-question instrument was developed by Sallis et al. to determine the perception of neighborhood design features hypothesized to be related to physical activity. The questionnaire includes questions about types of residences (to assess density), proximity of stores and facilities in the neighborhood, perceived access to these places, street characteristics (to assess connectivity), facilities for walking and cycling, neighborhood aesthetics, and safety regarding traffic and crime. The San Diego instrument was originally developed for self-administration and was therefore adapted for telephone administration in our study. A reliability study of a self-administered version of this instrument was completed in San Diego.27 The South Carolina instrument. This 61-question instrument was developed by Ainsworth et al. and includes an assessment of the physical and social environments, including perceptions of the community environment (e.g., whether the neighborhood is pleasant), safety, access to recreation and shopping destinations, and conditions of the neighborhood and facilities. Thirteen items focus on the neighborhood, which is defined as a half-mile radius or a 10-minute walk from the respondents home, and 13 items focus on the community, which is defined as a 10-mile radius or a 20-minute drive from the respondents home. Additional physical activity questions from the BRFSS incorporate an assessment of employment activity as well as moderate and vigorous physical activities and global walking behaviors. This instrument was previously tested for reliability and validity among 1200 adults who lived in Sumter County, SC.28 The St. Louis instrument. This 104-question survey was developed by Brownson et al. to measure physical activity and environmental influences on physical activity across the United States.22,29,30 Several constructs in the St Louis instrument were used to develop and evaluate physical activity interventions in rural settings.14,31 The questionnaire includes a detailed assessment of walking behavior, places to walk, barriers to being physically active, neighborhood infrastructure for walking and cycling, perceptions about places for walking, social assets, social support for physical activity, community assets, policy attitudes, and sedentary behaviors. An earlier version of this instrument was tested for reliability in a US sample of ethnically diverse women aged 40 years and older.32
Data Collection
Analyses
Compared with the overall US population,26 our sample tended to overrepresent females, Whites, and persons who had more than a high school education. Statistically significant differences (P < 0.01) across the 3 samples were present for age group, education level, and employment status (Table 1
For the San Diego questionnaire (Table 2
Results for 19 questions about the community and physical environment are shown for the South Carolina instrument (Table 3
The highest proportion of questions within the St Louis questionnaire (Table 4
There is growing recognition that it is essential to understand, and eventually intervene on, environmental and policy factors if we are to increase population rates of physical activity.9,10,36 To conduct research studies that test these environmental hypotheses, it is essential to improve measurement of environmental variables. There are at least 2 ways in which these environmental factors can be measured. First, unobtrusive indicators or measures are those on which data can be collected without an individuals or communitys awareness.20 They often include examining physical surroundings, archival (public) records, sales records, institutional records, and personal documents, as well as observational measures recorded for specific events.3740 Recently, systematic direct observations of features of the physical environment within communities has been shown to be a useful and reliable method for collecting data.41 Regardless of how data are collected, they can be mapped and analyzed with geographic information system technologies.42,43 The other main source of environmental measures is from survey or surveillance data on individuals that can be aggregated to some larger unit (e.g., zip code) and compared across subgroups. The 3 instruments used in our study are useful for this type of data collection and analysis. Although each instrument was designed for a slightly different purpose, most of the variables were reasonably reliable in a diverse sample of adults. In spite of differences in content and response formats, all 3 surveys showed evidence of reliability, and most items are now ready for use in research and in public health surveillance. Several patterns in our data deserve mention:
When determining which questionnaire scales to use in a particular study, there is a trade-off between the ability to comprehensively measure all domains and the feasibility of collecting data efficiently. It is necessary to match environmental variables with the physical activity outcomes of interest, and very specific hypotheses may need to be developed. For example, walking for transportation is likely to be related to the presence of shops nearby, and walking for recreation may be more related to neighborhood aesthetics. Bicycling is expected to be related to accessibility of cycling facilities, and other types of recreational physical activity may be related to presence, condition, and accessibility of recreational facilities. The 3 questionnaires we evaluated can assess a wide range of environmental variables that allow researchers to test multiple hypotheses. The next research priority is to test hypotheses about the relationship between environmental variables and physical activity. Because it is not clear whether perceived or objectively measured environmental variables provide more explanatory power, the use of triangulationapplying multiple methods of data collection to determine points of concordance or disagreement44,45is recommended. A broad range of populations should be studied for several reasons; for example, children and older adults are likely to do physical activity in different settings. Both cross-sectional and longitudinal studies are needed in multiple settings that range from urban to rural locales. To adequately explain physical activity, researchers should examine the separate and interactive contributions of psychological, social, and environmental variables.12 As consensus is reached on the most important correlates or predictors of physical activity, these variables can be incorporated into national surveillance systems. There are several limitations to our study that deserve mention. We relied on selfreported telephone survey data for which there are several potential biases (e.g., possible underrepresentation of lower socioeconomic status segments of the population).4648 Our questions about the environment were selfreported and did not include separate objective measures that would allow assessment of validity (i.e., presence of some "gold standard"). However, 1 recent study found statistically significant associations between selfreported and objectively measured (with geographic information systems) characteristics of trails that may influence physical activity.43 We do not intend to imply that perceived environment measures are preferred over objective measures. At this early phase in this field of research, it is important to evaluate both perceived and objective measures of the environment as they relate to physical activity. Our response rate for the initial survey was lower than anticipated. However, because our study was not developed to measure prevalence, and because the follow-up response rate was reasonable (64%), our reliability results should not be subject to substantial bias. Other similar reliability studies of questionnaires on physical activity have shown response rates from 13% to 54%,4951 and many reliability studies have relied on convenience samples. Both the length of the survey/completion time and the content areas may be factors in the low baseline response rate. Surveillance of chronic diseases has focused primarily on the diseases themselves until recently, when national systems began tracking behavioral risk factors and changes in preventive health practices.25,5254 Our study suggests that numerous dimensions of the physical environment can be measured reliably with telephone survey methods. Multisite collaborations such as ours allow for the testing of multiple instruments simultaneously. Surveillance systems need to begin capturing key aspects of the physical and social environments in addition to the main focus on the behavior of physical activity.20,54
Our study contributes to the growing understanding about the ability to measure peoples perceptions of their physical and social environments in community settings. These surveys have been shown to be reliable in diverse adult samples and are now available for use in further studies. Additional studies are needed to establish the validity of perceptions about environmental variables that may be related to physical activity.
This project was funded through the Centers for Disease Control and Prevention contract U48/CCU710806 (Centers for Research and Demonstration of Health Promotion and Disease Prevention), the Robert Wood Johnson Foundation (awarded to Saint Louis University), and through National Institutes of Health grant HL67350 (awarded to San Diego State University).
Human Participant Protection
Contributors R. C. Brownson led the development of the St Louis questionnaire, conceptualized the original study, and wrote the article. J. J. Chang assisted in data collection and conducted the analyses. A. A. Eyler helped design the St Louis questionnaire, oversaw data collection, and contributed to writing. B. E. Ainsworth led the development of the South Carolina questionnaire and contributed to writing. K. A. Kirtland assisted with the design of the South Carolina questionnaire and contributed to writing. B. E. Saelens assisted with the development of the San Diego questionnaire, contributed analytic algorithms, and contributed to writing. J. F. Sallis led the development of the San Diego questionnaire and contributed to writing. Accepted for publication March 12, 2003.
1. Hahn RA, Teutsch SM, Rothenberg RB, Marks JS. Excess deaths from nine chronic diseases in the United States, 1986. JAMA. 1990;264:26542659. 2. McGinnis JM. The public health burden of a sedentary lifestyle. Med Sci Sports Exerc. 1992;6(suppl):S196S200.
3. McGinnis JM, Foege WH. Actual causes of death in the United States. JAMA. 1993;270:22072212. 4. Powell KE, Blair SN. The public health burden of sedentary living habits: theoretical but realistic estimates. Med Sci Sports Exerc. 1994;26:851856.[Web of Science][Medline] 5. US Dept of Health and Human Services. Healthy People 2010. Volume II. Conference Edition. Washington, DC: US Dept of Health and Human Services; 2000. 6. Centers for Disease Control and Prevention. Physical activity trendsUnited States, 19901998. MMWR Morb Mortal Wkly Rep. March 9, 2001;50(09):166169. 7. World Health Organization. Promoting Physical Activity for Health and Wellbeing: An Achievable Collective Challenge. Available at: http://www.who.int/hpr/physactiv/publications.htm. Accessed June 29, 2002. 8. Giles-Corti B, Donovan RJ. The relative influence of individual, social and physical environment determinants of physical activity. Soc Sci Med. 2002;54(12):17931812. 9. Sallis JF, Bauman A, Pratt M. Environmental and policy interventions to promote physical activity. Am J Prev Med. 1998;15(4):379397.[Web of Science][Medline] 10. Humpel N, Owen N, Leslie E. Environmental factors associated with adults participation in physical activity. A review. Am J Prev Med. 2002;22(3):188199.[Web of Science][Medline] 11. Saelens BE, Sallis JF, Frank LD. Environmental correlates of walking and cycling: findings from the transportation, urban design and planning literatures. Ann Behav Med. 2002;25:8091. 12. Booth SL, Sallis JF, Ritenbaugh C, et al. Environmental and societal factors affect food choice and physical activity: rationale, influences, and leverage points [discussion]. Nutr Rev. 2001;59(3 Pt 2):S21S39; S57S65.[Web of Science][Medline] 13. Frank L, Engelke P, Schmid T. Health and Community Design. The Impact of the Built Environment on Physical Activity. Washington, DC: Island Press; 2003. 14. Brownson RC, Housemann RA, Brown DR, et al. Promoting physical activity in rural communities: walking trail access, use, and effects. Am J Prev Med. 2000;18:235241.[Web of Science][Medline]
15. Parks SE, Housemann RA, Brownson RC. Differential correlates of physical activity in urban and rural adults of various socioeconomic backgrounds in the United States. J Epidemiol Community Health. 2003;57(1):2935.
16. Wilcox S, Castro C, King AC, Housemann R, Brownson RC. Determinants of leisure time physical activity in rural compared with urban older and ethnically diverse women in the United States. J Epidemiol Community Health. 2000;54:667672. 17. Ainsworth BE, Montoye HJ, Leon AS. Methods of assessing physical activity during leisure and work. In: Bouchard C, Shephard RJ, Stephens T, eds. Physical Activity, Fitness, and Health: International Proceedings and Consensus Statement. Champaign, Ill: Human Kinetics; 1994:146159. 18. Kriska AM, Caspersen CJ. A collection of physical activity questionnaires for health-related research. Med Sci Sports Exerc. 1997;29(suppl):S3S205. 19. Bauman A, Sallis JF, Owen N. Environmental and policy measurement in physical activity research. In: Welk GJ, ed. Physical Activity Assessments for Health-Related Research. Champaign, Ill: Human Kinetics; 2002. 20. Baker EA, Brennan LK, Brownson RC, Housemann RA. Measuring the determinants of physical activity in the community: current and future directions. Res Q Exerc Sport. 2000;71:146158. 21. Bauman A, Smith B, Stoker L, Bellew B, Booth M. Geographical influences upon physical activity participation: evidence of a "coastal effect." Aust N Z J Public Health.1999;23:322344.[Web of Science][Medline]
22. Brownson RC, Baker EA, Housemann RA, Brennan LK, Bacak SJ. Environmental and policy determinants of physical activity in the United States. Am J Public Health. 2001;91(12):19952003. 23. Sallis JF, Hovell MF, Hofstetter CR, et al. Distance between homes and exercise facilities related to frequency of exercise among San Diego residents. Public Health Rep. 1990;105(2):179185.[Web of Science][Medline] 24. Gentry EM, Kalsbeek WD, Hogelin GC, et al. The behavioral risk factor surveys: II. Design, methods, and estimates from combined state data. Am J Prev Med. 1985;1(6):914.[Medline] 25. Remington PL, Smith MY, Williamson DF, Anda RF, Gentry EM, Hogelin GC. Design, characteristics, and usefulness of state-based behavioral risk factor surveillance: 19811987. Public Health Rep. 1988;103:366375.[Web of Science][Medline] 26. US Dept of Commerce. US Census Bureau. Available at: http://www.census.gov. Accessed July 14, 2002.
27. Saelens BE, Sallis JF, Black JB, Chen D. Neighborhood-based differences in physical activity: an environment scale evaluation. Am J Public Health. 2003;93:15521558. 28. Ainsworth BA, Bassett DRJ, Strath SJ, et al. Comparison of three methods of measuring time spent in physical activity. Med Sci Sports Exerc. 2000;32(suppl 9):S457S464.[Web of Science][Medline] 29. Eyler AA, Brownson RC, Donatelle RJ, King AC, Brown D, Sallis JF. Physical activity social support and middle- and older-aged minority women: results from a US survey. Soc Sci Med. 1999;49:781789. 30. King AC, Castro C, Wilcox S, Eyler AA, Sallis JF, Brownson RC. Personal and environmental factors associated with physical inactivity among different racial-ethnic groups of US middle-aged and older-aged women. Health Psychol. 2000;19(4):354364.[Web of Science][Medline] 31. Brownson RC, Schmid TL, King AC, et al. Support for policy interventions to increase physical activity in rural Missouri. Am J Health Promotion. 1998;12(4):263266.[Web of Science][Medline]
32. Brownson RC, Eyler AA, King AC, Shyu Y-L, Brown DR, Homan SM. Reliability of information on physical activity and other chronic disease risk factors among US women aged 40 years or older. Am J Epidemiol. 1999;149:379391. 33. CASRO Task Force on Completion Rates. On the Definitions of Response Rates. Special Report. New York, NY: Council of American Survey Research Organizations; 1982. 34. Fisher RA.Statistical Methods for Research Workers. 14th ed. New York, NY: Hafner; 1973. 35. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159174.[Web of Science][Medline] 36. King AC, Jeffery RW, Fridinger F, et al. Environmental and policy approaches to cardiovascular disease prevention through physical activity: issues and opportunities. Health Educ Q. 1995;22(4):499511.[Web of Science][Medline] 37. Webb EJ, Campbell DT, Schwartz RD, Sechrest L. Unobtrusive Measures: Non-Reactive Research in the Social Sciences. Chicago, Ill: Rand McNally & Co.; 1966. 38. Sechrest L, Phillips M. Unobtrusive measures: an overview. In: Sechrest L, ed. Unobtrusive Measurement Today. San Francisco, Calif: Jossey-Bass; 1979:117. 39. Cheadle A, Wagner E, Koepsell T, et al. Environmental indicators: a tool for evaluating community-based health-promotion programs. Am J Prev Med. 1992;8(6):345350.[Web of Science][Medline]
40. Cheadle A, Sterling TD, Schmid TL, Fawcett SB. Promising community-level indicators for evaluating cardiovascular health-promotion programs. Health Educ Res. 2000;15(1):109116. 41. Pikora T, Bull F, Jamrozik K, Knuiman M, Giles-Corti B, Donovan R. Developing a reliable audit instrument to measure the physical environment for physical activity. Am J Prev Med. 2002;23(3):187.[Web of Science][Medline]
42. Moore DA, Carpenter TE. Spatial analytical methods and geographic information systems: use in health research and epidemiology. Epidemiol Rev. 1999;21(2):143161. 43. Troped PJ, Saunders RP, Pate RR, Reininger B, Ureda JR, Thompson SJ. Associations between self-reported and objective physical environmental factors and use of a community rail-trail. Prev Med. 2001;32(2):191200.[Web of Science][Medline] 44. Denzin NK.The Research Act in Sociology. London, England: Butterworth; 1970. 45. Steckler A, McLeroy KR, Goodman RM, Bird ST, McCormick L. Toward integrating qualitative and quantitative methods: an introduction. Health Educ Q. 1992;19(1):18.[Web of Science][Medline] 46. Marcus AC, Crane LA. Telephone surveys in public health research. Med Care. 1986;24(2):97112.[Web of Science][Medline] 47. Centers for Disease Control and Prevention. Using Chronic Disease Data: A Handbook for Public Health Practitioners. Atlanta, Ga: Centers for Disease Control and Prevention; 1992. 48. Ford ES. Characteristics of survey participants with and without a telephone: findings from the third National Health and Nutrition Examination Survey. J Clin Epidemiol. 1998;51(1):5560.[Web of Science][Medline] 49. Pols MA, Peeters PH, Ocke MC, et al. Relative validity and repeatability of a new questionnaire on physical activity. Prev Med. 1997;26(1):3743.[Web of Science][Medline]
50. Blair SN, Dowda M, Pate RR, et al. Reliability of long-term recall of participation in physical activity by middle-aged men and women. Am J Epidemiol. 1991;133(3):266275.
51. Chasan-Taber L, Erickson JB, McBride JW, Nasca PC, Chasan-Taber S, Freedson PS. Reproducibility of a self-administered lifetime physical activity questionnaire among female college alumnae. Am J Epidemiol. 2002;155(3):282289.
52. Berkelman RL, Buehler JW. Public health surveillance of non-infectious chronic diseases: the potential to detect rapid changes in disease burden. Int J Epidemiol. 1990;19(3):628635. 53. Thacker SB, Stroup DF, Rothenberg RB. Public health surveillance for chronic conditions: a scientific basis for decisions. Stat Med. 1995;14(57):629641.[Web of Science][Medline] 54. Macera CA, Pratt M. Public health surveillance of physical activity. Res Q Exerc Sport. 2000;71(suppl 2):S97S103.[Web of Science][Medline] This article has been cited by other articles:
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||