Abstract
The RE-AIM (reach, effectiveness, adoption, implementation, maintenance) framework, which provides a practical means of evaluating health interventions, has primarily been used in studies focused on changing individual behaviors. Given the importance of the built environment in promoting health, using RE-AIM to evaluate environmental approaches is logical. We discussed the benefits and challenges of applying RE-AIM to evaluate built environment strategies and recommended modest adaptations to the model. We then applied the revised model to 2 prototypical built environment strategies aimed at promoting healthful eating and active living. We offered recommendations for using RE-AIM to plan and implement strategies that maximize reach and sustainability, and provided summary measures that public health professionals, communities, and researchers can use in evaluating built environment interventions.
The RE-AIM (reach, effectiveness, adoption, implementation, maintenance) framework1 was developed to enhance the impact of health promotion interventions by evaluating the dimensions considered most relevant to real-world implementation, such as the capacity to reach underserved populations and to be adopted within diverse settings.2,3 Briefly, the reach dimension of the framework refers to the percentage and characteristics of individuals receiving the intervention; effectiveness refers to the impact of the intervention, including anticipated as well as unanticipated outcomes; adoption concerns the percentage and representativeness of settings that adopt the intervention; implementation refers to the consistency and cost of delivering the intervention; and maintenance refers to long-term sustainability at both the setting and individual levels (see http://www.re-aim.org for more information about the framework.).1,4,5
The RE-AIM model was intended to guide planning and evaluation of evidence-based interventions6,7 that address the different levels of the socioecological model, such as those that target individual health behavior change by increasing intrapersonal, organizational, and community resource support.8 It has been used to evaluate programmatic and policy9 interventions addressing a wide range of health conditions (e.g., diabetes, obesity, and hypertension)10–12 and health behaviors (e.g., physical activity, dietary behaviors, and smoking).13–15
Despite RE-AIM's efficacy as a public health planning and evaluation framework, it has not been formally applied to interventions targeting the social or built (i.e., manmade features of the environment that provide the settings for human activity)16,17 environment. As public health continues to expand its focus beyond surveillance and epidemiology to address root factors affecting community health, we need models that help frame the planning and implementation of multilevel health interventions and guide comprehensive evaluations of the processes, effects, and outcomes18 associated with such interventions. Holistic evaluations of changes in public spaces (e.g., changes in transportation and land use) are critical given the complexity of such changes and their strong potential to positively affect social capital and cohesion or to exacerbate social and health inequities.
Here we focus on applying RE-AIM to built environment interventions, although many of the issues and recommendations are also applicable to social environment interventions, and intended or unintended social consequences of interventions are included within the RE-AIM model. Our specific goals are to provide a rationale for using RE-AIM to plan and evaluate built environment changes that promote health behavior, discuss definitions and measures of the dimensions of RE-AIM and propose adaptations to them, illustrate applications of the dimensions through examples of built environment changes, and establish practical RE-AIM summary measures for built environment interventions.
The increased understanding among behavioral scientists, public health practitioners, and planning experts of the built environment's role in promoting healthy behavior and reducing health risks (e.g., pollution, inactivity, accidents)19 offers an opportunity to use a transdisciplinary approach to addressing major risk factors associated with many of the leading causes of death (e.g., cancer, respiratory and heart diseases, unintentional injuries).20 Furthermore, because emphasizing the physical location where individuals encounter an intervention will influence which populations are reached, how often they are reached, and whether the environmental change has a positive, neutral, or negative effect (e.g., does transit-oriented development21 increase a community's access to desirable retail services or lead to gentrification and displacement of low-income residents?22), strategic selection of the location for the built environment change during the planning stage is critical.
As a result of public concern about obesity and health disparities and the inequitable burden of chronic diseases, especially in poor neighborhoods,23,24 there is social pressure on policymakers to address local inequities related to accessing healthful food and safe physical activity venues.19,25–28 Recommendations for evidence-based environmental changes prioritized by the Centers for Disease Control and Prevention in a recent report addressing obesity29 included improved geographic availability of full-service grocery stores30,31 and farmers' markets32 to ensure communities' access to healthful, affordable foods. The report also recommended increased residential access to nearby (i.e., within a half-mile) public outdoor recreational facilities33 and improved infrastructure (e.g., bike lanes, sidewalks) for active transport.34–37
Successful implementation of such projects could be aided by the use of a planning and evaluation framework that explicitly requires identification of the target population, as well as appropriate settings, institutions, and partners, with the goal of increasing the probability that the project will maximize access and health outcomes in a sustainable way. The RE-AIM framework fits well given its attention to the representativeness of both participants and settings or, in the present case, the intervention's geographic location and the agents involved.1 Application of the model requires knowledge of or collection of data on the target population and the potential settings and organizations (e.g., clinics, worksites, schools) that can implement the intervention. Defining and specifying target populations and institutional “adopters” is less clear, however, in the planning of built environment interventions, as described subsequently.
Jilcott et al.9 offered recommendations for applying RE-AIM to health policies, and their initial set of background questions provided a useful starting point for considering the complexities of built environment interventions intended to change behavior and, by so doing, improve health. These questions (reworded to fit built environment issues) are as follows: Whose health behaviors and health are to be improved? What stakeholders need to be included in the planning process, and what agencies are responsible for approving the environmental change? What agencies are responsible for implementing the change? What agencies are responsible for maintaining the change? What funding needs to be secured to implement and maintain the change?
Applying RE-AIM to evaluate built environment changes is not straightforward for several reasons. Each RE-AIM dimension, although conceptually the same as originally defined by Glasgow et al.,1 requires assessment indicators different from those used for evaluating programs or treatments (Table 1).
TABLE 1 Definitions, Challenges, and Metrics of Applying RE-AIM to Built Environment Projects
| RE-AIM Dimension | Definition | Questions and Challenges | Built Environment–Specific Metricsa |
| Reach | No. of people and percentage of the target population affected and the extent to which the individuals reached are representative and include those most at risk. | If space is redesigned to improve accessibility to the community, how can it be calculated whether people who live and work nearby visit? | Estimate number of people reached based on the population living or working within a specific distance of the change, observe and describe visitors at varied times and days of the week, conduct intercept surveys to determine whether people visiting are from the surrounding neighborhoods, use setting-level proxies such as sales receipt volumes or daily usage, and track changes in reach over time. |
| Effectiveness | A measure of effects on health behaviors, including positive, negative, and unanticipated consequences. | Users of the space may demonstrate both desirable and undesirable behaviors, so does the change produce equal effects across subgroups? | Observe and map population behaviors (both positive and negative) occurring before and after the environmental change, document how robust or consistent the outcomes are across key subgroups, assess any unanticipated consequences (including both positive and negative behaviors), enlist adopters/agencies to help describe and quantify behaviors (e.g., food purchases, park attendance) before and after the change, identify public data sources (e.g., crime or accident data) that can be used to quantify changes. |
| Adoption (inclusion and approval) | No. and percentage of settings participating, and the extent to which the settings selected are representative of settings that the target population will use or visit. | Because settings do not “adopt” built environment changes, who are the adopters (e.g., target population, business owners, city council)? | Assess the representativeness of those making decisions with regard to selection of the setting and design of the change; assess the inclusion of those needed to approve the project (city council, neighborhood association), implement the change (public works), and maintain the space (parks and recreation, police); planning stage—calculate the percentage of key stakeholders involved; implementation stage (qualitative)—evaluate whether the agency or group approving the change is viewed positively, and if it is able to maintain the change? |
| Implementation (installation) | Level of adherence to implementation principles or guidelines, the extent to which all versus selected elements are implemented, and the cost. | When the environmental project is completed, does it meet established design principles or plans for attracting visitors from the target population? Are the ongoing costs sustainable? | Are standards or guidelines for implementing or installing the built environment change followed or only partially implemented? Do barriers or deterrents to use remain (address via intercept surveys and observations)? Where and when is the cost of change incurred? Who pays? Document other changes needed to support the project (e.g., law enforcement or traffic engineering). |
| Maintenance (sustainability) | Individual level—individuals continue to exhibit the desired health behavior changes | What agencies or groups are in a position to monitor individual behavior and setting maintenance over time? Is there a policy or program in place that will support ongoing improvements and maintenance to sustain use and address evolving issues not initially anticipated? Are there adequate resources and plans for covering ongoing maintenance costs? | Individual level—the long-term impact on health behaviors as novelty erodes (6 months or more after installation). |
| Setting level—change is maintained and deterioration or development of new barriers to use is prevented or mitigated. | Setting level—the approving/enforcing agency continues to provide upkeep and necessary support (e.g., lighting, police patrol), and budget and staff are allocated each year to ensure that space is maintained. |
For example, reach (absolute number, percentage, and representativeness of those affected by the environmental change) is challenging to calculate when considering potential and actual users of public space. To paraphrase a line from the movie Field of Dreams (Universal Pictures, 1989), “If you build it, will they come?” is the reach question relevant for built environment interventions. For example, if a neighborhood makes environmental improvements such as sidewalk and bike lane additions and traffic calming initiatives (e.g., stop signs, curb extenders) to increase active (i.e., pedestrian and bike) transportation, who is being reached? Identifying the target population that could potentially use the sidewalks and bike lanes—in this case, residents of the neighborhood where improvements were made—and then capturing who actually uses them requires collecting data on the target population and then conducting observational or survey research before and after installation.
In instances in which geographic boundaries for the designated target population are not clearly defined, researchers will often use buffer zones, or circular areas, around the specific geographic location approximating the catchment area for expected users.41 The size of the buffer zone may vary according to the ubiquity of the destination (e.g., coffee shop vs specialty food store), its importance to a community's daily life, and the location of the intended users (target population). Thus, a coffee shop's buffer zone may be a few blocks, and a specialty food store's buffer zone may be the entire city.
Continuing with the active transportation example, assessing effectiveness may require measuring whether there are different effects across different subgroups9 (e.g., did the installation of bike lanes, sidewalks, and destinations increase active transportation or reduce the number of car trips among those who will most benefit, and were there any unintended negative outcomes,42 including social justice issues?). Thus, measuring whether a built environment change results in health behavior changes among the members of the target population may require data collection methods that have origins in urban planning and marketing research (as opposed to health research methods, wherein known participants volunteer for programs).43
Some methods that have been used with success in such instances include systematic observational approaches such as behavior mapping, in which the number and characteristics of people using the space and the way they use the space are sampled and recorded at various times44; telephone or door-to-door surveys in which household data are gathered45; and street-intercept survey techniques to collect data from potential users.46 Street-intercept surveys have been shown to be more successful than are telephone interviews in capturing a representative sample of the target population within specific geographic boundaries, especially in the case of low-income and culturally diverse populations, among whom face-to-face methods may promote trust.
Adoption from the RE-AIM perspective has traditionally been defined with respect to the settings (e.g., worksites, clinics, schools) in which programmatic or policy changes take place. Similar to reach at the individual level, adoption can be used to evaluate the characteristics of institutions or organizations that adopt or decline the intervention and whether those that adopt it are representative of all eligible or invited institutions or organizations. Identification of potential adopters is less concrete in the case of changes in the built environment, in which key adopters may change over the course of the project.
The process of changing public spaces to promote health behaviors involves a geographic component (i.e., where the project is built influences the populations that are reached) as well as a multiagency component (i.e., specific agencies have the authority to approve the change being made and the know-how and resources to implement and maintain it over time). Policymakers, planners, traffic engineers, law enforcement personnel, residents, and other stakeholders should be involved in site selection as well as the design and evaluation of the project.47 Diffusion of the intervention to other settings may be less important than ensuring widespread community access and attending to unforeseen consequences for adjacent property owners (e.g., a park that increases the number of cars parked on adjacent streets).
The implementation dimension of RE-AIM has traditionally been used to examine the consistency with which an intervention is delivered and the cost of such delivery. Although the evidence supporting built environment changes is increasing,48–51 there has been a lack of research on the ways in which study findings are translated and adapted to ensure that such changes produce desired improvements in health behaviors. In addition, the agents involved in implementing changes may vary according to the stage of implementation, and not all of these agents may be well versed in best practices related to health promotion.
The complexity of implementing wide-scale changes in public spaces was underscored when the Americans with Disabilities Act (104 Stat 327) was implemented in 1990.52 Cities needed to evaluate and communicate compliance standards across agencies and organizations with differing goals and priorities (e.g., government, planners, engineers, transportation, law enforcement, and people with disabilities). Published best practices for changing public spaces, including design principles and research-based evaluation guidelines, are available23,44,53–55; however, there is a need for assessment of the degree of fidelity to these principles, how they are translated in diverse situations, and the subsequent effects on health behaviors when they are only partially followed.
Maintenance includes assessment at both the individual level (i.e., are desired health behaviors sustained?) and the setting level (i.e., do adopting institutions integrate the intervention into regular practices and provide staffing and budgetary support?). In the case of built environment interventions, individual maintenance implies continued use of the space by a high percentage of the target population. Setting maintenance often requires the involvement of community members and public entities to preserve the quality of the space and to prevent other changes that create obstacles and diminish its use by the target population.
Economic pressures on communities, changes in neighborhood demographics, crime, and upkeep are factors that influence continued use of public spaces. Thus, the RE-AIM framework needs to be modified to include diverse indicators such as factors that influence construction and maintenance costs.56 Engaging citizen groups, law enforcement officials, and local government representatives at the initiation of the project; providing a mechanism for collecting systematic cost (e.g., annual upkeep) and usage data; and creating long-term plans to monitor environmental or social changes that may threaten continued appropriate use of the space will help ensure its maintenance.
One advantage of built environment interventions is that they can influence the behavior of large and diverse segments of the population. In addition, once built, such projects are likely to be sustained, although maintenance will be required to retain their intended use. Because construction costs for built environment changes can be high, careful planning that includes the intended users as well as those who will need to approve, construct, and maintain the environmental change is essential.
Each dimension of the RE-AIM framework can be used as a blueprint for planning (Table 2). We recommend planning for evaluations of the intervention from the start,7 including identifying metrics readily available from public sources (e.g., crime and accident statistics), and identifying means by which behaviors can be tracked routinely and efficiently (e.g., store and restaurant register receipts, electronic benefit transfer machines at farmers' markets that allow use of food stamps, and routine customer surveys). Training community groups in the use of qualitative methods, such as systematic observation and walkability audit tools44,57 may encourage the involvement of the community in maintaining the environmental change (Table 1). Finally, identifying milestones up front and establishing ways to frequently report progress and celebrate the achievement of milestones are important for projects that may require months or even years to complete.58
TABLE 2 Application of RE-AIM to the 2 Example Built Environment Strategies
| RE-AIM Dimension | Planning Stage | Farmers' Market | Complete Streets |
| Reach | Identify target population whose health or health behavior could benefit. | ||
| Numerator | Postimplementation observation of no. of shoppers at various times and days and assessment of their demographics (age, gender, race). | Postimplementation observation of no. of visitors per day arriving at the retail district at various times and assessment of their demographics (age, gender, race). | |
| Denominator | All residents residing within 1 mi of the market. | All residents within 3 mi of the surrounding retail district. | |
| Effectiveness | Identify desired health or behavioral outcomes and estimate probability that target population will engage in those behaviors if the environmental change is made (with consideration of cultural norms, convenience, and alternatives). | Average number and percentage of market customers per day who purchase fruits and vegetables, changes over time with regard to volume of fruit and vegetable sales (as a proxy for direct measurement of consumer eating behaviors), increased perceived access to fruits and vegetables among patrons of the market (market survey), and data collected via intercept surveys of residents living within 1 mi of the market. | Average number of visitors per day who walk, bike, or take public transportation to commute to the retail district; increases in observed foot and bike traffic; increase in public transportation volume to the revitalized retail district; decreases in observed motor vehicle traffic to destinations within the revitalized district; and decreased accidents involving pedestrians or bicyclists and cars. |
| Adoption (inclusion/approval) | Identify and include key stakeholders to ensure that the project is designed to fit the target population and that all the organizations needed to approve the project (city council, neighborhood association), implement the change (public works), and maintain the space (parks and recreation, police) are involved. | ||
| Numerator | Planning stage—residents/target population are included to allow an understanding of preferences and to address potential barriers. | Planning stage—agencies, organizations, and residents/target population are involved in planning the project. | |
| Approval stage—agencies and organizations approve use of the space for a farmers' market. Implementation stage—vendors accept invitation to sell their produce at the market. | Approval stage—agencies and organizations necessary for approving the project see it through. | ||
| Implementation stage—extent to which project is approved as planned and resources are approved to support implementation of the change. | |||
| Denominator | Planning stage—comprehensive list of agencies and organizations are invited to participate in establishing the farmers' market. | Planning stage—comprehensive list of agencies and organizations are invited to participate in the design of the revitalization project. | |
| Approval stage—the goal is to ensure that the correct agencies and stakeholders are involved. | Approval stage—the goal is to ensure that the correct agencies and stakeholders are involved. | ||
| Implementation | Identify standards and guidelines, including universal design and smart growth principles; perform health impact assessment to minimize unintended adverse consequences; anticipate barriers and delays that might affect costs and timelines; and provide a forum for airing community concerns and unanticipated backlash. | Postimplementation—the extent to which setting for farmers' market addresses transportation, pricing, and other barriers to food access for target population; the variety of fruits and vegetables and local foods are consistently available; linguistic and culturally competent customer service in place; and that food is handled safely. | Planning stage—setting for project addresses transportation, type of retail, pricing, and other barriers for both target population and retailers. |
| Implementation stage—necessary supports and resources are consistently provided (e.g., clean-up, police patrols, lighting). | |||
| Maintenance | Identify and include in planning all agencies or groups that may be in a position to monitor usage of the space and its maintenance over time. | ||
| Individual | Postimplementation—customers continue to purchase or increase demand for fruits and vegetables. | Postimplementation—customers continue to visit central retail district via foot, bike, or public transport. | |
| Setting | Postimplementation—farmers' market vendors continue to offer fresh fruits and vegetables and increase or maintain profits. | Postimplementation—retail establishments continue to thrive and attract local customers, and annual budget and staff resources continue to be allocated to maintaining the environmental change. |
To demonstrate how the revised RE-AIM framework can be applied to built environment interventions, we described 2 exemplars based on composites of actual community strategies employed in Colorado during the past 3 years (see http://www.livewellcolorado.org). These exemplars are also summarized in Table 2. The example strategies have been endorsed by the Centers for Disease Control and Prevention as ways to combat obesity29 and are representative of built environment strategies now being implemented by communities across the country. The first strategy, “farmers' market,” addresses barriers related to fruit and vegetable access and consumption. The second strategy, “complete streets,” encourages active transport.
A coalition was formed to address obesity issues in a low-income community. Plans were drafted for an evening farmers' market that would be situated in a centrally located church parking lot to address the lack of a grocery store within the predominantly Latino neighborhood. The coalition defined the denominator for calculating reach as the estimated number of households within 1 mile (1.6 km) of the proposed market site, given that households beyond 1 mile tended to be composed of non-Latino Caucasians, a group that was not the primary focus of the market. Plans to track customers included observing the number of visitors to the market (i.e., to estimate the numerator for reach) and using vendor sales information to determine the volume of fruit and vegetable purchases.
To ensure that the farmers' market would be approved and would appeal to the target population, the following partners were included in the planning process: the neighborhood association, the police department, the parent–teacher organization, local family farmers and ranchers, the church priest, and a nearby Latino social organization. The implementation was assessed in both quantitative and qualitative terms. Quantitative data included number of vendors per week and variety of fruits and vegetables offered. Qualitative data included information gathered from a focus group of community members formed to help provide an understanding of food needs, pricing, and the optimal location for the market. Maintenance plans were not discussed, although a potential future need to relocate the market was raised as a result of concerns about liability from the church and complaints about increased traffic from some of the neighbors.
This strategy applied smart growth principles54 related to land use planning and transit-oriented development to revitalize a city's central retail district and encourage commerce in a historic low-income area. Plans called for the surrounding street network to be retrofitted according to complete streets guidelines, which promote roadway designs that increase safety and accessibility for users (e.g., bicyclists, pedestrians, transit users, and motorists) of all ages and abilities. Complete streets designs typically include sidewalks wide enough to accommodate wheelchair users, bike lanes, and traffic calming elements (e.g., reduced speed limits). Because public transportation improvements associated with the revitalized space served residents within 3 miles, the target population was defined by the city as those living within a 3-mile buffer. A desired behavioral outcome was an increase in active transportation behaviors among individuals commuting to the revitalized district.
Bicyclist, pedestrian, and transportation data were assessed through periodic observations and intercept surveys conducted within the district. Adopters included in the planning, approval, and design of the project were government officials (city manager, public works personnel, and traffic engineering personnel), representatives of businesses (chamber of commerce, grocery stores, and restaurants), and resident groups (bicycle organizations, seniors groups, and neighborhood associations). The coalition charged with implementing the project assessed fidelity to smart growth principles by evaluating the city's master plan and recommending ways to adapt it to meet land use guidelines. Maintenance plans included ongoing tracking of perceived barriers and business satisfaction and profitability; this information was collected through town hall meetings hosted by the coalition and the city council. The ultimate goal was to add language to the city's master plan to ensure application of smart growth and complete streets principles to all future land use projects (Table 2).
Given the diversity of environmental approaches and the potential cost and time commitment associated with projects that involve changes to the built environment, one practical use of RE-AIM is to compare projects whose target populations and target behaviors differ. Three straightforward ways of providing a “bottom-line” summary score are potentially appropriate.
First, scores on each RE-AIM dimension can be set as 0.0 to 1.0 (or 0% to 100%), reflecting an estimated proportion (or percentage) of the criteria met. Scores can then be averaged across all dimensions. In Table 3, the complete streets and farmers' market examples just described are used to assign a score to each RE-AIM dimension, as well as a summary score that allows for comparison between these 2 very different projects.
TABLE 3 Scores on RE-AIM Dimensions for Farmers' Market and Complete Streets Built Environment Strategies
| RE-AIM Dimension | Farmers' Market | Complete Streets |
| Reach | ||
| Numerator | Observed average no. of daily shoppers (100) | Observed average no. of daily visitors (2000) |
| Denominator | Residents within a 1-mi buffer of the market (1000) | Residents within a 3-mi buffer of the district (7000) |
| Scorea | 0.10 (100/1000) | 0.28 (2000/7000) |
| Effectiveness | ||
| Description | Average no. of customers per day who purchase fruits and vegetables (60) | Average no. of visitors per day who walk, bike, or take public transportation to commute to the retail district (100) |
| Scorea | 0.60 (60/100) | 0.05 (100/2000) |
| Adoption (inclusion/approval) | ||
| Numerator | No. of agencies and organizations accepting the invitation and participating (9) | No. of agencies and organizations accepting the invitation and participating (20) |
| Denominator | Total no. of agencies and organizations invited to participate in establishing the farmers' market (10) | Total no. of agencies and organizations invited to participate in establishing the revitalized district (25) |
| Scorea | 0.90 (9/10) | 0.80 (20/25) |
| Implementation | ||
| Description | The community identified a location for the market on a side street that can be closed off to traffic and is adjacent to retail and restaurants, which will increase visibility and be mutually beneficial to the market and the adjacent businesses | The project addressed public transportation, sidewalks, and bike lanes between low-income neighborhoods and the redeveloped space; the community was unsuccessful in attracting a grocery store to address a major need; and barriers to implementing traffic calming measures are being addressed |
| Scoreb | 0.75 | 0.50 |
| Maintenance (projected) | ||
| Description | No plans have been discussed for sustaining the farmers' market as a permanent structure | Commitment to continuously improve the district by adding green areas and expanding the pedestrian and biking infrastructure has been written into the city's 10-year budget and master plan |
| Scorec | 0.0 | 1.0 |
| RE-AIM summary score (average across dimensions) | 0.47 | 0.53 |
a Scores range from 0.0 to 1.0 (or 0% to 100%), reflecting an estimated proportion (or percentage) of the criteria met.
b Score is a subjective rating ranging from 0.0 (no criteria met) to 1.0 (all criteria met), of how closely the actual implementation matched the planned criteria.
c Score is a subjective rating ranging from 0.0 (unlikely) to 1.0 (very likely), of the likelihood that built environment changes (and resulting reach and effectiveness) will be sustained.
We calculated reach by observing and counting the number of visitors to either the farmers' market or the revitalized retail district and dividing this value by the number of people residing in the predesignated geographic area. We calculated effectiveness as the proportion of visitors engaged in the desired health behavior (i.e., purchasing fruit and vegetables or actively commuting to the retail district).
Adoption was calculated as the percentage of invited agencies and individuals participating in the planning and approval process (including those involved in implementing and maintaining the change). We rated implementation using an anchored scale based on the extent to which implementation deviated from preestablished criteria (e.g., for the farmers' market, adherence to the planned number of vendors and the diversity and cost of food and, for the complete streets example, adherence to established design guidelines).
Finally, we estimated maintenance using a similarly constructed anchored scale based on the likelihood that the environmental change (and resulting reach and effectiveness) would be sustained (and measured subsequently via periodic observations). The summary scores for the 2 projects were close (0.47 and 0.53) despite wide variation on the separate dimensions. This suggests that important information, such as whether a sustainability plan has been discussed, may be obscured if summary scores alone are used.
A second approach is to form a composite score by multiplying the 0.0 to 1.0 reach score by the 0.0 to 1.0 effectiveness score (R × E). The scores for the 2 examples (0.06 and 0.014) mask large differences in effectiveness (0.60 and 0.05; Table 3). Although the R × E score is relatively simple to calculate (because it eliminates the less straightforward adoption, implementation, and maintenance ratings), it removes those aspects of RE-AIM that are most likely to affect reach and sustainability (i.e., the participation of adopters with the authority to approve the project and the likelihood that it will be maintained).
A third and related index recommended by Glasgow4 is the “efficiency index,” in which the cost of the built intervention is divided by the R × E metric. Including cost information may appeal to decision makers and investors tasked with allocating scarce resources. However, estimating true costs may not be practical for large, multifaceted infrastructure changes, particularly given that large capital investments may be offset by civic and social benefits (e.g., increased commerce and jobs, traffic and crime safety), in addition to improved health behaviors. One way to address such situations would be to parse out the costs most directly related to the targeted health behavior, such as the costs of walking and biking infrastructure improvements.
In general, the efficiency index method may be best suited to projects (e.g., community gardens, trails, or playgrounds) in which the direct costs of implementation and maintenance are closely related to the R × E score. The issue of who collects, analyzes, and summarizes these data for decision makers is a complex one whose detailed discussion is beyond the scope of this article. We recommend that a neutral party, such as a state health representative or an independent evaluation firm, conduct these analyses.
Presenting RE-AIM data in a way that resonates with the general public is another complex issue. Use of graphic representations, such as charts that illustrate the relative strength of each dimension, may better facilitate communication and decision making than use of numerical scores.59
The 5 RE-AIM dimensions, with some modification of definitions, seem to be applicable to built environment interventions and provide added value given their usefulness in anticipating impact, planning for sustainability, and addressing unexpected or adverse consequences. The greatest modification with respect to both planning and evaluation was associated with the adoption dimension. Because built environment interventions do not involve acceptance by a specified set of institutions or organizations such as schools or worksites, identifying the participation and characteristics of adopters is less central than is identifying and including those with the authority to approve the project and those involved in its implementation, enforcement, or maintenance. Although the specific adopters may change as the project moves from planning and design to implementation, anticipating and including all critical stakeholders and end users during the planning stage will reduce the likelihood of costly delays, revisions, or cancellations.
An advantage of using RE-AIM is that it ties together key concepts that can be used in both planning and evaluating built environment projects. The model can be applied to various scenarios to compare and make decisions regarding how a proposed project's location affects reach, which agencies and organizations need to be brought to the table, and the relative costs of different project scenarios. A disadvantage of using RE-AIM is its conceptual nature; that is, the framework does not provide guidelines on what specific data to collect, how to collect these data, or how to monitor “exposure” to the project and its impact on behaviors over time. Thus, community groups may find it challenging to address all 5 dimensions of RE-AIM in a practical manner.
On a related note, measurement recommendations for built environment interventions may demand expertise that is beyond the means of many community organizations. Recruiting individuals and measuring their health and health behaviors longitudinally is often not an option. However, changes in behavior can be adequately captured by community volunteers trained in using qualitative techniques such as systematic observation and behavior mapping.60
In addition, if diverse partners are involved in the design and implementation phases, data already collected for other purposes (e.g., sales receipts, crime and accident statistics) can also be used to quantify reach and effectiveness. Even if it is not possible to measure all aspects of the RE-AIM framework for a given built environment intervention, consideration of all dimensions in the planning stage, including qualitative assessments of relevant metrics (e.g., the characteristics of who is, and who is not, participating and benefiting61), can enhance the success of the intervention.
A follow-up question that emerges from this application of RE-AIM is whether a particular RE-AIM dimension should be weighted more heavily than others or whether a summary score can suffice. The answer to this question depends on the situation. As our examples showed, comparing an average summary score across RE-AIM dimensions may obscure important elements such as inclusion of key stakeholders or plans for maintaining the change. Also, a summary score is not meaningful in and of itself because it has no referents or norms. Comparing the scores for each dimension across strategies may be the most useful method and may also be easier to communicate visually to constituents.
With an increasing number of communities using socioecological approaches, including policies, programs, and environmental changes, to promote health behavior, the ability to apply a single framework such as RE-AIM across different types of interventions is advantageous.5,9 We propose furthering the framework's usefulness by adapting and applying the 5 RE-AIM criteria to the planning of built environment interventions with maximal reach and effectiveness.
Acknowledgments
This work was supported in part by the Kaiser Permanente Colorado Community Benefit Department and the National Institute of Diabetes and Digestive and Kidney Diseases (grant 2 R01 DK035524-21).
We thank Jessica Osbourne from the Colorado Physical Activity and Nutrition Program for providing valuable input on evaluating active community environments and LiveWell Colorado for inspiring the exemplars.
Human Participant Protection
Because of the conceptual nature of this study, no protocol approval was needed.
