In public health, a major gap exists between the evidence-based interventions (EBIs), practices, and policies shown to promote health and prevent disease (“what we know works”) and what is actually delivered in real-world settings (“what we do”). Dissemination and implementation (D&I) science seeks to bridge this gap by studying methods and strategies to facilitate adoption, use, and sustainability of EBIs among diverse settings and populations.1 Although recognized as an important domain within D&I science, sustainability has been notably understudied, related in part to the logistical and methodological challenges involved in studying and addressing this complex issue.2,3 We present six recommendations to advance understanding of sustainability to benefit research, policy, and practice.

Within D&I science, tremendous progress has been made to advance measurement, study designs, frameworks, and implementation strategies for EBIs. However, research has largely focused on initial adoption and implementation, and much less is known about sustaining EBIs and policies to provide continued health benefits.2–4 This has been identified by D&I experts as “one of the most significant translational research problems of our time.”1 Enhancing the impact of EBIs requires a commitment to advancing sustainability research.

Historically, it has been challenging to capture sustainability given variability in definition and measurement,2,4 but there is a growing consensus.3 Moore et al. defined sustainability as follows: “(1) after a defined period of time, (2) the program, clinical intervention, and/or implementation strategies continue to be delivered and/or (3) individual behavior change (i.e., clinician, patient) is maintained; (4) the program and individual behavior change may evolve or adapt while (5) continuing to produce benefits for individuals/systems.”5

We argue that it is critical to prioritize sustainability research and offer the following recommendations to advance sustainability research in public health.

1.

Conceptually and operationally specify key dimensions of sustainability. There has been a move away from conceptualizing sustainability as static (e.g., institutionalization) and focusing on it as dynamic (e.g., as the population or context changes and new evidence emerges, adaptation or deimplementation may be warranted).6 Researchers should collaborate with stakeholders to specify, in alignment with established definitions and conceptualizations, key dimensions of sustainability. As highlighted in previous work,2,4 it is critical that researchers move beyond measuring sustainability dichotomously and use multiple sustainability outcomes, including continuing or improving health benefits, maintaining community or organizational partnerships (e.g., coalitions) and capacity, and continuing intervention activities or core components (determined a priori).3 There is debate about the extent to which fidelity to core components should be maintained and what “counts” as the original EBI. Thus, it is critical to determine whether there are sustainability thresholds and to measure EBI adaptations (e.g., which components, why, what impact). Streamlining definitions, dimensions, and measures of sustainability will advance research rigor in this area and facilitate comparisons and generalizations across studies.

2.

Test and evaluate existing sustainability conceptual frameworks. Most sustainability frameworks have not been empirically tested to assess their validity or identify predictors of sustainability.2,3 We encourage working from existing sustainability frameworks to facilitate consistency.3 This will help enable more precise theory-based research on the specific causal mechanisms through which different factors affect sustainability outcomes. Testing these frameworks is complex but will help advance understanding of which factors are most critical, in what combination, and across what contexts. For example, although funding may be critically important for sustainability, it is unclear whether it is sufficient in the absence of other factors or whether other factors (e.g., partnerships) can compensate for limited funding. In addition, frameworks that identify possible determinants of sustainability are often similar to implementation frameworks in structure (e.g., factors are grouped as outer contextual, inner contextual, processes, implementer characteristics, and EBI characteristics).3 Working from existing frameworks can help determine the extent to which commonly identified factors shape both implementation and sustainability.

3.

Use rigorous designs to study sustainability at multiple levels over time. It is important to match sustainability research questions to study designs that are robust and feasible. Much prior work has been in the form of descriptive or qualitative single-site studies, with variable and typically brief time periods for measurement.2,4 We recommend assessing sustainability over multiple time points, via a prospective design, to capture the potential dynamic nature of sustainability.2,3 It is critical to specify units of analysis and measure sustainability at the appropriate levels on the basis of what is meaningful for the EBI or policy (e.g., continued health benefits, delivery of the EBI among implementers, capacity at the organizational or community level). Quantitative surveys can be used to investigate predictors of sustainability in combination with qualitative research among stakeholders to contextualize findings. Given the dynamic nature of sustainability and the potential interplay of multiple factors influencing sustainability, other methods (e.g., systems science approaches, dynamic models, network analysis, time series, survival analysis) may also be useful in capturing this complexity. Multisite studies are strongly encouraged to address concerns about limited power and external validity.

4.

Recognize adaptation as central to sustainability. It is critical that researchers explicitly address, document, and evaluate adaptation as central to studying sustainability2,4 and investigate the impact of planned adaptations on intervention effectiveness and sustainability. The Dynamic Sustainability Framework suggests that continued learning and evaluation in new and changing contexts is an inherent part of studying sustainability.6 Concepts such as flexible fidelity and adaptability acknowledge the importance of differentiating core components from adaptive components, potentially guided by existing program planning models, other forms of “practice-based evidence” (e.g., community-based participatory research), and emerging adaptation models. Researchers might consider comparing sustainability outcomes for EBIs that are adapted and those that are implemented with full fidelity.

5.

Plan for sustainability by developing and testing sustainability strategies. As more evidence becomes available on the factors that influence sustainability, an important new direction is to develop and test sustainability strategies that proactively support the continuation of EBIs and policies. It may be useful for researchers to build on existing taxonomies of implementation strategies that have been developed in the field and determine which strategies are similar or different with respect to sustainability. In addition, emerging sustainability-specific instruments (e.g., the Program Sustainability Assessment Tool) can help identify an organization’s capacity in key areas (e.g., funding). This is an area in which experimental designs, adaptive and factorial trial designs, and hybrid effectiveness–implementation studies may be particularly relevant.

6.

Prioritize the study of policy sustainability. Although some research is beginning to examine policy termination, very little has placed a primary emphasis on policy sustainability. This is an important next step to advance policy D&I research.4 The Dynamic Sustainability Framework has potential for examining processes and outcomes of evidence-based policy-making and policy implementation over time.6 This and other innovations are needed to guide research on how long-term implementation of social and health policies affects diverse communities and population health trends. Ultimately, we must understand whether the sustainability of policies matters when it comes to reducing disparities and promoting health equity. Doing so can demonstrate the value and population return on investment of macro-level structural interventions (systems, policy). Adaptation is also central in studying policy-making innovations. Rather than designing policies ill equipped to respond to unintended consequences and implementation barriers, we must consider adaptive policies that respond to emerging evidence over time.7

We strongly encourage researchers, policymakers, and practitioners to apply these recommendations to advance both knowledge and action related to the sustainability of interventions and policies. Ultimately, these recommendations put into action will make more evident the value of investing in sustainability.

ACKNOWLEDGMENTS

Thank you to Laura Brotzman for her assistance in reviewing and editing this editorial. Matthew Lee is a Rowe Family Fellow through the Lerner Center for Public Health Promotion at Columbia University and a Robert Wood Johnson Foundation Health Policy Research Scholar.

CONFLICTS OF INTEREST

No conflicts of interest.

References

1. Proctor E, Luke D, Calhoun A, et al. Sustainability of evidence-based healthcare: research agenda, methodological advances, and infrastructure support. Implement Sci. 2015;10(1):88. Crossref, MedlineGoogle Scholar
2. Wiltsey Stirman S, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci. 2012;7:17. Crossref, MedlineGoogle Scholar
3. Shelton RC, Cooper BR, Stirman SW. The sustainability of evidence-based interventions and practices in public health and health care. Annu Rev Public Health. 2018;39:5576. Crossref, MedlineGoogle Scholar
4. Scheirer M, Dearing J. An agenda for research on the sustainability of public health programs. Am J Public Health. 2011;101(11):20592067. LinkGoogle Scholar
5. Moore JE, Mascarenhas A, Bain J, Straus SE. Developing a comprehensive definition of sustainability. Implement Sci. 2017;12(1):110. Crossref, MedlineGoogle Scholar
6. Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013;8(1):117. Crossref, MedlineGoogle Scholar
7. Carey G, Crammond B, Malbon E, Carey N. Adaptive policies for reducing inequalities in the social determinants of health. Int J Health Policy Manag. 2015;4(11):763767. Crossref, MedlineGoogle Scholar

Related

No related items

TOOLS

SHARE

ARTICLE CITATION

Rachel C. Shelton, ScD, MPH, and Matthew Lee, MPHBoth authors are with the Department of Sociomedical Sciences, Mailman School of Public Health, Columbia University, New York, NY. Rachel C. Shelton is also a Guest Editor for this supplement issue. “Sustaining Evidence-Based Interventions and Policies: Recent Innovations and Future Directions in Implementation Science”, American Journal of Public Health 109, no. S2 (February 1, 2019): pp. S132-S134.

https://doi.org/10.2105/AJPH.2018.304913

PMID: 30785794