Objectives. Awareness of and support for systems thinking and modeling in the public health field are growing, yet there are many practical challenges to implementation. We sought to identify and describe these challenges from the perspectives of practicing public health professionals.

Methods. A systems-based methodology, concept mapping, was used in a study of 133 participants from 2 systems-based public health initiatives (the Initiative for the Study and Implementation of Systems and the Syndemics Prevention Network). This method identified 100 key challenges to implementation of systems thinking and modeling in public health work.

Results. The project resulted in a map identifying 8 categories of challenges and the dynamic interactions among them.

Conclusions. Implementation by public health professionals of the 8 simple rules we derived from the clusters in the map identified here will help to address challenges and improve the organization of systems that protect the public’s health.

Modern public health practice encompasses a complex, loosely coupled system1 of actors including governmental entities at the international, national, regional, and local levels; a diverse conglomeration of nongovernmental organizations (such as foundations, advocacy and special interest groups, coalitions and partnerships, for-profit and nonprofit medical systems, and businesses); and citizens in the public at large. The broad array of threats to well-being, ranging from obesity and tobacco use to violence and infectious diseases, can be most aptly portrayed from a complex and adaptive system perspective.

Systems thinking and modeling are broad classes of intellectual endeavors that are being incorporated increasingly into contemporary public health. Research has proven both the general potential of systems thinking214 and applications in specific areas.1528 Empirical studies related to complex systems have appeared of late in notable medical journals, including the Journal of the American Medical Association, Lancet, and the New England Journal of Medicine.2935 The authors of an Institute of Medicine report, Crossing the Quality Chasm: A New Health System for the 21st Century,11(pp8–9) used a systems perspective to delineate 10 “simple rules to guide the redesign of the health care system” and described the entire health care system as a complex adaptive system:

A health care system can be defined as a set of connected or interdependent parts or agents—including caregivers and patients—bound by a common purpose and acting on their knowledge. Health care is complex because of the great number of interconnections within and among small care systems.11(p64)

Systems thinking encompasses and is consonant with ecological models3638 familiar to public health practitioners, including the ideas of human ecology, population health, and the social determinants of public health. But it goes beyond these models, incorporating advances over the past decades, particularly in fields such as system dynamics and complexity theory.

It is relatively easy to identify examples of public health issues that can be understood accurately only by examining the complex and dynamic part-and-whole interactions that make up systems. For instance, consider the area of tobacco control. At the policy level, it is reasonable to argue that the 1964 surgeon general’s report on smoking39 had profound effects on the policy debate with consequences for smoking prevalence and consumption to this day. The report itself was the product of a complex series of events that led to its production. In turn, it set off a cascade of events and changes. It is virtually impossible to determine the effects of that important event in isolation, as a part that is separable from the whole.

For instance, the report was most likely an important catalyst in creating a public policy climate that enabled the litigation that led to the Tobacco Settlement Agreement several decades later, to increased taxation of cigarettes by states, to legal restrictions on smoking in public places, and to tobacco counteradvertising. By the same token, the report may have led to unanticipated “negative” consequences by spurring the tobacco industry to adapt its product, marketing, lobbying, and public relations and perhaps indirectly contributed to the creation of front groups and covert efforts to undermine tobacco control research.40

There are countless other examples. Smokers may react to fears about the harmful effects of smoking by switching to so-called “light” cigarettes (i.e., low tar and nicotine formulations as determined by machine smoking tests). Yet, despite their manufacturers’ claims, “light” cigarettes may actually increase the prevalence of a more lethal type of lung cancer, probably because of the way their ventilated filters alter the physical act of smoking.41,42 Or consider the way the industry responds to youth advertising restrictions. Billboard advertising was outlawed by the Master Settlement Agreement. In response, the tobacco companies increased advertising and promotion in retail stores, effectively saturating the retail environment with product images.43 Thus, banning billboard advertising has led to even more children being exposed to protobacco messages as they go about their daily lives.

Even the cigarette itself is a complex system, a highly engineered product designed for extreme elasticity of delivery.44 And tobacco control as a public endeavor can be viewed as a set of systems challenges regarding how we might best balance the complex configurations of individuals, government agencies, and organizations that are engaged in fighting tobacco use. In tobacco control, as in all other areas of public health work, systems problems are legion, and the need for systems thinking and modeling is ubiquitous.

Despite the growing cognizance of and support for “systems thinking” in public health, implementation of effective systems approaches remains challenging.12,13,45,46 We sought to identify and describe the challenges that must be addressed by public health leaders in implementing effective systems-based approaches. Here we address 3 major topics. First, we discuss systems thinking and modeling and their most recent developments. Second, we provide an overview of public health initiatives that are exploring and using systems thinking and modeling, particularly the Initiative for the Study and Implementation of Systems (ISIS).1 Finally, we present the results of an initial empirical study in which we used a systems-based method (concept mapping) with a self-selected group of public health professionals in an attempt to identify the challenges facing those who support systems thinking and modeling in public health.

Systems thinking is a general conceptual orientation concerned with the interrelationships between parts and their relationships to a functioning whole, often understood within the context of an even greater whole. It is ancient in origin and familiar to us all, but it is also something very modern. We engage in a type of systems thinking in our everyday lives when we contemplate the complex interactions of our relationships with families and friends, when we organize in our communities or workplaces, and when we try to puzzle out the dynamics of the economy. But systems thinking also encompasses some of the most advanced and sophisticated recent work in contemporary science. Systems modeling is a methodological tradition that involves the use of formal models or simulations as explicit aids to increase our understanding of complex systems and improve the effectiveness of our actions within them. Computational modeling and simulation, as a complement to experimentation and theory, are hallmarks of recent systems thinking and the systems sciences.47

The depth and breadth of systems science can be bewildering, particularly as one first is introduced to its underlying principles and formulations. Consider just a few of the topics associated with contemporary systems thinking: causal feedback48; stock–flow structures and open and closed systems49; centralized, decentralized, heterarchical, hierarchical, and self-organizing systems50,51; autopoiesis5254; nonlinear systems and chaos55; complex adaptive systems5659; boundary conditions, scaling, power laws, phase transitions, universality, and renormalization55,60; silo effects61; emergence62,63; cellular automata64; fractal self-similarity46; general systems theory65; cybernetics66,67; control theory68; information theory69; computational simulation47,70; decision and game theory71; system dynamics7277; evolution, biology, and ecology7881; small world phenomena8284; and set, graph, and network theory.53,79,80,82,83,85,86

The vastness of the literature alone can be overwhelming, and it is not easily summarized. We offer 2 organizing ideas (dynamics and complexity) and 2 influential metaphors (mechanical and biological) that can help us understand this daunting array. In addition, we consider 2 common misconceptions about systems thinking that are important to an understanding of these ideas and metaphors.

Dynamics and Complexity

Whether a system settles into a state of equilibrium, changes in repeating cycles, or changes in even more complicated ways, a common theme is change. A field called dynamics, with its own rich history dating back to Isaac Newton, provides a vocabulary and a methodology for understanding these changes.55 Terms such as the “butterfly effect”87,88 and the “tipping point,”89 which are now part of the public vernacular, have their source in the study of chaotic systems. In chaos, “a deterministic system exhibits aperiodic behavior that depends sensitively on the initial conditions, thereby rendering long-term prediction impossible.”55(p3) This concept is just one of the many useful ideas in the field of dynamics.


Most systems in the public health arena are complex in that they consist of many interacting stakeholders with often different and competing interests. Agents in these networks must constantly adapt to the actions of others and to a changing environment that is in turn affected by the actions of the agents themselves. Such systems are not controlled centrally; they are self-organizing. Complexity theory, or the study of complex adaptive systems,57,59 focuses on understanding systems of this type. Most definitions of such systems include some notion of the relationship between the emergent or unpredictable behavior of a system and autonomous agents self-organizing by simple rules. For example, one description of a complex adaptive system is independent variables following simple local rules leading to emergent complexity,90(p17) and another description suggests that a working definition of a complex system is “one whose properties are not fully explained by an understanding of its component parts.”91(px) Simple rules, networks of adaptive agents, feedback, self-organization, and emergence are hallmarks of complex adaptive systems.

A good example of a complex adaptive system that is familiar to virtually everyone is the emergency medical services (EMS) system that comes into play in some way in virtually every medical or health emergency that an individual experiences. It illustrates well how a complex adaptive system is made up of various independent agents following simple rules and interacting locally with other independent agents in the system. In the EMS, each agent has a role to play and a simple set of rules to follow. These roles and rules are graduated in a linked chain from first responder to emergency room doctor to rehabilitation specialist.

For a citizen in first responder role, the rules typically instruct the agent to activate the EMS system by “calling 911” and then administer basic care until professional help arrives. Next in the chain, the professional rescuer, firefighter, or emergency medical technician (EMT) plays a different role and follows a different rule set with primary responsibility for advanced field care and transport. In turn, the EMT initiates the involvement of the next set of agents by communicating with the emergency room. If needed, the emergency room connects with the agents of hospital inpatient care, and these agents in turn join subsequently with posthospital rehabilitation.

Today, except in rural and more isolated communities in the United States, we take the EMS system for granted, and it is difficult to remember a time when it did not exist to help individuals in an emergency. But the recent horrific hurricane in New Orleans and surrounding areas reminds us that a comparable regional-, state-, or federal-level emergency disaster response system has not yet evolved sufficient capacity to deal with large-scale catastrophes.

Although the vast array of participating governmental, medical, public health, and nonprofit organizations involved in the EMS system have coordinated their efforts in various ways for several decades,92 it is not a centrally and hierarchically controlled system. Tens of thousands of people are trained in various roles by a wide variety of entities ranging from medical and public health schools to nonprofit organizations such as the Red Cross.93 Although individual agents may be aware of the existence of the broader system, their training concentrates on their specific part in it and how it connects with adjacent parts—the role they play and the rules they follow—leading to a system that can adapt to a great variety of individual medical emergencies.

Mechanical and Biological Systems

In addition to the 2 broad organizing ideas just discussed, it is useful to distinguish 2 metaphors for systems that are both prevalent and influential78,80,81,94,95: systems as mechanical and systems as biological. In the mechanical metaphor, systems are construed as machines made up of parts or subsystems that interact in complex ways to produce certain characteristic behaviors. In the biological metaphor, systems are living and evolving entities, in turn often composed of subsystems that are themselves evolving and adapting to the environment. Studies of systems, influenced by both types of metaphors, have led to many significant scientific discoveries. The biological metaphor appears to be increasingly prevalent, but some systems, even complex and nonlinear systems, behave more like machines than like biological organisms. There are also mixes of the metaphors, such as in bioengineering, in which cells are thought of as tiny biological machines.96 Although systems thinking inherently is not either mechanistic or biological, particular phenomena may be aptly characterized by one or the other metaphor or by some combination of the 2 metaphors.


Finally, 2 misconceptions about the systems approach need to be addressed. First, systems thinking is not a rejection of traditional scientific views that are linear, reductionist, mechanistic, or atomistic and framed by mechanical, spatial, or temporal metaphors.65,79,80,90,97,98 In a study focusing on the enablers of, barriers to, and precursors to systems thinking, Davidz et al. noted: “It is important to remember the embedded nature of systems. What is considered a holistic, systems view is considered a reductionist view when the boundaries of the system are redrawn.”99(pp1–2) Contrary to popular claims, systems thinking encompasses and includes reductionism; it does not replace or reject it.

A second misconception is that systems thinking lacks scientific rigor.1 This fallacy probably stems from popular literature portraying systems thinking as “soft” or in opposition to scientific or analytic thinking. According to Von Bertalanffy, systems epistemology “shares the same scientific attitude” with scientific or analytic thinking.65(p423) Systems thinkers achieve a holistic view of complex phenomena80,90,94,95,97,100 precisely because they approach the study of relationships as a distinct and legitimate form of inquiry.101 Consequently, most of the techniques used for systems thinking and modeling are rooted in mathematics as well as the physical, biological, and social sciences, and they have been used to conduct some of the most rigorous and sophisticated experiments ever devised.

The field of public health is adapting to the evolution of systems thinking and its accompanying modeling approaches. More scholars are studying and writing on the topic, more research is emphasizing a systems view, and ambitious attempts are under way to focus practitioners on improving overall system performance.102 A good example is ISIS, a project of the Tobacco Control Research Branch of the US National Cancer Institute. The purpose of ISIS is to explore whether systems thinking can serve as a foundation for more effective public health efforts to combat tobacco use, particularly in the face of countervailing forces such as the efforts of the tobacco industry.

ISIS brought together a transdisciplinary group of leaders in fields such as system dynamics, network analysis, knowledge management and informatics, tobacco control, management sciences, and health policy to develop a framework for systems action. This network of thinkers considered some core questions: How can the flow in both directions between research and practice be optimized? How can systems structure and function be best characterized to be useful to the public health community? Which approaches can be used for better understanding and optimization of networks? Through which strategies do information and knowledge become the currency for change?

The ISIS team concluded that systems thinking in public health cannot be encompassed by a single discipline or even a single approach to “systems thinking” (e.g., system dynamics); instead, it consists of a trans-disciplinary integration of public health approaches that strive to understand and reconcile linear and nonlinear, qualitative and quantitative, and reductionist and holist thinking and methods into a federation of approaches to systems thinking and modeling.103

The ISIS team also recognized that the complexity and breadth of systems thinking may be dismissed as being too complicated. If the public health community, from clinicians to policymakers, is to value systems thinking as a guiding approach, it must be practical, manageable, and accessible. Toward that end, ISIS supported efforts that resulted in practical examples of systems ideas in public health contexts: development of a system dynamics model for characterizing the complex state of tobacco use and its control, creation of a map of the social network of tobacco control organizations, a concept mapping project to promote better understanding of how to integrate research and practice, and a knowledge management map to guide the use of information in tobacco control. In addition, ISIS supported actual networks for global tobacco research and reduction of harm from tobacco and produced a monograph1 summarizing the 2-year effort and serving as a road map for future approaches to systems thinking in public health.

ISIS is hardly the only effort to assess the value of systems thinking and modeling in public health work. The Syndemics Prevention Network,104109 supported by the Centers for Disease Control and Prevention, studies how recognition of mutually reinforcing health problems (substance abuse, violence, AIDS) expands the conceptual, methodological, and moral dimensions of public health work. This group seeks to learn how innovative ways of thinking about health as a system—along with the methodological techniques they inspire—lead to more effective and ethical action.110112 Examples of other relevant efforts include a major Institute of Medicine report,11 the Community–University Partnerships Initiative113 sponsored by the W. K. Kellogg Foundation, the community-based participatory research efforts sponsored jointly by the Agency for Healthcare Research and Quality and the W. K. Kellogg Foundation,114 the Community–Campus Partnerships for Health,115 the efforts of the Institute for Healthcare Improvement,116 the Healthy Cities movement,117 the Partnership for the Public’s Health,118 the Turning Point Program,119 and the efforts of the World Health Organization’s Commission on Social Determinants of Health.120

If systems thinking and modeling are to be successfully integrated into public health practice, the associated practical dilemmas and challenges need to be identified and addressed. To this end, we conducted an initial study with systems thinkers (individuals involved in ISIS and the Syndemics Prevention Network) in which we used a systems methodology—structured concept mapping—to describe the challenges of systems thinking and modeling in public health.


Invitations to participate in the project were sent to 359 individuals who were on the e-mail distribution lists of the Syndemics Prevention Network and the ISIS project as of December 15, 2004. The Syndemics Prevention Network included more than 300 members from 11 countries, and the ISIS list included approximately 60 participants from Canada, Australia, and the United States. Most of these individuals are practicing public health professionals (e.g., researchers, program managers, policymakers); however, a significant percentage also identified themselves as having special expertise and training in systems thinking and modeling. (More details on the methods and results of this project are available from the first author on request.)

Concept Mapping

Concept mapping is a systems method that enables a group to describe its ideas on any topic121 and represent these ideas visually in a map. The general procedure for concept mapping has been described in detail by Trochim.121 The method has been used in a wide range of fields,122 including health services research123,124 and public health.125

To accomplish this project, participants brainstormed or free listed a large set of statements addressing an agreed-upon focus statement for the project. All participation was via the Web. Each generated statement completed the following focus prompt: “One specific practical challenge that needs to be addressed to encourage and support effective systems thinking and modeling in public health work is. . . .” The group generated 318 statements that were synthesized and edited to a final set of 100 statements used during the remainder of the project.

Each participant was invited to sort these statements into groups of similar ones and rate each statement in terms of its relative importance as a challenge that must be addressed to encourage and support systems thinking and modeling in public health. These data were assessed in a sequence of multivariate statistical analyses that included multidimensional scaling126,127 and hierarchical cluster analysis128,129 methods. The resulting maps showed each of the statements, with more similar ones located nearer each other, and illustrated how the statements were grouped into clusters. Initial interpretation of the maps followed the general process described by Trochim.121

In the brainstorming phase, 133 participants visited the Web page. In the sorting and rating phase, 56 participants completed the sorting task, and 54 completed the ratings. The stress value is the usual statistic reported in multidimensional scaling analyses to indicate goodness of fit, with a lower stress value indicating a better fit. In a study of the reliability of concept mapping, Trochim130 reported an average stress value across 33 projects of 0.285, with a range from 0.155 to 0.352. The stress value in the present analysis was 0.300. An 8-cluster solution was selected as the one that preserved the most detail and yielded substantively interpretable clusters of statements. The key materials used in the interpretation of the results included the statements produced through brainstorming, listed by cluster; the point map showing each statement; and the cluster map showing the 8-cluster solution.

Figure 1 shows the final map with the cluster labels arrived at through a consensus process that involved a subgroup of the participants. Table 1 lists, for each cluster, the 3 challenges to implementing systems thinking that were assigned the highest average importance ratings.

Here we describe each cluster briefly, moving from highest to lowest in cluster average importance rating as listed in Table 1. The cluster labeled “Expand Cross-Category Funding” consisted of 10 statements primarily related to financial issues. These statements challenged traditional funding categories and explicitly encouraged a more integrative, systems-based view of financing. The cluster labeled “Support Dynamic and Diverse Networks” contained 8 statements about encouraging networks, collaborations, teams, and partnerships that span traditional disciplines and boundaries and value diverse perspectives.

The “Use Systems Measures and Models” cluster (10 statements) related to the creation of methods and tools for systems-based measurement and data collection, use and evaluation of systems methods and models, and development of systems tools and approaches. “Inspire Integrative Learning” included the most statements (23), reflecting the fact that the topics contained in this cluster were most frequently brainstormed by participants. Examples are training and education, dissemination and diffusion of systems thinking and approaches, use of interactive and Web-based resources, and a broad emphasis on understanding the area of systems thinking and modeling.

“Explore Systems Paradigms and Perspectives” included 15 statements addressing values and implications of a potential paradigm change involved in systems thinking and the influence of such thinking on the perspectives people bring to public health work. The 11 statements in the “Show Potential of Systems Approaches” cluster described the value and impact of systems thinking and modeling. “Foster Systems Planning and Evaluation” (9 statements) suggested the integration of systems thinking and modeling into traditional planning and evaluation. Finally, “Utilize Systems Incentives” (14 statements) emphasized the need to address political and social factors that influence use of systems thinking, including issues of bureaucracy, people’s fears (e.g., apprehensions about job loss, job difficulty, or change), jurisdictional conflicts among organizations, and factors in academic environments that limit adoption or use of systems approaches.

The map developed in this project depicts 8 distinct clusters of practical challenges that need to be addressed to encourage and support effective systems thinking and modeling in public health. The label for each cluster of challenges was carefully reverse checked with the cluster statements to ensure that they were adequately represented. In many cases, single words in the cluster name represent entire fields of inquiry such as integrative learning, networks, planning, and evaluation, and each term in the cluster name modifies the others. Thus, the modifying effect of “systems” on “planning” in the Foster Systems Planning and Evaluation cluster is not merely semantic; it reflects a perspective that differs dramatically from traditional planning in which planning precedes action and evaluation follows in a rational and linear fashion. Instead, the statements suggest 2 challenges to systems thinking in public health: that planning and evaluation are not yet sufficiently systemic and that planning should be continuous and adaptive, with constant feedback among planning, action, and evaluation.

The cluster in the center of the map (Figure 1), Show Potential of Systems Approaches, can be considered central both graphically and conceptually. In systems jargon, it might be thought of as a central attractor in the dynamic cycle of the overall map. As the clusters in the exterior ring interact in various ways, their activity converges on the central cluster, where assessment and dissemination (e.g., research into what works) are represented; the disseminated content in turn excites new activity in the exterior “ring” of clusters.

The Figure 1 map can be viewed through the lens of systems thinking itself. Earlier, we offered 2 broad organizing ideas that help to make sense of the often bewildering and diverse landscape of systems thinking: Systems are dynamic and systems are complex. From the perspective of dynamic systems, the map can be interpreted as a collection of interacting “cluster agents” each affecting the other. From the perspective of complex systems, clusters can be viewed as simple rules that encourage emergence and adaptation. Each is considered briefly.

In terms of dynamics, the clusters can be thought of as interacting conceptual or thematic agents that can influence other cluster agents when coupled. Each cluster resembles a semi-autonomous agent functioning in a highly integrated system. When one cluster interacts with another, they affect each other. For example, when interacting with Explore Systems Paradigms and Perspectives, the Inspire Integrative Learning cluster takes on a slightly different meaning than when it is considered in connection with Use Systems Measurement and Modeling. In the first case, learning is centered on systems paradigms and perspectives at a conceptual or epistemological level. In the second case, learning is more formal and adaptive; systems-based measurement and modeling are used to inform decisionmaking and action.

In another example, the Support Dynamic and Diverse Networks cluster interacts with Expand Cross-Category Funding to create diverse and dynamic networks of monetary flows and feedback loops. In contrast, when Show Potential of Systems Approaches, which contains statements relating to research, researchers, and best practices, is combined with Support Dynamic and Diverse Networks, innovative notions about supporting diverse networks of researchers and practitioners emerge. The cluster-as-agent interpretation generates 56 possible pairings. Increasing in complexity, the clusters can be thought of as combinations interacting in triples or larger groupings. This interpretation is dynamic because, with just 8 simple clusters, a diverse and complex number of possible innovative ideas can result. For example, the map might lead us to ask what a dynamic network of best practices in systems thinking and modeling would look like, with differing combinations and juxtapositions of clusters suggesting specific novel practices.

With respect to complexity, the map can be viewed from the perspective of the theory of complex adaptive systems, which holds that simple local rules lead to emergent complexity. Each cluster label is worded as a simple object-action “rule” that can be used by public health professionals to encourage and manage a complex and adaptive system. Each label completes the prompt that was the focus of this project and constitutes a practical challenge to be addressed. The especially intriguing hypothesis from a systems point of view is that if multiple independent agents (individuals and organizations) follow the 8 rules represented by the cluster names and are provided with appropriate feedback about what is occurring as a result, systems-oriented organizations and networks will naturally emerge in the public health system.

This study involved several limitations. The sample consisted of self-selected participants invited from the e-mail lists of 2 prominent initiatives for systems thinking in public health, so the generalizability of our results is limited to similar groups. However, generalizability was not a major purpose of this study, which was closer in nature to a focus group or expert panel than to a sample survey. In addition, this was an unfunded project accomplished under considerable time constraints that may have limited participation. Brainstorming occurred during late December and early January and conflicted with busy holiday seasons. The entire project was accomplished within 2 months.

Because of time and technology constraints, it was not feasible to engage a broad cross section of the entire participant group in an interpretation of the map results or to engage the broader public health community directly. To help address this limitation, we created a Web site (available at: http://www.greaterthanthesum.net) to distribute detailed results of this project and encourage further interaction and discussion of these issues and the resulting concept map. Many of the limitations of this study would be best ameliorated through independent replications and studies involving the use of alternative methods for identifying practical challenges to support effective systems thinking and modeling in public health work.

This project is significant for several reasons. First, the concept map results provide a basis for subsequent action by various actors in public health agencies (and beyond). Any group or organization can examine the clusters, or the statements contained within them, and determine the degree to which they constitute or suggest actions they might take to address practical challenges to systems thinking and modeling in public health. Second, the results clearly point to a need for education and learning in systems thinking and modeling, including everything from potential curriculum topics (participatory methods, nonlinear dynamics, simulations) to enhancing learning capacity (development of centers and of electronic materials).

Third, the map provides a conceptual model that serves as a basis for developing role play simulations that can enable public health organizations to try out different actions and explore in a controlled context how their adoption could potentially change the outcomes of public health efforts. The map constitutes a structure and a set of rules that different agents can use depending on their roles in the public health system. Such simulations would make it possible for the field of public health to learn more dynamically about the effects of systems thinking and modeling and to anticipate better where funding resources might be most effectively allocated to encourage systems evolution.

Finally, a significant aspect of having done the concept mapping is that it helps to establish an appropriate and widely shared boundary for thinking about the many issues in question, particularly at a relatively early stage in our efforts to build institutional support for a systems orientation. The number of clusters and the diversity of their themes now serve as a check against planning and capacity-building initiatives that might otherwise be scoped too narrowly or too abstractly. Equipped with the practical insights from the concept mapping, we can now embark on more productive multistakeholder dialogues and think together about precisely what kinds of supports are needed if an authentic system orientation is to thrive in public health agencies.

This study provides an initial identification of the challenges, a map that can be used to navigate them, and a set of 8 simple rules for facing these challenges and moving toward effective implementation of systems approaches in public health efforts. It has practical utility in terms of helping organizational practitioners, researchers, policymakers, and the general public face these challenges. The results reported here, major reports such as Crossing the Quality Chasm, and initiatives such as the Syndemics Prevention Network and ISIS can be viewed from a systems perspective as dynamically interacting components in the growing awareness and support of systems thinking and modeling in public health, and they offer the promise that more effective public health systems will consequently emerge.

TABLE 1— Participants’ Ratings of Practical Challenges to Effective Systems Thinking and Modeling in Public Health
TABLE 1— Participants’ Ratings of Practical Challenges to Effective Systems Thinking and Modeling in Public Health
Cluster and ChallengeRating
Expand Cross-Category Funding (average: 3.86)
    Identify and develop funding sources that will encourage systems approaches to public health4.13
    Develop funding for demonstration projects that validate systems approaches to public health4.09
    Increase funding for transdisciplinary and interagency collaborative projects with a systems focus4.06
Support Dynamic and Diverse Networks (average: 3.50)
    Encourage collaborations between researchers and practitioners by clarifying the link of systems thinking and modeling to everyday practice in public health3.74
    Use participatory action approaches to partner with communities to co-define public health problems, challenges, needs, assets, and resources3.74
    Sustain multidisciplinary teams from a broad range of health and science backgrounds and thinking (e.g., deductive/inductive, research/practice)3.65
Use Systems Measures and Models (average: 3.39)
    Development of methods and tools that encourage systems approaches in research and evaluation3.93
    Develop new evaluation approaches that will help demonstrate the value of systems approaches, such as syndemics, in public health3.78
    Identify priority public health issues (e.g., tobacco, HIV, obesity) as possible tipping points for early examples of systems thinking and modeling3.46
Inspire Integrative Learning (average: 3.38)
    Identify and disseminate examples of “best practices” or “what works” in systems thinking inside and outside of public health3.93
    A critical mass of practitioners who are able to approach public health from a nonlinear perspective3.72
    Training and education in systems research techniques for public health professionals3.70
Explore Systems Paradigms and Perspectives (average: 3.26)
    Recognize the importance of a systems paradigm to public health (e.g., ecological, systemic, holistic, participatory, multidimensional, constructivist, adaptive, complex, and nonlinear frameworks)4.00
    Recognize the limitations of the dominant paradigm in public health (e.g., linear causality, reductionism, positivism, objectivism, the medical model, logic models, program-focused, disease-focused frameworks)3.57
    Getting government and public health officials at state and federal levels to appreciate the value of community-based approaches and highlight citizenship and local governance in public health3.54
Show Potential of Systems Approaches (average: 3.25)
    Rigorous research that demonstrates the value of systems thinking, methods, approaches, and research3.76
    Set priorities by analyzing system-wide issues rather than simply ranking by disease burden or attributable risk3.74
    Connect systems thinking and modeling to the series of recent Institute of Medicine reports (e.g., bridging the quality chasm, reducing health care errors, eliminating health and health care disparities)3.63
Foster Systems Planning and Evaluation (average: 3.20)
    Integrate organizational planning and evaluation functions around a systems approach3.72
    Apply systems thinking to physical and mental health problems affecting individuals, families, and communities throughout the human life cycle3.65
    Develop a unified mission/vision across sectors (e.g., public health, education, public safety, behavioral health) and between layers (e.g., national, state, community) regarding the systems approach3.50
Utilize System Incentives (average: 3.05)
    Provide incentives that encourage systems thinking3.56
    Reduce the overemphasis on immediate positive program impacts by taking a longer term view3.54
    Address issues of politics and bureaucracy that hinder systems thinking (e.g., politicians’ ignorance of how their systems work, public employee unions that avoid employee accountability, civil service systems that encourage stagnation)3.35

Note. Shown are the 3 challenges in each cluster with the highest average importance ratings.


1. Committee on Assuring the Health of the Public in the 21st Century. The Future of the Public’s Health in the 21st Century. Washington, DC: Institute of Medicine, 2002. Google Scholar
2. Green LW, McGinnis M, Phillips TS, Devereaux M, Montes H. Strategies for Promoting Health for Specific Populations. Washington, DC: US Office of Health Information and Health Promotion; 1981. DHHS publication PHS 81 50169. Google Scholar
3. Green LW. Should health education abandon attitude change strategies: perspectives from recent research. Health Educ Monogr. 1970;1:24–48. CrossrefGoogle Scholar
4. Anderson RA, Issel LM, McDaniel RR. Nursing homes as complex adaptive systems. Nurs Res. 2003; 52:12–21. Crossref, MedlineGoogle Scholar
5. Anderson RA, McDaniel RRJ. Managing health care organizations: where professionalism meets complexity science. Health Care Manage Rev. 2000;25: 83–92. Crossref, MedlineGoogle Scholar
6. Anonymous. Ways of understanding. BMJ [serial online]. 2001:323. Available at: http://bmj.bmjjournals.com/cgi/content/full/323/7315/0/a. Accessed November 27, 2005. Google Scholar
7. Suchman AL. Linearity, complexity and well-being. Med Encounter. 2002;16:17–19. Google Scholar
8. Ashmos DP, Duchon D, McDaniel R. Physicians and decisions: a simple rule for increasing connections in hospitals. Health Care Manage Rev. 2000;25: 109–115. Crossref, MedlineGoogle Scholar
9. Begun JW, Zimmerman B, Dooley K. Health care organizations as complex adaptive systems. In: Mick SS, Wyltenbach ME, eds. Advances in Health Care Organization Theory. San Francisco, Calif: Jossey-Bass; 2002: 253–288. Google Scholar
10. Glouberman S, Zimmerman B. Complicated and Complex Systems: What Would Successful Reform of Medicare Look Like? Toronto, Ontario, Canada: Commission on the Future of Health Care in Canada; 2002. Google Scholar
11. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001. Google Scholar
12. Plsek PE, Greenhalgh T. The challenge of complexity in health care. BMJ. 2001;323:625–628. Crossref, MedlineGoogle Scholar
13. Plsek PE, Wilson T. Complexity, leadership, and management in healthcare organisations. BMJ. 2001; 323:746–749. Crossref, MedlineGoogle Scholar
14. McDaniel RR, Driebe DJ, Blair J, Fottler M, Savage G, eds. Complexity Science and Health Care Management. Amsterdam, the Netherlands: Elsevier Science Ltd; 2001. Google Scholar
15. Ashmos DP, Duchon D, McDaniel R. Physicians and decisions: a simple rule for increasing connections in hospitals. Health Care Manage Rev. 2000;25:109–115. Crossref, MedlineGoogle Scholar
16. Baskin K, Goldstein J, Lindberg C. Merging, de-merging, and emerging at Deaconess Billings Clinic. Physician Executive. 2000;26(3):20–25. MedlineGoogle Scholar
17. Dardik I. The origin of disease and health, heart waves: the single solution to heart rate variability and ischemic preconditioning. Frontier Perspect. 1997;6: 18–32. Google Scholar
18. Dekker J, Crow RS, Folsom AR, et al. Low heart rate variability in a 2-minute rhythm strip predicts risk of coronary heart disease and mortality from several causes. Circulation. 2000;102:1239–1244. Crossref, MedlineGoogle Scholar
19. Frumkin H. Beyond toxicity: human health and the natural environment. Am J Prev Med. 2001;30: 234–244. CrossrefGoogle Scholar
20. Godin PJ, Buchman TG. Uncoupling biological oscillators: a complementary hypothesis concerning multiple organ dysfunction syndrome. Crit Care Med. 1996; 24:1107–1116. Crossref, MedlineGoogle Scholar
21. Goldberger AL. Fractal variability versus pathologic periodicity: complexity loss and stereotypy in disease. Perspect Biol Med. 1997;40:543–561. Crossref, MedlineGoogle Scholar
22. Goodwin JS. Chaos and the limits of modern medicine. JAMA. 1997;278:1399–1400. Crossref, MedlineGoogle Scholar
23. Lindberg C, Herzog A, Merry M, Goldstein J. Life at the edge of chaos—health care applications of complexity science. Physician Executive. 1998;24(1):6–20. MedlineGoogle Scholar
24. McDaniel RD, Dean J. Complexity science and health care management. In: Blair J, Fottler M, Savage G, eds. Advances in Health Care Management. Vol. 2. Greenwich, Conn: JAI Press; 2001:11–36. Google Scholar
25. Miller W. Understanding change in primary care practice using complexity theory. J Fam Pract. 1998; 46:369–376. MedlineGoogle Scholar
26. Seely A, Nicolas V. Multiple organ dysfunction syndrome: exploring the paradigm of complex nonlinear systems. Crit Care Med. 2000;28:2193–2200. Crossref, MedlineGoogle Scholar
27. Weibel ER. Fractal geometry: a design principle for living organisms. Am J Physiol. 1991;261: 361–369. Google Scholar
28. Zimmerman B, Lindberg C, Plsek P. Edgeware: Lessons From Complexity Science for Health Care Leaders. Dallas, Tex: VHA Inc; 2001. Google Scholar
29. Cole C. Heart-rate recovery immediately after exercise as a predictor of mortality. N Engl J Med. 1999; 341:1351–1357. Crossref, MedlineGoogle Scholar
30. Ivanov PC, Amaral LAN, Goldberger AL, et al. Multifractality in human heartbeat dynamics. Nature. 1999;399:461–465. Crossref, MedlineGoogle Scholar
31. Pennisi E. In nature, animals that stop and start win the race. Science. 2000;288:83–85. Crossref, MedlineGoogle Scholar
32. Schmidt G. Heart-rate turbulence after ventricular premature beats as a predictor of mortality after acute myocardial infarction. Lancet. 1999;353:1390–1396. Crossref, MedlineGoogle Scholar
33. Lipsitz LA, Goldberger AL. Loss of ‘complexity’ and aging: potential applications of fractals and chaos theory to senescence. JAMA. 1992;267:1806–1809. Crossref, MedlineGoogle Scholar
34. Goldberger AL, Rigney DR, West BJ. Chaos and fractals in human physiology. Sci Am. 1990;262: 42–49. Crossref, MedlineGoogle Scholar
35. Goldberger AL. Non-linear dynamics for clinicians: chaos theory, fractals, and complexity at the bedside. Lancet. 1996;347:1312–1314. Crossref, MedlineGoogle Scholar
36. Syme SL, Balfour JL. Social determinants of disease. In: Wallace RB, ed. Public Health and Preventive Medicine. 14th ed. Stamford, Conn: Appleton & Lange; 1998:795. Google Scholar
37. Healthy People 2010. Washington, DC: US Dept of Health and Human Services; 2000. Google Scholar
38. Stokols D, Pelletier KR, Fielding JE. The ecology of work and health: research and policy directions for the promotion for employee health. Health Educ Q. 1996;23:137–158. Crossref, MedlineGoogle Scholar
39. Smoking and Health: Report of the Advisory Committee to the Surgeon General of the Public Health Service. Washington, DC: US Dept of Health, Education, and Welfare; 1964. Google Scholar
40. Trochim W, Stillman F, Clark P, Schmitt C. Development of a model of the tobacco industry’s interference with tobacco control programs. Tob Control. 2003;12:140–147. Crossref, MedlineGoogle Scholar
41. Hoffmann D, Hoffmann I. The changing cigarette, 1950–1995. J Toxicol Environ Health. 1997;50: 307–364. Crossref, MedlineGoogle Scholar
42. Wynder EL, Hoffmann D. Smoking and lung cancer: scientific challenges and opportunities. Cancer Res. 1994;54:5284–5295. MedlineGoogle Scholar
43. Wakefield MA, Terry YM, Chaloupka FJ, et al. Changes at the point-of-sale for tobacco following the 1999 tobacco billboard ban. Am J Public Health. 2002; 92:937–939. LinkGoogle Scholar
44. Clark PI, Djordjevic MJ. The role of smoking topography in assessing human smoking and its utility for informing machine-smoking protocols. Report to the World Health Organization, Scientific Advisory Council on Tobacco Regulation. 2003:1–23 Google Scholar
45. Stacey RD, Griffin D, Shaw P. Complexity and Management: Fad or Radical Challenge to Systems Thinking? New York, NY: Routledge; 2000. Google Scholar
46. McKelvey B. Complexity theory in organization science: seizing the promise or becoming a fad? Emergence. 1999;1:5–32. CrossrefGoogle Scholar
47. Gilbert GN, Troitzsch KG. Simulation for the Social Scientist. Philadelphia, Pa: Open University Press; 1999. Google Scholar
48. Richardson GP. Feedback Thought in Social Science and Systems Theory. Philadelphia, Pa: University of Pennsylvania Press; 1991. Google Scholar
49. Sterman JD. Business Dynamics: Systems Thinking and Modeling for a Complex World. New York, NY: McGraw-Hill/Irwin; 2000. Google Scholar
50. Kauffman SA. At Home in the Universe: The Search for Laws of Self-Organization and Complexity. New York, NY: Oxford University Press Inc; 1995. Google Scholar
51. Kauffman SA. The Origins of Order: Self-Organization and Selection in Evolution. New York, NY: Oxford University Press Inc; 1993. Google Scholar
52. Maturana HR. Autopoiesis: reproduction, heredity and evolution. In: Zeleny M, ed. Autopoiesis, Dissipative Structures and Spontaneous Social Order. Boulder, Colo: Westview Press; 1981:48–80. Google Scholar
53. Maturana HR, Varela FJ. Autopoiesis and Cognition: The Realization of the Living. Dordrecht, the Netherlands: D. Reidel Publishing Co; 1980. Google Scholar
54. Maturana HR, Varela FJ. The Tree of Knowledge: The Biological Roots of Human Understanding. Boston, Mass: Shambala; 1992. Google Scholar
55. Strogatz SH. Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry, and Engineering. Reading, Mass: Addison-Wesley; 1994. Google Scholar
56. Gell-Mann M. Let’s call it plectics. Complexity. 1995;1:5. CrossrefGoogle Scholar
57. Gell-Mann M. The Quark and the Jaguar: Adventures in the Simple and the Complex. London, England: Abacus; 2003. Google Scholar
58. Holland J. Hidden Order: How Adaptation Builds Complexity. Reading, Mass: Perseus Books; 1995. Google Scholar
59. Waldrop MM. Complexity: The Emerging Science at the Edge of Order and Chaos. New York, NY: Simon & Schuster; 1992. Google Scholar
60. Yeomans JM. Statistical Mechanics of Phase Transitions. Oxford, England: Oxford University Press Inc; 1992. Google Scholar
61. Côté M. A matter of trust and respect. CA Magazine [serial online]. March 2002. Available at: http://www.camagazine.com/index.cfm/ci_id/6798/la_id/1.htm. Accessed November 27, 2005. Google Scholar
62. Holland JH. Emergence: From Chaos to Order. Reading, Mass: Addison-Wesley; 1998. Google Scholar
63. Johnson S. Emergence: The Connected Lives of Ants, Brains, Cities, and Software. New York, NY: Scribner; 2001. Google Scholar
64. Wolfram S. A New Kind of Science. Champaign, Ill: Wolfram Media; 2002. Google Scholar
65. Von Bertalanffy L. The history and status of general systems. In: Klir J, ed. Trends in General Systems Theory. New York, NY: Wiley-Interscience; 1972:423. Google Scholar
66. Francois C. International Encyclopedia of Systems and Cybernetics. 2nd ed. Munich, Germany: KG Saur; 2004. Google Scholar
67. Young JF. Cybernetics. New York, NY: Elsevier Publishing Co; 1969. Google Scholar
68. Sontag ED. Mathematical Control Theory: Deterministic Finite Dimensional Systems. 2nd ed. New York, NY: Springer; 1998. Google Scholar
69. Shannon C. A mathematical theory of communication. Bell Syst Tech J. 1948;27:379–423. CrossrefGoogle Scholar
70. Resnick M. Turtles, Termites, and Traffic Jams: Explorations in Massively Parallel Microworlds. Cambridge, Mass: MIT Press; 1994. Google Scholar
71. von Neumann J, Morgenstern O. Theory of Games and Economic Behavior. Princeton, NJ: Princeton University Press; 1944. Google Scholar
72. Forrester JW. Industrial Dynamics. Cambridge, Mass: MIT Press; 1961. Google Scholar
73. Forrester JW. Learning through system dynamics as preparation for the 21st century. Paper presented at: Systems Thinking and Dynamic Modeling Conference for K–12 Education, June 1994, Concord, Mass. Google Scholar
74. Forrester JW. Roadmaps: A guide to learning system dynamics. Available at: http://web.mit.edu/sdg/www/roadmaps.html. Accessed March 18, 2003. Google Scholar
75. Richardson GP. Problems for the future of system dynamics. Syst Dynamics Rev. 1996;12:141–157. CrossrefGoogle Scholar
76. Sterman J. System dynamics modeling: tools for learning in a complex world. Calif Manage Rev. 2001; 43:8–25. CrossrefGoogle Scholar
77. System Dynamics Society. What is system dynamics? Available at: http://www.systemdynamics.org. Accessed December 19, 2002. Google Scholar
78. Capra F. The Web of Life: A New Synthesis of Mind and Matter. London, England: Flamingo; 1997. Google Scholar
79. Capra F. The Hidden Connections: Integrating the Hidden Connections Among the Biological, Cognitive, and Social Dimensions of Life. New York, NY: Doubleday; 2002. Google Scholar
80. Capra F. From the Parts to the Whole: Systems Thinking in Ecology and Education. Mill Valley Calif: Mill Valley School District; 1994. Google Scholar
81. New Paradigm Thinking [audiotape]. San Francisco, Calif: New Dimensions Foundation; 1993. Google Scholar
82. Strogatz SH. Sync: The Emerging Science of Spontaneous Order. New York, NY: Hyperion; 2003. Google Scholar
83. Watts DJ. Small Worlds: The Dynamics of Networks Between Order and Randomness. Princeton, NJ: Princeton University Press; 1999. Google Scholar
84. Watts DJ. The Structure and Dynamics of Small-World Systems [dissertation]. Ithaca, NY: Cornell University; 1997. Google Scholar
85. Newman MEJ. The structure and function of complex networks. arXiv. March 25, 2003:1. Google Scholar
86. Watts DJ. Six Degrees: The Science of a Connected Age. New York: WW Norton & Co; 2003. Google Scholar
87. Lorenz E. Predictability: does the flap of a butterfly’s wings in Brazil set off a tornado in Texas? Paper presented at: Annual Meeting of the American Association for the Advancement of Science, December 1979, Washington, DC. Google Scholar
88. Gleick J. Chaos: Making a New Science. New York, NY: Viking; 1987. Google Scholar
89. Gladwell M. The Tipping Point: How Little Things Can Make a Big Difference. Boston, Mass: Little Brown & Co; 2000. Google Scholar
90. Cabrera DA. Patterns of knowledge: knowledge as a complex, evolutionary system, an educational imperative. In: Miller R, ed. Creating Learning Communities [online edition]. Brandon, Vt: Solomon Press; 2002. Google Scholar
91. Lewin R. Complexity: Life at the Edge of Chaos. 2nd ed. Chicago, Ill: University of Chicago Press; 1999. Google Scholar
92. Institute of Medicine. Accidental Death and Disability: The Neglected Disease of Modern Society. Washington, DC: National Academy of Sciences; 1966. Google Scholar
93. American Red Cross. CPR/AED for the Professional Rescuer. Yardely, Pa: Staywell; 2002. Google Scholar
94. Mindwalk [videotape]. Hollywood, Calif: Paramount; 1992. Google Scholar
95. The Turning Point: A Transformative Vision for an Ecological Age [audiotape]. Emeryville, Calif: Enhanced Audio Systems; 1991. Google Scholar
96. Cabrera DA. Bacterial burritos, mixed metaphors and a man in the middle: a scientific life at the crossroads of biology, chemistry and engineering. Available at: http://www.genomics.cornell.edu/news/news_story.cfm?id=52. Accessed November 24, 2005. Google Scholar
97. Von Bertalanffy L. The History and Status of General Systems Theory. New York, NY: Wiley-Interscience; 1972. Google Scholar
98. Valera FJ. On being autonomous: the lessons of natural history for systems theory. In: Klir G, ed. Applied Systems Research. New York, NY: Plenum Press; 1977:77–85. Google Scholar
99. Davidz HL, Nightingale DJ, Rhodes DH. Enablers, Barriers, and Precursors to Systems Thinking Development: The Urgent Need for More Information. Cambridge, Mass: MIT Press; 2004. Google Scholar
100. Von Bertalanffy L. General System Theory: Foundations, Development, Applications. New York, NY: Braziller; 1969. Google Scholar
101. Richmond B. Systems thinking: critical thinking skills for the 1990s and beyond. Syst Dynamics Rev. 1993;9:113–134. CrossrefGoogle Scholar
102. The World Health Report 2000: Health Systems: Improving Performance. Geneva, Switzerland: World Health Organization; 2000. Report 924156198X. Google Scholar
103. Krygiel AJ. Behind the Wizard’s Curtain: An Integration Environment for a System of Systems. Washington, DC: Institute for National Strategic Studies; 1999. Google Scholar
104. Spotlight on Syndemics. Atlanta, Ga: Centers for Disease Control and Prevention; 2001. Google Scholar
105. Baer HA, Singer M, Susser I. Medical Anthropology and the World System. 2nd ed. Westport, Conn: Praeger; 2003. Google Scholar
106. Singer M. A dose of drugs, a touch of violence, a case of AIDS: conceptualizing the SAVA syndemic. Free Inquiry Creative Sociol. 1996;24:99–110. Google Scholar
107. Singer M, Clair S. Syndemics and public health: reconceptualizing disease in bio-social context. Med Anthropol Q. 2003;17:423–441. Crossref, MedlineGoogle Scholar
108. Singer M, Snipes C. Generations of suffering: experiences of a treatment program for substance abuse during pregnancy. J Health Care Poor Underserved. 1992;3:222–234. Crossref, MedlineGoogle Scholar
109. Milstein B. Syndemics. In: Encyclopedia of Evaluation. Newbury Park, Calif: Sage Publications; 2004: 404–405. Google Scholar
110. Bammer G. Integration and implementation sciences: building a new specialization. Ecology Soc [serial online]. 2005;10(2):6. Available at: http://www.ecologyandsociety.org/vol10/iss2/art6. Accessed November 24, 2005. CrossrefGoogle Scholar
111. Emirbayer M. Manifesto for a relational sociology. Am J Sociol. 1997;103:281–317. CrossrefGoogle Scholar
112. Midgley G. System Intervention: Philosophy, Methodology, and Practice. New York, NY: Kluwer Academic; 2000. Google Scholar
113. WK Kellogg Foundation. Partnerships: a powerful tool for improving the well-being of families and neighborhoods. Available at: http://www.wkkf.org/Pubs/YouthED/Pub591.pdf. Accessed November 24, 2005. Google Scholar
114. US Dept of Health and Human Services. Creating partnerships, improving health: the role of community-based participatory research. Available at: http://www.ahrq.gov/research/cbprrole.htm. Accessed November 24, 2005. Google Scholar
115. University of Washington. Community-Campus Partnerships for Health (CCPH). Available at:. Accessed November 24, 2005. Google Scholar
116. Institute for Healthcare Improvement. About us. Available at: http://www.ihi.org/ihi/about. Accessed September 15, 2004. Google Scholar
117. International Healthy Cities Foundation. What is the Healthy Cities movement? Available at: http://www.healthycities.org. Accessed October 5, 2004. Google Scholar
118. Partnership for the Public’s Health. About the partnership. Available at: http://www.partnershipph.org/col1/about/overview.html. Accessed April 20, 2005. Google Scholar
119. Turning Point National Program Office. From Silos to Systems: Using Performance Management to Improve the Public Health. Seattle, Wash: University of Washington; 2003. Google Scholar
120. World Health Organization, Commission on Social Determinants of Health. From knowledge to action: global country partners address social determinants of health. Available at: http://www.who.int/social_determinants/en. Accessed March 18, 2005. Google Scholar
121. Trochim W. An introduction to concept mapping for planning and evaluation. Eval Program Plann. 1989;12:1–16. CrossrefGoogle Scholar
122. Trochim W. Concept mapping: soft science or hard art? Eval Program Plann. 1989;12:87–110. CrossrefGoogle Scholar
123. Batterham R, Southern D, Appleby N, et al. Construction of a GP integration model. Soc Sci Med. 2002;54:1225–1241. Crossref, MedlineGoogle Scholar
124. DeRidder D, Depla M, Severens P, Malsch M. Beliefs on coping with illness: a consumer’s perspective. Soc Sci Med. 1997;44:553–559. Crossref, MedlineGoogle Scholar
125. Trochim W, Milstein B, Wood B, Jackson S, Pressler V. Setting objectives for community and systems change: an application of concept mapping for planning a statewide health improvement initiative. Health Promotion Pract. 2004;5:8–19. Crossref, MedlineGoogle Scholar
126. Davison ML. Multidimensional Scaling. New York, NY: John Wiley & Sons Inc; 1983. Google Scholar
127. Kruskal JB, Wish M. Multidimensional Scaling. Beverly Hills, Calif: Sage Publications; 1978. Google Scholar
128. Anderberg MR. Cluster Analysis for Applications. New York, NY: Academic Press Inc; 1973. Google Scholar
129. Everitt B. Cluster Analysis. 2nd ed. New York, NY: Halsted Press; 1980. Google Scholar
130. Trochim W. Reliability of concept mapping. Paper presented at: Annual Conference of the American Evaluation Association, November 3–6, 1993, Dallas, Tex. Google Scholar


No related items




William M. Trochim, PhD, Derek A. Cabrera, MA, Bobby Milstein, MPH, Richard S. Gallagher, BS, and Scott J. Leischow, PhDWilliam M. Trochim is with the Department of Policy Analysis and Management, Cornell University, Ithaca, NY. Derek A. Cabrera is with the Department of Education, Cornell University. Bobby Milstein is with the Centers for Disease Control and Prevention, Atlanta, Ga. Richard S. Gallagher is with Gallagher and Associates, Ithaca, NY. At the time of the study, Scott J. Leischow was with the National Cancer Institute, Bethesda, Md. “Practical Challenges of Systems Thinking and Modeling in Public Health”, American Journal of Public Health 96, no. 3 (March 1, 2006): pp. 538-546.


PMID: 16449581