# Rigor AND Relevance

August 28, 2005 draft of a paper for the September 13-15 symposium:

Expanding Research and Evaluation Designs and Methods to Improve the
Science Base for Health Care and Public Health Quality Improvement

Kurt C. Stange, MD, PhD

Editor, *Annals of Family Medicine*

Gertrude Donnelly Hess, MD Professor of Oncology Research

Professor of Family Medicine, Epidemiology & Biostatistics, Sociology
and Oncology

Case Western Reserve University

Associate Director for Prevention, Control and Population Research Case
Comprehensive Cancer Center

## The problem

The problem addressed in the papers by Ornstein and Ruhe (and to a large
extent by this conference) is the classic tradeoff between rigor and
relevance.^1^

Rigor typically is thought of as a focus on internal validity --
developing tightly-specified intervention approaches that can be
evaluated with objectivity and implemented with fidelity. Relevance is
conceptualized as developing, implementing and evaluating approaches to
research and quality improvement (QI) that have meaning to the
participants and end-users.

This conceptualization of the problem is itself a part of the problem.
The proper formulation of the predicament is not as a polarity between
rigor and relevance, but as the challenge of optimizing both. In order
to meet this challenge, we need fundamentally different ways of
approaching health care and quality improvement research. These involve:

-   An integrative epistemology

-   A transdisciplinary, whole systems approach

-   A commitment to developing and sustaining community "laboratories"

-   Participatory approaches that incorporate both research and
    development

-   Bringing together both quantitative and qualitative methods

-   Relevant theoretical underpinnings including those from complexity
    science

Taken together, these approaches result in a greater emphasis on
context, process and meaning than the current zeitgeist. They lead to
efforts to develop lines of inquiry that are integrated with development
and application, rather than conducting and disseminating isolated
studies. They allow the questions and the methods for answering those
questions to evolve over time and place. They ask not only what the QI
intervention is and what its outcomes are, but how it can be implemented
in different settings, where different approaches are relevant, when
they should be put into practice, who should conduct different aspects
of the intervention, and why it is important.

## An integrative epistemology

As identified by Wilber^2-4^ and applied to health care research by
Stange, Miller and McWhinney,^5^ different ways of knowing are relevant
for understanding and improving health and health care. As shown in
Figure 1, this Generalist Wheel of Knowledge represents knowledge in
four quadrants that inform primary care research and practice. These
represent the perspectives of the clinician, the
patient/family/community, the health care system, and disease.
Additional ways of knowing are represented by the border regions between
these four quadrants, and the integrative function is represented at the
center. (See the Appendix for an additional depiction.)

Currently, an over-reliance on a reductionist biomedical paradigm
(Quadrant 4 in this scheme) has led to an under-valuing of knowledge
based on understanding of systems and of the perspectives of patients,
families, communities and front-lines clinicians. However, the different
ways of knowing are always present, although we may not see them because
our focus and methods give attention to only one way of knowing at a
time. Broadening our lens, so that even in narrowly focused studies, we
pay at least a little attention to different ways of knowing has the
potential to bring together rigor and relevance. In many ways, this is
an epistemological struggle -- an often unrecognized disagreement in
understanding of the nature of the relationship between the knower /
inquirer and the known / knowable.

As elucidated by Guba,^6^ the dominant way of knowing is the
post-positivist paradigm in which objective, value-free and
generalizable knowledge is identified by experimental methods. By
comparison, a constructivist paradigm stresses the social construction
of knowledge, in which information carries with it the history of its
creation, and to become valid in another context, must be reinterpreted
to establish their relevance and meaning.

A focus on rigor tends to emphasize a positivist epistemology, and often
leads to concern about translating research into practice (TRIP),^7,\ 8^
since the resulting QI approaches often turn out to be poorly
transportable into settings outside the research environment. An
emphasis on relevance often leads to use of a social constructivist
epistemology, which emphasizes translating practice into research. This
can lead to a different kind of transportability problem, if the
resulting insights are so locally adapted that their generalizable
lessons are not elucidated.

The dominance of positivist epistemology underlies efforts to find a
"magic bullet" intervention that can then be easily transported *en
masse* to other settings from that in which it was developed. The
tremendous advances in biomedical science that have resulted from the
application of this epistemology leave us breathless, but often unable
to see that our world view is a central part of the TRIP problem that we
seek to solve. A solely positivist approach can result in an
over-reliance on methods that divorce the phenomenon under study from
its sources of meaning and context. Thus, our approach to the TRIP
problem often is part of the problem.

A social constructivist approach may be more appropriate for the
inherently social and context-dependent settings in which QI
interventions live. Such approaches emphasize that reality is
co-constructed by participants, each of whom has different experiences
of the phenomenon, and each of whose perspective is valid.

The danger that such a social constructivist epistemology can lead to a
Tower of Babel can be overcome in part by focusing on the development of
general theories and principles that transcend and include the different
perspectives of the key stakeholders in QI. The danger that a positivist
epistemology can isolate the phenomenon of interest from the context
that gives it meaning can be overcome in part by methods that
simultaneously or sequentially focus on context. Recognizing the utility
of both epistemologies, and allowing them to co-exist within a research
team, can have the best of both world views.

## A transdisciplinary whole systems approach

NIH Roadmap calls for an increasing focus on team science.^9^ Those who
have experienced true transdisciplinary teams see them as a tremendous
means for advancing understanding. Diverse teams can be conceptualized
as a progression from multidisciplinary to interdisciplinary to
transdisciplinary.^5,\ 10^ In multidisciplinary research, participants
from different disciplines each contribute their piece to solving a
problem - like different chapters in an edited book, with the reader
responsible for synthesizing meaning across the chapters.
Interdisciplinary research is a collaborative health care team, with
each member working together on solving a common problem.
Transdisciplinary teams create a sustained conversation across and
beyond disciplinary boundaries. Often, they can be recognized by their
creation of a new shared language.

Not all projects are worthy of the extra effort needed to move from
multi- to inter- to trans-disciplinary teams. However, transdisciplinary
teams may hold the greatest potential for the kind of grounded
innovation and pursuit of a sustained lines of inquiry that are needed
to make quantum improvements in health care and health.

Whole systems approaches^11-14^ go even further by asking that all
levels of the interacting organizations and groups that influence health
care be considered, and ideally brought to the table in pursing research
and development for quality improvement. This can dramatically alter the
focus of such research and quality improvement, and such efforts often
are deemed to be "messy." However, in this messiness is the needed grist
for the mill of sustainable quality improvement.

## A commitment to developing and sustaining QI "laboratories"

The NIH Roadmap^9^ also calls for moving the clinical research
enterprise out of the biomedical medical center into the clinical and
community settings in which most people get most of their care most of
the time. In order to be effective, this movement needs to be supported
by infrastructure and relationships that transcend any single research
or quality improvement project.

In recent years there has been an explosion in the development of
practice-based research networks (PBRNs). ^15-17^ Practice-Based
Research Networks (PBRNs) have been defined as

> *\"a group of ambulatory practices devoted principally to the primary
> care of patients, affiliated with each other (and often with an
> academic or professional organization) in order to investigate
> questions related to community-based practice. Typically, PBRNs draw
> on the experience and insight of practicing clinicians to identify and
> frame research questions whose answers can improve the practice of
> primary care. By linking these questions with rigorous research
> methods, the PBRN can produce research findings that are immediately
> relevant to the clinician and, in theory, more easily assimilated into
> everyday practice.\"* (<http://ahrq.gov/research/pbrnfact.htm>)

PBRNs have an administrative structure that supports clinician-initiated
projects, and a mission that transcends the conduct of a single study.
The most productive networks have developed partnerships with academic
organizations. By engaging primary care practices and the communities
they serve in generating and answering questions, PBRNs have great
potential to diminish the problem of translating research into practice
and community application.^16,\ 18-23^ Despite important but
under-funded efforts by AHRQ,^24,\ 25^ RWJF^26^ and others, PBRN
infrastructure remains woefully underdeveloped^27^ in comparison to
other critical research and QI infrastructure, and in comparison to the
potential of PBRNs to serve as an engine for knowledge generation and
translation around some of the most important problems in health and
health care.^17,\ 23,\ 28,\ 29^

## Participatory approaches to both research and development

Community-based participatory research (CBPR), and its cousin action
research,^30^ both involve learning cycles that explicitly engage the
users of knowledge in defining the question, designing and implementing
the methods and interpreting and applying the findings.^21,\ 31^

Recent efforts by the NIH have recognized the potential of CBPR to
enlighten and overcome current problems in health and healthcare
disparities. In addition, like PBRNs, CBPR infrastructure is
under-funded in comparison to its potential to be a major part of the
solution to some of our most intractable health and health care
problems. Infrastructure support is needed for long-term relationship
and process development work that transcends a single study or line of
investigation. The NIH research center model has potential applicability
for both PBRN and CBPR support.

In addition to PBRN and CBPR approaches that are grounded in the
practices and communities where health and health care knowledge is
applied, there is a fundamental need to consider development as well as
research. Development involves building capacity. In industry, research
seldom is considered without considerable emphasis on development. In
health care, research and development have been divorced, with the
result that most research never is translated into practice or community
application. Considering the development and application needs for
research at the outset is important.

## Bringing together both quantitative and qualitative methods

The strength and the weakness of quantitative methods is that they
isolate a phenomenon from its context. Combining quantitative and
qualitative methods fosters understanding both of what can be measured
and of meaning and context.^32,\ 33^ Mixed methods foster emergent
understanding of meaning and context at the same time as they allow
testing of a priori hypotheses.^34,\ 35^

The STEP-UP project described by Ms. Ruhe is an excellent example. In a
group randomized design,^36,\ 37^ a priori hypotheses were tested using
quantitative measures. Qualitative methods were used to understand the
practice settings and to tailor the intervention based on this
understanding. Both numerical and narrative data on the process of
change were gathered and used to develop new intervention approaches
that were tested in a pre-post design in the delayed intervention
(control) practices. A subsequent comparative case study analysis^38^
was used to develop new theories^39-41^ and to guide the development of
the next intervention study, involving an enlarged research team, an
expanded PBRN and new community and health care system partners.

## A complexity science underpinning

The tenets of complexity the complexity sciences can reframe both the
generation and application about knowledge for quality
improvement.^42,\ 43^ These tenets note that complex behavior emerges
from relationships among agents, and often can be understood through
simple rules. Recurrent patterns occur at multiple levels of an
organization. Organizations co-evolve with other agents in the system,
based in part on a strong dependence on initial conditions. Complex
adaptive systems exhibit non-linearity.^40,\ 44-46^

Applying these leads researchers to work across multiple levels of the
system and to look for opportunities to understand and facilitate
positive co-evolution. These principles jibe better with the reality of
practice and community settings than linear thinking. They lead
participants to expect that sometimes large interventions can have
minimal effects, whereas in other situations a small intervention can
result in a large change. Complexity science leads to focusing on
relationships rather than, or in addition to focusing on tools and
isolated variables.

## The Presentations by Ornstein and Ruhe

The lines of investigation and specific studies presented by Dr.
Ornstein and Ms. Ruhe exhibit the thoughtful management of trade-offs
that front lines quality improvement investigators and implementers must
address every day. Their efforts show the challenge of applying
theoretical models and research methods that don't quite match the
on-the-ground reality. Their efforts to move forward despite these
challenges are heroic. More importantly, their use of both qualitative
and quantitative methods allows them to link their work to other studies
using similar measures while learning from the process to inform the
work as it progresses, and to help us to learn the larger lessons with
them.

Both projects used a multifaceted intervention strategy that previous
research has shown to be most effective.^22,\ 47-51^ Both used
facilitation, audit and feedback, participatory planning and
implementation, patient activation and practice-individualized tools.
Both applied a best practices approach and principles from complexity
science. Such a comprehensive scheme avoids the shortcomings of a
"one-size fits all" approach^52^ and increases the odds that the right
approach will be available at the right time.

Ornstein used the electronic medical record both as a fundamental
structural underpinning and as a framework for practice
recruitment.^53,\ 54^ As it becomes more common in practice, the EMR
will have growing applicability. However, as recent research
shows,^55,\ 56^ without the comprehensive and locally adapted QI
approach used by Ornstein and by Ruhe, it is magical thinking to imagine
that the EMR is a total solution to TRIP.

In addition to a multifaceted intervention strategy, it is remarkable
that both projects focused on multiple quality measures. This flies in
the face of the researcher's natural tendency to focus on a manageable
number of factors. However, this seeming lack of focus reduces concerns
about competing demands -- the idea that improvement in one area may
lead to reductions in other areas.^57,\ 58^ Rather, this broad focus is
consistent with the idea that the proactive development of systems may
lead to cross-benefit. For example, systems to improve chronic disease
management may improve preventive service delivery.^59^

During the course of their investigations, both research teams learned
the importance of linking the intervention to participants' motivations.
Ruhe's team identified participants' motivation for other practice
improvement domains than the focus of the intervention. Despite the
broad focus of the intervention, this energy for positive change may
have been "left on the table" when it could not be linked to the focus
of the QI project.

Ornstein's choice of a pre/post design without a concurrent comparison
group for his second study shows a classic trade-off (or perhaps a
logical progression) between optimizing internal validity with an RCT
compared to enhancing external validity by engaging practices that would
not wish to be randomized or to contribute data without getting
something back. The idea that participants might be offered different
interventions or the same intervention at different times may provide a
partial solution to this dilemma. It would be interesting to formally
compare the differences between participants and intervention effect
sized for the group randomized trial and pre/post studies.

Ruhe's progression to the next phase project (EPOCHS) shows an important
recognition of the need to intervene at multiple levels. However, this
creates additional challenges assessing the process and the outcome at
from the perspective of the patient, practice agents, health care system
and community.

The importance of local adaptation and individualization of intervention
strategies is a transcendent lesson from both lines of inquiry.

.

## Putting it together

The lessons learned by Ornstein and Ruhe are consistent with the RE-AIM
framework ([http://www.re-aim.org](http://www.re-aim.org/)) elucidated
by Glasgow and colleagues.^60-62^ This framework (reach,
efficacy/effectiveness, adoption, implementation, and maintenance)
provides a model for considering both internal validity and
transportability that encourages continual contextualization.

Another important framework is proposed by Grenhaugh and colleagues.^63^
This model synthesizes the evidence for the diffusion of innovations in
health service organizations and identifies knowledge gaps where further
research is needed.

The Medical Research Council's framework for design and evaluation of
complex interventions to improve health,^64^ while acknowledging the
benefit of using both qualitative and quantitative methods, still takes
a linear approach that moves from theory to modeling to exploratory and
definitive trials, followed by long-term implementation. A more
iterative model would fit better with the non-linear way in which
knowledge is created and applied.

What would it look like if the ideas and approaches described throughout
this paper were employed together as part of a new mainstream of
integrated knowledge development and application?

Practices, patients, health care systems, community agencies and groups,
academics and policymakers are at the table together to develop and
refine research and development questions, and the methods used to
pursue them. The questions tend to focus on whole people and systems,
rather than just isolated parts of people and systems. Participants
clearly state their epistemological stance and work to understand how
theory informs practical application and how theory development can be
an explicit goal of planned research. Explicitly diverse participants
use respectful discussions of both practice and theory to develop
transdisciplinary teams, continually engaging new partners as needed to
challenge evolving understanding and practical application. The design
of individual projects involves a mixture of methods to develop
knowledge in the settings in which it will be applied, and from the
perspectives of those who will apply and pay for the knowledge.
Individual projects are viewed within the context of multiple
interrelated lines of investigation and application. Support for
communication, research methods and databases is provided by shared
enabling resources supported by NIH, AHRQ and CDC research centers for
integrative research and application, with additional support by local
foundations and endowments to catalyze community infrastructure
development, application and evaluation. These resources are applied to
increasing understanding of some of the most important problems
affecting health and confronting health care. The resulting
understanding is applied and evaluated in the course of studying it, and
refinements and development needed for continued evolution of knowledge
and practice is pursued in knowledge communities that involve the
evolving tapestry of relevant stakeholders. Human subjects protection
becomes less of an issue than assuring participation by affected groups
and individuals.

##  

## References

**1.** Slawson DC, Shaughnessy AF, Barry H. Which should come first:
rigor or relevance? *J Fam Pract.* 2001;50(3):209-210.

**2.** Wachtel TJ, O\'Sullivan P. Practice guidelines to reduce testing
in the hospital. *J Gen Intern Med.* 1990;5:335-341.

**3.** Wilber K. *Sex, Ecology, Spirituality: The Spirit of Evolution*.
Boston: Shambhala Publications, Inc.; 1995.

**4.** Wilber K. *A Brief History of Everything*. Boston: Shambhala
Publications, Inc.; 1996.

**5.** Stange KC, Miller WL, McWhinney I. Developing the knowledge base
of family practice. *Fam Med.* Apr 2001;33(4):286-297.

**6.** Guba E. *The Paradigm Dialog*. Newbury Park: Sage; 1990.

**7.** Col NF. Challenges in translating research into practice. *J
Womens Health.* 2005;14(1):87-95.

**8.** Hait W. Translating research into clinical practice:
deliberations from the American Association for Cancer Research. *Clin
Cancer Res.* 2005;11(12):4275-4277.

**9.** NIH. Re-engineering the clinical research enterprise. Available
at: nihroadmap.nih.gov/clinicalresearch/index.asp.

**10.** Miller WL. Common space: Creating a collaborative research
conversation. In: Crabtree BF, Miller WL, Addison RB, Gilchrist VJ,
Kuzel A, eds. *Exploring Collaborative Research in Primary Care*.
Thousand Oaks, CA: Sage Publications; 1994:265-288.

**11.** Thomas P, While A. Increasing research capacity and changing the
culture of primary care towards reflective inquiring practice: the
experience of the West London Research Network (WeLReN). *J
Interprofessional Care.* 2001;15(2):133-139.

**12.** Thomas P. *The Liverpool Primary Health Care Facilitation
Project 1989-1994*: Liverpool Family Health Services Authority; 1994.

**13.** Thomas P. *Whole system learning and change in primary health
care: The role of primary care research networks* \[MD\]: Medicine,
University of Liverpool; 2003.

**14.** Thomas P, Griffiths F, Kai J, O\'Dwyer A. Networks for research
in primary health care. *BMJ.* 2001;322:588-590.

**15.** Stange KC. Practice-based research networks: Their current level
of validity, generalizability, and potential for wider application.
*Arch Fam Med.* 1993;2:921-923.

**16.** Nutting P, Green L. Practice-based research networks: reuniting
practice and research around the problems most of the people have most
of the time. *J Fam Pract.* 1994;38:335-336.

**17.** Nutting P, Beasley J, Werner J. Practice-based research networks
answer primary care questions. *JAMA.* 1999;281:686-688.

**18.** Nutting P. Why can\'t clinical policies be relevant to practice?
*J Fam Pract.* 1997;44:350-352.

**19.** Nutting PA, Stange KC. Practice-based research: The opportunity
to create a learning discipline. In: Rakel RE, ed. *The Textbook of
Family Practice*; 2002.

**20.** Green L, Nutting P. Family physicians as researchers in their
own practices. *J Am Board Fam Pract.* 1994;7:261-263.

**21.** Macaulay AC, Commanda LE, Freeman WL, et al. Participatory
research maximises community and lay involvement. *BMJ.*
1999;319:774-778.

**22.** Davis DA, Taylor-Vaisey A. Translating guidelines into practice:
A systematic review of the theoretic concepts, practical experience and
research evidence in the adoption of clinical practice guidelines. *Can
Med Ass J.* 1997;157:408-416.

**23.** Nutting PA. Practice-based research: laboratories for outcomes
and effectivness research. Paper presented at: AHCPR Conference
Proceedings. Preimary Care Research: Theory and Methods, 1991;
Washington, D.C.

**24.** Lanier D. Primary Care Practice-Based Research Comes of Age in
the United States. *Ann Fam Med.* May 1, 2005 2005;3(suppl_1):S2-4.

**25.** Nutting P, Stange K, eds. *Contemporary Challenges for
Practice-Based Research Networks*. 3 (suppl 1): Ann Fam Med; 2005.

**26.** Cifuentes M, Fernald DH, Green LA, et al. Prescription for
Health Changing primary care practice to foster healthy behaviors. *Ann
Fam Med.* 2005;3 (suppl 2):S4-S11.

**27.** Green LA, White LL, Barry HC, Nease DE, Jr, Hudson BL.
Infrastructure Requirements for Practice-Based Research Networks. *Ann
Fam Med.* May 1, 2005 2005;3(suppl_1):S5-11.

**28.** Green LA, Hames CG, Nutting PA. Potential of practice-based
research networks: Experiences from ASPN. *J Fam Pract.*
1994;38:400-406.

**29.** Green LA, Dovey SM. Practice based primary care research
networks. They work and are ready for full development and support.
*BMJ.* 2001;322:567-568.

**30.** Minkler M. Using participatory action research to build healthy
communities. *Public Health Rep.* 2000;115:191-197.

**31.** Green LW, Mercer SL. Can public health researchers and agencies
reconcile the push from funding bodies and the pull from communities?
*Am J Pub Health.* 2001;91(12):1926-1929.

**32.** Stange KC, Zyzanski SJ. The integrated use of quantitative and
qualitative research methods. *Fam Med.* 1989;21:448-451.

**33.** Stange KC, Miller WL, Crabtree BF, O\'Connor PJ, Zyzanski SJ.
Multimethod research: approaches for integrating qualitative and
quantitative methods. *J Gen Intern Med.* 1994;9:278-282.

**34.** Borkan JM. Mixed methods studies: a foundation for primary care
research. *Ann Fam Med.* 2004;2(1):4-6.

**35.** Stange KC, Miller WL, McWhinney IR. Developing the knowledge
base of family practice. *Fam Med.* 2001;33(4):286-297.

**36.** Goodwin MA, Zyzanski SJ, Zronek S, et al. A clinical trial of
tailored office systems for preventive service delivery: The Study to
Enhance Prevention by Understanding Practice (STEP-UP). *Am J Prev Med.*
2001;21(1):20-28.

**37.** Stange KC, Goodwin MA, Zyzanski SJ, Dietrich AJ. Sustainability
of a practice-individualized preventive service delivery intervention.
*Am J Prev Med.* 2003;25:296-300.

**38.** Ruhe MC, Weyer SM, Zronek S, Wilkinson A, Wilkinson PS, Stange
KC. Facilitating practice change: Lessons from the STEP-UP clinical
trial. *Prev Med.* 2005;40(6):729-734.

**39.** Cohen D, McDaniel RR, Jr., Crabtree BF, et al. A practice change
model for quality improvement in primary care practice. *J Healthc
Manag.* 2004;49(3):155-168; discussion 169-170.

**40.** Stroebel CK, McDaniel RR, Crabtree BF, Miller WL, Nutting PA,
Stange KC. Using complexity science to inform a reflective practice for
improvement in primary care. *Jt Comm J Qual Saf.* 2005:(in press).

**41.** Stange KC, Ruhe MC, Weyer SM, et al. Practice change models:
insights from multimethod practice assessment of change over time
(IMPACT). *Ann Fam Med.* 2005;(revision under review).

**42.** Miller WL, Crabtree BF, McDaniel R, Stange KC. Understanding
change in primary care practice using complexity theory. *J Fam Pract.*
1998;46:369-376.

**43.** Miller WL, McDaniel RR, Jr, Crabtree BF, Stange KC. Practice
jazz: understanding variation in family practices using complexity
science. *J Fam Pract.* 2001;50(10):872-880.

**44.** Albrecht G, Freeman S, Higginbotham N. Complexity and human
health: The case for a transdisciplinary paradigm. *Culture, Medicine
and Psychiatry.* 1998;22:55-92.

**45.** Plsek PE, Greenhalgh T. Complexity science: The challenge of
complexity in health care. *BMJ.* 2001;323:625-628.

**46.** Stacey RD. *Complexity and creativity in organizations*. San
Francisco: Berrett-Koehler; 1996.

**47.** Davis DA, Thomson MA, Freemantle N, et al. Impact of Formal
Continuing Medical Education: Do Conferences, Workshops, Rounds, and
Other Traditional Continuing Education Activities Change Physician
Behavior or Health Care Outcomes? *JAMA.* 1999;282(9):867-874.

**48.** Davis DA, Thomson MA, Oxman AD, Haynes B. Changing physician
performance: A systematic review of the effect of continuing medical
education strategies. *JAMA.* 1995;274:700-705.

**49.** Davis DA, Thomson MA, Oxmas AD, Haynes RB. Systematic
practice-based interventions are better than conference for improving
professional practice. *JAMA.* 1995;274:700-705.

**50.** Grol R. Improving the quality of medical care: Building bridges
among professional pride, payer profit, and patient satisfaction.
*JAMA.* 2001;84(20):2578-2585.

**51.** Grol R, Grimshaw J. Evidence-based implementation of
evidence-based medicine. *J Comm J Qual Improv.* 1999;25(10):503-513.

**52.** Stange KC. \"One size doesn\'t fit all.\" Multimethod research
yields new insights into interventions to improve preventive service
delivery in family practice. *J Fam Pract.* 1996;43(4):358-360.

**53.** Feifer C, Fifield J, Ornstein S, Karson AS, Bates DW, Jones KR.
From research to daily clinical practice: what are the challenges in
\"translation\"? *Jt Comm J Qual Saf.* 2004;30(5):235-245.

**54.** Feifer C, Ornstein SM. Strategies for increasing adherence to
clinical guidelines and improving patient outcomes in small primary care
practices. *Jt Comm J Qual Saf.* 2004;30(8):432-441.

**55.** O\'Connor PJ, Crain AL, Rush WA, Sperl-Hillen JM, Gutenkauf JJ,
Duncan JE. Impact of an electronic medical record on diabetes quality of
care. *Ann Fam Med.* 2005;3(4):300-306.

**56.** Crosson JJ, Stroebel C, Scott JG, Stello B, Crabtree BF.
Implementing an electronic medical record in a family medicine practice:
communication, decision making and conflict. *Ann Fam Med.*
2005;3(4):307-311.

**57.** Jaén CR, Stange KC, Nutting PA. The competing demands of primary
care: A model for the delivery of clinical preventive services. *J Fam
Pract.* 1994;38:166-171.

**58.** Jaén CR, McIlvain H, Pol L, Phillips RL, Jr , Flocke SA,
Crabtree BF. Tailoring tobacco counseling to the competing demands in
the clinical encounter. *J Fam Pract.* 2001;50:859-863.

**59.** Glasgow RE, Orleans CT, Wagner EH, Curry SJ, Solberg LI. Does
the chronic care model serve also as a template for improving
prevention? *Millbank Quarterly.* 2001;79(4):579-612.

**60.** Glasgow RE, McKay HG, Piette JD, Reynolds KD. The RE-AIM
framework for evaluating interventions: what can it tell us about
approaches to chronic illness management? *Patient Educ Couns.* Aug
2001;44(2):119-127.

**61.** Glasgow R, Magid D, Beck A, Ritzwoller D, Estabrooks P.
Practical clinical trials for translating research to practice: design
and measurement recommendations. *Med Care.* 2005;43(6):551-557.

**62.** Klesges LM EP, Dzewaltowski DA, Bull SS, Glasgow RE,. Beginning
with the application in mind: designing and planning health behavior
change interventions to enhance dissemination. *Ann Behav Med.* 2005;29
(2 Suppl):66-75.

**63.** Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O.
Diffusion of innovations in service organizations: systematic review and
recommendations. *Milbank Q.* 2004;82(4):581-629.

**64.** Campbell M, Fitzpatrick R, Haines A, et al. Framework for design
and evaluation of complex interventions to improve health. *Bmj.* Sep 16
2000;321(7262):694-696.

**Figure 1**

**The Generalist Wheel of Knowledge, Understanding, and Inquiry\***

**INFORMATION**

(Quadrant 1) **MASTERY** (Quadrant 4)

**CLINICIAN** Evidence-Based **DISEASE**

Self-Awareness Medicine Science

*Reflection, Journaling Learning Epidemiology &*

*Experimentation*

**RELATIONSHIP INTEGRATION PRIORITIZATION**

Human Interactions Health Care & Healing Value

*Participant Observation Transdisciplinary, Cost-Effectiveness*

*Multimethod, Analysis*

*Participatory*

(Quadrant 2)

**PATIENT,** (Quadrant 3)

**FAMILY SYSTEM**

**COMMUNITY** Organizations

Personal Values *Health Services*

*Depth Interviews* **JUSTICE** *Research*

*Living in Place* Social Values

*Policy Analysis*

\* **[KEY:]{.underline}**

Line 1: (capitalized words): **FOCUS OF KNOWLEDGE**

Line 2: Task of Understanding

Line 3 (italicized words): *Mode of Inquiry*

\[From: Stange KC, Miller WL, McWhinney I. Developing the knowledge base
of family practice. *Fam Med.* 2001;33:286-297.\]

**Appendix:**

![](media/image1.wmf){width="4.995833333333334in"
height="3.745138888888889in"}