**Computerized Tailored Interventions Workgroup Meeting**

**Renaissance Providence Hotel**

**Providence, Rhode Island**

**August 2 - 3, 2007**

**Full Report**

**TABLE OF CONTENTS**

#### 

I.  Executive Summary 3

II. Meeting Report 8

III. Appendix A -- Current CTIs 21

IV. Appendix B -- Emerging CTIs 24

V.  Appendix C -- Future CTIs 29

VI. Appendix D -- Dissemination of CTIs via RTIPs: Breakout Group A 32

VII. Appendix E -- Dissemination of CTIs via RTIPs: Breakout Group B 36

VIII. Appendix F -- Dissemination of CTIs via RTIPs: Breakout Group C 42

IX. Action Items 45

X.  Participants List 46

***Executive Summary***

**Overview of the Meeting**

The Computerized Tailored Interventions (CTI) Workgroup met August 2-3,
2007, to discuss the translation of CTIs into practice. NCI's
overarching goal is to improve population health and quality of life
through cancer prevention and control. Because nearly two-thirds of
cancers are attributable to behavior, the implementation of effective
behavioral interventions represents a significant opportunity for cancer
control. However, along NCI's discovery-delivery continuum, research to
link discovery to effective delivery of evidence-based interventions is
lacking. Given changes in technology and increasing access to
technology, the time is ripe to impact the population's health behavior
in part through delivery of CTIs. CTIs that can deliver pertinent
information will assist patients in actively participating in their own
care and making better health decisions, which will lead to increased
compliance with treatment and prevention activities, a healthier
population, and potentially lower health care costs across the country.
The Workgroup's discussion over the two days addressed the designing of
CTIs for broad dissemination with emerging and future technologies, and
dissemination of existing CTIs in the absence of available support from
the CTI developer.

At this meeting, opportunities for dissemination of current (already
developed), emerging (i.e., funded and in development), and future
(i.e., in the next 5 years), CTIs were discussed, along with
infrastructure and technology barriers and challenges to designing CTIs
for broad dissemination and implementation. Relevant delivery audiences
(i.e., those delivering CTIs to users) were identified that included
entities and sectors of both the research and practice communities who
should be involved in the development of strategies to disseminate and
implement CTIs.

Participants proposed ideas for consideration to promote the
dissemination of existing CTIs not supported by developers via the
Research-tested Intervention Programs (RTIPs) mechanism on Cancer
Control P.L.A.N.E.T. (Plan, Link, Act, Network with Evidence-based
Tools). Interventions available through RTIPS have been funded by
peer-review research grants and published in peer-reviewed journals.
RTIPs have been proven to be effective in research environments and may
be adapted and used in community or clinical settings. The website
primarily assists public health practitioners and planners with
identifying programs and materials suitable for a given population, or
programs that can be adapted for that population. Topic areas on RTIPs
include breast, cervical, and colorectal cancer screening promotion;
diet/nutrition; informed decision making; sun safety; physical activity;
tobacco control; and survivorship. Along with challenges for
disseminating existing CTIs via RTIPs, participants were asked to
identify infrastructure, technology, and any other challenges for
integrating CTIs into the RTIPs structure, or to propose alternative
models or tools for disseminating CTIs.

**Dissemination of Current, Emerging, and Future CTIs**

The three discussion groups identified similar delivery audiences. These
included insurers, health care providers (for-profit and non-profit),
health care promotion groups (such as the American Cancer Society
\[ACS\]), state and county health departments, health maintenance
organizations (HMOs), schools, and workplaces.

*[Current]{.underline}*

Barriers to dissemination of current CTIs developed in academia included
their lack of attractiveness to the intended audience and requirements
for extensive IT support. Many CTIs must be adapted for use by the
intended audience (for example, the CTI must be made culturally relevant
and usable in the intended setting) and often target a narrow range of
behaviors when the users would prefer a more global approach (i.e., a
suite of interventions such as exercise, diet, smoking cessation). Many
users also may have difficulty using a CTI if it requires an extensive
initial IT investment and continuing support. As solutions to these
barriers, this group recommended centralizing access to technical
assistance and to funds for the adaptation of CTIs for dissemination.
Non-profit organizations such as ACS and the Lance Armstrong Foundation
could help provide such assistance. The group also suggested the
establishment of a consortium of researchers to provide a venue for
discussions of how to design CTIs with dissemination in mind,
particularly in terms of developing CTIs that are more relevant to the
intended audience or easy to adapt to the audience's needs and goals. It
was recognized, however, that it is not always possible to develop an
intervention for dissemination and use in a broad audience before it has
been determined that the intervention is effective. Development of a set
of standards or guidelines (i.e., preferred programming languages) and
promoting the use of adaptive and flexible technologies compatible with
those used by the intended audience also was suggested.

*[Emerging]{.underline}*

Barriers to dissemination of emerging CTIs were identified as issues of
ownership/intellectual property, value proposition, sustainability,
customizability, and fragmentation. Problems related to intellectual
property issues could be addressed by the development of public-private
partnerships, licensing agreements, and co-branding opportunities to
promote dissemination of CTIs. NCI could assist in this area by
promoting open-source CTI software, or creating a fund or providing seed
money to support the commercialization of CTIs. To increase the
attractiveness of CTIs for dissemination, plans for dissemination should
consider both the social value and business value. The value of the CTI
should be expressed in metrics valued by the adopters. For example,
value could be defined as the satisfaction of HMO members or numbers of
people who adopted a healthy habit after using the CTI. Also, a
compelling case for "return on investment" must be made. Many users
often wish to customize programs, but the fidelity, and integrity of the
CTIs should be considered. CTIs also should be sustainable and evolve to
fit the users' needs, based on feedback from users and consumers.
Developers also should be encouraged to design CTIs using flexible
technology that does not quickly become obsolete and can be easily
sustained. NCI could promote research to identify incentive structures
for adoption and sustainability and to help developers and adopter
organizations better understand the cost of maintaining systems over
time.

*[Future]{.underline}*

Discussions on dissemination of future CTIs resulted in the suggestion
that CTI developers focus more on the interests and needs of the
end-user audience, which will help ensure "pull" for the CTI (i.e., the
intended audience will want to use it). Existing effective dissemination
strategies, such as HMO member sites, electronic medical records, or
widely used IT platforms (such as Google) should be considered.
Opportunities for large scale dissemination of interventions that
utilize these dissemination vehicles should be developed. Another
important issue for future dissemination efforts includes overcoming the
perceived lack of value for evidence-based behavior change programming
from industry, clinics, and the public, many of whom consider CTIs to be
no better than common sense or unnecessary for promoting desired
behavior changes. NCI could play a valuable role in this effort by
communicating strongly the power of prevention and the importance of
tested behavioral change interventions. To improve dissemination
strategies, developers should seek input from or examine the strategies
used by other disseminators of information, such as HMO member sites or
search sites such as Google. Thought-leaders (policymakers, futurists,
health economists) and CTI researchers should be convened to discuss
strategies for dissemination that can help appropriately guide CTI
development.

Funding issues for dissemination research and growth of the field itself
was a topic of interest that arose during the meeting. Interventions are
often developed or studied in a defined or targeted group for cost
reasons, but inevitably will need to be redesigned to work in a broader
audience. The funding timeline means that interventions developed using
R21 funding may be obsolete by the time the investigator procures R01
funding to study the broader applicability of the intervention. Also,
support for and an understanding of dissemination science by members of
study sections is lacking. Participants suggested that the Special
Emphasis Panel for the Dissemination and Diffusion Research Program
Announcement be made into an independent study section. Additionally,
investigators should consider dissemination as they develop their
intervention; a description of proposed dissemination strategies could
be required in research proposals. NCI could assist in this regard by
developing a step-by-step guide to illustrate the activities necessary
for moving an intervention from a research to a practice (i.e., broader
audience) setting. To build a community of investigators interested in
intervention and dissemination research, NCI could provide opportunities
for interactions between intervention and dissemination researchers in
other relevant fields such as statistics, behavioral science, computer
science, the medical sciences, and the communication sciences.
Trans-disciplinary training and communication opportunities should be
encouraged.

A number of suggestions that cut across all three discussion groups were
identified:

1.  Create standards and guidance for technology development.

2.  Change the review process and funding mechanisms and create study
    sections for dissemination research to increase funding of this
    research.

3.  Convene thought leaders and members of the technology industry to
    gain new perspectives and ideas for dissemination strategies. NCI
    should plan workshops and meetings to provide venues for these
    discussions.

4.  Disseminate findings from this meeting to other groups such as the
    Society for Behavioral Medicine, the e-Health Congress, and other
    informatics groups. NCI also could sponsor exhibits at vendor
    meetings for health IT to showcase funded CTIs and to show products
    that vendors could implement by licensing the CTI from the company
    that developed it.

**Dissemination of Existing CTIs via the RTIPs Mechanism and/or
Alternative Approaches**

Dissemination of CTIs via RTIPs could be improved by using the site to
create a community of users and developers and a place where tools for
both CTI researchers and developers could be housed. A registration page
for those using RTIPs could help connect developers, disseminators, and
users. Technical improvements such as the incorporation of Web 2.0
elements could help create discussion groups for sharing information on
topics such as costs for developing CTIs, ways to customize or tailor
CTIs, and strategies for dissemination. A guidance document for RTIPs to
improve users' understanding and tailoring of CTIs for different topic
areas might also help users implement CTIs. Creation of a Wiki function
on RTIPs, such as a page for best practices, also was suggested. Clear
descriptions of interventions could be housed on the RTIPs site, which
will be of use to those seeking to implement an intervention and help
researchers conduct meta-analyses of CTIs.

Improving the dissemination of CTIs via RTIPs or by other mechanisms
will benefit from broadening the types of groups and people involved in
dissemination, such as clinical systems, large health IT companies,
policymakers, and employers. Several participants suggested developing a
"business-to business" model, in which a CTI developed by a research
investigator could be acquired or licensed by a company that would make
the program available for public use. To help ensure that underserved
audiences are still reached by the interventions, a "pro bono" model, in
which NCI requires a business to provide an intervention to underserved
groups in exchange for funds to aid dissemination, was suggested. NCI
also could help identify audiences that might not be reached if industry
disseminates the CTI.

NCI could serve in a convening function to promote inclusion of
businesses and other entities in CTI dissemination by helping develop a
consultant network to support adaptation and provide technological
support for CTIs. Companies and contractors could work with NCI to adapt
CTIs for users as needed. NCI also could help develop new strategic
partnerships with entities such as clinical systems, large health
information technology (IT) companies, policymakers, and employers. NCI
could bring in third-party contractors to provide a centralized system
for hosting an intervention, collecting data, and providing customer
support.

Another important function for RTIPs could be to serve is to vet CTIs to
promote selection and adaptation of the "best" CTIs. Participants
recommended that NCI develop a Program Announcement for upgrading
existing CTIs for dissemination, perhaps through the Small Business
Innovation Research program (or a modification of this mechanism). NCI
also could create research dissemination contracts to encourage
independent contractors to develop ways to disseminate existing CTIs.

**Meeting Outcomes for Day 1**

Ten potential topic areas were initially identified from the group's
discussion. These were consolidated into seven potential workgroups.
Meeting attendees were asked to indicate their interest in participating
in one of these seven workgroups:

1.  Develop Guidance on IT Standards/Best Practices for CTI Development
    (e.g., software with broad platform utility, open-source CTI
    software)

2.  Develop Criteria on Dissemination Potential for NIH CTI Grant Review
    Process

3.  New NIH Funding Initiatives Targeting CTI Development to Fit
    Existing Dissemination Systems with Broad Reach

4.  Convening Meetings of Thought Leaders, Policy Makers, and Decision
    Makers to Explore the Future Business Models in Health IT

5.  Develop a Research Agenda for Disseminating CTI

6.  Develop Special Initiatives to Support CTI Dissemination to Limited
    Resource Delivery Systems for the Underserved

7.  Convene IT Workshops for Software Experts to Develop CTI Authoring
    Tools

Two other unique topic areas that were not consolidated into the seven
workgroups but will be considered for future consideration in the
dissemination of CTIs include:

\* Training of New Researchers

\* Communicate High Impact of "Power of Prevention" and Appreciation of
Evidence

**Meeting Outcomes for Day 2:**

The breakout groups identified the challenges in disseminating CTIs
through the existing RTIPs structure. Other approaches & tools were
proposed for NCI to consider that could facilitate the dissemination of
existing CTIs on RTIPs. These were organized into three broad
categories: 1) funding mechanisms to stimulate adaptation of CTIs for
dissemination, 2) intermediaries for dissemination, and 3)
tools/resources.

**Funding Mechanisms to Stimulate Adaptation of CTIs for Dissemination**

1.  Develop a Program Announcement for upgrading existing CTIs for
    dissemination (SBIR)

2.  Develop a research dissemination contract.

3.  Select and fund adaptation of the "best" CTIs possibly through a
    dissemination grant.

The review process for these mechanisms would need to be considered.

**Intermediaries**

1.  NCI should develop a consultant network to support adaptation and
    technological support for CTIs.

2.  Companies and contractors can work with NCI to adapt CTIs (i.e.,
    bundled) for users &/or build installers into RTIPs for components
    that can be disseminated, such as algorithms and message libraries,
    as needed.

3.  The user community should be expanded to include businesses.

4.  New strategic partnerships (such as clinical systems, large health
    IT companies, policymakers, and employers) should be developed.

5.  Continuous quality improvement (CQI) with current users (determine
    value).

6.  The role of RTIPs as a quality indicator or vetting source to
    identify the best CTIs based on results and ease of use and offer
    these for adaptation to a common platform

7.  Develop a guidance document for RTIPs to improve users'
    understanding and tailoring of CTIs for different topic areas
    (toolkit concept).

8.  NCI can bring in third party contractors to provide a centralized
    system for hosting the intervention, collecting data, and providing
    customer support.

**Tools**

1.  Create a community of users and developers

2.  Technical improvement: Web 2.0 type elements (discussion groups)

3.  Tools for researchers and developers

4.  Create a Wiki function on RTIPs, such as a page for best practices;
    develop a registration page; provide a location for users to ask
    questions.

***Meeting Report***

**Setting the Stage, Workgroup Expectations, and Meeting Outcome
Discussions**

*Jon Kerner, Ph.D.*

*Deputy Director for Research Dissemination and Diffusion*

*Division of Cancer Control and Population Sciences (DCCPS)*

*National Cancer Institute (NCI)*

The goal of this meeting was to discuss the translation of Computerized
Tailored Interventions (CTIs) into practice. Current, emerging, and
future technologies were discussed, along with opportunities for
dissemination of CTIs, and barriers and challenges to designing CTIs for
broad dissemination and implementation. NCI's dissemination of existing
CTIs via the Research Tested Intervention Programs (RTIPs) mechanism on
Cancer Control Plan, Link, Act Network with Evidence-based Tools
(P.L.A.N.E.T.) or other dissemination tools and the feasibility of
adapting CTI technology and software for dissemination to public health
and clinical practitioners also were presented. From a practice
perspective, plans for dissemination must consider whether the tailoring
effort is worth the resources of the practice community, based on the
results. Meeting participants were asked to consider the resources, both
time and financial, of practitioners and practice settings with respect
to implementation of CTIs.

Along NCI's discovery-delivery continuum, research to link discovery to
effective delivery is lacking. Additionally, only a small percentage of
DCCPS' funding portfolio accounts for research on the dissemination of
evidence-based interventions. In the past few years, options for
delivery of interventions have changed, in part due to the increasing
availability of Internet access to larger numbers of people.
Nonetheless, dissemination and implementation of CTIs have lagged. In
the third round of applications for dissemination and implementation
research grants, not one application for dissemination and
implementation of a CTI was received. To address this issue, a trans-NIH
meeting focusing on dissemination and implementation sciences will be
held September 10, 2007.

Current efforts to disseminate evidence-based cancer control
interventions include RTIPs on DCCPS' Cancer Control P.L.A.N.E.T. Web
portal. Current CTIs on RTIPs include two tobacco control interventions,
two screening CTIs and one on diet and nutrition. Seven CTIs (4
screening, 2 tobacco, 1 diet/nutrition) are "on hold" until plans for
dissemination can be developed. A more active approach to research
dissemination, in contrast to current passive diffusion processes such
as publication of scientific articles and presentations at scientific
meetings, is needed.

Practitioners will adopt CTIs into their everyday practice if the CTIs
are easy to use and effective; payers, such as insurance companies or
the Centers for Medicaid and Medicare Services (CMS) must also be
willing to pay for the CTI. Moving evidence-based cancer control
interventions into practice will require scientific push (i.e.,
documenting, improving, and communicating the intervention for wide
population use), delivery capacity (i.e., building the capacity of
relevant systems to deliver the intervention), and market pull or demand
(i.e., building a market and demand for the intervention). Delivery
capacity is an especially critical issue, given infrastructure barriers
to integrating CTIs into practice, along with the tendency of new health
care options to exacerbate health disparities, given differences in
resources and access. Increasing the number of systems and practitioners
providing evidence-based cancer control programs, along with increasing
the number of individuals participating in evidence-based cancer control
programs will help attain NCI's ultimate goal of improving population
health and well-being.

*Bradford Hesse, Ph.D.*

*Branch Chief, Health Communication and Informatics Research Branch*

*Behavioral Research Program*

*NCI*

NCI's overarching goal is to improve population health and quality of
life through cancer prevention and control. Improvement has been made in
controlling diagnoses and deaths from cancer, and since it has been
recognized that a significant number of cancer cases, perhaps
two-thirds, are attributable to behaviors, the behavioral component
represents a significant opportunity for intervention for cancer
control. The Cancer Center Director's Report contained numerous
recommendations for accelerating success in the cancer prevention arena,
including public and private organizations working together to focus on
ways to improve the public's understanding of cancer prevention and
early detection and to better communicate and implement evidence-based
cancer control programs. NCI's Health Communications and Informatics
branch should take the lead in activities to improve understanding of
which communications approaches to cancer prevention are most effective
for a given population in a given setting.

The Internet initially was a restrictive information environment.
However, in response to new information on the way people used the
Internet, adaptations such as easier interfaces (graphic interface) and
ways to link thoughts (hypertext) increased use. The structure of the
Web needed to change to accommodate how people use and integrate
information. Early health information technologies (IT) programs are no
longer around, although they did generate large numbers of publications.
The goal was to have technology that acted like a person; however,
computers instead gather information for people to use to improve
knowledge and behavior.

As artificial intelligence evolved, adaptations such as embedding
artificial intelligence in a larger system (Spell Check) and embedding
social network analysis (Google and Amazon recommendations) became
common as more was learned about how people gather and use information
using computers. Several medical information programs such as Quicken
Health and Intuit were developed, although these were problematic
because the information contained in these programs was not always
evidence-based. As an example of an overextension of the desire to have
computers act socially, Microsoft's "Bob" interface, which evolved into
Microsoft Word's "Clippy" help function was universally disliked by
users. A critical lesson from this experience that can be applied to
e-health research is that interfaces must be consistent, must not be
autonomous, and must not be anthropomorphic. Through dissemination
research it was learned that user interface design goals should be
cognitively comprehensible, consistent, predictable, and controllable,
and should be affectively acceptable, providing a sense of mastery,
satisfaction, and responsibility.

The ways in which medical literature is disseminated also have evolved.
Medical literature changes rapidly and quality care is dependent on
dissemination of "evidence-based" practices. Tools were needed to ensure
that the best, most current evidence is available to clinicians at the
time of care. Historically, infrastructure barriers hindered
dissemination of the most up-to-date information---quickly outdated CDs
were used to deliver personal medical information, computer lookups
required special training, search processes were cumbersome, and access
was not compatible with clinical demands. As the Internet has matured,
development of sites such as PubMed and the use of Digital Object
Identifiers and hypertext links have improved access to literature
describing evidence-based interventions.

Emerging trends in e-health include information gathered by the Health
Information National Trends Survey (HINTS), which monitors how Americans
find and use information on cancer. Increasing numbers of Americans are
seeking health information using the Internet, a trend that likely will
increase with subsequent generations. Consumers are being encouraged to
take more responsibility for their health care. United Kingdom Health
System Goals to empower people to live healthier lives are based on
findings that showed when levels of public engagement in relation to
health are high, life expectancy increases beyond forecasts, health
status improves dramatically, people are confident in the health system
and demand high quality care, use of resources is more efficient, and
the health services are responsive with high rates of technology uptake,
especially in relation to disease prevention.

Given trends and changes in infrastructure, the time is ripe to
personalize health care. The Internet and computer kiosks can provide
pertinent information that permits patients to actively participate in
their own care and make better health decisions, leading to increased
compliance with treatment, a healthier population, and potentially lower
health care costs across the country. Electronic personal health records
will provide opportunities for tutorials, tailoring and further
personalization of care.

Companies such as Intel are becoming interested in ubiquitous computing.
This approach could provide monitoring alerts to initial signs of
problems, prompts for taking medications and how to take them, remote
monitoring of health conditions, and long term monitoring to assess
changes in abilities and behaviors, thus providing continuously updated
records of health. Despite its potential benefits, this approach may
appear too similar to a "Big Brother" type of situation for some
consumers. Gathering evidence about the impact of such monitoring could
help avoid this.

NCI has many resources that can be of help in the field of dissemination
research and dissemination. Cancer Control P.L.A.N.E.T. provides links
to comprehensive cancer control resources for public health
practitioners. Dissemination grants are available, as is funding through
the Small Business Innovation Research (SBIR) Program, which seeks to
increase private-sector commercialization of federal research and
development, particularly by small business and minority and
disadvantaged companies' participation in federally funded research and
development. The NCI Usability Laboratory links intervention researchers
to usability experts. NCI has the "power to convene" experts in various
fields through meetings and workshops. Lastly, NCI also can help to set
standards for personal health records.

**Organization of the Meeting**

*Dr. Kerner*

The participants were divided into three break-out groups and asked to
discuss challenges and barriers to the implementation of CTIs. The first
day's discussion focused on identifying the range of delivery audiences
(i.e., those delivering CTIs to users) relevant to the design of CTIs
for dissemination and implementation. Participants also were asked to
identify infrastructure, technology, and other barriers and challenges
to delivering CTIs across multiple audiences and those specific to a
given delivery audience. Participants were then asked to develop
solutions to these barriers and challenges. Each group was asked to
consider either current, emerging (i.e., funded and in development), or
future (i.e., in the next 5 years) CTIs. The groups also were asked to
consider other entities and sectors of both the research and practice
communities who should be involved in the development of strategies to
disseminate and implement CTIs.

The second day's discussion focused on challenges for disseminating
existing CTIs that are not supported by developers via RTIPs.
Participants were asked to discuss and propose ways to integrate current
CTIs into the RTIPs structure, including infrastructure, technology, and
any other challenges that NCI may need to address. The break-out groups
were all asked to identify the highest priority changes needed on the
RTIPs Web site, if RTIPs was determined by the group to be a viable CTI
dissemination tool. If RTIPs was determined to be inadequate, the groups
were asked to propose alternative models or tools for dissemination of
existing CTIs and also resources needed to support these alternative
models.

**Discussion**

Dr. Vic Strecher suggested that NCI consider the pharmaceutical
development model. NCI funds development of new cancer drugs or
treatment procedures, but relies on pharmaceutical companies to
disseminate the drugs and knowledge about the new drugs and treatments.
This results in widespread access to new cancer therapeutics. Dr. Kerner
answered that it may be difficult to apply this model to public health
interventions and that this development model has not always led to
equal access to newly developed therapeutics and interventions.

Mr. Tom Eng suggested that the main role of NCI and other federal
agencies should be to test concepts and ensure that they meet safety
standards and serve as independent, trusted agencies that evaluate
products for the consumer. NCI can assist the dissemination process by
evaluating and deciding whether an intervention is successful in a
defined setting. Dr. Wayne Velicer noted that although technology is
improving communications and allowing for more sophisticated means of
communication, different people will be reached by different channels
(i.e., print, radio, Internet) and suggested that the group consider
advantages and disadvantages for each channel.

Dr. Alexander Prokhorov noted that many of the CTIs in question have a
"shelf-life" and that the group needs to balance being on the cutting
edge of current dynamic technologies with the consideration that testing
and developing the evidence base for interventions can take time. Dr.
Kerner agreed that this was a significant challenge. For example, RTIPs
provides access to publications and other information that may be over
25 years old, yet these same concepts are re-invented time and again.
Technology sometimes can move faster than the ability of research to
evaluate its applicability.

Dr. Kerner clarified that the trans-NIH dissemination PAR was
established because of a lack of research to analyze and develop
effective strategies for dissemination. He noted the variety of
expertise, knowledge, and experience of the participants and asked them
to propose priorities that should be acted on immediately or in the
future, and to consider how they themselves could be involved in these
action steps. Dr. Kara Hall added that participants should focus on the
delivery audience (i.e., practitioners who deliver an intervention to
the user). Some examples of delivery audiences include the Veterans
Administration (VA), a work site, or public health practitioners.

Dr. Strecher asked if the participants should consider how delivery of
an intervention would be funded. Dr. Kerner answered that reimbursement
could be considered in the context of an infrastructure challenge.
Reimbursement should not be excluded from the discussion, but should not
dominate and strategies should not be eliminated because there is no
current mechanism to fund them. Dr. Hesse agreed that participation of
payers at future meetings could be useful.

Dr. Seth Noar asked whether NCI has a record of visitors to the RTIPs
site and users of RTIPs programs. Dr. Kerner answered that an evaluation
of Cancer Control P.L.A.N.E.T. has determined some characteristics of
the user audience (i.e., public health workers, clinical workers).
Programs are frequently downloaded from the site, but there is no
information concerning contact made to the researchers who posted the
intervention. Questions concerning use of the programs after downloading
have found that only a small number of users have implemented the
intervention as developed; instead, many users adapt the program to
their own setting. A significant amount of community-based intervention
work appears to be funded by various philanthropies. NCI has identified
approximately 500 philanthropies that fund cancer control interventions
and service delivery, and is considering whether partnerships can be
developed with them to ensure that they fund evidence-based
interventions. One goal is to link philanthropies to Cancer Control
P.L.A.N.E.T. to increase the number of evidence-based grant applications
submitted to the philanthropies.

Dr. William Rakowksi asked for a definition of "computerized" as it
pertains to CTIs. Dr. Hall answered that in this setting, the term
"computerized" includes protocols that require a computer to drive the
intervention. For this meeting, a CTI was defined as "an intervention
using a computer to generate tailored feedback and suggestions on an
individual basis, in this case, to promote healthy behavior change."

Dr. Kerner reminded the participants that only a small portion of the
NCI budget is targeted to dissemination and diffusion research, but the
goal is to expand this; thus participants are asked to help guide NCI's
decisions for allocating dissemination and diffusion research resources.
Dr. Strecher noted that without NCI, ProChange and Health Media would
not exist; thus, NCI should take some credit for effective
dissemination. Dr. James Prochaska agreed that NCI should take credit
for the grants it has supported, which has led to the development of new
science, interventions, and businesses which have had a significant
impact on cancer control. Dr. Strecher suggested that representatives
from WebMD, Google, Intel, and other such companies be asked to
participate in future meetings, since the business of these companies is
to disseminate information.

**Break-out Groups**

Participants were divided into three break-out groups and asked to
consider barriers and challenges to dissemination of current, emerging
(i.e., funded and in development), or future (i.e., in the next 5 years)
CTIs, along with other entities that should be involved in the
development of strategies to disseminate and implement CTIs. Detailed
reports of each groups' discussion are found in Appendix A (current
CTIs), Appendix B (emerging CTIs), and Appendix C (future CTIs). An
overview of the discussions for all three groups is provided below.

**Report-back from Breakout Groups**

**Current technologies**

Adopters of CTIs who should be considered in these discussions include
insurers, providers of health care, health care promotion groups (i.e.,
ACS), for-profit companies, county health departments, HMOs, schools,
and workplaces. Within each group there are large and small, public and
private, non-profit and for-profit entities. CTIs must be attractive to
their intended target audience and plans for dissemination must consider
the target audience. Barriers to dissemination of CTIs were categorized
as either technology-based, related to cost or expense, or related to
fit (i.e., is the CTI what the intended audience wants or needs).

Related to the technology barrier, many developed CTIs do not stand
alone but instead require IT support or may need to be adapted for use
by a broader audience than that on which the CTI was developed. Some
CTIs may not be compatible with current technology platforms used by the
intended audience and also may not be easily updated (made compatible
with current technology or updated because of changes in health
guidelines or recommendations). However, it was recognized that it is
not always possible to develop an intervention for dissemination and use
in a broad audience before it has been determined that the intervention
is effective. Solutions include designing the intervention with
dissemination in mind, perhaps through the development of a set of
standards or guidelines (i.e., addressing programming language).

The cost of implementing and adapting interventions for use by specific
target audiences also was considered a barrier to dissemination.
Centralized access to technology assistance for making adaptations and
running programs, instead of requiring each audience to adapt an
intervention on their own could help increase use of CTIs. A centralized
mechanism to support and provide resources for CTI dissemination could
include non-profit groups (such as LAF or ACS) to help provide
infrastructure, IT support, and support for adapting interventions for
use in specific populations was proposed. Development of a consortium of
researchers to provide a forum for discussion of how to design CTIs with
dissemination in mind also could help ensure that CTIs are easily
adapted to real world use.

CTIs also need to fit with the requirements and desires of the target
audience. A CTI as designed may not be culturally relevant and also may
be too specific. For example, a practitioner may wish to provide a
general intervention on cancer screening, rather than specific screening
interventions for each type of cancer. To encourage development of
broader CTIs, reviewers must be trained to value generalizability,
dissemination ability, and potential for use in the real world.

**Emerging Technologies**

The delivery audiences can be divided into two categories: 1) adopter
organizations such as schools, employers, or community health
organizations, and 2) delivery organizations such as companies and
for-profit and non-profit health organizations. CTI barriers and
challenges include issues of ownership/intellectual property, value
proposition, sustainability, customizability, and fragmentation. These
challenges apply to most organizations.

To help resolve issues of ownership and intellectual property,
public-private partnerships to promote dissemination of CTIs, licensing
agreements, and co-branding should be developed. NCI could provide
assistance with intellectual property issues by promoting open-source
CTI software or the creation of a fund or provision of seed money to
support commercialization of CTIs.

Value proposition refers to distinguishing between the social value and
the business value of a CTI. The values of all organizations involved in
development, dissemination, and use of a CTI must be understood, and
plans for dissemination should take advantage of these values. The
express value of the CTI should be framed in terms of metrics valued by
adopters (i.e., satisfaction of members, number of people who stop
smoking). A compelling "return on investment" model is needed to
encourage adoption of CTIs.

CTIs must be sustainable; they must evolve to fit the users' needs,
based on feedback from users and consumers, and support for CTI
evolution must be provided. NCI could promote research to identify
incentive structures for adoption and sustainability. The technology
platforms of CTIs may become obsolete; this must be addressed by
ensuring that the relevant technology is sustained and works properly
over time. For example, implementation of electronic medical records has
suffered a 40 percent failure rate in the practice setting because there
has been inadequate support for sustaining use of these records. Both
developers and adopter organizations must understand the cost of
maintaining systems over time.

Customizability also is an important issue, because people and
organizations often wish to put their own stamp on programs.
Customizability is desirable, with the caveat that changes do not
negatively impact the sustainability, fidelity, or integrity of the CTI.
A solution for this issue is to develop standards for customizability.

**Future Technologies**

CTIs often fail to address the interests or needs of both intermediary
and end-user audiences. For example, an HMO may be most interested in
the usability of a CTI, a school may be most interested in its impact on
test scores, and an employer may be most interested in the CTI's impact
on productivity. Developers should consider how to ensure there is
"pull" for CTIs and that the intended audiences want to use them. To
address this issue, existing effective dissemination strategies, such as
HMO member sites, electronic medical records, or widely used IT
platforms (such as Google) should be considered and opportunities for
large scale interventions that fit within these dissemination strategies
should be developed.

A lack of perceived value for evidence-based behavior change programming
from industry, clinics, and the public is another barrier to CTI
dissemination. Many consider some CTIs to be no better than common sense
or believe that they may be able to promote desired behavior change
without the use of CTIs. To address this issue, NCI should communicate
strongly the power of prevention and the importance of behavioral change
interventions as applicable to CTIs.

The business model in health IT continues to change and is shifting
responsibility to the end user; CTIs must acknowledge this change.
Although electronic medical records do not currently focus on
prevention, they will be a major tool that can either limit or
accelerate the implementation of CTIs. Convening thought leaders
(policy-makers, futurists, and health economists) together with CTI
researchers will help guide CTI development in the appropriate
direction.

**Discussion**

Dr. Kerner asked the participants to consider how intervention
development should be changed, for example, whether dissemination should
be considered at the time of development rather than after the fact. An
issue related to this is how evaluation of dissemination potential can
be integrated into the peer review process. He suggested a "second tier"
review in which dissemination experts would advise NCI on the
dissemination potential of a proposed CTI. Dr. Kerner noted the
suggestion by Group 3 that well-established dissemination channels be
considered in the design of interventions, and noted that CRN and the
Health Services Research Branch could be helpful in this matter. Dr.
Kerner also suggested creating a Master Agreement Order, which could
provide for creation of a group of people who compete to provide support
and adaptation services for compelling interventions, although rules
would need to be established to ensure that smaller companies can
compete for this work. NCI could play a convening role by bringing
together groups that normally do not interact.

Mr. Ed Saunders commented that he preferred requiring investigators to
describe dissemination strategies on their research proposals rather
than having a two-tiered grant review. Also, scientists who review these
proposals need to be informed about intervention dissemination, which
will be challenging to integrate into the standard peer-review process.
The gap between the establishment of efficacy and dissemination is
significant. Many investigators may not be motivated to apply for
dissemination grants and may not fully understand what is needed to
properly disseminate their interventions. NCI could help develop a
step-by-step guide to illustrate the steps necessary to move a CTI from
a research setting to one of broad applicability. CTI developers also
should be encouraged to have their CTI deliverable by different
mechanisms. Although it will be difficult to integrate dissemination
potential into the efficacy review process, this will be important to
ensure that more CTIs are moved into practice. However, Mr. Saunders
cautioned that including criteria for evaluation of dissemination in the
review process should not increase the length of time it takes to have a
grant reviewed and funded. A streamlined process in which dissemination
is part of the review process is preferable to a two-tier system. In
addition, although CTIs could be built on successful dissemination
platforms, Mr. Saunders cautioned against having NCI exert influence
over the commercial arena, which could hinder creativity. Dr. Kerner
agreed that NCI should be cautious about dictating implementation
specifics. NCI could develop platforms on which developers could build,
but not specific programming solutions. Locking developers into a
technology that might be obsolete in the future should be avoided.
Establishment of a set of best practices for both content and delivery
also could be useful.

Regarding the lack of perceived value for behavioral change programming,
this relates to dissemination of all three CTI groups (current,
emerging, and future) because it impacts what will be adopted and paid
for by users. Mr. Saunders expressed concern that the public does not
appear to realize that behavior programming is important because roughly
75 percent of cancers are caused by behaviors. He suggested that
literature and meta-analyses of behavior change research and its impact
are lacking. A case for population-attributable benefit of behavior
change, specifically related to cancer prevention, needs to be made,
because it is significantly more cost-effective to prevent cancer rather
than treat it when it occurs.

Many organizations have well-developed strategies and incentives (both
positive and negative) to encourage participation in behavior change
programs and developers of CTIs need to better understand these
strategies. The lack of literature on population health management may
be related to the lack of infrastructure for disease registries and
electronic health records; thus, the information is not easily
consolidated into a single review. A cultural change also will be needed
for the population to understand the benefit and importance of behavior
changes, much as the benefits of pharmaceutical interventions are
understood. The media must be influenced to promote effective behavioral
interventions more strongly. Researchers should be educated on how to
contact and interact with the media to advocate for behavioral change
research and interventions.

Concerning effectiveness, an intervention must be shown to work, but
also must have applicability in an uncontrolled environment and must
retain its effectiveness when widely disseminated. During the
development process, the investigator should consider how to best
communicate the effectiveness and importance of the intervention and how
it will be disseminated. Grants to encourage commercialization could be
useful, but may be hindered by lack of knowledge about the future
technological and business environments. Standards for software should
ensure that CTIs can be tailored, re-used, and sustained by users; the
ability to interact with other interventions also should be considered.
An R21 mechanism or other funding mechanism could be used to develop a
proposal for development of software that could be used to make CTIs
more broadly applicable. The success of such a mechanism would depend on
ensuring that experts in CTIs and dissemination serve on study sections
reviewing such grants.

To increase public and industry awareness of the value of behavior
change interventions, better marketing of such interventions is needed.
For example, one study has found that a CTI alone is as effective as a
CTI plus counseling for depression treatment. Delivery of a CTI in
conjunction with something familiar (such as counseling) could encourage
use of CTIs. Adding relevant outcome measures to a CTI also could help
dissemination. For example, a CTI could measure the user's satisfaction
with the HMO that delivered the intervention, which would be a metric
that could increase the HMO's desire to use a CTI.

Dr. Hesse suggested that the National Institute of Standards and
Technology could be helpful in developing technology guidelines. He
noted, however, that too many requirements and barriers could inhibit
the creativity and innovation of technology developers. He also asked
whether use of open source programming should be a requirement. Dr.
Hesse suggested that an action step arising from this meeting could be
to ask meeting participants to work with NCI to develop guidelines for
dissemination technology.

Regarding the shift to a consumer-directed approach to health care, many
insurance companies wish to shift the burden of health care costs to the
consumers without understanding the ability, or inability, of consumers
to make health care decisions. This could be an opportunity for CTIs to
provide important decision-support tools that may be critical to
consumer decision-making. Also, foundations that work with technology,
such as the Gates Foundation, may be interested in working in the arena
of health behaviors.

A meeting to specifically follow up on discussions and issues raised at
this meeting also could be planned, and participants were asked to
suggest others who should be invited. It was suggested that informatics
groups working to counteract the fragmentation of health care should be
invited. For example, Google works to integrate information and
knowledge. Representatives from smaller organizations, such as community
health departments that are interested in using CTIs also should be
invited. Similarly, those who interact with minority populations should
be included in future meetings. Dissemination science researchers who
can discuss core dissemination issues not specific to CTIs also should
be invited. Another issue of interest is that of training a new
generation of CTI researchers. Training in this field involves a number
of fields, including statistics, behavioral science, computer science,
communication sciences, and medical sciences. The participants of this
meeting were asked to consider how to encourage transdisciplinary
training and communication opportunities to lead to better quality CTIs.

Dr. Hall reviewed crosscutting themes arising from all three breakout
group discussions:

-   Develop standards and guidance for technology development.

```{=html}
<!-- -->
```
-   Change the review process and funding mechanisms and create study
    sections for dissemination research to increase funding of this
    research.

```{=html}
<!-- -->
```
-   Convene thought leaders and members of the technology industry to
    gain new perspectives and ideas for dissemination strategies. NCI
    should plan workshops and meetings to provide venues for these
    discussions.

```{=html}
<!-- -->
```
-   Disseminate findings from this meeting to other groups such as the
    Society for Behavioral Medicine, the e-Health Congress, and other
    informatics groups. NCI also could sponsor exhibits at vendor
    meetings for health IT to showcase funded CTIs and to show products
    that vendors could implement by licensing the CTI from the company
    that developed it.

Participants were asked to indicate interest in participating in any
action steps developed from these discussions, with the idea that NCI
would contact participants at a later date.

**August 3, 2007**

**Setting the Stage**

*Irene Prabhu Das, Ph.D.*

*Office of the Director, Research Dissemination and Diffusion*

*DCCPS\
NCI*

Friday's discussions were to focus on dissemination of existing CTIs
through the RTIPs mechanism on Cancer Control P.L.A.N.E.T. Two types of
RTIPs are available: those that can be ordered from the program
developer and those that can be downloaded directly from RTIPs with no
developer support. Challenges arise when the CTI developer no longer
wishes to provide support.

RTIPs have been evaluated by peer-reviewed research grant committees,
published in peer-reviewed journals, and can be adapted and used in
community or clinical settings. RTIPs have been tested and proven to be
effective in research environments. This resource can be used to
identify programs and materials suitable for a given population, provide
program materials that can be adapted for another population or for
delivery in a related setting, identify programs to include in a
replication or dissemination research project, and provide a means to
contact a principal investigator for collaboration purposes or to obtain
consulting support for a project. Topic areas on RTIPs include breast,
cervical, and colorectal cancer screening; diet/nutrition; informed
decision making; sun safety; physical activity; and tobacco control. A
new adaptation tool called "Using What Works" is now available to teach
users how to adapt a program on RTIPs for use in their own communities.
The Cancer Information Service (CIS) trains practitioners and other
state and local partners on this tool.

Dr. Prabhu Das reviewed an example to demonstrate the challenge in
disseminating CTIs via RTIPs. The Personally Relevant Information about
Screening Mammography (PRISM) provides the user with a summary of the
program, and informs users about the integrity of the research, its
dissemination capability, the cultural, gender, and age appropriateness
for program use, as well as the effect size and reach of the
intervention (intervention impact). Currently, PRISM provides
downloadable materials that can be printed as a booklet for the user's
context, but does not provide the resources needed for it to be
implemented as a tailored intervention.

The meeting participants were asked to determine if and how existing
CTIs can be disseminated through RTIPs. If existing CTIs cannot be
disseminated through RTIPs, participants were asked to propose
alternatives to disseminating existing CTIs for NCI's consideration.
Priorities for dissemination of existing CTIs also were to be
considered. The participants were reminded that discussion should focus
on the dissemination of CTIs when the developer does not wish or is
unable to provide support for the user.

**Discussion**

Mr. Saunders asked if NCI was interested in hosting a CTI delivery
system on the Web. Dr. Kerner answered that at this point NCI cannot
determine to whom the CTIs would be delivered. Public health
practitioners who had participated in Cancer Control P.L.A.N.E.T.
training were surveyed to learn how they used the web portal. However,
the survey did not show who specifically adapted RTIPs for their own
use. Cancer Control P.L.A.N.E.T. delivers information to the
practitioners and program planners, but the information is not tailored.
NCI will not be the primary conveyor of information to its audience or
the main provider of technical support; instead, the goal is to provide
tools to adapt RTIPs to those who can deliver interventions to users.
Currently, this can be done for non-tailored, but not tailored,
interventions.

Ms. Cynthia Vinson explained that usability testing of current programs
on RTIPs is in progress. Simplified implementation protocols are being
created for use by people in the community, but resources including
staff and computers are needed. Dr. Kerner noted that each step on
Cancer Control P.L.A.N.E.T. and RTIPs provides the user with a "Contact
Us" option. Innovations on RTIPs are driven by the community of
practice, and a new kind of dialogue with the users of CTIs may be
needed. Dr. Velicer commented that current laws for intellectual
property provide an investigator's institution with rights to anything
developed by the investigator, which should be considered when ideas for
marketing are discussed. Also, although shareware can be useful, it
becomes quickly dated because of a lack of support. Protected software
usually has more support for maintenance. Dr. Kerner explained that
approximately 60 percent of RTIPs use shareware and the remaining 40
percent use proprietary codes and require permission for use. Dr. David
Ahern noted that for CTIs to be useful, further tailoring usually is
required; a consulting industry exists for performing tailoring to adapt
the CTI to the users' settings and target populations.

Each breakout group was asked to discuss potential solutions and
recommendations to disseminate existing CTIs that are not supported by
the developers. Groups were also asked to discuss alternatives to RTIPs
for dissemination of existing CTIs and the challenges and solutions to
developing these alternatives.

**Breakout Groups**

Participants were divided into three breakout groups and all three
groups were asked to consider the dissemination of existing CTIs via the
RTIPs website or other appropriate mechanisms. Detailed reports of each
groups' discussion are found in Appendix D, E, and F. An overview of all
three groups' discussions is provided below.

**Dissemination of Existing CTIs via RTIPs**

Challenges in disseminating CTIs through the existing RTIPs structure
were identified. Other approaches & tools were also proposed for NCI to
consider that could facilitate the dissemination of existing CTIs on
RTIPs. These were organized into three broad categories: 1) funding
mechanisms to stimulate adaptation of CTIs for dissemination, 2)
intermediaries for dissemination, and 3) tools/resources.

**Funding Mechanisms to Stimulate Adaptation of CTIs for Dissemination**

1.  Develop a Program Announcement for upgrading existing CTIs for
    dissemination (SBIR)

2.  Develop a research dissemination contract.

3.  Select and fund adaptation of the "best" CTIs possibly through a
    dissemination grant.

The review process for these mechanisms would need to be considered.

**Intermediaries**

1.  NCI should develop a consultant network to support adaptation and
    technological support for CTIs.

2.  Companies and contractors can work with NCI to adapt CTIs (i.e.,
    bundled) for users &/or build installers into RTIPs for components
    that can be disseminated, such as algorithms and message libraries,
    as needed.

3.  The user community should be expanded to include businesses.

4.  New strategic partnerships (such as clinical systems, large health
    IT companies, policymakers, and employers) should be developed.

5.  Continuous quality improvement (CQI) with current users (determine
    value).

6.  The role of RTIPs as a quality indicator or vetting source to
    identify the best CTIs based on results and ease of use and offer
    these for adaptation to a common platform

7.  Develop a guidance document for RTIPs to improve users'
    understanding and tailoring of CTIs for different topic areas
    (toolkit concept).

8.  NCI can bring in third party contractors to provide a centralized
    system for hosting the intervention, collecting data, and providing
    customer support.

**Tools/Resources**

1.  Create a community of users and developers

2.  Technical improvement: Web 2.0 type elements (discussion groups)

3.  Tools for researchers and developers

4.  Create a Wiki function on RTIPs, such as a page for best practices;
    develop a registration page; provide a location for users to ask
    questions.

**Discussion**

Mr. Eng recommended that any third party contractors considered for
participation in this work have experience with commercialization of
health care tools and a solid understanding of the context, users, and
market for such tools. Dr. Kerner asked for input on the information a
contractor or developer might need for development of implementation and
dissemination strategies for CTIs. Dr. Kerry Evers suggested seeking
input from those in the development field and providing them with
descriptions of the CTI with attention to resources needed to implement
it and appropriate audiences and added that clear descriptions of
interventions will help develop meta-analyses of CTIs. Dr. Velicer
agreed that this information was important and that more journals that
publish research dissemination and diffusion research are needed. Dr.
Kerner noted that a meeting of journal editors was convened to discuss
publication limitations of clinical intervention implementation and
dissemination research. Some journals (such as the *Journal of Public
Health Practice and Management*) now require authors to address the
clinical practice implications of their work.

Dr. Celette Skinner recommended that the Web site provide information on
the cost of an intervention fairly early in the process of selecting an
intervention. Dr. Kerner explained that it is difficult to gather this
information because it is complicated to segregate research costs from
practice costs. This issue speaks to the need for cost-effectiveness
analyses. Dr. Hesse suggested that community feedback or discussion
boards could provide information on the costs of using an intervention.

Dr. Matthew Kreuter stated that RTIPs as it is currently structured may
not be appropriate for dissemination of complex CTIs. Dr. Kerner
suggested that the idea of using intermediaries could address this
issue. A program from RTIPs could be downloaded or sent to a developer
or contractor who administers the CTI. Contractors also could take
pieces of an evidence-based CTI and re-formulate it in a way such that
it can be delivered to users from a single source. One issue relevant to
this is how to convey the value of CTIs to potential users.

Dr. Velicer described three alternative dissemination channels. The
first involves conveying a CTI to a business that will license and
distribute it, similar to the pharmaceutical industry model. CTIs also
could be delivered to the public through health departments. The third
model involves connecting the researcher who developed the CTI with
other researchers who will further develop the CTI for broader use. Dr.
Hall commented that NCI may need to assess the potential role of
industry in dissemination and also can identify audiences that might not
be reached if industry disseminates the CTI.

Dr. Strecher argued that significantly more impact would be achieved by
marketing CTIs through businesses rather than through public entities
such as health departments. NCI could, for example, partner with Google
Health to provide them with evidence-based health promotion tools.
Independent contractors funded by NCI to adapt CTIs for public use also
could help disseminate these interventions more widely. Dr. Kerner
agreed that dissemination by large commercial enterprises will be
effective for certain audiences, but contractors might be needed to more
effectively reach underserved audiences. Dr. Hall suggested a "pro bono
model" in which NCI provides funds for development to businesses, with
the requirement that the business must also provide the intervention to
underserved populations. Dr. Karen Emmons noted that this model may also
apply to cooperation with nonprofit groups. Dr. Strecher explained that
businesses such as Health Media work with state health departments and
underserved groups. Retrofitting CTIs for a real world audience can be
expensive and done more cost-effectively by larger businesses, which
then can provide the CTI to underserved groups at a lower cost.

Dr. Velicer commented that development costs of CTIs include more than
just changes to the software. Dr. Kerner noted that this points to the
need for case studies and cost analyses to understand the real costs
from inception to dissemination. Dr. Hesse observed that individually
deployed CTIs usually end up fragmented and do not usually lead to
significant health changes. CTIs that are more broadly based might be
easier to deliver. Dr. Ahern noted that CTIs created 3 or 4 years ago
largely stand alone. A "portal experience," which draws from original
tools to present a single program to a client will not stand alone. Dr.
Evers added that dissemination strategies must consider which systems
and programs will be most relevant in the near future (approximately 5
years). Dr. Kreuter provided an example of costs to disseminate a CTI:
an approximate cost of \$200,000 to develop and another \$50,000 to
develop kiosks to disseminate the CTI after efficacy had been shown.

Dr. Skinner agreed that most CTIs are not feasible to deliver to
organizations without some changes. She suggested development of a
"tailoring engine" that could create a quasi-tailored version of the CTI
that more closely meets the needs of the user. Dr. Kerner suggested a PA
that calls for ways to adapt an intervention to make it more useful in a
certain setting. Research would be needed to determine if the adapted
version had the same effect as the original CTI. Dr. Kreuter suggested
creating profiles of audience segments to help determine parameters
specific to different audiences, such as motivation.

Dr. Prokhorov cautioned that some populations may have little or no
access to the Internet and other means of dissemination of CTIs
(television, radio, telephone help lines). Dr. Strecher commented that
analyses of tailored versus untailored interventions are needed, along
with how to determine major elements that could be targeted instead of
tailored. Empirical research to determine the variables that must be
tailored is needed. Dr. Kreuter commented that given an existing CTI, it
is possible to determine which tailoring variables vary. Those that did
not vary when the intervention was successful would be good variables
for targeting. Dr. Velicer agreed that determining the effect size of
various constructs is the most practical aspect of tailored or targeted
intervention research.

A general consensus was that complex CTIs need to be simplified.
Technology changes along with increased use of new devices and programs
will impact this. Dr. Kerner explained that one of the aims of the
Cancer Control P.L.A.N.E.T. initiative is to make information available
about technologies that will work alongside and complement the
technological tools that are used in the private sector. Dr. Stephen
Intille suggested partnering with the National Science Foundation (NSF)
and recruiting talented undergraduate students from NSF's Research
Experiences for Undergraduates program to perform research on
simplification of CTIs for dissemination as proof-of-concept projects.
This would have the added advantage of exposing students interested in
technology to the field of public health. Dr. Hesse added that students
in the Cancer Communication Internship Program also could work on such
projects.

Dr. Kerner agreed that NCI would work to increase interest in
dissemination in new groups of people and constituencies. In response to
questions concerning feedback from the meeting, Ms. Vinson explained
that a summary or guidance document for RTIPs would be developed, which
also will be helpful for enlisting participation from third party
contractors in the future. The discussion points will be used to develop
new possibilities for RTIPs. Participants were encouraged to contact NCI
with new ideas or follow-up questions or comments from this meeting.

Dr. Kerner explained that the feedback and ideas generated at this
meeting will be compiled and prioritized within the Research
Dissemination and Diffusion Team with consideration of resources and
funding. The Team will collaborate with other Offices, Divisions, and
Centers within the NCI to make the best use of existing resources and
programs. Participants of this meeting will also be contacted based on
interest expressed at this meeting. Drs. Hall and Prabhu Das thanked
everyone for their participation and perspectives provided by both
technology and behavior research participants.

***Appendix A***

**Breakout Group 1: Current CTIs**

*Facilitators: Irene Prabhu Das and Sarah Kobrin*

Group 1 discussed ways to help providers (practitioners, community
health clinics, government groups, advocacy groups, state cancer
coalitions, large employers, etc.) use and deliver computer-tailored,
evidence-based interventions. Other delivery audiences identified
include the Health Resources and Services Administration (HRSA), Centers
for Medicaid and Medicare Services (CMS), insurance companies,
for-profit hospitals, the VA, clinics, federally funded community health
centers, schools, health maintenance organizations (HMOs),
non-governmental organizations (NGOs) for the poor, churches, and other
entities that cater to minorities or the economically disadvantaged.

Barriers were described as technical, financial, or lack of fit (i.e.,
the intervention does not meet the needs of the user). Issues pertaining
to the dissemination research field itself also were discussed.

**Technology Barriers:**

1.  An intervention that is not "stand- alone" (i.e., other input or
    support is necessary before it can be used) may be difficult to
    deliver, due to financial reasons or lack of staff. However, fully
    automating an intervention may change the intervention.

2.  An intervention may be based on obsolete technology. The speed of
    changing technology raises barriers to dissemination of
    evidence-based interventions because the technology used may be
    obsolete by the time all necessary evidence is gathered.

3.  Dissemination to a broader audience raises security issues (there
    will be more health information to secure).

**Technology Solutions:**

1.  To avoid obsolescence, newly developed interventions must be
    designed to be compatible with or easily adapted to future (i.e., 5
    years) technology and should be back-compatible. Interventions also
    must be adaptable to the users' technology. Advice should be sought
    from those in the IT sector to help determine systems and platforms
    least likely to quickly become obsolete.

2.  Web-based interventions should be encouraged because these are less
    likely to become technologically obsolete. The Web has standards set
    by an international consortium that should be adhered to because the
    Web is global.

3.  Standards should be developed for storage and security of content
    and the content of the intervention should be easily updated (i.e.,
    update guidelines that change over time).

4.  Guidelines for R21 grants should encourage design of interventions
    usable on multiple platforms.

5.  An R01 to convert old software to newer technologies with a broader
    reach should be considered.

6.  A group should be established to develop a set of general
    dissemination guidelines to help investigators ensure that an
    intervention is flexible and can be used in the future. Use of
    international standards for networking, Web development, and
    database design should be encouraged. Usability.gov also has
    standard rules based on research on the best ways to build Web
    sites, etc., that could help with standardization of CTI software.

7.  Use of common technologies (i.e., Java) and programming should be
    encouraged. The use of the newest technology (before it is proven to
    endure) should be discouraged. Unusual choices should be justified
    by a specific research finding (for example, a mass mailing may use
    an unusual envelope size because research has found unusual
    envelopes to be more noticeable).

**Cost/Expense Barriers:**

1.  Lack of insurance coverage for an intervention a provider wishes to
    deliver.

2.  Costs for purchasing, training for implementation, maintaining,
    updating, and sustaining an intervention.

3.  Lack of cost-effectiveness analyses to encourage large for-profit
    organizations to use a CTI.

**Cost/Expense Solutions:**

1.  Centralize CTIs and dissemination through advocacy groups and other
    health organizations (American Cancer Society \[ACS\], CMS, Lance
    Armstrong Foundation \[LAF\]); this could help low resource settings
    gain access to the CTIs. Organizations also should be encouraged to
    invest in sustaining interventions for real world use.

2.  Develop marketing efforts to reach out to existing delivery systems.

3.  Partner with funding organizations (Gates Foundation, computer
    companies for CTIs) and involve payers early in the process of
    intervention development. These groups also could help customize
    interventions.

4.  Work with the Cancer Information Services (CIS), Cancer Research
    Network (CRN), and other NCI resources.

5.  Consider changes to the SBIR/STTR funding mechanisms to make them
    more attractive to small companies who may be able to help with
    customization and dissemination of CTIs for the delivery audiences.

**Fit/Appropriateness Barriers:**

1.  There is often a disconnect between the available interventions and
    the mission of delivery organizations. The intended audience must
    want to use the intervention.

2.  Tailored interventions as developed are usually disease-specific and
    users may need broader interventions (i.e., an intervention for
    screening for breast, cervical, and colorectal cancer together).
    Institutions may wish to adapt an intervention to a number of
    diseases or behaviors for financial reasons, but this may change a
    valid, evidence-based intervention.

3.  An intervention may be inappropriate for the target population (for
    reasons of culture, literacy, ethnicity, etc.).

4.  The research audience on whom the intervention was developed may be
    narrow (because of strict recruitment guidelines), which makes it
    difficult to disseminate an intervention to the broader practice
    audience.

**Fit/Appropriateness Solutions:**

1.  Develop a mechanism that permits tailoring of successful
    interventions for a specific target audience.

2.  Develop a funding mechanism to support broader interventions (i.e.,
    combine three separate cancer screening programs into one program
    that screens for all three cancers) and educate reviewers on the
    need for broader-based interventions and dissemination studies.

**Dissemination Research and CTI Development Issues:**

1.  Tension exists between the research and practice environments.
    Concept testing is different than large scale dissemination, but
    performing the initial study on a large scale would be too costly.
    If an intervention is studied in a small group, it may need to be
    redesigned to work on a broader audience.

2.  Support for and understanding of dissemination research by study
    sections is lacking.

3.  The funding timeline means that CTIs developed with R21 funding may
    be obsolete by the time R01 funding is obtained to study broader
    applicability of the CTI.

**Dissemination Research and CTI Development Solutions:**

1.  Provide CTI researchers with the resources to make the CTI stand
    alone.

2.  Engage practitioners who will deliver the CTI in the development
    process.

3.  The Special Emphasis Panel for the Dissemination and Diffusion PAR
    should become an independent study section.

4.  Develop a consortium of tailoring intervention scientists to provide
    a forum for discussing evolving CTI standards.

***Appendix B***

**Breakout Group 2: Emerging CTIs**

*Facilitators: Audie Atienza and Cynthia Vinson*

***Morning Session: Barriers and Challenges***

**Audiences**

This breakout group examined currently funded, emerging research, the
results of which will be available in the near future, and determined
relevant audiences and the challenges of delivery. Emerging research
includes research currently being evaluated at NCI. Current research
includes technologies that have been developed and through the
peer-review and RTIPs process, whose outcomes that have been published,
whereas emerging research includes work that has been funded but whose
outcomes have not been published. Emerging research is not necessarily
the same as an emerging technology.

NCI has identified 38 programs it considers eligible for dissemination,
especially those with CTIs that will be available in the next few years.
NCI also is interested in tailoring, especially new types of tailoring
that can help exploit new technologies for use in development and
implementation of CTIs. Evidence of scientific evaluation of programs
also is necessary. Additionally, health disparity is a significant
concern; how to disseminate CTIs to reach disadvantaged populations
should be a priority.

The group identified a number of delivery audiences including: public
health practitioners, schools/universities, voluntary health
organizations, HMOs, Medicare/Medicaid, organizations that are
financially liable for their constituents (e.g., employers, Veterans
Health Administration, retirement centers), state and local public
health departments, disseminators to employers and other organizations,
clinicians, private industry/for-profit deliverers, information
intermediaries (e.g., WebMD.com), and hospitals and other health care
organizations.

**Barriers and Challenges**

Barriers and challenges faced by these audiences were discussed, and the
consensus was that most barriers will be at the end level:

1.  Schools do not have sophisticated or reliable IT capabilities,
    making it impossible to count on high-end transmission of multimedia
    materials.

2.  The ability to maintain the integrity of the CTI following
    modifications and customization must be explored and parameters to
    maintain integrity must be established.

3.  There are common barriers across various audiences (e.g., cost,
    technology shelf-life, research lag, consumer/organizational
    acceptance, lack of standards, credibility, and sustainability).

4.  Each audience level must be considered for successful CTI
    dissemination.

5.  Sustainability is a major problem (i.e., the lag between the
    research and the technologies).

6.  Organizations need a sufficient quantity of users to justify an
    investment in CTIs.

7.  Poorly informed decision-makers are another barrier; this is an area
    in which NCI could help because it is not necessarily in the best
    interest of the deliverers to deal with well-informed
    decision-makers.

8.  The No Child Left Behind Act may be a major barrier at the school
    level.

9.  There are competing priorities within an organization to adopt a
    CTI, and the socio-technological aspect of an organization (i.e.,
    how technology is integrated into an organization) must be
    considered.

10. Most CTIs address only one problem; they need to be much more
    comprehensive and integrated.

11. Content ownership, especially for for-profit companies, and control
    (i.e., the value in maintaining control vs. allowing modifications
    for dissemination) are challenges.

12. Some organizations allow only a short amount of time to determine if
    clientele will benefit from an intervention to view the effort as
    worthwhile. For example, the BlueCross^®^ BlueShield^®^ Association
    must see results within 6 months for it to deem any program
    worthwhile and continue it.

The group examined NCI's strengths, competitive advantages, and
weaknesses for providing solutions and determining appropriate partners:

1.  A solution to the barrier of low-tech IT in schools might be to
    design something that allows everyone to have the CTIs on their
    system.

2.  Within the public health system, many individuals who promote health
    and disease prevention chose this profession because they enjoy
    creating things; these individuals can be recruited to create
    customizations for CTIs.

3.  Tying CTIs into health education may be a solution to the barrier
    created by the No Child Left Behind Act.

4.  The values of the practitioner need to be considered; some value
    social good, whereas others value profit and return on investment.
    These values can be used and articulated to the appropriate audience
    to increase dissemination.

5.  NCI can explore ways to synthesize evidence-based research into a
    comprehensive program to minimize localized and fragmented CTIs.
    Integrated CTIs may be more desirable in the future, but funding
    will be an issue. Creating comprehensive CTIs is the job of the
    practitioners, not the researchers developing the interventions. The
    private sector must take the outcomes developed by NCI-funded
    researchers and modify them to suit their needs.

6.  NCI can help establish standards to enhance dissemination of CTIs
    and surmount some barriers and challenges.

7.  The SBIR Program could be used to develop and create tools for
    dissemination if the relevant businesses are interested in pursuing
    this type of project.

8.  The Office of Research Dissemination can provide expertise and
    funding to help develop solutions for dissemination and diffusion
    issues.

***Afternoon Session: Proposed Solutions and Considerations for
Designing Future CTI Interventions to Facilitate Dissemination and
Implementation***

The afternoon's discussion focused on leveraging NCI's strengths to
disseminate CTIs and foster their adoption in a wider audience. Barriers
and solutions also were discussed, including common barriers and those
unique to emerging research.

**Barriers:**

*Sustainability*. An unusable technology is a barrier for both the
deliverers and the adopters, but the burden lies with the deliverers.
Fidelity and sustainability are not the same. To address this barrier
interested groups could take a "recipe" for a CTI and adopt it to their
needs.

> *Out-of-date technology*. This is a subcomponent of sustainability.
> Obsolescence is an inherent characteristic of CTIs, and this barrier
> affects both deliverers and adopters.

*Cost reimbursement*. This is a major issue for NCI, and for-profit
organizations must be concerned with research and development costs.
Cost reimbursement is a subset of the value issue; the CTI must have
some value. NCI must consider the target markets.

*Decision-maker training*. NCI has a competitive edge in this area
because the training of decision-makers can benefit the social good,
which is an interest for NCI. Messages can be tailored to the target
communities using either a rational or emotional approach.

*Competing priorities*. Most deliverers do not have competing
priorities, but the adopter organizations are concerned with this
subject. Politics often play a role in this area. There is not always a
dichotomy between barriers that affect deliverers versus adopters and
this is one of those barriers, but it is less of an issue for deliverers
than adopters. If it is a problem for the adopters, it becomes a problem
for the deliverers; therefore, this barrier is relevant to both
deliverers and adopters but to varying degrees. The overall category
that captures this barrier is value proposition, which also includes
overall value and sufficient reach. Sufficient reach has multiple
relationships. Schools are often inundated with competing priorities,
whereas business views promoting health as decreasing future costs and
increasing efficiency.

During the discussion, it was decided to prioritize the barriers, taking
into account NCI's strength as an agency that gathers scientific
evidence, increases dissemination, supports a remarkable grantee
portfolio, and fosters beneficial partnerships. Each member of the group
voted for their top four priorities among the list of 10 categories of
barriers generated during brainstorming The top four priorities as voted
on by the breakout group were: (1) ownership/intellectual property, (2)
value proposition, (T3) sustainability, and (T3) customizability.

**Solutions for the top priority barriers and challenges:**

*Ownership/intellectual property*.

1.  One possible solution is an open source approach to the material
    that is contained in the CTI, making non-ownership valuable; NCI can
    promote a non-ownership approach. Industry is heading in this
    direction and still making profits. If it is known that this project
    will be open source from the beginning, then industry will get on
    board. Profitability also must be considered with issues of
    ownership. One question is where licensing plays a role; some
    licensing allows rewards.

2.  Some intellectual property could be made available through
    marketing. NCI could be responsible for the public health market,
    industry could be responsible for the employers market, and so
    forth. However, for-profit and not-for-profit organizations have
    different motives and some organizations may want the CTI to be
    completely customized, branded, and local even if they are not
    interested in making a profit.

3.  The ownership/intellectual property issue is a smaller question of
    the larger public/private relationship. Some models address
    intellectual property within their framework. NCI should provide
    examples of models that have resolved intellectual property issues
    for both public and private entities.

4.  NCI should consider providing seed money for commercialization,
    because the SBIR program can be cumbersome. This would permit
    organizations to use NCI seed money to commercialize CTIs without
    needing to find bank funding or private investors, who may or may
    not understand the value of CTIs.

5.  One possible solution to the ownership/intellectual property issue
    may be developing standards and agreements that include the level of
    changes that can be made to a CTI. This approach will allow others
    to customize within an agreed upon framework and increase options in
    terms of ownership.

*Value proposition*. This issue covers the motivation of organizations
and companies. Solutions should be geared toward solving conflicts in
value proposition or to foster increased acceptance of CTIs given the
motivations of companies and organizations.

1.  Explaining the benefits using terminology familiar to the audience,
    such as in the pharmaceutical example that was presented in the
    morning's plenary session, could increase interest in CTIs. Pairing
    the benefits of the CTI with other potential benefits important to
    the organization could help accomplish this.

2.  NCI could help fund cost-benefit analysis studies, with the caveat
    that not all concepts need to be proved. For example, it is not
    necessary to prove that smoking cessation improves health; however,
    the actual degree to which smoking cessation decreases health care
    costs is not known.

*Sustainability and customizability*. The last two priorities were
discussed together, although they are separate (but related) entities.
Adopters scrutinize their costs for sustainability and customizability.

1.  Branding is one method of customization.

2.  Guidelines and standards should be established that include
    allowances for customization.

3.  A study found that if physicians are reimbursed, they prefer to
    handle interventions themselves and not refer patients elsewhere.
    Google and WebMD could help ensure that people have access to
    accurate health information.

4.  The number one question that any organization considers when
    adopting a CTI is reach. Organizations often solve the problem of
    technical support themselves, but NCI is uniquely positioned to help
    determine strategies for increasing reach.

5.  Investigating how incentives are managed, implemented, and
    structured is valuable, including determining possible unforeseen
    consequences as a result of incentives. Incentives fall under reach
    strategies and having effective reach strategies are of interest to
    adopters, making this a particularly valuable approach.

6.  A method to educate the financial decision-makers and stress cost
    savings in the public sector is needed. Effectively communicating
    current or specific savings is a possible solution to this issue.

7.  Adopters can be given a toolkit to use for tailoring. If NCI does
    not support customization, other groups must. Expanding support
    services for technology to include support services for the whole
    CTI process including infrastructure may be a solution.

Possible partners to be involved in providing solutions for the four
priority barriers and challenges are:

1.  Representatives of organizations and agencies that might provide
    seed money.

2.  Representatives from adopter organizations (departments of health,
    schools, etc.).

3.  American Cancer Society (ACS) can be a partner in helping NCI to
    identify core competencies and determine which should be maintained.

4.  Organizational sociologists.

***Appendix C***

**Breakout Group 3: Future CTIs**

*Facilitators: Kara Hall and Brad Hesse*

**Audiences**

The group identified a number of audiences who each might have different
levels of health education, expertise in technology, and access to
delivery mechanisms. These audiences include: public health
practitioners, universities, voluntary health organizations, HMOs,
Medicare/Medicaid, employers, state organizations, disseminators to
employers and other organizations, clinicians, Centers for Disease
Control and Prevention (CDC), the private IT industry, the
consumer-based industry (e.g., Web MD), hospitals, and nursing/physician
professional organizations.

**Barriers and Challenges**

Barriers and challenges faced by these audiences---and particularly
issues that NCI would need to address for reaching the audiences---were
discussed. Issues included:

1.  There is no clear channel to the consumer, unlike the pharmaceutical
    industry.

2.  Some entities are underserved by the health industries.

3.  The identified audiences have varying levels of knowledge of health
    and technical capability, including adapting to changes. The
    delivery products (and the level of products) that are needed will
    vary with each organization.

4.  Effectiveness is another challenge.

5.  There is no existing broad infrastructure. CTI projects are defined
    in scope.

6.  Incentives are need to provide a reason to use technology. There is
    concern about productivity: companies want to save time, save money,
    and increase revenue.

Several suggestions were given on how the NCI might begin addressing
these issues:

1.  NCI might have a role in educating the marketplace. NCI also might
    play a role in surveillance and the meeting of needs; the quality of
    tools being built; and the receptiveness of the marketplace.

2.  NCI should consider existing models that dictate where universities
    must go to obtain assistance (e.g., the National Registry of
    Effective Registry Programs, under SAMHSA).

3.  NCI could consider targeting a few audiences that need full
    infrastructure to try to delineate and assist with specific issues.
    It was noted that some infrastructure issues are related to the
    capacity to use technology. Additionally, a multi-risk factor
    approach likely would be helpful.

The group discussed what the ideal technology or delivery mechanism
situation would be in 20 years. It must be fun, engaging, and easy to
integrate. It must be comfortable for doctors, nurses, and others to
use. Questions were raised about the deliverer, as the role of
scientists likely will shift in part toward that of a technician. The
amount that employers will pay in the future for health premiums will
have a huge impact on health services. NCI should take a long view
toward health issues, particularly prevention; those organizations with
a short view are a barrier to prevention. Groups that are underserved by
employers are a possible target for NCI.

**Dissemination**

Dissemination issues at the individual and population levels were
discussed. The market is consumer-driven, and older people have the
greatest incentives to use health services. Implications for
dissemination thus affect pharmaceutical companies who aim to create end
user demand. It was noted that SBIRs could provide secondary support.
Additionally, NCI could take lessons from those concerned with global
warming, who used high-impact communication vehicles to educate the
public and illustrate the benefits of stopping global warming to the
consumer.

The system is forcing patients to take more responsibility, particularly
for cost reasons. In 10 years, people may be given a monetary allotment
to pay for their health services, with smokers or other high-risk
people, for example, required to pay more. Mediated health care might
have a significant impact, particularly in their attention to or
ignoring of prevention. Direct consumers are looking for guidance.

NCI could consider developing a directory of guidelines, as well as
supporting human factor studies. It also could modify its future
solicitations to require that the applicant meet evaluation of adoption
criteria---that is, demonstrate that a target audience would want to use
the technology or service without significant advertisement or cost to
the project. In addition, NCI could provide seed money through the SBIR
to get the technology started and help create links with science to
refine the evolving product as it is being used. Additionally, it could
educate the market and review groups. A suggestion was that NCI should
consider return on investment, but without discouraging experimentation.
Another comment was that NCI could benefit significantly from awarding
small R01 and SBIR grants, rather than allocating large amounts of funds
to only a few grants.

**Other Barriers and Solutions**

Dr. Hall asked the group to share additional challenges (and possible
steps) that NCI could consider. These included:

1.  From the software user perspective, many artifacts are unavailable.
    They should be open source and follow standard operating procedures
    (SOPs).

2.  For the consumer, the industry should offer tools to lower-resource
    settings. This includes tailoring products and teaching relevant
    players (such as at the county level) how to use them.

3.  Ideographic research is needed to determine how individuals change.
    Tailored communications would help determine why people do or do not
    use health services. The quality of the product and marketing to
    individuals via new methods could be improved.

4.  Often, the intervention does not match the target audience.

5.  The disconnect between commercial and academic arenas can cause
    problems, as these groups approach needs quite differently (e.g.,
    case study vs. research study) and sometimes lack the willingness to
    communicate with each other. Additionally, the business industry
    lacks incentives to engage with academia.

6.  In the health care system, doctors, nurses, and pharmacists are not
    perceiving prevention as important and not taking it seriously. This
    is particularly true in their training.

7.  Behavioral science is not appreciated. For example, the Web site
    "Web MD" spends little time reviewing the science.

8.  There is a lack of software consistency and compatibility in tools
    and language.

9.  The future (i.e., 20 years from now) could hold many significant
    changes in shareware and Internet access. Compatibility and open
    source software and platforms are an increasing problem because of
    rapid changes in technology.

10. The issue of proprietary software (intellectual property rights and
    copyrights) will need to be addressed. The movement toward shareware
    should be encouraged.

11. Adaptation remains a concern.

12. Current evaluation tools are myopic. It is important to evaluate and
    select the appropriate tool. Evaluation should consider both the
    business and health perspectives. Business and health objectives and
    values should be aligned; currently there are different foci of
    motivations and outcomes.

13. Interventions should address the interest, needs, and concerns of
    consumers.

14. There is a lack of information management (which leads to consumers'
    lack of trust in information sources), compounded by an overload of
    health information and lack of basis to select among choices. This
    gives conflicting messages to consumers, even as the system shifts
    more responsibility to patients.

15. There also is a lack of transparency in the origin of technology.
    This is further exacerbated by the number of hybridized platforms.

16. The limited scope of electronic medical records and personal health
    records is a current problem.

***Appendix D***

**Dissemination of CTIs via RTIPs: Breakout Group A**

*Facilitators: Cynthia Vinson, Brad Hesse*

A number of CTIs on RTIPs are not supported by the developer. The
priorities of investigators tend to focus on development of CTIs rather
than dissemination. CTIs also are often developed as prototypes to test
concepts and ideas and may not be easily adapted for dissemination and
use in a broader population. The cost of adapting such CTIs may be high,
and use of a CTI in the real world may bring other complications such as
liability, a need for customer support, and quality control issues. Two
alternatives to RTIPs were proposed, a "business-to-business" model and
a "business-to-consumer" model. Challenges, solutions, and NCI's role in
potential development of each model were discussed, as well as issues
pertaining to the perceived lack of a robust CTI developer community and
ways to promote RTIPs as it currently exists.

[Business-to-Business Model]{.smallcaps}

This model is based on the current pharmaceutical development model. A
CTI developed by a research investigator could be acquired or licensed
by a company that would make the program available for public use. The
company would provide the necessary customer support and access
resources. A similar model has been used to distribute new statistical
analysis programs though SAS.

The large consulting industry that tailors health care delivery could
help implement CTIs. Groups such as ProChange, WebMD, and other health
media companies could serve as conduits for dissemination of CTIs and
marketing of RTIPs. Many such companies often are able to reach large,
diverse populations.

[Business-to-Consumer Model]{.smallcaps}

This model involves distribution of CTIs directly to the customer (such
as local health departments). Open source software would be essential
for this model, and the CTI would have to largely stand alone to avoid
requiring a significant investment in infrastructure and technical
support by the customer.

**Challenges**

-   The universities at which an intervention was developed often hold
    the licensing rights to the products or tools developed by their
    investigators. This could represent a particular challenge for the
    business-to-business model.

-   In general, there is less support for CTIs developed using
    shareware. CTIs built using proprietary code likely will have
    sustained support, particularly if licensed by a for-profit company.

-   Companies that may be interested in disseminating CTIs have large
    markets and can afford to pay for CTI adaptation and dissemination.
    However, entities such as schools, clinics, and state or local
    health agencies may not be able to pay. There have been some cases
    of large companies successfully marketing and making their products
    available for schools (Scholastic).

-   Many users of CTIs, whether for-profit or non-profit, prefer to
    adapt an intervention for their populations and settings. However,
    changes may impact the effectiveness of a CTI, even if it has been
    previously shown to be effective; maintaining the fidelity of a CTI
    or demonstrating its effectiveness after changes are made is a
    concern. A balance must be maintained between users' desires to
    "brand" an intervention and maintaining effectiveness.

-   Infrastructure issues pose a problem to the delivery of CTIs. As
    health care IT evolves, the infrastructure will change and tailored
    tools may become more widely available. Currently, the availability
    of health care IT varies greatly across the US.

-   CTIs require memory; without a database, data is not collected, the
    system has no memory, and there can be no building upon what was
    learned. Maintaining connections to the necessary databases or
    message libraries is a concern. If linked to a Web site, the site
    has to retain information so CTI responses can be tailored to the
    user's progress.

**Solutions and NCI's role**

-   A developer effort should be created to build open source software
    and/or adapt existing CTIs to common, consistent, and widely
    accessible platforms. Making CTIs web-based would increase
    accessibility.

-   NCI could serve a vetting function, identifying the best CTIs based
    on results and ease of use and offer these for adaptation to a
    common platform.

-   NCI could help demonstrate the value of CTIs to the end users and
    serve as a trusted source of information for consumers and
    businesses.

-   Because many users prefer broader-based CTIs, NCI could add more
    push toward multiple risk factor grants, but the ultimate decision
    rests with study sections who prefer to fund single focus CTIs.

-   NCI could help connect developers with the health informatics
    industry.

-   NCI could increase industry interest and capabilities in
    disseminating CTIs by offering a workshop to present information on
    what to tailor and how, including communications strategies as well
    as technology.

-   NCI could offer contracts to small IT developers to build CTIs into
    common consistent platforms (i.e., Microsoft Word or Web-based
    programs). NCI should not define specifics such as programming
    languages, but should encourage the developer to use platforms that
    will not quickly become obsolete.

-   NCI could help identify good datasets to use to build systems. There
    also should be a push to make the collected data publicly available
    and develop common data elements to help foster inter-connection.

-   NCI could develop guidelines to teach users to adapt CTIs
    appropriately with regard to content and define parameters to
    consider when making changes.

-   NCI could provide funding support to the original developer to adapt
    CTIs for broader use and facilitate creation of teams to build CTIs
    that can be disseminated using the raw materials of the original
    CTIs.

-   NCI could help develop partnerships with federal organizations such
    as the Substance Abuse and Mental Health Services Administration
    (SAMHSA) and CDC that may have more experience with disseminating
    prevention behaviors/tools.

-   NCI could promote partnering with advocacy organizations or
    philanthropies to fund CTI dissemination.

[The CTI Community]{.smallcaps}

The CTI community is small, which could create problems for the
business-to-business model of dissemination, because businesses usually
wish to draw on a large population of experts. Very few investigators
build truly effective CTIs and the knowledge base is concentrated in a
small number of research groups.

**Solutions**

-   Develop a program to help investigators build CTIs, including
    software tailoring programs or computer-assisted survey interface
    tools that would help put data into any format and ease adaptation
    and improvement of CTIs. An infrastructure of software tools that
    would create consistency among all the best CTIs would help
    dissemination. Interoperability of tools will be important for use
    in the clinical setting.

-   Encourage use of open source software to help build a new generation
    of CTI researchers and facilitate interaction with businesses. This
    would help create more public-private partnerships and help develop
    the critical mass needed to garner more support for a better
    infrastructure.

[Promoting RTIPs]{.smallcaps}

The RTIPs program is not well-publicized. It is not linked from the main
NCI page because the intended audience for RTIPs is public health
practitioners, rather than the general public. Over 1,000 researchers
have been trained by the Cancer Information Service (CIS) on how to use
RTIPs using a train-the-trainer model. To increase knowledge about this
program, Cancer Control P.L.A.N.E.T. has booths at the relevant meetings
such as the American Public Health Association annual meeting, staff
presents at other meetings, and has a listserv. ACS is also a regional
partner, and ACS and CDC investigators have been trained on the web
portal.

**Solutions**

-   NCI should help build and connect a community of users of RTIPs.
    RTIPs could provide the space for sharing between developers and
    adapters, implementers, and end users to help support CTI tools.
    This also could be a space for discussions between those who have
    used a CTI and those who wish to develop it for a broader or
    different use.

-   NCI should advertise RTIPs with groups such as the International
    Communication Association, Society of Behavioral Medicine, and
    American Psychological Association.

-   Efforts by Cancer Control P.L.A.N.E.T. to work with schools of
    public health to develop curricula around P.L.A.N.E.T. should be
    continued, with a focus on how to implement an evidence-based
    program.

-   NCI should develop a Program Announcement (PA) or Request for
    Proposals (RFP) that calls for programmers and/or independent
    contractors to build the most effective existing RTIPs (as
    determined by NCI) onto more easily disseminated platforms.
    Guidelines or requirements such as database structure, ability to
    tailor, and ability to be self-contained could be requirements of
    the RFP. The goal of this project would be to build the RTIPs on to
    platforms that are widely and easily used so that the end users do
    not have to provide a great deal of technical support.

-   Open software to other researchers who work on dissemination of
    CTIs. Everything on RTIPs should be open to any researcher who wants
    to work on adapting it for dissemination. Tools for this could be
    provided on RTIPs.

-   NCI could develop an RFP for vendors or small programmers to take
    the best RTIPs and turn them into publicly available software.

***Appendix E***

**Dissemination of CTIs via RTIPs: Breakout Group B**

*Facilitators: Audie Atienza and Irene Prabhu Das*

***Session #1: Challenges in Disseminating Tested CTIs Via RTIPs***

Existing tools are geared toward paper-based interventions, not
necessarily toward tailoring and technologies. Additionally, capacity
and infrastructure issues regarding insufficient methods to move
technologies forward exist.

**Challenges:**

-   Lack of perceived benefits.

-   Cost and time investments.

-   The ability of the infrastructure to disseminate components in a
    comprehensive and meaningful way.

-   Generalizability of interventions.

-   Lack of time or financial resources for the researcher/developer to
    assist potential adopters with their tailoring requests

-   The perception that researchers/developers are uncooperative and
    proprietary.

-   Lack of a common platform.

-   Ensuring the fidelity of CTIs when only the parts are provided.

-   Lack of knowledge concerning which parts of CTIs to choose and what
    items predict specific outcomes.

-   Intellectual property concerns.

-   The necessity of further developing the majority of interventions on
    RTIPs before they can be used.

-   The perception that researchers/developers who undertake NCI-funded
    intervention research must be prepared to become entrepreneurs as a
    result of lack of support from NCI.

-   Duplication of effort.

-   The disconnect related to incentives (i.e., incentives to
    researchers do not match with NCI's incentives for dissemination or
    incentives for academic promotion).

-   Confidentiality issues, including human subject issues.

-   Whether individuals downloading CTIs have the skills and resources
    necessary to properly use/adapt them.

-   Obsolescence of technologies.

**Solutions:**

-   The various CTI components could be made available so that users
    could choose the components appropriate for their needs and then put
    them together in a way that is beneficial to them.

-   Several high priority and/or best quality interventions could be
    selected to receive funding for adaptation to facilitate
    dissemination. The selection criteria could be decided via a grant
    competition or dissemination project.

-   Interventions that are already licensed can be described on RTIPs
    with external links. RTIPs can market CTIs in this manner without
    actually providing the CTI and violating intellectual
    property/licensing policies.

-   Instructions can be added to the message library. CTIs should be
    categorized and the instructions should direct those interested in
    adapting interventions to choose from each category; an explanation
    that individual items should not be used alone should be included.

-   If excessive effort is needed to renovate the existing CTIs, a
    consultant could be hired to examine the existing CTIs and create a
    new CTI using the best of what is available.

-   Projects can be placed on the RTIPs server with a link to the
    developer as the contact for support. Developers can keep their own
    methods but provide a common method by which others can use the CTI
    (e.g., integration of technology via application programming
    interfaces \[APIs\]).

-   Technology transfer experts could help researchers commercialize
    their products. These experts should not only have expertise in
    commercializing but also field expertise to ensure that they also
    have the knowledge to improve the content of the product.

-   RTIPs could provide a clearinghouse for various components (message
    libraries, algorithms, etc.) to allow interested parties to take the
    components of the clearinghouse to an expert and have the expert
    adapt them to fit their needs.

-   Educate users about the effort involved in adapting interventions.
    The labor and cost needed for adaptation should be revealed up
    front, and a category in RTIPs should be created that includes
    computer interventions that cannot be downloaded. These will be
    described, including how they can be adapted and who would be needed
    to assist with the adaptation.

-   A somewhat standardized, evidence-based, generalized DVD could be
    sent to public health departments to help prevent duplication of
    effort.

-   Links from the RTIPs Web Site to developers could help solve
    proprietary issues.

-   Directly involving consultants in the creation/development of items
    in RTIPs could help address deficits in skills and resources of the
    users.

-   A hybrid approach that uses algorithms could provide a targeted, if
    not tailored, program to individuals.

-   The RTIPs Web Site should explain that there are "middle men" who
    have the expertise and resources to tailor interventions offered on
    the Web site.

-   Information concerning the amount of times a program has been
    downloaded could be added to a researcher's curriculum vitae to
    address requirements for academic promotion.

-   A description of the Web site and its benefits would be helpful for
    those who do not understand the advantages of using RTIPs. The
    instructions and descriptions should be kept simple and should not
    refer to literature or books outside the Web site.

-   The RTIPs Web Site could be converted into a wiki so that everyone
    can contribute. Developers and users could use this mechanism to
    establish a knowledge base that would continue to develop and
    improve over time. Collaborators could be required to register
    before contributing so that NCI could screen individuals, thereby
    eliminating the need for moderation of each addition to the wiki and
    requiring minimal time and effort on the part of NCI staff. This is
    possible because the Web site is targeted to developers and not the
    general public.

-   Confidentiality and maintenance of privacy issues could be included
    in descriptions on the RTIPs Web Site.

The group then discussed the feasibility of providing a message library
and algorithms so that individuals and organizations can build the
interventions themselves.

-   Standards could be created to decrease the likelihood of improper
    use of CTIs provided in parts.

-   RTIPs must be populated with interventions that can be applied to a
    wide variety of people and places.

-   The NIH SBIR Web Site has a directory of companies that are
    interested in partnering; this could be used as a model for RTIPs.

-   University departments also could be investigated as partners.

-   University researchers could be matched with consultants to
    distribute CTIs.

-   Intermediaries to assist researchers who do not have the time or
    resources to help interested individuals understand, adapt, and use
    the interventions. A disclaimer should be added that there are no
    implied warranties to using the interventions; they are provided as
    is. If NCI is interested in disseminating these interventions, it
    can assist in adapting the interventions for dissemination.

-   Criteria could be developed to select CTIs most likely to benefit
    the widest audience. If all CTIs cannot be made available, this
    method could help identify the best.

-   The role of RTIPs must be defined. A possible role could be that of
    a support network for developers and interested third-party
    contractors, whereas other roles might focus on advocacy, promotion,
    and education.

-   Other models (e.g., CDC HIV Best-Evidence Interventions) should be
    considered, including mental health and injury prevention models.

***Session #2: Proposed Solutions and Recommendations to Disseminate
Tested CTIs via RTIPs or Other Dissemination Tools***

The group was asked to consider possible designs of alternative models
to disseminate CTIs, within or out of RTIPs, including what they would
look like, what safeguards would be included, and who the best
intermediaries would be. Within the current model, components of CTIs
are placed on the RTIPs Web Site. Possible alternative models include
NCI providing resources to develop CTIs or NCI acting as a broker to
link researchers to developers.

**Proposed alternative models:**

-   Capitalism is the ideal model for dissemination. If something can be
    made profitable, businesses will invest in it.

-   SBIR incentives could be employed. A Phase 3 could be added to SBIR
    that allows for the connection of researchers with commercialization
    support. SBIR could provide supplemental funds with a strict time
    limit.

-   Researchers willing to sign over their program to NCI could allow
    NCI to work with third parties to develop the programs for
    dissemination.

-   Researchers could partner with private companies from the beginning
    of development. The caveat to this may be that programs are rarely
    developed by one researcher but usually by a group that includes
    members from academia and the private sector; a private company may
    already be involved. Even if the members of academia are willing to
    sign their program over to NCI and/or a third party developer, the
    private sector members of the project team may not be willing or
    able to agree to this. Additionally, the team may not be working
    together by this point. In this type of case, an agreement may
    possibly be negotiated to obtain commitment from all involved
    parties.

-   Practitioners could be notified early that additional funding will
    be needed to afford the interventions.

**Challenges for adopting the above models:**

-   Self-employed people and self-insured companies have a financial
    incentive to adopt early cancer prevention strategies. Interventions
    will likely need to be bundled together (e.g., smoking, exercise,
    nutrition) for the programs to be attractive, depending on the
    audience, cost, and so forth. Many companies choose to have multiple
    topics included to provide a variety of materials that can be
    beneficial to a wider audience.

-   In addition to an interested company, an expert who understands the
    bridge between research and commercialization will be needed.
    Product knowledge could be transmitted to sales and marketing
    personnel to help them increase interest in the products.

-   Creating a demand when practitioners have small budgets may be
    addressed by the fact that many practitioners belong to
    organizations, such as hospitals and HMOs, that may be able to
    subsidize some of the cost.

-   Decision-makers, especially those that make budget decisions, need
    to be involved in creating demand for evidence-based interventions
    in the population of local and state health departments. Higher
    levels of organizations and funding agencies must be reached; they
    must begin to realize the importance of these interventions.

-   The types of CTIs that can be developed now are increasingly
    sophisticated and complex. A five-variable survey with three message
    elements for each variable could be employed and would by definition
    be a tailored intervention. For example, a community public health
    clinic could give the survey to individuals, and an algorithm card
    could tell the surveyor what to do and what materials to provide
    given the answers to the questions.

-   There are two different audiences who deliver CTIs: (1) for-profit
    and non-profit organizations, who must be convinced of the benefits
    and effectiveness of using interventions, and (2) practitioners, who
    have limited budgets in carrying out their goal of improving the
    health of the community.

-   Public health is divided into different segments, and each segment
    (e.g., government agencies, non-profit community groups, for-profit
    organizations, etc.) has different resources and different methods
    by which it obtains funding (e.g., grants).

-   Increased research on dissemination would be beneficial, but R21
    Dissemination and Implementation Research in Health Grants are of
    little value.

-   Partnerships involving researchers, organizations that will use the
    interventions, second-line developers for testing, brokers,
    disseminators, and investors would be beneficial to dissemination.

-   Determining whether interventions are effective for organizations is
    difficult because commercialization partners do not care about this
    type of outcome; the research data should indicate whether the
    intervention is beneficial.

-   County health departments are interested in interventions and
    understand their benefits; commercialization partners should market
    products to them.

-   It is necessary to carefully choose commercialization partners
    because some will be interested in outright ownership of the
    product. It is beneficial to choose a partner that shares a
    philosophy, not just the best disseminator.

-   An installer could be built for RTIPs, but this option is time and
    labor intensive and there currently are no incentives for this.

-   NCI should consider seeking outside funding to permit small,
    independent developers to hire staff or purchase needed tools to
    create CTIs.

-   Potential adopters could be directed to mechanisms to assist with
    development and use of CTIs on the RTIPs Web Site. NCI can provide
    the infrastructure, and the adapter can pay for the intermediate
    developer.

-   If a specific investor develops enough intermediate relationships,
    profits will increase, which will influence researchers and
    developers to plan for commercialization from the beginning.
    However, commercialization cannot occur until the research is proven
    to be effective.

Cost and marketing issues, given the resources that currently exist:

-   Capitalism---you can afford what you can---be allowed to play out.

-   Incentives for researchers, will promote planning for
    commercialization and inclusion of these plans in grant
    applications.

-   Researchers must be comfortable that their research will be used
    appropriately and that their intellectual property ownership will
    not be compromised.

-   A continuum of CTIs, from generalizable to specific, are needed,
    including those that will never be commercially viable.

***Appendix F***

**Dissemination of CTIs via RTIPs: Breakout Group C**

*Facilitators: Kara Hall and Sarah Kobrin*

**Challenges in Disseminating Tested CTIs Via RTIPs**

The group discussed a number of challenges to disseminate tested CTIs
via RTIPs, particularly tailoring interventions, security issues, and
RTIPs versus other alternatives.

**Tailoring Interventions**

-   Space for more CTIs is available on RTIPs, but encouraging users to
    access and select CTIs from RTIPs is problematic. A number of
    questions were raised about CTIs: When is a tailored intervention
    necessary? Should CTIs be displayed on RTIPs in some way other than
    by category, such as in relation to other CTIs?

-   Instruction manuals do not exist that explain how to install a CTI.
    A detailed list of specifications (e.g., the needed files, software,
    hardware, and infrastructure) would be helpful. In addition, some
    tailored communications include copyrighted elements, such as
    images. A contractor could be hired to assist with these issues, or
    people (users) could be surveyed to determine if they would use the
    information (marketing materials, etc.) as it is presented, or if
    they would extract parts of it to tailor the information to a
    specific context.

-   The NCI could develop a generic leaflet that includes
    recommendations on when to tailor or not tailor materials. It was
    noted, however, that people do not usually look to the government
    for suggestions on where to access health or CTI programs.

-   Disclaimers should be developed and included with guidelines for
    tailoring interventions. Caution should be taken to avoid
    detrimental effects (i.e., harm to a population through an
    intervention).

**Security Issues**

-   Security protocols need to be followed. Questions were raised about
    responsibility for Health Insurance Portability and Accountability
    Act (HIPAA) requirements and programmer developers' level of
    liability. Confidentiality must be maintained. To address the
    intellectual property issue, only content should be made available.

**RTIPs and Alternatives**

RTIPs were designed for public health officials and workers; however,
there is no defined audience and materials are submitted by a variety of
authors. The most current users likely are researchers. One participant
described a successful model for a neonatal health calendar, in which an
individual baby's photo is integrated, along with the dates for
vaccinations, immunizations, regular visits to the doctor, and other
relevant health date information; this could serve as a model for RTIPs.

The use of RTIPs as a fully packaged vehicle to archive and download
data was discussed. Two alternative ideas were mentioned:

-   Involve investigators who would take a tested intervention and work
    with practitioners (i.e., community network practice) to translate
    it in the RFA process.

-   Contract with a third party who wants to provide evidence-based
    information, with the NCI serving as a link. A (small) department
    still will be needed to provide support and answer questions. RTIPs
    could be adapted or restructured for the intermediary.

Participants agreed that any solution should connect a more friendly
system to the user, and that incentives will be needed for both the
researcher and the initial users/adopters. The NCI could consider a
return on investment---perhaps in the form of a bonus if the
intervention is deployed in the community. However, the NCI should not
be liable for drawbacks. Additionally, it is unlikely that investigators
would update recommendations, and plans should be made to address this.
To help track usage, those who download the CTIs could be required to
register; it was noted that such tracking would stretch the limits of
RTIPs.

**Additional Comments**

-   The many iterations of programs, a number of which are either
    paper-based or 10 years or older, can pose a problem.

-   Not all tested programs should be disseminated to all areas. For
    example, programs that work in inner cities often are unsuccessful
    in rural areas, for ethnic and cultural reasons.

-   It might be ideal to remove separate hosting and put everything into
    a central area. There is, however, a distinction between account
    management and cancer data hosting.

-   Tailoring is one-third of the problem; infrastructure accounts for
    two-thirds of the problem.

```{=html}
<!-- -->
```
-   Potential deliverers should collaborate to work through the process
    of implementation.

-   A central organization or intermediary is needed for central
    hosting. However, local hosting offers better protection of patient
    data.

**Proposed Solutions and Recommendations to Disseminate Tested CTIs Via
RTIPs or Other Dissemination Tools**

*Facilitators: Kara Hall and Sarah Kobrin*

Concerns related to dissemination and use of CTIs via RTIPs or other
tools included the different constraints, policies, and needs of the
broad group of public health practitioners---for example, state versus
local officials. In addition, the descriptions of CTI programs need to
be made more accessible to the non-research community; some of the
current text appears to be describing a study, not a program.

The group offered the following comments:

-   Incentives need to be linked to existing networks or partnerships.

-   A firewall should be in place to create an interface so that others
    can use the system independent of the developer and it should
    include an incentive to customize to new groups.

-   There is a role for the NCI to convene industry and solicit ideas
    and possible partnerships. The NCI could then identify the audiences
    not reached and better target them.

-   A centralized service model should be adopted.

-   Regarding the third party contractor alternative, several models
    could be proposed and tested to provide a "menu of options."

-   The NCI could focus on a specific area and identify researchers to
    contact. This could be carried out via Wiki or bulletin boards, and
    further expanded to develop a social networking forum, allowing
    potential users to contact existing users for comments and reactions
    regarding the various models.

**Options for Disseminating CTIs**

The group recommended the following solutions and alternatives to RTIPs:

-   Target to particular organizations (i.e., market to that group)

-   Create simple software to generate tailored interventions

-   Hire a third party contractor to host the CTI, through one of the
    following models:

1)  support model

2)  centralized service model

3)  University license model

-   Create a process for the practitioner to request an intervention and
    include a process for assessing the "credibility" of a request

-   Use a vehicle (such as a Wiki or bulletin board) for a network of
    practitioners to provide feedback on their experience with the
    intervention or express interest in particular interventions

-   Offer incentives to the researcher to generate a program that is
    transferable; test it through the RFA mechanism --- that is, work
    with practitioners to translate tested interventions (e.g.,
    community network programs)

-   Generate a new process for future usability of programs

***Action Items***

***Designing for Dissemination***

Participants indicated their interest for the respective workgroups. NCI
will follow-up with participants to begin working on selected priority
areas from the seven workgroups:

1.  Develop Guidance on IT Standards/Best Practices for CTI Development
    (e.g., software with broad platform utility, open-source CTI
    software)

2.  Develop Criteria on Dissemination Potential for NIH CTI Grant Review
    Process

3.  New NIH Funding Initiatives Targeting CTI Development to Fit
    Existing Dissemination Systems with Broad Reach

4.  Convening Meetings of Thought Leaders, Policy Makers, and Decision
    Makers to Explore the Future Business Models in Health IT

5.  Develop a Research Agenda for Disseminating CTI

6.  Develop Special Initiatives to Support CTI Dissemination to Limited
    Resource Delivery Systems for the Underserved

7.  Convene IT Workshops for Software Experts to Develop CTI Authoring
    Tools

***Dissemination via RTIPs***

The proposed approaches within the three broad categories will be
reviewed and prioritized by NCI's Research Dissemination and Diffusion
Team taking into consideration available funding and existing resources
and programs.

1.  Funding Mechanisms to Stimulate Adaptation of CTIs for Dissemination

2.  Intermediaries

3.  Tools/Resources

> ***Participants List***

**David Ahern, Ph.D.**

National Program Director

Health e-Technologies Initiative

Brigham and Women's Hospital

1249 Boylston Street, Third Floor

Boston, MA 02215

Telephone: (617) 525-6166

Fax: (617) 525-6169

E-mail: dahern@partners.org

**Audie Atienza, Ph.D.**

Behavioral Scientist

Health Promotion Research Branch

Behavioral Research Program

Division of Cancer Control and Population Sciences National Cancer
Institute

National Institutes of Health

Executive Plaza North, Room 4070

6130 Executive Boulevard

Bethesda, MD 20892

Telephone: (301) 451-9530

E-mail: atienzaa@mail.nih.gov

**Tim Bickmore, Ph.D.**

Assistant Professor

College of Computer and Information Science

Northeastern University

WVH202

360 Huntington Avenue

Boston, MA 02115

Telephone: (617) 373-5477

Fax: (617) 812-2589

E-mail: bickmore@ccs.neu.edu

**Lee Campbell**

Software Developer

University of North Carolina at Chapel Hill

Chapel Hill, NC

Telephone: (919) 475-5193

E-mail: ljc@graffiti.net

**Victoria Champion, D.N.S., R.N.**

Associate Dean for Research

School of Nursing

Indiana University

1111 Middle Drive

Indianapolis, IN 46202

Telephone: (317) 274-4187

Fax: (317) 278-2021

E-mail: vchampio@iupui.edu

**Arie Dijkstra, Ph.D.**

Associate Professor

Department of Social and Organizational Psychology

University of Groningen

Grote Kruisstraat 2/1

9712 TS Groningen

The Netherlands

Telephone: 31-50-3638729

E-mail: arie.dijkstra@rug.nl

**Karen Emmons, Ph.D.**

Professor and Deputy Director

Center for Community-Based Research

Dana-Farber Cancer Institute and Harvard School of Public Health

44 Binney Street

Boston, MA 02115

Telephone: (617) 632-2188

E-mail: karen_m_emmons@dfci.harvard.edu

**Tom Eng, V.M.D., M.P.H.**

President and Founder

Healia/Meredith Corporation

40 Lake Bellevue Drive, Suite 350

Bellevue, WA 98005

Telephone: (425) 646-6030

Fax: (425) 650-9888

E-mail: tom@evalumetrix.com

**Kerry Evers, Ph.D.**

Director

Pro-Change Behavior Systems, Inc.

P.O. Box 775

West Kingston, RI 02892

Telephone: (401) 874-4118

Fax: (401) 874-4103

E-mail: kevers@prochange.com

**David Farrell, M.P.H.**

President

People Designs

1304 Broad Street

Durham, NC 27705

Telephone: (919) 416-9341

Fax: (919) 416-9081

E-mail: dfarrell@peopledesigns.com

**Amanda Graham, Ph.D.**

Assistant Professor

Lombardi Comprehensive Cancer Center

Georgetown Medical Center

3300 Whitehaven Street, N.W., Suite 4100

Washington, DC 20007

Telephone: (202) 687-7905

Fax: (202) 687-0305

E-mail: alg45@georgetown.edu

**Kara Hall, Ph.D.**

Health Scientist

Office of the Director, Research Dissemination and Diffusion

Division of Cancer Control and Population Sciences

National Cancer Institute

National Institutes of Health

Executive Plaza North, Room 5128

6130 Executive Boulevard

Bethesda, MD 20892

Telephone: (301) 594-1237

Fax: (301) 594-6787

E-mail: hallka@mail.nih.gov

**Bradford Hesse, Ph.D.**

Branch Chief

Health Communication and Informatics Research Branch

Behavioral Research Program

Division of Cancer Control and Population Sciences

National Cancer Institute

National Institutes of Health

Executive Plaza North, Room 4068

6130 Executive Boulevard

Bethesda, MD 20892

Telephone: (301) 594-9904

E-mail: hesseb@mail.nih.gov

**Stephen Intille, Ph.D.**

Technology Director, House_n

MIT Department of Architecture

Massachusetts Institute of Technology

1 Cambridge Center, Fourth Floor

Cambridge, MA 02142

Telephone: (617) 452-2346

Fax: (617) 225-0027

E-mail: intille@mit.edu

**Jon Kerner, Ph.D.**

Deputy Director for Research Dissemination and Diffusion

Office of the Director, Research Dissemination and Diffusion

Division of Cancer Control and Population Sciences

National Cancer Institute

National Institutes of Health

Executive Plaza North, Room 6142

6130 Executive Boulevard

Bethesda, MD 20892

Telephone: (301) 594-7294

Fax: (301) 594-6787

E-mail: kernerj@mail.nih.gov

**Sarah Kobrin, Ph.D., M.P.H.**

Program Director

Applied Cancer Screening Research Branch

Behavioral Research Program

Division of Cancer Control and Population Sciences

National Cancer Institute

National Institutes of Health

Executive Plaza North, Room 4096

6130 Executive Boulevard

Bethesda, MD 20892

Telephone: (301) 435-8662

E-mail: kobrins@mail.nih.gov

**Matthew Kreuter, Ph.D., M.P.H.**

Professor and Director

Health Communications Research Laboratory

Saint Louis University School of Public Health

3545 Lafayette Avenue, Room 429

St. Louis, MO 63104

Telephone: (314) 977-8132

Fax: (314) 977-4016

E-mail: kreuter@slu.edu

**Mario Luca, M.S.**

Applications Programmer

The University of Texas M.D. Anderson Cancer Center

P.O. Box 301439, Unit 1330

Houston, TX 77230

Telephone: (713) 745-4473

Fax: (713) 745-4286

E-mail: mluca@mdanderson.org

**Jeffrey McLaughlin**

Principal and Founder

Radiant Creative Group

1315 Sterrett Street

Houston, TX 77002

Telephone: (832) 618-1070

Fax: (713) 223-4216

E-mail: jeff@radiant-creative.com

**Seth Noar, Ph.D.**

Assistant Professor

Department of Communication

University of Kentucky

248 Grehan Building

Lexington, KY 40506

Telephone: (859) 257-7809

Fax: (859) 257-4103

E-mail: noar@uky.edu

**Patricia Nolan, M.D., M.P.H.**

Clinical Associate Professor of Community Health

Department of Community Health

Warren Alpert Medical School

Brown University

Box GS121

121 S. Main Street, Room 204

Providence, RI 02912

Telephone: (401) 863-6416

Fax: (401) 863-3533

E-mail: patricia_nolan@brown.edu

**Mike Nowak, B.S., M.I.L.S.**

Technology Group Lead

Center for Health Communications Research

University of Michigan

300 North Ingalls, 5D04

Ann Arbor, MI 48103

Telephone: (734) 763-9944

Fax: (734) 647-7343

E-mail: mnowak@umich.edu

**Irene Prabhu Das, Ph.D.**

Partnership Coordinator

Office of the Director, Research Dissemination and Diffusion

Division of Cancer Control and Population Sciences

National Cancer Institute

National Institutes of Health

Executive Plaza North, Room 5124

6130 Executive Boulevard

Bethesda, MD 20892

Telephone: (301) 451-5803

Fax: (301) 594-6787

E-mail: prabhudasi@mail.nih.gov

**James Prochaska, Ph.D.**

Professor and Director

Cancer Prevention Research Center

University of Rhode Island

2 Chaffee Road

Kingston, RI 02881

Telephone: (401) 874-2830

Fax: (401) 874-5562

E-mail: jop@uri.edu

**Alexander Prokhorov, M.D., Ph.D.**

Professor and Director

Tobacco Outreach Education Program

Department of Behavioral Science

The University of Texas M.D. Anderson Cancer Center

P.O. Box 301439, Unit 1330

Houston, TX 77230

Telephone: (713) 745-2382

Fax: (713) 745-4286

E-mail: aprokhor@mdanderson.org

**William Rakowski, Ph.D.**

Professor of Medical Science

Department of Community Health

Brown University

121 S. Main Street, Second Floor

Providence, RI 02912

Telephone: (401) 863-3263

Fax: (401) 863-3713

E-mail: william_rakowski@brown.edu

**Ed Saunders, M.S.**

Deputy Director

Center for Health Communications Research

University of Michigan

300 North Ingalls, 5D04

Ann Arbor, MI 48109

Telephone: (734) 763-4666

Fax: (734) 647-7343

E-mail: saunders@umich.edu

**Celette Skinner, Ph.D.**

Professor

Division of Behavioral and Communication Sciences

Department of Clinical Sciences

University of Texas Southwestern Medical Center

5323 Harry Hines Boulevard, E-506

Dallas, TX 75390

Telephone: (214) 648-5499

Fax: (214) 648-3934

E-mail: celette.skinner@utsouthwestern.edu

**Vic Strecher, Ph.D., M.P.H.**

Professor and Director

Center for Health Communications Research

Department of Health Behavior and Health Education

University of Michigan School of Public Health

300 N. Ingalls Road, 5D-04 (0471)

Ann Arbor, MI 48109

Telephone: (734) 763-6099

Fax: (734) 647-7343

E-mail: strecher@umich.edu

**Wayne Velicer, Ph.D.**

Professor of Psychology and Co-Director

Cancer Prevention Research Center

University of Rhode Island

2 Chaffee Road

Kingston, RI 02881

Telephone: (401) 874-4254

Fax: (401) 874-5562

E-mail: velicer@uri.edu

**Cynthia Vinson, M.P.A.**

Dissemination and Diffusion Coordinator

Office of the Director, Research Dissemination and Diffusion

Division of Cancer Control and Population Sciences

National Cancer Institute

National Institutes of Health

Executive Plaza North, Room 6140

6130 Executive Boulevard

Bethesda, MD 20892

Telephone: (301) 594-5906

Fax: (301) 594-6787

E-mail: cvinson@mail.nih.gov

**Lindsey Volckmann, B.A.**

Cancer Research Training Award Fellow

Health Communication and Informatics Research Branch

Behavioral Research Program

Division of Cancer Control and Population Sciences

National Cancer Institute

National Institutes of Health

Executive Plaza North, Room 4074C

6130 Executive Boulevard

Bethesda, MD 20892

Telephone: (301) 435-8663

E-mail: volckmal@mail.nih.gov