November 2004

Evaluation Plan for the\
National Cancer Institute's Community Networks Program:\
Phases I and II

Final Report

Prepared for

Ken Chu, Ph.D.

Center to Reduce Cancer Health Disparities

National Cancer Institute

6116 Executive Boulevard

Suite 602 MSC 8341

Rockville, MD 20852

Prepared by

Debra J. Holden, Ph.D.

Brenda Stone-Wiggins, M.P.H.

LaShawn Wordlaw-Stinson, Ph.D.

RTI International

3040 Cornwallis Road

Research Triangle Park, NC 27709

RTI Project Number 09293.000

RTI Project Number

09293.000

Evaluation Plan for the\
National Cancer Institute's Community Networks Program:

Phases I and II

Final Report

November 2004

Prepared for

**Ken Chu, Ph.D.**

Center to Reduce Cancer Health Disparities

National Cancer Institute

6116 Executive Boulevard

Suite 602 MSC 8341

Rockville, MD 20852

Prepared by

Debra J. Holden, Ph.D.

Brenda Stone-Wiggins, M.P.H.

LaShawn Wordlaw-Stinson, Ph.D.

RTI International

3040 Cornwallis Road

Research Triangle Park, NC 27709

\*RTI International is a trade name of Research Triangle Institute.

Contents

Executive Summary ES-1

1\. Introduction [2](#introduction)

2\. Evaluation Plan Development [2](#evaluation-plan-development)

2.1 Overview of Evaluation Planning
[2](#overview-of-evaluation-planning)

2.2 Steps Completed for Evaluation Planning
[2](#steps-completed-for-evaluation-planning)

2.3 Engage Stakeholders in the CNP Evaluation (Step 1)
[2](#engage-stakeholders-in-the-cnp-evaluation-step-1)

2.4 Describe the CNP (Step 2) [2](#describe-the-cnp-step-2)

2.4.1 Program Building Blocks [2](#program-building-blocks)

2.4.2 Focus of Activities [2](#focus-of-activities)

2.4.3 Outcomes of Interest to the CNP
[2](#outcomes-of-interest-to-the-cnp)

2.5 Focus the CNP Evaluation (Step 3)
[2](#focus-the-cnp-evaluation-step-3)

2.6 Next Steps [2](#next-steps)

References [2](#references)

Appendix A: Planning Matrix A-1

Exhibits

> Exhibit 1. CNP Phases and Accompanying Goals
> [2](#__RefHeading___Toc88540566)
>
> Exhibit 2. CDC's Framework for Program EvaluationAdapted Steps for
> CNP Evaluation Plan Development [2](#__RefHeading___Toc88540567)
>
> Exhibit 3. Proposed Interconnectedness of the Local Community Advisory
> Groups and the National Evaluation Advisory Committee
> [2](#__RefHeading___Toc88540568)
>
> Exhibit 4. Comparison of Conventional and Participatory Evaluation
> Approaches [2](#__RefHeading___Toc88540569)
>
> Exhibit 5. CNP Draft Conceptual Framework
> [2](#__RefHeading___Toc88540570)
>
> Exhibit 6. Nine Principles of Community-Based Participatory Research
> [2](#__RefHeading___Toc88540571)
>
> Exhibit 7. Possible Data Collection Methodologies for the CNP
> [2](#__RefHeading___Toc88540572)

## Executive Summary

Cancer mortality has been declining for most populations in the United
States, but this benefit is less obvious for some racial and ethnic
groups, as well as others defined by geographic region or income. As an
example, African-Americans are more likely to develop and die from
cancer than persons of any other racial and ethnic group (ACS, 2003).
Through a closer examination of the literature and of statistical
trends, it is clear that the benefits of current knowledge about
state-of-the-art cancer care are not shared equally by all members of
our society (Aziz and Rowland, 2002). When survival rates are broken
down by race/ethnicity, significant differences exist across ethnic
minority and medically underserved populations with respect to the risk
of developing and dying from cancer. Many reasons for these disparities
are known, while some are still under study, such as the biological
bases that could be contributing to the disparities. These causes
include limited access to health care (e.g., increasing numbers of
uninsured Americans, decreasing resources for Medicaid coverage,
geographic location of health facilities), differences in medical
practices provided to patients, lack of knowledge among people about the
types of care that are of greatest benefit, and others. Many of these
causes are grounded within the local community where these groups live
and can best be addressed in that setting.

Through the development of the Community Networks Program (CNP), the
National Cancer Institute (NCI) has emphasized the need to
systematically address these health disparities with regard to cancer
care. NCI plans to fund up to 22 initiatives to conduct community-based
participatory research, training, and education programs that are
focused on reducing local cancer disparities. Groups of individuals who
may benefit from these efforts include those of different racial and/or
ethnic backgrounds (e.g., African-Americans, Asians, Hispanics) and/or
those belonging to underserved populations (e.g., Appalachia, rural).
The primary goal of this program is to "significantly improve access to
and utilization of beneficial cancer interventions in communities with
cancer disparities."

RTI International (RTI) worked with staff from the NCI's Center to
Reduce Cancer Health Disparities (CRCHD) to conduct a feasibility study
for evaluating the CNP. The purpose of this study was to better
understand the goals and objectives of the CNP and to explore mechanisms
for conducting a comprehensive evaluation. The CNP is comprised of three
phases: one to develop and increase local capacity, a second to create
community-based participatory research and training programs, and a
third to establish credibility and sustainability of the CNP. Through
this effort, RTI has designed a program evaluation plan for Phases I and
II of the CNP. In conducting the feasibility study, RTI sought answers
to the following questions:

-   What is the purpose and scope of the evaluation?

-   What evaluation questions are important to NCI?

-   What practical issues need to be addressed in planning for program
    evaluation (e.g., political, cultural, financial, and methodological
    constraints)?

-   What will be the interface between the grantees and the evaluators?

-   What is the best evaluation approach, both philosophically and
    practically? For example, it is not practical to expect to see
    reductions in cancer morbidity and mortality within 5 years.
    Therefore, what is the ultimate hope for achievement through this
    effort?

-   What are the ultimate outcomes for the CNP effort?

This report summarizes RTI's approach to developing the Phase I and II
evaluation plans, as well as the theoretical foundations that were used
to organize and support the evaluation approach. The report also
provides tools RTI created, such as a CNP conceptual framework, that NCI
can use to implement a comprehensive national evaluation plan of Phases
I and II of the CNP. ***Exhibit ES-1*** presents a compliance matrix
summarizing how this report addresses each of the technical requirements
of this project.

Exhibit ES-1. Compliance Matrix of Final Report to Technical
Requirements of the Evaluation Plan Project

+---------------------+---------------------------------+--------------+
| **Detailed          | **Description**                 | **Report     |
| Technical           |                                 | Section      |
| Requirement**       |                                 | Where        |
|                     |                                 | Addressed**  |
+---------------------+---------------------------------+--------------+
| A. Identify data    | Discuss the types of data       | 2.5,         |
| elements and        | elements that could be assessed |              |
| information         | through the evaluation of the   | Appendix A   |
| necessary to assess | CNP.                            | (Possible    |
| each of the CNP     |                                 | Data         |
| objectives.         |                                 | Sources)     |
+---------------------+---------------------------------+--------------+
| B. Give the design  | Describe how the data could be  | 2.5,         |
| of the data         | collected for the CNP           |              |
| collection system   | evaluation of Phases I and II.  | 2.6          |
| that can be used to |                                 |              |
| collect these data. |                                 |              |
+---------------------+---------------------------------+--------------+
| C. Indicate the     | Describe types of measures that | 2.2,         |
| types of            | could be used for evaluating    |              |
| qualitative and     | the CNP.                        | 2.4.3,       |
| quantitative        |                                 |              |
| measures that will  |                                 | 2.5 (Exhibit |
| be used on the data |                                 | 7), Appendix |
| and information for |                                 | A (Possible  |
| each CNP objective. |                                 | Data         |
|                     |                                 | Sources)     |
+---------------------+---------------------------------+--------------+
| D. Describe the     | Overview of how Phases I and II | 2.3,         |
| scope of work for   | could be evaluated, including a |              |
| the evaluation.     | description of the role of the  | 2.4.1,       |
|                     | national evaluator.             |              |
|                     |                                 | 2.6          |
+---------------------+---------------------------------+--------------+

## 1. Introduction

Cancer-related health disparities reflect the disconnect between
research discovery, system development, and service delivery (NIH/NCI,
2001). Identifying and eliminating the barriers to cancer prevention and
control, treatment, and communication is a necessary step to achieving
the Healthy People 2010 and the National Cancer Institute (NCI)
Director's 2015 goals of reducing these disparities (USDHHS, 2003,
2004). An interdisciplinary community science model that integrates
prevention science and community-centered practice is one strategy for
bridging that gap (Wandersman, 2003). In April 2005, the NCI will invest
\$24 million per year for 5 years to fund up to 22 community cancer
networks that will collaborate with local cancer control partners and
NCI's Centers, Divisions, and Offices to improve access to and
utilization of preventive cancer care. A key component of these local
programs is to engage their community in every aspect of development and
implementation, from identifying cancer care needs and priorities, to
addressing gaps in cancer care prevention in their local environment, to
evaluating their efforts for program longevity. Because disparities
exist particularly among the four leading causes of cancer-related
deathsbreast, cervical, prostate, and colorectaleach Community
Networks Program (CNP) will address some aspect of care (e.g.,
encouraging health preventive behaviors, such as smoking cessation;
increasing screening rates for early detection; improving access to
clinical trials) for at least one of these cancers. CNPs can also elect
to address other needs that are identified by their community partners
as important with regard to cancer care. As shown in ***Exhibit 1***,
the CNP will consist of three phases. Each phase is dedicated to a goal
for the overall development and implementation of the program.

[]{#__RefHeading___Toc88540566 .anchor}Exhibit 1. CNP Phases and
Accompanying Goals

  ---------- ------------------------------------------------------------
  Phase      Goal

  I          Develop and increase capacity building to support community
             education, research, and training to reduce cancer
             disparities.

  II         Increase community-based research and training programs to
             reduce cancer disparities.

  III        Establish credibility and sustainability of the Community
             Networks Program.
  ---------- ------------------------------------------------------------

The CNP is a complex, multifaceted cancer control program that aims to
create change through community-based participatory research. Levels of
change that will be incorporated into a comprehensive evaluation of the
CNP have been organized at three socio-ecological levels: individual,
community (including organizational and system-level changes within a
community), and policy. The diversity of the economic, cultural, social,
and political context within which these networks will function requires
flexibility to design and implement initiatives that are relevant and
capture the distinctive characteristics present in the disparate
communities (e.g., the type of cancer on which each focuses).
Consequently, each CNP will differ in focus and evaluation capacity.
These factors present unique opportunities and challenges that
necessitate technical assistance for improving the programs and
evaluating the effectiveness of the CNP. It is critical that all CNPs
successfully implement culturally-relevant, community-based, and
evidence-based prevention, early detection, treatment, and communication
interventions if they are to effectively address cancer care for
disparate populations. Therefore, it is important to provide for all
stakeholders a "road map" that defines outcomes at several points and
provides linkages between the socio-ecological levels and the expected
levels of outcomes.

The NCI Center to Reduce Cancer Health Disparities (CRCHD) worked with
RTI International (RTI) to develop the "best overall approach and most
appropriate measures" for conducting a comprehensive CNP evaluation.
RTI's initial step in developing a comprehensive national evaluation
plan is to understand NCI's philosophy and expectations for the
evaluation. In conducting this feasibility study, RTI sought answers to
the following questions:

-   What is the purpose and scope of the evaluation?

-   What evaluation questions are important to NCI?

-   What practical issues need to be addressed in planning for program
    evaluation (e.g., political, cultural, financial, and methodological
    constraints)?

-   What will be the interface between the grantees and the evaluators?

-   What is the best evaluation approach, both philosophically and
    practically? For example, it is not practical to expect to see
    reductions in cancer morbidity and mortality within 5 years.
    Therefore, what is the ultimate hope for achievement through this
    effort?

-   What are the ultimate outcomes for the CNP effort?

In collaboration with NCI, RTI identified several principles to guide
the CNP evaluation planning process:

 It must be a community-focused, participatory, and collaborative
process. In other words, the CNP must be active participants in all
aspects of planning and implementing the evaluation.

 It must provide feedback to the grantees throughout the program to
strengthen and improve their evaluation capacity. For example, each
project will be measured against its progress from baseline and not
against the progress of other sites.

 It must demonstrate to NCI if the CNP worked as planned. In other
words, were CNPs successful in accomplishing what they set out to do?

This report summarizes RTI's approach to developing the evaluation plan,
as well as the theoretical foundations that were used to organize and
support the evaluation approach. The report also presents tools that NCI
can use to implement a comprehensive national evaluation of Phases I and
II of the CNP.

## 2. Evaluation Plan Development

### 2.1 Overview of Evaluation Planning

Evaluation planning and implementation is an important process to
utilize in program development. Its ultimate goal is to determine what
components of a program are effective and impact health outcomes. As
specified in the concept statement for the CNP, NCI takes a
comprehensive approach to decreasing health disparities related to
cancer by addressing three main goals (see Exhibit 1). As summarized
above, these goals encompass such community-based participatory research
principles as capacity building, partnerships and collaboration to
engage community members and researchers in a joint process with
equitable contributions, system development and improvement, and a
balance between research and action (Israel et al., 1998). Each of the
funded programs will work to address these goals through objectives
specified by the NCI. The complexity of the CNP raises a number of
important questions that must be addressed by a comprehensive national
evaluation:

 To what extent are the three goals of the overall program being met?

 Does the CNP impact disparities in breast, cervical, prostate, and
colorectal cancers?

 Is each CNP able to improve access to primary and secondary prevention
services among the targeted communities?

 What are the characteristics of the programs that effectively impact
individual, community, and/or policy changes?

 What level of capacity is required at the community level in order to
effectively address education, research, and training needs related to
impacting cancer disparities?

Answering these questions is critical to implementing a comprehensive
evaluation of the CNP. In this section, we provide an overview of how we
have begun to organize and create a comprehensive evaluation plan for
CNP. This process of plan development incorporates the evaluation's
objectives within a conceptual framework that depicts the activities and
direct impacts and outcomes of each program component, as viewed by key
stakeholders. The plan developed to date is comprehensive, yet relies on
existing data sources where possible and provides an approach that is
feasible. In this report, we present the evaluation plan developed for
Phases I and II.  A similar process will need to be completed for Phase
III.

### 2.2 Steps Completed for Evaluation Planning

***Exhibit 2*** is adapted from CDC's "Framework for Program Evaluation
in Public Health" (1999). Based on this framework, we have developed a
series of steps, accompanied by tools, that have guided the development
and implementation of a comprehensive national evaluation plan for the
first two phases of the CNP. In the remainder of this section, we
describe the steps completed to date and provide an overview of how this
step-by-step approach can result in a comprehensive plan that meets the
needs of CNP key stakeholders.

[]{#__RefHeading___Toc88540567 .anchor}Exhibit 2. CDC's Framework for
Program EvaluationAdapted Steps for CNP Evaluation Plan Development

![](media/image1.wmf)

Source: Centers for Disease Control and Prevention (CDC). 1999.
"Framework for Program Evaluation in Public Health." *Morbidity and
Mortality Weekly Report* 48(RR11):1-40.

The framework in Exhibit 2 highlights six steps in the process of
developing an evaluation plan. For this study, steps 1 and 2 were
completed for Phases I and II of the CNP. Step 3 was initiated for these
two phases but will need to be addressed further, as will steps 4
through 6, once the CNPs are awarded funding and evaluation planning is
completed. The next section provides an overview of the process followed
to complete steps 1 through 3.

***Step 1---Engage Stakeholders***

The RTI team initiated the evaluation planning process with a
face-to-face meeting among key stakeholders of the CNP staff and RTI
team leaders to review the goals of the evaluation planning process,
answer questions about NCI's concept of how the program would work
locally, and collect any documents that needed to be reviewed. Key
stakeholders are defined as those individuals who have a "stake" in the
program under study and/or the evaluation findings (Patton, 1997). So
far, these stakeholders have included a number of staff within the CRCHD
who will be involved in overseeing and managing the CNP once it is
underway. Stakeholders are important team members in the development of
an evaluation plan because they are generally experts in the program
itself and understand how it can or should impact the local community.
During the conduct of this feasibility study, drafts of the evaluation
planning matrix and conceptual framework, presented below, were
presented to NCI team members for review and comment. These documents
were then revised to incorporate team members' perspectives into the
final documents. As this evaluation planning process moves forward,
other key stakeholders, such as the CNP Principal Investigators (PIs) or
their designated evaluator, will need to be included in developing the
evaluation plan for Phase III.

***Step 2---Describe the Program***

After meeting initially with key stakeholders, the RTI team reviewed
program documents to gain a deeper understanding of the program and its
phases. Based on this understanding, we developed a draft conceptual
framework to examine and explain the relationships that exist among
program goals, objectives, and outcomes (CDC, 1999). We then developed
an evaluation planning matrix that specifies the program's goals and
objectives and begins to state potential evaluation outcomes that may be
of interest to the NCI team. This matrix also includes details about
data sources that will be required to measure program activities and
outcomes specified in the evaluation planning matrix. Below, we provide
an overview of the process followed to date to develop a comprehensive
evaluation plan for Phases I and II of the CNP.

**[Document Review]{.underline}**

Members of our interdisciplinary team reviewed all existing documents of
the CNP (e.g., the Concept Statement for the CNP, Program Announcement,
notes from meetings with CRCHD) and worked together to integrate this
knowledge into the conceptual framework and evaluation planning matrix,
which provided the basis for development of the evaluation plan. At this
stage of evaluation plan development, we do not know which local
programs will be funded or what their proposed activities will be.
However, once that determination is made, the evaluation planning matrix
can be updated and refined to better match what activities are to be
developed at the local level.

**[Conceptual Framework]{.underline}**

Conceptual frameworks typically include a visual depiction of how the
program is expected to impact change. A conceptual framework is
generally more theoretically based and conceptual than a logic model. A
logic model tends to be program specific and provides a more detailed
description of the planned activities and outputs of a program. A key
advantage of a conceptual framework, in the context of a multifaceted
program such as the CNP, is that it identifies the proposed
interrelationships across major program phases and activities (e.g.,
capacity building, partnership and collaboration development) and the
expected relationship between these and the program's outcomes.

**[Evaluation Planning Matrix]{.underline}**

An evaluation planning matrix of activities, outcomes, and data sources
is a useful organizational tool for evaluation planning that flows from
the conceptual framework. We created a matrix that relates to
appropriate "program building blocks" and begins to specify both
short-term (or process) and intermediate (or impact) outcomes that might
be expected from the community-based programs. The evaluation planning
matrix developed for Phases I and II of the CNP is presented in
***Appendix A***. Each section of the matrix is described below:

 *Goal*: The stated goals for Phases I and II are taken directly from
the Request for Applications (RFA) for the CNP respondents.

 *Objectives*: NCI specified objectives for each of the program goals
in the RFA that are provided in the first column of the matrix.

 *Activities*: Activities are those tasks that would typically need to
be conducted to meet a stated objective. Many of the activities in the
current matrix were stated in the RFA prepared by NCI (those appear in
**bold** in the matrix). However, for some objectives, all of the
activities that seemed necessary to accomplish an objective were not
specified and have been incorporated into the matrix as much as possible
at this time (in regular text). The activities of the current matrix
will need to be further refined once we know which programs received
funding since some of the specified activities will likely not be
addressed by any of the CNPs.

 *Outputs/Products*: Based on details from the RFA for the CNP and
other documents, outputs were specified for each of the program
objectives. Outputs are those tangible products specific to an objective
that indicate that an activity was completed (e.g., educational
materials, curriculum for trainings, organizational chart with CNP staff
specified).

 *Process Outcomes*: These outcomes are measures that assess how well
activities are being accomplished. These measures are often referred to
as "formative" because they focus on assessing how well a program is
being implemented, what is going well or not, and what lessons have been
learned that could inform ongoing program improvement. These outcomes
are often the most feasible to collect since they focus more on
"program-level" measures, which is a more accessible sample than those
they serve (e.g., the community members the program impacted). In the
matrix, initial process outcomes have been specified for each objective.

 *Impact Outcomes*: These outcomes are measures that assess the extent
to which the activities have resulted in change. For the CNP, the types
of change that are of greatest importance are at the individual (e.g.,
change in screening behaviors), community (e.g., changes in how
community-based organizations interact, local referral practices), and
policy (e.g., changes in eligibility for Medicaid coverage) levels.

 *Possible Data Sources*: For the matrix, we have begun to specify some
possible data sources that can be used to assess the outcomes for each
of the program objectives. These sources are described in detail in
Section 2.5. However, much of the work to determine which data sources
are the most applicable for and feasible to the CNP will need to be
completed after the programs have received funding. This work is part of
the process to "focus the evaluation," which is described in detail
below.

***Step 3---Focus the Evaluation Plan***

The evaluation must be focused to assess the issues of greatest
importance to stakeholders, given the available resources dedicated to
evaluation and surveillance (CDC, 1999). To focus the evaluation, NCI
and the future evaluator will need to work with other key stakeholders
to prioritize needs to address in the evaluation plan. Based on these
priorities, feasible evaluation strategies can be developed and
integrated into the CNP evaluation plan. During this process, it will be
important to incorporate both qualitative and quantitative data elements
into the evaluation plan to obtain a comprehensive overview of the CNP.
Examples of these types of data elements are described below.

The methods that could be used to understand the effectiveness of the
CNP include multilevel or hierarchical models that can account for how
multiple levels of influences can directly impact behavior, either
independently (e.g., a provider's recommendation to obtain screening) or
in combination with another level(s) of influence (e.g., a provider's
recommendation in combination with a local mobile mammography made
available to increase access). The evaluation would apply these methods
to develop complementary evaluation designs to address the proximal and
distal outcomes of the program. We recommend formative, process
evaluations to explore the fidelity and quality of program
implementation and outcome evaluations of progress toward proximal and
distal outcomes. In the following subsections, we describe how
quantitative and qualitative methods could be incorporated into the
final comprehensive national evaluation plan for the CNP.

**[Quantitative Methods]{.underline}**

The heart of our quantitative strategies focuses on the notion that
individuals will differ in their exposure to various program activities
(e.g., pilot projects, research, training). By relating these exposures
to outcomes (by way of the conceptual framework), quantitative methods
can be used to better understand how program components work
independently and jointly to contribute to the attainment of program
goals. For example, a local cultural competency training held for area
gynecologists could lead to a number of local practices scheduling
appointment times during hours after work. This training then impacts
both the cultural compassion that providers have for their patients and
access to services among local community members.

By making use of the mix of program activities across communities and
organizations, the evaluation can better measure the impact of each
program component on key outcomes to determine program successes and
failures. In assessing program effectiveness, the evaluation must also
attend to the possibility that the context within which these program
activities occur will influence program outcomes. Sociodemographic
characteristics and the capacity of the communities to organize and
deliver education, research, and training to disparate populations may
influence program effectiveness.

Multilevel models (hierarchical linear models) are a multivariate tool
to relate individual outcome measures to exposure to program activities
that occur in organizations, communities, and society at large. These
models can also account for the context within which the program
activities occur. Individual outcomes of interest include

 improvement in attitudes about obtaining cancer screenings,

 reduction in negative health behaviors (e.g., smoking, not using
sunscreen),

 increase in survivors obtaining care through clinical trials, and

 increase in access to screening services among members of disparate
populations.

For measures that are reliably captured in community-based surveys, as
an example, we can rely on self-reported data on program activities (to
the extent feasible). However, to get a more complete understanding of
the program's effects, it is also desirable to gather quantitative data
on program activities and policies from local organizations and provider
practices, as well as the communities, to understand the context within
which individuals make decisions about their cancer care.

These quantitative methods can help derive measures of program
effectiveness for many of the program efforts. Quantitative measures can
also be used to assess changes at the community and policy levels. We
recognize the limitations of quantitative methods, however, and suggest
complementary qualitative methods to strengthen the quantitative data
conclusions. The aim of these efforts is to better understand the
context within which change may be occurring and the "how" and "why" of
program implementation.

**[Qualitative Methods]{.underline}**

Qualitative methods generally consist of three kinds of data collection:
(1) in-depth, open-ended interviews; (2) direct observation; and (3)
review of written documents (Patton, 1990). In evaluation research,
qualitative methods are typically used to study programs for the
following reasons:

 Process or formative evaluation of a program to inform improvement of
its development or to provide understanding of contextual factors that
could be impacting its successful implementation

 A complement to quantitative methods to provide validation or a richer
explanation of the results

 An in-depth exploratory tool for use in cases where quantitative
methods are either not feasible or inappropriate

Qualitative tools for a comprehensive evaluation should be interwoven
across program phases or components to understand how program activities
may be mutually reinforcing. To illustrate, focus groups could be
conducted with a diverse representation of community members to explore
a variety of topics related to the program, such as how involved they
have been in each aspect of program implementation, what their exposure
to the program has been, how their access to services has changed (or
not), and other relevant issues across each of the CNP phases. Data from
these focus groups could be combined with other data from key
stakeholder interviews, record abstractions, network analysis, and
various ethnographic techniques to further elucidate changes relating to
the key program areas of capacity building, education and training, or
partnership development.

Evaluation research stresses the importance of developing measures that
provide for a "triangulation" of methods to ensure that the conclusions
drawn from qualitative analysis are reliable (Patton, 1990). Our
experience has taught us that this triangulation of data provides for a
much more in-depth overview of how the program has impacted change.

***Steps 4 through 6***

As shown in Exhibit 2, CDC recommends that six steps be completed for
evaluation implementation. Since this is an evaluation planning effort,
Steps 4 through 6 are beyond the scope of this project and are not
addressed here but should help guide the process for evaluation
implementation once that work begins. In the following sections, we
provide an overview of how the evaluation plan for CNP has been
completed to date and recommendations for developing and implementing a
national evaluation.

### 2.3 Engage Stakeholders in the CNP Evaluation (Step 1)

As shown in Exhibit 2, the CDC's "Framework for Program Evaluation in
Public Health" provides "a systematic way to improve and account for
public health actions by involving procedures that are useful, feasible,
ethical and accurate" (1999, p. 4). Step 1 in the evaluation process
involves engaging stakeholders. So far, the key stakeholders that have
been involved in developing the current evaluation plan represent the
national level, through the CRCHD. It is also important to identify and
engage local key stakeholders. Early in the program planning, NCI will
need to give the grantees guidance in (1) determining which local key
stakeholders should be involved and (2) to what extent they will be
involved in planning, implementing, and evaluating local efforts. This
subsection provides discussion points relevant for the process of
engaging stakeholders for the CNP.

Engaging stakeholders is a fundamental component of community-based
participatory research that can occur throughout the process of program
implementationfrom identifying program goals and objectives to
conducting the evaluation. However, the depth of involvement from the
community should be discussed prior to or in the early stages of the
program. The role of the community can best be thought of as a
continuum. This means that involvement can range from not being very
involved (such as providing feedback on materials and research developed
by the program) to being very involved (such as completing tasks that
have a direct impact on what the program does and accomplishes). Since
the CNP is required to have a community-based, participatory approach,
it is likely that each Community Advisory Group (CAG), a required
component of Phase I, will play an active role in every stage of program
development, implementation, and evaluation.

The evaluation process begins with an investment from the stakeholders
in what will be learned and what will be done with the evaluation. For
the local-level evaluation, to be conducted by each individual CNP, the
stakeholders to engage in this process will likely be members of the
CAG. However, some CNPs may decide that additional, guest members need
to be invited to participate in order to provide additional expertise on
evaluation. Prior research has identified three principal groups of
potential stakeholders to include in evaluation planning and
implementation (Mathie and Greene, 1997):

 Those involved in program operations (e.g., partners, collaborators,
funding officials, administrators, managers, and staff)

 Those served or affected by the program (e.g., individuals,
researchers, providers)

 Primary users of the evaluation (e.g., NCI)

In a community-based participatory approach, key stakeholders include
community members, organizational representatives, and researchers who
work in partnership on all aspects of the research process (Israel et
al., 2003). Since the "communities" differ for each level of evaluation
(i.e., national, regional, or local), the stakeholders for the national
evaluation, to be conducted through the CRCHD, will likely belong to
different groups than those involved with each local CNP evaluation. The
"community" for the national evaluation includes the programs
themselves, since they will be supplying the measures for the national
evaluation. Key stakeholders in the national evaluation will include
each PI or his/her designated replacement and CRCHD staff members. Other
stakeholders may be invited experts on program evaluation and/or cancer
control and representatives from some local communities involved in CNP.
To be truly "participatory" at the national level, the key stakeholders
should include a mix of people with different backgrounds and
perspectives. Given the size of the CNP, the number of stakeholders
involved will need to be streamlined and a communication plan will need
to be developed to ensure that the stakeholders working with the
national evaluators are effectively relaying progress to all of their
counterparts.

One structure to consider for ensuring cross-collaboration and a
participatory approach at the national level is to convene a group of
stakeholders for the national evaluation fairly soon (ideally within 90
days) after award of the CNPs. NCI and the evaluator could work with the
(up to) 22 CNPs to determine who will be members of this group, using a
list of criteria developed in collaboration with the PIs (e.g., two
members are to represent the management from local CNPs, two should be
community members of CNPs). The purpose of the initial meeting will be
to provide an overview of the work that has been completed to date and
engage the group in establishing a mechanism to obtain their ongoing
input in the national evaluation plan. One structure that has worked
well for several other national evaluations is the development of an
Evaluation Advisory Committee (EAC). These Committees tend to include
key stakeholders of the program, as well as experts in program
evaluation, to ensure that the resulting evaluation plan is feasible for
the programs and will realistically achieve outcomes of interest
(Patton, 1997). It is ideal for these groups to have approximately 10 to
12 regular members in order to provide productive meetings that result
in direct guidance to the evaluation plan. During the first year of CNP
implementation, this Committee should probably meet fairly regularly to
complete such tasks as

 reviewing versions of the conceptual framework as it is revised to
better incorporate local efforts,

 providing guidance on research methods to use at the local and
national levels,

 prioritizing evaluation efforts so that the outcomes of greatest
importance are incorporated into the evaluation design, and

 critiquing evaluation instruments and tools to be used in the field.

Another role of the EAC would be to ensure adequate communication
between what is happening with each local CNP and the national
evaluation. The people designated from each CNP to be members of the EAC
should be willing to report regularly to their own CAG about what is
being advised for the national evaluation. Data that are collected
locally will be used to inform the national evaluation, so the CAG's
input and guidance into what is done for evaluation will be important.
The ideal model for interaction is depicted in ***Exhibit 3*** with
overlapping circles. One circle depicts the national EAC, and the others

[]{#__RefHeading___Toc88540568 .anchor}Exhibit 3. Proposed
Interconnectedness of the Local Community Advisory Groups and the
National Evaluation Advisory Committee

  -----------------------------------------------------------------------

  -----------------------------------------------------------------------

illustrate how each local CAG would potentially be contributing members
to that national group.

Ideally, these groups would be interchanging information so that each is
aware of what is expected but also knows what is realistic and of
interest to local community members so that their thoughts can be taken
into consideration in designing the national evaluation. The national
evaluator could help to facilitate this communication by convening
monthly conference calls among all of the CNPs and using these calls to
inform stakeholders of the progress being made. An additional strategy
for obtaining input is to create a Web-based "bulletin board" where
draft tools and instruments for the evaluation are shared so that each
stakeholder can provide input. This board could also be used as a place
for each site to post their local evaluation efforts if they are in need
of input and technical assistance. This model is somewhat different from
the conventional approach to program evaluation. As shown in ***Exhibit
4***, there are distinct differences in how a community-based
participatory evaluation would occur and who would be involved, as
opposed to conventional approaches to evaluation research.

Exhibit 4 summarizes the stark contrast between the two approaches;
however, a mixed method approach can be used, with varying degrees of
input from the community. For the national evaluation, because the need
for outcomes of accountability may be greater, the focus may be more on
conventional approaches to evaluation, whereas at the local level, the
focus may be more on participatory evaluation, since this is required by
the RFA. NCI will need to decide early on in the process of program
implementation which of these approaches to use in guiding the CNPs on
their evaluation and in overseeing the work for the national evaluation.

[]{#__RefHeading___Toc88540569 .anchor}Exhibit 4. Comparison of
Conventional and Participatory Evaluation Approaches

  --------------------- ------------------------ -------------------------
  Issues for            Conventional Evaluation  Participatory Evaluation
  Consideration                                  

  Who                   External experts conduct Community leads the
                        the evaluation           efforts, supported by
                                                 project staff

  What                  Predetermined indicators People identify their own
                        of success are measured, indicators of success
                        primarily cost and       
                        health outcomes          

  How                   Focus is on "scientific  Focus on self-evaluation
                        objectivity," uniform    with the use of simple
                        and complex              methods adapted to local
                        methodologies, and       culture; open, immediate
                        delayed access to        sharing of results
                        results                  

  Why                   Accountability of        To empower local people
                        funding, usually impact  to initiate, control, and
                        related, to determine if take corrective action
                        the funding should       
                        continue                 
  --------------------- ------------------------ -------------------------

Source: Adapted from Springett, 2003.

### 2.4 Describe the CNP (Step 2) 

As described in Section 2.2, the second step in the evaluation process
is to describe the program that will be evaluated. For this project, we
reviewed all available documents about the CNP and met with CRCHD staff
to discuss the overall program goals and objectives. We developed two
tools for the evaluation plan. First, we presented a draft conceptual
framework for the overall program to NCI team members for their input
and feedback. Individuals who were involved in providing feedback
included those who will serve as Project Officers to the funded programs
and/or those with specific expertise in one or more of the areas of
outcomes (e.g., community, policy). Based on their feedback, the
framework was then revised and has been used extensively in our
evaluation planning work. The resulting conceptual framework (***Exhibit
5***) is described in detail in the following sections.

The CNP conceptual framework provides a "top of the trees" view of how
the CNP is proposed to work. It shows the relationship between its three
major program components: the *Program Building Blocks* or foundation
that the local initiatives will draw from; the primary *Focus of
Activities* that the Program envisioned; and the desired short-term,
intermediate, long-term, and ultimate *Outcomes*. It provides a logical
sequence of how the resources invested by the CNP will lead to program
improvements and the desired results.

Second, we created an evaluation planning matrix that is presented in
Appendix A. This matrix provides details on how Phases I and II of the
CNP are expected to be implemented and what the resulting outcomes may
include. The matrix is much more detailed than the conceptual framework
and can be mapped directly to the structure of the CNP. In the following
sections, we describe each of the three major program components as
depicted in the CNP conceptual framework. For each component, we
describe how the details from the

[]{#__RefHeading___Toc88540570 .anchor}Exhibit 5. CNP Draft Conceptual
Framework

![](media/image2.wmf){width="8.5in" height="5.073611111111111in"}

evaluation planning matrix should be used to guide the addition of Phase
III to the evaluation plan and the implementation of the evaluation once
awards have been made.

#### 2.4.1 Program Building Blocks

The Program Building Blocks (see Exhibit 5) consist of the
infrastructure, partners, and collaborators that will direct and
facilitate development and implementation of the Program activities.
These interrelated conceptual building blocks link directly to the first
three objectives of Phase I, in the left-hand column of the evaluation
planning matrix (see Appendix A, pp. 1-4) and are described separately
below:

 Objective I.1: Develop core organizational infrastructure for local
CNP (p. A-1)

 Objective I.2: Develop partnerships with communities experiencing
cancer disparities and organizations that can aid in reducing cancer
disparities (p. A-2)

 Objective I.3: Have at least four collaborations with NCI
Centers/Divisions/Offices (p. A-4)

In order for the CNPs to successfully address each of these objectives,
they will need to be versed in the principles of community-based
participatory research. The value of participatory research lies in its
ability to improve evaluations, utilize evaluations, and empower
participants (Patton, 1997; Gergen and Gergen, 1991; Morgan, 1983;
Papineau and Kiely, 1996; Reason, 1988). The characteristics of
successful participatory research are based on recognizing and planning
for the challenges that accompany it. Such challenges include overcoming
power differences (Israel et al., 1998; Greene, 2000; Mathie and Greene,
1997), determining what is considered program success, adequately
documenting success, gaining access to data to measure outcomes
(Nichols, 2002), and juggling the differing perspectives and priorities
of the community and funding sources of the researcher (Schulz et al.,
2003). However, proponents of participatory research suggest that the
"active engagement of those who will be affected by the research is
necessary in order to challenge power differentials that currently
exclude some from the process of creating knowledge" (Schulz et al.,
2003, p. 295). Therefore, if the challenges are addressed in the
planning stage of the programs, and a supporting infrastructure provides
a solid foundation for community work, then a climate of program success
can ensue and thrive.

**[Infrastructure]{.underline}**

The first Program Building Block in the CNP conceptual framework (see
Exhibit 5) corresponds to the infrastructure created by the CNP, stated
as "establish and maintain an infrastructure for addressing cancer
disparities." The CNP model assumes an asset perspective to create
participatory, collaborative relationships between the researchers and
cultural and/or racial/ethnic minority and underserved communities. The
infrastructure brought to the community by CNP will provide the
foundation for conducting local research, training, education, and other
initiatives. However, studies have shown that engaging community
residents in the design, implementation, and evaluation of interventions
in their communities can substantially contribute to efforts to improve
health. As the programs build their infrastructure, they will need to
plan a structure that lends itself to supporting community-based
participatory research efforts.

The coalition development literature provides some insight into
essential structural components to support community-based participatory
research. In their review of 252 collaborative partnerships, Roussos and
Fawcett (2000) identified seven interconnected and modifiable structural
components that enhanced their individual and population-level health
outcomes:

 A clear vision and mission of what they wanted to accomplish

 Effective leadership

 Regular time to plan what they want to do and strategies to accomplish
their goals

 Regular communication to update partners on progress

 Technical assistance

 Ongoing funding mechanism

 A comprehensive evaluation plan

Leadership was the most important organizational factor that contributed
to partnership effectiveness. Effective leaders are able to engage broad
and active participation, ensure broad-based influence and control, and
facilitate productive group dynamics. Communication, facilitating
meetings, negotiating with partners, and networking were identified as
core skills of effective leaders (Roussos and Fawcett, 2000).

During Phase I, each grantee must hire and maintain core administrative
and evaluation staff who will provide the CNP leadership. These staff
members include a PI, project manager, research coordinator, policy
analyst, and program evaluator and two to three administrative staff.
Only the project manager and administrative staff are full-time.
Additionally, the grantees must also create a Project Steering
Committee, Regional Advisory Committee, and a CAG as formal structures
to ensure community participation. The CNP will determine who will
participate and the mix among these groups; however, NCI will expect the
local CAG to have a strong presence in each CNP. A formal structure of
the CAG will likely be required, such that community members are
afforded the opportunity to provide ongoing input and direction to the
CNP. To accomplish this, regular meetings and communication plans will
need to be established early on by each CNP to ensure ongoing CAG
involvement. To transition to Phase II, the grantees must demonstrate
that they have developed the necessary infrastructure to support the
CNP.

Ongoing management processes include collaborating with NCI to provide
training and technical assistance to CNP staff, advisory group members,
and community partners; updating the strategic plan; observing and
communicating progress toward benchmarks; and attending CNP and NCI
meetings. The third column of the evaluation planning matrix (see
Appendix A) presents some possible outputs and products from these
infrastructure-building activities, including resumes, organizational
charts, budgets, cost reports, strategic plan, communication plan,
annual progress reports, written communications (e.g., e‑mails, memos,
faxes, meeting minutes), membership lists, and a CNP database.

To evaluate how well they build and maintain this infrastructure,
evaluation questions of interest for the CNPs include the following:

 What is the climate within the CNP? And within the larger lead or
institutional agency?

 What is the local community context that could be impacting how well
the CNP can function?

 How are the values of the CNP and lead agency similar or dissimilar?

 How do these values facilitate or impede CNP efforts?

 How are funds administered? What proportion of funds are provided
directly to the community?

 What is the timeline on which organizational structure was established
and operationalized?

 What are the turnover rates for the staff and advisory groups?

 What is the structure and size of the CNP?

 How were the advisory group members recruited? How were they selected?

 What is the structure and size of the CAG? What is the
interrelationship between the CNP and CAG?

 What are the skills, experiences, and worldview of the CNP leadership
and staff?

 What are the roles and responsibilities of advisory groups?

**[Partnerships]{.underline}**

The second and third Program Building Blocks in the CNP conceptual
framework (see Exhibit 5) relate to partnerships. Since partnerships are
the most essential component of the CNP, a distinction has been made
between two groups of partners. The first is partnerships with
community-based organizations (CBOs) and local community leaders,
particularly those who work directly with members of the disparate
populations. These partnerships would focus on assessing local community
resources and identifying the gaps in services that are impacting cancer
care among disparities. CBO partners would likely be members of each
CNP's Community Advisory Group (CAG) and would help to facilitate
leveraging of local resources to address needs in the community. The
second type of partnership (or the third Program Building Block) is with
the organizations that can aid in reducing disparities. These
organizations may include representatives from local hospitals, private
foundations and business groups, government entities such as the local
health department, and policy makers. These partnerships are essential
to impacting change and would also be ideal members to include on each
CAG. As the community identifies its assets and needs, these partners
can potentially provide the infrastructure, resources, and funding
needed to fill any gaps in services. The following provides an overview
of key elements to consider in establishing and evaluating these
partnerships.

Partnership, coalition, network, and alliance are all terms used to
describe the goal of community-based initiatives, such as the CNP.
Developing since the 1980s, coalition development literature provides
extensive case studies on the ideal structure and process of these
partnerships. For both the structure and process, the Program Building
Blocks represent the need for the CNP to identify, create, and/or renew
partnerships with different groups of people in their community.
However, establishing these partnerships is just the beginning of the
process and will need to be followed with ongoing efforts to nurture the
relationships and ensure "buy in" from all essential partners.

The community researcher is a central member of the community-based
partnerships. Participatory researchers share their knowledge and
expertise, serve as facilitators of the process, and coach in developing
the capacity of the community members (Small, 1995). But each partner
shares their "unique strengths and shared responsibilities" (Green et
al., 1995, p. 12) in order to better understand the social and cultural
dynamics of the community, and integrate this knowledge with action to
improve the local health and well-being (Schulz et al., 1998). Experts
in community-based participatory research have identified nine
principles that should be followed and integrated into their work to
develop partnerships (***Exhibit 6***).

These nine principles provide a list of ideals for community-based
researchers to strive to achieve in establishing their local
partnerships. However, there are many challenges to this process of
partnership development. After an in-depth review of community-based
participatory research, Wallerstein (1999) identified several
difficulties with putting this methodology into practice, including
creating a rigorous evaluation design, defining "community," and
measuring "community" due to inadequacies of existing datasets at the
community level. Perhaps even more importantly, Boyce (2001) in his
review of 100 health promotion projects found that 41 percent of
community members participated in planning activities, 29 percent
participated in implementation, 19 percent regularly attended meetings,
and only 11 percent participated in evaluation. These findings suggest
that community involvement is easier to accomplish earlier in a project
than it is on an ongoing, long-term basis. Obtaining community buy-in or
ownership into a research project requires a long-term commitment to
addressing disparities within that community for years to come.
Therefore, it is critical that researchers approach this process with
the idea that the CNP is just the starting point (or possibly a
continuation) to this partnership and will serve as a building block for
identifying resources and additional partnerships to improve the local
health.

In evaluating these partnerships, it is important to be aware that they
develop over time, and therefore measures to assess their development
should be sensitive to this process and the structure within which they
are occurring. According to Butterfoss, Goodman, and Wandersman (1993),
coalitions or community-based partnerships evolve through four steps:
formation, implementation, maintenance, and accomplishment of goals or
outcomes. Florin, Mitchell, and Stevenson (1993) proposed a more
extensive seven-stage process: initial mobilization, establishing
organizational structure, building capacity for action,

[]{#__RefHeading___Toc88540571 .anchor}Exhibit 6. Nine Principles of
Community-Based Participatory Research

+-----------------------------+----------------------------------------+
| Principle                   | > Description of Principle             |
+-----------------------------+----------------------------------------+
| CBPR recognizes community   | > Approach incorporates an attempt to  |
| as a unit of identity.      | > identify and work with existing      |
|                             | > communities of identity and to       |
|                             | > strengthen a sense of community      |
|                             | > through collective engagement.       |
+-----------------------------+----------------------------------------+
| CBPR builds on strengths    | > Approach is "asset" focused, as      |
| and resources within the    | > opposed to "deficit" focused and     |
| community.                  | > starts by identifying and building   |
|                             | > on strengths, resources, and         |
|                             | > relationships that exist within      |
|                             | > communities of identity.             |
+-----------------------------+----------------------------------------+
| CBPR facilitates            | > All parties participate and share    |
| collaborative, equitable    | > responsibility for all aspects of    |
| partnership in all phases   | > the program, including problem       |
| of the research.            | > definition, data collection,         |
|                             | > interpretation of results, and       |
|                             | > application of results to address    |
|                             | > local needs.                         |
+-----------------------------+----------------------------------------+
| CBPR promotes co-learning   | > Approach facilitates the reciprocal  |
| and capacity building among | > transfer of knowledge, skills, and   |
| all partners.               | > capacity or resources.               |
+-----------------------------+----------------------------------------+
| CBPR integrates and         | > Approach involves a commitment to    |
| achieves a balance between  | > translating and integrating research |
| research and action for the | > results with community change        |
| mutual benefit of all       | > efforts underway with the goal of    |
| partners.                   | > benefiting all partners involved.    |
+-----------------------------+----------------------------------------+
| CBPR emphasizes local       | > Approach addresses public health     |
| relevance of public health  | > problems that are locally relevant   |
| problems and ecological     | > and considers the concept of health  |
| perspectives that recognize | > from a positive model, incorporating |
| and attend to the multiple  | > the integration of environmental     |
| determinants of health and  | > influences into the research with an |
| disease.                    | > ecological, broad-based perspective. |
+-----------------------------+----------------------------------------+
| CBPR involves system        | > Approach seeks to build the          |
| development through a       | > competencies within a system in      |
| cyclical and iterative      | > order to enhance learning and        |
| process.                    | > sustainability.                      |
+-----------------------------+----------------------------------------+
| CBPR disseminates findings  | > All partners receive the results of  |
| and knowledge gained to all | > the research and are involved with   |
| partners and involves all   | > determining the process of relating  |
| partners in the             | > the results.                         |
| dissemination process.      |                                        |
+-----------------------------+----------------------------------------+
| CBPR involves a long-term   | > Approach involves the development of |
| process and commitment.     | > trust in working with the community  |
|                             | > to reduce disparities, which is a    |
|                             | > long-term process and involves a     |
|                             | > longer-term commitment than one      |
|                             | > research project allows. CBPR        |
|                             | > highlights the need for communities  |
|                             | > to find mechanisms for               |
|                             | > sustainability.                      |
+-----------------------------+----------------------------------------+

Note: CPBR = community-based participatory research.\
Source: Adapted from Israel et al., 2003.

planning for action, implementation, refinement, and
institutionalization. Any evaluation designed to assess both types of
partnerships essential to CNP would need to be developed so that the
effects of the process are taken into account in interpreting the
findings.

Structure can also influence group interaction and dynamics that shape
whether the partnership is effective (Schulz, Israel, and Lanz, 2003).
Perceived collective efficacy, a group-level interdependence construct,
asserts that people will put forth conjoint effort toward a shared
belief to exercise control over community life. Research findings show
that the stronger the perceived collective efficacy, the higher the
group aspires, the more motivated the group members are to achieve, the
stronger their staying power, and the greater their performance
(Bandura, 2001). Partnership provides an effective structure for
collective community action but needs to be planned for and adequately
managed by each of the CNPs.

NCI is requiring that the CNP develop at least one partnership with
community members, providers, and researchers to progress to Phase II.
The motivation for and purpose of the partnerships will vary according
to the community and drive the structure and outcome. The CNP's local
context will vary significantly in the amount and type of assets
available to each community. The partnerships will undoubtedly reflect
that diversity. During the early phases of partnership development, it
will be important for the CAG to assess existing cancer control
resources, assets, and needs within the community of interest. This type
of context assessment or study can also identify political, social,
cultural, and environmental opportunities and barriers. Doing this type
of assessment would initially help the CNPs to plan relevant and
effective interventions for that specific context. For example,
availability of cancer resources may be more of an issue in a rural
setting or primary and secondary providers may be available in an urban
setting but not accessible due to clinic hours. Once the CNPs are up and
running, this type of effort can be helpful in assessing and
understanding factors that influence the Program's progress toward
outcomes. Engaging broad community participation in this contextual
assessment is essential for developing and maintaining the partnership.

Some possible outputs and products from these partnerships are listed in
the third column of the evaluation planning matrix (see Appendix A).
Possible outputs/products include a map of existing and new cancer
control programs, partnerships, and providers; a matrix of community
partners by focus areas; and memoranda of understanding (MOU) and
memoranda of agreement (MOA) developed with each CNP partner.

Local relationships and capacity building will affect what the CNP is
able to accomplish. Potential evaluation questions to answer include the
following:

 What partnerships exist within the disparate communities?

 What is the history of programs and providers working together?

 Who are the key stakeholders?

 What types of partnerships have been developed?

-   What CBOs are involved with CNP? What is their role? What has been
    their contribution, if any?

-   What business or government entities are involved with CNP? What is
    their role? What has been their contribution, if any?

 How are the CAGs structured? Are they representative of the community?
What CBOs or local business and government entities are not involved
with CNP and possibly should be?

 How was the community involved in the development and implementation
of the CNP plan?

 How many MOAs and MOUs are in place?

 What are partners' perceptions of the strengths and weaknesses of the
CNP?

 What are the opportunities and threats from the external environment
that could be impacting how well the CNP can influence individual,
community, and/or policy changes?

**[Collaborations]{.underline}**

The fourth Program Building Block in the CNP conceptual framework is
collaborations, which is to "develop NCI collaborations (including with
CIS)." In some partnership models, collaboration is the most involved,
most complex, most difficult stage in a continuum for characterizing
relationships among partners (Himmelman, 2001). It is the ultimate
engagement of community members. Lasker and Weiss (2003) define it as
synergy, "the breakthrough in thinking and action that are produced when
a collaborative process successfully combines the complementary
knowledge, skills, and resources of a group of participants" (p. 25).
These distinctions highlight the error of interchanging the terms
"partnership" and "collaboration."

Lasker and Weiss' (2003) Community Governance Model provides a
conceptual map of the collaborative process. It hypothesizes that
leadership and management are critical to participation and
collaboration. It describes a structured method for identifying who
should be involved by focusing on three levels of collaboration:
empowering individuals through their active involvement, building
interpersonal relationships with other people who are involved, and
creating a synergy through the combined assets of all partners. The
result is the collaborative problem solving that creates social
changethe synergy. Everything in the collaborative process depends on
the people and organizations that participate in that process. Getting
and keeping all of the right people at the table is a challenge.

For the CNP, the focus of collaboration is between the NCI and the local
program. Programs are required to establish a collaborative effort with
the Cancer Information Service (CIS) and with other NCI entities as
recommended by their Project Officer. NCI will provide guidance as to
which collaborations may be best suited for each CNP. Possible outputs
or products of these collaborations may include culturally and
linguistically appropriate education materials and procurement of
additional funding resources.

To evaluate how well these collaborations occur, possible evaluation
questions include the following:

 What factors facilitate or impede collaborations?

 How are collaborative relationships maintained?

 What was NCI's role in facilitating the collaborations?

 What has been the community's role in building the collaborations?

 What have been the outcomes of each collaboration?

#### 2.4.2 Focus of Activities

As shown in Exhibit 5, the "focus of activities" relates to the actions
the local programs will take to impact cancer-related disparities.
Initially, once the Program Building Blocks are in place, the resulting
action will be to "develop, through a community-based participatory
process, an understanding of relevant resources, assets, and needs to
address cancer disparities by creating a synergy of efforts and
leveraging local resources." As described in Section 2.2, the Program
Building Blocks are hypothesized to result in a synergy of effort at the
local level (depicted as an oval in the second column of the CNP
conceptual framework \[see Exhibit 5\]). This synergy will influence the
focus and intensity of activities implemented in the CNP communities.
Many factors can influence how well the CNP is able to inspire their
local CAG to implement effective initiatives. Key factors include
whether the CAG provides a broad and deep representation of the
disparate community, such as if the local community leaders are
involved, the agencies they represent, and/or their peers and family.
Hays et al. (2000) demonstrated that a partnership's ability to change
policy was positively related to having broad sector representation.
Serving as advocate for the partnership and community, this "boundary
spanner" plays a critical role in community organizing across diverse
sectors and between organizations, by breaking down the boundaries that
separate these nontraditional partners (McLeroy et al., 1988;
Brinkerhoff, 2002). This factor implies that a broad array of needs and
assets within the community should be identified so that gaps in the
cancer care of disparate populations can be targeted. As the local
partners are working together to identify the community's assets and
needs, they will undoubtedly learn about the work of others and the
available, additional resources that may be used to leverage the CNP and
adequately address gaps in care. It is through this combined effort that
the CNP will be able to identify efforts that need to be implemented to
effect local change. Two objectives of Phase I of the CNP relate
directly to this component of the conceptual framework and are outlined
in detail in the evaluation planning matrix (see Appendix A, pp. 4-5):

> Objective I.4: Increase the utilization of beneficial interventions to
> reduce disparities in the community (p. A-4)
>
> Objective I.5: Obtain non-CRCHD funding for community-based education
> and training activities directed at reducing cancer disparities (p.
> A-5)

Possible evaluation questions to answer in order to assess the extent to
which these objectives were met include the following:

 What do the local partners have to offer the community? What
interventions were already in place before CNP funding?

 How were interventions created or developed?

 What was the role of the CAG in the development and implementation of
the interventions?

 Are the interventions implemented "evidence-based"?

 What additional funds were sought? What funds were received?

 What is the evidence that the community was able to strategically
leverage CNP funding to address local gaps in services?

Once the gaps in the community have been identified, local programs will
conduct three levels or types of activities to effect change:
individual, community, and policy. These levels of activities have been
conceptualized through the reliance on existing health behavior theory,
using the socio-ecological model as a basis for how these activities can
work to effect change. In general, ecological models direct attention to
both behavior and its individual and environmental determinants. They
examine multiple levels of influence and provide the basis of the
socio-ecological framework suggested to evaluate the CNP. These types of
models are one of the principles of community-based participatory
research in that they take into account the individual, the immediate
contexts within which s/he lives (such as family or social network), and
the larger contexts in which these are embedded (such as community or
society) (Bronfenbrenner, 1990). Given this attention to multiple units
of influence and the complex set of determinants of health, "CBPR
efforts strive to achieve broad-scale social changes aimed at
eliminating health disparities" (Israel et al., 2003, p. 57).

The typologies of socio-ecological models may vary. However, they all
capture micro, meso, exo, and macro system levels. Microsystems focus on
the face-to-face influences in specific settings or interactions such as
within one's family, informal networks, and work groups. Mesosystems
target the interrelations among the settings that the individual is
involved in, such as family, peer groups, and church. The exosystem
refers to the forces within the larger social system in which the
individual is embedded, such as economic instability, unemployment, and
others. The macrosystem reflects cultural beliefs and values that
influence both the micro and macrosystem (e.g., cultural beliefs about
smoking, seeking health care). Overall, ecological models support the
idea of "a reciprocal causation between the individual and the
environment" (McLeroy et al., 1988, p. 354; see also Pervin, 1968;
Bronfenbrenner, 1979). Adapting from the McLeroy et al. model (1988),
RTI has identified three socio-ecological levels described by the CNPs
that may effect change in their efforts to reduce cancer disparities:

 Individual-level change depicts the attitudes, beliefs, perceptions,
and behaviors of the people targeted by the local CNP (e.g., the
relationship between screening behaviors and attitudes). At this level
of change, as depicted in the CNP conceptual framework (see Exhibit 5),
the CNP will "develop and conduct community-based education programs and
activities."

 Community-level change imparts the significance of how system-wide
change can influence the health behaviors of local individuals. Examples
of community-level changes relevant to the CNP could include whether
organizations within the community work together more effectively to
address cancer care or if community norms are influenced to the point of
impacting individuals to change their cancer prevention behaviors. This
level of change is referred to in Exhibit 5 and presented similar to
policy-level changes, since both are more of a systems-level approach to
impacting cancer disparities than the individual-level initiatives the
CNP may implement.

 Policy and public advocacy have important implications for communities
and can include whether local agencies work together to provide services
to low-income, uninsured peoples, or a program changes their eligibility
requirements to allow for more people in the community to be served.
This level of change, along with community-level change, is referred to
in the final box of the CNP conceptual framework as "develop and conduct
community-based provider training, recruitment and training of minority
students, and relevant research (e.g., pilot projects)."

Overall, the socio-ecological model provides a framework for examining
the focus of the activities to determine their broadness and
comprehensiveness in addressing cancer health disparities. Objectives
specifically related to this aspect of activity development are in Phase
II of the CNP and include the following:

> Objective II.1: Develop cancer disparities research that focuses on
> the spectrum of research issues necessary to reduce cancer disparities
> (p. A-7)
>
> Objective II.2: Train researchers, particularly those from
> racial/ethnic minority and underserved populations, in community-based
> participatory intervention research to reduce cancer disparities (p.
> A-8)
>
> Objective II.3: Develop pilot projects in community-based
> participatory disparities research (p. A-9).

The following evaluation questions are important to consider when
assessing progress made toward meeting each of these three objectives:

 What existing sources of data were collected to build knowledge of
cancer disparities?

 What research is conducted? How is the research used to impact future
work?

 What was the quality of community involvement in the research?

 What current policies impact cancer control in the community? What
about them needs to be changed? Has any change occurred over time?

 How many researchers from racial/ethnic minorities attended training?

 Did the training accomplish what was intended (e.g., change in
attitude, career goals)?

 What population is targeted by the pilot project?

 How well did the pilot projects identify and/or address the needs of
the communities?

 What is the role of the community and the CAG in determining which
interventions, research, training, and pilot projects to implement?

#### 2.4.3 Outcomes of Interest to the CNP

The resulting evaluation plan focuses on four different sets of outcomes
that are based on their timing. First, the CNP conceptual framework
(Exhibit 5) and the evaluation planning matrix (Appendix A) specify
short-term or process outcomes. These measures focus on the formative
aspects of the program that are used to assess the extent to which the
program is being implemented as planned. Common process measures are
related to elements of change that are precursors to behavioral or
system changes. To illustrate these types of short-term and intermediate
changes, the Precede-Proceed model has been incorporated into the CNP
conceptual framework (Green and Krueter, 1991). This model was
originally designed as a program planning model to use in applying
underlying theories (e.g., socio-ecological level) to an intervention
with the goal of identifying and implementing the most appropriate
strategies (Glanz et al., 1997). The model has its origin in the same
principles used in medical diagnosis. For example, a medical diagnosis
*precedes* a treatment plan; thus, an educational diagnosis should
precede an intervention plan and evaluation plan.

The Precede part of the model was developed in the 1970s, and it is an
acronym representing factors that influence any given health behavior
(Green and Krueter, 1991). A description of these factors follows:

 *Predisposing factors* are those antecedents to behavior that provide
the rationale or motivation for individual behavior. As shown in the
short-term outcomes in the CNP conceptual framework (Exhibit 5), these
predisposing factors might include changes in individual knowledge,
beliefs, attitudes, and values with regard to cancer-related issues.

 *Reinforcing factors* provide the continuing reward or incentive
(e.g., reinforcement) for the targeted behavior. These are generally
system- or community-level changes that support individual behavioral
change. Depending on how the community is defined, examples related to
the CNP could include increases in health professionals' knowledge and
sensitivity related to cultural compassion, increases in understanding
of issues impacting cancer control among disparate populations, and
mobilization of the community to support CNP efforts and potentially
positively impact community norms related to cancer care (see Exhibit 5,
short-term outcomes).

 *Enabling factors* provide the facilitation of a motivation for
behavior change to be realized. For the CNP, these factors relate
primarily to community-based changes, such as improved local referral
patterns among providers; improved provider interactions with members of
the disparate groups; increased access to preventive, screening,
diagnostic, and treatment services among disparate groups; and the
leveraging of funding among community partners to effectively address
gaps in services. All of these changes would enable an individual to
receive more adequate cancer care and are depicted in the CNP conceptual
framework as intermediate or impact outcomes (see Exhibit 5).

The Proceed part of the model was developed in the early 1990s and was
added to Precede to highlight the important role of environmental
factors as determinants of health and health behaviors. The Proceed
component of the model is more specific to program planning and is
therefore not highlighted here but is presented in detail in the 1991
book by Green and Krueter.

In the following sections, we provide a brief overview of the four types
of outcomes presented in Exhibit 5.

**[Short-term (Process) Outcomes]{.underline}**

Predisposing and reinforcing factors have been conceptualized as the
short-term outcomes desired from the CNP. These factors/outcomes are
precursors to behavior, system, or policy changes that might be
accomplished in the 5 years of funding for the CNP. Many researchers
refer to these outcomes as "process" measures since they generally
capture program-level activities and the extent to which the program was
successful in implementing plans. For the CNP, short-term measures at
the program, individual, community, and policy levels will be important
to incorporate into the national evaluation plan.

Within 1 to 2 years of program initiation, individual outcomes such as
improvements in knowledge, attitudes, beliefs, and values of community
members with regard to cancer issues should be measurable through a
national evaluation. If these individual-level changes are occurring,
then we can assume that changes in behavior will likely follow for most
individuals.

Community and policy short-term outcomes are also related to the
knowledge and values of the local systems. Within the community, the
knowledge and attitudes of local health care providers with regard to
cancer care for disparate populations have important influences, for
example, on how often patients are able to obtain recommended services.
The evaluation could measure the extent to which local efforts to
provide training in cultural competence changed the attitudes of
providers and increased their willingness to better accommodate
disparate groups through their practice, by including clinic hours after
work and on weekends, providing transportation, or opening a new office
in an inner-city location because their other office is too far for many
people in the community. Within 2 years, it would also be possible to
determine if changes have begun in the community norms related to cancer
care. In some communities, there may be a great deal of fear about
obtaining screening for cancers, such as prostate or colorectal
screenings. With ongoing community-based education efforts, people could
come to realize the benefits of obtaining this screening so that they
are less fearful of the results. As more people obtain screenings and
talk to their peer group about their experience, the result is that the
community norms gradually change. The precursors for changes in
community norms are an increased understanding among the community of
issues impacting cancer control and information on how the community
members can obtain the care they need.

The policy-level change involves those laws, regulations, formal and
informal rules, and understandings that are adopted on a collective
basis to guide individual and collective behavior (Wallack, 1990). It
has several venues: policy development, policy advocacy, and policy
analysis. Research suggests that programs that recognize the need to
facilitate behavior change by removing policy (see Schmidt, 1995) and/or
creating policy can encourage health-promoting behaviors. The CNP plans
to address health-promoting behaviors such as screening for the targeted
cancers: cervical, breast, colorectal, and prostate. Therefore,
examining policy for the CRCHD could yield a worthwhile understanding of
the political climate related to cancer research and activities.
Short-term outcomes at this level could include increased awareness
among policy makers with regard to policies that impact cancer care and
how they can be revised to positively influence the community's health.
If the local CNPs are working with local policy makers to address cancer
issues, the evaluation would be able to capture the extent to which
their increased awareness has influenced their willingness to impact
change.

**[Intermediate (Impact) Outcomes]{.underline}**

Impact measures, or those measuring change, have been conceptualized as
the intermediate outcomes of the CNP. These outcomes, measured within 3
to 5 years of program initiation, would focus on the extent to which
changes have occurred among individuals, the community, and local
policy. When utilizing the concepts of community-based participatory
research, the unit of analysis is often the community itself but that
level of analysis is often not within the scope of the evaluation being
conducted. When individuals within the community are assessed, impact
measures would determine the extent to which local screening rates for
the four major cancers (breast, cervical, prostate, and colorectal) have
increased; whether people have attempted to change their related health
behaviors, such as quitting smoking or using sunscreen; and if local
cancer survivors have more readily utilized clinical trials in their
treatment regimen. At the individual level, the issue of causality is
difficult to address. Oftentimes, depending on the evaluation methods
used, if increases are noted, they cannot be linked directly to the
program under study. This issue can cause difficulty for funding sources
since they generally want to be able to state how their program or funds
made a difference within the local community. For that reason, community
or policy-level changes are often the most feasible to ascertain with an
evaluation design.

For the CNP, community changes would be those resulting differences in
the local norms related to cancer care that are described above. As
people become more knowledgeable about cancer issues, and see their
friends and family change their own behavior, there will be a notable
shift in the community norms within that area. More easily measured
changes within the community would include the extent to which CNP funds
have been used to leverage local resources to address gaps in services
that were beyond the scope of this program. This aspect of the
evaluation would ascertain whether there have been increases in local
resources from other sources and how the partnerships developed through
the CNP have impacted these changes in services. Additional changes
within the community could include providers being more centrally
located and providing more accessible clinic hours, people reporting
higher levels of satisfaction about their interactions with their
providers and perceiving an increased sense of cultural compassion, and
an increased number of minority members either seeking or entering the
health care profession than prior to the CNP initiative.

Policy-level changes are also more readily accessible through an
evaluation design and not as costly to measure as individual-level
changes. As previously described, policy can act as a conduit in support
of individual-level behavior change (see Farquhar, 1978; Advisory Board
International Heart Health Conference, 1992) and similarly as a barrier
to individual-level behavior change. Intermediate outcomes at the policy
level would examine the extent to which policies have been revised,
developed, and/or implemented to support the individual and
community-level efforts. An example of this type of outcome includes
whether a local political leader influenced organizations to decrease
their eligibility levels for services, thereby allowing more people to
have access to cancer care.

**[Long-term and Ultimate Outcomes]{.underline}**

Long-term and ultimate outcomes are generally measured over a period of
time that is beyond the scope of one initiative. Since the CNP funding
would last for 5 years, it is not likely that long-term and ultimate
outcomes can be measured through the national evaluation of the program.
However, in order to conceptualize the overarching goals of the CNP, we
worked with the NCI to specify these outcomes so that they could be
depicted in the conceptual framework (see Exhibit 5).

Long-term outcomes are those that could potentially occur within 5 to 7
years of program implementation. The long-term outcome that CNP hopes to
achieve is to reduce disparities by

 achieving a shift in diagnosis from later to earlier stages;

 improving cancer survivorship; and

 narrowing the gap between the discovery, development, and delivery of
care for disparate populations and other groups.

These particular outcomes relate to ensuring longer survival rates
across all populations. Since the stage of diagnosis among the majority
population tends to be earlier in the process of cancer growth, this
population is more likely to survive cancer. NCI has an overarching
desire to reduce the disparities across cancers so that minority
populations, including those of racial/ethnic minorities, low
socio-economic status, or with minimal access to cancer care such as
rural or inner-city populations, receive the same level of care as
others and therefore find out if they have cancer at earlier stages than
is presently the case. The final long-term outcome of "achieving
sustainability of efforts for disparate groups" relates to Phase III of
the CNP and to the desire to maintain and even improve positive outcomes
over time. All of these outcomes relate directly to the ultimate outcome
to "decrease morbidity and mortality from cancer among disparate
populations to equate rates among other groups."

The remainder of the CNP conceptual framework (see Exhibit 5)
illustrates the need for evaluation to be an integral part of program
implementation with the arrows drawn from each of the columns in the
framework down to the arrow, which is generally thought of as a feedback
loop. A critical role for evaluation research is to provide information
that can be used for ongoing program improvement. As each component of
the CNP is evaluated, the lessons learned should be communicated back to
the Programs so that they can be implemented, where feasible, to improve
local implementation. Each step illustrated in the model, from whether
the Program Building Blocks are in place, to how the synergy has been
facilitated or deterred in the community, what activities or research
have been conducted, and what the outcomes of each have been, should all
be evaluated and findings shared with all key stakeholders. These
stakeholders in turn would help to interpret the findings so that they
can be used to guide ongoing program management and improvements for
every CNP.

### 2.5 Focus the CNP Evaluation (Step 3)

As described in Section 2.2, Step 3 of the CDC Framework (see Exhibit 2)
generally consists of identifying the key measures to be obtained during
the evaluation. During this process, key stakeholders are involved in
prioritizing the evaluation questions that should be answered over the
course of the project. Much of the work to focus the evaluation cannot
be completed until each CNP has been awarded funding. Once it is known
which CNPs will be funded, aspects of their programs can be taken into
consideration to revise the evaluation planning matrix. In addition,
Phase III of the CNP will need to be incorporated into the matrix to
thoroughly provide process and impact outcomes over the 5 years of
funding. The goal in using the evaluation planning matrix is to have a
document that comprehensively addresses all potential evaluation
questions for the program. It is then during the "focusing" that key
stakeholders review that matrix and begin to select those evaluation
measures that must be addressed and those that are not as critical to
meeting the goals of the national evaluation.

Once the stakeholders have determined the priority evaluation measures,
the evaluator would work to develop the data sources necessary to
capture each. At this stage in the development of the evaluation plan,
it is somewhat premature to attempt to identify these data sources and
elements, but we have provided possibilities in the last column of the
evaluation planning matrix (see Appendix A). As described in Section
2.2, our approach to the national evaluation would be to develop data
sources that provide for a triangulation of findings so that one type of
data (e.g., stakeholder interviews) can be used to inform the
development of a second type of data (e.g., a program monitoring
system). These different data sources can be analyzed in order to
combine findings and provide a more comprehensive overview of results
than any one data set can on its own. ***Exhibit 7*** provides an
overview of potential data sources and the types of data elements each
could capture.

As shown in Exhibit 7, a variety of evaluation methods and data
collection strategies could be incorporated into the CNP national
evaluation, once it is clear which questions are a priority to answer.
Working with NCI and the EAC described in Section 2.2, the evaluator
should finalize these methods and the collection of the data elements to
minimize the burden of the local CNPs while still addressing the primary
goals of the national evaluation.

[]{#__RefHeading___Toc88540572 .anchor}Exhibit 7. Possible Data
Collection Methodologies for the CNP

+----------+---------------------+----------------+-------------------+
| P        | Description of      | Who's          | Examples of       |
| otential | Methodology         | Responsible    | Possible Data     |
| Met      |                     | for            | Elements          |
| hodology |                     | Collecting?    |                   |
|          |                     | (Local Program |                   |
|          |                     | or National    |                   |
|          |                     | Evaluator)     |                   |
+==========+=====================+================+===================+
| Record   | Review of all       | Local program  | Composition of    |
| Abs      | written documents   | provides       | local partnership |
| traction | that provide some   | e-mail copies  |                   |
|          | information about   | to national    | Roles and         |
|          | one or more program | evaluator for  | responsibilities  |
|          | components,         | review and     | of CNP staff, CAG |
|          | including minutes   | abstraction    | members, etc.     |
|          | from community      |                |                   |
|          | meetings,           |                | Number of staff   |
|          | organizational      |                | hired for CNP     |
|          | charts, educational |                |                   |
|          | materials           |                | Level of effort   |
|          | developed, media    |                | for local         |
|          | articles on program |                | meetings (e.g.,   |
|          | components, program |                | how many, what is |
|          | implementation      |                | accomplished)     |
|          | and/or evaluation   |                |                   |
|          | plan, etc. While    |                | Quality of CNP    |
|          | reviewing           |                | implementation    |
|          | documents, abstract |                | and/or evaluation |
|          | details that are    |                | plan              |
|          | applicable to       |                |                   |
|          | answering the       |                | Details on        |
|          | evaluation          |                | trainings         |
|          | questions.          |                | conducted (who    |
|          |                     |                | attended, etc.)   |
+----------+---------------------+----------------+-------------------+
| Program  | A Web-based, or     | Local program  | Annual budget and |
| Mo       | other user-friendly | would enter    | allocations to    |
| nitoring | data system, such   | data quarterly | local community   |
| System   | as an Access data   | or monthly     |                   |
|          | file, that is       | into a system  | Progress toward   |
|          | created for the NCI | developed and  | reaching          |
|          | CNP to collect      | managed by     | benchmarks in     |
|          | details on ongoing  | national       | Phases I and II   |
|          | program activities  | evaluator      |                   |
|          |                     |                | Details on        |
|          |                     |                | partners invited  |
|          |                     |                | to participate in |
|          |                     |                | CNP and their     |
|          |                     |                | response          |
|          |                     |                |                   |
|          |                     |                | Types of          |
|          |                     |                | activities        |
|          |                     |                | planned versus    |
|          |                     |                | implemented and   |
|          |                     |                | barr              |
|          |                     |                | iers/facilitators |
|          |                     |                | for completing    |
|          |                     |                | each              |
+----------+---------------------+----------------+-------------------+

(continued)

Exhibit 7. Possible Data Collection Methodologies for the CNP
(continued)

+----------+---------------------+----------------+-------------------+
| P        | Description of      | Who's          | Examples of       |
| otential | Methodology         | Responsible    | Possible Data     |
| Met      |                     | for            | Elements          |
| hodology |                     | Collecting?    |                   |
|          |                     | (Local Program |                   |
|          |                     | or National    |                   |
|          |                     | Evaluator)     |                   |
+==========+=====================+================+===================+
| Case     | An in-depth         | National       | Lessons learned   |
| Study    | community-based     | evaluator      | on each aspect of |
|          | study that collects | would conduct  | program           |
|          | primarily           | annual visits  | development       |
|          | qualitative data    | to selected    |                   |
|          | (e.g., through      | sites, with    | Barriers and      |
|          | face-to-face        | local programs | facilitators to   |
|          | interviews, focus   | participating  | completion of     |
|          | groups) from        | in recruitment | program           |
|          | management,         | for interviews | objectives        |
|          | partners,           | and focus      |                   |
|          | recipients, and     | groups         | Quality of local  |
|          | other key           |                | partnerships      |
|          | stakeholders to     |                |                   |
|          | assess lessons      |                | Extent to which   |
|          | learned on critical |                | CNP has conducted |
|          | aspects of program  |                | community-based   |
|          | development         |                | participatory     |
|          |                     |                | research          |
|          |                     |                |                   |
|          |                     |                | Key stakeholder   |
|          |                     |                | perceptions of    |
|          |                     |                | program successes |
+----------+---------------------+----------------+-------------------+
| Survey   | Standardized        | Developed and  | Data elements to  |
| ins      | instruments         | administered   | be captured would |
| truments | developed           | by the         | depend on the     |
|          | specifically for    | national       | evaluation        |
|          | CNP to capture an   | evaluator with | question to be    |
|          | important aspect of | guidance from  | addressed. The    |
|          | the program. An     | the EAC        | method and data   |
|          | example would be to |                | collection        |
|          | administer an       |                | strategy would    |
|          | instrument to local |                | need to be        |
|          | partners that       |                | developed for     |
|          | measures their      |                | each topic to be  |
|          | satisfaction with   |                | studied.          |
|          | the CNP, their      |                |                   |
|          | level of            |                |                   |
|          | involvement, and    |                |                   |
|          | the degree to which |                |                   |
|          | synergy has been    |                |                   |
|          | created             |                |                   |
+----------+---------------------+----------------+-------------------+

### 

### 2.6 Next Steps

As previously described, the work completed to date only focuses on
Phases I and II of the CNP. The evaluation plan presented here is
comprehensive in scope and will need to be refined once the CNPs have
received their awards. At that point, three major steps will need to be
taken by the national evaluator. First, the evaluator will need to meet
with key stakeholders of the national evaluation plan and review the
draft CNP conceptual framework. Using input from these stakeholders,
perhaps through the development of an EAC, the framework can then be
refined and distributed for additional review by community partners
working with each of the local CNPs. The evaluator should plan to work
with NCI to determine the extent of review and input desired from the
various groups of stakeholders who will be interested in the outcomes
and then establish a process to accommodate their involvement.

Second, as the framework is being revised, the evaluator should revise
the evaluation planning matrix (see Appendix A) to incorporate Phase III
into the eventual plan. During this process, it will be important for
NCI and the EAC (or similar entity) to state the overall goals of the
evaluation. If the EAC determines that the focus of the evaluation
should be on measuring health outcomes, for example, this will have
great implications on what types of methodologies and data collection
strategies should be used in the national evaluation. A process similar
to the one followed to date could be used to engage NCI stakeholders in
first specifying what they see as the outcomes of each objective and
then working to provide overarching goals or purposes for the national
evaluation. This input could be incorporated into a revised evaluation
planning matrix and comprehensive plan for data collection and analysis.
The complete matrix could then be shared with the local CNP for review
and comment on the priorities to address through the evaluation plan and
their role in ongoing data collection and analysis.

Third, once the evaluation planning matrix is revised and close to
finalized, work can then be done to identify the specific data sources
to be used in the national evaluation. We have begun to identify data
sources and methods for Phases I and II in this report (see Exhibit 7)
but, as noted, until the overarching purposes and goals of the national
evaluation are agreed upon, these aspects of the plan cannot be
finalized. Therefore, the evaluator will need to work with NCI and
others to prioritize the evaluation measures of greatest concern and
identify the most cost efficient and feasible data sources to develop
and implement to comprehensively assess CNP achievements.

## References

Advisory Board International Heart Health Conference. 1992. *The
Victoria Declaration on Heart Health.* Victoria, British Columbia,
Canada: British Columbia Ministry of Health.

American Cancer Society (ACS). 2003. *Cancer Facts & Figures 2003*.
Atlanta, GA: American Cancer Society.

Aziz, N.M., and J.H. Rowland. 2003. "Trends and Advances in Cancer
Survivorship Research: Challenge and Opportunity." *Seminars in
Radiation Oncology* 13:248-266.

Bandura, A. 2001. "Social Cognitive Theory: An Agentic Perspective."
*Annual Review of Psychology* 52:1-26.

Boyce, W.F. 2001. "Disadvantaged Persons' Participation in Health
Promotion Projects: Some Structural Dimensions." *Social Science &
Medicine* 52(10):1551-1564.

Brinkerhoff, J.M. 2002. "Assessing and Improving Partnership
Relationships and Outcomes: A Proposed Framework." *Evaluation and
Program Planning* 25:215-231.

Bronfenbrenner, U. 1979. *The Ecology of Human Development*. Cambridge,
MA: Harvard University Press.

Bronfenbrenner, U. 1990. *The Ecology of Human Development: Experiments
by Nature and Design*. Cambridge, MA: Harvard University Press.

Butterfoss, F.D., R.M. Goodman, and A. Wanderman. 1993. "Community
Coalitions for Prevention and Health Promotion." *Health Education
Research* 8:315-330.

Centers for Disease Control and Prevention. 1999. "Framework for Program
Evaluation in Public Health." *Morbidity and Mortality Weekly Report*
48(RR11):1-40.

Dalton, J.H., M.J. Elias, and A. Wandersman. 2001. *Community
Psychology: Linking Individuals and Communities*, pp. 60-86. Stamford,
CT: Thomson Learning.

Farquhar, J.W. 1978. *The American Way of Life Need Not Be Hazardous to
Your Health*. New York, NY: WW Norton & Co.

Florin, P., R. Mitchell, and J. Stevenson. 1993. "Identifying Training
and Technical Assistance Needs in Coalitions: A Developmental Approach."
*Health Education Research Theory and Practice* 8:417-432.

Gergen, K.J., and M.M. Gergen. 1991. "From Theory to Reflexivity in
Research Practice." In *Research and Reflexivity*, F. Steier, ed.
Newbury Park: Sage Publications.

Gielen, A.C., and E.M. McDonald. 1997. "Chapter 17: The Procede-Proceed
Planning Model." In *Health Behavior and Health Education: Theory,
Research, and Practice*, 2nd edition, K. Glanz, F.M. Lewis, and B.
Rimer, eds., pp. 359-383. San Francisco, CA: Jossey-Bass Publications.

Glanz, K., F.M. Lewis, and B.K. Rimer. 1997. *Health Behavior and Health
Education: Theory, Research, and Practice,* 2nd edition. San Francisco,
CA: Jossey-Bass.

Green, L.W., and M.W. Krueter. 1991. "Chapter 5: Educational and
Organizational Diagnosis: Factors Affecting Health-Related Behavior and
Environments." In *Health Promotion Planning*, 2nd edition. Mountain
View: Mayfield Publishing Company.

Green, L.W., M.A. George, M. Daniel, C.J. Frankish, C.P. Herber, W.R.
Bowie, and M. O'Neill. 1995. "Study of Participatory Research in Health
Promotion: Review and Recommendations for the Development of
Participatory Research in Health Promotion in Canada." Vancouver,
British Columbia: Royal Society of Canada.

Greene, J.C. 2000. "Challenges in Practicing Deliberative Democratic
Evaluation." *New Directions from Evaluation*, 85:13-26.

Hays, C.E., S.P. Hays, J.O. DeVille, and P.F. Mulhall. 2000. "Capacity
for Effectiveness: The Relationship between Coalition Structure and
Community Impact." *Evaluation and Program Planning* 23(3):373-379.

Himmelman, A.T. 2001. "On Coalitions and the Transformation of Power
Relations: Collaborative Betterment and Collaborative Empowerment."
*American Journal of Community Psychology* 29(2):277-284.

Israel, B.A., A.J. Schulz, E.A. Parker, and A.B. Becker. 1998. "Review
of Community-Based Research: Assessing Partnership Approaches to Improve
Public Health." *Annual Review of Public Health* 19:173-202.

Israel, B.A., A.J. Schulz, E.A. Parker, A.B. Becker, A.J. Allen, and
J.R. Guzman. 2003. "Critical Issues in Developing and Following
Community Based Participatory Research Principles." In *Community Based
Participatory Research for Health*, M. Minkler and N. Wallerstein, eds.,
pp. 53-76. California: Jossey-Bass.

Lasker, R.D., and E.S. Weiss. 2003. "Broadening Participation in
Community Problem Solving: A Multidisciplinary Model to Support
Collaborative Practice and Research." *Journal of Urban Health: Bulletin
of the New York Academy of Medicine* 80(1):14-60.

Mathie, A., and J. Greene. 1997. "Stakeholder Participation in
Evaluation: How Important is Diversity?" *Evaluation & Program Planning*
20(3):279-85.

McLeroy, K.R., D. Bibeau, A. Steckler, and K. Glanz. 1988. "An
Ecological Perspective on Health Promotion Programs." *Health Education
Quarterly* 15(4):351-377.

Morgan, G. 1983. "Research Strategies: Modes of Engagement." In Beyond
Method: Strategies for Social Research, pp. 11-22. Beverly Hills: Sage
Publications.

National Institute of Health, National Cancer Institute (NIH/NCI). 2001.
Voices of a Broken System: Real People, Real Problems. President's
Cancer Panel. Report of the Chairman, 2000-2001. Bethesda, MD: National
Institute of Health, National Cancer Institute.
\<http://156.40.135.142:8080/webisodes/pcpvideo/voices_files/PDFfiles/
PCPbook.pdf\>.

Nichols, L. 2002. "Participatory Program Planning Including Program
Participants and Evaluators." *Evaluation & Program Planning* 25:1-14.

Papineau, D., and M.C. Kiely. 1996. "Participatory Evaluation in a
Community Organization: Fostering Stakeholder Empowerment and
Utilization." *Evaluation & Program Planning* 19(1):79-93.

Patton, M.A. 1990. *Qualitative Evaluation and Research Methods*.
California: Sage Publications.

Patton, M.Q. 1997. *Utilization-Focused Evaluation: The New Century
Text* (Edition 3). California: Sage Publications.

Pervin, L.A. 1968. "Performance and Satisfaction as a Function of
Individual-Environment Fit." *Psychological Bulletin* 69:56-68.

Reason, P. 1988. *Human Inquiry in Action*. Newbury Park: Sage
Publications.

Roussos, S.T., and S.B. Fawcett. 2000. "A Review of Collaborative
Partnerships as a Strategy for Improving Community Health." *Annual
Review of Public Health* 21:369‑402.

Schulz, A.J., B.A. Israel, and P. Lantz. 2003. "Instrument for
Evaluating Dimensions of Group Dynamics within Community-Based
Participatory Research Partnerships." *Evaluation and Program Planning*
26(3):249-262.

Schulz, A.J., B.A. Israel, E.A. Parker, M. Lockett, Y.R. Hill, and R.
Wills. 2003. "Engaging Women in Community Participatory Research for
Health: The East Side Village Health Worker Partnership." In *Community
Based Participatory Research for Health*, M. Minkler and N. Wallerstein,
eds., pp. 293-315. California: Jossey-Bass.

Schulz, A.J., B.A. Israel, S.M. Selig, and I.S. Bayer. 1998.
"Development and Implementation of Principles for Community-Based
Research in Public Health." In *Research Strategies for Community
Practice*, R.H. MacNair, ed., pp. 83-110. NY: Haworth Press.

Small, S.A. 1995. "Action-Oriented Research: Models and Methods."
*Journal of Marriage and the Family* 57:941-955.

Springett, J. 2003. "Issues in Participatory Evaluation." In *Community
Based Participatory Research for Health*, M. Minkler and N. Wallerstein,
eds., pp. 263-287. California: Jossey-Bass.

U.S. Department of Health and Human Services (USDHHS), Public Health
Service, National Cancer Institute. 125th National Cancer Advisory
Board, Summary of Meeting, February 11-12, 2003. Bethesda, MD: National
Institutes of Health.
\<http://deainfo.nci.nih.gov/ADVISORY/ncab/125_0203/11feb03mins.pdf\>.

U.S. Department of Health and Human Services (USDHHS). March 2004.
Making Cancer Health Disparities History. Report of the Trans-HHS Cancer
Health Disparities Progress Review Group.
\<http://www.chdprg.omhrc.gov/pdf/chdprg.pdf\>.

Wallerstein, N. 1999. "Power Between Evaluator and Community: Research
Relationships within New Mexico's Healthier Communities." *Social
Science & Medicine* 49(1):39-53.

Wandersman, A. 2003. "Community Science: Bridging the Gap between
Science and Practice with Community-Centered Models." *American Journal
of Community Psychology* 31(3/4):227-242.

# Appendix A: National Cancer Institute's Community Networks Program Evaluation Matrix

National Cancer Institute's Community Networks Program Evaluation Matrix

+--------+----------------+---------+------------+---------+---------+
| *      |                |         |            |         |         |
| *Phase |                |         |            |         |         |
| I      |                |         |            |         |         |
| Goal:  |                |         |            |         |         |
| D      |                |         |            |         |         |
| evelop |                |         |            |         |         |
| and    |                |         |            |         |         |
| in     |                |         |            |         |         |
| crease |                |         |            |         |         |
| ca     |                |         |            |         |         |
| pacity |                |         |            |         |         |
| bu     |                |         |            |         |         |
| ilding |                |         |            |         |         |
| to     |                |         |            |         |         |
| s      |                |         |            |         |         |
| upport |                |         |            |         |         |
| com    |                |         |            |         |         |
| munity |                |         |            |         |         |
| -based |                |         |            |         |         |
| p      |                |         |            |         |         |
| artici |                |         |            |         |         |
| patory |                |         |            |         |         |
| educ   |                |         |            |         |         |
| ation, |                |         |            |         |         |
| res    |                |         |            |         |         |
| earch, |                |         |            |         |         |
| and    |                |         |            |         |         |
| tr     |                |         |            |         |         |
| aining |                |         |            |         |         |
| to     |                |         |            |         |         |
| reduce |                |         |            |         |         |
| cancer |                |         |            |         |         |
| dispa  |                |         |            |         |         |
| rities |                |         |            |         |         |
| (Years |                |         |            |         |         |
| 1      |                |         |            |         |         |
| --5)** |                |         |            |         |         |
+========+================+=========+============+=========+=========+
| **     | **Activities** | **Outp  | **Process  | *       | **P     |
| Object |                | uts/Pro | Outcomes** | *Impact | ossible |
| ives** | *(Organize by  | ducts** |            | Out     | Data    |
|        | perspective:   |         | *          | comes** | So      |
|        | CNP Project    |         | (Formative |         | urces** |
|        | Activities,    |         | measures   | *(M     |         |
|        | CNP Evaluation |         | that       | easures |         |
|        | Team, NCI-CNP  |         | assess how | the     |         |
|        | Program        |         | well       | extent  |         |
|        | Director)*     |         | activities | to      |         |
|        |                |         | are being  | which   |         |
|        |                |         | acco       | the     |         |
|        |                |         | mplished)* | act     |         |
|        |                |         |            | ivities |         |
|        |                |         |            | have    |         |
|        |                |         |            | r       |         |
|        |                |         |            | esulted |         |
|        |                |         |            | in      |         |
|        |                |         |            | system, |         |
|        |                |         |            | indi    |         |
|        |                |         |            | vidual, |         |
|        |                |         |            | or      |         |
|        |                |         |            | policy  |         |
|        |                |         |            | c       |         |
|        |                |         |            | hange)* |         |
+--------+----------------+---------+------------+---------+---------+
| **I.1  | a.  **Hire and | Organiz | Leadership | **Met   | Prior   |
| D      |     maintain   | ational | style of   | organiz | funding |
| evelop |     required   | chart   | Principal  | ational | history |
| core   |     staff:**   |         | In         | infrast | (       |
| or     |                | Impleme | vestigator | ructure | initial |
| ganiza | -   Principal  | ntation |            | requ    | and     |
| tional |                | plan    | **Staff of | irement | conti   |
| in     |   Investigator |         | multidi    | to      | nuation |
| frastr |     (10%)      | CNP     | sciplinary | tra     | pro     |
| ucture |                | d       | pro        | nsition | posals) |
| for    | -   Project    | atabase | fessionals | to      |         |
| local  |     Manager    |         | to         | Phase   | CNP     |
| Com    |     (100%)     | *       | implement  | II (CP: | grant   |
| munity |                | *Annual | program    | 12)     | appli   |
| N      | -   2--3 Staff | p       | hired and  | (RFA:   | cations |
| etwork |     (100%)     | rogress | maintained | 6,      |         |
| P      |                | re      | as         | 14)**   | Impleme |
| rogram | -   Research   | ports** | required   |         | ntation |
| (C     |                |         | (CP: 9,    |         | plans   |
| NP).** |    Coordinator | **Add   | 12)**      |         |         |
|        |     (20%)      | itional |            |         | Annual  |
|        |                | reports | **CNP      |         | p       |
|        | -   Policy     | as      | staffing   |         | rogress |
|        |     Analyst    | sp      | (% time    |         | reports |
|        |     (20%)      | ecified | and        |         | (RFA:   |
|        |                | by NCI  | position)  |         | 9)      |
|        | -   Program    | Program | (CNP:      |         |         |
|        |     Evaluator  | Dir     | 13)**      |         | Grant   |
|        |     (20%)      | ector** |            |         | budget  |
|        |                |         | **C        |         | and     |
|        | b.  **Develop  | **      | omposition |         | expen   |
|        |     required   | Written | of Project |         | ditures |
|        |     advisory   | communi | Steering   |         |         |
|        |     bodies:**  | cations | Committee  |         | Sem     |
|        |                | (e.g.,  | met**      |         | iannual |
|        | -   Project    | e-mail, |            |         | meeting |
|        |     Steering   | memos,  | **Size and |         | with    |
|        |     Committee  | minutes | represen   |         | CRCHD   |
|        |                | of      | tativeness |         | D       |
|        | -   Regional   | meet    | of the     |         | irector |
|        |     Advisory   | ings)** | Commu      |         | agenda  |
|        |     Committee  |         | nity-based |         | and     |
|        |                |         | Advisory   |         | minutes |
|        | -   C          |         | Group      |         |         |
|        | ommunity-based |         | (role and  |         | Summit  |
|        |     Advisory   |         | respons    |         | Con     |
|        |     Group      |         | ibility)** |         | ference |
|        |                |         |            |         | proc    |
|        | c.  **         |         | **Number   |         | eedings |
|        | Operationalize |         | and        |         |         |
|        |     roles to   |         | frequency  |         | Program |
|        |     orient and |         | of         |         | Mon     |
|        |     train      |         | meetings   |         | itoring |
|        |     community  |         | (RFA: 9)** |         | Sys     |
|        |                |         |            |         | tem/CNP |
|        |    partnership |         | **CNP      |         | D       |
|        |     members.** |         | office     |         | atabase |
|        |                |         | space**    |         | (con    |
|        | d.  Develop an |         | (square    |         | sisting |
|        |                |         | footage    |         | of data |
|        | Implementation |         | per staff) |         | e       |
|        |     Plan for   |         | **(CNP:    |         | lements |
|        |     the        |         | 13)**      |         | as      |
|        |     Project.   |         |            |         | agreed  |
|        |                |         | **Ins      |         | upon by |
|        | e.  Provide    |         | titutional |         | NCI     |
|        |     needed     |         | resources  |         | Program |
|        |     technical  |         | provided   |         | Di      |
|        |     assistance |         | (e.g.,     |         | rector, |
|        |     (e.g.,     |         | lower      |         | CNPs,   |
|        |     logic      |         | indirect   |         | and the |
|        |     models,    |         | cost,      |         | Eva     |
|        |     grant      |         | waiving    |         | luator) |
|        |     writing,   |         | indirect   |         |         |
|        |                |         | cost for   |         |         |
|        |    evaluation) |         | pilots)**  |         |         |
|        |     for local  |         |            |         |         |
|        |     Project    |         | **New      |         |         |
|        |     staff.     |         | orga       |         |         |
|        |                |         | nizational |         |         |
|        | f              |         | structures |         |         |
|        | .  Collaborate |         | are        |         |         |
|        |     with NCI   |         | e          |         |         |
|        |     Project    |         | stablished |         |         |
|        |     Director   |         | (CP: 5)**  |         |         |
|        |     and        |         |            |         |         |
|        |     National   |         |            |         |         |
|        |     Evaluator  |         |            |         |         |
|        |     to develop |         |            |         |         |
|        |     **"core    |         |            |         |         |
|        |     program    |         |            |         |         |
|        |     evaluation |         |            |         |         |
|        |     data       |         |            |         |         |
|        |     elements"\ |         |            |         |         |
|        |     (RFA:      |         |            |         |         |
|        |     9)**.      |         |            |         |         |
+--------+----------------+---------+------------+---------+---------+

(continued)

National Cancer Institute's Community Networks Program Evaluation Matrix
(continued)

+--------+----------------+---------+------------+---------+---------+
| *      |                |         |            |         |         |
| *Phase |                |         |            |         |         |
| I      |                |         |            |         |         |
| Goal:  |                |         |            |         |         |
| D      |                |         |            |         |         |
| evelop |                |         |            |         |         |
| and    |                |         |            |         |         |
| in     |                |         |            |         |         |
| crease |                |         |            |         |         |
| ca     |                |         |            |         |         |
| pacity |                |         |            |         |         |
| bu     |                |         |            |         |         |
| ilding |                |         |            |         |         |
| to     |                |         |            |         |         |
| s      |                |         |            |         |         |
| upport |                |         |            |         |         |
| com    |                |         |            |         |         |
| munity |                |         |            |         |         |
| -based |                |         |            |         |         |
| p      |                |         |            |         |         |
| artici |                |         |            |         |         |
| patory |                |         |            |         |         |
| educ   |                |         |            |         |         |
| ation, |                |         |            |         |         |
| res    |                |         |            |         |         |
| earch, |                |         |            |         |         |
| and    |                |         |            |         |         |
| tr     |                |         |            |         |         |
| aining |                |         |            |         |         |
| to     |                |         |            |         |         |
| reduce |                |         |            |         |         |
| cancer |                |         |            |         |         |
| dispa  |                |         |            |         |         |
| rities |                |         |            |         |         |
| (Years |                |         |            |         |         |
| 1      |                |         |            |         |         |
| --5)** |                |         |            |         |         |
+========+================+=========+============+=========+=========+
| **     | **Activities** | **Outp  | **Process  | *       | **P     |
| Object |                | uts/Pro | Outcomes** | *Impact | ossible |
| ives** | *(Organize by  | ducts** |            | Out     | Data    |
|        | perspective:   |         | *          | comes** | So      |
|        | CNP Project    |         | (Formative |         | urces** |
|        | Activities,    |         | measures   | *(M     |         |
|        | CNP Evaluation |         | that       | easures |         |
|        | Team, NCI-CNP  |         | assess how | the     |         |
|        | Program        |         | well       | extent  |         |
|        | Director)*     |         | activities | to      |         |
|        |                |         | are being  | which   |         |
|        |                |         | acco       | the     |         |
|        |                |         | mplished)* | act     |         |
|        |                |         |            | ivities |         |
|        |                |         |            | have    |         |
|        |                |         |            | r       |         |
|        |                |         |            | esulted |         |
|        |                |         |            | in      |         |
|        |                |         |            | system, |         |
|        |                |         |            | indi    |         |
|        |                |         |            | vidual, |         |
|        |                |         |            | or      |         |
|        |                |         |            | policy  |         |
|        |                |         |            | c       |         |
|        |                |         |            | hange)* |         |
+--------+----------------+---------+------------+---------+---------+
|        | g              |         | **Ability  |         | **C     |
|        | .  Communicate |         | to develop |         | riteria |
|        |     progress   |         | infr       |         | for     |
|        |     toward     |         | astructure |         | dete    |
|        |                |         | (CP: 3)**  |         | rmining |
|        |    benchmarks. |         |            |         | "q      |
|        |                |         | **Number,  |         | uality" |
|        | h.  Attend     |         | role, and  |         | of the  |
|        |     regular    |         | chara      |         | Impleme |
|        |     Project    |         | cteristics |         | ntation |
|        |     meetings   |         | of         |         | Plan**  |
|        |     convened   |         | i          |         |         |
|        |     by NCI:    |         | ndividuals |         |         |
|        |                |         | sponsored  |         |         |
|        | -   Annual     |         | for Annual |         |         |
|        |     Cancer     |         | Cancer     |         |         |
|        |                |         | D          |         |         |
|        |    Disparities |         | isparities |         |         |
|        |     Summit (up |         | Summit     |         |         |
|        |     to 8       |         | Conference |         |         |
|        |     conference |         | (CP: 12)** |         |         |
|        |                |         |            |         |         |
|        |   participants |         |            |         |         |
|        |     per CNP)   |         |            |         |         |
|        |                |         |            |         |         |
|        | -   PI         |         |            |         |         |
|        |     meetings   |         |            |         |         |
|        |     with CRCHD |         |            |         |         |
|        |     Director   |         |            |         |         |
|        |                |         |            |         |         |
|        |   (semiannual) |         |            |         |         |
+--------+----------------+---------+------------+---------+---------+
| **I.2  | a.  Assess     | **      | Number and | **Met   | Survey  |
| D      |     existing   | Listing | type of    | part    | of key  |
| evelop |     cancer     | of      | **MOAs     | nership | leaders |
| partne |     control    | cancer  | with roles | requ    | of the  |
| rships |     resources  | control | and        | irement | organi  |
| with   |     for        | pr      | respon     | to move | zations |
| commu  |     community  | ograms, | sibilities | to      | of      |
| nities |     champions  | partne  | specified  | Phase   | i       |
| experi |     (i.e.,     | rships, | (RFA: 3)** | II**    | nterest |
| encing |                | and     |            |         |         |
| cancer |    survivors), | pr      | Number and | **In    | Program |
| dispa  |     such as:   | oviders | type of    | creased | Mon     |
| rities |                | for     | p          | l       | itoring |
| and    | -   Lay health | each    | artnership | inkages | Sys     |
| o      |     workers,   | focus   | activities | between | tem/CNP |
| rganiz |     c          | area    | (level of  | pa      | D       |
| ations | ommunity-level | (e.g.,  | in         | rtners, | atabase |
| that   |                | primary | volvement) | in      |         |
| can    |  organizations | and     |            | cluding | Part    |
| aid in |     (e.g.,     | se      | **Total    | i       | nership |
| re     |     health     | condary | number of  | mproved | and     |
| ducing |     services,  | prev    | cancer     | communi | MOAs at |
| cancer |     education  | ention, | control    | cation, | b       |
| di     |     outreach   | di      | a          | coordi  | aseline |
| sparit |     groups,    | agnosis | ctivities\ | nation, | and     |
| ies.** |                | and     | (CP: 13)** | netw    | ben     |
|        |    faith-based | tre     |            | orking, | chmarks |
|        |     groups,    | atment, | **Total    | etc.**  |         |
|        |                | surviv  | number of  |         | Gr      |
|        |    coalitions) | orship, | int        | *       | antees' |
|        |     (CP: 9)    | po      | erventions | *Effect | Phase   |
|        |                | licy)** | developed  | iveness | II      |
|        | -   Primary    |         | (CP: 13)** | of      | Impleme |
|        |     prevention | *       |            | cancer  | ntation |
|        |                | *Matrix | **Quality  | aw      | Plan    |
|        |    *providers* | of      | of         | areness |         |
|        |     (centers)  | co      | pa         | act     | Co      |
|        |     and        | mmunity | rtnerships | ivities | mmunity |
|        |     programs   | p       | with       | (CP:    | c       |
|        |     (e.g.,     | artners | co         | 12)**   | apacity |
|        |     smoking    | by      | mmunities\ |         | scale   |
|        |     cessation  | focus   | (CP: 3)**  | **In    |         |
|        |     and        | areas** |            | creased |         |
|        |     hepatitis  |         | **CNP      | leve    |         |
|        |                | **Me    | budget     | raging/ |         |
|        |    vaccination | moranda | allocation | sharing |         |
|        |     programs)  | of      | provided   | of      |         |
|        |     (CP:9),    | Unders  | directly   | funds   |         |
|        |     cessation  | tanding | to         | among   |         |
|        |     and        | (MOU)** | c          | i       |         |
|        |     hepatitis  |         | ommunity** | nvolved |         |
|        |                |         |            | o       |         |
|        |    vaccination |         |            | rganiza |         |
|        |     programs   |         |            | tions** |         |
|        |     (CP: 9)    |         |            |         |         |
+--------+----------------+---------+------------+---------+---------+

(continued)

National Cancer Institute's Community Networks Program Evaluation Matrix
(continued)

+--------+----------------+---------+------------+---------+---------+
| *      |                |         |            |         |         |
| *Phase |                |         |            |         |         |
| I      |                |         |            |         |         |
| Goal:  |                |         |            |         |         |
| D      |                |         |            |         |         |
| evelop |                |         |            |         |         |
| and    |                |         |            |         |         |
| in     |                |         |            |         |         |
| crease |                |         |            |         |         |
| ca     |                |         |            |         |         |
| pacity |                |         |            |         |         |
| bu     |                |         |            |         |         |
| ilding |                |         |            |         |         |
| to     |                |         |            |         |         |
| s      |                |         |            |         |         |
| upport |                |         |            |         |         |
| com    |                |         |            |         |         |
| munity |                |         |            |         |         |
| -based |                |         |            |         |         |
| p      |                |         |            |         |         |
| artici |                |         |            |         |         |
| patory |                |         |            |         |         |
| educ   |                |         |            |         |         |
| ation, |                |         |            |         |         |
| res    |                |         |            |         |         |
| earch, |                |         |            |         |         |
| and    |                |         |            |         |         |
| tr     |                |         |            |         |         |
| aining |                |         |            |         |         |
| to     |                |         |            |         |         |
| reduce |                |         |            |         |         |
| cancer |                |         |            |         |         |
| dispa  |                |         |            |         |         |
| rities |                |         |            |         |         |
| (Years |                |         |            |         |         |
| 1      |                |         |            |         |         |
| --5)** |                |         |            |         |         |
+========+================+=========+============+=========+=========+
| **     | **Activities** | **Outp  | **Process  | *       | **P     |
| Object |                | uts/Pro | Outcomes** | *Impact | ossible |
| ives** | *(Organize by  | ducts** |            | Out     | Data    |
|        | perspective:   |         | *          | comes** | So      |
|        | CNP Project    |         | (Formative |         | urces** |
|        | Activities,    |         | measures   | *(M     |         |
|        | CNP Evaluation |         | that       | easures |         |
|        | Team, NCI-CNP  |         | assess how | the     |         |
|        | Program        |         | well       | extent  |         |
|        | Director)*     |         | activities | to      |         |
|        |                |         | are being  | which   |         |
|        |                |         | acco       | the     |         |
|        |                |         | mplished)* | act     |         |
|        |                |         |            | ivities |         |
|        |                |         |            | have    |         |
|        |                |         |            | r       |         |
|        |                |         |            | esulted |         |
|        |                |         |            | in      |         |
|        |                |         |            | system, |         |
|        |                |         |            | indi    |         |
|        |                |         |            | vidual, |         |
|        |                |         |            | or      |         |
|        |                |         |            | policy  |         |
|        |                |         |            | c       |         |
|        |                |         |            | hange)* |         |
+--------+----------------+---------+------------+---------+---------+
| **I.2  | -   Secondary  | **Me    |            |         | In      |
| D      |     prevention | moranda |            |         | ventory |
| evelop |     programs   | of      |            |         | of      |
| partne |     (s         | Ag      |            |         | e       |
| rships | creening/early | reement |            |         | xisting |
| with   |     detection  | (MOA)** |            |         | cancer  |
| commu  |                |         |            |         | pr      |
| nities |    *providers* |         |            |         | ograms, |
| experi |                |         |            |         | pr      |
| encing |    \[centers\] |         |            |         | ojects, |
| cancer |     and        |         |            |         | pr      |
| dispa  |     programs)  |         |            |         | oviders |
| rities |     (CP: 9,    |         |            |         | in the  |
| and    |     10),       |         |            |         | ide     |
| o      |     NCI-funded |         |            |         | ntified |
| rganiz |     research   |         |            |         | comm    |
| ations |                |         |            |         | unities |
| that   |   institutions |         |            |         |         |
| can    |     (CP: 9),   |         |            |         | Co      |
| aid in |     cancer     |         |            |         | mmittee |
| re     |     research   |         |            |         | and     |
| ducing |                |         |            |         | group   |
| cancer |   organization |         |            |         | minutes |
| di     |     and/or     |         |            |         | and     |
| sparit |     cancer     |         |            |         | att     |
| ies.** |     treatment  |         |            |         | endance |
|        |     facilities |         |            |         | rosters |
|        |     (e.g.,     |         |            |         |         |
|        |     community  |         |            |         |         |
|        |     hospitals) |         |            |         |         |
|        |     (CP: 9),   |         |            |         |         |
|        |     diagnosis  |         |            |         |         |
|        |     and        |         |            |         |         |
|        |     treatment  |         |            |         |         |
|        |     facilities |         |            |         |         |
|        |                |         |            |         |         |
|        |   (providers). |         |            |         |         |
|        |     Phase II   |         |            |         |         |
|        |     (CP: 12)   |         |            |         |         |
|        |     (RFA: 6),  |         |            |         |         |
|        |     national   |         |            |         |         |
|        |     cancer     |         |            |         |         |
|        |     prevention |         |            |         |         |
|        |     and        |         |            |         |         |
|        |     control    |         |            |         |         |
|        |                |         |            |         |         |
|        |  organizations |         |            |         |         |
|        |     (e.g.,     |         |            |         |         |
|        |     ACS);      |         |            |         |         |
|        |     other      |         |            |         |         |
|        |                |         |            |         |         |
|        |   governmental |         |            |         |         |
|        |     agencies   |         |            |         |         |
|        |     (e.g.,     |         |            |         |         |
|        |     CDC, HRSA, |         |            |         |         |
|        |     CMMS) (14) |         |            |         |         |
|        |                |         |            |         |         |
|        | b.  Determine  |         |            |         |         |
|        |     gaps and   |         |            |         |         |
|        |     needs.     |         |            |         |         |
|        |                |         |            |         |         |
|        | c.  Develop    |         |            |         |         |
|        |     and        |         |            |         |         |
|        |     maintain   |         |            |         |         |
|        |                |         |            |         |         |
|        |  partnerships. |         |            |         |         |
|        |                |         |            |         |         |
|        | d.  Implement  |         |            |         |         |
|        |     activities |         |            |         |         |
|        |     and        |         |            |         |         |
|        |                |         |            |         |         |
|        | interventions. |         |            |         |         |
+--------+----------------+---------+------------+---------+---------+

(continued)

National Cancer Institute's Community Networks Program Evaluation Matrix
(continued)

+--------+----------------+---------+------------+---------+---------+
| *      |                |         |            |         |         |
| *Phase |                |         |            |         |         |
| I      |                |         |            |         |         |
| Goal:  |                |         |            |         |         |
| D      |                |         |            |         |         |
| evelop |                |         |            |         |         |
| and    |                |         |            |         |         |
| in     |                |         |            |         |         |
| crease |                |         |            |         |         |
| ca     |                |         |            |         |         |
| pacity |                |         |            |         |         |
| bu     |                |         |            |         |         |
| ilding |                |         |            |         |         |
| to     |                |         |            |         |         |
| s      |                |         |            |         |         |
| upport |                |         |            |         |         |
| com    |                |         |            |         |         |
| munity |                |         |            |         |         |
| -based |                |         |            |         |         |
| p      |                |         |            |         |         |
| artici |                |         |            |         |         |
| patory |                |         |            |         |         |
| educ   |                |         |            |         |         |
| ation, |                |         |            |         |         |
| res    |                |         |            |         |         |
| earch, |                |         |            |         |         |
| and    |                |         |            |         |         |
| tr     |                |         |            |         |         |
| aining |                |         |            |         |         |
| to     |                |         |            |         |         |
| reduce |                |         |            |         |         |
| cancer |                |         |            |         |         |
| dispa  |                |         |            |         |         |
| rities |                |         |            |         |         |
| (Years |                |         |            |         |         |
| 1      |                |         |            |         |         |
| --5)** |                |         |            |         |         |
+========+================+=========+============+=========+=========+
| **     | **Activities** | **Outp  | **Process  | *       | **P     |
| Object |                | uts/Pro | Outcomes** | *Impact | ossible |
| ives** | *(Organize by  | ducts** |            | Out     | Data    |
|        | perspective:   |         | *          | comes** | So      |
|        | CNP Project    |         | (Formative |         | urces** |
|        | Activities,    |         | measures   | *(M     |         |
|        | CNP Evaluation |         | that       | easures |         |
|        | Team, NCI-CNP  |         | assess how | the     |         |
|        | Program        |         | well       | extent  |         |
|        | Director)*     |         | activities | to      |         |
|        |                |         | are being  | which   |         |
|        |                |         | acco       | the     |         |
|        |                |         | mplished)* | act     |         |
|        |                |         |            | ivities |         |
|        |                |         |            | have    |         |
|        |                |         |            | r       |         |
|        |                |         |            | esulted |         |
|        |                |         |            | in      |         |
|        |                |         |            | system, |         |
|        |                |         |            | indi    |         |
|        |                |         |            | vidual, |         |
|        |                |         |            | or      |         |
|        |                |         |            | policy  |         |
|        |                |         |            | c       |         |
|        |                |         |            | hange)* |         |
+--------+----------------+---------+------------+---------+---------+
| **I.3  | **a. Establish | F       | \# and     | I       |         |
| Have   | projects with  | indings | type of    | ncrease |         |
| at     | different      | from    | activities | in      |         |
| least  | Office,        | eva     | with NCI   | collabo |         |
| four   | Centers,       | luation | groups     | rations |         |
| co     | Divisions of   | of each |            | among   |         |
| llabor | NCI.**         | project | Perception | NCI     |         |
| ations |                |         | among NCI  | e       |         |
| with   | **b. Develop   |         | Project    | ntities |         |
| NCI    | objectives and |         | Officers   | and     |         |
| Ce     | evaluation     |         | of         | local   |         |
| nters/ | strategy for   |         | evaluation | CNP     |         |
| Divi   | each project** |         | of CNP     |         |         |
| sions/ |                |         | role       |         |         |
| Offi   |                |         |            |         |         |
| ces.** |                |         | **Number   |         |         |
|        |                |         | and types  |         |         |
|        |                |         | of         |         |         |
|        |                |         | projects   |         |         |
|        |                |         | offered by |         |         |
|        |                |         | NCI.**     |         |         |
|        |                |         |            |         |         |
|        |                |         | **Number   |         |         |
|        |                |         | and types  |         |         |
|        |                |         | of         |         |         |
|        |                |         | projects   |         |         |
|        |                |         | f          |         |         |
|        |                |         | acilitated |         |         |
|        |                |         | by CNP**   |         |         |
+--------+----------------+---------+------------+---------+---------+
| **I.4  | a              | Public  | **Number   | **I     | Program |
| In     | .  **Determine | Service | and type   | ncrease | Mon     |
| crease |     the best   | Announ  | of         | in      | itoring |
| the    |     strategies | cements | activities | collabo | Sys     |
| utili  |     for        |         | (e.g.,     | rations | tem/CNP |
| zation |                | **News  | health     | across  | D       |
| of     |  collaboration | letters | fairs,     | local   | atabase |
| bene   |     to meet    | (CP:    | annual     | pr      |         |
| ficial |     Project    | 13)**   | events,    | ograms/ |         |
| i      |                |         | ACS        | par     |         |
| nterve |  objectives.** | *       | events)**  | tners** |         |
| ntions |                | *Videos |            |         |         |
| to     | b              | (CP:    | **Number   | U       |         |
| reduce | .  **Establish | 13)**   | and type   | ltimate |         |
| dispa  |     mandated   |         | of         | changes |         |
| rities |                | *       | community  | in      |         |
| in the |  collaboration | *Awards | education  | b       |         |
| commun |     with       | for     | materials  | ehavior |         |
| ity.** |     Cancer     | presen  | d          | as      |         |
|        |                | tation\ | eveloped** | sp      |         |
|        |    Information | (CP:    | (language, | ecified |         |
|        |     Service    | 13)**   | community/ | for     |         |
|        |     (CIS)      |         | i          | Phase   |         |
|        |                | **Ad    | ndividuals | III     |         |
|        |    partnership | dresses | targeted)  |         |         |
|        |                | of Web  | (**CPN:    |         |         |
|        |    programs.** | sites   | 13)**      |         |         |
|        |                | de      |            |         |         |
|        | c.  **Develop  | veloped |            |         |         |
|        |     at least   | (CP:    |            |         |         |
|        |     three      | 13**)   |            |         |         |
|        |     additional |         |            |         |         |
|        |     NCI        |         |            |         |         |
|        |     c          |         |            |         |         |
|        | ollaborations, |         |            |         |         |
|        |     through    |         |            |         |         |
|        |     guidance   |         |            |         |         |
|        |     from the   |         |            |         |         |
|        |     NCI        |         |            |         |         |
|        |     Project    |         |            |         |         |
|        |                |         |            |         |         |
|        |    Officers.** |         |            |         |         |
|        |                |         |            |         |         |
|        | d              |         |            |         |         |
|        | .  **Implement |         |            |         |         |
|        |     the        |         |            |         |         |
|        |                |         |            |         |         |
|        |  collaborative |         |            |         |         |
|        |                |         |            |         |         |
|        |    projects.** |         |            |         |         |
+--------+----------------+---------+------------+---------+---------+

(continued)

National Cancer Institute's Community Networks Program Evaluation Matrix
(continued)

+--------+----------------+---------+------------+---------+---------+
| *      |                |         |            |         |         |
| *Phase |                |         |            |         |         |
| I      |                |         |            |         |         |
| Goal:  |                |         |            |         |         |
| D      |                |         |            |         |         |
| evelop |                |         |            |         |         |
| and    |                |         |            |         |         |
| in     |                |         |            |         |         |
| crease |                |         |            |         |         |
| ca     |                |         |            |         |         |
| pacity |                |         |            |         |         |
| bu     |                |         |            |         |         |
| ilding |                |         |            |         |         |
| to     |                |         |            |         |         |
| s      |                |         |            |         |         |
| upport |                |         |            |         |         |
| com    |                |         |            |         |         |
| munity |                |         |            |         |         |
| -based |                |         |            |         |         |
| p      |                |         |            |         |         |
| artici |                |         |            |         |         |
| patory |                |         |            |         |         |
| educ   |                |         |            |         |         |
| ation, |                |         |            |         |         |
| res    |                |         |            |         |         |
| earch, |                |         |            |         |         |
| and    |                |         |            |         |         |
| tr     |                |         |            |         |         |
| aining |                |         |            |         |         |
| to     |                |         |            |         |         |
| reduce |                |         |            |         |         |
| cancer |                |         |            |         |         |
| dispa  |                |         |            |         |         |
| rities |                |         |            |         |         |
| (Years |                |         |            |         |         |
| 1      |                |         |            |         |         |
| --5)** |                |         |            |         |         |
+========+================+=========+============+=========+=========+
| **     | **Activities** | **Outp  | **Process  | *       | **P     |
| Object |                | uts/Pro | Outcomes** | *Impact | ossible |
| ives** | *(Organize by  | ducts** |            | Out     | Data    |
|        | perspective:   |         | *          | comes** | So      |
|        | CNP Project    |         | (Formative |         | urces** |
|        | Activities,    |         | measures   | *(M     |         |
|        | CNP Evaluation |         | that       | easures |         |
|        | Team, NCI-CNP  |         | assess how | the     |         |
|        | Program        |         | well       | extent  |         |
|        | Director)*     |         | activities | to      |         |
|        |                |         | are being  | which   |         |
|        |                |         | acco       | the     |         |
|        |                |         | mplished)* | act     |         |
|        |                |         |            | ivities |         |
|        |                |         |            | have    |         |
|        |                |         |            | r       |         |
|        |                |         |            | esulted |         |
|        |                |         |            | in      |         |
|        |                |         |            | system, |         |
|        |                |         |            | indi    |         |
|        |                |         |            | vidual, |         |
|        |                |         |            | or      |         |
|        |                |         |            | policy  |         |
|        |                |         |            | c       |         |
|        |                |         |            | hange)* |         |
+--------+----------------+---------+------------+---------+---------+
| **I.5  | a.  Identify   | *       | Number and | **I     | Award   |
| Obtain |     unmet      | *Matrix | type of    | ncrease | notices |
| non    |     cancer     | of the  | proposals  | in      |         |
| -CRCHD |     control    | type,   | submitted  | no      | State   |
| f      |     needs in   | amount, |            | n-CRCHD | and     |
| unding |     the        | and     | Proportion | funding | local   |
| for    |                | source  | of         | (CP:    | funding |
| com    |    communities | of      | ap         | 2)**    | p       |
| munity |     of         | funding | plications |         | olicies |
| -based |     interest.  | for     | won/lost   |         |         |
| edu    |                | cancer  |            |         | Annual  |
| cation | b.  **Assess   | control | Assessment |         | reports |
| and    |     funding    | in      | of costs   |         | and     |
| tr     |     needs and  | general | and        |         | other   |
| aining |     available  | and     | con        |         | o       |
| acti   |     resources  | cancer  | tributions |         | fficial |
| vities |     (local,    | disp    | made by    |         | commun  |
| di     |     state,     | arities | sources    |         | ication |
| rected |     private,   | in      | other than |         | from    |
| at     |     public)    | parti   | NCI        |         | Foun    |
| re     |     (CP: 10)** | cular** |            |         | dations |
| ducing |                |         |            |         | and     |
| cancer | c.  Develop    |         |            |         | other   |
| di     |     and submit |         |            |         | funding |
| sparit |     proposals  |         |            |         | sources |
| ies.** |     for        |         |            |         |         |
|        |     funding.   |         |            |         | Program |
|        |                |         |            |         | Mon     |
|        | d.  Leverage   |         |            |         | itoring |
|        |     in-kind    |         |            |         | Sys     |
|        |                |         |            |         | tem/CNP |
|        |  contributions |         |            |         | D       |
|        |     of space,  |         |            |         | atabase |
|        |     staff      |         |            |         |         |
|        |     support,   |         |            |         |         |
|        |     materials, |         |            |         |         |
|        |     etc.       |         |            |         |         |
+--------+----------------+---------+------------+---------+---------+

(continued)

National Cancer Institute's Community Networks Program Evaluation Matrix
(continued)

+--------+----------------+---------+------------+---------+---------+
| *      |                |         |            |         |         |
| *Phase |                |         |            |         |         |
| I      |                |         |            |         |         |
| Goal:  |                |         |            |         |         |
| D      |                |         |            |         |         |
| evelop |                |         |            |         |         |
| and    |                |         |            |         |         |
| in     |                |         |            |         |         |
| crease |                |         |            |         |         |
| ca     |                |         |            |         |         |
| pacity |                |         |            |         |         |
| bu     |                |         |            |         |         |
| ilding |                |         |            |         |         |
| to     |                |         |            |         |         |
| s      |                |         |            |         |         |
| upport |                |         |            |         |         |
| com    |                |         |            |         |         |
| munity |                |         |            |         |         |
| -based |                |         |            |         |         |
| p      |                |         |            |         |         |
| artici |                |         |            |         |         |
| patory |                |         |            |         |         |
| educ   |                |         |            |         |         |
| ation, |                |         |            |         |         |
| res    |                |         |            |         |         |
| earch, |                |         |            |         |         |
| and    |                |         |            |         |         |
| tr     |                |         |            |         |         |
| aining |                |         |            |         |         |
| to     |                |         |            |         |         |
| reduce |                |         |            |         |         |
| cancer |                |         |            |         |         |
| dispa  |                |         |            |         |         |
| rities |                |         |            |         |         |
| (Years |                |         |            |         |         |
| 1      |                |         |            |         |         |
| --5)** |                |         |            |         |         |
+========+================+=========+============+=========+=========+
| **     | **Activities** | **Outp  | **Process  | *       | **P     |
| Object |                | uts/Pro | Outcomes** | *Impact | ossible |
| ives** | *(Organize by  | ducts** |            | Out     | Data    |
|        | perspective:   |         | *          | comes** | So      |
|        | CNP Project    |         | (Formative |         | urces** |
|        | Activities,    |         | measures   | *(M     |         |
|        | CNP Evaluation |         | that       | easures |         |
|        | Team, NCI-CNP  |         | assess how | the     |         |
|        | Program        |         | well       | extent  |         |
|        | Director)*     |         | activities | to      |         |
|        |                |         | are being  | which   |         |
|        |                |         | acco       | the     |         |
|        |                |         | mplished)* | act     |         |
|        |                |         |            | ivities |         |
|        |                |         |            | have    |         |
|        |                |         |            | r       |         |
|        |                |         |            | esulted |         |
|        |                |         |            | in      |         |
|        |                |         |            | system, |         |
|        |                |         |            | indi    |         |
|        |                |         |            | vidual, |         |
|        |                |         |            | or      |         |
|        |                |         |            | policy  |         |
|        |                |         |            | c       |         |
|        |                |         |            | hange)* |         |
+--------+----------------+---------+------------+---------+---------+
| **I.6  | a.  Work with  | Tools   | Amount of  | **      | Program |
| NCI to |     Projects   | for     | TA         | Quality | Mon     |
| faci   |     and        | effe    | requested/ | of      | itoring |
| litate |     National   | ctively | received   | collabo | Sys     |
| and    |     Evaluator  | pr      | from       | rations | tem/CNP |
| s      |     to develop | oviding | NCI/CIS    | with    | D       |
| upport |     "**core    | TA to   |            | other   | atabase |
| local  |     program    | local   | Number and | NCI     |         |
| CNP    |     evaluation | CNPs    | types of   | Center  | Int     |
| d      |     data       |         | **NCI      | s/Divis | erviews |
| evelop |                |         | -sponsored | ions/Of | with    |
| ment** |    elements**" |         | coll       | fices** | Pr      |
|        |     (RFA: 9).  |         | aborations |         | incipal |
|        |                |         | (CP: 13)** | System  | Invest  |
|        | b.  Assess     |         |            | change  | igators |
|        |     Project's  |         |            | (e.g.,  |         |
|        |                |         |            | geo     |         |
|        |   achievements |         |            | graphic |         |
|        |     to         |         |            | and     |         |
|        |     determine  |         |            | appo    |         |
|        |     if Phase I |         |            | intment |         |
|        |     benchmarks |         |            | sch     |         |
|        |     have been  |         |            | eduling |         |
|        |     met.       |         |            | a       |         |
|        |                |         |            | ccessib |         |
|        | c.  Provide    |         |            | ility)\ |         |
|        |     ongoing    |         |            | **(CP:  |         |
|        |     technical  |         |            | 3)**    |         |
|        |     assistance |         |            |         |         |
|        |     (TA) to    |         |            |         |         |
|        |     CNPs.      |         |            |         |         |
|        |                |         |            |         |         |
|        | d.  Identify   |         |            |         |         |
|        |     problems   |         |            |         |         |
|        |     with CNPs  |         |            |         |         |
|        |     that are   |         |            |         |         |
|        |     behind in  |         |            |         |         |
|        |     meeting    |         |            |         |         |
|        |     benchmarks |         |            |         |         |
|        |     and        |         |            |         |         |
|        |     address    |         |            |         |         |
|        |                |         |            |         |         |
|        | deficiencies.. |         |            |         |         |
|        |                |         |            |         |         |
|        | e.  Organize   |         |            |         |         |
|        |     regular    |         |            |         |         |
|        |     meetings   |         |            |         |         |
|        |     with CNPs: |         |            |         |         |
|        |                |         |            |         |         |
|        | -   Annual     |         |            |         |         |
|        |     Cancer     |         |            |         |         |
|        |                |         |            |         |         |
|        |    Disparities |         |            |         |         |
|        |     Summit (up |         |            |         |         |
|        |     to 8       |         |            |         |         |
|        |     conference |         |            |         |         |
|        |                |         |            |         |         |
|        |   participants |         |            |         |         |
|        |     per CNP)   |         |            |         |         |
|        |                |         |            |         |         |
|        | -   PI         |         |            |         |         |
|        |     Meetings   |         |            |         |         |
|        |     with CRCHD |         |            |         |         |
|        |     Director   |         |            |         |         |
|        |                |         |            |         |         |
|        |   (semiannual) |         |            |         |         |
|        |                |         |            |         |         |
|        | **f. Assess    |         |            |         |         |
|        | with which NCI |         |            |         |         |
|        | entities       |         |            |         |         |
|        | collaboration  |         |            |         |         |
|        | is feasible    |         |            |         |         |
|        | and facilitate |         |            |         |         |
|        | collaboration  |         |            |         |         |
|        | between local  |         |            |         |         |
|        | CNPs and any   |         |            |         |         |
|        | three NCI      |         |            |         |         |
|        | entities.**    |         |            |         |         |
+--------+----------------+---------+------------+---------+---------+

(continued)

National Cancer Institute's Community Networks Program Evaluation Matrix
(continued)

+--------+----------------+----------+-----------+---------+--------+
| *      |                |          |           |         |        |
| *Phase |                |          |           |         |        |
| II     |                |          |           |         |        |
| Goal:  |                |          |           |         |        |
| Create |                |          |           |         |        |
| com    |                |          |           |         |        |
| munity |                |          |           |         |        |
| -based |                |          |           |         |        |
| re     |                |          |           |         |        |
| search |                |          |           |         |        |
| and    |                |          |           |         |        |
| tr     |                |          |           |         |        |
| aining |                |          |           |         |        |
| pr     |                |          |           |         |        |
| ograms |                |          |           |         |        |
| to     |                |          |           |         |        |
| reduce |                |          |           |         |        |
| cancer |                |          |           |         |        |
| dispa  |                |          |           |         |        |
| rities |                |          |           |         |        |
| (Years |                |          |           |         |        |
| 2      |                |          |           |         |        |
| --5)** |                |          |           |         |        |
+========+================+==========+===========+=========+========+
| **     | **Activities\  | **O      | **Process | *       | **Po   |
| Object | ***(Organize   | utputs** | O         | *Impact | ssible |
| ives** | by             |          | utcomes** | Ou      | Data   |
|        | perspective:   |          |           | tcomes\ | Sou    |
|        | CNP Project    |          | *(        | ***(M   | rces** |
|        | Activities,    |          | Formative | easures |        |
|        | CNP Evaluation |          | measures  | the     |        |
|        | Team, NCI-CNP  |          | that      | extent  |        |
|        | Program        |          | assess    | to      |        |
|        | Director)*     |          | how well  | which   |        |
|        |                |          | a         | the     |        |
|        |                |          | ctivities | act     |        |
|        |                |          | are being | ivities |        |
|        |                |          | accom     | have    |        |
|        |                |          | plished)* | r       |        |
|        |                |          |           | esulted |        |
|        |                |          |           | in      |        |
|        |                |          |           | system, |        |
|        |                |          |           | indi    |        |
|        |                |          |           | vidual, |        |
|        |                |          |           | or      |        |
|        |                |          |           | policy  |        |
|        |                |          |           | c       |        |
|        |                |          |           | hange)* |        |
+--------+----------------+----------+-----------+---------+--------+
| **II.1 | a.  Collect    | **Matrix | Number of | R       | Ho     |
| D      |     and review | of       | research  | esearch | spital |
| evelop |     **baseline | Health   | studies   | f       | (e.g., |
| cancer |     data to**  | Dis      | proposed  | indings | ER,    |
| dispa  |     educate    | parities | and       | to      | phy    |
| rities |     and raise  | Re       | completed | support | sician |
| re     |     awareness  | search** |           | new     | of     |
| search |     about      |          | Level of  | evidenc | fices) |
| that   |     cancer     | -        | in        | e-based | and    |
| f      |                |   Report | volvement | cancer  | clinic |
| ocuses |    disparities |          | of        | control | data   |
| on the |     in the     |   number | priority  | interve |        |
| sp     |                |     of   | po        | ntions, | BRFSS  |
| ectrum |   communities, |     d    | pulations | po      |        |
| of     |     such as    | isparity | in        | licies, | Com    |
| re     |                |          | d         | etc.,   | munity |
| search | -   preval     | research | eveloping | to      | health |
| issues | ence/incidence |          | the       | address | s      |
| nec    |     of cancer  |    areas | research  | f       | urveys |
| essary |     by type,   |     a    |           | indings |        |
| to     |     sex, and   | ddressed | Fidelity  | from    | P      |
| reduce |     age;       |          | of the    | r       | rogram |
| cancer |                | -        | research  | esearch | Moni   |
| di     | -   stage of   |   Report | plan to   |         | toring |
| sparit |     cancer at  |     type | what was  | Tran    | Syst   |
| ies.** |     diagnosis; |     of   | i         | slation | em/CNP |
|        |                |     d    | dentified | of      | Da     |
|        | -   screening  | isparity | as gaps   | r       | tabase |
|        |     behavior   |          | in        | esearch |        |
|        |     patterns   | research | unde      | to      |        |
|        |     by cancer  |          | rstanding | p       |        |
|        |     type;      |    areas | and what  | ractice |        |
|        |                |     a    | was       |         |        |
|        | -   follow-up  | ddressed | u         |         |        |
|        |     rates from |          | ltimately |         |        |
|        |     abnormal   | -        | studied   |         |        |
|        |                |   Report |           |         |        |
|        |    screenings; |     type | Increased |         |        |
|        |     and        |     of   | unde      |         |        |
|        |                |          | rstanding |         |        |
|        | -   treatment  |  cancers | of the    |         |        |
|        |                |          | issues    |         |        |
|        |    utilization | targeted | impacting |         |        |
|        |     practices  |     (CP: | cancer    |         |        |
|        |     by cancer  |     14)  | control   |         |        |
|        |     type.      |          | among     |         |        |
|        |                | -        | disparate |         |        |
|        | b.  Prioritize |   Report | po        |         |        |
|        |     community  |          | pulations |         |        |
|        |     cancer     | priority |           |         |        |
|        |     control    |          |           |         |        |
|        |     needs.     |    needs |           |         |        |
|        |                |     to   |           |         |        |
|        | c.  Determine  |     be   |           |         |        |
|        |     efficient  |     a    |           |         |        |
|        |     use of     | ddressed |           |         |        |
|        |     resources  |          |           |         |        |
|        |     to address | C        |           |         |        |
|        |     needs.     | ommunity |           |         |        |
|        |                | Cancer   |           |         |        |
|        | d.  Develop    | Profile  |           |         |        |
|        |                |          |           |         |        |
|        |  participatory |          |           |         |        |
|        |     c          |          |           |         |        |
|        | ommunity-based |          |           |         |        |
|        |     research   |          |           |         |        |
|        |     proposals. |          |           |         |        |
|        |                |          |           |         |        |
|        | e.  **Conduct  |          |           |         |        |
|        |     research   |          |           |         |        |
|        |     to         |          |           |         |        |
|        |     identify   |          |           |         |        |
|        |     cancer     |          |           |         |        |
|        |     c          |          |           |         |        |
|        | ontrol-related |          |           |         |        |
|        |                |          |           |         |        |
|        |    policies.** |          |           |         |        |
+--------+----------------+----------+-----------+---------+--------+

(continued)

National Cancer Institute's Community Networks Program Evaluation Matrix
(continued)

+--------+----------------+----------+-----------+---------+--------+
| *      |                |          |           |         |        |
| *Phase |                |          |           |         |        |
| II     |                |          |           |         |        |
| Goal:  |                |          |           |         |        |
| Create |                |          |           |         |        |
| com    |                |          |           |         |        |
| munity |                |          |           |         |        |
| -based |                |          |           |         |        |
| re     |                |          |           |         |        |
| search |                |          |           |         |        |
| and    |                |          |           |         |        |
| tr     |                |          |           |         |        |
| aining |                |          |           |         |        |
| pr     |                |          |           |         |        |
| ograms |                |          |           |         |        |
| to     |                |          |           |         |        |
| reduce |                |          |           |         |        |
| cancer |                |          |           |         |        |
| dispa  |                |          |           |         |        |
| rities |                |          |           |         |        |
| (Years |                |          |           |         |        |
| 2      |                |          |           |         |        |
| --5)** |                |          |           |         |        |
+========+================+==========+===========+=========+========+
| **     | **Activities\  | **O      | **Process | *       | **Po   |
| Object | ***(Organize   | utputs** | O         | *Impact | ssible |
| ives** | by             |          | utcomes** | Ou      | Data   |
|        | perspective:   |          |           | tcomes\ | Sou    |
|        | CNP Project    |          | *(        | ***(M   | rces** |
|        | Activities,    |          | Formative | easures |        |
|        | CNP Evaluation |          | measures  | the     |        |
|        | Team, NCI-CNP  |          | that      | extent  |        |
|        | Program        |          | assess    | to      |        |
|        | Director)*     |          | how well  | which   |        |
|        |                |          | a         | the     |        |
|        |                |          | ctivities | act     |        |
|        |                |          | are being | ivities |        |
|        |                |          | accom     | have    |        |
|        |                |          | plished)* | r       |        |
|        |                |          |           | esulted |        |
|        |                |          |           | in      |        |
|        |                |          |           | system, |        |
|        |                |          |           | indi    |        |
|        |                |          |           | vidual, |        |
|        |                |          |           | or      |        |
|        |                |          |           | policy  |        |
|        |                |          |           | c       |        |
|        |                |          |           | hange)* |        |
+--------+----------------+----------+-----------+---------+--------+
| **II.2 | a.  **Develop  | **       | Number of | I       | P      |
| Train  |     and        | Training | those     | ncrease | rogram |
| resear |     outline    | curr     | recruited | in      | Moni   |
| chers, |     training   | iculum** | for       | mi      | toring |
| partic |     procedures | *(CP:    | trainings | nority/ | Syst   |
| ularly |     (CP:       | 14)*     |           | unde    | em/CNP |
| those  |     10).**     |          | *         | rserved | Da     |
| from   |                | **       | *Increase | c       | tabase |
| r      | b.  **Recruit  | Training | in        | ommunit |        |
| acial/ |     junior-**  | w        | provider  | y-based | Pre-tr |
| ethnic |     and        | orkshops | knowledge | cancer  | aining |
| mi     |     senior-(?) | con      | about     | control | surv   |
| nority |     **level    | ducted** | recomm    | practi  | ey/int |
| and    |                |          | endations | tioners | erview |
| under  |    researchers |          | for       | and     |        |
| served |     for        |          | care**    | rese    | P      |
| popula |     training   |          |           | archers | ost-tr |
| tions, |     program    |          | Increase  |         | aining |
| in     |     (CP:       |          | in        | I       | surv   |
| com    |     10).**     |          | provider  | ncrease | ey/int |
| munity |                |          | knowledge | in      | erview |
| -based | c.  **Conduct  |          | about the | m       |        |
| p      |                |          | b         | inority | Clinic |
| artici |   trainings.** |          | arriers/c | r       | observ |
| patory |                |          | hallenges | ecruits | ations |
| interv | d.  **Develop  |          | related   | to      |        |
| ention |     mentoring  |          | to health | medical |        |
| re     |     program    |          | care      | and     |        |
| search |     between    |          | services  | health  |        |
| to     |     junior and |          | in the    | prof    |        |
| reduce |     senior     |          | community | essions |        |
| cancer |                |          |           | in      |        |
| di     | researchers.** |          | Increase  | general |        |
| sparit |                |          | in        |         |        |
| ies.** |                |          | providers | *       |        |
|        |                |          | who state | *Change |        |
|        |                |          | they are  | in      |        |
|        |                |          | dedicated | p       |        |
|        |                |          | to        | rovider |        |
|        |                |          | serving   | prac    |        |
|        |                |          | disparate | tices** |        |
|        |                |          | po        |         |        |
|        |                |          | pulations | -   E   |        |
|        |                |          |           | xtended |        |
|        |                |          |           |         |        |
|        |                |          |           |  office |        |
|        |                |          |           |         |        |
|        |                |          |           |   hours |        |
|        |                |          |           |         |        |
|        |                |          |           | -   M   |        |
|        |                |          |           | obile/o |        |
|        |                |          |           | utreach |        |
|        |                |          |           |     s   |        |
|        |                |          |           | ervices |        |
|        |                |          |           |         |        |
|        |                |          |           | -   D   |        |
|        |                |          |           | octor-- |        |
|        |                |          |           | patient |        |
|        |                |          |           |         |        |
|        |                |          |           |  commun |        |
|        |                |          |           | ication |        |
|        |                |          |           |     str |        |
|        |                |          |           | ategies |        |
|        |                |          |           |         |        |
|        |                |          |           | I       |        |
|        |                |          |           | ncrease |        |
|        |                |          |           | in      |        |
|        |                |          |           | c       |        |
|        |                |          |           | ultural |        |
|        |                |          |           | com     |        |
|        |                |          |           | petency |        |
|        |                |          |           | of      |        |
|        |                |          |           | trained |        |
|        |                |          |           | pr      |        |
|        |                |          |           | oviders |        |
+--------+----------------+----------+-----------+---------+--------+

(continued)

National Cancer Institute's Community Networks Program Evaluation Matrix
(continued)

+--------+----------------+----------+-----------+---------+--------+
| *      |                |          |           |         |        |
| *Phase |                |          |           |         |        |
| II     |                |          |           |         |        |
| Goal:  |                |          |           |         |        |
| Create |                |          |           |         |        |
| com    |                |          |           |         |        |
| munity |                |          |           |         |        |
| -based |                |          |           |         |        |
| re     |                |          |           |         |        |
| search |                |          |           |         |        |
| and    |                |          |           |         |        |
| tr     |                |          |           |         |        |
| aining |                |          |           |         |        |
| pr     |                |          |           |         |        |
| ograms |                |          |           |         |        |
| to     |                |          |           |         |        |
| reduce |                |          |           |         |        |
| cancer |                |          |           |         |        |
| dispa  |                |          |           |         |        |
| rities |                |          |           |         |        |
| (Years |                |          |           |         |        |
| 2      |                |          |           |         |        |
| --5)** |                |          |           |         |        |
+========+================+==========+===========+=========+========+
| **     | **Activities** | **O      | **Process | *       | **Po   |
| Object |                | utputs** | O         | *Impact | ssible |
| ives** |                |          | utcomes** | Out     | Data   |
|        |                |          |           | comes** | Sou    |
|        |                |          |           |         | rces** |
+--------+----------------+----------+-----------+---------+--------+
| **II.3 | a.  **Develop  | **Number | *         | I       | Pilot  |
| D      |     criteria   | of pilot | *Fidelity | ncrease | p      |
| evelop |     for pilot  | project  | of pilot  | in      | roject |
| pilot  |     project    | s        | projects  | deve    | r      |
| pr     |     (e.g.,     | ubmitted | to        | lopment | eports |
| ojects |                | by       | planned   | and     |        |
| in     |   resource).** | CNPs**   | events**  | collab  | P      |
| com    |                |          |           | oration | rogram |
| munity | b.  **Prepare  | **Number | Increase  | of new  | Moni   |
| -based |     a request  | of       | in        | and     | toring |
| p      |     for        | papers   | knowledge | e       | Syst   |
| artici |     proposal   | publ     | of        | xisting | em/CNP |
| patory |     document   | ished/in | f         | c       | Da     |
| dispa  |     for        | press    | acilities | ommunit | tabase |
| rities |     d          | based on | that      | y-based |        |
| re     | istribution.** | work of  | conduct   | cancer  | Survey |
| search |                | C        | s         | control | data   |
| (      | c.             | ommunity | creenings | p       | col    |
| option |  **Collaborate | Ne       | for free  | rograms | lected |
| to     |     and pilot  | tworks** | or at low |         | during |
| fund   |     projects   |          | cost to   | **I     | p      |
| up to  |     of the     | **Number | decrease  | mproved | roject |
| four   |     Community  | of       | cost      | access  | (will  |
| pilot  |     Network    | talks/   | barrier   | to and  | depend |
| pr     |     Program    | prese    |           | util    | on     |
| ojects |     (CNP)      | ntations | Increase  | ization | what's |
| per    |     sites (CP: | of CNP   | in        | of      | done)  |
| ye     |     10).**     | work at  | knowledge | ben     |        |
| ar).** |                | prof     | about     | eficial |        |
|        | -   Determine  | essional | effective | cancer  |        |
|        |     the focus  | me       | outreach  | interve |        |
|        |     of the     | etings** | s         | ntions\ |        |
|        |     pilot      |          | trategies | (CP:    |        |
|        |     project    | \% CNPs  |           | 2)**    |        |
|        |     (e.g.,     | u        | Increase  |         |        |
|        |                | tilizing | in cancer | I       |        |
|        | accessibility, | eviden   | awareness | ncrease |        |
|        |     follow-up, | ce-based | and       | in      |        |
|        |                | practice | primary   | re      |        |
|        |    treatment). | (C       | p         | sources |        |
|        |                | ommunity | revention | (e.g.,  |        |
|        | -   Outline    | Health   | (e.g.,    | support |        |
|        |     the goals  | Gui      | diet,     | groups) |        |
|        |     of the     | delines) | exercise, | for     |        |
|        |     pilot      |          | smoking   | cancer  |        |
|        |     project.   |          | c         | su      |        |
|        |                |          | essation) | rvivors |        |
|        |                |          |           |         |        |
|        |                |          | Increase  | I       |        |
|        |                |          | in cancer | ncrease |        |
|        |                |          | awareness | in      |        |
|        |                |          | and       | funding |        |
|        |                |          | secondary | level   |        |
|        |                |          | p         | from    |        |
|        |                |          | revention | other   |        |
|        |                |          | (e.g.,    | sources |        |
|        |                |          | s         | for     |        |
|        |                |          | creening, | ongoing |        |
|        |                |          | symptoms) | work    |        |
|        |                |          |           | with    |        |
|        |                |          | Increase  | di      |        |
|        |                |          | in        | sparate |        |
|        |                |          | knowledge | popu    |        |
|        |                |          | related   | lations |        |
|        |                |          | to cancer |         |        |
|        |                |          | risk      |         |        |
|        |                |          | factors   |         |        |
+--------+----------------+----------+-----------+---------+--------+
| **II.4 | a.  Review the | **Number | Improved  | I       | P      |
| NCI to |     proposals  | of pilot | knowledge | mproved | rogram |
| faci   |     for pilot  | projects | among     | ability | Moni   |
| litate |     projects.  | approved | CNPs of   | of CNPs | toring |
| and    |                | by NCI** | NCI       | to      | Syst   |
| s      | ### Select/no  |          | exp       | succe   | em/CNP |
| upport | tify pilot pro | **Number | ectations | ssfully | Da     |
| local  | ject awardees. | of pilot |           | apply   | tabase |
| CNP    |                | projects |           | for     |        |
| eff    | b.  Award      | funded   |           | pilot   |        |
| orts** |     funding    | and      |           | project |        |
|        |     for pilot  | c        |           | funding |        |
|        |     projects.  | ompleted |           |         |        |
|        |                | (with    |           |         |        |
|        | c.  Monitor    | written  |           |         |        |
|        |     pilot      | reports) |           |         |        |
|        |     projects.  | each     |           |         |        |
|        |                | year     |           |         |        |
|        | d.  Provide    | (CP:     |           |         |        |
|        |     ongoing TA | 14)**    |           |         |        |
|        |     to local   |          |           |         |        |
|        |     programs.  |          |           |         |        |
|        |                |          |           |         |        |
|        | e.  Identify   |          |           |         |        |
|        |     problems   |          |           |         |        |
|        |     with CNPs  |          |           |         |        |
|        |     that are   |          |           |         |        |
|        |     behind in  |          |           |         |        |
|        |     meeting    |          |           |         |        |
|        |     benchmarks |          |           |         |        |
|        |     and        |          |           |         |        |
|        |     address    |          |           |         |        |
|        |                |          |           |         |        |
|        |  deficiencies. |          |           |         |        |
+--------+----------------+----------+-----------+---------+--------+