A ROADMAP FOR 

COLLABORATIVE 

AND EFFECTIVE 

EVALUATION IN 

TRIBAL COMMUNITIES 
This page intentionally left blank. 
Acknowledgments 

Building on the momentum created during the 2011 National Child Welfare Evaluation Summit, the Children’s Bureau 

convened three Child Welfare Research and Evaluation Workgroups. Each workgroup examined a particular evaluation 

topic or issue with the goal of improving child welfare research and evaluation and strengthening the link between 

research and practice. This product was created by the Tribal Evaluation Workgroup. Workgroup members included: 

Melinda Baldwin, MSW, LCSW
 

Federal Co-lead
 

Children’s Bureau
 

Marla Jean Big Boy, JD
 

National Indian Child Welfare Association
 

Dolores Subia Bigfoot, PhD*
 

Indian Country Child Trauma Center at the University 


of Oklahoma*
 

Brian Deakins, MSW
 

Federal Lead
 

Children’s Bureau
 

Nancy Dufraine, MEd
 

Confederated Tribes of the Chehalis Reservation
 

Lucille Echohawk 


Denver Indian Family Resource Center
 

Erin Geary, MSW
 

Administration on Children, Youth and Families
 

Rosie Gomez, MS
 

Children’s Bureau
 

Carol Hafford, PhD
 

NORC at the University of Chicago
 

Francine Eddy Jones, MSW
 

Central Council of Tlingit and
 

Haida Indian Tribes of Alaska
 

Joan LaFrance, EdD
 

Mekinak Consulting
 

Molly Mee
 

Children’s Bureau
 

Aleta Meyer, PhD*
 

Office of Planning, Research & Evaluation
 

Sylvia Murray, LMSW-Macro
 

Saginaw Chippewa Indian Tribe of Michigan
 

Deborah Painte, MPH
 

Native American Training Institute
 

Mary Jane Peck, MSSW
 

Workgroup Coordinator


 

JBS International, Inc.
 

Carlette Randall, MSW*
 

JBS International, Inc.
 

Paulette Running Wolf, PhD*
 

Running Wolf & Associates
 

Malia Villegas, EdD* 


National Congress of American Indians
 

Joe Walker, MPM
 

National Resource Center for Tribes
 

Eileen West
 

Children’s Bureau
 

Virginia Whitekiller, EdD, MSW
 

Northeastern State University
 

Nancy Whitesell, PhD*
 

Centers for American Indian and Alaska Native Health 


at the University of Colorado
 

*Denotes Steering Team member. The Steering Team was primarily

responsible for developing this Tribal Evaluation Workgroup product. 

Suggested citation: 

Tribal Evaluation Workgroup. “A Roadmap for Collaborative and Effective Evaluation in Tribal 

Communities.” Children’s Bureau, Administration for Children and Families, U.S. Department of 

Health and Human Services. September 2013. 

This product was created by JBS International, Inc., under Contract No. HHSP23320095638WC, 

funded by the Children’s Bureau, Administration for Children and Families, U.S. Department of 

Health and Human Services. The content of this product does not necessarily reflect the official 

views of the Children’s Bureau. 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

> Table of Contents 

Executive Summary ............................................................ 1
 

Preface: Creating a Roadmap for Co-Creating Collaborative & Effective Evaluation 


to Improve Tribal Child Welfare Programs .......................................... 3
 

Children’s Bureau Child Welfare Research and Evaluation Workgroups ................. 3
 

Tribal Evaluation Workgroup ................................................... 3
 

Creating the Roadmap ........................................................ 4
 

Becoming a Workgroup ..................................................... 4
 

Workgroup Members....................................................... 5
 

The Workgroup's Goal of Envisioning a New Narrative ............................ 6
 

Using the Roadmap........................................................... 6
 

The Visual Roadmap ............................................................ 7
 

Understanding the Roadmap .................................................... 9
 

Values...................................................................... 9
 

Value #1: Indigenous Ways of Knowing ........................................ 9
 

Value #2: Respect for Tribal Sovereignty ....................................... 10
 

Value #3: Strengths Focus................................................... 11
 

Value #4: Cultural & Scientific Rigor .......................................... 11
 

Value #5: Community Engagement........................................... 12
 

Value #6: Ethical Practices .................................................. 13
 

Value #7: Knowledge Sharing ............................................... 13
 

Historical Context ........................................................... 13
 

Indigenous World Views Are Undervalued..................................... 13
 

Intergenerational and Community Trauma .................................... 15
 

Distrust ................................................................. 15
 

Invasive, Imposed Evaluation ................................................ 16
 

Judgmental Evaluation..................................................... 17
 

Fears Related to Evaluation.................................................... 18
 

Fear of Being Evaluated .................................................... 18
 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Fear of Doing an Evaluation................................................. 18
 

Priorities for the Roadmap of Evaluation in Child Welfare ........................... 18
 

Priorities Related to Relationship Building ..................................... 18
 

Priorities Related to Knowledge and Skill Building............................... 19
 

Secondary Priorities ....................................................... 21
 

Building a New Narrative....................................................... 23
 

Collaborative, Culturally Responsive Evaluation ................................... 23
 

System Improvement......................................................... 23
 

Locally Guided Questions, Data, and Insight...................................... 24
 

Meaningful Analyses......................................................... 24
 

Bidirectional Learning ........................................................ 25
 

AI/AN Evaluation Skills ....................................................... 25
 

Envisioning the New Narrative ................................................. 26
 

Stakeholders ................................................................. 27
 

Primary Stakeholders......................................................... 27
 

Program Directors......................................................... 27
 

Tribal Evaluators .......................................................... 30
 

Evaluation Partners........................................................ 32
 

Program Staff ............................................................ 34
 

Secondary Stakeholders . .................................................... 36
 

Tribal Government Leaders ................................................. 36
 

Policymakers ............................................................. 38
 

Community Members...................................................... 40
 

Cultural Authorities and Spiritual Leaders...................................... 42
 

Tribal Colleges............................................................ 44
 

Universities .............................................................. 46
 

Professional Organizations.................................................. 48
 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Existing Resources That Address Priorities ........................................ 51
 

Theory and Background Resources ............................................. 51
 

Research Indigenous Knowledge and Indigenous Methods ....................... 51
 

Community-Based Participatory Research & Tribal Participatory Research ........... 52
 

History and Context of Evaluation and Research in Tribal Contexts................. 52
 

Evidence-Based Practice, Translation, and Adaptation ........................... 52
 

Best Practices and Practical Guidelines........................................... 52
 

Research and Evaluation in Tribal Contexts .................................... 52
 

Evaluation Design ......................................................... 53
 

Program Implementation................................................... 53
 

Outcomes and Indicators................................................... 53
 

Guidelines for Adaptation .................................................. 54
 

What Is Needed To Move Forward ............................................... 54
 

Guidelines for Stakeholders ................................................... 54
 

Training Opportunities ....................................................... 54
 

Additional Resources......................................................... 55
 

Evaluation Design ......................................................... 55
 

Ethics and Institutional Review Board Processes................................... 56
 

Outcome Measures........................................................ 56
 

Dissemination ............................................................ 56
 

References ................................................................... 57
 
> Executive Summary 

Many Tribal* communities feel the impact of intergenerational trauma as a result of the experiences of prior generations 

exposed to adverse and devastating events and conditions. Tribal communities can recount negative experiences that 

have created a distrust of research and evaluation. Evaluation activities have generally been imposed on Native communities by funding agencies that view evaluation from the dominant cultural paradigm. These approaches often failed to 

recognize the sovereignty of Tribes and to take advantage of long traditions of successful evaluation strategies that draw 

on indigenous practice. Research was often invasive and offered little benefit to the community. In some cases, research 

actually harmed and exploited Native culture and ignored community rights. 

These experiences have contributed to a fear of evaluation in Tribal communities – fear of doing an evaluation and fear 

of being evaluated. Evaluation efforts are often met with fearful reluctance or outright refusal. 

To address these challenges with respect to child welfare, the Children’s Bureau formed a workgroup comprising representatives from Tribal child welfare programs, evaluators, university researchers, technical assistance providers, and Federal 

program partners. The workgroup developed this Roadmap for Co-Creating Collaborative & Effective Evaluation To Improve 

Tribal Child Welfare Programs. This tool can be used to create a shared vision for the future of Tribal child welfare evaluation 

and provide a common language for Tribal communities and evaluators as they improve evaluation practice. 

CO-CREATING COLLABORATIVE & EFFECTIVE EVALUATION TO IMPROVE TRIBAL CHILD WELFARE PROGRAMS

Values

Indigenous Ways of Knowing | Respect for Tribal Sovereignty 

Strengths Focus | Cultural and Scientific Rigor | Community Engagement 

Ethical Practices | Knowledge Sharing 

Program 

Directors 

Tribal 

Evaluators 

Evaluation 

Partners 

Program 

Staff 

Tribal 

Government 

Leaders 

Policy Makers 

Community 

Members 

Cultural 

Authorities & 

Spiritual Leaders 

Tribal Colleges 

Universities 

Professional 

Organizations 

Relationship Building Knowledge & Skill Building 

Evaluation practice that involves the 

community in determining priorities 

Dialogue with community to 

anchor evaluation within cultural & 

ethical practices 

Clear guidelines & examples of 

evaluation components important to 

the community 

Clear guidelines & examples of 

required evaluation components tied 

to funding 

Transparency in evaluation & 

translation for community 

Cultural protocols for communityengaged, comprehensive evaluation 

Mentoring for evaluation design 

& implementation 

Guidelines for selecting, adapting, 

developing, administering, & 

interpreting evaluation instruments 

Storytelling to support staff & 

community interpretation of 

evaluation data 

Training in use, interpretation, 

& storage of data 

Consolidation & coordination of data 

across community programs 

Building a 

New Narrative 

Collaborative, Culturally 

Responsive Evaluation 

System Improvement 

Locally Guided Questions, 

Data, & Insight 

Meaningful Analyses 

Bidirectional Learning

 AI/AN Evaluation Skills 

Developing policies, establishing clear guidelines, 

& building Tribal infrastructure for evaluation 

Indigenous world views undervalued 

Intergenerational and community trauma 

Invasive, imposed evaluation 

Judgmental evaluation 

Distrust 

Historical Context

* Although the word "Tribal" is used throughout this document, the Roadmap is intended to be broadly applicable to a variety of

American Indian and Alaska Native (AI/AN) communities and organizations. 

1 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
An overarching idea in the Roadmap is the concept of “growing our own,” with an emphasis on supporting the training 

and career development of Tribal members who are working to become evaluators and researchers. Many Tribal nations 

have made great strides in taking ownership of research and evaluation within their communities, whereas others are 

just beginning this effort. 

The Roadmap includes priorities related to building relationships, knowledge, and skills that will contribute to the 

creation of a new narrative for evaluation practice in Tribal child welfare. The five relationship-building priorities highlight 

the need for community involvement in determining priorities and anchoring evaluations within appropriate cultural and 

ethical practices. Building relationships in communities ensures that evaluation plans honor cultural ways and respect 

local values. The task of developing and providing clear guidelines is an essential part of building relationships. Providing 

clear guidelines and examples of how local program questions can be answered using evaluation results allows program 

directors and staff to see the benefits of program evaluation. 

The six priority areas related to knowledge and skill building include ideas such as cultural protocols for communityengaged evaluation. Meaningful evaluations in Tribal communities depend on the integration of rigorous scientific 

methods and rigorous cultural adaptation, including guidelines for adapting the evaluation to specific Tribal contexts. 

Another priority is the use of storytelling to support the interpretation of data. Evaluations are opportunities for 

communities to express their concerns, to seek answers to important questions, and to create a story that describes 

activities in their child welfare programs. The data and information generated from the evaluation becomes part of the 

local story. 

The relationship-, knowledge-, and skill-building priorities contribute to building a new narrative in Tribal child welfare 

evaluation by addressing common fears of evaluation and ensuring that the community is actively involved in the design 

and understands the benefits and outcomes of the evaluation. The goal of this effort is to foster evaluation practice that 

is collaborative and culturally responsive and ultimately leads to system improvement. It is guided by local questions, 

data, and insight. This new way of evaluation practice includes meaningful analysis of local issues and is sensitive to 

cultural contexts. The heart of this innovative practice is bidirectional learning; communities have the opportunity to learn 

evaluation skills while also educating outside evaluators about their cultures, programs, and communities. 

The Roadmap is designed to support 11 stakeholder groups that play important roles in the evaluation process. This 

guide explains how these stakeholders can contribute to and benefit from efforts to build the new narrative and provides resources that address the six priorities. 

The Roadmap is a tool to facilitate discussions, partnerships, planning, policy making, and the development of new 

methods among stakeholders for Tribal child welfare evaluation. It will help communities outline the evaluations they 

envision for their programs and assist Federal funders in developing evaluation requirements that promote the inclusion 

of tribally identified evaluation questions. Ultimately, the Roadmap is intended to encourage a new way of conducting 

evaluations in and with Tribal communities. 

2 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
> Preface: Creating a Roadmap for Co-Creating Collaborative & Effective 

Evaluation to Improve Tribal Child Welfare Programs 

Children’s Bureau Child Welfare Research and Evaluation Workgroups 

The Roadmap for Co-Creating Collaborative & Effective Evaluation To Improve Tribal Child Welfare Programs is 

the product of a project launched by the Children’s Bureau in April 2012 to capitalize on the momentum created during 

the 2011 National Child Welfare Evaluation Summit. The Children’s Bureau convened three Child Welfare Research and 

Evaluation Workgroups to engage stakeholders in conversations about how to build and disseminate evidence of effective child welfare practices, strengthen evaluation practice, and promote the use of findings for making sound decisions 

in child welfare programs and systems. 

In its background materials for the creation of these workgroups, the Children’s Bureau summarized the purpose as follows: 

Child welfare systems are striving to achieve better outcomes for the children, youth, and families they serve. As 

expectations for accountability and effectiveness have risen, public child welfare agencies and private providers 

have been called upon to adopt programs and services that have been previously demonstrated to work. Child welfare systems have also been pushed to seek better ways to collect, manage, and use data to ensure quality, inform 

decision-making, and improve service delivery. Likewise, program directors and evaluators have been encouraged 

to pursue more rigorous evaluation approaches that have the potential to contribute new knowledge to the field 

and strengthen a growing evidence base. While this momentum toward building evidence, strengthening practice, 

and relying on data to guide decision-making offers great promise, it also presents unique challenges to Tribal communities and the agencies and organizations that serve them. 

Tribal Evaluation Workgroup 

One of the three Child Welfare Research and Evaluation Workgroups was charged with focusing specifically on 

evaluation within Tribal contexts. In forming this group, the Children’s Bureau noted: 

Some research and evaluation approaches and the heightened expectations to strengthen them are typically not 

well-aligned with Tribes’ experiences, worldviews, values, and resources. Many Tribes can recount past experiences of having been unwilling subjects of outsiders’ research, refused access to the data collected about them, 

or harmed by conclusions drawn from data that were deemed more valid than their own. Despite being equally 

committed to the well-being of their children, Tribes are unlikely to embrace opportunities to investigate research 

questions proposed by funding entities (State, Federal, or academic research agendas) or to assess their own 

performance without numerous fears, apprehension and concern based on their predominately negative historical 

experiences with evaluation. Evaluation questions and methods that fail to acknowledge and respect familial and 

community social structures, caretaking norms and traditions, cultural values, political and economic contexts, 

and worldviews (about concepts such as life, the human condition, family, permanency, well-being, and knowledge) may only serve to reinforce their fears and/or concerns. And, even in those instances in which programs are 

designed to empower a Tribal community to design its own evaluation or to build a data collection system that is 

tailored to its needs, mandates for particular evaluation designs and/or limitations in Tribal capacity – in areas like 

expertise and financial resources – can stifle a project before it gets underway. 

Despite these challenges, many Tribes are finding ways to meet these rising expectations while taking an active 

role in guiding and designing evaluations that will produce useful and meaningful information and create 

knowledge that will help their communities to reach their goals. 

This workgroup is charged with identifying and producing one or more key deliverables that will address the 

challenges described above. The workgroup is expected to take an approach that is informed by Tribal knowledge, 

culture, and tradition while advancing the goal of building strong evidence about what works that will be credible, 

adaptable, transferable, and relevant across communities. 

3 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
The workgroup will develop one or more practical and useful products that will accomplish two or more of the 

following objectives: 

1.	 Increase Tribal involvement and investment in evaluation practice as a means to develop knowledge to improve

the well-being of Tribal children, youth, and families and the child welfare systems that serve them (including but

not limited to CQI [continuous quality improvement])

2.	 Empower Tribal communities to direct, guide, and/or conduct efforts to assess and/or build evidence about the

effectiveness of new or existing Tribal child welfare practices, programs, or policies

3.	 Improve the strength and quality of evaluation practice in and/or with Tribal communities

4.	 Increase the spread and use of culturally appropriate evaluation methods that are likely to produce credible and

useful findings for both the communities of interest and the broader field of child welfare

Creating the Roadmap

> Becoming a Workgroup 

The Children’s Bureau convened the Tribal Evaluation Workgroup in the summer of 2012. A total of 21 members served 

on the workgroup, including representatives from Tribal child welfare programs, evaluators with extensive experience 

in Tribal contexts, university researchers working with AI/AN populations, technical assistance providers, and Federal 

program partners from the Children’s Bureau and the Office of Planning, Research, and Evaluation within the Administration for Children and Families. The majority of the workgroup members were Tribal members. They represented 

geographic regions across the country and diverse cultural communities. 

Initial workgroup meetings took place via conference call as members collectively considered the charge given to them 

by the Children’s Bureau. The respectful collective decision-making processes typically adhered to by many Tribal communities resulted in a relationship-based bonding process. Workgroup interactions were based on mutual respect and 

regard for individual opinions and experiences, as well as a commitment to general improvements in evaluation efforts in 

Indian Country, particularly within Tribal child welfare programs. Discussions quickly focused on the need for fundamental change in the way evaluation is practiced within Tribal contexts and for guidance on how to move toward evaluation 

practice that is culturally and scientifically rigorous. Workgroup members decided that an important step would be to 

create a strategic plan for moving evaluation practice in Tribal contexts forward. They created the Roadmap as a tool to 

increase capacity in the Tribal evaluation community and in supportive institutions (colleges, universities, funding agencies, and professional organizations). 

The workgroup outlined the Roadmap in its initial conference calls. While these discussions were fruitful, the greatest 

strides came in a September 2012 face-to-face meeting in the Washington, D.C. area. At this meeting, members built 

relationships and solidified their identity as a group with a shared mission. The workgroup was transformed from disembodied voices on a phone to a very human, very real, very collaborative partnership team. The resulting environment of 

trust fostered candid exchanges of ideas and brainstorming that moved the Roadmap from a vaguely formed idea to a 

well-articulated vision of the future of evaluation practice in Tribal child welfare. The richness of this meeting was pivotal 

and exemplifies the kind of relationship building that is critical to evaluation partnerships in Tribal communities depicted 

in the Roadmap itself. The Roadmap is grounded in the personal and/or professional knowledge and experience of the 

workgroup members (listed below) and builds upon the current literature base in this field. 

4 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
> Workgroup Members 

Workgroup 

•	 Marla Jean Big Boy, JD (Oglala Lakota Tribe), Board of Directors, National Indian Child Welfare Association

•	 Dolores Subia Bigfoot, PhD (Caddo Nation of Oklahoma), Indian Country Child Trauma Center at the University of

Oklahoma* 

•	 Nancy Dufraine, MEd, Confederated Tribes of the Chehalis Reservation

•	 Lucille Echohawk (Pawnee Nation of Oklahoma), Denver Indian Family Resource Center

•	 Carol Hafford, PhD, NORC at the University of Chicago

•	 Francine Eddy Jones, MSW (Eagle/Wolf of the Yanyeidi Clan of the Taku Tlingit people), Central Council of Tlingit

and Haida Indian Tribes of Alaska

•	 Joan LaFrance, EdD (Turtle Mountain Band of Chippewa Indians), Mekinak Consulting

•	 Sylvia Murray, LMSW-Macro (Lake Superior Band of Chippewa Indians, Keweenaw Bay Ojibwe), Saginaw Chippewa

Indian Tribe of Michigan

•	 Deborah Painte, MPH (Arikara), Native American Training Institute

•	 Carlette Randall, MSW (Oglala Sioux Tribe/Lakota), JBS International, Inc.* 

•	 Paulette Running Wolf, PhD (Blackfeet Tribe), Running Wolf & Associates* 

•	 Malia Villegas, EdD (Alutiiq/Sugpiaq – Alaska Native), National Congress of American Indians* 

•	 Joe Walker, MPM (Delaware Tribe of Western Oklahoma), National Resource Center for Tribes

•	 Virginia Whitekiller, EdD, MSW (Cherokee), Northeastern State University

•	 Nancy Whitesell, PhD, Centers for American Indian and Alaska Native Health at the University of Colorado* 

Federal Staff 

•	 Melinda Baldwin, MSW, LCSW, Children’s Bureau, Administration for Children and Families

•	 Brian Deakins, MSW, Children’s Bureau, Administration for Children and Families

•	 Erin Geary, MSW, Administration on Children, Youth and Families

•	 Rosie Gomez, MS, Children’s Bureau, Administration for Children and Families

•	 Molly Mee, Children’s Bureau, Administration for Children and Families

•	 Aleta Meyer, PhD, Office of Planning, Research & Evaluation, Administration for Children and Families* 

•	 Mary Jane Peck, MSSW, JBS International, Inc.

•	 Eileen West, Children’s Bureau, Administration for Children and Families

* Member of the Workgroup Steering Team

5 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
> The Workgroup's Goal of Envisioning a New Narrative 

The overarching goal of the Roadmap is to create a shared vision for the future of Tribal child welfare evaluation and 

identify mutual goals for both Tribal child welfare agencies and a broad range of Federal funding initiatives as well as 

educational systems (e.g., Tribal colleges, universities) that train evaluators. Tribal infrastructure has evolved over the past 

30 years. Program development, professional development, and sustainability have been challenging issues for Tribal 

nations and organizations given the geographic isolation, inadequate employment pool, and the Tribal-State-Federal 

histories of oppression and discriminatory policies and practices. The Roadmap specifies areas for capacity building and 

provides a common language to assist Tribal communities and evaluators as they strengthen evaluation practice. It will 

be a useful tool for clarifying expectations, roles, and responsibilities. The Roadmap's collaborative approach and focus 

on building relationships and improving skills are intended to empower Tribes and encourage them to continue to grow 

evaluation capacity within their communities. It embodies the “process of becoming” by promoting the creation and 

ongoing support of culturally appropriate evaluation in Indian Country, a gradual and iterative process that will continue 

well beyond the efforts of the workgroup. 

In developing the Roadmap, the workgroup was mindful of the importance of incorporating lessons from a difficult 

history of evaluation and research in Tribal communities, a history that has created fear and distrust regarding the 

evaluation process. Workgroup members agreed that they must move beyond the past to formulate a new way to 

conduct evaluations (or build a new narrative) for the future, one in which evaluation can be embraced as an important 

way to reflect on current practice and the structure of the system as well as to gain insights into improving the wellbeing of American Indian and Alaska Native (AI/AN) children and families. Fundamental to building this new narrative is 

empowering Native communities to move beyond fear and distrust of evaluation by working together to build relationships 

across the stakeholders in Tribal child welfare; stakeholders are defined as all the individuals and groups that are invested in 

the well-being of Tribal children and families. Equally critical is building knowledge and skills in all stakeholder groups that 

will foster more effective evaluation practice. These components make up the pathways of the Roadmap. 

Using the Roadmap

The Children’s Bureau and the Tribal Evaluation Workgroup hope that the Roadmap will facilitate discussion, partnerships, planning, policy making, and the development of new methods among the stakeholders in Tribal child welfare 

evaluation. Possible uses for the Roadmap include: 

11. Program directors could share the Roadmap with evaluation partners to outline the kind of evaluation they 

envision for their programs and the priorities they must address.
 

22. Grantees could reference the Roadmap in grant applications to summarize the components of rigorous evaluation

in Tribal contexts.

33. Tribal colleges or university faculty could include the Roadmap in required reading for students who are preparing

for careers in evaluation and research in Tribal communities.

44. Federal and State funders could use the Roadmap to establish more consistent and responsive evaluation 


requirements that promote the inclusion of tribally identified evaluation questions, communicate consistent 


expectations to Tribal grantees, improve the quality of evaluations, and prioritize building the new narrative.
 

55. Tribal governments, research review boards (RRBs), or institutional review boards (IRBs) could incorporate

components of the Roadmap into their guidelines for researchers and evaluators working within their communities.

Additional ideas on how the Roadmap might be used to benefit individuals and groups invested in the well-being of 

Tribal children and families (stakeholders) – and how these individuals and groups can, in turn, contribute to achieving 

the goals of the Roadmap – are included in the Stakeholders section below. 

6 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
 

> The Visual Roadmap

The visual Roadmap provides a graphic representation of the workgroup's strategic plan. It is, essentially, an overview of 

Roadmap components and their interrelationships. 

Values

Indigenous Ways of Knowing | Respect for Tribal Sovereignty 

Strengths Focus | Cultural and Scientific Rigor | Community Engagement 

Ethical Practices | Knowledge Sharing 

CO-CREATING COLLABORATIVE & EFFECTIVE EVALUATION TO IMPROVE TRIBAL CHILD WELFARE PROGRAMS

Program 

Directors 

Tribal 

Evaluators 

Evaluation 

Partners 

Program 

Staff 

Tribal 

Government 

Leaders 

Policy Makers 

Community 

Members 

Cultural 

Authorities & 

Spiritual Leaders 

Tribal Colleges 

Universities 

Professional 

Organizations 

Relationship Building Knowledge & Skill Building 

Evaluation practice that involves the 

community in determining priorities 

Dialogue with community to 

anchor evaluation within cultural & 

ethical practices 

Clear guidelines & examples of 

evaluation components important to 

the community 

Clear guidelines & examples of 

required evaluation components tied 

to funding 

Transparency in evaluation & 

translation for community 

Cultural protocols for communityengaged, comprehensive evaluation 

Mentoring for evaluation design 

& implementation 

Guidelines for selecting, adapting, 

developing, administering, & 

interpreting evaluation instruments 

Storytelling to support staff & 

community interpretation of 

evaluation data 

Training in use, interpretation, 

& storage of data 

Consolidation & coordination of data 

across community programs 

Building a 

New Narrative 

Collaborative, Culturally 

Responsive Evaluation 

System Improvement 

Locally Guided Questions, 

Data, & Insight 

Meaningful Analyses 

Bidirectional Learning

 AI/AN Evaluation Skills 

Developing policies, establishing clear guidelines, 

& building Tribal infrastructure for evaluation 

Indigenous world views undervalued 

Intergenerational and community trauma 

Invasive, imposed evaluation 

Judgmental evaluation 

Distrust 

Historical Context

Many members of the workgroup talked about the importance of the “process of becoming,” of creating an ongoing, iterative process in which stakeholders work together to create and continually improve the way Tribal child welfare programs 

are built, evaluated, and improved. Workgroup members attempted to capture the nature of this process in the Roadmap. 

This collaborative, iterative “process of becoming” is reflected in the visual Roadmap: 

•	 The circular form represents the continuous cycle of program improvement through evaluation. The immediate

focus is on improving evaluation practice, but the ultimate goal is to use evaluation results to inform child welfare

practice and service delivery systems, which ultimately improve the well-being of children and families in Tribal

communities.

•	 The overlap and linkages across Relationship Building and Knowledge & Skill Building reflect the interdependence

of these priorities and the importance of attending to both sets of priorities.

•	 The centrality of Building a New Narrative within the Roadmap emphasizes the goal of creating a new way of

doing evaluation in Tribal child welfare.

7 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
•	 The outer circle of Historical Context shows how history has shaped current practice and how lessons learned can

improve practice.

•	 Values are at the top of the figure to show their influence on all items in the graphic and their importance in

charting a course for the future. Values play a central role in shaping priorities and practice in building the new

narrative.

•	 The multiple Stakeholders placed around the Roadmap priorities show the importance of engaging several,

interconnected individuals and groups that are committed to ensuring the well-being of children and families in

Tribal communities. These stakeholders represent the diverse perspectives, priorities, and skill sets being brought to

bear on evaluation in Tribal child welfare contexts.

8 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
> Understanding the Roadmap

This document describes each item in the visual Roadmap and suggests ways to address priorities. It provides resources, 

including references and links to information and tools that address the goal of building a new narrative for conducting 

evaluations. The goal is to make resources easily accessible to stakeholders in Tribal evaluation practice. 

Values 

The workgroup identified four concepts that are fundamental to the values identified in the Roadmap: 

•	 Appreciation for those using the program services being evaluated: Elders, leaders, families, and children

who participate in programs, as well as service providers, have wisdom to share and should be included when

planning for and implementing evaluations.

•	 Self-determination: Community members are uniquely positioned to determine the important questions to ask

and to identify the best ways to gather information to answer these questions meaningfully and within the cultural

context. Communities have rights to own evaluation information and to determine with whom it can be shared.

•	 High standards for evaluations: To fully understand a program’s activities and successes, Tribes need information

that reflects different perspectives and is gathered over time using a variety of methods.

•	 Generosity: Lessons learned through evaluation should be shared both wisely and widely using language and

media that are familiar and friendly. Learning should also be shared with those who support the work of Tribes and

contribute resources to Tribal communities.

These concepts underlie several specific values in the Roadmap. Respect for these values is essential to the “process of 

becoming” culturally adept and maintaining cultural integrity in conducting evaluations. These values should be considered at every step in the journey, grounding the approach to evaluating Tribal child welfare programs. 

These seven values include: 

•	 Indigenous Ways of Knowing

•	 Respect for Tribal Sovereignty

•	 Strengths Focus

•	 Cultural and Scientific Rigor

•	 Community Engagement

•	 Ethical Practices

• Knowledge Sharing
 

Descriptions of how the seven values apply to evaluating Tribal programs are provided below.
 

> Value #1: Indigenous Ways of Knowing 

Native people have always been evaluators, using traditional ways of understanding (garnered from historically based 

experiences and generational knowledge passed down) what works and does not work and using oral tradition, 

storytelling, and narrative to disseminate findings. Evaluations in Tribal contexts should rigorously use the best scientific 

methods available but with a clear emphasis on respecting and abiding by cultural protocols. Evaluators should: 

•	 Identify who can speak for the Tribe in approving evaluation and research projects

•	 Ensure that the design of the evaluation is appropriate for the questions to be addressed and the Tribal context

•	 Determine how to gather information

9 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
•	 Identify how to approach administrators or Tribal leaders in respectful and culturally appropriate ways

•	 Carefully decide what questions to ask

•	 Know who to ask for information about specific topics

•	 Review specific evaluation instruments for cultural appropriateness (e.g., disrespectful items, evaluation burden for

participants)

•	 Interpret findings in ways that incorporate cultural and contextual factors

•	 Use appropriate and effective ways to share information

The Importance of Oral Tradition 

Western researchers sometimes discount oral traditions because they consider the written word true or more true than 

spoken ideas or stories. Historically, oral tradition has been the primary mode of transmission of culture and values in 

many Tribal communities. It is central to preserving ceremonies, cultural protocols, language, and other elements of Native culture. Native traditions consider both spoken and written words sacred. Understanding the importance and value 

of oral tradition is critical to both gathering and disseminating information in Tribal communities. 

Evaluation should be grounded in the Native community’s cultural values and ways of understanding, which may include 

the community’s unique perspectives on what it means to come to know, how to establish research outcomes and data 

collection methods, and what the appropriate role of culture is in evaluation research. Evaluations should tell stories that 

teach lessons. Evaluation, like storytelling, is about learning. 

> Value #2: Respect for Tribal Sovereignty 

Sovereignty is at the core of Tribal values. Sovereignty flows from language, culture, and governance. Evaluators and other professionals working with Tribes should recognize that many Tribal communities are involved in nation building, as 

they work to create internal infrastructures for child welfare programs and for evaluation practice to improve those programs. Nation building is an evolving process; some Tribes are further along the path whereas others are just beginning. 

Some Tribal nations and organizations have well-established cultural protocols for evaluation and research, developed 

to provide oversight and guidance to studies within their communities. Others are starting their journey. And still others 

are somewhere in between. Evaluators and researchers should respect where the community is in this process and work 

closely with Tribal authorities to ensure appropriate input and cultural rigor in their work. Grants to Tribal nations and organizations that require evaluation provide opportunities for Tribes to exercise their sovereignty by identifying evaluation 

questions, engaging in evaluation design, and establishing indigenous evaluation protocols that can provide information 

to inform improvements in child welfare in their communities. Recognition of each Tribe’s sovereignty is important to 

supporting the evaluation practice across Tribal communities. 

The first priority when planning or performing an evaluation with Tribes should be to recognize and value sovereignty and 

evaluate using methods that are based on cultural ways of knowing and respectful of cultural definitions of success. Within 

the United States, 566 American Indian Tribes and Alaska Native villages have Federal recognition as sovereign nations. 

Many more Tribes do not have Federal recognition: some have State recognition, and some have neither State nor Federal 

recognition. Regardless of recognition status, Tribal interests with regard to the welfare of children and families must be 

respected. Tribal nations have sovereignty over research that happens on their land and with their citizens, including evaluation research. Understanding and abiding by the sovereignty of Tribal nations is critical to appropriate evaluation practice. 

The second priority should be that evaluation and research both protect and benefit Tribal peoples. Tribal ownership 

of data is an important concern related to Tribal sovereignty. Many Tribal nations and organizations assert their rights 

to own their data. When universities or Federal funders work with Tribal nations and organizations, questions of data 

ownership often come to the forefront. Tribes, universities, and funders all claim some aspect of ownership. Collabora10 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
tively establishing protocols for shared data ownership is important in moving forward, as is attaining Tribal approval 

for publication of research based on an evaluation. Tribal nations and organizations should determine how and where 

information is presented to public audiences. Evaluators and Tribes need to work collaboratively on protocols regarding 

publication review and approval processes. 

> Value #3: Strengths Focus 

The goal of Tribal child welfare programs is to support the strong and healthful development of children and families. 

Evaluation should always focus on how child welfare programs promote positive development and family and community protective factors, not just on how they reduce or prevent risk. 

While it is important to monitor how program interventions assist in closing the gap between Native and non-Native 

achievements, a strengths-based approach focuses on exploring Native-to-Native comparisons that emphasize what 

is working well while closely collaborating with other stakeholders to leverage these strengths across communities. 

Although the individual “uniqueness” of Tribal communities can make comparisons across Tribal communities difficult, 

focusing on common strengths provides a potential baseline on which to build. 

Scientific Rigor 

Rigor in Native communities requires careful 

attention to evaluation designs that ensure 

information allows for learning about the value 

of a program within the Tribal or organizational 

local setting. The goal of evaluation is to learn 

how to support the population’s well-being, 

not to test a model to export to other Tribal 

communities; however, Tribal nations and 

organizations are always willing to share what 

they have learned and describe the practices 

that worked well in their communities. 

From a Native perspective, rigor must include 

using (as appropriate to the situation) local 

cultural protocols that promote a fundamental 

respect for knowledge of cultural leaders who 

can provide meaningful insight/explanations to 

important questions. For example: 

> Cultural rigor might include incorporation 

of oral traditions. 

> Cultural rigor might include engagement 

of the entire community in the evaluation 

process. 

Rigorous evaluation in Tribal communities 

means that sound scientific methods need 

to be employed but that they must also be 

grounded in sound cultural methods.

> Value #4: Cultural & Scientific Rigor 

Rigor often means that, for the information resulting from an 

evaluation to be useful, the evaluation must be done in ways 

that make the information real and valid; rigor is important 

when ruling out alternative explanations for findings and providing reliable answers to important questions. Sometimes, 

rigor seems to be code for specific types of experimental or 

quasi-experimental evaluation designs that require treatment 

and control or comparison groups. These types of evaluation designs are often encouraged to yield findings that are 

considered evidence based and support treatment models 

that can be scaled up or exported to other Tribal or non-Tribal 

communities. However, the term rigor often carries connotations of rigidity – implying methods that are inflexible and not 

respectful of local cultural and community ways. 

Rigorous evaluation is a useful tool to inform Tribal communities. It provides reliable information to communities as they 

continue to improve programs and better serve children and 

families. Rigorous evaluation is often a requirement of funding for child welfare programs, but even when not required, it 

serves a very important purpose in guiding programs. 

Tribal communities should benefit from the highest quality 

evaluations that are informed by Western science to answer 

local program questions. Ongoing negotiations of the understanding of indigenous ways of knowing and the concepts of 

scientific rigor validate both Native and Western perspectives. 

For example: 

• Tribal nations and organizations should not get second-class evaluations because evaluators consider working with

them to be different, difficult, or time consuming.

11 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
•	 Evaluations of Tribal programs should be of the highest

quality and address Tribe-specific history and contexts,

beliefs, protocols, and program needs.

•	 Scientific rigor may need to be redefined in Tribal

contexts, to include rigorous application of cultural

knowledge and methods as well as rigorous application

of the scientific method.

What is rigorous in other populations may not be rigorous in 

Native communities. Importing methodology without regard 

to cultural context will result in non-rigorous evaluation. 

Establishing a concept of rigorous indigenous evaluation adds 

richness to the conversation because it encompasses recognition of community input and spiritual awareness of the work. 

> Value #5: Community Engagement 

Within a framework that centers on Tribal sovereignty, community engagement may extend beyond participation to emphasize tribally driven research and to strengthen the capacity 

and authority of communities to oversee evaluation. A critical 

examination of decision-making processes in research can 

provide important information about the extent of community engagement. 

Community engagement includes the understanding of 

relationships and the importance of belonging within the 

community. Engagement cannot be superficial; it must include a commitment to the community. Engagement, and the 

relationship building that is integral to engagement, takes time to develop; it cannot be rushed. Engagement involves 

getting to know the community and its people, building trust, deference, and mutual respect. Both knowledge sharing 

and cultural rigor in evaluation are grounded in community engagement. 

Evaluation should incorporate meaningful community input in all phases, including: 

•	 Determination of key questions (What do you want to know about the program, intervention, or service being

provided?)

•	 Design of evaluation plan (How is information gathered to answer the key questions? How burdensome is it to

participate?)

•	 Selection of appropriate measures (What tools, surveys, interview questions will be used?)

•	 Interpretation of findings (What does the information that has been gathered mean? Does it answer the key

questions?)

•	 Dissemination – returning the knowledge gained to the community (How is the information shared? What

publications, journals, newspapers, community forums, and so forth will disseminate the information?)

Western Science 

Science generally describes the “systematic 

study of the structure and behavior of the 

physical and natural world through observation 

and experiment” (“Science,” 2005). “Western 

science” in this document refers to scientific 

approaches commonly accepted within the 

dominant culture that employ analytical and 

reductionist methods. Western scientific studies 

often prioritize objectivity and quantitative 

methods when attempting to answer research 

questions and build evidence. Western 

science typically uses academic and written 

communication to document and transfer 

information. This contrasts with traditional or 

indigenous knowledge, which often relies on 

subjective, qualitative, intuitive, and holistic 

methods to explain human experience and 

humans’ connectedness with the world. 

Indigenous knowledge is grounded in 

context and local experience and conditions. 

Community leaders and elders share indigenous 

knowledge without attempting to separate 

empirical and sacred lessons.This knowledge 

is often orally transmitted from generation to 

generation (Mazzocchi, 2006). 

12 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
> Value #6: Ethical Practices 

Ethical concerns should be considered and respected at each phase of evaluation. The Belmont Report (U.S. Department 

of Health and Human Services, 1979) notes three ethical principles: 

• Respect for persons (informed consent, protection of vulnerable populations)

• Beneficence (risk-benefit analysis, minimized risk)

• Justice (fair selection of participants, return knowledge gained to community)

These principles may need to be expanded to include protection for communities, especially in contexts where cultural 

knowledge is held by a wide range of community members. In addition, evaluators and researchers should consider 

how consent and data sharing are managed with individuals and Tribal governments. Specifically, how consent and data 

sharing are managed with individuals and Tribal governments needs to be considered. Ethical standards for research 

and evaluation in Tribal communities should not just be responsive to a history of violations, but also adhere to cultural 

principles that emphasize both protection and benefit. 

Evaluators, researchers, and others working with Tribal nations and organizations will be much more effective if they can 

recognize the ongoing forces of racism, colonialism, and discrimination that impact modern Tribal communities. Oppressive legacies remain embedded in policies, institutions, and social systems (institutional and structural racism) and 

continue to be perpetuated in the practices, belief systems, and behaviors of many individuals (individual racism). 

> Value #7: Knowledge Sharing 

Indigenous Knowledge 

Native people understand their communities and their children. They have important knowledge to share with evaluators, and sharing that knowledge is critical to valid evaluation. For some Tribal communities, knowledge sharing is based 

on Tribal perceptions of the rights of individuals to give and receive the knowledge. For example, elders often have 

elevated status as holders of wisdom; it may be disrespectful for youth to offer opinions or share knowledge. This can 

have a significant impact on the collection, analysis, and interpretation of information. 

Scientific Knowledge 

Evaluators and researchers understand scientific methods and evaluation strategies. They have important knowledge 

to share with Native communities and other community stakeholders. Sharing this knowledge is critical to ensuring an 

evaluation that provides useful information to improve practice. 

Historical Context 

To understand the Roadmap and use it to evaluate Tribal child welfare programs, it is important to understand the 

historical context in which evaluation operates. 

> Indigenous World Views Are Undervalued 

Evaluation activities have generally been imposed on Native communities by funding agencies that view evaluation from 

the dominant cultural paradigm. Use of standard evaluation approaches (e.g., research design, instrument selection, 

data collection strategies, data interpretation, reporting issues) has been required in Native communities as a condition 

of funding for many child welfare programs. Until recently, Tribal communities had limited or no authority on evaluation 

design, instrument selection, data collection, interpretation of findings, or dissemination of information being reported 

about their communities. These practices have often ignored the sovereignty of Tribal nations and have failed to take 

advantage of long traditions of successful evaluation strategies that draw on indigenous practice. 

13 | Child Welfare Research & Evaluation Workgroups: Sponsored by the Children’s Bureau 
Relational World View 

A relational world view is fundamental to the cultures of 

many American Indian Tribes and Alaska Native villages. “The 

relational worldview, sometimes called the cyclical worldview, is intuitive, non-time–oriented, and fluid. The balance 

and harmony in relationships between multiple variables, 

including spiritual forces, make up the core of the thought 

system. Every event is understood in relation to all other 

events regardless of time, space, or physical existence. In the 

relational worldview, helpers and healers are taught to understand problems through the balances and imbalances in the 

person’s relational world” (Cross, 1997). 

The relational world view is also sometimes known as the 

Circle Theory. This is “Old Wisdom” that was applied for 

generation upon generation, but the transmission of these 

teachings and applications was interrupted when the structure of the indigenous social composition was attacked and 

almost destroyed. 

Indigenous Ways of Knowing 

Native people have historically and collaboratively used evaluation skills, such as assessing, comparing, and interpreting. 

Indigenous relational world views and indigenous ways of 

knowing and evaluating have been discounted and considered inferior to Western scientific methods. 

Native people today continue to have keen observational 

skills developed from childhood and practiced throughout 

adult- hood. Native elders pass these skills on to younger 

generations. These skills are instrumental in planning seasonal activities and designing shelter and clothing. Observing, 

listening, and measuring change (e.g., animal and insect 

activity, vegetation growth patterns) are used to determine 

when to fish, hunt, and harvest. Construction of shelters, 

designs of canoes and other seaworthy vessels, and use of 

various fibers for clothing (including weaving and tanning) 

have long required careful observation and evaluation of the 

effectiveness of different methods and materials. The use of 

medicinal plants was developed from observing their effects 

on the body and on health challenges. Ceremonies have been 

developed to preserve this knowledge and support the success of proven practices within Tribal communities. 

Historical Trauma 

The term “Historical Trauma” was coined by 

Maria Brave Heart Yellow Horse (Lakota) to 

explain the effects of what American Indians 

and Alaska Natives experience today as a result 

of past generations’ exposure to adverse and 

devastating events and conditions. These 

experiences have been variously conceptualized 

as intergenerational posttraumatic stress 

disorder (Duran & Duran, 1995) and historical 

grief (Brave Heart, 1998; Brave Heart & 

DeBruyn, 1998). Historical trauma can best be 

understood as individual and group responses 

to conditions that are life-threatening and 

present constant concerns. It is a legacy for 

American Indian and Alaska Native people who 

have faced numerous social, political, economic, 

and psychological assaults. 

Traumatic events that undermine a civilization 

disrupt not only individual lives but also the 

capacity for the community to regroup or 

rebuild in a healthful and meaningful way. 

Constant assaults on communities trying 

to cope with disruptive trauma diminishes 

their capacity even more. Understanding the 

impact of historical trauma on AI/AN people 

requires recognition of pervasive cultural and 

intergenerational trauma that has accumulated 

over centuries of exposure to racism, warfare, 

oppression, displacement, violence, and 

catastrophic diseases. 

Intergenerational trauma occurs when the 

trauma of an event is not resolved and is 

subsequently internalized and passed from 

one generation to the next through a variety 

of poor practices or impaired capacity to 

adequately care for oneself or others. These 

effects are evident in inappropriate parenting 

or relational skills and inability of communities 

to intervene when families are overwhelmed 

or are self-destructing (e.g., out-of-home 

placements, foster care, residential care). 

Contributions of Indigenous Science 

Indigenous knowledge has been periodically mined by Western science and has resulted in many contributions to medicine and health practices in society today (Warriner, G. & Engelstad K, 1985). Few Tribal contributions to the Western 

scientific world of medicine and healing have been acknowledged. Much of this knowledge has been borrowed without 

14 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
acknowledgment or compensation. Although Tribal communities have only recently begun to formally document evidence regarding the effectiveness of traditional healing methods, herbs, and ceremonies, western science and evaluation can benefit from understanding indigenous practice. 

> Intergenerational and Community Trauma 

Historical Events That Disrupted Families 

The consequences of the U.S. Government's boarding school policies in the late 19th century are a prime example of 

traumatic events experienced by Tribal peoples. These policies attempted to assimilate Native Americans (commonly 

referred to as "savages" during this period) by removing thousands of Tribal children from their homes to be educated. 

Often without parental permission, Tribal children were sent to boarding schools, and parents were rarely, if ever, 

allowed to visit. Boarding schools were often administered by retired military staff whose disciplinary philosophies 

included corporal punishment. 

Native children were also adopted out of Tribal communities. One example is the Indian Adoption Project, a collaboration between the Child Welfare League of America (CWLA) and the Bureau of Indian Affairs. From 1958 to 1967, the 

CWLA provided Federal funding to member agencies and public child welfare agencies to place 395 Indian children with 

white adoptive families (Kreisher, 2002). As a result of this program and other similar programs, surveys conducted in 

1969 and 1974 found that approximately 25–35 percent of all Indian children were separated from their families and 

placed in foster homes, adoptive homes, or institutions (Unger, 1977). The passage of the Indian Child Welfare Act in 

1978 was intended to reverse these policies (“Indian Child Welfare Act,” n.d.). 

> Distrust 

In the 20th century, many policies and experiences instilled a widespread fear of evaluation throughout Indian Country. This fear further entrenched the impact of intergenerational trauma. Efforts to engage Native people in evaluation 

are often met with fearful reluctance or outright refusal. Evaluation is to this day considered invasive, particularly when 

imposed by outside funding agencies. 

Research is also met with distrust in many Native communities. This distrust of research is a direct result of a legacy of 

intrusive studies that contributed to scientific knowledge with little direct benefit to Indian communities. In some cases, 

misguided research harmed communities, exploited Native culture, and ignored the rights of communities (e.g., Barrow 

Alcohol Study; Foulks, 1989). Despite recent efforts to conduct more responsible, community-engaged, and communitydriven research, distrust of research continues to permeate indigenous communities (LaFrance & Nichols, 2010). 

Earning Trust 

Trust is earned when individuals demonstrate they are dependable. To earn the trust of Tribal communities, evaluators 

must allow time for positive experiences to occur to demonstrate trustworthiness. Evaluators may benefit from participating in a face-to-face meeting to increase interactivity, develop group cohesion, and better facilitate a dynamic and 

meaningful conversation about evaluation issues. 

Native people are highly observant. Earning their trust is often time consuming and unrewarding (e.g., providing and 

supporting lengthy discussions on evaluation topics that may or may not result in group consensus). Many Native communities use leadership styles that require almost 100 percent consensus for most decisions. Evaluators may be tempted 

not to participate in the decision-making process or believe that being an objective observer enhances their research 

ability. However, not joining with the community further demonstrates a lack of connectedness. The evaluator’s skills, 

abilities, and intent will continue to be perceived as questionable and not trustworthy. Successful non-Native evaluators 

recognize the multi-dimensional nature of Tribal communities and build relationships across a wide spectrum of community members such as Tribal government leaders, service providers, youth, parents, elders, traditional cultural leaders, 

and faith-based leaders. 

15 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
Distrust of Research and Evaluation in Tribal Communities 

Federal Termination Policy: A survey conducted by the U.S. Senate in 1943 found that residents of 

reservations were living in extreme poverty. The Bureau of Indian Affairs was found to be at fault because 

of mismanagement. By 1953, the U.S. Government developed an official termination policy to “make 

Indians within the territorial limits of the United States subject to the same laws and entitled to the same 

privileges and responsibilities as are applicable to other citizens of the United States” (House Concurrent 

Resolution 108). As a result, the Federal Government terminated recognition of 109 Tribal governments 

as sovereign nations, and Federal responsibility and jurisdiction was turned over to State governments. 

Approximately 2.5 million acres of trust land was removed from protected status and 12,000 Native 

Americans lost Tribal affiliation. (http://www.nrcprograms.org/site/PageServer?pagename=airc_hist_ 

terminationpolicy) 

Havasupai Tribe Diabetes Project (1989): In 2004, the Havasupai Tribe filed a lawsuit against Arizona Board 

of Regents and Arizona State University researchers for misuse of the Tribal members’ DNA samples. 

Although the Havasupai people consented to giving blood samples for Type II diabetes research, it was 

discovered later that blood samples and subsequent data findings were shared with other researchers for 

numerous studies across the United States on migration, inbreeding, and schizophrenia. The lawsuit cited 

concerns of lack of informed consent (specificity), violation of civil rights through mishandling of blood 

samples, unapproved use of data, and violation of medical confidentiality, along with other complaints. 

(http://genetics.ncai.org/case-study/havasupai-Tribe.cfm) 

Barrow Alcohol Study (1979): The Center for Research on the Acts of Man (subcontracted by Intersect, 

a consulting firm in Seattle, WA) conducted a survey on the use of alcohol among the Inupiat people of 

Barrow, AK, in 1979. Although the contracted research was to enable the Department of Public Safety in 

Barrow, AK, to establish effective programs for reducing traumatic deaths associated with alcohol abuse 

in the community, it was extremely difficult to establish consensus of community attitudes and values 

involving alcohol use/abuse as well as the cause of alcohol use/abuse. In 1980, Intersect and the Barrow 

Department of Public Safety prepared a press release with hopes to shock the community into action. 

The mass media sensationalized the research findings with damaging headlines such as “Alcohol Plagues 

Eskimos, and Sudden Wealth Sparks Epidemic of Alcoholism—What We Have Here is a Society of Alcoholics” 

(Foulkes, 1989). 

Outside evaluators and researchers often overly rely on certain individuals who purport to speak for the community. 

The information gathered from interviews is treated as valid for the entire community. This can create trust issues.The 

assumption that a few individuals can represent the entire voice of the community is tenuous and, depending on the 

individuals interviewed, can provide a limited perspective. Relying on only a few key voices represents a missed opportunity to establish relationships more broadly and experience the richness of the community. 

Until Tribal communities are welcomed to the table as equal partners in this journey, evaluation activities and knowledge 

gained are, at best, questionable and, at worst, further traumatizing. 

> Invasive, Imposed Evaluation 

Evaluation as a Requirement, Rather Than as a Useful Tool 

As a result of this traumatic history, evaluation in Indian Country is typically considered invasive, with Federal and State 

funding resources requiring local programs and families to assume the evaluation burden and provide often intimate information to unknown outsiders, often without any benefit to Native families, programs, or communities in the process. 

Evaluation is often regarded as a requirement rather than as a tool for addressing local questions and priorities and providing information of local use and value. Evaluation information has often been filed away without an attempt to share 

the information or use it to improve program services. 

16 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
Evaluation as Something Outsiders Do 

An additional issue when conducting evaluation in Indian Country is the lack of local skills and understanding of evaluation processes. Skilled local Native evaluators (e.g., statisticians, data entry technicians, or skilled interviewers) remain 

scarce. As a result, many Tribal communities seek out contracted evaluators from regional universities. These individuals 

typically have little understanding or awareness of the barriers 

to developing evaluation designs or conducting evaluation in 

Tribal communities. 

Because Tribal advisory councils and program directors often 

lack an understanding of evaluation processes, they view external or university-based evaluators as individuals with power or 

knowledge and believe they must conform to these outsiders’ 

opinions, ideas, and suggestions. 

Tribal colleges and local educational institutions need to develop 

educational programming to increase the availability of skilled 

evaluators and evaluation staff and improve the evaluation 

knowledge base of Tribal social service administrators. By building local capacity (“growing our own”), an evaluation workforce 

can be built that can integrate both rigorous evaluation methods 

generated from a Western scientific model and rigorous evaluation methods emanating from indigenous ways of knowing. 

Evaluation as Foreign to Indigenous Language and Ways 

Evaluation is about learning and telling a story. In order to 

share what has been learned, evaluators most often rely on the English language, which has progressively become 

more technical and increasingly uses science-based concepts often unfamiliar in Indian Country. Because English is 

typically used in reports, the language is often perceived as another barrier that contributes to the fear of evaluation. 

Evaluation language and processes must be demystified to prevent further perpetuating the community’s fears, distrust, and conflicting values and to address concern that biased judgments are being made with evaluation efforts in 

Tribal communities. 

Mentoring to Support Evaluation 

Non-Native evaluators should invest time in 

seeking mentoring opportunities that enhance 

their cultural empathy and understanding of 

cultural protocols. These reciprocal mentorship 

opportunities would, at the same time, 

provide opportunities for building evaluation 

knowledge among Tribal members. By taking 

a collaborative approach, evaluators have the 

opportunity for rich learning that can better 

prepare them to use their skills in ways that 

are culturally appropriate and rigorous. NonNative evaluators should seek out Native staff 

or community members who have the cultural 

knowledge, understanding, skills, and passion 

that can contribute to instrument review, 

data collection, analysis, interpretation, 

and dissemination. By engaging community 

partners, evaluation designs can be culturally 

and scientifically rigorous. 

> Judgmental Evaluation 

Evaluation as a Threat 

Tribal communities continue to fear that evaluation efforts will result in programs being shut down; that funding will be 

pulled; and that Tribal programs, children, and families will be found lacking and judged harshly. Evaluation is often equated with judgment. It is considered a way of providing approval or disapproval of culture-based family structures, relationships, clinical skills, services, and service delivery structures, rather than a way of providing useful information for improving services or service delivery processes. It is critical to shift from evaluation-as-judgment to evaluation-as-learning and to 

show that evaluation can be a tool that improves programs and finds better ways to serve Tribal children and families. 

Evaluation as Focusing on Deficits 

Until recently, the majority of evaluation efforts in Native communities served only to highlight the deficits apparent in 

the newly developing Tribal child welfare systems. Evaluations rarely sought to identify strengths of Tribal programs. 

Non-Native, urban-based child welfare programs now recognize the strengths of many Tribal child welfare programs and 

incorporate these strengths into their own approaches to formulate services for the entire family unit. This philosophical 

17 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
switch can be attributed to the recognition that the entire family is crucial to a child’s safety, permanency, and wellbeing. Not only are the service needs of the family important, the family is also a crucial resource that can influence the 

effectiveness of service delivery to children. 

Some Tribal child welfare programs have developed culture-based family assessments and programming to meet families’ cultural needs. These efforts have been shown to be instrumental in engaging and supporting families in Tribal child 

welfare systems. One example of this is the work of the White Earth Tribe, whose cultural assessment tool was showcased at the 2013 National Indian Child Welfare Association conference. 

Fears Related to Evaluation 

Discussions of evaluation practice in Tribal child welfare programs often include an acknowledgment of the fears engendered when evaluations are required by funding agencies. Two key types of fears are often articulated: 1) fear of being 

evaluated; and 2) fear of doing an evaluation. 

> Fear of Being Evaluated 

The fear of being evaluated can be traced to the troublesome history of evaluation in Tribal communities (Historical Context discussion above). Critical to addressing this fear is a commitment to the priorities related to relationship building, a 

commitment to creating atmosphere of trust and open collaboration among partners in the evaluation process. 

> Fear of Doing an Evaluation

The fear of doing an evaluation is based on the fact that resources and training to complete the evaluation are in short 

supply in many Tribal programs and communities and that training for non-Native evaluators to conduct culturally valid 

evaluations are equally inadequate. The bidirectional priorities included in the Roadmap help address this fear by identifying areas for building evaluation knowledge and skills. 

Priorities for the Roadmap of Evaluation in Child Welfare 

The Roadmap suggests several priorities for building a new narrative for evaluation in Tribal child welfare. These include 

Relationship Building and Knowledge and Skill Building priorities, with a priority that bridges both skill areas – “growing 

our own.” 

> Cross-Cutting Priority: Growing Our Own: Developing Policies, Establishing Clear Guidelines, and 

Buildin g Tribal Infrastructure for Evaluation 

Many Tribal nations and organizations have made great strides in recent years in taking ownership of research and evaluation within their borders; many other Tribal nations and organizations are just beginning on this journey. The path is 

not easy or clear, but persistent commitment to the importance of Tribal oversight has led to greater protection of Tribal 

members by ensuring that research and evaluation practice is ethical, culturally appropriate, and respectful. “Growing 

our own” has become a priority in many Tribal communities, with a focus on the training and career development of 

Tribal members who are becoming evaluators and researchers. 

The next generation of evaluation practice in Tribal child welfare will need to support emerging infrastructure for evaluation 

oversight and practice in Tribal communities. This support must involve Tribal research oversight bodies (e.g., institutional 

review boards, research review boards, and Tribal councils) on evaluation and research protocols and on issues of data ownership and sharing. Support for mentoring and training opportunities to emerging investigators will also be important. 

18 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
> Priorities Related to Relationship Building 

Five key priorities for relationship building have been identified. 

Relationship Building Priority #1: Evaluation Practice That Involves the Community in Determining Priorities 

Involving the community in determining evaluation priorities increases the chances that evaluation of Tribal child welfare 

programs will be successful. The evaluation must be firmly anchored in questions that are meaningful to local programs 

and community members. Evaluation imposed from the “outside” by policymakers or external evaluators may not capture questions meaningful to the local community; thus, the results of evaluation will not be useful to local programs. 

All evaluation planning should begin with intensive discussion with community program directors and program staff and 

ideally with Tribal government leaders, community members, cultural authorities, and spiritual leaders. Open discussions 

with these community stakeholders will ensure that the evaluation includes questions that directly address community 

needs and priorities. 

Evaluators and Tribal child welfare program directors jointly bear the responsibility of ensuring that community voices 

are heard when identifying questions and designing evaluations. Community stakeholders are responsible for representing their communities in these discussions, thinking broadly about community needs and resources, and speaking out to 

ensure that the evaluation strategies are responsive to community priorities. 

Relationship Building Priority #2: Dialogue with Community to Anchor Evaluation Within Cultural & Ethical 

Practices 

Not only is it critical that the questions in an evaluation emanate from community priorities, it is equally important that 

the process of evaluation be grounded in cultural practices and proceed ethically within this context. Conversations with 

community stakeholders are essential to ensuring such grounding. Only by getting to know the community and the 

culture can evaluators create an evaluation plan that honors cultural ways and respects local values. 

Relationship Building Priority #3: Clear Guidelines and Examples of Evaluation Components Important to the 

Community 

An important way to help program directors and staff see the benefit of program evaluation is to provide examples of successful and informative evaluations in other communities. These examples should demonstrate how local questions about 

program practice can be answered using evaluation and how the answers can, in turn, inform program improvement. 

Such examples also serve an important role for Tribal evaluators, evaluation partners, and others involved in the evaluation 

enterprise, demonstrating the effective integration of cultural and scientific rigor in addressing relevant local questions. 

Relationship Building Priority #4: Clear Guidelines and Examples of Required Evaluation Components Tied to 

Funding 

Too often evaluation requirements attached to program funding are a mystery to program directors and staff in Tribal communities. An important goal, therefore, is to clearly explain why evaluation is required, why it is important, and how it can 

inform program improvement. Evaluators should carefully articulate what the community stands to gain from evaluation. 

Evaluators must also clearly explain the structure of the evaluation. Critical components should be clearly specified and 

reinforced with practical examples. 

Relationship Building Priority #5: Transparency in Evaluation and Translation for Community 

The importance of transparency in evaluation is difficult to overstate. Most AI/AN communities have experiences with 

evaluation and research that have been insensitive to local needs and have, in some cases, done direct harm to communities. Many Tribal members are understandably skeptical of evaluation and research and therefore resist efforts to gather 

information in their communities. Effective and appropriate use of evaluation requires open and candid conversation with 

19 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
community stakeholders about the purpose, methods, and results of evaluation. Evaluators must work hard to explain 

each component in plain English. They must also diligently to ensure that community members understand what is being 

done and have opportunities to ask questions and provide input into the entire process. 

> Priorities Related to Knowledge and Skill Building 

The workgroup identified six key priorities for knowledge and skill building. 

Knowledge and Skill Building Priority #1: Cultural Protocols for Community-Engaged, Comprehensive Evaluation 

Rigorous evaluation in Tribal child welfare programs will ultimately depend on the integration of rigorous scientific 

methods with rigorous cultural adaptation. Many models exist for community-engaged evaluation and research, such 

as Community-Based Participatory Research (CBPR) (Cashman et al., 2008). Other examples include models like Tribal 

Participatory Research that specifically highlight special practices when working with Tribal communities (Fisher & Ball, 

2003). Standard protocols for evaluation practice in Tribal communities, with guidelines for adapting work within specific Tribal contexts, form the foundation of culturally and scientifically rigorous evaluations. 

Knowledge and Skill Building Priority #2: Mentoring and Technical Assistance for Evaluation Design and 

Implementation 

Tribal child welfare programs are often required to evaluate programs, but are not always designed to do so. Evaluation 

practice comprises a set of skills that requires training, particularly in the complex milieu of Tribal child welfare programs. 

Local program staff members are unlikely to be trained evaluators or too often find requirements confusing and unnecessary. Nevertheless, these are the very people on whom rigorous evaluation depends; without staff engagement in the 

process, evaluation is unlikely to be successful. Thus, it is critical that local staff receive appropriate and timely technical 

assistance during the grant award phase to identify questions that can be answered through evaluation, to share their 

expertise on the local culture and context, and to guide culturally rigorous evaluation protocols. 

Knowledge and Skill Building Priority #3: Guidelines for Selecting, Adapting, Developing, Administering, & 

Interpreting Evaluation Instruments 

A rigorous evaluation grounded in locally relevant questions and built around a scientifically and culturally rigorous 

design will produce meaningful results if the outcomes of interest are well measured. Good data come from good measures. However, even a well-designed evaluation can produce meaningless results if the outcomes are poorly measured. 

Unfortunately, there is a great deal of uncertainty about how to identify appropriate measurement instruments for use 

with children and families in AI/AN communities. Although numerous instruments are available to assess most outcomes of interest in child welfare, few have been established as appropriate for use with Tribal populations. Many standard measures that are created and normed for non-Native populations are likely to work well for American Indians and 

Alaska Natives, but others are likely to provide misleading information unless they are adapted to reflect cultural differences. It is important to provide information about instruments that have been used with American Indians and Alaska 

Natives and how well they have performed for these populations. It is also important important to provide guidelines for 

evaluators on when to adapt standard measures, how to determine whether an adapted measure is reliable and valid, 

and how to proceed when no appropriate measure can be found. 

Knowledge and Skill Building Priority #4: Storytelling to Support Staff and Community Interpretation of 

Evaluation Data 

In many Tribal communities, information is communicated through storytelling, reflecting rich oral traditions. Evaluations 

are opportunities for local communities to express their concerns, to seek answers to questions of importance to them, 

and to create a story of what is happening in child welfare in their community. Participants at all levels – children, fami20 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
lies, providers, program staff, evaluators – need to be able to share their stories. The data are one component of the stories. The information from the evaluation becomes part of the local story and Tribal legacy and should be respected and 

readily apparent during the human subject review process, especially regarding participant confidentiality and privacy. 

Knowledge and Skill Building Priority #5: Training in Use, Interpretation, and Storage of Data 

Gathering data is a critical step in any evaluation. Too often, data are collected but not fully used. Thinking strategically 

about how data can be used to help guide Tribal child welfare practice is important. Local staff must receive training in 

how to use, interpret, and store data that have been collected. Although it is unrealistic to expect that child welfare staff 

will become proficient in complex data analyses, it is possible to better prepare local evaluators to fully use data that are 

collected as a part of evaluations. 

Knowledge and Skill Building Priority #6: Consolidation & Coordination of Data Across Community Programs 

In most Tribal communities, while data are being collected for evaluation of Tribal child welfare programs, other data 

are being collected for evaluation of other Tribal programs and services. In some cases, this may lead to duplication of 

effort. In other cases, different programs may be collecting data that, while not the same, might be usefully combined 

to provide more complete information. Efficiency could be created by consolidating and coordinating data collection 

across programs within communities and by sharing data in ways that can add to the richness of results. In addition, 

sharing data collected within a single community across federally funded evaluations could be very useful to Tribal 

programs. Identifying these potential sources of data and how sharing data might reduce the burden of data collection 

within programs should be a priority. Furthermore, while many Tribal nations have begun to establish Tribal archives, it 

is important that Tribal child welfare, Tribal social and family services, and Tribal health providers begin the process of 

establishing a centralized data collection center that would effectively address many of the issues discussed above while 

providing storage and archival services. This would be an effective approach to improving the knowledge base in Tribal 

communities, while facilitating Tribal ownership of the data and the planning opportunities it represents – an important 

development of Tribal infrastructure. 

> Secondary Priorities 

Four additional priorities identified as important to building a new narrative for Tribal evaluation, but not depicted on the 

Roadmap are described below. 

Secondary Priority #1: Protocols for Tribal Approval of Evaluation Research 

Tribal oversight of research and evaluation varies dramatically across Tribal nations and organizations. Some have federally recognized IRBs that govern human subjects research within Tribal boundaries. Others have RRBs that work with 

recognized IRBs at universities, Tribal colleges, the Indian Health Service, or other institutions to provide human subjects 

protections for Tribal members. Still others have processes through which Tribal councils or council committees review 

proposed research and provide Tribal resolutions in support of approved studies. There is constant change as Tribal nations and organizations find better ways to oversee research with their peoples. Understanding the review requirements 

of the Tribe in which a study is being conducted, knowing how to navigate the review process, and being respectful of 

Tribal requirements while conducting evaluation or research and disseminating findings are fundamental to appropriate 

evaluation practice in Tribal communities. 

Secondary Priority #2: Cultural Orientation Mentorship and Models 

Many priorities outlined in the Roadmap address the importance of providing scientific training and mentorship to Tribal 

members working in child welfare and evaluators of those programs; equally important, however, are providing cultural training and mentorship to members of the evaluation and research community who are learning to work in partnership with 

Tribal child welfare programs. Some training can be at a global level - preparing evaluators and researchers to work with a 

variety of Tribal communities and making them aware of the importance of general principles of working in Tribal contexts. 

21 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
Some training, however, must be within the context of
 

the particular Tribal community in which an evaluation 

is taking place.
 

Secondary Priority #3: Building Capacity for 

Teaching Evaluation in Tribal Colleges 

Tribal colleges are uniquely suited to support the goal 

of “growing our own,” providing educational opportunities for 

AI/AN individuals to gain the skills needed to lead 

evaluations within their home communities. It will 

be important to build the capacity of Tribal colleges 

to meet this need, including consulting with Tribal 

college faculty and administrators in the creation of 

evaluation curricula, degree programs, and specialized 

evaluation training workshops. 

Secondary Priority #4: Supporting the Education 

of a Tribal Evaluation and Research Workforce 

A recurring theme throughout the Roadmap is the 

process of helping Tribal members gain the skills and 

experience to lead culturally and scientifically rigorous evaluation that addresses questions important to 

Tribal priorities for child welfare. Figure 1 depicts the 

process of building a Tribal evaluation and research 

workforce. As the figure illustrates, equipping Tribal members to be leaders of evaluation and research activities, both 

within their communities and beyond, will require extensive commitment to training future leaders.  The process begins 

when Tribal members are exposed to research and evaluation and have opportunities. This process requires substantial 

commitment from individual Tribal members seeking to become researchers and from mentors and research training 

programs that support the development of a qualified Tribal research workforce. 

Figure 1: Research Training "Pipeline"
 

Exploration 

(involvement with 

evaluation & research) 

Commitment 

Practice 

Leadership 

Exposure 

Curiosity 

Practice 

Practice 

Specialization 

(advanced education/ 

graduate school) 

Training 

(basic education/ 

undergraduate) 

22 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
> Building a New Narrative 

The purpose of addressing the priorities that compose the Roadmap is to foster evaluation that transforms current 

practice by achieving six key goals: 

1 Collaborative, Culturally Responsive Evaluation

2 System Improvement

3 Locally Guided Questions, Data, and Insight

4 Meaningful Analyses

5 Bidirectional Learning

6 AI/AN Evaluation Skills

Achieving these goals will add to the richness and rigor of what can be learned through evaluation. These goals involve 

community engagement, identification of meaningful questions, appropriate interpretation of data, and awareness of 

spiritual and cultural meaning. The assumption that a variable can be isolated and its impact quantified and extracted, 

independent of relationships surrounding it, is anathema to Tribal concepts of interconnectedness and holistic world 

views. Indigenous evaluation practice embraces holistic world views and balances cultural and scientific rigor to support 

meaningful evaluation. 

Collaborative, Culturally Responsive Evaluation 

Most Importantly, an evaluation should be designed to benefit the community and to answer questions that are locally 

relevant and culturally resonant. Satisfying the requirements of program funders is critical, but should not take precedence over the primary goal of gaining information that can be directly used by the community. Evaluation should support decision making about whether a particular program is a good fit for the community, whether it is effective in the 

community, and specific steps for program improvement. 

Evaluation should be tailored to the cultural context of the community and responsive to cultural ways of life, including 

cultural values for parenting and child well-being. 

Indigenous ways of knowing should shape evaluation plans, complementing scientific methods and anchoring them 

within local culture. They should inform and guide all steps of the evaluation process – from conceptualization of questions, to selection of design and measurement strategies, to communication of findings. 

Cultural ways of knowing should be joined with the best of scientific practices to create evaluation strategies that are 

culturally and scientifically rigorous and that can provide valid, comprehensive, and useful information to support ongoing improvement in Tribal child welfare programs and healthier outcomes for Native children and families. Evaluation 

practice should be built on the wealth of resources available across Tribal programs, evaluation partners, Tribal governments, and funding agencies. 

System Improvement 

Evaluation should inform practice, program, and system improvement, providing information to answer questions that 

local program directors and staff have about how to better serve the children and families in their communities. It 

should focus on improving systems by identifying what works well in existing practice, what does not work as well, and 

how it can be improved. The goal is not to grade or critique systems, but rather to provide useful information for ongoing system improvement. 

23 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
 

 

 

Locally Guided Questions, Data, and Insight 

Evaluation should be grounded in local issues, answering questions that come from program directors, staff members, 

families, and community members. The value of an evaluation is directly proportional to the extent that it is linked to 

meaningful local issues. 

Local program directors and staff know best what questions to ask and what information they need to improve their 

programs. Therefore, their questions should drive evaluation plans. Evaluators should work closely with program directors and staffs to identify key questions, examine relevant local current and historical data, develop appropriate questions, and design an appropriate evaluation plan. 

Evaluators should continue working with program directors and staff when reviewing and interpreting the collected data 

and sharing the story of the evaluation with other stakeholders. Local insight is critical to correctly interpreting evaluation 

findings, and local storytelling is critical to sharing results in a meaningful way. 

In essence, evaluation practice should reflect authentic CBPR, as discussed in Knowledge and Skill Building Priority #1: 

Cultural Protocols for Community-Engaged, Comprehensive Evaluation. 

Meaningful Analyses 

Analyses should be rooted firmly in the questions identified in collaborative discussions with communities and in the 

questions asked by funding sources. For example, if a Tribal community is interested in exploring whether a cultural 

adaptation to a program would improve outcomes for children and families, the evaluator should work with community 

stakeholders to: 

4 Provide analyses of outcome data that can directly answer the question of how the adaptation affects outcomes (e.g., 


statistical tests of group differences)
 

3 Design an evaluation plan to allow clear comparison of the effects of the cultural adaptation on those outcomes (e.g., one

group with standard program, a second [comparison] group with cultural adaptation) 

2 Select or create sensitive measures of those outcomes (e.g., program sessions attended, score on a parenting scale) 

1 Identify the exact outcomes that are anticipated to be affected by the cultural component (e.g., engagement with the

program, parenting self-efficacy) 

Analyses should be designed and presented in ways that are meaningful and interpretable and that provide opportunities to directly inform system improvement. In planning any evaluation, evaluators should consider how to disseminate 

the results to program staff and others in the community in ways that can fully inform ongoing program efforts. Scientific journal publications rarely reach the on-the-ground audience that can benefit most from the knowledge gained in 

evaluation studies; when they do, they are often written in technical jargon that is too complex for those without extensive training. If findings are expected to impact practice, evaluators must disseminate them in ways that resonate with 

practitioners. This includes consideration of both the venues in which findings are reported (e.g., program staff meetings, newsletters distributed through early childhood networks) and the format of presentation (e.g., narrative and/ or 

storytelling formats consistent with oral traditions in Tribal communities rather than dense scientific journal formats; see 

Knowledge and Skill Building Priority #4: Storytelling to Support Staff and Community Interpretation of Evaluation Data). 

Evaluators should consult closely with Tribal community partners to determine both where and how to most effectively 

present findings. 

24 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
Bidirectional Learning 

Bidirectional learning is inherent in the Roadmap. For example, program directors and staff should have opportunities 

to learn about the evaluation process and develop skills to participate in and inform local evaluations. They will benefit 

from education about the scientific process of evaluation and will be better equipped to contribute actively to rigorous 

and informative evaluations. Program directors and staff also have opportunities to educate outside evaluators and other 

partners (e.g., policymakers, universities) about their programs, their cultures, and their communities and to develop 

cultural protocols for appropriate local evaluation practice. 

Outside evaluators should have opportunities to learn about cultural ways (e.g., ways of knowing, ways of parenting, 

child development techniques) and how to build evaluations that use rigorous cultural methods (e.g., guidelines for 

building strong and equitable community–evaluator partnerships, templates for incorporating cultural protocols in data 

collection procedures, strategies for disseminating findings in settings and formats that are likely to be most effective in 

Tribal contexts). They also have many opportunities to help Tribal partners build skills in evaluation and to provide mentorship in rigorous scientific methods. 

AI/AN Evaluation Skills 

Although AI/AN evaluation skills could be subsumed under the goal of bidirectional learning, they are emphasized here 

because of their central importance in improving evaluation practice in Tribal child welfare programs. The importance of 

an evaluation workforce that embodies both cultural rigor and scientific rigor is high. “Growing our own,” or creating an 

evaluation workforce within Tribal communities that has the best blend of experiences and skills to support an evaluation that is highly sensitive to cultural contexts and values and committed to scientific principles ensures the credibility 

and usefulness of evaluation products. The Roadmap thus articulates the centrality of this goal as evaluation practice 

moves forward. 

25 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
> Envisioning the New Narrative 

It is difficult to summarize the breadth and depth of the new narrative the workgroup envisions, but the following chart provides examples of how the narrative of evaluation should be changed. 

Old Narrative New Narrative 

Evaluation is mandated by a funding agency (e.g., national 

cross-site evaluations are often a grant requirement) 

Funding-agency-mandated evaluations generate dialogue to 

develop an evaluation approach by and for the community, 

to answer local questions, and to improve local practice 

Funders’ evaluation requirements do not consider the unique 

contexts of Tribal programs 

Funders understand the importance of cultural rigor as an 

integral component of rigorous evaluation 

Program staff and local evaluators do what has to be done 

to satisfy external funders 

Program staff and local evaluators conduct an evaluation 

that serves local program needs and produces knowledge to 

benefit other communities 

An evaluation plan is brought into the community to be 

implemented 

An evaluation plan is created through consultation with local 

program staff and community stakeholders 

Evaluation is conducted by experts with scientific training 

but little or no training or awareness of Tribal history, 

world views, values, cultural protocols, or connection to 

community 

Evaluation is conducted by evaluation experts who blend 

scientific expertise, cultural expertise, and connection to 

community 

Program directors rely on scientific experts to direct 

evaluations in their communities 

Program directors rely on their training in scientific methods 

and their cultural expertise to partner with evaluators who 

have expertise in Tribal settings; together they create teamled evaluations built on mutual respect 

Tribal communities are required to use evaluation 

instruments that are burdensome and have not been 

normed for their community 

Tribal communities are involved in a collaborative process to 

review, pilot, and select evaluation instruments 

Data are collected using protocols developed for other 

populations, generally not sensitive to Tribal world views, 

values, or communication styles 

Data collection is tailored to local ways, including respect for 

oral tradition and cultural protocols 

Conclusions are generated by outside evaluators who 

interpret data out of context, do not recognize Tribal 

trauma history, and receive little or no input from local 

cultural experts 

Conclusions and interpretations are firmly grounded within 

the cultural context — with program staff, cultural experts, 

and others in the community being engaged to interpret 

findings 

Evaluation findings are reported to funders but never fully 

disseminated to local programs and communities 

Evaluation planning includes dissemination as a priority 

Reports are created in technical language that is often 

inaccessible to community stakeholders 

Evaluation narratives are created to tell the stories of 

program outcomes to Tribal governments and their 

community stakeholders in language that is clear and easy 

to understand 

26 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
> Stakeholders 

The workgroup identified 11 groups as playing important roles in evaluation of Tribal child welfare programs. The first 

four are the primary stakeholders – those who are at the heart of evaluation practice in Tribal programs and for whom 

the Roadmap is designed. The remaining seven are supportive stakeholders who provide support, guidance, and context 

for the evaluation process; they often collaborate with primary stakeholders. 

For each stakeholder group, the workgroup first provides an overview of who are included in the group and why they 

are important stakeholders. Next, for each group, the workgroup provides two depictions of what it sees as different 

layers of the Roadmap: one highlighting what the group can contribute to meeting Roadmap priorities, and the second 

highlighting how meeting Roadmap priorities can benefit the group’s work. 

Primary Stakeholders 

> Program Directors 

Who They Are 

Program directors oversee child welfare programs and are responsible for running the programs. 

Why They Are Important Stakeholders 

Program directors are responsible for ensuring that child welfare programs in their communities operate effectively to 

serve children and families. They are central to the evaluation process because they are the decision-makers, and evaluation plays a critical role in informing their decisions. They are responsible for identifying or responding to key evaluation 

questions, hiring evaluators, overseeing the evaluation process, and using evaluation results to inform programmatic 

planning. Although program directors may not have specific training in evaluation or be immersed in the details of evaluation planning and process, they are the first-line consumers of evaluation findings. 

27 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
What They Can Contribute to Meeting Roadmap Priorities 

Program directors play a pivotal role in addressing Roadmap priorities by sharing their program needs and priorities, 

educating evaluators about the community and cultural context in which their programs are embedded, and serving 

as key liaisons to program staff in interpreting evaluation findings and determining how they can be used to improve 

program practice. 

The figure below highlights the five evaluation practice priorities to which program directors significantly contribute. 

Values

Indigenous Ways of Knowing | Respect for Tribal Sovereignty 

Strengths Focus | Cultural and Scientific Rigor | Community Engagement 

Ethical Practices | Knowledge Sharing 

CO-CREATING COLLABORATIVE & EFFECTIVE EVALUATION TO IMPROVE TRIBAL CHILD WELFARE PROGRAMS

Tribal 

Evaluators 

Evaluation 

Partners 

Program 

Staff 

Policy Makers 

Community 

Members 

Cultural 

Authorities & 

Spiritual Leaders 

Tribal Colleges 

Universities 

Professional 

Organizations 

Relationship Building Knowledge & Skill Building 

Evaluation practice that involves the 

community in determining priorities 

Cultural protocols for communityengaged, comprehensive evaluation 

Mentoring for evaluation design 

& implementation 

Guidelines for selecting, adapting, 

developing, administering, & 

interpreting evaluation instruments 

Storytelling to support staff & 

community interpretation of 

evaluation data 

Consolidation & coordination of data 

across community programs 

Building a 

New Narrative 

Collaborative, Culturally 

Responsive Evaluation 

System Improvement 

Locally Guided Questions, 

Data, & Insight 

Meaningful Analyses 

Bidirectional Learning

 AI/AN Evaluation Skills 

Indigenous world views undervalued 

Intergenerational and community trauma 

Invasive, imposed evaluation 

Judgmental evaluation 

Distrust 

Historical Context

Program 

Directors 

Dialogue with community to 

anchor evaluation within cultural & 

ethical practices 

Clear guidelines & examples of 

evaluation components important to 

the community 

Clear guidelines & examples of 

required evaluation components tied 

to funding 

Transparency in evaluation & 

translation for community 

Developing policies, establishing clear guidelines, 

& building Tribal infrastructure for evaluation 

Tribal 

Government 

Leaders 

Training in use, interpretation, 

& storage of data 

28 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
How Meeting Roadmap Priorities Can Benefit Their Work 

Addressing Roadmap priorities can help program directors and all stakeholders gain a better understanding of the 

overall goals of evaluation of Tribal child welfare programs and of the various components that need to be in place to 

ensure effective evaluation and use of evaluation findings. Specifically, there are three Relationship Building priorities 

and two Knowledge & Skill Building priorities that will benefit program directors. 

Values

Indigenous Ways of Knowing | Respect for Tribal Sovereignty 

Strengths Focus | Cultural and Scientific Rigor | Community Engagement 

Ethical Practices | Knowledge Sharing 

CO-CREATING COLLABORATIVE & EFFECTIVE EVALUATION TO IMPROVE TRIBAL CHILD WELFARE PROGRAMS

Evaluation 

Partners 

Program 

Staff 

Policy Makers 

Community 

Members 

Cultural 

Authorities & 

Spiritual Leaders 

Tribal Colleges 

Universities 

Professional 

Organizations 

Relationship Building Knowledge & Skill Building 

Evaluation practice that involves the 

community in determining priorities 

Cultural protocols for communityengaged, comprehensive evaluation 

Mentoring for evaluation design 

& implementation 

Guidelines for selecting, adapting, 

developing, administering, & 

interpreting evaluation instruments 

Building a 

New Narrative 

Collaborative, Culturally 

Responsive Evaluation 

System Improvement 

Locally Guided Questions, 

Data, & Insight 

Meaningful Analyses 

Bidirectional Learning

 AI/AN Evaluation Skills 

Program 

Directors 

Tribal 

Evaluators 

Storytelling to support staff & 

community interpretation of 

evaluation data 

Consolidation & coordination of data 

across community programs 

Indigenous world views undervalued 

Intergenerational and community trauma 

Invasive, imposed evaluation 

Judgmental evaluation 

Distrust 

Historical Context

Dialogue with community to 

anchor evaluation within cultural & 

ethical practices 

Clear guidelines & examples of 

evaluation components important to 

the community 

Clear guidelines & examples of 

required evaluation components tied 

to funding 

Transparency in evaluation & 

translation for community 

Developing policies, establishing clear guidelines, 

& building Tribal infrastructure for evaluation 

Tribal 

Government 

Leaders 

Training in use, interpretation, 

& storage of data 

29 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
> Tribal Evaluators 

Who They Are 

Tribal evaluators are directly involved in the process of evaluating Tribal child welfare programs. They are involved in the 

day-to-day operation of evaluation activities. Some Tribal evaluators may lead the evaluation activities in their program; 

others may work collaboratively with or under the guidance of outside evaluation partners to coordinate activities. 

Why They Are Important Stakeholders 

Tribal evaluators play a critical role in designing and implementing evaluation plans in their communities. Their cultural 

knowledge as members of the community and scientific knowledge as evaluators offers an important blend of 

perspectives that is critical to culturally and scientifically rigorous evaluation. 

What They Can Contribute to Meeting Roadmap Priorities 

These stakeholders are central to addressing two Relationship-Building and three Knowledge & Skill Building priorities, 

and one priority that cuts across these contexts. 

Values

Indigenous Ways of Knowing | Respect for Tribal Sovereignty 

Strengths Focus | Cultural and Scientific Rigor | Community Engagement 

Ethical Practices | Knowledge Sharing 

CO-CREATING COLLABORATIVE & EFFECTIVE EVALUATION TO IMPROVE TRIBAL CHILD WELFARE PROGRAMS

Evaluation 

Partners 

Program 

Staff 

Policy Makers 

Community 

Members 

Cultural 

Authorities & 

Spiritual Leaders 

Tribal Colleges 

Universities 

Professional 

Organizations 

Relationship Building Knowledge & Skill Building 

Evaluation practice that involves the 

community in determining priorities 

Cultural protocols for communityengaged, comprehensive evaluation 

Mentoring for evaluation design 

& implementation 

Guidelines for selecting, adapting, 

developing, administering, & 

interpreting evaluation instruments 

Storytelling to support staff & 

community interpretation of 

evaluation data 

Consolidation & coordination of data 

across community programs 

Building a 

New Narrative 

Collaborative, Culturally 

Responsive Evaluation 

System Improvement 

Locally Guided Questions, 

Data, & Insight 

Meaningful Analyses 

Bidirectional Learning

 AI/AN Evaluation Skills 

Program 

Directors 

Indigenous world views undervalued 

Intergenerational and community trauma 

Invasive, imposed evaluation 

Judgmental evaluation 

Distrust 

Historical Context

Tribal 

Evaluators 

Dialogue with community to 

anchor evaluation within cultural & 

ethical practices 

Clear guidelines & examples of 

evaluation components important to 

the community 

Clear guidelines & examples of 

required evaluation components tied 

to funding 

Transparency in evaluation & 

translation for community 

Developing policies, establishing clear guidelines, 

& building Tribal infrastructure for evaluation 

Training in use, interpretation, 

& storage of data 

Tribal 

Government 

Leaders 

30 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
How Meeting Roadmap Priorities Can Benefit Their Work 

The Roadmap can help Tribal evaluators better understand the contexts and challenges of evaluation of Tribal child 

welfare programs and the components necessary to ensure effective evaluation and use of evaluation findings. Specific 

resources linked to the Roadmap can provide useful guidance in navigating evaluation planning and process. Tribal 

evaluators, like program directors, directly benefit from clear guidelines and increased transparency in evaluation. In 

addition, all Knowledge & Skill Building priorities and the cross-cutting priority hold promise for supporting the work 

of Tribal evaluators. Many in Tribal communities emphasize the importance of “growing our own”; addressing the 

Roadmap priorities for Tribal evaluators can help move in this direction. 

Values

Indigenous Ways of Knowing | Respect for Tribal Sovereignty 

Strengths Focus | Cultural and Scientific Rigor | Community Engagement 

Ethical Practices | Knowledge Sharing 

CO-CREATING COLLABORATIVE & EFFECTIVE EVALUATION TO IMPROVE TRIBAL CHILD WELFARE PROGRAMS

Evaluation 

Partners 

Program 

Staff 

Policy Makers 

Community 

Members 

Cultural 

Authorities & 

Spiritual Leaders 

Tribal Colleges 

Universities 

Professional 

Organizations 

Relationship Building Knowledge & Skill Building 

Evaluation practice that involves the 

community in determining priorities Building a 

New Narrative 

Collaborative, Culturally 

Responsive Evaluation 

System Improvement 

Locally Guided Questions, 

Data, & Insight 

Meaningful Analyses 

Bidirectional Learning

 AI/AN Evaluation Skills 

Program 

Directors 

Tribal 

Evaluators 

Cultural protocols for communityengaged, comprehensive evaluation 

Storytelling to support staff & 

community interpretation of 

evaluation data 

Consolidation & coordination of data 

across community programs 

Mentoring for evaluation design 

& implementation 

Guidelines for selecting, adapting, 

developing, administering, & 

interpreting evaluation instruments 

Indigenous world views undervalued 

Intergenerational and community trauma 

Invasive, imposed evaluation 

Judgmental evaluation 

Distrust 

Historical Context

Dialogue with community to 

anchor evaluation within cultural & 

ethical practices 

Clear guidelines & examples of 

evaluation components important to 

the community 

Clear guidelines & examples of 

required evaluation components tied 

to funding 

Transparency in evaluation & 

translation for community 

Developing policies, establishing clear guidelines, 

& building Tribal infrastructure for evaluation 

Tribal 

Government 

Leaders 

Training in use, interpretation, 

& storage of data 

31 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
> Evaluation Partners 

Who They Are 

Evaluation partners provide evaluation services to Tribal child welfare programs. They are involved in the planning and 

design of evaluations and oversee the day-to-day operation of evaluation activities. 

Why They Are Important Stakeholders 

Evaluation partners play a critical role in designing and implementing evaluation plans, working in partnership with 

Tribal communities. Some evaluation partners have extensive experience working in Tribal communities and with Tribal 

programs, whereas others have limited or no experience in Tribal contexts. The effectiveness of evaluation partners 

depends largely on their ability to understand the cultural contexts in which they work and to adapt evaluation methods 

to provide culturally and scientifically rigorous evaluation plans to be implemented within these contexts. 

What They Can Contribute to Meeting Roadmap Priorities 

Evaluation partners play an important role in addressing every Relationship Building and Knowledge & Skill Building 

Priority and contribute to almost all priorities identified in the Roadmap. These stakeholders can provide important 

guidance to Tribal partners, work with them to establish culturally appropriate protocols for evaluation, and serve as 

liaisons and translators between other stakeholders and program directors and staff. 

Values

Indigenous Ways of Knowing | Respect for Tribal Sovereignty 

Strengths Focus | Cultural and Scientific Rigor | Community Engagement 

Ethical Practices | Knowledge Sharing 

CO-CREATING COLLABORATIVE & EFFECTIVE EVALUATION TO IMPROVE TRIBAL CHILD WELFARE PROGRAMS

Evaluation 

Partners 

Program 

Staff 

Policy Makers 

Community 

Members 

Cultural 

Authorities & 

Spiritual Leaders 

Tribal Colleges 

Universities 

Professional 

Organizations 

Relationship Building Knowledge & Skill Building 

Evaluation practice that involves the 

community in determining priorities Building a 

New Narrative 

Collaborative, Culturally 

Responsive Evaluation 

System Improvement 

Locally Guided Questions, 

Data, & Insight 

Meaningful Analyses 

Bidirectional Learning

 AI/AN Evaluation Skills 

Program 

Directors 

Tribal 

Evaluators 

Cultural protocols for communityengaged, comprehensive evaluation 

Storytelling to support staff & 

community interpretation of 

evaluation data 

Consolidation & coordination of data 

across community programs 

Mentoring for evaluation design 

& implementation 

Guidelines for selecting, adapting, 

developing, administering, & 

interpreting evaluation instruments 

Indigenous world views undervalued 

Intergenerational and community trauma 

Invasive, imposed evaluation 

Judgmental evaluation 

Distrust 

Historical Context

Dialogue with community to 

anchor evaluation within cultural & 

ethical practices 

Clear guidelines & examples of 

evaluation components important to 

the community 

Clear guidelines & examples of 

required evaluation components tied 

to funding 

Transparency in evaluation & 

translation for community 

Developing policies, establishing clear guidelines, 

& building Tribal infrastructure for evaluation 

Tribal 

Government 

Leaders 

Training in use, interpretation, 

& storage of data 

32 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
How Meeting Roadmap Priorities Can Benefit Their Work 

Evaluation partners benefit from many specific resources linked to the Roadmap, particularly those for navigating 

work with sovereign nations, building relationships with Tribal partners, and selecting and adapting culturally appropriate measures and designs. Just as these stakeholders can contribute to progress on all the identified priorities in the 

Roadmap, they can also benefit from the work of other stakeholders in addressing these priorities. For example, while 

evaluation partners can provide mentoring for evaluation design and implementation to Tribal evaluators, they can 

receive mentoring from university partners, professional organizations, Tribal colleges, and policymakers. Just as they can 

contribute to dialogue with community to anchor evaluation within cultural and ethical practices, they can gain a great 

deal from these conversations to improve their own evaluation strategies. 

Values

Indigenous Ways of Knowing | Respect for Tribal Sovereignty 

Strengths Focus | Cultural and Scientific Rigor | Community Engagement 

Ethical Practices | Knowledge Sharing 

CO-CREATING COLLABORATIVE & EFFECTIVE EVALUATION TO IMPROVE TRIBAL CHILD WELFARE PROGRAMS

Evaluation 

Partners 

Program 

Staff 

Policy Makers 

Community 

Members 

Cultural 

Authorities & 

Spiritual Leaders 

Tribal Colleges 

Universities 

Professional 

Organizations 

Relationship Building Knowledge & Skill Building 

Building a 

New Narrative 

Collaborative, Culturally 

Responsive Evaluation 

System Improvement 

Locally Guided Questions, 

Data, & Insight 

Meaningful Analyses 

Bidirectional Learning

 AI/AN Evaluation Skills 

Program 

Directors 

Tribal 

Evaluators 

Cultural protocols for communityengaged, comprehensive evaluation 

Storytelling to support staff & 

community interpretation of 

evaluation data 

Consolidation & coordination of data 

across community programs 

Mentoring for evaluation design 

& implementation 

Guidelines for selecting, adapting, 

developing, administering, & 

interpreting evaluation instruments 

Evaluation practice that involves the 

community in determining priorities 

Indigenous world views undervalued 

Intergenerational and community trauma 

Invasive, imposed evaluation 

Judgmental evaluation 

Distrust 

Historical Context

Dialogue with community to 

anchor evaluation within cultural & 

ethical practices 

Clear guidelines & examples of 

evaluation components important to 

the community 

Clear guidelines & examples of 

required evaluation components tied 

to funding 

Transparency in evaluation & 

translation for community 

Developing policies, establishing clear guidelines, 

& building Tribal infrastructure for evaluation 

Tribal 

Government 

Leaders 

Training in use, interpretation, 

& storage of data 

33 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
> Program Staff 

Who They Are 

Program staff members work in Tribal child welfare programs. 

Why They Are Important Stakeholders 

Program staff members interact with children and families daily. Their work is evaluated when Tribal child welfare programs are evaluated. Program staff members are central to the evaluation process because they carry out the program 

components and are often involved in gathering evaluation information (e.g., completing questionnaires, collecting 

information from children and families). The results of the evaluation can directly impact the way they do their work. The 

goal of a program evaluation is to provide program staff members with valuable information that can help them improve 

the services they deliver; they, along with program directors, are first-line consumers of evaluation findings. 

What They Can Contribute to Meeting Roadmap Priorities 

Program staff members play a critical role in conversations with evaluators. They educate them about program priorities, 

key questions for evaluations to address, and cultural practices that need to be considered and incorporated into evaluation tools and designs. 

Values

Indigenous Ways of Knowing | Respect for Tribal Sovereignty 

Strengths Focus | Cultural and Scientific Rigor | Community Engagement 

Ethical Practices | Knowledge Sharing 

CO-CREATING COLLABORATIVE & EFFECTIVE EVALUATION TO IMPROVE TRIBAL CHILD WELFARE PROGRAMS

Policy Makers 

Community 

Members 

Cultural 

Authorities & 

Spiritual Leaders 

Tribal Colleges 

Universities 

Professional 

Organizations 

Relationship Building Knowledge & Skill Building 

Building a 

New Narrative 

Collaborative, Culturally 

Responsive Evaluation 

System Improvement 

Locally Guided Questions, 

Data, & Insight 

Meaningful Analyses 

Bidirectional Learning

 AI/AN Evaluation Skills 

Program 

Directors 

Tribal 

Evaluators 

Cultural protocols for communityengaged, comprehensive evaluation 

Storytelling to support staff & 

community interpretation of 

evaluation data 

Consolidation & coordination of data 

across community programs 

Mentoring for evaluation design 

& implementation 

Guidelines for selecting, adapting, 

developing, administering, & 

interpreting evaluation instruments 

Evaluation practice that involves the 

community in determining priorities 

Evaluation 

Partners 

Program 

Staff 

Indigenous world views undervalued 

Intergenerational and community trauma 

Invasive, imposed evaluation 

Judgmental evaluation 

Distrust 

Historical Context

Dialogue with community to 

anchor evaluation within cultural & 

ethical practices 

Clear guidelines & examples of 

evaluation components important to 

the community 

Clear guidelines & examples of 

required evaluation components tied 

to funding 

Transparency in evaluation & 

translation for community 

Developing policies, establishing clear guidelines, 

& building Tribal infrastructure for evaluation 

Tribal 

Government 

Leaders 

Training in use, interpretation, 

& storage of data 

34 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
How Meeting Roadmap Priorities Can Benefit Their Work 

In addition to helping program staff members understand the broad context and importance of evaluation of Tribal 

child welfare programs, the Roadmap is particularly useful in engaging staff members to identify evaluation goals 

and determine processes, reinforcing that they are active partners in the evaluation, and addressing fears about the 

evaluation. Three priorities are particularly relevant. 

Values

Indigenous Ways of Knowing | Respect for Tribal Sovereignty 

Strengths Focus | Cultural and Scientific Rigor | Community Engagement 

Ethical Practices | Knowledge Sharing 

CO-CREATING COLLABORATIVE & EFFECTIVE EVALUATION TO IMPROVE TRIBAL CHILD WELFARE PROGRAMS

Policy Makers 

Community 

Members 

Cultural 

Authorities & 

Spiritual Leaders 

Tribal Colleges 

Universities 

Professional 

Organizations 

Relationship Building Knowledge & Skill Building 

Building a 

New Narrative 

Collaborative, Culturally 

Responsive Evaluation 

System Improvement 

Locally Guided Questions, 

Data, & Insight 

Meaningful Analyses 

Bidirectional Learning

 AI/AN Evaluation Skills 

Program 

Directors 

Tribal 

Evaluators 

Cultural protocols for communityengaged, comprehensive evaluation 

Mentoring for evaluation design 

& implementation 

Guidelines for selecting, adapting, 

developing, administering, & 

interpreting evaluation instruments 

Evaluation practice that involves the 

community in determining priorities 

Evaluation 

Partners 

Program 

Staff 

Consolidation & coordination of data 

across community programs 

Storytelling to support staff & 

community interpretation of 

evaluation data 

Indigenous world views undervalued 

Intergenerational and community trauma 

Invasive, imposed evaluation 

Judgmental evaluation 

Distrust 

Historical Context

Dialogue with community to 

anchor evaluation within cultural & 

ethical practices 

Clear guidelines & examples of 

evaluation components important to 

the community 

Clear guidelines & examples of 

required evaluation components tied 

to funding 

Transparency in evaluation & 

translation for community 

Developing policies, establishing clear guidelines, 

& building Tribal infrastructure for evaluation 

Tribal 

Government 

Leaders 

Training in use, interpretation, 

& storage of data 

35 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
Secondary Stakeholders

> Tribal Government Leaders 

Who They Are 

Tribal government leaders include Tribal council members, presidents, and other officials within Tribal communities charged with 

governance. These stakeholders bear the ultimate responsibility for child welfare within their communities and set policy that 

affects child welfare practice. 

Why They Are Important Stakeholders 

Because of their power and influence over Tribal child welfare programs, tribal government leaders must understand 

the importance of evaluation and how evaluation findings can inform policy decisions to improve practice. Supporting 

the development of a Tribal evaluation workforce is important for nation building and empowering Tribes to use local 

resources to address local concerns. 

What They Can Contribute to Meeting Roadmap Priorities 

Although Tribal government leaders may not be directly involved in evaluation practice, they make important 

contributions to addressing Roadmap priorities through participating in dialogues that identify local priorities, cultural 

protocols for evaluation, and guidelines for evaluation within their communities; supporting policies that enable efficient 

sharing of data across Tribal programs; and investing in the Tribal infrastructure for evaluation. 

Values

Indigenous Ways of Knowing | Respect for Tribal Sovereignty 

Strengths Focus | Cultural and Scientific Rigor | Community Engagement 

Ethical Practices | Knowledge Sharing 

CO-CREATING COLLABORATIVE & EFFECTIVE EVALUATION TO IMPROVE TRIBAL CHILD WELFARE PROGRAMS

Policy Makers 

Community 

Members 

Cultural 

Authorities & 

Spiritual Leaders 

Tribal Colleges 

Universities 

Professional 

Organizations 

Relationship Building Knowledge & Skill Building 

Building a 

New Narrative 

Collaborative, Culturally 

Responsive Evaluation 

System Improvement 

Locally Guided Questions, 

Data, & Insight 

Meaningful Analyses 

Bidirectional Learning

 AI/AN Evaluation Skills 

Program 

Directors 

Tribal 

Evaluators 

Cultural protocols for communityengaged, comprehensive evaluation 

Mentoring for evaluation design 

& implementation 

Guidelines for selecting, adapting, 

developing, administering, & 

interpreting evaluation instruments 

Evaluation practice that involves the 

community in determining priorities 

Evaluation 

Partners 

Program 

Staff 

Consolidation & coordination of data 

across community programs 

Storytelling to support staff & 

community interpretation of 

evaluation data 

Indigenous world views undervalued 

Intergenerational and community trauma 

Invasive, imposed evaluation 

Judgmental evaluation 

Distrust 

Historical Context

Dialogue with community to 

anchor evaluation within cultural & 

ethical practices 

Clear guidelines & examples of 

evaluation components important to 

the community 

Clear guidelines & examples of 

required evaluation components tied 

to funding 

Transparency in evaluation & 

translation for community 

Developing policies, establishing clear guidelines, 

& building Tribal infrastructure for evaluation 

Tribal 

Government 

Leaders 

Training in use, interpretation, 

& storage of data 

36 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
How Meeting Roadmap Priorities Can Benefit Their Work 

Addressing Roadmap priorities can support the work of Tribal government leaders by clearly translating evaluation 

findings to inform the development of child welfare policy. 

Values

Indigenous Ways of Knowing | Respect for Tribal Sovereignty 

Strengths Focus | Cultural and Scientific Rigor | Community Engagement 

Ethical Practices | Knowledge Sharing 

CO-CREATING COLLABORATIVE & EFFECTIVE EVALUATION TO IMPROVE TRIBAL CHILD WELFARE PROGRAMS

Policy Makers 

Community 

Members 

Cultural 

Authorities & 

Spiritual Leaders 

Tribal Colleges 

Universities 

Professional 

Organizations 

Relationship Building Knowledge & Skill Building 

Building a 

New Narrative 

Collaborative, Culturally 

Responsive Evaluation 

System Improvement 

Locally Guided Questions, 

Data, & Insight 

Meaningful Analyses 

Bidirectional Learning

 AI/AN Evaluation Skills 

Program 

Directors 

Tribal 

Evaluators 

Cultural protocols for communityengaged, comprehensive evaluation 

Mentoring for evaluation design 

& implementation 

Guidelines for selecting, adapting, 

developing, administering, & 

interpreting evaluation instruments 

Evaluation practice that involves the 

community in determining priorities 

Evaluation 

Partners 

Program 

Staff 

Consolidation & coordination of data 

across community programs 

Storytelling to support staff & 

community interpretation of 

evaluation data 

Indigenous world views undervalued 

Intergenerational and community trauma 

Invasive, imposed evaluation 

Judgmental evaluation 

Distrust 

Historical Context

Dialogue with community to 

anchor evaluation within cultural & 

ethical practices 

Clear guidelines & examples of 

evaluation components important to 

the community 

Clear guidelines & examples of 

required evaluation components tied 

to funding 

Transparency in evaluation & 

translation for community 

Developing policies, establishing clear guidelines, 

& building Tribal infrastructure for evaluation 

Tribal 

Government 

Leaders 

Training in use, interpretation, 

& storage of data 

37 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
> Policymakers 

Who They Are 

Policymakers often work in Tribal, State, and Federal governments and are responsible for overseeing and funding child 

welfare programs in Tribal communities (e.g., the Administration for Children and Families, the Children’s Bureau, State 

child welfare offices). The broad category of policymakers also includes elected government officials who are responsible 

for setting policy and allocating funding to programs. 

Why They Are Important Stakeholders 

Policymakers provide funding for Tribal child welfare programs and set requirements for program evaluations. It is critical that policymakers understand the unique context of evaluation of Tribal child welfare programs and recognize the importance of requiring 

culturally and scientifically rigorous evaluation to inform decisions. 

What They Can Contribute to Meeting Roadmap Priorities 

Policymakers play a pivotal role in addressing Roadmap priorities by responding to local priorities and protocols and by 

providing clear guidelines for the evaluation and the dissemination of results. 

Values

Indigenous Ways of Knowing | Respect for Tribal Sovereignty 

Strengths Focus | Cultural and Scientific Rigor | Community Engagement 

Ethical Practices | Knowledge Sharing 

CO-CREATING COLLABORATIVE & EFFECTIVE EVALUATION TO IMPROVE TRIBAL CHILD WELFARE PROGRAMS

Community 

Members 

Cultural 

Authorities & 

Spiritual Leaders 

Tribal Colleges 

Universities 

Professional 

Organizations 

Relationship Building Knowledge & Skill Building 

Building a 

New Narrative 

Collaborative, Culturally 

Responsive Evaluation 

System Improvement 

Locally Guided Questions, 

Data, & Insight 

Meaningful Analyses 

Bidirectional Learning

 AI/AN Evaluation Skills 

Program 

Directors 

Tribal 

Evaluators 

Cultural protocols for communityengaged, comprehensive evaluation 

Mentoring for evaluation design 

& implementation 

Guidelines for selecting, adapting, 

developing, administering, & 

interpreting evaluation instruments 

Evaluation practice that involves the 

community in determining priorities 

Evaluation 

Partners 

Program 

Staff 

Consolidation & coordination of data 

across community programs 

Storytelling to support staff & 

community interpretation of 

evaluation data 

Indigenous world views undervalued 

Intergenerational and community trauma 

Invasive, imposed evaluation 

Judgmental evaluation 

Distrust 

Historical Context

Dialogue with community to 

anchor evaluation within cultural & 

ethical practices 

Clear guidelines & examples of 

evaluation components important to 

the community 

Clear guidelines & examples of 

required evaluation components tied 

to funding 

Transparency in evaluation & 

translation for community 

Developing policies, establishing clear guidelines, 

& building Tribal infrastructure for evaluation 

Tribal 

Government 

Leaders 

Training in use, interpretation, 

& storage of data 

Policy Makers 

38 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
How Meeting Roadmap Priorities Can Benefit Their Work 

The Roadmap can help policymakers better understand the complexities of evaluation in Tribal contexts, including the 

importance of building relationships with Tribal communities and programs, working with sovereign nations, adapting 

measurement strategies and evaluation designs to be culturally rigorous, and appropriately disseminating findings. 

Values

Indigenous Ways of Knowing | Respect for Tribal Sovereignty 

Strengths Focus | Cultural and Scientific Rigor | Community Engagement 

Ethical Practices | Knowledge Sharing 

CO-CREATING COLLABORATIVE & EFFECTIVE EVALUATION TO IMPROVE TRIBAL CHILD WELFARE PROGRAMS

Community 

Members 

Cultural 

Authorities & 

Spiritual Leaders 

Tribal Colleges 

Universities 

Professional 

Organizations 

Relationship Building Knowledge & Skill Building 

Building a 

New Narrative 

Collaborative, Culturally 

Responsive Evaluation 

System Improvement 

Locally Guided Questions, 

Data, & Insight 

Meaningful Analyses 

Bidirectional Learning

 AI/AN Evaluation Skills 

Program 

Directors 

Tribal 

Evaluators 

Mentoring for evaluation design 

& implementation 

Guidelines for selecting, adapting, 

developing, administering, & 

interpreting evaluation instruments 

Evaluation 

Partners 

Program 

Staff 

Storytelling to support staff & 

community interpretation of 

evaluation data 

Policy Makers 

Cultural protocols for communityengaged, comprehensive evaluation 

Evaluation practice that involves the 

community in determining priorities 

Indigenous world views undervalued 

Intergenerational and community trauma 

Invasive, imposed evaluation 

Judgmental evaluation 

Distrust 

Historical Context

Consolidation & coordination of data 

across community programs 

Dialogue with community to 

anchor evaluation within cultural & 

ethical practices 

Clear guidelines & examples of 

evaluation components important to 

the community 

Clear guidelines & examples of 

required evaluation components tied 

to funding 

Transparency in evaluation & 

translation for community 

Developing policies, establishing clear guidelines, 

& building Tribal infrastructure for evaluation 

Tribal 

Government 

Leaders 

Training in use, interpretation, 

& storage of data 

39 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
> Community Members 

Who They Are 

Community members live in the communities being served by Tribal child welfare programs. This group includes families, 

children, elders, and others with an interest in child welfare. For American Indian reservations and Alaska Native villages, 

the community can be defined by geographical boundaries; in urban AI/AN settings, the community can be dispersed 

and defined by connections to local community or service centers. 

Why They Are Important Stakeholders 

Community members are, ultimately, the beneficiaries of child welfare services and have the most to gain from culturally 

and scientifically rigorous evaluation practices that inform the improvement of services. 

What They Can Contribute to Meeting Roadmap Priorities 

Community members play an integral role in educating other stakeholders about the community. By being engaged in 

the evaluation process, community members can contribute to evaluation designs and measures that are culturally appropriate and responsive to local questions and concerns. 

Values

Indigenous Ways of Knowing | Respect for Tribal Sovereignty 

Strengths Focus | Cultural and Scientific Rigor | Community Engagement 

Ethical Practices | Knowledge Sharing 

CO-CREATING COLLABORATIVE & EFFECTIVE EVALUATION TO IMPROVE TRIBAL CHILD WELFARE PROGRAMS

Cultural 

Authorities & 

Spiritual Leaders 

Tribal Colleges 

Universities 

Professional 

Organizations 

Relationship Building Knowledge & Skill Building 

Building a 

New Narrative 

Collaborative, Culturally 

Responsive Evaluation 

System Improvement 

Locally Guided Questions, 

Data, & Insight 

Meaningful Analyses 

Bidirectional Learning

 AI/AN Evaluation Skills 

Program 

Directors 

Tribal 

Evaluators 

Mentoring for evaluation design 

& implementation 

Guidelines for selecting, adapting, 

developing, administering, & 

interpreting evaluation instruments 

Evaluation 

Partners 

Program 

Staff 

Storytelling to support staff & 

community interpretation of 

evaluation data 

Policy Makers 

Cultural protocols for communityengaged, comprehensive evaluation 

Evaluation practice that involves the 

community in determining priorities 

Consolidation & coordination of data 

across community programs 

Indigenous world views undervalued 

Intergenerational and community trauma 

Invasive, imposed evaluation 

Judgmental evaluation 

Distrust 

Historical Context

Community 

Members 

Dialogue with community to 

anchor evaluation within cultural & 

ethical practices 

Clear guidelines & examples of 

evaluation components important to 

the community 

Clear guidelines & examples of 

required evaluation components tied 

to funding 

Transparency in evaluation & 

translation for community 

Developing policies, establishing clear guidelines, 

& building Tribal infrastructure for evaluation 

Tribal 

Government 

Leaders 

Training in use, interpretation, 

& storage of data 

40 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
How Meeting Roadmap Priorities Can Benefit Their Work 

Community members benefit from discussions with evaluators and policymakers. They also benefit from increased 

understanding of the guidelines for evaluation and from the development of Tribal infrastructure for evaluation 

oversight; both outcomes provide greater protection for community members who participate in the research process. 

In terms of Knowledge & Skill Building, community members benefit greatly from better cultural protocols for evaluation 

practice and from efforts to improve dissemination through storytelling, provide training opportunities, and consolidate 

data collection across programs. 

Values

Indigenous Ways of Knowing | Respect for Tribal Sovereignty 

Strengths Focus | Cultural and Scientific Rigor | Community Engagement 

Ethical Practices | Knowledge Sharing 

CO-CREATING COLLABORATIVE & EFFECTIVE EVALUATION TO IMPROVE TRIBAL CHILD WELFARE PROGRAMS

Cultural 

Authorities & 

Spiritual Leaders 

Tribal Colleges 

Universities 

Professional 

Organizations 

Relationship Building Knowledge & Skill Building 

Building a 

New Narrative 

Collaborative, Culturally 

Responsive Evaluation 

System Improvement 

Locally Guided Questions, 

Data, & Insight 

Meaningful Analyses 

Bidirectional Learning

 AI/AN Evaluation Skills 

Program 

Directors 

Tribal 

Evaluators 

Mentoring for evaluation design 

& implementation 

Guidelines for selecting, adapting, 

developing, administering, & 

interpreting evaluation instruments 

Evaluation 

Partners 

Program 

Staff 

Policy Makers 

Community 

Members 

Cultural protocols for communityengaged, comprehensive evaluation 

Evaluation practice that involves the 

community in determining priorities 

Indigenous world views undervalued 

Intergenerational and community trauma 

Invasive, imposed evaluation 

Judgmental evaluation 

Distrust 

Historical Context

Dialogue with community to 

anchor evaluation within cultural & 

ethical practices 

Clear guidelines & examples of 

evaluation components important to 

the community 

Clear guidelines & examples of 

required evaluation components tied 

to funding 

Transparency in evaluation & 

translation for community 

Developing policies, establishing clear guidelines, 

& building Tribal infrastructure for evaluation 

Storytelling to support staff & 

community interpretation of 

evaluation data 

Consolidation & coordination of data 

across community programs 

Tribal 

Government 

Leaders 

Training in use, interpretation, 

& storage of data 

41 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
> Cultural Authorities and Spiritual Leaders 

Who They Are 

These leaders are intimately involved with the cultural and spiritual lives of their communities. They include leaders in 

culture-based spirituality as well as religious and faith-based leaders in the Tribal community. 

Why They Are Important Stakeholders 

In Native communities, spirituality is a pervasive part of life. Cultural and spiritual leaders are important resources for 

evaluators developing protocols that are sensitive and responsive to the local cultural context. Involving these leaders in 

dissemination efforts holds great promise for sharing finding with community members in meaningful and informative ways. 

What They Can Contribute to Meeting Roadmap Priorities 

Cultural and spiritual leaders can play a pivotal role in educating other stakeholders about the cultural context of child 

welfare practice in their communities and in informing efforts to tell the story of child welfare and evaluation findings. 

Values

Indigenous Ways of Knowing | Respect for Tribal Sovereignty 

Strengths Focus | Cultural and Scientific Rigor | Community Engagement 

Ethical Practices | Knowledge Sharing 

CO-CREATING COLLABORATIVE & EFFECTIVE EVALUATION TO IMPROVE TRIBAL CHILD WELFARE PROGRAMS

Tribal Colleges 

Universities 

Professional 

Organizations 

Relationship Building Knowledge & Skill Building 

Building a 

New Narrative 

Collaborative, Culturally 

Responsive Evaluation 

System Improvement 

Locally Guided Questions, 

Data, & Insight 

Meaningful Analyses 

Bidirectional Learning

 AI/AN Evaluation Skills 

Program 

Directors 

Tribal 

Evaluators 

Mentoring for evaluation design 

& implementation 

Guidelines for selecting, adapting, 

developing, administering, & 

interpreting evaluation instruments 

Evaluation 

Partners 

Program 

Staff 

Policy Makers 

Community 

Members Storytelling to support staff & 

community interpretation of 

evaluation data 

Cultural protocols for communityengaged, comprehensive evaluation 

Evaluation practice that involves the 

community in determining priorities 

Consolidation & coordination of data 

across community programs 

Cultural 

Authorities & 

Spiritual Leaders 

Indigenous world views undervalued 

Intergenerational and community trauma 

Invasive, imposed evaluation 

Judgmental evaluation 

Distrust 

Historical Context

Dialogue with community to 

anchor evaluation within cultural & 

ethical practices 

Clear guidelines & examples of 

evaluation components important to 

the community 

Clear guidelines & examples of 

required evaluation components tied 

to funding 

Transparency in evaluation & 

translation for community 

Developing policies, establishing clear guidelines, 

& building Tribal infrastructure for evaluation 

Tribal 

Government 

Leaders 

Training in use, interpretation, 

& storage of data 

42 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
How Meeting Roadmap Priorities Can Benefit Their Work 

The benefits to cultural and spiritual leaders are primarily improvements in sharing evaluation findings with community 

members through increased transparency in evaluation practice and storytelling to disseminate findings. 

Values

Indigenous Ways of Knowing | Respect for Tribal Sovereignty 

Strengths Focus | Cultural and Scientific Rigor | Community Engagement 

Ethical Practices | Knowledge Sharing 

CO-CREATING COLLABORATIVE & EFFECTIVE EVALUATION TO IMPROVE TRIBAL CHILD WELFARE PROGRAMS

Tribal Colleges 

Universities 

Professional 

Organizations 

Relationship Building Knowledge & Skill Building 

Building a 

New Narrative 

Collaborative, Culturally 

Responsive Evaluation 

System Improvement 

Locally Guided Questions, 

Data, & Insight 

Meaningful Analyses 

Bidirectional Learning

 AI/AN Evaluation Skills 

Program 

Directors 

Tribal 

Evaluators 

Mentoring for evaluation design 

& implementation 

Guidelines for selecting, adapting, 

developing, administering, & 

interpreting evaluation instruments 

Evaluation 

Partners 

Program 

Staff 

Policy Makers 

Community 

Members Storytelling to support staff & 

community interpretation of 

evaluation data 

Cultural protocols for communityengaged, comprehensive evaluation 

Evaluation practice that involves the 

community in determining priorities 

Consolidation & coordination of data 

across community programs 

Cultural 

Authorities & 

Spiritual Leaders 

Indigenous world views undervalued 

Intergenerational and community trauma 

Invasive, imposed evaluation 

Judgmental evaluation 

Distrust 

Historical Context

Dialogue with community to 

anchor evaluation within cultural & 

ethical practices 

Clear guidelines & examples of 

evaluation components important to 

the community 

Clear guidelines & examples of 

required evaluation components tied 

to funding 

Transparency in evaluation & 

translation for community 

Developing policies, establishing clear guidelines, 

& building Tribal infrastructure for evaluation 

Tribal 

Government 

Leaders 

Training in use, interpretation, 

& storage of data 

43 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
> Tribal Colleges 

Who They Are 

Tribal colleges serve key roles in providing education that is culturally grounded and locally relevant. The stakeholders in 

this group are the administrators, faculty, and researchers at those colleges. 

Why They Are Important Stakeholders 

Tribal colleges can educate Tribal members in evaluation and provide local opportunities to build the research infrastructure in Tribes. Through educational programs, these colleges help many Tribes grow their own cadre of professional 

evaluators who are well equipped to provide culturally and scientifically rigorous evaluations for child welfare and other 

programs in Tribal communities. 

What They Can Contribute to Meeting Roadmap Priorities 

Tribal colleges can play an important role in meeting most of the Roadmap priorities related to Knowledge & Skill Building, but they also play a critical role in addressing priorities related to Relationship Building. 

Values

Indigenous Ways of Knowing | Respect for Tribal Sovereignty 

Strengths Focus | Cultural and Scientific Rigor | Community Engagement 

Ethical Practices | Knowledge Sharing 

CO-CREATING COLLABORATIVE & EFFECTIVE EVALUATION TO IMPROVE TRIBAL CHILD WELFARE PROGRAMS

Universities 

Professional 

Organizations 

Relationship Building Knowledge & Skill Building 

Building a 

New Narrative 

Collaborative, Culturally 

Responsive Evaluation 

System Improvement 

Locally Guided Questions, 

Data, & Insight 

Meaningful Analyses 

Bidirectional Learning

 AI/AN Evaluation Skills 

Program 

Directors 

Tribal 

Evaluators 

Mentoring for evaluation design 

& implementation 

Guidelines for selecting, adapting, 

developing, administering, & 

interpreting evaluation instruments 

Evaluation 

Partners 

Program 

Staff 

Policy Makers 

Community 

Members Storytelling to support staff & 

community interpretation of 

evaluation data 

Cultural protocols for communityengaged, comprehensive evaluation 

Evaluation practice that involves the 

community in determining priorities 

Consolidation & coordination of data 

across community programs 

Cultural 

Authorities & 

Spiritual Leaders 

Indigenous world views undervalued 

Intergenerational and community trauma 

Invasive, imposed evaluation 

Judgmental evaluation 

Distrust 

Historical Context

Dialogue with community to Tribal Colleges anchor evaluation within cultural & 

ethical practices 

Clear guidelines & examples of 

evaluation components important to 

the community 

Clear guidelines & examples of 

required evaluation components tied 

to funding 

Transparency in evaluation & 

translation for community 

Developing policies, establishing clear guidelines, 

& building Tribal infrastructure for evaluation 

Tribal 

Government 

Leaders 

Training in use, interpretation, 

& storage of data 

44 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
How Meeting Roadmap Priorities Can Benefit Their Work 

Tribal colleges can serve as the conduit through which resources identified and developed under the Roadmap can be 

shared with local students of evaluation to build a strong local evaluation workforce. 

Values

Indigenous Ways of Knowing | Respect for Tribal Sovereignty 

Strengths Focus | Cultural and Scientific Rigor | Community Engagement 

Ethical Practices | Knowledge Sharing 

CO-CREATING COLLABORATIVE & EFFECTIVE EVALUATION TO IMPROVE TRIBAL CHILD WELFARE PROGRAMS

Universities 

Professional 

Organizations 

Relationship Building Knowledge & Skill Building 

Building a 

New Narrative 

Collaborative, Culturally 

Responsive Evaluation 

System Improvement 

Locally Guided Questions, 

Data, & Insight 

Meaningful Analyses 

Bidirectional Learning

 AI/AN Evaluation Skills 

Program 

Directors 

Tribal 

Evaluators 

Evaluation 

Partners 

Program 

Staff 

Policy Makers 

Community 

Members Storytelling to support staff & 

community interpretation of 

evaluation data 

Cultural protocols for communityengaged, comprehensive evaluation 

Evaluation practice that involves the 

community in determining priorities 

Consolidation & coordination of data 

across community programs 

Cultural 

Authorities & 

Spiritual Leaders 

Tribal Colleges Mentoring for evaluation design 

& implementation 

Guidelines for selecting, adapting, 

developing, administering, & 

interpreting evaluation instruments 

Indigenous world views undervalued 

Intergenerational and community trauma 

Invasive, imposed evaluation 

Judgmental evaluation 

Distrust 

Historical Context

Dialogue with community to 

anchor evaluation within cultural & 

ethical practices 

Clear guidelines & examples of 

evaluation components important to 

the community 

Clear guidelines & examples of 

required evaluation components tied 

to funding 

Transparency in evaluation & 

translation for community 

Developing policies, establishing clear guidelines, 

& building Tribal infrastructure for evaluation 

Tribal 

Government 

Leaders 

Training in use, interpretation, 

& storage of data 

45 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
> Universities 

Who They Are 

University partners include faculty, such as teachers and researchers. They provide educational services related to 

evaluation and research. 

Why They Are Important Stakeholders 

University partners can support the education of a Tribal evaluation workforce through course offerings and 

development of course syllabi. They can serve as evaluators or as consultants to evaluators. 

What They Can Contribute to Meeting Roadmap Priorities 

University partners can mentor and train Tribal evaluators, establish clear guidelines for evaluation, and translate 

evaluation reports for communities. 

Values

Indigenous Ways of Knowing | Respect for Tribal Sovereignty 

Strengths Focus | Cultural and Scientific Rigor | Community Engagement 

Ethical Practices | Knowledge Sharing 

CO-CREATING COLLABORATIVE & EFFECTIVE EVALUATION TO IMPROVE TRIBAL CHILD WELFARE PROGRAMS

Professional 

Organizations 

Relationship Building Knowledge & Skill Building 

Building a 

New Narrative 

Collaborative, Culturally 

Responsive Evaluation 

System Improvement 

Locally Guided Questions, 

Data, & Insight 

Meaningful Analyses 

Bidirectional Learning

 AI/AN Evaluation Skills 

Program 

Directors 

Tribal 

Evaluators 

Evaluation 

Partners 

Program 

Staff 

Policy Makers 

Community 

Members Storytelling to support staff & 

community interpretation of 

evaluation data 

Cultural protocols for communityengaged, comprehensive evaluation 

Evaluation practice that involves the 

community in determining priorities 

Consolidation & coordination of data 

across community programs 

Cultural 

Authorities & 

Spiritual Leaders 

Tribal Colleges Mentoring for evaluation design 

& implementation 

Guidelines for selecting, adapting, 

developing, administering, & 

interpreting evaluation instruments 

Indigenous world views undervalued 

Intergenerational and community trauma 

Invasive, imposed evaluation 

Judgmental evaluation 

Distrust 

Historical Context

Universities 

Dialogue with community to 

anchor evaluation within cultural & 

ethical practices 

Clear guidelines & examples of 

evaluation components important to 

the community 

Clear guidelines & examples of 

required evaluation components tied 

to funding 

Transparency in evaluation & 

translation for community 

Developing policies, establishing clear guidelines, 

& building Tribal infrastructure for evaluation 

Tribal 

Government 

Leaders 

Training in use, interpretation, 

& storage of data 

46 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
How Meeting Roadmap Priorities Can Benefit Their Work 

Addressing the Roadmap priorities can help university evaluators gain a better understanding of rigorous cultural 

protocols for evaluation and form effective relationships with Tribal partners. 

Values

Indigenous Ways of Knowing | Respect for Tribal Sovereignty 

Strengths Focus | Cultural and Scientific Rigor | Community Engagement 

Ethical Practices | Knowledge Sharing 

CO-CREATING COLLABORATIVE & EFFECTIVE EVALUATION TO IMPROVE TRIBAL CHILD WELFARE PROGRAMS

Professional 

Organizations 

Relationship Building Knowledge & Skill Building 

Building a 

New Narrative 

Collaborative, Culturally 

Responsive Evaluation 

System Improvement 

Locally Guided Questions, 

Data, & Insight 

Meaningful Analyses 

Bidirectional Learning

 AI/AN Evaluation Skills 

Program 

Directors 

Tribal 

Evaluators 

Evaluation 

Partners 

Program 

Staff 

Policy Makers 

Community 

Members Storytelling to support staff & 

community interpretation of 

evaluation data 

Consolidation & coordination of data 

across community programs 

Cultural 

Authorities & 

Spiritual Leaders 

Tribal Colleges Mentoring for evaluation design 

& implementation 

Guidelines for selecting, adapting, 

developing, administering, & 

interpreting evaluation instruments 

Universities 

Cultural protocols for communityengaged, comprehensive evaluation 

Evaluation practice that involves the 

community in determining priorities 

Indigenous world views undervalued 

Intergenerational and community trauma 

Invasive, imposed evaluation 

Judgmental evaluation 

Distrust 

Historical Context

Dialogue with community to 

anchor evaluation within cultural & 

ethical practices 

Clear guidelines & examples of 

evaluation components important to 

the community 

Clear guidelines & examples of 

required evaluation components tied 

to funding 

Transparency in evaluation & 

translation for community 

Developing policies, establishing clear guidelines, 

& building Tribal infrastructure for evaluation 

Tribal 

Government 

Leaders 

Training in use, interpretation, 

& storage of data 

47 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
> Professional Organizations 

Who They Are 

Several professional organizations have the potential to help address Roadmap priorities: 

• American Academy of Pediatrics http://www.aap.org

• American Evaluation Association http://www.eval.org

• American Indian Higher Education Consortium http://www.aihec.org

• American Psychological Association http://www.apa.org

• Association of American Indian Physicians http://www.aaip.org

• Canadian Mental Health Association http://www.cmha.ca

• First Nations Behavioral Health Association http://www.fnbha.org

• National Association of Social Workers http://www.naswdc.org

• National Indian Child Welfare Association http://www.nicwa.org

• Native Research Network http://www.nativeresearchnetwork.org

• Society of Indian Psychologists http://www.aiansip.org

• Society for Prevention Research http://www.preventionresearch.org

• Society for Research in Child Development http://www.srcd.org

• Society for Research on Adolescence http://www.s-r-a.org

Why They Are Important Stakeholders 

Professional organizations educate the evaluation workforce, provide opportunities for evaluators to learn about new 

developments in evaluation practice, and provide networking opportunities for evaluators to learn from one another. 

48 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
What They Can Contribute to Meeting Roadmap Priorities 

Professional organizations provide mentoring, training, and guidelines for evaluation practice. 

Values

Indigenous Ways of Knowing | Respect for Tribal Sovereignty 

Strengths Focus | Cultural and Scientific Rigor | Community Engagement 

Ethical Practices | Knowledge Sharing 

CO-CREATING COLLABORATIVE & EFFECTIVE EVALUATION TO IMPROVE TRIBAL CHILD WELFARE PROGRAMS

Relationship Building Knowledge & Skill Building 

Building a 

New Narrative 

Collaborative, Culturally 

Responsive Evaluation 

System Improvement 

Locally Guided Questions, 

Data, & Insight 

Meaningful Analyses 

Bidirectional Learning

 AI/AN Evaluation Skills 

Program 

Directors 

Tribal 

Evaluators 

Evaluation 

Partners 

Program 

Staff 

Policy Makers 

Community 

Members Storytelling to support staff & 

community interpretation of 

evaluation data 

Consolidation & coordination of data 

across community programs 

Cultural 

Authorities & 

Spiritual Leaders 

Tribal Colleges Mentoring for evaluation design 

& implementation 

Guidelines for selecting, adapting, 

developing, administering, & 

interpreting evaluation instruments 

Universities 

Cultural protocols for communityengaged, comprehensive evaluation 

Evaluation practice that involves the 

community in determining priorities 

Professional 

Organizations 

Indigenous world views undervalued 

Intergenerational and community trauma 

Invasive, imposed evaluation 

Judgmental evaluation 

Distrust 

Historical Context

Dialogue with community to 

anchor evaluation within cultural & 

ethical practices 

Clear guidelines & examples of 

evaluation components important to 

the community 

Clear guidelines & examples of 

required evaluation components tied 

to funding 

Transparency in evaluation & 

translation for community 

Developing policies, establishing clear guidelines, 

& building Tribal infrastructure for evaluation 

Tribal 

Government 

Leaders 

Training in use, interpretation, 

& storage of data 

49 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
How Meeting Roadmap Priorities Can Benefit Their Work 

Professional organizations can benefit from the Roadmap by increasing understanding of the importance of cultural 

protocols in evaluation and the intricacies of partnering with Tribal nations in evaluation. 

Values

Indigenous Ways of Knowing | Respect for Tribal Sovereignty 

Strengths Focus | Cultural and Scientific Rigor | Community Engagement 

Ethical Practices | Knowledge Sharing 

CO-CREATING COLLABORATIVE & EFFECTIVE EVALUATION TO IMPROVE TRIBAL CHILD WELFARE PROGRAMS

Relationship Building Knowledge & Skill Building 

Building a 

New Narrative 

Collaborative, Culturally 

Responsive Evaluation 

System Improvement 

Locally Guided Questions, 

Data, & Insight 

Meaningful Analyses 

Bidirectional Learning

 AI/AN Evaluation Skills 

Program 

Directors 

Tribal 

Evaluators 

Evaluation 

Partners 

Program 

Staff 

Policy Makers 

Community 

Members Storytelling to support staff & 

community interpretation of 

evaluation data 

Consolidation & coordination of data 

across community programs 

Cultural 

Authorities & 

Spiritual Leaders 

Tribal Colleges Mentoring for evaluation design 

& implementation 

Guidelines for selecting, adapting, 

developing, administering, & 

interpreting evaluation instruments 

Universities 

Evaluation practice that involves the 

community in determining priorities 

Professional 

Organizations 

Cultural protocols for communityengaged, comprehensive evaluation 

Indigenous world views undervalued 

Intergenerational and community trauma 

Invasive, imposed evaluation 

Judgmental evaluation 

Distrust 

Historical Context

Dialogue with community to 

anchor evaluation within cultural & 

ethical practices 

Clear guidelines & examples of 

evaluation components important to 

the community 

Clear guidelines & examples of 

required evaluation components tied 

to funding 

Transparency in evaluation & 

translation for community 

Developing policies, establishing clear guidelines, 

& building Tribal infrastructure for evaluation 

Tribal 

Government 

Leaders 

Training in use, interpretation, 

& storage of data 

50 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
> Existing Resources That Address Priorities 

Many resources are available for individuals, programs, and Tribes working to improve child welfare practice through 

evaluation. The sections below highlight key publications, documents, Web sites, and other resources. Resources are 

divided two categories: Theory and Background Resources and Best Practices and Practical Guidelines. 

Theory and Background Resources 

The workgroup identified resources that provide important perspectives on the underlying issues and challenges of evaluation and research in the context of Tribal communities. These resources are useful for setting the stage for the work to be 

done, providing important context, asking critical questions, and suggesting fundamental strategies for moving forward. 

> Research Indigenous Knowledge and Indigenous Methods 

•	 Utilizing Research Methods that Respect and Empower Indigenous Knowledge. Faircloth, S.C., & 

Tippeconnic, J.W. (2004). Tribal College, 16(2), 24-27. http://www.tribalcollegejournal.org/archives/10665. 

•	 Culturally Competent Research with American Indians and Alaska Natives: Findings and 

Recommendations of the First Symposium of the Work Group on American Indian Research and Program 

Evaluation Methodology (AIRPEM). Caldwell, J.Y., Davis, J., Du Bois, B., Echo-Hawk, H., Erickson, J.S., Goins, 

R.T., & Stone, J.B. (2005). American Indian and Alaska Native Mental Health Research, 12(1). http://www.ucdenver. 

edu/academics/colleges/PublicHealth/research/centers/CAIANH/journal/Documents/Volume%2012/12(1)_ 

Caldwell_Culturally_Competent_1-21.pdf. 

•	 AIRPEM Monograph: Symposium on Research and Evaluation Methodology: Lifespan Issues Related to 

American Indians/Alaska Natives with Disabilities. Davis, J.D., Erickson, J.S., Johnson, S.R., Marshall, C.A., 

Running Wolf, P., & Santiago, R.L. (Eds.), Flagstaff: Northern Arizona University, Institute for Human Development, 

Arizona University Center on Disabilities, American Indian Rehabilitation Research and Training Center (2002). 

http://www.fnbha.org/pdf/AIRPEMMonograph.pdf. 

•	 American Indian Higher Education Consortium – Indigenous Evaluation Framework Workshop Series. 

http://indigeval.aihec.org/Pages/IndigHome.aspx. 

•	 Tribal Early Childhood Research Center Summer Institute. Course in Early Childhood Research with Tribal 

Communities, University of Colorado and Johns Hopkins University. http://www.tribalearlychildhood.org. 

•	 Culture Writes the Script: The Centrality of Context in Indigenous Evaluation. LaFrance, J., Nichols, R., & 

Kirkhart, K.E. (2012). New Directions for Evaluation, 2012(135), 59-74. 

•	 Researching Ourselves Back to Life: Taking Control of the Research Agenda in Indian Country. LaFrance, J., 

& Crazy Bull, C. (2009). Handbook of social research ethics, 135-149. 

•	 Reframing Evaluation: Defining an Indigenous Evaluation Framework. LaFrance, J., & Nichols, R. (2008). 

Canadian Journal of Program Evaluation, 23(2), 13-32. http://www.aihec.org/programs/documents/ NSF-TCUP/ 

DefiningIndigenousEvaluationFramework_LaFrance-NicholsNov2010.pdf. 

•	 The Cultural Context of Educational Evaluation. The National Science Foundation. http://www.nsf.gov/ 

pubs/2003/nsf03032/nsf03032.pdf. 

•	 Planning Your Rigorous Evaluation for Tribal Home Visiting [PPT slides]. Meyer, A.L., Lyon, 

K., Barlow, A., & Walkup, J. (2011). Tribal Maternal, Infant, and Early Childhood Home Visiting 

Program, Administration for Children and Families, Office of Child Care. https://docs.google.com/ 

viewer?a=v&pid=sites&srcid=bnJjNHRyaWJlcy5vcmd8d2lraXxneDo3MWViNGIzMWM1YzUxOGVk. 

51 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
 

> Community-Based Participatory Research & Tribal Participatory Research 

•	 Community-Based Participatory Research in American Indian and Alaska Native Communities. Sahota, P.C. 

(2010). NCAI Policy Research Center publication. http://www.ncaiprc.org/files/CBPR%20Paper%20FINAL.pdf. 

•	 Lessons Learned from Community-Based Participatory Research in Indian Country. Burhansstipanov, L., & 

Schumacher, S. C. S. A. (2005). Cancer control: journal of the Moffitt Cancer Center, 12(Suppl 2), 70. 

http://hlmcc.org/CCJRoot/v12s5/ pdf/70.pdf. 

•	 The Indian Family Wellness Project: An Application of the Tribal Participatory Research Model. Fisher, P.A. & 

Ball, T.J. (2002). Prevention Science: The Official Journal Of The Society For Prevention Research, 3(3), 235-24. 

•	 Urban Indian Voices: A Community-Based Participatory Research Health and Needs Assessment. Johnson, 

C.V., Bartgis, J., Worley, J.A., Hellman, C.M., & Burkhart, R. (2010). American Indian and Alaska Native Mental 

Health Research online, 17(1), 49-70. http://www.niams.nih.gov/about_us/Mission_and_ Purpose/Community_ 

Outreach/Multicultural_Outreach/AIAN_WG/Bartgis_UrbanIndianVoicesCBPR.pdf. 

> History and Context of Evaluation and Research in Tribal Contexts 

•	 National Child Welfare Resource Center for Tribes (NRC4Tribes) Needs Assessment. 

http://www.nrc4tribes.org/files/NRCT%20Needs%20Assessment%20Findings_APPROVED.pdf. 

> Evidence-Based Practice, Translation, and Adaptation 

•	 Culture Based Interventions: The Native Aspirations Project. Walker, R.D., Bigelow, D.A., Louden, L., 

Silk-Walker, P., & Singer, M.J. (2008). Eugene, One Sky Center. 

•	 Home-Visiting Intervention to Improve Child Care Among American Indian Adolescent Mothers: A 

Randomized Trial. Barlow, A.,Varipatis-Baker, E., Speakman, K., Ginsburg, G., Friberg, I., Goklish, N., & Walkup, J. 

(2006). Archives of Pediatrics & Adolescent Medicine, 160(11), 1101. http://www.jhsph.edu/research/centers-andinstitutes/center-for-american-indian-health/Resources/Barlow_HomeVisitsToImproveChildCare.pdf. 

Best Practices and Practical Guidelines 

The workgroup identified resources that offer best practices and practical guidelines for evaluation practice. The 

resources below contain more concrete guidance for research and evaluation in Tribal contexts, evaluation design, 

program implementation, outcomes and indicators, and guidelines for adaptation. 

> Research and Evaluation in Tribal Contexts 

•	 American Evaluation Association Statement on Cultural Competence in Evaluation. 

http://www.eval.org/p/cm/ld/fid=92. 

•	 Culture Card: A Guide to Build Cultural Awareness. The Substance Abuse and Mental Health Services 

Administration (SAMHSA). http://store.samhsa.gov/shin/content/SMA08-4354/SMA08-4354.pdf. 

•	 Indigenous Evaluation Framework: Telling Our Story in Our Place and Time. LaFrance, J., & Nichols, R. (2008). 

Alexandria, VA: American Indian Higher Education Consortium (AIHEC). http://indigeval.aihec.org/Pages/Documents.aspx. 

•	 Tribal Research Assessment Checklist. Northern Plains Tribal Epidemiology Center. http://www.nptao.arizona. 

edu/research/NPTAOResearchProtocolsWebPage/NPTAO_Research_OtherResources.doc/NorthernPlains/ 

NorthernPlainsResearchChecklist.pdf 

•	 Research Regulation in American Indian/Alaska Native Communities: Policy and Practice Considerations. 

Sahota, P.C. (2009). http://ncaiprc.org/files/Research%20Regulation%20in%20AI%20AN%20Communities%20 

-%20Policy%20and%20Practice.pdf. 

52 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
•	 Research Regulation in American Indian/Alaska Native Communities: A Guide to Reviewing Research 

Studies. Sahota, P.C. (2009). http://ncaiprc.org/files/Research%20Regulation%20in%20AI%20AN%20 

Communities%20-%20Guide%20to%20Reviewing%20Research%20Studies.pdf. 

•	 Research Regulation Options for American Indian and Alaska Native Communities. NCAI Policy Research 

Center. http://www.ncaiprc.org/files/Research%20Regulation%20Options%20for%20AIAN%20Communities.pdf. 

•	 Data Control Options in American Indian and Alaska Native Communities. NCAI Policy Research Center. 

http://genetics.ncai.org/data-control-options.cfm. 

•	 Research Review Checklist for American Indian and Alaska Native Communities. NCAI Policy Research 

Center. http://www.fws.gov/nativeamerican/pdf/tek-research.pdf. 

•	 Brief Chronology of the Havasupai Case and Settlement – with Reminders and Possible Implications for 

HRPPs, IRBs, Researchers, and the Common Rule. Freeman, W.L. (2010). PowerPoint Presentation. 

•	 Intersecting Interests: Tribal Knowledge and Research Communities. The University of Montana. 

http://iers.umt.edu/docs/intersectinginterestsdocs/Intersecting%20Interests%20Compendium%20Final.pdf. 

•	 Model Tribal Research Code: With Materials for Tribal Regulation for Research and Checklist for Indian 

Health Boards. American Indian Law Center (1999). http://www.ihs.gov/Research/pdf/mdl-code.pdf. 

•	 Process of Conducting Research on the Navajo Nation, Arizona Cooperative Extension. Tuttle, S., Moore, G., 

& Benally, J. (2008). http://arizona.openrepository.com/arizona/bitstream/10150/144738/1/az1472-2008.pdf. 

•	 Autonomous Individuals or Self-Determined Communities? The Changing Ethics of Research among 

Native Americans. Smith-Morris, C. (2007). Human Organization, 66(3), 327-336. 

http://faculty.arts.ubc.ca/menzies/documents/Smith-Morris.pdf. 

> Evaluation Design 

•	 Research that Benefits Native People: A Guide for Tribal Leaders. NCAI (20009). 

http://www.ncaiprc.org/research-curriculum-guide. 

•	 Medicine Wheel Evaluation Framework. Atlantic Council for International Cooperation. 

http://www.acic-caci.org/storage/Medicine_Wheel_Evaluation_Framework.pdf. 

•	 American Indian Adult Tobacco Survey Implementation. U.S. Department of Health and Human Services, 

Centers for Disease Control and Prevention. http://www.cdc.gov/tobacco/data_statistics/surveys/american_indian/. 

> Program Implementation 

•	 Key Considerations and Best Practices for Tribal Title IV-E Data Collection and Reporting. Geary, E., & Day, P.A. 

(2010). http://childwelfare.ncaiprc.org/documentlibrary/2010/04/Data%20Management%20Paper%20-%20Final.pdf. 

> Outcomes and Indicators 

•	 Tribal Early Childhood Research Center Compendium of Measures that have been used with American 

Indian and Alaska Native children and families. http://www.ucdenver.edu/academics/colleges/PublicHealth/ 

research/centers/CAIANH/trc/Pages/TRCPublicationsPresentation.aspx. 

> Guidelines for Adaptation 

•	 Tribal Maternal, Infant, and Early Childhood Home Visiting Program: Model Selection and Cultural 

Adaptation of Home Visiting Models. Webinar hosted by the National Resource Center for Tribes (July 2011).https:// 

docs.google.com/viewer?a=v&pid=sites&srcid=bnJjNHRyaWJlcy5vcmd8d2lraXxneDo1ODM5NzQxNzAzYzRjMDEw. 

53 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
> What Is Needed To Move Forward 

Although there are many resources available to guide evaluation practice in Tribal communities, the workgroup identified several areas where the field would benefit from additional resource development. These potential resources could 

contribute to the new narrative for evaluation practice. 

Guidelines for Stakeholders 

•	 Guidelines for evaluators working with Tribal communities could provide steps for engaging community 

stakeholders in identifying evaluation questions and in adapting standard evaluation practice to fit within the 

cultural contexts of the Tribal child welfare programs. 

•	 Guidelines for program directors are needed including targeted steps for identifying key questions for evaluation 

and for communicating these to evaluators (Tribal evaluators, evaluation partners, and policymakers), as well as: 

>	 Suggestions for how to identify what information would be most useful in improving their programs, articulating 

questions for evaluation, prioritizing questions, and communicating with evaluators about these priorities 

>	 Guidelines for evaluating the appropriateness of a proposed evaluation plan for the community, identifying 

potential misalignment between evaluation strategies and community practices, and negotiating adaptations to 

ensure a community-responsive evaluation plan 

•	 Guidelines for community members should also be developed to help those engaged by evaluators and program 

directors understand the importance of their role in the process, support their contributions to the collaborative 

process, and elicit information about local culture that will be critical to culturally rigorous evaluation models. 

•	 Guidelines for policymakers could provide suggestions for language and structure in evaluation requirements tied to 

funding and provide concrete examples of effective local evaluation. 

Training Opportunities 

•	 An introduction to evaluation for Tribal child welfare staff could be shared with local program staff at the beginning 

of an evaluation or periodically as part of staff development and could help demystify the process and engage staff 

members as partners in evaluation. 

•	 Evaluators could serve as mentors and engage local program staff members throughout the evaluation process, 

encouraging them to participate in identifying questions, reviewing evaluation plans, interpreting data, and 

disseminating findings. Similar approaches have been used, such as a joint effort of the Administration for Children 

and Families and the Office of Planning, Research and Evaluation for the Temporary Assistance for Needy Families 

Child Welfare Coordination Study, and should be expanded to include additional projects. Through this mentorship 

model, local staff gain skills to become active, authentic partners in the evaluation process. Relevant evaluation skills 

include (but are not limited to): 

> Evaluation design and conceptualization
 

> Identifying theory of change
 

> Instrument development
 

> Data collection (qualitative and quantitative)
 

> Data analysis and interpretation
 

> Reporting and dissemination
 

> IRB review and human subjects protection
 

> Community disclosure
 

54 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
> Data security and management
 

> Interacting with stakeholders
 

> Implementation science
 

> Translational research
 

•	 Training opportunities for Tribal evaluators and researchers should be developed, specifically targeted to the 

needs of Tribal members interested in careers in research and evaluation. Existing training opportunities should be 

identified for Tribal members who have an interest in and aptitude for evaluation and research. Opportunities for 

financial assistance should also be identified. Federal cross-agency collaboration is important for identifying and 

developing training opportunities for Tribal researchers and evaluators. 

•	 Creative training options should be provided for Tribal members who are interested in furthering their education 

in evaluation and research but have limited options for doing so. Distances to educational institutions offering the 

necessary courses may be too great, and tuition may be too expensive. Multiple options should address this barrier, 

including working with Tribal colleges to increase local educational options, providing distance learning through 

regional colleges and universities, or engaging a national training network through Webinars and other online 

resources. 

•	 Continuing education for evaluators on data management and analysis could focus on challenges unique to Tribal 

evaluation. 

•	 Technical assistance could help make evaluation requirements clear and attainable for Tribal programs and provide 

mentoring opportunities for local evaluators. 

•	 Support for Tribal college evaluation programs and courses could include: 

> Syllabi for evaluation coursework at Tribal colleges that do not offer evaluation courses 

> Consultation to Tribal college administrators to create or improve evaluation courses and programs 

> Support for peer networks to foster information and the exchange of best practices between Tribal colleges 

with existing evaluation courses and/or programs and colleges that might develop courses/programs 

Additional Resources 

> Evaluation Design 

•	 Templates for evaluation practice in Tribal communities could be drawn from the protocols established by Tribal 

communities that are further along in their evaluation efforts (e.g., Navajo Nation, Cherokee Nation, Oglala Sioux 

Tribe). These templates could be shared with other communities that need guidance on how to proceed and how to 

build their own infrastructure for research and evaluation practice. 

•	 Guidelines for local programs on cultural education for evaluators/researchers could provide specific suggestions for 

program directors and Tribal evaluators to help focus the kind of information they provide about their community 

when they begin a partnership with an outside evaluator or researcher. 

> Ethics and Institutional Review Board Processes 

•	 A compendium of Tribal review procedures could be developed based on human subjects review procedures 

from various Tribes. This compendium could include a list of contacts responsible for overseeing research in Tribal 

communities and links to related resources (e.g., Tribal review applications, instructions). This resource could give 

evaluators and researchers information on where to begin the review process in a particular community. Evaluators 

and researchers could use the compendium to identify Tribes that have formal processes so that they do not waste 

valuable time trying to identify the appropriate process or fail to follow established procedures. 

55 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
> Outcome Measures 

•	 A compendium of examples of successful evaluations in Tribal child welfare programs that describe local questions 

and priorities, requirements of funders, and both cultural and scientific rigor could be helpful for evaluators working 

in Tribal contexts. 

•	 An instrument clearinghouse for evaluators could share information about evaluation measures (e.g., things that 

worked well, problems with use). Evaluators could use the clearinghouse to ask for suggestions for assessing 

particular outcomes and seek guidance from peers working in other Tribal communities. 

•	 Instrument selection guidelines could help evaluators identify existing measures, evaluate the utility of those 

measures for Tribal populations, and make appropriate adaptations. 

> Dissemination 

•	 Protocols for community dissemination could include establishing appropriate practices for community 

dissemination, highlighting effective strategies for effective communication of evaluation findings to important 

community stakeholders, and providing examples of clear and culturally appropriate dissemination efforts. 

Guidelines could also include opportunities for community feedback on study findings. 

•	 Storytelling guidelines could guide evaluators on how to develop the dissemination plan. These guidelines could 

include suggestions about which community members to consult to help identify how, when, where, and to whom 

the story should be told. 

•	 Outreach to students and individuals working in Tribal child welfare explaining the value of evaluation and research 

could identify those who might be interested in pursuing evaluation/research careers. 

•	 Creating and supporting infrastructure for data collection, management, and archiving are challenging and 

complex undertakings. Historically, research data have been collected in Tribal communities, but the findings of 

these studies were often not reported to the Tribal nations or not useful to the community. Many Tribes lack the 

infrastructure to support ongoing data collection and management of data sets that are relevant to improving child 

welfare practice and outcomes. Supporting the collection, management, and archiving of data will allow for Tribal 

nations and organizations to improve outcomes for children and families. 

56 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau 
> References 

Brave Heart, M. (1998). The return to the sacred path: Healing the historical trauma and historical unresolved grief 


response among the Lakota through psychoeducational group intervention. Smith College Studies in Social Work, 68(3), 


287-305.
 

Brave Heart, M., & DeBruyn, L. (1998). The American Indian holocaust: Healing historical unresolved grief. American 


Indian and Alaska Native Mental Health Research: Journal of the National Center, 8(2), 56-78.
 

Cashman, S., Adeky, S., Allen, A., Corburn, J., Israel, B., Montaño, J., & Eng, E. (2008). The power and the promise: 


Working with communities to analyze data, interpret findings, and get to outcomes. American Journal of Public Health, 


98(8), 1407-1417.
 

Cross, T. (1997). Understanding the relational worldview in Indian families. Pathways Practice Digest, 12(4).
 

Duran, E. & Duran, B. (1995). Native American postcolonial psychology. Albany, NY: SUNY Press.
 

Fisher, P. A., & Ball, T. J. (2003). Tribal participatory research: Mechanisms of a collaborative model. American Journal of 


Community Psychology, 32(3/4), 207-216. 

Indian child welfare act (ICWA). (n.d.). In The adoption history project. Retrieved December 16, 2013, from http://pages.uoregon.edu/adoption/topics/ICWA.html.
 

Kreisher, K. (2002). Coming home: The lingering effects of the Indian adoption project. Retrieved December 16, 2013, 


from http://www.cwla.org/articles/cv0203indianadopt.html.
 

LaFrance, J., & Nichols, R. (2010). Reframing evaluation: Defining an indigenous evaluation framework. The Canadian 


Journal of Program Evaluation, 23(2), 13-31.
 

Mazzocchi, F. (2006). Western science and traditional knowledge: Despite their variations, different forms of knowledge 


can learn from each other. Science and Society, 7(5), 463-466.
 

Science. (2005). In Oxford English Dictionary online. Retrieved September 24, 2013, from http://oxforddictionaries.com/ 


definition/english/science.
 

Unger, S. (1977). Destruction of American Indian families. New York, NY: Association on American Indian Affairs.
 

U.S. Department of Health and Human Services. (1979). Ethical principles and guidelines for the protection of human subjects of research. Retrieved December 16, 2013, from http://www.hhs.gov/ohrp/humansubjects/guidance/belmont.html. 

Warriner, G. & Engelstad K. (Directors and Producers). (1985). More than bows and arrows [Motion picture]. United 

States: Camera One. 

57 | Child Welfare Research & Evaluation Workgroups are a Project of the Children’s Bureau