# Evaluation Theories/Week 5: Evaluation Theory Panel: Patton, Cronbach, Chen, Guba & Lincoln

yawnnotes Click here for the Google Doc with class notes - * These need to be imported and cleaned up for Wikiversity!

### Q&A & Feedback

Today; while T1 hearts Cronbach, I heart Huey T. Chen;
Sophisticated Saying: “If it ain’t broke, don’t fix it.” - With Moderating; T1 did it so well that we’ll keep her moderating.
In Week 7: TODO: Turn in a 2-page Reflection paper.
You will have gone through the 12 Theorists; and we want this paper to be an opportunity to take you across theorists.
* 2. 1. Top three salient issues
* 3. Brief: To pages; double spaced; Times New Roman; 1” Margins - For Week 7.
Come prepared with a question (J: ???)

## Patton

Utility Feasibility Propriety and Accuracy;
Sarah: Used to be an evangelical preacher; said talking to people about whether their life was consistent with their values.
13:17:11 Lee Joseph Cronbach
Evaluation as a ____ for education; adress particular needs; create a forum where everyone can interact; educate; influence social policy.

## Guba

PhD Stats. Qual Research Special Interest Group Eval Center Western Mich. Teas A&M

## Lincoln:

Construct knowledge for the specific context;

## Huey Chen

* Born & Raised in Taiwan; U Alabama Birmingham Public Health
* (WHY THE NAME OF AWARDS?!!?! :))
* National Evaluation System for Evaluating a CDC-Funded HIV Education Program (Josh: WOOT! lol, I should go evaluate the Sudan one I produced!)
Pseudo-Patton: How do you think your academic training / background influenced how you think?
AS PSEUDO-PATTON, I THINK THE ORGANIZATIONAL PART OF MY DEGREE WAS IMPORTANT; - I KEEP UP WITH MY TEACHING BACKGROUNDWITH THE INTERNATIONAL DEVELOPMENT EVALUATION TRAINING; I DO A LOT OF TRAINING
CDC: TAXONomies for how to develop a useful evaluation. (J: I want ot make the taxonomies of EVALUATION - the use I can’ 13:30:52ake a big impact in; too many brilliant people are already working in it. The bigger picture is not together though. I love being in that position, because i’m usually
_____ completely reshaped his ideas of what evaluation is.

## Cronbach

…. Unlike BLakc-box Evaluation;

## GUBA

Valueing: Basic approach: construct knowledge that is context bound.
(J: What is good about this system: it’s good for looking at who is well developed in what area. . .but mutually exclusive and collectively exhaustive (MECE) (Ethan Rasiel, 1999, 'The McKinsey Way', p. 6; and which I just ran across in MIT’s introduction to Probability course…)

## Misc.

13:54:29 - ____ Most well developed is SOcial Programming;
Because…
Mentor: - Cronbach a bit abstract often
14:19:09 - Cronbach UTOS:
External & Internal Validity: Campbell & Cronbach:
14:26:08 - Guber ;
Stakeholder APproach; Social Sciences Approach; ____ Approach.
Former Theory &
Balance between Stakeholder and Scientific Credibility.
* Type 1: False Positive
* Type 2: You think there isn’t an effect when there really is.
* Type 3:
* - the right answer to the wrong question. This is sometimes called a Type 0 error
* - Type III error occurs when you correctly conclude that the two groups are statistically different, but you are wrong about the direction of the difference. (http://graphpad.com/support/faqid/1080/)
* - Chen: Theor
y was good; implementation was flawed. (Tiffany Berry P315Z; 2014-02-19)

## Hermeneutic exchange

Definition:
* 8-Step Process:
* 1. You don’t know everything
* 2. Look at Stakeholders separately (using Maximum variation sampling)
* 3.
* 4.
Q:What information do you gather from these stakeholders? -
Do you educate stakeholders to ensure they can keep on doing things after you leave?
- What makes this different from “Including Everyone else’s point of view?
Cronbach - Policy-Making; Info for decision-makin:
I’d advise: 1. Multiple perspectives 2. Embrace people usually excluded (J: Rawls) but not while silencing those who are already in control 3. Explicate dynamics of power and privilege interculturalhe program. 4. ___ context; not idly serve ____
Multiple Stakeholder Groups: You’ve Identified some: Most important groups of stakeholders that you would involve: 1. Public Servants (those who make the program for the public) (J: Our favourite public servants: Google. Re: Google Books Project. Until you @#$@# them and they became evil when they realized their idealism wouldn’t fly in the real world) 2. Members of the Program (who it’s made for)
Mentioned Leverage: Previous Ambiguity; Adjustment at the Margins?

## INTEGRATIVE VALIDITY MODEL

three components. Campbelian Typology of Validity
* Viability
* Effectuality
* Transferability
Superior to Campbell because Campbell’s approach Lab -> Real world; mine goes: “Can it work in Real world?” - then “Actually works in real world?” then ____
Transferrable validity: Revision of Campbell that takes into account the stakeholder’s perspective (Lisa ____, 2014-02-19)
- very practice oriented; more expansive idea of validity than the traditional typology.
Viable Validity: Should we even bother; do we have the resources; buy-in from stakeholders? Is it cost effective?
Typologies: External Validity; Focus is on how to rule out threats to validity. Becomes less and less relevant to stakeholders.
Viable Vlaidity: a world-class center doesn’t matter at all if you put it on a hill and there’s no escalator up: IT doesn’t matter at all if we can’t get to it; can’t access it.
Really good thing about Halcolm from ___ (girl in the pink jacket)

## Student Generated Questions:

Gets continued throughout the process: at every stage you’re doing this exchange; ensuring that entire population is informed; contributing; feeding back information through entire profcess. I don’t think you write as clearly about the process of evaluation as ____. . . (Guba & Lincoln)

## Q&A:

Constant implementation failure but people persist in pursuing the overarching causal or program theory; what advice would you give in that situation? -
A: We deal with that all the time. Implmeentation is always flawed: it’s just levels of flawedness. Sometimes its so severe. . . one project wer’e advising now, Dosage is a big part of the implmentation; but once they looked at the [dosage] - they were serving these kids 2 hours per year, and were supposed to improve their resilience and grit; . .. ____ - (Great personal story)
Implementation is always flawed: that’s why it’s absolutely critical to measure it, always.