# Documentary Impact Evaluation

One model for measuring the impact of documentaries is presented in a 2014 paper by Diesner, Pak, Kim, Soltani, & Aleyasen

## Elements in the Model

* Dimensions: Content, Medium, Target, Impact,
** Level: multiple levels exist within each dimension, Diesner et. al defines this as a "set of sub-categories of evaluation criteria"
*** Evaluation Factors (what Diesner et. al call "Index") the set of evaluation factors in a given level.
*** Analytics = analytic methods that can be used for each evaluation factor
* Items: sets of "specific features to be measured" within each evaluation factor.

## Dimension 1: Content
### Levels:

* Message = This can be identified by interviews with the film-maker, document analysis of governing and planning documents for the film, and analysis of the content of the film itself. Diesner et. al define a message as "the main message that a film wants to convey." Reflection questions: 1) Do you think that this definition is necessary and sufficient for the "Message" of a documentary? 2) What would you add or subtract from this definition? 3) Why?
* Expected Outcome = These can be found in the grant application, through interviews with the film-makers, etc.
* Evaluation Priority = Create a list of priorities based on the outcomes identified in the step above, and rank them in order of importance.
* Resource = The resources needed to produce the film. This level is necessary for any evaluation of cost-effectiveness.

### Evaluation Factors:

* "Guiding Factor"

### Analytics:

* Description
* Ranking
* Weighing

### Items:

* Reports by producers (Diesner et al, 2014, Table 2, p.6)
* Reports by funding agencies (Diesner et al, 2014, Table 2, p.6)

## Dimension 4: Impact
### Levels:
#### Level 1

* Individual
* Communal
* Societal
* Global
Diesner et al identify four categories of measurement that For each of these levels:
* Cognitive <- Evaluation factor "Awareness", analytical methods: "Statistics, Text mining, web analytics, network analysis", as well as surveys
* Attitudinal <- Evaluation Factor "Sentiment", analytical method: "Sentiment Analysis"
* Behavioral <- analytical methods: text mining, web analytics, network analysis, as well as observational protocols.
** Engagement <-
** Enactment
** Connectedness
** Capacity
** Expansiveness
** Centralization
* Temporal <- Evaluation factor: "Impact Dynamics", analytical method: "Longitudinal analysis"
Diesner et all call these the "four stimulous dimensions" (p. 8)

### Analytics:

* Text Mining (mentioned twice, for Behavioral and Cognitive levels)
* Web Analytics (mentioned twice, for Behavioral and Cognitive levels)
* Network Analysis (mentioned twice, for Behavioral and Cognitive levels)
* Longitudinal analysis (mentioned once, for Impact Dynamics)
* Sentiment Analysis (mentioned once, for Attitudinal level)
Applicable to Temporal categories:
* Longitudinal Analysis

### Items:

* ↑ (Diesner, Pak, Kim, Soltani, & Aleyasen, 2014, Diesner, Jana, Susie Pak, Jinseok Kim, Kiumars Soltani, and Amirhossein Aleyasen. “Computational Assessment of the Impact of Social Justice Documentaries.” iConference 2014 Proceedings, 2014. https://www.ideals.illinois.edu/handle/2142/47312.
* ↑ (Diesner, Pak, Kim, Soltani, & Aleyasen, 2014)