# Optimized Propaganda with Bayesian Networks: Comment on "Articulating Lay Theories Through Graphical Models"

Derek Powell, Kara Weisman, and Ellen M. Markman's ["Articulating Lay Theories Through Graphical Models: A Study of Beliefs Surrounding Vaccination Decisions"](http://www.derekmpowell.com/publication/lay-theories-cogsci) (a conference paper from [CogSci 2018](https://cognitivesciencesociety.org/past-conferences/)) represents an exciting advance in marketing research, showing how to use [causal graphical models](https://www.lesswrong.com/posts/hzuSDMx7pd2uxFc5w/causal-diagrams-and-causal-models) to study why ordinary people have the beliefs they do, and how to intervene to make them be [less wrong](https://tvtropes.org/pmwiki/pmwiki.php/Main/TitleDrop).

The specific case our authors examine is that of childhood vaccination decisions: some parents don't give their babies the recommended vaccines, because they're afraid that vaccines cause autism. [(Not true.)](https://en.wikipedia.org/wiki/MMR_vaccine_and_autism) This is pretty bad—not only are those unvaccinated kids more likely to get sick themselves, but declining vaccination rates undermine the population's [herd immunity](https://en.wikipedia.org/wiki/Herd_immunity), leading to [new outbreaks of highly-contagious diseases like the measles in regions where they were once eradicated](https://en.wikipedia.org/wiki/Measles_resurgence_in_the_United_States).

What's wrong with these parents, huh?! But that doesn't have to just be a rhetorical question—Powell _et al._ show how we can use statistics to make the rhetorical [hypophorical](https://en.wikipedia.org/wiki/Hypophora) and model _specifically_ what's wrong with these people! Realistically, people aren't going to just have a raw, "atomic" dislike of vaccination _for no reason_: parents who refuse to vaccinate their children do so _because_ they're (irrationally) afraid of giving their kids autism, and not afraid enough of letting their kids get infectious diseases. Nor are beliefs about vaccine effectiveness or side-effects _uncaused_, but instead depend on other beliefs.

To unravel the structure of the web of beliefs, our authors got [Amazon Mechanical Turk](https://en.wikipedia.org/wiki/Amazon_Mechanical_Turk) participants to take surveys about vaccination-related beliefs, rating statements like "Natural things are always better than synthetic alternatives" or "Parents should trust a doctor's advice even if it goes against their intuitions" on a 7-point [Likert-like scale](https://en.wikipedia.org/wiki/Likert_scale) from "Strongly Agree" to "Strongly Disagree".

Throwing some [off-the-shelf Bayes-net structure-learning software](https://www.bnlearn.com/) at a [training set](https://en.wikipedia.org/wiki/Training,_validation,_and_test_sets) from the survey data, plus some ancillary assumptions (more-general "theory" beliefs like "skepticism of medical authorities" can cause more-specific "claim" beliefs like "vaccines have harmful additives", but not _vice versa_) produces a range of probabilistic models that can be depicted with [graphs](https://en.wikipedia.org/wiki/Graph_(discrete_mathematics)) where nodes representing the different beliefs are connected by arrows that show which beliefs "cause" others: an arrow from a _naturalism_ node (in this context, denoting a worldview that prefers natural over synthetic things) to a _parental expertise_ node means that people think parents know best _because_ they think that nature is good, not the other way around.

Learning [these kinds of models](https://www.lesswrong.com/posts/jnjjzkH8Fdzg4D6EK/causality-a-chapter-by-chapter-review) is feasible because not all possible causal relationships are consistent with the data: if $A$ and $B$ are [statistically independent](https://en.wikipedia.org/wiki/Independence_(probability_theory)) of each other, but each dependent with $C$ (and are [_conditionally_](https://en.wikipedia.org/wiki/Conditional_independence) dependent given the value of $C$), it's kind of hard to make sense of this except to posit that $A$ and $B$ are causes with the common effect $C$.

Simpler models with fewer arrows might sacrifice a little bit of predictive accuracy for the benefit of being more intelligible to humans. Powell _et al._ ended up choosing a model that can predict responses from the [test set](https://en.wikipedia.org/wiki/Cross-validation_(statistics)) at [_r_](https://en.wikipedia.org/wiki/Pearson_correlation_coefficient) = .825, [explaining](https://en.wikipedia.org/wiki/Explained_variation) 68.1% of the variance. Not bad?!—check out the full 14-node graph in Figure 2 on page 4 of [the PDF](https://mindmodeling.org/cogsci2018/papers/0183/0183.pdf).

Causal graphs are useful as a guide for planning interventions: the graph encodes predictions about what would happen if you _changed_ some of the variables. Our authors point out that since [previous work](https://www.pnas.org/content/112/33/10321) showed that people's beliefs about vaccine dangers were difficult to influence, that suggests trying to intervene on the _other_ parents of the intent-to-vaccinate node in the model: if the _hoi polloi_ won't listen to you when you tell them the costs are minimal (vaccines are safe), instead tell them about the benefits (diseases are really bad and vaccines prevent disease).

To make sure I really understand this, I want to adapt it into a simpler example with made-up numbers where I can do the arithmetic myself. Let me consider a graph with just three nodes—

![vaccines are safe → vaccinate against measles ← measles are dangerous](https://i.imgur.com/NuYrnik.png)

Suppose this represents a [structural equation model](https://en.wikipedia.org/wiki/Structural_equation_modeling) where an anti-vaxxer-leaning parent-to-be's propensity-to-vaccinate-against-measles $C$ is expressed in terms of belief-in-vaccine-safety $A$ and belief-in-measles-danger $B$ as—

$$C = 0.7 \cdot A + 0.3 \cdot B $$

And suppose that we're a public health authority trying to decide whether to spend our budget (or what's left of it after recent funding cuts) on a public education initiative that will increase $A$ by 0.1, or one that will increase $B$ by 0.3.

We should choose the program that intervenes on $B$, because $(0.3)(0.3) = 0.09$ is bigger than $(0.7)(0.1) = 0.07$. That's actionable advice that we couldn't have derived without a quantitative model of how the lay audience thinks. Exciting!

At this point, some readers may be wondering why I've described this work as "marketing research" about constructing "optimized propaganda." A couple of those words usually have _negative_ connotations, but educating people about the importance of vaccines is a _positive_ thing. What gives?

The thing is, "Learn the causal graph of why they think that and compute how to intervene on it to make them think something else" is a [symmetric weapon](https://web.archive.org/web/20200521005958/http://slatestarcodex.com/2017/03/24/guided-by-the-beauty-of-our-weapons/)—a _fully general_ persuasive technique that doesn't [depend on whether the thing you're trying to convince them of is _true_](http://benjaminrosshoffman.com/humility-argument-honesty/).

In my simplified example, the choice to intervene on $B$ was based on numerical assumptions that amount to the claim that it's sufficiently easier to change $B$ than it is to change $A$, such that intervening on $B$ is more effective at changing $C$ than intervening on $A$ (even though $C$ depends on $A$ more than it does on $B$). But this methodology is _completely indifferent_ to what $A$, $B$, and $C$ _mean_. It would have worked just as well, and _for the same reasons_ if the graph had been—

![Coca-Cola isn't unhealthy → drink Coca-Cola ← Coca-Cola tastes great](https://i.imgur.com/lQmo66J.png)

Suppose that we're advertising executives for the Coca-Cola Company trying to decide how to spend our budget (or what's left of it after recent funding cuts). If consumers won't listen to us when we tell them the costs of drinking Coke are minimal (lying that it isn't unhealthy), we should instead tell them about the benefits (Coke tastes good).

Or with different assumptions about the parameters—maybe $C = 0.8 \cdot A + 0.2 \cdot B$ actually—then intervening to increase belief in "Coca-Cola isn't unhealthy" _would_ be the right move (because $(0.8)(0.1) = 0.08 > 0.06 = (0.2)(0.3)$). The [marketing algorithm](https://www.lesswrong.com/posts/P3FQNvnW8Cz42QBuA/dialogue-on-appeals-to-consequences?commentId=bAQBHN2svqS6BfmSM) that just computes _what belief changes will flip the decision node_, doesn't have any way to notice or care whether those belief changes are in the direction of more or less accuracy.

To be clear—and I really _shouldn't_ have to say this—this is not a criticism of Powell–Weisman–Markman's research! The "Learn the causal graph of why they think that" methodology is genuinely really cool! It doesn't _have_ to be deployed as a marketing algorithm: the process of figuring out which belief change would flip some downstream node is the same thing as what we call locating a [crux](https://www.lesswrong.com/posts/exa5kmvopeRyfJgCy/double-crux-a-strategy-for-resolving-disagreement).[^crux] The difference is just a matter of [forwards or backwards direction](https://www.lesswrong.com/posts/SFZoEBpLo9frSJGkc/rationalization): whether you _first_ figure out if the measles vaccine or Coca-Cola are safe and [_then_ use whatever answer you come up with to guide your decision](https://www.lesswrong.com/posts/9f5EXt8KNNxTAihtZ/a-rational-argument), or whether you [write the bottom line first](https://www.lesswrong.com/posts/34XxbRFe54FycoCDw/the-bottom-line).

[^crux]: Thanks to [Anna Salamon](https://www.lesswrong.com/users/annasalamon) for this observation.

Of course, most people on most issues don't have the time or expertise to do their own research. For the most part, we can only hope that the sources we trust as authorities are doing their best to use their [limited bandwidth](https://www.lesswrong.com/posts/4ZvJab25tDebB8FGE/you-have-about-five-words) to keep us genuinely informed, rather than merely computing what [signals to emit](https://www.lesswrong.com/posts/4hLcbXaqudM9wSeor/philosophy-in-the-darkest-timeline-basics-of-the-evolution) in order to control our decisions.

If that's _not_ true, we might be in trouble—perhaps increasingly so, if technological developments grant new advantages to the propagation of disinformation over the discernment of truth. In [a possible future world](https://www.lesswrong.com/posts/HBxe6wdjxK239zajf/what-failure-looks-like) where _most_ words are produced by AIs running a "Learn the causal graph of why they think that and intervene on it to make them think something else" algorithm hooked up to a next-generation [GPT](https://www.lesswrong.com/tag/gpt), even [reading plain text from an untrusted source could be dangerous](https://www.alignmentforum.org/posts/5bd75cc58225bf06703754b9/autopoietic-systems-and-difficulty-of-agi-alignment?commentId=5bd75cc58225bf06703754c1).