# Frequentist vs Bayesian breakdown: interpretation vs inference

Suppose we have two different human beings, Connor and Diane, who agree to **interpret** their subjective anticipations as probabilities, thereby commonly earning them the title "Bayesian".  On a particular project or venture, they might disagree on Trick A or Trick B to decide the next step in the project.  It might be that Trick A is commonly labelled a "Frequentist **inference** method" and B is a "Bayesian **inference** method".  Why might they disagree?

As far as I can see, there are 3 disagreements that get labelled "Bayesian vs Frequentist" debates, and conflating them is a problem:

(1) Whether to **interpret** all _subjective anticipations as probabilities_.

(2) Whether to **interpret** all _probabilities as subjective anticipations_.

(3) Whether, on a particular project, to _use Statistical Trick B instead of Statistical Trick A_ to **infer** the best course of action, when B is commonly labelled a "Bayesian method" and A is a "Frequentist method".

(Regarding 3, UC Berkeley professor Michael Jordan offers a good heuristic for how statistical tricks get labelled as Bayesisn or Frequentist, in terms of which terms in a loss function one treats as fixed or variable.  I recommend [watching the first twenty minutes of his video lecture](/lw/2xe/interesting_talk_on_bayesians_and_frequentists/) on this if you're not familiar.)

The question "is Connor a Bayesian or a Frequentist?" is commonly posed as though Connor's position on 1, 2, and 3 must be either "yes, yes, yes" or "no, no, no".  I don't believe this is so often the case. For example, my position is:

(1) - Yes.  Insofar as we have subjective anticipations, I agree normatively that they should behave and update as probabilities.

(2) - Don't care much. Expressions like P(X|Y) and P(X and Y) are useful for denoting both _subjective anticipations_ and proportions of a whole, and in particular, _proportions of real future events_.  Whether to use the word "probability" is a terminological question.  Personally I try to reserve the word "probability" for when they mean subjective anticipations, and say "proportion" when they mean proportions of real future, but this is word choice.  Unfortunately this word choice is strongly associated and confused with positions on (1) and (3).

(3) - It depends.  In statistical inference, we commonly consider data sets **x**, world models **M**, and parameters **θ** that specify the model **M** more precisely.  I consider the separation of belief into **M** and **θ** to be purely formal. When guessing the next data set **y**, one considers expressions of the form **P**(**x|M,****θ**) in some way.  If I'm already very confident in a specific world model **M**, and expect **θ** to actually vary from situation to situation, I'll probably try to estimate the parameters **θ** from **x** in a way that has the best expected success rate across all possible data sets **M** would generate. You might say here that I "trust the model more than the data" (though what I really don't trust are the changing model parameters), and this is a trick commonly referred to as "Frequentist".  If I'm not confident in the model **M**, or expect the parameters **θ** to the be the same in many future situations, I'll probably try to estimate **M,****θ** from **x** in a way that has the best expected success rate** assuming x.**  You might say here that I "trust the data more than the model", and label this a "Bayesian" trick.  

Throughout (3), since my position in (1) is not changing, a member of the Bayes Tribe will say I'm "really a Bayesian all along", but I don't want to continue with this conflation of position names.  It's true that if I use the "Frequentist trick", it will be because I've updated in favor of it, i.e. my subjective confidence levels in the various theory elements are appropriate for it. 

... But from now on, when term "Bayesian" or "Frequentist" arises in a debate, _my plan is to taboo the terms immediately_, and proceed to either dissolve the issue into (1), (2), and (3) above, or change the conversation if people don't have the energy or interest for that length of conversation.

Do people agree with this breakdown?  I think I could be persuaded otherwise and would of course appreciate it if I were :)

* * *

ETA: I think the wisdom to [treat beliefs as anticipation controllers](/lw/i3/making_beliefs_pay_rent_in_anticipated_experiences/) and [update our confidences based on evidence](http://wiki.lesswrong.com/wiki/Belief_update) might be too precious to alienate people from it with the label "Bayesian", especially if the label is as ambiguous as my breakdown has found it to be.