# Bayesian probability theory as extended logic -- a new result

I have a new paper that strengthens the case for strong Bayesianism, a.k.a. [One Magisterium Bayes](/r/discussion/lw/p71/onemagisterium_bayes/ "One Magisterium Bayes"). The paper is entitled "[From propositional logic to plausible reasoning: a uniqueness theorem](http://authors.elsevier.com/a/1VIqc,KD6ZCKMf)." (The preceding link will be good for a few weeks, after which only the [preprint version](https://arxiv.org/abs/1706.05261) will be available for free. I couldn't come up with the $2500 that Elsevier makes you pay to make your paper open-access.)

Some background: E. T. Jaynes took the position that (Bayesian) probability theory is an extension of propositional logic to handle degrees of certainty -- and appealed to Cox's Theorem to argue that probability theory is the **only** viable such extension, "the unique consistent rules for conducting inference (i.e. plausible reasoning) of any kind." This position is sometimes called _strong Bayesianism_. In a nutshell, frequentist statistics is fine for reasoning about frequencies of repeated events, but that's a very narrow class of questions; most of the time when researchers appeal to statistics, they want to know what they can conclude with what degree of certainty, and that is an _epistemic_ question for which Bayesian statistics is the right tool, according to Cox's Theorem.

You can find a "guided tour" of Cox's Theorem [here](http://ksvanhorn.com/bayes/papers.html) (see "Constructing a logic of plausible inference"). Here's a very brief summary. We write _A | X_ for "the reasonable credibility" (plausibility) of proposition _A_ when _X_ is known to be true. Here _X_ represents whatever information we have available. We are **not** at this point assuming that _A | X_ is any sort of probability. A system of plausible reasoning is a set of rules for evaluating _A | X_. Cox proposed a handful of intuitively-appealing, qualitative requirements for any system of plausible reasoning, and showed that these requirements imply that any such system is just probability theory in disguise. That is, there necessarily exists an order-preserving isomorphism between plausibilities and probabilities such that _A | X_, after mapping from plausibilities to probabilities, respects the laws of probability.

Here is one (simplified and not 100% accurate) version of the assumptions required to obtain Cox's result:

1.  _A | X_ is a real number.
2.  _(A | X) = (B | X)_ whenever _A_ and _B_ are logically equivalent; furthermore, (_A | X) ≤ (B | X)_ if _B_ is a tautology (an expression that is logically true, such as (_a or not a)_).
3.  We can obtain (_not A | X)_ from _A | X_ via some non-increasing function _S_. That is, _(not A | X) = S(A | X)_.
4.  We can obtain (_A and B | X)_ from _(B | X)_ and _(A | B and X)_ via some continuous function _F_ that is strictly increasing in both arguments: _(A and B | X) = F((A | B and X), B | X)_.
5.  The set of triples _(x,y,z)_ such that _x = A|X_, _y = (B | A and X)_, and _z = (C | A and B and X)_ for some proposition _A_, proposition _B_, proposition _C_, and state of information _X_, is dense. Loosely speaking, this means that if you give me any _(x',y',z')_ in the appropriate range, I can find an _(x,y,z)_ of the above form that is arbitrarily close to _(x',y',z')_.

The "guided tour" mentioned above gives detailed rationales for all of these requirements.

  

Not everyone agrees that these assumptions are reasonable. My paper proposes an alternative set of assumptions that are intended to be less disputable, as every one of them is simply a requirement that some property _already true of propositional logic_ continue to be true in our extended logic for plausible reasoning. Here are the alternative requirements:

1.  If _X_ and _Y_ are logically equivalent, and _A_ and _B_ are logically equivalent assuming _X_, then _(A | X) = (B | Y)_.
2.  We may define a new propositional symbol _s_ without affecting the plausibility of any proposition that does not mention that symbol. Specifically, if _s_ is a propositional symbol not appearing in _A_, _X_, or _E_, then (_A | X) = (A | (s ↔ E) and X)_.
3.  Adding irrelevant background information does not alter plausibilities. Specifically, if _Y_ is a satisfiable propositional formula that uses no propositional symbol occurring in _A_ or _X_, then (_A | X) = (A | Y and X)_.
4.  The implication ordering is preserved: if  _A → B_ is a logical consequence of _X_, but _B → A_ is not, then then _A | X_ < _B | X_; that is, _A_ is strictly less plausible than _B_, assuming _X_.

Note that I **do not** assume that _A | X_ is a real number. Item 4 above assumes only that there is some _partial_ ordering on plausibility values: in _some_ cases we can say that one plausibility is greater than another.

I also explicitly take the state of information _X_ to be a propositional formula: all the background knowledge to which we have access is expressed in the form of logical statements. So, for example, if your background information is that you are tossing a six-sided die, you could express this by letting _s1_ mean "the die comes up 1," _s2_ mean "the die comes up 2," and so on, and your background information _X_ would be a logical formula stating that exactly one of _s1_, ..., _s6_ is true, that is,

  

_(s1 or s2 or s3 or s5 or s6) and_

_not (s1 and s2) and not (s1 and s3) and not (s1 and s4) and_

_not (s1 and s5) and not (s1 and s6) and not (s2 and s3) and_

_not (s2 and s4) and not (s2 and s5) and not (s2 and s6) and_

_not (s3 and s4) and not (s3 and s5) and not (s3 and s6) and_

_not (s4 and s5) and not (s4 and s6) and not (s5 and s6)._

  

Just like Cox, I then show that there is an order-preserving isomorphism between plausibilities and probabilities that respects the laws of probability.

  

My result goes further, however, in that it gives actual numeric values for the probabilities. Imagine creating a truth table containing one row for each possible combination of truth values assigned to each atomic proposition appearing in either _A_ or _X_. Let _n_ be the number of rows in this table for which _X_ evaluates true. Let _m_ be the number of rows in this table for which both _A_ and _X_ evaluate true. If _P_ is the function that maps plausibilities to probabilities, then _P(A | X) = m / n_.

  

For example, suppose that _a_ and _b_ are atomic propositions (not decomposable in terms of more primitive propositions), and suppose that we only know that at least one of them is true; what then is the probability that _a_ is true? Start by enumerating all possible combinations of truth values for _a_ and _b_:

1.  _a_ false, _b_ false: _(a or b)_ is false, _a_ is false.
2.  _a_ false, _b_ true : _(a or b)_ is true,  _a_ is false.
3.  _a_ true,  _b_ false: _(a or b)_ is true,  _a_ is true.
4.  _a_ true,  _b_ true : _(a or b)_ is true,  _a_ is true.

There are 3 cases (2, 3, and 4) in which _(a or b)_ is true, and in 2 of these cases (3 and 4) _a_ is also true. Therefore,

  

    _P(a | a or b) = 2/3._

  

This concords with the classical definition of probability, which Laplace expressed as

  

> The probability of an event is the ratio of the number of cases favorable to it, to the number of possible cases, when there is nothing to make us believe that one case should occur rather than any other, so that these cases are, for us, equally possible.

  

This definition fell out of favor, in part because of its apparent circularity. My result validates the classical definition and sharpens it. We can now say that a “possible case” is simply a truth assignment satisfying the premise _X_. We can simply drop the problematic phrase “these cases are, for us, equally possible.” The phrase “there is nothing to make us believe that one case should occur rather than any other” means that we possess no additional information that, if added to _X_, would expand by differing multiplicities the rows of the truth table for which _X_ evaluates true.

  

For more details, see the paper linked above.