# Unlearning and Seyab's theorem: a dialogue about updating probability

P.G.L. Porta Mana

<figure><img src="image_1.png" /></figure>

## To cite this version:

P.G.L. Porta Mana. Unlearning and Seyab's theorem: a dialogue about updating probability. 2018. [hal-01726127](https://hal.science/hal-01726127)

HAL Id: hal-01726127

https://hal.science/hal-01726127

Preprint submitted on 8 Mar 2018

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Unlearning and Seyab's theorem
a dialogue about updating probability

P.G.L. Porta Mana <mana@kth.se>

27 January 2018

> "You can never know everything," Lan said quietly, "and part of what you know is always wrong. Perhaps even the most important part. A portion of wisdom lies in knowing that. A portion of courage lies in going on anyway."

(Jordan 2010)

*The interlocutors Salviati and Simplicio are two students very passionate about the probability calculus, logic, and philosophy.*

**Salv.** It's beautifully simple how Bayes's theorem,

$$P(B|A \land I) = \frac{P(A|B \land I) P(B|I)}{P(A|I)} \quad (1)$$

relates the probabilities we give to a proposition *B* in the two cases: when we know only *I*, and when we also know *A* besides *I*. Isn't it, Simplicio?

**Simp.** It surely is, Salviati. Don't forget that the theorem requires you to specify either P(*A* | *B* ∧ *I*) and P(*A* ∧ *I*), or P(*A* ∧ *B* | *I*) and P(*A* | ¬*B* ∧ *I*), or some similar combination, to get the latter of the two probabilities you mention from the former.

**Salv.** Of course. And as you just said, we usually calculate P(*B* | *A* ∧ *I*) given P(*B* | *I*), don't we?

**Simp.** Yes, that's quite natural, because the theorem tells us how to update our probability about *B* when we learn *A*. That's why we call P(*B* | *A* ∧ *I*) the *posterior* probability and P(*B* | *I*) the *prior* probability.

**Salv.** You see, I was thinking about something curious connected with this *prior* and *posterior* business. In our lives we often learn new things and acquire new knowledge. But at times we also unlearn, forget, lose data, or discover that some of our knowledge is unfounded. Don't we?

**Simp.** Absolutely! Something like that happened to me recently. Do you remember what they told us in our classes in quantum mechanics
– that if you prepare a particle with a very precise position, then a measurement of its momentum will give a completely uncertain result?

**Salv.** I remember. Of course, position and momentum are conjugate quantities.

**Simp.** Well, what our teachers said isn't really true. It turns out that if you make one measurement of the momentum of that particle, you can get a definite result, as precise as you like.

**Salv.** Heresy!

**Simp.** It's true and actually very sensible. Consider a simpler case: you prepare an electron with spin in the positive x direction, say, and then make one measurement of its spin in the y direction. The result of the measurement is one definite spot on the screen at the exit of the measuring Stern-Gerlach apparatus (Peres 1995 fig. 1.5, § 1-5, p. 14): we know precisely the result, either +ħ/2 or -ħ/2 in the y direction. Yet the measurement error, according to the standard formulae, is ħ/2.

**Salv.** That sounds true. But then how do you reconcile this with the formulae? Were our teachers lying?

**Simp.** The point is that the formulae refer to the statistics of *many* measurements, not just one. If you repeat the spin measurement with the same experimental setup, you'll observe a new and precise result, either +ħ/2 or -ħ/2. As you repeat this kind of experiment, both results will be observed in about a fifty-fifty ratio. The "measurement error" refers to the standard deviation in *many* measurements. The situation with position and momentum is analogous. Our teachers were not lying but using a too vague or slightly wrong terminology about what the "measurement error" was. Read Ballentine (1970), or take a look at the experimental results in Leonhardt (1997) – e.g. his fig. 2.1, § 2.2, p. 23 – to have a better understanding of the situation. No paradoxes.

**Salv.** Amazing, it looks like I have also unlearnt something today.

**Simp.** As you said, it happens. But how does this relate to Bayes's theorem?

**Salv.** Right. You see, I was thinking about the three rules of the probability calculus,

$$ 
\begin{gathered}
P(\neg A | I) = 1 - P(A | I), \qquad P(B \wedge A | I) = P(B | A \wedge I) P(A | I), \\ 
P(A \lor B | I) = P(B | I) + P(A | I) - P(B \wedge A | I).
\end{gathered} 
\qquad (2) 
$$
These rules, combined, give us relations among probabilities with different conditionals. By the way, do you mind if I use the term *context* to refer to the conditional?

**Simp.** It isn't standard terminology I think, but it sounds like a sensible term. Maybe you could even use *situation*, similarly to Barwise's (1989) notion in Logic; I've been wondering for a long time whether Barwise's notion of *situation* in Logic has deeper connections with probability conditionals. But that's a discussion topic for another time. Please go on.

**Salv.** Thank you. Bayes's theorem is an example of what I was saying about relations between contexts: it relates the probabilities for *B* in the context where we know *I* and in the context where we know *A* ∧ *I*. Now here's my point: the probability rules tell us nothing about which of these two contexts comes first in time; they don't say which should be *prior* and which *posterior*. Now imagine a situation in which we have knowledge of *A* ∧ *I*, but some time later we forget whether *A* is true. Then P(*B* | *A* ∧ *I*) would be our *prior* probability, and P(*B* | *I*) our *posterior*.

**Simp.** Mmm... In that case wouldn't we also forget the numerical value of P(*B* | *A* ∧ *I*)? Or, if we have saved the value of this probability somewhere, it would remind us that we knew *A* was true, so we should still need to consider P(*B* | *A* ∧ *I*), shouldn't we?

**Salv.** Right, my example wasn't very logical. Consider this then: we are initially sure about *A*, besides *I*, so we use probabilities with the context *A* ∧ *I*. But then we somehow discover that *A* isn't actually certain; our certainty was unfounded. Upon this discovery we should start using probabilities with context *I* only. Our probability for *B* should be updated in reverse, using “Seyab's theorem”:

$$P(B|I) = P(A|I) \frac{P(B|A \land I)}{P(A|B \land I)}, \quad (3)$$

which expresses our *unlearning* of *A*. The probability P(*B* | *A* ∧ *I*) is our *prior*, and P(*B* | *I*) our *posterior*. The probability P(*A* | *I*) appears as a normalization factor just like in Bayes's theorem.

**Simp.** I feel that there's an inconsistency in your example. If the context *I* doesn't give an extremal, 0 or 1 probability to *A*, then that context does not contradict the possibility that *A* might be true. So in that context there's the possibility that we discover later that *A* is true, and
therefore move A into the context. But once A enters the context, and we use the new context A ∧ I, the possibility that A might be uncertain contradicts this new context. Hence we can't revert to using I alone.

**Salv.** Yours is an interesting assertion. You seem to be invoking an additional rule on how to use the three probability rules. Do the three rules (2) say that you can't take a proposition out of a context, once you're using that context for your plausibility judgements? Where or how do the rules exactly say this?

**Simp.** Well, the probability rules don't prescribe which context we should be using, for that matter. I think it's implicit in their use that the context of probability judgement should reflect our state of knowledge or whatever working hypotheses we're entertaining. Just like in truth logic we use axioms that express what we know, when we want to reason out a conclusion deductively.

**Salv.** I agree with the point of view you just expressed. But I think it endorses what I was saying: if we are initially sure about A and I, then we should use the context A ∧ I; if we later become uncertain about A, then we should switch to the context I. I thought you agreed with me that in real life we can happen to become unsure about something we were previously sure of.

**Simp.** Your reasoning appears formally correct, but I still can't shake off the feeling that it contains a contradiction. OK, Let me change my argument a little. I think that we shouldn't have used the context A ∧ I in the first place. You say: "we're initially certain about A"; but I rather imagine that we initially didn't know A directly: we had some other kind of knowledge instead, say expressed by the proposition O, that gave almost unit probability to A:

$$P(A|O \land I) = 1 - \epsilon, \quad \epsilon \ll 1, \qquad (4)$$

and, by the probability rules,

$$ \begin{aligned} P(B|O \land I) &= P(B|A \land O \land I) P(A|O \land I) + \\ &P(B|\neg A \land O \land I) P(\neg A|O \land I), \end{aligned} \qquad (5) \\ \approx P(B|A \land O \land I). $$

So O ∧ I, not A ∧ I, was our initial context, leading to probability judgements very similar to assuming A to be true. When you say "we discover that A is uncertain", I imagine that we learn some new fact, say
expressed by $O'$, that makes $A$ again very uncertain, even nullifying our knowledge of $O$:

$$P(A|O' \land O \land I) = P(A|I), \qquad (6)$$

and in this new context, $O' \land O \land I$, our probability judgement for $B$ is numerically very close to $P(B|I)$. I think this analysis gives numerical values similar to yours, but is closer to what would happen in a real situation of “unlearning”.

**Salv.** Your analysis is absolutely valid and can be an accurate representation of many a real situation. But I think that it lends a hand to my own argument. First, I can still imagine a situation in which we become unsure about $O$, the proposition that you assumed to be certain. In this case I suppose we could make an analysis similar to the one you just made, but with $O$ in place of $A$, introducing some new proposition $N$ relating to $O$ as $O$ related to $A$. But this game could go on forever.

**Simp.** You seem to be saying that indubitable, certain knowledge is impossible to acquire. I completely disagree with such philosophical position.

**Salv.** That wasn't my intention. My point is to show that your analysis doesn't deny the possibility that we might become unsure about something we were sure of. Your analysis doesn't show any contradiction in my position; it only sidesteps it.

**Simp.** That's true. I feel the need to sidestep it because, to me, saying that $A$ is certain seems to exclude the possibility that we can become uncertain about it. Doesn't this possibility contradict the notion of "certainty" itself? Wouldn't it mean that we were *not* completely certain about $A$ in the first place?

**Salv.** That's an important philosophical and semantic question. I partly understand what you mean. And yet I'm not convinced that being sure about something at some time contradicts being unsure about it some time later. In any case, what's important is that the probability rules make allowance for this possibility; don't they? Seyab's theorem, derived from them, seems to suggest exactly this possibility.

**Simp.** I agree, I don't have a philosophical proof of my view about "certainty". To be honest I'm not acquainted with any philosophical works about this concept. I was just giving voice to my common sense; but it seems yours is at variance with mine. My analysis shows that
we can make sense of your “unlearning”, though, by adding more propositions to the context rather than by deleting them.

**Salv.** The numerical probability values of your analysis are practically identical with the ones we would obtain with Seyab's theorem (3). Or at least, your analysis doesn't show that direct application of Seyab's theorem would lead to inconsistent values. So why can't we just apply it? Why can't we simply move *A* out of the context?

**Simp.** With some more thinking I might find a contradiction in your view. But for the moment let me ask you a different question: in which situations could your Seyab theorem be used? Do you have any concrete applications in mind?

**Salv.** No, I don't. As I said, it was a curiosity and prompted my question about updating contexts. But we live in an age when knowledge and information are stored at a rate never experienced before. For this reason, loss and deterioration of knowledge and information become more likely, too. The outcome of a laboratory experiment could be misrecorded, the results of a medical test could be lost, a meteorological report could be misinterpreted. Couldn't Seyab's theorem turn out to be useful in such cases?

**Simp.** I don't think it likely, but as a scientist I'll give you the benefit of doubt of course. We never know.

## Thanks

... to Jakob Jordan and Alper Yegenoglu for many insightful discussions about probability conditionals. To Mari & Miri for continuous encouragement and affection. To Buster Keaton and Saitama for filling life with awe and inspiration. To the developers and maintainers of LATEX, Emacs, AUCTEX, Open Science Framework, biorXiv, PhilSci, Hal archives, Python, Inkscape, Sci-Hub for making a free and unfiltered scientific exchange possible. ❖

## Bibliography

Ballentine, L. E. (1970): *The statistical interpretation of quantum mechanics*. Rev. Mod. Phys. **42**号, 358-381.

Barwise, J. (1989): *The Situation in Logic*. (CSLI, Stanford).

Jordan, R. (2010): *Winter's Heart*. (Tor, New York). First publ. 2000.

Leonhardt, U. (1997): *Measuring the Quantum State of Light*. (Cambridge University Press, Cambridge).

Peres, A. (1995): *Quantum Theory: Concepts and Methods*. (Kluwer, Dordrecht).