# Improving the modal UDT optimality result


$$\newcommand{\box}{{\square}}$$
$$\newcommand{\q}[1]{\ulcorner{#1}\urcorner}$$
$$\newcommand{\PA}{\mathrm{PA}}$$
$$\newcommand{\GL}{\mathrm{GL}}$$
$$\newcommand{\NN}{\mathbb{N}}$$
$$\newcommand{\lr}{\leftrightarrow }$$
$$\newcommand{\UDT}{{\mathrm{UDT}}}$$
$$\newcommand{\UDTp}{\UDT^{(\varphi)}}$$
$$\newcommand{\T}{\vec T(\vec u)}$$
$$\newcommand{\P}{\vec P(\vec a)}$$
$$\newcommand{\A}{\vec A}$$
$$\newcommand{\AT}{\A^{(T)}}$$
$$\newcommand{\U}{\vec U}$$
$$\newcommand{\UT}{\U^{(T)}}$$

I recently [posted about an optimality result for modal UDT](/item?id=50), which shows that for every [modal decision problem](/item?id=47) $\vec P(\vec a)$, there is a closed modal formula $\varphi$ such that the version of modal UDT that searches for proofs in $\PA + \varphi$ will perform optimally on $\vec P(\vec a)$.

Paul commented on this post and [suggested a stronger version](/item?id=56): For every modal decision theory $\vec T(\vec u)$ and every provably extensional modal decision problem $\vec P(\vec a)$, modal UDT will do at least as well on $\vec P(\vec a)$ as $\vec T(\vec u)$ does if it is using a proof system that can prove what action $\vec T(\vec u)$ chooses on this decision problem, and which outcome it obtains as a result. In this post, I give a detailed proof of this.

**Prerequisite**: [An optimality result for modal UDT](/item?id=50), and the prerequisites therein.

***

Let $(\AT,\UT)$ be the fixed point of $\vec T(\vec u)$ and $\vec P(\vec a)$, and recall my notation $\vec\chi^{(i)}$ for the sequence of formulas which has $\top$ as the $i$'th entry, and $\bot$ as all of its other entries (with the length of the sequence being clear from context). Now suppose that $\varphi$ is a true closed formula in the language of $\GL$ such that
$$\GL\,\vdash\,\varphi\to\big((\AT\lr\vec\chi^{(i^*_T)})\wedge(\UT\lr\vec\chi^{(j^*_T)})\big);$$
that is, $\GL + \varphi$ proves that $\vec T(\vec u)$ chooses action $i^*_T$, and achieves the outcome $j^*_T$ as a result. (By saying that $\varphi$ is a "true" formula we mean that its translation to the language of arithmetic is true about the standard natural numbers: $\NN\vDash\varphi$.) It's ok to talk about $\AT$ and $\UT$ in the definition of $i^*_T$ and $j^*_T$ because fixed points are unique (up to provable equivalence), and $\GL$ can prove that the fixed point is in fact a fixed point.

My claim, then, is that $\vec\UDT^{(\varphi)}(\vec u)$ will perform at least as well as $\vec T(\vec u)$ on the decision problem $\vec P(\vec a)$; that is, the outcome it achieves will be ranked $\le j^*_T$.

Intuitively, this is straight-forward. $\vec\UDTp(\vec u)$ searches through all pairs $(j,i)$ of outcomes and actions in lexicographical order, until it finds a pair such that it can prove that if it takes action $i$, it will achieve outcome $j$; as soon as it finds such a pair, it takes action $i$. (This is justified becaues it searches outcomes best-first, so it takes an action that leads to as good an outcome as it's able to prove it can get.) So if it can prove that taking action $i^*_T$ will lead to outcome $j^*_T$, it will either take that action and get that outcome, or there's some pair $(j,i) < (j^*_T,i^*_T)$ such that it takes action $i$ and obtains outcome $j\le j^*_T$. (Remember that outcomes are numbered from best to worst.)

Let's go through the details of showing that it actually works out that way.

***

Let's write $(\A,\U)$ for the fixed point of $\vec\UDTp(\vec u)$ with $\vec P(\vec a)$. The part in our argument that we need to check carefully is that UDT will in fact stop at the pair $(j^*_T,i^*_T)$ if it hasn't already stopped before that; i.e.,
$$\GL\,\vdash\,\varphi\to\big(\UDTp_{i^*_T}(\U)\to U_{j^*_T}\big).$$
If this is satisfied, then we're done: We know that there will be some pair $(j^*,i^*)\le(j_T^*,i_T^*)$ such that $\vec\UDTp(\U)$ outputs $i^*$ and such that
$$\GL\,\vdash\,\varphi\to\big(\UDTp_{i^*}(\U)\to U_{j^*}\big),$$
and hence, since (a) $\GL$ is sound on $\NN$, (b) $\NN\vDash\varphi$, and (c) by assumption, $\NN\vDash\UDTp_{i^*}$, it follows that $\NN\vDash U_{j^*}$, i.e., $\UDTp(\U)$ achieves the outcome $j^*\le j_T^*$. Thus, let's check that $\GL$ does indeed prove $\varphi\to(\UDTp_{i^*_T}(\U)\to U_{j^*_T})$.

To do so, we make use of provable extensionality, that is, of the fact that
$$\GL\,\vdash\big(\vec a\lr\vec b\big)\to\big(\P\lr\vec P(\vec b)\big).$$
Since as a modal decision theory, $\vec\UDTp$ is a p.m.e.e. sequence (provably mutually exclusive and exhaustive), $\GL$ proves that $\UDTp_{i^*_T}(\U)$ implies $\vec\UDTp(\U)\lr\vec\chi^{(i^*_T)}$, i.e., $\A\lr\vec\chi^{(i^*_T)}$. Hence, together with provable extensionality, we obtain
$$\GL\,\vdash\,\UDTp_{i^*_T}(\U)\to\big(\U\lr\vec P(\chi^{(i^*_T)})\big)$$
(since $\GL\vdash\vec P(\A)\lr\U$ by definition of $\U$). But on the other hand, recall our initial assumption that
$$\GL\,\vdash\,\varphi\to\big((\AT\lr\vec\chi^{(i^*_T)})\wedge(\UT\lr\vec\chi^{(j^*_T)})\big);$$
again by provable extensionality, this implies
$$\GL\,\vdash\,\varphi\to\big(\big(\vec P(\AT)\lr\vec P(\vec\chi^{(i^*_T)})\big)\wedge\big(\UT\lr\vec\chi^{(j^*_T)}\big)\big),$$
and since $\GL\vdash\vec P(\AT)\lr\UT$ by definition of $\UT$, this simplifies to
$$\GL\,\vdash\,\varphi\to\big(P(\vec\chi^{(i^*_T)})\lr\vec\chi^{(j^*_T)}\big).$$
But together with our earlier result, this implies
$$\GL\,\vdash\,\big(\varphi\wedge\UDTp_{i^*_T}(\U)\big)\to\big(\U\lr\vec\chi^{(j_T^*)}\big),$$
which is equivalent to
$$\GL\,\vdash\,\varphi\to\big(\UDTp_{i^*_T}(\U)\to U_{j_T^*}\big),$$
as desired.