# Reflective oracles as a solution to the converse Lawvere problem

_(This post was originally published on Nov 30th 2017, and is 1 of 4 posts brought forwarded today as part of the AI Alignment Forum launch sequence on fixed points.)_

**1 Introduction**

Before the work of Turing, one could justifiably be skeptical of the idea
 of a universal computable function.
 After all, there is no computable function $f\colon\mathbb{N}\times\mathbb{N}\to\mathbb{N}$ such that for all computable $g\colon\mathbb{N}\to\mathbb{N}$ there is some index $i_{g}$ such that $f\left(i_{g},n\right)=g\left(n\right)$ for all $n$.
 If there were, we could pick $g\left(n\right)=f\left(n,n\right)+1$, and then
$$g\left(i_{g}\right)=f\left(i_{g},i_{g}\right)+1=g\left(i_{g}\right)+1,$$
a contradiction.
 Of course, universal Turing machines don't run into this obstacle; as Gödel
 put it, "By a kind of miracle it is not necessary to distinguish orders, and the
 diagonal procedure does not lead outside the defined notion." [1]

The miracle of Turing machines is that there is a partial computable function
 $f\colon\mathbb{N}\times\mathbb{N}\to\mathbb{N}\cup\left\{ \bot\right\}$ such that for all partial computable $g\colon\mathbb{N}\to\mathbb{N}\cup\left\{ \bot\right\}$ there is an index $i$ such that $f\left(i,n\right)=g\left(n\right)$ for all $n$.
 Here, we look at a different "miracle", that of reflective oracles [[2](https://arxiv.org/abs/1508.04145),[3](https://intelligence.org/files/ReflectiveSolomonoffAIXI.pdf)].
 As we will see in Theorem 1, given a reflective oracle $O$, there is a (stochastic) $O$-computable function $f\colon\mathbb{N}\times\mathbb{N}\to\left\{ 0,1\right\}$ such that for any (stochastic) $O$-computable function $g\colon\mathbb{N}\to\left\{ 0,1\right\}$, there is some index $i$ such that $f\left(i,n\right)$ and $g\left(n\right)$ have the same distribution for all $n$.
 This existence theorem seems to skirt even closer to the contradiction
 mentioned above.

We use this idea to answer "in spirit" the converse Lawvere problem posed in [[4](https://agentfoundations.org/item?id=1356)]. These methods also generalize to prove a similar analogue of the ubiquitous converse Lawvere problem from [[5](https://agentfoundations.org/item?id=1372)]. The original questions, stated in terms of topology, remain open, but I find that the model proposed here, using computability, is equally satisfying from the point of view of studying reflective agents. Those references can be consulted for more motivation on these problems from the perspective of reflective agency.

Section 3 proves the main lemma, and proves the converse Lawvere theorem for reflective oracles. In section 4, we use that to give a (circular) proof of Brouwer's fixed point theorem, as mentioned in [[4](https://agentfoundations.org/item?id=1356)]. In section 5, we prove the ubiquitous converse Lawvere theorem for reflective oracles.

# 2 Definitions

For any measurable space $X$, the set of probability measures on $X$ is denoted $\Delta X$.

A *probabilistic oracle* is a map $\mathbb{N}\to\left[0,1\right]$.
 A *probabilistic oracle machine* is a probabilistic Turing machine that can query a probabilistic oracle
 $O$.
 On a given query $n\in\mathbb{N}$, it gets the answer $1$ with probability $O\left(n\right)$ and the answer $0$ otherwise.
 Different queries are independent events, so this completely determines
 the behavior of such a machine.

We denote by $\phi_{i}^{O}$ the stochastic partial $O$-computable function $\mathbb{N}\to\Delta\left(\mathbb{N}\cup\left\{ \bot\right\} \right)$, where $\bot$ represents nonhalting, computed by the probabilistic Turing machine with
 index $i$.
 The notation $\phi_{i}^{O}\left(n\right)\downarrow$ indicates the event that $\phi_{i}^{O}$ halts on input $n$, and $\phi_{i}^{O}\left(n\right)\downarrow=m$ is the event that $\phi_{i}^{O}\left(n\right)$ halts and outputs $m$.
 Finally, $\phi_{i}^{O}\left(n\right)\uparrow$ is the event that $\phi_{i}^{O}$ does not halt on input $n$.

We use $\left\langle \cdot\right\rangle$ to represent a computable coding function, in order to biject arbitrary
 countable sets to the naturals for the purpose of computability.

A *reflective oracle* is a probabilistic oracle $O$ such that for all $i,n\in\mathbb{N}$ and $p\in\left[0,1\right]_{\mathbb{Q}}$,
\begin{align*}
\mathbb{P}\left(\phi_{i}^{O}\left(n\right)\downarrow=1\right)>p & \qquad\Longrightarrow\qquad O\left(\left\langle i,n,p\right\rangle \right)=1\\
\mathbb{P}\left(\neg\phi_{i}^{O}\left(n\right)\downarrow=0\right)<p & \qquad\Longrightarrow\qquad O\left(\left\langle i,n,p\right\rangle \right)=0.
\end{align*}
For more on reflective oracles, see [Fallenstein et al., 2015 [2]](https://arxiv.org/abs/1508.04145).

A function $f\colon\mathbb{N}\to\left[0,1\right]$ is *$O$-computable* if there is an index $i$ such that for all $n\in\mathbb{N}$, we have
$$\mathbb{P}\left(\phi_{i}^{O}\left(n\right)\downarrow\in\left\{ 0,1\right\} \right)=1$$
and
$$\mathbb{P}\left(\phi_{i}^{O}\left(n\right)\downarrow=1\right)=f\left(n\right).$$
That is, $\phi_{i}^{O}$ represents $f$ by probabilistically outputting either $0$ or $1$.

For any $m\in\mathbb{N}$, a function $f\colon\mathbb{N}\to\left[0,1\right]^{m}$ is *$O$-computable* if each coordinate $f_{j}\colon\mathbb{N}\to\left[0,1\right]$ for $1\le j\le m$ is $O$-computable.

A function $f\colon\mathbb{N}\to\left[0,1\right]^{\mathbb{N}}$ is *$O$-computable* if the corresponding function $\mathbb{N}\to\left[0,1\right]$ given by $\left\langle n,m\right\rangle \mapsto f\left(n\right)\left(m\right)$ is $O$-computable.

For any point $p\in\left[0,1\right]$, a probabilistic oracle $O$ is *compatible* with $p$ if for all $r\in\left[0,1\right]_{\mathbb{Q}}$, we have $O\left(\left\langle r\right\rangle \right)=0$ if $p<r$ and $O\left(\left\langle r\right\rangle \right)=1$ if $p>r$.
 More generally, for any $m\in\mathbb{N}$ and any $p\in\left[0,1\right]^{m}$, a probabilistic oracle $O$ is *compatible* with $p$ if for all $j$ such that $1\le j\le m$ and all $r\in\left[0,1\right]_{\mathbb{Q}}$, we have $O\left(\left\langle j,r\right\rangle \right)=0$ if $p_{j}<r$ and $O\left(\left\langle j,r\right\rangle \right)=1$ if $p_{j}>r$.

A function $f\colon\left[0,1\right]^{m}\to\left[0,1\right]^{m}$ is *$O$-computable* if for each coordinate $f_{j}\colon\left[0,1\right]^{m}\to\left[0,1\right]$, there is an index $i$ such that whenever $P$ is compatible with $p$, we have
$$\mathbb{P}\left(\phi_{i}^{O,P}\left(0\right)\downarrow\in\left\{ 0,1\right\} \right)=1$$
and
$$\mathbb{P}\left(\phi_{i}^{O,P}\left(0\right)\downarrow=1\right)=f_{j}\left(p\right).$$


A map $f\colon\mathbb{N}\to\left[0,1\right]^{\mathbb{N}}$ is *$O$-computably ubiquitous* if for every $O$-computable map $e\colon\mathbb{N}\to\left[0,1\right]^{\mathbb{N}}$, there is some $n\in\mathbb{N}$ such that $f\left(n\right)=e\left(n\right)$.

# 3 Converse Lawvere property

We first need a lemma that tells us that we can replace certain partial
 $O$-computable functions with total ones when working relative to a reflective
 oracle.
 As discussed in the introduction, this contrasts strongly with the situation for computable functions.
 All of our theorems will make essential use of this lemma.

**Lemma 1** (totalizing): There is a computable map $\tau\colon\mathbb{N}\to\mathbb{N}$ such that for all $i,n\in\mathbb{N}$ and any reflective oracle $O$, we have
$$\mathbb{P}\left(\phi_{\tau\left(i\right)}^{O}\left(n\right)\downarrow\in\left\{ 0,1\right\} \right)=1$$
and for $b\in\left\{ 0,1\right\}$,
$$\mathbb{P}\left(\phi_{\tau\left(i\right)}^{O}\left(n\right)\downarrow=b\right)\ge\mathbb{P}\left(\phi_{i}^{O}\left(n\right)\downarrow=b\right).$$


**Proof**: We construct $\tau$ using a recursive procedure that ensures that $\mathbb{P}\left(\phi_{\tau\left(i\right)}^{O}\left(n\right)\downarrow=b\right)$ upper-bounds $\mathbb{P}\left(\phi_{i}^{O}\left(n\right)\downarrow=b\right)$ using what may be thought of as repeated subdivision or binary search.
 This is essentially the same as the function ${\tt flip}$ in [[3](https://intelligence.org/files/ReflectiveSolomonoffAIXI.pdf)], but we handle computability issues differently.
 Let $S\subseteq\left[0,1\right]\times\left[0,1\right]$ be the set of pairs of dyadic rationals $\left(p,q\right)$ such that $p<q$.
 We recursively define a stochastic $O$-computable function $t\colon\mathbb{N}\times\mathbb{N}\times S\to\Delta\left\{ 0,1\right\}$; the intent is to have $\mathbb{P}\left(t\left(i,n,p,q\right)=1\right)$ equal
$$\frac{\mathbb{P}\left(\phi_{\tau\left(i\right)}^{O}\left(n\right)\downarrow=1\right)-p}{q-p}$$
if that quantity is in the interval $\left[0,1\right]$, and take the closest possible value, either $0$ or $1$, otherwise.
 Then, we will be able to define $\phi_{\tau\left(i\right)}^{O}\left(n\right)$ to be $t\left(i,n,0,1\right)$.

Construct $t$ so that a call $t\left(i,n,p,q\right)$ first queries $O\left(\left\langle i,n,r\right\rangle \right)$, where $r=\frac{p+q}{2}$, and also flips a fair coin $C$.
 Then, it either outputs $0$, $1$, or the result of a recursive call, as follows:
$$t\left(i,n,p,q\right)=\begin{cases}
0 & \text{if $O\left(\left\langle i,n,r\right\rangle \right)=0$ and $C=0$}\\
t\left(i,n,p,r\right) & \text{if $O\left(\left\langle i,n,r\right\rangle \right)=0$ and $C=1$}\\
1 & \text{if $O\left(\left\langle i,n,r\right\rangle \right)=1$ and $C=0$}\\
t\left(i,n,r,q\right) & \text{if $O\left(\left\langle i,n,r\right\rangle \right)=1$ and $C=1$.}
\end{cases}$$
We can now choose $\tau$ so that $\phi_{\tau\left(i\right)}^{O}\left(n\right)=t\left(i,n,0,1\right)$.

This algorithm upper bounds the probabilities $\mathbb{P}\left(\phi_{i}^{O}\left(n\right)\downarrow=0\right)$ and $\mathbb{P}\left(\phi_{i}^{O}\left(n\right)\downarrow=1\right)$ by binary search.
 Once the initial call $t\left(i,n,0,1\right)$ has recursed to $t\left(i,n,p,q\right)$, it has already halted outputting $1$ with probability $p$, confirming that this is an upper bound since it received answer $1$ from a call to $O\left(\left\langle i,n,p\right\rangle \right)$.
 Similarly, it has output $0$ with probability $1-q$, confirming this bound since it received the answer $0$ from a call to $O\left(\left\langle i,n,q\right\rangle \right)$.
 Further, since each call to $t$ halts without recursing with probability $\frac{1}{2}$, $t$ halts almost surely.
 Thus, we get the totality property
$$\mathbb{P}\left(\phi_{\tau\left(i\right)}^{O}\left(n\right)\downarrow\in\left\{ 0,1\right\} \right)=1$$
and the bounds
$$\mathbb{P}\left(\phi_{\tau\left(i\right)}^{O}\left(n\right)\downarrow=b\right)\ge\mathbb{P}\left(\phi_{i}^{O}\left(n\right)\downarrow=b\right)$$
for $b\in\left\{ 0,1\right\}$.
 $\square$

Now we can prove our main theorem.

**Theorem 1** (converse Lawvere for reflective oracles): Let $O$ be a reflective oracle.
 Then, there is an $O$-computable map $f\colon\mathbb{N}\to\left[0,1\right]^{\mathbb{N}}$ such that for all $O$-computable $g\colon\mathbb{N}\to\left[0,1\right]$, there is some index $i$ such that $g=f\left(i\right)$.

**Proof**: Using $\tau$ from the totalizing lemma, let
$$f\left(i\right)\left(n\right)=\mathbb{P}\left(\phi_{\tau\left(i\right)}^{O}\left(n\right)\downarrow=1\right).$$
Given any $O$-computable $g\colon\mathbb{N}\to\left[0,1\right]$, there is some $i$ such that
\begin{align*}
\mathbb{P}\left(\phi_{i}^{O}\left(n\right)\downarrow=1\right) & =g\left(n\right)\\
\mathbb{P}\left(\phi_{i}^{O}\left(n\right)\downarrow=0\right) & =1-g\left(n\right).
\end{align*}
Then,
$$f\left(i\right)\left(n\right)=\mathbb{P}\left(\phi_{\tau\left(i\right)}^{O}\left(n\right)\downarrow=1\right)\ge\mathbb{P}\left(\phi_{i}^{O}\left(n\right)\downarrow=1\right)=g\left(n\right)$$
and similarly $1-f\left(i\right)\left(n\right)\ge1-g\left(n\right)$, so $f\left(i\right)=g$.
 $\square$

This theorem gives a computable analogue to the problem posed in [[4](https://agentfoundations.org/item?id=1356)].
 The analogy would be strengthened if we worked in a cartesian closed category
 where the present notion of $O$-computability gave the morphisms, and where $\left[0,1\right]^{\mathbb{N}}$ is an exponential object.
 In addition, the totalizing lemma would have a nice analogue in this setting.
 I expect that all this can be done using something like an effective topos
 [6], but I leave this for future work.

# 4 Recovering Brouwer's fixed point theorem

As mentioned in [[4](https://agentfoundations.org/item?id=1356)], the intermediate value theorem would follow from the
 existence of a space with the converse Lawvere property, that is, a space
 $X$ that surjects onto the mapping space $\left[0,1\right]^{X}$.
 Further, Brouwer's fixed point theorem on the $n$-ball, $B^{n}$, would follow from the existence of a topological space $X$ with a surjection $X\to\left(B^{n}\right)^{X}$.
 We can do something similar to conclude Brouwer's fixed point theorem from
 the converse Lawvere theorem for reflective oracles.
 Of course, this is circular; Kakutani's fixed point theorem, a generalization
 of Brouwer's fixed point theorem, is used to prove the existence of reflective
 oracles.
 Still, it is interesting to see how this can be done.

We start with two technical lemmas telling us some basic facts about reflective-oracle-computable functions $\left[0,1\right]^{m}\to\left[0,1\right]^{m}$.
 Using these, it will be easy to derive Brouwer's fixed point theorem.

**Lemma 2** (continuous implies relatively computable): Take $m\in\mathbb{N}$ and let $h\colon\left[0,1\right]^{m}\to\left[0,1\right]^{m}$ be continuous.
 Then, there is a (deterministic) oracle $O$ such that $h$ is $O$-computable.

**Proof**: For each coordinate $h_{j}$ of $h$, each rectangle
$$R=\left[\ell_{1},u_{1}\right]\times\dots\times\left[\ell_{m},u_{m}\right]\subseteq\left[0,1\right]^{m}$$
with rational endpoints, and each pair of rationals $\ell_{0},u_{0}\in\left[0,1\right]_{\mathbb{Q}}$ with $\ell_{0}<u_{0}$, let $O\left(\left\langle j,\ell_{0},u_{0},\dots,\ell_{m},u_{m}\right\rangle \right)$ be $1$ if $h_{j}\left(R\right)\subseteq\left[\ell_{0},u_{0}\right]$ and $0$ otherwise.
 To see that $h$ is $O$-computable, we compute any $h_{j}\left(p\right)$ for any $j$ with $1\le j\le m$, and any $p\in\left[0,1\right]^{m}$, making use of $O$ and any oracle $P$ compatible with $p$.

We proceed by a search process similar to the argument in the totalizing
 lemma.
 Start with $R^{0}=\left[0,1\right]^{m}$, $\ell_{0}^{0}=0$ and $u_{0}^{0}=1$.
 At each step $s$, perform an exhaustive search for a rectangle
$$R^{s+1}=\left[\ell_{1}^{s+1},u_{1}^{s+1}\right]\times\dots\times\left[\ell_{m}^{s+1},u_{m}^{s+1}\right]\subseteq R^{s}$$
and points $\ell_{0}^{s+1},u_{0}^{s+1}$ such that a query to $P\left(\left\langle k,\ell_{k}^{s+1}\right\rangle \right)$ returns $1$ for all $k$, a query to any $P\left(\left\langle k,u_{k}^{s+1}\right\rangle \right)$ returns 0, a query to $O\left(\left\langle j,\ell_{0}^{s+1},u_{0}^{s+1},R^{s+1}\right\rangle \right)$ returns $1$, and where either $\ell_{0}^{s+1}=\ell_{0}^{s}$ and $u_{0}^{s+1}=\frac{1}{3}\ell_{0}^{s}+\frac{2}{3}u_{0}^{s}$, or $\ell_{0}^{s+1}=\frac{2}{3}\ell_{0}^{s}+\frac{1}{3}u_{0}^{s}$ and $u_{0}^{s+1}=u_{0}^{s}$.
 In the first case, output $0$ with probability $\frac{1}{3}$ and continue with probability $\frac{2}{3}$, and in the second case, output $1$ with probability $\frac{1}{3}$ and continue with probability $\frac{2}{3}$.

By construction, $p\in R^{s}$ and $h_{j}\left(R^{s}\right)\subseteq\left[\ell_{0}^{s},u_{0}^{s}\right]$ at each stage $s$.
 Since $h_{j}$ is continuous, there is some neighbourhood of $p$ on which its image is contained in either $\left[\ell_{0}^{s},\frac{1}{3}\ell_{0}^{s}+\frac{2}{3}u_{0}^{s}\right]$ or $\left[\frac{2}{3}\ell_{0}^{s}+\frac{1}{3}u_{0}^{s},u_{0}^{s}\right]$.
 There are two possibilities to consider.
 If $p\in{\rm int}\:R^{s}$, then there is some rectangle $R$ contained in such a neighbourhood of $p$, and with $p\in{\rm int}\:R$.
 This rectangle would be chosen if considered, so the algorithm will move
 beyond step $s$.

The remaining possibility is that $p$ is on the boundary of $R^{s}$; say, $p_{k}=\ell_{k}^{s}$.
 Since $R^{s}$ was chosen previously though, we know that querying $P\left(k,\ell_{k}^{s+1}\right)$ has returned $1$ at least once, so $P\left(k,\ell_{k}^{s+1}\right)\ne0$.
 Thus, the algorithm will almost surely eventually accept $R^{s}$ or another rectangle.

Putting this together, this algorithm almost surely halts, outputting either
 $0$ or $1$.
 By stage $s$, it has halted outputting $1$ with probability $\ell_{0}^{s}$ and outputting $0$ with probability $1-u_{0}^{s}$, so overall it outputs $1$ with probability $h_{j}\left(p\right)$.
 Thus, $h$ is $O$-computable.
 $\square$

**Lemma 3** (composition): Let $O$ be a reflective oracle and let $g\colon\mathbb{N}\to\left[0,1\right]^{m}$ and $h\colon\left[0,1\right]^{m}\to\left[0,1\right]^{m}$ be $O$-computable.
 Then, $h\circ g\colon\mathbb{N}\to\left[0,1\right]^{m}$ is $O$-computable.

**Proof**: For each $j$ with $1\le j\le m$, take $i_{j}\in\mathbb{N}$ such that $\phi_{i_{j}}^{O}$ witnesses the computability of $g_{j}$.
 Then, $O\left(\left\langle i_{j},n,r\right\rangle \right)=0$ if $g_{j}\left(n\right)<r$ and $O\left(\left\langle i_{j},n,r\right\rangle \right)=1$ if $g_{j}\left(n\right)>r$, so $O$ lets us simulate a probabilistic oracle compatible with $g\left(n\right)$.
 Hence, by the $O$-computability of $h$, for each $k$ with $1\le k\le m$, we have a probabilistic $O$-machine that always halts, outputting either $0$ or $1$, and that on input $n$ outputs $1$ with probability $h_{k}\circ g\left(n\right)$.
 $\square$

**Theorem 2** (Brouwer's fixed point theorem): Take $m\in\mathbb{N}$ and $h\colon\left[0,1\right]^{m}\to\left[0,1\right]^{m}$.
 Then, $h$ has a fixed point.

**Proof**: By Lemma 2, we have an oracle $O$ such that $h$ is $O$-computable.
 By relativizing the construction of a reflective oracle [[2](https://arxiv.org/abs/1508.04145),[3](https://intelligence.org/files/ReflectiveSolomonoffAIXI.pdf)] to $O$, we get a reflective oracle $\widetilde{O}$ above $O$.
 Notice that $h$ is $\widetilde{O}$-computable.
 Letting $f$ be the $\widetilde{O}$-computable function
$$f\left(i\right)\left(n\right)=\mathbb{P}\left(\phi_{\tau\left(i\right)}^{\widetilde{O}}\left(n\right)\downarrow=1\right)$$
constructed in the converse Lawvere theorem for reflective oracles, define
 $f_{m}\colon\mathbb{N}\to\left(\left[0,1\right]^{m}\right)^{\mathbb{N}}$ by
$$f_{m}\left(\left\langle i_{1},\dots,i_{m}\right\rangle \right)\left(n\right)=\left(f\left(i_{1}\right)\left(n\right),\dots,f\left(i_{m}\right)\left(n\right)\right).$$
The rest will now follow the proof of Lawvere's fixed point theorem [7].

Define $g\colon\mathbb{N}\to\left[0,1\right]^{m}$ by $g\left(n\right)=h\left(f_{m}\left(n\right)\left(n\right)\right)$; this is $\widetilde{O}$-computable by Lemma 3.
 Now, by converse Lawvere theorem, for each coordinate $1\le j\le m$ of $g$, there is some $i_{j}\in\mathbb{N}$ such that $g_{j}=f\left(i_{j}\right)$.
 Letting $i=\left\langle i_{1},\dots,i_{m}\right\rangle$, we have
$$g_{j}\left(i\right)=h_{j}\left(f_{m}\left(i\right)\left(i\right)\right)=h_{j}\left(g\left(i\right)\right),$$
so $g\left(i\right)=h\left(g\left(i\right)\right)$, and so $g\left(i\right)$ is a fixed point of $h$.
 $\square$

# 5 Ubiquitous converse Lawvere property

**Theorem 3** (ubiquitous converse Lawvere): Let $O$ be a reflective oracle.
 Then, there is an $O$-computable, $O$-computably ubiquitous map $f\colon\mathbb{N}\to\left[0,1\right]^{\mathbb{N}}$.

**Proof**: This follows by a combination of the recursion theorem and the totalizing
 lemma.
 We use the same map $f$ from Theorem 1.
 Let $e\colon\mathbb{N}\to\left[0,1\right]^{\mathbb{N}}$ be any $O$-computable map.
 There is a computable map $s\colon\mathbb{N}\to\mathbb{N}$ such that for all $i,n\in\mathbb{N}$, we have
$$\mathbb{P}\left(\phi_{s\left(i\right)}^{O}\left(n\right)\downarrow\in\left\{ 0,1\right\} \right)=1,$$
and
$$e\left(i\right)\left(n\right)=\mathbb{P}\left(\phi_{s\left(i\right)}^{O}\left(n\right)\downarrow=1\right).$$
By the recursion theorem, there is some $i$ such that $\phi_{s\left(i\right)}^{O}=\phi_{i}^{O}$.
 Then,
\begin{align*}
e\left(i\right)\left(n\right) & =\mathbb{P}\left(\phi_{s\left(i\right)}^{O}\left(n\right)\downarrow=1\right)=\mathbb{P}\left(\phi_{i}^{O}\left(n\right)\downarrow=1\right)\\
 & =\mathbb{P}\left(\phi_{\tau\left(i\right)}^{O}\left(n\right)\downarrow=1\right)=f\left(i\right)\left(n\right),
\end{align*}
so $e\left(i\right)=f\left(i\right)$. $\square$

# References

[1] Kurt Gödel.
 1946.
 "Remarks before the Princeton bicentennial conference of problems in mathematics." Reprinted in: Martin Davis.
 1965.
 "The Undecidable.
 Basic Papers on Undecidable Propositions, Unsolvable Problems, and Computable
 Functions." Raven Press.

[2] Benja Fallenstein, Jessica Taylor, and Paul F.
 Christiano.
 2015.
 ["Reflective Oracles: A Foundation for Classical Game Theory." arXiv: 1508.04145.](https://arxiv.org/abs/1508.04145)

[3] Benya Fallenstein, Nate Soares, and Jessica Taylor.
 2015.
 ["Reflective variants of Solomonoff induction and AIXI."](https://intelligence.org/files/ReflectiveSolomonoffAIXI.pdf) In *Artificial General Intelligence*.
 Springer.

[4] Scott Garrabrant.
 2017.
 "Formal Open Problem in Decision Theory." [https://agentfoundations.org/item?id=1356](https://agentfoundations.org/item?id=1356).

[5] Scott Garrabrant.
 2017.
 "The Ubiquitous Converse Lawvere Problem." [https://agentfoundations.org/item?id=1372](https://agentfoundations.org/item?id=1372)

[6] Jaap van Oosten.
 2008.
 "Realizability: an introduction to its categorical side." *Studies in Logic and the Foundations of Mathematics*, vol.
 152.
 Elsevier.

[7] F.
 William Lawvere.
 1969.
 "Diagonal arguments and cartesian closed categories." In *Category Theory, Homology Theory and their Applications, II (Battelle Institute
 Conference, Seattle, Wash., 1968, Vol.
 Two)*, pages 134–145.
 Springer.

***

_This post was originally published on Nov 30th 2017, and has been brought forwarded as part of the AI Alignment Forum launch sequences._

_Tomorrow's AIAF sequences post will be 'Iterated Amplification and Distillation' by Ajeya Cotra, in the sequence on iterated amplification._