## 6 Open Problems regarding Uniqueness of AIXI  
As a unification of two optimal theories, it is plausible that AIXI is optimal in the \"union\" of their domains, which has been affirmed but not finally settled by the positive results derived so far. In the absence of a definite answer, one should be open to alternative models, but no convincing competitor exists to date. Most of the following items describe ideas which, if worked out, might result in alternative models:  
* **Action with expert advice.** Expected performance bounds for predictions based on [PERSON]'s prior exist. Inspired by [PERSON] induction, a dual, currently very popular approach, is \"prediction with expert advice\" (PEA) [11, 160, 161]. Whereas PEA performs well in any environment, but only relative to a given set of experts, [PERSON]'s predictor competes with _any_ other predictor, but only in expectation for environments with computable distribution. It seems philosophically less compromising to make assumptions on prediction strategies than on the environment, however weak. PEA has been generalized to active learning [11, 162], but the full reinforcement learning case is still open [52]. If successful, it could result in a model dual to AIXI, but I expect the answer to be negative, which on the positive side would show the distinguishedness of AIXI. Other ad-hoc approaches like [126, 163] are also unlikely to be competitive.
* **Actions as random variables.** There may be more than one way for the choice of the generalized \(M\) in the AIXI model. For instance, instead of defining \(M\) as in [27] one could treat the agent's actions \(a\) also as universally distributed random variables and then conditionalize \(M\) on \(a\).
* **Structure of AIXI.** The algebraic properties and the structure of AIXI has barely been investigated. It is known that the value of AI\(\mu\) is a linear function in \(\mu\) and the value of AIXI is a convex function in \(\mu\), but this is neither very deep nor very specific to AIXI. It should be possible to extract all essentials from AIXI which finally should lead to an axiomatic characterization of AIXI. The benefit is as in any _axiomatic approach_: It would clearly exhibit the assumptions, separate the essentials from technicalities, simplify understanding and, most importantly, guide in finding proofs.
* **Parameter dependence.** The AIXI model depends on a few parameters: the choice of observation and action spaces \(\mathcal{O}\) and \(\mathcal{A}\), the horizon \(m\), and the universal machine \(U\). So strictly speaking, AIXI is only (essentially) unique, if it is (essentially) independent of the parameters. I expect this to be true, but it has not been proven yet. The \(U\)-dependence has been discussed in problem 4.h. Countably infinite \(\mathcal{O}\) and \(\mathcal{A}\) would provide a rich enough interface for all problems, but even binary \(\mathcal{O}\) and \(\mathcal{A}\) are sufficient by sequentializing complex observations and actions. For special classes one could choose \(m\!\rightarrow\!\infty\)[20]; unfortunately, the universal environment \(M\) does not belong to any of these special classes. See [27, 32, 164] for some preliminary considerations.