## 5 Open Problems regarding Optimality of AIXI  
AIXI has been shown to be Pareto-optimal and a variant of AIXI to be self-optimizing [137]. These are important results supporting the claim that AIXI is universally optimal. More results can be found in [27]. Unlike the induction case, the results are not strong enough to alley all doubts. Indeed, the major problem is not to prove optimality but to come up with a sufficiently strong but still satisfiable optimality notion in the reinforcement learning case. The following items list four potential approaches towards a solution:  
* **What is meant by universal optimality? ** A \"learner\" (like AIXI) may converge to the optimal informed decision maker (like AI\(\mu\)) in several senses. Possibly relevant concepts from statistics are, _consistency_, _self-tuningness_, _self-optimizingness_, _efficiency_, _unbiasedness_, _asymptotically_ or _finite convergence_[109], _Pareto-optimality_, and some more defined in [27]. Some concepts are stronger than necessary, others are weaker than desirable but suitable to start with. It is necessary to investigate in more breadth which properties the AIXI model satisfies. * **Limited environmental classes. ** The problem of defining and proving general value bounds becomes more feasible by considering, in a first step, restricted concept classes. One could analyze AIXI for known classes (like Markov or factorizable environments) and especially for the new classes (_forgetful_, _relevant_, _asymptotically learnable_, _farsighted_, _uniform_, and _(pseudo-)passive_) defined in [27]. * **Generalization of AIXI to general Bayes mixtures. ** Alternatively one can generalize AIXI to AI\(\xi\), where \(\xi(\cdot)\!=\!\sum_{\
u\in\mathcal{M}}\!w_{\
u}\
u(\cdot)\) is a general Bayes-mixture of distributions \(\
u\) in some class \(\mathcal{M}\) and prior \(w_{\
u}\). If \(\mathcal{M}\) is the multi-set of all enumerable semi-measures, then AI\(\xi\) coincides with AIXI. If \(\mathcal{M}\) is the (multi)set of passive semi-computable environments, then AIXI reduces to [PERSON]'s optimal predictor [18]. The key is not to prove absolute results for specific problem classes, but to prove _relative results_ of the form \"if there exists a policy with certain desirable properties, then AI\(\xi\) also possesses these desirable properties\". If there are tasks which cannot be solved by any policy, AI\(\xi\) should not be blamed for failing. * **Intelligence Aspects of AIXI. ** Intelligence can have many faces. As argued in [27], it is plausible that AIXI possesses all or at least most properties an intelligent rational agent should posses. Some of the following properties could and should be investigated mathematically: _creativity_, _problem solving_, _pattern recognition_, _classification_, _learning_, _induction_, _deduction_, _building analogies_, _optimization_, _surviving in an environment_, _language processing_, _planning_. Sources of inspiration can be previously proven loss bounds for Solomonoff sequence prediction generalized to unbounded horizon, optimality results from the adaptive control literature, and the asymptotic self-optimizingness results for the related AI\(\xi\) model. Value bounds for AIXI are expected to be, in a sense, weaker than the loss bounds for Solomonoff induction because the problem class covered by AIXI is much larger than the class of sequence prediction problems. In the same sense as [PERSON] solution to the bandit problem and [PERSON]' rule for Bernoulli sequences, AIXI may simply be regarded as (Bayes-)optimal by construction. Even when accepting this \"easy way out\", the above questions remain significant: Theorems relating AIXI to AI\(\mu\) would no longer be regarded as optimality proofs of AIXI, but just as how much harder it becomes to operate when \(\mu\) is unknown, i.e. progress on the items above is simply reinterpreted. A weaker goal than to prove optimality of AIXI is to ask for reasonable convergence properties:  
* **Posterior convergence for unbounded horizon.