SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\Lambda_ {\mathrm {B}} ^ {0, 1} = \left\{\lambda_ {\mathrm {B}} ^ {0}, \lambda_ {\mathrm {B}} ^ {1} \right\}, \quad \Lambda_ {\mathrm {B}} ^ {2, 3} = \left\{\lambda_ {\mathrm {B}} ^ {2}, \lambda_ {\mathrm {B}} ^ {3} \right\}. \tag {C.18}
$$
Next notice that there cannot be a singleton basis by Proposition C.6, and thus, both  $\Lambda_{\mathrm{B}}^{0,1}$  and  $\Lambda_{\mathrm{B}}^{2,3}$  satisfy (1)  $|\Lambda_{\mathrm{B}}^{0,1}| = |\Lambda_{\mathrm{B}}^{2,3}| = \mathrm{rank}(\Lambda)$ , and (2) both  $\Lambda_{\mathrm{B}}^{0,1} \models \Lambda$ ,  $\Lambda_{\mathrm{B}}^{2,3} \models \Lambda$ .
Beyond the lack of redundancy of a basis, we may also be interested in their expressive power. For instance, if we compare two minimal bases,  $\Lambda_{\mathrm{B}}^{1}$  and  $\Lambda_{\mathrm{B}}^{2}$ , how might we justify which to use? To address this question, we consider another desirable property of a basis: universality.
Definition C.7. An agent basis  $\Lambda_{\mathrm{B}}$  is universal if  $\Lambda_{\mathrm{B}} \models \mathbb{A}$ .
Clearly, it might be desirable to work with a universal basis, as doing so ensures that the set of agents we consider in our design space is as rich as possible. We next show that there is at least one natural basis that is both minimal and universal.
Proposition C.9. The basis,
$$
\Lambda_ {\mathrm {B}} ^ {\circ} = \left\{\lambda : \mathcal {O} \rightarrow \Delta (\mathcal {A}) \right\}, \tag {C.19}
$$
is a minimal universal basis:
1.  $\Lambda_{\mathrm{B}}^{\circ} \models \mathbb{A}$ : The basis uniformly generates the set of all agents.
2.  $\Lambda_{\mathrm{B}}^{\circ}$  is minimal.
Proof of Proposition C.9.
We prove each property separately.
$$
I. \Lambda_ {\mathrm {B}} ^ {\circ} \vDash \Lambda
$$
First, we show that the basis is universal:  $\Lambda_{\mathrm{B}}^{\circ} \models \mathbb{A}$ . Recall that this amounts to showing that,
$$
\forall_ {\lambda \in \mathcal {A}} \forall_ {h \in \mathcal {H}} \exists_ {\lambda^ {\prime} \in \Lambda_ {\mathrm {B}} ^ {\circ}} \lambda (h) = \lambda^ {\prime} (h). \tag {C.20}
$$
Let  $\lambda \in \mathbb{A}$  and  $h\in \mathcal{H}$  be fixed but arbitrary. Now, let us label the action distribution produced by  $\lambda (h)$  as  $p_{\lambda (h)}$  . Let  $o$  refer to the last observation contained in  $h$  (or  $\emptyset$  if  $h = h_0 = \emptyset$  ). Now, construct the agent  $\lambda_{\mathrm{B}}^{\circ}:o\mapsto p_{\lambda (h)}$  . By construction of  $\Lambda_{\mathrm{B}}^{\circ}$  , this agent is guaranteed to be a member of  $\Lambda_{\mathrm{B}}^{\circ}$  , and furthermore, we know that  $\lambda_{\mathrm{B}}^{\circ}$  produces the same output as  $\lambda$  on  $h$  . Since both  $\lambda$  and  $h$  were chosen arbitrarily, the construction will work for any choice of  $\lambda$  and  $h$  and we conclude that at every history, there exists a basis agent  $\Lambda_{\mathrm{B}}^{\circ}\in \Lambda_{\mathrm{B}}^{\circ}$  that produces the same probability distribution over actions as any given agent. Thus, the first property holds.
2.  $\Lambda_{\mathrm{B}}^{\circ}$  is minimal.
Second, we show that  $\Lambda_{\mathrm{B}}^{\circ}$  is a minimal basis of  $\mathbb{A}$ . Recall that since  $\mathrm{rank}(\Lambda_{\mathrm{B}}^{\circ}) = \infty$ , the definition of a minimal basis means that:
$$
\forall_ {\Lambda_ {B} \subseteq \Lambda_ {B} ^ {\circ}} \Lambda_ {B} \models \wedge \Rightarrow \Lambda_ {B} = \Lambda_ {B} ^ {\circ}. \tag {C.21}
$$
To do so, fix an arbitrary proper subset of  $\Lambda_{\mathrm{B}} \in \mathcal{P}(\Lambda_{\mathrm{B}}^{\circ})$ . Notice that since  $\Lambda_{\mathrm{B}}$  is a proper subset, there exists a non-empty set  $\overline{\Lambda_{\mathrm{B}}}$  such that,
$$
\Lambda_ {\mathrm {B}} \cup \overline {{\Lambda_ {\mathrm {B}}}} = \Lambda_ {\mathrm {B}} ^ {\circ}.
$$
Now, we show that  $\Lambda_{\mathrm{B}}$  cannot uniformly generate  $\triangleleft$  by constructing an agent from  $\overline{\Lambda_{\mathrm{B}}}$ . In particular, consider the first element of  $\overline{\Lambda_{\mathrm{B}}}$ , which, by construction of  $\Lambda_{\mathrm{B}}^{\circ}$ , is some mapping from  $O$  to a choice of probability distribution over  $\mathcal{A}$ . Let us refer to this agent's output probability distribution over actions as  $\overline{p}$ . Notice that there cannot exist an agent in  $\Lambda_{\mathrm{B}}^{\circ}$  that chooses  $\overline{p}$ , otherwise  $\Lambda_{\mathrm{B}}$  would not be a proper subset of  $\Lambda_{\mathrm{B}}^{\circ}$ . Notice further that in the set of all agents, there are infinitely many agents that output  $\overline{p}$  in at least one history. We conclude that  $\Lambda_{\mathrm{B}}$  cannot uniformly generate  $\mathbb{A}$ , as it does not contain any base element that produces  $\overline{p}$ . The set  $\overline{\Lambda_{\mathrm{B}}}$  was chosen arbitrarily, and thus the claim holds for any proper subset of  $\Lambda_{\mathrm{B}}^{\circ}$ , and we conclude.
This completes the proof of both statements.
Corollary C.10. As a direct consequence of Proposition C.9, every universal basis has infinite rank.
C.1.2 Orthogonal and Parallel Agent Sets
Drawing inspiration from vector spaces, we introduce notions of orthogonal and parallel agent bases according to the agent sets they generate.
Definition C.8. A pair of agent bases  $(\Lambda_{\mathrm{B}}^{1},\Lambda_{\mathrm{B}}^{2})$  are orthogonal if any pair  $(\Lambda^1,\Lambda^2)$  they each uniformly generate
$$
\Lambda_ {\mathrm {B}} ^ {1} \vDash \Lambda^ {1}, \quad \Lambda_ {\mathrm {B}} ^ {2} \vDash \Lambda^ {2}, \tag {C.22}
$$
$$
\Lambda^ {1} \cap \Lambda^ {2} = \emptyset . \tag {C.23}
$$
Naturally this definition can be modified to account for environment-relative generation  $(\mathbb{I}^{\varepsilon})$ , or to be defined with respect to a particular set of learning rules in which case two bases are orthogonal with respect to the learning rule set just when they generate different agent sets under the given learning rules. As with the variants of the two operators, we believe the details of such formalisms are easy to produce.
A few properties hold of any pair of orthogonal bases.
Proposition C.11. If two bases  $\Lambda_{\mathrm{B}}^{1},\Lambda_{\mathrm{B}}^{2}$  are orthogonal, then the following properties hold:
1.  $\Lambda_{\mathrm{B}}^{1}\cap \Lambda_{\mathrm{B}}^{2} = \emptyset$
2. Neither  $\Lambda_{\mathrm{B}}^{1}$  nor  $\Lambda_{\mathrm{B}}^{2}$  are universal.
Proof of Proposition C.11.
We prove each property independently.
$$
I. \Lambda_ {\mathrm {B}} ^ {1} \cap \Lambda_ {\mathrm {B}} ^ {2} = \emptyset
$$
We proceed toward contradiction. That is, suppose that both  $\Lambda_{\mathrm{B}}^{1}$  is orthogonal to  $\Lambda_{\mathrm{B}}^{2}$ , and that  $\Lambda_{\mathrm{B}}^{1} \cap \Lambda_{\mathrm{B}}^{2} \neq \emptyset$ . Then, by the latter property, there is at least one agent that is an element of
both bases. Call this agent  $\lambda_{\mathrm{B}}^{\circ}$ . It follows that the set  $\Lambda_{\mathrm{B}}^{\circ} = \{\lambda_{\mathrm{B}}^{\circ}\}$  is a subset of both  $\Lambda_{\mathrm{B}}^{1}$  and  $\Lambda_{\mathrm{B}}^{2}$ . By Proposition C.2, it follows that  $\Lambda_{\mathrm{B}}^{1} \models \Lambda_{\mathrm{B}}^{\circ}$  and  $\Lambda_{\mathrm{B}}^{2} \models \Lambda_{\mathrm{B}}^{\circ}$ . But this contradicts the fact that  $\Lambda_{\mathrm{B}}^{1}$  is orthogonal to  $\Lambda_{\mathrm{B}}^{2}$ , and so we conclude.
2. Neither  $\Lambda_{\mathrm{B}}^{1}$  nor  $\Lambda_{\mathrm{B}}^{2}$  are universal.
We again proceed toward contradiction. Suppose without loss of generality that  $\Lambda_{\mathrm{B}}^{1}$  is universal. Then, we know  $\Lambda_{\mathrm{B}}^{1} \models \wedge$ . Now, we consider two cases: either  $\Lambda_{\mathrm{B}}^{2}$  generates some non-empty set,  $\Lambda^{2}$ , or it does not generate any sets. If it generates a set  $\Lambda^{2}$ , then we arrive at a contradiction as  $\Lambda^{2} \cap \wedge \neq \emptyset$ , which violates the definition of orthogonal bases. If if does not generate a set, this violates the definition of a basis, as any basis is by construction non-empty, and we know that containing even a single element is sufficient to generate at least one agent set by Proposition C.6. Therefore, in either of the two cases, we arrive at a contradiction, and thus conclude the argument.
This concludes the proof of each statement.
Corollary C.12. For any non-universal agent basis  $\Lambda_{\mathrm{B}}$ , there exists an orthogonal agent basis,  $\Lambda_{\mathrm{B}}^{\dagger}$ .
Conversely, two agent bases  $\Lambda_{\mathrm{B}}^{1},\Lambda_{\mathrm{B}}^{2}$  are parallel just when they generate the same agent sets.
Definition C.9. A pair of agent bases  $(\Lambda_{\mathrm{B}}^{1},\Lambda_{\mathrm{B}}^{2})$  are parallel if for every agent set  $\Lambda$ ,  $\Lambda_{\mathrm{B}}^{1} \models \Lambda$  if and only if  $\Lambda_{\mathrm{B}}^{2} \models \Lambda$ .
Proposition C.13. If two bases  $\Lambda_{\mathrm{B}}^{1},\Lambda_{\mathrm{B}}^{2}$  are parallel, then the following properties hold:
1. Both  $\Lambda_{\mathrm{B}}^{1} \models \Lambda_{\mathrm{B}}^{2}$  and  $\Lambda_{\mathrm{B}}^{2} \models \Lambda_{\mathrm{B}}^{1}$ .
2.  $\mathrm{rank}(\Lambda_{\mathrm{B}}^{1}) = \mathrm{rank}(\Lambda_{\mathrm{B}}^{2})$
3.  $\Lambda_{\mathrm{B}}^{1}$  is universal if and only if  $\Lambda_{\mathrm{B}}^{2}$  is universal.
Proof of Proposition C.13.
1. Both  $\Lambda_{\mathrm{B}}^{1} \models \Lambda_{\mathrm{B}}^{2}$  and  $\Lambda_{\mathrm{B}}^{2} \models \Lambda_{\mathrm{B}}^{1}$ .
The claim follows directly from the definition of parallel bases. An agent set  $\Lambda$  is uniformly generated by  $\Lambda_{\mathrm{B}}^{1}$  if and only if it is uniformly generated by  $\Lambda_{\mathrm{B}}^{2}$ . Since by Proposition C.2 we know both  $\Lambda_{\mathrm{B}}^{1} \models \Lambda_{\mathrm{B}}^{1}$  and  $\Lambda_{\mathrm{B}}^{2} \models \Lambda_{\mathrm{B}}^{2}$ , we conclude that both  $\Lambda_{\mathrm{B}}^{1} \models \Lambda_{\mathrm{B}}^{2}$  and  $\Lambda_{\mathrm{B}}^{2} \models \Lambda_{\mathrm{B}}^{1}$ .
2.  $\mathrm{rank}(\Lambda_{\mathrm{B}}^{1}) = \mathrm{rank}(\Lambda_{\mathrm{B}}^{2}).$
Recall that the definition of rank refers to the size of the smallest basis that uniformly generates it,
$$
\operatorname {r a n k} (\Lambda) = \min  _ {\Lambda_ {B} \subset \mathbb {A}} | \Lambda_ {B} | \qquad \text {s . t .} \qquad \Lambda_ {B}   \models   \Lambda .
$$
Now, note that by property (1.) of the proposition, both sets uniformly generate each other. Therefore, we know that
$$
\operatorname {r a n k} \left(\Lambda_ {\mathrm {B}} ^ {1}\right) \leq \min  \left\{\left| \Lambda_ {\mathrm {B}} ^ {1} \right|, \left| \Lambda_ {\mathrm {B}} ^ {2} \right| \right\}, \quad \operatorname {r a n k} \left(\Lambda_ {\mathrm {B}} ^ {2}\right) \leq \min  \left\{\left| \Lambda_ {\mathrm {B}} ^ {1} \right|, \left| \Lambda_ {\mathrm {B}} ^ {2} \right| \right\},
$$
since the smallest set that generates each basis is no larger than the basis itself, or the other basis.
3.  $\Lambda_{\mathrm{B}}^{1}$  is universal if and only if  $\Lambda_{\mathrm{B}}^{2}$  is universal.
The claim again follows by combining the definitions of universality and parallel: If  $\Lambda_{\mathrm{B}}^{1}$  is universal, then by definition of parallel bases,  $\Lambda_{\mathrm{B}}^{2}$  must uniformly generate all the same agent sets including  $\mathbb{A}$ , and therefore  $\Lambda_{\mathrm{B}}^{2}$  is universal, too. Now, if  $\Lambda_{\mathrm{B}}^{1}$  is not universal, then it does not uniformly generate  $\mathbb{A}$ . By the definition of parallel bases, we conclude  $\Lambda_{\mathrm{B}}^{2}$  does not generate  $\mathbb{A}$  as well. Both directions hold for each labeling of the two bases without loss of generality, and we conclude.
This completes the argument for each property, and we conclude.
C.2 Analysis: Reaches
We now establish other properties of the reaches operator. Several of these results are based on a third modality of the reaches operator: always reaches, in which an agent eventually reaches an agent basis in all histories realizable in a given environment. We define this precisely as follows.
Definition C.10. We say agent  $\lambda \in \mathbb{A}$  always reaches  $\Lambda_{\mathrm{B}}$ , denoted  $\lambda \sqcup \stackrel{\sim}{\succ} \Lambda_{\mathrm{B}}$ , if and only if
$$
\forall_ {h \in \bar {\mathcal {H}}} \exists_ {t \in \mathbb {N} _ {0}} \forall_ {h ^ {\circ} \in \bar {\mathcal {H}} _ {h} ^ {t: \infty}} \exists_ {\lambda_ {\mathrm {B}} \in \Lambda_ {\mathrm {B}}} \forall_ {h ^ {\prime} \in \bar {\mathcal {H}} _ {h}} \lambda (h h ^ {\circ} h ^ {\prime}) = \lambda_ {\mathrm {B}} (h h ^ {\circ} h ^ {\prime}). \tag {C.24}
$$
The nested quantifiers allows the agent to become equivalent to different base behaviors depending on the evolution of the interaction stream. For example, in an environment that flips a coin to determine whether  $a_{\mathrm{heads}}$  or  $a_{\mathrm{tails}}$  is optimal, the  $\lambda$  might output  $a_{\mathrm{heads}}$  indefinitely if the coin is heads, but  $a_{\mathrm{tails}}$  otherwise. In this case, such an agent will still always reach the basis  $\Lambda_{\mathrm{B}} = \{\lambda_{\mathrm{B}}^{1}: h \mapsto a_{\mathrm{heads}}, \lambda_{\mathrm{B}}^{2}: h \mapsto a_{\mathrm{tails}}\}$ . Notice that we here make use of the notation,  $\bar{\mathcal{H}}_h^{t:\infty}$ , which refers to all history suffixes of length  $t$  or greater, defined precisely as
$$
\bar {\mathcal {H}} _ {h} ^ {t: \infty} = \left\{h ^ {\prime} \in \bar {\mathcal {H}} _ {h}: | h ^ {\prime} | \geq t \right\} \tag {C.25}
$$
We first show that the always reaches operator implies sometimes reaches.
Proposition C.14. If  $\lambda \rightsquigarrow \Lambda$ , then  $\lambda \rightsquigarrow \Lambda$ .
Proof of Proposition C.14.
Assume  $\lambda \rightsquigarrow \Lambda$ . That is, expanding the definition of always reaches, we assume
$$
\forall_ {h \in \bar {\mathcal {H}}} \exists_ {t \in \mathbb {N} _ {0}} \forall_ {h ^ {\circ} \in \bar {\mathcal {H}} _ {h} ^ {t: \infty}} \exists_ {\lambda_ {\mathrm {B}} \in \Lambda_ {\mathrm {B}}} \forall_ {h ^ {\prime} \in \bar {\mathcal {H}} _ {h}} \lambda (h h ^ {\circ} h ^ {\prime}) = \lambda_ {\mathrm {B}} (h h ^ {\circ} h ^ {\prime}). \tag {C.26}
$$
Further recall the definition of can reach  $\lambda \rightsquigarrow \Lambda_{\mathrm{B}}$  is as follows
$$
\exists_ {h \in \bar {\mathcal {H}}} \exists_ {\lambda_ {\mathrm {B}} \in \Lambda_ {\mathrm {B}}} \forall_ {h ^ {\prime} \in \bar {\mathcal {H}} _ {h}} \lambda (h h ^ {\prime}) = \lambda_ {\mathrm {B}} (h h ^ {\prime}). \tag {C.27}
$$
Then, the claim follows quite naturally: pick any realizable history  $h \in \mathcal{H}$ . By our initial assumption that  $\lambda \boxdot \stackrel{\sim}{\to} \Lambda$ , it follows (by Equation C.26) that there is a time  $t$  and a realizable history suffix  $h^{\circ}$  for which
$$
\exists_ {\lambda_ {\mathrm {B}} \in \Lambda_ {\mathrm {B}}} \forall_ {h ^ {\prime} \in \bar {\mathcal {H}} _ {h}} \lambda (h h ^ {\circ} h ^ {\prime}) = \lambda_ {\mathrm {B}} (h h ^ {\circ} h ^ {\prime}).
$$
By construction of  $hh^{\circ} \in \bar{\mathcal{H}}_h$ , we know  $hh^{\circ}$  is a realizable history. Therefore, there exists a realizable history,  $h^{*} = hh^{\circ}$ , for which  $\exists_{\lambda_{\mathrm{B}} \in \Lambda_{\mathrm{B}}} \forall_{h' \in \bar{\mathcal{H}}_h} \lambda(h^{*}h') = \lambda_{\mathrm{B}}(h^{*}h')$  holds. But this is exactly the definition of can reach, and therefore, we conclude the argument.
Next, we highlight the fact that every agent in a basis also reaches that basis.
Proposition C.15. For any agent set  $\Lambda$ , it holds that  $\lambda \sqcup \aleph \Rightarrow \Lambda$  for every  $\lambda \in \Lambda$ .
Proof of Proposition C.15.
Corollary C.16. As a corollary of Proposition C.15, any pair of agent sets  $(\Lambda_{\mathrm{small}}, \Lambda_{\mathrm{big}})$  where  $\Lambda_{\mathrm{small}} \subseteq \Lambda_{\mathrm{big}}$ , satisfies
$$
\forall_ {\lambda \in \Lambda_ {\text {s m a l l}}} \lambda \square^ {\Leftrightarrow} \Lambda_ {\text {b i g}}. \tag {C.28}
$$
We further show that, unlike sometimes and never reaches, always reaches is transitive.
Proposition C.17. Always reaches is transitive.
Proof of Proposition C.17.
We proceed by assuming that both  $\forall_{\lambda^1 \in \Lambda^1} \lambda^1 \boxdot \Leftrightarrow \Lambda^2$  and  $\forall_{\lambda^2 \in \Lambda^2} \lambda^2 \boxdot \Leftrightarrow \Lambda^3$  and show that it must follow that  $\forall_{\lambda^1 \in \Lambda^1} \lambda^1 \boxdot \Leftrightarrow \Lambda^3$ . To do so, pick a fixed but arbitrary  $\lambda^1 \in \Lambda^1$ , and expand  $\lambda^1 \boxdot \Leftrightarrow \Lambda^2$  as
$$
\forall_ {h \in \bar {\mathcal {H}}} \exists_ {t \in \mathbb {N} _ {0}} \forall_ {h ^ {\circ} \in \bar {\mathcal {H}} _ {h} ^ {\mathrm {t}: \infty}} \exists_ {\lambda^ {2} \in \Lambda^ {2}} \forall_ {h ^ {\prime} \in \bar {\mathcal {H}} _ {h}} \lambda^ {1} (h h ^ {\circ} h ^ {\prime}) = \lambda^ {2} (h h ^ {\circ} h ^ {\prime}).
$$
Now, consider for any realizable history  $hh^\circ h'$ , we know that the corresponding  $\lambda^2$  that produces the same action distribution as  $\lambda^1$  also satisfies  $\lambda^2 \Box \Leftrightarrow \Lambda^3$ . Thus, there must exist some time  $\bar{t}$  at which, any realizable history  $\bar{h} \bar{h}^\circ$ , will satisfy  $\exists_{\lambda^3 \in \Lambda^3} \forall_{\bar{h}' \in \bar{\mathcal{H}}} \lambda^2 (\bar{h} \bar{h}^\circ \bar{h}') = \lambda^3 (\bar{h} \bar{h}^\circ \bar{h}')$ . But then there exists a time  $(\bar{t})$ , that ensures every  $\lambda^2 \in \Lambda^2$  will have a corresponding  $\lambda^3 \in \Lambda^3$  with the same action distribution at all subsequent realizable histories.
$$
\forall_ {h \in \bar {\mathcal {H}}} \exists_ {t ^ {\prime} \in \mathbb {N} _ {0}} \forall_ {h ^ {\circ} \in \bar {\mathcal {H}} _ {h t ^ {\prime}: \infty}} \exists_ {\lambda^ {2} \in \Lambda^ {2}} \forall_ {h ^ {\prime} \in \bar {\mathcal {H}} _ {h}} \lambda^ {1} (h h ^ {\circ} h ^ {\prime}) = \underbrace {\lambda^ {2} (h h ^ {\circ} h ^ {\prime})} _ {\exists_ {\lambda^ {3} \in \Lambda^ {3}} = \lambda^ {3} (h h ^ {\circ} h ^ {\prime})}.
$$
Thus, rewriting,
$$
\forall_ {h \in \bar {\mathcal {H}}} \exists_ {t ^ {\prime} \in \mathbb {N} _ {0}} \forall_ {h ^ {\circ} \in \bar {\mathcal {H}} _ {h} ^ {t ^ {\prime}: \infty}} \exists_ {\lambda^ {3} \in \Lambda^ {3}} \forall_ {h ^ {\prime} \in \bar {\mathcal {H}} _ {h}} \lambda^ {1} (h h ^ {\circ} h ^ {\prime}) = \lambda^ {3} (h h ^ {\circ} h ^ {\prime}).
$$
But this is precisely the definition of always reaches, and thus we conclude.
Next, we show two basic properties of the set of all agents: it uniformly generates all agent sets, and it is always reached by all agents.
Proposition C.18. For any  $e$ , the set of all agents  $\bigtriangleup (i)$  uniformly generates all other agent sets, and (ii) is always reached by all agents:
$$
\begin{array}{l l} (i) & \forall_ {\Lambda \subseteq \mathbb {A}} \wedge \vDash \Lambda , \\ & (i i) \quad \forall_ {\lambda \in \mathbb {A}} \lambda \square^ {\Leftrightarrow} \wedge . \end{array} \tag {C.29}
$$
Proof of Proposition C.18.
$$
(i). \forall_ {\Lambda \subseteq \mathbb {A}} \mathbb {A} \vDash \Lambda
$$
The property holds as a straightforward consequence of Proposition C.2: Since any set  $\Lambda$  is a subset of  $\mathbb{A}$ , it follows that  $\mathbb{A} \models \Lambda$ .
$$
\left(i i\right). \forall_ {\lambda \in \mathbb {A}} \lambda \square^ {\Leftrightarrow} \Lambda
$$
The property holds as a straightforward consequence of Proposition C.15: Since every agent satisfies  $\lambda \in \mathbb{A}$ , it follows that  $\lambda \rightsquigarrow \mathbb{A}$ .
This concludes the argument of both statements.
C.3 Figure: Set Relations in CRL
Finally, in Figure 3 we present a visual depicting the set relations in CRL between an agent basis  $\Lambda_{\mathrm{B}}$ , an agent set it generates  $\Lambda$ , and the three agent sets corresponding to those agents in  $\Lambda$  that (i) sometimes, (ii) never, or (iii) always reach the basis. First, we highlight that we visualize  $\Lambda_{\mathrm{B}}$  as a subset of  $\Lambda$  since we define  $\Lambda_{\mathrm{B}} \subset \Lambda$  in CRL (Definition 4.2). However, there can exist triples  $(\Lambda_{\mathrm{B}}, \Lambda, e)$  such that  $\Lambda_{\mathrm{B}} \notin \Lambda$ , but that  $\Lambda_{\mathrm{B}}$  is not a subset of  $\Lambda$ . Such cases are slightly peculiar, since it means that the basis contains agents that cannot be expressed by the agent set  $\Lambda$ . Such cases are not in line with our definition of CRL, so we instead opt to visualize  $\Lambda_{\mathrm{B}}$  as a subset of  $\Lambda$ . Next, notice that the basis is a subset of both the agents that always reach the basis and the agents that sometimes reach the basis—this follows directly from the combination of Proposition C.14 and point (3.) of Theorem 4.3. By similar reasoning from Proposition C.14, we know that the set of agents that always reaches  $\Lambda_{\mathrm{B}}$  is a subset of the agents that sometimes reach the basis. Further, since sometimes and never reaches are negations of one another (Remark 3.2), observe that the two sets are disjoint, and together comprise the entirety of  $\Lambda$ . Lastly, we know that the set of optimal agents,  $\Lambda^{*}$ , contains only agents that never reach the basis, and thus the set  $\Lambda^{*}$  is disjoint from  $\Lambda_{\mathrm{B}}$  and the set of agents that sometimes reach  $\Lambda_{\mathrm{B}}$ .
Figure 3: A depiction of the division of a set of agents  $\Lambda$  relative to a basis  $\Lambda_{\mathrm{B}}$  through the reaches operator in CRL.