SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Junqi Wang
Department of Math & CS
Newark, NJ, 07102
junqi.wang@rutgers.edu
Pei Wang
peiwang@rutgers.edu
Patrick Shafto
shaft@rutgers.edu
Human learners have ability to adopt appropriate learning approaches depending on constraints such as prior on the hypothesis, urgency of decision, and drift of the environment. However, existing learning models are typically considered individually rather than in relation to one and other. To build agents that have the ability to move between different modes of learning over time, it is important to understand how learning models are related as points in a broader space of possibilities. We introduce a mathematical framework, Generalized Belief Transport (GBT), that unifies and generalizes prior models, including Bayesian inference, cooperative communication and classification, as parameterizations of three learning constraints within Unbalanced Optimal Transport (UOT). We visualize the space of learning models encoded by GBT as a cube which includes classic learning models as special points. We derive critical properties of this parameterized space including proving continuity and differentiability which is the basis for model interpolation, and study limiting behavior of the parameters, which allows attaching learning models on the boundaries. Moreover, we investigate the long-run behavior of GBT, explore convergence properties of models in GBT mathematical and computationally, document the ability to learn in the presence of distribution drift, and formulate conjectures about general behavior. We conclude with open questions and implications for more unified models of learning.
Learning and inference are subject to internal and external constraints. Internal constraints include the availability of relevant prior knowledge. External constraints include the availability of time to accumulate evidence versus the need make the best decision now or environmental non-stationarity. Standard models of machine learning tend to view different constraints as different problems, which impedes development of unified learning agents.
These internal and external constraints map onto classic dichotomies in machine learning. Availability of prior knowledge maps onto the Frequentist-Bayesian dichotomy in which the latter uses prior knowledge as a constraint on posterior beliefs, while the former does not. Within Bayesian theory, a classic debate pertains to uninformative, or minimally informative, settings of priors [Jeffreys, 1946, Robert et al., 2009]. Availability of time to accumulate evidence informs the use of generative versus discriminative approaches [Ng and Jordan, 2001], and static or drift/dynamic models [Dagum et al., 1992, Murphy, 2002]. Combining constraints on probability of beliefs and costs of data models cooperative communication [Wang et al., 2020b].
Learning agents must interpolate between modes of reasoning as necessary given the constraints of the moment. Imagine observing an agent behaving in an environment. As an observer, one may wish to learn about the environment from the agent's actions. However, any inferences depend on one's model of the agent and their constraints. How is the agent updating their beliefs? Do they have stable goals, or are they changing over time? Perhaps the agent is selecting actions to communicate what they know? In order to draw inferences over these possibilities, one must parameterize the space,
ideally in such a way one could optimize over the possibilities. Indeed, in order to implement these possibilities, the agent themselves must parameterize the space in order to interpolate between classic dichotomies such as Bayesian and frequentist, static and dynamic environments, and helpful versus neutral agent, given constraints.
We introduce Generalized Belief Transport (GBT), based on Unbalanced Optimal Transport (Sec. 1), which parameterizes and interpolates between known reasoning modes (Sec. 2.2), with four major contributions. First, we prove continuity in the parameterization and differentiability on the interior of the parameter space (Sec. 2.1). Second, we analyze the behavior under variations in the parameter space (Sec. 2.3). Third, we study sequential learning, where learners may (not) track the empirically observed data frequencies in (Sec. 3). Fourth, we investigate predictive performance under environmental drift (Sec. 4).
Notations.  $\mathbb{R}_{\geq 0}$  denotes the non-negative reals. Vector  $\mathbf{1} = (1,\dots ,1)$ . The  $i$ -th component of vector  $v$  is  $v(i)$ .  $\mathcal{P}(\bar{A})$  is the set of probability distributions over  $A$ . For a matrix  $M$ ,  $M_{ij}$  represents its  $(i,j)$ -th entry,  $M_{(i, - )}$  denotes its  $i$ -th row, and  $M_{(-j)}$  denotes its  $j$ -th column. Probability is  $\mathbb{P}(\cdot)$ .
1 Learning as a problem of unbalanced optimal transport
Consider a general learning setting: an agent, which we call a learner, updates their belief about the world based on observed data subject to constraints. There is a finite set  $\mathcal{D} = \{d^1,\dots,d^n\}$  of all possible data, that defines the interface between the learner and the world. The world is defined by a true hypothesis  $h^{*}$ , whose meaning is captured by a probability mapping  $\mathbb{P}(d|h^{*})$  onto observable data. For instance, the world can either be the environment in classic Bayesian inference [Murphy, 2012] or a teacher in cooperative communication [Wang et al., 2020b].
A learner is equipped with a set of hypotheses  $\mathcal{H} = \{h^1, \ldots, h^m\}$  which may NOT contain  $h^*$ ; an initial belief on the hypotheses set, denoted by  $\theta_0 \in \mathcal{P}(\mathcal{H})$ ; and a non-negative cost matrix  $C = (C_{ij})_{m \times n}$ , where  $C_{ij}$  measures the underlying cost of mapping  $d^i$  into  $h^{j-1}$ . The cost matrix can be derived from other matrices that record the relation between  $\mathcal{D}$  and  $\mathcal{H}$ , such as likelihood matrices in classic Bayesian inference or consistency matrices in cooperative communication (see details in Section 2.2). This setting reflects an agent's learning constraints: pre-selected hypotheses, and the relations between them and the communication interface (data set).
A learner observes data in sequence. At round  $k$ , the learner observes a data  $d_k$  that is sampled from  $\mathcal{D}$  by the world according to  $\mathbb{P}(d|h^*)$ . Then the learner updates their beliefs over  $\mathcal{H}$  from  $\theta_{k-1}$  to  $\theta_k$  through a learning scheme, where  $\theta_{k-1}, \theta_k \in \mathcal{P}(\mathcal{H})$ . For instance, in Bayesian inference, the learning scheme is defined by Bayes rule; while in discriminative models, the learning scheme is prescribed by a code book.
The learner transforms the observed data into a belief on hypotheses  $h \in \mathcal{H}$  with a minimal cost, subject to appropriate constraints, with the goal of learning the exact map  $\mathbb{P}(d|h^*)$ . We can naturally cast this learning problem as Unbalanced Optimal Transport.
1.1 Unbalanced Optimal Transport
Unbalanced Optimal Transport (UOT), introduced by Liero et al. [2018], is a generalization of (entropic) Optimal Transport [Villani, 2008, Cuturi, 2013, Peyre and Cuturi, 2019], that relaxes the marginal constraints. Formally, for non-negative scalar parameters  $\epsilon = (\epsilon_{P},\epsilon_{\eta},\epsilon_{\theta})$ , the UOT plan is,
$$
P ^ {\epsilon} (C, \eta , \theta) = \underset {P \in (\mathbb {R} _ {\geq 0}) ^ {n \times m}} {\arg \min } \left\{\langle C, P \rangle - \epsilon_ {P} H (P) + \epsilon_ {\eta} \mathrm {K L} (P \mathbf {1} | \eta) + \epsilon_ {\theta} \mathrm {K L} (P ^ {T} \mathbf {1} | \theta) \right\}. \tag {1}
$$
Here,  $\langle C,P\rangle = \sum_{i,j}C_{ij}P_{ij}$  is the inner product between  $C$  and  $P$ ,  $H(P) = -\sum_{ij}P_{ij}(\log P_{ij} - 1)$  is the entropy of  $P$ , and  $\mathrm{KL}(\mathbf{a}|\mathbf{b})\coloneqq \sum_{i}(a_{i}\log (a_{i} / b_{i}) - a_{i} + b_{i})$  is the Kullback-Leibler divergence between vectors. It is shown in Chizat et al. [2018] that UOT plans can be solved efficiently via Algorithm 1: Given a cost  $C$ ,  $P^{\epsilon}$  can be obtained by applying  $(\eta ,\theta ,\epsilon)$ -unbalanced Sinkhorn scaling on  $K^{\epsilon}\coloneqq e^{-\frac{1}{\epsilon_P} C} = (e^{-\frac{1}{\epsilon_P} C_{ij}})_{m\times n}$ , with convergence rate  $\tilde{\mathcal{O}} (\frac{mn}{\epsilon_P})$  [Pham et al., 2020].
To guarantee the hypotheses are distinguishable, we assume that  $C$  does not contain two columns that are only differ by an additive scalar.
Proposition 1. The UOT problem with cost matrix  $C$ , marginals  $\theta, \eta$  and parameters  $\epsilon = (\epsilon_{P}, \epsilon_{\eta}, \epsilon_{\theta})$  generates the same UOT plan as the UOT problem with  $tC$ ,  $\theta, \eta, t\epsilon = (t\epsilon_{P}, t\epsilon_{\eta}, t\epsilon_{\theta})$  for any  $t \in (0, \infty)$ . Therefore, the analysis on  $\epsilon$  and  $t\epsilon$  are the same for general cost  $C$ .
Thus a positive common factor on  $C,\epsilon_P,\epsilon_\eta ,\epsilon_\theta$  does not affect the solution of Eq. (1). Therefore, for the later analysis, we fix  $\epsilon_{P} = 1$  unless otherwise stated.
**Framework:** Generalized Belief Transport (GBT). Learning, efficiently transport one's belief with constraints, is naturally a UOT problem, i.e. a Generalized Belief Transport. Each round, a learner, defined by a choice of  $\epsilon = (\epsilon_{P}, \epsilon_{\eta}, \epsilon_{\theta})$ , updates their beliefs as follows. Let  $\eta_{k-1}, \theta_{k-1}$  be the learner's estimations of the data distribution and the belief over hypotheses  $\mathcal{H}$  after round  $k-1$ , respectively. At round  $k$ , the learner first improves their estimation of the mapping between  $\mathcal{D}$  and  $\mathcal{H}$ , denoted by  $M_{k}$ , through solving the UOT plan Eq. (1) with  $(C, \eta_{k-1}, \theta_{k-1})$ , i.e.  $M_{k} = P^{\epsilon}(C, \eta_{k-1}, \theta_{k-1})$ . Then with data observation  $d_{k}$ , the learner updates their beliefs over  $\mathcal{H}$  using corresponding row of  $M_{k}$ , i.e. suppose  $d_{k} = d^{i}$  for some  $d^{i} \in \mathcal{D}$ , the learner's belief  $\theta_{k}$  is defined to be the row normalization of the  $i$ -th row of  $M_{k}$ . Finally, the learner updates their data distribution to  $\eta_{k}$  by increment of the  $i$ -th element of  $\eta_{k-1}$ , see Algorithm 2.
Algorithm 1 Unbalanced Sinkhorn Scaling
input:  $C,\theta ,\eta ,\epsilon = (\epsilon_P,\epsilon_\eta ,\epsilon_\theta),N$  stopping
condition  $\omega$
output:  $P^{\epsilon}(C,\eta ,\theta)$
initialize:  $\mathbf{K} = \exp (-\epsilon_P C)$ $\mathbf{v}^{(0)} = \mathbf{1}_m$
while  $k < N$  and not  $\omega$  do
$$
\mathbf {u} ^ {(k)} \leftarrow \left(\frac {\eta}{\mathbf {K v} ^ {(k - 1)}}\right) ^ {\frac {\epsilon_ {\eta}}{\epsilon_ {\eta} + \epsilon_ {P}}},
$$
$$
\mathbf {v} ^ {(k)} \leftarrow \left(\frac {\theta}{\mathbf {K} ^ {T} \mathbf {u} ^ {(k)}}\right) ^ {\frac {\epsilon_ {\theta}}{\epsilon_ {\theta} + \epsilon P}}
$$
$$
P ^ {\epsilon} (C, \eta , \theta) = \operatorname {d i a g} (u) \mathbf {K} \operatorname {d i a g} (v)
$$
Algorithm 2 Generalized Belief Transport
input:  $C$ ,  $\theta_0$ ,  $\eta_0$ ,  $h^*$ ,  $N$ , data sampler  $\tau$
based on  $\mathbb{P}(d|h^{*})$ , stopping condition  $\omega$
output:  $M,\theta$
initialize:  $k \gets 1$
while  $k < N$  and not  $\omega (\theta)$  do
$$
M \leftarrow P ^ {\epsilon} (C, \eta_ {k - 1}, \theta_ {k - 1})
$$
get data  $d^i$  sampled from  $\tau$
$\eta_{k}\gets \text{update} (\eta_{k - 1},d^{i})$  via update rule
$$
\mathbf {v} \leftarrow M _ {(i, \dots)}
$$
$$
\theta_ {k} \leftarrow \mathbf {v} / \sum_ {h \in \mathcal {H}} \mathbf {v} (h)
$$
$$
k \gets k + 1
$$
2 Generalized Belief Transport
Many learning models—including Bayesian inference, Frequentist inference, Cooperative learning, and Discriminative learning—are unified under our GBT framework under choice of  $\epsilon$ . In this section, we focus on the single-round behavior of the GBT model, i.e., given a pair of marginals  $(\theta, \eta)$ , how different learners update beliefs with a single data observation. We first visualize the entire learner set as a cube (in terms of parameters), see Figure 1. Then, we study the topological properties of the learner set through continuous deformations of parameters  $\epsilon$ . In particular, we show that existing models including Bayesian inference, cooperative inference and discriminative learning are learners with parameters  $(1,0,\infty)$ ,  $(1,\infty,\infty)$  and  $(0,\infty,\infty)$  respectively in our UOT framework.
2.1 The parameter space of GBT model
The space of learners in GBT are parameterized by three regularizers for the underlying UOT problem (1):  $\epsilon_{P},\epsilon_{\eta}$  and  $\epsilon_{\theta}$ , each ranges in  $[0,\infty)$ . Therefore, the constraint space for GBT is  $\mathbb{R}_{\geq 0}^{3}$ , with the standard topology. When  $C,\theta$  and  $\eta$  are fixed (assume  $\eta \in \mathbb{R}_{>0}^{m}$ ), the map  $\epsilon = (\epsilon_{P},\epsilon_{\eta},\epsilon_{\theta})\mapsto (P^{\epsilon})$  bears continuous properties:
Proposition 2. The UOT plan  $P$  in Equation (1), as a function of  $\epsilon$ , is continuous in  $(0, \infty) \times [0, \infty)^2$ . Furthermore,  $P$  is differentiable with respect to  $\epsilon$  in the interior of its domain  $\mathbb{R}_{\geq 0}^3$ .
Continuity on  $\epsilon$  provides the basis for interpolation between different learning agents. The proof of Proposition 2 also implies the continuity on  $\eta$  and  $\theta$ . Further, towards the boundaries of the parameter space (where theories like Bayesian, Cooperative Communication live in), we show:
2Proofs of all claims are included in the Supplementary Materials.
Proposition 3. Let  $s_P, s_\eta, s_\theta \geq 0$  be arbitrary finite numbers, the following holds:
(1) The limit of  $P^{\epsilon}$  exists as  $\epsilon$  approaches  $(\infty, s_{\eta}, s_{\theta})$ . In fact,  $\lim_{\epsilon \to (\infty, s_{\eta}, s_{\theta})} P_{ij}^{\epsilon} = 1$  for all  $i, j$ .
(2) As  $\epsilon \to (s_P, \infty, s_\theta)$ ,  $P^\epsilon$  converges to the solution to
$$
\min  \langle C, P \rangle - s _ {P} H (P) + s _ {\theta} K L \left(P ^ {T} \mathbf {1} | \theta\right), w i t h c o n s t r a i n t P \mathbf {1} = \eta ,
$$
(3) Similarly, as  $\epsilon \rightarrow (s_P,s_\eta ,\infty)$ ,  $P^{\epsilon}$  converges to the solution to
$$
\min  \langle C, P \rangle - s _ {P} H (P) + s _ {\eta} K L (P \mathbf {1} | \eta), w i t h c o n s t r a i n t P ^ {T} \mathbf {1} = \theta .
$$
(4) And when  $\epsilon \rightarrow (s_P, \infty, \infty)$ , the matrix  $P^{\epsilon}$  converges to the EOT solution:
$$
\min  \langle C, P \rangle - s _ {P} H (P), \text {w i t h c o n s t r a i n t s} P ^ {T} \mathbf {1} = \theta \text {a n d} P \mathbf {1} = \eta .
$$
(5) When  $\epsilon \rightarrow (\infty, \infty, s_{\theta}), (\infty, s_{\eta}, \infty)$  or  $(\infty, \infty, \infty)$ , the limit does not exist, but the directional limits can be calculated.
Figure 1: The parameter space  $S$  of GBT. Parameters  $\epsilon = (\epsilon_{P},\epsilon_{\eta},\epsilon_{\theta})$  can take the value  $\infty$ , rendering the corresponding regularization to a strict constraint. The two dashed edges with  $\epsilon_{P} = \infty$  are not generally well-defined since the limits do not exist. The vertices corresponding to  $\theta \otimes \eta$ , Frequentist  $(\eta \otimes \mathbf{1})$  and  $\mathbf{1} \otimes \theta$  are the limits taken along the vertical edges. Given  $(C,\theta,\eta)$  as shown in the left corner, each colored map plots each GBT learner (differ by constraints)'s estimation of the mapping between hypotheses and data (UOT plan).
The parameter space for GBT with its boundaries can be visualized in Fig. 1. Proposition 3 implies that the parameter space is  $\mathcal{S} = [0,\infty ]^3\backslash (\{(\infty ,\infty ,x):x\in [0,\infty ]\} \cup$ $\{(x,x,\infty):x\in [0,\infty ]\})$  . In Fig. 1, segment  $[0,\infty)$  is mapped to  $[0,1)$  by sigmoid  $(\log (x))$  Then boundaries are added to the image cube  $[0,1)^3$  . The dashed lines on top of the cube indicates limits that do not exist.
2.2 Special points in the parameter space
Bayesian Inference. Given observed data, a Bayesian learner (BI) [Murphy, 2012] derives posterior belief  $\mathbb{P}(h|d)$  based on prior belief  $\mathbb{P}(h)$  and likelihood matrix  $\mathbb{P}(d|h)$ , according to the Bayes rule. Intuitively, due to soft time constraint  $(\epsilon_P = 1)$ , a Bayesian learner is a generative agent who puts a hard internal constraint on their prior belief  $(\epsilon_{\theta} = \infty)$ , and omits the estimated data distribution  $\eta$  in the learning process,  $(\epsilon_{\eta} = 0)$ .
Corollary 4. Consider a UOT problem with
cost  $C = -\log \mathbb{P}(d|h)$ , marginals  $\theta = \mathbb{P}(h)$ ,  $\eta \in \mathcal{P}(\mathcal{D})$ . The optimal UOT plan  $P^{(1,\epsilon_{\eta},\epsilon_{\theta})}$  converges to the posterior  $\mathbb{P}(h|d)$  as  $\epsilon_{\eta} \to 0$  and  $\epsilon_{\theta} \to \infty$ . Thus, Bayesian inference is a special case of GBT with  $\epsilon = (1,0,\infty)$ .
Frequentist Inference. A frequentist updates their belief from data observations by increasing the corresponding frequencies of datum. Intuitively, a frequentist is an agent who puts a hard constraint on the data distribution  $\eta$  ( $\epsilon_{\eta} = \infty$ ), and omits prior knowledge  $\theta$  ( $\epsilon_{\theta} = 0$ ) in a learning process without time constraint ( $\epsilon_{P} = \infty$ ). Formally we show:
Corollary 5. Consider a UOT problem with  $\theta \in \mathcal{P}(\mathcal{H})$ ,  $\eta = \mathbb{P}(d)$ . The optimal UOT plan  $P^{(\epsilon_P, \infty, 0)}$  converges to  $\eta \otimes 1$  as  $\epsilon_P \to \infty$ . Frequentist Inference is a special case of GBT with  $\epsilon = (\infty, \infty, 0)$ .
Cooperative Communication. Two cooperative agents, a teacher and a learner, are considered in Yang et al. [2018], Wang et al. [2020b], Shafto et al. [2021]. Cooperative learners (CI) draw inferences about hypotheses based on which data would be most effective for the teacher to choose Given a data observation, a cooperative learner derives an optimal plan  $L = \mathbb{P}(\mathcal{H},\mathcal{D})$  based on a prior belief  $\mathbb{P}(h)$ , a shared data distribution  $\mathbb{P}(d)$  and a matrix  $M$  specifies the consistency between data and hypotheses (such as  $M_{ij}$  records the co-occurrence of  $d^i$  and  $h^j$ ). Intuitively, a cooperative learner is also a generative agent who puts hard constraints on both data and hypotheses ( $\epsilon_{\eta} = \infty, \epsilon_{\theta} = \infty$ ), and aims to align with the true belief asymptotically, ( $\epsilon_P = 1$ ). Thus we show:
Corollary 6. Let cost  $C = -\log M$ , marginals  $\theta = \mathbb{P}(h)$  and  $\eta = \mathbb{P}(d)$ . The optimal UOT plan  $P^{(1,\epsilon_{\eta},\epsilon_{\theta})}$  converges to the optimal plan  $L$  as  $\epsilon_{\eta}\rightarrow \infty$  and  $\epsilon_{\theta}\rightarrow \infty$ . Cooperative Inference is a special case of GBT with  $\epsilon = (1,\infty ,\infty)$ , which is exactly entropic Optimal Transport [Cuturi, 2013].
Discriminative learning. A discriminative learner decodes an uncertain, possibly noise corrupted, encoded message, which is a natural bridge to information theory [Cover, 1999, Wang et al., 2020b]. A discriminative learner builds an optimal map to hypotheses  $\mathcal{H}$  conditioned on observed data  $\mathcal{D}$ . The map is perfect when, for all messages, encodings are uniquely and correctly decoded. Intuitively, a discriminative learner aims to quickly build a deterministic code book (implies  $\epsilon_{P} = 0$ ) that matches the marginals on  $\mathcal{H}$  and  $\mathcal{D}$ . We show that discriminative learner is GBT with  $\epsilon = (0,\infty ,\infty)$ :
Corollary 7. Consider a UOT problem with cost  $C = -\log \mathbb{P}(d,h)$ ,  $m = n$ , and marginals  $\theta = \eta$  are uniform. The optimal UOT plan  $P^{(\epsilon_P,\epsilon_\eta,\epsilon_\theta)}$  approaches to a diagonal matrix as  $\epsilon_{\eta},\epsilon_{\theta}\to \infty$  and  $\epsilon_P\rightarrow 0$ . In particular, discriminative learner is a special case of GBT with  $\epsilon = (0,\infty ,\infty)$ , which is exactly classical Optimal Transport [Villani, 2008].
Many other interesting models are unified under GBT framework as well. GBT with  $\epsilon = (0,\infty ,0)$  denotes Row Greedy learner which is widely used in Reinforcement learning community [Sutton and Barto, 2018];  $\epsilon = (\infty ,\infty ,\infty)$  yields  $\eta \otimes \theta$  which is independent coupling used in  $\chi^2$  [Fienberg et al., 1970];  $\epsilon = (\epsilon_{P},\epsilon_{\theta},\infty)$  is used for adaptive color transfer studied in [Rabin et al., 2014]; and  $\epsilon = (0,\epsilon_{\theta},\epsilon_{\eta})$  is UOT without entropy regularizer developed in [Chapel et al., 2021]. Other points in the GBT parameter space are also of likely interest, past or future.
2.3 General properties on the transportation plans
The general GBT framework builds a connection between the above theories, and the behavior of theory varies according to the change of parameters. In particular, each factor of  $\epsilon = (\epsilon_{P},\epsilon_{\eta},\epsilon_{\theta})$  expresses different constraints of the learner. Given  $(C,\theta ,\eta)$  as shown in the top-left corner of Fig. 1, we plot each learner's UOT plan with darker color representing larger elements.
$\epsilon_{P}$  controls a learner's learning horizon. When  $\epsilon_{P} \to 0$ , a learner's UOT plan is concentrated on a clear leading diagonal which allows them to make fast decisions. This corresponding to agents who are under the time pressure of making immediate decision, i.e. discriminative learner, or row greedy learner on the bottom of the cube (Fig. 1). Most of the time, one datum is enough to identify the true hypothesis and convergence is achieved within every data observation. When  $\epsilon_{P} \to \infty$ , GBT converges to a reticent learner, such as learners on the top of the cube. Data do not constrain the true hypothesis, and learners draw their conclusions independent of the data. In between, GBT provides a generative (probabilistic) learner. When  $\epsilon_{P} = 1$ , we have Bayesian learner and Cooperative learner, for whom data accumulate to identify the true hypothesis in a manner broadly consistent with probabilistic inference, and consistency is asymptotic.
$\epsilon_{\eta}$  controls a learner's knowledge on the data distribution  $\eta$ . When  $\epsilon_{\eta} \to \infty$ , GBT converges to a learner who is aware of the data distribution and reasons about the observed data according to the probabilities/costs of possible outcomes. Examples include the Discriminative and Cooperative learners on the front of the cube. When  $\epsilon_{\eta} \to 0$ , GBT converges to a learner who updates their belief without taking  $\eta$  into consideration, such as Bayesian learners on the back of the cube, and the Tyrant who does not care about data nor cost and is impossible to be changed by anybody.
$\epsilon_{\theta}$  controls the strength of the learner's prior knowledge. When  $\epsilon_{\theta} \to 0$ , GBT converges to learners who utilizes no prior knowledge. Hence, they do NOT maintain beliefs over  $\mathcal{H}$ , and draws their conclusions purely on the data distribution, such as a Frequentist learner on the left of the cube. When  $\epsilon_{\theta} \to \infty$ , GBT converges to a learner who enforces a strict prior such as Bayesian, Cooperative and Discriminative learners on the right of the cube. In particular, we show that:
Proposition 8. In GBT with  $\epsilon_{\theta} = \infty$ , cost  $C$  and current belief  $\theta$ . The learner updates  $\theta$  with UOT plan in the same way as applying Bayes rule with likelihood from  $P^{\epsilon}(C, \eta, \theta)$ , and prior  $\theta$ .
3 Sequential GBT - Static