Generalized Belief Transport
Authors: Junqi Wang
PEI WANG
Patrick Shafto
Venue: Submitted to ICLR 2023 (2023)

Abstract: Human learners have ability to adopt appropriate learning approaches depending on constraints such as prior on the hypothesis and urgency of decision. However, existing learning models are typically considered individually rather than in relation to one and other. To build agents that have the ability to move between different modes of learning over time, it is important to understand how learning models are related as points in a broader space of possibilities. We introduce a mathematical framework, Generalized Belief Transport (GBT), that unifies and generalizes prior models, including Bayesian inference, cooperative communication and classification, as parameterizations of three learning constraints within Unbalanced Optimal Transport (UOT). We visualize the space of learning models encoded by GBT as a cube which includes classic learning models as special points. We derive critical properties of this parameterized space including proving continuity and differentiability which is the basis for model interpolation, and study limiting behavior of the parameters, which allows attaching learning models on the boundaries. Moreover, we investigate the long-run behavior of GBT, explore convergence properties of models in GBT mathematical and computationally, and formulate conjectures about general behavior. We conclude with open questions and implications for more unified models of learning.

Under review as a conference paper at ICLR 2023
GENERALIZED BELIEF TRANSPORT
Anonymous authors
Paper under double-blind review
ABSTRACT
Human learners have ability to adopt appropriate learning approaches depending constraints such as prior on the hypothesis and urgency of decision. However, existing learning models are typically considered individually rather than in re- lation and other. To build agents that have the ability move between different modes of learning over time, it is important to understand how learning models are related as points in broader space of possibilities. We introduce a mathematical framework, Generalized Belief Transport (GBT), that unifies and generalizes prior models, including Bayesian inference, cooperative communica- tion and classification, as parameterizations of three learning constraints within Unbalanced Optimal Transport (UOT). We visualize the space of learning models encoded by GBT cube which includes classic learning models as special points. We derive critical properties of this parameterized space including proving continuity and differentiability which is the basis for model interpolation, and study limiting behavior of the parameters, which allows attaching learning models on the boundaries. Moreover, investigate the long-run behavior of GBT, explore convergence properties of models in GBT mathematical and computationally, and formulate conjectures about general behavior: We conclude with open questions and implications for more unified models of learning.
INTRODUCTION
Learning and inference are subject internal and external constraints. Internal constraints include the availability of relevant prior knowledge, which may be brought to bear on inferences based on data. External constraints include the availability of time to accumulate evidence or make the best decision now. Human learners appear to be capable of moving between these constraints as necessary. However, standard models of machine learning tend to view constraints as different problems, which impedes development of unified view of learning agents.
Indeed, internal and external constraints on learning map onto classic dichotomies machine learning. Internal constraints such as the availability of prior knowledge maps onto the Brqueentit-Bayesiiin dichotomy in which the latter uses prior knowledge as a constraint on posterior beliefs, while the former does not. Within Bayesian theory, classic debate pertains uninformative, or minimally informative, settings of priors (Jeffreys 1946; Robert et al.) 2009). External constraints such as availability of time to accumulate evidence versus the need to make the best possible decision now informs the use of generative versus discriminative approaches (Ng and Jordan 2001). Despite the fundamental nature of these debates, and the usefulness of all approaches the appropriate contexts, we are unaware of prior efforts to unify these perspectives and study the full space of possible models.
We introduce Generalized Belief Transport (GBT), based on Unbalanced Optimal Transport (Sec. which paramterizes and interpolates between known reasoning modes (Sec. 3.2). with four major contributions. First, we prove continuity the parameterization and differentiability on the interior of the parameter space (Sec. 3.1)). Second, analyze the behavior under variations in the parameter space (Sec. (33).. Third, we consider sequential learning, where learners may (not) track the empirically observed data frequencies. And finally state our theoretical results, simulations and conjectures about the sequential behaviors for various parameters for generic costs and priors (Sec. (.2)).
Notations. R>o denotes the non-negative reals. Vector (1,.... 1). The i-th component of vector v is e(). P(A) is the set of probability distributions over A. For a matrix M, My represents its ji entry, Mi__ denotes its i-th row, and M MC,)) denotes its j-th column. Probability is TP(

---

Under review as a conference paper at ICLR 2023
2 LEARNING AS A PROBLEM OF UNBALANCED OPTIMAL TRANSPORT
Consider general learning setting: an agent, which we call a learner, updates their belief about the world based on observed data subject to constraints. There is a finite set D = =dd "'d'...,,'' ,,') of all
possible data, that defines the interface between the learner and the world. The world is defined by true hypothesis h", whose meaning is captured by a probability mapping P(d[h") onto observable
data. For instance, the world can either be the environment in classic Bayesian inference (Murphy 2012) or a teacher in cooperative communication (Wang et al. 2020b)
A learner is equipped with set of hypotheses H {h' ..... hmm which may not contain h"; an initial belief on the hypotheses set, denoted by 6g E P(H); and a non-negative cost matrix C (Gig m where C.j measures the underlying cost of mapping d' into h The cost matrix can be derived from other matrices that record the relation between 'D and H. such as likelihood matrices in classic Bayesian inference or consistency matrices in cooperative communication (see details in Section B.2). This setting reflects an agent's 's learning constraints: pre-selected hypotheses, and the relations between them and the communication interface (data set).
A learner observes data in sequence. At round k, the learner observes data dę that is sampled from 'D by the world according to P(dlh"). Then the learner updates their beliefs over H from 0/-1 to 6g through learning scheme, where - 0g E P(H). For instance, in Bayesian inference, the learning scheme is defined by Bayes rule; while in discriminative models, the learning scheme is prescribed by code book.
The learner transforms the observed data into a belief on hypotheses h E H with a minimal cost, subject appropriate constraints, with the goal of learning the exact map IPd()t). We can naturally cast this learning problem as Unbalanced Optimal Transport.
2.1 UNBALANCED OPTIMAL TRANSPORT
Unbalanced optimal transport is generalization of (entropic) Optimal Transport. Optimal trans- port infers a coupling that minimizes the cost of transporting between two marginal probability distributions (Monge 1781 Kantorovich] 2006; Villani 20088. Entropic Optimal Transport adds regularization term based on the entropy of the inferred coupling, which has desirable computational consequences (Cuturi 2013 Peyré and Cuturi] 2019). Unbalanced OT further relaxes the problem by allowing one to approximately match marginal probability distributions.
and 0 distribution matrix = (Pij)axm is called a transport plan or coupling between 17 and if P has 7 and f as its marginals. Given cost matrix C (Cij)nxm (Rzz))mxX, mmx, En- tropy regularized optimal transport (EOT) (Cuturi 2013) solves the optimal transport plan pep that minimizes the entropy regularized cost of transporting n) into 0. Thus for a parameter ep > 0: per = arg minppe(()))) P) - -PpH(P), where (n,, is the set of all trans- port plans between 17 and 0, (C,P) = Ei, CipPu is the inner product between C' and P, and H(P) = Eng Paj log Più + Pij is the entropy of P.
Let 17
be two probability distributions. A joint
Unbalanced Optimal Transport (UOT), introduced by Liero et al. (2018), is generalization of EOT that relaxes the marginal constraints. The degree of relaxation controlled by two regularization terms. Formally, for non-negative scalar parameters = (cp, Cy. co), the UOT plan is,
(C,,,, = arg min
+
+ KaKL(P" 10)).
(I)
Pe(Rgo)nxm
Here KL(a|b)= := E; a; log(a;/bb) /b:) + b; is the Kullback-Leible divergence between vectors. UOT differs from BOT in relaxing the hard constraint that satisfy the given marginals 1 and 0, to soft constraints that penalize the marginals being far from 7) or 0 2 In particular, as e,, and éq oc, we recover the EOT problem.
To guarantec the hypotheses are distinguishable, we assume that does not contain two columns that are only differ by an additive scalar. UOT also generalizes measures of arbitrary mass, i.c. the total mass of n does need equal to 0.
2

---

Under review as a conference paper at ICLR 2023
Proposition 1. The UOT problem with cost matrix C. marginals 8, 7, and parameters (pp.Cnn generates the same UOT plan as the UOT problem with tC, 8, 7, te = (tép, tep, tea) for any e (0, oc). Therefore. the analysis on e and te are the same for general cost C.
Eyp Eg)
The objective function in Eq. ①T) is linear on C,cp, Ey. co. so a positive common factor does not affect the solution. In the discussion under general cost matrix C, properties that hold for also valid for all Le (t > 0).
UOT plans can be solved efficiently via Sinkhorn-style algorithm (Sinkhorn and Knopp 1967). Roughly speaking, (1), 0, < Juunbalanced Sinkhorn scaling of a matrix M is iterated alternation of row and column normalizations of M with respect to (1, 0, (see Algorithm Chizat et al. (2018) shows
that: given a cost C, the UOT plan P can be obtained by applying (1) 0, -unbalancee Sinkhorn scaling on K := 1 (e
+ "p C22) mmxn with convergence rate
mn cp
Pham al. 2020).
Generalized Belief Transport. Learning, efficiently transport one's belief with constraints, is
naturally a UOT problem. Each round, learner, defined by choice of (ep, f E0), updates their beliefs as follows. Let 1, 0-- be the learner's estimations of the data distribution and the belief over hypotheses H. after round k 1, respectively. At round k; the learner first improves their estimation of the mapping between D and H. denoted by Mr. through solving the UOT plan Eq. ([) with (C, 0 i.c. M₁ P (C, ms- m Then with data observation dx. the learner updates their beliefs over H using corresponding row of Mr. i.e. suppose de = d' for some d' E D the learner's belief 0g is defined to be the row normalization of the i-th row of M4. Finally, the learner updates their data distribution to T)s by increment of the i-th element of 1 I. see Algorithm
Algorithm 1 Generalized Belief Transport
Algorithm 2 Unbalanced Sinkhorn Scaling
input: C. Oo. 1o. h", NN, data sampler T
input: C, 0, 17, (ep, En: En: N stopping
based on P(d h"), stopping condition
condition w
output: M, 0
output: P 1,,
initialize: k 4-
initialize: K = exp(- (() = 1m
while k < N and not w()) do
while k < and not w do
M PC(C, (.CC: Ox-1)
u(4)
71
get data d' sampled from T
Kv(k-1)
T f update(M:-1. d') via update rule
v t -MM On + V/ENEN v(h)
(M)
T6
(
A + k +1
end while Pa (C, n, 0) = diag(u Kdiag(u))
end while
3 GENERALIZED BELIEF TRANSPORT (GBT)
Many learning models with different constraints-includee Bayesian inference, Frequentist inference, Cooperative learning, and Discriminative learning- are unified under our GBT framework by varying the choice of In this section, we focus the single-round behavior of the GBT model, i.c., given pair of marginals (0, n)), how different learners update beliefs. We first visualize the entire learner set cube (in terms of parameters), see Figure []] Then, study the topological properties of the learner set through continuous deformations of parameters e. In particular, we show that existing models including Bayesian inference, cooperative inference and discriminative learning are learners with parameters (1, 0, oo), (1, oc, oo) and (0, co, oo) respectively in our UOT framework.
3.1 THE PARAMETER SPACE OF GBT MODEL
The space of constrained belief-updating learners in GBT are parameterized by three regularizers for the underlying UOT problem (I EP, e, and ég, each ranges in [0, oo). Therefore, the constraint space for GBT is R3 with the standard topology. When C, 0 and v) fixed (assume 17 E RRT)), the map c (cp, Cy ca) + (Pt) bears continuous properties:
3

---

Under review as a conference paper at ICLR 2023
Proposition 2. Furthermore, P is differentiable with respect to e in the interior.
The UOT plan P in Equation L function of is continuous in (0, (0,,00x0,,00)
Continuity on e provides the basis for interpolation between different learning agents. The proof of Propositionaaasee implies the continuity on 17 and 6. Further, towards the boundaries of the parameter space (where theories like Bayesian, Cooperative Communication live in), we show:
Proposition 3. For any finite sp, Sq. se 2 O, the limit of P exists as approaches (oo, Sq, So). In
fact, lim (20.,,.00))) P.j = for all j. Moreover, P converges to the solution to
min(C, P) (P) soKL(P" with constraint P1 1/
a + (sp, 00, so). Similarly, 1e converges to the solution to
min(C, P) - -pp + s, (I)) with constraint p = =, a - (sp, Sn, dc). And when e - (sp; x0, oo), the matrix P converges to the EOT solution:
min(C, P) - HpH(( (P), with constraints pT = and P1 =
When c + (oc, 00, s0), (00, 8y, 00) or (oo, 00, 00), the limit does not exist, but the directional limits can be calculated.
Spare
Frequentis
Sayesion
Row Groedy
Column Gready
%, m)
nalive dgmostiogg..0
Csser
Figure 1: The parameter space S of GBT. Parameters é = (EP, Enyy 6o) can take the value oo, rendcring the corresponding regularization to a strict constraint. The two dashed edges with ep = Do are not generally well-defined since the limits do not exist. The vertices corresponding to 0 @ 1), Frequentist (17 & 1) and 1 & 0 the limits taken along the vertical edges. Given (C, 0, 7) shown in the left comer, each colored map plots each GBT learner (differ by constraints)'s 's estimation of the mapping between hypotheses and data (UOT plan).
The generalized parameter space for UOT with its boundaries can be visualized in Fig. I Function sigmoid (1og(x)) (igmoid(log(z)) maps segment [0, 5o) to [0, 1) smoothly. Then
add boundaries to the image cube [0, 1)3. The dashed lines in the figure indicates limits that do not exist. The parameter space then S [0, ({)o) o0, : x [0, coo} U {(oc, x, o0) x [0, oc]]). Later, we may still mention (oc, oo, ea) and (oo, En, x.), only for case where the direction is vertical (along axis of ep).
3.2 SOME SPECIAL POINTS IN THE PARAMETER SPACE
Bayesian Inference. Given data observation, Bayesian learner (BI) (Murphy) 2012) derives posterior belief P(hjd) based prior belief P(h) and likelihood matrix ((dh)), according to the Bayes rule. Intuitively, due to soft time constraint (Ep = 1), Bayesian learner a generative agent who puts hard internal constraint their prior belief (ca o0), and omits the estimated data distribution 77 in the learning process, (E, = 0). As a direct application of Prop 3 we show:
Corollary 4. Consider a UOT problem with cost C log (d]h)) marginals f P(h), r P(D).
The optimal UOT plan p().s)(o)) converges the posterior (hild as en + 0 and eg + co. Bayesian
inference is a special case of GBT with (1, 0o).
Moreover, by relaxing the constraint on the prior (i.c., 0 << Eq < oo), one obtains a parameterized family of less informative priors.
Proofs of all claims included in the Appendices.
4

---

Under review as a conference paper at ICLR 2023
Frequentist Inference. A frequentist updates their belief from data observations by increasing the corresponding frequencies of datum. Intuitively, a frequentist is an agent who puts a hard constraint on the data distribution 1) (e, oo), and omits prior knowledge 0 (Eg = 0) in a learning process without time constraint (ep = oc). Formally we show:
Corollary 5. Consider a UOT problem with I E P(H), = P(d). The optimal UOT plan P(er,c0,0) converges yi @ 1 as 6p - co. Frequentist Inference is special case of GBT with 00,,000,00 0).
Cooperative Communication. Two cooperative agents. a teacher and learner, are considered in Yang et al. (2018); Wang et al. (2020b): Shafto et al. (202T). Cooperative learners (CI) draw inferences about hypotheses based on which data would be most effective for the teacher to choose (see brief model summary in the Appendix A Given a data observation, cooperative learner derives an optimal plan L P(H, D) based on prior belief P(h), shared data distribution P(d) and a matrix M specifies the consistency between data and hypotheses (such as Mij records the co-occurrence of d' and h²). Intuitively, a cooperative learner also generative agent who puts hard constraints on both data and hypotheses (e,, = oc, oc), and aims to align with the true belief asymptotically, (ep = 1). Thus as a direct application of Proposition we show: Corollary 6. Let cost C
log M, marginals p(l,6n,5)) converges to the optimal plan L Cq oo and co - oc. Cooperative Inference is special case of GBT with = (1,00,00) oc, which is exactly entropic Optimal Transport (Cuturi 2013).
(P))) and rj = P(d). The optimal UOT plan
Discriminative learning. A discriminative learner decodes an uncertain. possibly noise corrupted, encoded message, which is natural bridge to information theory (Cover 1999 Wang et al., 2020b). discriminative learner builds an optimal map to hypotheses H conditioned on observed data D. The map is perfect when, for all messages, encodings are uniquely and correctly decoded. Intuitively, discriminative learner aims to quickly build a deterministic code book (implies <p = 0) that matches the marginals on H. and D. Thus, discriminative learner is GBT with (0, oc, co):
Corollary 7. Consider a UOT problem with cost C = log (dd, h), m = n, and marginals 0 = are uniform. The optimal UOT plan p(tprenits) approaches to a diagonal matrix as cy. co - OC and cp + O. In particular, discriminative learner is a special case of GBT with c 00, 00, oc), which is exactly classical Optimal Transport (Villani 2008).
Many other interesting models are unified under GBT framework as well. GBT with c = (0, oo, 0) denotes Row Greedy learner which is widely used in Reinforcement learning community (Sutton and Barto 2018); (oo, p0, 00) yields 17 @ 9 which is independent coupling used in x² (Fienberg et al. 1970); (ep, eg, oo) is used for adaptive color transfer studied in (Rabin et al. (014); and
6 (0, 60, 6,) UOT without entropy regularizer developed in (Chapel et al. 202 l). Other points in
the GBT parameter space are also of likely interest, past or future.
3.3 GENERAL PROPERTIES ON THE TRANSPORTATION PLANS
The general GBT framework builds a connection between the above theories, and the behavior of theory varies according the change of parameters. In particular, each factor of e = (ep, Eny Eg) expresses different constraints of the learner. Given (C, 8, 17) as shown in the top-left corner of Fig. we plot each learner's UOT plan with darker color representing larger elements.
EP controls a learner's 's learning horizon. When ep + O, agents are under the time pressure of making immediate decision, hence GBT converges a discriminative learner, or Row Greedy learner on the bottom of the cube (Fig. [].. Their UOT plans have a clear leading diagonal which allows them to make fast decisions. Most of the time, one datum is enough to identify the true hypothesis and convergence is achieved within every data observation. When ép co, GBT converges to reticent learner, such as learners on the top of the cube. Data do not constrain the true hypothesis, and learners draw their conclusions independent of the data. In between, GBT provides generative (probabilistic) learner. When ep = 1, we have Bayesian learner and Cooperative learner, for whom data accumulate to identify the true hypothesis in manner broadly consistent with probabilistic inference, and consistency is asymptotic.
En controls leamer's 's knowledge the data distribution n) When 6, co, GBT converges to a learner who is aware of the data distribution and reasons about the observed data according to the probabilities/costs of possible outcomes. Examples include the Diseriminative and Cooperative
5

---

Under review as a conference paper at ICLR 2023
learners the front of the cube. When c, 0, GBT converges to learner who updates their belief without taking 7 into consideration, such as Bayesian learners on the back of the cube, and the Tyrant who does not care about data nor cost and is impossible to be changed by anybody.
Eo controls the strength of availability of prior knowledge for the learner. When Eo + oo, GBT converges a learner who enforces prior Discriminative learners on the right of the cube. Actually, BGT follows Bayes rule when co = oo (Prop gp. When eg 0, GBT converges to learners who utilizes no prior knowledge. Hence they do NOT maintain belief's over H. and draws their conclusions purely on the data distribution, such as Frequentist learner 7) @ 1 on the left of the cube.
the hypotheses, such as Bayesian, Cooperative and
Proposition 8. In GBT with 6g = oc, cost C and current belief t. The learner updates 0 with UOT plan in the same way as applying Bayes rule with likelihood from P (C, 17 0), and prior O.
4 SEQUENTIAL GBT: ASYMPTOTIC BEHAVIOR
One interesting difference between the one-shot case considered above and the sequential case is the possibility of observing many data points. In addition the learning models in the GBT parameter space, in this section, we consider whether the learner's marginal on the data is fixed a priori, or accumulates evidence based on experience.
4.1 BASICS
The sequential GBT model consists of a teacher and a learner. The teacher samples data from probability distribution (not necessarily related to some h E H), and the learner follows GBT with cost C, and parameter The learner starts with prior 6u. and applies in each round k GBT with mh-1 and fx-1 to generate 0% through the UOT solution Mg. the Preliminary sequential model (PS), we assume Tik- = 1/ for all k. However, in practice, learner does not have access fj PPd(h") Instead, each round the learner may choose to use the current statistical distribution data as an estimation of 7), i.e., 11 (d) x i : k, d; = d} + nu(d) according to the observed data sequence, where no (d)
in add-one smoothing (Murphy) 2012)) the prior counts to avoid zero in Thus we have the Real sequential model (RS) where 7) 0.5, 17. It is easy to see that the sequence of posteriors form time-homogeneous Markov chain on P(H).
0 (e.g.,
In statistics, a model is said to be consistent crrongly--onsistenn when, for every fixed hypothesis E H. the model's belief over the hypotheses H converges on in probability (almost surely) as more and more data are sampled from " IP(d))), when 0's are considered random variables. The consistency has been well studied for Bayesian Inference since Bernstein and von Mises and Doob (Doob) 1949), and recently demonstrated for Cooperative Communication (Wang al. 2020a). The challenging problem arises when one tries learn h² that contained in the pre-selected hypothesis space H. It is not clear which h E H is the 'correct' target to converge to. Thus consistency does not fit the situation in sequential GBT.
For sequential GBT models, we state the properties directly in the language of posterior sequence (Ox) as random variables, and name them if necessary. We focus on whether the sequence converges (and in which sense), and how conclusive (how likely to provide stable, fixed h E H as the result) the sequence is. We provide some theoretical conclusions, and fill the gaps with empirical results and conjectures.
4.2 RESULTS AND CONTECTURES
Results this section are stated on different 68 values. According to Prop. I we could focus on Ep 1 for generic cost matrix C, and general result of (Ep, 66) becomes the same as the (1, /.,,/e, case. So we choose Ep = 1 in simulations.
co Conclusive and Bayesian-style. These are located the right side of Fig. 1 and contain many well-studied learners: Bayesian, Cooperative, Discriminative, Row Greedy etc. According to Prop & learners in this class perform "Bayesian" style learning.
There are two theoretical results: e, = 0 (Bayesian) and e, = oo (SCBI learner Wang et al. 2020a Others are explored in simulations.

---

Under review as a conference paper at ICLR 2023
Theorem 9 (Doob 1949). Wang et al. 2020a)). In GBT sequential model (both (PS) and (RS)) with é = (Cp, O, oo) where Ep E (0, oo), the sequence 2₂ converges to some òn almost surely, is the closest column of e C/sp to nj in the sense of KL-divergence, when 0 is positive (on each entry).
When En = Eg oe, the models (PS) and (RS) have slightly different behaviors.
Lemma 10. For (ep, 00, 00), ep & (0, o0), given cost C' with initial belief 6 P(H)) and fixed teaching and learning distribution Te = E (D) for all k, i.c., model (PS), then the belief random variables (+&)ee have the same expectation on h: Es, [0(h)] (n)).
63
0.
02
loxt update, 10.1g anett Ann overt
1ast0 optate of
She 5n aaeer
spddta of sentet
20x10 of
100e1
10v30
2542t
peat
1001 mt
Figure 2: Evidence of general consistency: we plot the percentage of episodes that reaches a threshold (0.999) by round number (in colors of the bars). Each bar represents size of matrix. for each bar 100 matrices randomly sampled, and 1000 rounds were simulated per matrix. "exact" means learner uses 7 = n) (PS). "update" means learner uses statistics on current data in the episode (RS). "uot" takes = (1, 40, 40) and "ot" comes with exact and é (1, 2o, 0o).
Theorem (PS). Consider a learning problem with initial belief Oo P(H), and the true hypothexis h" defined by 7 E P(D). If the learner's data distribution Ti 17, then belief random variables (Ox)ee converge to the random variable Y in probability, where Y = Enen (o(h) and Y is supported on {Ōx heH with P(Y 5x) = (h) ( for G, Cg = оc and Cp E (0,
Corollary 12. Given fixed data sequence d, sampled from n,, if 0g converges to و,s then the j-th column of M2 converges to
Thus a GBT learner, with access to the data distribution and using strict marginal constraints, converges to distribution D same with probability 1. Moreover, the probability of which column shaped into 7j is determined by their prior 0g. That is, GBT learners converge to the truth by changing of their original hypotheses into the true hypothesis.
For the (RS) model, the result is similar, but Lemma I0 fails to hold:
Proposition 13. Consider a learning problem with cost C. initial belief Gg P(H). the true hypothesis h- defined by E P(D). For the (RS) problem, the belief random variables (Qx)ee
satisfies that for any k-+00 BEW
lim (P) > 8) 1. a consequence, Mj as the transport
plan has a dominant column (h²) with total weights > 1 s, and (MM))s The (i)) <s
In fact, as long as sequence 7)s as random variables converges to 7 in probability, the above proposition holds. The limit lim Σ P(O(h) 1 s) measures how conclusive the model is.
hEH
In contrast with standard Bayesian other inductive learners, Proposition 13 showws that GBT learner able to learn any hypothesis mapping P(d)h") up given threshold s with probability In addition to unifying disparate models of learning, GBT enables a fundamentally more powerful approach to learning by empirically monitoring the data marginal.
Fig. 2 illustrates convergence over learning problems and episodes. In each bar, we sample 100 learning problem (C, h²) from Dirichlet distribution with hyperparameters the vector 1. Then we sample 1000 data sequences (episodes) of maximal length N 10000. The learner learns with Algorithm I where the stopping condition w set to mахдеи 0(h) > 1 - -s with & = 0.001. The y-axis in the plots represents the percentage of total episode converged. The color indicates in how many rounds the episode converges. For instance, in the bar corresponding to '10 x 10. s___ppate_uu uot', with 10 data points (yellow portion), about 50% episodes satisfy the stopping condition.
7

---

with ranges in [10, 25, 50, 100]. Here "exact' and "update" indicate the problem is (PS) or (RS), respectively. For parameters, uot represents the parameter choice (er eg En 40) of represents the parameter choice (Ep = 1, Ey oo). The first plot illustrates that learners that do not have access to the true hypothesis (empirically builds estimation of 77) learn faster than learners who have full access. The second plot indicates with fixed number of hypotheses, learning is faster when the dimension of 'D Under review as a conference paper at ICLR 2023
In Figure| the first plot shows results for 10 10 and 5 3 matrices. The second plot shows results for rectangular matrices of dimension m x 10 with m ranges in [5, 10, 25, 50, 100]. The third plot shows results for square matrices of dimension
increases. The third plot shows that the GBT learner scales well with the dimension of the problem.
0.5
distance
0/2
Bat
0.02
20
$0.0
0.2
IIv my-aa dajl IIv b/b
8P
Figure 3: Lefft: Behavior of models spanning the line segment hetween BI and CI. ep = 1 and co = oc, when varies from to the theory changes from BI to CI. Each bar graphs the Monte-Cart result of 400,000 teaching sequences, changes from BI to CI continuously from 8(h") by Bernstein-von Mises (h) by Theorem 11 Right: the Euclidean distances of each coefficient a(h) BT result (blue crosses), and CI result (orange dots).
empirically observe that the coefficients a(h) of the limit terms of >Ee kee ((h),,
Then study the learners that interpolate between Bayesian and Cooperative learners (located on the line connecting CI and BI in Fig Consider a fixed learning problem (C, 0g h²). Consistency of Bayesian inference states that asymptotically, the learner Bayesian converges hypothesis hg E H almost surely where hú is the hypothesis closest to h" under KI. divergence. Theorem [1 indicates that GBT cooperative learner modifies of the hypotheses into h" in probability I. The probability of le' converges to ht is determined by (o(h).
a particular
In Fig. 3 study the asymptotic behavior of the learners corresponding to 6n oo). with en E (.,0.02 0.2, 0.5, 1, 2, 5, 50, oo). We sample a learning problem with a dimension 5 x from Dirichlet distribution with hyperparameters the vector Each learner (I, cy. oo) equipped with a fixed C, Oo and their convergence summary in the bar graph. continuous transition from a Bayesian learner to cooperative learner can be empirically observed: the coefficients a(h) of the limit in terms of Enen a(h)6, changes from 8(h²) by Bernstein-von Mises to (h) by Theorem
17 for all k. We run 400, 000 learning episodes per learner, and plot
From the previous empirical results, we conclude the following conjecture:
Conjecture 14. When c = (ep, Gn oc), where ep E c0), the sequence of posteriors O₃ from generic C, t,, 0 and e as rundom variables satisfy lim Σ P()Ox(h) 1|< for any e >> k-OOOEH
We further report an empirical property observed in simulation, which suggests a possible rate of convergence. For given C, Op and 7) fix cp 1 and cæ oc, as G, changes from 0 to cc, pick out those episodes with (h) > 0.95 and plot the values (e) 0g (h) - In(1 - 0g (h)) for each h against (Fig. bottom). Near linear relations observed away from the first several rounds and before the values reaches the precision threshold. These are empirical estimates of the rate of convergence.
There is a special case on the boundary, the Independent Coupling (oo, oc, oo), whose limit is taken vertically along cp-axis, see Sec. 3.1l Independent Coupling has fixed posterior, where Law( deg. the normalization of each row of p(c0,00,00) is Og.
60 0: Inconclusive and independent. The following holds for both (PS) and (RS):
Proposition 15. For e = (Ep, Eny 0) with ep E (0, oo), as Th - n almost surely, the sequence O₂ of posteriors as sequence of random variables converges in probability variable O where
8

---

Under review as a conference paper at ICLR 2023
60-
10
20
0
0.02
Figure 4: Top: For a learning problem C, behaviors different learners with Ep = 1, 6g DC and various en (denoted in figure) on conclusion distributions, a(h) in bar graph, plots below bars are estimated convergence rates E (h)/(( (x())) averaged on episodes converging h, per hypothesis.
P(O = - and v² = -P/L (Ej= and P - ENER (h) - < = for generic (for all but in a closed subset) cost and r).
(C, Therefore, for any 8 > 0,
With ca the constraint column-sum (c,-term) fails to affect the transport plan, thus the O₂ the sequence are independent from each other, in contrast that in all other cases the adjacent ones correlated via nondegenerate transition distribution. The independence makes the sequence of posterior-samples in one episode behave totally random, thus rarely converge as points in P(H). Furthermore, when consider the natural coupling (Qx-1. θx) from Markov transition measure for co 0 (which is independent), (10x-1 1 O 2) converges to the variance Var(n). In contrast, for Eg oc, (10,-1 θ₂ 12) converges if Conj. holds.
co (0, co): partially conclusive. From Conj. 14 and Prop. 15 together with the continuity of the transition distribution on e, we conjecture the following continuity on conclusiveness when Cp E (0,
Conjecture 16. For both (PS) and (RS) models, when = (ep, En, Eq) with ep, eg (0, oc), the pos- terior sequence 2g, generated from generic C. m,, 9 and satisfy that limg ΣnER (h)- 11 < s) exists, and (0, 1). for any > O.
RELATED WORK
Prior work defines and outlines basic properties of Unbalanced Optimal Transport (Liero et al.) 2018
Chizat et al.) 2018 Pham et al., 2020). Bayesian approaches are prominent in machine learning
(Murphy 2012) and beyond (Jaynes 2003 Gelman al., 1995). There is also research cooperative
learning (Wang et al. 2019 2020b a) see also (Liu et al. [021]] Yuan et al., 202T Zhu 2015 Liu
al. 2017 Shafto and Goodman, 2008 Shafto al. 2014]] Frank and Goodman 2012
Goodman
and Frank 2016 Fisac et al. 2017 Ho et al. 2018 Laskey et al. (017). Discriminative learning is
reciprocal problem which one sees data and asks which hypothesis best explains it (Ng and
Jordan 200T Mandler 1980). We are unaware of any work that attempts to unify and analyze the
general problem learning in which each of these are instances.
CONCLUSIONS
We have introduced Generalized Belief Transport (GBT), which unifies and parameterizes classic instances of learning including Bayesian inference, Cooperative Inference, and Discrimination. as Unbalanced Optimal Transport (UOT). We show that each instance a point in a continuous, differentiable on the interior, 3-dimensional space defined by the regularization parameters of UOT. In addition to supporting generalized learning, we prove and illustrate asymptotic consistency and estimate rates of convergence, including convergence to hypotheses with zero prior support. In summary, GBT unifies very different modes of learning, yielding a powerful, general framework for modeling learning agents.

---

Under review as a conference paper at ICLR 2023
ETHIC STATEMENT
The main contributions of this paper are theoretical, rather than practical, in nature. While understand- ing learning and inference more unified and generalized ways may have broad impact including causing any ethic problems, nothing is likely to be realized as direct consequence of this work.
REFERENCES
Lactitia Chapel, Rémi Flamary, Haoran Wu, Cédric Févotte, and Gilles Gasso. Unbalanced optimal transport through non non-negative penalized linear regression. Advances in Neural Information Processing Systems, 34:23270-23222 -22822, 2021.
Lenaic Chizat, Gabriel Peyré, Bernhard Schmitzer, and François-Xavier Vialard. Scaling algorithms for unbalanced optimal transport problems. Mathematics of Computation, 87(314):2563-2676 2018.
Thomas M Cover. Elements of information theory. John Wiley Sons, 1999.
Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. In Advances in neural information processing systems, pages 2292-2300, 2013.
Joseph Doob. Application of the theory of martingales. Le calcul des probabilites et ses applications, pages 23-27, 1949.
Stephen E Fienberg al. An iterative procedure for estimation in contingency tables. The Annals of Mathematical Statistics, 41(3)9977-917, 1970.
Jaime Fisac, Monica Gates, Jessica B Hamrick, Chang Liu, Dylan Hadfield- Menell,, Malayandi Palaniappan, Dhruv Malik, S Shankar Sastry, Thomas L Griffiths, and Anca D Dragan. Pragmatic- pedagogic value alignment. arXiv preprint arXiv: 177770033555 2017.
Michael C Frank and Noah D Goodman. Predicting pragmatic reasoning in language games. Science, 336(6084):998 998, 2012.
Andrew Gelman, John B Carlin, Hal S Stern, and Donald B Rubin. Bayesian data analysis. Chapman and Hall/CRC, 1995.
Noah D Goodman and Michael C Frank. Pragmatic language interpretation probabilistic inference. Trends in cognitive sciences, 20(11 11818-889 2016.
Mark K Ho, Michael L Littman, Fiery Cushman, and Joseph L Austerweil. Effectively learning from pedagogical demonstrations. Proceedings of the Annual Conference of the Cognitive Science Soccityy 2018.
Edwin T Jaynes. Probability theory: The logic of science. Cambridge university press, 2003.
Harold Jeffreys. An invariant form for the prior probability in estimation problems. Proceedings of the Royal Society of London, Series A. Mathematical and Physical Sciences, 186(1007):453-661, 1946.
Leonid V Kantorovich. On the translocation of masses. Journal of Mathematical Sciences, 133(4): 1381 1382, 2006.
Michael Laskey, Caleb Chuck, Jonathan Lee, Jeffrey Mahler, Sanjay Krishnan, Kevin Jamieson, Anca Dragan, and Ken Goldberg. Comparing human-centric and robot-centric sampling for robot deep learning from demonstrations. In Robotics and Automation (ICRA), 2017 IEEE International Conference on, pages 358-365. IEEE, 2017.
Matthias Liero, Alexander Mielke, and Giuseppe Savaré. Optimal entropy-transpon problems and new hellinger-kamee kantorovich distance between positive measures. Inventiones mathematicae, 211(3): 969-1117, 2018.
10

---

Under review as a conference paper at ICLR 2023
Weiyang Liu, Bo Dai, Ahmad Humayun, Charlene Tay, Chen Yu, Linda B Smith, James M Rehg, and Le Song. Iterative machine teaching. In International Conference on Machine Learning, pages 2149-2158 PMLR, 2017.
Weiyang Liu, Zhen Liu, Hanchen Wang, Liam Paull, Bernhard Schölkopf, and Adrian Weller. Iterative teaching by label synthesis. Advances in Neural Information Processing Systems, 34, 2021.
George Mandler. Recognizing: The judgment of previous occurrence. Psychological review, 87(3): 252. 1980.
Gaspard Monge. Memory on the theory of excavations and embankments. History of the Royal Academy of Sciences of Paris, 1781.
Kevin P Murphy. Machine learning: probabillisicc perspective. MIT press, 2012.
Andrew Ng and Michael Jordan. On discriminative vs. generative classifiers: A comparison of logistic regression and naive bayes. Advances in neural information processing systems, 14, 2001.
Gabriel Peyré and Marco Cuturi. Computational optimal transport. Foundations and Trends in Machine Learning, 1 115-6))355-607, 2019.
Khiem Pham, Khang Le, Nhat Ho. Tung Pham, and Hung Bui. On unbalanced optimal transport: An analysis of sinkhorn algorithm. In International Conference on Machine Learning, pages 7673-7682. PMLR, 2020.
Julien Rabin, Sira Ferradans, and Nicolas Papadakis. Adaptive color transfer with relaxed optimal transport. In 2014 IEEE international conference on image processing (ICIP), pages 4852 4856. IEEE, 2014.
Christian P Robert, Nicolas Chopin, and Judith Rousseau. Harold jeffreys's theory of probability revisited. Statistical Science, 24(2):141-172, 2009.
Patrick Shafto and Noah D. Goodman. Teaching games: Statistical sampling assumptions for learning in pedagogical situations. In Proceedings of the 30th annual conference of the Cognitive Science Society, Austin, TX, 2008. Cognitive Science Society.
Patrick Shafto, Noah D Goodman, and Thomas L Griffiths. A rational account of pedagogical reasoning: Teaching by, and learning from, examples. Cognitive Psychology. 71:55-89, 2014.
Patrick Shafto, Junqi Wang, and Pei Wang. Cooperative communication belief transport. Trends in cognitive sciences, 25( 25(10)822-822, 2021.
Richard Sinkhorn and Paul Knopp. Concerning nonnegative matrices and doubly stochastic matrices. Pacific Journal of Mathematics, 21(2):333-445 1967.
Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018.
Cédric Villani. Optimal transport: old and new, volume 338. Springer Science & Business Media, 2008.
Junqi Wang, Pei Wang, and Patrick Shafto. Sequential cooperative bayesian inference. In International Conference on Machine Learning, pages 10039-10049, PMLR, 2020a.
Pei Wang, Pushpi Paranamana, and Patrick Shafto. Generalizing the theory of cooperative inference. AlStats, 2019.
Pei Wang, Junqi Wang, Pushpi Paranamana, and Patrick Shafto. A mathematical theory of cooperative communication. Advances in Neural Information Processing Systems, 33, 2020b.
Scott Cheng-Hsin Yang, Yue Yu, Arash Givchi, Pei Wang, Wai Keen Vong, and Patrick Shafto. Optimal cooperative inference. In AISTATS, volume 84 Proceedings of Machine Learning Research, pages 375-385. PMLR, 2018.
11

---

Under review as a conference paper at ICLR 2023
Luyao Yuan, Dongruo Zhou, Junhong Shen, Jingdong Gao, Jeffrey L Chen, Quanquan Gu, Ying Nian Wu, and Song-Chum Zhu. Iterative teacher-aware learning. Advances in Neural Information Processing Systems, 34:29231-29222 2021.
Xiaojin Zhu. Machine teaching: An inverse problem machine learning and an approach toward optimal education. In Twenty-Ninhh AAA/ Conference on Artificial Intelligence, 2015.
12

---

Under review as a conference paper at ICLR 2023
A ADDITIONAL MATERIALS
Algorithm 3 Unbalanced Sinkhorn Scaling
input: C. 0. 17) (Ep, En, Ep), NN stopping condition w
initialize: K exp(- exp(-ep(), v()) 1m
while & < N and not w do
(A) < (Kv(TTT) opter, v(k) t KT Tptop
end while
output: M diag (w) (olk)Kdaag (d)
Cooperative Communication. Cooperative communication formalizes single problem com- prised of interactions between two processes: teaching and learning. The teacher and learner have beliefs about hypotheses, which are represented as probability distributions. The process teaching is to select data that move the learner's beliefs from some initial state, to a final desired state. The process of learning is then, given the data selected by the teacher, infer the beliefs of the teacher. The teacher's selection and learner's inference incur costs. The agents minimize the cost to achieve their goals. Communication is successful when the learner's helief, given the teacher's 's data, is moved to the target distribution.
Formally, denote the common ground between agents: the shared priors on H and D by P(h) and P(d), the shared initial matrix over "D and H by M of size [D] x |H). In general, up normalization, M is simply non-negative matrix which also specifies the consistency between data and hypotheses"
In cooperative communication, learner's goal is minimize the cost of transforming the observed data distribution P(D) to the shared prior over hypotheses P(H). A learner's cost matrix C+ (C |M|x [74] defined as Ct₁, log M. A learning plan is a joint distribution Lij PL (d, h2) represents the probability of the learner inferring h, given d, It is proved in (Wang
(Lig). where
et al. 2019) that:
Proposition 17. Optimal cooperative communication plans. L, is the EOT plan with cost C and marginals being r P(d) and Phh).
B PROOFS
Proposition I The UOT problem with cost matrix C, marginals 0, and parameters = (Ep,en,
Enj Eq)
generates the same UOT plan as the UOT problem with LC, 0, 7) = (tep, Len, tea) for any E (0, oc).
Proof. Consider that the UOT problem solution is
() arg min Pe(Rgo)nxm
tkL((")10)) (2)
where the objective function is linear on C and e.
ple(tCn 0) =
arg min Pe(
{(tC,) P) tt teoKL(P²110)
arg min PE(Ro))
L. L.C,, - (P
KL 1
P OC,
(3)
11
Proposition 2. Furthermore, P is differentiable with respect to in the interior.
The UOT plan P in Equation ( as a function of e, is continuous in (0, o0) x [0, o0)
Data, d, consistent with hypothesis, hy, when Mgj > 0.
13

---

Under review as a conference paper at ICLR 2023
Proof. For simplicity, in this proof, for vector v, use both i, and o(1) represent component of v.
By definition, the UOT plan P minimizes the objective function e)(;:)) (C, epH (P) e, K(P1()) + e L(KL(P")))) Since Ω2 is a strict convex function on P. there is only one minimal P. So the UOT plan P is the solution to V y= = O. From direct calculation,
(Vpl)e = Cij + cplll P.j + cy 6,(10
P.,) Inn(()) + co(In( Pky) In (j))
k=1
k=1
and
epdady
Enpii Pat
EgOJI E Paj
Pss
EE
As we assume that P. > 0 for all i, all the terms above are well-defined. Besides, Vp is C on 77 and e. Therefore, we can show P" (C, 17. is continuous not only on e but also on 1) and 0 after checking Hessian. From implicit function theorem, if we show the above Hessian is invertible for cp > 0, then the results of the proposition are true. Equivalently, it suffices to show that det H # where matrix H is the flattened V2.2 by mapping ki, () + (im + j, km + 1).
Invertibility of H. Let r be the vector of reciprocals of row sums of i... T3 1/ (E, Po). and similarly, let be the vector of reciprocals of column sums of P, P.e.e. (2, P.z). Then
cpdu15J Psj
(V20)
+ C,,ar + codgic..
Let be the map (i.j) + (im + j), then & induces a reshaping of P to a vector of size mn, denoted by P When there is no ambiguity, we may omit the o superscript.
Further define pd as vector of dimension mn where P cp/P.. By definition, H e (dddiag(p))) + e @ (diag(r)) <o(dia(()) & 1, where 1 is the k x k matrix of ones, and 3 B Kronecker product (tensor product of matrices). Decompose H + G where D (p(diag(p")) and G cnLm @ (diag(r)) + eo (d) & In.
From now on, we may use P-row, P-column to represent i, j style indices, and G-row, G-column or simply row/column to represent those of G. or the ones in range [1, mn]. D is diagonal, and det G = 0. Furthermore,
(x) any row or column of G with index k can be represented by an entry position (i.j) of P by inverse of o, and any rows of indices kn k2 k3, ką corresponding to (1): (21, (12, (12, (i.c.. determined as intersections of two P-rows and two P-columns) is linearly dependent: G(k,.)) + G(kk.)) G(ka..) - G O, we denote this property as (*).
Structure of det Н: Let D = diag(pi, P2.... Pmnn) then det H is polynomial on pk's with constant term O. Each term in det H is of form f(I) (TI Pk) for each subset I C (1,2..... mn). and the coefficient (T) = det G(Z,T) where G(z,z) is the submatrix with lines indices not in I, i.e. the entries of G(I,I) are of the form Gi with i E I and j E I.
Next we show that ((Z) is nonnegative for all I, then with Ph > 0 for all k. we can conclude that det H 0. Since I S. (1, mn). o d'((I) C (1,2...., .n} (1. m), and & is bijection, we may not distinguish I from o (I), in order to make the statement neater.
[Operation-(s) on ZJ: We want to investigate the operations on producing subset J such that f(Z) f(J). By the properties of determinant, (*) induces one operation: when I con- taining integer pairs which can form the vertices of rectangle, (I)
Moreover, for any ki, ka, ka, ha such indices in (+), we generate row G(ka.)) by G(ka_)) G (
then if {ki, k2, k3} C I, we can build G(ka..) on any G(k:.), thus the determinant det (I.z) = + det G(I,z) (positive for ke and ka, negative for ki ). Similarly, if we follow the same operation on columns, we have det Geol + det G(z,z)- And when doing both, det Geol-row det G
(I.I)
14

---

Under review as a conference paper at ICLR 2023
Therefore, know that if A2, hts E I, and J {ka} (K; for any i 1,2,3, then f = f(J). Such operations changing I to J is denoted by operation-*. In short, an operation-* moves an end of a small "L-shaped" set of pairs along P-row or P-column, producing another I-shaped set of 3 pairs.
2. [Regularized form of I, and decomposition of nondegenerate regularized form I into L-shaped subsets]: Once I or any J equivalent to I via operations-* contains pairs satisfying condition (+), f(Z) O, then call I degenerate. In decomposing I, when we find it degenerate, we stop since
((Z) is known.
We decompose I of pairs inductively in the following way before stopping. Start with any (i.j) E I, we look for pairs of form (i, () and (k.j) in I, adding them into the subset A(i.)) containing j). Then check the degeneracy, by looking for whether I contains a point (k. 1) with (i,1), (i, k) A(uJ). whenever Z is degenerate, changing the set I to regularized form using operation-*'s ss For each (ks [) with (i,1) E A(.))) then (k, can be constructed on (k, I) via operation-* with (i,j) and (i, 1). Thus we modify I into J (i.0) UI
stop since ((Z) O. Next we enlarge A(..) by
(k,1) () that f(I) = ([T) and adding (i, into set A(i.)) Similar process can be done for those (k.,) 1) I with (k E A
After regularizing I and enlarging A(i.)) maximum about (i,j), get a regularized form I of I, with fZ) f(J), and component A(.,) of L-shape. The set of J A(,) has elements of form (k,2) with (i, 1) E A(a or (k,j) E A(r,)) as they are already moved to A(i.)) by operation- 8. Therefore, J(A(),) is supported on rectangular region by deleting all P-rows (k_ and P-columns (_,)'s where k,''s occur in A(i.))
Repeating the L-shaped component construction above for A(,.s))> we can transform I into regularized form (not unique or standard) I and we have a decomposition Z = - A(a,s)) into L-shaped components, which do not intersect with each other. The name "regularized form" is given to the transformed set with a L-shaped decomposition, and since only operation-* is applied, fII)= f(
3. [Properties between the L-shaped subsets:] For each I which we did not conclude f(Z) = 0 in the last step, we get I and a decomposition I U.er A, into L-shaped subsets.
The construction of components A, induces such a property: for two distinct components A, there no elements (i.j) E A₁ and (k, I) E A,, in normal words, the A₁ occupies certain P-rows and P-columns which is distinct from those of Ag.
For (i,j) and (ks, 1) with i k and j * I, Gim+ij.mt from the formula that G.m+ Grrdd + coe,ō5t Therefore, the decomposition It Uier A, induces a decomposition of matrix G 24) into blockwise diagonal matrix
GA 0
0 GA
0
0
0
GA
So for a decomposition I U.er AG, we haves f(It) IheT f(A))
4. [A) for an L-shaped componentt) The last part is to show f(A) 0 for all L-shaped components. Recall that Gim+ = c,r,6sh + Cuczōzı, so for A an L-shaped component with s P-rows and P-columns, G(A.A) in general is of form
V1 + C1
T1
0
ri
TI
Ct-
T1
G(A,A)
+
Cr
(5)
0
C₁
C
r2
Ce
:
0
Ct
+
15

---

Under review as a conference paper at ICLR 2023
Recall the formula det
C D = de and the matrix determinant lemma
det(diag(c) + +r11") = (1 - rl diag(c) 11 det(diag(c)) = II c1 + -(r/aa)-
If 1ort the determinant ((a,A) can be calculated directly by the matrix determinant lemma above.
E B where E contains the upper left t x t C
If $ > 1 and t > 1, we cut Eq. (5) into 4 blocks
D
part, zero but the last row, zero but the last column, D a matrix in a similar form as E.
According to the characters of B.C C.C. stated above, it can be found that CF B of &IF which an 8 x s-matrix. The entry E,, det F(tst-,,:tt))/ det where E(1 is the matrix E without the last row and last column, moreover, E (IT EI+
< 1/c.. There- fore, CE-IB A11 with X C and D CE drag((2:s) + (ct A)117, whose determinant is positive according to the matrix determinant lemma.
(IT G(1 E((r/aa)) = a(1 + E (n//a)
As consequence, det ((,,) > for each L-shaped components A. So combining the discussions in [1 [1-4], we have det H = det(D G) 0.
Then the implicit function theorem implies the differentiability of P on c.
Proposition 3 For any finite sp, 8y, 2 O, the limit of Ps exists approaches to (oc, 8ns se). In fact, lim, (00.5..50) P. = for all i,j (Limit 1). Moreover, P converges to the solution to
min(C, P) - Hp + (oK 10)), with constraint P1 as c (sp, 80) s0) (Limit 2). Similarly, P converges to the solution
min(C, P) sP) + K,,KL(P1))) with constraint PT = 0,
(7)
as (sp, Sy. 0o) (Limit 3). And in the case when e - 00), the matrix P converges to the EOT solution (Limit 4):
min (C, P) sp)(P), with constraints PT and P1 V).
When e + (oo, 00, se), sn, 20) or (oo, oo, 00), the limit does not exist, but the directional limits can be calculated.
and H(P) is strictly concave, therefore H has unique maximum mmn at Pej 1, denoted by 1. Similarly, K KL(bb) (a, In bx) - a, + b;), V, L(ab),: = In In by KL is strictly convex, therefore KL has minimum 0 a; 6, for all
Proof. Recall that HPP) - Ei, In Paj., (VpH)) = - Pn -n
Limit 1. Shown by contradiction: When e + (oo, sn,sa) sa). suppose the limit lim (30.ss))))) Ps for some (i,j) does not exist, or is not 1. Thus there is >> 0 that, for any ō > 0 and N > 0, there exists parameter e = (EP, En, c0) such that ep N, len syl and lea sol satisfying IPG - 1|
However, for any 0 < < 1/2, let 1, let E = (L-t)
min ((P:e) S
Q(1:c) < for some G > 0 where (1)iz = 1 for all (i.j), and any E ((Cp, co) : : < cn <
/s, < co < 3sp 3sa/2,1 2, }. So there is N > 0 such that NE > G + max;; Cij + mn + L
where inf (P1)n) + (('' P'1 0)), meaning those P with P. > e for some
(i,j) is not minimizing Ω.
The contradiction indicates that lim Pú for all i,j-
Limit 2 & 3: The situation of cg - 20 and Cn + oc are similar, so we only prove for co + 0o case.
Let P denote the solution Eq. 77
16

---

Under review as a conference paper at ICLR 2023
Let P be the solution to the optimization with constraints. (sp Ek=1 Pky -z-
We
first
show
that
This is similar to limit 1. Suppose the limit either does not exist or is not 0, then there exists an e such that for any N > 0, 6 > O, there exists ep N, |6n syl < 8 and (ep spl < 6, such that
P
0g
>
for some j. Thus (L((P') E' for some E' > O. Consider that P) 2 0, H(P) -mn and K (P) 2 are lower bounded, we can take sufficiently large N such that the p satisfying Eq. D satisfy (+) > S(P:e) making pe fail to optimize ((;e), which is a contradiction. Thus we have (s), k= Pkj 0g.
For each e cn 60), let 0r denote the (P')"1, then for any e, the solution p is also the solution to
min(C, P) + <pll(P) e,, e, L(PIl)) with constraint pT 0".
(10)
Denote (p
(C, =c,
&, When cr E (0, 20), the new objective
function & (PP, En) is continuous on P and Ep,,,, and each minimization problem gets a unique solution since the objective function is strictly convex. Therefore, the limit (im,-(s:: P. We show this via contradiction:
Suppose the opposite, there exists some such that ||P -Pll2 & for e arbitrarily close to (sp, Sq, 00). Let
inf pT PT (P,,81,8)) (P,, 8n).
0 since the minimum P is unique and the objective is strictly convex. The sets pT è 0c is compact since it is closed and bounded, so there exists bounds b (b,,b, b1,bb),))) for = (EP, Enp kny such that the bound where lep - sssl < b1, |e, syl 62 and εş b3. max ( s,) (PP, Ep, en) a/3 for with pTe and pa its Euclidean projection { }. and max (PP, EP, 6n) 4(P", Sp,, sn) a/3 for with pT lr and pd its Euclidean projection to {PT e 0).
Let e be a parameter in the above bound b to (SP, Sn, oo), where p= a L in & far from P. Then 4(P, 6n) O(P", s,y) a/3 ((P, sp, sn) +2/3a > cp,)) a/3 1 (Po, cp, cy), which is a contradiction the assumption that the argmin.
Limit 4: Similar to the previous two limits, we can say that lim,-osp. Σx=1 Pk O and
lim, lim,--(sp,o0,00) Σ k= P 17.- Then the problem becomes the EOT problem, which has unique solution.
Boundaries at 6, 0 éœ 0: It is simple check the continuity when 6, - 0 or cœ + 0. From the continuity and differentiability hold for e, - 0 or Eg + 0 when ép > 0.
Prop. 2
Nonexistence of the limits when 6p, + oc, and directional limits: Let sequence é1, 6 where = (Ep, e eg) satisfy lim ep = lim e oc and lim(é,/epp) = f. then the limit P of p satisfy P.j = ((nn ej - (tn)/(1+ + 1), since the limit minimizes the following objective function
H(P) + (KL(P))))
The reason is, as Σnh = 1, (P) and (L())(n) cannot vanish for the same P. thus the minima of objective function approaches to infinity, therefore the finite terms (C, P) and cp ( tend to have no effect on the minimal point as cp, Cn increases to infinity.
A direct consequence of the above discussion is, when changes. the limits P of those sequences changes, which indicates that the limit of p as e - (oc, 0o, sp) fails to exist. And similar situation happens when (0o, 8n. 80;
xc)
17

---

Under review as a conference paper at ICLR 2023
Nonexistence of the limits when Cp, co oc, and directional limits: Similar to the discus-
sions above, let the sequence é1, 6.,... where (f Sy. 6)) satisfy lim;- G = (oo, 2o, oo).
Further let lim lim(c,/ u, lim(ce/ cppp)) w, then ps converges to the solution the problem
H(P) ox
which could considered as another UOT problem with cost function constantly
Corollary 4
Consider a UOT problem with cost C' = = Pd]lh), marginals 0 = P(h), n P(D).
The optimal UOT plan p()..)50)) converges to the posterior P(hld) as En + 0 and - cc. Bayesian inference is a special case of GBT with (1, 0, oo).
Proof. As direct application of Limit of Proposition 3 we only need show that the optimal plan p(1,0,00) is propositional the posterior P(hld).
p(1,0,00 = arg min K(P) == arg min -
(11)
PEU(!)
PEU(0)
where U(0)= (P MIDx m))p"1=
Let À E R+* consider the corresponding Lagrangian problem:
L(P,A) = PC (A
Partial derivatives Pp., = 0 and 02 L = 0 result the following system of equations:
log P. log P(d b4 P(h))
(12)
Calculation shows that the solution to Equation 12 is Pıj P(d,),))P(hs) P(hy)d:). Hence the proof is completed.
= P(dll))P(h))) x
Σ. P(d.1h,)
Corollary 5 Consider UOT problem with 0 E P(H), P(d). The optimal UOT plan p(ep.co,0) converges to 17 @ 1 as Ep - oc. Frequentist Inference is special case of GBT with e = (oo, 00, 0).
Proof. As direct application of Proposition 3 we only need to show that p(oc,0o,)) 2 1. Notice that Eq is equivalent to
arg min H(P), with constraint P1 17
(13)
PE(Ra))mxm
Hence plo0,00.0) = &1 1.
Corollary 6 Let cost C = - log M, marginals = (P)) and = P(d). The optimal UOT plan
p(l.ea.a)) converges to the optimal plan L as + oo and Eq + oc. Cooperative Inference is
special case of GBT with (1, xc, 20), which is exactly entropic Optimal Transport (Cuturi 2013).
Proof. According to proposition 17 L p(1,00,0)) and the convergence result a direct application of Limit 4 of Proposition
Li
Corollary 7 Consider UOT problem with cost C log (d, m n, and marginals 0 uniform. The optimal UOT plan p(certust) approaches diagonal matrix c,,, eg - and Ep - O. In particular, discriminative learner is a special case of GBT with e = (0, 00, oo), which is exactly classical Optimal Transport (Villani 2008
18

---

Under review as a conference paper at ICLR 2023
Proof. Limit 4 of Proposition 3 implies the convergence of p(en-(nr0)) - p(0,0,,00) Eys Co 00 and Ep O. When n,, p(0,00,00) is permutation matrix is the result of Wang al. (P000oorroposiin 8].
Proposition 8 In GBT with cg = oc, cost C and current belief f. The learner updates 0 with UOT plan in the same way applying Bayes rule with likelihood from (C(C, n. 0), and prior f.
Proof. From Algorithm for a general data point d' chosen, the GBT takes the vector normalization of some row Ps, i..., 0' / Pt))
On the other hand, when we apply Bayes rule to P prior is f P(h), likelihood P(d]h) is the column normalization of P", satisfying P(d' |(d" = P (, P.j) = /0 The last equality is because ((i) En Pi when 6o co. So the posterior P(hld*) ) is the vector normalization of P(d' (h)), by P(d PP()) / x 0g = P.j. Therefore, P(h'ld') = (hh).
Now, we introduce some notations will be used in the following proofs.
Notations. Denote the set of all possible belief by A P(H)). Distribution of O₂ is denoted by pa- We only consider the case where no two hypotheses are the same in H. Hence we make the following assumption that columns of exp(-epC) are not differ by a multiplicative scalar, i.c. columns of C not differ by additive scalar.
Lemma 10. For (cp, 00, oo), E (0, oo), given cost C with initial belief 8o P(H) and fixed teaching and learning distribution Th = 1] E P(D) ( for all k, then the belief random variables (Ow)keR have the same expectation h: Ee, [0))] (h)..
Proof. We start the proof by showing Es, [0(h]]]]]]]]]]]] Ee, [0(h) for 1: 2 Notice that given cost C and data marginal an observed data d E D and UOT planning uniquely determines a map from learner's initial belief 0g-1 1 to one's posterior belief 0x- Denote this map by Ta 0z-1 + 0g. Let the distribution of Ox-1 over P(H) be f4-1,, denote its support by Sk-1. Then the following holds:
Mn(i,)
Be, [[(h)] = Σ
1k-1((0)
Σ (O)()))
Σ pk- 1k-1(()
Σ n
DE
AED
BES
d'ED
Σ
1k-1 (0) Σ Ma(i.j) Σ
Mk 1 (0))(()))))) Ee.
[0(h)
8ESk-1
dED
BESt-1
Hence (0(h) = (Be-,(0(h)) = _... Ee. [0()) = (b)).
Consider a learning problem with initial belief 0g E P(H), and the true hypothesis h" defined by n E P(D). , If the learner's data distribution тe 1) then belief random variables (Ox) converge to the random variable Y in probability, where Y = Σnen 0o(h)o, and Y is supported on (Ōr bee with P(Y = = (h( for =g oo and ep E (0)
Theorem 11
Proof. Step 1: First, show the following claim inspired the proof proposition 5.1 in Wang et al. (020a)
Claim: limg-roo | (A.) 0, for any where A. {0 A 0(h) < c, vh E H).
Assume the claim does not hold, then there exists a > 0 and a subsequence ( )ie such that
Mkx (A,) for all
Let the center of A be u, we define L(u) = (0), where f(0) = ||0- -ull2 (f may also be chosen entropy H (0)). Then ((p+11 Ey,, (Eddn (OT))))).
19

---

Under review as a conference paper at ICLR 2023
Notice that is strictly convex, by Jensen's inequality,
Ex (Ta(O)) 2 ((Ean)Ta(O)) ( 1())
(4)
(14)
Here (b) holds because:
E Ta(0)
Σ dDED
n (M)() = Σ Ma(i,)
(15)
D'ED
(c), (d) hold since Mr has marginals 0.
Moreover, equality holds in (a) if and only if (0) 0 for all d Thus rows of M, are the same up to a scalar. This implies either (1) only one column of M2 is none zero, thus O₂ òū for some h or (2) Mg has at least two columns are differed by scalar.
In the case of (1), if 8 & on. θ ò₁ is contradict to Lemma 10 Otherwise, Y = 5n the result holds. the case of (2), according to Wang al. (2019), Ms cross-ratio equivalent to exp(-epC)) hence exp( - pC) has two columns differ by multiplicative scalar, contradict to the assumption.
Thus for any E A., J
f(0). Therefore [(u=11)= [(ux) for any k.
Moreover, notice that A, is compact, there is a lower bound 3 > 0, such that Ea (Ta(()) f(0) for all Ac. Therefore:
[( EEEL,HEE, (Barn (Ba + (Ean, BoO,ES, (f(0)) + F., edis. +a*
(16)
= -[(Kk) +
Thus L(uR+++) > L(ux,) + s a + 00 as - co. On the other hand, by definition, f(0) is bounded above by the diameter of A under 1 norm, so L(μ) also bounded above. Contradiction! Therefore, the Claim holds.
Step We show lim P(O, Af-)) limg (x(Ai (h()), for all h H where A/_. = {0 A ((h) > 1 e).
For fixed h E H. we have:
(h) (8) Ee, (f(h)) (b) Eo,esp (B(h)) EBLEAT (B(h)) + EELEE.(((h))
(o)
- |n(A .1 (R(H-))) c c+ (x(A.)
uu ( +E+
where AT_, denotes the union of all the other corners of A, i.e. AV_ = UNNExN Here (a) direct application Lemmal 10 (b) holds since A A UA A UA. (c) holds because in general 0(h2) < 1, and ((h²) < for any 0 e A. Therefore, S 0o
S e+ + as I + 2o hold for any choice of c. Pick a sequence of e - 0, we have that lim Ma (A/_.) = (h).
Hence combining results from Step and Step we have shown O₂ converges to Y in probability: P([O, Y <) < (A.) + (0e (a() - 0 as k + co. Hence the proof is completed.
Corollary 12
Given fixed data sequence d; sampled from if 0g. converges to ōns, then the j-th column of Ms converges to n.
Proof. For > 0, there exists N > 0 such that 0g (x())) > 1 - for any & > N. So Σ Jj Me(i,j") < for any d, D, on the other hand Eg Me(i,j") 7/. This implies that Y): Ma(i,) < Th so Mi n O. Therefore the j-th column of Mg converges 17.
20

---

Under review as a conference paper at ICLR 2023
Consider a learning problem with cost C, initial belief 8g P(H). the true hypothesis h defined by 7) E P(D). If the learner updates the estimation Tis with observed data (sampled from 7) stated above, then belief random variables (Qx)ket satisfies that for any O, limg--oo EREn ((0(h) > - 8) = 1. As a consequence, Mg as the transport plan has a dominant column (h²) with total weights > 1 - 8, and ((Mx)i -
Proposition 13
mhe [x) < 8. In fact, as long as the sequence of T" as random variables converges to 17 in probability, the above proposition holds.
Proof. The proof is similar Step I of Theorem [ The major difference is that data are sampled from 7j in each step, whereas the learner only has an estimation 7) at round k. Therefore, under current condition, equality (b) of Eq 14 need to be modified following:
L = O Vk. TB
Ta(Ou))((
Σ n (M& d'EDD
Σ Me(i, d'ED
(17)
where Vk= = ( is a vector of the size of the data set D, and @ represents element- product.
(
Hence (Ta = [02 @ vk) holds for all 02 E A. Since 1/ - 1l as k + op. For any
a+3 > 0, there exists N > 0 such that for k > N, |1 z *
V
0+8 2n
Hence: -
a+3 Then corresponding Eq 16 for k; > N, we have:
2
1(pk++1)= = Be EA,
+ Eon. (Benn
> Ee,, Ea. (f(Op O vk)) + Bo,, EDIA, (f(Os 3 vk)) +
x*
> Eo,, EA, (f(Ok) + EAr,, (f(b)) x
+ +
2
= L(px) + 2
Hence the contradiction on the upper bound of L(ux+1)) still holds, which shows the claim that:
lim M4 (A.) = So lim Σnen P(O(h) > - 8) = 1. The proof for the second part of
the proposition follows exactly Corollary 12
O
Proposition 15
For c (cp,Cg. 0) with cp (0, oc). ті - 1) almost surely, the sequence
O of postertors as a sequence of random variables converges in probability to variable θ, where
P(O = v²) -n(i) and v = PP/.// (E Pis) and P p (C, n1 Therefore, for any > < s) = 0 for generic (for all but in closed subset) cost C and n, f.
limg-+o0 P(Ex
Proof. First, cg = means that P* P*(C, 17. 8) is independent of 0. Therefore, M₁ = P* Me. 0) and has a limit (C, 17, 0), regardless of the concrete posterior Oz. From construction of GBT, the posterior O₂ is determined by IP(O; = wi.) 711) where w (MR) (M))) Given the coupling (Oz, θ) by setting only IP(O; = w θ v²) = (7( for each i, we may calculate P([O; θ < 8) converge as Mj converge (C, n, 0).
For generic C, 17, 0, the probability of PC, 0) having row with only one nonzero entry is 0.
As 7) + 17 almost surely, for any > O, there exists N > O. such that, when k > N, the probability of having Mh e-close to is 1. Thus in almost all episodes, with generic C, 17. 8, when e is small enough, for any /// m/l < e (using p - oc norm, same for below), the row-normalized (to 1,,) UOT plans
Remark:
max || PS(C, (') - P; T/. 0 < min (C, ()6 PF 0-)ll b
where PE is the row normalization of Pe.
21

---

Under review as a conference paper at ICLR 2023
Therefore, for such e, we may find N 0 such that for any k, k' (C, The 0). However, for generic 7), say, no entry of 7, is O. ||0, &'ll < when k, > N and
N, (C 7k. 0) *
dg o dg. Thus the posterior sequence of almost every episode fails to converge.
C ADDITIONAL SIMULATIONS
Interpolation between learning models can be investigated properly under GBT. Human learners appear to be capable of moving between different learning models gradually. Consider an individual carnival who is playing a game. At each of 10 trials, bit of information provided, but the available reward decreases. The individual has a pool of tickets with which they can bet on the outcome at each trial. The question how the individual should update their beliefs in order to maximize their rewards. On the first trial, their belief update, in order to accurately reflect the evidence, should follow Bayes rule. However, for the last trial, one should focus bets on the most probable outcome in order to maximize chances for rewards, that is, their beliefs should be optimized for discriminating among the possible outcomes. GBT offers coherent way of interpolating between these two approaches to provide candidate strategies are common where there is an explicit constraint on the time horizon after which point no further evidence can be obtained, and there are incentives to act early, rather than to wait until evidence has fully accumulated; for example, identifying dangerous situations (tiger or not? poisonous or not?).
the intermediate steps. Such situations
We now demonstrate how continuity of GBT (section 3.1) allows one to gradually interpolate between Bayesian and discriminative learning over steps (rather than a sharp switch).
C.I SIMULATION SETUP
Suppose a learner who observes data sampled from a true hypothesis P(dlh",, and needs to make a conclusion on whether ht is one of the hypotheses in H within a fixed number N of observations.
Here we compare a baseline learner who utilizes Bayesian inference (e (1, O, oo)) on the first N - 1 observations, and switch to discriminative learning (c = (0, co, oo)) on the last observation, against learners who interpolate from Bayesian to discriminative learning gradually along a sequence of models curves in GBT. Two curves along with intermediate models are shown red and orange in Figure [
We take random sampled M of shape 4 x 4 as example,
0.014886 0.225779
0.433787
0.050735
0.613779
0.322347
0.172658
0.109262
M
0.069799
0.620178
0.29083
0.243635
0.090643
0.042588
0.102725
0.596368
Thus |7[| = [D] = 4. Set N 10 and start from uniform (0.25, 0.25, 0.25, 0.25).
Simulation details: We perform 40000 trials in total. For each trial (or say each episode), we uniformly sample X, E P(H), and let the true hypothesis h be the covex combination of elements in H. with coefficients given by X,. While teaching the episode, in each round, we sample hypothesis E H. following X,, then sample a data d following the column of M corresponding to d. During inference, we 7/ by counting the frequency of each d D (starting from 1 to avoid ( in m)) and then normalize, as stated in (RS) model Sec.
C.2 RESULTS
Following paths shown in Fig. 5 for baseline (blue, left), path 1 (orange, middle), and path 2 (red, right), the distribution of maximal component of each posterior at round 10 are shown in histograms of 30, and the entropy of these posteriors are plotted in the lower three figures.
In the upper figures, comparing to the baseline (blue), weights are concentrated more on the right bars for the gradual interpolations (orange and red). Thus learning tends to be more conclusive along these paths. Here conclusiveness means that the ability of getting conclusion (one component of the posterior eventually becoming dominant). Furthermore, the entropy distributions shown in the lower figures also illustrate this point, as compare baseline, gradual interpolations have lower entropy.
22

---

Under review as a conference paper at ICLR 2023
Bayeslan
Path
siggoid(loglag))
Pakn
signoid(logies,)
Discriminative
Figure 5: Baseline (sharp change) and two paths we follow on the parameter space of GBT.
Numerical results: entropy of baseline: mean 0.1888, standard deviation 0.2858, entropy along path 1: mean 0.0097, standard deviation 0.0686 entropy along path 2: mean 0.0571, standard deviation 1584.
It is necessary to consider that, the two paths and interpolations are chosen for demonstration purpose, by no means they are optimal. However, we believe GBT capable of facilitating exploration of such optimization.
10
10a
10c
10a
10a
10a
10a
10a
102
10
10
10
10a
10a
10a
0.4
1.0
10
10
10a
10a
10a
10a
10
10
10
10a
10a
10a
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50 0.75
1.00
0.00
0.25
0.50
0.75
1.00
Figure 6: Results. Upper: distribution of maximal component of posterior. Lower: Entropy distribution of posteriors. Left: baseline. Middle: along path 1. Right, along path 2.
23


Reviews:
[{"review_id": "t5IVS2bUW8", "reviewer": "Reviewer_pJWT", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a unified theoretical framework, called generalized belief transport (GBT), that generalizes subclasses of machine learning algorithms including Bayesian inference, cooperative communication, and classification. The authors visualize the parameter space of learning models encoded by GBT as a cube, with the aforementioned subclasses of classic learning algorithms as particular points. Finally, the authors investigate online learning in GBT framework, where the learner’s marginal on the data is not fixed a priori, but accumulates evidence based on experience. Asymptotic convergence properties are provided.", "summary_of_the_review": "This paper proposes a unified theoretical framework, called generalized belief transport (GBT), that generalizes subclasses of machine learning algorithms including Bayesian inference, cooperative communication, and classification. This allows one to visualize the parameter space of learning models encoded by GBT as a cube, with the subclasses of classic learning algorithms as special points. It would be appreciated if the authors could demonstrate how the GBT framework leads to new algorithms that improve over current baselines. However, I find this theoretical framework novel and could have an impact across fields of machine learning.", "strength_and_weaknesses": "This paper provides a unified theoretical framework that generalizes several classic machine learning algorithms. This framework, called GBT, provides a novel perspective of these classic learning algorithms using the theory of optimal transport. The cubic view of the parameter space of GBT is intuitive, and how classic algorithms lie on this cubic space as particular points is well explained.\n\nOverall, I found this new framework intriguing. However, it is unclear how existing methods could benefit from such an alternative perspective. Does GBT provide improvement in the classic learning settings? Does GBT permit us to learn in settings that were challenging before? It would be appreciated if the authors could provide examples to show how this novel framework permits us to improve over current baselines.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "qc9okOnr3U", "reviewer": "Reviewer_TGqt", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors proposed a framework that can align a set of hypotheses models with a set of datasets. Specifically, the hypotheses and datasets are viewed as empirical distributions and certain loss functions can serve as the transport cost of moving one hypothesis to a dataset. Thus, this alignment can be reviewed as an unbalanced optimal transport problem. In the proposed formulation, the coefficients that control the KL for regularizing the marginals have a new interpretation. The authors showed that the whole learning framework is interpolating between several learning paradigms, including Bayesian inference, frequentist estimation, cooperative communication and discriminative learning. The authors also provided a detailed theoretical analysis to justify the consistency properties.", "summary_of_the_review": "From my perspective, this draft is well presented and the proposed idea shares a new perspective on connecting several classical ML perspectives. Thus, I lean toward the acceptance of this work. ", "strength_and_weaknesses": "Pro:\n- First of all, this paper is well-written. The relationship between different learning perspectives is introduced comprehensively. \n- Overall the idea of formulating such a learning problem as unbalanced optimal transport is quite novel based on some existing works connecting OT with cooperative communication.\n- The authors introduced their theoretical results quite rigorously. \n\nCons and questions:\n- In the algorithm 1, where is the $\\eta$ updated? What is the data sampler $\\tau$? In addition, in each iteration there will be new data samples obtained from the sampler, will that lead to a different cost matrix $C$ in each iteration?  \n- Maybe one limitation of this work is that, I can see its potential in unifying several ML perspectives in an elegant way however it could be great if the authors could specify some applications of the belief. It seems to me that this framework maybe is related to autoML or model selection and the authors might consider some tasks on real-world datasets.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "QRcA9ILsZK", "reviewer": "Reviewer_wkW3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": " This paper introduces a new unified formalism where Bayesian\n inference, Discriminate learning, Cooperative learning, and Maximum\n Likelihood Estimation are all expressible using a single algorithm\n inspired by the Sinkhorn algorithm from the optimal transport literature.\n\n The algorithm's generality is expressed with a cube where one could in\n principle express not just any of the above algorithms but linear combinations\n of them.", "summary_of_the_review": " This is an original idea that would greater benefit from additional clarity and better\n exposition of the value of the presented formalism.\n", "strength_and_weaknesses": " The formalism introduced in the paper is novel, but I'm not particularly sure\n that it is all that useful. While I can see how exploring a Bayesian inference where\n the prior is made progressively more or less informative has some value, I don't\n see what benefits come from some of the other mixtures.\n\n I found the paper very difficult to follow. Large amounts of time is spent on\n introducing the work with fairly basic ideas being defined that likely reader\n would already be familiar with. I also am unsure some of the terminology is being\n used precisely enough. I don't know what it means to do Frequentist inference. From\n context it seems to be maximum likelihood estimation. Bayesian inference seems\n to be restricted to discrete space of parameters. None of these limitations\n are explicitly mentioned, I just don't see how the algorithm presented could\n generalize otherwise. It would be very helpful if the authors clarified precisely\n which methods their formalism can represent instead of just saying \"Bayesian inference\"\n and \"Discriminate Learning\".\n\nThe author's response did help clarify some of what these interpolations could mean, but I\nremain unsure all pairs of interpolations within the cube yields an explainable learning algorithm. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "LkLrORxkD6F", "reviewer": "Reviewer_KwJp", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper aims at unifying Bayesian inference, cooperative communication, and discriminative learning in a general learning framework, called Generalized Belief Transport (GBT). The cornerstone of this unifying framework lies in the paradigm of entropy-regularized unbalanced optimal transport defined using three scaling parameters, from which classical learning models can be derived using special points. The paper examines both the batch (Section 3) and the sequential (Section 4) versions of GBT, where the latter opens the door to several questions related to the convergence of online GBT for several scaling points.\n", "summary_of_the_review": "Arguably, unifying Bayesian inference, communication inference, and discriminative learning is very challenging.\nThis paper aims at paving the way in this direction but, as indicated above, it is quite difficult to assess the significance of the results, \ndue to clarity issues and the lack of a detailed related work section.", "strength_and_weaknesses": "Overall, this general framework is conveying some interesting results for the study of learning agents, but it is quite difficult to assess their significance due to clarity issues and the absence of a detailed related work section (see comment below). \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Decision: Reject