Title: Generalized Belief Transport
Venue: NeurIPS 2023
Accepted: yes
Keywords: Bayesian theory

Belief transport

Unbalanced optimal transport

parametrization

asymptotic behavior

environment drift detection
Abstract: Human learners have ability to adopt appropriate learning approaches depending on constraints such as prior on the hypothesis, urgency of decision, and drift of the environment. However, existing learning models are typically considered individually rather than in relation to one and other. To build agents that have the ability to move between different modes of learning over time, it is important to understand how learning models are related as points in a broader space of possibilities. We introduce a mathematical framework, Generalized Belief Transport (GBT), that unifies and generalizes prior models, including Bayesian inference, cooperative communication and classification, as parameterizations of three learning constraints within Unbalanced Optimal Transport (UOT). We visualize the space of learning models encoded by GBT as a cube which includes classic learning models as special points. We derive critical properties of this parameterized space including proving continuity and differentiability which is the basis for model interpolation, and study limiting behavior of the parameters, which allows attaching learning models on the boundaries. Moreover, we investigate the long-run behavior of GBT, explore convergence properties of models in GBT mathematical and computationally, document the ability to learn in the presence of distribution drift, and formulate conjectures about general behavior. We conclude with open questions and implications for more unified models of learning.

Generalized Belief Transport

Abstract

Human learners have ability to adopt appropriate learning approaches depending on constraints such as prior on the hypothesis, urgency of decision, and drift of the environment. However, existing learning models are typically considered individually rather than in relation to one and other. To build agents that have the ability to move between different modes of learning over time, it is important to understand how learning models are related as points in a broader space of possibilities. We introduce a mathematical framework, Generalized Belief Transport (GBT), that unifies and generalizes prior models, including Bayesian inference, cooperative communication and classification, as parameterizations of three learning constraints within Unbalanced Optimal Transport (UOT). We visualize the space of learning models encoded by GBT as a cube which includes classic learning models as special points. We derive critical properties of this parameterized space including proving continuity and differentiability which is the basis for model interpolation, and study limiting behavior of the parameters, which allows attaching learning models on the boundaries. Moreover, we investigate the long-run behavior of GBT, explore convergence properties of models in GBT mathematical and computationally, document the ability to learn in the presence of distribution drift, and formulate conjectures about general behavior. We conclude with open questions and implications for more unified models of learning.

Learning and inference are subject to internal and external constraints. Internal constraints include the availability of relevant prior knowledge. External constraints include the availability of time to accumulate evidence versus the need make the best decision now or environmental non-stationarity. Standard models of machine learning tend to view different constraints as different problems, which impedes development of unified learning agents.

These internal and external constraints map onto classic dichotomies in machine learning. Availability of prior knowledge maps onto the Frequentist-Bayesian dichotomy in which the latter uses prior knowledge as a constraint on posterior beliefs, while the former does not. Within Bayesian theory, a classic debate pertains to uninformative, or minimally informative, settings of priors [Jeffreys, 1946, Robert et al., 2009]. Availability of time to accumulate evidence informs the use of generative versus discriminative approaches [Ng and Jordan, 2001], and static or drift/dynamic models [Dagum et al., 1992, Murphy, 2002]. Combining constraints on probability of beliefs and costs of data models cooperative communication [Wang et al., 2020b].

Learning agents must interpolate between modes of reasoning as necessary given the constraints of the moment. Imagine observing an agent behaving in an environment. As an observer, one may wish to learn about the environment from the agent's actions. However, any inferences depend on one's model of the agent and their constraints. How is the agent updating their beliefs? Do they have stable goals, or are they changing over time? Perhaps the agent is selecting actions to communicate what they know? In order to draw inferences over these possibilities, one must parameterize the space,

ideally in such a way one could optimize over the possibilities. Indeed, in order to implement these possibilities, the agent themself must parameterize the space in order to interpolate between classic dichotomies such as Bayesian and frequentist, static and dynamic environments, and helpful versus neutral agent, given constraints.

We introduce Generalized Belief Transport (GBT), based on Unbalanced Optimal Transport (Sec. 1), which paramterizes and interpolates between known reasoning modes (Sec. 2.2), with four major contributions. First, we prove continuity in the parameterization and differentiability on the interior of the parameter space (Sec. 2.1). Second, we analyze the behavior under variations in the parameter space (Sec. 2.3). Third, we study sequential learning, where learners may (not) track the empirically observed data frequencies in (Sec. 3). Fourth, we investigate predictive performance under environmental drift (Sec. 4).

Notations. R ≥ 0 denotes the non-negative reals. Vector 1 = (1 , . . . , 1) . The i -th component of vector v is v ( i ) . P ( A ) is the set of probability distributions over A . For a matrix M , M ij represents its ( i, j ) -th entry, M ( i, _ ) denotes its i -th row, and M ( _ ,j ) denotes its j -th column. Probability is P ( · ) .

1 Learning as a problem of unbalanced optimal transport

Consider a general learning setting: an agent, which we call a learner , updates their belief about the world based on observed data subject to constraints. There is a finite set D = { d 1 , . . . , d n } of all possible data, that defines the interface between the learner and the world. The world is defined by a true hypothesis h ∗ , whose meaning is captured by a probability mapping P ( d | h ∗ ) onto observable data. For instance, the world can either be the environment in classic Bayesian inference [Murphy, 2012] or a teacher in cooperative communication [Wang et al., 2020b].

A learner is equipped with a set of hypotheses H = { h 1 , . . . , h m } which may NOT contain h ∗ ; an initial belief on the hypotheses set, denoted by θ 0 ∈ P ( H ) ; and a non-negative cost matrix C = ( C ij ) m × n , where C ij measures the underlying cost of mapping d i into h j 1 . The cost matrix can be derived from other matrices that record the relation between D and H , such as likelihood matrices in classic Bayesian inference or consistency matrices in cooperative communication (see details in Section 2.2).This setting reflects an agent's learning constraints: pre-selected hypotheses, and the relations between them and the communication interface (data set).

A learner observes data in sequence. At round k , the learner observes a data d k that is sampled from D by the world according to P ( d | h ∗ ) . Then the learner updates their beliefs over H from θ k -1 to θ k through a learning scheme , where θ k -1 , θ k ∈ P ( H ) . For instance, in Bayesian inference, the learning scheme is defined by Bayes rule; while in discriminative models, the learning scheme is prescribed by a code book.

The learner transforms the observed data into a belief on hypotheses h ∈ H with a minimal cost, subject to appropriate constraints, with the goal of learning the exact map P ( d | h ∗ ) . We can naturally cast this learning problem as Unbalanced Optimal Transport.

1.1 Unbalanced Optimal Transport

Unbalanced Optimal Transport (UOT), introduced by Liero et al. [2018], is a generalization of (entropic) Optimal Transport [Villani, 2008, Cuturi, 2013, Peyré and Cuturi, 2019], that relaxes the marginal constraints. Formally, for non-negative scalar parameters ϵ = ( ϵ P , ϵ η , ϵ θ ) , the UOT plan is,

Here, ⟨ C, P ⟩ = ∑ i,j C ij P ij is the inner product between C and P , H ( P ) = -∑ ij P ij (log P ij -1) is the entropy of P , and KL ( a | b ) := ∑ i ( a i log( a i /b i ) -a i + b i ) is the Kullback-Leibler divergence between vectors. It is shown in Chizat et al. [2018] that UOT plans can be solved efficiently via Algorithm 1 : Given a cost C , P ϵ can be obtained by applying ( η, θ, ϵ ) -unbalanced Sinkhorn scaling on K ϵ := e -1 ϵ P C = ( e -1 ϵ P C ij ) m × n , with convergence rate ˜ O ( mn ϵ P ) [Pham et al., 2020].

Proposition 1. The UOT problem with cost matrix C , marginals θ, η and parameters ϵ = ( ϵ P , ϵ η , ϵ θ ) generates the same UOT plan as the UOT problem with tC , θ , η , tϵ = ( tϵ P , tϵ η , tϵ θ ) for any t ∈ (0 , ∞ ) . Therefore, the analysis on ϵ and tϵ are the same for general cost C .

Thus a positive common factor on C, ϵ P , ϵ η , ϵ θ does not affect the solution of Eq. (1). Therefore, for the later analysis, we fix ϵ P = 1 unless otherwise stated.

Framework: Generalized Belief Transport (GBT). Learning, efficiently transport one's belief with constraints, is naturally a UOT problem, i.e. a Generalized Belief Transport . Each round, a learner, defined by a choice of ϵ = ( ϵ P , ϵ η , ϵ θ ) , updates their beliefs as follows. Let η k -1 , θ k -1 be the learner's estimations of the data distribution and the belief over hypotheses H after round k -1 , respectively. At round k , the learner first improves their estimation of the mapping between D and H , denoted by M k , through solving the UOT plan Eq. (1) with ( C, η k -1 , θ k -1 ) , i.e. M k = P ϵ ( C, η k -1 , θ k -1 ) . Then with data observation d k , the learner updates their beliefs over H using corresponding row of M k , i.e. suppose d k = d i for some d i ∈ D , the learner's belief θ k is defined to be the row normalization of the i -th row of M k . Finally, the learner updates their data distribution to η k by increment of the i -th element of η k -1 , see Algorithm 2.

Algorithm 1 Unbalanced Sinkhorn Scaling

2 Generalized Belief Transport

Many learning models-including Bayesian inference, Frequentist inference, Cooperative learning, and Discriminative learning-are unified under our GBT framework under choice of ϵ . In this section, we focus on the single-round behavior of the GBT model, i.e., given a pair of marginals ( θ, η ) , how different learners update beliefs with a single data observation. We first visualize the entire learner set as a cube (in terms of parameters), see Figure 1. Then, we study the topological properties of the learner set through continuous deformations of parameters ϵ . In particular, we show that existing models including Bayesian inference, cooperative inference and discriminative learning are learners with parameters (1 , 0 , ∞ ) , (1 , ∞ , ∞ ) and (0 , ∞ , ∞ ) respectively in our UOT framework.

2.1 The parameter space of GBT model

The space of learners in GBT are parameterized by three regularizers for the underlying UOT problem (1): ϵ P , ϵ η and ϵ θ , each ranges in [0 , ∞ ) . Therefore, the constraint space for GBT is R 3 ≥ 0 , with the standard topology. When C , θ and η are fixed (assume η ∈ R m > 0 ), the map ϵ = ( ϵ P , ϵ η , ϵ θ ) ↦→ ( P ϵ ) bears continuous properties:

Proposition 2. 2 The UOT plan P in Equation (1) , as a function of ϵ , is continuous in (0 , ∞ ) × [0 , ∞ ) 2 . Furthermore, P is differentiable with respect to ϵ in the interior of its domain R 3 ≥ 0 .

Continuity on ϵ provides the basis for interpolation between different learning agents. The proof of Proposition 2 also implies the continuity on η and θ . Further, towards the boundaries of the parameter space (where theories like Bayesian, Cooperative Communication live in), we show:

Algorithm 2 Generalized Belief Transport

Proposition 3. Let s P , s η , s θ ≥ 0 be arbitrary finite numbers, the following holds:

(1) The limit of P ϵ exists as ϵ approaches ( ∞ , s η , s θ ) . In fact, lim ϵ → ( ∞ ,s η ,s θ ) P ϵ ij = 1 for all i, j .

(2) As ϵ → ( s P , ∞ , s θ ) , P ϵ converges to the solution to

(4) And when ϵ → ( s P , ∞ , ∞ ) , the matrix P ϵ converges to the EOT solution:

(5) When ϵ → ( ∞ , ∞ , s θ ) , ( ∞ , s η , ∞ ) or ( ∞ , ∞ , ∞ ) , the limit does not exist, but the directional limits can be calculated.

n

sigmqid(log(Ep))

M=exp（-C)

（∞,∞,∞）:7θ

（∞，0,∞）:

Tyrant

Tyrant

(∞,o,O):Frequentist

(1,0,∞）):

Bayesian

nθ

Bayesian

RowGreedy

(0,∞,0):

CC

ColumnGreedy

(0,0,∞）):

LG

sigmoid(log(ε))

（0，00，00）：

RG

Discriminative

sigmoid(log(εn))

CooperativeCommunication

(1,∞0，∞0）：

The parameter space for GBT with its boundaries can be visualized in Fig. 1. Proposition 3 implies that the parameter space is S = [0 , ∞ ] 3 \ ( { ( ∞ , ∞ , x ) : x ∈ [0 , ∞ ] } ∪ { ( ∞ , x, ∞ ) : x ∈ [0 , ∞ ] } ) . In Fig. 1, segment [0 , ∞ ) is mapped to [0 , 1) by sigmoid (log( x )) . Then boundaries are added to the image cube [0 , 1) 3 . The dashed lines on top of the cube indicates limits that do not exist.

2.2 Special points in the parameter space

Bayesian Inference. Given observed data, a Bayesian learner (BI) [Murphy, 2012] derives posterior belief P ( h | d ) based on prior belief P ( h ) and likelihood matrix P ( d | h ) , according to the Bayes rule. Intuitively, due to soft time constraint ( ϵ P = 1 ), a Bayesian learner is a generative agent who puts a hard internal constraint on their prior belief ( ϵ θ = ∞ ), and omits the estimated data distribution η in the learning process, ( ϵ η = 0 ).

Corollary 4. Consider a UOT problem with cost C = -log P ( d | h ) , marginals θ = P ( h ) , η ∈ P ( D ) . The optimal UOT plan P (1 ,ϵ η ,ϵ θ ) converges to the posterior P ( h | d ) as ϵ η → 0 and ϵ θ →∞ . Thus, Bayesian inference is a special case of GBT with ϵ = (1 , 0 , ∞ ) .

Frequentist Inference. A frequentist updates their belief from data observations by increasing the corresponding frequencies of datum. Intuitively, a frequentist is an agent who puts a hard constraint on the data distribution η ( ϵ η = ∞ ), and omits prior knowledge θ ( ϵ θ = 0 ) in a learning process without time constraint ( ϵ P = ∞ ). Formally we show:

Corollary 5. Consider a UOT problem with θ ∈ P ( H ) , η = P ( d ) . The optimal UOT plan P ( ϵ P , ∞ , 0) converges to η ⊗ 1 as ϵ P →∞ . Frequentist Inference is a special case of GBT with ϵ = ( ∞ , ∞ , 0) .

Cooperative Communication. Two cooperative agents, a teacher and a learner, are considered in Yang et al. [2018], Wang et al. [2020b], Shafto et al. [2021]. Cooperative learners (CI) draw inferences about hypotheses based on which data would be most effective for the teacher to choose Given a data observation, a cooperative learner derives an optimal plan L = P ( H , D ) based on a prior belief P ( h ) , a shared data distribution P ( d ) and a matrix M specifies the consistency between data and hypotheses (such as M ij records the co-occurrence of d i and h j ). Intuitively, a cooperative learner is also a generative agent who puts hard constraints on both data and hypotheses ( ϵ η = ∞ , ϵ θ = ∞ ), and aims to align with the true belief asymptotically, ( ϵ P = 1 ). Thus we show:

Corollary 6. Let cost C = -log M , marginals θ = P ( h ) and η = P ( d ) . The optimal UOT plan P (1 ,ϵ η ,ϵ θ ) converges to the optimal plan L as ϵ η → ∞ and ϵ θ → ∞ . Cooperative Inference is a special case of GBT with ϵ = (1 , ∞ , ∞ ) , which is exactly entropic Optimal Transport [Cuturi, 2013].

Discriminative learning. A discriminative learner decodes an uncertain, possibly noise corrupted, encoded message, which is a natural bridge to information theory [Cover, 1999, Wang et al., 2020b]. Adiscriminative learner builds an optimal map to hypotheses H conditioned on observed data D . The map is perfect when, for all messages, encodings are uniquely and correctly decoded. Intuitively, a discriminative learner aims to quickly build a deterministic code book (implies ϵ P = 0 ) that matches the marginals on H and D . We show that discriminative learner is GBT with ϵ = (0 , ∞ , ∞ ) :

Corollary 7. Consider a UOT problem with cost C = -log P ( d, h ) , m = n , and marginals θ = η are uniform. The optimal UOT plan P ( ϵ P ,ϵ η ,ϵ θ ) approaches to a diagonal matrix as ϵ η , ϵ θ →∞ and ϵ P → 0 . In particular, discriminative learner is a special case of GBT with ϵ = (0 , ∞ , ∞ ) , which is exactly classical Optimal Transport [Villani, 2008].

Many other interesting models are unified under GBT framework as well. GBT with ϵ = (0 , ∞ , 0) denotes Row Greedy learner which is widely used in Reinforcement learning community [Sutton and Barto, 2018]; ϵ = ( ∞ , ∞ , ∞ ) yields η ⊗ θ which is independent coupling used in χ 2 [Fienberg et al., 1970]; ϵ = ( ϵ P , ϵ θ , ∞ ) is used for adaptive color transfer studied in [Rabin et al., 2014]; and ϵ = (0 , ϵ θ , ϵ η ) is UOT without entropy regularizer developed in [Chapel et al., 2021]. Other points in the GBT parameter space are also of likely interest, past or future.

2.3 General properties on the transportation plans

The general GBT framework builds a connection between the above theories, and the behavior of theory varies according to the change of parameters. In particular, each factor of ϵ = ( ϵ P , ϵ η , ϵ θ ) expresses different constraints of the learner. Given ( C, θ, η ) as shown in the top-left corner of Fig. 1, we plot each learner's UOT plan with darker color representing larger elements.

ϵ P controls a learner's learning horizon. When ϵ P → 0 , a learner's UOT plan is concentrated on a clear leading diagonal which allows them to make fast decisions. This corresponding to agents who are under the time pressure of making immediate decision, i.e. discriminative learner, or row greedy learner on the bottom of the cube (Fig. 1). Most of the time, one datum is enough to identify the true hypothesis and convergence is achieved within every data observation. When ϵ P →∞ , GBT converges to a reticent learner, such as learners on the top of the cube. Data do not constrain the true hypothesis, and learners draw their conclusions independent of the data. In between, GBT provides a generative (probabilistic) learner. When ϵ P = 1 , we have Bayesian learner and Cooperative learner, for whom data accumulate to identify the true hypothesis in a manner broadly consistent with probabilistic inference, and consistency is asymptotic.

ϵ η controls a learner's knowledge on the data distribution η . When ϵ η → ∞ , GBT converges to a learner who is aware of the data distribution and reasons about the observed data according to the probabilities/costs of possible outcomes. Examples include the Discriminative and Cooperative learners on the front of the cube. When ϵ η → 0 , GBT converges to a learner who updates their belief without taking η into consideration, such as Bayesian learners on the back of the cube, and the Tyrant who does not care about data nor cost and is impossible to be changed by anybody.

ϵ θ controls the strength of the learner's prior knowledge. When ϵ θ → 0 , GBT converges to learners who utilizes no prior knowledge. Hence, they do NOT maintain beliefs over H , and draws their conclusions purely on the data distribution, such as a Frequentist learner on the left of the cube. When ϵ θ →∞ , GBT converges to a learner who enforces a strict prior such as Bayesian, Cooperative and Discriminative learners on the right of the cube. In particular, we show that:

Proposition 8. In GBT with ϵ θ = ∞ , cost C and current belief θ . The learner updates θ with UOT plan in the same way as applying Bayes rule with likelihood from P ϵ ( C, η, θ ) , and prior θ .

3 Sequential GBT - Static

The sequential GBT captures the asymptotic behavior of a learning problem ( C, θ 0 , h ∗ ) . Static world where there exists a fixed true hypotheses h ∗ is considered in this section. Data is sampled from

η = P ( d | h ∗ ) (not necessarily related to some h ∈ H ). Then the learner follows GBT with cost C , and parameter ϵ , starts with a prior θ 0 , then in each round k , applies GBT with η k -1 and θ k -1 to generate θ k .

We investigate two cases. In the Preliminary sequential model ( PS ), we assume η k = η for all k . In practice, often a learner does not have access η . Instead, in each round the learner may choose to use the current observed data distribution η k ( d ) as an estimation of η , Thus we study the Real sequential model ( RS ) where η k a.s. - - → η .

In statistics, a model is said to be consistent when, for every fixed hypothesis h ∈ H , the model's belief θ over the hypotheses set H converges to δ h in probability as more data are sampled from η = P ( d | h ) . Such consistency has been well studied for Bayesian Inference since Bernstein and von Mises and Doob [Doob, 1949], and recently demonstrated for Cooperative Communication [Wang et al., 2020a]. However, the challenge arises when one tries to learn a h ∗ that is not contained in the pre-selected hypothesis space H . It is not clear which h ∈ H is the 'correct' target to converge to.

In this section, we demonstrate GBT's ability of learning new hypothesis. Analogize to consistency, the properties are stated directly in the language of posterior sequence (Θ k ) ∞ k =1 as random variables, focusing on whether the sequence converges (and in which sense), and how conclusive (how likely to a stable new hypothesis is learned) the sequence is.

For a learning problem ( C, θ 0 , h ∗ ) , results in this section are organized based on different ϵ θ values.

Conclusive and Bayesian-style: ϵ θ = ∞ . These learners are located on the right side of Cube Fig. 1. Many well-studied learners are in this class: Bayesian, Cooperative, Discriminative, Row Greedy etc. According to Prop 8, learners in this class perform 'Bayesian' style learning.

When ϵ η = 0 , i.e. the learners who update their belief without considering data distribution, ( PS ) and ( RS ) are essentially the same. The following holds:

Theorem 9 ([Doob, 1949],[Wang et al., 2020a]) . In GBT sequential model (both ( PS ) and ( RS )) with ϵ = ( ϵ P , 0 , ∞ ) where ϵ P ∈ (0 , ∞ ) , the sequence Θ k converges to some δ h almost surely, h is the closest column of e -C/ϵ P to η in the sense of KL-divergence.

When ϵ η = ∞ , the models ( PS ) and ( RS ) present slightly different behaviors.

Theorem 10 ( PS ) . When ϵ η = ϵ θ = ∞ , for the PS problem belief random variables of a GBT learner (Θ k ) k ∈ N converge to the random variable Y in probability, where Y = ∑ h ∈H θ 0 ( h ) δ h and Y is supported on { δ h } h ∈H with P ( Y = δ h ) = θ 0 ( h ) for ϵ η = ϵ θ = ∞ and ϵ P ∈ (0 , ∞ ) .

Corollary 11. Given a fixed data sequence d i sampled from η , if θ k converges to δ h j , then the j -th column of M k converges to η .

Thus a GBT learner, with access to the data distribution and using strict marginal constraints, converges to the true hypothesis mapping η with probability 1 . Moreover, the probability of which h ∈ H is shaped into η is determined by their prior θ 0 . That is, GBT learners converge to the truth by reforming one of their original hypotheses into the true hypothesis.

Proposition 12. When ϵ η = ϵ θ = ∞ , for the ( RS ) problem, the belief random variables of a GBT learner (Θ k ) k ∈ N satisfies that for any s > 0 , lim k →∞ ∑ h ∈H P (Θ( h ) > 1 -s ) = 1 . As a consequence, M k as the transport plan has a dominant column ( h j ) with total weights > 1 -s , and | ( M k ) ij -η k ( i ) | < s .

In fact, as long as the sequence of η k as random variables converges to η in probability, the above proposition holds. The limit lim k →∞ ∑ h ∈H P (Θ( h ) > 1 -s ) measures how conclusive the model is.

In contrast with standard Bayesian or other inductive learners, Proposition 12 shows that a GBT learner is able to learn any hypothesis mapping η = P ( d | h ∗ ) up to a given threshold s with probability 1 . In addition to unifying disparate models of learning, GBT enables a fundamentally more powerful approach to learning by empirically monitoring the data marginal.

Fig. 2 illustrates convergence over learning problems and episodes. In each bar, we sample 100 learning problems ( C, θ 0 , h ∗ ) from Dirichlet distribution with hyperparameters the vector 1 . Then we sample 1000 data sequences (episodes) of maximal length N = 10000 . The learner learns with Algo. 2 where the stopping condition ω is set to be max h ∈H θ ( h ) > 1 -s with s = 0 . 001 . The

1.0

1.0 -

1.0

8'0

8°0

0.8

0.6

F9°0

3000

2 15 10 5

0.6

1000

0.4

0.2

1000

100

10000

15

10

30

0.4 -

100

50

25

0.4

25 20

100

50

15

10

5

50

0

0.2

0.0

_uot

ot

0.0

0.0

_exact_uot

aepdnotxot

10x10_exact_uot

10x10_

5x3_

jon"ajepdn"exs

5x10_ot

10x10_ot

100

1001X05

100x10_ot

10x10_ot

25x25_ot

50x50_ot

100x100_ot

y -axis in the plots represents the percentage of total episode converged. The color indicates in how many rounds the episode converges. For instance, in the bar corresponding to ' 10 × 10 _update_uot', with 10 data points (yellow portion), about 50% episodes satisfy the stopping condition.

The first plot shows results for 10 × 10 and 5 × 3 matrices. The second plot shows results for rectangular matrices of dimension m × 10 with m ranges in [5 , 10 , 25 , 50 , 100] . The third plot shows results for square matrices of dimension m × m with m ranges in [10 , 25 , 50 , 100] . Here 'exact' and 'update' indicate the problem is ( PS ) or ( RS ), respectively. For parameters, uot represents the parameter choice ( ϵ P = 1 , ϵ θ = ϵ η = 40) vs. ot represents the parameter choice ( ϵ P = 1 , ϵ θ = ϵ η = ∞ ) .

The first plots demonstrates that learners that do not have access to the true hypothesis (empirically builds estimation of η ) learn faster than learners who have full access. The second plot indicates with a fixed number of hypotheses, learning is faster when the dimension of D increases. The third plot shows that the GBT learner scales well with the dimension of the problem.

1.0

0.8

distance

0123

4

234

01

234

01234

0 1 2 3 4

0.6

Bar of θ

E：

B1

0.02

0.2

015

1.0

2.0

5.0

50.0

0.4

0.2

IIv-63l2

IIv-0ll2

0.0

01234

01234

01234

01234

01234

B2

2

5

0

o0

5

Then we study the learners that interpolate between Bayesian and Cooperative learners (located on the line connecting CI and BI in Fig 1). Consider a fixed learning problem ( C, θ 0 , h ∗ ) . Consistency of Bayesian inference states that asymptotically, the learner Bayesian converges to a particular hypothesis h b ∈ H almost surely where h b is the hypothesis closest to h ∗ under KL divergence. Theorem 10 indicates that a GBT cooperative learner modifies one of the hypotheses into h ∗ in probability 1 . The probability of h j converges to h ∗ is determined by θ 0 ( h j ) .

In Fig. 3, we study the asymptotic behavior of the learners corresponding to ϵ = (1 , ϵ η , ∞ ) , with ϵ η ∈ { 0 , 0 . 02 , 0 . 2 , 0 . 5 , 1 , 2 , 5 , 50 , ∞} . We sample a learning problem with a dimension 5 × 5 from Dirichlet distribution with hyperparameters the vector 1 . Each learner ϵ = (1 , ϵ η , ∞ ) is equipped with a fixed C , θ 0 and η k = η for all k . We run 400 , 000 learning episodes per learner, and plot their convergence summary in the bar graph. A continuous transition from a Bayesian learner to a cooperative learner can be empirically observed: the coefficients a ( h ) of the limit in terms of ∑ h ∈H a ( h ) δ h changes from δ ( h 3 ) by Bernstein-von Mises to θ 0 ( h ) by Theorem 10.

From the previous empirical results, we conclude the following conjecture:

Conjecture 13. When ϵ = ( ϵ P , ϵ η , ∞ ) , where ϵ P ∈ (0 , ∞ ) , the sequence of posteriors Θ k from generic C , η , θ and ϵ as random variables satisfy lim k →∞ ∑ h ∈H P ( | Θ k ( h ) -1 | <e ) = 1 for any e > 0 .

Further, we pick out those episodes with θ N ( h ) > 0 . 95 , plot the values E θ N ( h ) > 0 . 95 [ln θ k ( h ) -ln(1 -θ k ( h ))] for each h against k in Fig. 4. Near linear relations are observed away from the first several rounds and before the values reaches the precision threshold. These are empirical estimates of the rate of convergence.

1

0.5

0

60

0

2

4

0

2

4

0

2

4

0

2

4

B0

0

2

t

0

2

4

0

2

4

0

2

4

30

0

2

4

40

5

10

5

h0

5

h0

20

40

20

30

20

20

20

5

5

5

10

20

10

10

10

0

0

0

0

0

0

0

0

0

0

100

200

0

50

100

0

50

100

0

50

100

0

50

100

0

100

0

20

40

0

20

40

0

20

40

bi

0.02

0.2

0.5

1

2

5

50

Inconclusive and independent: ϵ θ = 0 . The following holds for both ( PS ) and ( RS ):

Proposition 14. For ϵ = ( ϵ P , ϵ η , 0) with ϵ P ∈ (0 , ∞ ) , as η k → η almost surely, the sequence Θ k of posteriors as a sequence of random variables converges in probability to variable Θ , where P (Θ = v i ) = η ( i ) and v i = P ( i, _ ) / ( ∑ m j =1 P ij ) and P = P ϵ ( C, η, θ ) . Therefore, for any s > 0 , lim k →∞ ∑ h ∈H P ( | Θ k ( h ) -1 | < s ) = 0 for generic (for all but in a closed subset) cost C and η , θ .

With ϵ θ = 0 , the constraint on column-sum ( ϵ η -term) fails to affect the transport plan, thus the Θ k 's in the sequence are independent from each other, in contrast that in all other cases the adjacent ones are correlated via a nondegenerate transition distribution. The independence makes the sequence of posterior-samples in one episode behave totally random, thus rarely converge as points in P ( H ) . Furthermore, when consider the natural coupling (Θ k -1 , Θ k ) from Markov transition measure for ϵ θ = 0 (which is independent), E ( | Θ k -1 -Θ k | 2 ) converges to the variance V ar ( η ) . In contrast, for ϵ θ = ∞ , E ( | Θ k -1 -Θ k | 2 ) converges to 0 if Conj. 13 holds.

ϵ θ ∈ (0 , ∞ ) : partially conclusive. From Conj. 13 and Prop. 14, together with the continuity of the transition distribution on ϵ , we conjecture the following continuity on conclusiveness:

Conjecture 15. For both ( PS ) and ( RS ) models, when ϵ = ( ϵ P , ϵ η , ϵ θ ) with ϵ P , ϵ θ ∈ (0 , ∞ ) , the posterior sequence Θ k from generated from generic C , η , θ and ϵ satisfy that lim k →∞ ∑ h ∈H P ( | Θ k ( h ) -1 | < s ) = L exists, and L ∈ (0 , 1) , for any s > 0 .

An Exploration on Interpolation It is popular in the state of art machine learning models that an agent learns probabilistically, but makes decisions greedily. This heuristic represents a path where a big leap on the cube was taken at the last step. An interesting question is under what circumstances this is optimal, what are the trade-offs, and under what conditions smoother trajectories are preferable. Instead of the giant leap, small steps along two paths are explored in the cube.

Our exploration takes a slightly different situation where the last step is replaced by a discriminative learning strategy ϵ = (0 , ∞ , ∞ ) . We choose a matrix randomly of shape 4 × 4 (see Supplementary for detail), set total steps or total data points taught N = 10 , and uniform prior θ ∈ P ( H ) on hypotheses. Three learners are postulated: Blue performs Bayesian in first 9 steps and Discriminative in the last. Orange and Red follows two different interpolation curves with the same endpoints on the cube drawn in Fig. 5a. The curves are line and parabola segments on the cube sides.

Results of the three learners are shown in Fig. 5b. We sample h ∗ uniformly from the 4 columns of (the column-normalized) M , 40000 repeats for each learner. Conclusiveness (minimal l 1 distance between posterior and a 1-hot vector) and posterior entropy are plotted as histograms. The results show that the smoother path may lead to a more conclusive posterior. Numerical results: Conclusiveness of Blue: mean 0.9406, standard deviation 0.1300. Conclusiveness of Orange: mean 0.9964, standard deviation

0.0327. Conclusiveness of Red: mean 0.9834, standard deviation 0.0676. Furthermore, compared with a sudden jump, gradual interpolations have lower entropy. Numerical results: entropy of Blue: mean 0.1261, standard deviation 0.2435, entropy of Orange: mean 0.0079, standard deviation 0.0629; entropy of Red: mean 0.0388, standard deviation 0.1336.

104

104

104

103

103

103

102

102

102

101

10

101

100.

100 1

100 1

Path 2 Bayesian

0.4

0.6

0.8

1.0

0.4

0.6

0.8

1.0

0.4

0.6

0.8

1.0

104

104

104

Path1

sigmoid(log(ε)

103

103

103

102

102

10²

Discriminative

101

101

10

100

0.000.250.500.751.00

100

0.000.250.500.751.00

00

0.000.250.500.751.00

From this experiment, in the 10-sample learning, two smoother learners behave more conclusive in their posterior with smaller posterior entropy. Meanwhile, we still know very little about the interpolation behavior, such as which path works better, how to distribute vertices on the curve, etc.

4 Sequential GBT - Dynamic

While static models are frequently studied, in many cases world changes dynamically. In this section, we take a first step in this direction by exploring the sequential behaviors of GBT learners assuming the world changes periodically. Demonstrate that unlike existing learners, GBT learner is capable of detecting the non-static property of a given problem.

Let integer p > 0 be the period, given a set of true hypotheses distributions ⃗ η = { η 0 , η 1 , . . . , η p -1 } ⊆ P ( D ) , datum d t is sampled from d k % p where k % p represents the remainder of k under division by p . Proposition 16. For a Bayesian learner, the posterior sequence { Θ i } converges almost surely to the average of true hypotheses η = 1 p ∑ p -1 k =0 η k .

For random variables of learner's posterior sequence (Θ k ) ∞ k =1 , group them by period, we denote ⃗ Θ t := (Θ tp , Θ tp +1 , . . . , Θ ( t +1) p -1 ) . Here k represents the time step, t denotes the period index.

Proposition 17. For ϵ in the interior of the cube, for ( PS ) problem, the sequence { ⃗ Θ t } (random variables over P ( H ) p ) form a time-homogeneous Markov chain. For ( RS ) problem, { ( ⃗ Θ t , 1 pt ∑ p -1 k =0 tη k ) } , the random variable sequence producing samples { ( ⃗ θ t , 1 pt ∑ pt -1 k =0 δ d k ) } , forms a Markov chain.

Next we compare different learners' behavior empirically for ( RS ) problem. For visualization, M is taken of shape 3 × 3 , thus P ( D ) and P ( H ) are both of dimension 2. If the Markov chain defined in Prop. 17 stabilizes, E [Θ k ] will be periodic, matching the pattern of ⃗ Θ t . In fact, the period could be p , or a factor of p , or stabilizes where the period can be considered as 1 . Thus we analyze ⃗ η in P ( D ) and E [Θ k ] in P ( H ) , obtained from Monte-Carlo sampling along certain amount of episodes.

Fig. 6 (a) assume the true hypothesis travel along the triangular path connecting the 3 columns of M (shown in blue crosses). We found that GBT learners with ϵ in the interior of the cube (general GBT) produce a posterior path of period p , while the posteriors of Bayesian and SCBI learners tend to converge (Fig. 6 (b-e)). Thus a general GBT learner can naturally detect the periodicity of the world. We simulate up to k = 400 steps and 10240 repeats for each learner.

Moreover, we discovered that a general GBT learner's posteriors converge to a curve whose area is proportional to the area of the path of true hypotheses. In Fig. 7, as the path of true hypotheses vary by radius and shapes, the ratio (shown as slope) between both areas tends to be the same, it is 0.1620 in (a) and 0.1616 in (d), which suggests that this ratio is independent from the path of ⃗ η .

(a).P(D)

(b).P(H)

(1, 0,∞)

solid

0.175

(1, 0,∞)

Bayesiary

[ll,

0.150

(1,10, 10)

0.20

dash-dots

(1, 10, 1)

0.125

(1,1, 10)

一

0.100

(1,00,∞0)

(1, 1, 1)

0.15

0.075

0.10

(c).P(H)

(d).P(H)

(1,10,10)

(1, ∞, ∞)

0.050

SCBI

mean

0.025

0.05

0.000

0.00

0

3

6

9

12

15

18

21

(e).numberofperiods(t)

0.9

10

8

0.8

0.7

8

0.6

6

0.6

0.8

(b).

Teachers in (a).

4

P(D):Mandteachers

P(D):M and teachers

4

2

0.8

2

P(H):posteriors

0.6

P(H):posteriors

0

0

(a).

0

1

Areaofteachingcycle(x10-2)

2

3

4

5

(c).Teachers in (d).

0.6

0.8

(d).

0

1

Areaofteachingcycle(x10-2)

2

3

4

5

Related Work. Prior work defines and outlines basic properties of Unbalanced Optimal Transport [Liero et al., 2018, Chizat et al., 2018, Pham et al., 2020]. Bayesian approaches are prominent in machine learning [Murphy, 2012] and beyond [Jaynes, 2003, Gelman et al., 1995]. There is also research on cooperative learning [Wang et al., 2019, 2020b,a] see also [Liu et al., 2021, Yuan et al., 2021, Zhu, 2015, Liu et al., 2017, Shafto and Goodman, 2008, Shafto et al., 2014, Frank and Goodman, 2012, Goodman and Frank, 2016, Fisac et al., 2017, Ho et al., 2018, Laskey et al., 2017]. Discriminative learning is the reciprocal problem in which one sees data and asks which hypothesis best explains it [Ng and Jordan, 2001, Mandler, 1980]. We are unaware of any work that attempts to unify and analyze the general problem of learning in which each of these are instances.

5 Conclusions

We have introduced Generalized Belief Transport (GBT), which unifies and parameterizes classic instances of learning including Bayesian inference, Cooperative Inference, and Discrimination, as Unbalanced Optimal Transport (UOT). We show that each instance is a point in a continuous, differentiable on the interior, 3-dimensional space defined by the regularization parameters of UOT. Moreover, to demonstrate general GBT's capacity of supporting generalized learning, we prove and illustrate asymptotic consistency and estimate rates of convergence, including convergence to hypotheses with zero prior support, and ability of gripping dynamic of the world. In summary, GBT unifies very different modes of learning, yielding a powerful, general framework for modeling learning agents.

Acknowledgments and Disclosure of Funding

This research was supported in part by DARPA awards HR0011-23-9-0050 and W912CG22C0001 and NSF MRI 2117429 to P.S.

References

Laetitia Chapel, Rémi Flamary, Haoran Wu, Cédric Févotte, and Gilles Gasso. Unbalanced optimal transport through non-negative penalized linear regression. Advances in Neural Information Processing Systems , 34:23270-23282, 2021.

Lenaic Chizat, Gabriel Peyré, Bernhard Schmitzer, and François-Xavier Vialard. Scaling algorithms for unbalanced optimal transport problems. Mathematics of Computation , 87(314):2563-2609, 2018.

Thomas M Cover. Elements of information theory . John Wiley & Sons, 1999.

Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. In Advances in neural information processing systems , pages 2292-2300, 2013.

Paul Dagum, Adam Galper, and Eric Horvitz. Dynamic network models for forecasting. In Uncertainty in artificial intelligence , pages 41-48. Elsevier, 1992.

Joseph L Doob. Application of the theory of martingales. Le calcul des probabilites et ses applications , pages 23-27, 1949.

Stephen E Fienberg et al. An iterative procedure for estimation in contingency tables. The Annals of Mathematical Statistics , 41(3):907-917, 1970.

Jaime F Fisac, Monica A Gates, Jessica B Hamrick, Chang Liu, Dylan Hadfield-Menell, Malayandi Palaniappan, Dhruv Malik, S Shankar Sastry, Thomas L Griffiths, and Anca D Dragan. Pragmaticpedagogic value alignment. arXiv preprint arXiv:1707.06354 , 2017.

Michael C Frank and Noah D Goodman. Predicting pragmatic reasoning in language games. Science , 336(6084):998-998, 2012.

Andrew Gelman, John B Carlin, Hal S Stern, and Donald B Rubin. Bayesian data analysis . Chapman and Hall/CRC, 1995.

Noah D Goodman and Michael C Frank. Pragmatic language interpretation as probabilistic inference. Trends in cognitive sciences , 20(11):818-829, 2016.

Mark K Ho, Michael L Littman, Fiery Cushman, and Joseph L Austerweil. Effectively learning from pedagogical demonstrations. In Proceedings of the Annual Conference of the Cognitive Science Society , 2018.

Edwin T Jaynes. Probability theory: The logic of science . Cambridge university press, 2003.

Harold Jeffreys. An invariant form for the prior probability in estimation problems. Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences , 186(1007):453-461, 1946.

Michael Laskey, Caleb Chuck, Jonathan Lee, Jeffrey Mahler, Sanjay Krishnan, Kevin Jamieson, Anca Dragan, and Ken Goldberg. Comparing human-centric and robot-centric sampling for robot deep learning from demonstrations. In Robotics and Automation (ICRA), 2017 IEEE International Conference on , pages 358-365. IEEE, 2017.

Matthias Liero, Alexander Mielke, and Giuseppe Savaré. Optimal entropy-transport problems and a new hellinger-kantorovich distance between positive measures. Inventiones mathematicae , 211(3): 969-1117, 2018.

Weiyang Liu, Bo Dai, Ahmad Humayun, Charlene Tay, Chen Yu, Linda B Smith, James M Rehg, and Le Song. Iterative machine teaching. In International Conference on Machine Learning , pages 2149-2158. PMLR, 2017.

Weiyang Liu, Zhen Liu, Hanchen Wang, Liam Paull, Bernhard Schölkopf, and Adrian Weller. Iterative teaching by label synthesis. Advances in Neural Information Processing Systems , 34, 2021.

George Mandler. Recognizing: The judgment of previous occurrence. Psychological review , 87(3): 252, 1980.

Kevin P Murphy. Machine learning: a probabilistic perspective . MIT press, 2012.

Kevin Patrick Murphy. Dynamic bayesian networks: representation, inference and learning . University of California, Berkeley, 2002.

Andrew Ng and Michael Jordan. On discriminative vs. generative classifiers: A comparison of logistic regression and naive bayes. Advances in neural information processing systems , 14, 2001.

Gabriel Peyré and Marco Cuturi. Computational optimal transport. Foundations and Trends in Machine Learning , 11(5-6):355-607, 2019.

Khiem Pham, Khang Le, Nhat Ho, Tung Pham, and Hung Bui. On unbalanced optimal transport: An analysis of sinkhorn algorithm. In International Conference on Machine Learning , pages 7673-7682. PMLR, 2020.

Julien Rabin, Sira Ferradans, and Nicolas Papadakis. Adaptive color transfer with relaxed optimal transport. In 2014 IEEE international conference on image processing (ICIP) , pages 4852-4856. IEEE, 2014.

Christian P Robert, Nicolas Chopin, and Judith Rousseau. Harold jeffreys's theory of probability revisited. Statistical Science , 24(2):141-172, 2009.

Patrick Shafto and Noah D. Goodman. Teaching games: Statistical sampling assumptions for learning in pedagogical situations. In Proceedings of the 30th annual conference of the Cognitive Science Society , Austin, TX, 2008. Cognitive Science Society.

Patrick Shafto, Noah D Goodman, and Thomas L Griffiths. A rational account of pedagogical reasoning: Teaching by, and learning from, examples. Cognitive Psychology , 71:55-89, 2014.

Patrick Shafto, Junqi Wang, and Pei Wang. Cooperative communication as belief transport. Trends in cognitive sciences , 25(10):826-828, 2021.

Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction . MIT press, 2018.

Cédric Villani. Optimal transport: old and new , volume 338. Springer Science & Business Media, 2008.

Junqi Wang, Pei Wang, and Patrick Shafto. Sequential cooperative bayesian inference. In International Conference on Machine Learning , pages 10039-10049. PMLR, 2020a.

Pei Wang, Pushpi Paranamana, and Patrick Shafto. Generalizing the theory of cooperative inference. AIStats , 2019.

Pei Wang, Junqi Wang, Pushpi Paranamana, and Patrick Shafto. A mathematical theory of cooperative communication. Advances in Neural Information Processing Systems , 33, 2020b.

Scott Cheng-Hsin Yang, Yue Yu, Arash Givchi, Pei Wang, Wai Keen Vong, and Patrick Shafto. Optimal cooperative inference. In AISTATS , volume 84 of Proceedings of Machine Learning Research , pages 376-385. PMLR, 2018.

Luyao Yuan, Dongruo Zhou, Junhong Shen, Jingdong Gao, Jeffrey L Chen, Quanquan Gu, Ying Nian Wu, and Song-Chun Zhu. Iterative teacher-aware learning. Advances in Neural Information Processing Systems , 34:29231-29245, 2021.

Xiaojin Zhu. Machine teaching: An inverse problem to machine learning and an approach toward optimal education. In Twenty-Ninth AAAI Conference on Artificial Intelligence , 2015.

Peer reviews:
comments: SUMMARY:
This paper describes a unifying view at various learning settings in machine learning, such as Bayesian inference, optimal communication, supervised classification and frequentist inference. All three learning settings are shown as special cases of an objective function, which can be interpreted from the lens of optimal transport.

STRENGTHS:
The proposed objective function generalises such learning concepts in a general equation. This generalisation sheds new light on the various methods.

WEAKNESSES:
The current paper generalises the learning paradigms, but what new insight can be gained from this? Does this generalisation give rise to a new learning method or a more efficient version of an existing model that would solve an existing problem?

QUESTIONS:
* Does this work relate to Generalised Belief Propagation [1][2]? Or fit in this [3] hierarchy of inference methods?

[1] Yedidia, Jonathan S., William Freeman, and Yair Weiss. "Generalized belief propagation." NeurIPS 2000

[2] Teh, Yee, and Max Welling. "The unified propagation and scaling algorithm." NeurIPS 2001

[3] Rosen-Zvi, Michal, Michael I. Jordan, and Alan Yuille. "The dlr hierarchy of approximate inference." arXiv (2012).

LIMITATIONS:
-
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-05

comments: SUMMARY:
This work aims to unify learning approaches by specifying a 3D space where the dimensions represent modalities of learning that can be combined to specify various learning approaches, e.g., Bayesian and Frequentist approaches. The 3D space is defined by three learning constraints within Unbalanced Optimal transport (UOT).

STRENGTHS:
This work's conceptual aims are quite interesting and potentially illuminating and instructive.

WEAKNESSES:
Many details are glossed over in the proofs, making them hard to read and understand. This work is limited to finite datasets and finite hypotheses, which significantly limits the generality of the work. Minimally, the hypothesis class' finiteness limits this unification's informativeness.

QUESTIONS:
Does this unification hold with infinite hypotheses or datasets? How might one use this unification to design new algorithms?

LIMITATIONS:
This work is not presented in a digestible manner; the impact of this work is not effectively communicated or demonstrated.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nStandard models of machine learning treat different internal constraints (e.g., prior knowledge) and external constraints (e.g., time availability, environmental non-stationarity) as separate problems, and thus hinder the development of unified learning agents. This paper proposes a framework called Generalized Belief Transport (GBT), which builds upon Unbalanced Optimal Transport, to unify existing learning models. GBT offers a parameterization that allows for interpolation between different modes such as Bayesian inference, Frequentist inference, cooperative learning, and discriminative learning. The authors provide theoretical analyses and empirical investigations to support their claims.\n\nIn short, the paper makes the following contributions:\n- It proposes a parameterization that unifies existing learning approaches and shows its continuity and differentiability. \n- It analyzes the behavior of GBT under variations in the parameter space. \n- It studies sequential GPT for both static and non-static settings.\n\nSTRENGTHS:\n**Originality & Significance**\n- The paper presents an interesting point of view that unifies various learning models with different internal and external constraints. The proposed GBT framework allows flexible combinations of $(\\epsilon_P, \\epsilon_\\eta, \\epsilon_\\theta)$ and helps the development of unified learning agents.\n\n**Quality & Clarity**\n- The paper is well-structured and provides a clear explanation of the challenges and the proposed framework. The theoretical analyses conducted to establish continuity, differentiability, and behavior under parameter variations help improve the understanding of the framework. The empirical investigations on sequential learning and predictive performance under environmental drift also demonstrate the practical applicability of GBT. The authors provide the proofs for the key theorems and the code for reproducing experiments in the supplementary material.\n\nWEAKNESSES:\n**Empirical significance**\n- It is very interesting to unify seemingly different learning models like the Frequentist and the Bayesian. However, the paper touches vaguely on the practical implications of such a unified framework. For instance, are there trade-offs for taking different points in the parameter space? How does the proposed framework extrapolate? Does it lead to new learning methods that are previously under-explored? What are some practical future steps for improving/utilizing the proposed framework?\n\n\n**Presentation**\n- The paper assumes a very high level of familiarity with concepts such as the Frequentist-Bayesian dichotomy and generative vs. discriminative approaches. Readers without extensive prior knowledge in these areas may find it challenging to fully grasp the nuances of this work. Including additional explanations or references to relevant background material would enhance the accessibility of the paper.\n\n**Other details**\n- Section 1: missing definition of $\\eta$ (estimation of the data distribution) in the first 4 paragraphs.\n- The arrangement of the color bars in Figure 2 (left) can be more organized. Currently it’s hard to read and distinguish the setting names.\n- Color legend is missing for Figure 3.\n\nQUESTIONS:\n- How is the area for the GBT learner’s posteriors measured for Figure 6 (the curves do not seem to be closed)?\n\n- Are there any existing efforts to combine discriminative learning methods with Bayesian approaches (class priors) to tackle a learning problem? How does the proposed framework handle such integration?\n\nLIMITATIONS:\nGiven the lack of previous work attempting to unify and analyze the general problem of learning under constraints, what are the potential research directions and open questions in this area? How can the field benefit from a unified framework for learning and inference?", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors introduce the concept of Generalized Belief Transport to unify and parameterize 3 different axes of learning from within the formalism of Unbalanced Optimal Transport.  Corresponding limits points on the \"cube\" of these 3 axes recapitulate many common learning paradigms from the literature (e.g., bayesian inference, frequentist inference, cooperative communication, etc.).  The authors further demonstrate an algorithm for solving some synthetic instances of these various problems, and demonstrate various tradeoffs as one moves from one type of learning to another.\n\nSTRENGTHS:\nOriginality:  The connections between these formalisms through the lens of UoT (and the proofs) are certainly novel.\n\nQuality: The overall quality of presentation is high, and I particularly like the \"cube\" device for visualizing the different limit points.\n\nClarity: The presentation is incredibly straightforward, and the proofs are quite clear.\n\nSignificance: This is less clear, but a more general formalism that can automatically facilitate many different, familiar forms of learning seems instrumentally valuable for further methods development.\n\nWEAKNESSES:\nHow useful this formalism will _actually_ be for practitioners is my primary point of uncertainty.  Typically, any given practitioner doesn't really have any doubt about where they are on the \"cube\" for their learning problem, and thus they throw the most powerful technique available to them at that particular point of learning-agent-space.  This work might be better served by a less synthetic example, i.e., an example problem that _requires_ moving between points in learning-agent-space, to which this formalism would actually be uniquely well suited.  But a good (better yet, a *convincing*) example of such a thing is not obvious to me.  tldr: how do I actually use this formalism, if I know exactly where I am in learning agent space, and why wouldn't I use one of the more well-known tools?\n\nQUESTIONS:\nsee above\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

comments: SUMMARY:
This paper proposes a framework called Generalized Belief Transport that unifies different types of machine learning models. The authors have proven a number of properties about their proposed framework.

STRENGTHS:
* The problem proposed seems relevant given the different number of approaches to machine learning. If successful the proposed framework can greatly unify prior work.

WEAKNESSES:
* I do not have any prior knowledge about the problem studied in this paper at all, so I found the paper to be difficult to understand. That might not necessarily be the fault of the authors, but I will defer to the experts in this area to comment.

QUESTIONS:
N/A

LIMITATIONS:
N/A
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-25