Resolving label uncertainty with implicit generative models
Authors: Esther Rolf
Nikolay Malkin
Alexandros Graikos
Ana Jojic
Caleb Robinson
Nebojsa Jojic
Venue: ICLR 2022 Submitted (2022)
Keywords: weakly supervised learning
generative models
image segmentation

Abstract: In prediction problems, coarse and imprecise sources of input can provide rich information about labels, but are not readily used by discriminative learners. In this work, we propose a method for jointly inferring labels across a collection of data samples, where each sample consists of an observation and a prior belief about the label. By implicitly assuming the existence of a generative model for which a differentiable predictor is the posterior, we derive a training objective that allows learning under weak beliefs. This formulation unifies various machine learning settings: the weak beliefs can come in the form of noisy or incomplete labels, likelihoods given by a different prediction mechanism on auxiliary input, or common-sense priors reflecting knowledge about the structure of the problem at hand. We demonstrate the proposed algorithms on diverse problems: classification with negative training examples, learning from rankings, weakly and self-supervised aerial imagery segmentation, co-segmentation of video frames, and coarsely supervised text classification.

Under review as a conference paper at ICLR 2022
RESOLVING LABEL UNCERTAINTY WITH IMPLICIT GENERATIVE MODELS
Anonymous authors Paper under double-blind review
ABSTRACT
In prediction problems, coarse and imprecise sources of input can provide rich information about labels. but are not readily used by discriminative learners. this work, we propose method for jointly inferring labels across a collection of data samples, where each sample consists of an observation and a prior be- lief about the label. By implicitly assuming the existence of a generative model for which a differentiable predictor is the posterior, we derive a training objective that allows learning under weak beliefs. This formulation unifies various machine learning settings; the weak beliefs can come in the form noisy or incomplete labels, likelihoods given by different prediction mechanism auxiliary input, or common-sense priors reflecting knowledge about the structure of the problem at hand. We demonstrate the proposed algorithms on diverse problems: classifi- cation with negative training examples, learning from rankings, weakly and self- supervised aerial imagery segmentation, co-segmentation of video frames, and coarsely supervised text classification.
INTRODUCTION
We consider the problem of joint inference of latent label variables & in collection of data samples indexed by consisting of observations (features) X; and corresponding prior beliefs about their latent label variables p;(().
Two illustrative examples are shown in Fig. I In the first example, the x; are 784-dimensional vectors representing 28x28 MNIST digits. We aim to infer the digit classes 4 {0. .). all images in the given collection based on data in which we are given just one negative label per sample, i.e., the prior beliefs (((() (top row) are uniform over all classes except for one incorrect class. The procedure described in this paper produces inferred distributions over labels (bottom row) that are usually peaky and place the maximum at the correct digit 97% of the time (Fig. 3b.
9} for
In the second example, the observations x; are image patches centered around each pixel coordinate in Surrealist painting, with patch size (1 x1x1 equal to the receptive field of a 5-layer convolutional
neural network used our inference procedure. The prior beliefs (f(() are distributions over
classes (sky, boat, water) depending the coordinate i. The joint inference of all labels in this image yields a feasible segmentation despite the high similarity in colors and textures (see $E.4
In both examples, the inference technique needs to estimate statistical links between observations xj and corresponding latents & that would both be highly confident (i.e., lead to low entropy in the posterior distributions) and explain the varying prior beliefs, which typically have low confidence (high entropy in the prior distributions). This problem training on weak beliefs, in some form, is learning, domain
often encountered in machine learning, c.g, weak supervision and semi-superviss transfer, and integration of modalities - settings where coarse, partial, or inexact sources of data can provide rich information about the state of a prediction instance, though not always "ground truth" label for each instance.
In many such settings, fusing the weak input data into probability distribution over classes is more natural alternative to transforming the weak input into hard labels (26). However, because supervised models target the distribution over labels, training machine learning models with super- vision from probabilistic prior beliefs results in uncertain predictions. Most approaches to resolve these uncertainties involve iterative generation of hard pseudo labels (56), loss functions promot-

---

Under review as a conference paper at ICLR 2022
7
(a) Le séducteur, René Magritte (b) Boat Prior, anonymous artist
(c) Inferred segmentation
Inference of latent MNIST digit classes with negative label supervision using small CNN trained on the RQ criterion 2 Below: Joint inference of latent pixel classes in an image (a). The prior beliefs (C) over three classes sky (red), boat (green), water (blue) are manually set (b). A small CNN using no data except (a.b) infers the posterior classes (c).
Figure Above:
ing low entropy of predictions [35 55 59 54]. Typically, these approaches are ieppiiaationspeciife
10
57
[2 Further discussion of related work is provided in {Bandd $C
Our key modeling insight is to associate the output distribution forward neural network q, with implicit generative model (.2. of features and to consider the given prior belief as part of that model. We show that without estimating the full generative model, is possible to learn the network that performs inference in it and reap the benefits of generative modeling, including high certainty in the posterior under soft priors and rich opportumities to model structure in the prior beliefs. We validate the effectiveness of our approach with experiments ($4) {E that highlight: prior beliefs as a natural way to fuse weak inputs, graceful degradation of per- formance with increasingly noisy incomplete inputs, and comparison of our implicit generative model with explicitly generative modeling approaches.
discriminative model, a feed-
BACKGROUND AND APPROACH
Supervised learning on prior beliefs. Supervised learning models, including many neural nets, are typically trained to minimize the cross-entrope Σ; Σ, (t) log (E( between the data distri- bution over labels (f) and the distribution ((( q({|x;; output by predictor q using data features X1. This is equivalent minimizing the KL divergence Σ, KL Ilq)) which not well- suited to training with uncertain labels defined by prior beliefs. If we were to set p (f) to equal a much softer prior over latent labels, ((),) the minimum would be attained when the two distribu- tions (() and (() are equal: when the prior belief soft, the trained model q will also be highly uncertain. Turning soft labels into hard training targets, (c) 1[e arg max, Pr())), by sampling), introduces the opposite bias. Now, the cost would indeed be minimized if the predictions had zero entropy, but learning such a prediction function faces difficulty with overconfident labels which are often wrong, as well as the possibility that certain labels often receive substantial weight in the prior, but never the maximum. These issues illustrated in Figure D.3
Generative modeling resolves the prior's uncertainty. The approach classification problems through generative modeling, instead of targeting the conditional probability of latents given the data features, assumes that there is forward (generative) distribution p(x,le) and optimizes the log- ligeliiel of the observed features, Σ, log(x,) Σ, log Σe p(x]((p; (O), with respect to the parameters of the forward distribution. The posterior under the model p(f(x;) p(x)()p;((t) is then used to infer latent labels for individual data points [44]. As result, the generative modeling approach does not suffer from uncertainty in the posterior distribution over latents given the input
2

---

Under review as a conference paper at ICLR 2022
features, even when the priors ((( are soft generative modeling approaches is given in S
Further discussion relating method with existing
However, expressive generative models are typically harder and more expensive to train compared to supervised training of neural networks, as they often require sampling (c.g., sampling of the posterior in VAEs [1 and sampling of the generative model in GANs (8). Furthermore, the modeling often requires doubling of parameters to express both the forward (generative) model and the reverse (posterior) model. And, in case of GANs, the learning algorithms may not cover all modes in the data, which would prevent joint inference for all data points.
2.1 OPTIMIZATION UNDER IMPLICIT GENERATIVE MODELS
Suppose that there is generative model p(x(f) of observed features conditioned on latent labels. Optimization of the log-likelihood of observed features, Σ, log(x;) = Σ, log Ee p(x](()p;(() can be achieved by introducing a variational distribution (f) over the latent variable for each instance Kj, then minimizing the variational free energy, which we review next.
Recall that the free energy, also known the negative evidence lower bound (ELBO), is defined as
F: - L 9i((t) p(x1())p((() (i())
Σ
(KL(q)))))) - log
where r() = p(x,Opp((O) Er p(x(()pp:: is the posterior distribution under the generative model. As the KL distance is always positive, the free energy is minimized when qi ri, in which case the free energy equals the negative log-likelihood of the data.
Free energy minimization is used in various approaches to latent variable modeling. The expectation-maxxiiiztoon (EM) 6 algorithm minimizes (T) by iteratively setting qi to equal the posteriors r and updating the parameters of the forward distributions p(x;(() while keeping qi fixed, until guaranteed convergence to a local minimum. In applications of EM predating deep generative models, the qi distributions are not given by a predictor taking x; input; they are are auxiliary distributions used only as part of the learning procedure and replaced at each EM iteration. Their dependence on x; is implicit in energy minimization.
In variational auto-encoders (VAEs) (1I), both the generative model px([h) and the posterior (h(xx) are represented neural networks. Here p a deep nonlinear generative model with hidden latents h (not necessearily corresponding to labels), and evaluation of the full posterior is intractable. Instead, q is learned as a neural network (h)x,, 0) jointly with p. VAE training involves sampling q(x[xx 0) in learning to improve the agreement between the forward (generative)
from q: (h) model p (and its true posterior (h)) and the reverse (posterior) neural network model q.
Implicit generative models. To derive our training objectives, we will assume the existence of the generative model p(x(() without ever parametrizing it or sampling from it. Instead, we parametrize only the reverse (variational posterior) model (f) = q 0) in the form of a neural net taking x as input. The trained model q the direct output of our algorithms.
Given the variational distributions qi. we can minimize (I] with respect to the forward probabilities p(x(()) for all x; and such that Σ, p(x](() for all t. The optimum achieved by:
pxx(()= (() Σイ)
(2)
We refer this as the implicit generative model associated to q. Much like the posterior implicit in the free energy minimization in the EM algorithm. here instead the forward model is just matrix of numbers Pi,z, implicit free energy minimization the opposite, in some sense, to what is done in EM optimization. The link x l is left entirely to the neural network q to capture explicitly. The probability mass of the implicit generative model is not uniformly distributed, rather, the data points for which the variational posterior (C) is more certain are considered more likely under that latent e, unless it corresponds to a popular class to which many other data points are also assigned.
For intuition, consider a model where & is the mixture index in mixture of high sigh-dimensionll Gaussians. If the observations x; naturally cluster into several well-defined clusters, then the prior pf(f) may be flat, hut the posterior distributions under the model assign data points to clusters with high certainty.
3

---

Under review as a conference paper at ICLR 2022
The posterior under the implied generative model is
batchusize, nuclasses Log-Likeebbodds from
prior: size, nuclasses tikelshoods
((()).(1) ri(1) Σj9((0)
cet ce torstts,,,, priart:
(3)
return -foog priort.sum(1)
tossoos priori:
Nog., leg_se leg(1 (eg.el Log.sttogsoooooo
Note that we can compute rt) by multiplying the re-normalized model outputs with the prior at each instance, so that for each instance i we have two outputs: qi and ri.
(logue (ol.a.ep(())ssmm11 Hlog_r (gg.0.5o11)...mm)
det ra lossoo,, priarts
tog_r 1oog., leg, softtoottt) + (egtrr 3oftraxt33 return (log_r eum Ulog, (sgrr.exs())) sunii1
Figure 2:
Cross-entropy and implicit QR / RQ
We propose two methods of optimizing the free energy with respect to the parameters & by gra- dient steps. Each method iterates the following:
losses in PyTorch. Note that normalization in 2 done within batch, rather than across the en- tire dataset. In practice, this may be sufficient if batches are large and representative of the diver- sity in the data. Otherwise, the denominator in (2) may need to be updated in an online fashion.
(1) Calculate the posterior distributions ri in terms of qi as in (3)
(2) Update the parameters of q with gradient step:
Option QR: 0 V Σ. (L.l)))))).
Option RQ: 9 - 0 - V Σ, KL(rill1)..
Gradients of the model parameters propagated to the expression of r; through qi (see Fig. 2 Both losses have stable point when q, =r.. A discussion of the relative benefits and limitations of the QR and RQ loss formulations is given in A along with practical considerations for implement- ing these methods.
Option QR uses the KL distance in the direction it appears in ① and thus guarantees continual improvements in free energy and convergence to a local minimum (with the exception for the effects of stochasticity in minibatch sampling). Substituting r from (3) the free energy (I] becomes:
F = De9(( log Earc)) -E (o) -o (pi(O) i.
(4)
This criterion does not encourage entropy of individual qi distributions, but of their average. The second term alone would be minimized if the q network could put all the mass on arg max, (f( for each data point, but the first term promotes diversity in assignment of latents (labels) & across the entire dataset. Thus network q can optimize (4) if it makes different confident predictions for different data points. To illustrate this, consider the case when p:(() p()), i.e. all data points have p((), and this can be achieved when the network learns a constant distribution (((xx;;)) = p(). But the free energy also minimized if the network predicts only single label for each data point with high certainty, but it varies in predictions so that the counts of label predictions match the prior.
the same prior. Then 4 is minimized when 1 Σ, (h)
As demonstrated in Fig. I and in our experiments, avoiding degenerate solutions is not hard. We attribute this to two factors. First, the situations of interest typically involve uncertain, but varying distributions p.(E) which break symmetries that could lead to ignoring the data features x;- Second, the neural networks used as posterior models come with their own constraints in parametrization (e.g. a multi-layer convolutional net with certain number of filters in each layer), and, the inductive biases as a consequence of gradient descent learning in nonlinear, multilayer networks. In fact, as discussed in $ and $E. even unsupervised clustering is possible with suitably chosen priors that break symmetry, allowing this approach to be used for self-supervised training. See also $ for more on relationships with other approaches.
SOURCES OF LABEL PRIORS
In this section, we describe range of machine learning settings where priors P:(() emerge. These settings are illustrated in experiments $4 with additional experiments $
Negative or partial labels ($4.1). When we are given a set of equally possible labels L; for each point data point i, instead of a single label f, then we set the prior pr(E) 1 E L,]. An
extreme example is when one negative label is given, as shown in Fig. [
4

---

Under review as a conference paper at ICLR 2022
Joint labels and learning from rankings ($4.2). Priors may also come in the form of joint dis- tributions over labels of multiple instances. For example, ranking supervision - the knowledge of which example in pair is greater with respect to an ordering of the labels gives prior beliefs about pairs of labels. Suppose our data is organized into pairs of images of digits 7 = (,,2) and for each pair we are told which image represents the digit (0-9) which is greater. This sets prior p(4,, (2) over pairs of labels in each pair, represented by either an upper or a lower triangular matrix, depending on which digit in the pair is known to be greater, with all nonzero entries equal to 1/5s.
We assume the implicit generative model has the form X2 f2) = p(x1 (222(()) We aim to fit posterior model q([x;; 0). For each pair Tj, we have two outputs of the predictor network, ((111z,1) and ((22,1,22) for the two images in the pair. The joint posterior under the implicit generative model is
pll, ((G,
()((,,6) & pl,, plG,,)))GGs x ,,9((1l,,,, Σ, ((22),2))
(5)
and we can now use QR or RQ loss to fit ((11)1)))) to the marginal (()) and ((21x)22) to ((2).
We often have side infor- mation z associated to each instance i that allows setting the priors (f( = p(flz, for each point directly by hand. These include situations when we have beliefs about labels for different points, as in the Seducer example (Fig. D. but also more interesting weak supervision settings, such as ones that sometimes arise in remote sensing ($4.3) and medical pathology ($E.2) applications. For ex- ample, in a task of segmenting aerial imagery into land cover classes, we often have coarse labels c associated to large blocks of pixels, but not the target labels E for individual pixels. If the conditional p([cc) is known, it sets a belief about the high-resolution labels E for pixels in a block of class c.
Coarse data in weakly supervised segmentation ($4.3
Fusing models and data sources ( 444 44.5). Auxiliary information may not always come with a known correspondence p((z)) In the land cover mapping problem, auxiliary information may include different modalities and resolutions (road maps, sparse point labels, etc.). While these sources can be fused into a prior by hand-coded rules, the prior may be more accurately set as the output of a model p((lz)) trained on separate dataset of points (4, This is especially useful when the data X; (e.g. satellite imagery) is more informative about the latents 4, but is prone to domain shift problems, while the auxiliary data zı does not suffer from domain shift issues but not sufficient on its own to predict the labels. In a text classification problem, might be the encoding of text x; by pretrained language model, and p(f] noisy distribution over labels given by their likelihoods under the language model as continuations of prompt.
Priors for self-supervision (E..). In 12.1 we discussed the pitfalls of using constant prior (((() = p(f) for all data points in training models under the QR loss as a potential method for un- supervised clustering. However, in I we give example of joint learning of the posterior model and an energy model (Markov random field) on the latent labels & that expresses local structure of labels in an image. This results in unsupervised clusterings that are useful in downstream segmen- tation tasks. Such an approach is an example of a benefit of generative modeling the possibility of learning of a parametrized distribution over latents being inherited by implicit generative models.
Priors with latent structure E.E.).. Implicit generative modeling allows building hicrarchical latent structure into the prior (another benefit of classical generative models), we demonstrate in $E.3on on a video segmentation task. The prior is an admixture of possible segmentations with a structure similar to [17], but using set of mask proposals p(G]lm) from Mask R-CNN model [II) indexed by a latent m. The prior is pf(()= = Σm P(C]Im))(m where (mm), a probabilistic selection of the masks for the admixture in the given frame, is estimated by minimizing the free energy.
4 EXPERIMENTS
4.1 PARTIAL LABELS IN MNIST AND CIFAR-10
In this illustrative experiment, we compare algorithms for learning with partial labels on two 10-way image classification datasets, MNIST and CIFAR-10. To each training example x,, we randomly assign a set N; of & negative labels, chosen from the 9 labels distinct from the ground truth. The prior (f(E) is set to be uniform over & & N; and 0 for e E N,. We vary & from 1 (one negative label per example) to 9 (one-hot prior, full supervision). The data of k negative labels carries - (1-k bits of label information if k 1, 22x less label information than in the fully supervised setting.
5

---

Under review as a conference paper at ICLR 2022
MNIST
CIFAR-10
tran accuracy
best BOOLEREY
taan BOCuracy
tent accuracy
1.
10
0.2
02
nogative looes exammpl
negative lobels exampe
Regotive label exomple
negative libel/ exampte
Step 10
step 20
step 40
step 80
step 160
step 320
step 640
step 1280
Figure 3: Above: Accuracies MNIST and CIFAR-IO classifiers trained with varying numbers of negative labels per example; the lighter variant of each color and marker shows the peak accuracy over 300 training epochs. (Average of 10 runs with standard error region.) Below: Confusion ma- trices of MNIST classifiers in the course of training on batches of 128 pairs of digits. The trajectory of convergence to the diagonal shows that uncertainty is first resolved for the digits 0 and 9, then and 8, etc.
For both datasets, the base model q is taken to be small convolutional network, with four layers of ReLU-activated 3 x convolutions with stride 2 and a linear map to the 10 output logits (-33k learnable parameters for MNIST, ~34k for CIFAR-10). We experiment with four training losses:
CE: cross-entropy between predictions ((]);; 0) and the prior (6).
NLL (union): negative logarithm of the sum of likelihoods assigned by q to labels in E & N,, equivalently, log Σ. (f 0), as done, e.g.. by 15 19
The QR and RQ losses.
The CE, NLL (union), and RQ losses are equivalent when k = 9, and the RQ and NLL (union) losses are equivalent when Σ, (C) is uniform over C, which approximately holds late in training.
All models are trained for 300 epochs on batches of 256 images with the Adam optimizer (0 and a learning rate of 10-4, After each epoch, we compute the accuracy of the predictor q on the ground truth labels the train and test sets. Fig. 3 shows the final accuracies, as well as the maximum accuracies over epochs, averaged over 10 choices of partial label sets and random initializations. Models trained on RQ loss perform best, with the greatest benefit over CE seen for very few negative labels. We hypothesize that the small advantage of RQ regularization in early training. Meanwhile, QR performs as well as CE for very uncertain priors at the peak epoch, but its predictions degenerate with longer training.
NLL (union) loss be attributed to
4.2 MULTIPLE--NSSANCEE SUPERVISION: LEARNING FROM RANKS
We train a CNN of the same architecture as in $4.1 on MNIST, but with the only supervision coming in the form of pairs of images in which it is known which image represents the greater digit. The training set of 60k images is divided into pairs that are fixed throughout the training procedure; each digit appears in exactly one pair. We optimize to match the predictor 4 with the implicit generative model's 's posterior (5) using the RQ loss. Fig. 3 shows the confusion matrices at initial iterations of training. The learned classifier has 97% accuracy on both training and testing sets, which means that from pairwise comparisons alone, we can group the digit images and place them in order.
4.3 LABEL SUPER-RESOLLNE RESOLUTION: CHESAPEAKE BAY LAND COVER MAPPING
We benchmark our method's performance on the Chesapeake Land Cover datase Im-resolution land cover dataset used previously for label super-resolution 27 42
large
It con-
https://1ila httpa//aaaaaaaeeaaaaaeee/eeeaaaa

---

Under review as a conference paper at ICLR 2022
Imagery
Prior (blurred NLCD) predictions
K) predictions
Labels
Chesapeake classes
Pennsy vania
Water
lrees and Forest
Grass and Herbsceous
Impervious Surface
New York
Figure 4: Predictions of models trained with QR loss on the NLCD-only prior in the Chesapeake region, shown on regions of 1000x 1000 pixels in Pennsylvania and 500x500 pixels in New York.
sists of several aligned data layers, including: NAIP (4-channel high-resolution aerial im- agery at about Im/px), NLCD (16-class, 30m-resolution coarse land cover labels), and high-resolution land cover labels (LC) in four classes. The task is to train high-resolution segmenta- tion models, in the four target classes, using only NI.CD labels as supervision. Although the NL.DD layer is at 30x lower resolution than the imagery and target labels and follows a different class scheme, the cooccurrence statistics of NLCD classes c and LC labels C are assumed to be known.
To form prior over land cover classes l at each pixel position, we map the NLCD classes to proba- bilities over the target LC classes using these known cooceurrence counts and apply a spatial blur to reduce low-resolution block artifacts (Fig. "Prior"), We then train small convolutional networks (receptive field 11 X 11) to predict high-resolution land cover from input imagery. We evaluate both the QR and RQ variants of approach on the two states that comprise the "Chesapeake North" test set: Pennsylvania (PA) and New York (NY), and the two states combined, after picking hy- perparameters based depiction of the data and prediction results is given in Figure 4
independent validation set in Delaware (details in Appendix D.1.3). A
Table compares our performance against the algorithmic technique with the best published per- formance on the Chesapake dataset, self epitoepiiomicc LSR (28) and the hard naïve baseline from (27]. Self-epitomic LSR, a generative modeling approach that explicitly produces likelihoods pxx((), an- alyzes small patches of data by making a large number of comparisons between sampled 7x7 image patches and all other image patches. It does produce a trained feedforward inference model, and the inference procedure is at least an order of magnitude slower than evaluation of our convolutional model. The hard na've baseline maps the NLCDD classes to LCC classes based given concurrence matrix, then trains standard semantic segmentation model on these pscudo-labels.
Not only does training on the QR loss achieve comparable performance with self-epitomic LSR (Table []. but the implicit generative model for p(x|c) from (2) is largely consistent with the epito- mie generative model (Fig. D.4). Moreover, our method handles batched input, where self-epitomic LSR trains on one tile at a time, and similar approaches have been shown to degrade in performance and exhaust computation capacity when training on multiple tiles (28)) Thus, optimization under an implied generative model has the computational ad advantage of scaling naturally to large training data while maintaining the benefits of leading generative modeling approaches. (See also (E.2)
4.4 DATA FUSION AND LEARNED PRIORS: ENVIROATI.AS
In this set of experiments, augment NLCD with information about the presence of buildings, road networks, water bodics, and waterways from public sources (see Fig. 5gad SD...I). To evaluate the ability of models to generalize new geographic regions, the geographically diverse EnviroAtlas dataset (39 in four cities in the US: Pittsburgh, PA, Durham, NC, Austin, TX, and Phoenix, AZ. The NLCD-basse prior model from $4.3 is augmented with the auxiliary information to obtain a hand-coded prior for each image (see $D.1.2) These types of priors can be made every in the United States, while hard labels are rarely available.
1m 5-class land cover labels from
The standard alternative to performing local inference under such priors is to simply apply super- vised models trained hard labels elsewhere, hoping that the domain shift is tolerable. Table

---

Under review as a conference paper at ICLR 2022
Table 1: Pixel accuracy and class mean intersection over union the Chesapeake Land Cover dataset. All models use only coarse NLCD labels as supervision. For our proposed methods, we evaluate both the trained predictor (qt) and the posterior under the implicit generative model (rz).
PA
NY
Chesapeake
Model
%
IoU %
%
IoU %
%
IoU
Self-epitomic LSR 228
86.2
67.6
86.4
70.5
86.3
69.7
Hard naive 27
85.3
63.0
83.6
59.8
83.6
59.7
QR (g)
84.2
66,6
86.2
71.0
84.6
67.7
QR (r)
85.0
68.1
86.9
72.4
85.4
69.4
RQ (g)
82.2
63.4
79.0
61.4
78.7
59.4
RQ (r)
82.3
63.5
79.0
61.6
78.8
59.7
M.OD
Learned
Labe's
Pitsorggh Pi
Whrter
Eancn
Trees and fures
Grass and Her boceous
Dunaan NC
Serubs (nst precirted)
Open Water
Deceloped intersilt
PuseeL TX
Developed inturiiiy
Developed High nitesity
Decidkiuu Porest
Evergroon Forest
Mixed Forest
Protndd 32
Y6thorbozcous
Resiurettayy
Woody Weslends
Freergeent Her bebbrrms Wellands
Figure 5: Prior generation process for land cover mapping: "NLCD only prior" as used in $4.3 and
"hand-coded prior" and 'learned prior" used in $4.4
compares the performance of a model (of the same architecture as in $4.3) trained on Pittsburgh high-resolution data (HR) in each of the three other cities with that of models tuned on the hand- coded prior in each other city. The QR method trained on the local handmade prior outperforms the HR model in each evaluation city. This may be attributed the extra data in each city given to our method in the form of prior beliefs. To isolate this effect, we also compare to a high-resolution model that consumes the prior belief to inpul data, concatenated with the NAIP imagery (HR + aux). While the HR + aux model does increase performance substantially from the HR model with NAIP imagery alone as input, the QR model remains the gighest-fidelity approach. These results illustrate that information that generalizes across domains may find its best use within separate model to build prior in our setting and then used to supervise local inference.
A prior belief could be crafted by domain expert to reflect the uniquities in geographic and struc- tural features for each city. We emulate incorporating such context-specific knowledge by training (on disjoint set of instances) neural network that consumes the inputs to the handmade prior function (NLCD and auxiliary map data), and predicts high-resolution labels (Figure 5 "Learned prior"). Alongside structural interactions between the inputs that generalize across cities (c.g. tree canopy supersedes rivers, road supersede water), the learned prior captures region-specine knowl- edge (e.g., buildings in Durham, tend be have grass surrounding them and trees farther out, while in Austin, this is reversed, and in Phoenix, riverbeds surrounded by barren land are likely to be dry). As shown in Table 2 these tailored prior beliefs tend increase scores.
The final row in Table benchmarks the performance imagery and labels over the entire contiguous US 42. This large model takes NAIP, Landsat 8 satellite imagery, and building footprints as inputs. Small, local models with priors created from only weak supervision outperform the US- wide model in all cities. (See D.1..f evaluation details.)
high-resolution land cover model trained
8

---

Under review as a conference paper at ICLR 2022
Table 2: Land cover classification experiments for generalizing across cities. In each column, the score of the best model not depending on auxiliary data as input is italicized. The score of the best overall model is bolded. (A larger set of experimental results is given in Table D.I)
Durham, NC
Austin, TX
Phoemix, AZ
Train region
Model
IoU
acc %
IoU %
acc
IoU %
Pittsburgh
HR HR + aux
72.9
34.8
71.9
36.8
6.7
7.5
(supervised)
78.9
47.7
77.2
50.6
66.0
27.9
Local
QR (q)
78.9
46.6
79.3
51.7
73.2
39.1
(hand-coded prior)
QR (r)
79.1
48.8
79.6
53.0
73.4
40.6
Local
QR (q)
78.7
47.4
79.8
51.1
39.6
(learned prior)
QR (r)
78.9
49.6
80.3
52.6
75.3
42.2
Full US 42
U-Net Large
77.4
48.8
76.4
51.4
24.5
23.2
Table 3: Fl-scores of various models on the coarsely supervised text classification task. The first five columns are taken from 32.. The last two columns use the GPT-2 prior defined in $4.5 as weak supervision with cross-entropy and RQ loss, respectively (mean of 10 random initializations).
pseudolaheling
GPT-2 prior, trigram features
WeSTClass 33
ConWen B1
LOTClass
34
X-Class [1
C2F 32
prior urgmax
CE
RQ
Micro-Fl %
76.23
73.96
15.00
91.16 81.09
92.62
86.33
87.18
93.18
Macro-F1 %
69,82
65.03
20.21
87.01
77.61
77.90
84.26
4.5 UNCERTAIN TEACHERS: TEXT CLASSIFICATION
This experiment follows the very recent work of 32 and illustrates the effectiveness of learning on prior beliefs beyond computer vision. We work with dataset of 12k New York Times news articles. Each article belongs to one of 20 fine categories (e.g., 'energy companies', 'tennis", 'golf"), which are grouped into coarse categories (e.g.. 'business', 'sports"). The goal is to train text classifiers that predict fine labels, but only the coarse label for each article is available in training.
Some external knowledge about the fine categories necessary to resolve the coarse labels into fine labels. Past work on this problem 33 31; 34 51] has trained supervised models on pseudolabels created by mechanisms such as propagation of seed words and querying large pretrained language models. On the other hand, [32) create training data by sampling featwres (articles) from finetuned version of the large generative language model GPT-2 [41] conditioned on fine categories, then tune classifier based the almost equally large model BERT 7 in supervised manner.
We obtain comparable results with an elementary predictor, far less computation, and no finetuning of massive language models. We form a prior (r(() on the fine class C of each article x; by querying GPT-2 for the likelihood of each fine category name l compatible with the known coarse label following the prompt "[article text] Topic: and normalizing over L. We then divide p(( by the mean likelihood of over all articles x; and renormalize. We represent each article as a vector of alphabetic trigram counts (263 features, of which only 8k ever nonzero) and train logistic regression with the RQ objective against the 'GPT-2 prior'. After ten epochs of training (~ on our hardware), the trained classifier nears exceeds the performance of models requiring least 100x longer to train, not to mention to generate pscudo-training data for (Table
5 CONCLUSIONS
We found that the generative distribution in a free energy criterion can be left implicit to the mini- mization process in posterior (diseriminative) model training. This allowed to unite the training of neural networks (C[x;; 0) for prediction of labels l from features x and the modeling of the prior ((),,, possibly with its own latent structure. Implicit modeling of the conditional generative dis- tributions removes the burden of training accurate (and therefore large or deep) generative models, but allows natural generative approaches modeling priors. We expect interesting new modeling paradigms that integrate different modalities and models to arise from this formulation.

---

Under review as a conference paper at ICLR 2022
REPRODUCIBILITY STATEMENT
This submission comes with two code directories that illustrate our algorithms for partial-label learn- ing and weakly supervised segmentation. The provided code is sufficient to reproduce results sembling those in Fig. Upon publication, we intend to release code for all land cover mapping experiments that compatible with the TorchGeo library.
ETHICS STATEMENT
The authors anticipate that the immediate consequences arising from this work will have nonnegative social impacts. In general, learning from weak, coarse, or imprecise data may be susceptible to data biases in differential quality or precision data sources across sub-populations or geographies. We encourage further work to examine these issues, especially in cases where uncertain labels used in machine learning systems that impact people or decision-making systems. Overall, we hope that the ability to learn from variety of weak data sources can aid applicability of machine learning systems in diverse settings.
REFERENCES
[1] Qianyue Bao, Yang Liu, Zixiao Zhang, Dafan Chen, Yuting Yang, Licheng Jiao, and Fang Liu. Mrta: Multi-resolution training algorithm for multitemporal semantic change detection. International Geoscience and Remote Sensing Symposium (IGARSS), 2021.
[2] Geoff Boeing. Osmnx: New methods for acquiring, constructing, analyzing. and visualizing complex street networks. Computers, Environment and Urban Systems, 65:126-139, 2017.
[3] Caelles, K.K. Maninis, Pont-Tuset, L. Leal-Taixé, D. Cremers, and L. Van Gool. One-shot video object segmentation. Computer Vision and Pattern Recognition (CVPR), 2017.
[4] Ho Kei Cheng, Yu-Wing Tai, and Chi-Keung Tang. Rethinking space-time networks with improved memory coverage for efficient video object segmentation. Neural Information Pro- cessing Systems (NeurIPS), 2021.
[5] J. Cheng, Y.-H. Tsai, W.-C. Hung, S. Wang, and M.-H. Yang. Fast and accurate online video object segmentation via tracking parts. Computer Vision and Puttern Recognition (CVPR), 2018.
[6] A. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society B, 19(1))1-33, 1977.
[7] Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-traine tre-training of deep bidirectional transformers for language understanding. North American Chapter of the Association for Computational Linguistics (NAACL). 2019.
[8] Ian J. Goodfellow, Jean Couget-Abadic Mehdi Mirza, Bing Xu, David Warde-Farley. Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Neural Information Processing Systems (NeurIPS), 2014.
[9] Mordechai Haklay and Patrick Weber. OpenStreetMap: User-generated street maps. IEEE Pervasive Computing. 7(4): 2008.
[10] Junwei Han, Dingwen Zhang, Gong Cheng, Lei Guo, and Jinchang Ren. Object detection in optical remote sensing images based on weakly supervised learning and high-level feature learning. IEEE Transactions on Geoscience and Remote Sensing, 53(6):332533377 2014.
[11] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask R-CNN. International Conference Computer Vision (ICCV), 2017.
[12] Geoffrey F. Hinton, Peter Dayan, Brendan J. Frey, and R M Neal. The "wake-sleep" algorithm for unsupervised neural networks. Science, 268 1114:158-61, 1995.
10

---

Under review as a conference paper at ICLR 2022
[13] Le Hou, Vu Nguyen, Ariel B Kanevsky, Dimitris Samaras, Tahsin M Kure, Tianhao Zhao, Ra- jarsi R Gupta, Yi Gao, Wenjin Chen, David Foran, et al. Sparse autoencoder for unsupervised nucleus detection and representation in histopathology images. Pattern Recognition, 2019.
[14] Neal Jean, Sherrie Wang, Anshul Samar, George Azzari, David Lobell, and Stefano Ermon. Tile2vec: Unsupervised representation learning for spatially distributed data. Association for the Advancement of Artificial Intelligence (AAAI), 2019.
[15] Rong Jin and Zoubin Ghahramani. Learning with multiple labels. Neural Information Pro- cessing Systems (NeurIPS), 2002.
[16] Joakim Johnander, Martin Danelljan, Emil Brissman, Fahad Shahbaz Khan, and Michael Fels- berg. A generative appearance model for end-to-end video object segmentation. Computer Vision and Puttern Recognition (CVPR), 2019.
[17] Nehojsa Jojic, Alessandro Perina, Marco Cristani, Vittorio Murino, and Brendan Frey. Stel component analysis: Modeling spatial correlations in image class structure. In 2009 IEEE conference on computer vision and pattern recognition, pp. 2044-2051. IEEE, 2009.
[18] A. Khoreva, R. Benenson, E. 1lg, T. Brox, and B. Schiele. Lucid data dreaming for object tracking. The 2017 DAVIS Challenge on Video Object Segmentation CVPR Workshops, 2017.
[19] Youngdong Kim, Junho Yim, Juseung Yun, and Junmo Kim. NLNL: Negative learning for noisy labels. International Conference on Computer Vision (ICCV), 2019.
[20] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.6980, 2014.
[21] Diederik P. Kingma and Max Welling, Auto-encoding variational bayes. International Con- ference on Learning Representations (ICLR), 2014.
[22] Zhuohong Li, Fangxiao Lu, Hongyan Zhang. Guangyi Yang, and Liangpei Zhang. Change cross-detection based on label improvements and multi-model fusion for multi-temporal re- mote sensing images. International Geoscience and Remote Sensing Symposium (IGARSS), 2021.
[23] Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pictro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO: common objects in context. Euro- pean Conference on Computer Vision (ECCV), 2014.
[24] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint 17X 2017.
[25] Jonathon Luiten, Paul Voigtlaender, and Bastian Leibe. Premvos: Proposal-generatio re- finement and merging for video object segmentation. Asian Conference on Computer Vision (ACCV), 2018
[26] Oisin Mac Aodha, Elijah Cole, and Pietro Perona. Presence-only geographical priors for fine- grained image classification. International Conference Computer Vision (ICCV), 2019.
[27] Nikolay Malkin, Caleb Robinson, Le Hou, Rachel Soobitsky, Jacob Czawlytko, Dimitris Samaras, Joel Saltz, Lucas Joppa, and Nebojsa Jojic. Label super-resolution networks. In- ternational Conference on Learning Representations (ICLR), 2019.
[28] Nikolay Malkin, Anthony Ortiz, and Nebojsa Jojic. Mining self-similarity: Label super- resolution with epitomic representations. European Conference on Computer Vision (ECCV), 2020.
[29] Kevis-Kokitsi Maninis, Sergi Caelles, Yuhua Chen, Jordi Pont-Tuset, Laura Leal-Taixé, Daniel Cremers, and Luc Van Gool. Video object segmentation without temporal information. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2018.
[30] Tim Meinhardt and Laura Leal-Taixe. Make one-shot video object segmentation efficient again. Neural Information Processing Systems (NeurIPS), 2020.
11

---

Under review as a conference paper at ICLR 2022
[31] Dheeraj Mekala and Jingbo Shang. Contextualized weak supervision for text classification. Association for Computational Linguistics (ACL), 2020.
[32] Dheeraj Mekala, Varun Gangal, and Jingbo Shang. Coarse2Fine: Fine-grained text classifica- tion on coarsely-grained annotated data. Empirical Methods in Natural Language Processing (EMNLP), 2021.
[33] Yu Meng, Jiaming Shen, Chao Zhang, and Jiawei Han. Weakly-supervised neural text classi- fication. International Conference on Information and Knowledge Management, 2018.
[34] Yu Meng, Yunyi Zhang, Jiaxin Huang, Chenyan Xiong, Heng Ji, Chao Zhang, and Jiawei Han. Text classification using label names only: A language model self-training approach. Empirical Methods in Natural Language Processing (EMNLP), 2020.
[35] Nam Nguyen and Rich Caruana. Classification with partial labels. Knowledge Discovery and Data Mining (KDD), 2008.
[36] Scoung Wug Oh, Joon-Young Young Lee, Ning Xu, and Seon Joo Kim. Video object segmentation using space-time memory networks. International Conference on Computer Vision (ICCV), 2019.
[37] Perazzi, J. Pont-Tuset, B. McWilliams, A benchmark dataset and evaluation methodology for video object segmentation. Computer Vision and Puttern Recognition (CVPR), 2016.
L. Van Gool, M. Gross, and A. Sorkine-Horunng
[38] Federico Perazzi, Anna Khoreva, Rodrigo Benenson, Bernt Schiele, and Alexander Sorkine- Hornung. Learning video object segmentation from static images. Computer Vision and Pat- tern Recognition (CVPR), 2017.
[39] Brian R Pickard, Jessica Daniel, Megan Mehaffey, Laura E Jackson, and Anne Neale. Envi- roAtlas: A new geospatial tool foster ecosystem services science and resource management. Ecosystem Services, 14:45-55, 2015.
[40] Andrew Pilant, Keith Endres, Daniel Rosenbaum, and Gillian Gundersen. US EPA EnviroAtlas meter-scale urban land cover (MULC): 1-m pixel land cover class definitions and guidance. Remote Sensing, 12(12), 2020.
[41] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. 2019.
[42] Caleb Robinson, Le Hou, Nikolay Malkin, Rachel Soobitsky, Jacob Czawlytko, Bistra Dilkina, and Nebojsa Jojic. Large scale high-resolution land cover mapping with multi-resolution data. Computer Vision and Pattern Recognition (CVPR), 2019.
[43] Caleb Robinson, Anthony Ortiz, Nikolay Malkin, Blake Elias, Andi Peng, Dan Morris, Bis- tra Dilkina, and Nebojsa Jojic. Human-machine collaboration for fast land cover mapping. Association for the Advancement of Artificial Intelligence (AAAI), 2020.
[44] Matthias Seeger. Learning with Labeled and Unlabeled Data. 2002.
URL https
pdf
[45] Balaji Lakshminarayanan Shakir Mohamed. Learning in implicit generative models. Interna- tional Conference on Learning Representations (ICLR), 2017.
[46] Ravid Shwartz-Ziv and Naftali Tishby. Opening the black box of deep neural networks via information. arXiv preprint arXiv: 1703:0/003.08/000 2017.
[47] Matthew Tancik, Pratul P. Srinivasan, Ben Mildenhall, Sara Kridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T. Barron, and Ren Ng. Fourier features let networks leam high frequeney functions in low dimensional domains. Neural Information Processing Systems (NeurIPS), 2020.
[48] TorchGeo contributors. TorchGeo.
2021.
12

---

Under review as a conference paper at ICLR 2022
[49] Paul Voigtlaender and Bastian Leibe. Online adaptation of convolutional neural networks for video object segmentation. British Machine Vision Conference (BVMC), 2017.
[50] Paul Voigtlaender, Yuning Chai, Florian Schroff, Hartwig Adam, Bastian Leibe, and Liang- Chieh Chen. FEELVOS: Fast end-to-end embedding learning for video object segmentation. Conference on Computer Vision and Pultern Recognition (CVPR), 2019.
[51] Zihan Wang, Dheeraj Mekala, and Jingbo Shang. X-Class: Text classification with extremely weak supervision. North American Chapter of the Association for Computational Linguisties (NAACL), 2021.
[52] Linjie Yang, Yanran Wang, Xuehan Xiong, Jianchao Yang, and Aggelos K. Katsaggelos. Effi- cient video object segmentation via network modulation. Computer Vision and Pattern Recog- nition (CVPR), 2018.
[53] Zongxin Yang, Yunchao Wei, and Yi Yang. Collaborative video object segmentation by integration. European Conference on Computer Vision (ECCV), 2020.
[54] Yao Yao, Jiehui Deng, Xiuhua Chen, Chen Gong, Jianxin Wu, and Jian Yang. Deep discrimina- tive CNN with temporal ensembling for ambiguously-labeled image classification. Association for the Advancement of Artificial Intelligence (AAAZ). 2020.
[55] Fei Yu and Min-Ling Zhang. Maximum margin partial label learning. In Asian conference on machine learning. pp. 96- 16-111. PMLR, 2016.
[56] Xiao Zhang, Yixiao Ge, Qiao, and Hongsheng Li. Refining pseudo labels with clustering consensus over generations for unsupervised object re-identification. Computer Vision and Puttern Recognition (CVPR), 2021.
[57] Zhuo Zheng, Yinhe Liu, Shiqi Tian, Junjue Wang, Ailong Ma, and Yanfei Zhong. Weakly su- pervised semantic change detection via label refinement framework. International Geoscience and Remote Sensing Symposium (IGARSS), 2021.
[58] Naiyun Zhou, Xiaxia Yu, Tianhao Zhao, Si Wen, Fusheng Wang, Wei Zhu, Tahsin Kurc, Allen Tannenbaum, Joel Saltz, and Yi Gao. Evaluation of nucleus segmentation in digital pathology images through large scale image synthesis. In Medical Imaging 2017: Digital Pathology, volume 10140. International Society for Optics and Photonics, 2017.
[59] Yuliang Zou, Zizhao Zhang, Han Zhang, Chun-Liang Li, Xiao Bian, Jia-Bin Huang, and Tomas Pfister. Pseudoseg: Designing pseudo labels for semantic segmentation. arXiv preprint a:Xiv:2010.09713, 2020.
13

---

Under review as a conference paper at ICLR 2022
A PRACTICAL CONSIDERATIONS
Figure shows PyTorch implementation of the QR and RQ loss functions, where loss is computed over batches of training data. Our experiments validate that so long as these batches are large enough to include enough diversity of (xi, (i)) pairs, our method works when eq. (2 and eq. (3) are applied directly to batches. As discussed in $4.44 handling batched input is important for leveraging the scale of large training datasets. As discussed in the caption of Figure [ should mini-batch training become an issuc in future implementations, it may be beneficial to estimate the
Mini-batches:
denominator of eq. 2 across multiple batches.
Relative benefits/limiatiios of the QR and RQ loss formulations: The algorithm presented in [2.1 details two loss options: a QR option and an RQ option, both with unique strengths. The QR algorithm is guaranteed converge as each step reduces loss (except for randomness in the learning algorithm). The RQ algorithm, on the other hand, has the appealing property that it reduces to standard minimization of cross entropy loss in the case of hard labels. In $C we discuss connections between QR option and variational auto-encoders (VAEs), and between the RQ option and the wake- sleep algorithm. Ultimately, though, we find that which option works better may depend on the application, with RQ working across all applications we tried but sometimes being slightly beaten by QR.
Simple ways to avoid degenerate solutions As discussed in 2.1 minimizing eq. ① can lead to degenerate solutions. However, avoiding these solutions can be quite simple, and in most of our experiments we did not make any interventions to explicitly avoid such local minima. In targeted experiment in table D.I we show that pre pre-training on hard labels (even out-of-domain) or using sharper learned priors can help break symmetries during carly training phases. When hard labels not available, belief, and then switch to RQ or QR loss. The intuition that first training to minimize cross-entropy breaks the symmetry at the start, while implicit generative modeling sharpens the predictions in later iterations.
could similarly start the training process with a cross-entropy loss on the prior
B ADDITIONAL RELATED WORK
There are several approaches learning with uncertain, weak, coarse labels under different as- sumptions and settings. Work on partial-label learning often employs loss functions that aim to decrease prediction entropy B5 54 55 In contrast to our work, these approaches do not use a generative formulation in these loss functions, makes them ill-suited to problems with more var- ied forms of uncertainty encoded priors. Another line of work studies the generation and use of pseudo labels in learning settings. Specifically, [59] relies on domain-specific augmentation pro- cedure for semantic segmentation with image-level labels, and, (56) studies unsupervised clustering Application-specine solutions also include object detection in remote sensing images [0] and change detection with temporal satellite imagery [57 22).
applied to object re-identification.
In our experimental setups, chose mix of baselines to compare algorithm design and benchmark performance on certain tasks.
algorithmic basis, we compare to the negative logarithm of the sum of likelihoods (NLL), which is used in prior works to handle multiple ambiguous labels [15] and negative labels 19). As we discuss in $4.1 NLL and RQ are equivalent when Σ, 4;({) is uniform over c, evidenced by the comparable performance between the two in Figure 3 We compare to self- epitomic LSR (28) an algorithmic comparison by which to contrast method with an "explicit" generative modeling approach. Our similar performance to self-epitomic LSR in regimes where self-epitomic LSR has been shown to perform well (super-resolution in land cover mapping ($4.3) and the tumor-infiltratin lymphocytes task ( (E.2) is an important validation of our motivation in
To compare approach
$2
To benchmark performance of our approach across tasks, we compare to state-of-the -tate-o-thhe-rtt pseudo- labeling methods in supervised text classification (see $4.5), an established Im resolution map of land cover predictions across the United States [42] and best-performing published results for the land cover mapping tasks we study 28
43 the best known published results for the tumor-
14

---

Under review as a conference paper at ICLR 2022
infiltrating lymphocyte segmentation task ((27) (8) and host of comparisons for the video instance segmentation task (see Table E.3 for a full list).
Lastly, it is worth noting that the term "implicit generative model" has also been used in prior literature to refer to amortized sampling procedures for nonparametric (or not specified) energy functions, such generative adversarial models (e.g.. [5]). Although we do not make explicit connection with such models, our formulation also does not assume a parametrization of the data distribution. However, we assume tractability of sampling from posterior certain distinguished latents (classes) conditioned on observed data (features, e.g., images), rather than directly sampling latents.
C RELATIONSHIPS WITH EM, VAES, AND WAKE-SLEEP ALGORITHM
As discussed in $ 2. the QR loss guarantees continual improvements in the free energy ( On the other hand, option RQ is equivalent to performing a gradient step on the cross-entropy of qi and ra and gradient step the negative entropy of In the case that the priors pi(() are hard (supported only on one ground truth label), the same true of r, and the RQ loss is equivalent to cross- cross-entropy. This option reverses the KL distance in manner reminiscent of the training procedure in the wake- sleep algorithm [12], where parameter updates for the forward and reverse models are iterated, but the KL distance optimized always places the probabilities under the model being optimized in the second position in the KI. distance (inside the logarithm), so that the generative and the inference models each optimize log-likelihoods of their predictions. The wake-sleep algorithm, however, also trains a generative model rather than treating it as an auxiliary distribution like we do, and that requires sampling. As opposed VAEs, the wake-sleep algorithm samples the generative model, not the posterior.
It is interesting to contrast our approach to the EM formulation. In standard EM, the q distributions are considered auxiliary, rather than parametrized as direct functions of the inputs x. The (i({) is simply matrix of numbers normalized across f. Its dependence iterative re-estimation of the minimum of the free energy. The link x C is modeled explicitly in the parametrized forward distribution pxx((). We instead treat forward probabilities p(xy(() as auxiliary parameters, a matrix of numbers Pi.e normalized across i that we fit to minimize the free energy at each data point, and optimize only the parameters of the q model which explicitly models the link x-f. This then allows us to capture nonlinear (and 'deep') ) structure and benefit from inductive biases inherent to training deep models with SGD, but without the cost of training an actual parametrized generative model and other problems associated with deep generative model fitting. The resulting y network approximates the posterior in a generative model which (locally) maximizes the log likelihood of the data and it is usually highly confident (as seen in Fig.
the data is only implicit in the
The implicit modeling of the posterior in EM does not lead over-fitiing of the generative model. But, given that degenerate solutions to optimization with implicit generative models are possible when the prior is constant across all data points ($2. we can imagine that our approach of im- plicit generative modeling might lead to degenerate solutions. As demonstrated in Fig. [ and in our experiments, avoiding degenerate solutions not too hard. We attribute this to two factors. First, the situations of interest typically involve uncertain, but varying distributions (,(E) which break symmetries that could lead to ignoring the data features X; as in the example above of a constant prior. Second, the neural networks to be used posterior models q come with their own constraints in parametrization, and, equally importantly, the inductive biases that come as a consequence of gradient descent learning in nonlincar, multilayer networks.
D EXPERIMENT DETAILS
D.1 LAND COVER MAPPING
D.1.1 DATASETS
Imagery Data Our land cover mapping experiments imagery from the National Agriculture
Imagery Program (NAIP), which is 4-channel aerial imagery at a S Im/px resolution taken in the United States (US).
15

---

Under review as a conference paper at ICLR 2022
Chesapeake Conservancy land cover dataset The Chesapeake Conservancy land cover dataset consists of several raster layers of both imagery and labels covering parts of 6 states in the Northeast- United States: Maryland, Delaware, Virginia, West Virginia, Pennsylvania, and New York [42 The raster layers include: high resolution ( (Im/px) NAIP imagery, high resolution (1m/px) land cover labels created semi-autonomously by the Chesapeake Conservancy, low resolution (30m/px) Landsat-8 mosaics imagery, low resolution (30m/px) land cover labels from the National Land Cover Database (NLCD), and building footprint masks from the Microsoft Building Footprint dataset. The dataset is partitioned into train, validation, and test splits per-state, where each split is a set of 7km x 6km tiles containing the aligned raster layers.
EPA EnviroAtlas data The EnviroAtlas land cover data consists of high resolution (Im/px) land cover maps over 30 cities in the US. and collected and hosted by the US Environmental Protection Agency (EPA) [3 A detailed description of the dataset and its land cover definitions is provided by 40 As with most high-resolution land cover datasets (including the Chesapeake Conservancy land cover labels), the EnviroAtlas land cover labels are themselves derived by remote sensing and learning procedures, and thus are not themselves a perfect "ground truth" representation of land cover. For example, the estimated accuracy of the provided labels is 86.5% in Pittsburgh, PA, 83.0% in Durham, NC, 86.5% Austin, TX, and 69.2% in Phoenix, AZ
The high-resolution label files were aligned to match the extent of the NAIP tiles from the clos- est available years to the years that the EnviroAtlas labels were collected: for Pittsburgh, PA and Phoenix, AZ, we used data from 2010 and for Durham, NC and Austin, TX, we used data from 2012. We chose these four cities to get a wide coverage across the United States (US), and due to a mostly consistent set of classes being used between the four cities.
National Land Cover Database (NLCD) The National Land Cover Database is produced by the United States Geological Survey (USGS) and uses 16 land cover classes. Maps are generated every 2-3 years, with spatial resolution of 30m/px. Data and more information can be found at: https: 7/www.
Microsoft Building Footprint dataset The Microsoft Building Footprint dataset consists of pre- dicted building polygons over the continental US from Bing Maps imagery. As of the time of writing, the most updated Microsoft Building Footprints dataset in the US can be accessed at: https: bttps://github. www////irroooo///SSS......o htt.a//wwww.......ooooo////aaaaa
Open Street Map (OSM) data Open Street Map (https: www. oopenstreetnnn org/) an ongoing effort to make publicly available and editable map of the world, generated largely from volunteer efforts. The data is available under the Open Database License. From the many different sources of information provided by OSM (. we download raster data for road networks, waterways, and water bodies, using the OSM python package (2].
Data splits and data processing For experiments using the Chesapeake Conservancy dataset (Ta- [I) used established train, test, and validation splits. In particular, used the 20 test tiles in New York (NY) and the 20 test tiles in Pennsylvania on which to conduct our experiments. Here a tile matches the extent of NAIP tile, roughly 7km x 6km. To facilitate comparison of our results with previous published results on this dataset, we condensed the labels into four classes: (1) water, (2) impervious surfaces (roads, buildings, barren land), (3) grass/field, and (4) tree canopy.
For experiments with the EnviroAtlas dataset (Table 2) we aligned the high resolution land cover data, NLCD, OSM, and Microsoft Building Footprints data with NAIP imagery tiles. matching years as closely as possible to the EnviroAtlas data collection year for NLCD and NAIP. We instantiated a split of 10 train, 8 validation, and 10 test tiles in Pittsburgh, and 10 test tiles in Durham, NC, Austin, TX, and Phoenix, AZ. For Pittsburgh we assigned tiles to splits randomly from the set of 28 tiles that had no missing labels. There were not enough such tiles in Durham to follow the same procedure, so we chose the ten evaluation tiles at random from a set with no number of missing labels per tile. For Austin and Phoenix, we chose the 10 evaluation tiles random from the tiles each city that had no agriculture class (as it is not present in Pittsburgh or Durham) and no missing labels.
Dataset can
be
downloaded
from:
:tttss://1100 a..ila.aii sc wwciienneeaaeo datasets/
chesapeake andcover
16

---

Under review as a conference paper at ICLR 2022
Chesapeake Conservancy dataset
Enviroatlas
DE
Pittsburgh, PA
Mete
Vider
epeairted hed
Thems Phrme
PA
Durham, NC
Meter
Tharforer-
Austin, TX
Damen
operiis
Trees and warmo
Ohrelient Con resentty
Caviooed Mediin retentr
Upve Fowet
Phoenix. AZ
Cartoce
NLCD Class
Onen aochone
Covioped of
conaiiin Vigi
color
NLCD Class
Figure D.1: Cooccurrence matrices between NLCD classes and high resolution land cover labels for each region we study.
We set aside separate tiles in each city for use in "learning the prior" (in Pittsburgh these 5 tiles are a subset of the 8 validation tiles). As above, each tile corresponds one NAIP tile. The tiles in these constructed sets for Pittsburgh, Durham, and Austin contain five unique labels: (1) water, (2) impervious surfaces (roads, buildings). (2) barren land, (4) grass/field, and (5) trees. Phoenix additionally has "shrub" class; when forming the prior merge this class with trees, and we ignore shrub class when evaluating in Phoenix. We cropped all data tiles to ensure no spatial overlap in any tiles between within the train/val/test splits.
1.1.2 FORMING THE PRIORS
To form the priors for the land cover classification tasks, we first spatially smooth the NLCD labels by applying 2D Gaussian filter (with standard deviation 31 pixels) across every channel in a one- hot representation of the NLCD classes. The main reason for applying this smoothing is to reduce artifacts due to the 30m boundaries of the NLCD data: to undo the blocking procedure induced by the aggregation to 30m 30m extents, to incorporate the spatial correlations between nearby NLCD blocks, and to remove erroneous sharp differentials between inputs that can cause artifacts during later training stages.
We then remap the blurred NLCD layers to the classes of interest by multiplying by a matrix of co- occurrence counts between the (unblurred) NLCD data and the high resolution labels in each region. For the Chesapeake region, we use the train tiles provided with the Chesapeake Conservancy land compute
cover dataset to define co-occurrence matrices in NY and PA. For EnviroAtlas, occurrences using the entire city (excluding tiles with agriculture in Phoenix AZ, and Austin, TX). The co-occurrence matrices for each region we study
shown in Figure D.I
The priors for the Chesapeake Conservancy dataset are then generated by normalizing the blurred and remapped NLCD data that summing over all five classes gives probability 1 for each pixel.
17

---

Under review as a conference paper at ICLR 2022
For the EnviroAtlas data, augment this prior with publicly available data on buildings, road networks, water bodies, and waterways. We obtain building maps from the Microsoft Buildings Footprint database and road, water bodies, and waterways data from Open Street Map. using the OSMix tool [2] to download the data (see Appendix [...I). We apply a small spatial blur to each of these input sources to account for (a) vector representation of roads and waterways being unre- alistically thin, and (b) possible data-image misalignment on the order of pixels. Where this results in probability mass impervious surfaces or water, add these probability masses to the blurred NLCD prior, and then renormalize to obtain a valid set of probabilities for each pixel.
In $4.4 we describe method for "learning the prior," which uses more sophisticated process to aggregate the individually weak and coarse inputs that we use in the handmade prior. In this method, we train neural net to take input the blurred, remapped NLCD representation (5 classes) concatenated with the 4 classes of additional data: buildings, roads, waterways, water bodies, and to predict high-resolution labels in each city. We train these networks using tiles of imagery and high-resolution labels from the EnviroAtlas Dataset in each city which are distinct from the 10 test tiles in each city. The training procedure for these prior generation networks is described in in $D.1.3 To create the priors that we then train our method on ( ('learned prior' rows in Table 2 we ran these learned models forward on (blurred and remapped NLCD, buildings, roads, waterways, and waterbodies) input for each of the 10 evaluation tiles in each city.
D.1.3 EXPERIMENTAL PROCEDURE
We use priors generated as described in Appendix D.1.2 with Gaussian spatial smoothing standard deviation of 31 pixels, and co-occurrence matrix determined via the training splits in each city/state. We apply an additive smoothing constant of lc-4 to that is applied pixel-wise the the probability vectors output by the neural network as well as to the prior probability vectors used as the model su- pervision data. This additive smoothing constant ensures that there are no extremely low probability classes in either the prior or the predicted outputs during training.
Experiments summarized in Table I and Table 2 use a 5-layer fully connected network with kernel sizes of at each layer, 128 filters per layer, and leaky ReLUs between layers. Note that the receptive field of this model is only 11 x 11 pixels. We use batch sizes of 128 instances during training, where each image instance is a cropped 128 x 128 pixels from a larger tile. Training is and model evaluation is done within the torchgeo framework for geo-spatial machine learning 48] All models use the AdamW optimizer 24 during training and torchgeo defaults unless otherwise noted.
Comparison to previous label super-resolution for L.C mapping To obtain the parameter set- ting used for the runs in New York (NY) and Pennsylvania (PA) in Table [ we first perform a hyperparameter search with the 20 tiles test set in Delaware (DE) from the same overall dataset. We use a learning rate schedule that decreases learning rate when the validation loss platcaus, as well as early stopping to prevent over training of models. Of the grid of learning rates in [le-3, le-5, we describe below, we pick learning rate as le-4 for both QR and RQ variants of our method, as this is the setting that minimizes the IoU of the q output on the 20 DE tiles for both variants.
When training on NY and PA jointly ("Chesapeake" in Table D. we use the per-state co-occurrence matrices. This ensure that the co-occurrence matrices used are consistent between our method and the self-epitomic LSR benchmark across all columns in Table
Generalization across cities. For the high-resolution model with NAIP imagery from Pittsburgh as input, we consider learning rates in 2 10-3, 1,10-,1000 10-5) and pick based on the best validation performance on the validation set in Pittsburgh. The chosen learning rate is 1e-3. We search over the same set of learning rates for the model with NAIP imagery and the prior concatenated as input; the chosen learning rate is also Ic-3. For this model with concatenated image and prior as input, only the number of input channels changes in the fully connected network model architecture. When training on the high-resolution land cover labels, we use a very small additive constant 1e-8 for the last layer of the model.
When training our method, we initialize model weights using the best NAIP image input model from the Pittsburgh validation set runs, and then train using the priors and the training procedure described in the main text. We pick the learning rate for this training step using again the validation set in Pittsburgh; search learning rates in (10-3, 10-4, 10-5), and pick le-5 the learning rate, since
18

---

Under review as a conference paper at ICLR 2022
resulted in the best performance for both QR and RQ in the Pittsburgh validation set. We discuss the results of a similar procedure using randomly initialized model weights in Appendix D.1.4
For the learned prior, 128 filters per layer and leaky ReLUs between layers. For each city, we train this model on the prior inputs (blurred and remapped NLCD, roads, buildings, waterways, and water bodies) using a validation set of 5 tiles separate from from the 10 evaluation tiles each city. We considered learning rates in {10-3, 10 10-55 for learning the prior in each city, and chose le-4 as it gave most often resulted in the highest accuracies of each validation set. For learning on this learned prior, we searched in condensed space of learning rate in (000",, 10-5), and chose le-5 for evaluation runs in each city based on validation set performance in Pittsburgh, PA.
layer fully connected network is kernel sizes of 11,7, respectively,
Table D.I: Supplementary results to accompany Table 2
Pittsburgh, PA
Durham, NC
Austin, TX
Phoenix, AZ
Train region
Model
acc %
ToU
acc %
ToU
acc
IaU
acc
ToU
Pittshurgh
HR
89.55
70.03
72.87
34.83
71.86
36.81
6.69
7.49
(supervised)
HR + aux
89.48
70.75
78.86
47.67
77.20
50.62
66.04
27.92
Same
QR (g)
80.46
56.81
77.77
43.92
79.59
$1.93
72.20
21.86
(random
QR (r)
80.62
57.44
77.91
44.86
79.59
52.30
72.39
22.32
initialization)
RQ (q)
51.74
10.35
70.63
26.83
74.32
45.99
66.96
19.16
RQ (r)
2.95
0.59
70.63
26.83
74.33
46.00
66.96
19.16
Same as test
QR (q)
85.41
64.71
78.90
46.56
79.28
$1.67
73.21
39.09
(pretrained
QR (r)
85.38
65.74
79.09
48.80
79.63
53.01
73.44
40.62
in Pittsburgh)
RQ (g)
87.35
67.88
78.80
44.43
78.65
$1.90
70.59
34.94
RQ (r)
87.45
68.59
78.83
44.61
78.79
52.47
70.70
36.10
Same test
QR (q)
87.21
68.54
78.74 78.74
47.35
79.83
51.10
74.57
39.59
(learned prior)
QR (r)
86.97
69.27
78.93
49.57
80.27
52.60
75.31
42.22
Full US* [42
U-Net Lrg.
79.51
61.62
77.35
48.78
76.40
$1.35
24.49
23.21
1.1.4
ADDITIONAL RESULTS
Comparison of the Full US* U-Net Large [42 map predictions when evaluated the full classes considered in Table 2 (water, grass/field, trees/shrub, impervious surfaces, and barren land) and evaluated on the four prediction classes predicted by the model (where barren land and impervious surfaces are merged as single class), and when barren is post-facto assigned whenever the predicted class is "impervious surfaces" and the label class is "barren land".
Table D.2:
Pittsburgh, PA
Durham, NC
Austin, TX
Phoenix, AZ.
Classication Scheme
acc %
IoU %
acc %
IoU %
acc %
IoU %
acc %
IoU %
5 Classes
79.29
55.01
76.86
42.62
76.05
48.80
18.06
18.49
4 Classes
79.29
68.76
77.35
53.13
76.40
59.84
24.49
16.33
Barren reassigned
79.51
61.62
77.35
48.78
76.40
51.35
24.49
23.21
Extended results for generalizing across EnviroAthe Alas cities. The extended results for generaliz- ing across cities with the EnviroAtlas datasets in Table D. contaii the results of the RQ runs trained on the handmade prior in each city, and additionally shows the evaluation results in Pittsburgh, PA, give further context full comparison of generalization across cities by each method.
Table D.I also details the result of initializing the model weights randomly for the QR method. again search over learning rate in [le-3,1e-4,1e-5] in Pittsburgh, this time
For this experiment, resulting in a choice of le-4 for the QR evaluation runs and le-3 for the RQ evaluation runs. This seems to be suboptimal choice of learning rate for the RQ method in other cities (see Table 2), so the most fruitfull comparison between initialization schemes is between the QR variant. Table shows that the choice of model initialization can be important for method this is most apparent
19

---

Under review as a conference paper at ICLR 2022
model predissons
Hand-ooded
Prlo) predistions
fi model
EnviroAtias classes
Water
Faybuigh
Impervious Surface
Soll and Barren
Trees and forest
Herbacoous
Dumhm MC
Shrubs (not prodicted)
Phooente
Figure D.2: Example predictions on the and learned prior in each EnviroAtlas city we study.
in Pittsburgh, PA (unsurprisingly since the high-resolution model was trained in Pittsburgh) and Phoenix, AZ. In Phoenix, much of the handmade prior is consistent across geographies and the randomly initialized model has trouble distinguishing between infrequent classes that most often occur together in the handmade prior. The results in Table D. suggest that using pre-trained models as starting point for our method can help break some these symmetry issues resolving the information in the prior. Results in Table suggest that using more detailed prior map may help with this as well.
Evaluating the Full US map from [42). Recall that the row for the full US Map 42 in Table reflects the performance of the model evaluated on all 5 classes we consider in our experiments, where we give the map predictions the "benefit of the doubt" in that any prediction of "impervious surfaces" where the true label is "barren land" gets assigned correct classification "barren land." The results reported in Table 2 are thus a sort of upper bound on the predictive performance of the method that generated the predictive maps. was important for us to keep the barren class while evaluating across cities, as it is the dominant class in Phoenix, AZ. In the remaining three cities, the barren class is challenging to predict as it is infrequent. In Table D.22, we compare this classification scheme with alternatives: 5 class scheme that will penalizes the map predictions for never predicts the barren class, and 4 class scheme that merges the barren land and impervious surfaces classes in evaluation. Table D.2 shows that while the choice of evaluation scheme does greatly effect accuracy (outside of Phoenix, AZ, where the accuracy of the Full US Map is low for both classification schemes), the average IoU drops significantly for all cities apart from Phoenix.
Comparing loss functions: qualitative results with land cover mapping. Figure D.3 compares predictions under different loss functions with an illustrative example. Here the prior is similar the "hand-coded"" prior described in Appendix [.1.2] but with the prior defined over all NL.CD classes. We train each model (a slight variant on the network used in experimental results) on the single NAIP tile region encompassing the zoom-in in the figure for 2000 iterations with the Adam algorithm (20), a batch size of 64, and learning rate fixed at Ic-4 during training. Qualitative comparisons show that predictions made by the QR and RQ loss functions are more certain (sharper colors in plots) than training with cross entropy or squared-error loss on the soft priors, and, in most places, arrive at better solutions than training with standard cross entropy loss on the argmax of the prior.
E ADDITIONAL EXPERIMENTS
E. SELF-SUPERV FOR UNSUPERVISED IMAGE CLUSTERING
Neural networks are usually trained on large amounts of hard-labeled data (xi, 4), yet, due to the biases induced by the typical architectures and learning algorithms, lot of the modeling power of
20

---

Under review as a conference paper at ICLR 2022
naip image
prede using cross-entropy loss on argmaxed prior
argmaxed prior
preds using cross-antropy loss soft prior
soft prior
NLCD Legend
Water
Developed Open Space
Developed Low Intensity
Developed Med. Intensity
Developed High Intensity
Barren Land
Deciduous Forest
preds using squared prior
prior
Evergreen Forest
Shrub/Scrub
Pasture/Hay
Cultivated Crops
Woody Wellands
Emergent Herbaceous
Wetlands
preds using QR loss
preds using QR loss
soft prior
greds using
a preds using RO loss
soft prior
Figure D.3: Comparison of different loss functions on hard and soft prior.
21

---

Under review as a conference paper at ICLR 2022
P(X Water)
P(x Trem)
P(X|== Low Veg)
PiXiE Impervious)
Imagery,
Implied
Epitome model
Figure D.4: Comparison of forward model likelihoods under the implicit generative model trained with QR loss (above) and the likelihood under epitome model [28 for part of test tile from
these networks seem to focus on correlations in the input space [6 This means that a network trained for one application, i.e., for one label space l L., can be adopted another application, i.e,, a different labels space e E L2, as long as the input features are in a similar domain. The canonical example of this is the use of lower levels of the networks pre-trained on ImageNet as part of the networks solving a completely different set of image classification problems. Pretrained networks require smaller training sets in fine tuning, as long as they have learned represent the variation in the input space well. Self-supervised models attempt to go step further and learn these representations without any labels. In our framework, self-supervision can simply be seen as the appropriate choice of subset priors p(lr) ) over appropriately chosen tuples of labels.
To discuss the pitfalls and opportunities, consider the QR loss
F= Σ qi(() log - a)) log L -,0 1,
(6)
i,
If we were to simply set Pi(() to a constant (e.g.. uniform) distribution p(() for all data points then the optimal solution would be any function (f) q(([x,) such that 1 Σ, q((xx,) p(t). Thus simply using the uniform prior may not lead to appropriate unsupervised clustering (or self- supervised learning of the network q). The inductive biases in the network architecture and training may not help, because one solution is (f(xx) = p()), which can be achieved by zeroing out all weights except for biases in the last softmax layer that outputs probabilities for labels l. As the softmax bias vector is the closest to the top in gradient descent, it will quickly be learned to match log p() and this will not only slow down the propagation of gradients into the network, but can eventually stop it completely, as this solution is a global optimum. Another optimal solution would be a function satisfying y Σ; (([x;) = p(f), but where individual entropies for each data point are small:- Eee (((x;) log q({]x;) < E, which motivates alternative cost criterion
F == Σ (0) log (O) 1.1
Σ (() log( Σ ()))) 1.8
where the first term promotes certainty in predictions ((([x;) for each point i and the second is pro- moting the diversity of the predictions across the different inputs, i.e., a high entropy of the average N En (h). This prevents learning a network with constant output (h) = p(h) and forces the model to find some statistics in the input data that break it into clusters indexed by labels t. The result will be highly dependent on the inductive biases associated with the network architecture and SGD method used, we can imagine degenerate solutions here well. For example, we can ig- nore completely some subset of features and still train a network that is certain in its modeling of the remaining ones, and achieves high diversity of predicted classes across the dataset. Obviously, this may be dangerous if the features omitted end up being the most important ones for the down- stream task. However, due to the stochastic gradient descent training as well as their architecture, it has been difficult to prevent neural networks from learning statistics involving all the input features. For example, training a neural network using weak generative model teacher corresponds to
22

---

Under review as a conference paper at ICLR 2022
using a simpler mixture model, whose posterior is used as target Pi(() and then learning neural network that can approximate it. The inductive bias then leads to networks that do not match pr ()) exactly but learn more complex statistics instead.
Equation 7 can be seen as a degenerate example of using a tuple prior where the tuple has the same data point repeated and the prior simply expects the two predictions to be the same. In many applications, there are natural constraints involving multiple data points that are easily modeled with priors over tuples for an example. is usually expected that nearby pixels should belong to the same class (or a small subset of classes), and that faraway pixels more likely to belong to different subset of classes. This belief is typically modeled in terms of Markov random field models of joint probabilities of labels in the image,
the entire collection of labels. Consider unsupervised image segmentation,
p((() or exp Σ (la (lGTTNN))
(8)
We experimented with potentials of the form
$(4 = =C(G)))eN) = at
l[c= [[(=
1 [Zi
Σ JEL.
[[c=
(9)
|S
JES;
where for pixel i, S is a small (5 x 5) neighborhood around it and L, is larger (50 x 50) neighbor- hood. If we set a₂ 1, Br = -1 for all C, then we consider this a contrastive prior, as it favors labels f, to match the labels found more concentrated in its immediate neighborhood than the larger scope. On the other hand az. and Br can be estimated based on the current statistics in the label dis- tribution using logistic regression, and we refer this as a self-similaritty prior p( P(G)):a ae:::eeBB Be. yc) with parameters which are periodically fit to the current statistics in the predictions Σ jes, ((()x)), and Eje, ((([x)) to promote similar label patterns across the image. The criterion (7) can also be seen as degenerate version of this setting with S being x I and L being infinite for the whole image).
The contrastive version of this prior relies on the insight previously pursued in image self- supervision, e.g., [14]. However, contrasting is accomplished without sampling triplets, but con- sidering all the data jointly, simply by expressing the goal of contrasting with far away regions within the prior in our framework.
As example of self-supervised pretraining in our framework, in Fig. E. of clustering of a large tile of aerial imagery into 12 classes using 5 layer FCN as network 4 of the architecture used in $4.4 The clustering is achieved by updating the prior every 50 steps of gradient descent on batches of 200 256 x 256 patches. The prior is initialized to a contrasting prior, and then updated through gradient descent. After 7 iterations, the result is sharpened by continuing training using
we show an example
NAIP image
Implicit clastering
Chater assignment
Figure E. Unsupervised clustering using implicit QR loss (middle) of a NAIP tile (left). On the right, we show the assignment of the 12 clusters to land cover labels: water (blue), tall vegetation (darker green), low vegetation (lighter green) and impervious/barren (gray).
This tile recently used in testing the fine-tuning of pretrained model with minimal amount of new labels in a new region 43. Both the pre-training region, the state of Maryland, and the testing region, the tiles in New York State, come from the 4-class Chesapeake Land Cover dataset ($4.3)
23

---

Under review as a conference paper at ICLR 2022
Yet, the slight shift in geography results in reduction of accuracy from around 90% in Maryland down to around 72.5% in New York. In [43]. various techniques for quick model adaptation are studied, on labels acquirable in up to 15 minutes of human labeling effort per tile. In Table E. ww compare the tunability of our self-suuer models on the four 85km² regions tested in 43 with active learning approaches to tuning pre-trained Maryland model with 400 labeled points. We show in the table the accuracy and mean intersection over union from [43] for tuning the pretrained model's last 64 x 4 layer with different active learning strategies for selecting points to be labeled. A random selection of 400 points for which the labels are provided yields an average accuracy improvement from 72.5% to 80.6%.
On the other hand, recall that we have created an unsupervised segmentation into 12 clusters, with posteriors over the clusters (f) To investigate how well these clusters align with ground truth land cover labels, we compute a simple assignment of clusters to land cover labels. Given a set of labeled points ((i, ci)rie,, we infer a mapping from clusters to four target labels.
p(cle) & Σ qi((). iEl:;=c
The label of any point/ j can now be inferred as Ej arg max, Σ, (((pp((()) This procedure, using 400 randomly selected labeled points, yields an average accuracy of 81.1% (averaged over 50 random collections of labeled points), which is above the performance of the pretrained model tuned on as many randomly selected points, and on par with the more sophisticated methods for point selection and the use of the pretrained model. (Note that the large model pretrained was trained on a large similar dataset nearby state). (Table E.
Table E.1:
Finetuning a pre-trained model by gradient descent label assignment in low-label regimes.
43
versus implicit QR clustering
pretrained model in
43
Implicit QR
Query method
No tuning
Random
Entropy
Min-margin
Random
Tuned parameters
64x4
64x4
64x4
12x4
Accuracy
72.5
80.6
73.6
81.1
81.1
IoU %
51.0
60.8
50.1
60.8
59.8
E.2 IUMOR-INEITTRATING LYMPHOCYTE SEGMENTATION
The setup of this experiments mimics that of the land cover label super-resolution experiment in $4.3 The training data consists of 50k 240 x 240 crops of H&E-stained histological imagery at 0.5pm/px resolution, paired with coarse estimates of the density of tumor-infiltratin lymphocytes (TILs) created by a simple classifier, at the resolution of 100 x 100 blocks. The goal is to produce models for high-resolution TIL segmentation. Models are evaluated on a held-out set of I 1786 images with high-resolution point labels for the center pixel.
The coarse density estimates c belong to one of 11 classes, from (no TILs) 9 (highest estimated THI. density). We use an estimated conditional likelihood p((c) of the likelihood of the positive THI label at pixels with each low-resolution class c to construct a prior p:(() over the TIL label probability. Notice that this prior is the same for all pixels in any given low-resolution, coarsely labeled block F
We train a small CNN with receptive field 11 x 11 (five ReLU-activated convolutional layers with 64 filters) under the RQ loss against this prior for 200 epochs with learning rate 10-5, then evaluate on the held-out testing set. Inspired by (28. we apply a spatial blur of 11 pixels to the predicted log-likelihoods (again correcting for the model's small receptive field and the dataset bias).
We experimented with setting p; to conditional likelihoods estimated from a held-out set and with simply setting ( = 1|c = 0) = 0.05, py = 1/e 1) = 0.15, .... P)(C I/c = 9) = 0.95. The latter gave better results, perhaps due to the bias of the evaluation set, in which every image is known to be centered on cell of some kind.
24

---

Under review as a conference paper at ICLR 2022
Table E.2: Area under ROC curve for various predictors on the TII. segmentation task.
fully supervised
weakly supervised
Model
SVM 58 13
CNN [3
CSP-CNN 13
U-Net 27
Epitome (28)
RQ
AUC
0.713
0.494
0.786
0.783
0.801
0,802
The AUC scores of this model and of the baselines are shown in Table E.2 Interestingly, the best- performing models RQ and epitomic super-resolution (a generative model) both have receptive fields of 11 X 11, much smaller than those of the U-Net and fully supervised CNNs. This means that prediction of TIL likelihood is possible using only local image data, but the challenge is learning to resolve highly uncertain label information. Unlike U-Nets and deep CNN autoencoders, small models are not able to learn and overfit to distant spurious clues to the classes of nearby pixels.
E.3 VIDEO SEGMENTATION WITH A STRUCTURED PRIOR
To demonstrate the use of priors with latent structure, we set up the problem of video segmentation as follows. Given a frame 1, we tune networks The prior in each frame comes from a Mask R-CNN model I1I pre-trained on still images in the COCO dataset [23]. The Mask R-CNN model finds several possible instances of objects of different categories and outputs the soft object masks in form of confidence scores for each pixel. We convert this into probability distribution over the index f (foreground/backkrrundd) of the form p(fi.I(m;) where m, are different detected instances by the model, and the distributions P(fr. |m,) are the soft masks for these instances converted to probability distributions. i.c. value of the probability of foreground differs for each pixel and each instance based on the Mask R-CNN confidence scores. Although the COCO dataset may not have had instances of object of interest in our frame X,, we assume that some admixture of detected instances (likely involving unrelated types of objects) does model reasonably well the foreground segmentation in the frame. Mathematically, p(fi,) = Σm, pffi,rlm) (m;), where p(m) expresses the probabilistic selection of the foreground masks for different instances from which the foreground is constructed. (One can think of instances m, as topics topic models, which are also admixture models). To complete the prior, fix the distribution (C]ff as fixed binary L X 2 matrix assigning subset of L pixel classes to foreground and the rest to the background. (For example, we assign first 3 classes to foreground and the remaining 5 to the background for total of L=8 pixel classes). Therefore,
predicting one of L pixel classes for
pixel at coordinate in frame
p(a, = =)= Σ ELALD -((h===(m>) flm)) f
(10)
my
We can now select the instances m; in each frame by optimizing the free energy with this prior over p(m)). The procedure involves standard variational inference of the posterior distribution over possible instances m, for each pixel in frame which involves the posterior qr (fi,)))) In practice we found that it is enough to do this inference once, using the network qr-1 estimated in the previous frame.
This requires the inference of m, for each pixel i:
sy (m,), or exp( EEeesnaa fxx..) log p(f. = flm)))(mm).
(11)
and then optimizing Pm, as the count of times each instance is used,
p(m)) oc Σ (m,)))
(12)
Selection of instances m, in frame / therefore involves comparing the predictions from the network q, (G., flx,,) grouped into foreground/backgroum, segmentation with the foreground/backgroon segmentation for different instances from Mask R-CNN, and making a selection of a subset (proba- bilistically in p(m,)) based which instances most overlap with the predictions from network qi- While the above two equations should in principle be iterated, and iterated with updates to network q, (Cr, f) we found that in practice it is sufficient just select the instances m, based on their
25

---

Under review as a conference paper at ICLR 2022
Figure E.2: Example of inferring the foreground mask on a video frame.
intersection with the network predictions once, at the very beginning, to make soft fixed prior, and leave to optimizing the prediction network with the RQ loss to find confident segmentation (Fig.
We tested the approach on the DAVIS 2016 dataset [37] The dataset is comprised of 50 unique scenes, accompanied by per-pixel foreground/background segmentation masks. The objective is to produce foreground segmentation masks for all frames in a scene, given only the ground truth an- notations of the first frame (Semi-Supervised) We evaluated our method on the 20-scene validation set at 480p resolution.
The network q used in this experiment combines both the pixel intensities and spatial position in- formation for its predictions. At cach pixel location i, we augment the intensity information with learned Fourier features [sin(W [s.n(W[i.j]'' cos (W[i, )r [47). The image and spatial position first processed separately; 4-layer, 64-channel, cully-convolutional network with 3 x kernels, ReLU activations and Batch Normalization produces the image features. A 3-layer, 16-channel, pixel-wise MLP with ReLU activations and Batch Normalization processes the learned Fourier fea- tures. These two are concatenated and passed through single 3 x convolution-Rel -eetiinnnnRL------ Reee--LLUBBBtee Normalization layer before being mapped to output predictions. We also experimented with adding optical flow another auxiliary input the network.
For each scene, the network go is trained on the first frame, using the given ground truth annota- tions split uniformly between 3 foreground and 5 background classes prior, for 300 iterations. This network is then used to predict the foreground pixels the next frame and after computing the intersection
union between the predicted foreground pixels and the Mask R-CNN output masks, we select masks that overlap more than pre-specified threshold. The chosen masks are then summated, weighted by their Mask R-CNN confidence scores (0-1), form the prior for the next frame. The process of selecting masks from the Mask-RCNN predictions and forming the prior for frame showcased in Figure E.3 The network qo is then fine-tuned for 10 iterations to obtain qı and this process repeats for all subsequent frames. We used the Adam optimizer, with starting learning rate of 10-3 for the first frame, reduced 10 for fine-tuning, and trained with batches of 128 64x64 patches.
To infer the foreground pixels first need a pre-trained the Mask R-CNN on the COCO dataset. Then, for each scene we only require ~ Im of training time on the ground truth antthannottted first frame and ~3s per every following frame for the entire process of forming the prior and inferring the foreground pixels. We do not train on any video data, in contrast to most video object segmentation methodologies that rely on both a pre- pre--rrinee trained network on static image datasets (such as COCO) and additionally on offline training
video sequences. In Table F.3 we compare our results on the DAVIS 2016 validation set to other video object segmentation algorithms from 2017 present.
26

---

Under review as a conference paper at ICLR 2022
Seom:
Mascc Lsbeb: pereee Scom: 0.59
Mext Label: "iee Snow: 0.5
Mesk: "person" Scure: 0:38
Label "oie glasa
Scrnc
Labet "wie glass" Score:
Combined Mascs Print
Pradict or
Labels
Figure E.3: Video segmentation procedure. Starting with network 9:-1 trained on frame 1, apply q11 on frame / to get a rough foreground estimation (top). By running the pre-trained Mask R-CNN model frame and selecting only the masks that overlap with the qi-1 prediction get the candidate object masks (middle). The prior constructed as the sum of the candidate masks, weighted by their corresponding Mask R-CNN scores, and gr-1 is finetuned frame with this prior to get the detailed predictions (bottom).
27

---

Under review as a conference paper at ICLR 2022
Table E.3: Jaccard and F1 measures for various algorithms the video instance segmentation task.
Model
J&FT
Mean
Recall
Decay
Mean
Recall f
Decay 1
Year
OSVOS
80.2
79.8
93.6
14.9
80.6
92.6
15
2017
MSK 38
77.55
79.7
93.1
8.9
75.4
87.1
9
2017
OnAVOS
49
85.5
86.1
96.1
5.2
84.9
89.7
5.8
2017
Lucid 18
82.95
83.9
95
9.1
82
88.1
9.7
2017
OSVOS-S 29
86.55
85.6
96.8
5.5
87.5
95.9
8.2
2018
FAVOS 5
80.95
82.4
96.3
4.5
79.5
89.4
5.5
2018
PReMVOS 25
86.75
84.9
96.1
8.8
88.6
94.7
9.8
2018
OSMN
73.45
87.6
9
72.9
84
10.6
2018
AGAME 16
81.85
81.5
93.6
9.4
82.2
90.3
9.8
2019
STM 36
89.4
88.7
97.4
5
90.1
95.2
4.2
2019
FEELVOS 50
81.65
81.1
90.5
13.7
82.2
86.6
14.1
2019
CFBI 58
89.4
88.3
90.5
2020
e-OSVOS
30
86.6
2020
STCN 4
91.7
90.4
98.1
4.1
93
97.1
4.3
2021
Ours
83.8
84
96.2
8.4
83.6
94.2
10.2
Ours (+flow)
83.9
83.2
95.5
9.5
84.6
93.3
9.1
E.4 SEDUCERS SEDUCING SEDUCERS
One of the conclusions from our experiments on EnviroAtlas ($4.4) is that training a network with the goal of generalizing to new input data is often inferior to simply performing in-collection in- ference. In other words, given the collection of pairs x;, (E),,, learning the posterior q under the implicit generative model is optimized for resolving ambiguities in that collection, and possibly that collection alone. As pointed out in [28]. which performs collection inference using large generative models to mine self-similaris among the examples in the collection, this is appropriate when can expect our data to always come paired with prior beliefs p(G). is interesting to reconsider the Seducer example from Fig. [ The artist created several versions of that painting, and shown E.4 collection inference applied separately to each of these painting works equally well. However, using a learned q network from one image onto others yields inferior segmentations (Fig. E.5). as the learned network specialized for inference in the data it saw. (A fully generative model would similarly overtrain on the input data features x;, as would supervised neural network trained on hard-labeled pairs (x;, 4) due to the domain shift.) Yet, if we know we will always be given collections with beliefs in the form of priors Pi()), local (collection) inference is all we need.
in Fig.
28

---

Under review as a conference paper at ICLR 2022
Figure E.4: Two additional versions of Le séducteur (left), hand-made priors (middle) and inferred segmentations (right).
(b)
(c)
Figure E.5: Result of applying network y trained to infer (b), on all three Le séducteur versions.
29


Reviews:
[{"review_id": "xH5aiMamQSY", "reviewer": "Reviewer_Da9G", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n- This work considers a general form of weakly-supervised problems, which can be applied to a wide range of scenarios, including learning from partial or coarse labels. \n- The paper proposes a new strategy by constructing an implicit generative model that incorporates the weak label priors in a unified manner.\n- On a range of vision and text classification tasks, the proposed method seems to achieve competitive performances with higher efficiency. \n\nConcerns:\n- The proposed implicit generative model seems expensive to compute due to its normalization factor. As shown in Equation 2, the forward probability requires a summation over the entire training dataset, which is also used in the loss in Equation 4. It is unclear how to implement this in a mini-batch mode. Note the summation within the log should be over i instead of l. \n- As discussed in the paper, there are potentially different degenerate solutions for the loss function, and it is unclear in which conditions those solutions can be avoided. The strategy of breaking symmetries used in this work seems a bit ad hoc and no general guideline is provided. \n-  The motivation for adopting the RQ loss is unclear. Unlike the QR objective, the RQ loss does not correspond to any properly-defined loss function. More importantly, it seems difficult to choose which one to use in a specific task. In this paper, RQ is adopted for partial labels and text classification, while QR is more effective for land cover segmentation.\n- The experimental evaluation seems less convincing due to the following reasons:\n1) For the partial label task in 4.1, the improvement over a simple NLL baseline is marginal, as shown in Figure 3 top. \n2) For the learning from rank task in 4.2, it lacks comparisons with other learning to rank learning objectives.\n3) For the land cover segmentation task in 4.3, it also lacks comparisons with a simple baseline trained with pseudo/inferred labeling. \n4) For the learned prior task in 4.4, the pre-training seems to play a key role and it is unclear how important the proposed strategy is. The performance is much worse if the network is trained from scratch.  \n5) For the text classification in 4.5, it lacks detailed ablative study for the proposed model. It is unclear whether the improvement is due to a better prior adopted in this work or the RQ loss. ", "summary_of_the_paper": "The paper presents a weakly supervised learning strategy, which exploits instance labels in a form of label prior distributions for training classifiers. The main idea of this work is to build an implicit generative model from a probabilistic label prediction network, which is then trained with an ELBO loss. The paper applies this framework to several types of weakly supervised learning scenarios, including classification with negative labels or labels from ranking, semantic segmentation and text classification from coarse labels, and other tasks with structured label priors, etc.  The experimental evaluation validates the efficacy of the proposed learning strategy on the aforementioned tasks with comparisons to the prior works. ", "summary_of_the_review": "The proposed implicit generative model seems interesting and achieves good performance for a range of weakly-supervised tasks. However, I am slightly leaning towards the negative side due to the lack of clarity in the model design and missing comparisons in the experimental evaluation.   ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "WaIu_Opa56x", "reviewer": "Reviewer_HHeb", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength:\n\n+The idea of implicit generative models with label priors to handle coarse and imprecise data input is interesting and easy to follow.\n\n+The experiment validation covers five different tasks in both CV and NLP fields, which shows a potential good and positive impact for implicit generative models to the CV and NLP communities.\n\nWeakness:\n\n-The paper is not well written and organized. Especially in the introduction section, the background of the problem and the motivation of introducing implicit generative models are not well stated. I  suggest the authors to move Section A in the Appendix into the Section 2 as a separative subsection.\n\n-The experiment validation still has room to improve. As the goal of this paper is to resolve label uncertainty, the authors should compare the proposed methods with some state-of-the-art approaches like [35; 54; 53; 55; 58] to make the experiment validation more solid.\n\n-It should be better if the authors can provide discussion on limit and failure cases of the proposed method existing in different tasks.\n", "summary_of_the_paper": "In this paper, the authors present implicit generative models in a free energy criterion with a combination of both the training of neural networks for label prediction and modeling of a label prior. They also discuss multiple sources of label priors to handle label uncertainty for coarse and imprecise data input. The extensive experiments conducted on five different tasks and the experimental results seem to well validate the efficacy of the proposed method.", "summary_of_the_review": "The idea is interesting and easy to follow. It has a very positive impact to the AI community. However, the paper writing and organization, experiment design and presentation still have room to improve. Therefore, I initially tend to marginally accept this paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "SadbukLGEH5", "reviewer": "Reviewer_5Mem", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper discussed an approach to take priors into account in machine learning with a slightly different formula, which I have doubts on its correctness. In the experiments, they showed that performance in some tasks can be significantly improved by cleverly choosing class priors. The experiment results look good, but I do not think it fully justifies their algorithm because of the lack of baselines and inappropriate interpretations of an equation.\n\nThe authors mentioned a lot of different types of priors, however many of them (including most of those used in their experiments) could be characterized as cost-sensitive learning, a.k.a. taking priors into account in classification problems. In this regard, the citations to cost-sensitive learning literature and comparisons are vastly insufficient. e.g. papers that should be cited include:\n\nZadrozny et al. Cost-sensitive learning by cost-proportionate example weighting. ICDM 2003\n\nBranco et al. A survey of predictive modeling on imbalanced domains. ACM Computing Surveys 2016 is a recent survey paper on cost-sensitive learning.\n\nThey did not compare against any prior work on cost-sensitive classification, which is a major problem that lead me to the decision of rejection. A recent popular general approach to deal with class priors is focal loss, which should be cited and compared as well.\n \nLin et al. Focal Loss for Dense Object Detection. ICCV 2017\n\nDespite the authors claims, eq. (4) still seems fundamentally flawed in that in the case of 50%/50% 2-class classification problems, the classifier could have entirely flipped one class with another and still optimize eq. (4). I believe what is working is NOT eq. (4), but the practice of applying eq. (4) on mini-batches. Because each mini-batch is much smaller, it becomes difficult to keep the balance in a mini-batch and the multiple resampling of each mini-batch from different epochs may lead to the effect of normalization, similar to the effects that we can see in minibatch regularization in the training of GANs. Hence, I have strong doubts that similar balance-breaking would not necessarily happen so reliably if mini-batches are not used.\n\nThe choice between QR and RQ is quite arbitrary in different experiments. Is there any principle about it? Or just trial-and-error?\n\nThere is an equation typo in eq. (4), the log-sum term should be log(\\sum_i q_i(l)), not log(\\sum_l q_i(l)). The interpretation in the subsequent paragraph is although correct in that it favors predicting a high-confidence label.\n\nMinor: The authors should mention that q_i(l) is an energy function (logits) instead of a probabilistic distribution, otherwise they should have \\sum_j q_j(l) = 1 and the derivation would be in a different place (e.g. the \\sum_j q_j term won't exist).\n", "summary_of_the_paper": "This paper discussed an approach to take priors into account in machine learning with a slightly different formula, which I have doubts on its correctness. In the experiments, they showed that performance in some tasks can be significantly improved by cleverly choosing class priors.\n", "summary_of_the_review": "This paper discussed an approach to take priors into account in machine learning with a slightly different formula, which I have doubts on its correctness. In the experiments, they showed that performance in some tasks can be significantly improved by cleverly choosing class priors. The experiment results look good, but I do not think it fully justifies their algorithm because of the lack of baselines and inappropriate interpretations of an equation.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "AfIOOUfgHN9", "reviewer": "Reviewer_cjDc", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper addresses an impressive range of learning settings under the same formulation. The paper is clear and well-presented.\n\nOn the positive side, I like how the authors connect fields that are in general not address under the same umbrella frameworks, which makes their approach very complete and appealing. \n\nAlso, I wanted to highlight the very thorough experimental design and analysis. The choice of case studies is very adequate and in some cases, inspiring.\n\nOn the negative side, I would have liked to have seen a more elaborate discussion of the relationship with existing methods for each of the problems the paper addresses. The literature review lacks a bit and it makes it difficult to assess the overlap with previous works. \n\nAlso, although the experimental work is very good, I was missing any form of theoretical analysis of the proposed approach.", "summary_of_the_paper": "The paper presents an approach base on implicit generative models to address a number of existing (and potentially new) scenarios, including weak labels (noisy or incomplete), likelihoods given by a different prediction mechanism on auxiliary input,\nor priors reflecting knowledge about the structure of the problem. The experiment presents a selection of very diverse case studies.", "summary_of_the_review": "I like the paper and its aim, particularly on the experimental front. There is room for improvement on the theoretical analysis and related work.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors have addressed several of the issues raised by the reviewers, and they are strongly encouraged in include the additional experiments, and sections, that they propose, in a revised submission. The reviewers also recognized the novelty and extend of applications the proposed methodology has. Nevertheless, the paper would significantly benefit from a rigorous and thorough comparison to related work, placing it well within the context of the literature brought up by reviewers. Experimental comparisons to competitors, even if the latter address more restrictive settings, would strengthen the paper. Most importantly, the authors should consider including a comprehensive related work section, that convincingly discusses and compares to related/adjacent methods.
Decision: Reject