Resolving label uncertainty with implicit generative models

1 INTRODUCTION . We consider the problem of joint inference of latent label variables ℓ8 in a collection of data samples indexed by 8 consisting of observations ( features ) G8 and corresponding prior beliefs about their latent label variables ? 8 ( ℓ ) . Two illustrative examples are shown in Fig . 1 . In the first example , the G8 are 784-dimensional vectors representing 28×28 MNIST digits . We aim to infer the digit classes ℓ8 ∈ { 0 , 1 , ... , 9 } for all images in the given collection based on data in which we are given just one negative label per sample , i.e. , the prior beliefs ? 8 ( ℓ ) ( top row ) are uniform over all classes except for one incorrect class . The procedure described in this paper produces inferred distributions over labels ( bottom row ) that are usually peaky and place the maximum at the correct digit 97 % of the time ( Fig . 3 ) . In the second example , the observations G8 are image patches centered around each pixel coordinate 8 in a Surrealist painting , with patch size ( 11×11 ) equal to the receptive field of a 5-layer convolutional neural network used in our inference procedure . The prior beliefs ? 8 ( ℓ ) are distributions over 3 classes ( sky , boat , water ) depending on the coordinate 8 . The joint inference of all labels in this image yields a feasible segmentation despite the high similarity in colors and textures ( see §E.4 ) . In both examples , the inference technique needs to estimate statistical links between observations G8 and corresponding latents ℓ8 that would both be highly confident ( i.e. , lead to low entropy in the posterior distributions ) and explain the varying prior beliefs , which typically have low confidence ( high entropy in the prior distributions ) . This problem of training on weak beliefs , in some form , is often encountered in machine learning , e.g , weak supervision and semi-supervised learning , domain transfer , and integration of modalities – settings where coarse , partial , or inexact sources of data can provide rich information about the state of a prediction instance , though not always a “ ground truth ” label for each instance . In many such settings , fusing the weak input data into a probability distribution over classes is a more natural alternative to transforming the weak input into hard labels [ 26 ] . However , because supervised models target the distribution over labels , training machine learning models with supervision from probabilistic prior beliefs results in uncertain predictions . Most approaches to resolve these uncertainties involve iterative generation of hard pseudo labels [ 56 ] , or loss functions promot- ( a ) Le séducteur , René Magritte ( b ) Boat Prior , anonymous artist ( c ) Inferred segmentation Figure 1 : Above : Inference of latent MNIST digit classes with negative label supervision using a small CNN trained on the RQ criterion ( §2.1 ) . Below : Joint inference of latent pixel classes in an image ( a ) . The prior beliefs ? 8 ( ℓ ) over three classes – sky ( red ) , boat ( green ) , water ( blue ) – are manually set ( b ) . A small CNN using no data except ( a , b ) infers the posterior classes ( c ) . ing low entropy of predictions [ 35 ; 55 ; 59 ; 54 ] . Typically , these approaches are application-specific [ 10 ; 57 ; 1 ; 22 ] . Further discussion of related work is provided in §B and §C . Our key modeling insight is to associate the output distribution of a discriminative model , a feedforward neural network @ , with an implicit generative model ( §2.1 ) of features and to consider the given prior belief as part of that model . We show that without estimating the full generative model , it is possible to learn the network that performs inference in it and reap the benefits of generative modeling , including high certainty in the posterior under soft priors and rich opportunities to model structure in the prior beliefs . We validate the effectiveness of our approach with experiments ( §4 , §E ) that highlight : prior beliefs as a natural way to fuse weak inputs , graceful degradation of performance with increasingly noisy or incomplete inputs , and comparison of our implicit generative model with explicitly generative modeling approaches . 2 BACKGROUND AND APPROACH . Supervised learning on prior beliefs . Supervised learning models , including many neural nets , are typically trained to minimize the cross-entropy −∑8 ∑ℓ ? 83 ( ℓ ) log @ 8 ( ℓ ) between the data distribution over labels ? 3 8 ( ℓ ) and the distribution @ 8 ( ℓ ) = @ ( ℓ |G8 ; \ ) output by a predictor @ using data features G8 . This is equivalent to minimizing the KL divergence ∑ 8 KL ( ? 38 ‖ @ 8 ) which is not wellsuited to training with uncertain labels defined by prior beliefs . If we were to set ? 3 8 ( ℓ ) to equal a much softer prior over latent labels , ? 8 ( ℓ ) , the minimum would be attained when the two distributions ? 8 ( ℓ ) and @ 8 ( ℓ ) are equal : when the prior belief is soft , the trained model @ will also be highly uncertain . Turning soft labels into hard training targets , ( ? 3 8 ( ℓ ) = 1 [ ℓ = arg maxℓ ? 8 ( ℓ ) ] , or by sampling ) , introduces the opposite bias . Now , the cost would indeed be minimized if the predictions had zero entropy , but learning such a prediction function faces difficulty with overconfident labels which are often wrong , as well as the possibility that certain labels often receive substantial weight in the prior , but never the maximum . These issues are illustrated in Figure D.3 . Generative modeling resolves the prior ’ s uncertainty . The approach to classification problems through generative modeling , instead of targeting the conditional probability of latents given the data features , assumes that there is a forward ( generative ) distribution ? ( G8 |ℓ ) and optimizes the log-likelihood of the observed features , ∑ 8 log ( G8 ) = ∑ 8 log ∑ ℓ ? ( G8 |ℓ ) ? 8 ( ℓ ) , with respect to the parameters of the forward distribution . The posterior under the model ? ( ℓ |G8 ) ∝ ? ( G8 |ℓ ) ? 8 ( ℓ ) is then used to infer latent labels for individual data points [ 44 ] . As a result , the generative modeling approach does not suffer from uncertainty in the posterior distribution over latents given the input features , even when the priors ? 8 ( ℓ ) are soft.1 Further discussion relating our method with existing generative modeling approaches is given in §C . However , expressive generative models are typically harder and more expensive to train compared to supervised training of neural networks , as they often require sampling ( e.g. , sampling of the posterior in VAEs [ 21 ] and sampling of the generative model in GANs [ 8 ] ) . Furthermore , the modeling often requires doubling of parameters to express both the forward ( generative ) model and the reverse ( posterior ) model . And , in case of GANs , the learning algorithms may not even cover all modes in the data , which would prevent joint inference for all data points . 2.1 OPTIMIZATION UNDER IMPLICIT GENERATIVE MODELS . Suppose that there is a generative model ? ( G |ℓ ) of observed features conditioned on latent labels . Optimization of the log-likelihood of observed features , ∑ 8 log ( G8 ) = ∑ 8 log ∑ ℓ ? ( G8 |ℓ ) ? 8 ( ℓ ) , can be achieved by introducing a variational distribution @ 8 ( ℓ ) over the latent variable for each instance G8 , then minimizing the variational free energy , which we review next . Recall that the free energy , also known as the negative evidence lower bound ( ELBO ) , is defined as : = − ∑ 8 @ 8 ( ℓ ) log ? ( G8 |ℓ ) ? 8 ( ℓ ) @ 8 ( ℓ ) = ∑ 8 ( KL ( @ 8 ‖A8 ) − log ? ( G8 ) ) , ( 1 ) where A8 ( ℓ ) = ? ( G8 |ℓ ) ? 8 ( ℓ ) ∑ ℓ ? ( G8 |ℓ ) ? 8 ( ℓ ) is the posterior distribution under the generative model . As the KL distance is always positive , the free energy is minimized when @ 8 = A8 , in which case the free energy equals the negative log-likelihood of the data . Free energy minimization is used in various approaches to latent variable modeling . The expectation-maximization ( EM ) [ 6 ] algorithm minimizes ( 1 ) by iteratively setting @ 8 to equal the posteriors A8 and updating the parameters of the forward distributions ? ( G8 |ℓ ) while keeping @ 8 fixed , until guaranteed convergence to a local minimum . In applications of EM predating deep generative models , the @ 8 distributions are not given by a predictor taking G8 as input ; they are are auxiliary distributions used only as part of the learning procedure and replaced at each EM iteration . Their dependence on G8 is implicit in energy minimization . In variational auto-encoders ( VAEs ) [ 21 ] , both the generative model ? ( G |ℎ ) and the posterior @ ( ℎ|G ) are represented as neural networks . Here ? is a deep nonlinear generative model with hidden latents ℎ ( not necessearily corresponding to labels ) , and evaluation of the full posterior is intractable . Instead , @ is learned as a neural network @ ( ℎ|G8 , \ ) jointly with ? . VAE training involves sampling from @ 8 ( ℎ ) = @ ( G |G8 , \ ) in learning to improve the agreement between the forward ( generative ) model ? ( and its true posterior A8 ( ℎ ) ) and the reverse ( posterior ) neural network model @ . Implicit generative models . To derive our training objectives , we will assume the existence of the generative model ? ( G |ℓ ) without ever parametrizing it or sampling from it . Instead , we parametrize only the reverse ( variational posterior ) model @ 8 ( ℓ ) = @ ( ℓ |G8 ; \ ) in the form of a neural net taking G8 as input . The trained model @ is the direct output of our algorithms . Given the variational distributions @ 8 , we can minimize ( 1 ) with respect to the forward probabilities ? ( G8 |ℓ ) for all G8 and ℓ such that ∑ 8 ? ( G8 |ℓ ) = 1 for all ℓ . The optimum is achieved by : ? ( G8 |ℓ ) = @ 8 ( ℓ ) ∑ 9 @ 9 ( ℓ ) . ( 2 ) We refer to this as the implicit generative model associated to @ . Much like the posterior is implicit in the free energy minimization in the EM algorithm , here instead the forward model is just a matrix of numbers ? 8 , ℓ , implicit to free energy minimization – the opposite , in some sense , to what is done in EM optimization . The link G − ℓ is left entirely to the neural network @ to capture explicitly . The probability mass of the implicit generative model is not uniformly distributed , rather , the data points for which the variational posterior @ 8 ( ℓ ) is more certain are considered more likely under that latent ℓ , unless it corresponds to a popular class ℓ to which many other data points are also assigned . 1For intuition , consider a model where ℓ is the mixture index in a mixture of high-dimensional Gaussians . If the observations G8 naturally cluster into several well-defined clusters , then the prior ? 8 ( ℓ ) may be flat , but the posterior distributions under the model assign data points to clusters with high certainty . The posterior under the implied generative model is A8 ( ℓ ) ∝ ? 8 ( ℓ ) @ 8 ( ℓ ) ∑ 9 @ 9 ( ℓ ) . ( 3 ) Note that we can compute A8 ( ℓ ) by multiplying the re-normalized model outputs with the prior at each instance , so that for each instance 8 we have two outputs : @ 8 and A8 . We propose two methods of optimizing the free energy with respect to the parameters \ by gradient steps . Each method iterates the following : ( 1 ) Calculate the posterior distributions A8 in terms of @ 8 as in ( 3 ) ( 2 ) Update the parameters of @ with a gradient step : • Option QR : \ ← \ − [ ∇\ ∑ 8 KL ( @ 8 ‖A8 ) . • Option RQ : \ ← \ − [ ∇\ ∑ 8 KL ( A8 ‖ @ 8 ) . Gradients of the model parameters are propagated to the expression of A8 through @ 8 ( see Fig . 2 ) . Both losses have a stable point when @ 8 = A8 . A discussion of the relative benefits and limitations of the QR and RQ loss formulations is given in §A , along with practical considerations for implementing these methods . Option QR uses the KL distance in the direction it appears in ( 1 ) and thus guarantees continual improvements in free energy and convergence to a local minimum ( with the exception for the effects of stochasticity in minibatch sampling ) . Substituting A8 from ( 3 ) , the free energy ( 1 ) becomes : = ∑ 8 , ℓ @ 8 ( ℓ ) log ( ∑ 9 @ 9 ( ℓ ) ) − ∑ 8 , ℓ @ 8 ( ℓ ) log ( ? 8 ( ℓ ) ) ( 4 ) This criterion does not encourage entropy of individual @ 8 distributions , but of their average . The second term alone would be minimized if the @ network could put all the mass on arg maxℓ ? 8 ( ℓ ) for each data point , but the first term promotes diversity in assignment of latents ( labels ) ℓ across the entire dataset . Thus a network @ can optimize ( 4 ) if it makes different confident predictions for different data points . To illustrate this , consider the case when ? 8 ( ℓ ) = ? ( ℓ ) , i.e . all data points have the same prior . Then ( 4 ) is minimized when 1 # ∑ 8 @ 8 ( ℎ ) = ? ( ℓ ) , and this can be achieved when the network learns a constant distribution @ ( ℓ |G8 ; \ ) = ? ( ℓ ) . But the free energy is also minimized if the network predicts only a single label for each data point with high certainty , but it varies in predictions so that the counts of label predictions match the prior . As demonstrated in Fig . 1 and in our experiments , avoiding degenerate solutions is not hard . We attribute this to two factors . First , the situations of interest typically involve uncertain , but varying distributions ? 8 ( ℓ ) which break symmetries that could lead to ignoring the data features G8 . Second , the neural networks used as posterior models @ come with their own constraints in parametrization ( e.g . a multi-layer convolutional net with certain number of filters in each layer ) , and , the inductive biases as a consequence of gradient descent learning in nonlinear , multilayer networks . In fact , as discussed in §3 and §E.1 , even unsupervised clustering is possible with suitably chosen priors that break symmetry , allowing this approach to be used for self-supervised training . See also §C for more on relationships with other approaches .

Summary: The paper presents a weakly supervised learning strategy, which exploits instance labels in a form of label prior distributions for training classifiers. The main idea of this work is to build an implicit generative model from a probabilistic label prediction network, which is then trained with an ELBO loss. The paper applies this framework to several types of weakly supervised learning scenarios, including classification with negative labels or labels from ranking, semantic segmentation and text classification from coarse labels, and other tasks with structured label priors, etc.  The experimental evaluation validates the efficacy of the proposed learning strategy on the aforementioned tasks with comparisons to the prior works.