Implicit Maximum Likelihood Estimation

1 INTRODUCTION . Generative modelling is a cornerstone of machine learning and has received increasing attention . Recent models like variational autoencoders ( VAEs ) ( Kingma & Welling , 2013 ; Rezende et al. , 2014 ) and generative adversarial nets ( GANs ) ( Goodfellow et al. , 2014 ; Gutmann et al. , 2014 ) , have delivered impressive advances in performance and generated a lot of excitement . Generative models can be classified into two categories : prescribed models and implicit models ( Diggle & Gratton , 1984 ; Mohamed & Lakshminarayanan , 2016 ) . Prescribed models are defined by an explicit specification of the density , and so their unnormalized complete likelihood can be usually expressed in closed form . Examples include models whose complete likelihoods lie in the exponential family , such as mixture of Gaussians ( Everitt , 1985 ) , hidden Markov models ( Baum & Petrie , 1966 ) , Boltzmann machines ( Hinton & Sejnowski , 1986 ) . Because computing the normalization constant , also known as the partition function , is generally intractable , sampling from these models is challenging . On the other hand , implicit models are defined most naturally in terms of a ( simple ) sampling procedure . Most models take the form of a deterministic parameterized transformation Tθ ( · ) of an analytic distribution , like an isotropic Gaussian . This can be naturally viewed as the distribution induced by the following sampling procedure : 1 . Sample z ∼ N ( 0 , I ) 2 . Return x : = Tθ ( z ) The transformation Tθ ( · ) often takes the form of a highly expressive function approximator , like a neural net . Examples include generative adversarial nets ( GANs ) ( Goodfellow et al. , 2014 ; Gutmann et al. , 2014 ) and generative moment matching nets ( GMMNs ) ( Li et al. , 2015 ; Dziugaite et al. , 2015 ) . The marginal likelihood of such models can be characterized as follows : pθ ( x ) = ∂ ∂x1 · · · ∂ ∂xd ∫ { z|∀i ( Tθ ( z ) ) i≤xi } φ ( z ) dz where φ ( · ) denotes the probability density function ( PDF ) of N ( 0 , I ) . In general , attempting to reduce this to a closed-form expression is hopeless . Evaluating it numerically is also challenging , since the domain of integration could consist of an exponential number of disjoint regions and numerical differentiation is ill-conditioned . These two categories of generative models are not mutually exclusive . Some models admit both an explicit specification of the density and a simple sampling procedure and so can be considered as both prescribed and implicit . Examples include variational autoencoders ( Kingma & Welling , 2013 ; Rezende et al. , 2014 ) , their predecessors ( MacKay , 1995 ; Bishop et al. , 1998 ) and extensions ( Burda et al. , 2015 ) , and directed/autoregressive models , e.g. , ( Neal , 1992 ; Bengio & Bengio , 2000 ; Larochelle & Murray , 2011 ; van den Oord et al. , 2016 ) . 1.1 CHALLENGES IN PARAMETER ESTIMATION . Maximum likelihood ( Fisher , 1912 ; Edgeworth , 1908 ) is perhaps the standard method for estimating the parameters of a probabilistic model from observations . The maximum likelihood estimator ( MLE ) has a number of appealing properties : under mild regularity conditions , it is asymptotically consistent , efficient and normal . A long-standing challenge of training probabilistic models is the computational roadblocks of maximizing the log-likelihood function directly . For prescribed models , maximizing likelihood directly requires computing the partition function , which is intractable for all but the simplest models . Many powerful techniques have been developed to attack this problem , including variational methods ( Jordan et al. , 1999 ) , contrastive divergence ( Hinton , 2002 ; Welling & Hinton , 2002 ) , score matching ( Hyvärinen , 2005 ) and pseudolikelihood maximization ( Besag , 1975 ) , among others . For implicit models , the situation is even worse , as there is no term in the log-likelihood function that is in closed form ; evaluating any term requires computing an intractable integral . As a result , maximizing likelihood in this setting seems hopelessly difficult . A variety of likelihood-free solutions have been proposed that in effect minimize a divergence measure between the data distribution and the model distribution . They come in two forms : those that minimize an f -divergence , and those that minimize an integral probability metric ( Müller , 1997 ) . In the former category are GANs , which are based on the idea of minimizing the distinguishability between data and samples ( Tu , 2007 ; Gutmann & Hyvärinen , 2010 ) . It has been shown that when given access to an infinitely powerful discriminator , the original GAN objective minimizes the Jensen-Shannon divergence , the − logD variant of the objective minimizes the reverse KL-divergence minus a bounded quantity ( Arjovsky & Bottou , 2017 ) , and later extensions ( Nowozin et al. , 2016 ) minimize arbitrary f -divergences . In the latter category are GMMNs which use maximum mean discrepancy ( MMD ) ( Gretton et al. , 2007 ) as the witness function . In the case of GANs , despite the theoretical results , there are a number of challenges that arise in practice , such as mode dropping/collapse ( Goodfellow et al. , 2014 ; Arora & Zhang , 2017 ) , vanishing gradients ( Arjovsky & Bottou , 2017 ; Sinn & Rawat , 2017 ) and training instability ( Goodfellow et al. , 2014 ; Arora et al. , 2017 ) . A number of explanations have been proposed to explain these phenomena and point out that many theoretical results rely on three assumptions : the discriminator must have infinite modelling capacity ( Goodfellow et al. , 2014 ; Arora et al. , 2017 ) , the number of samples from the true data distribution must be infinite ( Arora et al. , 2017 ; Sinn & Rawat , 2017 ) and the gradient ascent-descent procedure ( Arrow et al. , 1958 ; Schmidhuber , 1992 ) can converge to a global pure-strategy Nash equilibrium ( Goodfellow et al. , 2014 ; Arora et al. , 2017 ) . When some of these assumptions do not hold , the theoretical guarantees do not necessarily apply . A number of ways have been proposed that alleviate some of these issues , e.g. , ( Zhao et al. , 2016 ; Salimans et al. , 2016 ; Donahue et al. , 2016 ; Dumoulin et al. , 2016 ; Arjovsky et al. , 2017 ; Hjelm et al. , 2017 ; Li et al. , 2017 ; Zhu et al. , 2017 ) , but a way of solving all three issues simultaneously remains elusive . 1.2 OUR CONTRIBUTION . In this paper , we present an alternative method for estimating parameters in implicit models . Like the methods above , our method is likelihood-free , but can be shown to be equivalent to maximizing likelihood under some conditions . Our result holds when the capacity of the model is finite and the number of data examples is finite . The idea behind the method is simple : it finds the nearest sample to each data example and optimizes the model parameters to pull the sample towards it . The direction in which nearest neighbour search is performed is important : the proposed method ensures each data example has a similar sample , which contrasts with an alternative approach of pushing each sample to the nearest data example , which would ensure that each sample has a similar data example . The latter approach would permit all samples being similar to one data example . Such a scenario would be heavily penalized by the former approach . The proposed method could sidestep the three issues mentioned above : mode collapse , vanishing gradients and training instability . Modes are not dropped because the loss ensures each data example has a sample nearby at optimality ; gradients do not vanish because the gradient of the distance between a data example and its nearest sample does not become zero unless they coincide ; training is stable because the estimator is the solution to a simple minimization problem . By leveraging recent advances in fast nearest neighbour search algorithms ( Li & Malik , 2016 ; 2017 ) , this approach is able to scale to large , high-dimensional datasets . 2 IMPLICIT MAXIMUM LIKELIHOOD ESTIMATOR . 2.1 DEFINITION . We are given a set of n data examples x1 , . . . , xn and some unknown parameterized probability distribution Pθ with density pθ . We also have access to an oracle that allows us to draw independent and identically distributed ( i.i.d . ) samples from Pθ . Let x̃θ1 , . . . , x̃ θ m be i.i.d . samples from Pθ , where m ≥ n. For each data example xi , we define a random variable Rθi to be the distance between xi and the nearest sample . More precisely , Rθi : = min j∈ [ m ] ∥∥x̃θj − xi∥∥22 where [ m ] denotes { 1 , . . . , m } . The implicit maximum likelihood estimator θ̂IMLE is defined as : θ̂IMLE : = arg min θ ERθ1 , ... , Rθn [ n∑ i=1 Rθi ] = arg min θ Ex̃θ1 , ... , x̃θm [ n∑ i=1 min j∈ [ m ] ∥∥x̃θj − xi∥∥22 ] 2.2 ALGORITHM . We outline the proposed parameter estimation procedure in Algorithm 1 . In each outer iteration , we drawm i.i.d . samples from the current model Pθ . We then randomly select a batch of examples from the dataset and find the nearest sample from each data example . We then run a standard iterative optimization algorithm , like stochastic gradient descent ( SGD ) , to minimize a sample-based version of the Implicit Maximum Likelihood Estimator ( IMLE ) objective . Algorithm 1 Implicit maximum likelihood estimation ( IMLE ) procedure Require : The dataset D = { xi } ni=1 and a sampling mechanism for the implicit model Pθ Initialize θ to a random vector for k = 1 to K do Draw i.i.d . samples x̃θ1 , . . . , x̃ θ m from Pθ Pick a random batch S ⊆ { 1 , . . . , n } σ ( i ) ← arg minj ∥∥xi − x̃θj∥∥22 ∀i ∈ S for l = 1 to L do Pick a random mini-batch S̃ ⊆ S θ ← θ − η∇θ ( n |S̃| ∑ i∈S̃ ∥∥∥xi − x̃θσ ( i ) ∥∥∥2 2 ) end for end for return θ Because our algorithm needs to solve a nearest neighbour search problem in each outer iteration , the scalability of our method depends on our ability to find the nearest neighbours quickly . This was traditionally considered to be a hard problem , especially in high dimensions . However , this is no longer the case , due to recent advances in nearest neighbour search algorithms ( Li & Malik , 2016 ; 2017 ) . Note that the use of Euclidean distance is not a major limitation of the proposed approach . A variety of distance metrics are either exactly or approximately equivalent to Euclidean distance in some non-linear embedding space , in which case the theoretical guarantees are inherited from the Euclidean case . This encompasses popular distance metrics used in the literature , like the Euclidean distance between the activations of a neural net , which is often referred to as a perceptual similarity metric ( Salimans et al. , 2016 ; Dosovitskiy & Brox , 2016 ) . The approach can be easily extended to use these metrics , though because this is the initial paper on this method , we focus on the vanilla setting of Euclidean distance in the natural representation of the data , e.g . : pixels , both for simplicity/clarity and for comparability to vanilla versions of other methods that do not use auxiliary sources of labelled data or leverage domain-specific prior knowledge . For distance metrics that can not be embedded in Euclidean space , the analysis can be easily adapted with minor modifications as long as the volume of a ball under the metric has a simple dependence on its radius .

Summary: The paper proposes a new algorithm for implicit maximum likelihood estimation based on a fast Nearest Neighbor search. The algorithm can be used to implicitly maximize the likelihood of models for which the former quantity is not intractable but for which sampling is easy which is typically the case for implicit models.  The paper shows that under some conditions the optimal solution of the algorithm corresponds to the MLE solution and provides some experimental evidence that the method leads to a higher likelihood. However, The paper lacks clarity and the experiments are not really convincing. Here are some remarks: