Multi-objective training of Generative Adversarial Networks with multiple discriminators

Recent literature has demonstrated promising results on the training of Generative Adversarial Networks by employing a set of discriminators , as opposed to the traditional game involving one generator against a single adversary . Those methods perform single-objective optimization on some simple consolidation of the losses , e.g . an average . In this work , we revisit the multiple-discriminator approach by framing the simultaneous minimization of losses provided by different models as a multi-objective optimization problem . Specifically , we evaluate the performance of multiple gradient descent and the hypervolume maximization algorithm on a number of different datasets . Moreover , we argue that the previously proposed methods and hypervolume maximization can all be seen as variations of multiple gradient descent in which the update direction computation can be done efficiently . Our results indicate that hypervolume maximization presents a better compromise between sample quality and diversity , and computational cost than previous methods . 1 INTRODUCTION . Generative Adversarial Networks ( GANs ) ( Goodfellow et al. , 2014 ) offer a new approach to generative modeling , using game-theoretic training schemes to implicitly learn a given probability density . Prior to the emergence of GAN architectures , realistic generative modeling remained elusive . When offering unparalleled realism , GAN training remains fraught with stability issues . Commonly reported shortcomings involved in the GAN game are the lack of useful gradients provided by the discriminator , and mode collapse , i.e . lack of diversity in the generator ’ s samples . Considerable research effort has been devoted in recent literature in order to overcome training instability 1 within the GAN framework . Some architectures such as BEGAN ( Berthelot et al. , 2017 ) have applied auto-encoders as discriminators and proposed a new loss to help stabilize training . Methods such as TTUR ( Heusel et al. , 2017 ) , in turn , have attempted to define schedules for updating the generator and discriminator differently . The PacGAN algorithm ( Lin et al. , 2017 ) proposes to modify the discriminator ’ s architecture which will receive m concatenated samples as input , while modifications to alternate updates in SGD were introduced in ( Yadav et al. , 2017 ) . These samples are jointly classified as either real or generated , and authors show that this enforces sample diversity . In SNGAN ( Miyato et al. , 2018 ) , authors introduce spectral normalization on the discriminator aiming to ensure Lipschitz continuity , which is empirically shown to consistently yield high quality samples when different sets of hyperparameters are used . Recent works have proposed to tackle GANs instability issues using multiple discriminators . Neyshabur et al . ( 2017 ) propose a GAN variation in which one generator is trained against a set of discriminators , where each discriminator sees a fixed random projection of the inputs . Prior work , including GMAN ( Durugkar et al. , 2016 ) has also explored training against multiple discriminators . 1Instability in the sense commonly used in GANs literature , i.e . divergence and mode-collapse of the generator when the discriminator is able to easily distinguish real and fake samples during training ( Neyshabur et al. , 2017 ; Arjovsky et al. , 2017 ; Berthelot et al. , 2017 ) . In this paper , we build upon Neyshabur et al. ’ s introduced framework and propose reformulating the average loss minimization aiming to further stabilize GAN training . Specifically , we propose treating the loss signal provided by each discriminator as an independent objective function . To achieve this , we simultaneously minimize the losses using multi-objective optimization techniques . Namely , we exploit previously introduced methods in literature such as the multiple gradient descent algorithm ( MGD ) ( Désidéri , 2012 ) . However , due to MGD ’ s prohibitively high cost in the case of large neural networks , we propose the use of more efficient alternatives such as maximization of the hypervolume of the region defined between a fixed , shared upper bound on those losses , which we will refer to as the nadir point η∗ , and each of the component losses . In contrast to Neyshabur et al . ( 2017 ) ’ s approach , where the average loss is minimized when training the generator , hypervolume maximization ( HV ) optimizes a weighted loss , and the generator ’ s training will adaptively assign greater importance to feedback from discriminators against which it performs poorly . Experiments performed on MNIST show that HV presents a good compromise in the computational cost-samples quality trade-off , when compared to average loss minimization or GMAN ’ s approach ( low quality and cost ) , and MGD ( high quality and cost ) . Also , the sensitivity to introduced hyperparameters is studied and results indicate that increasing the number of discriminators consequently increases the generator ’ s robustness along with sample quality and diversity . Experiments on CIFAR10 indicate the method described produces higher quality generator samples in terms of quantitative evaluation . Moreover , image quality and sample diversity are once more shown to consistently improve as we increase the number of discriminators . In summary , our main contributions are the following : 1 . We offer a new perspective on multiple-discriminator GAN training by framing it in the context of multi-objective optimization , and draw similarities between previous research in GANs variations and MGD , commonly employed as a general solver for multi-objective optimization . 2 . We propose a new method for training multiple-discriminator GANs : Hypervolume maximization , which weighs the gradient contributions of each discriminator by its loss . The remainder of this document is organized as follows : Section 2 introduces definitions on multiobjective optimization and MGD . In Section 3 we describe prior relevant literature . Hypervolume maximization is detailed in Section 4 , with experiments and results presented in Section 5 . Conclusions and directions for future work are drawn in Section 6 . 2 PRELIMINARIES . In this section we provide some definitions regarding multi-objective optimization literature which will be useful in the next sections . Henceforth , the boldface notation will be used to indicate vector-valued variables . Multi-objective optimization . A multi-objective optimization problem is defined as ( Deb , 2001 ) : min F ( x ) = [ f1 ( x ) , f2 ( x ) , ... , fK ( x ) ] T , x ∈ Ω , ( 1 ) where K is the number of objectives , Ω is the variables space and x = [ x1 , x2 , ... , xn ] T ∈ Ω is a decision vector or possible solution to the problem . F : Ω→ RK is a set of K-objective functions that maps the n-dimensional variables space to the K-dimensional objective space . Pareto-dominance . Let x1 and x2 be two decision vectors . x1 is said to dominate x2 ( denoted by x1 ≺ x2 ) if and only if fi ( x1 ) ≤ fi ( x2 ) for all i ∈ { 1 , 2 , . . . , K } and fj ( x1 ) < fj ( x2 ) for some j ∈ { 1 , 2 , . . . , K } . If a decision vector x is dominated by no other vector in Ω , x is said to be non-dominated . Pareto-optimality . A decision vector x∗ ∈ Ω is said to be Pareto-optimal if and only if there is no x ∈ Ω such that x ≺ x∗ , i.e . x∗ is a non-dominated solution . The Pareto-optimal Set ( PS ) is defined as the set of all Pareto-optimal solutions x ∈ Ω , i.e. , PS = { x ∈ Ω|x is Pareto optimal } . The set of all objective vectors F ( x ) such that x is Pareto-optimal is called Pareto front ( PF ) , that is PF = { F ( x ) ∈ RK |x ∈ PS } . Pareto-stationarity . Pareto-stationarity is a necessary condition for Pareto-optimality . For fk differentiable everywhere for all k , F is said to be Pareto-stationary at the point x if there exists a set of scalars αk , k ∈ { 1 , . . . , K } , such that : K∑ k=1 αk∇fk = 0 , K∑ k=1 αk = 1 , αk ≥ 0 ∀k . ( 2 ) Multiple Gradient Descent . Multiple gradient descent ( Désidéri , 2012 ; Schäffler et al. , 2002 ; Peitz & Dellnitz , 2018 ) was proposed for the unconstrained case of multi-objective optimization of F ( x ) assuming a convex , continuously differentiable and smooth fk ( x ) for all k. MGD finds a common descent direction for all fk by defining the convex hull of all ∇fk ( x ) and finding the minimum norm element within it . Consider w∗ given by : w∗ = argmin||w|| , w = K∑ k=1 αk∇fk ( x ) , s.t . K∑ k=1 αk = 1 , αk ≥ 0 ∀k . ( 3 ) w∗ will be either 0 in which case x is a Pareto-stationary point , or w∗ 6= 0 and then w∗ is a descent direction for all fi ( x ) . Similar to gradient descent , MGD consists in finding the common steepest descent direction w∗t at each iteration t , and then updating parameters with a learning rate λ according to xt+1 = xt − λ w ∗ t ||w∗t || . 3 RELATED WORK . 3.1 TRAINING GANS WITH MULTIPLE DISCRIMINATORS . While we would prefer to always have strong gradients from the discriminator during training , the vanilla GAN makes this difficult to ensure , as the discriminator quickly learns to distinguish real and generated samples ( Goodfellow , 2016 ) , thus providing no meaningful error signal to improve the generator thereafter . Durugkar et al . ( 2016 ) proposed the Generative Multi-Adversarial Networks ( GMAN ) which consist in training the generator against a softmax weighted arithmetic average of K different discriminators , according to Eq . 4 . LG = K∑ k=1 αkLDk , ( 4 ) where αk = e βLDk∑K j=1 e βLDj , β ≥ 0 , and LDk is the loss of discriminator k and defined as LDk = −Ex∼pdata logDk ( x ) − Ez∼pz log ( 1−Dk ( G ( z ) ) ) , ( 5 ) where Dk ( x ) and G ( z ) are the outputs of the k-th discriminator and the generator , respectively . The goal of using the proposed averaging scheme is to privilege worse discriminators and thus providing more useful gradients to the generator during training . Experiments were performed with β = 0 ( equal weights ) , β → ∞ ( only worst discriminator is taken into account ) , β = 1 , and β learned by the generator . Models with K = { 2 , 5 } were tested and evaluated using a proposed metric and the Inception score ( Salimans et al. , 2016 ) . However , results showed that the simple average of discriminator ’ s losses provided the best values for both metrics in most of the considered cases . Opposed to GMAN , Neyshabur et al . ( 2017 ) proposed training a GAN with K discriminators using the same architecture . Each discriminator Dk sees a different randomly projected lower-dimensional version of the input image . Random projections are defined by a randomly initialized matrix Wk , which remains fixed during training . Theoretical results provided show that the distribution induced by the generator G will converge to the real data distribution pdata , as long as there is a sufficient number of discriminators . Moreover , discriminative tasks in the projected space are harder , i.e . real and fake samples are more alike , thus avoiding early convergence of discriminators , which leads to common stability issues in GAN training such as mode-collapse ( Goodfellow , 2016 ) . Essentially , the authors trade one hard problem for K easier subproblems . The losses of each discriminator LDk are the same as shown in Eq . 5 . However , the generator loss LG is defined as simply the sum of the losses provided by each discriminator , as shown in Eq . 6 . This choice of LG does not exploit available information such as the performance of the generator with respect to each discriminator . LG = − K∑ k=1 Ez∼pz logDk ( G ( z ) ) . ( 6 )

Summary: The work is a clear introduction/overview of this area of research. The reviewer enjoyed the connections to Multiple-Gradient Descent and clear distinctions/contrasts with previous approaches to weighting the outputs of multiple discriminators. All in all, the paper is quite clear in what its contributions are and how it differs from previous approaches. The details and motivations of the Hypervolume Maximization  (HVM) method (especially as it relates to and interacts with the slack method of picking the nadir point) were a bit harder to follow intuitively given the standalone information in the paper.