Deep Learning of Determinantal Point Processes via Proper Spectral Sub-gradient

1 INTRODUCTION . Diversity is desired in multiple machine learning and computer vision tasks ( e.g. , image hashing ( Chen et al. , 2017 ; Carreira-Perpinán & Raziperchikolaei , 2016 ) , descriptor learning ( Zhang et al. , 2017 ) , metric learning ( Mishchuk et al. , 2017 ) and video summarization ( Sharghi et al. , 2018 ; Liu et al. , 2017 ) ) , in which sub-sampled points or learned features need to spread out through a specific bounded space . Originated from quantum physics , determinantal point processes ( DPP ) have shown its power in delivering such properties ( Kulesza et al. , 2012 ; Kulesza & Taskar , 2011b ) . Compared with other diversity-oriented techniques ( e.g. , entropy ( Zadeh et al. , 2017 ) and orthogonality ( Zhang et al. , 2017 ) ) , DPP shows its superiority as it incorporates only one single metric and delivers genuine diversity on any bounded space ( Kulesza et al. , 2012 ; Affandi et al. , 2013 ; Gillenwater et al. , 2012 ) . Therefore , DPP has been utilized in a large body of diversity-oriented tasks . In general , sample points from a DPP tend to distribute diversely within a bounded space A ( Kulesza et al. , 2012 ) . Given a positive semi-definite kernel function κ : A × A → R , the probability of a discrete point set X ⊂ A under a DPP with kernel function κ can be characterized as : Pκ ( X ) ∝ det ( LX ) ( 1 ) where L is a |X | × |X | matrix with entry Lij = κ ( xi , xj ) and xi , xj ∈ X . L is called L-ensemble . Note that A is a continuous space , whereas X is finite . In the Hilbert space associated with κ , larger determinant implies larger spanned volume , thus the mapped points tend not to be similar or linearly dependent . DPP can be viewed from two perspectives : sampling and learning . A comprehensive introduction to mathematical fundamentals of DPP for sampling from a discrete space can be found in Kulesza et al . ( 2012 ) . Based on this , a line of works has been proposed ( Kulesza & Taskar , 2011a ; Kang , 2013 ; Hennig & Garnett , 2016 ) . In this paper , we concentrate on learning DPPs . In learning of DPP , the term det ( L ) is typically treated as a singleton diversity measurement and is extended to learning paradigms on continuous space ( Chao et al. , 2015 ; Kulesza & Taskar , 2010 ; Affandi et al. , 2014 ) . There are generally two lines of strategies to learn DPPs : Approximation . This type of methods is to convert DPP into a simpler format which can ease and stabilize the computation . low-rank approximation proves powerful in easing the computational burden ( Gartrell et al. , 2017 ) , in which the gram matrix is factorized as L = BB > where B ∈ < n×m with m n. This decomposition can also reduce the complexity which is originally a cubic time of |L| . Kulesza & Taskar ( 2011b ) explicitly expressed the kernel with κ ( x , y ) = σ1σ2δ ( x ) > δ ( y ) , where σ measures the intrinsic quality of the feature and δ ( · ) is function mapping input x to a feature space . In this sense , the pairwise similarity is calculated in Euclidean feature space with cosine distance . Elfeki et al . ( 2019 ) suggest approximating a given distribution by approximating the eigenvalues of the corresponding DPP . As such , the computation can be eased and become stable . Following this , DPP is also applied on some visual tasks , such as video summarization ( Sharghi et al. , 2018 ) , ranking ( Liu et al. , 2017 ) and image classification ( Xie et al. , 2017 ) . It can be noted that the approximation is not straightforward for DPP , thus can not fully deliver the diversity property ( e.g . resulting in rank-deficiency ) . Direct optimization . While the aforementioned methods optimize DPP with specific approximation , a series of efforts also seek to optimize the DPP term directly ( Gillenwater et al. , 2014 ; Mariet & Sra , 2015 ; Bardenet & Titsias , 2015 ) . In this setting , the whole gram matrix L corresponding to the pairwise similarity among features is updated directly , which allows accommodating more flexible feature mapping functions rather than an approximation . Gillenwater et al . ( 2014 ) proposed an Expectation-Maximization algorithm to update marginal kernel DPP K = L ( L + I ) −1 , together with a baseline K-Ascent derived from projected gradient ascent ( Levitin & Polyak , 1966 ) . Mariet & Sra ( 2015 ) extended DPP from a fixed-point perspective and Bardenet & Titsias ( 2015 ) proposed to optimize DPP upon a lower bound in variational inference fashion . A key problem of such line of works is that the computation is not differentiable , making it difficult to be used in deep learning frameworks . To the best of our knowledge , there is no previous method incorporating DPP as a feature-level diversity metric in deep learning . A key difficulty in doing so is that the calculation of the gradient of det ( L ) involves matrix inversion , which can be unstable and inaccurate in GPUs . Though KAscent seems to be a naive rule , it still needs explicit matrix inversion in the first step before the projection procedure . This fact greatly hinders the tight integration of DPP with deep networks . Some alternative methods seek to reach diversity under more constrained settings . For example , Zhang et al . ( 2017 ) resorted to a global pairwise orthogonality constraint in hyper-sphere and Zadeh et al . ( 2017 ) employed statistical moments to measure the diversity . However , compared with DPP , such measurements are unable to fully characterize diversity in an arbitrary bounded space . In this paper , rather than providing more efficient DPP solvers , we concentrate on delivering a feasible feature-level DPP integration under the deep learning framework . To this end , we revisit the spectral decomposition of DPP and propose a sub-gradient generation method which can be tightly integrated with deep learning . Our method differs from either approximation or direct optimization by introducing a “ differentiable direct optimization ” procedure , thus can produce genuinely diverse features in continuous bounded space . Our method is stable and scalable to the relatively large dataset with a specific mini-batch sampling strategy , which is verified by several experiments on various tasks . Notations : Bold lower case x and bold upper case K represent vector and matrix , respectively . det ( · ) and Tr ( · ) calculate the determinant and trace of a matrix , respectively . A⊗B is the element-wise product of matrices A and B . |X | and |x| measure the cardinality of a finite set X and the L2 length of a vector x , respectively . 〈x , y〉 calculates the inner product of the two vectors . x = diag ( X ) transforms a diagonal matrix X into its vector form x , and vice versa . We refer “ positive semi-definite ” and “ positive definite ” to PSD and PD , respectively . Denote < the real numbers . 2 BACKGROUND . 2.1 DETERMINANTAL POINT PROCESS . L-ensemble expression of DPP requires L to be PSD , whereas kernel expression further constrains K < I ( each eigenvalue of K is less than 1 ) . A conversion from L to K can thus be written as K = L ( L + I ) −1 following the truth ∑ X det ( LX ) = det ( L + I ) , which is the marginal normalization constant given a specific L. While there is always conversion from L to K , the inverse may not exist ( Kulesza et al. , 2012 ) . In practice , one may construct L-ensemble first , then normalize it into a marginal kernel . This fact may give rise to the difficulty of deep networks . Since a conversion from K to L might not exist , the network needs carefully adjusting the gradients under specific constraints to ensure the updated L to be valid . As L and K share the same eigenvectors vi , a pair of L and K holds the relation : K = ∑ i λiviv > i ⇐⇒ L = ∑ i λi 1− λi viv > i ( 2 ) where λi is the ith eigenvalue . It is seen that such conversion is not straightforward to be directly integrated with deep learning framework . Therefore , we optimize ensemble L directly in this paper . 2.2 GAUSSIAN KERNEL . We briefly introduce Gaussian kernel in this section , which works on Hilbert space with infinite dimension . Mercer ’ s theorem Friedman et al . ( 2001 ) ensures the PSD properties when constructing new kernels with existing ones under a specific procedure . Such procedure is also employed in multiple kernel learning paradigms ( Affandi et al. , 2014 ; Kulesza & Taskar , 2011b ; Chao et al. , 2015 ) , which is out of the scope of this paper . A Gaussian kernel is defined as κ ( xi , xj ) = exp ( −|xi − xj |2/σ2 ) , where σ is a controlling parameter . Thus an L-ensemble matrix becomes Lij = κ ( xi , xj ) . According to the definition , Lii = 1 and for any element in the matrix we have Lij ∈ ( 0 , 1 ] . With Gaussian kernel , we have a nice property 0 ≤ det ( L ) ≤ 1 . This can be easily verified by applying geometric inequality to the eigenvalues of L. Although not tight , this property shows that the determinant value with Gaussian kernel is bounded . This fact inspires one version of our algorithm detailed in the next section . Throughout this paper , our discussion is based on the Gaussian kernel unless specified . 3 METHOD . Given vectorized inputs Ii ∈ Rh where i = 1 , ... , n , our goal is to learn a map f such that the features xi = f ( Ii ) can spread out within a bounded feature space xi ∈ S. Hereafter we refer space to an Euclidean bounded space ( e.g. , [ −1 , 1 ] d ) without loss of generality . Given any loss function J , the chain rule of gradient involving DPP is written as : ∆J = ∂J ∂ det ( L ) ∂ det ( L ) ∂L ∂L ∂X ( 3 ) where X refer to the features before DPP layer . While calculating ∂J/∂ det ( L ) and ∂L/∂X is straightforward , the main difficulty lies on the calculation of ∂ det ( L ) /∂L . We will discuss the calculation of this term under two case : 1 ) When the inversion L−1 can be stably obained , we will derive the gradient of DPP det ( L ) on Sec 3.1 ; When L is not invertible or L−1 is difficult to calculate , we give the procedure to handle the case by generating valid sub-gradient in Sec 3.2 . Since our objective is to diverse features , det ( L ) will serve as a ( partial ) objective term to be directly maximized . Algorithm 1 DPPSG Input : K , tolerance ∆ ; Output : K̄ UΛU > ← K ( σ1 , ... , σn ) ← diag ( K ) for i in { 1 , ... , n } do if σi < ∆ then σi ← ∆ end if end for Λ̂← diag ( σ1 , ... , σn ) K̂← UΛ̂U > K̄← K̂−K Algorithm 2 DPPSG * Input : K , tolerance θ ; Output : K̄ UΛU > ← K ( σ1 , ... , σn ) ← diag ( K ) b = ( 1− θ ) ( σ1 , ... , σn ) − θ ( 1 , ... , 1 ) Λ̂← diag ( b ) K̂← UΛ̂U > K̄← K̂−K

Summary: Determinantal Point Processes (DPPs) are statistical models that allow efficient sampling of diverse solutions - a problem that is hard with most machine learning modeling frameworks.  However, learning diverse features via DPPs with deep learning frameworks is challenging due the instability in computing the gradient which involves a matrix inverse operation. Better marriage between deep learning and DPPs is an important problem as diversity is crucial in many machine learning problems (even beyond computer vision - the experimental domain of this paper - e.g. in machine translation and document summarization).