Signed Graph Metric Learning via Gershgorin Disc Perfect Alignment

Given a convex and differentiable objective <inline-formula><tex-math notation="LaTeX">$Q({\mathbf M})$</tex-math><alternatives><mml:math><mml:mrow><mml:mi>Q</mml:mi><mml:mo>(</mml:mo><mml:mi mathvariant="bold">M</mml:mi><mml:mo>)</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="yang-ieq1-3091682.gif"/></alternatives></inline-formula> for a real symmetric matrix <inline-formula><tex-math notation="LaTeX">${\mathbf M}$</tex-math><alternatives><mml:math><mml:mi mathvariant="bold">M</mml:mi></mml:math><inline-graphic xlink:href="yang-ieq2-3091682.gif"/></alternatives></inline-formula> in the positive definite (PD) cone—used to compute Mahalanobis distances—we propose a fast general metric learning framework that is entirely projection-free. We first assume that <inline-formula><tex-math notation="LaTeX">${\mathbf M}$</tex-math><alternatives><mml:math><mml:mi mathvariant="bold">M</mml:mi></mml:math><inline-graphic xlink:href="yang-ieq3-3091682.gif"/></alternatives></inline-formula> resides in a space <inline-formula><tex-math notation="LaTeX">${\mathcal S}$</tex-math><alternatives><mml:math><mml:mi mathvariant="script">S</mml:mi></mml:math><inline-graphic xlink:href="yang-ieq4-3091682.gif"/></alternatives></inline-formula> of generalized graph Laplacian matrices corresponding to balanced signed graphs. <inline-formula><tex-math notation="LaTeX">${\mathbf M}\in {\mathcal S}$</tex-math><alternatives><mml:math><mml:mrow><mml:mi mathvariant="bold">M</mml:mi><mml:mo>∈</mml:mo><mml:mi mathvariant="script">S</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="yang-ieq5-3091682.gif"/></alternatives></inline-formula> that is also PD is called a graph metric matrix. Unlike low-rank metric matrices common in the literature, <inline-formula><tex-math notation="LaTeX">${\mathcal S}$</tex-math><alternatives><mml:math><mml:mi mathvariant="script">S</mml:mi></mml:math><inline-graphic xlink:href="yang-ieq6-3091682.gif"/></alternatives></inline-formula> includes the important diagonal-only matrices as a special case. The key theorem to circumvent full eigen-decomposition and enable fast metric matrix optimization is Gershgorin disc perfect alignment (GDPA): given <inline-formula><tex-math notation="LaTeX">${\mathbf M}\in {\mathcal S}$</tex-math><alternatives><mml:math><mml:mrow><mml:mi mathvariant="bold">M</mml:mi><mml:mo>∈</mml:mo><mml:mi mathvariant="script">S</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="yang-ieq7-3091682.gif"/></alternatives></inline-formula> and diagonal matrix <inline-formula><tex-math notation="LaTeX">${\mathbf S}$</tex-math><alternatives><mml:math><mml:mi mathvariant="bold">S</mml:mi></mml:math><inline-graphic xlink:href="yang-ieq8-3091682.gif"/></alternatives></inline-formula>, where <inline-formula><tex-math notation="LaTeX">$S_{ii} = 1/v_i$</tex-math><alternatives><mml:math><mml:mrow><mml:msub><mml:mi>S</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mn>1</mml:mn><mml:mo>/</mml:mo><mml:msub><mml:mi>v</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow></mml:math><inline-graphic xlink:href="yang-ieq9-3091682.gif"/></alternatives></inline-formula> and <inline-formula><tex-math notation="LaTeX">${\mathbf v}$</tex-math><alternatives><mml:math><mml:mi mathvariant="bold">v</mml:mi></mml:math><inline-graphic xlink:href="yang-ieq10-3091682.gif"/></alternatives></inline-formula> is the first eigenvector of <inline-formula><tex-math notation="LaTeX">${\mathbf M}$</tex-math><alternatives><mml:math><mml:mi mathvariant="bold">M</mml:mi></mml:math><inline-graphic xlink:href="yang-ieq11-3091682.gif"/></alternatives></inline-formula>, we prove that Gershgorin disc left-ends of similarity transform <inline-formula><tex-math notation="LaTeX">${\mathbf B}= {\mathbf S}{\mathbf M}{\mathbf S}^{-1}$</tex-math><alternatives><mml:math><mml:mrow><mml:mi mathvariant="bold">B</mml:mi><mml:mo>=</mml:mo><mml:mi mathvariant="bold">S</mml:mi><mml:mi mathvariant="bold">M</mml:mi><mml:msup><mml:mrow><mml:mi mathvariant="bold">S</mml:mi></mml:mrow><mml:mrow><mml:mo>-</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup></mml:mrow></mml:math><inline-graphic xlink:href="yang-ieq12-3091682.gif"/></alternatives></inline-formula> are perfectly aligned at the smallest eigenvalue <inline-formula><tex-math notation="LaTeX">$\lambda _{\min }$</tex-math><alternatives><mml:math><mml:msub><mml:mi>λ</mml:mi><mml:mo movablelimits="true" form="prefix">min</mml:mo></mml:msub></mml:math><inline-graphic xlink:href="yang-ieq13-3091682.gif"/></alternatives></inline-formula>. Using this theorem, we replace the PD cone constraint in the metric learning problem with tightest possible linear constraints per iteration, so that the alternating optimization of the diagonal / off-diagonal terms in <inline-formula><tex-math notation="LaTeX">${\mathbf M}$</tex-math><alternatives><mml:math><mml:mi mathvariant="bold">M</mml:mi></mml:math><inline-graphic xlink:href="yang-ieq14-3091682.gif"/></alternatives></inline-formula> can be solved efficiently as linear programs via the Frank-Wolfe method. We update <inline-formula><tex-math notation="LaTeX">${\mathbf v}$</tex-math><alternatives><mml:math><mml:mi mathvariant="bold">v</mml:mi></mml:math><inline-graphic xlink:href="yang-ieq15-3091682.gif"/></alternatives></inline-formula> using Locally Optimal Block Preconditioned Conjugate Gradient (LOBPCG) with warm start as entries in <inline-formula><tex-math notation="LaTeX">${\mathbf M}$</tex-math><alternatives><mml:math><mml:mi mathvariant="bold">M</mml:mi></mml:math><inline-graphic xlink:href="yang-ieq16-3091682.gif"/></alternatives></inline-formula> are optimized successively. Experiments show that our graph metric optimization is significantly faster than cone-projection schemes, and produces competitive binary classification performance.