manifold based constraints for operations in face space in this paper we constrain faces to points on a manifold within the parameter space of a linear statistical model the manifold is the subspace of faces which have maximally likely distinctiveness and different points correspond to unique identities we provide a detailed empirical validation for the chosen manifold we show how the log and exponential maps for a hyperspherical manifold can be used to replace linear operations such as warping and averaging with operations on this manifold finally we use the manifold to develop a new method for fitting a statistical face shape model to data which is both robust avoids overfitting and overcomes model dominance is not susceptible to local minima close to the mean face we provide experimental results for fitting a dense 3d morphable face model to data using two different objective functions one underconstrained and one with many local minima our method outperforms generic nonlinear optimisers based on the bfgs quasi newton method and the levenberg marquardt algorithm when fitting using the basel face model highlightswe decompose statistical face models into identity and distinctiveness subspaces the identity subspace forms a hyperspherical manifold that we validate empirically the manifold provides non linear alternatives to warping and averaging we use the manifold to constrain optimisation based model fitting this outperforms two existing algorithms on over and under constrained problems

efficient rank reduction of correlation matrices geometric optimisation algorithms are developed that efficiently find the nearest low rank correlation matrix we show in numerical tests that our methods compare favourably with the existing methods in the literature the connection with the lagrange multiplier method is established along with an identification of whether a local minimum is a global minimum an additional benefit of the geometric approach is that any weighted norm can be applied the problem of finding the nearest low rank correlation matrix occurs as part of the calibration of multi factor interest rate market models to correlation

alternating projections on manifolds we prove that if two smooth manifolds intersect transversally then the method of alternating projections converges locally at a linear rate we bound the speed of convergence in terms of the angle between the manifolds which in turn we relate to the modulus of metric regularity for the intersection problem a natural measure of conditioning we discuss a variety of problem classes where the projections are computationally tractable and we illustrate the method numerically on a problem of finding a low rank solution of a matrix equation

low rank nonnegative matrix factorization on stiefel manifold abstract low rank is an important but ill posed problem in the development of nonnegative matrix factorization nmf algorithms because the essential information is often encoded in a low rank intrinsic data matrix whereas noise and outliers are contained in a residue matrix most existing nmf approaches achieve low rank by directly specifying the dimensions of the factor matrices a few others impose the low rank constraint on the factor matrix and use the alternating direction method of multipliers to solve the optimization problem in contrast to previous approaches this paper proposes a novel method for low rank nonnegative matrix factorization on a stiefel manifold lnmfs which utilizes the low rank structure of intrinsic data and transforms it into a frobenius norm of the latent factors to obtain orthogonal factors as distinct patterns we further impose orthogonality constraints by assuming that the basis matrix lies on a stiefel manifold in addition to improve the robustness of the data in a manifold structure we incorporate the graph smoothness of the coefficient matrix finally we develop an efficient alternative iterative algorithm to solve the optimization problem and provide proof of its convergence extensive experiments on real world datasets demonstrate the superiority of the proposed method compared with other representative nmf based algorithms

a riemannian gradient sampling algorithm for nonsmooth optimization on manifolds in this paper an optimization method for nonsmooth locally lipschitz functions on complete riemannian manifolds is presented the method is based on approximating the subdifferential of the cost function at every iteration by the convex hull of transported gradients from tangent spaces at randomly generated nearby points to the tangent space of the current iterate and can hence be seen as a generalization of the well known gradient sampling algorithm to a riemannian setting a convergence result is obtained under the assumption that the cost function is bounded below and continuously differentiable on an open set of full measure and that the employed vector transport and retraction satisfy certain conditions which hold for instance for the exponential map and parallel transport then with probability one the algorithm produces iterates at which the cost function is differentiable and each cluster point of the iterates is a clarke stationary point modifications yielding only varepsilon stationary

comparison results between linear and quasilinear elliptic boundary value problems on considere des resultats de comparaison du type suivant on suppose que u c 2 c satisfait un probleme aux valeurs limites elliptique quasi lineaire div a x u u u b x u u dans u sur et on suppose que v est la solution classique du probleme lineaire v const dans v sur on etudie l existence de situations ou u x v x x

heterogeneous tensor decomposition for clustering via manifold optimization tensor clustering is an important tool that exploits intrinsically rich structures in real world multiarray or tensor datasets often in dealing with those datasets standard practice is to use subspace clustering that is based on vectorizing multiarray data however vectorization of tensorial data does not exploit complete structure information in this paper we propose a subspace clustering algorithm without adopting any vectorization process our approach is based on a novel heterogeneous tucker decomposition model taking into account cluster membership information we propose a new clustering algorithm that alternates between different modes of the proposed heterogeneous tensor model all but the last mode have closed form updates updating the last mode reduces to optimizing over the multinomial manifold for which we investigate second order riemannian geometry and propose a trust region algorithm numerical experiments show that our proposed algorithm compete effectively with state of the art clustering algorithms that are based on tensor factorization

a nonlinear programming algorithm for solving semidefinite programs via low rank factorization in this paper we present a nonlinear programming algorithm for solving semidefinite programs sdps in standard form the algorithm s distinguishing feature is a change of variables that replaces the symmetric positive semidefinite variable x of the sdp with a rectangular variable r according to the factorization x rr t the rank of the factorization i e the number of columns of r is chosen minimally so as to enhance computational speed while maintaining equivalence with the sdp fundamental results concerning the convergence of the algorithm are derived and encouraging computational results on some large scale test problems are also presented

riemannian optimization for high dimensional tensor completion tensor completion aims to reconstruct a high dimensional data set where the vast majority of entries is missing the assumption of low rank structure in the underlying original data allows us to cast the completion problem into an optimization problem restricted to the manifold of fixed rank tensors elements of this smooth embedded submanifold can be efficiently represented in the tensor train or matrix product states format with storage complexity scaling linearly with the number of dimensions we present a nonlinear conjugate gradient scheme within the framework of riemannian optimization which exploits this favorable scaling numerical experiments and comparison to existing methods show the effectiveness of our approach for the approximation of multivariate functions finally we show that our algorithm can obtain competitive reconstructions from uniform random sampling of few entries compared to adaptive sampling techniques such as cross approximation

reduction of nonlinear embedded boundary models for problems with evolving interfaces embedded boundary methods alleviate many computational challenges including those associated with meshing complex geometries and solving problems with evolving domains and interfaces developing model reduction methods for computational frameworks based on such methods seems however to be challenging indeed most popular model reduction techniques are projection based and rely on basis functions obtained from the compression of simulation snapshots in a traditional interface fitted computational framework the computation of such basis functions is straightforward primarily because the computational domain does not contain in this case a fictitious region this is not the case however for an embedded computational framework because the computational domain typically contains in this case both real and ghost regions whose definitions complicate the collection and compression of simulation snapshots the problem is exacerbated when the interface separating both regions evolves in time this paper addresses this issue by formulating the snapshot compression problem as a weighted low rank approximation problem where the binary weighting identifies the evolving component of the individual simulation snapshots the proposed approach is application independent and therefore comprehensive it is successfully demonstrated for the model reduction of several two dimensional vortex dominated fluid structure interaction problems

a direct formulation for sparse pca using semidefinite programming given a covariance matrix we consider the problem of maximizing the variance explained by a particular linear combination of the input variables while constraining the number of nonzero coefficients in this combination this problem arises in the decomposition of a covariance matrix into sparse factors or sparse principal component analysis pca and has wide applications ranging from biology to finance we use a modification of the classical variational representation of the largest eigenvalue of a symmetric matrix where cardinality is constrained and derive a semidefinite programming based relaxation for our problem we also discuss nesterov s smooth minimization technique applied to the semidefinite program arising in the semidefinite relaxation of the sparse pca problem the method has complexity o n 4 sqrt log n epsilon where n is the size of the underlying covariance matrix and epsilon is the desired absolute accuracy on the optimal value of the problem

the two stage iteration algorithms based on the shortest distance for low rank matrix completion abstract despite matrix completion requiring the global solution of a non convex objective there are many computational efficient algorithms which are effective for a broad class of matrices based on these algorithms for matrix completion with given rank problem we propose a class of two stage iteration algorithms for general matrix completion in this paper the inner iteration is the scaled alternating steepest descent algorithm for the fixed rank matrix completion problem presented by tanner and wei 2016 the outer iteration is used two iteration criterions the gradient norm and the distance between the feasible part with the corresponding part of reconstructed low rank matrix the feasibility of the two stage algorithms are proved finally the numerical experiments show the two stage algorithms with shorting the distance are more effective than other algorithms

a broyden class of quasi newton methods for riemannian optimization this paper develops and analyzes a generalization of the broyden class of quasi newton methods to the problem of minimizing a smooth objective function f on a riemannian manifold a condition on vector transport and retraction that guarantees convergence and facilitates efficient computation is derived experimental evidence is presented demonstrating the value of the extension to the riemannian broyden class through superior performance for some problems compared to existing riemannian bfgs methods in particular those that depend on differentiated retraction

minimal subspace rotation on the stiefel manifold for stabilization and enhancement of projection based reduced order models for the compressible navier stokes equations for a projection based reduced order model rom of a fluid flow to be stable and accurate the dynamics of the truncated subspace must be taken into account this paper proposes an approach for stabilizing and enhancing projection based fluid roms in which truncated modes are accounted for a priori via a minimal rotation of the projection subspace attention is focused on the full non linear compressible navier stokes equations in specific volume form as a step toward a more general formulation for problems with generic non linearities unlike traditional approaches no empirical turbulence modeling terms are required and consistency between the rom and the navier stokes equation from which the rom is derived is maintained mathematically the approach is formulated as a trace minimization problem on the stiefel manifold the reproductive as well as predictive capabilities of the method are evaluated on several compressible flow problems including a problem involving laminar flow over an airfoil with a high angle of attack and a channel driven cavity flow problem

global registration of multiple point clouds using semidefinite programming consider n points in r d and m local coordinate systems that are related through unknown rigid transforms for each point we are given possibly noisy measurements of its local coordinates in some of the coordinate systems alternatively for each coordinate system we observe the coordinates of a subset of the points the problem of estimating the global coordinates of the n points up to a rigid transform from such measurements comes up in distributed approaches to molecular conformation and sensor network localization and also in computer vision and graphics the least squares formulation of this problem although nonconvex has a well known closed form solution when m 2 based on the singular value decomposition svd however no closed form solution is known for m geq 3 in this paper we demonstrate how the least squares formulation can be relaxed into a convex program namely a semidefinite program sdp by setting up connections between the uniqueness of this sdp and results fr

the geometry of algorithms using hierarchical tensors in this paper the differential geometry of the novel hierarchical tucker format for tensors is derived the set ht k of tensors with fixed tree t and hierarchical rank k is shown to be a smooth quotient manifold namely the set of orbits of a lie group action corresponding to the non unique basis representation of these hierarchical tensors explicit characterizations of the quotient manifold its tangent space and the tangent space of ht k are derived suitable for high dimensional problems the usefulness of a complete geometric description is demonstrated by two typical applications first new convergence results for the nonlinear gauss seidel method on ht k are given notably and in contrast to earlier works on this subject the task of minimizing the rayleigh quotient is also addressed second evolution equations for dynamic tensor approximation are formulated in terms of an explicit projection operator onto the tangent space of ht k in addition a numerical comparison is made between this dynamical approach and the standard one based on truncated singular value decompositions 2013 elsevier inc

riemannian optimal model reduction of linear port hamiltonian systems abstract in this paper we describe the development of a riemannian optimal model reduction method for linear stable port hamiltonian systems this development is motivated by the fact that there remains room for improvement in existing methods the model reduction problem is formulated as an optimization problem on the product manifold of the set of skew symmetric matrices the manifold of the symmetric positive definite matrices and euclidean space the reduced systems constructed using the optimal solutions to the problem preserve the original structure i e stability passivity and the port hamiltonian form the riemannian gradient is derived to relate our problem to another problem in some studies and the hessian is also derived to solve our problem using a riemannian trust region method the initial point in the proposed method is chosen by using the output of the iterative rational krylov algorithm for linear port hamiltonian systems irka ph a numerical experiment illustrates that the proposed method considerably improves the results of irka ph when the reduced model dimension is small

toeplitz matrix completion via smoothing augmented lagrange multiplier algorithm abstract toplitz matrix completion tmc is to fill a low rank toeplitz matrix from a small subset of its entries based on the augmented lagrange multiplier alm algorithm for matrix completion in this paper we propose a new algorithm for the tmc problem using the smoothing technique of the approximation matrices the completion matrices generated by the new algorithm are of toeplitz structure throughout iteration which save computational cost of the singular value decomposition svd and approximate well the solution convergence results of the new algorithm are proved finally the numerical experiments show that the augmented lagrange multiplier algorithm with smoothing is more effective than the original alm and the accelerated proximal gradient apg algorithms

convergence results for projected line search methods on varieties of low rank matrices via łojasiewicz inequality the aim of this paper is to derive convergence results for projected line search methods on the real algebraic variety mathcal m le k of real m times n matrices of rank at most k such methods extend riemannian optimization methods which are successfully used on the smooth manifold mathcal m k of rank k matrices to its closure by taking steps along gradient related directions in the tangent cone and afterwards projecting back to mathcal m le k considering such a method circumvents the difficulties which arise from the nonclosedness and the unbounded curvature of mathcal m k the pointwise convergence is obtained for real analytic functions on the basis of a ojasiewicz inequality for the projection of the antigradient to the tangent cone if the derived limit point lies on the smooth part of mathcal m le k i e in mathcal m k this boils down to more or less known results but with the benefit that asymptotic convergence rate estimates for specific step sizes

low rank matrix completion by riemannian optimization the matrix completion problem consists of finding or approximating a low rank matrix based on a few samples of this matrix we propose a new algorithm for matrix completion that minimizes the least square distance on the sampling set over the riemannian manifold of fixed rank matrices the algorithm is an adaptation of classical nonlinear conjugate gradients developed within the framework of retraction based optimization on manifolds we describe all the necessary objects from differential geometry necessary to perform optimization over this low rank matrix manifold seen as a submanifold embedded in the space of matrices in particular we describe how metric projection can be used as retraction and how vector transport lets us obtain the conjugate search directions finally we prove convergence of a regularized version of our algorithm under the assumption that the restricted isometry property holds for incoherent matrices throughout the iterations the numerical experiments indicate that our approach

derivatives of spectral functions a spectral function of a hermitian matrix x is a function which depends only on the eigenvalues of x 1x 2x nx and hence may be written f 1x 2x nx for some symmetric function f such functions appear in a wide variety of matrix optimization problems we give a simple proof that this spectral function is differentiable at x if and only if the function f is differentiable at the vector x and we give a concise formula for the derivative we then apply this formula to deduce an analogous expression for the clarke generalized gradient of the spectral function a similar result holds for real symmetric matrices

a new approach to queer differential equations abstract queer differential equations first arose in the work of harold grad on controlled thermonuclear fusion these relate in particular to models for the slow adiabatic evolution and resistive diffusion of a plasma these are queer in that these share aspects of partial ordinary and functional differential equations in this document the authors give a new way of thinking of these equations by relating these to free boundary problems this is the first of a series of papers intending to demonstrate that solutions of such a queer differential equation can be thought of as limits of solutions of free boundary problems with n free boundaries a long term hope is that this work will complement and further refine existing numerical schemes for finding such solutions

preserving lagrangian structure in nonlinear model reduction with application to structural dynamics this work proposes a model reduction methodology that preserves lagrangian structure and achieves computational efficiency in the presence of high order nonlinearities and arbitrary parameter dependence as such the resulting reduced order model retains key properties such as energy conservation and symplectic time evolution maps we focus on parameterized simple mechanical systems subjected to rayleigh damping and external forces and consider an application to nonlinear structural dynamics to preserve structure the method first approximates the system s lagrangian ingredients the riemannian metric the potential energy function the dissipation function and the external force and subsequently derives reduced order equations of motion by applying the forced euler lagrange equation with these quantities from the algebraic perspective key contributions include two efficient techniques for approximating parameterized reduced matrices while preserving symmetry and positive definiteness matri

sparse representation for 3d shape estimation a convex relaxation approach we investigate the problem of estimating the 3d shape of an object defined by a set of 3d landmarks given their 2d correspondences in a single image a successful approach to alleviating the reconstruction ambiguity is the 3d deformable shape model and a sparse representation is often used to capture complex shape variability but the model inference is still challenging due to the nonconvexity in the joint optimization of shape and viewpoint in contrast to prior work that relies on an alternating scheme whose solution depends on initialization we propose a convex approach to addressing this challenge and develop an efficient algorithm to solve the proposed convex program we further propose a robust model to handle gross errors in the 2d correspondences we demonstrate the exact recovery property of the proposed method the advantage compared to several nonconvex baselines and the applicability to recover 3d human poses and car models from single images

majorized proximal alternating imputation for regularized rank constrained matrix completion abstract low rank approximation in the presence of missing data is ubiquitous in many areas of science and engineering as an np hard problem spectral regularization e g nuclear norm schatten p norm is widely used however solving this problem directly can be computationally expensive due to the unknown rank of variables and full rank singular value decompositions svd bilinear factorization is efficient and effectively benefits from the fast numerical optimization however most of existing methods require explicit knowledge of the rank and often tend to be flatlining or over fitting to balance and take advantage of both we formulate the matrix completion problem under the regularized rank constrained conditions we relax this problem to the surrogates with majorization minimization mm by employing a proximal alternating minimization method to optimize the corresponding surrogates we propose an efficient and accurate algorithm named majorized proximal alternating imputation mpa impute algorithm the proposed method iteratively replaces the missing data at each step with the most recent estimate and only involves simple matrix computation and small scale svd theoretically the proposed method is guaranteed to converge to the stationary point under mild conditions moreover the proposed method can be generalized to a family of non convex regularization problems extensive experiments over synthetic data and real world dataset demonstrate the superior performance of our method

low rank matrix completion via preconditioned optimization on the grassmann manifold we address the numerical problem of recovering large matrices of low rank when most of the entries are unknown we exploit the geometry of the low rank constraint to recast the problem as an unconstrained optimization problem on a single grassmann manifold we then apply second order riemannian trust region methods rtrmc 2 and riemannian conjugate gradient methods rcgmc to solve it a preconditioner for the hessian is introduced that helps control the conditioning of the problem and we detail preconditioned versions of riemannian optimization algorithms the cost of each iteration is linear in the number of known entries the proposed methods are competitive with state of the art algorithms on a wide range of problem instances in particular they perform well on rectangular matrices we further note that second order and preconditioned methods are well suited to solve badly conditioned matrix completion tasks

nonconvex phase synchronization we estimate n phases angles from noisy pairwise relative phase measurements the task is modeled as a nonconvex least squares optimization problem it was recently shown that this problem can be solved in polynomial time via convex relaxation under some conditions on the noise in this paper under similar but more restrictive conditions we show that a modified version of the power method converges to the global optimum this is simpler and empirically faster than convex approaches empirically they both succeed in the same regime further analysis shows that in the same noise regime as previously studied second order necessary optimality conditions for this quadratically constrained quadratic program are also sufficient despite nonconvexity

conic geometric optimization on the manifold of positive definite matrices we develop geometric optimization on the manifold of hermitian positive definite hpd matrices in particular we consider optimizing two types of cost functions i geodesically convex g convex and ii log nonexpansive ln g convex functions are nonconvex in the usual euclidean sense but convex along the manifold and thus allow global optimization ln functions may fail to be even g convex but still remain globally optimizable due to their special structure we develop theoretical tools to recognize and generate g convex functions as well as cone theoretic fixed point optimization algorithms we illustrate our techniques by applying them to maximum likelihood parameter estimation for elliptically contoured distributions a rich class that substantially generalizes the multivariate normal distribution we compare our fixed point algorithms with sophisticated manifold optimization methods and obtain notable speedups

quantitative risk assessment for teratological effects abstract this article presents a quantitative procedure for using a benchmark dose to obtain low dose risk estimates for reproductive and developmental toxic effects this procedure combines the best features of the previously proposed methods for handling litter effects for teratology data and the currently used methods for quantitative risk assessment the beta binomial distribution is used to account for litter effects and the weibull dose response model is used for modeling teratogenic effects a benchmark dose defined to be the lowest dose at which the excess risk does not exceed 1 with 95 confidence is proposed to replace the no observed effect level noel the noel is generally the highest experimental dose that is not statistically different from the control the noel approach does not use experimental data effectively for quantitative risk estimation in this article a lower limit on the safe dose is estimated by linearly extrapolating downward from the benchmark dose this procedure is

on the asymptotics of constrained m estimation limit theorems for an m estimate constrained to lie in a closed subset of r d are given under two different sets of regularity conditions a consistent sequence of global optimizers converges under chernoff regularity of the parameter set a n consistent sequence of local optimizers converges under clarke regularity of the parameter set in either case the asymptotic distribution is a projection of a normal random vector on the tangent cone of the parameter set at the true parameter value limit theorems for the optimal value are also obtained agreeing with chernoff s result in the case of maximum likelihood with global optimizers

comparison theorems for a class of nonlinear elliptic boundary value problems on compare des solutions d une classe de problemes de dirichlet quasi lineaires aux solutions de problemes lineaires d ordre 2 plus simples definis sur le meme domaine

matrix completion by least square low rank and sparse self representations abstract conventional matrix completion methods are generally based on rank minimization these methods assume that the given matrix is of low rank and the data points are drawn from a single subspace of low dimensionality therefore they are not effective in completing matrices where the data are drawn from multiple subspaces in this paper we establish a novel matrix completion framework that is based on self representation specifically least square low rank and sparse self representations based matrix completion algorithms are provided the underlying idea is that one data point can be efficiently reconstructed by other data points belonging to a common subspace where the missing entries are recovered so as to fit the common subspace the proposed algorithms actually maximize the weighted correlations among the columns of a given matrix we prove that the proposed algorithms are approximations for rank minimization of the incomplete matrix in addition they are able to complete high rank or even full rank matrices when the data are drawn from multiple subspaces comparative studies are conducted on synthetic datasets natural image inpainting tasks temperature prediction task and collaborative filtering tasks the results show that the proposed algorithms often outperform other state of the art algorithms in various tasks

stability analysis of hierarchical tensor methods for time dependent pdes abstract we address the question of whether it is possible to stably integrate time dependent high dimensional linear pdes with hierarchical tensor methods and explicit time stepping schemes to this end we develop sufficient conditions for stability and convergence of tensor solutions evolving on tensor manifolds with constant rank we also argue that the applicability of pde solvers with explicit time stepping may be limited by time step restriction dependent on the dimension of the problem numerical applications are presented and discussed for variable coefficients linear hyperbolic and parabolic pdes

matrix completion from noisy entries given a matrix m of low rank we consider the problem of reconstructing it from noisy observations of a small random subset of its entries the problem arises in a variety of applications from collaborative filtering the netflix problem to structure from motion and positioning we study a low complexity algorithm introduced by keshavan montanari and oh 2010 based on a combination of spectral techniques and manifold optimization that we call here optspace we prove performance guarantees that are order optimal in a number of circumstances

projection like retractions on matrix manifolds this paper deals with constructing retractions a key step when applying optimization algorithms on matrix manifolds for submanifolds of euclidean spaces we show that the operation consisting of taking a tangent step in the embedding euclidean space followed by a projection onto the submanifold is a retraction we also show that the operation remains a retraction if the projection is generalized to a projection like procedure that consists of coming back to the submanifold along admissible directions and we give a sufficient condition on the admissible directions for the generated retraction to be second order this theory offers a framework in which previously proposed retractions can be analyzed as well as a toolbox for constructing new ones illustrations are given for projection like procedures on some specific manifolds for which we have an explicit easy to compute expression

matrix completion from a few entries let m be an n n matrix of rank r and assume that a uniformly random subset e of its entries is observed we describe an efficient algorithm which we call optspace that reconstructs m from e o rn observed entries with relative root mean square error 1 2 rmse c nr e 1 2 with probability larger than 1 1 n3 further if r o 1 and m is sufficiently unstructured then optspace reconstructs it exactly from e o n log n entries with probability larger than 1 1 n3 this settles in the case of bounded rank a question left open by candes and recht and improves over the guarantees for their reconstruction algorithm the complexity of our algorithm is o e r log n which opens the way to its use for massive data sets in the process of proving these statements we obtain a generalization of a celebrated result by friedman kahn szemeredi and feige ofek on the spectrum of sparse random matrices

direct estimation of the derivative of quadratic mutual information with application in supervised dimension reduction a typical goal of linear supervised dimension reduction is to find a low dimensional subspace of the input space such that the projected input variables preserve maximal information about the output variables the dependence maximization approach solves the supervised dimension reduction problem through maximizing a statistical dependence between projected input variables and output variables a well known statistical dependence measure is mutual information mi which is based on the kullback leibler kl divergence however it is known that the kl divergence is sensitive to outliers quadratic mi qmi is a variant of mi based on the distance which is more robust against outliers than the kl divergence and a computationally efficient method to estimate qmi from data least squares qmi lsqmi has been proposed recently for these reasons developing a supervised dimension reduction method based on lsqmi seems promising however not qmi itself but the derivative of qmi is needed for subspace search

riemannian optimal control and model matching of linear port hamiltonian systems this paper presents a unified controller design method for h 2 optimal control and model matching problems of linear port hamiltonian systems the controller design problems are formulated as optimization problems on the product manifold of the set of skew symmetric matrices the manifold of the symmetric positive definite matrices and euclidean space a riemannian metric is chosen for the manifold in such a manner that the manifold is geodesically complete i e the domain of the exponential map is the whole tangent space for every point on the manifold in order to solve these problems the riemannian gradients of the objective functions are derived and these gradients are used to develop a riemannian steepest descent method on the product manifold the geodesic completeness of the manifold guarantees that all points generated by the steepest descent method are on the manifold numerical experiments illustrate that our method is able to solve the two specified problems

harmonic mean iteratively reweighted least squares for low rank matrix recovery we propose a new iteratively reweighted least squares irls algorithm for the recovery of a matrix x cd1 d2 of rank r min d1 d2 from incomplete linear observations solving a sequence of low complexity linear problems the easily implementable algorithm which we call harmonic mean iteratively reweighted least squares hm irls optimizes a non convex schatten p quasi norm penalization to promote low rankness and carries three major strengths in particular for the matrix completion setting first we observe a remarkable global convergence behavior of the algorithm s iterates to the low rank matrix for relevant interesting cases for which any other state of the art optimization approach fails the recovery secondly hm irls exhibits an empirical recovery probability close to 1 even for a number of measurements very close to the theoretical lower bound r d1 d2 r i e already for significantly fewer linear observations than any other tractable approach in the literature thirdly hm irls exhibits a locally superlinear rate of convergence of order 2 p if the linear observations fulfill a suitable null space property while for the first two properties we have so far only strong empirical evidence we prove the third property as our main theoretical result

noisy matrix completion understanding statistical guarantees for convex relaxation via nonconvex optimization this paper studies noisy low rank matrix completion given partial and noisy entries of a large low rank matrix the goal is to estimate the underlying matrix faithfully and efficiently arguably o

a geometric newton method for oja s vector field newton s method for solving the matrix equation runs up against the fact that its zeros are not isolated this is due to a symmetry of f by the action of the orthogonal group we show how differential geometric techniques can be exploited to remove this symmetry and obtain a geometric newton algorithm that finds the zeros of f the geometric newton method does not suffer from the degeneracy issue that stands in the way of the original newton method

on l_q optimization and matrix completion rank minimization problems which consist of finding a matrix of minimum rank subject to linear constraints have been proposed in many areas of engineering and science a specific problem is the matrix completion problem in which a low rank data matrix can be recovered from incomplete samples of its entries by solving a rank penalized least squares problem the rank penalty is in fact the l0 norm of the matrix singular values a recent convex relaxation of this penalty is the commonly used l1 norm of the matrix singular values in this paper we bridge the gap between these two penalties and propose the lq 0 q 1 penalized least squares problem for matrix completion an iterative algorithm is developed by solving a non standard optimization problem and a non trivial convergence result is proved we illustrate with simulations comparing the reconstruction quality of the three matrix singular value penalty functions l0 l1 and lq 0 q 1

local minima and convergence in low rank semidefinite programming the low rank semidefinite programming problem lrsdpr is a restriction of the semidefinite programming problem sdp in which a bound r is imposed on the rank of x and it is well known that lrsdpr is equivalent to sdp if r is not too small in this paper we classify the local minima of lrsdpr and prove the optimal convergence of a slight variant of the successful yet experimental algorithm of burer and monteiro 5 which handles lrsdpr via the nonconvex change of variables x rrt in addition for particular problem classes we describe a practical technique for obtaining lower bounds on the optimal solution value during the execution of the algorithm computational results are presented on a set of combinatorial optimization relaxations including some of the largest quadratic assignment sdps solved to date

reinforced robust principal component pursuit high dimensional data present in the real world is often corrupted by noise and gross outliers principal component analysis pca fails to learn the true low dimensional subspace in such cases this is the reason why robust versions of pca which put a penalty on arbitrarily large outlying entries are preferred to perform dimension reduction in this paper we argue that it is necessary to study the presence of outliers not only in the observed data matrix but also in the orthogonal complement subspace of the authentic principal subspace in fact the latter can seriously skew the estimation of the principal components a reinforced robustification of principal component pursuit is designed in order to cater to the problem of finding out both types of outliers and eliminate their influence on the final subspace estimation simulation results under different design situations clearly show the superiority of our proposed method as compared with other popular implementations of robust pca this paper also showcases possible applications of our method in critically tough scenarios of face recognition and video background subtraction along with approximating a usable low dimensional subspace from real world data sets the technique can capture semantically meaningful outliers

riemannian gradient descent methods for graph regularized matrix completion abstract low rank matrix completion is the problem of recovering the missing entries of a data matrix by using the assumption that the true matrix admits a good low rank approximation much attention has been given recently to exploiting correlations between the column row entities to improve the matrix completion quality in this paper we propose preconditioned gradient descent algorithms for solving the low rank matrix completion problem with graph laplacian based regularizers experiments on synthetic data show that our approach achieves significant speedup compared to an existing method based on alternating minimization experimental results on real world data also show that our methods provide low rank solutions of similar quality in comparable or less time than the state of the art method

an alternative to em for gaussian mixture models batch and stochastic riemannian optimization we consider maximum likelihood estimation for gaussian mixture models gmm s this task is almost invariably solved in theory and practice via the expectation maximization em algorithm em owes its success to various factors of which is its ability to fulfill positive definiteness constraints in closed form is of key importance we propose an alternative to em grounded in the riemannian geometry of positive definite matrices using which we cast gmm parameter estimation as a riemannian optimization problem surprisingly such an out of the box riemannian formulation completely fails and proves much inferior to em this motivates us to take a closer look at the problem geometry and derive a better formulation that is much more amenable to riemannian optimization we then develop riemannian batch and stochastic gradient algorithms that outperform em often substantially we provide a non asymptotic convergence analysis for our stochastic method which is also the first to our knowledge such global analysis for riemannian stochastic gradient numerous empirical results are included to demonstrate the effectiveness of our methods

detecting approximate reflection symmetry in a point set using optimization on manifold we propose an algorithm to detect approximate reflection symmetry present in a set of volumetrically distributed points belonging to mathbb r d containing a distorted reflection symmetry pattern we pose the problem of detecting approximate reflection symmetry as the problem of establishing correspondences between the points which are reflections of each other and we determine the reflection symmetry transformation we formulate an optimization framework in which the problem of establishing the correspondences amounts to solving a linear assignment problem and the problem of determining the reflection symmetry transformation amounts to solving an optimization problem on a smooth riemannian product manifold the proposed approach estimates the symmetry from the geometry of the points and is descriptor independent we evaluate the performance of the proposed approach on the standard benchmark dataset and achieve the state of the art performance we further show the robustness of our approach by varying the amount of distortion in a perfect reflection symmetry pattern where we perturb each point by a different amount of perturbation we demonstrate the effectiveness of the method by applying it to the problem of 2 d two dimensional and 3 d reflection symmetry detection along with comparisons

near optimal matrix recovery from random linear measurements in matrix recovery from random linear measurements one is interested in recovering an unknown m by n matrix x 0 from n m n measurements y i t r a i x 0 where each a i is an m by n measurement matrix with i i d random entries i 1 n we present a matrix recovery algorithm based on approximate message passing which iteratively applies an optimal singular value shrinker a nonconvex nonlinearity tailored specifically for matrix estimation our algorithm typically converges exponentially fast offering a significant speedup over previously suggested matrix recovery algorithms such as iterative solvers for nuclear norm minimization nnm it is well known that there is a recovery tradeoff between the information content of the object x 0 to be recovered specifically its matrix rank r and the number of linear measurements n from which recovery is to be attempted the precise tradeoff between r and n beyond which recovery by a given algorithm becomes possible traces the so called phase transition curve of that algorithm in the r n plane the phase transition curve of our algorithm is noticeably better than that of nnm interestingly it is close to the information theoretic lower bound for the minimal number of measurements needed for matrix recovery making it not only state of the art in terms of convergence rate but also near optimal in terms of the matrices it successfully recovers

iterative methods for low rank approximation of graph similarity matrices in this paper we analyze an algorithm to compute a low rank approximation of the similarity matrix s introduced by blondel et al in 1 this problem can be reformulated as an optimization problem of a continuous function s tr stm2 s where s is constrained to have unit frobenius norm and m2 is a non negative linear map we restrict the feasible set to the set of matrices of unit frobenius norm with either k nonzero identical singular values or at most k nonzero not necessarily identical singular values we first characterize the stationary points of the associated optimization problems and further consider iterative algorithms to find one of them we analyze the convergence properties of our algorithm and prove that accumulation points are stationary points of s we finally compare our method in terms of speed and accuracy to the full rank algorithm proposed in 1

preconditioned low rank riemannian optimization for linear systems with tensor product structure the numerical solution of partial differential equations on high dimensional domains gives rise to computationally challenging linear systems when using standard discretization techniques the size of the linear system grows exponentially with the number of dimensions making the use of classic iterative solvers infeasible during the last few years low rank tensor approaches have been developed that allow one to mitigate this curse of dimensionality by exploiting the underlying structure of the linear operator in this work we focus on tensors represented in the tucker and tensor train formats we propose two preconditioned gradient methods on the corresponding low rank tensor manifolds a riemannian version of the preconditioned richardson method as well as an approximate newton scheme based on the riemannian hessian for the latter considerable attention is given to the efficient solution of the resulting newton equation in numerical experiments we compare the efficiency of our riemannian algorit

robust low rank matrix completion by riemannian optimization low rank matrix completion is the problem where one tries to recover a low rank matrix from noisy observations of a subset of its entries in this paper we propose rmc a new method to deal with the problem of robust low rank matrix completion i e matrix completion where a fraction of the observed entries are corrupted by non gaussian noise typically outliers the method relies on the idea of smoothing the ell 1 norm and using riemannian optimization to deal with the low rank constraint we first state the algorithm as the successive minimization of smooth approximations of the ell 1 norm and we analyze its convergence by showing the strict decrease of the objective function we then perform numerical experiments on synthetic data and demonstrate the effectiveness on the proposed method on the netflix dataset

adaptive regularization with cubics on manifolds adaptive regularization with cubics arc is an algorithm for unconstrained non convex optimization akin to the trust region method its iterations can be thought of as approximate safe guarded newton steps for cost functions with lipschitz continuous hessian arc has optimal iteration complexity in the sense that it produces an iterate with gradient smaller than varepsilon in o 1 varepsilon 1 5 iterations for the same price it can also guarantee a hessian with smallest eigenvalue larger than sqrt varepsilon in this paper we study a generalization of arc to optimization on riemannian manifolds in particular we generalize the iteration complexity results to this richer framework our central contribution lies in the identification of appropriate manifold specific assumptions that allow us to secure these complexity guarantees both when using the exponential map and when using a general retraction a substantial part of the paper is devoted to studying these assumptions relevant beyond arc and providing user friendly sufficient conditions for them numerical experiments are encouraging

complete dictionary recovery over the sphere ii recovery by riemannian trust region method we consider the problem of recovering a complete i e square and invertible matrix a 0 from y in mathbb r n times p with y boldsymbol a 0 x 0 provided x 0 is sufficiently sparse this recovery problem is central to theoretical understanding of dictionary learning which seeks a sparse representation for a collection of input signals and finds numerous applications in modern signal processing and machine learning we give the first efficient algorithm that provably recovers a 0 when x 0 has o left n right nonzeros per column under suitable probability model for x 0 our algorithmic pipeline centers around solving a certain nonconvex optimization problem with a spherical constraint and hence is naturally phrased in the language of manifold optimization in a companion paper we have showed that with high probability our nonconvex formulation has no spurious local minimizers and around any saddle point the objective function has a negative directional curvature in this paper we take advantage of the particular geometric structure and describe a riemannian trust region algorithm that provably converges to a local minimizer with from arbitrary initializations such minimizers give excellent approximations to the rows of x 0 the rows are then recovered by a linear programming rounding and deflation

low rank optimization with trace norm penalty the paper addresses the problem of low rank trace norm minimization we propose an algorithm that alternates between fixed rank optimization and rank one updates the fixed rank optimization is characterized by an efficient factorization that makes the trace norm differentiable in the search space and the computation of duality gap numerically tractable the search space is nonlinear but is equipped with a riemannian structure that leads to efficient computations we present a second order trust region algorithm with a guaranteed quadratic rate of convergence overall the proposed optimization scheme converges superlinearly to the global solution while maintaining complexity that is linear in the number of rows and columns of the matrix to compute a set of solutions efficiently for a grid of regularization parameters we propose a predictor corrector approach that outperforms the naive warm restart approach on the fixed rank quotient manifold the performance of the proposed algorithm is illustrated on p

low rank tensor recovery via iterative hard thresholding abstract we study extensions of compressive sensing and low rank matrix recovery matrix completion to the recovery of low rank tensors of higher order from a small number of linear measurements while the theoretical understanding of low rank matrix recovery is already well developed only few contributions on the low rank tensor recovery problem are available so far in this paper we introduce versions of the iterative hard thresholding algorithm for several tensor decompositions namely the higher order singular value decomposition hosvd the tensor train format tt and the general hierarchical tucker decomposition ht we provide a partial convergence result for these algorithms which is based on a variant of the restricted isometry property of the measurement operator adapted to the tensor decomposition at hand that induces a corresponding notion of tensor rank we show that subgaussian measurement ensembles satisfy the tensor restricted isometry property with high probability under a certain almost optimal bound on the number of measurements which depends on the corresponding tensor format these bounds are extended to partial fourier maps combined with random sign flips of the tensor entries finally we illustrate the performance of iterative hard thresholding methods for tensor recovery via numerical experiments where we consider recovery from gaussian random measurements tensor completion recovery of missing entries and fourier measurements for third order tensors

direct density derivative estimation estimating the derivatives of probability density functions is an essential step in statistical data analysis a naive approach to estimate the derivatives is to first perform density estimation and then compute its derivatives however this approach can be unreliable because a good density estimator does not necessarily mean a good density derivative estimator to cope with this problem in this letter we propose a novel method that directly estimates density derivatives without going through density estimation the proposed method provides computationally efficient estimation for the derivatives of any order on multidimensional data with a hyperparameter tuning method and achieves the optimal parametric convergence rate we further discuss an extension of the proposed method by applying regularized multitask learning and a general framework for density derivative estimation based on bregman divergences applications of the proposed method to nonparametric kullback leibler divergence approximation and bandwidth matrix selection in kernel density estimation are also explored

a newton like method for solving rank constrained linear matrix inequalities this paper presents a newton like algorithm for solving systems of rank constrained linear matrix inequalities though local quadratic convergence of the algorithm is not a priori guaranteed or observed in all cases numerical experiments including application to an output feedback stabilization problem show the effectiveness of the algorithm

the power of convex relaxation near optimal matrix completion this paper is concerned with the problem of recovering an unknown matrix from a small fraction of its entries this is known as the matrix completion problem and comes up in a great number of applications including the famous netflix prize and other similar questions in collaborative filtering in general accurate recovery of a matrix from a small number of entries is impossible but the knowledge that the unknown matrix has low rank radically changes this premise making the search for solutions meaningful this paper presents optimality results quantifying the minimum number of entries needed to recover a matrix of rank r exactly by any method whatsoever the information theoretic limit more importantly the paper shows that under certain incoherence assumptions on the singular vectors of the matrix recovery is possible by solving a convenient convex program as soon as the number of entries is on the order of the information theoretic limit up to logarithmic factors this convex program simply finds among all matrices consistent with the observed entries that with minimum nuclear norm as an example we show that on the order of nr log n samples are needed to recover a random n x n matrix of rank r by any method and to be sure nuclear norm minimization succeeds as soon as the number of entries is of the form nr polylog n

an implementable proximal point algorithmic framework for nuclear norm minimization the nuclear norm minimization problem is to find a matrix with the minimum nuclear norm subject to linear and second order cone constraints such a problem often arises from the convex relaxation of a rank minimization problem with noisy data and arises in many fields of engineering and science in this paper we study inexact proximal point algorithms in the primal dual and primal dual forms for solving the nuclear norm minimization with linear equality and second order cone constraints we design efficient implementations of these algorithms and present comprehensive convergence results in particular we investigate the performance of our proposed algorithms in which the inner sub problems are approximately solved by the gradient projection method or the accelerated proximal gradient method our numerical results for solving randomly generated matrix completion problems and real matrix completion problems show that our algorithms perform favorably in comparison to several recently proposed state of the art algorithms interestingly our proposed algorithms are connected with other algorithms that have been studied in the literature

low rank optimization on the cone of positive semidefinite matrices we propose an algorithm for solving optimization problems defined on a subset of the cone of symmetric positive semidefinite matrices this algorithm relies on the factorization x yy t where the number of columns of y fixes an upper bound on the rank of the positive semidefinite matrix x it is thus very effective for solving problems that have a low rank solution the factorization x yy t leads to a reformulation of the original problem as an optimization on a particular quotient manifold the present paper discusses the geometry of that manifold and derives a second order optimization method with guaranteed quadratic convergence it furthermore provides some conditions on the rank of the factorization to ensure equivalence with the original problem in contrast to existing methods the proposed algorithm converges monotonically to the sought solution its numerical efficiency is evaluated on two applications the maximal cut of a graph and the problem of sparse principal component analysis

fitting nonlinear and constrained generalized estimating equations with optimization software summary in this article we present an estimation approach for solving nonlinear constrained generalized estimating equations that can be implemented using object oriented software for nonlinear programming such as nlminb in splus or fmincon and lsqnonlin in matlab we show how standard estimating equation theory includes this method as a special case so that our estimates when unconstrained will remain consistent and asymptotically normal to illustrate this method we fit a nonlinear dose response model with nonnegative mixed bound constraints to clustered binary data from a developmental toxicity study satisfactory confidence intervals are found using a nonparametric bootstrap method when a common correlation coefficient is assumed for all the dose groups and for some of the dose specific groups

multivariate regression with gross errors on manifold valued data we consider the topic of multivariate regression on manifold valued output that is for a multivariate observation its output response lies on a manifold moreover we propose a new regression model to deal with the presence of grossly corrupted manifold valued responses a bottleneck issue commonly encountered in practical scenarios our model first takes a correction step on the grossly corrupted responses via geodesic curves on the manifold then performs multivariate linear regression on the corrected data this results in a nonconvex and nonsmooth optimization problem on riemannian manifolds to this end we propose a dedicated approach named palmr by utilizing and extending the proximal alternating linearized minimization techniques for optimization problems on euclidean spaces theoretically we investigate its convergence property where it is shown to converge to a critical point under mild conditions empirically we test our model on both synthetic and real diffusion tensor imaging data and show that our model outperforms other multivariate regression models when manifold valued responses contain gross errors and is effective in identifying gross errors

accelerated low rank matrix approximate algorithms for matrix completion abstract in this paper we propose two accelerated algorithms for the low rank approximate method in wang et al 0000 for matrix completion the main idea is to use the successive over relaxation technique based on the successive over relaxation method for the feasible matrices or projection matrices the low rank matrix approximate method is modified and accelerated meanwhile we discuss the convergence of the over relaxation algorithm for the feasible matrix finally the numerical experiments show them to be effective

a path following algorithm for the graph matching problem we propose a convex concave programming approach for the labeled weighted graph matching problem the convex concave programming formulation is obtained by rewriting the weighted graph matching problem as a least square problem on the set of permutation matrices and relaxing it to two different optimization problems a quadratic convex and a quadratic concave optimization problem on the set of doubly stochastic matrices the concave relaxation has the same global minimum as the initial graph matching problem but the search for its global minimum is also a hard combinatorial problem we therefore construct an approximation of the concave problem solution by following a solution path of a convex concave problem obtained by linear interpolation of the convex and concave formulations starting from the convex relaxation this method allows to easily integrate the information on graph label similarities into the optimization problem and therefore perform labeled weighted graph matching the algorithm is compared with some of the best performing graph matching methods on four data sets simulated graphs qaplib retina vessel images and handwritten chinese characters in all cases the results are competitive with the state of the art

linear dimensionality reduction survey insights and generalizations linear dimensionality reduction methods are a cornerstone of analyzing high dimensional data due to their simple geometric interpretations and typically attractive computational properties these methods capture many data features of interest such as covariance dynamical structure correlation between data sets input output relationships and margin between data classes methods have been developed with a variety of names and motivations in many fields and perhaps as a result the connections between all these methods have not been highlighted here we survey methods from this disparate literature as optimization programs over matrix manifolds we discuss principal component analysis factor analysis linear multidimensional scaling fisher s linear discriminant analysis canonical correlations analysis maximum autocorrelation factors slow feature analysis sufficient dimensionality reduction undercomplete independent component analysis linear regression distance metric learning and more this optimization framework gives insight to some rarely discussed shortcomings of well known methods such as the suboptimality of certain eigenvector solutions modern techniques for optimization over matrix manifolds enable a generic linear dimensionality reduction solver which accepts as input data and an objective to be optimized and returns as output an optimal low dimensional projection of the data this simple optimization framework further allows straightforward generalizations and novel variants of classical methods which we demonstrate here by creating an orthogonal projection canonical correlations analysis more broadly this survey and generic solver suggest that linear dimensionality reduction can move toward becoming a blackbox objective agnostic numerical technology

intrinsic representation of tangent vectors and vector transports on matrix manifolds the quasi newton methods on riemannian manifolds proposed thus far do not appear to lend themselves to satisfactory convergence analyses unless they resort to an isometric vector transport this prompts us to propose a computationally tractable isometric vector transport on the stiefel manifold of orthonormal p frames in mathbb r n rn specifically it requires o np 2 o np2 flops which is considerably less expensive than existing alternatives in the frequently encountered case where n gg p n p we then build on this result to also propose computationally tractable isometric vector transports on other manifolds namely the grassmann manifold the fixed rank manifold and the positive semidefinite fixed rank manifold in the process we also propose a convenient way to represent tangent vectors to these manifolds as elements of mathbb r d rd where d is the dimension of the manifold we call this an intrinsic representation as opposed to extrinsic representations as elements of mathbb r w rw where w is the dimension of the embedding space finally we demonstrate the performance of the proposed isometric vector transport in the context of a riemannian quasi newton method applied to minimizing the brockett cost function

a riemannian optimization algorithm for joint maximum likelihood estimation of high dimensional exploratory item factor analysis there has been regained interest in joint maximum likelihood jml estimation of item factor analysis ifa recently primarily due to its efficiency in handling high dimensional data and numerous latent factors it has been established under mild assumptions that the jml estimator is consistent as both the numbers of respondents and items tend to infinity the current work presents an efficient riemannian optimization algorithm for jml estimation of exploratory ifa with dichotomous response data which takes advantage of the differential geometry of the fixed rank matrix manifold the proposed algorithm takes substantially less time to converge than a benchmark method that alternates between gradient ascent steps for person and item parameters the performance of the proposed algorithm in the recovery of latent dimensionality response probabilities item parameters and factor scores is evaluated via simulations

regression on fixed rank positive semidefinite matrices a riemannian approach the paper addresses the problem of learning a regression model parameterized by a fixed rank positive semidefinite matrix the focus is on the nonlinear nature of the search space and on scalability to high dimensional problems the mathematical developments rely on the theory of gradient descent algorithms adapted to the riemannian geometry that underlies the set of fixed rank positive semidefinite matrices in contrast with previous contributions in the literature no restrictions are imposed on the range space of the learned matrix the resulting algorithms maintain a linear complexity in the problem size and enjoy important invariance properties we apply the proposed algorithms to the problem of learning a distance function parameterized by a positive semidefinite matrix good performance is observed on classical benchmarks

generalized reduced rank latent factor regression for high dimensional tensor fields and neuroimaging genetic applications we propose a generalized reduced rank latent factor regression model grrlf for the analysis of tensor field responses and high dimensional covariates the model is motivated by the need from imaging genetic studies to identify genetic variants that are associated with brain imaging phenotypes often in the form of high dimensional tensor fields grrlf identifies from the structure in the data the effective dimensionality of the data and then jointly performs dimension reduction of the covariates dynamic identification of latent factors and nonparametric estimation of both covariate and latent response fields after accounting for the latent and covariate effects grllf performs a nonparametric test on the remaining factor of interest grrlf provides a better factorization of the signals compared with common solutions and is less susceptible to overfitting because it exploits the effective dimensionality the generality and the flexibility of grrlf also allow various statistical models to be handled in a unified framework and solutions can be efficiently computed within the field of neuroimaging it improves the sensitivity for weak signals and is a promising alternative to existing approaches the operation of the framework is demonstrated with both synthetic datasets and a real world neuroimaging example in which the effects of a set of genes on the structure of the brain at the voxel level were measured and the results compared favorably with those from existing approaches

a gradient sampling method on algebraic varieties and application to nonsmooth low rank optimization in this paper a nonsmooth optimization method for locally lipschitz functions on real algebraic varieties is developed to this end the set valued map varepsilon conditional subdifferential x

manopt a matlab toolbox for optimization on manifolds optimization on manifolds is a rapidly developing branch of nonlinear optimization its focus is on problems where the smooth geometry of the search space can be leveraged to design efficient numerical algorithms in particular optimization on manifolds is well suited to deal with rank and orthogonality constraints such structured constraints appear pervasively in machine learning applications including low rank matrix completion sensor network localization camera network registration independent component analysis metric learning dimensionality reduction and so on the manopt toolbox available at www manopt org is a user friendly documented piece of software dedicated to simplify experimenting with state of the art riemannian optimization algorithms by dealing internally with most of the differential geometry the package aims particularly at lowering the entrance barrier

multivariate regression analyses for categorical data summary it is common to observe a vector of discrete and or continuous responses in scientific problems where the objective is to characterize the dependence of each response on explanatory variables and to account for the association between the outcomes the response vector can comprise repeated observations on one variable as in longitudinal studies or genetic studies of families or can include observations for different variables this paper discusses a class of models for the marginal expectations of each response and for pairwise associations the marginal models are contrasted with log linear models two generalized estimating equation approaches are compared for parameter estimation the first focuses on the regression parameters the second simultaneously estimates the regression and association parameters the robustness and efficiency of each is discussed the methods are illustrated with analyses of two data sets from public health research

constrained maximum likelihood exemplified by isotonic convex logistic regression abstract maximum likelihood estimates in problems in which the likelihood is smooth and the parameter space is defined by linear or smooth nonlinear inequality constraints can be obtained using available nonlinear optimization packages such as npsol this class of models includes generalized linear models with order restrictions convexity or smoothness constraints on the parameters smoothness constraints being in the form of bounds on finite differences of the regression function methods for such models are demonstrated by an analysis of data from four published studies of the incidence of down s syndrome in single year maternal age intervals constrained logistic regression of the incidence of down s syndrome on maternal age shows that the data are fitted well by a nondecreasing convex function p values for the likelihood ratio test of this model against alternatives are obtained by the parametric bootstrap and iterated parametric bootstrap the iterated bootstrap is used as a diagnostic tool it

sparse exploratory factor analysis sparse principal component analysis is a very active research area in the last decade it produces component loadings with many zero entries which facilitates their interpretation and helps avoid redundant variables the classic factor analysis is another popular dimension reduction technique which shares similar interpretation problems and could greatly benefit from sparse solutions unfortunately there are very few works considering sparse versions of the classic factor analysis our goal is to contribute further in this direction we revisit the most popular procedures for exploratory factor analysis maximum likelihood and least squares sparse factor loadings are obtained for them by first adopting a special reparameterization and second by introducing additional formula see text norm penalties into the standard factor analysis problems as a result we propose sparse versions of the major factor analysis procedures we illustrate the developed algorithms on well known psychometric problems our sparse solutions are critically compared to ones obtained by other existing methods