sequence set design with good correlation properties via majorization minimization sets of sequences with good correlation properties are desired in many active sensing and communication systems e g multiple input multiple output mimo radar systems and code division multiple access cdma cellular systems in this paper we consider the problems of designing complementary sets of sequences css and also sequence sets with both good auto and cross correlation properties algorithms based on the general majorization minimization method are developed to tackle the optimization problems arising from the sequence set design problems all the proposed algorithms can be implemented by means of the fast fourier transform fft and thus are computationally efficient and capable of designing sets of very long sequences a number of numerical examples are provided to demonstrate the performance of the proposed algorithms

scalable and flexible multiview max var canonical correlation analysis generalized canonical correlation analysis gcca aims at finding latent low dimensional common structure from multiple views feature vectors in different domains of the same entities unlike principal component analysis that handles a single view g cca is able to integrate information from different feature spaces here we focus on max var gcca a popular formulation that has recently gained renewed interest in multilingual processing and speech modeling the classic max var gcca problem can be solved optimally via eigen decomposition of a matrix that compounds the whitened correlation matrices of the views but this solution has serious scalability issues and is not directly amenable to incorporating pertinent structural constraints such as nonnegativity and sparsity on the canonical components we posit regularized max var gcca as a nonconvex optimization problem and propose an alternating optimization based algorithm to handle it our algorithm alternates between inexact solutions of a regularized least squares subproblem and a manifold constrained nonconvex subproblem thereby achieving substantial memory and computational savings an important benefit of our design is that it can easily handle structure promoting regularization we show that the algorithm globally converges to a critical point at a sublinear rate and approaches a global optimal solution at a linear rate when no regularization is considered judiciously designed simulations and large scale word embedding tasks are employed to showcase the effectiveness of the proposed algorithm

maximin joint optimization of transmitting code and receiving filter in radar and communications in this paper we conduct the joint design of transmitting sequence s and receiving filters subject to the peak to average ratio par constraint in radar and communications applications we consider optimizing the worst case performance and the resulting optimization problem takes a maximin format we propose two algorithms based on the mm majorization minimization or minorization maximization method as opposed to the traditional epigraph based smooth reformulation on top of that both algorithms are guaranteed to converge to a b oulingand stationary solution and b stationarity is the appropriate stationarity condition for problems with a nonconvex constraint set the proposed algorithms successively solve a series of simple convex problems that enjoy low computational complexity numerical simulations have shown that the proposed algorithms empirically achieve slightly higher objective values and converge faster in terms of cpu time than the existing methods

a unified convergence analysis of block successive minimization methods for nonsmooth optimization the block coordinate descent bcd method is widely used for minimizing a continuous function f of several block variables at each iteration of this method a single block of variables is optimized while the remaining variables are held fixed to ensure the convergence of the bcd method the subproblem of each block variable needs to be solved to its unique global optimal unfortunately this requirement is often too restrictive for many practical scenarios in this paper we study an alternative inexact bcd approach which updates the variable blocks by successively minimizing a sequence of approximations of f which are either locally tight upper bounds of f or strictly convex local approximations of f the main contributions of this work include the characterizations of the convergence conditions for a fairly wide class of such methods especially for the cases where the objective functions are either nondifferentiable or nonconvex our results unify and extend the existing convergence results

prime phase retrieval via majorization minimization this paper considers the phase retrieval problem in which measurements consist of only the magnitude of several linear measurements of the unknown e g spectral components of a time sequence we develop low complexity algorithms with superior performance based on the majorization minimization mm framework the proposed algorithms are referred to as prime phase retrieval via the majorization minimization technique they are preferred to existing benchmark methods since at each iteration a simple surrogate problem is solved with a closed form solution that monotonically decreases the original objective function in total three algorithms are proposed using different majorization minimization techniques experimental results validate that our algorithms outperform existing methods in terms of successful recovery and mean square error under various settings

fast unit modulus least squares with applications in beamforming unit modulus least squares uls problems arise in many applications including phase only beamforming sensor network localization synchronization phase retrieval and radar code design uls formulations can always be recast as unit modulus quadratic programs to which semidefinite relaxation sdr can be applied and is often the state of the art approach however sdr lifts the problem dimension i e the number of variables from n to n 2 which drastically increases the memory burden and computational cost when the problem size is already large e g when designing phase only beamformer weights for massive multiple input multiple output systems this paper focuses on scalable first order algorithms for the uls problem and some of its variants it advocates using simple gradient projection gp as a starting point for solving the uls problem establishes global convergence of gp to a karush kuhn tucker point for this np hard problem and bounds its iteration complexity then it proposes uls extensions tailored to reflect practical beamformer design objectives bringing in and exploiting new degrees of freedom to improve the beampattern designs simple variants of gp are proposed to deal with these extended uls problems simulations are used to showcase the effectiveness of the proposed algorithms in both the plain uls problem and in the context of phase only beamforming

around a multivariate schmidt spitzer theorem given an arbitrary complex valued infinite matrix infmata a ij i 1 dotsc infty j 1 dotsc infty and a positive integer n we introduce anaturally associated polynomial basis pol

per stream mse based linear transceiver design for mimo interference channels with csi error this paper studies the linear transceiver design for multi stream multiple input multiple output mimo interference channels with imperfect channel state information csi two csi error models are considered the same coding and modulation scheme is assumed for all users and data streams in the first model where the csi error is statistical we minimize the average maximum per stream mean square error mse in the second model where the csi error is norm bounded the worst case maximum per stream mse is optimized since the problems are not jointly convex the alternate optimization approach is adopted to obtain suboptimal transceivers for the statistical csi error a low complexity algorithm utilizing lagrangian duality is proposed which has a semi closed form solution and can be implemented distributively compared with the classical algorithm using the second order cone programming the proposed algorithm significantly reduces the computational complexity without sacrificing the performance for the bounded csi error we present two approaches one uses s procedure to obtain an equivalent problem and the other one solves an approximated problem simulation results demonstrate the robustness of the proposed algorithms and their improved performance over the sum mse and the maximum per user mse based methods the lower complexity of the lagrangian duality based algorithm is also verified numerically

nonlocal sparse model with adaptive structural clustering for feature extraction of aero engine bearings abstract fault information of aero engine bearings presents two particular phenomena i e waveform distortion and impulsive feature frequency band dispersion which leads to a challenging problem for current techniques of bearing fault diagnosis moreover although many progresses of sparse representation theory have been made in feature extraction of fault information the theory also confronts inevitable performance degradation due to the fact that relatively weak fault information has not sufficiently prominent and sparse representations therefore a novel nonlocal sparse model coined nlsm and its algorithm framework has been proposed in this paper which goes beyond simple sparsity by introducing more intrinsic structures of feature information this work adequately exploits the underlying prior information that feature information exhibits nonlocal self similarity through clustering similar signal fragments and stacking them together into groups within this framework the prior information is transformed into a regularization term and a sparse optimization problem which could be solved through block coordinate descent method bcd is formulated additionally the adaptive structural clustering sparse dictionary learning technique which utilizes k nearest neighbor knn clustering and principal component analysis pca learning is adopted to further enable sufficient sparsity of feature information moreover the selection rule of regularization parameter and computational complexity are described in detail the performance of the proposed framework is evaluated through numerical experiment and its superiority with respect to the state of the art method in the field is demonstrated through the vibration signals of experimental rig of aircraft engine bearings

nonlinear system identification via direct weight optimization a general framework for estimating nonlinear functions and systems is described and analyzed in this paper identification of a system is seen as estimation of a predictor function the considered predictor function estimate at a particular point is defined to be affine in the observed outputs and the estimate is defined by the weights in this expression for each given point the maximal mean square error or an upper bound of the function estimate over a class of possible true functions is minimized with respect to the weights which is a convex optimization problem this gives different types of algorithms depending on the chosen function class it is shown how the classical linear least squares is obtained as a special case and how unknown but bounded disturbances can be handled most of the paper deals with the method applied to locally smooth predictor functions it is shown how this leads to local estimators with a finite bandwidth meaning that only observations in a neighborhood of the target point will be used in the estimate the size of this neighborhood the bandwidth is automatically computed and reflects the noise level in the data and the smoothness priors the approach is applied to a number of dynamical systems to illustrate its potential

extended admm and bcd for nonseparable convex minimization models with quadratic coupling terms convergence analysis and insights in this paper we establish the convergence of the proximal alternating direction method of multipliers admm and block coordinate descent bcd method for nonseparable minimization models with quadratic coupling terms the novel convergence results presented in this paper answer several open questions that have been the subject of considerable discussion we firstly extend the 2 block proximal admm to linearly constrained convex optimization with a coupled quadratic objective function an area where theoretical understanding is currently lacking and prove that the sequence generated by the proximal admm converges in point wise manner to a primal dual solution pair moreover we apply randomly permuted admm rpadmm to nonseparable multi block convex optimization and prove its expected convergence for a class of nonseparable quadratic programming problems when the linear constraint vanishes the 2 block proximal admm and rpadmm reduce to the 2 block cyclic proximal bcd method and randomly permuted bcd rpbcd our study provides the first iterate convergence result for 2 block cyclic proximal bcd without assuming the boundedness of the iterates we also theoretically establish the expected iterate convergence result concerning multi block rpbcd for convex quadratic optimization in addition we demonstrate that rpbcd may have a worse convergence rate than cyclic proximal bcd for 2 block convex quadratic minimization problems although the results on rpadmm and rpbcd are restricted to quadratic minimization models they provide some interesting insights 1 random permutation makes admm and bcd more robust for multi block convex minimization problems 2 cyclic bcd may outperform rpbcd for nice problems and rpbcd should be applied with caution when solving general convex optimization problems especially with a few blocks

piecewise linear solution paths with application to direct weight optimization recently pathfollowing algorithms for parametric optimization problems with piecewise linear solution paths have been developed within the field of regularized regression this paper presents a generalization of these algorithms to a wider class of problems it is shown that the approach can be applied to the nonparametric system identification method direct weight optimization dwo and be used to enhance the computational efficiency of this method the most important design parameter in the dwo method is a parameter l controlling the bias variance trade off and the use of parametric optimization with piecewise linear solution paths means that the dwo estimates can be efficiently computed for all values of l simultaneously this allows for designing computationally attractive adaptive bandwidth selection algorithms one such algorithm for dwo is proposed and demonstrated in two examples

inexact block coordinate descent methods for symmetric nonnegative matrix factorization symmetric nonnegative matrix factorization snmf is equivalent to computing a symmetric nonnegative low rank approximation of a data similarity matrix it inherits the good data interpretability of the well known nonnegative matrix factorization technique and has better ability of clustering nonlinearly separable data in this paper we focus on the algorithmic aspect of the snmf problem and propose simple inexact block coordinate decent methods to address the problem leading to both serial and parallel algorithms the proposed algorithms have guaranteed convergence to stationary solutions and can efficiently handle large scale and or sparse snmf problems extensive simulations verify the effectiveness of the proposed algorithms compared to recent state of the art algorithms

computing b stationary points of nonsmooth dc programs motivated by a class of applied problems arising from physical layer based security in a digital communication system in particular by a secrecy sum rate maximization problem this paper studies a nonsmooth difference of convex dc minimization problem the contributions of this paper are i clarify several kinds of stationary solutions and their relations ii develop and establish the convergence of a novel algorithm for computing a d stationary solution of a problem with a convex feasible set that is arguably the sharpest kind among the various stationary solutions iii extend the algorithm in several directions including a randomized choice of the subproblems that could help the practical convergence of the algorithm a distributed penalty approach for problems whose objective functions are sums of dc functions and problems with a specially structured nonconvex dc constraint for the latter class of problems a pointwise slater constraint qualification is introduced that facilitates the ver

phaseless near field far field transformation utilizing combinations of probe signals phaseless near field far field transformations based on nonconvex optimization suffer from local stationary points in order to avoid the corresponding suboptimal solutions the incorporation of combinations of probe signals into the cost functional is proposed and investigated the nonconvex optimization based on the wirtinger calculus relies on forward and adjoint radiation operator evaluations via the fast irregular antenna field transformation algorithm fiafta and is performed by a memory limited quasi newton method this allows the solution of large scale problems with utilization of the full flexibility of the fiafta in terms of measurement probe correction and arbitrary sample locations results for synthetic and measured near field data reveal significant improvements when the knowledge of probe signal combinations is utilized especially in applications with a large number of unknowns

phase retrieval via wirtinger flow theory and algorithms we study the problem of recovering the phase from magnitude measurements specifically we wish to reconstruct a complex valued signal boldsymbol x in mathbb c n about which we have phaseless samples of the form y r left langle boldsymbol a r boldsymbol x rangle right 2 r 1 ldots m knowledge of the phase of these samples would yield a linear system this paper develops a nonconvex formulation of the phase retrieval problem as well as a concrete solution algorithm in a nutshell this algorithm starts with a careful initialization obtained by means of a spectral method and then refines this initial estimate by iteratively applying novel update rules which have low computational complexity much like in a gradient descent scheme the main contribution is that this algorithm is shown to rigorously allow the exact retrieval of phase information from a nearly minimal number of random measurements indeed the sequence of successive iterates provably converges to the solution at a geometric rate so that the proposed scheme is efficient both in terms of computational and data resources in theory a variation on this scheme leads to a near linear time algorithm for a physically realizable model based on coded diffraction patterns we illustrate the effectiveness of our methods with various experiments on image data underlying our analysis are insights for the analysis of nonconvex optimization schemes that may have implications for computational problems beyond phase retrieval

lazy learning for local modelling and control design this paper presents local methods for modelling and control of discrete time unknown non linear dynamical systems when only input output data are available we propose the adoption of lazy learning a memory based technique for local modelling the modelling procedure uses a query based approach to select the best model configuration by assessing and comparing different alternatives a new recursive technique for local model identification and validation is presented together with an enhanced statistical method for model selection a lso three methods to design controllers based on the local linearization provided by the lazy learning algorithm are described in the first method the lazy technique returns the forward and inverse models of the system which are used to compute the control action to take the second is an indirect method inspired by self tuning regulators where recursive least squares estimation is replaced by a local approximator the third method combines the linearization provided by t

robust matrix factorization by majorization minimization l 1 norm based low rank matrix factorization in the presence of missing data and outliers remains a hot topic in computer vision due to non convexity and non smoothness all the existing methods either lack scalability or robustness or have no theoretical guarantee on convergence in this paper we apply the majorization minimization technique to solve this problem at each iteration we upper bound the original function with a strongly convex surrogate by minimizing the surrogate and updating the iterates accordingly the objective function has sufficient decrease which is stronger than just being non increasing that other methods could offer as a consequence without extra assumptions we prove that any limit point of the iterates is a stationary point of the objective function in comparison other methods either do not have such a convergence guarantee or require extra critical assumptions extensive experiments on both synthetic and real data sets testify to the effectiveness of our algorithm the speed of our method is also highly competitive

a coordinate descent framework to design low psl isl sequences this paper is focused on the design of phase sequences with good aperiodic autocorrelation properties in terms of peak sidelobe level and integrated sidelobe level the problem is formulated as a biobjective pareto optimization forcing either a continuous or a discrete phase constraint at the design stage an iterative procedure based on the coordinate descent method is introduced to deal with the resulting optimization problems that are nonconvex and np hard in general each iteration of the devised method requires the solution of a nonconvex min max problem it is handled either through a novel bisection or an fft based method respectively for the continuous and the discrete phase constraint additionally a heuristic approach to initialize the procedures employing the l p norm minimization technique is proposed simulation results illustrate that the proposed methodologies can outperform some counterparts providing sequences with good autocorrelation features especially in the discrete phase binary case

maximum pseudolikelihood estimation for model based clustering of time series data mixture of autoregressions moar models provide a model based approach to the clustering of time series data the maximum likelihood ml estimation of moar models requires evaluating products of large numbers of densities of normal random variables in practical scenarios these products converge to zero as the length of the time series increases and thus the ml estimation of moar models becomes infeasible without the use of numerical tricks we propose a maximum pseudolikelihood mpl estimation approach as an alternative to the use of numerical tricks the mpl estimator is proved to be consistent and can be computed with an em expectation maximization algorithm simulations are used to assess the performance of the mpl estimator against that of the ml estimator in cases where the latter was able to be calculated an application to the clustering of time series data arising from a resting state fmri experiment is presented as a demonstration of the methodology

fusion rings and geometry the algebraic structure of fusion rings in rational conformal field theories is analyzed in detail in this paper a formalism which closely parallels classical tools in the study of the cohomology of homogeneous spaces is developed for fusion rings in general and for current algebra theories in particular it is shown that fusion rings lead to a natural orthogonal polynomial structure the rings are expressed through generators and relations the relations are then derived from some potentials leading to an identification of the fusion rings with deformations of affine varieties in general the fusion algebras are mapped to affine varieties which are the locus of the relations the connection with modular transformations is investigated in this picture it is explained how chiral algebras arising in n 2 superconformal field theory can be derived from fusion rings in particular it is argued that theories of the type su n k su n 1 are the n 2 counterparts of grassmann manifolds and that there is a natural identification of the chiral fields with schubert varieties which is a graded algebra isomorphism

smoothed state estimates under abrupt changes using sum of norms regularization the presence of abrupt changes such as impulsive and load disturbances commonly occur in applications but make the state estimation problem considerably more difficult than in the standard setting with gaussian process disturbance abrupt changes often introduce a jump in the state and the problem is therefore readily and often treated by change detection techniques in this paper we take a different approach the state smoothing problem for linear state space models is here formulated as a constrained least squares problem with sum of norms regularization a generalization of 1 regularization this novel formulation can be seen as a convex relaxation of the well known generalized likelihood ratio method by willsky and jones another nice property of the suggested formulation is that it only has one tuning parameter the regularization constant which is used to trade off fit and the number of jumps good practical choices of this parameter along with an extension to nonlinear state space models are given

a flexible and efficient algorithmic framework for constrained matrix and tensor factorization we propose a general algorithmic framework for constrained matrix and tensor factorization which is widely used in signal processing and machine learning the new framework is a hybrid between alternating optimization ao and the alternating direction method of multipliers admm each matrix factor is updated in turn using admm hence the name ao admm this combination can naturally accommodate a great variety of constraints on the factor matrices and almost all possible loss measures for the fitting computation caching and warm start strategies are used to ensure that each update is evaluated efficiently while the outer ao framework exploits recent developments in block coordinate descent bcd type methods which help ensure that every limit point is a stationary point as well as faster and more robust convergence in practice three special cases are studied in detail non negative matrix tensor factorization constrained matrix tensor completion and dictionary learning extensive simulations and experiments with real data are used to showcase the effectiveness and broad applicability of the proposed framework

an iteratively weighted mmse approach to distributed sum utility maximization for a mimo interfering broadcast channel consider the multiple input multiple output mimo interfering broadcast channel whereby multiple base stations in a cellular network simultaneously transmit signals to a group of users in their own cells while causing interference to each other the basic problem is to design linear beamformers that can maximize the system throughput in this paper we propose a linear transceiver design algorithm for weighted sum rate maximization that is based on iterative minimization of weighted mean square error mse the proposed algorithm only needs local channel knowledge and converges to a stationary point of the weighted sum rate maximization problem furthermore the algorithm and its convergence can be extended to a general class of sum utility maximization problem the effectiveness of the proposed algorithm is validated by numerical experiments

robust covariance matrix estimation in heterogeneous low rank context this paper addresses the problem of robust covariance matrix cm estimation in the context of a disturbance composed of a low rank lr heterogeneous clutter plus an additive white gaussian noise the lr clutter is modeled by a spherically invariant random vector with assumed high clutter to noise ratio in such a context adaptive process should require less training samples than classical methods to reach equivalent performance as in a full rank clutter configuration the main issue is that classical robust estimators of the cm cannot be computed in the undersampled configuration to overcome this issue the current approach is based on regularization methods nevertheless most of these solutions are enforcing the estimate to be well conditioned while in our context it should be lr structured this paper therefore addresses this issue and derives an algorithm to compute the maximum likelihood estimator of the cm for the considered disturbance model several relaxations and robust generalizations of the result are discussed performance is finally illustrated on numerical simulations and on a space time adaptive processing for airborne radar application

tensor decompositions and applications this survey provides an overview of higher order tensor decompositions their applications and available software a tensor is a multidimensional or n way array decompositions of higher order tensors i e n way arrays with n geq 3 have applications in psycho metrics chemometrics signal processing numerical linear algebra computer vision numerical analysis data mining neuroscience graph analysis and elsewhere two particular tensor decompositions can be considered to be higher order extensions of the matrix singular value decomposition candecomp parafac cp decomposes a tensor as a sum of rank one tensors and the tucker decomposition is a higher order form of principal component analysis there are many other tensor decompositions including indscal parafac2 candelinc dedicom and paratuck2 as well as nonnegative variants of all of the above the n way toolbox tensor toolbox and multilinear engine are examples of software packages for working with tensors

hypergeometric polynomials are optimal with any integer convex polytope p subset mathbb r n we associate a multivariate hypergeometric polynomial whose set of exponents is mathbb z n cap p this polynomial is defined uniquely up to a constant multiple and satisfies a holonomic system of partial differential equations of horn s type we prove that under certain nondegeneracy conditions any such polynomial is optimal in the sense of 7 i e that the topology of its amoeba 13 is as complex as it could possibly be using this we derive optimal properties of several classical families of multivariate hypergeometric polynomials

path design for best emitter location using two mobile sensors we consider the design of a pair of moving sensors trajectories for the purpose of optimally localizing a stationary emitter based on time difference of arrival measurements the localization error covariance matrix is predicted by the cramer rao bound as an optimization criterion for the localization error we propose the maximization of the smallest eigenvalue of the fisher information matrix that is associated with the major principle axis of the confidence ellipsoid we establish the path design problem under a set of constraints rising from speed maneuvering safety and no fly zones limitations we propose a solution based on a nonconvex alternating direction method of multipliers we examine the results of the algorithm for the case of a pair of sensor against the basin hopping global optimizer with impressive results

the concave convex procedure the concave convex procedure cccp is a way to construct discrete time iterative dynamical systems that are guaranteed to decrease global optimization and energy functions monotonically this procedure can be applied to almost any optimization problem and many existing algorithms can be interpreted in terms of it in particular we prove that all expectation maximization algorithms and classes of legendre minimization and variational bounding algorithms can be reexpressed in terms of cccp we show that many existing neural network and mean field theory algorithms are also examples of cccp the generalized iterative scaling algorithm and sinkhorn s algorithm can also be expressed as cccp by changing variables cccp can be used both as a new way to understand and prove the convergence of existing optimization algorithms and as a procedure for generating new algorithms

a unified successive pseudoconvex approximation framework in this paper we propose a successive pseudoconvex approximation algorithm to efficiently compute stationary points for a large class of possibly nonconvex optimization problems the stationary points are obtained by solving a sequence of successively refined approximate problems each of which is much easier to solve than the original problem to achieve convergence the approximate problem only needs to exhibit a weak form of convexity namely pseudoconvexity we show that the proposed framework not only includes as special cases a number of existing methods for example the gradient method and the jacobi algorithm but also leads to new algorithms that enjoy easier implementation and faster convergence speed we also propose a novel line search method for nondifferentiable optimization problems which is carried out over a properly constructed differentiable function with the benefit of a simplified implementation as compared to the state of the art line search techniques that directly operate on the original nondifferentiable objective function the advantages of the proposed algorithm are shown both theoretically and numerically by several example applications namely multi input multi output mimo broadcast channel capacity computation energy efficiency maximization in massive mimo systems and ell 1 regularized least square problems in sparse signal recovery

detection of faults in rotating machinery using periodic time frequency sparsity abstract this paper addresses the problem of extracting periodic oscillatory features in vibration signals for detecting faults in rotating machinery to extract the feature we propose an approach in the short time fourier transform stft domain where the periodic oscillatory feature manifests itself as a relatively sparse grid to estimate the sparse grid we formulate an optimization problem using customized binary weights in the regularizer where the weights are formulated to promote periodicity in order to solve the proposed optimization problem we develop an algorithm called augmented lagrangian majorization minimization algorithm which combines the split augmented lagrangian shrinkage algorithm salsa with majorization minimization mm and is guaranteed to converge for both convex and non convex formulation as examples the proposed approach is applied to simulated data and used as a tool for diagnosing faults in bearings and gearboxes for real data and compared to some state of the art methods the results show that the proposed approach can effectively detect and extract the periodical oscillatory features

maximum block improvement and polynomial optimization in this paper we propose an efficient method for solving the spherically constrained homogeneous polynomial optimization problem the new approach has the following three main ingredients first we establish a block coordinate descent type search method for nonlinear optimization with the novelty being that we accept only a block update that achieves the maximum improvement hence the name of our new search method maximum block improvement mbi convergence of the sequence produced by the mbi method to a stationary point is proved second we establish that maximizing a homogeneous polynomial over a sphere is equivalent to its tensor relaxation problem thus we can maximize a homogeneous polynomial function over a sphere by its tensor relaxation via the mbi approach third we propose a scheme to reach a kkt point of the polynomial optimization provided that a stationary solution for the relaxed tensor problem is available numerical experiments have shown that our new method works very efficiently for a majority of the test instances that we have experimented with the method finds the global optimal solution at a low computational cost

some comments on cp i study the typical configuration of a cp plot when the number of variables in the regression problem is large and there are many weak effects i show that a particular configuration that is very commonly seen can arise in a simple way i give a formula by means of which the risk incurred by the minimum cp rule can be estimated

piecewise linear regularized solution paths we consider the generic regularized optimization problem argmin l y x j efron hastie johnstone and tibshirani ann statist 32 2004 407 499 have shown that for the lasso that is if l is squared error loss and j 1 is the l 1 norm of the optimal coefficient path is piecewise linear that is is piecewise constant we derive a general characterization of the properties of loss l penalty j pairs which give piecewise linear coefficient paths such pairs allow for efficient generation of the full regularized coefficient paths we investigate the nature of efficient path following algorithms which arise we use our results to suggest robust versions of the lasso for regression and classification and to develop new efficient algorithms for existing problems in the literature including mammen and van de geer s locally adaptive regression splines

robust volume minimization based matrix factorization for remote sensing and document clustering this paper considers volume minimization volmin based structured matrix factorization volmin is a factorization criterion that decomposes a given data matrix into a basis matrix times a structured coefficient matrix via finding the minimum volume simplex that encloses all the columns of the data matrix recent work showed that volmin guarantees the identifiability of the factor matrices under mild conditions that are realistic in a wide variety of applications this paper focuses on both theoretical and practical aspects of volmin on the theory side exact equivalence of two independently developed sufficient conditions for volmin identifiability is proven here thereby providing a more comprehensive understanding of this aspect of volmin on the algorithm side computational complexity and sensitivity to outliers are two key challenges associated with real world applications of volmin these are addressed here via a new volmin algorithm that handles volume regularization in a computationally simple way and automatically detects and iteratively downweights outliers simultaneously simulations and real data experiments using a remotely sensed hyperspectral image and the reuters document corpus are employed to showcase the effectiveness of the proposed algorithm

identification of switched linear regression models using sum of norms regularization this paper proposes a general convex framework for the identification of switched linear systems the proposed framework uses over parameterization to avoid solving the otherwise combinatorially forbidding identification problem and takes the form of a least squares problem with a sum of norms regularization a generalization of the 1 regularization the regularization constant regulates the complexity and is used to trade off the fit and the number of submodels

on calculation of generating functions of chebyshev polynomials in several variables we propose a new method of calculation of generating functions of chebyshev polynomials in several variables associated with root systems of the simple lie algebras we obtain the generating functions of the polynomials in two variables corresponding to the lie algebra c2 as an illustration

phase retrieval using alternating minimization phase retrieval problems involve solving linear equations but with missing sign or phase for complex numbers information more than four decades after it was first proposed the seminal error reduction algorithm of gerchberg and saxton and fienup is still the popular choice for solving many variants of this problem the algorithm is based on alternating minimization i e it alternates between estimating the missing phase information and the candidate solution despite its wide usage in practice no global convergence guarantees for this algorithm are known in this paper we show that a resampling variant of this approach converges geometrically to the solution of one such problem finding a vector bf x from bf y bf a where bf y vert bf a t bf x vert and vert bf z vert denotes a vector of element wise magnitudes of bf z under the assumption that bf a is gaussian empirically we demonstrate that alternating minimization performs similar to recently proposed convex techniques for this problem which are based on lifting to a convex matrix problem in sample complexity and robustness to noise however it is much more efficient and can scale to large problems analytically for a resampling version of alternating minimization we show geometric convergence to the solution and sample complexity that is off by log factors from obvious lower bounds we also establish close to optimal scaling for the case when the unknown vector is sparse our work represents the first theoretical guarantee for alternating minimization albeit with resampling for any variant of phase retrieval problems in the non convex setting

locally weighted regression an approach to regression analysis by local fitting abstract locally weighted regression or loess is a way of estimating a regression surface through a multivariate smoothing procedure fitting a function of the independent variables locally and in a moving fashion analogous to how a moving average is computed for a time series with local fitting we can estimate a much wider class of regression surfaces than with the usual classes of parametric functions such as polynomials the goal of this article is to show through applications how loess can be used for three purposes data exploration diagnostic checking of parametric models and providing a nonparametric regression surface along the way the following methodology is introduced a a multivariate smoothing procedure that is an extension of univariate locally weighted regression b statistical procedures that are analogous to those used in the least squares fitting of parametric functions c several graphical methods that are useful tools for understanding loess estimates and checking the a

toward fast transform learning this paper introduces a new dictionary learning strategy based on atoms obtained by translating the composition of k k convolutions with s s sparse kernels of known support the dictionary update step associated with this strategy is a non convex optimization problem we propose a practical formulation of this problem and introduce a gauss seidel type algorithm referred to as alternative least square algorithm for its resolution the search space of the proposed algorithm is of dimension ks ks which is typically smaller than the size of the target atom and much smaller than the size of the image moreover the complexity of this algorithm is linear with respect to the image size allowing larger atoms to be learned as opposed to small patches the conducted experiments show that we are able to accurately approximate atoms such as wavelets curvelets sinc functions or cosines for large values of k the proposed experiments also indicate that the algorithm generally converges to a global minimum for large values of k k and s s

brief an algorithm for multi parametric quadratic programming and explicit mpc solutions explicit solutions to constrained linear model predictive control problems can be obtained by solving multi parametric quadratic programs mp qp where the parameters are the components of the state vector we study the properties of the polyhedral partition of the state space induced by the multi parametric piecewise affine solution and propose a new mp qp solver compared to existing algorithms our approach adopts a different exploration strategy for subdividing the parameter space avoiding unnecessary partitioning and qp problem solving with a significant improvement of efficiency

undersampled sparse phase retrieval via majorization minimization in the undersampled phase retrieval problem the goal is to recover an n dimensional complex valued signal from only m intensity measurements without phase information this inverse system is not only nonconvex but also underdetermined in this paper we propose to exploit the sparsity in the original signal and develop two low complexity algorithms with superior performance based on the majorization minimization framework the proposed algorithms are preferred to existing benchmark methods since at each iteration a simple convex surrogate problem is solved with a closed form solution that monotonically decreases the objective function value when the unknown signal is sparse in the standard basis the first algorithm c prime can produce a stationary point of the corresponding nonconvex phase retrieval problem when the unknown signal is not sparse in the standard basis the second algorithm sc prime can find a coordinatewise stationary point of the more challenging phase retrieval problem through sparse coding experimental results validate that the proposed algorithms have higher successful recovery rate and less normalized mean square error than existing up to date methods under the same setting

majorization minimization algorithms in signal processing communications and machine learning this paper gives an overview of the majorization minimization mm algorithmic framework which can provide guidance in deriving problem driven algorithms with low computational cost a general introduction of mm is presented including a description of the basic principle and its convergence results the extensions acceleration schemes and connection to other algorithmic frameworks are also covered to bridge the gap between theory and practice upperbounds for a large number of basic functions derived based on the taylor expansion convexity and special inequalities are provided as ingredients for constructing surrogate functions with the pre requisites established the way of applying mm to solving specific problems is elaborated by a wide range of applications in signal processing communications and machine learning

cognitive radar based sequence design via sinr maximization the ambiguity function plays an important role in radar systems in fact many radar design problems can be interpreted from the perspective of persuing desired ambiguity functions to adapt to various application scenes in this paper we consider designing a radar sequence subject to a peak to average power ratio par constraint to maximize the signal to interference plus noise ratio which can also be interpreted as designing a sequence with a desired ambiguity function from an optimization point of view this is equivalent to optimizing a complex quartic function with the par constraint an efficient algorithm based on the general majorization minimization mm method is developed to solve this problem with guaranteed convergence to a stationary point under some mild conditions in addition the unit modulus constraint as a special case is considered and another algorithm is proposed which is the combination of the general mm and the coordinate descent method numerical experiments show that the proposed algorithms can shape a desired ambiguity function based on the prior information and the performance is much better compared with the existing methods

the explicit linear quadratic regulator for constrained systems we present a technique to compute the explicit state feedback solution to both the finite and infinite horizon linear quadratic optimal control problem subject to state and input constraints we show that this closed form solution is piecewise linear and continuous as a practical consequence of the result constrained linear quadratic regulation becomes attractive also for systems with high sampling rates as on line quadratic programming solvers are no more required for the implementation

phase recovery maxcut and complex semidefinite programming phase retrieval seeks to recover a signal x in mathbb c p x c p from the amplitude a x a x of linear measurements ax in mathbb c n a x c n we cast the phase retrieval problem as a non convex quadratic program over a complex phase vector and formulate a tractable relaxation called phasecut similar to the classical maxcut semidefinite program we solve this problem using a provably convergent block coordinate descent algorithm whose structure is similar to that of the original greedy algorithm in gerchberg and saxton optik 35 237 246 1972 where each iteration is a matrix vector product numerical results show the performance of this approach over three different phase retrieval problems in comparison with greedy phase retrieval algorithms and matrix completion formulations

chebyshev polynomials in several variables and the radial part of the laplace beltrami operator chebyshev polynomials of the first and the second kind in n variables 1 z25 n zn are introduced the yariables n z2 zn are the characters of the representations of sl n 1 c corresponding to the fundamental weights the chebyshev polynomials are eigenpolynomials of a second order linear partial differential operator which is in fact the radial part of the laplace beltrami operator on certain symmetric spaces we give an explicit expression of this operator in the coordinates z1 ff z2 s zn and then show how many results in the literature on differential equations satisfied by chebyshev polynomials in several variables follow immediately from well known results on the radial part of the laplace beltrami operator related topics like orthogonality symmetry relations generating functions and recurrence relations are also discussed finally we note that the chebyshev polynomials are a special case of a more general class of orthogonal polynomials in several variables

incremental majorization minimization optimization with application to large scale machine learning majorization minimization algorithms consist of successively minimizing a sequence of upper bounds of the objective function these upper bounds are tight at the current estimate and each iteration monotonically drives the objective function downhill such a simple principle is widely applicable and has been very popular in various scientific fields especially in signal processing and statistics we propose an incremental majorization minimization scheme for minimizing a large sum of continuous functions a problem of utmost importance in machine learning we present convergence guarantees for nonconvex and convex optimization when the upper bounds approximate the objective up to a smooth error we call such upper bounds first order surrogate functions more precisely we study asymptotic stationary point guarantees for nonconvex problems and for convex ones we provide convergence rates for the expected objective function value we apply our scheme to composite optimization and obtain a new incremental proximal gradient algorithm with linear convergence rate for strongly convex functions our experiments show that our method is competitive with the state of the art for solving machine learning problems such as logistic regression when the number of training samples is large enough and we demonstrate its usefulness for sparse estimation with nonconvex penalties

scalable solvers of random quadratic equations via stochastic truncated amplitude flow a novel approach termed stochastic truncated amplitude flow staf is developed to reconstruct an unknown n dimensional real complex valued signal boldsymbol x from m phaseless quadratic equations of the form psi i langle boldsymbol a i boldsymbol x rangle this problem also known as phase retrieval from magnitude only information is np hard in general adopting an amplitude based nonconvex formulation staf leads to an iterative solver comprising two stages s1 orthogonality promoting initialization through a stochastic variance reduced gradient algorithm and s2 a series of iterative refinements of the initialization using stochastic truncated gradient iterations both stages involve a single equation per iteration thus rendering staf a simple scalable and fast approach amenable to large scale implementations that are useful when n is large when lbrace boldsymbol a i rbrace i 1 m are independent gaussian staf provably recovers exactly any boldsymbol x in mathbb r n exponentially fast based on order of n quadratic equations staf is also robust in the presence of additive noise of bounded support simulated tests involving real gaussian lbrace boldsymbol a i rbrace vectors demonstrate that staf empirically reconstructs any boldsymbol x in mathbb r n exactly from about 2 3n magnitude only measurements outperforming state of the art approaches and narrowing the gap from the information theoretic number of equations m 2n 1 extensive experiments using synthetic data and real images corroborate markedly improved performance of staf over existing alternatives

phaselift exact and stable signal recovery from magnitude measurements via convex programming suppose we wish to recover a signal input amssym font abc cmmib10 def bi 1 hbox abc 1 bi x in bbb c n from m intensity measurements of the form that is from data in which phase information is missing we prove that if the vectors are sampled independently and uniformly at random on the unit sphere then the signal x can be recovered exactly up to a global phase factor by solving a convenient semidefinite program a trace norm minimization problem this holds with large probability provided that m is on the order of and without any assumption about the signal whatsoever this novel result demonstrates that in some instances the combinatorial phase retrieval problem can be solved by convex programming techniques finally we also prove that our methodology is robust vis a vis additive noise 2012 wiley periodicals inc