fisher information for a complex gaussian random variable beamforming applications for wave propagation in a random medium we derive a general expression for the fisher information for a special class of complex gaussian random variables whose mean and covariance may be expressed in terms of a specific parameterized linear transformation the deterministic unknown parameter case is considered this class is descriptive of signals that propagate in a homogeneous or inhomogeneous medium and also occurs in numerous beamforming applications the fisher information derived here encompasses a wide range of previously derived well known results for complex gaussian signals propagating in a homogeneous medium and is also applicable to more recent models for signals propagating in an inhomogenous medium the conditions for the estimates of the azimuth and elevation to decouple from the estimates of the nuisance parameters are studied for monochromatic plane wave and spherical wave propagation in a random medium for propagation in an anisotropic medium it is found that in general the cramer rao lower bounds crlbs of the entire unknown parameter set will be coupled for special cases of propagation in an isotropic medium it is found that for plane waves the crlbs of the angles of arrival will be approximately decoupled from the crlbs of the other unknowns and for spherical waves the crlbs of the angles of arrival will be always be coupled to the crlbs of all other unknowns although the couplings may be minimized under certain conditions

a redistributed proximal bundle method for nonconvex optimization proximal bundle methods have been shown to be highly successful optimization methods for unconstrained convex problems with discontinuous first derivatives this naturally leads to the question of whether proximal variants of bundle methods can be extended to a nonconvex setting this work proposes an approach based on generating cutting planes models not of the objective function as most bundle methods do but of a local convexification of the objective function the corresponding convexification parameter is calculated on the fly in such a way that the algorithm can inform the user as to what proximal parameters are sufficiently large that the objective function is likely to have well defined proximal points this novel approach shown to be sound from both the objective function and subdifferential modelling perspectives opens the way to create workable nonconvex algorithms based on nonconvex mathcal vu theory both theoretical convergence analysis and some encouraging preliminary numerical experience are provided

a regularity result for the singular values of a transfer matrix and a quadratically convergent algorithm for computing its l norm the i th singular value of a transfer matrix need not be a differentiable function of frequency where its multiplicity is greater than one we show that near a local maximum however the largest singular value has a lipschitz second derivative but need not have a third derivative using this regularity result we give a quadratically convergent algorithm for computing the l norm of a transfer matrix

minimum density hyperplanes associating distinct groups of objects clusters with contiguous regions of high probability density high density clusters is central to many statistical and machine learning approaches to the classification of unlabelled data we propose a novel hyperplane classifier for clustering and semi supervised classification which is motivated by this objective the proposed minimum density hyperplane minimises the integral of the empirical probability density function along it thereby avoiding intersection with high density clusters we show that the minimum density and the maximum margin hyperplanes are asymptotically equivalent thus linking this approach to maximum margin clustering and semi supervised support vector classifiers we propose a projection pursuit formulation of the associated optimisation problem which allows us to nd minimum density hyperplanes efficiently in practice and evaluate its performance on a range of benchmark data sets the proposed approach is found to be very competitive with state of the art methods for clustering and semi supervised classification

controller design via nonsmooth multidirectional search we propose an algorithm which combines multidirectional search mds with nonsmooth optimization techniques to solve difficult problems in automatic control applications include static and fixed order output feedback controller design simultaneous stabilization h 2 h infty synthesis and much else we show how to combine direct search techniques with nonsmooth descent steps in order to obtain convergence certificates in the presence of nonsmoothness our technique is efficient when small and medium size controllers for plants with large state dimension are sought our numerical testing includes several benchmark examples for instance our algorithm needs 0 41 s to compute a static output feedback stabilizing controller for the boeing 767 flutter benchmark problem e e j davison ifac technical committee reports pergamon press oxford 1990 a system with 55 states the first static controller without performance specifications for this system was obtained in j burke a lewis and m overton siam j optim 15 2003 pp 751 779

a new sequential optimality condition for constrained nonsmooth optimization we introduce a sequential optimality condition for locally lipschitz constrained nonsmooth optimization verifiable just using derivative information and which holds even in the absence of any con

oblique projectors and group involutory matrices in this work conditions characterizing when a lineal combination of two projectors of the same order is a group involutory matrix are analyzed in particular a representation of an involutory matrix as a sum or difference of two projectors is given in addition some results are applied to control problems an application to obtain a state feedback control is given

analysis of the rational krylov subspace and adi methods for solving the lyapunov equation for large scale problems an effective approach for solving the algebraic lyapunov equation consists of projecting the problem onto a significantly smaller space and then solving the reduced order matrix equation although krylov subspaces have been used for a long time only more recent developments have shown that rational krylov subspaces can be a competitive alternative to the classical and very popular alternating direction implicit adi recurrence in this paper we develop a convergence analysis of the rational krylov subspace method rksm based on the kronecker product formulation and on potential theory moreover we propose new enlightening relations between this approach and the adi method our results provide solid theoretical ground for recent numerical evidence of the superiority of rksm over adi when the involved parameters cannot be computed optimally as is the case in many practical application problems

on the uniqueness of functions that maximize the crouzeix ratio abstract let a be an n by n matrix with numerical range w a q a q q c n q 2 1 we are interested in functions f that maximize f a 2 the matrix norm induced by the vector 2 norm over all functions f that are analytic in the interior of w a and continuous on the boundary and satisfy max z w a f z 1 it is known that there are functions f that achieve this maximum and that such functions are of the form b where is any conformal mapping from the interior of w a to the unit disk d extended to be continuous on the boundary of w a and b is a blaschke product of degree at most n 1 it is not known if a function f that achieves this maximum is unique up to multiplication by a scalar of modulus one we show that this is the case when a is a 2 2 nonnormal matrix or a jordan block but we give examples of some 3 3 matrices with elliptic numerical range for which two different functions f involving the same conformal mapping but blaschke products of different degrees achieve the same maximal value of f a 2

estimating tangent and normal cones without calculus we study simple estimates of the tangent and normal cones to a set with a nonsmooth boundary we seek estimates that are robust relative to the base point and that depend only on collections of nearby points known to lie inside and outside of the set

optimality conditions and a smoothing trust region newton method for nonlipschitz optimization regularized minimization problems with nonconvex nonsmooth perhaps non lipschitz penalty functions have attracted considerable attention in recent years owing to their wide applications in image restoration signal reconstruction and variable selection in this paper we derive affine scaled second order necessary and sufficient conditions for local minimizers of such minimization problems moreover we propose a global convergent smoothing trust region newton method which can find a point satisfying the affine scaled second order necessary optimality condition from any starting point numerical examples are given to demonstrate the effectiveness of the smoothing trust region newton method

algorithms for solving the inverse problem associated with k a k a s 1 in previous papers the authors introduced and characterized a class of matrices called k s 1 potent also they established a method to construct these matrices the purpose of this paper is to solve the associated inverse problem several algorithms are developed in order to find all involutory matrices k satisfying k a s 1 k a for a given matrix a c n n and a given natural number s the cases s 0 and s 1 are separately studied since they produce different situations in addition some examples are presented showing the numerical performance of the methods

on a disjoint idempotent decomposition for linear combinations produced from n commutative tripotent matrices abstract it has been established a 3 n term disjoint idempotent decomposition did for the linear combinations produced from n 2 commutative tripotent matrices their products and their products of power 2 at most the results obtained in this way generalize those in y tian a disjoint idempotent decomposition for linear combinations produced from two commutative tripotent matrices and its applications linear multilinear algebra 59 2011 1237 1246 moreover an algorithm to get a did has been provided finally a numerical example has been given to exemplify the results

smoothing projected gradient method and its application to stochastic linear complementarity problems a smoothing projected gradient spg method is proposed for the minimization problem on a closed convex set where the objective function is locally lipschitz continuous but nonconvex nondifferentiable we show that any accumulation point generated by the spg method is a stationary point associated with the smoothing function used in the method which is a clarke stationary point in many applications we apply the spg method to the stochastic linear complementarity problem slcp and image restoration problems we study the stationary point defined by the directional derivative and provide necessary and sufficient conditions for a local minimizer of the expected residual minimization erm formulation of slcp preliminary numerical experiments using the spg method for solving randomly generated slcp and image restoration problems of large sizes show that the spg method is promising

a riemannian subgradient algorithm for economic dispatch with valve point effect the economic load dispatch problem eldp is a classical problem in the power systems community it consists in the optimal scheduling of the output of power generating units to meet the required load demand subject to unit and system inequality and equality constraints this optimization problem is challenging on three different levels the geometry of its feasible set the non differentiability of its cost function and the multimodal aspect of its landscape for this reason eldp has received much attention in the past few years and numerous derivative free techniques have been proposed to tackle its multimodal and nondifferentiable characteristics in this work we propose a different approach exploiting the rich geometrical structure of the problem we show that the nonlinear equality constraint can be handled in the framework of riemannian manifolds and we develop a feasible all iterates satisfy the constraints subgradient descent algorithm to provide fast convergence to local minima to this end we show that clarke s calculus can be used to compute a deterministic admissible descent direction by solving a simple low dimensional quadratic program we test our approach on four real data sets the proposed method provides fast local convergence and scales well with respect to the problem dimension finally we show that the proposed algorithm being a local optimization method can be incorporated in existing heuristic techniques to provide a better exploration of the search space

zero forcing precoding and generalized inverses we consider the problem of linear zero forcing precoding design and discuss its relation to the theory of generalized inverses in linear algebra special attention is given to a specific generalized inverse known as the pseudo inverse we begin with the standard design under the assumption of a total power constraint and prove that precoders based on the pseudo inverse are optimal among the generalized inverses in this setting then we proceed to examine individual per antenna power constraints in this case the pseudo inverse is not necessarily the optimal inverse in fact finding the optimal matrix is nontrivial and depends on the specific performance measure we address two common criteria fairness and throughput and show that the optimal generalized inverses may be found using standard convex optimization methods we demonstrate the improved performance offered by our approach using computer simulations

error estimates and evaluation of matrix functions via the faber transform the need to evaluate expressions of the form f a or f a b where f is a nonlinear function a is a large sparse n times n matrix and b is an n vector arises in many applications this paper describes how the faber transform applied to the field of values of a can be used to determine improved error bounds for popular polynomial approximation methods based on the arnoldi process applications of the faber transform to rational approximation methods and in particular to the rational arnoldi process also are discussed

numerical range and quasi sectorial contractions we apply a method developed by one of the authors see cite arl1 to localize the numerical range of textit quasi sectorial contractions semigroups our main theorem establishes a relation between the numerical range of quasi sectorial contraction semigroups exp t s t ge 0 and the maximal sectorial generators s we also give a new prove of the rate o 1 n for the operator norm euler formula approximation exp t s lim limits n to infty i ts n n t ge 0 for this class of semigroups

a blind source separation technique using second order statistics separation of sources consists of recovering a set of signals of which only instantaneous linear mixtures are observed in many situations no a priori information on the mixing matrix is available the linear mixture should be blindly processed this typically occurs in narrowband array processing applications when the array manifold is unknown or distorted this paper introduces a new source separation technique exploiting the time coherence of the source signals in contrast with other previously reported techniques the proposed approach relies only on stationary second order statistics that are based on a joint diagonalization of a set of covariance matrices asymptotic performance analysis of this method is carried out some numerical simulations are provided to illustrate the effectiveness of the proposed method

a bundle newton method for nonsmooth unconstrained minimization an algorithm based on a combination of the polyhedral and quadratic approximation is given for finding stationary points for unconstrained minimization problems with locally lips chitz problem functions that are not necessarily convex or differentiable global convergence of the algorithm is established under additional assumptions it is shown that the algorithm generates newton iterations and that the convergence is superlinear some encouraging numerical experience is reported

tests for complete k spectral sets abstract let be a family of functions analytic in some neighborhood of a complex domain and let t be a hilbert space operator whose spectrum is contained in our typical result shows that under some extra conditions if the closed unit disc is complete k spectral for t for every then is complete k spectral for t for some constant k in particular we prove that under a geometric transversality condition the intersection of finitely many k spectral sets for t is again k spectral for some k k these theorems generalize and complement results by mascioni stessin stampfli badea beckermann crouzeix and others we also extend to non convex domains a result by putinar and sandberg on the existence of a skew dilation of t to a normal operator with spectrum in as a key tool we use the results from our previous paper 11 on traces of analytic uniform algebras

smoothing methods for nonsmooth nonconvex minimization we consider a class of smoothing methods for minimization problems where the feasible set is convex but the objective function is not convex not differentiable and perhaps not even locally lipschitz at the solutions such optimization problems arise from wide applications including image restoration signal reconstruction variable selection optimal control stochastic equilibrium and spherical approximations in this paper we focus on smoothing methods for solving such optimization problems which use the structure of the minimization problems and composition of smoothing functions for the plus function x many existing optimization algorithms and codes can be used in the inner iteration of the smoothing methods we present properties of the smoothing functions and the gradient consistency of subdifferential associated with a smoothing function moreover we describe how to update the smoothing parameter in the outer iteration of the smoothing methods to guarantee convergence of the smoothing methods to a stationary point of the original minimization problem

separation of an instantaneous mixture of gaussian autoregressive sources by the exact maximum likelihood approach this paper deals with the problem of blind separation of an instantaneous mixture of gaussian autoregressive sources without additive noise by the exact maximum likelihood approach the maximization of the likelihood function is divided using relaxation into two suboptimization problems solved by relaxation methods as well the first one consists of the estimation of the separating matrix when the autoregressive structure of the sources is fixed the second one aims at estimating this structure when the separating matrix is fixed we show that the first problem is equivalent to the determinant maximization of the separating matrix under nonlinear constraints we prove the existence and the consistency of the maximum likelihood estimator we also give the expression of fisher s information matrix then we study by computer simulations the performance of our estimator and show the improvement of its achievements w r t both quasimaximum likelihood and second order blind identification sobi estimators

convergence analysis of the extended krylov subspace method for the lyapunov equation the extended krylov subspace method has recently arisen as a competitive method for solving large scale lyapunov equations using the theoretical framework of orthogonal rational functions in this paper we provide a general a priori error estimate when the known term has rank one special cases such as symmetric coefficient matrix are also treated numerical experiments confirm the proved theoretical assertions

the numerical range and the spectrum of a product of two orthogonal projections the aim of this paper is to describe the closure of the numerical range of the product of two orthogonal projections in hilbert space as a closed convex hull of some explicit ellipses parametrized by points in the spectrum several improvements removing the closure of the numerical range of the operator using a parametrization after its eigenvalues are possible under additional assumptions an estimate of the least angular opening of a sector with vertex 1 containing the numerical range of a product of two orthogonal projections onto two subspaces is given in terms of the cosine of the friedrichs angle applications to the rate of convergence in the method of alternating projections and to the uncertainty principle in harmonic analysis are also discussed

on nesterov s nonsmooth chebyshev rosenbrock functions abstract we discuss two nonsmooth functions on r n introduced by nesterov we show that the first variant is partly smooth in the sense of lewis and that its only stationary point is the global minimizer in contrast we show that the second variant has 2 n 1 clarke stationary points none of them local minimizers except the global minimizer but also that its only mordukhovich stationary point is the global minimizer nonsmooth optimization algorithms from multiple starting points generate iterates that approximate all 2 n 1 clarke stationary points not only the global minimizer but it remains an open question as to whether the nonminimizing clarke stationary points are actually points of attraction for optimization algorithms

an approximation scheme for distributionally robust nonlinear optimization we consider distributionally robust optimization problems drops with nonlinear and nonconcave dependence on uncertain parameters the drop can be written as a nonsmooth nonlinear program with a

crouzeix s conjecture and perturbed jordan blocks abstract crouzeix conjectured in m crouzeix bounds for analytical functions of matrices integr equ oper theory 48 2004 461 477 that for any square matrix a and any polynomial p p a 2 max p z z w a where w a is the field of values of a and denotes the spectral norm in this paper we show that the conjecture holds for matrices of the form 1 1 where and are complex numbers the technique for 1 and for 1 is to show that if g is a bijective conformal mapping from the field of values of such a matrix a to the unit disk mapping to 0 then g a is similar to a contraction via a similarity transformation with condition number at most 2

a sequential quadratic programming algorithm for nonconvex nonsmooth constrained optimization we consider optimization problems with objective and constraint functions that may be nonconvex and nonsmooth problems of this type arise in important applications many having solutions at points of nondifferentiability of the problem functions we present a line search algorithm for situations when the objective and constraint functions are locally lipschitz and continuously differentiable on open dense subsets of mathbb r n our method is based on a sequential quadratic programming sqp algorithm that uses an ell 1 penalty to regularize the constraints a process of gradient sampling gs is employed to make the search direction computation effective in nonsmooth regions we prove that our sqp gs method is globally convergent to stationary points with probability one and illustrate its performance with a matlab implementation

a proof of crouzeix s conjecture for a class of matrices abstract crouzeix s conjecture is that for any square matrix a and any polynomial p we have p a 2 max p z z w a where w a is the field of values of a and denotes the spectral norm in this paper we show that the conjecture holds for the matrices of the form 1 n 1 n where c and 1 n c n

clarke subgradients for directionally lipschitzian stratifiable functions using a geometric argument we show that under a reasonable continuity condition the clarke subdifferential of a semi algebraic or more generally stratifiable directionally lipschitzian function admits a simple form the normal cone to the domain and limits of gradients generate the entire clarke subdifferential the characterization formula we obtain unifies various apparently disparate results that have appeared in the literature our techniques also yield a simplified proof that closed semialgebraic functions on rn have a limiting subdifferential graph of uniform local dimension n

an error analysis for rational galerkin projection applied to the sylvester equation in this paper we suggest a new formula for the residual of galerkin projection onto rational krylov spaces applied to a sylvester equation and establish a relation to three different underlying extremal problems for rational functions these extremal problems enable us to compare the size of the residual for the above method with that obtained by adi in addition we deduce several new a priori error estimates for galerkin projection onto rational krylov spaces both for the sylvester and for the lyapunov equation

stabilization via nonsmooth nonconvex optimization nonsmooth variational analysis and related computational methods are powerful tools that can be effectively applied to identify local minimizers of nonconvex optimization problems arising in fixed order controller design we support this claim by applying nonsmooth analysis and methods to a challenging belgian chocolate stabilization problem posed in 1994 find a stable minimum phase rational controller that stabilizes a specified second order plant although easily stated this particular problem remained unsolved until 2002 when a solution was found using an eleventh order controller our computational methods find a stabilizing third order controller without difficulty suggesting explicit formulas for the controller and for the closed loop system which has only one pole with multiplicity 5 furthermore our analytical techniques prove that this controller is locally optimal in the sense that there is no nearby controller with the same order for which the closed loop system has all its poles further left in the complex plane although the focus of the paper is stabilization once a stabilizing controller is obtained the same computational techniques can be used to optimize various measures of the closed loop system including its complex stability radius or hinfin performance

paraexp a parallel integrator for linear initial value problems a novel parallel algorithm for the integration of linear initial value problems is proposed this algorithm is based on the simple observation that homogeneous problems can typically be integrated much faster than inhomogeneous problems an overlapping time domain decomposition is utilized to obtain decoupled inhomogeneous and homogeneous subproblems and a near optimal krylov method is used for the fast exponential integration of the homogeneous subproblems we present an error analysis and discuss the parallel scaling of our algorithm the efficiency of this approach is demonstrated with numerical examples

approximating subdifferentials by random sampling of gradients many interesting real functions on euclidean space are differentiable almost everywhere all lipschitz functions have this property but so for example does the spectral abscissa of a matrix a non lipschitz function in practice the gradient is often easy to compute we investigate to what extent we can approximate the clarke subdifferential of such a function at some point by calculating the convex hull of some gradients sampled at random nearby points

a modified gautschi s method without order reduction when integrating boundary value nonlinear wave problems abstract in this paper we analyse the order reduction which turns up when integrating nonlinear wave problems with non homogeneous and time dependent boundary conditions with the well known gautschi s method moreover a technique is suggested to avoid that order reduction so that the classical local order 4 and global order 2 are recovered on the other hand the usual approximation for the derivative which is used together with this method is also analysed and a substantial improvement is suggested some numerical results are shown which corroborate the performed analysis

variational analysis of non lipschitz spectral functions we consider spectral functions f where f is any permutation invariant mapping from cn to r and is the eigenvalue map from the set of n n complex matrices to cn ordering the eigenvalues lexicographically for example if f is the function maximum real part then f is the spectral abscissa while if f is maximum modulus then f is the spectral radius both these spectral functions are continuous but they are neither convex nor lipschitz for our analysis we use the notion of subgradient extensively analyzed in variational analysis r t rockafellar and r j b wets springer 1998 we show that a necessary condition for y to be a subgradient of an eigenvalue function f at x is that y commutes with x we also give a number of other necessary conditions for y based on the schur form and the jordan form of x in the case of the spectral abscissa we refine these conditions and we precisely identify the case where subdifferential regularity holds we conclude by introducing the notion of a semistable program maximize a linear function on the set of square matrices subject to linear equality constraints together with the constraint that the real parts of the eigenvalues of the solution matrix are non positive semistable programming is a nonconvex generalization of semidefinite programming using our analysis we derive a necessary condition for a local maximizer of a semistable program and we give a generalization of the complementarity condition familiar from semidefinite programming

nonsmooth optimization via quasi newton methods we investigate the behavior of quasi newton algorithms applied to minimize a nonsmooth function f not necessarily convex we introduce an inex act line search that generates a sequence of nested intervals containing a set of points of nonzero measure that satisfy the armijo and wolfe conditions if f is absolutely continuous along the line furthermore the line search is guaranteed to terminate if f is semi algebraic it seems quite difficult to establish a convergence theorem for quasi newton methods applied to such general classes of functions so we give a care ful analysis of a special but illuminating case the euclidean norm in one variable using the inexact line search and in two variables assuming that the line search is exact in practice we find that when f is locally lipschitz and semi algebraic with bounded sublevel sets the bfgs broyden fletcher goldfarb shanno method with the inexact line search almost always generates sequences whose cluster points are clarke stationary and with function values converging r linearly to a clarke station ary value we give references documenting the successful use of bfgs in a variety of nonsmooth applications particularly the design of low order controllers for linear dynamical systems we conclude with a challenging open question

piecewise linear approximations in nonconvex nonsmooth optimization we present a bundle type method for minimizing nonconvex nondifferentiable functions of several variables the algorithm is based on the construction of both a lower and an upper polyhedral approximation of the objective function in particular at each iteration a search direction is computed by solving a quadratic program aiming at maximizing the difference between the lower and the upper model a proximal approach is used to guarantee convergence to a stationary point under the hypothesis of weak semismoothness

worst case complexity of smoothing quadratic regularization methods for non lipschitzian optimization in this paper we propose a smoothing quadratic regularization sqr algorithm for solving a class of nonsmooth nonconvex perhaps even non lipschitzian minimization problems which has wide applications in statistics and sparse reconstruction the proposed sqr algorithm is a first order method at each iteration the sqr algorithm solves a strongly convex quadratic minimization problem with a diagonal hessian matrix which has a simple closed form solution that is inexpensive to calculate we show that the worst case complexity of reaching an epsilon scaled stationary point is o epsilon 2 moreover if the objective function is locally lipschitz continuous the sqr algorithm with a slightly modified updating scheme for the smoothing parameter and iterate can obtain an epsilon clarke stationary point in at most o epsilon 3 iterations

an arnoldi extrapolation algorithm for computing pagerank the arnoldi type algorithm proposed by golub and greif g golub c greif an arnoldi type algorithm for computing pagerank bit 46 2006 759 771 is a restarted krylov subspace method for computing pagerank however this algorithm may not be efficient when the damping factor is high and the dimension of the search subspace is small in this paper we first develop an extrapolation method based on ritz values we then consider how to periodically knit this extrapolation method together with the arnoldi type algorithm the resulting algorithm is the arnoldi extrapolation algorithm the convergence of the new algorithm is analyzed numerical experiments demonstrate the numerical behavior of this algorithm

characterization of half radial matrices abstract numerical radius r a is the radius of the smallest ball with the center at zero containing the field of values of a given square matrix a it is well known that r a a 2 r a where is the matrix 2 norm matrices attaining the lower bound are called radial and have been analyzed thoroughly this is not the case for matrices attaining the upper bound where only partial results are available in this paper we consider matrices satisfying r a a 2 and call them half radial we summarize the existing results and formulate new ones in particular we investigate their singular value decomposition and algebraic structure and provide other necessary and sufficient conditions for a matrix to be half radial based on that we study the extreme case of the attainable constant 2 in crouzeix s conjecture the presented results support the conjecture of greenbaum and overton that the crabb choi crouzeix matrix always plays an important role in this extreme case

a maximum likelihood based minimum mean square error separation and estimation of stationary gaussian sources from noisy mixtures in the context of independent component analysis noisy mixtures pose a dilemma regarding the desired objective on one hand a maximally separating solution providing the minimal attainable interference to source ratio isr would often suffer from significant residual noise on the other hand optimal minimum mean square error mmse estimation would yield estimates that are the closest possible to the true sources often at the cost of compromised isr in this paper we consider noisy mixtures of temporally diverse stationary gaussian sources in a semi blind scenario which conveniently lends itself to either one of these objectives we begin by deriving the maximum likelihood ml estimates of the unknown deterministic parameters of the model the mixing matrix and the possibly different noise variances in each sensor we derive the likelihood equations for these parameters as well as the corresponding cramer rao lower bound and propose an iterative solution for obtaining the ml estimates mles based on these mles the asymptotically optimal maximally separating solution can be readily obtained however we also present the ml based mmse estimate of the sources alongside a frequency domain based computationally efficient scheme exploiting their stationarity we show that this estimate is asymptotically optimal and attains the oracle mmse lower bound furthermore for non gaussian signals we show that this estimate serves as a quasi ml based linear mmse lmmse estimate and attains the oracle lmmse lower bound asymptotically empirical results of three simulation experiments are presented corroborating our analytical derivations

a gradient sampling method on algebraic varieties and application to nonsmooth low rank optimization in this paper a nonsmooth optimization method for locally lipschitz functions on real algebraic varieties is developed to this end the set valued map varepsilon conditional subdifferential x

numerical investigation of crouzeix s conjecture abstract crouzeix s conjecture states that for all polynomials p and matrices a the inequality p a 2 p w a holds where the quantity on the left is the 2 norm of the matrix p a and the norm on the right is the maximum modulus of the polynomial p on w a the field of values of a we report on some extensive numerical experiments investigating the conjecture via nonsmooth minimization of the crouzeix ratio f p w a p a using chebfun to evaluate this quantity accurately and efficiently and the bfgs method to search for its minimal value which is 0 5 if crouzeix s conjecture is true almost all of our optimization searches deliver final polynomial matrix pairs that are very close to nonsmooth stationary points of f with stationary value 0 5 for which w a is a disk or smooth stationary points of f with stationary value 1 for which w a has a corner our observations have led us to some additional conjectures as well as some new theorems we hope that these give insight into crouzeix s conjecture which is strongly supported by our results

an algorithm for nonsmooth optimization by successive piecewise linearization we present an optimization method for lipschitz continuous piecewise smooth ps objective functions based on successive piecewise linearization since in many realistic cases nondifferentiabilities are caused by the occurrence of abs max and min we concentrate on these nonsmooth elemental functions the method s idea is to locate an optimum of a ps objective function by explicitly handling the kink structure at the level of piecewise linear models this piecewise linearization can be generated in its abs normal form by minor extension of standard algorithmic or automatic differentiation tools in this paper it is shown that the new method when started from within a compact level set generates a sequence of iterates whose cluster points are all clarke stationary numerical results including comparisons with other nonsmooth optimization methods then illustrate the capabilities of the proposed approach

positive trigonometric polynomials for strong stability of difference equations we follow a polynomial approach to analyse strong stability of continuous time linear difference equations with several delays upon application of the hermite stability criterion on the discrete time homogeneous characteristic polynomial assessing strong stability amounts to deciding positive definiteness of a multivariate trigonometric polynomial matrix this latter problem is addressed with a converging hierarchy of linear matrix inequalities lmis numerical experiments indicate that certificates of strong stability can be obtained at a reasonable computational cost for state dimension and number of delays not exceeding 4 or 5

upper and lower bounds on norms of functions of matrices abstract given an n by n matrix a we look for a set s in the complex plane and positive scalars m and m such that for all functions p bounded and analytic on s and throughout a neighborhood of each eigenvalue of a the inequalities m inf f l s f a p a p a m inf f l s f a p a hold we show that for 2 by 2 matrices if s is the field of values then one can take m 1 and m 2 we show that for a perturbed jordan block a matrix a that is an n by n jordan block with eigenvalue 0 except that its n 1 entry is with 0 1 if s is the unit disk then m m 1 we argue however that in general due to the behavior of minimal norm interpolating functions it may be very difficult or impossible to find such a set s for which the ratio m m is of moderate size

the smoothed spectral abscissa for robust stability optimization this paper concerns the stability optimization of parameterized matrices a x a problem typically arising in the design of fixed order or fixed structured feedback controllers it is well known that the minimization of the spectral abscissa function alpha a gives rise to very difficult optimization problems since alpha a is not everywhere differentiable and even not everywhere lipschitz we therefore propose a new stability measure namely the smoothed spectral abscissa tilde alpha epsilon a which is based on the inversion of a relaxed h 2 type cost function the regularization parameter epsilon allows tuning the degree of smoothness for epsilon approaching zero the smoothed spectral abscissa converges towards the nonsmooth spectral abscissa from above so that tilde alpha epsilon a leq0 guarantees asymptotic stability evaluation of the smoothed spectral abscissa and its derivatives w r t matrix parameters x can be performed at the cost of solving a primal dual lyapunov equation pair allowing for an efficient integration into a derivative based optimization framework two optimization problems are considered on the one hand the minimization of the smoothed spectral abscissa tilde alpha epsilon a x as a function of the matrix parameters for a fixed value of epsilon and on the other hand the maximization of epsilon such that the stability requirement tilde alpha epsilon a x leq0 is still satisfied the latter problem can be interpreted as an h 2 norm minimization problem and its solution additionally implies an upper bound on the corresponding h infty norm or a lower bound on the distance to instability in both cases additional equality and inequality constraints on the variables can be naturally taken into account in the optimization problem

solving maximum entropy sampling problems using factored masks we present a practical approach to anstreicher and lee s masked spectral bound for maximum entropy sampling and we describe favorable results that we have obtained with a branch and bound algorithm based on our approach by representing masks in factored form we are able to easily satisfy a semidefiniteness constraint moreover this representation allows us to restrict the rank of the mask as a means for attempting to practically incorporate second order information

decay properties for functions of matrices over c algebras abstract we extend previous results on the exponential off diagonal decay of the entries of analytic functions of banded and sparse matrices to the case where the matrix entries are elements of a c algebra

lemniscates and k spectral sets abstract we show how multicentric representation of functions provides a simple way to generalize the von neumann result that the unit disc is a spectral set for contractions in hilbert spaces in particular the sets need not be connected and the results can be applied to bounding riesz spectral projections

two numerical methods for optimizing matrix stability consider the affine matrix family a x a0 m 1 xkak mapping a design vector x r m into the space of n n real matrices we are interested in the question of how to choose x to optimize the stability of the matrix a x a typical motivation is that one wishes to control the stability of the dynamical system a x z a classic example is stabilization by output feedback we take two approaches let x denote the spectral abscissa the largest real part of the eigenvalues of a matrix x as is well known this quantity bounds the asymptotic decay rate of the trajectories of the associated dynamical system z xz our first approach to optimizing stability is to directly minimize the function a x the spectral abscissa x is a continuous but non smooth in fact non lipschitz function of the matrix argument x and finding a global minimizer of a x is hard we introduce a novel random gradient bundle method for approximating local minimizers motivated by recent work on nonsmooth analysis of the function x our second approach is to minimize a related function a x where is a robustness parameter in 0 1 one motivation for the definition of the robust spectral abscissa x is that it bounds transient peaks as well as asymptotic decay of trajectories of xz the function x is lipschitz but non convex for 0 1

convergence of the gradient sampling algorithm for nonsmooth nonconvex optimization we study the gradient sampling algorithm of burke lewis and overton for minimizing a locally lipschitz function f on mathbb r n that is continuously differentiable on an open dense subset we strengthen the existing convergence results for this algorithm and introduce a slightly revised version for which stronger results are established without requiring compactness of the level sets of f in particular we show that with probability 1 the revised algorithm either drives the f values to infty or each of its cluster points is clarke stationary for f we also consider a simplified variant in which the differentiability check is skipped and the user can control the number of f evaluations per iteration

blind separation of gaussian sources with general covariance structures bounds and optimal estimation we consider the separation of gaussian sources exhibiting general arbitrary not necessarily stationary covariance structures first assuming a semi blind scenario in which the sources covariance structures are known we derive the maximum likelihood estimate of the separation matrix as well as the induced cramer rao lower bound icrlb on the attainable interference to source ratio isr we then extend our results to the fully blind scenario in which the covariance structures are unknown we show that under a scaling convention the fisher information matrix in this case is block diagonal implying that the same icrlb as in the semi blind scenario applies in this case as well subsequently we demonstrate that the same semi blind optimal performance can be approached asymptotically in the fully blind scenario if the sources are sufficiently ergodic or if multiple snapshots are available

a nonderivative version of the gradient sampling algorithm for nonsmooth nonconvex optimization we give a nonderivative version of the gradient sampling algorithm of burke lewis and overton for minimizing a locally lipschitz function f on mathbb r n that is continuously differentiable on an open dense subset instead of gradients of f we use estimates of gradients of the steklov averages of f obtained by convolution with mollifiers which require f values only we show that the nonderivative version retains the convergence properties of the gradient sampling algorithm in particular with probability 1 it either drives the f values to infty or each of its cluster points is clarke stationary for f

on the convergence of krylov subspace methods for matrix mittag leffler functions in this paper we analyze the convergence of some commonly used krylov subspace methods for computing the action of matrix mittag leffler functions as is well known such functions find application in the solution of fractional differential equations we illustrate the theoretical results by some numerical experiments

efficient method for computing lower bounds on the p radius of switched linear systems abstract this paper proposes lower bounds on a quantity called l p norm joint spectral radius or in short p radius of a finite set of matrices despite its wide range of applications to for example stability analysis of switched linear systems and the equilibrium analysis of switched linear economical models algorithms for computing the p radius are only available in a very limited number of particular cases the proposed lower bounds are given as the spectral radius of an average of the given matrices weighted via kronecker products and do not place any requirements on the set of matrices we show that the proposed lower bounds theoretically extend and also can practically improve the existing lower bounds a markovian extension of the proposed lower bounds is also presented

a robust gradient sampling algorithm for nonsmooth nonconvex optimization let f be a continuous function on rl n and suppose f is continuously differentiable on an open dense subset such functions arise in many applications and very often minimizers are points at which f is not differentiable of particular interest is the case where f is not convex and perhaps not even locally lipschitz but is a function whose gradient is easily computed where it is defined we present a practical robust algorithm to locally minimize such functions based on gradient sampling no subgradient information is required by the algorithm when f is locally lipschitz and has bounded level sets and the sampling radius eps is fixed we show that with probability 1 the algorithm generates a sequence with a cluster point that is clarke eps stationary furthermore we show that if f has a unique clarke stationary point bar x then the set of all cluster points generated by the algorithm converges to bar x as eps is reduced to zero numerical results are presented demonstrating the robustness of the algorithm and its applicability in a wide variety of contexts including cases where f is not locally lipschitz at minimizers we report approximate local minimizers for functions in the applications literature which have not to our knowledge been obtained previously when the termination criteria of the algorithm are satisfied a precise statement about nearness to clarke eps stationarity is available a matlab implementation of the algorithm is posted at http www cs nyu edu overton papers gradsamp alg

an error analysis of galerkin projection methods for linear systems with tensor product structure recent results on the convergence of a galerkin projection method for the sylvester equation are extended to more general linear systems with tensor product structure in the hermitian positive definite case explicit convergence bounds are derived for galerkin projection based on tensor products of rational krylov subspaces the results can be used to optimize the choice of shifts for these methods numerical experiments demonstrate that the convergence rates predicted by our bounds appear to be sharp

stable als approximation in the tt format for rank adaptive tensor completion low rank tensor completion is a highly ill posed inverse problem particularly when the data model is not accurate and some sort of regularization is required in order to solve it in this article we focus on the calibration of the data model for alternating optimization we observe that existing rank adaption methods do not enable a continuous transition between manifolds of different ranks we denote this characteristic as instability under truncation as a consequence of this property arbitrarily small changes in the iterate can have arbitrarily large influence on the further reconstruction we therefore introduce a singular value based regularization to the standard alternating least squares als which is motivated by averaging in microsteps we prove its stability and derive a natural semi implicit rank adaption strategy we further prove that the standard als microsteps for completion problems are only stable on manifolds of fixed ranks and only around points that have what we define as internal tensor restricted isometry property itrip in conclusion numerical experiments are provided that show improvements of the reconstruction quality up to orders of magnitude in the new stable als approximation salsa compared to standard als and the well known riemannian optimization rttc

blind separation of mixture of independent sources through a quasi maximum likelihood approach we propose two methods for separating mixture of independent sources without any precise knowledge of their probability distribution they are obtained by considering a maximum likelihood ml solution corresponding to some given distributions of the sources and relaxing this assumption afterward the first method is specially adapted to temporally independent non gaussian sources and is based on the use of nonlinear separating functions the second method is specially adapted to correlated sources with distinct spectra and is based on the use of linear separating filters a theoretical analysis of the performance of the methods has been made a simple procedure for optimally choosing the separating functions is proposed further in the second method a simple implementation based on the simultaneous diagonalization of two symmetric matrices is provided finally some numerical and simulation results are given illustrating the performance of the method and the good agreement between the experiments and the theory

an algorithm twisted from generalized admm for multi block separable convex minimization models the alternating direction method with multipliers admm has been one of most powerful and successful methods for solving a two block linearly constrained convex minimization model whose objective function is the sum of two functions without coupled variables it is known that the numerical efficiency is inherited for a large number of applications but the convergence is not guaranteed if the admm is directly extended to a multiple block convex minimization model whose objective function has more than two functions this viewpoint was in fact the motivation for developing efficient algorithms that cannot only preserve the numerical advantages of the direct extension of admm but also guarantee convergence one way is to correct the output of the direct extension of admm slightly via a simple correction step and the other is to employ a simple proximal to solve inexactly each subproblem in the direct extension of admm in this paper in order to solve the multi block separable convex minimization model efficiently we present a method which is a combination of the above two ways that is we first solve each subproblem with a simple proximal then we correct the output via a simple correction step theoretically we derive global convergence results for this method and establish a worst case o 1 k iteration complexity numerically the efficiency of this method can be showed by testing the problem of recovering low rank and sparse components of matrices from incomplete and noisy observation

equivariant adaptive source separation source separation consists of recovering a set of independent signals when only mixtures with unknown coefficients are observed this paper introduces a class of adaptive algorithms for source separation that implements an adaptive version of equivariant estimation and is henceforth called equivariant adaptive separation via independence easi the easi algorithms are based on the idea of serial updating this specific form of matrix updates systematically yields algorithms with a simple structure for both real and complex mixtures most importantly the performance of an easi algorithm does not depend on the mixing matrix in particular convergence rates stability conditions and interference rejection levels depend only on the normalized distributions of the source signals closed form expressions of these quantities are given via an asymptotic performance analysis the theme of equivariance is stressed throughout the paper the source separation problem has an underlying multiplicative structure the parameter space forms a matrix multiplicative group we explore the favorable consequences of this fact on implementation performance and optimization of easi algorithms

descent methods for composite nondifferentiable optimization problems we present a framework for the development of globally defined descent algorithms for the minimization of non differentiable objective functionsf h o f withh convex within our structure the global convergence properties of the cauchy modified newton gauss newton and variable metric methods are easily established along with that of several new approaches examples illustrating the calculational techniques are provided

nonsmooth analysis of eigenvalues the eigenvalues of a symmetric matrix depend on the matrix nonsmoothly this paper describes the nonsmooth analysis of these eigenvalues in particular i present a simple formula for the approximate limiting frechet subdifferential of an arbitrary function of the eigenvalues subsuming earlier results on convex and clarke subgradients as an example i compute the subdifferential of the k th largest eigenvalue

regularized mathematical programs with stochastic equilibrium constraints estimating structural demand models the article considers a particular class of optimization problems involving set valued stochastic equilibrium constraints we develop a solution procedure that relies on an approxima tion scheme for the equilibrium constraints based on regularization we replace the approximated equilibrium constraints by those involving only single valued lipschitz continuous functions in ad dition sampling has the further effect of replacing the simplified equilibrium constraints by more manageable ones obtained by implicitly discretizing the given probability measure so as to render the problem computationally tractable convergence is obtained by relying in particular on the graphical convergence of the approximated equilibrium constraints the problem of estimating the characteristics of a demand model a widely studied problem in microeconometrics serves both as motivation and illustration of the regularization and sampling procedure

nonsmooth h_infty synthesis we develop nonsmooth optimization techniques to solve h infty synthesis problems under additional structural constraints on the controller our approach avoids the use of lyapunov variables and therefore leads to moderate size optimization programs even for very large systems the proposed framework is versatile and can accommodate a number of challenging design problems including static fixed order fixed structure decentralized control design of pid controllers and simultaneous design and stabilization problems our algorithmic strategy uses generalized gradients and bundling techniques suited for the h infty norm and other nonsmooth performance criteria we compute descent directions by solving quadratic programs and generate steps via line search convergence to a critical point from an arbitrary starting point is proved and numerical tests are included to validate our methods the proposed approach proves to be efficient even for systems with several hundreds of states

numerical range and functional calculus in hilbert space we prove an inequality related to polynomial functions of a square matrix involving the numerical range of the matrix we also show extensions valid for bounded and also unbounded operators in hilbert spaces which allow the development of a functional calculus

epidemic threshold and immunization on generalized networks abstract the susceptible infected susceptible sis model is widely adopted in the studies of epidemic dynamics when it is applied on contact networks these networks mostly consist of nodes connected by undirected and unweighted edges following certain statistical properties whereas in this article we consider the threshold and immunization problem for the sis model on generalized networks that may contain different kinds of nodes and edges which are very possible in the real situation we proved that an epidemic will become extinct if and only if the spectral radius of the corresponding parameterized adjacent matrix pam is smaller than 1 based on this result we can evaluate the efficiency of immune strategies and take several prevailing ones as examples in addition we also develop methods that can precisely find the optimal immune strategies for networks with the given pam

decay bounds for the numerical quasiseparable preservation in matrix functions abstract given matrices a and b such that b f a where f z is a holomorphic function we analyze the relation between the singular values of the off diagonal submatrices of a and b we provide a family of bounds which depend on the interplay between the spectrum of the argument a and the singularities of the function in particular these bounds guarantee the numerical preservation of quasiseparable structures under mild hypotheses we extend the dunford cauchy integral formula to the case in which some poles are contained inside the contour of integration we use this tool together with the technology of hierarchical matrices h matrices for the effective computation of matrix functions with quasiseparable arguments

a version of the bundle idea for minimizing a nonsmooth function conceptual idea convergence analysis numerical results during recent years various proposals for the minimization of a nonsmooth functional have been made amongst these the bundle concept turned out to be an especially fruitful idea based on this concept a number of authors have developed codes that can successfully deal with nonsmooth problems the aim of the paper is to show that by adding some features of the trust region philosophy to the bundle concept the end result is a distinguished member of the bundle family with a more stable behaviour than some other bundle versions the reliability and efficiency of this code is demonstrated on the standard academic test examples and on some real life problems

pseudospectra of linear operators if a matrix or linear operator a is far from normal its eigenvalues or more generally its spectrum may have little to do with its behavior as measured by quantities such as an or exp ta more may be learned by examining the sets in the complex plane known as the pseudospectra of a defined by level curves of the norm of the resolvent zi a 1 five years ago the author published a paper that presented computed pseudospectra of thirteen highly nonnormal matrices arising in various applications since that time analogous computations have been carried out for differential and integral operators this paper a companion to the earlier one presents ten examples each chosen to illustrate one or more mathematical or physical principles

saa regularized methods for multiproduct price optimization under the pure characteristics demand model utility based choice models are often used to determine a consumer s purchase decision among a list of available products to provide an estimate of product demands and when data on purchase decisions or market shares are available to infer consumers preferences over observed product characteristics these models also serve as a building block in modeling firms pricing and assortment optimization problems we consider a firm s multiproduct pricing problem in which product demands are determined by a pure characteristics model a sample average approximation saa method is used to approximate the expected market share of products and the firm profit we propose an saa regularized method for the multiproduct price optimization problem we present convergence analysis and numerical examples to show the efficiency and the effectiveness of the proposed method

a rational arnoldi approach for ill conditioned linear systems for the solution of full rank ill posed linear systems a new approach based on the arnoldi algorithm is presented working with regularized systems the method theoretically reconstructs the true solution by means of the computation of a suitable function of matrix in this sense the method can be referred to as an iterative refinement process numerical experiments arising from integral equations and interpolation theory are presented finally the method is extended to work in connection with the standard tikhonov regularization with the right hand side contaminated by noise

between the von neumann inequality and the crouzeix conjecture abstract a new concept of a deformed numerical range w t is introduced here t is a bounded linear operator or a matrix and 1 is a parameter each w t is a closed convex set that contains the spectrum of t furthermore w t is decreasing with respect to and w 2 t coincides with the numerical range it is also shown that w t is contained in the closed unit disc if and only if t has a unitary dilation in the sense of nagy foias the spectral constant of w t is investigated it is shown that it is monotone and continuous with respect to the parameter

on the perturbation of the group inverse and oblique projection for given matrices a and e which are n matrices and matrices and e we discuss upper bounds for b bb b a a and bb aa aa also a lower bound for b is derived

stability analysis of systems with stochastically varying delays abstract a stability analysis of linear systems with stochastically varying delay is performed it is assumed that the delay function has the form of a sawtooth with switches occurring at the arrival times of a homogeneous poisson process several notions of stochastic stability are considered and corresponding stability criteria are derived for two examples the different criteria are compared and the effect on stability of various deterministic approximations is examined

variational analysis of the crouzeix ratio let w a denote the field of values numerical range of a matrix a for any polynomial p and matrix a define the crouzeix ratio to have numerator max left p zeta zeta in w a right max p w a and denominator vert p a vert 2 p a 2 crouzeix s 2004 conjecture postulates that the globally minimal value of the crouzeix ratio is 1 2 over all polynomials p of any degree and matrices a of any order we derive the subdifferential of this ratio at pairs p a for which the largest singular value of p a is simple in particular we show that at certain candidate minimizers p a the crouzeix ratio is clarke regular and satisfies a first order nonsmooth optimality condition and hence that its directional derivative is nonnegative there in every direction in polynomial matrix space we also show that pairs p a exist at which the crouzeix ratio is not regular

idempotency of linear combinations of two idempotent matrices abstract a complete solution is established to the problem of characterizing all situations where a linear combination of two different idempotent matrices p 1 and p 2 is also an idempotent matrix including naturally three such situations known in the literature viz if the combination is either the sum p 1 p 2 or one of the differences p 1 p 2 p 2 p 1 and appropriate additional conditions are fulfilled the solution asserts that in the particular case where p 1 and p 2 are complex matrices such that p 1 p 2 is hermitian these three situations exhaust the list of all possibilities and that this list extends when the above assumption on p 1 and p 2 is violated a statistical interpretation of the idempotency problem considered in this note is also pointed out