a newton cg augmented lagrangian method for semidefinite programming we consider a newton cg augmented lagrangian method for solving semidefinite programming sdp problems from the perspective of approximate semismooth newton methods in order to analyze the rate of convergence of our proposed method we characterize the lipschitz continuity of the corresponding solution mapping at the origin for the inner problems we show that the positive definiteness of the generalized hessian of the objective function in these inner problems a key property for ensuring the efficiency of using an inexact semismooth newton cg method to solve the inner problems is equivalent to the constraint nondegeneracy of the corresponding dual problems numerical experiments on a variety of large scale sdp problems with the matrix dimension n up to 4 110 and the number of equality constraints m up to 2 156 544 show that the proposed method is very efficient we are also able to solve the sdp problem fap36 with n 4 110 and m 1 154 467 in the seventh dimacs implementation challenge much more accurately than in previous attempts

blind separation of positive sources by globally convergent gradient search the instantaneous noise free linear mixing model in independent component analysis is largely a solved problem under the usual assumption of independent nongaussian sources and full column rank mixing matrix however with some prior information on the sources like positivity new analysis and perhaps simplified solution methods may yet become possible in this letter we consider the task of independent component analysis when the independent sources are known to be nonnegative and well grounded which means that they have a nonzero pdf in the region of zero it can be shown that in this case the solution method is basically very simple an orthogonal rotation of the whitened observation vector into nonnegative outputs will give a positive permutation of the original sources we propose a cost function whose minimum coincides with nonnegativity and derive the gradient algorithm under the whitening constraint under which the separating matrix is orthogonal we further prove that in the stiefel manifold of orthogonal matrices the cost function is a lyapunov function for the matrix gradient flow implying global convergence thus this algorithm is guaranteed to find the nonnegative well grounded independent sources the analysis is complemented by a numerical simulation which illustrates the algorithm

global convergence analysis of a discrete time nonnegative ica algorithm when the independent sources are known to be nonnegative and well grounded which means that they have a nonzero pdf in the region of zero oja and plumbley have proposed a nonnegative principal component analysis pca algorithm to separate these positive sources generally it is very difficult to prove the convergence of a discrete time independent component analysis ica learning algorithm however by using the skew symmetry property of this discrete time nonnegative pca algorithm if the learning rate satisfies suitable condition the global convergence of this discrete time algorithm can be proven simulation results are employed to further illustrate the advantages of this theory

completely positive reformulations for polynomial optimization polynomial optimization encompasses a very rich class of problems in which both the objective and constraints can be written in terms of polynomials on the decision variables there is a well established body of research on quadratic polynomial optimization problems based on reformulations of the original problem as a conic program over the cone of completely positive matrices or its conic dual the cone of copositive matrices as a result of this reformulation approach novel solution schemes for quadratic polynomial optimization problems have been designed by drawing on conic programming tools and the extensively studied cones of completely positive and of copositive matrices in particular this approach has been applied to solve key combinatorial optimization problems along this line of research we consider polynomial optimization problems that are not necessarily quadratic for this purpose we use a natural extension of the cone of completely positive matrices namely the cone of completely positive tensors we provide a general characterization of the class of polynomial optimization problems that can be formulated as a conic program over the cone of completely positive tensors as a consequence of this characterization it follows that recent related results for quadratic problems can be further strengthened and generalized to higher order polynomial optimization problems also we show that the conditions underlying the characterization are conceptually the same regardless of the degree of the polynomials defining the problem to illustrate our results we discuss in further detail special and relevant instances of polynomial optimization problems

an adaptive linear approximation algorithm for copositive programs we study linear optimization problems over the cone of copositive matrices these problems appear in nonconvex quadratic and binary optimization for instance the maximum clique problem and other combinatorial problems can be reformulated as such problems we present new polyhedral inner and outer approximations of the copositive cone which we show to be exact in the limit in contrast to previous approximation schemes our approximation is not necessarily uniform for the whole cone but can be guided adaptively through the objective function yielding a good approximation in those parts of the cone that are relevant for the optimization and only a coarse approximation in those parts that are not using these approximations we derive an adaptive linear approximation algorithm for copositive programs numerical experiments show that our algorithm gives very good results for certain nonconvex quadratic problems

emergence of phase and shift invariant features by decomposition of natural images into independent feature subspaces olshausen and field 1996 applied the principle of independence maximization by sparse coding to extract features from natural images this leads to the emergence of oriented linear filters that have simultaneous localization in space and in frequency thus resembling gabor functions and simple cell receptive fields in this article we show that the same principle of independence maximization can explain the emergence of phase and shift invariant features similar to those found in complex cells this new kind of emergence is obtained by maximizing the independence between norms of projections on linear subspaces instead of the independence of simple linear filter outputs the norms of the projections on such independent feature subspaces then indicate the values of invariant features

a fixed point algorithm for blind source separation with nonlinear autocorrelation this paper addresses blind source separation bss problem when source signals have the temporal structure with nonlinear autocorrelation using the temporal characteristics of sources we develop an objective function based on the nonlinear autocorrelation of sources maximizing the objective function we propose a fixed point source separation algorithm furthermore we give some mathematical properties of the algorithm computer simulations for sources with square temporal autocorrelation and the real world applications in the analysis of the magnetoencephalographic recordings meg illustrate the efficiency of the proposed approach thus the presented bss algorithm which is based on the nonlinear measure of temporal autocorrelation provides a novel statistical property to perform bss

a quadratically constrained quadratic optimization model for completely positive cone programming we propose a class of quadratic optimization problems which can be reformulated by completely positive cone programming with the same optimal values the objective function can be any quadratic form the constraints of each problem are described in terms of quadratic forms with no linear terms and all constraints are homogeneous equalities except one inhomogeneous equality where a quadratic form is set to be a positive constant for the equality constraints a hierarchy of copositvity condition is assumed this model is a generalization of the standard quadratic optimization problem of minimizing a quadratic form over the standard simplex and covers many of the existing quadratic optimization problems studied for exact copositive cone and completely positive cone programming relaxations in particular it generalizes the recent results on quadratic optimization problems by burer and the set semidefinite representation by eichfelder and povh optim lett 7 2013 pp 1373 1386

facial reduction and partial polyhedrality we present fra poly a facial reduction algorithm fra for conic linear programs that is sensitive to the presence of polyhedral faces in the cone the main goals of fra and fra poly are the same

separation of non negative mixture of non negative sources using a bayesian approach and mcmc sampling this paper addresses blind source separation in the case where both the source signals and the mixing coefficients are non negative the problem is referred to as non negative source separation and the main application concerns the analysis of spectrometric data sets the separation is performed in a bayesian framework by encoding non negativity through the assignment of gamma priors on the distributions of both the source signals and the mixing coefficients a markov chain monte carlo mcmc sampling procedure is proposed to simulate the resulting joint posterior density from which marginal posterior mean estimates of the source signals and mixing coefficients are obtained results obtained with synthetic and experimental spectra are used to discuss the problem of non negative source separation and to illustrate the effectiveness of the proposed method

feature selection using a piecewise linear network we present an efficient feature selection algorithm for the general regression problem which utilizes a piecewise linear orthonormal least squares ols procedure the algorithm 1 determines an appropriate piecewise linear network pln model for the given data set 2 applies the ols procedure to the pln model and 3 searches for useful feature subsets using a floating search algorithm the floating search prevents the nesting effect the proposed algorithm is computationally very efficient because only one data pass is required several examples are given to demonstrate the effectiveness of the proposed algorithm

l_p norm regularization algorithms for optimization over permutation matrices optimization problems over permutation matrices appear widely in facility layout chip design scheduling pattern recognition computer vision graph matching etc since this problem is np hard due to the combinatorial nature of permutation matrices we relax the variable to be the more tractable doubly stochastic matrices and add an l p norm 0 p 1 regularization term to the objective function the optimal solutions of the l p regularized problem are the same as the original problem if the regularization parameter is sufficiently large a lower bound estimation of the nonzero entries of the stationary points and some connections between the local minimizers and the permutation matrices are further established then we propose an l p regularization algorithm with local refinements the algorithm approximately solves a sequence of l p regularization subproblems by the projected gradient method using a nonmonotone line search with the barzilai borwein step sizes its performance can be fu

compressed suffix arrays and suffix trees with applications to text indexing and string matching the proliferation of online text such as found on the world wide web and in online databases motivates the need for space efficient text indexing methods that support fast string searching we model this scenario as follows consider a text t consisting of n symbols drawn from a fixed alphabet sigma the text t can be represented in n lg sigma bits by encoding each symbol with lg sigma bits the goal is to support fast online queries for searching any string pattern p of m symbols with t being fully scanned only once namely when the index is created at preprocessing time the text indexing schemes published in the literature are greedy in terms of space usage they require omega n lg n additional bits of space in the worst case for example in the standard unit cost ram suffix trees and suffix arrays need omega n memory words each of omega lg n bits these indexes are larger than the text itself by a multiplicative factor of omega smash lg sigma n which is significant when sigma is of constant size such as in textsc ascii or textsc unicode on the other hand these indexes support fast searching either in o m lg sigma time or in o m lg n time plus an output sensitive cost o mathit occ for listing the mathit occ pattern occurrences we present a new text index that is based upon compressed representations of suffix arrays and suffix trees it achieves a fast smash o m lg sigma n lg sigma epsilon n search time in the worst case for any constant 0 epsilon leq 1 using at most smash bigl epsilon 1 o 1 bigr n lg sigma bits of storage our result thus presents for the first time an efficient index whose size is provably linear in the size of the text in the worst case and for many scenarios the space is actually sublinear in practice as a concrete example the compressed suffix array for a typical 100 mb textsc ascii file can require 30 40 mb or less while the raw suffix array requires 500 mb our theoretical bounds improve emph both time and space of previous indexing schemes listing the pattern occurrences introduces a sublogarithmic slowdown factor in the output sensitive cost giving o mathit occ smash lg sigma epsilon n time as a result when the patterns are sufficiently long we can use auxiliary data structures in o n lg sigma bits to obtain a total search bound of o m lg sigma n mathit occ time which is optimal

nonnegative blind source separation by sparse component analysis based on determinant measure the problem of nonnegative blind source separation nbss is addressed in this paper where both the sources and the mixing matrix are nonnegative because many real world signals are sparse we deal with nbss by sparse component analysis first a determinant based sparseness measure named d measure is introduced to gauge the temporal and spatial sparseness of signals based on this measure a new nbss model is derived and an iterative sparseness maximization ism approach is proposed to solve this model in the ism approach the nbss problem can be cast into row to row optimizations with respect to the unmixing matrix and then the quadratic programming qp technique is used to optimize each row furthermore we analyze the source identifiability and the computational complexity of the proposed ism qp method the new method requires relatively weak conditions on the sources and the mixing matrix has high computational efficiency and is easy to implement simulation results demonstrate the effectiveness of our method

the direct extension of admm for multi block convex minimization problems is not necessarily convergent the alternating direction method of multipliers admm is now widely used in many fields and its convergence was proved when two blocks of variables are alternatively updated it is strongly desirable and practically valuable to extend the admm directly to the case of a multi block convex minimization problem where its objective function is the sum of more than two separable convex functions however the convergence of this extension has been missing for a long time neither an affirmative convergence proof nor an example showing its divergence is known in the literature in this paper we give a negative answer to this long standing open question the direct extension of admm is not necessarily convergent we present a sufficient condition to ensure the convergence of the direct extension of admm and give an example to show its divergence

a cooperative recurrent neural network for solving l1 estimation problems with general linear constraints the constrained l1 estimation is an attractive alternative to both the unconstrained l1 estimation and the least square estimation in this letter we propose a cooperative recurrent neural network crnn for solving l1 estimation problems with general linear constraints the proposed crnn model combines four individual neural network models automatically and is suitable for parallel implementation as a special case the proposed crnn includes two existing neural networks for solving unconstrained and constrained l1 estimation problems respectively unlike existing neural networks with penalty parameters for solving the constrained l1 estimation problem the proposed crnn is guaranteed to converge globally to the exact optimal solution without any additional condition compared with conventional numerical algorithms the proposed crnn has a low computational complexity and can deal with the l1 estimation problem with degeneracy several applied examples show that the proposed crnn can obtain more accurate estimates than several existing algorithms

a lagrangian dnn relaxation a fast method for computing tight lower bounds for a class of quadratic optimization problems we propose an efficient computational method for linearly constrained quadratic optimization problems qops with complementarity constraints based on their lagrangian and doubly nonnegative dnn relaxation and first order algorithms the simplified lagrangian completely positive programming cpp relaxation of such qops proposed by arima kim and kojima in 2012 takes one of the simplest forms an unconstrained conic linear optimization problem with a single lagrangian parameter in a cpp matrix variable with its upper left element fixed to 1 replacing the cpp matrix variable by a dnn matrix variable we derive the lagrangian dnn relaxation and establish the equivalence between the optimal value of the dnn relaxation of the original qop and that of the lagrangian dnn relaxation we then propose an efficient numerical method for the lagrangian dnn relaxation using a bisection method combined with the proximal alternating direction multiplier and the accelerated proximal gradient methods numerical results on binary qops quadratic multiple knapsack problems maximum stable set problems and quadratic assignment problems illustrate the superior performance of the proposed method for attaining tight lower bounds in shorter computational time

anechoic blind source separation using wigner marginals blind source separation problems emerge in many applications where signals can be modeled as superpositions of multiple sources many popular applications of blind source separation are based on linear instantaneous mixture models if specific invariance properties are known about the sources for example translation or rotation invariance the simple linear model can be extended by inclusion of the corresponding transformations when the sources are invariant against translations spatial displacements or time shifts the resulting model is called an anechoic mixing model we present a new algorithmic framework for the solution of anechoic problems in arbitrary dimensions this framework is derived from stochastic time frequency analysis in general and the marginal properties of the wigner ville spectrum in particular the method reduces the general anechoic problem to a set of anechoic problems with non negativity constraints and a phase retrieval problem the first type of subproblem can be solved by existing algorithms for example by an appropriate modification of non negative matrix factorization nmf the second subproblem is solved by established phase retrieval methods we discuss and compare implementations of this new algorithmic framework for several example problems with synthetic and real world data including music streams natural 2d images human motion trajectories and two dimensional shapes

misspecified nonconvex statistical optimization for sparse phase retrieval existing nonconvex statistical optimization theory and methods crucially rely on the correct specification of the underlying true statistical models to address this issue we take a first step towards taming model misspecification by studying the high dimensional sparse phase retrieval problem with misspecified link functions in particular we propose a simple variant of the thresholded wirtinger flow algorithm that given a proper initialization linearly converges to an estimator with optimal statistical accuracy for a broad family of unknown link functions we further provide extensive numerical experiments to support our theoretical findings

a nonnegative pca algorithm for independent component analysis we consider the task of independent component analysis when the independent sources are known to be nonnegative and well grounded so that they have a nonzero probability density function pdf in the region of zero we propose the use of a nonnegative principal component analysis nonnegative pca algorithm which is a special case of the nonlinear pca algorithm but with a rectification nonlinearity and we conjecture that this algorithm will find such nonnegative well grounded independent sources under reasonable initial conditions while the algorithm has proved difficult to analyze in the general case we give some analytical results that are consistent with this conjecture and some numerical simulations that illustrate its operation

solving max cut to optimality by intersecting semidefinite and polyhedral relaxations we present a method for finding exact solutions of max cut the problem of finding a cut of maximum weight in a weighted graph we use a branch and bound setting that applies a dynamic version of the bundle method as bounding procedure this approach uses lagrangian duality to obtain a nearly optimal solution of the basic semidefinite max cut relaxation strengthened by triangle inequalities the expensive part of our bounding procedure is solving the basic semidefinite relaxation of the max cut problem which has to be done several times during the bounding process we review other solution approaches and compare the numerical results with our method we also extend our experiments to instances of unconstrained quadratic 0 1 optimization and to instances of the graph equipartition problem the experiments show that our method nearly always outperforms all other approaches in particular for dense graphs where linear programming based methods fail our method performs very well exact solutions are obtained in a reasonable time for any instance of size up to n 100 independent of the density for some problems of special structure we can solve even larger problem classes we could prove optimality for several problems of the literature where to the best of our knowledge no other method is able to do so

a convex geometry based blind source separation method for separating nonnegative sources this paper presents a convex geometry cg based method for blind separation of nonnegative sources first the unaccessible source matrix is normalized to be column sum to one by mapping the available observation matrix then its zero samples are found by searching the facets of the convex hull spanned by the mapped observations considering these zero samples a quadratic cost function with respect to each row of the unmixing matrix together with a linear constraint in relation to the involved variables is proposed upon which an algorithm is presented to estimate the unmixing matrix by solving a classical convex optimization problem unlike the traditional blind source separation bss methods the cg based method does not require the independence assumption nor the uncorrelation assumption compared with the bss methods that are specifically designed to distinguish between nonnegative sources the proposed method requires a weaker sparsity condition provided simulation results illustrate the performance of our method

cooperative recurrent neural networks for the constrained l_ 1 estimator the constrained least absolute deviation l1 estimator is an attractive alternative to both the unconstrained l1 estimator and the least square estimator this paper introduces a constrained l1 method and proposes two cooperative recurrent neural networks crnns for the constrained l1 estimator unlike existing cooperative neural networks the proposed two crnns have a novel weighting cooperation scheme to integrate individual neural network information automatically as a special case the proposed continuous time crnn includes the existing continuous time neural network for unconstrained l1 estimator compared with existing continuous time neural networks for the constrained l1 estimator the proposed continuous time crnn has a lower model complexity and the finite time convergence to the exact optimal solution without any additional condition furthermore compared with existing numerical algorithms for the constrained l1 estimator in addition to a low computational complexity the proposed two crnns are suitable for parallel implementation and can deal with the l1 estimation problem with degeneracy the proposed two crnns are applied to parameter estimation problems under non gaussian noise environments simulation results demonstrate that the proposed crnns are indeed effective in dealing with the l1 estimation problem with nonunique solutions and in obtaining a better solution than relevant algorithms

sparse and non negative bss for noisy data non negative blind source separation bss has raised interest in various fields of research as testified by the wide literature on the topic of non negative matrix factorization nmf in this context it is fundamental that the sources to be estimated present some diversity in order to be efficiently retrieved sparsity is known to enhance such contrast between the sources while producing very robust approaches especially to noise in this paper we introduce a new algorithm in order to tackle the blind separation of non negative sparse sources from noisy measurements we first show that sparsity and non negativity constraints have to be carefully applied on the sought after solution in fact improperly constrained solutions are unlikely to be stable and are therefore sub optimal the proposed algorithm named ngmca non negative generalized morphological component analysis makes use of proximal calculus techniques to provide properly constrained solutions the performance of ngmca compared to other state of the art algorithms is demonstrated by numerical experiments encompassing a wide variety of settings with negligible parameter tuning in particular ngmca is shown to provide robustness to noise and performs well on synthetic mixtures of real nmr spectra

rigorous error bounds for the optimal value in semidefinite programming a wide variety of problems in global optimization combinatorial optimization as well as systems and control theory can be solved by using linear and semidefinite programming sometimes due to the use of floating point arithmetic in combination with ill conditioning and degeneracy erroneous results may be produced the purpose of this article is to show how rigorous error bounds for the optimal value can be computed by carefully postprocessing the output of a linear or semidefinite programming solver it turns out that in many cases the computational costs for postprocessing are small compared to the effort required by the solver numerical results are presented including problems from the sdplib and the netlib lp library these libraries contain many ill conditioned and real life problems

fast nonlinear autocorrelation algorithm for source separation independent component analysis ica and blind source separation bss methods have been used for pattern recognition problems it is well known that ica and bss depend on the statistical properties of original sources or components such as non gaussianity in the paper using a statistical property nonlinear autocorrelation and maximizing the nonlinear autocorrelation of source signals we propose a fast fixed point algorithm for bss we study its convergence property and show that its convergence speed is at least quadratic simulations by the artificial signals and the real world applications verify the efficient implementation of the proposed method

approximation of the stability number of a graph via copositive programming lovasz and schrijver siam j optim 1 1991 pp 166 190 showed how to formulate increasingly tight approximations of the stable set polytope of a graph by solving semidefinite programs sdps of increasing size lift and project method in this paper we present a similar idea we show how the stability number can be computed as the solution of a conic linear program lp over the cone of copositive matrices subsequently we show how to approximate the copositive cone ever more closely via a hierarchy of linear or semidefinite programs of increasing size liftings the latter idea is based on recent work by parrilo structured semidefinite programs and semi algebraic geometry methods in robustness and optimization ph d thesis california institute of technology pasadena ca 2000 in this way we can compute the stability number alpha g of any graph g v e after at most alpha g 2 successive liftings for the lp based approximations one can compare this to the n alpha g 1 bound for the lp based lift and project scheme of lovasz and schrijver our approach therefore requires fewer liftings for families of graphs where alpha g o sqrt n we show that the first sdp based approximation for alpha g in our series of increasingly tight approximations coincides with the vartheta function of schrijver ieee trans inform theory 25 1979 pp 425 429 we further show that the second approximation is tight for complements of triangle free graphs and for odd cycles

on the copositive representation of binary and continuous nonconvex quadratic programs in this paper we model any nonconvex quadratic program having a mix of binary and continuous variables as a linear program over the dual of the cone of copositive matrices this result can be viewed as an extension of earlier separate results which have established the copositive representation of a small collection of np hard problems a simplification which reduces the dimension of the linear conic program and an extension to complementarity constraints are established and computational issues are discussed

space efficient algorithms for computing minimal shortest unique substrings abstract given a string t of length n a substring u t i j of t is called a shortest unique substring sus for an interval s t if a u occurs exactly once in t b u contains the interval s t i e i s t j and c every substring v of t with v u containing s t occurs at least twice in t given a query interval s t 1 n the interval sus problem is to output all the suss for the interval s t in this article we propose a 4 n o n bits data structure answering an interval sus query in output sensitive o o c c time where occ is the number of returned suss additionally we focus on the point sus problem which is the interval sus problem for s t here we propose a log 2 3 1 n o n bits data structure answering a point sus query in the same output sensitive time we also propose space efficient algorithms for computing the minimal unique substrings of t

succinct representations of permutations and functions we investigate the problem of succinctly representing an arbitrary permutation p on 0 n 1 so that p k i can be computed quickly for any i and any positive or negative integer power k a representation taking 1 e nlgn o 1 bits suffices to compute arbitrary powers in constant time for any positive constant e 1 a representation taking the optimal lgn o n bits can be used to compute arbitrary powers in o lgn lglgn time we then consider the more general problem of succinctly representing an arbitrary function f n n so that f k i can be computed quickly for any i and any integer power k we give a representation that takes 1 e nlgn o 1 bits for any positive constant e 1 and computes arbitrary positive powers in constant time it can also be used to compute f k i for any negative integer k in optimal o 1 f k i time we place emphasis on the redundancy or the space beyond the information theoretic lower bound that the data structure uses in order to support operations efficiently a number of lower bounds have recently been shown on the redundancy of data structures these lower bounds confirm the space time optimality of some of our solutions furthermore the redundancy of one of our structures surpasses a recent lower bound by golynski golynski soda 2009 thus demonstrating the limitations of this lower bound

algorithms for nonnegative independent component analysis we consider the task of solving the independent component analysis ica problem x as given observations x with a constraint of nonnegativity of the source random vector s we refer to this as nonnegative independent component analysis and we consider methods for solving this task for independent sources with nonzero probability density function pdf p s down to s 0 it is sufficient to find the orthonormal rotation y wz of prewhitened sources z vx which minimizes the mean squared error of the reconstruction of z from the rectified version y sup of y we suggest some algorithms which perform this both based on a nonlinear principal component analysis pca approach and on a geodesic search method driven by differential geometry considerations we demonstrate the operation of these algorithms on an image separation problem which shows in particular the fast convergence of the rotation and geodesic methods and apply the approach to a musical audio analysis task

suffix arrays a new method for on line string searches a new and conceptually simple data structure called a suffix array for on line string searches is introduced in this paper constructing and querying suffix arrays is reduced to a sort and search paradigm that employs novel algorithms the main advantage of suffix arrays over suffix trees is that in practice they use three to five times less space from a complexity standpoint suffix arrays permit on line string searches of the type is w a substring of a to be answered in time o p log n where p is the length of w and n is the length of a which is competitive with and in some cases slightly better than suffix trees the only drawback is that in those instances where the underlying alphabet is finite and small suffix trees can be constructed in o n time in the worst case versus o n log n time for suffix arrays however an augmented algorithm is given that regardless of the alphabet size constructs suffix arrays in o n expected time albeit with lesser space efficiency it is

some np complete problems in quadratic and nonlinear programming in continuous variable smooth nonconvex nonlinear programming we analyze the complexity of checking whether a a given feasible solution is not a local minimum and b the objective function is not bounded below on the set of feasible solutions we construct a special class of indefinite quadratic programs with simple constraints and integer data and show that checking a or b on this class is np complete as a corollary we show that checking whether a given integer square matrix is not copositive is np complete

global analysis of oja s flow for neural networks a detailed study of oja s learning equation in neural networks is undertaken in this paper not only are such fundamental issues as existence uniqueness and representation of solutions completely resolved but also the convergence issue is resolved it is shown that the solution of oja s equation is exponentially convergent to an equilibrium from any initial value moreover the necessary and sufficient conditions are given on the initial value for the solution to converge to a dominant eigenspace of the associated autocorrelation matrix as a by product this result confirms one of oja s conjectures that the solution converges to the principal eigenspace from almost all initial values some other characteristics of the limiting solution are also revealed these facilitate the determination of the limiting solution in advance using only the initial information two examples are analyzed demonstrating the explicit dependence of the limiting solution on the initial value in another respect it is found that oja s equation is the gradient flow of generalized rayleigh quotients on a stiefel manifold

minimum volume constrained nonnegative matrix factorization enhanced ability of learning parts nonnegative matrix factorization nmf with minimum volume constraint mvc is exploited in this paper our results show that mvc can actually improve the sparseness of the results of nmf this sparseness is l0 norm oriented and can give desirable results even in very weak sparseness situations thereby leading to the significantly enhanced ability of learning parts of nmf the close relation between nmf sparse nmf and the mvc nmf is discussed first then two algorithms are proposed to solve the mvc nmf model one is called quadratic programming mvc nmf qp mvc nmf which is based on quadratic programming and the other is called negative glow mvc nmf ng mvc nmf because it uses multiplicative updates incorporating natural gradient ingeniously the qp mvc nmf algorithm is quite efficient for small scale problems and the ng mvc nmf algorithm is more suitable for large scale problems simulations show the efficiency and validity of the proposed methods in applications of blind source separation and human face images analysis

quadratic assignment problems abstract this paper surveys quadratic assignment problems qap at first several applications of this problem class are described and mathematical formulations of qaps are given then some exact solution methods and good heuristics are outlined their computational behaviour is illustrated by numerical results further recent results on the asymptotic probabilistic behaviour of qaps are outlined

equivariant adaptive source separation source separation consists of recovering a set of independent signals when only mixtures with unknown coefficients are observed this paper introduces a class of adaptive algorithms for source separation that implements an adaptive version of equivariant estimation and is henceforth called equivariant adaptive separation via independence easi the easi algorithms are based on the idea of serial updating this specific form of matrix updates systematically yields algorithms with a simple structure for both real and complex mixtures most importantly the performance of an easi algorithm does not depend on the mixing matrix in particular convergence rates stability conditions and interference rejection levels depend only on the normalized distributions of the source signals closed form expressions of these quantities are given via an asymptotic performance analysis the theme of equivariance is stressed throughout the paper the source separation problem has an underlying multiplicative structure the parameter space forms a matrix multiplicative group we explore the favorable consequences of this fact on implementation performance and optimization of easi algorithms

projection pursuit based method for blind separation of nonnegative sources this paper presents a projection pursuit pp based method for blind separation of nonnegative sources first the available observation matrix is mapped to construct a new mixing model in which the inaccessible source matrix is normalized to be column sum to 1 then the pp method is proposed to solve this new model where the mixing matrix is estimated column by column through tracing the projections to the mapped observations in specified directions which leads to the recovery of the sources the proposed method is much faster than chan s method which has similar assumptions to ours due to the usage of optimal projection it is also more advantageous in separating cross correlated sources than the independence and uncorrelation based methods as it does not employ any statistical information of the sources furthermore the new method does not require the mixing matrix to be nonnegative simulation results demonstrate the superior performance of our method

space time trade offs for finding shortest unique substrings and maximal unique matches abstract given a string x 1 n and a position k 1 n a shortest unique substring of x covering k denoted by s k is a substring x i j of x which satisfies the following conditions i i k j ii i is the only position where there is an occurrence of x i j and iii j i is minimized current best known algorithms for finding s k require n words of working space and o n time let be a given parameter we present the following new results given a k 1 n we can compute s k in o n 2 log n time using x and an additional o n words of working space for every k 1 n we can compute s k in o n 2 log n time using x and an additional o n words and 4 n o n bits of working space we present an o n log c 1 n time randomized algorithm that uses n log c n words in addition to that mentioned above where c 0 is an arbitrary constant in this case the reported string is unique and covers k but with probability at most n o 1 may not be the shortest by choosing 1 our results imply the first sub linear space in addition to the input string solution to these problems we also present the following two results an algorithm that finds s k for every k 1 n using o n log bits of working space in o n log n time where n is the number of distinct symbols in x a 4 n o n bit index that can report s k for any k in o 1 time as a consequence of our techniques we also obtain similar space and time tradeoffs for a related problem of finding maximal unique matches of two strings delcher et al 1999 14

on stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix abstract in applications of signal processing and pattern recognition eigenvectors and eigenvalues of the statistical mean of a random matrix sequence are needed iterative methods are suggested and analyzed in which no sample moments are used convergence is shown by stochastic approximation theory

bayesian estimation of linear mixtures using the normal compositional model application to hyperspectral imagery this paper studies a new bayesian unmixing algorithm for hyperspectral images each pixel of the image is modeled as a linear combination of so called endmembers these endmembers are supposed to be random in order to model uncertainties regarding their knowledge more precisely we model endmembers as gaussian vectors whose means have been determined using an endmember extraction algorithm such as the famous n finder n findr or vertex component analysis vca algorithms this paper proposes to estimate the mixture coefficients referred to as abundances using a bayesian algorithm suitable priors are assigned to the abundances in order to satisfy positivity and additivity constraints whereas conjugate priors are chosen for the remaining parameters a hybrid gibbs sampler is then constructed to generate abundance and variance samples distributed according to the joint posterior of the abundances and noise variances the performance of the proposed methodology is evaluated by comparison with other unmixing algorithms on synthetic and real images

maintaining the integrity of sources in complex learning systems intraference and the correlation preserving transform the correlation preserving transform cpt is introduced to perform bivariate component analysis via decorrelating matrix decompositions while at the same time preserving the integrity of original bivariate sources specifically unlike existing bivariate uncorrelating matrix decomposition techniques cpt is designed to preserve both the order of the data channels within every bivariate source and their mutual correlation properties we introduce the notion of intraference to quantify the effects of interchannel mixing artifacts within recovered bivariate sources and show that the integrity of separated sources is compromised when not accounting for the intrinsic correlations within bivariate sources as is the case with current bivariate matrix decompositions the cpt is based on augmented complex statistics and involves finding the correct conjugate eigenvectors associated with the pseudocovariance matrix making it possible to maintain the physical meaning of the separated sources the benefits of cpt are illustrated in the source separation and clustering scenarios for both synthetic and real world data

doubly nonnegative relaxations for quadratic and polynomial optimization problems with binary and box constraints we propose a doubly nonnegative dnn relaxation for polynomial optimization problems pops with binary and box constraints this work is an extension of the work by kim kojima and toh in 2016 from quadratic optimization problems to pops the dense and sparse dnn relaxations are reduced to a simple conic optimization problem cop to which an accelerated bisection and projection bp algorithm is applied the cop involves a single equality constraint in a matrix variable which is restricted to the intersection of the positive semidefinite cone and a polyhedral cone representing the algebraic properties of binary and box constraints as well as the sparsity structure of the objective polynomial our dnn relaxation serves as a variant of the lasserre moment sos hierarchy of relaxations for binary and box constrained pops when the size of the cones is gradually increased to compute tighter lower bounds for their optimal values a significant part of the paper is devoted to deriving and analyzing a class of polyhedral cones to strengthen the dnn relaxation and the design of efficient computation of the metric projections onto these cones in the accelerated bp algorithm numerical results on various pops are available in ito et al acm trans math softw 45 article 34 2019

0 1 completely positive tensors and multi hypergraphs abstract completely positive graphs have been employed to associate with completely positive matrices for characterizing the intrinsic zero patterns as tensors have been widely recognized as a higher order extension of matrices the multi hypergraph regarded as a generalization of graphs is then introduced to associate with tensors for the study of complete positivity to describe the dependence of the corresponding zero pattern for a special type of completely positive tensors the 0 1 completely positive tensors the completely positive multi hypergraph is defined by characterizing properties of the associated multi hypergraph we provide necessary and sufficient conditions for any 0 1 associated tensor to be 0 1 completely positive furthermore a necessary and sufficient condition for a uniform multi hypergraph to be a completely positive multi hypergraph is proposed as well

copositive relaxation beats lagrangian dual bounds in quadratically and linearly constrained quadratic optimization problems we study nonconvex quadratic minimization problems under possibly nonconvex quadratic and linear constraints characterizing both lagrangian and semi lagrangian dual bounds in terms of conic optimization while the lagrangian dual is equivalent to the sdp relaxation which has been known for quite a while although the presented form incorporating explicitly linear constraints seems to be novel we show that the semi lagrangian dual is equivalent to a natural copositive relaxation and this has apparently not been observed before this way we arrive at conic bounds tighter than the usual lagrangian dual and thus than the sdp bounds any of the known tractable inner approximations of the copositive cone can be used for this tightening but in particular the above mentioned characterization with explicit linear constraints is a natural much cheaper relaxation than the usual zero order approximation by doubly nonnegative dnn matrices and still improves upon the lagrangian dual bounds these app