on g inverses and the nonsingularity of a bordered matrixabc0 abstract we consider the bordered matrix m a b c 0 with the rank additivity condition and g c 1 c 2 c 3 c 4 which is a g inverse of m we give the necessary and sufficient conditions for c 1 to be a g inverse outer inverse reflexive g inverse and moore penrose inverse of a as a special case we give necessary and sufficient conditions for m to be nonsingular and explicit expressions of c i in m 1 moreover we give a method for computing any outer inverse of a matrix

ideal spatial adaptation by wavelet shrinkage summary with ideal spatial adaptation an oracle furnishes information about how best to adapt a spatially variable estimator whether piecewise constant piecewise polynomial variable knot spline or variable bandwidth kernel to the unknown function estimation with the aid of an oracle offers dramatic advantages over traditional linear estimation by nonadaptive kernels however it is a priori unclear whether such performance can be obtained by a procedure relying on the data alone we describe a new principle for spatially adaptive estimation selective wavelet reconstruction we show that variable knot spline fits and piecewise polynomial fits when equipped with an oracle to select the knots are not dramatically more powerful than selective wavelet reconstruction with an oracle we develop a practical spatially adaptive method riskshrink which works by shrinkage of empirical wavelet coefficients riskshrink mimics the performance of an oracle for selective wavelet reconstruction as well as it is possible to do so a new inequality in multivariate normal decision theory which we call the oracle inequality shows that attained performance differs from ideal performance by at most a factor of approximately 2 log n where n is the sample size moreover no estimator can give a better guarantee than this within the class of spatially adaptive procedures riskshrink is essentially optimal relying only on the data it comes within a factor log 2 n of the performance of piecewise polynomial and variableknot spline methods equipped with an oracle in contrast it is unknown how or if piecewise polynomial methods could be made to function this well when denied access to an oracle and forced to rely on data alone

proximal thresholding algorithm for minimization over orthonormal bases the notion of soft thresholding plays a central role in problems from various areas of applied mathematics in which the ideal solution is known to possess a sparse decomposition in some orthonormal basis using convex analytical tools we extend this notion to that of proximal thresholding and investigate its properties providing in particular several characterizations of such thresholders we then propose a versatile convex variational formulation for optimization over orthonormal bases that covers a wide range of problems and we establish the strong convergence of a proximal thresholding algorithm to solve it numerical applications to signal recovery are demonstrated

a cramer rule for minimum norm t least squares s solution of inconsistent linear equations abstract in this paper the minimum norm t least squares s solution of inconsistent linear equations ax y a c m n r y r a is given a determinantal form and reduces to common cramer s rule if a is nonsingular

stability and bifurcations in neural fields with finite propagation speed and general connectivity a stability analysis is presented for neural field equations in the presence of finite propagation speed along axons and for a general class of connectivity kernels and synaptic proper ties sufficient conditions are given for the stability of equilibrium solutions it is shown that the propagation delays play a significant role in nonstationary bifurcations of equilibria whereas the stationary bifurcations depend only on the connectivity kernel in the case of nonstationary bifurca tions bounds are determined on the frequencies of the resulting oscillatory solutions a perturbative scheme is used to calculate the types of bifurcations leading to spatial patterns oscillations and traveling waves for high propagation speeds a simple method is derived that allows the determi nation of the bifurcation type by visual inspection of the fourier transforms of the kernel and its first moment results are numerically illustrated on a class of neurologically plausible systems with combinations of gaussian excitatory and inhibitory connections

a primal dual proximal algorithm for sparse template based adaptive filtering application to seismic multiple removal unveiling meaningful geophysical information from seismic data requires to deal with both random and structured noises as their amplitude may be greater than signals of interest primaries additional prior information is especially important in performing efficient signal separation we address here the problem of multiple reflections caused by wave field bouncing between layers since only approximate models of these phenomena are available we propose a flexible framework for time varying adaptive filtering of seismic signals using sparse representations based on inaccurate templates we recast the joint estimation of adaptive filters and primaries in a new convex variational formulation this approach allows us to incorporate plausible knowledge about noise statistics data sparsity and slow filter variation in parsimony promoting wavelet frames the designed primal dual algorithm solves a constrained minimization problem that alleviates standard regularization issues in finding hyperparameters the approach demonstrates significantly good performance in low signal to noise ratio conditions both for simulated and real field data

nonnegative idempotent matrices and the minus partial order abstract we describe the structure of nonnegative matrices dominated by a nonnegative idempotent matrix under the minus order

linearized bregman iterations for compressed sensing finding a solution of a linear equation au f with various minimization properties arises from many applications one such application is compressed sensing where an efficient and robust to noise algorithm to find a minimal l 1 norm solution is needed this means that the algorithm should be tailored for large scale and completely dense matrices a while au and a t u can be computed by fast transforms and the solution we seek is sparse recently a simple and fast algorithm based on linearized bregman iteration was proposed in 28 32 for this purpose this paper is to analyze the convergence of linearized bregman iterations and the minimization properties of their limit based on our analysis here we derive also a new algorithm that is proven to be convergent with a rate furthermore the new algorithm is simple and fast in approximating a minimal l 1 norm solution of au f as shown by numerical simulations hence it can be used as another choice of an efficient tool in compressed sensing

multiple bumps in a neuronal model of working memory we study a partial integro differential equation defined on a spatially extended domain that arises from the modeling of working or short term memory in a neuronal network the equation is capable of supporting spatially localized regions of high activity which can be switched on and off by transient external stimuli we analyze the effects of coupling between units in the network showing that if the connection strengths decay monotonically with distance then no more thanon e regionof high activity canpersist whereas if they decay inanoscillatory fashion then multiple regions can persist

perspective functions proximal calculus and applications in high dimensional statistics abstract perspective functions arise explicitly or implicitly in various forms in applied mathematics and in statistical data analysis to date no systematic strategy is available to solve the associated typically nonsmooth optimization problems in this paper we fill this gap by showing that proximal methods provide an efficient framework to model and solve problems involving perspective functions we study the construction of the proximity operator of a perspective function under general assumptions and present important instances in which the proximity operator can be computed explicitly or via straightforward numerical operations these results constitute central building blocks in the design of proximal optimization algorithms we showcase the versatility of the framework by designing novel proximal algorithms for state of the art regression and variable selection schemes in high dimensional statistics

hebbian learning of recurrent connections a geometrical perspective we show how a hopfield network with modifiable recurrent connections undergoing slow hebbian learning can extract the underlying geometry of an input space first we use a slow and fast analysis to derive an averaged system whose dynamics derives from an energy function and therefore always converges to equilibrium points the equilibria reflect the correlation structure of the inputs a global object extracted through local recurrent interactions only second we use numerical methods to illustrate how learning extracts the hidden geometrical structure of the inputs indeed multidimensional scaling methods make it possible to project the final connectivity matrix onto a euclidean distance matrix in a high dimensional space with the neurons labeled by spatial position within this space the resulting network structure turns out to be roughly convolutional the residual of the projection defines the nonconvolutional part of the connectivity which is minimized in the process finally we show how restricting the dimension of the space where the neurons live gives rise to patterns similar to cortical maps we motivate this using an energy efficiency argument based on wire length minimization finally we show how this approach leads to the emergence of ocular dominance or orientation columns in primary visual cortex via the self organization of recurrent rather than feedforward connections in addition we establish that the nonconvolutional or long range connectivity is patchy and is co aligned in the case of orientation learning

the nonunique parallel sum abstract this paper is a sequel to earlier study of the authors on the nonunique shorted matrix under the failure of regularity conditions when a and b are real symmetric nonnegative definite n n d matrices of the same order anderson and trapp present an alternative definition of the parallel sum of anderson and duffin which uses a shorted version of the matrix here a being real symmetric n n d the regularity conditions are trivially true in the more general case however a similar approach based on the nonunique shorted matrix leads to a nonunique parallel sum the extent to which the nonunique parallel sum retains the properties of parallel sum of anderson and duffin is examined an interesting statistical interpretation of the parallel sum operation is provided in an appendix

shorted matrices an extended concept and some applications the shorting of an operator hitherto considered by krein 11 and by anderson and trapp 3 only for positive operators is extended to rectangular matrices and square matrices not necessarily hermitian nonnegative definite some applications of the shorted matrix in mathematical statistics are discussed

inconsistent signal feasibility problems least squares solutions in a product space presents parallel projection methods to find least squares solutions to inconsistent convex set theoretic signal synthesis problems the problem of finding a signal that minimizes a weighted average of the squares of the distances to constraint sets is reformulated in a product space where it is equivalent to that of finding a point that lies in a particular subspace and at minimum distance from the cartesian product of the original sets a solution is obtained in the product space via methods of alternating projections which naturally lead to methods of parallel projections in the original space the convergence properties of the proposed methods are analyzed and signal synthesis applications are demonstrated

the use of noise properties in set theoretic estimation in most digital signal processing problems the goal is to estimate an object from noise corrupted observations of a physical system the authors describe how a wide range of probabilistic information pertaining to the noise process can be used in a general set theoretic estimation framework the basic principle is to constrain the sample statistics of the estimation residual to be consistent with those probabilistic properties of the noise which are available and to construct sets accordingly in the solution space adding these sets to the collection of sets describing the solution will yield a smaller feasibility set and hence more reliable estimates pieces of information relative to quantities such as range moments absolute moments and second and higher order probabilistic attributes are considered and properties of the corresponding sets are established simulations are provided to illustrate the theoretical developments

more on maximal and minimal ranks of schur complements with applications the maximal and minimal ranks of the schur complement d cgb are determined with respect to generalized inverse g of a matrix a where g is taken respectively as least squares g inverse a 1 3 minimum norm g inverse a 1 4 as well as a 1 2 3 a 1 2 4 and a 1 3 4 various consequences are also presented

a pseudo heuristic parameter selection rule for l1 regularized minimization problems abstract this paper considers the regularization parameter determination of l 1 regularized minimization problem we solve the l 1 regularized problem using iterative reweighted least squares irls which involves solving a linear system whose coefficient matrix has the form m 1 n 0 1 the aim of this paper is to find an efficient and computationally inexpensive algorithm to both choose the regularization parameter and solve the l 1 regularized problem in order to achieve this we propose an irls algorithm with adaptive regularization parameter selection based on a heuristic parameter determination rule de boor s parameter selection criterion compared with some of the state of the art algorithms and parameter selection rules the numerical experiments show the efficiency and robustness of the proposed method

stability analysis of dynamical neural networks in this paper we use the matrix measure technique to study the stability of dynamical neural networks testable conditions for global exponential stability of nonlinear dynamical systems and dynamical neural networks are given it shows how a few well known results can be unified and generalized in a straightforward way local exponential stability of a class of dynamical neural networks is also studied we point out that the local exponential stability of any equilibrium point of dynamical neural networks is equivalent to the stability of the linearized system around that equilibrium point from this some well known and new sufficient conditions for local exponential stability of neural networks are obtained

nonconvex nonsmooth optimization via convex nonconvex majorization minimization the class of majorization minimization algorithms is based on the principle of successively minimizing upper bounds of the objective function each upper bound or surrogate function is locally tight at the current estimate and each minimization step decreases the value of the objective function we present a majorization minimization approach based on a novel convex nonconvex upper bounding strategy for the solution of a certain class of nonconvex nonsmooth optimization problems we propose an efficient algorithm for minimizing the convex surrogate function based on the alternating direction method of multipliers a preliminary convergence analysis for the proposed approach is provided numerical experiments show the effectiveness of the proposed method for the solution of nonconvex nonsmooth minimization problems

nonlinear wavelet image processing variational problems compression and noise removal through wavelet shrinkage this paper examines the relationship between wavelet based image processing algorithms and variational problems algorithms are derived as exact or approximate minimizers of variational problems in particular we show that wavelet shrinkage can be considered the exact minimizer of the following problem given an image f defined on a square i minimize over all g in the besov space b11 l1 i the functional f g l2 i 2 g b11 l1 i we use the theory of nonlinear wavelet image compression in l2 i to derive accurate error bounds for noise removal through wavelet shrinkage applied to images corrupted with i i d mean zero gaussian noise a new signal to noise ratio snr which we claim more accurately reflects the visual perception of noise in images arises in this derivation we present extensive computations that support the hypothesis that near optimal shrinkage parameters can be derived if one knows or can estimate only two parameters about an image f the largest for which f bq lq i 1 q 2 1 2 and the norm f bq lq i both theoretical and experimental results indicate that our choice of shrinkage parameters yields uniformly better results than donoho and johnstone s visushrink procedure an example suggests however that donoho and johnstone s 1994 1995 1996 sureshrink method which uses a different shrinkage parameter for each dyadic level achieves a lower error than our procedure

schur complements and statistics abstract in this paper we discuss various properties of matrices of the type s h ge 1 f which we call the schur complement of e in a e f g h the matrix e is assumed to be nonsingular when e is singular or rectangular we consider the generalized schur complements s h ge f where e is a generalized inverse of e a comprehensive account of results pertaining to the determinant the rank the inverse and generalized inverses of partitioned matrices and the inertia of a matrix is given both for schur complements and for generalized schur complements we survey the known results in a historical perspective and obtain several extensions numerous applications in numerical analysis and statistics are included the paper ends with an exhaustive bibliography of books and articles related to schur complements

proximity for sums of composite functions we propose an algorithm for computing the proximity operator of a sum of composite convex functions in hilbert spaces and investigate its asymptotic behavior applications to best approximation and image recovery are described

on the equivalence of soft wavelet shrinkage total variation diffusion total variation regularization and sides soft wavelet shrinkage total variation tv diffusion tv regularization and a dynamical system called sides are four useful techniques for discontinuity preserving denoising of signals and images in this paper we investigate under which circumstances these methods are equivalent in the one dimensional case first we prove that haar wavelet shrinkage on a single scale is equivalent to a single step of space discrete tv diffusion or regularization of two pixel pairs in the translationally invariant case we show that applying cycle spinning to haar wavelet shrinkage on a single scale can be regarded as an absolutely stable explicit discretization of tv diffusion we prove that space discrete tv diffusion and tv regularization are identical and that they are also equivalent to the sides system when a specific force function is chosen afterwards we show that wavelet shrinkage on multiple scales can be regarded as a single step diffusion filtering or regularization of the laplacian pyramid of the signal we analyze possibilities to avoid gibbs like artifacts for multiscale haar wavelet shrinkage by scaling the thresholds finally we present experiments where hybrid methods are designed that combine the advantages of wavelets and pde variational approaches these methods are based on iterated shift invariant wavelet shrinkage at multiple scales with scaled thresholds

a predual proximal point algorithm solving a non negative basis pursuit denoising model this paper develops an implementation of a predual proximal point algorithm pppa solving a non negative basis pursuit denoising model the model imposes a constraint on the l 2 norm of the residual instead of penalizing it the pppa solves the predual of the problem with a proximal point algorithm ppa moreover the minimization that needs to be performed at each iteration of ppa is solved with a dual method we can prove that these dual variables converge to a solution of the initial problem our analysis proves that we turn a constrained non differentiable convex problem into a short sequence of nice concave maximization problems by nice we mean that the functions which are maximized are differentiable and their gradient is lipschitz the algorithm is easy to implement easier to tune and more general than the algorithms found in the literature in particular it can be applied to the basis pursuit denoising bpdn and the non negative basis pursuit denoising nnbpdn and it does not make any assumption on the dictionary we prove its convergence to the set of solutions of the model and provide some convergence rates experiments on image approximation show that the performances of the pppa are at the current state of the art for the bpdn

should penalized least squares regression be interpreted as maximum a posteriori estimation penalized least squares regression is often used for signal denoising and inverse problems and is commonly interpreted in a bayesian framework as a maximum a posteriori map estimator the penalty function being the negative logarithm of the prior for example the widely used quadratic program with an l1 penalty associated to the lasso basis pursuit denoising is very often considered as map estimation under a laplacian prior in the context of additive white gaussian noise awgn reduction this paper highlights the fact that while this is one possible bayesian interpretation there can be other equally acceptable bayesian interpretations therefore solving a penalized least squares regression problem with penalty x need not be interpreted as assuming a prior c exp x and using the map estimator in particular it is shown that for any prior px the minimum mean square error mmse estimator is the solution of a penalized least square problem with some penalty x which can be interpreted as the map estimator with the prior c exp x vice versa for certain penalties x the solution of the penalized least squares problem is indeed the mmse estimator with a certain prior px in general dpx x c exp x dx

the feasible solution in signal restoration the feasible solution to the signal restoration problem is defined as the one which satisfies all constraints which can be imposed on the true solution a very important set of constraints can be obtained by examining the statistics of the noise these and other constraints can be described as closed convex sets thus projection onto closed convex sets is the numerical method used to obtain a feasible solution examples of this method demonstrate its usefulness in one and two dimensional signal restoration the limitations of the method are discussed

a projection based algorithm for consistent and inconsistent constraints signal synthesis and reconstruction is considered when the signal is to be determined by n constraint sets ci the solution sought is required to minimize a weighted quadratic cost functional hat j emphasis is on cases in which the intersection of the sets ci is empty our proposed procedure employs a suitably weighted simultaneous projection iteration method it is shown that the iterates generated by the algorithm converge weakly to a global minimizer of hat j provided the set of fixed points of the algorithm is nonempty if the problem is consistent c o bigcap c i neq emptyset weak convergence is to an element in co however it is indicated that large classes of inconsistent problems which could not be treated by existing methods admit a solution as well

a semismooth newton method with multidimensional filter globalization for l_1 optimization due to their property of enhancing the sparsity of solutions l 1 regularized optimization problems have developed into a highly dynamic research area with a wide range of applications we present a class of methods for l 1 regularized optimization problems that are based on a combination of semismooth newton steps a filter globalization and shrinkage thresholding steps a multidimensional filter framework is used to control the acceptance and to evaluate the quality of the semismooth newton steps if the current newton iterate is rejected a shrinkage thresholding based step with quasi armijo stepsize rule is used instead global convergence and transition to local q superlinear convergence for both convex and nonconvex objective functions are established we present numerical results and comparisons with several state of the art methods that show the efficiency and competitiveness of the proposed method

parallel proximal algorithm for image restoration using hybrid regularization regularization approaches have demonstrated their effectiveness for solving ill posed problems however in the context of variational restoration methods a challenging question remains namely how to find a good regularizer while total variation introduces staircase effects wavelet domain regularization brings other artefacts e g ringing however a tradeoff can be made by introducing a hybrid regularization including several terms not necessarily acting in the same domain e g spatial and wavelet transform domains while this approach was shown to provide good results for solving deconvolution problems in the presence of additive gaussian noise an important issue is to efficiently deal with this hybrid regularization for more general noise models to solve this problem we adopt a convex optimization framework where the criterion to be minimized is split in the sum of more than two terms for spatial domain regularization isotropic or anisotropic total variation definitions using various gradient filters are considered an accelerated version of the parallel proximal algorithm is proposed to perform the minimization some difficulties in the computation of the proximity operators involved in this algorithm are also addressed in this paper numerical experiments performed in the context of poisson data recovery show the good behavior of the algorithm as well as promising results concerning the use of hybrid regularization techniques

sparse signal estimation by maximally sparse convex optimization this paper addresses the problem of sparsity penalized least squares for applications in sparse signal processing e g sparse deconvolution this paper aims to induce sparsity more strongly than l1 norm regularization while avoiding non convex optimization for this purpose this paper describes the design and use of non convex penalty functions regularizers constrained so as to ensure the convexity of the total cost function f to be minimized the method is based on parametric penalty functions the parameters of which are constrained to ensure convexity of f it is shown that optimal parameters can be obtained by semidefinite programming sdp this maximally sparse convex msc approach yields maximally non convex sparsity inducing penalty functions constrained such that the total cost function f is convex it is demonstrated that iterative msc imsc can yield solutions substantially more sparse than the standard convex sparsity inducing approach i e l1 norm minimization

modeling electrocortical activity through improved local approximations of integral neural field equations neural field models of firing rate activity typically take the form of integral equations with space dependent axonal delays under natural assumptions on the synaptic connectivity we show how one can derive an equivalent partial differential equation pde model that properly treats the axonal delay terms of the integral formulation our analysis avoids the so called long wavelength approximation that has previously been used to formulate pde models for neural activity in two spatial dimensions direct numerical simulations of this pde model show instabilities of the homogeneous steady state that are in full agreement with a turing instability analysis of the original integral model we discuss the benefits of such a local model and its usefulness in modeling electrocortical activity in particular we are able to treat patchy connections whereby a homogeneous and isotropic system is modulated in a spatially periodic fashion in this case the emergence of a lattice directed traveling wave predicted by a linear instability analysis is confirmed by the numerical simulation of an appropriate set of coupled pdes

filtered wavelet thresholding methods when working with nonlinear filtering algorithms for image denoising problems there are two crucial aspects namely the choice of the thresholding parameter and the use of a proper filter function both greatly influence the quality of the resulting denoised image in this paper we propose two new filters which are a piecewise quadratic and an exponential function of respectively arid we show how they can be successfully used instead of the classical donoho and johnstone s soft thresholding filter we exploit the increased regularity and flexibility of the new filters to improve the quality of the final results moreover we prove that our filtered approximation is a near minimizer of the functional which has to be minimized to solve the denoising problem we also show that the quadratic filter due to its shape yields good results if we choose as the donoho and johnstone universal threshold while the exponential one is more suitable if we use the recently proposed h curve criterion encouraging results in extensive numerical experiments on several test images confirm the effectiveness of our proposal

applications of splitting algorithm to decomposition in convex programming and variational inequalities recently han and lou proposed a highly parallelizable decomposition algorithm for minimizing a strongly convex cost over the intersection of closed convex sets it is shown that their algorithm is in fact a special case of a splitting algorithm analyzed by gabay for finding a zero of the sum of two maximal monotone operators gabay s convergence analysis for the splitting algorithm is sharpened and new applications of this algorithm to variational inequalities convex programming and the solution of linear complementarily problems are proposed for convex programs with a certain separable structure a multiplier method that is closely related to the alternating direction method of multipliers of gabay mercier and of glowinski marrocco but which uses both ordinary and augmented lagrangians is obtained

simultaneous low pass filtering and total variation denoising this paper seeks to combine linear time invariant lti filtering and sparsity based denoising in a principled way in order to effectively filter denoise a wider class of signals lti filtering is most suitable for signals restricted to a known frequency band while sparsity based denoising is suitable for signals admitting a sparse representation with respect to a known transform however some signals cannot be accurately categorized as either band limited or sparse this paper addresses the problem of filtering noisy data for the particular case where the underlying signal comprises a low frequency component and a sparse or sparse derivative component a convex optimization approach is presented and two algorithms derived one based on majorization minimization mm and the other based on the alternating direction method of multipliers admm it is shown that a particular choice of discrete time filter namely zero phase noncausal recursive filters for finite length data formulated in terms of banded matrices makes the algorithms effective and computationally efficient the efficiency stems from the use of fast algorithms for solving banded systems of linear equations the method is illustrated using data from a physiological measurement technique i e near infrared spectroscopic time series imaging that in many cases yields data that is well approximated as the sum of low frequency sparse or sparse derivative and noise components

nonlinear wavelet shrinkage with bayes rules and bayes factors abstract wavelet shrinkage the method proposed by the seminal work of donoho and johnstone is a disarmingly simple and efficient way of denoising data shrinking wavelet coefficients was proposed from several optimality criteria in this article a wavelet shrinkage by coherent bayesian inference in the wavelet domain is proposed the methods are tested on standard donoho johnstone test functions

fixed point continuation for ell_1 minimization methodology and convergence we present a framework for solving the large scale ell 1 regularized convex minimization problem min x 1 mu f x our approach is based on two powerful algorithmic ideas operator splitting and continuation operator splitting results in a fixed point algorithm for any given scalar mu continuation refers to approximately following the path traced by the optimal value of x as mu increases in this paper we study the structure of optimal solution sets prove finite convergence for important quantities and establish q linear convergence rates for the fixed point algorithm applied to problems with f x convex but not necessarily strictly convex the continuation framework motivated by our convergence results is demonstrated to facilitate the construction of practical algorithms

how to express a parallel sum of k matrices abstract we present an alternative expression for the parallel sum of k hermitian nonnegative definite matrices by using the moore penrose inverse of a block matrix

on parallel summability of matrices abstract it is shown that the generalized inverses characterize the parallel sum the almost positive definite a p d matrices introduced by duffin and morley 2 are of two types whose intersection is the class of quasi positive definite matrices mitra and puri 7 the a p d matrices of any one type form a saturated subclass of pairwise parallel summable a p d matrices

stochastic neural field theory and the system size expansion we analyze a master equation formulation of stochastic neurodynamics for a network of synaptically coupled homogeneous neuronal populations each consisting of n identical neurons the state of the network is specified by the fraction of active or spiking neurons in each population and transition rates are chosen so that in the thermodynamic or deterministic limit n rightarrow infty we recover standard activity based or voltage based rate models we derive the lowest order corrections to these rate equations for large but finite n using two different approximation schemes one based on the van kampen system size expansion and the other based on path integral methods both methods yield the same series expansion of the moment equations which at mathcal o 1 n can be truncated to form a closed system of equations for the first and second order moments taking a continuum limit of the moment equations while keeping the system size n fixed generates a system of integrodifferential equations for the

a fast algorithm for sparse reconstruction based on shrinkage subspace optimization and continuation we propose a fast algorithm for solving the ell 1 regularized minimization problem min x in mathbb r n mu x 1 ax b 2 2 for recovering sparse solutions to an undetermined system of linear equations ax b the algorithm is divided into two stages that are performed repeatedly in the first stage a first order iterative shrinkage method yields an estimate of the subset of components of x likely to be nonzero in an optimal solution restricting the decision variables x to this subset and fixing their signs at their current values reduces the ell 1 norm x 1 to a linear function of x the resulting subspace problem which involves the minimization of a smaller and smooth quadratic function is solved in the second phase our code fpc as embeds this basic two stage algorithm in a continuation homotopy approach by assigning a decreasing sequence of values to mu this code exhibits state of the art performance in terms of both its speed and its ability to recover sparse signals

the moore penrose inverses of m n block matrices and their applications abstract this article gives the expressions for the moore penrose inverses of m n block matrices when they satisfy rank additivity conditions and presents some of their special cases and applications

moreau s decomposition in banach spaces moreau s decomposition is a powerful nonlinear hilbertian analysis tool that has been used in various areas of optimization and applied mathematics in this paper it is extended to reflexive banach spaces and in the context of generalized proximity measures this extension unifies and significantly improves upon existing results

the minus partial order and the shorted matrix the minus partial order was defined by hartwig 6 weakening the conditions of the star partial order of drazin 5 several new properties of the minus partial order are established the minus partial order is used to redefine the shorted matrix 15 17 and to define the infimum a b and the supremum a b of a pair a b of matrices of the same order the definition of the shorted matrix given here is similar to the krein anderson trapp definition of the shorted positive operator

matrix partial orders through generalized inverses unified theory the unified theory presented here covers as special cases the star order of drazin the minus order of hartwig and nambooripad the sharp order and other partial orders introduced by the author

adapting to unknown smoothness via wavelet shrinkage abstract we attempt to recover a function of unknown smoothness from noisy sampled data we introduce a procedure sureshrink that suppresses noise by thresholding the empirical wavelet coefficients the thresholding is adaptive a threshold level is assigned to each dyadic resolution level by the principle of minimizing the stein unbiased estimate of risk sure for threshold estimates the computational effort of the overall procedure is order n log n as a function of the sample size n sureshrink is smoothness adaptive if the unknown function contains jumps then the reconstruction essentially does also if the unknown function has a smooth piece then the reconstruction is essentially as smooth as the mother wavelet will allow the procedure is in a sense optimally smoothness adaptive it is near minimax simultaneously over a whole interval of the besov scale the size of this interval depends on the choice of mother wavelet we know from a previous paper by the authors that traditional smoot

group sparse signal denoising non convex regularization convex optimization convex optimization with sparsity promoting convex regularization is a standard approach for estimating sparse signals in noise in order to promote sparsity more strongly than convex regularization it is also standard practice to employ non convex optimization in this paper we take a third approach we utilize a non convex regularization term chosen such that the total cost function consisting of data consistency and regularization terms is convex therefore sparsity is more strongly promoted than in the standard convex formulation but without sacrificing the attractive aspects of convex optimization unique minimum robust algorithms etc we use this idea to improve the recently developed overlapping group shrinkage ogs algorithm for the denoising of group sparse signals the algorithm is applied to the problem of speech enhancement with favorable results in terms of both snr and perceptual quality

on the properties of nonlinear nonlocal operators arising in neural field models abstract we study the existence and continuous dependence of stationary solutions of the one population wilson cowan model on the steepness of the firing rate functions we investigate the properties of the nonlinear nonlocal operators which arise when formulating the stationary one population wilson cowan model as a fixed point problem the theory is used to study the existence and continuous dependence of localized stationary solutions of this model on the steepness of the firing rate functions the present work generalizes and complements previously obtained results as we relax on the assumptions that the firing rate functions are given by smoothed heaviside functions

convergence of the linearized bregman iteration for l1 norm minimization one of the key steps in compressed sensing is to solve the basis pursuit problem min u n u au f bregman iteration was very successfully used to solve this problem in 40 also a simple and fast iterative algorithm based on linearized bregman iteration was proposed in 40 which is described in detail with numerical simulations in 35 a convergence analysis of the smoothed version of this algorithm was given in 11 the purpose of this paper is to prove that the linearized bregman iteration proposed in 40 for the basis pursuit problem indeed converges

on some matrix equalities for generalized inverses with applications abstract necessary and sufficient conditions are derived for the matrix equality a pn q to hold where a and n are generalized inverses of matrices some consequences and applications are also given in particular necessary and sufficient conditions are derived for the additive decompositions c a b and a a 1 a k to hold

regression analysis of locality preserving projections via sparse penalty recent studies have shown that linear subspace algorithms such as principal component analysis linear discriminant analysis and locality preserving projections have attracted tremendous attention in many fields of information processing however the projection results obtained by these algorithms are linear combination of the original features which is difficult to be interpreted psychologically and physiologically motivated by compressive sensing theory we formulate the generalized eigenvalue problem under cs framework which then allows us to apply a sparsity penalty and minimization procedure to locality preserving projections the proposed algorithm is called sparse locality preserving projections which performs locality preserving projections in the lasso regression framework that dimensionality reduction feature selection and classification are merged into one analysis the method is also extended to its regularized form to improve its generalization the proposed algorithm is a combination of locality preserving with sparse penalty additionally the algorithm can be performed in either supervised or unsupervised tasks experimental results on toy and real data sets show that our methods are effective and demonstrate much higher performance

on the wedderburn guttman theorem abstract let a be a u by v matrix of rank a and let m and n be u by g and v by g matrices respectively such that m an is nonsingular then rank a n m an 1m a a g where g rank an m an 1m a rank m an this is called wedderburn guttman theorem what happens if m an is rectangular and or singular in this paper we investigate conditions under which the regular inverse m an 1 can be replaced by a g inverse m an of some kind thereby extending the wedderburn guttman theorem the resultant conditions look similar to those arising in seemingly unrelated contexts namely cochran s and related theorems on distributions of quadratic forms involving a normal random vector

variable metric quasi fejer monotonicity the notion of quasi fejer monotonicity has proven to be an efficient tool to simplify and unify the convergence analysis of various algorithms arising in applied nonlinear analysis in this paper we extend this notion in the context of variable metric algorithms whereby the underlying norm is allowed to vary at each iteration applications to convex optimization and inverse problems are demonstrated

invariance properties of a triple matrix product involving generalized inverses abstract given complex valued matrices a b and c of appropriate dimensions this paper investigates certain invariance properties of the product axc with respect to the choice of x where x is a generalized inverse of b different types of generalized inverses are taken into account the purpose of the paper is three fold first to review known results scattered in the literature second to demonstrate the connection between invariance properties and the concept of extremal ranks of matrices and third to add new results related to the topic

absolute stability and synchronization in neural field models with transmission delays abstract neural fields model macroscopic parts of the cortex which involve several populations of neurons we consider a class of neural field models which are represented by integro differential equations with transmission time delays which are space dependent the considered domains underlying the systems can be bounded or unbounded a new approach called sequential contracting instead of the conventional lyapunov functional technique is employed to investigate the global dynamics of such systems sufficient conditions for the absolute stability and synchronization of the systems are established several numerical examples are presented to demonstrate the theoretical results

wavelet shrinkage asymptopia much recent effort has sought asymptotically minimax methods for recovering infinite dimensional objects curves densities spectral densities images from noisy data a now rich and complex body of work develops nearly or exactly minimax estimators for an array of interesting problems unfortunately the results have rarely moved into practice for a variety of reasons among them being similarity to known methods computational intractability and lack of spatial adaptivity we discuss a method for curve estimation based on n noisy data translate the empirical wavelet coefficients towards the origin by an amount 2 log n n the proposal differs from those in current use is computationally practical and is spatially adaptive it thus avoids several of the previous objections further the method is nearly minimax both for a wide variety of loss functions pointwise error global error measured in l p norms pointwise and global error in estimation of derivatives and for a wide range of smoothness classes including standard holder and sobolev classes and bounded variation this is a much broader near optimality than anything previously proposed we draw loose parallels with near optimality in robustness and also with the broad near eigenfunction properties of wavelets themselves finally the theory underlying the method is interesting as it exploits a correspondence between statistical questions and questions of optimal recovery and information based complexity

the least squares g inverse for the sum of multiple matrices in this paper we give the necessary and sufficient conditions for the set identities a 1 a 2 a k 1 3 a 1 1 3 a 2 1 3 a k 1 3 and a 1 a 2 a k 1 4 a 1 1 4 a 2 1 4 a k 1 4 which concern the least squares g inverse 1 3 inverse and the minimum norm g inverse 1 4 inverse of sum of k matrices

spatially structured activity in synaptically coupled neuronal networks i traveling fronts and pulses we consider traveling front and pulse solutions to a system of integro differential equations used to describe the activity of synaptically coupled neuronal networks in a single spatial dimension our first goal is to establish a series of direct links between the abstract nature of the equations and their interpretation in terms of experimental findings in the cortex and other brain regions this is accomplished first by presenting a biophysically motivated derivation of the system and then by establishing a framework for comparison between numerical and experimental measures of activity propagation speed our second goal is to establish the existence of traveling pulse solutions using more rigorous methods two techniques are presented the first a shooting argument reduces the problem from finding a specific solution to an integro differential equation system to finding any solution to an ode system the second a singular perturbation argument provides a construction of traveling pulse solutions un

upper and lower bounds for ranks of matrix expressions using generalized inverses abstract the maximal and minimal ranks of the matrix expression a 1 b 1 x 1 c 1 b 2 x 2 c 2 with respect to x 1 and x 2 are presented as applications the maximal and minimal ranks of a 1 b 1 xc 1 subject to a consistent matrix equation b 2 xc 2 a 2 are also determined in addition the maximal and minimal ranks of the schur complement d ca b with respect to the generalized inverse a of a and their various consequences are also considered

nonnegative group monotone matrices and the minus partial order bapat et al previously described a class of nonnegative matrices dominated by a nonnegative idempotent matrix under the minus partial order in this paper we improve upon that description by first presenting a more general result that gives the precise structure of nonnegative matrices dominated by a group monotone matrix under the minus partial order as a special case we derive the complete class of nonnegative matrices dominated by a nonnegative idempotent matrix that includes the class obtained by bapat et al

non convex sparse regularisation abstract we study the regularising properties of tikhonov regularisation on the sequence space l 2 with weighted non quadratic penalty term acting separately on the coefficients of a given sequence we derive sufficient conditions for the penalty term that guarantee the well posedness of the method and investigate to which extent the same conditions are also necessary a particular interest of this paper is the application to the solution of operator equations with sparsity constraints assuming a linear growth of the penalty term at zero we prove the sparsity of all regularised solutions moreover we derive a linear convergence rate under the assumptions of even faster growth at zero and a certain injectivity of the operator to be inverted these results in particular cover non convex l p regularisation with 0 p 1

enhanced sparsity by non separable regularization this paper develops a convex approach for sparse one dimensional deconvolution that improves upon l1 norm regularization the standard convex approach we propose a sparsity inducing non separable non convex bivariate penalty function for this purpose it is designed to enable the convex formulation of ill conditioned linear inverse problems with quadratic data fidelity terms the new penalty overcomes limitations of separable regularization we show how the penalty parameters should be set to ensure that the objective function is convex and provide an explicit condition to verify the optimality of a prospective solution we present an algorithm an instance of forward backward splitting for sparse deconvolution using the new penalty

some further remarks on the singular linear model abstract inspired by a recent article of searle 1994 we consider some specific features of estimation of x in the general linear model y x v where the model matrix x need not have full column rank and the dispersion matrix v can be singular particular attention is paid to the problems related to the invariance of some matrix expressions with respect to the choice of generalized inverses these invariance properties yield interesting matrix algebra while characterizing the best linear unbiased estimator of x

on group inverses and the sharp order abstract in this work the group inverse of a matrix is used to define the order on square matrices of index 1 the order is similar to the order of drazin 2 and the minus order of hartwig 6 10 and nambooripad 17 the order and the order are compared and contrasted many conditions are given which assure the equivalence of the various partial orders studied

sparse identification of posynomial models posynomials are nonnegative combinations of monomials with possibly fractional and both positive and negative exponents posynomial models are widely used in various engineering design endeavors such as circuits aerospace and structural design mainly due to the fact that design problems cast in terms of posynomial objectives and constraints can be solved efficiently by means of a convex optimization technique known as geometric programming gp however while quite a vast literature exists on gp based design very few contributions can yet be found on the problem of identifying posynomial models from experimental data posynomial identification amounts to determining not only the coefficients of the combination but also the exponents in the monomials which renders the identification problem hard in this paper we propose an approach to the identification of multivariate posynomial models based on the expansion on a given large scale basis of monomials the model is then identified by seeking coefficients of the combination that minimize a mixed objective composed by a term representing the fitting error and a term inducing sparsity in the representation which results in a problem formulation of the square root lasso type with nonnegativity constraints on the variables we propose to solve the problem via a sequential coordinate minimization scheme which is suitable for large scale implementations a numerical example is finally presented dealing with the identification of a posynomial model for a naca 4412 airfoil

restoration of poissonian images using alternating direction optimization much research has been devoted to the problem of restoring poissonian images namely for medical and astronomical applications however the restoration of these images using state of the art regularizers such as those based upon multiscale representations or total variation is still an active research area since the associated optimization problems are quite challenging in this paper we propose an approach to deconvolving poissonian images which is based upon an alternating direction optimization method the standard regularization or maximum a posteriori map restoration criterion which combines the poisson log likelihood with a nonsmooth convex regularizer log prior leads to hard optimization problems the log likelihood is nonquadratic and nonseparable the regularizer is nonsmooth and there is a nonnegativity constraint using standard convex analysis tools we present sufficient conditions for existence and uniqueness of solutions of these optimization problems for several types of regularizers total variation frame based analysis and frame based synthesis we attack these problems with an instance of the alternating direction method of multipliers admm which belongs to the family of augmented lagrangian algorithms we study sufficient conditions for convergence and show that these are satisfied either under total variation or frame based analysis and synthesis regularization the resulting algorithms are shown to outperform alternative state of the art methods both in terms of speed and restoration accuracy

a proximal iteration for deconvolving poisson noisy images using sparse representations we propose an image deconvolution algorithm when the data is contaminated by poisson noise the image to restore is assumed to be sparsely represented in a dictionary of waveforms such as the wavelet or curvelet transforms our key contributions are as follows first we handle the poisson noise properly by using the anscombe variance stabilizing transform leading to a nonlinear degradation equation with additive gaussian noise second the deconvolution problem is formulated as the minimization of a convex functional with a data fidelity term reflecting the noise properties and a nonsmooth sparsity promoting penalty over the image representation coefficients e g lscr1 norm an additional term is also included in the functional to ensure positivity of the restored image third a fast iterative forward backward splitting algorithm is proposed to solve the minimization problem we derive existence and uniqueness conditions of the solution and establish convergence of the iterative algorithm finally a gcv based model selection procedure is proposed to objectively select the regularization parameter experimental results are carried out to show the striking benefits gained from taking into account the poisson statistics of the noise these results also suggest that using sparse domain regularization may be tractable in many deconvolution applications with poisson noise such as astronomy and microscopy

projection and proximal point methods convergence results and counterexamples recently hundal has constructed a hyperplane h a cone k and a starting point y0 in 2 such that the sequence of alternating projections pk ph n y0 n n converges weakly to some point in h k but not in norm we show how this construction results in a counterexample to norm convergence for iterates of averaged projections hence we give an a rmative answer to a question raised by reich two decades ago furthermore new counterexamples to norm convergence for iterates of rmly nonexpansive maps a la genel and lindenstrauss and for the proximal point algorithm al a g uler are provided we also present a counterexample along with some weak and norm convergence results for the new framework of string averaging projection methods introduced by censor et al extensions to banach spaces and the situation for the hilbert ball are discussed as well 2003 elsevier ltd all rights reserved msc 47h09 47j25 90c25

monotone operator theory in convex optimization several aspects of the interplay between monotone operator theory and convex optimization are presented the crucial role played by monotone operators in the analysis and the numerical solution of convex minimization problems is emphasized we review the properties of subdifferentials as maximally monotone operators and in tandem investigate those of proximity operators as resolvents in particular we study new transformations which map proximity operators to proximity operators and establish connections with self dual classes of firmly nonexpansive operators in addition new insights and developments are proposed on the algorithmic front

absolute stability and complete synchronization in a class of neural fields models neural fields are an interesting option for modeling macroscopic parts of the cortex involving several populations of neurons like cortical areas two classes of neural field equations are considered voltage and activity based the spatio temporal behavior of these fields is described by nonlinear integro differential equations the integral term computed over a compact subset of mathbb r q q 1 2 3 involves space and time varying possibly nonsymmetric intracortical connectivity kernels contributions from white matter afferents are represented as external input sigmoidal nonlinearities arise from the relation between average membrane potentials and instantaneous firing rates using methods of functional analysis we characterize the existence and uniqueness of a solution of these equations for general homogeneous i e independent of the spatial variable and spatially locally homogeneous inputs in all cases we give sufficient conditions on the connectivity functions for the solutions to be absolutely stable that is to say asymptotically independent of the initial state of the field these conditions bear on some compact operators defined from the connectivity kernels the maximal slope of the sigmoids and the time constants used in describing the temporal shape of the postsynaptic potentials numerical experiments are presented to illustrate the theory to our knowledge this is the first time that such a complete analysis of the problem of the existence and uniqueness of a solution of these equations has been obtained another important contribution is the analysis of the absolute stability of these solutions more difficult but more general than the linear stability analysis which it implies the reason we have been able to complete this work is our use of the functional analysis framework and the theory of compact operators in a hilbert space which has allowed us to provide simple mathematical answers to some of the questions raised by modelers in neuroscience

proximal operator of quotient functions with application to a feasibility problem in query optimization in this paper we determine the proximity functions of the sum and the maximum of componentwise reciprocal quotients of positive vectors for the sum of quotients denoted by q 1 the proximity function is just a componentwise shrinkage function which we call q shrinkage this is similar to the proximity function of the 1 norm which is given by componentwise soft shrinkage for the maximum of quotients q the proximal function can be computed by first order primal dual methods involving epigraphical projections the proximity functions of q 1 are applied to solve convex problems of the form argmin x q a x b subject to x 0 1 x 1 such problems are of interest in selectivity estimation for cost based query optimizers in database management systems

exponential convergence of delayed dynamical systems we discuss some delayed dynamical systems investigating their stability and convergence we prove that under mild conditions these delayed systems are global exponential convergent