fast coupled adaptation for sparse impulse responses using a partial haar transform this work presents a novel scheme for identifying the impulse response of a sparse channel the scheme consists of two adaptive filters operating sequentially the first adaptive filter adapts using a partial haar transform of the input and yields an estimate of the location of the peak of the sparse impulse response the second adaptive filter is then centered about this estimate both filters are short in comparison to the delay uncertainty of the unknown channel the principle advantage of this scheme is that two short adaptive filters can be used instead of one long adaptive filter resulting in faster overall convergence and reduced computational complexity and storage the scheme is analyzed in detail for a least mean squares lms lms lms type of structure although it can be implemented using any combination of adaptive algorithms monte carlo simulations are shown to be in good agreement with the theoretical model for the behavior of the peak estimating filter as well as for the mean square error mse behavior of the second filter

a note on the neumann problem for a quasilinear elliptic problem of a nonmonotone type abstract the existence and uniqueness of a weak solution of a neumann problem is discussed for a second order quasilinear elliptic equation in a divergence form the note is a continuation of a recent paper where mixed boundary value problems were considered which guaranteed the coerciveness of the problem

consistency of spectral clustering consistency is a key property of all statistical procedures analyzing randomly sampled data surprisingly despite decades of work little is known about consistency of most clustering algorithms in this paper we investigate consistency of the popular family of spectral clustering algorithms which clusters the data with the help of eigenvectors of graph laplacian matrices we develop new methods to establish that for increasing sample size those eigenvectors converge to the eigenvectors of certain limit operators as a result we can prove that one of the two major classes of spectral clustering normalized clustering converges under very general conditions while the other unnormalized clustering is only consistent under strong additional assumptions which are not always satisfied in real data we conclude that our analysis provides strong evidence for the superiority of normalized spectral clustering

exploiting sparsity in adaptive filters this paper studies a class of algorithms called natural gradient ng algorithms the least mean square lms algorithm is derived within the ng framework and a family of lms variants that exploit sparsity is derived this procedure is repeated for other algorithm families such as the constant modulus algorithm cma and decision directed dd lms mean squared error analysis stability analysis and convergence analysis of the family of sparse lms algorithms are provided and it is shown that if the system is sparse then the new algorithms will converge faster for a given total asymptotic mse simulations are provided to confirm the analysis in addition bayesian priors matching the statistics of a database of real channels are given and algorithms are derived that exploit these priors simulations using measured channels are used to show a realistic application of these algorithms

sparse signal approximation via nonseparable regularization the calculation of a sparse approximate solution to a linear system of equations is often performed using either l1 norm regularization and convex optimization or nonconvex regularization and nonconvex optimization combining these principles this paper describes a type of nonconvex regularization that maintains the convexity of the objective function thereby allowing the calculation of a sparse approximate solution via convex optimization the preservation of convexity is viable in the proposed approach because it uses a regularizer that is nonseparable the proposed method is motivated and demonstrated by the calculation of sparse signal approximation using tight frames examples of denoising demonstrate improvement relative to l1 norm regularization

graph realizations associated with minimizing the maximum eigenvalue of the laplacian in analogy to the absolute algebraic connectivity of fiedler we study the problem of minimizing the maximum eigenvalue of the laplacian of a graph by redistributing the edge weights via semidefinite duality this leads to a graph realization problem in which nodes should be placed as close as possible to the origin while adjacent nodes must keep a distance of at least one we prove three main results for a slightly generalized form of this embedding problem first given a set of vertices partitioning the graph into several or just one part the barycenter of each part is embedded on the same side of the affine hull of the set as the origin second there is an optimal realization of dimension at most the tree width of the graph plus one and this bound is best possible in general finally bipartite graphs possess a one dimensional optimal embedding

online sparse system identification and signal reconstruction using projections onto weighted ell_ 1 balls this paper presents a novel projection based adaptive algorithm for sparse signal and system identification the sequentially observed data are used to generate an equivalent sequence of closed convex sets namely hyperslabs each hyperslab is the geometric equivalent of a cost criterion that quantifies data mismatch sparsity is imposed by the introduction of appropriately designed weighted l1 balls and the related projection operator is also derived the algorithm develops around projections onto the sequence of the generated hyperslabs as well as the weighted l1 balls the resulting scheme exhibits linear dependence with respect to the unknown system s order on the number of multiplications additions and an o llog2l dependence on sorting operations where l is the length of the system signal to be estimated numerical results are also given to validate the performance of the proposed method against the least absolute shrinkage and selection operator lasso algorithm and two very recently developed adaptive sparse schemes that fuse arguments from the lms rls adaptation mechanisms with those imposed by the lasso rational

block sparsity induced adaptive filter for multi clustering system identification in order to improve the performance of least mean square lms based adaptive filtering for identifying block sparse systems a new adaptive algorithm called block sparse lms bs lms is proposed in this paper the basis of the proposed algorithm is to insert a penalty of block sparsity which is a mixed l2 0 norm of adaptive tap weights with equal group partition sizes into the cost function of traditional lms algorithm to describe a block sparse system response we first propose a markov gaussian model which can generate a kind of system responses of arbitrary average sparsity and arbitrary average block length using given parameters then we present theoretical expressions of the steady state misadjustment and transient convergence behavior of bs lms with an appropriate group partition size for white gaussian input data based on the above results we theoretically demonstrate that bs lms has much better convergence behavior than l0 lms with the same small level of misadjustment finally numerical experiments verify that all of the theoretical analysis agrees well with simulation results in a large range of parameters

on minimizing the spectral width of graph laplacians and associated graph realizations extremal eigenvalues and eigenvectors of the laplace matrix of a graph form the core of many bounds on graph parameters and graph optimization problems for example the value of a uniform sparsest cut is bounded from below and above by the second smallest and the largest eigenvalue of the weighted laplacian divided by the number of nodes of the graph minimizing the difference between maximum and second smallest eigenvalue over edge weighted laplacians of a graph reduces the size of this interval we study this problem of minimizing the spectral width for its own sake with the goal of advancing the understanding of connections between structural properties of the graph and the corresponding eigenvectors and eigenvalues building on previous work where these eigenvalues were investigated separately we show that a corresponding dual problem allows us to view eigenvectors to optimized eigenvalues as graph realizations in euclidean space whose structure is tightly linked to the separator structure of the g

region based wavelet packet adaptive algorithm for identification of sparse impulse responses identification of systems with sparse impulse response encounters large applicability numerous techniques have been proposed to identify such systems efficiently one strategy that leads to specially good results is to detect the active nonzero response samples and update only the corresponding adaptive coefficients wavelet based approaches have been shown to be specially effective to this end this paper proposes a new region based wavelet packet rbwp algorithm for efficient identification of systems with sparse impulse responses with arbitrary frequency spectra and with any delay of the effective response the discrete wavelet packet transform dwpt is adaptively tailored to the energy distribution of the unknown system s response spectrum the new algorithm leads to a reduced number of active weights and to a reduced computational complexity when compared with competing wavelet based algorithms monte carlo simulation results show good performances regarding convergence speed and robustness to design parameter choice

reduction of symmetric semidefinite programs using the regular ast representation we consider semidefinite programming problems on which a permutation group is acting we describe a general technique to reduce the size of such problems exploiting the symmetry the technique is based on a low order matrix representation of the commutant centralizer ring of the matrix algebra generated by the permutation matrices we apply it to extending a method of de klerk et al that gives a semidefinite programming lower bound to the crossing number of complete bipartite graphs it implies that cr k 8 n 2 9299n 2 6n cr k 9 n 3 8676n 2 8n and for any m 9 lim n to infty frac rm cr k m n z m n geq 0 8594 frac m m 1 where z m n is the zarankiewicz number lfloor frac 1 4 m 1 2 rfloor lfloor frac 1 4 n 1 2 rfloor which is the conjectured value of cr k m n here the best factor previously known was 0 8303 instead of 0 8594

minimizing effective resistance of a graph the effective resistance between two nodes of a weighted graph is the electrical resistance seen between the nodes of a resistor network with branch conductances given by the edge weights the effective resistance comes up in many applications and fields in addition to electrical network analysis including for example markov chains and continuous time averaging networks in this paper we study the problem of allocating edge weights on a given graph in order to minimize the total effective resistance i e the sum of the resistances between all pairs of nodes we show that this is a convex optimization problem and can be solved efficiently either numerically or in some cases analytically we show that optimal allocation of the edge weights can reduce the total effective resistance of the graph compared to uniform weights by a factor that grows unboundedly with the size of the graph we show that among all graphs with n nodes the path has the largest value of optimal total effective resistance and the complete graph has the least

embedded in the shadow of the separator eigenvectors to the second smallest eigenvalue of the laplace matrix of a graph also known as fiedler vectors are the basic ingredient in spectral graph partitioning heuristics maximizing this second smallest eigenvalue over all nonnegative edge weightings with bounded total weight yields the absolute algebraic connectivity introduced by fiedler who proved tight connections of this value to the connectivity of the graph our objective is to gain a better understanding of the connections between separators and the eigenspace of this eigenvalue by studying the dual semidefinite optimization problem to the absolute algebraic connectivity by exploiting optimality conditions we show that this problem is equivalent to finding an embedding of the n nodes of the graph in n space so that their barycenter is the origin the distance between adjacent nodes is bounded by one and the nodes are spread as much as possible the sum of the squared norms is maximized for connected graphs we prove that for any separator in the graph at least one of the two separated node sets is embedded in the shadow with the origin being the light source of the convex hull of the separator furthermore we show that there always exists an optimal embedding whose dimension is bounded by the tree width of the graph plus one

designing optimal multiplex networks for certain laplacian spectral properties we discuss the design of interlayer edges in a multiplex network under a limited budget with the goal of improving its overall performance we analyze the following three problems separately first we maximize the smallest nonzero eigenvalue also known as the algebraic connectivity second we minimize the largest eigenvalue also known as the spectral radius and finally we minimize the spectral width maximizing the algebraic connectivity requires identical weights on the interlayer edges for budgets less than a threshold value however for larger budgets the optimal weights are generally nonuniform the dual formulation transforms the problem into a graph realization embedding problem that allows us to give a fuller picture namely before the threshold budget the optimal realization is one dimensional with nodes in the same layer embedded to a single point while beyond the threshold the optimal embeddings generally unfold into spaces with dimension bounded by the multiplicity of the algebraic connectivity finally for extremely large budgets the embeddings again revert to lower dimensions minimizing the largest eigenvalue is driven by the spectral radius of the individual networks and its corresponding eigenvector before a threshold the total budget is distributed among interlayer edges corresponding to the nodal lines of this eigenvector and the optimal largest eigenvalue of the laplacian remains constant for larger budgets the weight distribution tends to be almost uniform in the dual picture the optimal graph embedding is one dimensional and nonhomogeneous at first with the nodes corresponding to the layer with the largest spectral radius distributed on a line according to its eigenvector while the other layer is embedded at the origin beyond this threshold the optimal embedding expands to be multidimensional and for larger values of the budget the two layers fill the embedding space finally we show how these two problems are connected to minimizing the spectral width

transform domain lms algorithm the concept of transform domain adaptive filtering is introduced in certain applications filtering in the transform domain results in great improvements in convergence rate over the conventional time domain adaptive filtering the relationship between several existing frequency domain adaptive filtering algorithms is established applications of the discrete fourier transform dft and the discrete cosine transform dct domain adaptive filtering algorithms in the areas of speech processing and adaptive line enhancers are discussed

partial update lms algorithms partial updating of lms filter coefficients is an effective method for reducing computational load and power consumption in adaptive filter implementations this paper presents an analysis of convergence of the class of sequential partial update lms algorithms s lms under various assumptions and shows that divergence can be prevented by scheduling coefficient updates at random which we call the stochastic partial update lms algorithm spu lms specifically under the standard independence assumptions for wide sense stationary signals the s lms algorithm converges in the mean if the step size parameter spl mu is in the convergent range of ordinary lms relaxing the independence assumption it is shown that s lms and lms algorithms have the same sufficient conditions for exponential stability however there exist nonstationary signals for which the existing algorithms s lms included are unstable and do not converge for any value of spl mu on the other hand under broad conditions the spu lms algorithm remains stable for nonstationary signals expressions for convergence rate and steady state mean square error of spu lms are derived the theoretical results of this paper are validated and compared by simulation through numerical examples

multiwavelet prestack kirchhoff migration the problems of stretching and aliasing usually occur in prestack kirchhoff migration a multiwavelet based approach is proposed to provide an alternative approach to resolve these problems two steps are involved in the multiwavelet based algorithm the first step is the decomposition of the seismic traces with a series of wavelets of different dominant frequencies this step is based on the principle of basis pursuit and is enhanced for more accurate and sparse decomposition by adding an adaptive subdictionary that is associated with the minima and maxima of a seismic trace the second step is wavelet migration based on the kirchhoff formulation using a novel approach the outer iteration is the wavelets of the input seismic traces which ensures that an input wavelet is only used once in the entire migration process this enables output of multiple migrated images of different wavelets of dominant frequency ranges elimination of noisy images and composition of images for specific interpretation and reservoir characterization purposes with very little extra computational cost the stretching and aliasing problems are naturally resolved because interpolation on the seismic trace is not required in the migration we tested the algorithm using a synthetic model with a dipping layer and a 2d real seismic section we compared our results with results obtained by conventional kirchhoff migration

why simple shrinkage is still relevant for redundant representations shrinkage is a well known and appealing denoising technique introduced originally by donoho and johnstone in 1994 the use of shrinkage for denoising is known to be optimal for gaussian white noise provided that the sparsity on the signal s representation is enforced using a unitary transform still shrinkage is also practiced with nonunitary and even redundant representations typically leading to very satisfactory results in this correspondence we shed some light on this behavior the main argument in this work is that such simple shrinkage could be interpreted as the first iteration of an algorithm that solves the basis pursuit denoising bpdn problem while the desired solution of bpdn is hard to obtain in general we develop a simple iterative procedure for the bpdn minimization that amounts to stepwise shrinkage we demonstrate how the simple shrinkage emerges as the first iteration of this novel algorithm furthermore we show how shrinkage can be iterated turning into an effective algorithm that minimizes the bpdn via simple shrinkage steps in order to further strengthen the denoising effect

sparse signal estimation by maximally sparse convex optimization this paper addresses the problem of sparsity penalized least squares for applications in sparse signal processing e g sparse deconvolution this paper aims to induce sparsity more strongly than l1 norm regularization while avoiding non convex optimization for this purpose this paper describes the design and use of non convex penalty functions regularizers constrained so as to ensure the convexity of the total cost function f to be minimized the method is based on parametric penalty functions the parameters of which are constrained to ensure convexity of f it is shown that optimal parameters can be obtained by semidefinite programming sdp this maximally sparse convex msc approach yields maximally non convex sparsity inducing penalty functions constrained such that the total cost function f is convex it is demonstrated that iterative msc imsc can yield solutions substantially more sparse than the standard convex sparsity inducing approach i e l1 norm minimization

bounds on the laplacian spectral radius of graphs the spectral radius of a graph is the largest eigenvalue of adjacency matrix of the graph and its laplacian spectral radius is the largest eigenvalue of the laplacian matrix which is the difference of the diagonal matrix of vertex degrees and the adjacency matrix some sharp bounds are obtained for the laplacian spectral radii of connected graphs as consequences some sharp upper bounds of the nordhaus gaddum type are also obtained for the sum of laplacian spectral radii of a connected graph and its connected complement

block sparse recovery via convex optimization given a dictionary that consists of multiple blocks and a signal that lives in the range space of only a few blocks we study the problem of finding a block sparse representation of the signal i e a representation that uses the minimum number of blocks motivated by signal image processing and computer vision applications such as face recognition we consider the block sparse recovery problem in the case where the number of atoms in each block is arbitrary possibly much larger than the dimension of the underlying subspace to find a block sparse representation of a signal we propose two classes of nonconvex optimization programs which aim to minimize the number of nonzero coefficient blocks and the number of nonzero reconstructed vectors from the blocks respectively since both classes of problems are np hard we propose convex relaxations and derive conditions under which each class of the convex programs is equivalent to the original nonconvex formulation our conditions depend on the notions of mutual and cumulative subspace coherence of a dictionary which are natural generalizations of existing notions of mutual and cumulative coherence we evaluate the performance of the proposed convex programs through simulations as well as real experiments on face recognition we show that treating the face recognition problem as a block sparse recovery problem improves the state of the art results by 10 with only 25 of the training data

an iterative bayesian algorithm for sparse component analysis in presence of noise we present a bayesian approach for sparse component analysis sca in the noisy case the algorithm is essentially a method for obtaining sufficiently sparse solutions of underdetermined systems of linear equations with additive gaussian noise in general an underdetermined system of linear equations has infinitely many solutions however it has been shown that sufficiently sparse solutions can be uniquely identified our main objective is to find this unique solution our method is based on a novel estimation of source parameters and maximum a posteriori map estimation of sources to tackle the great complexity of the map algorithm when the number of sources and mixtures become large we propose an iterative bayesian algorithm iba this iba algorithm is based on the map estimation of sources too but optimized with a steepest ascent method the convergence analysis of the iba algorithm and its convergence to true global maximum are also proved simulation results show that the performance achieved by the iba algorithm is among the best while its complexity is rather high in comparison to other algorithms simulation results also show the low sensitivity of the iba algorithm to its simulation parameters

invariancy of sparse recovery algorithms in this paper a property for sparse recovery algorithms called invariancy is introduced the significance of invariancy is that the performance of the algorithms with this property is less affected when the sensing i e the dictionary is ill conditioned this is because for this kind of algorithms there exists implicitly an equivalent well conditioned problem which is being solved some examples of sparse recovery algorithms will also be considered and it will be shown that some of them such as sl0 basis pursuit using interior point lp solver focuss and hard thresholding algorithms are invariant and some others like matching pursuit and spgl1 are not then as an application example of the invariancy property a sparse decomposition based method for direction of arrival estimation is reviewed and it is shown that if an invariant algorithm is utilized for solving the corresponding sparse recovery problem the spatial characteristics of the sensors will have essentially no effect on the final estimation provided that the number of sensors is large enough

l0 norm based structural sparse least square regression for feature selection this paper presents a novel approach for feature selection with regard to the problem of structural sparse least square regression sslsr rather than employing the l1 norm regularization to control the sparsity we directly work with sparse solutions via l0 norm regularization in particular we develop an effective greedy algorithm where the forward and backward steps are combined adaptively to resolve the sslsr problem with the intractable l r 0 norm on the one hand features with the strongest correlation to classes are selected in the forward steps on the other hand redundant features which contribute little to the improvement of the objective function are removed in the backward steps furthermore we provide solid theoretical analysis to prove the effectiveness of the proposed method experimental results on synthetic and real world data sets from different domains also demonstrate the superiority of the proposed method over the state of the arts highlightswe impose l0 norm inequality constraint to build the structural sparse lsr problem we develop an adaptive algorithm to ensure the structural sparsity accurately theoretical results on the efficiency and effectiveness of our method are provided experimental results prove the superiority of our method over the state of the arts

the fastest mixing markov process on a graph and a connection to a maximum variance unfolding problem we consider a markov process on a connected graph with edges labeled with transition rates between the adjacent vertices the distribution of the markov process converges to the uniform distribution at a rate determined by the second smallest eigenvalue lambda 2 of the laplacian of the weighted graph in this paper we consider the problem of assigning transition rates to the edges so as to maximize lambda 2 subject to a linear constraint on the rates this is the problem of finding the fastest mixing markov process fmmp on the graph we show that the fmmp problem is a convex optimization problem which can in turn be expressed as a semidefinite program and therefore effectively solved numerically we formulate a dual of the fmmp problem and show that it has a natural geometric interpretation as a maximum variance unfolding mvu problem the problem of choosing a set of points to be as far apart as possible measured by their variance while respecting local distance constraints this mvu problem is closely related to a problem recently proposed by weinberger and saul as a method for unfolding high dimensional data that lies on a low dimensional manifold the duality between the fmmp and mvu problems sheds light on both problems and allows us to characterize and in some cases find optimal solutions

analysis of sparse representation and blind source separation in this letter we analyze a two stage cluster then l1 optimization approach for sparse representation of a data matrix which is also a promising approach for blind source separation bss in which fewer sensors than sources are present first sparse representation factorization of a data matrix is discussed for a given overcomplete basis matrix the corresponding sparse solution coefficient matrix with minimum l1 norm is unique with probability one which can be obtained using a standard linear programming algorithm the equivalence of the l1 norm solution and the l0 norm solution is also analyzed according to a probabilistic framework if the obtained l1 norm solution is sufficiently sparse then it is equal to the l0 norm solution with a high probability furthermore the l1 norm solution is robust to noise but the l0 norm solution is not showing that the l1 norm is a good sparsity measure these results can be used as a recoverability analysis of bss as discussed the basis matrix in this article is estimated using a clustering algorithm followed by normalization in which the matrix columns are the cluster centers of normalized data column vectors zibulevsky pearlmutter boll and kisilev 2000 used this kind of two stage approach in underdetermined bss our recoverability analysis shows that this approach can deal with the situation in which the sources are overlapped to some degree in the analyzed domain and with the case in which the source number is unknown it is also robust to additive noise and estimation error in the mixing matrix finally four simulation examples and an eeg data analysis example are presented to illustrate the algorithm s utility and demonstrate its performance

exploiting special structure in semidefinite programming a survey of theory and applications abstract semidefinite programming sdp may be seen as a generalization of linear programming lp in particular one may extend interior point algorithms for lp to sdp but it has proven much more difficult to exploit structure in the sdp data during computation we survey three types of special structures in sdp data 1 a common chordal sparsity pattern of all the data matrices this structure arises in applications in graph theory and may also be used to deal with more general sparsity patterns in a heuristic way 2 low rank of all the data matrices this structure is common in sdp relaxations of combinatorial optimization problems and sdp approximations of polynomial optimization problems 3 the situation where the data matrices are invariant under the action of a permutation group or more generally where the data matrices belong to a low dimensional matrix algebra such problems arise in truss topology optimization particle physics coding theory computational geometry and graph theory we will give an overview of existing techniques to exploit these structures in the data most of the paper will be devoted to the third situation since it has received the least attention in the literature so far

rapid identification of a sparse impulse response using an adaptive algorithm in the haar domain this paper proposes a fast convergence adaptive algorithm for identifying a sparse impulse response that is rich in spectral content a sparse impulse response is referred here as a discrete time impulse response that has a large number of zero or near zero coefficients the basic idea for rapid identification is to automatically determine the locations of the nonzero impulse response coefficients for their adaptation and eliminate the unnecessary adaptation of zero coefficients the proposed method which is called the haar basis algorithm employs a transform approach by modeling the sparse impulse response in the haar domain the haar transform has many basis sets and each of them contains basis vectors that span the entire time domain range this special nature of the haar transform allows for the selection of one small subset of adaptive filter coefficients whose basis vectors span the entire range of the impulse response these coefficients are adapted at the beginning and are then used subsequently to identify from the hierarchical structure of the haar transform the rest of the filter coefficients that must be adapted to correctly model the unknown sparse impulse response the consequence is avoiding adaptation of many zero coefficients leading to a dramatic improvement in either convergence speed or steady state excess mean square error ease while requiring no a priori knowledge such as the number of nonzero coefficients in the unknown sparse impulse response the proposed algorithm has been tested with a variety of unknown sparse systems using both white noise input and colored input whose spectrum closely resembles that of speech simulation results show that the new approach provides promising results

analysis of the normalized lms algorithm with gaussian inputs the lms adaptive filter algorithm requires a priori knowledge of the input power level to select the algorithm gain parameter for stability and convergence since the input power level is usually one of the statistical unknowns it is normally estimated from the data prior to beginning the adaptation process it is then assumed that the estimate is perfect in any subsequent analysis of the lms algorithm behavior in this paper the effects of the power level estimate are incorporated in a data dependent that appears explicitly within the algorithm the transient mean and second moment behavior of the modified lms nlms algorithm are evaluated taking into account the explicit statistical dependence of upon the input data the mean behavior of the algorithm is shown to converge to the wiener weight a constant coefficient matrix difference equation is derived for the weight fluctuations about the wiener weight the equation is solved for a white data covariance matrix and for the adaptive line enhancer with a single frequency input in steady state for small expressions for the misadjustment error are also presented it is shown for the white data covariance matrix case that the averaging of about ten data samples causes negligible degradation as compared to the lms algorithm in the ale application the steady state weight fluctuations are shown to be mode dependent being largest at the frequency of the input

iterative sparsification projection fast and robust sparse signal approximation in this paper we address recovery of sparse signals from compressed measurements and sparse signal approximation which have received considerable attention over the last decade first we revisit smoothed l0 sl0 a well known sparse recovery algorithm and give some insights into it that have not been noticed previously specifically we re derive the sl0 algorithm based on proximal methods and using recent results in solving nonconvex problems by proximal algorithms we provide a convergence guarantee for it in addition inspired by this derivation we propose a general family of algorithms which we call iterative sparsification projection isp having sl0 as a special member our algorithmic framework starts with an initial guess for the unknown sparse vector and then iteratively sparsifies it using a fixed threshold followed by projecting the result onto the admissible solution set the threshold is then decreased and the same process is repeated the algorithm terminates when the threshold becomes sufficiently small or another stopping criterion is satisfied we also propose a robust projection to handle the situations with observation noise or model uncertainties our extensive simulations confirm the promising performance of the isp algorithms compared with some well known algorithms

a fast approach for overcomplete sparse decomposition based on smoothed ell 0 norm in this paper a fast algorithm for overcomplete sparse decomposition called sl0 is proposed the algorithm is essentially a method for obtaining sparse solutions of underdetermined systems of linear equations and its applications include underdetermined sparse component analysis sca atomic decomposition on overcomplete dictionaries compressed sensing and decoding real field codes contrary to previous methods which usually solve this problem by minimizing the l 1 norm using linear programming lp techniques our algorithm tries to directly minimize the l 1 norm it is experimentally shown that the proposed algorithm is about two to three orders of magnitude faster than the state of the art interior point lp solvers while providing the same or better accuracy

improved semidefinite programming bounds for quadratic assignment problems with suitable symmetry semidefinite programming sdp bounds for the quadratic assignment problem qap were introduced in zhao et al j comb optim 2 71 109 1998 empirically these bounds are often quite good in practice but computationally demanding even for relatively small instances for qap instances where the data matrices have large automorphism groups these bounds can be computed more efficiently as was shown in klerk and sotirov math program a 122 2 225 246 2010 continuing in the same vein we show how one may obtain stronger bounds for qap instances where one of the data matrices has a transitive automorphism group to illustrate our approach we compute improved lower bounds for several instances from the qap library qaplib

blind source separation by sparse decomposition in a signal dictionary the blind source separation problem is to extract the underlying source signals from a set of linear mixtures where the mixing matrix is unknown this situation is common in acoustics radio medical signal and image processing hyperspectral imaging and other areas we suggest a two stage separation process a priori selection of a possibly overcomplete signal dictionary for instance a wavelet frame or a learned dictionary in which the sources are assumed to be sparsely representable followed by unmixing the sources by exploiting the their sparse representability we consider the general case of more sources than mixtures but also derive a more efficient algorithm in the case of a nonovercomplete dictionary and an equal numbers of sources and mixtures experiments with artificial signals and musical sounds demonstrate significantly better separation than other known techniques

on galerkin approximations of a quasilinear nonpotential elliptic problem of a nonmonotone type abstract we prove the convergence of galerkin approximations for a second order quasilinear elliptic problem of a divergence form with nonlinear mixed boundary conditions the existence and uniqueness of a weak and approximate solution is studied as well

greedy sparse rls starting from the orthogonal greedy least squares method we build an adaptive algorithm for finding online sparse solutions to linear systems the algorithm belongs to the exponentially windowed recursive least squares rls family and maintains a partial orthogonal factorization with pivoting of the system matrix for complexity reasons the permutations that bring the relevant columns into the first positions are restrained mainly to interchanges between neighbors at each time moment the storage scheme allows the computation of the exact factorization implicitly working on indefinitely long vectors the sparsity level of the solution i e the number of nonzero elements is estimated using information theoretic criteria in particular bayesian information criterion bic and predictive least squares we present simulations showing that for identifying sparse time varying fir channels our algorithm is consistently better than previous sparse rls methods based on the norm regularization of the rls criterion we also use our sparse greedy rls algorithm for computing linear predictions in a lossless audio coding scheme and obtain better compression than mpeg4 als using an rls lms cascade

underdetermined blind modal identification of structures by earthquake and ambient vibration measurements via sparse component analysis abstract sparse component analysis sca approach was adopted to handle underdetermined blind modal identification of structures where the number of sensors is less than the number of active modes to exploit the sparsity of structural responses in time frequency domain short time fourier transform stft was used in this study the proposed sca based approach has two main stages modal matrix estimation and modal displacement estimation in the first stage hierarchical clustering algorithm was used to estimate the modal matrix the clustering algorithm was preceded by a preprocessing step to select the points in time frequency domain that only one mode makes contribution in the responses these points were fed to the clustering algorithm as an input performing this analysis enhanced the modal matrix estimation accuracy and reduced the computational cost while conducting clustering analysis having estimated mixing matrix the complex valued modal responses in the transformed domain were recovered via smoothed zero norm sl 0 algorithm in a broad sense using the sl 0 algorithm permits researchers to use any kind of transform in seeking sparsity regardless of obtaining real valued or complex valued signals in transformed domain natural frequencies and damping ratios were extracted from the recovered modal responses performance of the proposed method was investigated using a synthetic example and a benchmark structure with earthquake and ambient excitation respectively

rism single modal image registration via rank induced similarity measure similarity measure is an important block in image registration most traditional intensity based similarity measures e g sum of squared difference correlation coefficient and mutual information assume a stationary image and pixel by pixel independence these similarity measures ignore the correlation between pixel intensities hence perfect image registration cannot be achieved especially in the presence of spatially varying intensity distortions here we assume that spatially varying intensity distortion such as bias field is a low rank matrix based on this assumption we formulate the image registration problem as a nonlinear and low rank matrix decomposition nllrmd therefore image registration and correction of spatially varying intensity distortion are simultaneously achieved we illustrate the uniqueness of nllrmd and therefore we propose the rank of difference image as a robust similarity in the presence of spatially varying intensity distortion finally by incorporating the gaussian noise we introduce rank induced similarity measure based on the singular values of the difference image this measure produces clinically acceptable registration results on both simulated and real world problems examined in this paper and outperforms other state of the art measures such as the residual complexity approach

a general criterion for analog tx rx beamforming under ofdm transmissions in this paper we study beamforming schemes for a novel mimo transceiver which performs adaptive signal combining in the radio frequency rf domain assuming perfect channel knowledge at the receiver side we consider the problem of designing the transmit and receive rf beamformers under orthogonal frequency division multiplexing ofdm transmissions in particular a general beamforming criterion is proposed which depends on a single parameter this parameter establishes a tradeoff between the energy of the equivalent siso channel after tx rx beamforming and its spectral flatness the proposed cost function embraces most reasonable criteria for designing analog tx rx beamformers hence for particular values of the proposed criterion reduces to the minimization of the mean square error mse the maximization of the system capacity or the maximization of the received signal to noise ratio snr in general the proposed criterion results in a nonconvex optimization problem however we show that the problem can be rewritten as a convex cost function subject to a couple of rank one constraints and hence it can be approximately solved by semidefinite relaxation sdr techniques since the computational cost of sdr for this problem is rather high and building on the observation that the minima of the original problem must be solutions of a pair of coupled eigenvalue problems we propose yet another simple and efficient gradient search algorithm which in practice provides satisfactory solutions with a moderate computational cost finally several numerical examples show the good performance of the proposed technique for both uncoded and 802 11a coded transmissions

mimo radar 3d imaging based on combined amplitude and total variation cost function with sequential order one negative exponential form in inverse synthetic aperture radar isar imaging a target is usually regarded as consist of a few strong specular scatterers and the distribution of these strong scatterers is sparse in the imaging volume in this paper we propose to incorporate the sparse signal recovery method in 3d multiple input multiple output radar imaging algorithm sequential order one negative exponential soone function which forms homotopy between ell 1 and ell 0 norms is proposed to measure the sparsity gradient projection is used to solve a constrained nonconvex soone function minimization problem and recover the sparse signal however while the gradient projection method is computationally simple it is not robust when a matrix in the algorithm is ill conditioned we thus further propose using diagonal loading and singular value decomposition methods to improve the robustness of the algorithm in order to handle targets with large flat surfaces a combined amplitude and total variation objective function is also proposed to regularize the shapes of the flat surfaces simulation results show that the proposed gradient projection of soone function method is better than orthogonal matching pursuit cosamp ell 1 magic bayesian method with laplace prior smoothed ell 0 method and ell 1 ell s in high snr cases for recovery of pm 1 random spikes sparse signal the quality of the simulated 3d images and real data isar images obtained using the new method is better than that of the conventional correlation method and minimum ell 2 norm method and competitive to the aforementioned sparse signal recovery algorithms

on the convergence behavior of the lms and the normalized lms algorithms it is shown that the normalized least mean square nlms algorithm is a potentially faster converging algorithm compared to the lms algorithm where the design of the adaptive filter is based on the usually quite limited knowledge of its input signal statistics a very simple model for the input signal vectors that greatly simplifies analysis of the convergence behavior of the lms and nlms algorithms is proposed using this model answers can be obtained to questions for which no answers are currently available using other perhaps more realistic models examples are given to illustrate that even quantitatively the answers obtained can be good approximations it is emphasized that the convergence of the nlms algorithm can be speeded up significantly by employing a time varying step size the optimal step size sequence can be specified a priori for the case of a white input signal with arbitrary distribution

sparls the sparse rls algorithm we develop a recursive l1 regularized least squares sparls algorithm for the estimation of a sparse tap weight vector in the adaptive filtering setting the sparls algorithm exploits noisy observations of the tap weight vector output stream and produces its estimate using an expectation maximization type algorithm we prove the convergence of the sparls algorithm to a near optimal estimate in a stationary environment and present analytical results for the steady state error simulation studies in the context of channel estimation employing multipath wireless channels show that the sparls algorithm has significant improvement over the conventional widely used recursive least squares rls algorithm in terms of mean squared error mse moreover these simulation studies suggest that the sparls algorithm with slight modifications can operate with lower computational requirements than the rls algorithm when applied to tap weight vectors with fixed support

finite element approximation of a nonlinear heat conduction problem in anisotropic media abstract this paper is a survey of results which we have obtained in solving a stationary nonlinear heat conduction problem by the finite element method in particular we present uniqueness theorems for the classical and weak solutions a comparison theorem existence theorems for the weak and finite element solutions approximation of a curved boundary and numerical integration a discrete maximum principle convergence without any regularity assumptions a priori error estimates nonlinear boundary conditions and numerical algorithms

proportionate adaptive algorithms for network echo cancellation by analyzing the coefficient adaptation process of the steepest descent algorithm the condition under which the fastest overall convergence will be achieved is obtained and the way to calculate optimal step size control factors to satisfy that condition is formulated motivated by the results and using the stochastic approximation paradigm the spl mu law pnlms mpnlms algorithm is proposed to keep in contrast to the proportionate normalized least mean square pnlms algorithm the fast initial convergence during the whole adaptation process in the case of sparse echo path identification modifications of the mpnlms algorithm are proposed to lower the computational complexity

symmetry in semidefinite programs this paper is a tutorial in a general and explicit procedure to simplify semidefinite programs which are invariant under the action of a symmetry group the procedure is based on basic notions of representation theory of finite groups as an example we derive the block diagonalization of the terwilliger algebra of the binary hamming scheme in this framework here its connection to the orthogonal hahn and krawtchouk polynomials becomes visible