tree structure sparsity pattern guided convex optimization for compressive sensing of large scale images cost efficient compressive sensing of large scale images with quickly reconstructed high quality results is very challenging in this paper we present an algorithm to solve convex optimization via the tree structure sparsity pattern which can be run in the operator to reduce computation cost and maintain good quality especially for large scale images we also provide convergence analysis and convergence rate analysis for the proposed method the feasibility of our method is verified through simulations and comparison with the state of the art algorithms

proximal thresholding algorithm for minimization over orthonormal bases the notion of soft thresholding plays a central role in problems from various areas of applied mathematics in which the ideal solution is known to possess a sparse decomposition in some orthonormal basis using convex analytical tools we extend this notion to that of proximal thresholding and investigate its properties providing in particular several characterizations of such thresholders we then propose a versatile convex variational formulation for optimization over orthonormal bases that covers a wide range of problems and we establish the strong convergence of a proximal thresholding algorithm to solve it numerical applications to signal recovery are demonstrated

event based parareal a data flow based implementation of parareal parareal is an iterative algorithm that in effect achieves temporal decomposition for a time dependent system of differential or partial differential equations a solution is obtained in a shorter wall clock time but at the expense of increased compute cycles the algorithm combines a fine solver that solves the system to acceptable accuracy with an approximate coarse solver the critical task for the successful implementation of parareal on any system is the development of a coarse solver that leads to convergence in a small number of iterations compared to the number of time slices in the full time interval and is at the same time much faster than the fine solver very fast coarse solvers may not lead to sufficiently rapid convergence and slow coarse solvers may not lead to significant gains even if the number of iterations to convergence is satisfactory we find that the difficulty of meeting these conflicting demands can be substantially eased by using a data driven event based implementation of parareal as a result tasks for one iteration do not wait for the previous iteration to complete but are started when the needed data are available for given convergence properties the event based approach relaxes the speed requirements on the coarse solver by a factor of k where k is the number of iterations required for a converged solution this may for many problems lead to an efficient parareal implementation that would otherwise not be possible or would require substantial coarse solver development in addition the framework used for this implementation executes a task when the data dependencies are satisfied and computational resources are available this leads to improved computational efficiency over previous approaches that pipeline or schedule groups of tasks to a particular processor or group of processors

a direct formulation for sparse pca using semidefinite programming given a covariance matrix we consider the problem of maximizing the variance explained by a particular linear combination of the input variables while constraining the number of nonzero coefficients in this combination this problem arises in the decomposition of a covariance matrix into sparse factors or sparse principal component analysis pca and has wide applications ranging from biology to finance we use a modification of the classical variational representation of the largest eigenvalue of a symmetric matrix where cardinality is constrained and derive a semidefinite programming based relaxation for our problem we also discuss nesterov s smooth minimization technique applied to the semidefinite program arising in the semidefinite relaxation of the sparse pca problem the method has complexity o n 4 sqrt log n epsilon where n is the size of the underlying covariance matrix and epsilon is the desired absolute accuracy on the optimal value of the problem

a modified forward backward splitting method for maximal monotone mappings we consider the forward backward splitting method for finding a zero of the sum of two maximal monotone mappings this method is known to converge when the inverse of the forward mapping is strongly monotone we propose a modification to this method in the spirit of the extragradient method for monotone variational inequalities under which the method converges assuming only the forward mapping is lipschitz continuous on some closed convex subset of its domain the modification entails an additional forward step and a projection step at each iteration applications of the modified method to decomposition in convex programming and monotone variational inequalities are discussed

combined helmholtz equation least squares method for reconstructing acoustic radiation from arbitrarily shaped objects a combined helmholtz equation least squares chels method is developed for reconstructing acoustic radiation from an arbitrary object this method combines the advantages of both the hels method and the helmholtz integral theory based near field acoustic holography nah as such it allows for reconstruction of the acoustic field radiated from an arbitrary object with relatively few measurements thus significantly enhancing the reconstruction efficiency the first step in the chels method is to establish the hels formulations based on a finite number of acoustic pressure measurements taken on or beyond a hypothetical spherical surface that encloses the object under consideration next enough field acoustic pressures are generated using the hels formulations and taken as the input to the helmholtz integral formulations implemented through the boundary element method bem the acoustic pressure and normal component of the velocity at the discretized nodes on the surface are then determined by solving two matrix equations using singular value decomposition svd and regularization techniques also presented are in depth analyses of the advantages and limitations of the chels method examples of reconstructing acoustic radiation from separable and nonseparable surfaces are demonstrated

convergence rates in forward backward splitting forward backward splitting methods provide a range of approaches to solving large scale optimization problems and variational inequalities in which structures conducive to decomposition can be utilized apart from special cases where the forward step is absent and a version of the proximal point algorithm comes out efforts at evaluating the convergence potential of such methods have so far relied on lipschitz properties and strong monotonicity or inverse strong monotonicity of the mapping involved in the forward step the perspective mainly being that of projection algorithms here convergence is analyzed by a technique that allows properties of the mapping in the backward step to be brought in as well for the first time in such a general setting global and local contraction rates are derived moreover they are derived in a form which makes it possible to determine the optimal step size relative to certain constants associated with the given problem insights are thereby gained into the effects of shifting strong monotonicity between the forward and backward mappings when a splitting is selected

a fixed point iterative method for low n rank tensor pursuit the linearly constrained tensor n rank minimization problem is an extension of matrix rank minimization it is applicable in many fields which use the multi way data such as data mining machine learning and computer vision in this paper we adapt operator splitting technique and convex relaxation technique to transform the original problem into a convex unconstrained optimization problem and propose a fixed point iterative method to solve it we also prove the convergence of the method under some assumptions by using a continuation technique we propose a fast and robust algorithm for solving the tensor completion problem which is called fp lrtc fixed point for low n rank tensor completion our numerical results on randomly generated and real tensor completion problems demonstrate that this algorithm is effective especially for easy problems

parallel in time approximation of the lattice boltzmann method for laminar flows abstract fluid dynamics simulations using grid based methods such as the lattice boltzmann equation can benefit from parallel in space computation however for a fixed size simulation of this type the efficiency of larger processor counts will saturate when the number of grid points per core becomes too small to overcome this fundamental strong scaling limit in space parallel approaches we present a novel time parallel version of the lattice boltzmann method using the parareal algorithm this method is based on a predictor corrector scheme combined with mesh refinement to enable the simulation of larger number of time steps we present results of up to a 32 increase in speed for a model system consisting of a cylinder with conditions for laminar flow the parallel gain obtainable is predicted with strong accuracy providing a quantitative understanding of the potential impact of this method

a projection method for convex constrained monotone nonlinear equations with applications in this paper we present a projection method to solve monotone nonlinear equations with convex constraints this method can be viewed as an extension of cg descent method which is one of the most effective conjugate gradient methods for solving unconstrained optimization problems because of derivative free and low storage the proposed method can be used to solve large scale nonsmooth monotone nonlinear equations its global convergence is established under some appropriate conditions preliminary numerical results show that the proposed method is effective and promising moreover we also successfully use the proposed method to solve the sparse signal reconstruction in compressive sensing

splitting methods for pseudomonotone mixed variational inequalities we consider and analyze some new splitting methods for solving monotone mixed variational inequalities by using the technique of updating the solution the modified methods converge for monotone and pseudomonotone continuous operators the new splitting methods differ from the existing splitting methods the new results are versatile and proof of convergence is very simple

on the linear convergence of descent methods for convex essentially smooth minimization consider the problem of minimizing over a polyhedral set the composition of an affine mapping with a strictly convex essentially smooth function a general result on the linear convergence of descent methods for solving this problem is presented by applying this result the linear convergence of both the gradient projection algorithm of goldstein and levitin and polyak and a matrix splitting algorithm using regular splitting is established the results do not require that the cost function be strongly convex or that the optimal solution set be bounded the key to the analysis lies in a new error bound for estimating the distance from a feasible point to the optimal solution set

a posteriori error bounds for the linearly constrained varitional inequality problem in this paper we derive a posteriori error bound of an approximate solution to a linearly constrained variational inequality problem three measures of error are used with each depending on a particular characterization of an exact solution and related to a certain class of iterative methods

a pseudo heuristic parameter selection rule for l1 regularized minimization problems abstract this paper considers the regularization parameter determination of l 1 regularized minimization problem we solve the l 1 regularized problem using iterative reweighted least squares irls which involves solving a linear system whose coefficient matrix has the form m 1 n 0 1 the aim of this paper is to find an efficient and computationally inexpensive algorithm to both choose the regularization parameter and solve the l 1 regularized problem in order to achieve this we propose an irls algorithm with adaptive regularization parameter selection based on a heuristic parameter determination rule de boor s parameter selection criterion compared with some of the state of the art algorithms and parameter selection rules the numerical experiments show the efficiency and robustness of the proposed method

nonlinear wavelet image processing variational problems compression and noise removal through wavelet shrinkage this paper examines the relationship between wavelet based image processing algorithms and variational problems algorithms are derived as exact or approximate minimizers of variational problems in particular we show that wavelet shrinkage can be considered the exact minimizer of the following problem given an image f defined on a square i minimize over all g in the besov space b11 l1 i the functional f g l2 i 2 g b11 l1 i we use the theory of nonlinear wavelet image compression in l2 i to derive accurate error bounds for noise removal through wavelet shrinkage applied to images corrupted with i i d mean zero gaussian noise a new signal to noise ratio snr which we claim more accurately reflects the visual perception of noise in images arises in this derivation we present extensive computations that support the hypothesis that near optimal shrinkage parameters can be derived if one knows or can estimate only two parameters about an image f the largest for which f bq lq i 1 q 2 1 2 and the norm f bq lq i both theoretical and experimental results indicate that our choice of shrinkage parameters yields uniformly better results than donoho and johnstone s visushrink procedure an example suggests however that donoho and johnstone s 1994 1995 1996 sureshrink method which uses a different shrinkage parameter for each dyadic level achieves a lower error than our procedure

communication aware adaptive parareal with application to a nonlinear hyperbolic system of partial differential equations in the strong scaling limit the performance of conventional spatial domain decomposition techniques for the parallel solution of pdes saturates when sub domains become small halo communication and other overheard come to dominate a potential path beyond this scaling limit is to introduce domain decomposition in time with one such popular approach being the parareal algorithm which has received a lot of attention due to its generality and potential scalability low efficiency particularly on convection dominated problems has however limited the adoption of the method in this paper we introduce a new strategy communication aware adaptive parareal caap to overcome some of the challenges with caap we choose time subdomains short enough that convergence of the parareal algorithm is quick yet long enough that the overheard of communicating time subdomain interfaces does not induce a new limit to parallel speed up furthermore we propose an adaptive work scheduling algorithm that overlaps consecutive parareal cycles and decouples the number of time subdomains and number of active node groups in an efficient manner to allow for comparatively high parallel eciency we demonstrate the viability of caap trough the parallel in time integration of a hyperbolic system of pdes in the form of the two dimensional nonlinear shallow water wave equation solved using a 3rd order accurate weno rk discretization for the computational cheap approximate operator needed as a preconditioner in the parareal corrections we use a lower order roe type discretization time parallel integration of purely hyperbolic type evolution problems is traditionally considered impractical trough large scale numerical experiments we demonstrate that with caap it is possible not only to obtain time parallel speedup on this class of evolution problems but also that we may obtain parallel acceleration beyond what is possible using conventional spatial domain decomposition techniques alone the approach is widely applicable for parallel in time integration over long time domains regardless of the class of evolution problem

fast and accurate algorithms for re weighted ell _ 1 norm minimization to recover a sparse signal from an underdetermined system we often solve a constrained l1 norm minimization problem in many cases the signal sparsity and recovery performance can be further improved by replacing the l1 norm with a weighted l1 norm without prior information about the signal s nonzero elements the procedure for selecting weights is iterative in nature common approaches update the weights at every iteration using the solution of a weighted l1 problem from the previous iteration this paper presents two homotopy based algorithms that efficiently solve reweighted l1 problems first we present an algorithm that quickly updates the solution of a weighted l1 problem as the weights change since the solution changes only slightly with small changes in weights we develop a homotopy algorithm that replaces old weights with new ones in a small number of computationally inexpensive steps second we propose an algorithm that solves a weighted l1 problem by adaptively selecting weights while estimating the signal this algorithm integrates the reweighting into every step along the homotopy path by changing the weights according to changes in the solution and its support allowing us to achieve a high quality signal reconstruction by solving a single homotopy problem we compare both algorithms performance in terms of reconstruction accuracy and computational complexity against state of the art solvers and show that our methods have smaller computational cost we also show that the adaptive selection of the weights inside the homotopy often yields reconstructions of higher quality

why simple shrinkage is still relevant for redundant representations shrinkage is a well known and appealing denoising technique introduced originally by donoho and johnstone in 1994 the use of shrinkage for denoising is known to be optimal for gaussian white noise provided that the sparsity on the signal s representation is enforced using a unitary transform still shrinkage is also practiced with nonunitary and even redundant representations typically leading to very satisfactory results in this correspondence we shed some light on this behavior the main argument in this work is that such simple shrinkage could be interpreted as the first iteration of an algorithm that solves the basis pursuit denoising bpdn problem while the desired solution of bpdn is hard to obtain in general we develop a simple iterative procedure for the bpdn minimization that amounts to stepwise shrinkage we demonstrate how the simple shrinkage emerges as the first iteration of this novel algorithm furthermore we show how shrinkage can be iterated turning into an effective algorithm that minimizes the bpdn via simple shrinkage steps in order to further strengthen the denoising effect

fast alternating linearization methods for minimizing the sum of two convex functions we present in this paper alternating linearization algorithms based on an alternating direction augmented lagrangian approach for minimizing the sum of two convex functions our basic methods require at most o 1 epsilon iterations to obtain an epsilon optimal solution while our accelerated i e fast versions of them require at most o 1 sqrt epsilon iterations with little change in the computational effort required at each iteration for both types of methods we present one algorithm that requires both functions to be smooth with lipschitz continuous gradients and one algorithm that needs only one of the functions to be so algorithms in this paper are gauss seidel type methods in contrast to the ones proposed by goldfarb and ma in fast multiple splitting algorithms for convex optimization columbia university 2009 where the algorithms are jacobi type methods numerical results are reported to support our theoretical conclusions and demonstrate the practical potential of our algorithms

regularization techniques and suboptimal solutions to optimization problems in learning from data various regularization techniques are investigated in supervised learning from data theoretical features of the associated optimization problems are studied and sparse suboptimal solutions are searched for rates of approximate optimization are estimated for sequences of suboptimal solutions formed by linear combinations of n tuples of computational units and statistical learning bounds are derived as hypothesis sets reproducing kernel hilbert spaces and their subsets are considered

fixed point continuation for ell_1 minimization methodology and convergence we present a framework for solving the large scale ell 1 regularized convex minimization problem min x 1 mu f x our approach is based on two powerful algorithmic ideas operator splitting and continuation operator splitting results in a fixed point algorithm for any given scalar mu continuation refers to approximately following the path traced by the optimal value of x as mu increases in this paper we study the structure of optimal solution sets prove finite convergence for important quantities and establish q linear convergence rates for the fixed point algorithm applied to problems with f x convex but not necessarily strictly convex the continuation framework motivated by our convergence results is demonstrated to facilitate the construction of practical algorithms

a primal dual active set algorithm with continuation for compressed sensing the success of compressed sensing relies essentially on the ability to efficiently find an approximately sparse solution to an under determined linear system in this paper we developed an efficient algorithm for the sparsity promoting l 1 regularized least squares problem by coupling the primal dual active set strategy with a continuation technique on the regularization parameter in the active set strategy we first determine the active set from primal and dual variables and then update the primal and dual variables by solving a low dimensional least square problem on the active set which makes the algorithm very efficient the continuation technique globalizes the convergence of the algorithm with provable global convergence under restricted isometry property rip further we adopt two alternative methods i e a modified discrepancy principle and a bayesian information criterion to choose the regularization parameter automatically numerical experiments indicate that our algorithm is very competitive with state of the art algorithms in terms of accuracy and efficiency without a priori information about the regularization parameter

linearized proximal alternating minimization algorithm for motion deblurring by nonlocal regularization abstract non blind motion deblurring problems are highly ill posed and so it is quite difficult to find the original sharp and clean image to handle ill posedness of the motion deblurring problem we use nonlocal total variation abbreviated as tv regularization approaches nonlocal tv can restore periodic textures and local geometric information better than local tv but since nonlocal tv requires weighted difference between pixels in the whole image it demands much more computational resources than local tv by using the linearization of the fidelity term and the proximal function our proposed algorithm does not require any inversion of blurring operator and nonlocal operator therefore the proposed algorithm is very efficient for motion deblurring problems we compare the numerical performance of our proposed algorithm with that of several state of the art algorithms for deblurring problems our numerical results show that the proposed method is faster and more robust than state of the art algorithms on motion deblurring problems

reconstruction of a sparse spike train from a portion of its spectrum and application to high resolution deconvolution an algorithm is proposed for the reconstruction of a sparse spike train from an incomplete set of its fourier components it is shown that as little as 20 25 percent of the fourier spectrum is sufficient in practice for a high quality reconstruction the method employs linear programming to minimize the l1 norm of the output because minimization of this norm favors solutions with isolated spikes given a wavelet this technique can be used to perform deconvolution of noisy seismograms when the desired output is a sparse spike series relative reliability of the data is assessed in the frequency domain and only the reliable spectral data are included in the calculation of the spike series equations for the unknown spike amplitudes are solved to an accuracy compatible with the uncertainties in the reliable data in examples with 10 percent random noise the output is superior to that obtained using conventional least squares techniques

a fast algorithm for sparse reconstruction based on shrinkage subspace optimization and continuation we propose a fast algorithm for solving the ell 1 regularized minimization problem min x in mathbb r n mu x 1 ax b 2 2 for recovering sparse solutions to an undetermined system of linear equations ax b the algorithm is divided into two stages that are performed repeatedly in the first stage a first order iterative shrinkage method yields an estimate of the subset of components of x likely to be nonzero in an optimal solution restricting the decision variables x to this subset and fixing their signs at their current values reduces the ell 1 norm x 1 to a linear function of x the resulting subspace problem which involves the minimization of a smaller and smooth quadratic function is solved in the second phase our code fpc as embeds this basic two stage algorithm in a continuation homotopy approach by assigning a decreasing sequence of values to mu this code exhibits state of the art performance in terms of both its speed and its ability to recover sparse signals

a unified approach to error bounds for structured convex optimization problems error bounds which refer to inequalities that bound the distance of vectors in a test set to a given set by a residual function have proven to be extremely useful in analyzing the convergence rates of a host of iterative methods for solving optimization problems in this paper we present a new framework for establishing error bounds for a class of structured convex optimization problems in which the objective function is the sum of a smooth convex function and a general closed proper convex function such a class encapsulates not only fairly general constrained minimization problems but also various regularized loss minimization formulations in machine learning signal processing and statistics using our framework we show that a number of existing error bound results can be recovered in a unified and transparent manner to further demonstrate the power of our framework we apply it to a class of nuclear norm regularized loss minimization problems and establish a new error bound for this class under a strict complementarity type regularity condition we then complement this result by constructing an example to show that the said error bound could fail to hold without the regularity condition we believe that our approach will find further applications in the study of error bounds for structured convex optimization problems

a unified approach for minimizing composite norms we propose a first order augmented lagrangian algorithm falc to solve the composite norm minimization problem begin aligned begin array ll min limits x in mathbb r m times n mu 1 vert sigma mathcal f x g vert alpha mu 2 vert mathcal c x d vert beta text subject text to mathcal a x b in mathcal q end array end aligned where sigma x denotes the vector of singular values of x in mathbb r m times n the matrix norm vert sigma x vert alpha denotes either the frobenius the nuclear or the ell 2 operator norm of x the vector norm vert vert beta denotes either the ell 1 norm ell 2 norm or the ell infty norm mathcal q is a closed convex set and mathcal a mathcal c mathcal f are linear operators from mathbb r m times n to vector spaces of appropriate dimensions basis pursuit matrix completion robust principal component pursuit pcp and stable pcp problems are all special cases of the composite norm minimization problem thus falc is able to solve all these problems in a unified manner we show that any limit point of falc iterate sequence is an optimal solution of the composite norm minimization problem we also show that for all epsilon 0 the falc iterates are epsilon feasible and epsilon optimal after mathcal o log epsilon 1 iterations which require mathcal o epsilon 1 constrained shrinkage operations and euclidean projection onto the set mathcal q surprisingly on the problem sets we tested falc required only mathcal o log epsilon 1 constrained shrinkage instead of the mathcal o epsilon 1 worst case bound to compute an epsilon feasible and epsilon optimal solution to best of our knowledge falc is the first algorithm with a known complexity bound that solves the stable pcp problem

iterative sparsification projection fast and robust sparse signal approximation in this paper we address recovery of sparse signals from compressed measurements and sparse signal approximation which have received considerable attention over the last decade first we revisit smoothed l0 sl0 a well known sparse recovery algorithm and give some insights into it that have not been noticed previously specifically we re derive the sl0 algorithm based on proximal methods and using recent results in solving nonconvex problems by proximal algorithms we provide a convergence guarantee for it in addition inspired by this derivation we propose a general family of algorithms which we call iterative sparsification projection isp having sl0 as a special member our algorithmic framework starts with an initial guess for the unknown sparse vector and then iteratively sparsifies it using a fixed threshold followed by projecting the result onto the admissible solution set the threshold is then decreased and the same process is repeated the algorithm terminates when the threshold becomes sufficiently small or another stopping criterion is satisfied we also propose a robust projection to handle the situations with observation noise or model uncertainties our extensive simulations confirm the promising performance of the isp algorithms compared with some well known algorithms

constructing test instances for basis pursuit denoising the number of available algorithms for the so called basis pursuit denoising problem or the related lasso problem is large and keeps growing similarly the number of experiments to evaluate and compare these algorithms on different instances is growing in this correspondence we present a method to produce instances with exact solutions that is based on a simple observation related to the so called source condition from sparse regularization

projected landweber iteration for matrix completion recovering an unknown low rank or approximately low rank matrix from a sampling set of its entries is known as the matrix completion problem in this paper a nonlinear constrained quadratic program problem concerning the matrix completion is obtained a new algorithm named the projected landweber iteration plw is proposed and the convergence is proved strictly numerical results show that the proposed algorithm can be fast and efficient under suitable prior conditions of the unknown low rank matrix

a family of second order methods for convex ell _1 l1 regularized optimization this paper is concerned with the minimization of an objective that is the sum of a convex function f and an ell 1 l1 regularization term our interest is in active set methods that incorporate second order information about the function f to accelerate convergence we describe a semismooth newton framework that can be used to generate a variety of second order methods including block active set methods orthant based methods and a second order iterative soft thresholding method the paper proposes a new active set method that performs multiple changes in the active manifold estimate at every iteration and employs a mechanism for correcting these estimates when needed this corrective mechanism is also evaluated in an orthant based method numerical tests comparing the performance of three active set methods are presented

development and optimization of regularized tomographic reconstruction algorithms utilizing equally sloped tomography we develop two new algorithms for tomographic reconstruction which incorporate the technique of equally sloped tomography est and allow for the optimized and flexible implementation of regularization schemes such as total variation constraints and the incorporation of arbitrary physical constraints the founding structure of the developed algorithms is est a technique of tomographic acquisition and reconstruction first proposed by miao in 2005 for performing tomographic image reconstructions from a limited number of noisy projections in an accurate manner by avoiding direct interpolations est has recently been successfully applied to coherent diffraction microscopy electron microscopy and computed tomography for image enhancement and radiation dose reduction however the bottleneck of est lies in its slow speed due to its higher computation requirements in this paper we formulate the est approach as a constrained problem and subsequently transform it into a series of linear problems which can be accurately solved by the operator splitting method based on these mathematical formulations we develop two iterative algorithms for tomographic image reconstructions through est which incorporate bregman and continuative regularization our numerical experiment results indicate that the new tomographic image reconstruction algorithms not only significantly reduce the computational time but also improve the image quality we anticipate that est coupled with the novel iterative algorithms will find broad applications in x ray tomography electron microscopy coherent diffraction microscopy and other tomography fields

sparse identification of posynomial models posynomials are nonnegative combinations of monomials with possibly fractional and both positive and negative exponents posynomial models are widely used in various engineering design endeavors such as circuits aerospace and structural design mainly due to the fact that design problems cast in terms of posynomial objectives and constraints can be solved efficiently by means of a convex optimization technique known as geometric programming gp however while quite a vast literature exists on gp based design very few contributions can yet be found on the problem of identifying posynomial models from experimental data posynomial identification amounts to determining not only the coefficients of the combination but also the exponents in the monomials which renders the identification problem hard in this paper we propose an approach to the identification of multivariate posynomial models based on the expansion on a given large scale basis of monomials the model is then identified by seeking coefficients of the combination that minimize a mixed objective composed by a term representing the fitting error and a term inducing sparsity in the representation which results in a problem formulation of the square root lasso type with nonnegativity constraints on the variables we propose to solve the problem via a sequential coordinate minimization scheme which is suitable for large scale implementations a numerical example is finally presented dealing with the identification of a posynomial model for a naca 4412 airfoil

a first order smoothed penalty method for compressed sensing we propose a first order smoothed penalty algorithm spa to solve the sparse recovery problem min x 1 ax b spa is efficient as long as the matrix vector product ax and a t y can be computed efficiently in particular a need not have orthogonal rows spa converges to the target signal by solving a sequence of penalized optimization subproblems and each subproblem is solved using nesterov s optimal algorithm for simple sets yu nesterov introductory lectures on convex optimization a basic course kluwer academic publishers norwell ma 2004 and yu nesterov math program 103 2005 pp 127 152 we show that the spa iterates x k are epsilon feasible i e ax k b 2 leq epsilon and epsilon optimal i e x k 1 x ast 1 leq epsilon after tilde mathcal o epsilon frac 3 2 iterations spa is able to work with ell 1 ell 2 or ell infty penalty on the infeasibility and spa can be easily extended to solve the relaxed recovery problem min x 1 ax b 2 leq delta

wavelet frame based algorithm for 3d reconstruction in electron microscopy in electron microscopy three dimensional 3d reconstruction is one key component in many computerized techniques for solving 3d structures of large protein assemblies using electron microscopy images of particles main challenges in 3d reconstruction include very low signal to noise ratio and very large scale of data sets involved in the computation motivated by the recent advances of sparsity based regularization in the wavelet frame domain for solving various linear inverse problems in imaging science we proposed a wavelet tight frame based 3d reconstruction approach that exploits the sparsity of the 3d density map in a wavelet tight frame system the proposed approach not only runs efficiently in terms of cpu time but also requires a much lower memory footprint than existing framelet based regularization methods the convergence of the proposed iterative scheme and the functional it minimizes is also examined together with the connection to existing wavelet frame based regularizations the numeric

computationally efficient truncated nuclear norm minimization for high dynamic range imaging matrix completion is a rank minimization problem to recover a low rank data matrix from a small subset of its entries since the matrix rank is nonconvex and discrete many existing approaches approximate the matrix rank as the nuclear norm however the truncated nuclear norm is known to be a better approximation to the matrix rank than the nuclear norm exploiting a priori target rank information about the problem in rank minimization in this paper we propose a computationally efficient truncated nuclear norm minimization algorithm for matrix completion which we call tnnm alm we reformulate the original optimization problem by introducing slack variables and considering noise in the observation the central contribution of this paper is to solve it efficiently via the augmented lagrange multiplier alm method where the optimization variables are updated by closed form solutions we apply the proposed tnnm alm algorithm to ghost free high dynamic range imaging by exploiting the low rank structure of irradiance maps from low dynamic range images experimental results on both synthetic and real visual data show that the proposed algorithm achieves significantly lower reconstruction errors and superior robustness against noise than the conventional approaches while providing substantial improvement in speed thereby applicable to a wide range of imaging applications

incremental majorization minimization optimization with application to large scale machine learning majorization minimization algorithms consist of successively minimizing a sequence of upper bounds of the objective function these upper bounds are tight at the current estimate and each iteration monotonically drives the objective function downhill such a simple principle is widely applicable and has been very popular in various scientific fields especially in signal processing and statistics we propose an incremental majorization minimization scheme for minimizing a large sum of continuous functions a problem of utmost importance in machine learning we present convergence guarantees for nonconvex and convex optimization when the upper bounds approximate the objective up to a smooth error we call such upper bounds first order surrogate functions more precisely we study asymptotic stationary point guarantees for nonconvex problems and for convex ones we provide convergence rates for the expected objective function value we apply our scheme to composite optimization and obtain a new incremental proximal gradient algorithm with linear convergence rate for strongly convex functions our experiments show that our method is competitive with the state of the art for solving machine learning problems such as logistic regression when the number of training samples is large enough and we demonstrate its usefulness for sparse estimation with nonconvex penalties

determination of the optimal number of expansion terms in the hels method the helmholtz equation least square hels method has been proven to be an efficient way of reconstructing acoustic radiation from a vibrating structure wu j acoust soc am 107 2511 2522 2000 in this method the optimal number of expansion terms is determined by a heuristic method of searching the minimum norm errors of the reconstructed acoustic pressures with respect to the measured data while this approach works it is time consuming in this paper a new method of a priori determination of the optimal number of expansions is developed based on the eigenvalue analysis of the measured information assuming a gaussian white measurement noise the hypothesis of equal noise eigenvalues is tested based on the statistical properties of the eigenvalues this hypothesis test yields an upper bound of the optimal number of expansions in general also developed is a method for testing the correlation coefficients after eliminating some principal components which gives a lower bound of the optimum numb

an inexact proximal path following algorithm for constrained convex minimization many scientific and engineering applications feature nonsmooth convex minimization problems over convex sets in this paper we address an important instance of this broad class where we assume that the nonsmooth objective is equipped with a tractable proximity operator and that the convex constraint set affords a self concordant barrier we provide a new joint treatment of proximal and self concordant barrier concepts and illustrate that such problems can be efficiently solved without the need of lifting the problem dimensions as in disciplined convex optimization approach we propose an inexact path following algorithmic framework and theoretically characterize the worst case analytical complexity of this framework when the proximal subproblems are solved inexactly to show the merits of our framework we apply its instances to both synthetic and real world applications where it shows advantages over standard interior point methods as a byproduct we describe how our framework can obtain points on t

hybrid near field acoustic holography hybrid near field acoustical holography nah is developed for reconstructing acoustic radiation from an arbitrary object in a cost effective manner this hybrid nah is derived from a modified helmholtz equation least squares hels formula that expands the acoustic pressure in terms of outgoing and incoming waves the expansion coefficients are determined by solving an overdetermined linear system of equations obtained by matching the assumed form solution to measured acoustic pressures through the least squares measurements are taken over a conformal surface around a source at close range so that the evanescent waves can be captured next the modified hels is utilized to regenerate as much acoustic pressures on the conformal surface as necessary and take them as input to the helmholtz integral formulation implemented numerically by boundary element method bem the acoustic pressures and normal velocities on the source surface are reconstructed by using a modified tikhnov regularization tr with its regularization parameter determined by generalized cross validation gcv method results demonstrate that this hybrid nah combines the advantages of hels and inverse bem this is because a majority of the input data are regenerated but not measured thus the efficiency of reconstruction is greatly enhanced meanwhile the accuracy of reconstruction is ensured by the helmholtz integral theory and modified tr together with gcv method provided that hels converges fast enough on the measurement surface numerical examples of reconstructing acoustic quantities on the surface of a simplified engine block are demonstrated work supported by nsf

projective robust nonnegative factorization nonnegative matrix factorization nmf has been successfully used in many fields as a low dimensional representation method projective nonnegative matrix factorization pnmf is a variant of nmf that was proposed to learn a subspace for feature extraction however both original nmf and pnmf are sensitive to noise and are unsuitable for feature extraction if data is grossly corrupted in order to improve the robustness of nmf a framework named projective robust nonnegative factorization prnf is proposed in this paper for robust image feature extraction and classification since learned projections can weaken noise disturbances prnf is more suitable for classification and feature extraction in order to preserve the geometrical structure of original data prnf introduces a graph regularization term which encodes geometrical structure in the prnf framework three algorithms are proposed that add a sparsity constraint on the noise matrix based on l1 2 norm l1 norm and l2 1 norm respectively robustness and classification performance of the three proposed algorithms are verified with experiments on four face image databases and results are compared with state of the art robust nmf based algorithms experimental results demonstrate the robustness and effectiveness of the algorithms for image classification and feature extraction

recovering low rank matrices from corrupted observations via the linear conjugate gradient algorithm the matrix nuclear norm minimization problem has received much attention in recent years largely because its highly related to the matrix rank minimization problem arising from controller design signal processing and model reduction the alternating direction method is a very popular way to solve this problem due to its simplicity low storage practical computation efficiency and nice convergence properties in this paper we propose an alternating direction method where one variable is determined explicitly and the other variable is computed by a linear conjugate gradient algorithm at each iteration the method involves a singular value thresholding and its convergence result is guaranteed in this literature extensive experiments illustrate that the proposed algorithm compares favorable with the state of the art algorithms fpca and iadm bb which were specifically designed in recent years

continuous location under the effect of refraction in this paper we address the problem of locating a new facility on a d dimensional space when the distance measure ell p lp or polyhedral norms is different at each one of the sides of a given hyperplane mathcal h h we relate this problem with the physical phenomenon of refraction and extend it to any finite dimensional space and different distances at each one of the sides of any hyperplane an application to this problem is the location of a facility within or outside an urban area where different distance measures must be used we provide a new second order cone programming formulation based on the ell p lp norm representation given in blanco et al comput optim appl 58 3 563 595 2014 that allows to solve the problem in any finite dimensional space with second order cone or semidefinite programming tools we also extend the problem to the case where the hyperplane is considered as a rapid transit media a different third norm is also considered over mathcal h h that allows the demand to travel whenever it is convenient through mathcal h h to reach the new facility extensive computational experiments run in gurobi are reported in order to show the effectiveness of the approach some extensions of these models are also presented

nonlinear magnetohydrodynamics simulation using high order finite elements a conforming representation composed of 2d finite elements and finite fourier series is applied to 3d nonlinear non ideal magnetohydrodynamics using a semi implicit time advance the self adjoint semi implicit operator and variational approach to spatial discretization are synergistic and enable simulation in the extremely stiff conditions found in high temperature plasmas without sacrificing the geometric flexibility needed for modeling laboratory experiments growth rates for resistive tearing modes with experimentally relevant lundquist number are computed accurately with time steps that are large with respect to the global alfven time and moderate spatial resolution when the finite elements have basis functions of polynomial degree p two or larger an error diffusion method controls the generation of magnetic divergence error convergence studies show that this approach is effective for continuous basis functions with p 2 where the number of test functions for the divergence control terms is less than the number of degrees of freedom in the expansion for vector fields anisotropic thermal conduction at realistic ratios of parallel to perpendicular conductivity is computed accurately with p 3 without mesh alignment a simulation of tearingmode evolution for a shaped toroidal tokamak equilibrium demonstrates the effectiveness of the algorithm in nonlinear conditions and its results are used to verify the accuracy of the numerical anisotropic thermal conduction in 3d magnetic topologies

a first order augmented lagrangian method for compressed sensing we propose a first order augmented lagrangian fal algorithm for solving the basis pursuit problem fal computes a solution to this problem by inexactly solving a sequence of ell 1 regularized least squares subproblems these subproblems are solved using an infinite memory proximal gradient algorithm wherein each update reduces to shrinkage or constrained shrinkage we show that fal converges to an optimal solution of the basis pursuit problem whenever the solution is unique which is the case with very high probability for compressed sensing problems we construct a parameter sequence such that the corresponding fal iterates are epsilon feasible and epsilon optimal for all epsilon 0 within mathcal o left log left epsilon 1 right right fal iterations moreover fal requires at most mathcal o epsilon 1 matrix vector multiplications of the form ax or a ty to compute an epsilon feasible epsilon optimal solution we show that fal can be easily extended to solve