dc programming and dca thirty years of developments the year 2015 marks the 30th birthday of dc difference of convex functions programming and dca dc algorithms which constitute the backbone of nonconvex programming and global optimization in this article we offer a short survey on thirty years of developments of these theoretical and algorithmic tools the survey is comprised of three parts in the first part we present a brief history of the field while in the second we summarize the state of the art results and recent advances we focus on main theoretical results and dca solvers for important classes of difficult nonconvex optimization problems and then give an overview of real world applications whose solution methods are based on dca the third part is devoted to new trends and important open issues as well as suggestions for future developments

multitask quantile regression under the transnormal model abstractwe consider estimating multitask quantile regression under the transnormal model with focus on high dimensional setting we derive a surprisingly simple closed form solution through rank based covariance regularization in particular we propose the rank based l1 penalization with positive definite constraints for estimating sparse covariance matrices and the rank based banded cholesky decomposition regularization for estimating banded precision matrices by taking advantage of the alternating direction method of multipliers nearest correlation matrix projection is introduced that inherits sampling properties of the unprojected one our work combines strengths of quantile regression and rank based covariance regularization to simultaneously deal with nonlinearity and nonnormality for high dimensional regression furthermore the proposed method strikes a good balance between robustness and efficiency achieves the oracle like convergence rate and provides the provable prediction interval und

an upwinding embedded boundary method for maxwell s equations in media with material interfaces 2d case we propose a new upwinding embedded boundary method to solve time dependent maxwell s equations in media with material interfaces a global second order finite difference method is obtained by combining central difference schemes away from the interfaces and upwinding technique with jump conditions near the interfaces the proposed finite difference method allows time step based on a uniform mesh independent of the locations and shapes of the interfaces moreover the scheme is simple to implement in multidimensional cases numerical tests of wave equations with various types of material interfaces and electromagnetic scattering of 2d cylinders confirm the stability uniform accuracy and ease of implementation of the method

lattice tiling and density conditions for subspace gabor frames abstract the well known density theorem in time frequency analysis establishes the connection between the existence of a gabor frame g g a b e 2 i b m x g x a n m n z d for l 2 r d and the density of the time frequency lattice a z d b z d this is also tightly related to lattice tiling and packing in this paper we investigate the density theorem for gabor systems in l 2 s with s being an a z d periodic subset of r d we characterize the existence of a gabor frame for l 2 s in terms of a condition that involves the haar measure of the group generated by a z d and b t 1 z d this new characterization is used to recover the density theorem and several related known results in the literature additionally we apply this approach to obtain the density theorems for multi windowed and super gabor frames for l 2 s

high order fdtd methods via derivative matching for maxwell s equations with material interfaces this paper introduces a series of novel hierarchical implicit derivative matching methods to restore the accuracy of high order finite difference time domain fdtd schemes of computational electromagnetics cem with material interfaces in one 1d and two spatial dimensions 2d by making use of fictitious points systematic approaches are proposed to locally enforce the physical jump conditions at material interfaces in a preprocessing stage to arbitrarily high orders of accuracy in principle while often limited by numerical instability orders up to 16 and 12 are achieved respectively in 1d and 2d detailed stability analyses are presented for the present approach to examine the upper limit in constructing embedded fdtd methods as natural generalizations of the high order fdtd schemes the proposed derivative matching methods automatically reduce to the standard fdtd schemes when the material interfaces are absent an interesting feature of the present approach is that it encompasses a variety of schemes of different orders in a single code another feature of the present approach is that it can be robustly implemented with other high accuracy time domain approaches such as the multiresolution time domain method and the local spectral time domain method to cope with material interfaces numerical experiments on both 1d and 2d problems are carried out to test the convergence examine the stability access the efficiency and explore the limitation of the proposed methods it is found that operating at their best capacity the proposed high order schemes could be over 2000 times more efficient than their fourth order versions in 2d in conclusion the present work indicates that the proposed hierarchical derivative matching methods might lead to practical high order schemes for numerical solution of time domain maxwell s equations with material interfaces

the balian low theorem for the symplectic form on r2d in this paper we extend the balian low theorem which is a version of the uncertainty principle for gabor weyl heisenberg systems to functions of several variables in particular we first prove the balian low theorem for arbitrary quadratic forms then we generalize further and prove the balian low theorem for differential operators associated with a symplectic basis for the symplectic form on r2d

covariance matrix selection and estimation via penalised normal likelihood summary we propose a nonparametric method for identifying parsimony and for producing a statistically efficient estimator of a large covariance matrix we reparameterise a covariance matrix through the modified cholesky decomposition of its inverse or the one step ahead predictive representation of the vector of responses and reduce the nonintuitive task of modelling covariance matrices to the familiar task of model selection and estimation for a sequence of regression models the cholesky factor containing these regression coefficients is likely to have many off diagonal elements that are zero or close to zero penalised normal likelihoods in this situation with l1 and l2 penalities are shown to be closely related to tibshirani s 1996 lasso approach and to ridge regression adding either penalty to the likelihood helps to produce more stable estimators by introducing shrinkage to the elements in the cholesky factor while because of its singularity the l1 penalty will set some elements to zero and produce interpretable models an algorithm is developed for computing the estimator and selecting the tuning parameter the proposed maximum penalised likelihood estimator is illustrated using simulation and a real dataset involving estimation of a 102 x 102 covariance matrix

a second order dispersive fdtd algorithm for transverse electric maxwell s equations with complex interfaces this work overcomes the difficulty of the finite difference time domain fdtd algorithm in solving the transverse electric te maxwell s equations with inhomogeneous dispersive media for such te problems the electric fields are discontinuous across the dispersive interfaces moreover such discontinuities are time variant a novel matched interface and boundary time domain mibtd method is proposed to solve such problems through new developments in both mathematical formulations and numerical algorithms mathematically instead of handling all zeroth and first order jump conditions in a local coordinate we directly construct the te jump conditions which are needed in the fdtd computations in the cartesian coordinate such cartesian direction conditions depend on the time as well as tangential and normal components of the electric flux driven by the jump condition modeling we adopt the standard maxwell s equations in coupling with the debye auxiliary differential equations for the electric flux as the governing equations computationally the leapfrog scheme is employed for integrating the maxwell system and time dependent jump conditions sophisticated interface treatments are developed in both producing the te jump conditions and enforcing them in the fdtd algorithm based on a staggered yee lattice the numerical accuracy stability and efficiency of the proposed scheme are investigated by considering dispersive interfaces of various shapes the mibtd method achieves a spatially second order of convergence in all tests to the best of our knowledge the present mibtd scheme is the first fdtd method in the literature that can restore second order accuracy in treating curved dispersive interfaces for the te maxwell system

high order matched interface and boundary methods for the helmholtz equation in media with arbitrarily curved interfaces this work overcomes the difficulty of the previous matched interface and boundary mib method in dealing with interfaces with non constant curvatures for optical waveguide analysis this difficulty is essentially bypassed by avoiding the use of local cylindrical coordinates in the improved mib method instead novel jump conditions are derived along global cartesian directions for the transverse magnetic field components effective interface treatments are proposed to rigorously impose jump conditions across arbitrarily curved interfaces based on a simple cartesian grid even though each field component satisfies the scalar helmholtz equation the enforcement of jump conditions couples two transverse magnetic field components so that the resulting mib method is a full vectorial approach for the modal analysis of optical waveguides the numerical performance of the proposed mib method is investigated by considering interface problems with both constant and general curvatures the mib method is shown to be able to deliver a fourth order of accuracy in all cases even when a high frequency solution is involved

gabor representation of generalized functions this paper deals with gabor representation of generalized functions by this we mean that a given function is to be expanded in a series involving gabor functions that are located in the points of a given lattice in the time frequency plane we shall mainly consider lattices in which the area of the elementary cells of the lattice equals 1 in 1946 gabor used these expansions for the simultaneous analysis of signals in time and frequency cf 181 gabor stated that the above mentioned expansions exist for every reasonable signal and that the coefficients in the expansion are uniquely determined by the signal this statement is true when interpreted carefully and the aim of this paper is to find out what kind of generalized functions are sufficiently well behaved to allow a development in a gabor series we shall also consider the question of the uniqueness of the coefficients in the expansions as well as questions concerning convergence it turns out that the uniqueness questions can be handled with the aid of the main results of io in particular 2 12 and 2 13 the questions on the existence of expansions are harder to answer and require special care depending on the particular function we consider lpfunctions 1 p 2 in detail and show existence of gabor representation for tempered distribution in general

vector valued weak gabor dual frames on discrete periodic sets the notion of weak dual frames is a generalization of that of dual frames gabor analysis on discrete periodic sets has potential applications in signal processing this paper addresses vector valued weak gabor dual frames on discrete periodic sets we introduce the notions of its weak oblique gabor dual weak gabor duals of types i and ii for a gabor system on a discrete periodic set using the zak transform matrix method we characterize these three kinds of weak duals and their uniqueness finally we give an explicit expression of a class of weak gabor duals and provide some examples the notion of weak dual frames is a generalization of that of dual frames gabor analysis on discrete periodic sets has potential applications in signal processing this paper addresses vector valued weak gabor dual frames on discrete periodic sets we introduce the notions of its weak oblique gabor dual weak gabor duals of types i and ii for a gabor system on a discrete periodic set using the zak transform matrix method we characterize these three kinds of weak duals and their uniqueness finally we give an explicit expression of a class of weak gabor duals and provide some examples

graphical methods for efficient likelihood inference in gaussian covariance models in graphical modelling a bi directed graph encodes marginal independences among random variables that are identified with the vertices of the graph we show how to transform a bi directed graph into a maximal ancestral graph that i represents the same independence structure as the original bi directed graph and ii minimizes the number of arrowheads among all ancestral graphs satisfying i here the number of arrowheads of an ancestral graph is the number of directed edges plus twice the number of bi directed edges in gaussian models this construction can be used for more efficient iterative maximization of the likelihood function and to determine when maximum likelihood estimates are equal to empirical counterparts

estimation of a covariance matrix with zeros summary we consider estimation of the covariance matrix of a multivariate random vector under the constraint that certain covariances are zero we first present an algorithm which we call iterative conditional fitting for computing the maximum likelihood estimate of the constrained covariance matrix under the assumption of multivariate normality in contrast to previous approaches this algorithm has guaranteed convergence properties dropping the assumption of multivariate normality we show how to estimate the covariance matrix in an empirical likelihood approach these approaches are then compared via simulation and on an example of gene expression

balian low phenomenon for subspace gabor frames in this work the balian low theorem is extended to gabor also called weyl heisenberg frames for subspaces and more particularly its relationship with the unique gabor dual property for subspace gabor frames is pointed out to achieve this goal the subspace gabor frames which have a unique gabor dual of type i resp type ii are defined and characterized in terms of the zak transform for the rational parameter case this characterization is then used to prove the balian low theorem for subspace gabor frames along the same line the same characterization is used to prove a duality theorem for the unique gabor dual property which is an analogue of the ron and shen duality theorem

error analysis of finite element methods for 3 d maxwell s equations in dispersive media we consider the time dependent maxwell s equations in dispersive media on a bounded three dimensional domain a semi discrete standard finite element method and a semi discrete mixed finite element method are developed and error estimates are provided we believe this is the first finite element error analysis carried out for maxwell s equations in dispersive media

gabor super frames with hermite functions we investigate vector valued gabor frames sometimes called gabor superframes based on hermite functions hn let h h0 h1 hn be the vector of the first n 1 hermite functions we give a complete characterization of all lattices lambda subseteq mathbb r 2 such that the gabor system rm e 2 pi i lambda 2 t bf h t lambda 1 lambda lambda 1 lambda 2 in lambda is a frame for l 2 mathbb r mathbb c n 1 as a corollary we obtain sufficient conditions for a single hermite function to generate a gabor frame and a new estimate for the lower frame bound the main tools are growth estimates for the weierstrass function a new type of interpolation problem for entire functions on the bargmann fock space and structural results about vector valued gabor frames

maximum likelihood estimation for linear gaussian covariance models summary we study parameter estimation in linear gaussian covariance models which are p dimensional gaussian models with linear constraints on the covariance matrix maximum likelihood estimation for this class of models leads to a non convex optimization problem which typically has many local maxima using recent results on the asymptotic distribution of extreme eigenvalues of the wishart distribution we provide sufficient conditions for any hill climbing method to converge to the global maximum although we are primarily interested in the case in which n p the proofs of our results utilize large sample asymptotic theory under the scheme n p 1 remarkably our numerical simulations indicate that our results remain valid for p as small as 2 an important consequence of this analysis is that for sample sizes n 14p maximum likelihood estimation for linear gaussian covariance models behaves as if it were a convex optimization problem

finite difference time domain study of guided modes in nano plasmonic waveguides a conformal dispersive finite difference time domain fdtd method is developed for the study of one dimensional 1 d plasmonic waveguides formed by an array of periodic infinite long silver cylinders at optical frequencies the curved surfaces of circular and elliptical inclusions are modelled in orthogonal fdtd grid using effective permittivities eps and the material frequency dispersion is taken into account using an auxiliary differential equation ade method the proposed fdtd method does not introduce numerical instability but it requires a fourth order discretisation procedure to the authors knowledge it is the first time that the modelling of curved structures using a conformal scheme is combined with the dispersive fdtd method the dispersion diagrams obtained using eps and staircase approximations are compared with those from the frequency domain embedding method it is shown that the dispersion diagram can be modified by adding additional elements or changing geometry of inclusions numerical simulations of plasmonic waveguides formed by seven elements show that row s of silver nanoscale cylinders can guide the propagation of light due to the coupling of surface plasmons

sparse estimation of a covariance matrix we suggest a method for estimating a covariance matrix on the basis of a sample of vectors drawn from a multivariate normal distribution in particular we penalize the likelihood with a lasso penalty on the entries of the covariance matrix this penalty plays two important roles it reduces the effective number of parameters which is important even when the dimension of the vectors is smaller than the sample size since the number of parameters grows quadratically in the number of variables and it produces an estimate which is sparse in contrast to sparse inverse covariance estimation our method s close relative the sparsity attained here is in the covariance matrix itself rather than in the inverse matrix zeros in the covariance matrix correspond to marginal independencies thus our method performs model selection while providing a positive definite estimate of the covariance the proposed penalized maximum likelihood problem is not convex so we use a majorize minimize approach in which we iteratively solve convex approximations to the original nonconvex problem we discuss tuning parameter selection and demonstrate on a flow cytometry dataset how our method produces an interpretable graphical display of the relationship between variables we perform simulations that suggest that simple elementwise thresholding of the empirical covariance matrix is competitive with our method for identifying the sparsity structure additionally we show how our method can be used to solve a previously studied special case in which a desired sparsity pattern is prespecified

a well conditioned and sparse estimation of covariance and inverse covariance matrices using a joint penalty we develop a method for estimating well conditioned and sparse covariance and inverse covariance matrices from a sample of vectors drawn from a sub gaussian distribution in high dimensional setting the proposed estimators are obtained by minimizing the quadratic loss function and joint penalty of l1 norm and variance of its eigenvalues in contrast to some of the existing methods of covariance and inverse covariance matrix estimation where often the interest is to estimate a sparse matrix the proposed method is flexible in estimating both a sparse and well conditioned covariance matrix simultaneously the proposed estimators are optimal in the sense that they achieve the mini max rate of estimation in operator norm for the underlying class of covariance and inverse covariance matrices we give a very fast algorithm for computation of these covariance and inverse covariance matrices which is easily scalable to large scale data analysis problems the simulation study for varying sample sizes and variables shows that the proposed estimators performs better than several other estimators for various choices of structured covariance and inverse covariance matrices we also use our proposed estimator for tumor tissues classification using gene expression data and compare its performance with some other classification methods

a multiple testing approach to the regularisation of large sample correlation matrices abstract this paper proposes a regularisation method for the estimation of large covariance matrices that uses insights from the multiple testing m t literature the approach tests the statistical significance of individual pair wise correlations and sets to zero those elements that are not statistically significant taking account of the multiple testing nature of the problem the effective p values of the tests are set as a decreasing function of n the cross section dimension the rate of which is governed by the nature of dependence of the underlying observations and the relative expansion rates of n and t the time dimension in this respect the method specifies the appropriate thresholding parameter to be used under gaussian and non gaussian settings the m t estimator of the sample correlation matrix is shown to be consistent in the spectral and frobenius norms and in terms of support recovery so long as the true covariance matrix is sparse the performance of the proposed m t estimator is compared to a number of other estimators in the literature using monte carlo experiments it is shown that the m t estimator performs well and tends to outperform the other estimators particularly when n is larger than t

debye function expansions of complex permittivity using a hybrid particle swarm least squares optimization approach with appropriate modifications the finite difference time domain fdtd method can be used to analyze propagation through linear isotropic dispersive media although materials characterized by the debye permittivity model can be analyzed accurately and efficiently using well established methods the treatment of other types of frequency dependence is more difficult this paper proposes the use of a weighted sum of debye functions to approximate more general complex permittivity functions a combination of the particle swarm optimization method and linear least squares optimization is used to find the relaxation frequencies and weights in the expansion which can then be accommodated in the fdtd method using one of the established methods two key advantages of the proposed approach are that the relaxation frequencies are bandlimited and the weights are always positive these two characteristics help to maintain the accuracy and stability of the fdtd solution it is also shown that the correlation between the imaginary parts of two debye functions is the same as that between the real parts

characterization of coorbit spaces with phase space covers abstract we show that coorbit spaces can be characterized in terms of arbitrary phase space covers which are families of phase space multipliers associated with partitions of unity this generalizes previously known results for time frequency analysis to include time scale decompositions as a by product we extend the existing results for time frequency analysis to an irregular setting

time domain matched interface and boundary mib modeling of debye dispersive media with curved interfaces a new finite difference time domain fdtd method is introduced for solving transverse magnetic maxwell s equations in debye dispersive media with complex interfaces and discontinuous wave solutions based on the auxiliary differential equation approach a hybrid maxwell debye system is constructed which couples the wave equation for the electric component with maxwell s equations for the magnetic components this hybrid formulation enables the calculation of the time dependent parts of the interface jump conditions so that one can track the transient changes in the regularities of the electromagnetic fields across a dispersive interface effective matched interface and boundary mib treatments are proposed to rigorously impose the physical jump conditions which are not only time dependent but also couple both cartesian directions and both magnetic field components based on a staggered yee lattice the proposed mib scheme can deal with arbitrarily curved interfaces and nonsmooth interfaces with sharped edges second order convergences are numerically achieved in solving dispersive interface problems with constant curvatures general curvatures and nonsmooth corners

gabor schauder bases and the balian low theorem the balian low theorem is a strong form of the uncertainty principle for gabor systems that form orthonormal or riesz bases for l2 r in this paper we investigate the balian low theorem in the setting of schauder bases we prove that new weak versions of the balian low theorem hold for gabor schauder bases but we constructively demonstrate that several variants of the blt can fail for gabor schauder bases that are not riesz bases we characterize a class of gabor schauder bases in terms of the zak transform and product a2 weights the riesz bases correspond to the special case of weights that are bounded away from zero and infinity

wishart distributions for decomposable covariance graph models gaussian covariance graph models encode marginal independence among the components of a multivariate random vector by means of a graph g these models are distinctly different from the traditional concentration graph models often also referred to as gaussian graphical models or covariance selection models since the zeros in the parameter are now reflected in the covariance matrix as compared to the concentration matrix 1 the parameter space of interest for covariance graph models is the cone p g of positive definite matrices with fixed zeros corresponding to the missing edges of g as in letac and massam ann statist 35 2007 1278 1323 we consider the case where g is decomposable in this paper we construct on the cone p g a family of wishart distributions which serve a similar purpose in the covariance graph setting as those constructed by letac and massam ann statist 35 2007 1278 1323 and dawid and lauritzen ann statist 21 1993 1272 1317 do in the concentration graph setting we proceed to undertake a rigorous study of these covariance wishart distributions and derive several deep and useful properties of this class first they form a rich conjugate family of priors with multiple shape parameters for covariance graph models second we show how to sample from these distributions by using a block gibbs sampling algorithm and prove convergence of this block gibbs sampler development of this class of distributions enables bayesian inference which in turn allows for the estimation of e even in the case when the sample size is less than the dimension of the data i e when n p otherwise not generally possible in the maximum likelihood framework third we prove that when g is a homogeneous graph our covariance priors correspond to standard conjugate priors for appropriate directed acyclic graph dag models this correspondence enables closed form expressions for normalizing constants and expected values and also establishes hyper markov properties for our class of priors we also note that when g is homogeneous the family iw qg of letac and massam ann statist 35 2007 1278 1323 is a special case of our covariance wishart distributions fourth and finally we illustrate the use of our family of conjugate priors on real and simulated data

vector valued nonstationary gabor frames abstract as an extension of gabor frames nonstationary gabor nsg frames were recently introduced in adaptive signal analysis they allow for efficient reconstruction with flexible sampling and varying window functions in this paper we generalize the notion of nsg frames from l 2 r to the vector valued hilbert space l 2 r c l and investigate the resulting vector valued nsg frames we derive a walnut s representation of the mixed frame operator and provide some necessary sufficient conditions for a vector valued nsg system to be a frame for l 2 r c l furthermore we show the existence of painless vector valued nsg frames and of vector valued nsg frames with fast decaying window functions

high order accurate fdtd schemes for dispersive maxwell s equations in second order form using recursive convolutions abstract we propose a novel finite difference time domain fdtd scheme for the solution of maxwell s equations in which linear dispersive effects are present the method uses high order accurate approximations in space and time for the dispersive maxwell s equations written as a second order vector wave equation with a time history convolution term the modified equation approach is combined with the recursive convolution rc method to develop high order approximations accurate to any desired order in space and time high order accurate centered approximations of the physical maxwell interface conditions are derived for the dispersive setting in order to fully restore accuracy at discontinuous material interfaces second and fourth order accurate versions of the scheme are presented and implemented in two spatial dimensions for the case of the drude linear dispersion model the stability of these schemes is analyzed finally our approach is also amenable to curvilinear numerical grids if used with an appropriate generalized laplace operator

a fourth order finite difference method for waveguides with curved perfectly conducting boundaries a novel high order finite difference method is introduced for optical waveguides with smoothly curved perfectly electric conducting pec boundaries the proposed method shares some similarities with our previous matched interface and boundary mib methods developed for treating dielectric interfaces of optical waveguides such as the use of a simple cartesian grid the standard finite difference schemes and fictitious values however the pec boundary conditions have a physical nature quite different from that of the jump conditions at the dielectric interfaces i e all six electric and magnetic field components are prescribed in the jump conditions while only three of them are known at the pec walls consequently the previously developed mib methods are not applicable to deal with the perfectly conducting boundaries to overcome this difficulty a novel ray casting fictitious domain method is constructed to enforce the pec conditions along the normal direction such a boundary implementation couples the transverse magnetic field components so that the resulting ray casting mib method is a full vectorial approach for the modal analysis of optical waveguides the new mib method is validated by considering both homogeneous and inhomogeneous waveguides numerical results confirm the designed fourth order of accuracy

matched interface and boundary mib for the implementation of boundary conditions in high order central finite differences high order central finite difference schemes encounter great difficulties in implementing complex boundary conditions this paper introduces the matched interface and boundary mib method as a novel boundary scheme to treat various general boundary conditions in arbitrarily high order central finite difference schemes to attain arbitrarily high order the mib method accurately extends the solution beyond the boundary by repeatedly enforcing only the original set of boundary conditions the proposed approach is extensively validated via boundary value problems initial boundary value problems eigenvalue problems and high order differential equations successful implementations are given to not only dirichlet neumann and robin boundary conditions but also more general ones such as multiple boundary conditions in high order differential equations and time dependent boundary conditions in evolution equations detailed stability analysis of the mib method is carried out the mib method is shown to be able to deliver high order accuracy while maintaining the same or similar stability conditions of the standard high order central difference approximations the application of the proposed mib method to the boundary treatment of other non standard high order methods is also considered

matched interface and boundary mib method for elliptic problems with sharp edged interfaces elliptic problems with sharp edged interfaces thin layered interfaces and interfaces that intersect with geometric boundary are notoriously challenging to existing numerical methods particularly when the solution is highly oscillatory this work generalizes the matched interface and boundary mib method previously designed for solving elliptic problems with curved interfaces to the aforementioned problems we classify these problems into five distinct topological relations involving the interfaces and the cartesian mesh lines flexible strategies are developed to systematically extends the computational domains near the interface so that the standard central finite difference scheme can be applied without the loss of accuracy fictitious values on the extended domains are determined by enforcing the physical jump conditions on the interface according to the local topology of the irregular point the concepts of primary and secondary fictitious values are introduced to deal with sharp edged interfaces for corner singularity or tip singularity an appropriate polynomial is multiplied to the solution to remove the singularity extensive numerical experiments confirm the designed second order convergence of the proposed method

a new approach to cholesky based covariance regularization in high dimensions summary in this paper we propose a new regression interpretation of the cholesky factor of the covariance matrix as opposed to the well known regression interpretation of the cholesky factor of the inverse covariance which leads to a new class of regularized covariance estimators suitable for high dimensional problems regularizing the cholesky factor of the covariance via this regression interpretation always results in a positive definite estimator in particular one can obtain a positive definite banded estimator of the covariance matrix at the same computational cost as the popular banded estimator of bickel levina 2008b which is not guaranteed to be positive definite we also establish theoretical connections between banding cholesky factors of the covariance matrix and its inverse and constrained maximum likelihood estimation under the banding constraint and compare the numerical performance of several methods in simulations and on a sonar data example

the concave convex procedure the concave convex procedure cccp is a way to construct discrete time iterative dynamical systems that are guaranteed to decrease global optimization and energy functions monotonically this procedure can be applied to almost any optimization problem and many existing algorithms can be interpreted in terms of it in particular we prove that all expectation maximization algorithms and classes of legendre minimization and variational bounding algorithms can be reexpressed in terms of cccp we show that many existing neural network and mean field theory algorithms are also examples of cccp the generalized iterative scaling algorithm and sinkhorn s algorithm can also be expressed as cccp by changing variables cccp can be used both as a new way to understand and prove the convergence of existing optimization algorithms and as a procedure for generating new algorithms

rational time frequency super gabor frames and their duals abstract this paper addresses general super gabor systems with rational time frequency product lattices a zak transform matrix method is developed for such super gabor systems we characterize complete super gabor systems and super gabor frames given a super gabor frame we obtain an explicit expression of its canonical dual and a parametrization of all its super gabor duals and prove that the canonical dual is the norm minimal one among all super gabor duals we also prove that a super gabor frame has a unique super gabor dual if and only if it is a riesz basis

oversampling in the gabor scheme a method for calculating the coefficients of the gabor expansion in the case of oversampling is presented the method is based on the concept of frames and utilizes the zak transform as such the zak transform highlights the meaning and importance of frames in the context of oversampling and other aspects of signal representation by means of nonorthogonal bases

high order matched interface and boundary method for elliptic equations with discontinuous coefficients and singular sources this paper introduces a novel high order interface scheme the matched interface and boundary mib method for solving elliptic equations with discontinuous coefficients and singular sources on cartesian grids by appropriate use of auxiliary line and or fictitious points physical jump conditions are enforced at the interface unlike other existing interface schemes the proposed method disassociates the enforcement of physical jump conditions from the discretization of the differential equation under study to construct higher order interface schemes the proposed mib method bypasses the major challenge of implementing high order jump conditions by repeatedly enforcing the lowest order jump conditions the proposed mib method is of arbitrarily high order in principle in treating straight regular interfaces we construct mib schemes up to 16th order for more general elliptic problems with curved irregular interfaces and boundary up to 6th order mib schemes have been demonstrated by employing the standard high order finite difference schemes to discretize the laplacian the present mib method automatically reduces to the standard central difference scheme when the interface is absent the immersed interface method iim is regenerated for a comparison study of the proposed method the robustness of the mib method is verified against the large magnitude of the jump discontinuity across the interface the nature of high efficiency and low memory requirement of the mib method is extensively validated via solving various elliptic immersed interface problems in two and three dimensions ee dimensions

the existence of tight gabor duals for gabor frames and subspace gabor frames abstract let k and l be two full rank lattices in r d we give a complete characterization for all the gabor frames that admit tight dual of the same type the characterization is given in terms of the center valued trace of the von neumann algebra generated by the left regular projective unitary representations associated with the time frequency lattice k l two applications of this characterization were obtained i we are able to prove that every gabor frame has a tight dual if and only if the volume of k l is less than or equal to 1 2 ii we are able to obtain sufficient or necessary conditions for the existence of tight gabor pseudo duals for subspace gabor frames in various cases in particular we prove that every subspace gabor frame has a tight gabor pseudo dual if either the volume v k l 1 2 or v k l 2 moreover if k z d l z d with 1 then a subspace gabor frame g g l k has a tight gabor pseudo dual only when g g l k itself is already tight

superconvergence analysis of nonconforming mixed finite element methods for time dependent maxwell s equations in isotropic cold plasma media in this paper based on our previous work of asymptotic expansion an initial effort of global superconvergence analysis of time domain maxwell s equations in isotropic cold plasma media is obtained for semi discrete mixed finite element scheme during this procedure superclose analysis of the interpolation of the smoothing solution and the solution of mixed finite element method is obtained for the first time by the post processing operators superconvergence phenomena can be found to our best knowledge this is the first global superconvergence analysis presented by nonconforming mixed finite element methods when isotropic cold plasma media is involved

variable selection using mm algorithms variable selection is fundamental to high dimensional statistical modeling many variable selection techniques may be implemented by maximum penalized likelihood using various penalty functions optimizing the penalized likelihood function is often challenging because it may be nondifferentiable and or nonconcave this article proposes a new class of algorithms for finding a maximizer of the penalized likelihood for a broad class of penalty functions these algorithms operate by perturbing the penalty function slightly to render it differentiable then optimizing this differentiable function using a minorize maximize mm algorithm mm algorithms are useful extensions of the well known class of em algorithms a fact that allows us to analyze the local and global convergence of the proposed algorithm using some of the techniques employed for em algorithms in particular we prove that when our mm algorithms converge they must converge to a desirable point we also discuss conditions under which this convergence may be guaranteed we exploit the newton raphson like aspect of these algorithms to propose a sandwich estimator for the standard errors of the estimators our method performs well in numerical tests

estimating the error variance in a high dimensional linear model the lasso has been studied extensively as a tool for estimating the coefficient vector in the high dimensional linear model however considerably less is known about estimating the error variance in this context in this paper we propose the natural lasso estimator for the error variance which maximizes a penalized likelihood objective a key aspect of the natural lasso is that the likelihood is expressed in terms of the natural parameterization of the multiparameter exponential family of a gaussian with unknown mean and variance the result is a remarkably simple estimator of the error variance with provably good performance in terms of mean squared error these theoretical results do not require placing any assumptions on the design matrix or the true regression coefficients we also propose a companion estimator called the organic lasso which theoretically does not require tuning of the regularization parameter both estimators do well empirically compared to preexisting methods especially in settings where successful recovery of the true support of the coefficient vector is hard finally we show that existing methods can do well under fewer assumptions than previously known thus providing a fuller story about the problem of estimating the error variance in high dimensional linear models

sampling theorems on locally compact groups from oscillation estimates we present a general approach to derive sampling theorems on locally compact groups from oscillation estimates we focus on the l2 stability of the sampling operator by using notions from frame theory this approach yields particularly simple and transparent reconstruction procedures we then apply these methods to the discretization of discrete series representations and to paley wiener spaces on stratified lie groups

model selection and estimation in the gaussian graphical model summary we propose penalized likelihood methods for estimating the concentration matrix in the gaussian graphical model the methods lead to a sparse and shrinkage estimator of the concentration matrix that is positive definite and thus conduct model selection and estimation simultaneously the implementation of the methods is nontrivial because of the positive definite constraint on the concentration matrix but we show that the computation can be done effectively by taking advantage of the efficient maxdet algorithm developed in convex optimization we propose a bic type criterion for the selection of the tuning parameter in the penalized likelihood methods the connection between our methods and existing methods is illustrated simulations and real examples demonstrate the competitive performance of the new methods

a p q version of bourgain s theorem let 1 p q satisfy 1 p 1 q 1 we construct an orthonormal basis b n for l 2 r such that p b n and q bn are both uniformly bounded in n here f inf r x a f x 2 dx 1 2 this generalizes a theorem of bourgain and is closely related to recent results on the balian low theorem

a new high order dispersive fdtd method for drude material with complex interfaces in this paper motivated by the needs of tracking the transient change in the regularity of the electromagnetic fields across a drude interface we propose a new maxwell drude formulation for transverse magnetic problems with inhomogeneous drude dispersive materials based on the auxiliary differential equation approach the proposed formulation couples the wave equation for the electric component with maxwell s equations for the magnetic components a new finite difference time domain fdtd algorithm is introduced for solving the proposed maxwell drude system in which the time dependent jump conditions across the drude interface are enforced through the matched interface and boundary mib method the proposed fdtd method achieves a second order of accuracy in solving drude interfaces with fluctuating curvatures and non smooth corners based on a simple yee lattice while it can be generalized to up to sixth order in dealing with a straight drude interface therefore the proposed fdtd method is more accurate and cost efficient than the classical fdtd methods for drude material with complex interfaces

dilation of the weyl symbol and balian low theorem the key result of this paper describes the fact that for an important class of pseudodifferential operators the property of invertibility is preserved under minor dilations of their weyl symbols this observation has two implications in time frequency analysis first it implies the stability of general gabor frames under small dilations of the time frequency set previously known only for the case that the time frequency set is a lattice secondly it allows us to derive a new balian low theorem blt for gabor systems with window in the standard window class and with general time frequency families in contrast to the classical versions of blt the new blt does not only exclude orthonormal bases and riesz bases at critical density but indeed it even excludes irregular gabor frames at critical density

discontinuous galerkin methods for dispersive and lossy maxwell s equations and pml boundary conditions in this paper we will present a unified formulation of discontinuous galerkin method dgm for maxwell s equations in linear dispersive and lossy materials of debye type and in the artificial perfectly matched layer pml regions an auxiliary differential equation ade method is used to handle the frequency dependent constitutive relations with the help of auxiliary polarization currents in the computational and pml regions the numerical flux for the dispersive lossy maxwell s equations with the auxiliary polarization current variables is derived various numerical results are provided to validate the proposed formulation

partial covariance based functional connectivity computation using ledoit wolf covariance regularization abstract functional connectivity refers to shared signals among brain regions and is typically assessed in a task free state functional connectivity commonly is quantified between signal pairs using pearson correlation however resting state fmri is a multivariate process exhibiting a complicated covariance structure partial covariance assesses the unique variance shared between two brain regions excluding any widely shared variance hence is appropriate for the analysis of multivariate fmri datasets however calculation of partial covariance requires inversion of the covariance matrix which in most functional connectivity studies is not invertible owing to rank deficiency here we apply ledoit wolf shrinkage l2 regularization to invert the high dimensional bold covariance matrix we investigate the network organization and brain state dependence of partial covariance based functional connectivity although rsns are conventionally defined in terms of shared variance removal of widely shared variance surprisingly improved the separation of rsns in a spring embedded graphical model this result suggests that pair wise unique shared variance plays a heretofore unrecognized role in rsn covariance organization in addition application of partial correlation to fmri data acquired in the eyes open vs eyes closed states revealed focal changes in uniquely shared variance between the thalamus and visual cortices this result suggests that partial correlation of resting state bold time series reflect functional processes in addition to structural connectivity

vector valued gabor frames associated with periodic subsets of the real line the notion of vector valued frame also called superframe was first introduced by balan in the context of multiplexing it has significant applications in mobile communication satellite communication and computer area network for vector valued gabor analysis existent literatures mostly focus on l 2 r c l instead of its subspace let a 0 and s be an a z periodic measurable set in r i e s a z s this paper addresses gabor frames in l 2 s c l with rational time frequency product they can model vector valued signals to appear periodically but intermittently and the projections of gabor frames in l 2 r c l onto l 2 s c l cannot cover all gabor frames in l 2 s c l if s r by introducing a suitable zak transform matrix we characterize completeness and frame condition of gabor systems obtain a necessary and sufficient condition on gabor duals of type i resp ii for a general gabor frame and establish a parametrization expression of gabor duals of type i resp ii all our conclusions are closely related to corresponding zak transform matrices this allows us to easily realize these conclusions by designing the corresponding matrix valued functions an example theorem is also presented to illustrate the efficiency of our method

sparsistency and rates of convergence in large covariance matrix estimation this paper studies the sparsistency and rates of convergence for estimating sparse covariance and precision matrices based on penalized likelihood with nonconvex penalty functions here sparsistency refers to the property that all parameters that are zero are actually estimated as zero with probability tending to one depending on the case of applications sparsity priori may occur on the covariance matrix its inverse or its cholesky decomposition we study these three sparsity exploration problems under a unified framework with a general penalty function we show that the rates of convergence for these problems under the frobenius norm are of order s n log p n n 1 2 where s n is the number of nonzero elements p n is the size of the covariance matrix and n is the sample size this explicitly spells out the contribution of high dimensionality is merely of a logarithmic factor the conditions on the rate with which the tuning parameter n goes to 0 have been made explicit and compared under different penalties as a result for the l 1 penalty to guarantee the sparsistency and optimal rate of convergence the number of nonzero elements should be small sn o pn at most among o pn2 parameters for estimating sparse covariance or correlation matrix sparse precision or inverse correlation matrix or sparse cholesky factor where sn is the number of the nonzero elements on the off diagonal entries on the other hand using the scad or hard thresholding penalty functions there is no such a restriction

sparse covariance matrix estimation by dca based algorithms this letter proposes a novel approach using the formula see text norm regularization for the sparse covariance matrix estimation scme problem the objective function of scme problem is composed of a nonconvex part and the formula see text term which is discontinuous and difficult to tackle appropriate dc difference of convex functions approximations of formula see text norm are used that result in approximation scme problems that are still nonconvex dc programming and dca dc algorithm powerful tools in nonconvex programming framework are investigated two dc formulations are proposed and corresponding dca schemes developed two applications of the scme problem that are considered are classification via sparse quadratic discriminant analysis and portfolio optimization a careful empirical experiment is performed through simulated and real data sets to study the performance of the proposed algorithms numerical results showed their efficiency and their superiority compared with seven state of the art methods

high order fdtd methods for transverse magnetic modes with dispersive interfaces a new finite difference time domain fdtd algorithm is introduced to solve two dimensional 2d transverse magnetic tm modes with a straight dispersive interface driven by the consideration of simplifying interface jump conditions the auxiliary differential equation of the debye constitution model is rewritten to form a new debye maxwell tm system interface auxiliary differential equations are utilized to describe the transient changes in the regularities of electromagnetic fields across a dispersive interface the resulting time dependent jump conditions are rigorously enforced in the fdtd discretization by means of a matched interface and boundary scheme higher order convergences are numerically achieved for the first time in the literature in 2d fdtd simulations of dispersive inhomogeneous media

the wavelet transform time frequency localization and signal analysis two different procedures for effecting a frequency analysis of a time dependent signal locally in time are studied the first procedure is the short time or windowed fourier transform the second is the wavelet transform in which high frequency components are studied with sharper time resolution than low frequency components the similarities and the differences between these two methods are discussed for both schemes a detailed study is made of the reconstruction method and its stability as a function of the chosen time frequency density finally the notion of time frequency localization is made precise within this framework by two localization theorems

the duals of gabor frames on discrete periodic sets due to its good potential for digital signal processing analysis the discrete version of gabor analysis has interested many mathematicians in this paper we focus on gabor analysis on discrete periodic sets which can model a signal to appear periodically but intermittently for a gabor frame on a discrete periodic set three types of dual frames with gabor structure are introduced and investigated in terms of discrete zak transform given a gabor frame on a discrete periodic set a characterization of a gabor system being separately its dual of types i iii is obtained a method to construct these three types of duals is presented an explicit expression of its canonical dual is established the uniqueness of these three types of duals is characterized some examples are also provided to illustrate the generality of the theory

positive definite estimators of large covariance matrices using convex optimization we construct a sparse estimator of the covariance matrix that is positive definite and performs well in high dimensional settings a lasso type penalty is used to encourage sparsity and a logarithmic barrier function is used to enforce positive definiteness consistency and convergence rate bounds are established as both the number of variables and sample size diverge an efficient computational algorithm is developed and the merits of the approach are illustrated with simulations and a speech signal classification example copyright 2012 oxford university press

efficient estimation of approximate factor models via penalized maximum likelihood we study the estimation of a high dimensional approximate factor model in the presence of both cross sectional dependence and heteroskedasticity the classical method of principal components analysis pca does not efficiently estimate the factor loadings or common factors because it essentially treats the idiosyncratic error to be homoskedastic and cross sectionally uncorrelated for the efficient estimation it is essential to estimate a large error covariance matrix we assume the model to be conditionally sparse and propose two approaches to estimating the common factors and factor loadings both are based on maximizing a gaussian quasi likelihood and involve regularizing a large covariance sparse matrix in the first approach the factor loadings and the error covariance are estimated separately while in the second approach they are estimated jointly extensive asymptotic analysis has been carried out in particular we develop the inferential theory for the two step estimation because the proposed approaches take into account the large error covariance matrix they produce more efficient estimators than the classical pca methods or methods based on a strict factor model

density and duality theorems for regular gabor frames abstract we investigate gabor frames on locally compact abelian groups with time frequency shifts along non separable closed subgroups of the phase space density theorems in gabor analysis state necessary conditions for a gabor system to be a frame or a riesz basis formulated only in terms of the index subgroup in the classical results the subgroup is assumed to be discrete we prove density theorems for general closed subgroups of the phase space where the necessary conditions are given in terms of the size of the subgroup from these density results we are able to extend the classical wexler raz biorthogonal relations and the duality principle in gabor analysis to gabor systems with time frequency shifts along non separable closed subgroups of the phase space even in the euclidean setting our results are new

the structure of shift modulation invariant spaces the rational case in this paper we study structural properties of shift modulation invariant smi spaces also called gabor subspaces or weyl heisenberg subspaces in the case when shift and modulation lattices are rationally dependent we prove the characterization of smi spaces in terms of range functions analogous to the well known description of shift invariant spaces c de boor r devore a ron the structure of finitely generated shift invariant spaces in l2 r d j funct anal 119 1994 37 78 m bownik the structure of shift invariant subspaces of l2 r n j funct anal 177 2000 282 309 h helson lectures on invariant subspaces academic press new york london 1964 we also give a simple characterization of frames and riesz sequences in terms on their behavior of the fibers of the range function next we prove several orthogonal decomposition results of smi spaces into simpler blocks called principal smi spaces then this is used to characterize operators invariant under both shifts and modulations in terms of families of linear maps acting on the fibers of the range function we also introduce the fundamental concept of the dimension function for smi spaces as a result this leads to the classification of unitarily equivalent smi spaces in terms of their dimension functions finally we show several results illustrating our fiberization techniques to characterize dual gabor frames

generalized thresholding of large covariance matrices we propose a new class of generalized thresholding operators that combine thresholding with shrinkage and study generalized thresholding of the sample covariance matrix in high dimensions generalized thresholding of the covariance matrix has good theoretical properties and carries almost no computational burden we obtain an explicit convergence rate in the operator norm that shows the tradeoff between the sparsity of the true model dimension and the sample size and shows that generalized thresholding is consistent over a large class of models as long as the dimension p and the sample size n satisfy log p n 0 in addition we show that generalized thresholding has the sparsistency property meaning it estimates true zeros as zeros with probability tending to 1 and under an additional mild condition is sign consistent for nonzero elements we show that generalized thresholding covers as special cases hard and soft thresholding smoothly clipped absolute deviation and adaptive lasso and comp

model selection through sparse maximum likelihood estimation for multivariate gaussian or binary data we consider the problem of estimating the parameters of a gaussian or binary distribution in such a way that the resulting undirected graphical model is sparse our approach is to solve a maximum likelihood problem with an added l1 norm penalty term the problem as formulated is convex but the memory requirements and complexity of existing interior point methods are prohibitive for problems with more than tens of nodes we present two new algorithms for solving problems with at least a thousand nodes in the gaussian case our first algorithm uses block coordinate descent and can be interpreted as recursive l1 norm penalized regression our second algorithm based on nesterov s first order method yields a complexity estimate with a better dependence on problem size than existing interior point methods using a log determinant relaxation of the log partition function wainwright and jordan 2006 we show that these same algorithms can be used to solve an approximate sparse maximum likelihood problem for the binary case we test our algorithms on synthetic data as well as on gene expression and senate voting records data

the analysis and design of windowed fourier frame based multiple description source coding schemes in this paper the windowed fourier encoding decoding scheme applied to the multiple description compression problem is analyzed in the general case four window functions are needed to define the encoder and decoder although this number can be reduced to three or two by using time shift or frequency shift division schemes the encoding coefficients are next divided into two groups according to the eveness of either the modulation or translation index the distortion on each channel is analyzed using the zak transform for the optimal windows explicit representation formulas are obtained and nonlocalization results are proved asymptotic formulas of the total distortion and transmission rate are established and the redundancy is shown to trade off between these two