pseudo skeleton approximations with better accuracy estimates abstract we propose a priori accuracy estimates for low rank matrix approximations that use just a small number of the rows and columns this number is greater than the approximation rank unlike the existing methods of pseudo skeleton approximation but the estimates are more accurate than previously known ones this paper generalizes the results of 12 13

randomized qlp decomposition abstract we present the randomized qlp rqlp algorithm and its enhanced version erqlp for computing a low rank approximation to a matrix a of size m n such that a q l p t where l is a rank k lower triangular matrix q and p are column orthogonal matrices the implementation of rqlp and erqlp needs o m n k flops which are mainly applied in blas 3 operations we derive the lower bounds on the l values which can track the singular values of a approximately our claims are supported by numerical experiments

efficient scalable algorithms for hierarchically semiseparable matrices hierarchically semiseparable hss matrix algorithms are emerging techniques in constructing the superfast direct solvers for both dense and sparse linear systems here we develope a set of novel parallel algorithms for the key hss operations that are used for solving large linear systems these include the parallel rank revealing qr factorization the hss constructions with hierarchical compression the ulv hss factorization and the hss solutions the hss tree based parallelism is fully exploited at the coarse level the blacs and scalapack libraries are used to facilitate the parallel dense kernel operations at the ne grained level we have appplied our new parallel hss embedded multifrontal solver to the anisotropic helmholtz equations for seismic imaging and were able to solve a linear system with 6 4 billion unknowns using 4096 processors in about 20 minutes the classical multifrontal solver simply failed due to high demand of memory to our knowledge this is the first successful demonstration of employing the hss algorithms in solving the truly large scale real world problems our parallel strategies can be easily adapted to the parallelization of the other rank structured methods

a fast direct solver for quasi periodic scattering problems abstract we consider the numerical solution of the scattering of time harmonic plane waves from an infinite periodic array of reflection or transmission obstacles in a homogeneous background medium in two dimensions boundary integral formulations are ideal since they reduce the problem to n unknowns on the obstacle boundary however for complex geometries and or higher frequencies the resulting dense linear system becomes large ruling out dense direct methods and often ill conditioned despite being 2nd kind rendering fast multipole based iterative schemes also inefficient we present an integral equation based solver with o n complexity which handles such ill conditioning using recent advances in fast direct linear algebra to invert hierarchically the isolated obstacle matrix this is combined with a recent periodizing scheme that is robust for all incident angles including wood s anomalies based upon the free space green s function kernel the resulting solver is extremely efficient when multiple incident angles are needed as occurs in many applications our numerical tests include a complicated obstacle several wavelengths in size with n 10 5 and solution error of 10 10 where the solver is 66 times faster per incident angle than a fast multipole based iterative solution and 600 times faster when incident angles are chosen to share bloch phases

accurate solution of polynomial equations using macaulay resultant matrices we propose an algorithm for solving two polynomial equations in two variables our algorithm is based on the macaulay resultant approach combined with new techniques including randomization to make the algorithm accurate in the presence of roundoff error the ultimate computation is the solution of a generalized eigenvalue problem via the qz method we analyze the error due to roundoff of the method showing that with high probability the roots are computed accurately assuming that the input data that is the two polynomials are well conditioned our analysis requires a novel combination of algebraic and numerical techniques

convergence to diagonal form of block jacobi type methods we provide sufficient conditions for the general sequential block jacobi type method to converge to the diagonal form for cyclic pivot strategies which are weakly equivalent to the column cyclic strategy given a block matrix partition a ij a i j of a square matrix mathbf a a the paper analyzes the iterative process of the form mathbf a k 1 mathbf p k mathbf a k mathbf q k a k 1 p k a k q k k ge 0 k 0 mathbf a 0 mathbf a a 0 a where mathbf p k p k and mathbf q k q k are elementary block matrices which differ from the identity matrix in four blocks two diagonal and the two corresponding off diagonal blocks in our analysis of convergence a promising new tool is used namely the theory of block jacobi operators typical applications lie in proving the global convergence of block jacobi type methods for solving standard and generalized eigenvalue and singular value problems

constructing nested bases approximations from the entries of non local operators in this article a method for constructing nested bases approximations to large scale fully populated discretizations of integral operators is introduced the scheme uses only few of the matrix entries for approximating the whole matrix in this sense it is similar to the adaptive cross approximation method however its computational complexity is improved

new studies of randomized augmentation and additive preprocessing abstract a standard gaussian random matrix hereafter referred to just as gaussian matrix has full rank with probability 1 and is well conditioned with a probability quite close to 1 and converging to 1 fast as the matrix deviates from the square shape and becomes more rectangular if we append sufficiently many gaussian rows or columns to any normalized and possibly rank deficient or ill conditioned matrix then the augmented matrix has full rank with probability 1 and is well conditioned with a probability close to 1 we specify and prove these properties of augmentation and extend them to additive preprocessing that is to adding a product of two rectangular gaussian matrices by applying our randomization techniques to a matrix that has numerical rank r we accelerate the known algorithms for the approximation of its trailing singular spaces associated with all its positive singular values except for the r largest values our algorithms use much fewer random parameters and run much faster when various random sparse and structured preprocessors replace gaussian empirically the outputs of the resulting algorithms are as accurate as the outputs under gaussian preprocessing our novel duality techniques provides formal support so far missing for these empirical observations and opens door to de randomization of our preprocessing and to further acceleration and simplification of our algorithms by using more efficient sparse and structured preprocessors our techniques and our progress can be applied to various other fundamental matrix computations such as the celebrated low rank approximation of a matrix by means of random sampling

randomized algorithms for the low rank approximation of matrices we describe two recently proposed randomized algorithms for the construction of low rank approximations to matrices and demonstrate their application inter alia to the evaluation of the singular value decompositions of numerically low rank matrices being probabilistic the schemes described here have a finite probability of failure in most cases this probability is rather negligible 10 17 is a typical value in many situations the new procedures are considerably more efficient and reliable than the classical deterministic ones they also parallelize naturally we present several numerical examples to illustrate the performance of the schemes

a deim induced cur factorization we derive a cur approximate matrix factorization based on the discrete empirical interpolation method deim for a given matrix bf a such a factorization provides a low rank approximate decomposition of the form bf a approx bf c bf u bf r where bf c and bf r are subsets of the columns and rows of bf a and bf u is constructed to make bf c bf u bf r a good approximation given a low rank singular value decomposition bf a approx bf v bf s bf w t the deim procedure uses bf v and bf w to select the columns and rows of bf a that form bf c and bf r through an error analysis applicable to a general class of cur factorizations we show that the accuracy tracks the optimal approximation error within a factor that depends on the conditioning of submatrices of bf v and bf w for very large problems bf v and bf w can be approximated well using an incremental qr algorithm that makes only one pass through bf a numer

rank revealing gaussian elimination by the maximum volume concept abstract a gaussian elimination algorithm is presented that reveals the numerical rank of a matrix and identifies a square nonsingular submatrix of maximum dimension the bounds on the singular values of the submatrix and its schur complement are similar to the best known bounds for rank revealing lu factorization but in contrast to existing methods the algorithm does not make use of the normal matrix an implementation for dense matrices is described whose computational cost is roughly twice the cost of an lu factorization with complete pivoting because of its flexibility in choosing pivot elements the algorithm is amenable to implementation with blocked memory access and for sparse matrices

reduction of multivariate mixtures and its applications abstract we consider fast deterministic algorithms to identify the best linearly independent terms in multivariate mixtures and use them to compute up to a user selected accuracy an equivalent representation with fewer terms importantly the multivariate mixtures do not have to be a separated representation of a function one algorithm employs a pivoted cholesky decomposition of the gram matrix constructed from the terms of the mixture to select what we call skeleton terms and the other uses orthogonalization for the same purpose both algorithms require o r 2 n p d r n operations where n is the initial number of terms in the multivariate mixture r is the number of selected linearly independent terms and p d is the cost of computing the inner product between two terms of a mixture in d variables for general gaussian mixtures p d d 3 since we need to diagonalize a d d matrix whereas for separated representations p d d there is no need for diagonalization due to conditioning issues the resulting accuracy is limited to about one half of the available significant digits for both algorithms we also describe an alternative algorithm that is capable of achieving higher accuracy but is only applicable in low dimensions or to multivariate mixtures in separated form we describe a number of initial applications of these algorithms to solve partial differential and integral equations and to address several problems in data science for data science applications in high dimensions we consider the kernel density estimation kde approach for constructing a probability density function pdf of a cloud of points a far field kernel summation method and the construction of equivalent sources for non oscillatory kernels used in both computational physics and data science and finally show how to use the new algorithm to produce seeds for subdividing a cloud of points into groups

a low rank control variate for multilevel monte carlo simulation of high dimensional uncertain systems abstract multilevel monte carlo mlmc is a recently proposed variation of monte carlo mc simulation that achieves variance reduction by simulating the governing equations on a series of spatial or temporal grids with increasing resolution instead of directly employing the fine grid solutions mlmc estimates the expectation of the quantity of interest from the coarsest grid solutions as well as differences between each two consecutive grid solutions when the differences corresponding to finer grids become smaller hence less variable fewer mc realizations of finer grid solutions are needed to compute the difference expectations thus leading to a reduction in the overall work this paper presents an extension of mlmc referred to as multilevel control variates mlcv where a low rank approximation to the solution on each grid obtained primarily based on coarser grid solutions is used as a control variate for estimating the expectations involved in mlmc cost estimates as well as numerical examples are presented to demonstrate the advantage of this new mlcv approach over the standard mlmc when the solution of interest admits a low rank approximation and the cost of simulating finer grids grows fast

fast linear algebra is stable in demmel et al numer math 106 2 199 224 2007 we showed that a large class of fast recursive matrix multiplication algorithms is stable in a normwise sense and that in fact if multiplication of n by n matrices can be done by any algorithm in o n operations for any 0 then it can be done stably in o n operations for any 0 here we extend this result to show that essentially all standard linear algebra operations including lu decomposition qr decomposition linear equation solving matrix inversion solving least squares problems generalized eigenvalue problems and the singular value decomposition can also be done stably in a normwise sense in o n operations

mixed h and passive control for singular systems with time delay via static output feedback in this paper the mixed h and passive control problem is studied for singular systems with time delay a sufficient condition is proposed to ensure the considered system to be regular impulse free and stable with mixed h and passivity performance level then the criterion is adopted to design a static output feedback controller in terms of bilinear and linear matrix inequalities an algorithm is established to cast the nonlinear feasibility problem into a sequential minimization problem finally a numerical example is presented to show the effectiveness of the proposed methods

a newton method for rigid body frictional impact with multiple simultaneous impact points in this paper a mathematical formulation and a numerical algorithm for the analysis of frictional impact of rigid bodies with multiple simultaneous contact points are developed a direct approach is used where the impenetrability condition and coulomb s law of friction are formulated as equations which are not differentiable in the usual sense and solved together with the equations of motion and necessary kinematical relations using newton s method an experiment has also been performed where a possible mechanism for the difference between static and dynamic friction is observed and results are compared with computations using the present algorithm

finding structure with randomness probabilistic algorithms for constructing approximate matrix decompositions low rank matrix approximations such as the truncated singular value decomposition and the rank revealing qr decomposition play a central role in data analysis and scientific computing this work surveys and extends recent research which demonstrates that randomization offers a powerful tool for performing low rank matrix approximation these techniques exploit modern computational architectures more fully than classical methods and open the possibility of dealing with truly massive data sets this paper presents a modular framework for constructing randomized algorithms that compute partial matrix decompositions these methods use random sampling to identify a subspace that captures most of the action of a matrix the input matrix is then compressed either explicitly or implicitly to this subspace and the reduced matrix is manipulated deterministically to obtain the desired low rank factorization in many cases this approach beats its classical competitors in terms of accuracy robustness and or speed these claims are supported by extensive numerical experiments and a detailed error analysis the specific benefits of randomized techniques depend on the computational environment consider the model problem of finding the k dominant components of the singular value decomposition of an m times n matrix i for a dense input matrix randomized algorithms require bigo mn log k floating point operations flops in contrast to bigo mnk for classical algorithms ii for a sparse input matrix the flop count matches classical krylov subspace methods but the randomized approach is more robust and can easily be reorganized to exploit multiprocessor architectures iii for a matrix that is too large to fit in fast memory the randomized techniques require only a constant number of passes over the data as opposed to bigo k passes for classical algorithms in fact it is sometimes possible to perform matrix approximation with a single pass over the data

accurate computation of the product induced singular value decomposition with applications we present a new algorithm for floating point computation of the singular value decomposition svd of the product b tau c where b and c are full row rank matrices the algorithm replaces the pair b c with an equivalent pair b c and then it uses the jacobi svd algorithm to compute the svd of the explicitly computed matrix b tau c in this way each nonzero singular value sigma is approximated with some sigma delta sigma where the relative error delta sigma sigma is up to a factor of the dimensions of order boldmath varepsilon unboldmath min delta in cal d kappa 2 delta b min delta in cal d kappa 2 delta c where cal d denotes the set of diagonal nonsingular matrices kappa 2 cdot denotes the spectral condition number and boldmath varepsilon unboldmath is the roundoff unit of floating point arithmetic the new algorithm is applied to the eigenvalue problem hm x lambda x with symmetric positive definite h and m it is shown that each eigenvalue lambda is computed with high relative accuracy and that the relative error delta lambda lambda of the computed approximation lambda delta lambda is up to a factor of the dimension of order boldmath varepsilon unboldmath min delta in cal d kappa 2 delta h delta min delta in cal d kappa 2 delta m delta the new algorithm can also be used for accurate svd computation of a single matrix g that admits an accurate factorization g b tau c

on the existence and computation of rank revealing lu factorizations abstract by exploring properties of schur complements this paper presents bounds on the existence of rank revealing lu factorizations that are comparable with those of rank revealing qr factorizations the new bounds provide substantial improvement over previously derived bounds this paper also proposes two algorithms using gaussian elimination with a block pivoting strategy to select a subset of columns from a given matrix which has a guaranteed relatively large smallest singular value each of these two algorithms is faster than its orthogonal counterpart for dense matrices if implemented appropriately these algorithms are faster than the corresponding rank revealing qr methods even when the orthogonal matrices are not explicitly updated based on these two algorithms an algorithm using only gaussian elimination for computing rank revealing lu factorizations is introduced

sampling methods for the nystrom method the nystrom method is an efficient technique to generate low rank matrix approximations and is used in several large scale learning applications a key aspect of this method is the procedure according to which columns are sampled from the original matrix in this work we explore the efficacy of a variety of fixed and adaptive sampling schemes we also propose a family of ensemble based sampling algorithms for the nystrom method we report results of extensive experiments that provide a detailed comparison of various fixed and adaptive sampling techniques and demonstrate the performance improvement associated with the ensemble nystrom method when used in conjunction with either fixed or adaptive sampling schemes corroborating these empirical findings we present a theoretical analysis of the nystrom method providing novel error bounds guaranteeing a better convergence rate of the ensemble nystrom method in comparison to the standard nystrom method

a parallel butterfly algorithm the butterfly algorithm is a fast algorithm which approximately evaluates a discrete analogue of the integral transform int mathbb r d k x y g y dy at large numbers of target points when the kernel k x y is approximately low rank when restricted to subdomains satisfying a certain simple geometric condition in d dimensions with o n d quasi uniformly distributed source and target points when each appropriate submatrix of k is approximately rank r the running time of the algorithm is at most o r 2 n d log n a parallelization of the butterfly algorithm is introduced which assuming a message latency of alpha and per process inverse bandwidth of beta executes in at most o r 2 frac n d p log n beta r frac n d p alpha log p time using p processes this parallel algorithm was then instantiated in the form of the open source textttdistbutterfly library for the special case where k x y exp i phi x y where phi x y is a black box sufficiently smooth

efficient scalable parallel higher order direct mom sie method with hierarchically semiseparable structures for 3 d scattering a novel fast scalable parallel algorithm is proposed for the solution of large 3 d scattering problems based on 1 the double geometrical and current approximation higher order dho method of moments mom in the surface integral equation sie formulation and 2 a direct solver for dense linear systems utilizing hierarchically semiseparable hss structures namely an hss matrix representation is used for compression factorization and solution of the system matrix in addition a rank revealing qr decomposition for memory compression is used with a stopping criterion in terms of the relative rank tolerance value a method for geometrical preprocessing of the scatterers based on the cobblestone distance sorting technique is employed in order to enhance the hss algorithm accuracy and parallelization numerical examples show how the accuracy of the dho hss mom sie method is easily controllable by using the relative tolerance for the matrix compression moreover the examples demonstrate low memory consumption as well as much faster simulation time when compared to the direct lu decomposition the method enables dramatically faster monostatic scattering computations than iterative solvers and reduced number of unknowns when compared to low order discretizations finally great scalability of the algorithm is demonstrated on more than one thousand processes

efficient low rank approximation of matrices based on randomized pivoted decomposition given a matrix bf a with numerical rank k the two sided orthogonal decomposition tsod computes a factorization bf a bf udv t where bf u and bf v are orthogonal and bf d is upper lower triangular tsod is rank revealing as the middle factor bf d reveals the rank of bf a the computation of tsod however is demanding in this paper we present an algorithm called randomized pivoted tsod rp tsod where the middle factor is lower triangular key in our work is the exploitation of randomization and rp tsod is primarily devised to efficiently construct an approximation to a low rank matrix we provide three different types of bounds for rp tsod i we furnish upper bounds on the error of the low rank approximation ii we bound the k approximate principal singular values and iii we derive bounds for the canonical angles between the approximate and the exact singular subspaces our bounds describe the characteristics and behavior of our proposed algorithm through numerical tests we show the effectiveness of the devised bounds as well as our proposed algorithm

effective matrix free preconditioning for the augmented immersed interface method we present effective and efficient matrix free preconditioning techniques for the augmented immersed interface method aiim aiim has been developed recently and is shown to be very effective for interface problems and problems on irregular domains gmres is often used to solve for the augmented variable s associated with a schur complement a in aiim that is defined along the interface or the irregular boundary the efficiency of aiim relies on how quickly the system for a can be solved for some applications there are substantial difficulties involved such as the slow convergence of gmres particularly for free boundary and moving interface problems and the inconvenience in finding a preconditioner due to the situation that only the products of a and vectors are available here we propose matrix free structured preconditioning techniques for aiim via adaptive randomized sampling using only the products of a and vectors to construct a hierarchically semiseparable matrix approximation to a several improvements over existing schemes are shown so as to enhance the efficiency and also avoid potential instability the significance of the preconditioners includes 1 they do not require the entries of a or the multiplication of a t with vectors 2 constructing the preconditioners needs only o log n matrix vector products and o n storage where n is the size of a 3 applying the preconditioners needs only o n flops 4 they are very flexible and do not require any a priori knowledge of the structure of a the preconditioners are observed to significantly accelerate the convergence of gmres with heuristical justifications of the effectiveness comprehensive tests on several important applications are provided such as navier stokes equations on irregular domains with traction boundary conditions interface problems in incompressible flows mixed boundary problems and free boundary problems the preconditioning techniques are also useful for several other problems and methods

sparse fourier transform via butterfly algorithm this paper introduces a fast algorithm for computing sparse fourier transforms with spatial and fourier data supported on curves or surfaces this problem appears naturally in several important applications of wave scattering digital signal processing and reflection seismology the main idea of the algorithm is that the interaction between a frequency region and a spatial region is approximately low rank if the product of their widths are bounded by the maximum frequency based on this property we can approximate the interaction between these two boxes accurately and compactly using a small number of equivalent sources the overall structure of the algorithm follows the butterfly algorithm the computation is further accelerated by exploiting the tensor product property of the fourier kernel in two and three dimensions the proposed algorithm is accurate and has the optimal complexity we present numerical results in both two and three dimensions

low rank approximation of a sparse matrix based on lu factorization with column and row tournament pivoting in this paper we present an algorithm for computing a low rank approximation of a sparse matrix based on a truncated lu factorization with column and row permutations we present various approaches for determining the column and row permutations that show a trade off between speed versus deterministic probabilistic accuracy we show that if the permutations are chosen by using tournament pivoting based on qr factorization then the obtained truncated lu factorization with column row tournament pivoting lu crtp satisfies bounds on the singular values which have similarities with the ones obtained by a communication avoiding rank revealing qr factorization experiments on challenging matrices show that lu crtp provides a good low rank approximation of the input matrix and it is less expensive than the rank revealing qr factorization in terms of computational and memory usage costs while also minimizing the communication cost we also compare the computational complexity of our algorithm with randomized a

a fast solver for poisson problems on infinite regular lattices the fast multipole method fmm provides a highly efficient computational tool for solving constant coefficient partial differential equations e g the poisson equation on infinite domains the solution to such an equation is given as the convolution between a fundamental solution and the given data function and the fmm is used to rapidly evaluate the sum resulting upon discretization of the integral this paper describes an analogous procedure for rapidly solving elliptic difference equations on infinite lattices in particular a fast summation technique for a discrete equivalent of the continuum fundamental solution is constructed the asymptotic complexity of the proposed method is o n s o u r c e where n s o u r c e is the number of points subject to body loads this is in contrast to fft based methods which solve a lattice poisson problem at a cost o n wlogn w independent of n s o u r c e where w is an artificial rectangular box containing the loaded points and n w is the number of lattice points in w

linear time cur approximation of bem matrices in this paper we propose linear time cur approximation algorithms for admissible matrices obtained from the hierarchical form of boundary element matrices we propose a new approach called geometric sampling to obtain indices of most significant rows and columns using information from the domains where the problem is posed our strategy is tailored to boundary element methods bem since it uses directly and explicitly the cluster tree containing information from the problem geometry our cur algorithm has precision comparable with low rank approximations created with the truncated qr factorization with column pivoting qrcp and the adaptive cross approximation aca with full pivoting which are quadratic cost methods when compared to the well known linear time algorithm aca with partial pivoting we show that our algorithm improves in general the convergence error and overcomes some cases where aca fails we provide a general relative error bound for cur approximations created with geometrical sampling finally we evaluate the performance of our algorithms on traditional bem problems defined over different geometries

a nondirective plane wave mlfma stable at low frequencies a novel method called the nondirective stable plane wave multilevel fast multipole algorithm nspwmlfma is presented to evaluate the low frequency lf interactions that cannot be handled by the multilevel fast multipole algorithm mlfma it is well known that the mlfma cannot be used for lf interactions since it suffers from numerical instability contrary to current techniques the proposed technique is not based on the spectral representation of the green function instead the addition theorem of the mlfma is manipulated into a form that allows numerically stable translations along the z axis the translation operator for these translations is derived in closed form a qr based method is devised to allow stable translations in all the other directions interpolations and anterpolations are also provided allowing a full multilevel algorithm since the nspwmlfma is based on the same mathematical foundations as the mlfma it requires limited adaptations to existing mlfma codes the fact that a qr is needed limits this algorithm to lf interactions however a coupling with the mlfma is straightforward allowing the easy construction of a broadband algorithm the dc limit of the algorithm is also presented and it is shown that the algorithm remains valid for static problems finally it is shown that the error introduced in the different steps of the algorithm is controllable and a single level vectorial version of the algorithm is applied to a generic scattering application to demonstrate its validity

efficient discretization of laplace boundary integral equations on polygonal domains we describe a numerical procedure for the construction of quadrature formulae suitable for the efficient discretization of boundary integral equations over very general curve segments while the procedure has applications to the solution of boundary value problems on a wide class of complicated domains we concentrate in this paper on a particularly simple case the rapid solution of boundary value problems for laplace s equation on two dimensional polygonal domains we view this work as the first step toward the efficient solution of boundary value problems on very general singular domains in both two and three dimensions the performance of the method is illustrated with several numerical examples

a tangent algorithm for computing the generalized singular value decomposition we present two new algorithms for floating point computation of the generalized singular values of a real pair a b of full column rank matrices and for floating point solution of the generalized eigenvalue problem hx lambda mx with symmetric positive definite matrices h and m the pair a b is replaced with an equivalent pair a b and the generalized singular values are computed as the singular values of the explicitly computed matrix f a b 1 the singular values of f are computed using the jacobi method the relative accuracy of the computed singular value approximations does not depend on column scalings of a and b that is the accuracy is nearly the same for all pairs ad 1 bd 2 with d 1 d 2 arbitrary diagonal nonsingular matrices similarly the pencil h lambda m is replaced with an equivalent pencil h lambda m and the eigenvalues of h lambda m are computed as the squares of the singular values of g l h l m 1 where l h l m are the cholesky factors of h m respectively and the matrix g is explicitly computed as the solution of a linear system of equations for the computed approximation lambda delta lambda of any exact eigenvalue lambda the relative error delta lambda lambda is of order p n mbox boldmath varepsilon max min delta in cal d kappa 2 delta h delta min delta in cal d kappa 2 delta m delta where p n is a modestly growing polynomial of the dimension of the problem mbox boldmath varepsilon is the round off unit of floating point arithmetic cal d denotes the set of diagonal nonsingular matrices and kappa 2 cdot is the spectral condition number furthermore floating point computation corresponds to an exact computation with h delta h m delta m where for all i j delta h ij sqrt h ii h jj and delta m ij sqrt m ii m jj are of order of mbox boldmath varepsilon times a modest function of n

fast coulomb matrix construction via compressing the interactions between continuous charge distributions the continuous fast multipole method cfmm is well known for its asymptotically linear complexity for constructing the coulomb matrix in quantum chemistry however in practice cfmm must evaluate

efficient algorithms for computing a strong rank revealing qr factorization given anm n matrixm withm n it is shown that there exists a permutation fi and an integer k such that the qr factorization myi q ak ckbk reveals the numerical rank of m the k k upper triangular matrix ak is well conditioned ilckll2 is small and bk is linearly dependent on ak with coefficients bounded by a low degree polynomial in n existing rank revealing qr rrqr algorithms are related to such factorizations and two algorithms are presented for computing them the new algorithms are nearly as efficient as qr with column pivoting for most problems and take o ran2 floating point operations in the worst case

reduced basis multifidelity approach for efficient parametric study of naca airfoils multifidelity methods are often used to reduce the cost of parameter space exploration with applications to design optimization and uncertainty quantification we present a multifidelity method to

using low rank approximation of the jacobian matrix in the newton raphson method to solve certain singular equations it is well known that the pseudoinverse newton raphson method converges locally if the rank of the jacobian matrix is constant a weaker assumption is considered a set of zeros z is a smooth manifold of dimension k and the rank of the jacobian is exactly n k at all zeros low rank approximation of the jacobian matrix is used it is proved that newton raphson quadratically converges in this case also the predictor corrector approach can be used to trace a curve of zeros if k 1 the application considered belongs to the field of computer aided geometric design the method is applied to trace a curve of tangential intersection of two parametric surfaces some experimental results are shown suggesting that the method is stable

a fast direct solver for the integral equations of scattering theory on planar curves with corners we describe an approach to the numerical solution of the integral equations of scattering theory on planar curves with corners it is rather comprehensive in that it applies to a wide variety of boundary value problems here we treat the neumann and dirichlet problems as well as the boundary value problem arising from acoustic scattering at the interface of two fluids it achieves high accuracy is applicable to large scale problems and perhaps most importantly does not require asymptotic estimates for solutions instead the singularities of solutions are resolved numerically the approach is efficient however only in the low and mid frequency regimes once the scatterer becomes more than several hundred wavelengths in size the performance of the algorithm of this paper deteriorates significantly we illustrate our method with several numerical experiments including the solution of a neumann problem for the helmholtz equation given on a domain with nearly 10000 corner points

stable and efficient gaussian process calculations the use of gaussian processes can be an effective approach to prediction in a supervised learning environment for large data sets the standard gaussian process approach requires solving very large systems of linear equations and approximations are required for the calculations to be practical we will focus on the subset of regressors approximation technique we will demonstrate that there can be numerical instabilities in a well known implementation of the technique we discuss alternate implementations that have better numerical stability properties and can lead to better predictions our results will be illustrated by looking at an application involving prediction of galaxy redshift from broadband spectrum data

improving cur matrix decomposition and the nystrom approximation via adaptive sampling the cur matrix decomposition and the nystrom approximation are two important low rank matrix approximation techniques the nystrom method approximates a symmetric positive semidefinite matrix in terms of a small number of its columns while cur approximates an arbitrary data matrix by a small number of its columns and rows thus cur decomposition can be regarded as an extension of the nystrom approximation in this paper we establish a more general error bound for the adaptive column row sampling algorithm based on which we propose more accurate cur and nystrom algorithms with expected relative error bounds the proposed cur and nystrom algorithms also have low time complexity and can avoid maintaining the whole data matrix in ram in addition we give theoretical analysis for the lower error bounds of the standard nystrom method and the ensemble nystrom method the main theoretical results established in this paper are novel and our analysis makes no special assumption on the data matrices

efficient methods for grouping vectors into low rank clusters we present a few practical algorithms for sorting vectors into low rank clusters these algorithms rely on a subdivision scheme applied to the space of projections from d dimensions to 1 dimension this subdivision scheme can be thought of as a higher dimensional generalization of quicksort given the ability to quickly sort vectors into low rank clusters one can efficiently search a matrix for low rank sub blocks of large diameter the ability to detect large diameter low rank sub blocks has many applications ranging from data analysis to matrix compression

strong rank revealing lu factorizations abstract for any m n matrix a we introduce a definition of strong rank revealing lu rrlu factorization related to the definition presented by pan but with some extensions similar to the notion of strong rank revealing qr factorization developed in the joint work of gu and eisenstat a pivoting strategy based on the idea of local maximum volumes is introduced and a proof of the existence of strong rrlu is given

rank deficient nonlinear least squares problems and subset selection we examine the local convergence of the levenberg marquardt method for the solution of nonlinear least squares problems that are rank deficient and have nonzero residual we show that replacing the jacobian by a truncated singular value decomposition can be numerically unstable we recommend instead the use of subset selection we corroborate our recommendations by perturbation analyses and numerical experiments

on the compression of low rank matrices a procedure is reported for the compression of rank deficient matrices a matrix a of rank k is represented in the form a u circ b circ v where b is a k times k submatrix of a and u v are well conditioned matrices that each contain a k times k identity submatrix this property enables such compression schemes to be used in certain situations where the singular value decomposition svd cannot be used efficiently numerical examples are presented

the qlp approximation to the singular value decomposition in this paper we introduce a new decomposition called the pivoted qlp decomposition it is computed by applying pivoted orthogonal triangularization to the columns of the matrix x in question to get an upper triangular factor r and then applying the same procedure to the rows of r to get a lower triangular matrix l the diagonal elements of r are called the r values of x those of l are called the l values numerical examples show that the l values track the singular values of x with considerable fidelity far better than the r values at a gap in the l values the decomposition provides orthonormal bases of analogues of row column and null spaces provided of x the decomposition requires no more than twice the work required for a pivoted qr decomposition the computation of r and l can be interleaved so that the computation can be terminated at any suitable point which makes the decomposition especially suitable for low rank determination problems the interleaved algorithm also suggests a new efficient 2 norm estimator

randomized lu decomposition using sparse projections abstract a fast algorithm that approximates a low rank lu decomposition is presented in order to achieve a low complexity the algorithm uses sparse random projections combined with fft based random projections the asymptotic approximation error of the algorithm is analyzed and a theoretical error bound is presented finally numerical examples illustrate that for a similar approximation error the sparse lu algorithm is faster than recent state of the art methods the algorithm is completely parallelizable and can fully run on a gpu the performance is tested on a gpu card showing a significant speed up improvement in the running time in comparison to a sequential execution

a fast algorithm for simulating multiphase flows through periodic geometries of arbitrary shape this paper presents a new boundary integral equation bie method for simulating particulate and multiphase flows through periodic channels of arbitrary smooth shape in two dimensions the authors consider a particular system multiple vesicles suspended in a periodic channel of arbitrary shape to describe the numerical method and test its performance rather than relying on the periodic green s function as classical bie methods do the method combines the free space green s function with a small auxiliary basis and imposes periodicity as an extra linear condition as a result we can exploit existing free space solver libraries quadratures and fast algorithms and handle a large number of vesicles in a geometrically complex channel spectral accuracy in space is achieved using the periodic trapezoid rule and product quadratures while a first order semi implicit scheme evolves particles by treating the vesicle channel interactions explicitly new constraint correction formulas are introduced that pre

a nonlinear optimization procedure for generalized gaussian quadratures we present a new nonlinear optimization procedure for the computation of generalized gaussian quadratures for a broad class of square integrable functions on intervals while some of the components of this algorithm have been previously published we present a simple and robust scheme for the determination of a sparse solution to an underdetermined nonlinear optimization problem which replaces the continuation scheme of the previously published works the new algorithm successfully computes generalized gaussian quadratures in a number of instances in which the previous algorithms fail four applications of our scheme to computational physics are presented the construction of discrete plane wave expansions for the helmholtz green s function the design of linear array antennae the computation of a quadrature for the discretization of laplace boundary integral equations on certain domains with corners and the construction of quadratures for the discretization of laplace and helmholtz boundary integral equations on smooth surfaces

on selecting a maximum volume sub matrix of a matrix and related problems given a matrix email protected r m x n n vectors in m dimensions we consider the problem of selecting a subset of its columns such that its elements are as linearly independent as possible this notion turned out to be important in low rank approximations to matrices and rank revealing qr factorizations which have been investigated in the linear algebra community and can be quantified in a few different ways in this paper from a complexity theoretic point of view we propose four related problems in which we try to find a sub matrix email protected r m x k of a given matrix email protected r m x n such that i s m a x c the largest singular value of c is minimum ii s m i n c the smallest singular value of c is maximum iii k c email protected m a x c s m i n c the condition number of c is minimum and iv the volume of the parallelepiped defined by the column vectors of c is maximum we establish the np hardness of these problems and further show that they do not admit ptas we then study a natural greedy heuristic for the maximum volume problem and show that it has approximation ratio 2 o k l o g k our analysis of the greedy heuristic is tight to within a logarithmic factor in the exponent which we show by explicitly constructing an instance for which the greedy heuristic is 2 w k from optimal when a has unit norm columns a related problem is to select the maximum number of vectors with a given volume we show that if the optimal solution selects k columns then greedy will select w k logk columns providing a logk approximation

subspace iteration randomization and singular value problems a classical problem in matrix computations is the efficient and reliable approximation of a given matrix by a matrix of lower rank the truncated singular value decomposition svd is known to provide the best such approximation for any given fixed rank however the svd is also known to be very costly to compute among the different approaches in the literature for computing low rank approximations randomized algorithms have attracted researchers attention recently due to their surprising reliability and computational efficiency in different application areas typically such algorithms are shown to compute with very high probability low rank approximations that are within a constant factor from optimal and are known to perform even better in many practical situations in this paper we present a novel error analysis that considers randomized algorithms within the subspace iteration framework and show with very high probability that highly accurate low rank approximations as well as singular values ca

on the existence and computation of lu factorizations with small pivots let a be an n by n matrix which may be singular with a one dimensional null space and consider the lu factorization of a when a is exactly singular we show conditions under which a pivoting strategy will produce a zero n th pivot when a is not singular we show conditions under which a pivoting strategy will produce an nth pivot that is o g or o k a where is the smallest singular value of a and k a is the condition number of a these conditions are expressed in terms of the elements of a in general but reduce to conditions on the elements of the singular vectors corresponding to when a is nearly or exactly singular they can be used to build a 2 pass factorization algorithm which is guaranteed to produce a small n th pivot for nearly singular matrices as an example we exhibit an lu factorization of the n by n upper triangular matrix

hierarchical interpolative factorization for elliptic operators differential equations this paper introduces the hierarchical interpolative factorization for integral equations hif ie associated with elliptic problems in two and three dimensions this factorization takes the form of an approximate generalized lu decomposition that permits the efficient application of the discretized operator and its inverse hif ie is based on the recursive skeletonization algorithm but incorporates a novel combination of two key features 1 a matrix factorization framework for sparsifying structured dense matrices and 2 a recursive dimensional reduction strategy to decrease the cost thus higher dimensional problems are effectively mapped to one dimension and we conjecture that constructing applying and inverting the factorization all have linear or quasilinear complexity numerical experiments support this claim and further demonstrate the performance of our algorithm as a generalized fast multipole method direct solver and preconditioner hif ie is compatible with geometric adaptivity and can handle both boundary and volume problems matlab codes are freely available 2015 wiley periodicals inc

on updating signal subspaces the authors develop an algorithm for adaptively estimating the noise subspace of a data matrix as is required in signal processing applications employing the signal subspace approach the noise subspace is estimated using a rank revealing qr factorization instead of the more expensive singular value or eigenvalue decompositions using incremental condition estimation to monitor the smallest singular values of triangular matrices the authors can update the rank revealing triangular factorization inexpensively when new rows are added and old rows are deleted experiments demonstrate that the new approach usually requires o n sup 2 work to update an n n matrix and that it accurately tracks the noise subspace

randomized interpolative decomposition of separated representations we introduce an algorithm to compute tensor interpolative decomposition dubbed ctd id for the reduction of the separation rank of canonical tensor decompositions ctds tensor id selects for a user defined accuracy a near optimal subset of terms of a ctd to represent the remaining terms via a linear combination of the selected terms ctd id can be used as an alternative to or in combination with the alternating least squares als algorithm we present examples of its use within a convergent iteration to compute inverse operators in high dimensions we also briefly discuss the spectral norm as a computational alternative to the frobenius norm in estimating approximation errors of tensor id we reduce the problem of finding tensor ids to that of constructing interpolative decompositions of certain matrices these matrices are generated via randomized projection of the terms of the given tensor we provide cost estimates and several examples of the new approach to the reduction of separation rank

fast algorithms for spherical harmonic expansions ii we provide an efficient algorithm for calculating at appropriately chosen points on the two dimensional surface of the unit sphere in r 3 the values of functions that are specified by their spherical harmonic expansions a procedure known as the inverse spherical harmonic transform we also provide an efficient algorithm for calculating the coefficients in the spherical harmonic expansions of functions that are specified by their values at these appropriately chosen points a procedure known as the forward spherical harmonic transform the algorithms are numerically stable and if the number of points in our standard tensor product discretization of the surface of the sphere is proportional to l 2 then the algorithms have costs proportional to l 2ln l at any fixed precision of computations several numerical examples illustrate the performance of the algorithms

bounded and monotone properties of solutions of second order quasilinear forced difference equations abstract the authors consider the quasilinear difference equation p n 1 y n 1 q n y n r n and obtain results on the asymptotic behavior of solutions of including sufficient conditions for all solutions to be bounded or unbounded some results on the existence and behavior of nonincreasing solutions of are also obtained examples are inserted to illustrate the results

a fast direct solver for boundary value problems on locally perturbed geometries abstract many applications including optimal design and adaptive discretization techniques involve solving several boundary value problems on geometries that are local perturbations of an original geometry this manuscript presents a fast direct solver for boundary value problems that are recast as boundary integral equations the idea is to write the discretized boundary integral equation on a new geometry as a low rank update to the discretized problem on the original geometry using the sherman morrison formula the inverse can be expressed in terms of the inverse of the original system applied to the low rank factors and the right hand side numerical results illustrate for problems where perturbation is localized the fast direct solver is three times faster than building a new solver from scratch

subspace orbit randomized decomposition for low rank matrix approximations an efficient accurate and reliable approximation of a matrix by one of lower rank is a fundamental task in numerical linear algebra and signal processing applications in this paper we introduce a new matrix decomposition approach termed subspace orbit randomized singular value decomposition sor svd which makes use of random sampling techniques to give a low rank approximation to an input matrix given a large and dense data matrix of size m times n the algorithm requires a few passes through data to compute a rank k approximation in o mnk floating point operations moreover the sor svd algorithm can utilize advanced computer architectures and as a result it can be optimized for maximum efficiency the sor svd algorithm is simple accurate and provably correct and outperforms previously reported techniques in terms of accuracy and efficiency our numerical experiments support these claims

a blas 3 version of the qr factorization with column pivoting the qr factorization with column pivoting qrp originally suggested by golub numer math 7 1965 206 216 is a popular approach to computing rank revealing factorizations using level 1 blas it was implemented in linpack and using level 2 blas in lapack while the level 2 blas version delivers superior performance in general it may result in worse performance for large matrix sizes due to cache effects we introduce a modification of the qrp algorithm which allows the use of level 3 blas kernels while maintaining the numerical behavior of the linpack and lapack implementations experimental comparisons of this approach with the linpack and lapack implementations on ibm rs 6000 sgi r8000 and dec axp platforms show considerable performance improvements

fast direct methods for gaussian processes a number of problems in probability and statistics can be addressed using the multivariate normal gaussian distribution in the one dimensional case computing the probability for a given mean and variance simply requires the evaluation of the corresponding gaussian density in the n dimensional setting however it requires the inversion of an n times n covariance matrix c as well as the evaluation of its determinant det c in many cases such as regression using gaussian processes the covariance matrix is of the form c sigma 2 i k where k is computed using a specified covariance kernel which depends on the data and additional parameters hyperparameters the matrix c is typically dense causing standard direct methods for inversion and determinant evaluation to require mathcal o n 3 work this cost is prohibitive for large scale modeling here we show that for the most commonly used covariance functions the matrix c can be hierarchically factored into a product of block low rank updates of the identity matrix yielding an mathcal o n log 2 n algorithm for inversion more importantly we show that this factorization enables the evaluation of the determinant det c permitting the direct calculation of probabilities in high dimensions under fairly broad assumptions on the kernel defining k our fast algorithm brings many problems in marginalization and the adaptation of hyperparameters within practical reach using a single cpu core the combination of nearly optimal scaling in terms of problem size with high performance computing resources will permit the modeling of previously intractable problems we illustrate the performance of the scheme on standard covariance kernels

rank revealing qr factorizations and the singular value decomposition t chan has noted that even when the singular value decomposition of a matrix a is known it is still not obvious how to find a rank revealing qr factorization rrqr of a if a has numerical rank deficiency this paper offers a constructive proof of the existence of the rrqr factorization of any matrix a of size m x n with numerical rank r the bounds derived in this paper that guarantee the existence of rrqr are all of order f i in comparison with chan s 0 2 it has been known for some time that if a is only numerically rank one deficient then the column permutation l7 of a that guarantees a small rnn in the qr factorization of a n can be obtained by inspecting the size of the elements of the right singular vector of a corresponding to the smallest singular value of a to some extent our paper generalizes this well known result we consider the interplay between two important matrix decompositions the singular value decomposition and the qr factorization of a matrix a in particular we are interested in the case when a is singular or nearly singular it is well known that for any a e r m x n a real matrix with rn rows and n columns where without loss of generality we assume rn n there are orthogonal matrices u and v such that where c is a diagonal matrix with nonnegative diagonal elements we assume that a 2 a2 2 2 on 2 0 the decomposition 0 1 is the singular value decomposition svd of a and the ai are the singular values of a the columns of v are the right singular vectors of a and the columns of u are the left singular vectors of a mathematically in terms of the singular values received december 1 1990 revised february 8 199 1 199 1 mathematics subject classification primary 65f30 15a23 15a42 15a15