on the sauer xu formula for the error in multivariate polynomial interpolation use of a new notion of multivariate divided difference leads to a short proof of a formula by sauer and xu for the error in interpolation by polynomials of total degree n in d variables at a correct point set

the complex parameter rao test the rao test is an important method in signal detection in the presence of unknown parameters the traditional approach to the problem when the unknown parameters are complex valued is to form a corresponding real valued parameter vector and then to use the real rao test alternatively we present a complex parameter rao test by reformulating the calculations with respect to the complex valued quantities directly two important examples of the application of the complex parameter rao test are given to illustrate the procedure

constructing nested bases approximations from the entries of non local operators in this article a method for constructing nested bases approximations to large scale fully populated discretizations of integral operators is introduced the scheme uses only few of the matrix entries for approximating the whole matrix in this sense it is similar to the adaptive cross approximation method however its computational complexity is improved

an introduction to mathematical models of coagulation fragmentation processes a discrete deterministic mean field approach abstract we summarise the properties and the fundamental mathematical results associated with basic models which describe coagulation and fragmentation processes in a deterministic manner and in which cluster size is a discrete quantity an integer multiple of some basic unit size in particular we discuss smoluchowski s equation for aggregation the becker doring model of simultaneous aggregation and fragmentation and more general models involving coagulation and fragmentation

effective algorithms with circulant block matrices abstract effective numerical algorithms for circulant block matrices a whose blocks are circulant are obtained the eigenvalues of such matrices are determined in terms of the eigenvalues of matrices of reduced dimension and systems of linear equations involving these matrices are solved efficiently using fast fourier transforms

on multivariate lagrange interpolation lagrange interpolation by polynomials in several variables is studied through a finite difference approach we establish an interpolation formula analogous to that of newton and a remainder formula both of them in terms of finite differences we prove that the finite difference admits an integral representation involving simplex spline functions in particular this provides a remainder formula for lagrange interpolation of degree n of a function f which is a sum of integrals of certain n 1 st directional derivatives of f multiplied by simplex spline functions we also provide two algorithms for the computation of lagrange interpolants which use only addition scalar multiplication and point evaluation of polynomials

deterministic and probabilistic conditions for finite completability of low tucker rank tensor we investigate the fundamental conditions on the sampling pattern i e locations of the sampled entries for finite completability of a low rank tensor given some components of its tucker rank in order to find the deterministic necessary and sufficient conditions we propose an algebraic geometric analysis on the tucker manifold which allows us to incorporate multiple rank components in the proposed analysis in contrast with the conventional geometric approaches on the grassmannian manifold this analysis characterizes the algebraic independence of a set of polynomials defined based on the sampling pattern which is closely related to finite completability of the sampled tensor where finite completability simply means that the number of possible completions of the sampled tensor is finite probabilistic conditions are then studied and a lower bound on the sampling probability is given which guarantees that the proposed deterministic conditions on the sampling patterns for finite completability hold with high probability furthermore using the proposed geometric approach for finite completability we propose a sufficient condition on the sampling pattern that ensures there exists exactly one completion of the sampled tensor

multigrid accelerated tensor approximation of function related multidimensional arrays in this paper we describe and analyze a novel tensor approximation method for discretized multidimensional functions and operators in mathbb r d based on the idea of multigrid acceleration the approach stands on successive reiterations of the orthogonal tucker tensor approximation on a sequence of nested refined grids on the one hand it provides a good initial guess for the nonlinear iterations to find the approximating subspaces on finer grids on the other hand it allows us to transfer from the coarse to fine grids the important data structure information on the location of the so called most important fibers in directional unfolding matrices the method indicates linear complexity with respect to the size of data representing the input tensor in particular if the target tensor is given by using the rank r canonical model then our approximation method is proved to have linear scaling in the univariate grid size n and in the input rank r the method is tested by three dimensional 3d electronic structure calculations for the multigrid accelerated low tucker rank approximation of the all electron densities having strong nuclear cusps we obtain high resolution of their 3d convolution product with the newton potential the accuracy of order 10 6 in max norm is achieved on large n times n times n grids up to n 1 6 cdot10 4 with the time scale in several minutes

computation of the output of a function with fuzzy inputs based on a low rank tensor approximation we apply a derivative free optimization method based on novel low rank tensor methods to the problem of propagating fuzzy uncertainty through a continuous real valued function adhering to zadeh s extension principle such a problem can be reformulated as a sequence of optimization problems over nested search spaces the optimization method we use is based on a low rank tensor approximation of the function sampled on a grid and a search for the minimal and maximal entries in this low rank tensor in contrast to classical fuzzy uncertainty propagation methods such as the vertex method and the transformation method the method we propose does not exhibit an inherent exponential scaling for increasing dimension of the search space obviously no derivative free optimization algorithm can exist which shows sub exponential scaling with the dimension for all possible continuous functions the algorithm that we present here however can exploit a specific type of structure and regularity beyond continuity that is often present in real world optimization problems we illustrate this with some high dimensional numerical examples where the presented method clearly outperforms some established derivative free optimization codes

a theory of pseudoskeleton approximations abstract let an m n matrix a be approximated by a rank r matrix with an accuracy e we prove that it is possible to choose r columns and r rows of a forming a so called pseudoskeleton component which approximates a with o e r m n accuracy in the sense of the 2 norm on the way to this estimate we study the interconnection between the volume i e the determinant in the absolute value and the minimal singular value r of r r submatrices of an n r matrix with orthogonal columns we propose a lower bound better than one given by chandrasekaran and ipsen and by hong and pan for the maximum of r over all these submatrices and formulate a hypothesis on a tighter bound

density matrix formulation for quantum renormalization groups a generalization of the numerical renormalization group procedure used first by wilson for the kondo problem is presented it is shown that this formulation is optimal in a certain sense as a demonstration of the effectiveness of this approach results from numerical real space renormalization group calculations for heisenberg chains are presented

hybrid cross approximation of integral operators the efficient treatment of dense matrices arising e g from the finite element discretisation of integral operators requires special compression techniques in this article we use the matrix representation that approximates the dense stiffness matrix in admissible blocks corresponding to subdomains where the underlying kernel function is smooth by low rank matrices the low rank matrices are assembled by a new hybrid algorithm hca that has the same proven convergence as standard interpolation but also the same efficiency as the heuristic adaptive cross approximation aca

a low rank approach to the computation of path integrals we present a method for solving the reaction diffusion equation with general potential in free space it is based on the approximation of the feynman kac formula by a sequence of convolutions on sequentially diminishing grids for computation of the convolutions we propose a fast algorithm based on the low rank approximation of the hankel matrices the algorithm has complexity of o n r m log m n r 2 m flops and requires o m r floating point numbers in memory where n is the dimension of the integral r n and m is the mesh size in one dimension the presented technique can be generalized to the higher order diffusion processes

recompression of hadamard products of tensors in tucker format the hadamard product features prominently in tensor based algorithms in scientific computing and data analysis due to its tendency to significantly increase ranks the hadamard product can represent a major computational obstacle in algorithms based on low rank tensor representations it is therefore of interest to develop recompression techniques that mitigate the effects of this rank increase in this work we investigate such techniques for the case of the tucker format which is well suited for tensors of low order and small to moderate multilinear ranks fast algorithms are attained by combining iterative methods such as the lanczos method and randomized algorithms with fast matrix vector products that exploit the structure of hadamard products the resulting complexity reduction is particularly relevant for tensors featuring large mode sizes i and small to moderate multilinear ranks r to implement our algorithms we have created a new julia library for tensors in tucker format

discrete total variation flows without regularization we propose and analyze an algorithm for the solution of the l 2 subgradient flow of the total variation functional the algorithm involves no regularization thus the numerical solution preserves the main features that motivate practitioners to consider this type of energy we propose an iterative scheme for the solution of the arising problems show that the iterations converge and develop a stopping criterion for them we present numerical experiments which illustrate the power of the method explore the solution behavior and compare with regularized flows

fast and accurate tensor approximation of a multivariate convolution with linear scaling in dimension in the present paper we present the tensor product approximation of a multidimensional convolution transform discretized via a collocation projection scheme on uniform or composite refined grids examples of convolving kernels are provided by the classical newton slater exponential and yukawa potentials 1 email protected e l x and e l x email protected with email protected r d for piecewise constant elements on the uniform grid of size n d we prove quadratic convergence o h 2 in the mesh parameter h 1 n and then justify the richardson extrapolation method on a sequence of grids that improves the order of approximation up to o h 3 a fast algorithm of complexity o dr 1r 2nlogn is described for tensor product convolution on uniform composite grids of size n d where r 1 r 2 are tensor ranks of convolving functions we also present the tensor product convolution scheme in the two level tucker canonical format and discuss the consequent rank reduction strategy finally we give numerical illustrations confirming a the approximation theory for convolution schemes of order o h 2 and o h 3 b linear logarithmic scaling of 1d discrete convolution on composite grids c linear logarithmic scaling in n of our tensor product convolution method on an nxnxn grid in the range email protected 16384

on the fast matrix multiplication in the boundary element method by panel clustering the boundary element method bem leads to a system of linear equations with a full matrix while fem yields sparse matrices this fact seems to require much computational work for the definition of the matrix for the solution of the system and in particular for the matrix vector multiplication which always occurs as an elementary in this paper a method for the approximate matrix vector multiplication is described which requires much less arithmetical work in addition the storage requirements are strongly reduced

generalizing the column row matrix decomposition to multi way arrays abstract in this paper we provide two generalizations of the cur matrix decomposition y cur also known as pseudo skeleton approximation method 1 to the case of n way arrays tensors these generalizations which we called fiber sampling tensor decomposition types 1 and 2 fstd1 and fstd2 provide explicit formulas for the parameters of a rank r r r tucker representation the core tensor of size r r r and the matrix factors of sizes i n r n 1 2 n based only on some selected entries of the original tensor fstd1 uses p n 1 p r n mode fibers of the original tensor while fstd2 uses exactly r fibers in each mode as matrix factors as suggested by the existence theorem provided in oseledets et al 2008 2 with a core tensor defined in terms of the entries of a subtensor of size r r r for n 2 our results are reduced to the already known cur matrix decomposition where the core matrix is defined as the inverse of the intersection submatrix i e u w 1 additionally we provide an adaptive type algorithm for the selection of proper fibers in the fstd1 model which is useful for large scale applications several numerical results are presented showing the performance of our fstd1 adaptive algorithm compared to two recently proposed approximation methods for 3 way tensors

efficient convolution with the newton potential in d dimensions the paper is concerned with the evaluation of the convolution integral int mathbb r d frac 1 left vert x y right vert f y rm d y in d dimensions usually d 3 when f is given as piecewise polynomial of possibly large degree i e f may be considered as an hp finite element function the underlying grid is locally refined using various levels of dyadically organised grids the result of the convolution is approximated in the same kind of mesh if f is given in tensor product form the d dimensional convolution can be reduced to one dimensional convolutions although the details are given for the kernel 1 left vert x right vert the basis techniques can be generalised to homogeneous kernels e g the fundamental solution const cdot left vert x right vert 2 d of the d dimensional poisson equation

an adaptive detector for deterministic signals in noise of unknown spectra using the rao test an adaptive detector for a known deterministic signal of unknown amplitude in gaussian noise of unknown spectra is described the detector is based on the rao test which is asymptotically equivalent to the generalized likelihood ratio the detector achieves constant false alarm probability in the presence of large changes in input noise bandwidth and variance while providing optimum detection performance the results are supported by simulation

tt cross approximation for multidimensional arrays abstract as is well known a rank r matrix can be recovered from a cross of r linearly independent columns and rows and an arbitrary matrix can be interpolated on the cross entries other entries by this cross or pseudo skeleton approximation are given with errors depending on the closeness of the matrix to a rank r matrix and as well on the choice of cross in this paper we extend this construction to d dimensional arrays tensors and suggest a new interpolation formula in which a d dimensional array is interpolated on the entries of some tt cross tensor train cross the total number of entries and the complexity of our interpolation algorithm depend on d linearly so the approach does not suffer from the curse of dimensionality we also propose a tt cross method for computation of d dimensional integrals and apply it to some examples with dimensionality in the range from d 100 up to d 4000 and the relative accuracy of order 10 10 in all constructions we capitalize on the new tensor decomposition in the form of tensor trains tt decomposition

on bivariate hermite interpolation with minimal degree polynomials a newton type approach is used to deal with bivariate polynomial hermite interpolation problems when the data are distributed in the intersections of two families of straight lines as a generalization of regular grids the interpolation operator is degree reducing and the interpolation space is a minimal degree space integral remainder formulas are given for the lagrange case then extended to the hermite case and finally used to obtain error estimates

tensor decomposition in electronic structure calculations on 3d cartesian grids in this paper we investigate a novel approach based on the combination of tucker type and canonical tensor decomposition techniques for the efficient numerical approximation of functions and operators in electronic structure calculations in particular we study applicability of tensor approximations for the numerical solution of hartree fock and kohn sham equations on 3d cartesian grids we show that the orthogonal tucker type tensor approximation of electron density and hartree potential of simple molecules leads to low tensor rank representations this enables an efficient tensor product convolution scheme for the computation of the hartree potential using a collocation type approximation via piecewise constant basis functions on a uniform nxnxn grid combined with the richardson extrapolation our approach exhibits o h 3 convergence in the grid size h o n 1 moreover this requires o 3rn r 3 storage where r denotes the tucker rank of the electron density with r o logn almost uniformly in n for example calculations of the coulomb matrix and the hartree fock energy for the ch 4 molecule with a pseudopotential on the c atom achieved accuracies of the order of 10 6 hartree with a grid size n of several hundreds since the tensor product convolution in 3d is performed via 1d convolution transforms our scheme markedly outperforms the 3d fft in both the computing time and storage requirements

on the invariance coincidence and statistical equivalence of the glrt rao test and wald test three common techniques to discriminate between alternatives in a binary hypothesis testing problem are the generalized likelihood ratio test glrt the rao test and the wald test in this paper we investigate some characteristics of the corresponding decision statistics and provide their expressions for some problems of particular interest in statistical signal processing first of all we focus on the invariance of the rao and wald tests with respect to transformations leaving the testing problem unaltered then we introduce necessary and sufficient conditions in order for their decision statistics to coincide with twice the logarithm of the glrt statistic finally we present some detection problems usually encountered in practical signal processing applications where the decision variables of the three quoted tests are equivalent namely related by strictly monotonic transformations

tensor decompositions and applications this survey provides an overview of higher order tensor decompositions their applications and available software a tensor is a multidimensional or n way array decompositions of higher order tensors i e n way arrays with n geq 3 have applications in psycho metrics chemometrics signal processing numerical linear algebra computer vision numerical analysis data mining neuroscience graph analysis and elsewhere two particular tensor decompositions can be considered to be higher order extensions of the matrix singular value decomposition candecomp parafac cp decomposes a tensor as a sum of rank one tensors and the tucker decomposition is a higher order form of principal component analysis there are many other tensor decompositions including indscal parafac2 candelinc dedicom and paratuck2 as well as nonnegative variants of all of the above the n way toolbox tensor toolbox and multilinear engine are examples of software packages for working with tensors

rapid solution of integral equations of classical potential theory an algorithm is described for rapid solution of classical boundary value problems dirichlet an neumann for the laplace equation based on iteratively solving integral equations of potential theory cpu time requirements for previously published algorithms of this type are proportional to n2 where n is the number of nodes in the discretization of the boundary of the region the cpu time requirements for the algorithm of the present paper are proportional to n making it considerably more practical for large scale problems

grid based electronic structure calculations we present a fully grid based approach for solving hartree fock and all electron kohn sham equations based on low rank approximation of three dimensional electron orbitals due to the low rank structure the total complexity of the algorithm depends linearly with respect to the one dimensional grid size linear complexity allows for the usage of fine grids e g 81923 and thus cheap extrapolation procedure we test the proposed approach on closed shell atoms up to the argon several molecules and clusters of hydrogen atoms all tests show systematical convergence with the required accuracy

tree adaptive approximation in the hierarchical tensor format the hierarchical tensor format allows for the low parametric representation of tensors even in high dimensions d the efficiency of this representation strongly relies on an appropriate hierarchical splitting of the different directions 1 ldots d such that the associated ranks remain sufficiently small this splitting can be represented by a binary tree which is usually assumed to be given in this paper we address the question of finding an appropriate tree from a subset of tensor entries without any a priori knowledge on the tree structure we propose an agglomerative strategy that can be combined with rank adaptive cross approximation techniques such that tensors can be approximated in the hierarchical format in an entirely black box way numerical examples illustrate the potential and the limitations of our approach

on the convergence of the aca algorithm for radiation and scattering problems the adaptive cross approximation aca algorithm when used to accelerate the numerical solution of integral equations for radiation and scattering problems sometimes suffers from inaccuracies these inaccuracies occur when the aca convergence criterion which is based on an approximation of the residual relative error is prematurely satisfied this paper identifies the two sources of this problem and proposes adaptations of the algorithm that remedy them

reduced basis methods from low rank matrices to low rank tensors we propose a novel combination of the reduced basis method with low rank tensor techniques for the efficient solution of parameter dependent linear systems in the case of several parameters this combination called rbtensor consists of three ingredients first the underlying parameter dependent operator is approximated by an explicit affine representation in a low rank tensor format second a standard greedy strategy is used to construct a problem dependent reduced basis third the associated reduced parametric system is solved for all parameter values on a tensor grid simultaneously via a low rank approach this allows us to explicitly represent and store an approximate solution for all parameter values at a time once this approximation is available the computation of output functionals and the evaluation of statistics of the solution become a cheap online task without requiring the solution of a linear system

accurate and efficient computation of nonlocal potentials based on gaussian sum approximation abstract we introduce an accurate and efficient method for the numerical evaluation of nonlocal potentials including the 3d 2d coulomb 2d poisson and 3d dipole dipole potentials our method is based on a gaussian sum approximation of the singular convolution kernel combined with a taylor expansion of the density starting from the convolution formulation of the nonlocal potential for smooth and fast decaying densities we make a full use of the fourier pseudospectral plane wave approximation of the density and a separable gaussian sum approximation of the kernel in an interval where the singularity the origin is excluded the potential is separated into a regular integral and a near field singular correction integral the first is computed with the fourier pseudospectral method while the latter is well resolved utilizing a low order taylor expansion of the density both parts are accelerated by fast fourier transforms fft the method is accurate 14 16 digits efficient o n log n complexity low in storage easily adaptable to other different kernels applicable for anisotropic densities and highly parallelizable

approximation of boundary element matrices this article considers the problem of approximating a general asymptotically smooth function in two variables typically arising in integral formulations of boundary value problems by a sum of products of two functions in one variable from these results an iterative algorithm for the low rank approximation of blocks of large unstructured matrices generated by asymptotically smooth functions is developed this algorithm uses only few entries from the original block and since it has a natural stopping criterion the approximative rank is not needed in advance

fast evaluation of singular bem integrals based on tensor approximations in this paper we propose a method for the fast evaluation of integrals stemming from boundary element methods including discretisations of the classical single and double layer potential operators our method is based on the parametrisation of boundary elements in terms of a d dimensional parameter tuple we interpret the integral as a real valued function f depending on d parameters and show that f is smooth in a d dimensional box a standard interpolation of f by polynomials leads to a d dimensional tensor which is given by the values of f at the interpolation points this tensor may be approximated in a low rank tensor format like the cp format or the mathcal h tucker format the tensor approximation has to be done only once and allows us to evaluate interpolants in mathcal o dr m 1 operations in the cp format or mathcal o dk 3 dk m 1 operations in the mathcal h tucker format where m denotes the interpolation order and the ranks r k are small integers we demonstrate that highly accurate integral values can be obtained at very moderate costs

multilevel toeplitz matrices generated by tensor structured vectors and convolution with logarithmic complexity we study the tensor structure of two operations the transformation of a given multidimensional vector into a multilevel toeplitz matrix and the convolution of two given multidimensional vectors we show that the low rank tensor structure of the input is preserved in the output and propose efficient algorithms for these operations in the newly introduced quantized tensor train qtt format consider a d dimensional 2n times cdots times 2n vector boldsymbol x if it is represented elementwise the number of parameters is 2n d however if we assume that boldsymbol x is given in a qtt representation with ranks bounded by p the number of parameters is reduced to mathcal o left dp 2 log n right under this assumption we show how the multilevel toeplitz matrix generated by boldsymbol x can be obtained in the qtt format with ranks bounded by 2p in mathcal o left dp 2 log n right operations we also describe how the convolution boldsymbol x star boldsymbol y of

numerical operator calculus in higher dimensions when an algorithm in dimension one is extended to dimension d in nearly every case its computational cost is taken to the power d this fundamental difficulty is the single greatest impediment to solving many important problems and has been dubbed the curse of dimensionality for numerical analysis in dimension d we propose to use a representation for vectors and matrices that generalizes separation of variables while allowing controlled accuracy basic linear algebra operations can be performed in this representation using one dimensional operations thus bypassing the exponential scaling with respect to the dimension although not all operators and algorithms may be compatible with this representation we believe that many of the most important ones are we prove that the multiparticle schrodinger operator as well as the inverse laplacian can be represented very efficiently in this form we give numerical evidence to support the conjecture that eigenfunctions inherit this property by computing the ground state eigenfunction for a simplified schrodinger operator with 30 particles we conjecture and provide numerical evidence that functions of operators inherit this property in which case numerical operator calculus in higher dimensions becomes feasible

breaking the curse of dimensionality or how to use svd in many dimensions for d dimensional tensors with possibly large d 3 an hierarchical data structure called the tree tucker format is presented as an alternative to the canonical decomposition it has asymptotically the same and often even smaller number of representation parameters and viable stability properties the approach involves a recursive construction described by a tree with the leafs corresponding to the tucker decompositions of three dimensional tensors and is based on a sequence of svds for the recursively obtained unfolding matrices and on the auxiliary dimensions added to the initial spatial dimensions it is shown how this format can be applied to the problem of multidimensional convolution convincing numerical examples are given

tensor structured factorized calculation of two electron integrals in a general basis in this paper the problem of efficient grid based computation of the two electron integrals tei in a general basis is considered we introduce the novel multiple tensor factorizations of the tei unfolding matrix which decrease the computational demands for the evaluation of tei in several aspects using the reduced higher order svd the redundancy free product basis set is constructed that diminishes dramatically the initial number o n b 2 of three dimensional 3 d convolutions defined over cross products of n b basis functions to o n b scaling the tensor structured numerical integration with the 3 d newton convolving kernel is performed in one dimensional 1 d complexity thus enabling high resolution over fine 3 d cartesian grids furthermore using the quantized approximation of long vectors ensures the logarithmic storage complexity in the grid size finally we present and analyze two approaches to compute the cholesky decomposition of the tei matrix based on two types of precomput

approximation of the electron density of aluminium clusters in tensor product format the tensor structured methods developed recently for the accurate calculation of the hartree and the non local exchange operators have been applied successfully to the ab initio numerical solution of the hartree fock equation for some molecules in the present work we show that the rank structured representation can be gainfully applied to the accurate approximation of the electron density of large aluminium clusters we consider the tucker type decomposition of the electron density of certain aluminium clusters originating from finite element calculations in the framework of the orbital free density functional theory numerical investigations of the tucker approximation of the corresponding electron density reveal the exponential decay of the approximation error with respect to the tucker rank the resulting low rank tensor representation reduces dramatically the storage needs and the computational complexity of the consequent tensor operations on the electron density as main result the rank of the tucker approximation for the accurate representation of the electron density is small and only weakly dependent on the system size for the systems studied here this shows good promise for resolving the electronic structure of materials using tensor structured techniques

frequency domain cramer rao bound for gaussian processes an expression is derived for the asymptotic frequency domain cramer rao bound crb on the parameters of gaussian processes with information in their means the general result extends whittle s 1953 formula which is applicable to gaussian processes with information in their covariance matrices or spectra it is useful in many fields such as system identification radar and sonar examples of its application to system deconvolution and time delay estimation in colored noise are given

alternating minimal energy methods for linear systems in higher dimensions we propose algorithms for the solution of high dimensional symmetrical positive definite spd linear systems with the matrix and the right hand side given and the solution sought in a low rank format similarly to density matrix renormalization group dmrg algorithms our methods optimize the components of the tensor product format subsequently to improve the convergence we expand the search space by an inexact gradient direction we prove the geometrical convergence and estimate the convergence rate of the proposed methods utilizing the analysis of the steepest descent algorithm the complexity of the presented algorithms is linear in the mode size and dimension and the demonstrated convergence is comparable to or even better than the one of the dmrg algorithm in the numerical experiment we show that the proposed methods are also efficient for non spd systems for example those arising from the chemical master equation describing the gene regulatory model at the mesoscopic scale

stable als approximation in the tt format for rank adaptive tensor completion low rank tensor completion is a highly ill posed inverse problem particularly when the data model is not accurate and some sort of regularization is required in order to solve it in this article we focus on the calibration of the data model for alternating optimization we observe that existing rank adaption methods do not enable a continuous transition between manifolds of different ranks we denote this characteristic as instability under truncation as a consequence of this property arbitrarily small changes in the iterate can have arbitrarily large influence on the further reconstruction we therefore introduce a singular value based regularization to the standard alternating least squares als which is motivated by averaging in microsteps we prove its stability and derive a natural semi implicit rank adaption strategy we further prove that the standard als microsteps for completion problems are only stable on manifolds of fixed ranks and only around points that have what we define as internal tensor restricted isometry property itrip in conclusion numerical experiments are provided that show improvements of the reconstruction quality up to orders of magnitude in the new stable als approximation salsa compared to standard als and the well known riemannian optimization rttc

algorithms for numerical analysis in high dimensions nearly every numerical analysis algorithm has computational complexity that scales exponentially in the underlying physical dimension the separated representation introduced previously allows many operations to be performed with scaling that is formally linear in the dimension in this paper we further develop this representation by i discussing the variety of mechanisms that allow it to be surprisingly efficient ii addressing the issue of conditioning iii presenting algorithms for solving linear systems within this framework and iv demonstrating methods for dealing with antisymmetric functions as arise in the multiparticle schrodinger equation in quantum mechanics numerical examples are given

efficient iterative solution of finite element discretized nonsmooth minimization problems abstract for the iterative solution of finite element discretized nonsmooth minimization problems the alternating direction method of multipliers admm is considered which is a flexible method to solve a large class of convex minimization problems particular features are its unconditional convergence with respect to the involved step size and its direct applicability in particular this article deals with the admm with variable step sizes and devises an adjustment rule for the step size relying on the monotonicity of the residual and discusses proper stopping criteria the proposed scheme is applied to finite element formulations of the obstacle problem and the rudin osher fatemi image denoising problem and the numerical experiments show significant improvements over established variants of the admm

optimization on the hierarchical tucker manifold applications to tensor completion in this work we develop an optimization framework for problems whose solutions are well approximated by hierarchical tucker ht tensors an efficient structured tensor format based on recursive subspace factorizations by exploiting the smooth manifold structure of these tensors we construct standard optimization algorithms such as steepest descent and conjugate gradient for completing tensors from missing entries our algorithmic framework is fast and scalable to large problem sizes as we do not require svds on the ambient tensor space as required by other methods moreover we exploit the structure of the gramian matrices associated with the ht format to regularize our problem reducing overfitting for high subsampling ratios we also find that the organization of the tensor can have a major impact on completion from realistic seismic acquisition geometries these samplings are far from idealized randomized samplings that are usually considered in the literature but are realizable in practical scenarios using these algorithms we successfully interpolate large scale seismic data sets and demonstrate the competitive computational scaling of our algorithms as the problem sizes grow

primal dual optimization algorithms over riemannian manifolds an iteration complexity analysis in this paper we study nonconvex and nonsmooth multi block optimization over euclidean embedded smooth riemannian submanifolds with coupled linear constraints such optimization problems naturally arise from machine learning statistical learning compressive sensing image processing and tensor pca among others by utilizing the embedding structure we develop an admm like primal dual approach based on decoupled solvable subroutines such as linearized proximal mappings where the duality is with respect to the embedded euclidean spaces first we introduce the optimality conditions for the afore mentioned optimization models then the notion of epsilon stationary solutions is introduced as a result the main part of the paper is to show that the proposed algorithms possess an iteration complexity of o 1 epsilon 2 to reach an epsilon stationary solution for prohibitively large size tensor or machine learning models we present a sampling based stochastic algorithm with the same iteration complexity bound in expectation in case the subproblems are not analytically solvable a feasible curvilinear line search variant of the algorithm based on retraction operators is proposed finally we show specifically how the algorithms can be implemented to solve a variety of practical problems such as the np hard maximum bisection problem the ell q regularized sparse tensor principal component analysis and the community detection problem our preliminary numerical results show great potentials of the proposed methods

identifying conductivity in electrical impedance tomography with total variation regularization in this paper we investigate the problem of identifying the conductivity in electrical impedance tomography from one boundary measurement a variational method with total variation regularization is here proposed to tackle this problem we discretize the pde as well as the conductivity with piecewise linear continuous finite elements we prove the stability and convergence of this technique for the numerical solution we propose a projected armijo algorithm finally a numerical experiment is presented to illustrate our theoretical results

detection of broadband planewaves in the presence of gaussian noise of unknown covariance asymptotically optimum tests using the 2 d autoregressive noise model this paper addresses the problem of detecting a broadband planewave in noise of unknown spatial and temporal covariance at a linear array of sensors results of asymptotic detection theory are applied to derive detectors that approach optimal performance for large data records a parametric approach is used to model the statistics of the data a 2 d autoregressive 2dar model is chosen to model the noise process two broadband planewave signal models are considered both models represent the signal as a sum of monochromatic planewaves in the gaussian model the amplitudes are assumed to be gaussian with a single variance parameter whereas in the deterministic assumption they are individual unknown parameters detectors based on asymptotic theory are derived for both models as part of the development of the asymptotically as optimum detector the fisher information matrix fim is derived a proof of the locally asymptotic normal lan property is provided for the gaussian model probability density function pdf both detectors however are as equivalent to the generalized likelihood ratio test glrt are as of constant false alarm rate cfar and perform as as well as the glrt constructed with full knowledge of the noise statistics the performance of both detectors are compared with each other and to a standard spatially normalized beamformer in a computer simulation

using the tensor train approach to solve the ground state eigenproblem for hydrogen molecules we consider the born oppenheimer approximation of the schrodinger equation for two hydrogen atoms in the case of large separation distances we show that the feschbach schur perturbation method can be used to solve the problem for the difference between a given separation and infinite separation this leads to a simplified problem which can be solved iteratively we show that this iteration converges for sufficiently large separation distances and we solve the arising sequence of six dimensional pdes with a finite element method in combination with low rank tensor techniques to make the computations tractable in particular we show how the discretized problems can be represented and solved in the tensor train format since the storage and computational complexity of this format scale linearly in the dimension a very large number of grid points can be employed which leads to accurate approximations of the ground state energy and ground state wavefunction various numerical experiments show the performa

higher order principal component analysis for the approximation of tensors in tree based low rank formats this paper is concerned with the approximation of tensors using tree based tensor formats which are tensor networks whose graphs are dimension partition trees we consider hilbert tensor spaces of multivariate functions defined on a product set equipped with a probability measure this includes the case of multidimensional arrays corresponding to finite product sets we propose and analyse an algorithm for the construction of an approximation using only point evaluations of a multivariate function or evaluations of some entries of a multidimensional array the algorithm is a variant of higher order singular value decomposition which constructs a hierarchy of subspaces associated with the different nodes of the tree and a corresponding hierarchy of interpolation operators optimal subspaces are estimated using empirical principal component analysis of interpolations of partial random evaluations of the function the algorithm is able to provide an approximation in any tree based format with either a prescribed rank or a prescribed relative error with a number of evaluations of the order of the storage complexity of the approximation format under some assumptions on the estimation of principal components we prove that the algorithm provides either a quasi optimal approximation with a given rank or an approximation satisfying the prescribed relative error up to constants depending on the tree and the properties of interpolation operators the analysis takes into account the discretization errors for the approximation of infinite dimensional tensors for a tensor with finite and known rank in a tree based format the algorithm is able to recover the tensor in a stable way using a number of evaluations equal to the storage complexity of the representation of the tensor in this format several numerical examples illustrate the main results and the behavior of the algorithm for the approximation of high dimensional functions using hierarchical tucker or tensor train tensor formats and the approximation of univariate functions using tensorization

randomized interpolative decomposition of separated representations we introduce an algorithm to compute tensor interpolative decomposition dubbed ctd id for the reduction of the separation rank of canonical tensor decompositions ctds tensor id selects for a user defined accuracy a near optimal subset of terms of a ctd to represent the remaining terms via a linear combination of the selected terms ctd id can be used as an alternative to or in combination with the alternating least squares als algorithm we present examples of its use within a convergent iteration to compute inverse operators in high dimensions we also briefly discuss the spectral norm as a computational alternative to the frobenius norm in estimating approximation errors of tensor id we reduce the problem of finding tensor ids to that of constructing interpolative decompositions of certain matrices these matrices are generated via randomized projection of the terms of the given tensor we provide cost estimates and several examples of the new approach to the reduction of separation rank

quasioptimality of maximum volume cross interpolation of tensors we consider a cross interpolation of high dimensional arrays in the tensor train format we prove that the maximum volume choice of the interpolation sets provides the quasioptimal interpolation accuracy that differs from the best possible accuracy by the factor which does not grow exponentially with dimension for nested interpolation sets we prove the interpolation property and propose greedy cross interpolation algorithms we justify the theoretical results and measure speed and accuracy of the proposed algorithm with numerical experiments

tensor train decomposition a simple nonrecursive form of the tensor decomposition in d dimensions is presented it does not inherently suffer from the curse of dimensionality it has asymptotically the same number of parameters as the canonical decomposition but it is stable and its computation is based on low rank approximation of auxiliary unfolding matrices the new form gives a clear and convenient way to implement all basic operations efficiently a fast rounding procedure is presented as well as basic linear algebra operations examples showing the benefits of the decomposition are given and the efficiency is demonstrated by the computation of the smallest eigenvalue of a 19 dimensional operator

fast multidimensional convolution in low rank tensor formats via cross approximation we propose a new cross conv algorithm for approximate computation of convolution in different low rank tensor formats tensor train tucker hierarchical tucker it has better complexity with respect to the tensor rank than previous approaches the new algorithm has a high potential impact in different applications the key idea is based on applying cross approximation in the frequency domain where convolution becomes a simple elementwise product we illustrate efficiency of our algorithm by computing the three dimensional newton potential and by presenting preliminary results for solution of the hartree fock equation on tensor product grids

wedderburn rank reduction and krylov subspace method for tensor approximation part 1 tucker case new algorithms are proposed for the tucker approximation of a 3 tensor accessed only through a tensor by vector by vector multiplication subroutine in the matrix case the krylov methods are methods of choice to approximate the dominant column and row subspaces of a sparse or structured matrix given through a matrix by vector operation using the wedderburn rank reduction formula we propose a matrix approximation algorithm that computes the krylov subspaces and can be generalized to 3 tensors the numerical experiments show that on quantum chemistry data the proposed tensor methods outperform the minimal krylov recursion of savas and elden

black box approximation of tensors in hierarchical tucker format we derive and analyse a scheme for the approximation of order d tensors a rn1 nd in the hierarchical h tucker format a dimension multilevel variant of the tucker format and strongly related to the tt tensor train format for a fixed rank parameter k the storage complexity of a tensor in h tucker format is odk3 k i 1dni and we present a heuristic algorithm that finds an approximation to a tensor in the h tucker format in odk4 log d k2 i 1dni by inspection of only odk3 log d k2 i 1dni entries under mild assumptions tensors in the h tucker format are reconstructed for general tensors we derive error bounds that are based on the approximability of matrices matricizations of the tensor by few outer products of its rows and columns the construction parallelizes with respect to the order d and we also propose an adaptive approach that aims at finding the rank parameter for a given target accuracy e automatically

on local convergence of alternating schemes for optimization of convex problems in the tensor train format alternating linear schemes als with the alternating least squares algorithm a notable special case provide one of the simplest and most popular choices for the treatment of optimization tasks by tensor methods an according adaptation of als for the recent tt tensor train format i v oseledets siam j sci comput 33 2011 pp 2295 2317 known in quantum computations as matrix product states has recently been investigated in s holtz t rohwedder and r schneider siam j sci comput 34 2012 pp a683 a713 with the present work the positive practical experience with tt als is backed up with an according local linear convergence theory for the optimization of convex functionals j the main assumption entering the proof is that the redundancy introduced by the tt parametrization tau matches the null space of the hessian of the induced functional j j circ tau and we give conditions under which this assumption can be expected to hold in particular this is the case if

riemannian optimization for high dimensional tensor completion tensor completion aims to reconstruct a high dimensional data set where the vast majority of entries is missing the assumption of low rank structure in the underlying original data allows us to cast the completion problem into an optimization problem restricted to the manifold of fixed rank tensors elements of this smooth embedded submanifold can be efficiently represented in the tensor train or matrix product states format with storage complexity scaling linearly with the number of dimensions we present a nonlinear conjugate gradient scheme within the framework of riemannian optimization which exploits this favorable scaling numerical experiments and comparison to existing methods show the effectiveness of our approach for the approximation of multivariate functions finally we show that our algorithm can obtain competitive reconstructions from uniform random sampling of few entries compared to adaptive sampling techniques such as cross approximation

numerical solution of the hartree fock equation in multilevel tensor structured format in this paper we describe a novel method for a robust and accurate iterative solution of the self consistent hartree fock equation in mathbb r 3 based on the idea of tensor structured computation of the electron density and the nonlinear hartree and nonlocal exchange operators at all steps of the iterative process we apply the self consistent field scf iteration to the galerkin discretization in a set of low separation rank basis functions that are solely specified by the respective values on a three dimensional cartesian grid the approximation error is estimated by o h 3 where h o n 1 is the mesh size of an n times n times n tensor grid while the numerical complexity to compute the galerkin matrices scales linearly in n log n we propose the tensor truncated version of the scf iteration using the traditional direct inversion in the iterative subspace scheme enhanced by the multilevel acceleration with the grid dependent termination criteria at each discretization level this implies that the overall computational cost scales almost linearly in the univariate problem size n numerical illustrations are presented for the all electron case of h 2 o and the pseudopotential case of ch 4 and ch 3 oh molecules the proposed scheme is not restricted to a priori given rank 1 basis sets allowing analytically integrable convolution transform with the newton kernel that opens further perspectives for promotion of the tensor structured methods in computational quantum chemistry