cramer rao induced bounds for candecomp parafac tensor decomposition this paper presents a cramer rao lower bound crlb on the variance of unbiased estimates of factor matrices in canonical polyadic cp or candecomp parafac cp decompositions of a tensor from noisy observations i e the tensor plus a random gaussian i i d tensor a novel expression is derived for a bound on the mean square angular error of factors along a selected dimension of a tensor of an arbitrary dimension the expression needs less operations for computing the bound o nr6 than the best existing state of the art algorithm o n3r6 operations where n and r are the tensor order and the tensor rank insightful expressions are derived for tensors of rank 1 and rank 2 of arbitrary dimension and for tensors of arbitrary dimension and rank where two factor matrices have orthogonal columns the results can be used as a gauge of performance of different approximate cp decomposition algorithms prediction of their accuracy and for checking stability of a given decomposition of a tensor condition whether the crlb is finite or not a novel expression is derived for a hessian matrix needed in popular damped gauss newton method for solving the cp decomposition of tensors with missing elements beside computing the crlb for these tensors the expression may serve for design of damped gauss newton algorithm for the decomposition

kruskal s uniqueness inequality is sharp abstract kruskal proved that a tensor in v 1 v 2 v m of rank r has a unique decomposition as a sum of r pure tensors if a certain inequality is satisfied we will show the uniqueness fails if the inequality is weakened we give two different constructions for counterexamples

constacyclic codes cocycles and a u v u v construction a connection between cohomology cocycles and constacyclic codes is explored it suggests an isomorphism between cyclic codes of length mn and a direct sum of m constacyclic codes of length n the isomorphism is used i to study the discrete fourier transforms and the decomposition of group ring codes ii to give a u v u v construction over gf q when q is odd the u v u v construction gives some of the best ternary cyclic codes and classes of nearly mds cyclic codes of length 2q 2 the symmetry of the construction is used to give a new proof that there are no cyclic self dual codes over gf q when q is odd

parameter estimation problems with singular information matrices the case of a singular fisher information matrix fim represents a significant complication for the theory of the cramer rao lower bound crb that is usually handled by resorting to the pseudoinverse of the fisher matrix we take a different approach in which the crb is derived as the solution to an unconstrained quadratic maximization problem which enables us to handle the singular case in a simple yet rigorous manner when the fisher matrix is singular except under unusual circumstances any estimator having the specified bias derivatives that figure in the crb must have infinite variance

a unified convergence analysis of block successive minimization methods for nonsmooth optimization the block coordinate descent bcd method is widely used for minimizing a continuous function f of several block variables at each iteration of this method a single block of variables is optimized while the remaining variables are held fixed to ensure the convergence of the bcd method the subproblem of each block variable needs to be solved to its unique global optimal unfortunately this requirement is often too restrictive for many practical scenarios in this paper we study an alternative inexact bcd approach which updates the variable blocks by successively minimizing a sequence of approximations of f which are either locally tight upper bounds of f or strictly convex local approximations of f the main contributions of this work include the characterizations of the convergence conditions for a fairly wide class of such methods especially for the cases where the objective functions are either nondifferentiable or nonconvex our results unify and extend the existing convergence results

brief paper a stability analysis with time delay of primal dual power control in optical links primal dual power control algorithms in optical networks must be designed properly to remain stable in the presence of time delays control algorithms at the signal transmitters continuously update their channel powers in response to dynamic pricing information from the network links the control algorithms optimize the optical signal to noise ratios osnrs of the channels in a distributive and non cooperative manner we consider the case of a single link with multiple channels and a constant time delay these results also apply to single sink general network configurations if the time delay represents the longest round trip time in the network we derive sufficient conditions for the stability of the closed loop system based on a tunable parameter in the control algorithm the work combines singular perturbation theory with lyapunov razumikhin time delay techniques we verify our results through simulations based on realistic network parameters

tensor decompositions for learning latent variable models this work considers a computationally and statistically efficient parameter estimation method for a wide class of latent variable models including gaussian mixture models hidden markov models and latent dirichlet allocation which exploits a certain tensor structure in their low order observable moments typically of second and third order specifically parameter estimation is reduced to the problem of extracting a certain orthogonal decomposition of a symmetric tensor derived from the moments this decomposition can be viewed as a natural generalization of the singular value decomposition for matrices although tensor decompositions are generally intractable to compute the decomposition of these specially structured tensors can be efficiently obtained by a variety of approaches including power iterations and maximization approaches similar to the case of matrices a detailed analysis of a robust tensor power method is provided establishing an analogue of wedin s perturbation theorem for the singular vectors of matrices this implies a robust and computationally tractable estimation approach for several popular latent variable models

on the non existence of optimal solutions and the occurrence of degeneracy in the candecomp parafac model the candecomp parafac cp model decomposes a three way array into a prespecified number of r factors and a residual array by minimizing the sum of squares of the latter it is well known that an optimal solution for cp need not exist we show that if an optimal cp solution does not exist then any sequence of cp factors monotonically decreasing the cp criterion value to its infimum will exhibit the features of a so called degeneracy that is the parameter matrices become nearly rank deficient and the euclidean norm of some factors tends to infinity we also show that the cp criterion function does attain its infimum if one of the parameter matrices is constrained to be column wise orthonormal

parallel factor analysis in sensor array processing this paper links multiple invariance sensor array processing mi sap to parallel factor parafac analysis which is a tool rooted in psychometrics and chemometrics parafac is a common name for low rank decomposition of three and higher way arrays this link facilitates the derivation of powerful identifiability results for mi sap shows that the uniqueness of single and multiple invariance esprit stems from uniqueness of low rank decomposition of three way arrays and allows tapping on the available expertise for fitting the parafac model the results are applicable to both data domain and subspace mi sap formulations the paper also includes a constructive uniqueness proof for a special parafac model

a theory of pseudoskeleton approximations abstract let an m n matrix a be approximated by a rank r matrix with an accuracy e we prove that it is possible to choose r columns and r rows of a forming a so called pseudoskeleton component which approximates a with o e r m n accuracy in the sense of the 2 norm on the way to this estimate we study the interconnection between the volume i e the determinant in the absolute value and the minimal singular value r of r r submatrices of an n r matrix with orthogonal columns we propose a lower bound better than one given by chandrasekaran and ipsen and by hong and pan for the maximum of r over all these submatrices and formulate a hypothesis on a tighter bound

computing the gradient in optimization algorithms for the cp decomposition in constant memory through tensor blocking the construction of the gradient of the objective function in gradient based optimization algorithms for computing an r term candecomp parafac cp decomposition of an unstructured dense tensor is a key computational kernel the best technique for efficiently implementing this operation has a memory consumption that scales linearly with the number of terms r and sublinearly with the number of elements of the tensor we consider a blockwise computation of the cp gradient reducing the memory requirements to a constant this reduction is achieved by a novel technique that we call implicit block unfoldings which combines the benefits of the block tensor unfoldings by ragnarsson and van loan siam j matrix anal appl 33 2012 pp 149 169 and the implicit unfoldings of phan tichavsk and cichocki ieee trans signal process 61 2013 pp 4834 4846 a heuristic algorithm for automatically choosing the division into subtensors is part of the proposed algorithm the throughput that can be attaine

an analytical constant modulus algorithm iterative constant modulus algorithms such as godard 1980 and cma have been used to blindly separate a superposition of cochannel constant modulus cm signals impinging on an antenna array these algorithms have certain deficiencies in the context of convergence to local minima and the retrieval of all individual cm signals that are present in the channel we show that the underlying constant modulus factorization problem is in fact a generalized eigenvalue problem and may be solved via a simultaneous diagonalization of a set of matrices with this new analytical approach it is possible to detect the number of cm signals present in the channel and to retrieve all of them exactly rejecting other non cm signals only a modest amount of samples is required the algorithm is robust in the presence of noise and is tested on measured data collected from an experimental set up

generalizing the column row matrix decomposition to multi way arrays abstract in this paper we provide two generalizations of the cur matrix decomposition y cur also known as pseudo skeleton approximation method 1 to the case of n way arrays tensors these generalizations which we called fiber sampling tensor decomposition types 1 and 2 fstd1 and fstd2 provide explicit formulas for the parameters of a rank r r r tucker representation the core tensor of size r r r and the matrix factors of sizes i n r n 1 2 n based only on some selected entries of the original tensor fstd1 uses p n 1 p r n mode fibers of the original tensor while fstd2 uses exactly r fibers in each mode as matrix factors as suggested by the existence theorem provided in oseledets et al 2008 2 with a core tensor defined in terms of the entries of a subtensor of size r r r for n 2 our results are reduced to the already known cur matrix decomposition where the core matrix is defined as the inverse of the intersection submatrix i e u w 1 additionally we provide an adaptive type algorithm for the selection of proper fibers in the fstd1 model which is useful for large scale applications several numerical results are presented showing the performance of our fstd1 adaptive algorithm compared to two recently proposed approximation methods for 3 way tensors

a flexible and efficient algorithmic framework for constrained matrix and tensor factorization we propose a general algorithmic framework for constrained matrix and tensor factorization which is widely used in signal processing and machine learning the new framework is a hybrid between alternating optimization ao and the alternating direction method of multipliers admm each matrix factor is updated in turn using admm hence the name ao admm this combination can naturally accommodate a great variety of constraints on the factor matrices and almost all possible loss measures for the fitting computation caching and warm start strategies are used to ensure that each update is evaluated efficiently while the outer ao framework exploits recent developments in block coordinate descent bcd type methods which help ensure that every limit point is a stationary point as well as faster and more robust convergence in practice three special cases are studied in detail non negative matrix tensor factorization constrained matrix tensor completion and dictionary learning extensive simulations and experiments with real data are used to showcase the effectiveness and broad applicability of the proposed framework

from k means to higher way co clustering multilinear decomposition with sparse latent factors co clustering is a generalization of unsupervised clustering that has recently drawn renewed attention driven by emerging data mining applications in diverse areas whereas clustering groups entire columns of a data matrix co clustering groups columns over select rows only i e it simultaneously groups rows and columns the concept generalizes to data boxes and higher way tensors for simultaneous grouping along multiple modes various co clustering formulations have been proposed but no workhorse analogous to k means has emerged this paper starts from k means and shows how co clustering can be formulated as a constrained multilinear decomposition with sparse latent factors for three and higher way data uniqueness of the multilinear decomposition implies that unlike matrix co clustering it is possible to unravel a large number of possibly overlapping co clusters a basic multi way co clustering algorithm is proposed that exploits multilinearity using lasso type coordinate updates various line search schemes are then introduced to speed up convergence and suitable modifications are proposed to deal with missing values the imposition of latent sparsity pays a collateral dividend it turns out that sequentially extracting one co cluster at a time is almost optimal hence the approach scales well for large datasets the resulting algorithms are benchmarked against the state of art in pertinent simulations and applied to measured data including the enron e mail corpus

blind parafac receivers for ds cdma systems this paper links the direct sequence code division multiple access ds cdma multiuser separation equalization detection problem to the parallel factor parafac model which is an analysis tool rooted in psychometrics and chemometrics exploiting this link it derives a deterministic blind parafac ds cdma receiver with performance close to non blind minimum mean squared error mmse the proposed parafac receiver capitalizes on code spatial and temporal diversity combining thereby supporting small sample sizes more users than sensors and or less spreading than users interestingly parafac does not require knowledge of spreading codes the specifics of multipath interchip interference doa calibration information finite alphabet constant modulus or statistical independence whiteness to recover the information bearing signals instead parafac relies on a fundamental result regarding the uniqueness of low rank three way array decomposition due to kruskal 1977 1988 and generalized herein to the complex valued case that guarantees identifiability of all relevant signals and propagation parameters these and other issues are also demonstrated in pertinent simulation experiments

esprit a subspace rotation approach to estimation of parameters of cisoids in noise the application of a subspace invariance approach esprit to the estimation of parameters frequencies and powers of cisoids in noise is described esprit exploits an underlying rotational invariance of signal subspaces spanned by two temporally displaced data sets the new approach has several advantages including improved resolution over pisarenko s technique for harmonic retrieval

tensor decomposition for signal processing and machine learning tensors or multiway arrays are functions of three or more indices i j k ldots similar to matrices two way arrays which are functions of two indices r c for row column tensors have a rich history stretching over almost a century and touching upon numerous disciplines but they have only recently become ubiquitous in signal and data analytics at the confluence of signal processing statistics data mining and machine learning this overview article aims to provide a good starting point for researchers and practitioners interested in learning about and working with tensors as such it focuses on fundamentals and motivation using various application examples aiming to strike an appropriate balance of breadth and depth that will enable someone having taken first graduate courses in matrix algebra and probability to get started doing research and or developing tensor algorithms and software some background in applied optimization is useful but not strictly required the material covered includes tensor rank and rank decomposition basic tensor factorization models and their relationships and properties including fairly good coverage of identifiability broad coverage of algorithms ranging from alternating optimization to stochastic gradient statistical performance analysis and applications ranging from source separation to collaborative filtering mixture and topic modeling classification and multilinear subspace learning

parallel algorithms for constrained tensor factorization via alternating direction method of multipliers tensor factorization has proven useful in a wide range of applications from sensor array processing to communications speech and audio signal processing and machine learning with few recent exceptions all tensor factorization algorithms were originally developed for centralized in memory computation on a single machine and the few that break away from this mold do not easily incorporate practically important constraints such as non negativity a new constrained tensor factorization framework is proposed in this paper building upon the alternating direction method of multipliers admom it is shown that this simplifies computations bypassing the need to solve constrained optimization problems in each iteration and it naturally leads to distributed algorithms suitable for parallel implementation this opens the door for many emerging big data enabled applications the methodology is exemplified using non negativity as a baseline constraint but the proposed framework can incorporate many other types of constraints numerical experiments are encouraging indicating that admom based non negative tensor factorization ntf has high potential as an alternative to state of the art approaches

fast alternating ls algorithms for high order candecomp parafac tensor factorizations candecomp parafac cp has found numerous applications in wide variety of areas such as in chemometrics telecommunication data mining neuroscience separated representations for an order n tensor most cp algorithms can be computationally demanding due to computation of gradients which are related to products between tensor unfoldings and khatri rao products of all factor matrices except one these products have the largest workload in most cp algorithms in this paper we propose a fast method to deal with this issue the method also reduces the extra memory requirements of cp algorithms as a result we can accelerate the standard alternating cp algorithms 20 30 times for order 5 and order 6 tensors and even higher ratios can be obtained for higher order tensors e g n 10 the proposed method is more efficient than the state of the art als algorithm which operates two modes at a time also2 in the eigenvector pls toolbox especially for tensors with order n 5 and high rank

tensor decompositions and applications this survey provides an overview of higher order tensor decompositions their applications and available software a tensor is a multidimensional or n way array decompositions of higher order tensors i e n way arrays with n geq 3 have applications in psycho metrics chemometrics signal processing numerical linear algebra computer vision numerical analysis data mining neuroscience graph analysis and elsewhere two particular tensor decompositions can be considered to be higher order extensions of the matrix singular value decomposition candecomp parafac cp decomposes a tensor as a sum of rank one tensors and the tucker decomposition is a higher order form of principal component analysis there are many other tensor decompositions including indscal parafac2 candelinc dedicom and paratuck2 as well as nonnegative variants of all of the above the n way toolbox tensor toolbox and multilinear engine are examples of software packages for working with tensors

uniqueness of nonnegative tensor approximations we show that for a nonnegative tensor a best nonnegative rank r approximation is almost always unique its best rank one approximation may always be chosen to be a best nonnegative rank one approximation and the set of nonnegative tensors with nonunique best rank one approximations forms an algebraic hypersurface we show that the last part holds true more generally for real tensors and thereby determine a polynomial equation so that a real or nonnegative tensor that does not satisfy this equation is guaranteed to have a unique best rank one approximation we also establish an analogue for real or nonnegative symmetric tensors in addition we prove a singular vector variant of the perron frobenius theorem for positive tensors and apply it to show that a best nonnegative rank r approximation of a positive tensor can never be obtained by deflation as an aside we verify that the euclidean distance ed discriminants of the segre variety and the veronese variety are hypersurfaces and give defining equations of these ed discriminants

the stability of nonlinear least squares problems and the cramer rao bound a number of problems of interest in signal processing can be reduced to nonlinear parameter estimation problems the traditional approach to studying the stability of these estimation problems is to demonstrate finiteness of the cramer rao bound crb for a given noise distribution we review an alternate deterministic notion of stability for the associated nonlinear least squares nls problem from the realm of nonlinear programming i e that the global minimizer of the least squares problem exists and varies smoothly with the noise furthermore we show that under mild conditions identifiability of the parameters along with a finite crb for the case of gaussian noise is equivalent to the deterministic stability of the nls problem finally we demonstrate the application of our result which is general to the problems of multichannel blind deconvolution and sinusoid retrieval to generate new stability results for these problems with little additional effort

a vandermonde code construction we use a vandermonde matrix to construct a code of length mn from m codes of length n a distance estimate for the construction is given when the starting codes are suitable constacyclic codes the constructed code is cyclic and from this we obtain new cyclic codes the results generalize those of hughes see ibid vol 46 p 674 80 2000 on the u spl upsi u spl upsi construction

lower bounds for parametric estimation with constraints a chapman robbins form of the barankin bound is used to derive a multiparameter cramer rao cr type lower bound on estimator error covariance when the parameter theta in r sup n is constrained to lie in a subset of the parameter space a simple form for the constrained cr bound is obtained when the constraint set theta sub c can be expressed as a smooth functional inequality constraint it is shown that the constrained cr bound is identical to the unconstrained cr bound at the regular points of theta sub c i e where no equality constraints are active on the other hand at those points theta in theta sub c where pure equality constraints are active the full rank fisher information matrix in the unconstrained cr bound must be replaced by a rank reduced fisher information matrix obtained as a projection of the full rank fisher matrix onto the tangent hyperplane of the full rank fisher matrix onto the tangent hyperplane of the constraint set at theta a necessary and sufficient condition involving the forms of the constraint and the likelihood function is given for the bound to be achievable and examples for which the bound is achieved are presented in addition to providing a useful generalization of the cr bound the results permit analysis of the gain in achievable mse performance due to the imposition of particular constraints on the parameter space without the need for a global reparameterization

some ternary and quaternary codes and associated sphere packings tables of good ternary and quaternary codes are presented and they are used in the construction of dense sphere packings results include 1 tables of the best ternary and quaternary constacyclic codes including cyclic codes up to block length 50 2 a class of optimal n 2 codes over gf q 3 the u v w mod 2u v mod u construction a new ternary code construction technique that can be used to construct the ternary reed muller codes and others and 4 tables of linear ternary and quaternary codes obtained by modifying and combining various codes here and in the literature packings are generated in even dimensions up to 100 using these codes and a sphere packing construction in dimensions 36 and 60 new record densities appear to have been achieved

non negative matrix factorization revisited uniqueness and algorithm for symmetric decomposition non negative matrix factorization nmf has found numerous applications due to its ability to provide interpretable decompositions perhaps surprisingly existing results regarding its uniqueness properties are rather limited and there is much room for improvement in terms of algorithms as well uniqueness aspects of nmf are revisited here from a geometrical point of view both symmetric and asymmetric nmf are considered the former being tantamount to element wise non negative square root factorization of positive semidefinite matrices new uniqueness results are derived e g it is shown that a sufficient condition for uniqueness is that the conic hull of the latent factors is a superset of a particular second order cone checking this condition is shown to be np complete yet this and other results offer insights on the role of latent sparsity in this context on the computational side a new algorithm for symmetric nmf is proposed which is very different from existing ones it alternates between procrustes rotation and projection onto the non negative orthant to find a non negative matrix close to the span of the dominant subspace simulation results show promising performance with respect to the state of art finally the new algorithm is applied to a clustering problem for co authorship data yielding meaningful and interpretable results

adaptive algorithms to track the parafac decomposition of a third order tensor the parafac decomposition of a higher order tensor is a powerful multilinear algebra tool that becomes more and more popular in a number of disciplines existing parafac algorithms are computationally demanding and operate in batch mode both serious drawbacks for on line applications when the data are serially acquired or the underlying model changes with time adaptive parafac algorithms that can track the sought decomposition at low complexity would be highly desirable this is a challenging task that has not been addressed in the literature and the topic of this paper given an estimate of the parafac decomposition of a tensor at instant t we propose two adaptive algorithms to update the decomposition at instant t 1 the new tensor being obtained from the old one after appending a new slice in the time dimension the proposed algorithms can yield estimation performance that is very close to that obtained via repeated application of state of art batch algorithms at orders of magnitude lower complexity the effectiveness of the proposed algorithms is illustrated using a mimo radar application tracking of directions of arrival and directions of departure as an example

a survey of multilinear subspace learning for tensor data increasingly large amount of multidimensional data are being generated on a daily basis in many applications this leads to a strong demand for learning algorithms to extract useful information from these massive data this paper surveys the field of multilinear subspace learning msl for dimensionality reduction of multidimensional data directly from their tensorial representations it discusses the central issues of msl including establishing the foundations of the field via multilinear projections formulating a unifying msl framework for systematic treatment of the problem examining the algorithmic aspects of typical msl solutions and categorizing both unsupervised and supervised msl algorithms into taxonomies lastly the paper summarizes a wide range of msl applications and concludes with perspectives on future research directions

bounds on mixed binary ternary codes upper and lower bounds are presented for the maximal possible size of mixed binary ternary error correcting codes a table up to length 13 is included the upper bounds are obtained by applying the linear programming bound to the product of two association schemes the lower bounds arise from a number of different constructions

generalizing caratheodory s uniqueness of harmonic parameterization to n dimensions consider a sum of f exponentials in n dimensions and let i sub n be the number of equispaced samples taken along the nth dimension it is shown that if the frequencies or decays along every dimension are distinct and spl sigma sub n 1 sup n i sub n spl ges 2f n 1 then the parameterization in terms of frequencies decays amplitudes and phases is unique the result can be viewed as generalizing a classic result of caratheodory to n dimensions the proof relies on a recent result regarding the uniqueness of low rank decomposition of n way arrays

on kruskal s uniqueness condition for the candecomp parafac decomposition let x be a real valued three way array the candecomp parafac cp decomposition is written as x under bar y under bar 1 y under bar r e where y r are rank 1 arrays and e under bar is a rest term each rank 1 array is defined by the outer product of three vectors a r b r and c r i e y ijk r a i r b j r c k t these vectors make up the r columns of the component matrices a b and c if 2r 2 is less than or equal to the sum of the k ranks of a b and c then the fitted part of the decomposition is unique up to a change in the order of the rank 1 arrays and rescaling counterscaling of each triplet of vectors a r b r c r forming a rank 1 array this classical result was shown by kruskal his proof is however rather inaccessible and does not seem intuitive in order to contribute to a better understanding of cp uniqueness this paper provides an accessible and intuitive proof of kruskal s condition the proof is both self contained and compact and can easily be adapted for the complex valued cp decomposition c 2006 elsevier inc all rights reserved

cramer rao lower bounds for low rank decomposition of multidimensional arrays unlike low rank matrix decomposition which is generically nonunique for rank greater than one low rank three and higher dimensional array decomposition is unique provided that the array rank is lower than a certain bound and the correct number of components equal to array rank is sought in the decomposition parallel factor parafac analysis is a common name for low rank decomposition of higher dimensional arrays this paper develops cramer rao bound crb results for low rank decomposition of three and four dimensional 3 d and 4 d arrays illustrates the behavior of the resulting bounds and compares alternating least squares algorithms that are commonly used to compute such decompositions with the respective crbs simple to check necessary conditions for a unique low rank decomposition are also provided

fundamentals of statistical signal processing estimation theory minimum variance unbiased estimation cramer rao lower bound linear models general minimum variance unbiased estimation best linear unbiased estimators maximum likelihood estimation least squares method of moments the bayesian philosophy general bayesian estimators linear bayesian estimators kalman filters summary of estimators extension for complex data and parameters

krylov type methods for tensor computations i several krylov type procedures are introduced that generalize matrix krylov methods for tensor computations they are denoted minimal krylov recursion maximal krylov recursion and contracted tensor product krylov recursion it is proved that for a given tensor a with multilinear rank p q r the minimal krylov recursion extracts the correct subspaces associated to the tensor in p q r number of tensor vector vector multiplications an optimized minimal krylov procedure is described that for a given multilinear rank of an approximation produces a better approximation than the standard minimal recursion we further generalize the matrix krylov decomposition to a tensor krylov decomposition the tensor krylov methods are intended for the computation of low multilinear rank approximations of large and sparse tensors but they are also useful for certain dense and structured tensors for computing their higher order singular value decompositions or obtaining starting points for the best low rank computations of tensors a set of numerical experiments using real world and synthetic data sets illustrate some of the properties of the tensor krylov methods

multilinear discriminant analysis for face recognition there is a growing interest in subspace learning techniques for face recognition however the excessive dimension of the data space often brings the algorithms into the curse of dimensionality dilemma in this paper we present a novel approach to solve the supervised dimensionality reduction problem by encoding an image object as a general tensor of second or even higher order first we propose a discriminant tensor criterion whereby multiple interrelated lower dimensional discriminative subspaces are derived for feature extraction then a novel approach called k mode optimization is presented to iteratively learn these subspaces by unfolding the tensor along different tensor directions we call this algorithm multilinear discriminant analysis mda which has the following characteristics 1 multiple interrelated subspaces can collaborate to discriminate different classes 2 for classification problems involving higher order tensors the mda algorithm can avoid the curse of dimensionality dilemma and alleviate the small sample size problem and 3 the computational cost in the learning stage is reduced to a large extent owing to the reduced data dimensions in k mode optimization we provide extensive experiments on orl cmu pie and feret databases by encoding face images as second or third order tensors to demonstrate that the proposed mda algorithm based on higher order tensors has the potential to outperform the traditional vector based subspace learning algorithms especially in the cases with small sample sizes

optimization based algorithms for tensor decompositions canonical polyadic decomposition decomposition in rank l_r l_r 1 terms and a new generalization the canonical polyadic and rank l r l r 1 block term decomposition cpd and btd respectively are two closely related tensor decompositions the cpd and recently btd are important tools in psychometrics chemometrics neuroscience and signal processing we present a decomposition that generalizes these two and develop algorithms for its computation among these algorithms are alternating least squares schemes several general unconstrained optimization techniques and matrix free nonlinear least squares methods in the latter we exploit the structure of the jacobian s gramian to reduce computational and memory cost combined with an effective preconditioner numerical experiments confirm that these methods are among the most efficient and robust currently available for computing the cpd rank l r l r 1 btd and their generalized decomposition

rank and optimal computation of generic tensors abstract the typical rank maximal border rank of tensors of a given size and the set of optimal bilinear computations of typical tensors of a given rank are investigated for the size n n 3 with n odd the complement of the set of tensors of maximal border rank is a hypersurface its equation is given

tensor train decomposition a simple nonrecursive form of the tensor decomposition in d dimensions is presented it does not inherently suffer from the curse of dimensionality it has asymptotically the same number of parameters as the canonical decomposition but it is stable and its computation is based on low rank approximation of auxiliary unfolding matrices the new form gives a clear and convenient way to implement all basic operations efficiently a fast rounding procedure is presented as well as basic linear algebra operations examples showing the benefits of the decomposition are given and the efficiency is demonstrated by the computation of the smallest eigenvalue of a 19 dimensional operator

almost sure identifiability of multidimensional harmonic retrieval two dimensional 2 d and more generally multidimensional harmonic retrieval is of interest in a variety of applications including transmitter localization and joint time and frequency offset estimation in wireless communications the associated identifiability problem is key in understanding the fundamental limitations of parametric methods in terms of the number of harmonics that can be resolved for a given sample size consider a mixture of 2 d exponentials each parameterized by amplitude phase and decay rate plus frequency in each dimension suppose that i equispaced samples are taken along one dimension and likewise j along the other dimension we prove that if the number of exponentials is less than or equal to roughly ij 4 then assuming sampling at the nyquist rate or above the parameterization is almost surely identifiable this is significant because the best previously known achievable bound was roughly i j 2 for example consider i j 32 our result yields 256 versus 32 identifiable exponentials we also generalize the result to n dimensions proving that the number of exponentials that can be resolved is proportional to total sample size

quasi newton methods on grassmannians and multilinear approximations of tensors in this paper we proposed quasi newton and limited memory quasi newton methods for objective functions defined on grassmannians or a product of grassmannians specifically we defined bfgs and limited memory bfgs updates in local and global coordinates on grassmannians or a product of these we proved that when local coordinates are used our bfgs updates on grassmannians share the same optimality property as the usual bfgs updates on euclidean spaces when applied to the best multilinear rank approximation problem for general and symmetric tensors our approach yields fast robust and accurate algorithms that exploit the special grassmannian structure of the respective problems and which work on tensors of large dimensions and arbitrarily high order extensive numerical experiments are included to substantiate our claims

exact probabilities for typical ranks of 2 2 2 and 3 3 2 tensors we show that the probability to be of rank 2 for a 2 2 2 tensor with elements from a standard normal distribution is 4 and that the probability to be of rank 3 for a 3 3 2 tensor is 1 2 in the

pseudocyclic maximum distance separable codes the n k pseudocyclic maximum distance separable mds codes modulo x sup n a over gf q are considered suppose that n is a divisor of q 1 if n is odd pseudocyclic mds codes exist for all k however if n is even nontrivial pseudocyclic mds codes exist for odd k but not for even k if a is a quadratic residue in gf q and they exist for even k but not for odd k if a is not a quadratic residue in gf q also considered is the case when n is a divisor of q 1 and it is shown that pseudocyclic mds codes exist if and only if the multiplicative order of a divides q 1 n and that when this condition is satisfied such codes exist for all k if the condition is not satisfied every pseudocyclic code of length n is the result of interleaving a shorter pseudocyclic code