discriminant subspace analysis a fukunaga koontz approach the fisher linear discriminant fld is commonly used in pattern recognition it finds a linear subspace that maximally separates class patterns according to the fisher criterion several methods of computing the fld have been proposed in the literature most of which require the calculation of the so called scatter matrices in this paper we bring a fresh perspective to fld via the fukunaga koontz transform fkt we do this by decomposing the whole data space into four subspaces with different discriminabilities as measured by eigenvalue ratios by connecting the eigenvalue ratio with the generalized eigenvalue we show where the fisher criterion is maximally satisfied we prove the relationship between fld and fkt analytically and propose a unified framework to understanding some existing work furthermore we extend our theory to the multiple discriminant analysis mda this is done by transforming the data into intraclass and extraclass spaces followed by maximizing the bhattacharyya distance based on our fkt analysis we identify the discriminant subspaces of mda fkt and propose an efficient algorithm which works even when the scatter matrices are singular or too large to be formed our method is general and may be applied to different pattern recognition problems we validate our method by experimenting on synthetic and real data

robust regression discriminative methods e g kernel regression svm have been extensively used to solve problems such as object recognition image alignment and pose estimation from images these methods typically map image features mathbf x to continuous e g pose or discrete e g object category values a major drawback of existing discriminative methods is that samples are directly projected onto a subspace and hence fail to account for outliers common in realistic training sets due to occlusion specular reflections or noise it is important to notice that existing discriminative approaches assume the input variables mathbf x to be noise free thus discriminative methods experience significant performance degradation when gross outliers are present despite its obvious importance the problem of robust discriminative learning has been relatively unexplored in computer vision this paper develops the theory of robust regression rr and presents an effective convex approach that uses recent advances on rank minimization the framework applies to a variety of problems in computer vision including robust linear discriminant analysis regression with missing data and multi label classification several synthetic and real examples with applications to head pose estimation from images image and video classification and facial attribute classification with missing data are used to illustrate the benefits of rr

interpreting canonical correlation analysis through biplots of structure correlations and weights this paper extends the biplot technique to canonical correlation analysis and redundancy analysis the plot of structure correlations is shown to the optimal for displaying the pairwise correlations between the variables of the one set and those of the second the link between multivariate regression and canonical correlation analysis redundancy analysis is exploited for producing an optimal biplot that displays a matrix of regression coefficients this plot can be made from the canonical weights of the predictors and the structure correlations of the criterion variables an example is used to show how the proposed biplots may be interpreted

an example of a generalized brownian motion we present an example of a generalized brownian motion it is given by creation and annihilation operators on a twisted fock space ofl2 these operators fulfill for a fixed 1 1 the relationsc f c g c g c f f g 1 f g l2 we show that the distribution of these operators with respect to the vacuum expectation is a generalized gaussian distribution in the sense that all moments can be calculated from the second moments with the help of a combinatorial formula we also indicate that our brownian motion is one component of ann dimensional brownian motion which is invariant under the quantum groups u n of woronowicz with v2

canonical correlation analysis and reduced rank regression in autoregressive models when the rank of the autoregression matrix is unrestricted the maximum likelihood estimator under normality is the least squares estimator when the rank is restricted the maximum likelihood estimator is composed of the eigenvectors of the effect covariance matrix in the metric of the error covariance matrix corresponding to the largest eigenvalues anderson t w 1951 ann math statist 22 327 351 the asymptotic distribution of these two covariance matrices under normality is obtained and is used to derive the asymptotic distributions of the eigenvectors and eigenvalues under normality these asymptotic distributions differ from the asymptotic distributions when the regressors are independent variables the asymptotic distribution of the reduced rank regression is the asymptotic distribution of the least squares estimator with some restrictions hence the covariance of the reduced rank regression is smaller than that of the least squares estimator this result does not depend on normality

partial envelopes for efficient estimation in multivariate linear regression we introduce the partial envelope model which leads to a parsimonious method for multivariate linear regression when some of the predictors are of special interest it has the potential to achieve massive efficiency gains compared with the standard model in the estimation of the coefficients for the selected predictors the partial envelope model is a variation on the envelope model proposed by cook et al 2010 but as it focuses on part of the predictors it has looser restrictions and can further improve the efficiency we develop maximum likelihood estimation for the partial envelope model and discuss applications of the bootstrap an example is provided to illustrate some of its operating characteristics

ensemble manifold regularized sparse low rank approximation for multiview feature embedding in computer vision and pattern recognition researches the studied objects are often characterized by multiple feature representations with high dimensionality thus it is essential to encode that multiview feature into a unified and discriminative embedding that is optimal for a given task to address this challenge this paper proposes an ensemble manifold regularized sparse low rank approximation emr slra algorithm for multiview feature embedding the emr slra algorithm is based on the framework of least squares component analysis in particular the low dimensional feature representation and the projection matrix are obtained by the low rank approximation of the concatenated multiview feature matrix by considering the complementary property among multiple features emr slra simultaneously enforces the ensemble manifold regularization on the output feature embedding in order to further enhance its robustness against the noise the group sparsity is introduced into the objective formulation to impose direct noise reduction on the input multiview feature matrix since there is no closed form solution for emr slra this paper provides an efficient emr slra optimization procedure to obtain the output feature embedding experiments on the pattern recognition applications confirm the effectiveness of the emr slra algorithm compare with some other multiview feature dimensionality reduction approaches highlightsthis paper proposes a novel emr slra algorithm for multiview feature embedding the least squares component analysis is generalized to multiview version the ensemble manifold regularization is enforced to explore the complementarity the group sparsity is introduced to promote the robustness against the noise an efficient iterative procedure is developed to solve emr slra

a novel framework for parsimonious multivariate analysis abstract this paper proposes a framework in which a multivariate analysis method mva guides a selection of input variables that leads to a sparse feature extraction this framework called parsimonious mva is specially suited for high dimensional data such as gene arrays digital pictures etc the feature selection relies on the analysis of consistency in the behaviour of the input variables through the elements of an ensemble of mva projection matrices the ensemble is constructed following a bootstrap that builds on an efficient and generalized mva formulation that covers pca cca and opls moreover it allows the estimation of the relative relevance of each selected input variable experimental results point out that the features extracted by the parsimonious mva have excellent discrimination power comparing favorably with state of the art methods and are potentially useful to build interpretable features besides the parsimonious feature extractor is shown to be robust against to parameter selection as we all computationally efficient

parsimonious tensor response regression abstractaiming at abundant scientific and engineering data with not only high dimensionality but also complex structure we study the regression problem with a multidimensional array tensor response and a vector predictor applications include among others comparing tensor images across groups after adjusting for additional covariates which is of central interest in neuroimaging analysis we propose parsimonious tensor response regression adopting a generalized sparsity principle it models all voxels of the tensor response jointly while accounting for the inherent structural information among the voxels it effectively reduces the number of free parameters leading to feasible computation and improved interpretation we achieve model estimation through a nascent technique called the envelope method which identifies the immaterial information and focuses the estimation based upon the material information in the tensor response we demonstrate that the resulting estimator is asymptotically efficient

can the strengths of aic and bic be shared a conflict between model indentification and regression estimation a traditional approach to statistical inference is to identify the true or best model first with little or no consideration of the specific goal of inference in the model identification stage can the pursuit of the true model also lead to optimal regression estimation in model selection it is well known that bic is consistent in selecting the true model and aic is minimax rate optimal for estimating the regression function a recent promising direction is adaptive model selection in which in contrast to aic and bic the penalty term is data dependent some theoretical and empirical results have been obtained in support of adaptive model selection but it is still not clear if it can really share the strengths of aic and bic model combining or averaging has attracted increasing attention as a means to overcome the model selection uncertainty can bayesian model averaging be optimal for estimating the regression function in a minimax sense we show that the answers to these questions are basically in the negative for any model selection criterion to be consistent it must behave suboptimally for estimating the regression function in terms of minimax rate of covergence and bayesian model averaging cannot be minimax rate optimal for regression estimation

fock space associated to coxeter groups of type b abstract in this article we construct a generalized gaussian process coming from coxeter groups of type b it is given by creation and annihilation operators on an q fock space which satisfy the commutation relation b q x b q y q b q y b q x x y i x y q 2 n where x y are elements of a complex hilbert space with a self adjoint involution x x and n is the number operator with respect to the grading on the q fock space we give an estimate of the norms of creation operators we show that the distribution of the operators b q x b q x with respect to the vacuum expectation becomes a generalized gaussian distribution in the sense that all mixed moments can be calculated from the second moments with the help of a combinatorial formula related with set partitions our generalized gaussian distribution is associated to the orthogonal polynomials called the q meixner pollaczek polynomials yielding the q hermite polynomials when 0 and free meixner polynomials when q 0

retargeted least squares regression algorithm this brief presents a framework of retargeted least squares regression relsr for multicategory classification the core idea is to directly learn the regression targets from data other than using the traditional zero one matrix as regression targets the learned target matrix can guarantee a large margin constraint for the requirement of correct classification for each data point compared with the traditional least squares regression lsr and a recently proposed discriminative lsr models relsr is much more accurate in measuring the classification error of the regression model furthermore relsr is a single and compact model hence there is no need to train two class binary machines that are independent of each other the convex optimization problem of relsr is solved elegantly and efficiently with an alternating procedure including regression and retargeting as substeps the experimental evaluation over a range of databases identifies the validity of our method

laplacian eigenmaps for dimensionality reduction and data representation one of the central problems in machine learning and pattern recognition is to develop appropriate representations for complex data we consider the problem of constructing a representation for data lying on a low dimensional manifold embedded in a high dimensional space drawing on the correspondence between the graph laplacian the laplace beltrami operator on the manifold and the connections to the heat equation we propose a geometrically motivated algorithm for representing the high dimensional data the algorithm provides a computationally efficient approach to nonlinear dimensionality reduction that has locality preserving properties and a natural connection to clustering some potential applications and illustrative examples are discussed

cm theory of damped wave equations with stabilisation abstract the aim of this note is to extend the energy decay estimates from j wirth wave equations with time dependent dissipation i non effective dissipation j differential equations 222 2006 487 514 to a broader class of time dependent dissipation including very fast oscillations this is achieved using stabilisation conditions on the coefficient in the spirit of f hirosawa on the asymptotic behavior of the energy for wave equations with time depending coefficients math ann 339 4 2007 819 839

global solvability for kirchhoff equation in special classes of non analytic functions abstract in this paper we derive the following two properties the first one is a precise representation of wkb solution to the cauchy problem of a linear wave equation with a variable coefficient with respect to time and the second one is the global solvability for kirchhoff equation in some special classes of nonreal analytic functions which is proved by grace of the first property

learning eigenfunctions links spectral embedding and kernel pca in this letter we show a direct relation between spectral embedding methods and kernel principal components analysis and how both are special cases of a more general learning problem learning the principal eigenfunctions of an operator defined from a kernel and the unknown data generating density whereas spectral embedding methods provided only coordinates for the training points the analysis justifies a simple extension to out of sample examples the nystrom formula for multidimensional scaling mds spectral clustering laplacian eigenmaps locally linear embedding lle and isomap the analysis provides for all such spectral embedding methods the definition of a loss function whose empirical average is minimized by the traditional algorithms the asymptotic expected value of that loss defines a generalization performance and clarifies what these algorithms are trying to learn experiments with lle isomap spectral clustering and mds show that this out of sample embedding formula generalizes well with a level of error comparable to the effect of small perturbations of the training set on the embedding

eigenfaces vs fisherfaces recognition using class specific linear projection we develop a face recognition algorithm which is insensitive to large variation in lighting direction and facial expression taking a pattern classification approach we consider each pixel in an image as a coordinate in a high dimensional space we take advantage of the observation that the images of a particular face under varying illumination but fixed pose lie in a 3d linear subspace of the high dimensional image space if the face is a lambertian surface without shadowing however since faces are not truly lambertian surfaces and do indeed produce self shadowing images will deviate from this linear subspace rather than explicitly modeling this deviation we linearly project the image into a subspace in a manner which discounts those regions of the face with large deviation our projection method is based on fisher s linear discriminant and produces well separated classes in a low dimensional subspace even under severe variation in lighting and facial expressions the eigenface technique another method based on linearly projecting the image space to a low dimensional subspace has similar computational requirements yet extensive experimental results demonstrate that the proposed fisherface method has error rates that are lower than those of the eigenface technique for tests on the harvard and yale face databases

optimal selection of reduced rank estimators of high dimensional matrices we introduce a new criterion the rank selection criterion rsc for selecting the optimal reduced rank estimator of the coefficient matrix in multivariate response regression models the corresponding rsc estimator minimizes the frobenius norm of the fit plus a regularization term proportional to the number of parameters in the reduced rank model the rank of the rsc estimator provides a consistent estimator of the rank of the coefficient matrix in general the rank of our estimator is a consistent estimate of the effective rank which we define to be the number of singular values of the target matrix that are appropriately large the consistency results are valid not only in the classic asymptotic regime when n the number of responses and p the number of predictors stay bounded and m the number of observations grows but also when either or both n and p grow possibly much faster than m we establish minimax optimal bounds on the mean squared errors of our estimators our finite sample performance bounds for the rsc estimator show that it achieves the optimal balance between the approximation error and the penalty term furthermore our procedure has very low computational complexity linear in the number of candidate models making it particularly appealing for large scale problems we contrast our estimator with the nuclear norm penalized least squares nnp estimator which has an inherently higher computational complexity than rsc for multivariate regression models we show that nnp has estimation properties similar to those of rsc albeit under stronger conditions however it is not as parsimonious as rsc we offer a simple correction of the nnp estimator which leads to consistent rank estimation we verify and illustrate our theoretical findings via an extensive simulation study

normalized cuts and image segmentation we propose a novel approach for solving the perceptual grouping problem in vision rather than focusing on local features and their consistencies in the image data our approach aims at extracting the global impression of an image we treat image segmentation as a graph partitioning problem and propose a novel global criterion the normalized cut for segmenting the graph the normalized cut criterion measures both the total dissimilarity between the different groups as well as the total similarity within the groups we show that an efficient computational technique based on a generalized eigenvalue problem can be used to optimize this criterion we applied this approach to segmenting static images as well as motion sequences and found the results to be very encouraging

discriminative elastic net regularized linear regression in this paper we aim at learning compact and discriminative linear regression models linear regression has been widely used in different problems however most of the existing linear regression methods exploit the conventional zero one matrix as the regression targets which greatly narrows the flexibility of the regression model another major limitation of these methods is that the learned projection matrix fails to precisely project the image features to the target space due to their weak discriminative capability to this end we present an elastic net regularized linear regression enlr framework and develop two robust linear regression models which possess the following special characteristics first our methods exploit two particular strategies to enlarge the margins of different classes by relaxing the strict binary targets into a more feasible variable matrix second a robust elastic net regularization of singular values is introduced to enhance the compactness and effectiveness of the learned projection matrix third the resulting optimization problem of enlr has a closed form solution in each iteration which can be solved efficiently finally rather than directly exploiting the projection matrix for recognition our methods employ the transformed features as the new discriminate representations to make final image classification compared with the traditional linear regression model and some of its variants our method is much more accurate in image classification extensive experiments conducted on publicly available data sets well demonstrate that the proposed framework can outperform the state of the art methods the matlab codes of our methods can be available at http www yongxu org lunwen html

realizability of a model in infinite statistics following greenberg and others we study a space with a collection of operatorsa k satisfying the q mutator relations a l a k a l k l corresponding forq 1 to classical bose and fermi statistics we show that then n matrixa n q representing the scalar products ofn particle states is positive definite for alln ifq lies between 1 and 1 so that the commutator relations have a hilbert space representation in this case this has also been proved by fivel and by bozejko and speicher we also give an explicit factorization ofa n q as a product of matrices of the form 1 q jt 1 with 1 j n andt a permutation matrix in particular a n q is singular if and only ifq m 1 for some integerm of the formk 2 k 2 k n

the design and use of steerable filters the authors present an efficient architecture to synthesize filters of arbitrary orientations from linear combinations of basis filters allowing one to adaptively steer a filter to any orientation and to determine analytically the filter output as a function of orientation steerable filters may be designed in quadrature pairs to allow adaptive control over phase as well as orientation the authors show how to design and steer the filters and present examples of their use in the analysis of orientation and phase angularly adaptive filtering edge detection and shape from shading one can also build a self similar steerable pyramid representation the same concepts can be generalized to the design of 3 d steerable filters

a survey of kernel and spectral methods for clustering clustering algorithms are a useful tool to explore data structures and have been employed in many disciplines the focus of this paper is the partitioning clustering problem with a special interest in two recent approaches kernel and spectral methods the aim of this paper is to present a survey of kernel and spectral clustering methods two approaches able to produce nonlinear separating hypersurfaces between clusters the presented kernel clustering methods are the kernel version of many classical clustering algorithms e g k means som and neural gas spectral clustering arise from concepts in spectral graph theory and the clustering problem is configured as a graph cut problem where an appropriate objective function has to be optimized an explicit proof of the fact that these two paradigms have the same objective is reported since it has been proven that these two seemingly different approaches have the same mathematical foundation besides fuzzy kernel clustering methods are presented as extensions of kernel k means clustering algorithm

a test of incomplete additivity in the multiplicative interaction model abstract consider the multiplicative interaction model defined by yij i j i j ij i 1 2 t j 1 2 b where it is assumed that i i j j i i j j 0 and i i 2 j j 2 1 it is also assumed that the ij are distributed nid 0 2 this article derives the likelihood ratio test of h 0 h 0 and g 0 vs ha h 0 or g 0 where h is a q t matrix of row contrasts of rank q and g is a r b matrix of row constrasts of rank r an approximation to the critical points of the test statistic is given and tables are given for a few selected values of b t q and r an improved estimator of 2 is derived and all results are illustrated with an example

robust canonical discriminant analysis a method for robust canonical discriminant analysis via two robust objective loss functions is discussed these functions are useful to reduce the influence of outliers in the data majorization is used at several stages of the minimization procedure to obtain a monotonically convergent algorithm an advantage of the proposed method is that it allows for optimal scaling of the variables in a simulation study it is shown that under the presence of outliers the robust functions outperform the ordinary least squares function both when the underlying structure is linear in the variables as when it is nonlinear furthermore the method is illustrated with empirical data

q gaussian processes non commutative and classical aspects we examine for 1 q 1 q gaussian processes i e families of operators non commutative random variables where the a t fulfill the q commutation relations for some covariance function equipped with the vacuum expectation state we show that there is a q analogue of the gaussian functor of second quantization behind these processes and that this structure can be used to translate questions on q gaussian processes into corresponding and much simpler questions in the underlying hilbert space in particular we use this idea to show that a large class of q gaussian processes possesses a non commutative kind of markov property which ensures that there exist classical versions of these non commutative processes this answers an old question of frisch and bourret fb

principal component analysis with complex kernel the widely linear model nonlinear complex representations via the use of complex kernels can be applied to model and capture the nonlinearities of complex data even though the theoretical tools of complex reproducing kernel hilbert spaces crkhs have been recently successfully applied to the design of digital filters and regression and classification frameworks there is a limited research on component analysis and dimensionality reduction in crkhs the aim of this brief is to properly formulate the most popular component analysis methodology i e principal component analysis pca in crkhs in particular we define a general widely linear complex kernel pca framework furthermore we show how to efficiently perform widely linear pca in small sample sized problems finally we show the usefulness of the proposed framework in robust reconstruction using euler data representation

sparse reduced rank regression for simultaneous dimension reduction and variable selection the reduced rank regression is an effective method in predicting multiple response variables from the same set of predictor variables it reduces the number of model parameters and takes advantage of interrelations between the response variables and hence improves predictive accuracy we propose to select relevant variables for reduced rank regression by using a sparsity inducing penalty we apply a group lasso type penalty that treats each row of the matrix of the regression coefficients as a group and show that this penalty satisfies certain desirable invariance properties we develop two numerical algorithms to solve the penalized regression problem and establish the asymptotic consistency of the proposed method in particular the manifold structure of the reduced rank regression coefficient matrix is considered and studied in our theoretical analysis in our simulation study and real data analysis the new method is compared with several existing variable selection methods for multivariate regression

energy estimates at infinity for hyperbolic equations with oscillating coefficients abstract we study the behaviour for t of the energy of the solutions to the cauchy problem for some strictly hyperbolic second order equations with coefficients very rapidly oscillating

principal component analysis with missing data and its application to polyhedral object modeling observation based object modeling often requires integration of shape descriptions from different views to overcome the problems of errors and their accumulation we have developed a weighted least squares wls approach which simultaneously recovers object shape and transformation among different views without recovering interframe motion we show that object modeling from a range image sequence is a problem of principal component analysis with missing data pcamd which can be generalized as a wls minimization problem an efficient algorithm is devised after we have segmented planar surface regions in each view and tracked them over the image sequence we construct a normal measurement matrix of surface normals and a distance measurement matrix of normal distances to the origin for all visible regions over the whole sequence of views respectively these two matrices which have many missing elements due to noise occlusion and mismatching enable us to formulate multiple view merging as a combination of two wls problems a two step algorithm is presented after surface equations are extracted spatial connectivity among the surfaces is established to enable the polyhedral object model to be constructed experiments using synthetic data and real range images show that our approach is robust against noise and mismatching and generates accurate polyhedral object models

une theorie combinatoire des series formelles abstract this paper presents a combinatorial theory of formal power series the combinatorial interpretation of formal power series is based on the concept of species of structures a categorical approach is used to formulate it a new proof of cayley s formula for the number of labelled trees is given as well as a new combinatorial proof due to g labelle of lagrange s inversion formula polya s enumeration theory of isomorphism classes of structures is entirely renewed recursive methods for computing cycle index polynomials are described a combinatorial version of the implicit function theorem is stated and proved the paper ends with general considerations on the use of coalgebras in combinatorics

testing subhypotheses in the multiplicative interaction model the problem of analyzing a two way cross classified treatment structure with only one observation per treatment combination is considered a test procedure is given that will enable the data analyst to determine subareas of the data in which the data are additive the procedure is developed by assuming that a multiplicative interaction model adequately fits the data such a model is given by y ij i j i j ij where i 1 2 t and j 1 2 b it is assumed that the ij are distributed independently and normally with mean zero and variance 2 the other parameters are assumed to be unknown constants in general the problem may be stated as one of testing h 0 h 0 versus h a h 0 where 1 t and h is a q t contrast matrix a likelihood ratio statistic for this testing problem is derived and approximate critical points are given for the cases q 1 and q 2 the procedures are illustrated with an example

a q deformation of the gauss distribution the q deformed commutation relation aa qa a 1 for the harmonic oscillator is considered with q 1 1 an explicit representation generalizing the bargmann representation of analytic functions on the complex plane is constructed in this representation the distribution of a a in the vacuum state is explicitly calculated this distribution is to be regarded as the natural q deformation of the gaussian

blow up of the energy at infinity for 2 by 2 systems abstract the goal of this paper is to study the long time behavior of the energy of solutions to 2 by 2 linear hyperbolic systems some blow up results at t are given for a large class of initial data of the cauchy problem we shall prove the optimality of these results for a special class of systems we present also an example of generalized energy conservation under a c m stabilization condition

weighted graph cuts without eigenvectors a multilevel approach a variety of clustering algorithms have recently been proposed to handle data that is not linearly separable spectral clustering and kernel k means are two of the main methods in this paper we discuss an equivalence between the objective functions used in these seemingly different methods in particular a general weighted kernel k means objective is mathematically equivalent to a weighted graph clustering objective we exploit this equivalence to develop a fast high quality multilevel algorithm that directly optimizes various weighted graph clustering objectives such as the popular ratio cut normalized cut and ratio association criteria this eliminates the need for any eigenvector computation for graph clustering problems which can be prohibitive for very large graphs previous multilevel graph partitioning methods such as metis have suffered from the restriction of equal sized clusters our multilevel algorithm removes this restriction by using kernel k means to optimize weighted graph cuts experimental results show that our multilevel algorithm outperforms a state of the art spectral clustering algorithm in terms of speed memory usage and quality we demonstrate that our algorithm is applicable to large scale clustering tasks such as image segmentation social network analysis and gene network analysis

reduced rank models for multiple time series summary this paper is concerned with the investigation of reduced rank coefficient models for multiple time series in particular autoregressive processes which have a structure to their coefficient matrices similar to that of classical multivariate reduced rank regression are studied in detail the estimation of parameters and associated asymptotic theory are derived the exact correspondence between the reduced rank regression procedure for multiple autoregressive processes and the canonical analysis of box tiao 1977 is briefly indicated to illustrate the methods u s hog data are considered

sparse envelope model efficient estimation and response variable selection in multivariate linear regression the envelope model allows efficient estimation in multivariate linear regression in this paper we propose the sparse envelope model which is motivated by applications where some response variables are invariant with respect to changes of the predictors and have zero regression coefficients the envelope estimator is consistent but not sparse and in many situations it is important to identify the response variables for which the regression coefficients are zero the sparse envelope model performs variable selection on the responses and preserves the efficiency gains offered by the envelope model response variable selection arises naturally in many applications but has not been studied as thoroughly as predictor variable selection in this paper we discuss response variable selection in both the standard multivariate linear regression and the envelope contexts in response variable selection even if a response has zero coefficients it should still be retained to improve the estimation efficiency of the nonzero coefficients this is different from the practice in predictor variable selection we establish consistency and the oracle property and obtain the asymptotic distribution of the sparse envelope estimator

long time asymptotics for 2 by 2 hyperbolic systems the goal of this paper is to study the behavior of the energy for 2 by 2 strictly hyperbolic systems on the one hand we are interested in generalized energy conservation which excludes blow up and decay of the energy for t on the other hand we present scattering results which take account of terms of order zero

linear dimensionality reduction survey insights and generalizations linear dimensionality reduction methods are a cornerstone of analyzing high dimensional data due to their simple geometric interpretations and typically attractive computational properties these methods capture many data features of interest such as covariance dynamical structure correlation between data sets input output relationships and margin between data classes methods have been developed with a variety of names and motivations in many fields and perhaps as a result the connections between all these methods have not been highlighted here we survey methods from this disparate literature as optimization programs over matrix manifolds we discuss principal component analysis factor analysis linear multidimensional scaling fisher s linear discriminant analysis canonical correlations analysis maximum autocorrelation factors slow feature analysis sufficient dimensionality reduction undercomplete independent component analysis linear regression distance metric learning and more this optimization framework gives insight to some rarely discussed shortcomings of well known methods such as the suboptimality of certain eigenvector solutions modern techniques for optimization over matrix manifolds enable a generic linear dimensionality reduction solver which accepts as input data and an objective to be optimized and returns as output an optimal low dimensional projection of the data this simple optimization framework further allows straightforward generalizations and novel variants of classical methods which we demonstrate here by creating an orthogonal projection canonical correlations analysis more broadly this survey and generic solver suggest that linear dimensionality reduction can move toward becoming a blackbox objective agnostic numerical technology

reduced rank stochastic regression with a sparse singular value decomposition summary for a reduced rank multivariate stochastic regression model of rank r the regression coefficient matrix can be expressed as a sum of r unit rank matrices each of which is proportional to the outer product of the left and right singular vectors for improving predictive accuracy and facilitating interpretation it is often desirable that these left and right singular vectors be sparse or enjoy some smoothness property we propose a regularized reduced rank regression approach for solving this problem computation algorithms and regularization parameter selection methods are developed and the properties of the new method are explored both theoretically and by simulation in particular the regularization method proposed is shown to be selection consistent and asymptotically normal and to enjoy the oracle property we apply the proposed model to perform biclustering analysis with microarray gene expression data

envelopes and reduced rank regression we incorporate the nascent idea of envelopes cook et al statist sinica 20 927 1010 into reduced rank regression by proposing a reduced rank envelope model which is a hybrid of reduced rank and envelope regressions the proposed model has total number of parameters no more than either of reduced rank regression or envelope regression the resulting estimator is at least as efficient as both existing estimators the methodology of this paper can be adapted to other envelope models such as partial envelopes su cook biometrika 98 133 46 and envelopes in predictor space cook et al j r statist soc b 75 851 77

on the asymptotic behavior of the energy for the wave equations with time depending coefficients we consider the asymptotic behavior of the total energy of solutions to the cauchy problem for wave equations with time dependent propagation speed the main purpose of this paper is that the asymptotic behavior of the total energy is dominated by the following properties of the coefficient order of the differentiability behavior of the derivatives as t and stabilization of the amplitude described by an integral moreover the optimality of these properties are ensured by actual examples

unique existence of evolution equations of hyperbolic type with countably many singular of degenerate points soit h un espace de hilbert et soit un operateur auto adjoint a spectre borne par dessous soit t 0 un nombre reel arbitraire avec 1 t 0 1 on etudie la resolubilite unique de l equation d evolution abstraite singuliere ou degeneree de type hyperbolique u t t u t 0 sur t 0 1 u t 0 u 0 u t 0 u 1 ou est une fonction non negative sur 1 1 avec des points singuliers et degeneres denombrables s accummulant en 0

symmetric hilbert spaces arising from species of structures symmetric hilbert spaces such as the bosonic and the fermionic fock spaces over some lsquo one particle space mathcal k are formed by certain symmetrization procedures performed on the full fock space we investigate alternative ways of symmetrization by building on joyal s notion of a combinatorial species any such species f gives rise to an endofunctor gamma f of the category of hilbert spaces with contractions mapping a hilbert space mathcal k to a symmetric hilbert space gamma f mathcal k with the same symmetry as the species f a general framework for annihilation and creation operators on these spaces is developed and compared to the generalised brownian motions of r speicher and m bozejko as a corollary we find that the commutation relation a ia j a j a i f n delta ij with na i a i n a i admits a realization on a symmetric hilbert space whenever f has a power series with infinite radius of convergence and positive coefficients

component analysis with different sets of constraints on different dimensions many of the classical multivariate data analysis and multidimensional scaling techniques call for approximations by lower dimensional configurations a model is proposed in which different sets of linear constraints are imposed on different dimensions in component analysis and classical multidimensional scaling frameworks a simple efficient and monotonically convergent algorithm is presented for fitting the model to the data by least squares the basic algorithm is extended to cover across dimension constraints imposed in addition to the dimensionwise constraints and to the case of a symmetric data matrix examples are given to demonstrate the use of the method

redundancy analysis for qualitative variables redundancy analysis also called principal components analysis of instrumental variables is a technique for two sets of variables one set being dependent of the other its aim is maximization of the explained variance of the dependent variables by a linear combination of the explanatory variables the technique is generalized to qualitative variables it then gives implicitly a simultaneous optimal scaling of the dependent qualitative variables examples are taken from the dutch life situation survey 1977 using satisfaction with life and happiness as dependent variables the analysis leads to one well being scale defined by the explanatory variables marital status schooling income and activity

reduced rank regression via adaptive nuclear norm penalization we propose an adaptive nuclear norm penalization approach for low rank matrix approximation and use it to develop a new reduced rank estimation method for high dimensional multivariate regression the adaptive nuclear norm is defined as the weighted sum of the singular values of the matrix and it is generally non convex under the natural restriction that the weight decreases with the singular value however we show that the proposed non convex penalized regression method has a global optimal solution obtained from an adaptively soft thresholded singular value decomposition the method is computationally efficient and the resulting solution path is continuous the rank consistency of and prediction estimation performance bounds for the estimator are established for a high dimensional asymptotic regime simulation studies and an application in genetics demonstrate its efficacy

tensor envelope partial least squares regression abstractpartial least squares pls is a prominent solution for dimension reduction and high dimensional regressions recent prevalence of multidimensional tensor data has led to several tensor versions of the pls algorithms however none offers a population model and interpretation and statistical properties of the associated parameters remain intractable in this article we first propose a new tensor partial least squares algorithm then establish the corresponding population interpretation this population investigation allows us to gain new insight on how the pls achieves effective dimension reduction to build connection with the notion of sufficient dimension reduction and to obtain the asymptotic consistency of the pls estimator we compare our method both analytically and numerically with some alternative solutions we also illustrate the efficacy of the new method on simulations and two neuroimaging data analyses supplementary materials for this article are available online

feature importance in nonlinear embeddings fine applications in digital pathology quantitative histomorphometry qh refers to the process of computationally modeling disease appearance on digital pathology images by extracting hundreds of image features and using them to predict disease presence or outcome since constructing a robust and interpretable classifier is challenging in a high dimensional feature space dimensionality reduction dr is often implemented prior to classifier construction however when dr is performed it can be challenging to quantify the contribution of each of the original features to the final classification result we have previously presented a method for scoring features based on their importance for classification on an embedding derived via principal components analysis pca however nonlinear dr involves the eigen decomposition of a kernel matrix rather than the data itself compounding the issue of classifier interpretability in this paper we present feature importance in nonlinear embeddings fine an extension of our pca based feature scoring method to kernel pca kpca as well as several nldr algorithms that can be cast as variants of kpca fine is applied to four digital pathology datasets to identify key qh features for predicting the risk of breast and prostate cancer recurrence measures of nuclear and glandular architecture and clusteredness were found to play an important role in predicting the likelihood of recurrence of both breast and prostate cancers compared to the t test fisher score and gini index fine was able to identify a stable set of features that provide good classification accuracy on four publicly available datasets from the nips 2003 feature selection challenge

active appearance models we describe a new method of matching statistical models of appearance to images a set of model parameters control modes of shape and gray level variation learned from a training set we construct an efficient iterative matching algorithm by learning the relationship between perturbations in the model parameters and the induced image errors

hierarchical aligned cluster analysis for temporal clustering of human motion temporal segmentation of human motion into plausible motion primitives is central to understanding and building computational models of human motion several issues contribute to the challenge of discovering motion primitives the exponential nature of all possible movement combinations the variability in the temporal scale of human actions and the complexity of representing articulated motion we pose the problem of learning motion primitives as one of temporal clustering and derive an unsupervised hierarchical bottom up framework called hierarchical aligned cluster analysis haca haca finds a partition of a given multidimensional time series into m disjoint segments such that each segment belongs to one of k clusters haca combines kernel k means with the generalized dynamic time alignment kernel to cluster time series data moreover it provides a natural framework to find a low dimensional embedding for time series haca is efficiently optimized with a coordinate descent strategy and dynamic programming experimental results on motion capture and video data demonstrate the effectiveness of haca for segmenting complex motions and as a visualization tool we also compare the performance of haca to state of the art algorithms for temporal clustering on data of a honey bee dance the haca code is available online

asymptotic distribution of the reduced rank regression estimator under general conditions in the regression model y bx z with z unobserved ez 0 and ezz zz the least squares estimator of b is b s yx s 1 xx if the rank of b is known to be k less than the dimensions of y and x the reduced rank regression estimator of b say b k depends on the first k canonical variates of y and x the asymptotic distribution of b k is obtained and compared with the asymptotic distribution of b the advantage of b k is characterized

multiclass linear dimension reduction by weighted pairwise fisher criteria we derive a class of computationally inexpensive linear dimension reduction criteria by introducing a weighted variant of the well known k class fisher criterion associated with linear discriminant analysis lda it can be seen that lda weights contributions of individual class pairs according to the euclidean distance of the respective class means we generalize upon lda by introducing a different weighting function

characterization of a family of algorithms for generalized discriminant analysis on undersampled problems a generalized discriminant analysis based on a new optimization criterion is presented the criterion extends the optimization criteria of the classical linear discriminant analysis lda when the scatter matrices are singular an efficient algorithm for the new optimization problem is presented the solutions to the proposed criterion form a family of algorithms for generalized lda which can be characterized in a closed form we study two specific algorithms namely uncorrelated lda ulda and orthogonal lda olda ulda was previously proposed for feature extraction and dimension reduction whereas olda is a novel algorithm proposed in this paper the features in the reduced space of ulda are uncorrelated while the discriminant vectors of olda are orthogonal to each other we have conducted a comparative study on a variety of real world data sets to evaluate ulda and olda in terms of classification accuracy

energy estimates for wave equations with time dependent propagation speeds in the gevrey class abstract the total energy of the wave equation is conserved with respect to time if the propagation speed is a constant but this is not true in general for time dependent propagation speeds indeed it is considered in hirosawa 2007 3 that the following properties of the propagation speed are crucial for the estimates of the total energy oscillating speed difference from the mean and the smoothness in c m category the main purpose of this paper is to derive a benefit of a further smoothness of the propagation speed in the gevrey class for the energy estimates

eigentracking robust matching and tracking of articulated objects using a view based representation this paper describes an approach for tracking rigid and articulated objects using a view based representation the approach builds on and extends work on eigenspace representations robust estimation techniques and parameterized optical flow estimation first we note that the least squares image reconstruction of standard eigenspace techniques has a number of problems and we reformulate the reconstruction problem as one of robust estimation second we define a subspace constancy assumption that allows us to exploit techniques for parameterized optical flow estimation to simultaneously solve for the view of an object and the affine transformation between the eigenspace and the image to account for large affine transformations between the eigenspace and the image we define a multi scale eigenspace representation and a coarse to fine matching strategy finally we use these techniques to track objects over long image sequences in which the objects simultaneously undergo both affine image motions and changes of view in particular we use this eigentracking technique to track and recognize the gestures of a moving hand

asymptotic theory of overparameterized structural models abstract a theory of overparameterized structural models is presented in such a model some redundant parameters are involved the parameter vector is not identified and the information matrix is not nonsingular the minimum discrepancy function mdf test statistic is shown to have an asymptotic chi squared distribution almost everywhere for a wide class of discrepancy functions asymptotic distribution properties of the mdf estimators are investigated the factor analysis model is discussed as an example

probabilistic principal component analysis principal component analysis pca is a ubiquitous technique for data analysis and processing but one which is not based upon a probability model in this paper we demonstrate how the principal axes of a set of observed data vectors may be determined through maximum likelihood estimation of parameters in a latent variable model closely related to factor analysis we consider the properties of the associated likelihood function giving an em algorithm for estimating the principal subspace iteratively and discuss the advantages conveyed by the definition of a probability density function for pca

canonical redundancy factoring analysis the interrelationships between two sets of measurements made on the same subjects can be studied by canonical correlation originally developed by hotelling 1936 the canonical correlation is the maximum correlation betweenlinear functions canonical factors of the two sets of variables an alternative statistic to investigate the interrelationships between two sets of variables is the redundancy measure developed by stewart and love 1968 van den wollenberg 1977 has developed a method of extracting factors which maximize redundancy as opposed to canonical correlation

generalised energy conservation law for wave equations with variable propagation speed abstract we investigate the long time behaviour of the l 2 energy of solutions to wave equations with variable speed of propagation the novelty of the approach is the combination of estimates for higher order derivatives of the coefficient with a stabilisation property

on the structure of non commutative white noises we consider the concepts of continuous bernoulli systems and non commutative white noises we address the question of isomorphism of continuous bernoulli systems and show that for large classes of quantum levy processes one can make quite precise statements about the time behaviour of their moments

probabilistic visual learning for object representation we present an unsupervised technique for visual learning which is based on density estimation in high dimensional spaces using an eigenspace decomposition two types of density estimates are derived for modeling the training data a multivariate gaussian for unimodal distributions and a mixture of gaussians model for multimodal distributions those probability densities are then used to formulate a maximum likelihood estimation framework for visual search and target detection for automatic object recognition and coding our learning technique is applied to the probabilistic visual modeling detection recognition and coding of human faces and nonrigid objects such as hands

envelopes and partial least squares regression summary we build connections between envelopes a recently proposed context for efficient estimation in multivariate statistics and multivariate partial least squares pls regression in particular we establish an envelope as the nucleus of both univariate and multivariate pls which opens the door to pursuing the same goals as pls but using different envelope estimators it is argued that a likelihood based envelope estimator is less sensitive to the number of pls components that are selected and that it outperforms pls in prediction and estimation

sparse and kernel opls feature extraction based on eigenvalue problem solving orthonormalized partial least squares opls is a popular multivariate analysis method to perform supervised feature extraction usually in machine learning papers opls projections are obtained by solving a generalized eigenvalue problem however in statistical papers the method is typically formulated in terms of a reduced rank regression problem leading to a formulation based on a standard eigenvalue decomposition a first contribution of this paper is to derive explicit expressions for matching the opls solutions derived under both approaches and discuss that the standard eigenvalue formulation is also normally more convenient for feature extraction in machine learning more importantly since optimization with respect to the projection vectors is carried out without constraints via a minimization problem inclusion of penalty terms that favor sparsity is straightforward in the paper we exploit this fact to propose modified versions of opls in particular relying on the 1 norm we propose a sparse version of linear opls as well as a non linear kernel opls with pattern selection we also incorporate a group lasso penalty to derive an opls method with true feature selection the discriminative power of the proposed methods is analyzed on a benchmark of classification problems furthermore we compare the degree of sparsity achieved by our methods and compare them with other state of the art methods for sparse feature extraction highlightswe establish the connection between opls and reduced rank regression problem evd formulation a novel sparse opls is proposed which enhances the solution obtained following the procrustres approach we propose a novel sparse kernel opls feature extractor for improved performance interpretability and efficiency

subspace procrustes analysis procrustes analysis pa has been a popular technique to align and build 2 d statistical models of shapes given a set of 2 d shapes pa is applied to remove rigid transformations later a non rigid 2 d model is computed by modeling the residual e g pca although pa has been widely used it has several limitations for modeling 2 d shapes occluded landmarks and missing data can result in local minima solutions and there is no guarantee that the 2 d shapes provide a uniform sampling of the 3 d space of rotations for the object to address previous issues this paper proposes subspace pa spa given several instances of a 3 d object spa computes the mean and a 2 d subspace that can model rigid and non rigid deformations of the 3 d object we propose a discrete dspa and continuous cspa formulation for spa assuming that 3 d samples of an object are provided dspa extends the traditional pa and produces unbiased 2 d models by uniformly sampling different views of the 3 d object cspa provides a continuous approach to uniformly sample the space of 3 d rotations being more efficient in space and time we illustrate the benefits of spa in two different applications first spa is used to learn 2 d face and body models from 3 d datasets experiments on the facewarehouse and cmu motion capture mocap datasets show the benefits of our 2 d models against the state of the art pa approaches and conventional 3 d models second spa learns an unbiased 2 d model from cmu mocap dataset and it is used to estimate the human pose on the leeds sports dataset

a unifying review of linear gaussian models factor analysis principal component analysis mixtures of gaussian clusters vector quantization kalman filter models and hidden markov models can all be unified as variations of unsupervised learning under a single basic generative model this is achieved by collecting together disparate observations and derivations made by many previous authors and introducing a new way of linking discrete and continuous state models using a simple nonlinearity through the use of other nonlinearities we show how independent component analysis is also a variation of the same basic generative model we show that factor analysis and mixtures of gaussians can be implemented in autoencoder neural networks and learned using squared error plus the same regularization term we introduce a new model for static data known as sensible principal component analysis as well as a novel concept of spatially adaptive observation noise we also review some of the literature involving global and local mixtures of the basic models and provide pseudocode for inference and learning for all the basic models

nonlinear component analysis as a kernel eigenvalue problem a new method for performing a nonlinear form of principal component analysis is proposed by the use of integral operator kernel functions one can efficiently compute principal components in high dimensional feature spaces related to input space by some nonlinear map for instance the space of all possible five pixel products in 16 16 images we give the derivation of the method and present experimental results on polynomial feature extraction for pattern recognition

decorrelation of neutral vector variables theory and applications in this paper we propose novel strategies for neutral vector variable decorrelation two fundamental invertible transformations namely serial nonlinear transformation and parallel nonlinear transformation are proposed to carry out the decorrelation for a neutral vector variable which is not multivariate gaussian distributed the conventional principal component analysis cannot yield mutually independent scalar variables with the two proposed transformations a highly negatively correlated neutral vector can be transformed to a set of mutually independent scalar variables with the same degrees of freedom we also evaluate the decorrelation performances for the vectors generated from a single dirichlet distribution and a mixture of dirichlet distributions the mutual independence is verified with the distance correlation measurement the advantages of the proposed decorrelation strategies are intensively studied and demonstrated with synthesized data and practical application evaluations

tensor decompositions and applications this survey provides an overview of higher order tensor decompositions their applications and available software a tensor is a multidimensional or n way array decompositions of higher order tensors i e n way arrays with n geq 3 have applications in psycho metrics chemometrics signal processing numerical linear algebra computer vision numerical analysis data mining neuroscience graph analysis and elsewhere two particular tensor decompositions can be considered to be higher order extensions of the matrix singular value decomposition candecomp parafac cp decomposes a tensor as a sum of rank one tensors and the tucker decomposition is a higher order form of principal component analysis there are many other tensor decompositions including indscal parafac2 candelinc dedicom and paratuck2 as well as nonnegative variants of all of the above the n way toolbox tensor toolbox and multilinear engine are examples of software packages for working with tensors

generalised brownian motion and second quantisation a new approach to the generalised brownian motion introduced by m bouzejko and r speicher is described based on symmetry rather than deformation the symmetrisation principle is provided by joyal s notions of tensorial and combinatorial species any such species v gives rise to an endofunctor fv of the category of hilbert spaces with contractions a generalised brownian motion is an algebra of creation and annihilation operators acting on fv h for arbitrary hilbert spaces h and having a prescription for the calculation of vacuum expectations in terms of a function t on pair partitions the positivity is encoded by a semigroup of broken pair partitions whose representation space with respect to t is v the existence of the second quantisation as functor t from hilbert spaces to noncommutative probability spaces is proved to be equivalent to the multiplicative property of the function t for a certain one parameter interpolation between the fermionic and the free brownian motion it is shown that the field algebras k are type ii1 factors when k is infinite dimensional

on the energy estimates of the wave equation with time dependent propagation speed asymptotically monotone functions abstract we consider the energy estimates for the wave equation with time dependent propagation speed it is known that the asymptotic behavior of the energy is determined by the interactions of the properties of the propagation speed smoothness oscillation and the difference from an auxiliary function the main purpose of the article is to show that if the propagation speed behaves asymptotically as a monotone decreasing function then we can extend the preceding results to allow faster oscillating coefficients moreover we prove that the regularity of the initial data in the gevrey class can essentially contribute for the energy estimate

a least squares framework for component analysis over the last century component analysis ca methods such as principal component analysis pca linear discriminant analysis lda canonical correlation analysis cca locality preserving projections lpp and spectral clustering sc have been extensively used as a feature extraction step for modeling classification visualization and clustering ca techniques are appealing because many can be formulated as eigen problems offering great potential for learning linear and nonlinear representations of data in closed form however the eigen formulation often conceals important analytic and computational drawbacks of ca techniques such as solving generalized eigen problems with rank deficient matrices e g small sample size problem lacking intuitive interpretation of normalization factors and understanding commonalities and differences between ca methods this paper proposes a unified least squares framework to formulate many ca methods we show how pca lda cca lpp sc and its kernel and regularized extensions correspond to a particular instance of least squares weighted kernel reduced rank regression ls wkrrr the ls wkrrr formulation of ca methods has several benefits 1 provides a clean connection between many ca techniques and an intuitive framework to understand normalization factors 2 yields efficient numerical schemes to solve ca techniques 3 overcomes the small sample size problem 4 provides a framework to easily extend ca methods we derive weighted generalizations of pca lda sc and cca and several new ca techniques

testing the rank of a matrix with applications to the analysis of interaction in anova abstract this article develops some general theory for testing the rank of a matrix applications include tests of interaction in two factor experiments that are more powerful than the usual f test for certain reasonable alternatives the particular tests studied are a the likelihood ratio lr test of rank m 0 versus rank m r where m is a matrix of expectations and b the union intersection ui test of rank m 0 versus rank m r in the two factor application m is the a x b matrix of interaction parameters the ui test that the interaction has rank 0 versus rank 1 is a simultaneous test that all product interaction contrasts are zero it is shown that the asymptotic distributions of the ui and lr test statistics are identical a distinct advantage of the ui test is that the small sample null distribution of the test statistic is known and can be computed tables of exact percentiles of the ui test statistic for testing rank m 0 against rank m 1 are given the ui test is illustr

the q t gaussian process the q t fock space fq t h introduced in this paper is a deformation of the q fock space of bozejko and speicher the corresponding creation and annihilation operators now satisfy the commutation relation aq t f aq t g qaq t g aq t f f g htn turn mathjax off a defining relation of the chakrabarti jagannathan deformed quantum oscillator algebra the moments of the deformed gaussian element sq t h aq t h aq t h are encoded by the joint statistics of crossings and nestings in pair partitions the q 0 t specialization yields a natural single parameter deformation of the full boltzmann fock space of free probability with the corresponding semicircular measure variously encoded via the rogers ramanujan continued fraction the t airy function the t catalan numbers of carlitz riordan and the first order statistics of the reduced wigner process

transformation invariant clustering using the em algorithm clustering is a simple effective way to derive useful representations of data such as images and videos clustering explains the input as one of several prototypes plus noise in situations where each input has been randomly transformed e g by translation rotation and shearing in images and videos clustering techniques tend to extract cluster centers that account for variations in the input due to transformations instead of more interesting and potentially useful structure for example if images from a video sequence of a person walking across a cluttered background are clustered it would be more useful for the different clusters to represent different poses and expressions instead of different positions of the person and different configurations of the background clutter we describe a way to add transformation invariance to mixture models by approximating the nonlinear transformation manifold by a discrete set of points we show how the expectation maximization algorithm can be used to jointly learn clusters while at the same time inferring the transformation associated with each input we compare this technique with other methods for filtering noisy images obtained from a scanning electron microscope clustering images from videos of faces into different categories of identification and pose and removing foreground obstructions from video we also demonstrate that the new technique is quite insensitive to initial conditions and works better than standard techniques even when the standard techniques are provided with extra data

maximum likelihood parameter and rank estimation in reduced rank multivariate linear regressions this paper considers the problem of maximum likelihood ml estimation for reduced rank linear regression equations with noise of arbitrary covariance the rank reduced matrix of regression coefficients is parameterized as the product of two full rank factor matrices this parameterization is essentially constraint free but it is not unique which renders the associated ml estimation problem rather nonstandard nevertheless the problem turns out to be tractable and the following results are obtained an explicit expression is derived for the ml estimate of the regression matrix in terms of the data covariances and their eigenelements furthermore a detailed analysis of the statistical properties of the ml parameter estimate is performed additionally a generalized likelihood ratio test glrt is proposed for estimating the rank of the regression matrix the paper also presents the results of some simulation exercises which lend empirical support to the theoretical findings

the importance of common cyclical features in var analysis a monte carlo study abstract despite the commonly held belief that aggregate data display short run comovement there has been little discussion about the econometric consequences of this feature of the data we use exhaustive monte carlo simulations to investigate the importance of restrictions implied by common cyclical features for estimates and forecasts based on vector autoregressive models first we show that the best empirical model developed without common cycle restrictions need not nest the best model developed with those restrictions this is due to possible differences in the lag lengths chosen by model selection criteria for the two alternative models second we show that the costs of ignoring common cyclical features in vector autoregressive modelling can be high both in terms of forecast accuracy and efficient estimation of variance decomposition coefficients third we find that the hannan quinn criterion performs best among model selection criteria in simultaneously selecting the lag length and rank of vector autoregressions