statistical consistency of kernel canonical correlation analysis while kernel canonical correlation analysis cca has been applied in many contexts the convergence of finite sample estimates of the associated functions to their population counterparts has not yet been established this paper gives a mathematical proof of the statistical convergence of kernel cca providing a theoretical justification for the method the proof uses covariance operators defined on reproducing kernel hilbert spaces and analyzes the convergence of their empirical estimates of finite rank to their population counterparts which can have infinite rank the result also gives a sufficient condition for convergence on the regularization coefficient involved in kernel cca this should decrease as n 1 3 where n is the number of data

prediction and classification in nonlinear data analysis something old something new something borrowed something blue prediction and classification are two very active areas in modern data analysis in this paper prediction with nonlinear optimal scaling transformations of the variables is reviewed and extended to the use of multiple additive components much in the spirit of statistical learning techniques that are currently popular among other areas in data mining also a classification clustering method is described that is particularly suitable for analyzing attribute value data from systems biology genomics proteomics and metabolomics and which is able to detect groups of objects that have similar values on small subsets of the attributes

a pseudo arclength continuation method for nonlinear eigenvalue problems a variant of the classical pseudo arclength continuation method is proposed basically the method can be viewed as pseudo arclength continuation in r lambda space where r is a functional of the solution another difference is a three parameter predictor instead of the standard euler step this predictor as well as the newton corrector iteration are justified and some numerical results for reaction diffusion equations are presented the method provides a simple algebraic check for symmetry breaking bifurcation the most common type of secondary bifurcation in physical examples

principal inertia components and applications we explore properties and applications of the principal inertia components pics between two discrete random variables x and y the pics lie in the intersection of information and estimation theory and provide a fine grained decomposition of the dependence between x and y moreover the pics describe which functions of x can or cannot be reliably inferred in terms of mmse given an observation of y we demonstrate that the pics play an important role in information theory and they can be used to characterize information theoretic limits of certain estimation problems in privacy settings we prove that the pics are related to the fundamental limits of perfect privacy

analysis of additive dependencies and concurvities using smallest additive principal components additive principal components are a nonlinear generalization of linear principal components their distinguishing feature is that linear forms i a i x i are replaced with additive functions i f i x i a considerable amount of flexibility for fitting data is gained when linear methods are replaced with additive ones our interest is in the smallest principal components which is somewhat uncommon smallest additive principal components amount to data descriptions in terms of approximate implicit equations i f i x i 0 estimation of such equations is a data analytic method in its own right competing in some cases with the more customary regression approaches it is also a diagnostic tool in additive regression for detection of so called concurvity this term describes degeneracies among predictor variables in additive regression models similar to collinearity in linear regression models concurvity may lead to statistically unstable contributions of variables to additive models as an example we show in a reanalysis of the ozone data from breiman and friedman that concurvity does indeed exist in this particular data a fact which may impact the interpretation of the additive fits in the second half of this paper we give some second order theory including the description of null situations and eigenexpansions derived from associated eigenproblems we show how ace and additive principal components are related and we outline some analytical methods for theoretical calculations of additive principal components lastly we consider methods of estimation and computation additive principal components have had a long tradition in psychometric research and correspondence analysis they have started receiving attention by statisticians only in recent years

rates of convergence for minimum contrast estimators we shall present here a general study of minimum contrast estimators in a nonparametric setting although our results are also valid in the classical parametric case for independent observations these estimators include many of the most popular estimators in various situations such as maximum likelihood estimators least squares and other estimators of the regression function estimators for mixture models or deconvolution the main theorem relates the rate of convergence of those estimators to the entropy structure of the space of parameters optimal rates depending on entropy conditions are already known at least for some of the models involved and they agree with what we get for minimum contrast estimators as long as the entropy counts are not too large but under some circumstances large entropies or changes in the entropy structure due to local perturbations the resulting the rates are only suboptimal counterexamples are constructed which show that the phenomenon is real for non parametric maximum likelihood or regression this proves that under purely metric assumptions our theorem is optimal and that minimum contrast estimators happen to be suboptimal

mean absolute deviations of sample means and minimally concentrated binomials this is a contribution to the theory of sums of independent random variables at the level of optimal explicit inequalities we compute the optimal constants in hornich s lower bounds for the mean absolute deviations of sample means this is done by reducing the original problem to the elementary one of determining the minimally concentrated binomial distributions bn p with fixed sample size parameter n

subadditivity generalized products of random matrices and operations research an elementary theorem on subadditive sequences provides the key to a far reaching theory of subadditive processes one important instance of this theory is the limit theory for products of stationary random matrices this paper shows that the subadditive inequality that governs the log of the norm of ordinary matrix products also governs other functions of several generalized matrix products these generalized matrix products are used to calculate minimal cost transportation routes schedules in manufacturing and minimal and maximal probabilities of multistage processes the application of subadditive ergodic theory to generalized products of stationary random matrices yields new information about the limiting behavior of generalized products exact calculations of the asymptotic behavior are possible in some examples

general notions of statistical depth function statistical depth functions are being formulated ad hoc with increasing popularity in nonparametric inference for multivariate data here we introduce several general structures for depth functions classify many existing examples as special cases and establish results on the possession or lack thereof of four key properties desirable for depth functions in general roughly speaking these properties may be described as affine invariance maximality at center monotonicity relative to deepest point and vanishing at infinity this provides a more systematic basis for selection of a depth function in particular from these and other considerations it is found that the halfspace depth behaves very well overall in comparison with various competitors

canonical analysis of two euclidean subspaces and its applications abstract we develop a canonical analysis both without constraints and under constraints for subspaces of euclidean space we also look into a canonical analysis of operators we then apply the definitions and some of the results to the field of probability and statistics

remarks on functional canonical variates alternating least squares methods and ace we discuss properties of some data analytic methods which are intimately related to each other alternating least squares als correspondence analysis and more recently breiman and friedman s ace algorithm the application of these methods to regression produces nonparametric estimators of nonlinear transformations both of the response and the predictors we point out some anomalies as well as some curiosities in the mathematics of these methods and we relate them to some areas in computer aided tomography projection pursuit regression and nonlinear devices in the theory of noise

generalized measures of correlation for asymmetry nonlinearity and beyond some antecedents on causality this note comments on the generalized measure of correlation gmc that was suggested by zheng shi and zhang the gmc concept was partly anticipated in some publications over 100 years earlier by

a general method for comparing probability assessors a probability assessor or forecaster is a person who assigns subjective probabilities to events which will eventually occur or not occur in this paper a general method for comparing forecasters after a finite number of trials is introduced the general method is proven to include calculating all proper scoring rules as special cases it also includes comparison of forecasters in all simple two decision problems as special cases the relationship between the general method and calibration is also explored the general method is also translated into a method for deciding who will give better forecasts in the future

stepsize selection in continuation procedures and damped newton s method in this work we explore the relation of the problem of satisfying a sufficient decrease criterion in a damped newton s method to the problem of stepsize selection for continuation methods we show that there is a strong connection between the two and that standard line search techniques used for computing damping parameters have direct application to the stepsize selection problem the performance of the resulting continuation technique is demonstrated for several standard example problems

construction of solution curves for large two dimensional problems of steady state flows of incompressible fluids this work represents a step toward advancing classical methods of bifurcation analysis in conjunction with very large scale scientific computing needed to model realistically processes which involve laminar and steady flows of incompressible fluids a robust method of analysis based on pseudoarclength continuation newton s method and direct solution of linear systems is proposed and applied to the analysis of a representative system of fluid mechanics namely the tilted lid driven cavity accurate solution curves possessing simple singular points were computed for reynolds numbers varying from 0 to 10 000 and for different angles of tilt the results demonstrate that two dimensional models with up to 1 000 000 algebraic equations can be studied feasibly using the methods described here with state of the art vector supercomputers

linear smoothers and additive models we study linear smoothers and their use in building nonparametric regression models in the first part of this paper we examine certain aspects of linear smoothers for scatterplots examples of these are the running mean and running line kernel and cubic spline smoothers the eigenvalue and singular value decompositions of the corresponding smoother matrix are used to describe qualitatively a smoother and several other topics such as the number of degrees of freedom of a smoother are discussed in the second part of the paper we describe how linear smoothers can be used to estimate the additive model a powerful nonparametric regression model using the backfitting algorithm we show that backfitting is the gauss seidel iterative method for solving a set of normal equations associated with the additive model we provide conditions for consistency and nondegeneracy and prove convergence for the backfitting and related algorithms for a class of smoothers that includes cubic spline smoothers

orthogonal grid generation for two dimensional ducts abstract we present an orthogonal trajectory method for grid generation for two dimensional ducts the method can be seen as a generalization of the polar grid for a wedge or a circular annulus transversal grid lines are formed by arcs of circles and orthogonal trajectories are found by solving initial value problems for ordinary differential equations in order to have the method working a condition involving the local width of the duct and the curvatures of the boundary curves must be satisfied this condition is easily computable and it should be checked in advance for illustration several computational examples are included

three dimensional face recognition an expression invariant 3d face recognition approach is presented our basic assumption is that facial expressions can be modelled as isometries of the facial surface this allows to construct expression invariant representations of faces using the bending invariant canonical forms approach the result is an efficient and accurate face recognition algorithm robust to facial expressions that can distinguish between identical twins the first two authors we demonstrate a prototype system based on the proposed algorithm and compare its performance to classical face recognition methods the numerical methods employed by our approach do not require the facial surface explicitly the surface gradients field or the surface metric are sufficient for constructing the expression invariant representation of any given face it allows us to perform the 3d face recognition task while avoiding the surface reconstruction stage

the efficiency of investment information we investigate how the description of a correlated information v improves the investment in the stock market x the objective is to maximize the growth rate of wealth in repeated investments we find a single letter characterization of the incremental growth rate spl delta r the maximum increase in growth rate when v is described to the investor at rate r the incremental growth rate specialized to the horse race market is related to source coding with side information of wyner and ahlswede kormer we provide two horse race examples jointly binary and jointly gaussian the initial efficiency spl delta 0 is the maximum possible increase in the growth rate per bit of description we show that the initial efficiency is related to the dependency between v and the market in particular for the horse race market the initial efficiency is the square of the hirschfeld gebelein renyi maximal correlation between v and x this provides a connection with the hypercontraction of the markov operator of ahlswede and gacs for the general market the initial efficiency is 1 when the side information v is equal to the stock market outcome x

multiset canonical correlations analysis and multispectral truly multitemporal remote sensing data this paper describes two and multiset canonical correlations analysis cca for data fusion multisource multiset or multitemporal exploratory data analysis these techniques transform multivariate multiset data into new orthogonal variables called canonical variates cvs which when applied in remote sensing exhibit ever decreasing similarity as expressed by correlation measures over sets consisting of 1 spectral variables at fixed points in time r mode analysis or 2 temporal variables with fixed wavelengths t mode analysis the cvs are invariant to linear and affine transformations of the original variables within sets which means for example that the r mode cvs are insensitive to changes over time in offset and gain in a measuring device in a case study cvs are calculated from landsat thematic mapper tm data with six spectral bands over six consecutive years both rand t mode cvs clearly exhibit the desired characteristic they show maximum similarity for the low order canonical variates and minimum similarity for the high order canonical variates these characteristics are seen both visually and in objective measures the results from the multiset cca r and t mode analyses are very different this difference is ascribed to the noise structure in the data the cca methods are related to partial least squares pls methods this paper very briefly describes multiset cca based multiset pls also the cca methods can be applied as multivariate extensions to empirical orthogonal functions eof techniques multiset cca is well suited for inclusion in geographical information systems gis

a monte carlo investigation of conjoint analysis index of fit goodness of fit significance and power researchers in the field of conjoint analysis know the index of fit values worsen as the judgmental error of evaluation increases this simulation study provides guidelines on the goodness of fit based on distribution of index of fit for different conjoint analysis designs the study design included the following factors number of profiles number of attributes algorithm used and judgmental model used critical values are provided for deciding the statistical significance of conjoint analysis results using these cumulative distributions the power of the test used to reject the null hypothesis of random ranking is calculated the test is found to be quite powerful except for the case of very small residual degrees of freedom

on a notion of data depth based on random simplices for a distribution f on mathbb r p and a point x in mathbb r p the simplical depth d x is introduced which is the probability that the point x is contained inside a random simplex whose vertices are p 1 independent observations from f mathematically and heuristically it is argued that d x indeed can be viewed as a measure of depth of the point x with respect to f an empirical version of d cdot gives rise to a natural ordering of the data points from the center outward the ordering thus obtained leads to the introduction of multivariate generalizations of the univariate sample median and l statistics this generalized sample median and l statistics are affine equivariant

kernel independent component analysis we present a class of algorithms for independent component analysis ica which use contrast functions based on canonical correlations in a reproducing kernel hilbert space on the one hand we show that our contrast functions are related to mutual information and have desirable mathematical properties as measures of statistical dependence on the other hand building on recent developments in kernel methods we show that these criteria and their derivatives can be computed efficiently minimizing these criteria leads to flexible and robust algorithms for ica we illustrate with simulations involving a wide variety of source distributions showing that our algorithms outperform many of the presently known algorithms

upper bounds for kruskal s stress in this paper the relationships between the two formulas for stress proposed by kruskal in 1964 are studied it is shown that stress formula one has a system of nontrivial upper bounds it seems likely that minimization of this loss function will be liable to produce solutions for which this upper bound is small these are regularly shaped configurations even though stress formula two yields less equivocal results it seems to be expected that minimization of this loss function will tend to produce configurations in which the points are clumped these results give no clue as to which of the two loss functions is to be preferred

strict definiteness of integrals via complete monotonicity of derivatives let k be a nonnegative integer and let 0 r be a c function with k k completely monotone and not constant if 6 0 is a signed measure on any euclidean space rd with vanishing moments up to order k 1 then the integral rd rd x y 2 d x d y is strictly positive whenever it exists for general d no larger class of continuous functions seems to admit the same conclusion examples and applications are indicated a section on bilinear integrability might be of independent interest

completeness of location families translated moments and uniqueness of charges a sufficient condition for statistical completeness of location families generated by a probability density in euclidean space is given as an application completeness of families generated by a symmetric stable law is proved our criterion complementing a classical result of wiener and recent work of isenbeck and ruschendorf is in terms of regularity of the generating density and zerofreeness of its characteristic function its proof rests on a local version of the convolution theorem for fourier transforms of tempered distributions a more general version of the criterion is applicable to apparently different problems as is illustrated by giving a simultaneous proof of a theorem on translated moments by p hall and a uniqueness result of m riesz in potential theory

game theory maximum entropy minimum discrepancy and robust bayesian decision theory we describe and develop a close relationship between two problems that have customarily been regarded as distinct that of maximizing entropy and that of minimizing worst case expected loss using a formulation grounded in the equilibrium theory of zero sum games between decision maker and nature these two problems are shown to be dual to each other the solution to each providing that to the other although topsoe described this connection for the shannon entropy over 20 years ago it does not appear to be widely known even in that important special case we here generalize this theory to apply to arbitrary decision problems and loss functions we indicate how an appropriate generalized definition of entropy can be associated with such a problem and we show that subject to certain regularity conditions the above mentioned duality continues to apply in this extended context this simultaneously provides a possible rationale for maximizing entropy and a tool for finding robust bayes acts we also describe the essential identity between the problem of maximizing entropy and that of minimizing a related discrepancy or divergence between distributions this leads to an extension to arbitrary discrepancies of a well known minimax theorem for the case of kullback leibler divergence the redundancy capacity theorem of information theory for the important case of families of distributions having certain mean values specified we develop simple sufficient conditions and methods for identifying the desired solutions we use this theory to introduce a new concept of generalized exponential family linked to the specific decision problem under consideration and we demonstrate that this shares many of the properties of standard exponential families finally we show that the existence of an equilibrium in our game can be rephrased in terms of a pythagorean property of the related divergence thus generalizing previously announced results for kullback leibler and bregman divergences

canonical correlation analysis when the data are curves summary it is not immediately straightforward to extend canonical correlation analysis to the context of functional data analysis where the data are themselves curves or functions the obvious approach breaks down and it is necessary to use a method involving smoothing in some way such a method is introduced and discussed with reference to a data set on human gait the breakdown of the unsmoothed method is illustrated in a practical context and is demonstrated theoretically a consistency theorem for the smoothed method is proved in an increasing number of problems in a wide range of fields the data observed are not the univariate or multivariate observations of classical statistics but are func tions observed continuously ramsay and dalzell 1991 give the name functional data analysis to the analysis of data of this kind in most cases the observations will be functions of time or a closely related variable but there are clearly applications where the functions are surfaces observed over two or three dimensional space the motivating example for the present paper is a set of data on children s gait collected by the motion analysis laboratory at children s hospital san diego california see olshen et al 1989 for full details for each of a number of children several angles made by the child s joints knee hip etc are observed during the child s gait cycle one aim of the study is to gain understanding of the gait cycle of a normal child to make comparisons with children suffering from walking diffi culties these data motivated rice and silverman 1991 to discuss the extension of principal component analysis to the functional setting and to explain how smoothing can be incorporated into the analysis in a natural way in a substantial complemen tary paper ramsay and dalzell 1991 discuss different approaches to regression and principal component analysis in functional data analysis illustrating their work by a meteorological example the insights of functional data analysis may be helpful in chemometrics for instance a standard chemometric problem is the analysis of spectra observed in chromatography and these are to all intents and purposes functional observations another obvious area of potential relevance is the analysis of growth curve data the focus in the present paper will be on canonical correlation analysis cca in the gait example one might ask how variability in the knee angle cycle is related to

ein lokal uberlinear konvergentes verfahren zur bestimmung von ruckkehrpunkten implizit definierter raumkurven a numerical technique is described for computing turning points of a space curvel implicitly defined by a nonlinear system ofn equations inn 1 variables the basic idea is a local parametrization ofl where the parameter that gives the next iterate is determined by applying one step of the well known method for minimizing a real function using cubic hermite interpolation with two nodes the method is shown to convergeq super linearly and withr order of at least two a numerical example concerning the analysis of nonlinear resistive circuits shows the algorithm to work effectively on real life problems

a family of inequalities originating from coding of messages this paper presents 96 new inequalities with common structure all elementary to state but many not elementary to prove for example if n is a positive integer and a l a1 an and b b1 bn are arbitrary vectors in r n 0 n and mi j is the spectral radius of an n n matrix with elements mi j then i jmin aiaj bibj i jmin aibj biaj 2mm i jmax ai aj bi bj i jmax ai bj bi aj min aiaj bibj min aibj biaj 4mm i jmin aiaj bibj xixj i jmin aibj biaj xixj forallrealxi i 1 n log f x f y g x g y d x d y log f x g y g x f y d x d y the second inequality is obtained from the first inequality which is due to g zb ganu a new inequality with applications in measure and information theories in proceedings of the romanian academy series a1 1 2000 pp 15 19 by replacing min with max and with and by reversing the direction of the inequality the third inequality is obtained from the first by replacing the summation by the spectral radius the fourth inequality is obtained from the first by taking each summand as a coefficient in a quadratic form the fifth inequality is obtained from the first by replacing both outer summations by products min by by and the non negative vectors a b by non negative measurable functions f g the proofs of these inequalities are mysteriously diverse a nice generalization of the first inequality is proved let be one of the four operations min and max on an appropriate interval j of r let a b jn denote by a a the n n matrix ai j ai aj then the matrix a a is more different from b b than a b is from b a precisely if a 1 i j n ai j then a a b b a b b a

another look at some new cauchy schwarz type inner product inequalities in this paper we take a refreshing look at a cauchy schwarz type inner product inequalities we also provide a cauchy schwarz based proof and a refinement of buzano s inequality in inner product spaces some elementary applications such as trace inequalities for unitary matrices and discrete or integral inequalities are given

elementary inequalities that involve two nonnegative vectors or functions abstract we report 96 inequalities with common structure all elementary to state but many not elementary to prove if n is a positive integer a a 1 an and b b 1 bn are arbitrary vectors in and mij is the spectral radius of an n n matrix with elements mij then for example the second inequality is obtained from the first inequality by replacing min with max and with and by reversing the direction of the inequality the third inequality is obtained from the first by replacing the summation by the spectral radius the fourth inequality is obtained from the first by taking each summand as a coefficient in a quadratic form the fifth inequality is obtained from the first by replacing both outer summations by products min by by and the nonnegative vectors a and b by nonnegative measurable functions f and g the proofs of these inequalities are mysteriously diverse

on a periodic schrodinger equation with nonlocal superlinear part we consider the choquard pekar equation open image in new window and focus on the case of periodic potential v for a large class of even functions w we show existence and multiplicity of solutions essentially the conditions are that 0 is not in the spectrum of the linear part v and that w does not change sign our results carry over to more general nonlinear terms in arbitrary space dimension n 2

strictly proper scoring rules prediction and estimation scoring rules assess the quality of probabilistic forecasts by assigning a numerical score based on the predictive distribution and on the event or value that materializes a scoring rule is proper if the forecaster maximizes the expected score for an observation drawn from the distributionf if he or she issues the probabilistic forecast f rather than g f it is strictly proper if the maximum is unique in prediction problems proper scoring rules encourage the forecaster to make careful assessments and to be honest in estimation problems strictly proper scoring rules provide attractive loss and utility functions that can be tailored to the problem at hand this article reviews and develops the theory of proper scoring rules on general probability spaces and proposes and discusses examples thereof proper scoring rules derive from convex functions and relate to information measures entropy functions and bregman divergences in the case of categorical variables we prove a rigorous version of the

detecting generalized synchronization between chaotic signals a kernel based approach a unified framework for analysing generalized synchronization in coupled chaotic systems from data is proposed the key of the proposed approach is the use of the kernel methods recently developed in the field of machine learning several successful applications are presented which show the capability of the kernel based approach for detecting generalized synchronization and dynamical change of the coupling strength between two chaotic systems can be captured by the proposed approach it is also discussed how the kernel parameter is suitably chosen from data

coherent dispersion criteria for optimal experimental design we characterize those coherent design criteria which depend only on the dispersion matrix assumed proper and nonsingular of the state of nature which may be a parameter vector or a set of future observables and describe the associated decision problems connections are established with the classical approach to optimal design theory for the normal linear model based on concave functions of the information matrix implications of the theory for more general models are also considered

nonlinear canonical analysis and independence tests measures of association between two random variables r v that are symmetric nondecreasing functions of the canonical coefficients provided by the nonlinear canonical analysis of the r v s are studied these measures can be used to characterize independence their estimators are obtained by estimating a suitable approximation to nonlinear canonical analysis when some conditions ae satisfied asymptotic distributions of the estimators both under the independence hypothesis and under dependence are given a class of tests of independence with asymptotic level of significance can be investigated 1 introduction several measures of association between two random variables r v based on the canonical analysis of these r v s have been introduced in the literature and a review of such measures can be found in cramer and nicewander 1979 and lazraq and cleroux 1988 a general enough study of these measures is proposed in lin 1987 and more recently in dauxois and nkiet 1997 where a global study of the induced tests is developed in these works the measures considered are constructed using linear canonical analysis lca of the r v s and thus characterize only lack of a linear relationship it can be interesting to look for another class of measures of association which characterize independence the properties of nonlinear canonical analysis nlca suggest that such a class can be derived from this analysis but there is little work in this direction the canonical coefficients from nlca have already been used for testing independence but uniquely when the related random variables are categorical see for instance tsai and sen 1990 that is restrictive because one knows that in this particular case nlca is a lca of suitable random vectors and thus it suffices to use classical measures of association consequently it appears that the use of nlca is more interesting when the related random variables are not categorical in this paper we propose a class of measures of association between random variables with values in any measurable spaces constructed using symmetric nondecreasing functions and the canonical coefficients derived from nlca the properties of these measures show that they are appropriate to characterize independence without any assumption on the distribution of

level crossing probabilities i one dimensional random walks and symmetrization we prove for an arbitrary random walk in r1 with independent increments that the probability of crossing a level at a given time n is o n 1 2 moment or symmetry assumptions are not necessary in removing symmetry the sharp inequality p x y 1 2p x y 1 for independent identically distributed x y is used in part ii we shall discuss the connection of this result to polygonal recurrence of higher dimensional walks and some conjectures on directionally reinforced random walks in the sense of mauldin monticino and von weizsacker r d mauldin m monticino h von weizsacker directionally reinforced random walks adv math 117 1996 239 252 5

a parallel and adaptive continuation method for semilinear bifurcation problems in this paper we propose an efficient continuation method for the following of solution branches of parametrized semilinear elliptic problems in order to limit the time required for the computation of a whole branch of accurate solutions we combine a mesh adaptation strategy with parallel programming on a computer network the first computer is devoted to the continuation on a coarse mesh the others share out the correction of every roughly generated solution on a sequence of a posteriori refined meshes a numerical example illustrating the performance of the method is given