one bit matching theorem for ica convex concave programming on polyhedral set and distribution approximation for combinatorics according to the proof by liu chiu and xu 2004 on the so called one bit matching conjecture xu cheung and amari 1998a all the sources can be separated as long as there is an one to one same sign correspondence between the kurtosis signs of all source probability density functions pdf s and the kurtosis signs of all model pdf s which is widely believed and implicitly supported by many empirical studies however this proof is made only in a weak sense that the conjecture is true when the global optimal solution of an independent component analysis criterion is reached thus it cannot support the successes of many existing iterative algorithms that usually converge at one of the local optimal solutions this article presents a new mathematical proof that is obtained in a strong sense that the conjecture is also true when any one of local optimal solutions is reached in helping to investigating convex concave programming on a polyhedral set theorems are also provided not only on partial separation of sources when there is a partial matching between the kurtosis signs but also on an interesting duality of maximization and minimization on source separation moreover corollaries are obtained on an interesting duality with supergaussian sources separated by maximization and subgaussian sources separated by minimization also a corollary is obtained to confirm the symmetric orthogonalization implementation of the kurtosis extreme approach for separating multiple sources in parallel which works empirically but lacks mathematical proof furthermore a linkage has been set up to combinatorial optimization from a distribution approximation perspective and a stiefel manifold perspective with algorithms that guarantee convergence as well as satisfaction of constraints

consistent distribution free k sample and independence tests for univariate random variables a popular approach for testing if two univariate random variables are statistically independent consists of partitioning the sample space into bins and evaluating a test statistic on the binned data the partition size matters and the optimal partition size is data dependent while for detecting simple relationships coarse partitions may be best for detecting complex relationships a great gain in power can be achieved by considering finer partitions we suggest novel consistent distribution free tests that are based on summation or maximization aggregation of scores over all partitions of a fixed size we show that our test statistics based on summation can serve as good estimators of the mutual information moreover we suggest regularized tests that aggregate over all partition sizes and prove those are consistent too we provide polynomial time algorithms which are critical for computing the suggested test statistics efficiently we show that the power of the regularized tests is excellent compared to existing tests and almost as powerful as the tests based on the optimal yet unknown in practice partition size in simulations as well as on a real data example

measuring and testing dependence by correlation of distances distance correlation is a new measure of dependence between random vectors distance covariance and distance correlation are analogous to product moment covariance and correlation but unlike the classical definition of correlation distance correlation is zero only if the random vectors are independent the empirical distance dependence measures are based on certain euclidean distances between sample elements rather than sample moments yet have a compact representation analogous to the classical covariance and correlation asymptotic properties and applications in testing independence are discussed implementation of the test and monte carlo results are also presented

multicategory support vector machines two category support vector machines svm have been very popular in the machine learning community for classification problems solving multicategory problems by a series of binary classifiers is quite common in the svm paradigm however this approach may fail under various circumstances we propose the multicategory support vector machine msvm which extends the binary svm to the multicategory case and has good theoretical properties the proposed method provides a unifying framework when there are either equal or unequal misclassification costs as a tuning criterion for the msvm an approximate leave one out cross validation function called generalized approximate cross validation is derived analogous to the binary case the effectiveness of the msvm is demonstrated through the applications to cancer classification using microarray data and cloud classification with satellite radiance profiles

an alternative perspective on adaptive independent component analysis algorithms this article develops an extended independent component analysis algorithm for mixtures of arbitrary subgaussian and supergaussian sources the gaussian mixture model of pearson is employed in deriving a closedform generic score function for strictly subgaussian sources this is combined with the score function for a unimodal supergaussian density to provide a computationally simple yet powerful algorithm for performing independent component analysis on arbitrary mixtures of nongaussian sources

independent component analysis via distance covariance abstractthis article introduces a novel statistical framework for independent component analysis ica of multivariate data we propose methodology for estimating mutually independent components and a versatile resampling based procedure for inference including misspecification testing independent components are estimated by combining a nonparametric probability integral transformation with a generalized nonparametric whitening method based on distance covariance that simultaneously minimizes all forms of dependence among the components we prove the consistency of our estimator under minimal regularity conditions and detail conditions for consistency under model misspecification all while placing assumptions on the observations directly not on the latent components u statistics of certain euclidean distances between sample elements are combined to construct a test statistic for mutually independent components the proposed measures and tests are based on both necessary and sufficient conditions for

one bit matching conjecture for independent component analysis the one bit matching conjecture for independent component analysis ica could be understood from different perspectives but is basically stated as all the sources can be separated as long as there is a one to one same sign correspondence between the kurtosis signs of all source probability density functions pdf s and the kurtosis signs of all model pdf s xu cheung amari 1998a this conjecture has been widely believed in the ica community and implicitly supported by many ica studies such as the extended infomax lee girolami sejnowski 1999 and the soft switching algorithm welling weber 2001 however there is no mathematical proof to confirm the conjecture theoretically in this article only skewness and kurtosis are considered and such a mathematical proof is given under the assumption that the skewness of the model densities vanishes moreover empirical experiments are demonstrated on the robustness of the conjecture as the vanishing skewness assumption breaks as a by product we also show that the kurtosis maximization criterion moreau macchi 1996 is actually a special case of the minimum mutual information criterion for ica

the conditional permutation test for independence while controlling for confounders we propose a general new method the conditional permutation test for testing the conditional independence of variables x and y given a potentially high dimensional random vector z that may contain confounding factors the test permutes entries of x non uniformly to respect the existing dependence between x and z and thus to account for the presence of these confounders like the conditional randomization test of candes and co workers in 2018 our test relies on the availability of an approximation to the distribution of x z whereas their test uses this estimate to draw new x values for our test we use this approximation to design an appropriate non uniform distribution on permutations of the x values already seen in the true data we provide an efficient markov chain monte carlo sampler for the implementation of our method and establish bounds on the type i error in terms of the error in the approximation of the conditional distribution of x z finding that for the worst case test statistic the inflation in type i error of the conditional permutation test is no larger than that of the conditional randomization test we validate these theoretical results with experiments on simulated data and on the capital bikeshare data set

blind source separation semiparametric statistical approach the semiparametric statistical model is used to formulate the problem of blind source separation the method of estimating functions is applied to this problem it is shown that an estimator of the mixing matrix or its learning version can be described in terms of an estimating function the statistical efficiencies of these algorithms are studied the main results are as follows 1 the space consisting of all the estimating functions is derived 2 the space is decomposed into the orthogonal sum of the admissible part and a redundant ancillary part for any estimating function one can find a better or equally good estimator in the admissible part 3 the fisher efficient that is asymptotically best estimating functions are derived 4 the stability of learning algorithms is studied

tracking for fully actuated mechanical systems a geometric framework we present a general framework for the control of lagrangian systems with as many inputs as degrees of freedom relying on the geometry of mechanical systems on manifolds we propose a design algorithm for the tracking problem the notions of error function and transport map lead to a proper definition of configuration and velocity error these are the crucial ingredients in designing a proportional derivative feedback and feedforward controller the proposed approach includes as special cases a variety of results on control of manipulators pointing devices and autonomous vehicles our design provides particular insight into both aerospace and underwater applications where the configuration manifold is a lie group

stabilization of relative equilibria for underactuated systems on riemannian manifolds steady motions of mechanical systems with symmetry are stabilized via a potential shaping and damping control a general methodology based on riemannian geometry tools provides exponential stabilization over the full phase space

ica mixture models for unsupervised classification of non gaussian classes and automatic context switching in blind signal separation an unsupervised classification algorithm is derived by modeling observed data as a mixture of several mutually exclusive classes that are each described by linear combinations of independent non gaussian densities the algorithm estimates the density of each class and is able to model class distributions with non gaussian structure the new algorithm can improve classification accuracy compared with standard gaussian mixture models when applied to blind source separation in nonstationary environments the method can switch automatically between classes which correspond to contexts with different mixing properties the algorithm can learn efficient codes for images containing both natural scenes and text this method shows promise for modeling non gaussian structure in high dimensional data and has many potential applications

unsupervised image classification segmentation and enhancement using ica mixture models an unsupervised classification algorithm is derived by modeling observed data as a mixture of several mutually exclusive classes that are each described by linear combinations of independent non gaussian densities the algorithm estimates the data density in each class by using parametric nonlinear functions that fit to the non gaussian structure of the data this improves classification accuracy compared with standard gaussian mixture models when applied to images the algorithm can learn efficient codes basis functions for images that capture the statistically significant structure intrinsic in the images we apply this technique to the problem of unsupervised classification segmentation and denoising of images we demonstrate that this method was effective in classifying complex image textures such as natural scenes and text it was also useful for denoising and filling in missing pixels in images with complex structures the advantage of this model is that image codes can be learned with increasing numbers of classes thus providing greater flexibility in modeling structure and in finding more image features than in either gaussian mixture models or standard independent component analysis ica algorithms

partial distance correlation with methods for dissimilarities distance covariance and distance correlation are scalar coefficients that characterize independence of random vectors in arbitrary dimension properties extensions and applications of distance correlation have been discussed in the recent literature but the problem of defining the partial distance correlation has remained an open question of considerable interest the problem of partial distance correlation is more complex than partial correlation partly because the squared distance covariance is not an inner product in the usual linear space for the definition of partial distance correlation we introduce a new hilbert space where the squared distance covariance is the inner product we define the partial distance correlation statistics with the help of this hilbert space and develop and implement a test for zero partial distance correlation our intermediate results provide an unbiased estimator of squared distance covariance and a neat solution to the problem of distance correlation for dissimilarities rather than distances

capturing correlation changes by applying kernel change point detection on the running correlations abstract change point detection methods signal the occurrence of abrupt changes in a time series non parametric approaches such as the gaussian kernel based change point kcp detection arlot et al 2012 are especially attractive because they impose less assumptions on the data yet a drawback of these methods is that most of them are sensitive to changes in the mean the variance etc making them less sensitive to specific kinds of changes we show that kcp can be adapted to detect a particular type of change only we focus here on correlation change which has been put forward by different theories but proved hard to trace in multivariate time series we propose kcp corr which boils down to applying kcp on the running correlations to confirm that kcp corr is more sensitive than merely applying kcp on the raw data kcp raw a simulation study was conducted in which the number of noise variables and the size of the correlation change were varied kcp corr emerged as the better method especially in the more difficult but realistic settings where the correlation change is minimal and or noise variables are present kcp corr also outperforms cusum a non parametric method that specifically targets the detection of correlation changes

a unifying information theoretic framework for independent component analysis abstract we show that different theories recently proposed for independent component analysis ica lead to the same iterative learning algorithm for blind separation of mixed independent sources we review those theories and suggest that information theory can be used to unify several lines of research pearlmutter and parra 1 and cardoso 2 showed that the infomax approach of bell and sejnowski 3 and the maximum likelihood estimation approach are equivalent we show that negentropy maximization also has equivalent properties and therefore all three approaches yield the same learning rule for a fixed nonlinearity girolami and fyfe 4 have shown that the nonlinear principal component analysis pca algorithm of karhunen and joutsensalo 5 and oja 6 can also be viewed from information theoretic principles since it minimizes the sum of squares of the fourth order marginal cumulants and therefore approximately minimizes the mutual information 7 lambert 8 has proposed different bussgang cost functions for multichannel blind deconvolution we show how the bussgang property relates to the infomax principle finally we discuss convergence and stability as well as future research issues in blind source separation

martingale difference divergence matrix and its application to dimension reduction for stationary multivariate time series in this article we introduce a new methodology to perform dimension reduction for a stationary multivariate time series our method is motivated by the consideration of optimal prediction and focu

decorrelation of neutral vector variables theory and applications in this paper we propose novel strategies for neutral vector variable decorrelation two fundamental invertible transformations namely serial nonlinear transformation and parallel nonlinear transformation are proposed to carry out the decorrelation for a neutral vector variable which is not multivariate gaussian distributed the conventional principal component analysis cannot yield mutually independent scalar variables with the two proposed transformations a highly negatively correlated neutral vector can be transformed to a set of mutually independent scalar variables with the same degrees of freedom we also evaluate the decorrelation performances for the vectors generated from a single dirichlet distribution and a mixture of dirichlet distributions the mutual independence is verified with the distance correlation measurement the advantages of the proposed decorrelation strategies are intensively studied and demonstrated with synthesized data and practical application evaluations

rate of chaotic mixing and boundary behavior we discuss rigorous results on the rate of mixing for an idealized model of a class of fluid mixing device these show that the decay of correlations of a scalar field is governed by the presence of boundaries in the domain and in particular by the behavior of the modeled fluid at such boundaries

three methods for estimating a range of vehicular interactions abstract we present three different approaches how to estimate the number of preceding cars influencing a decision making procedure of a given driver moving in saturated traffic flows the first method is based on correlation analysis the second one evaluates quantitatively deviations from the main assumption in the convolution theorem for probability and the third one operates with advanced instruments of the theory of counting processes statistical rigidity we demonstrate that universally accepted premise on short ranged traffic interactions may not be correct all methods introduced have revealed that minimum number of actively followed vehicles is two it supports an actual idea that vehicular interactions are in fact middle ranged furthermore consistency between the estimations used is surprisingly credible in all cases we have found that the interaction range the number of actively followed vehicles drops with traffic density whereas drivers moving in congested regimes with lower density around 30 vehicles per kilometer react on four or five neighbors drivers moving in high density flows respond to two predecessors only

testing for independence in arbitrary distributions summary statistics are proposed for testing the hypothesis that arbitrary random variables are mutually independent the tests are consistent and well behaved for any marginal distributions they can be used for example for contingency tables which are sparse or whose dimension depends on the sample size as well as for mixed data no regularity conditions data jittering or binning mechanisms are required the statistics are rank based functionals of cramer von mises type whose asymptotic behaviour derives from the empirical multilinear copula process approximate p values are computed using a wild bootstrap the procedures are simple to implement and computationally efficient and maintain their level well in moderate to large samples simulations suggest that the tests are robust with respect to the number of ties in the data can easily detect a broad range of alternatives and outperform existing procedures in many settings additional insight into their performance is provided through asymptotic local power calculations under contiguous alternatives the procedures are illustrated on traumatic brain injury data

copula index for detecting dependence and monotonicity between stochastic signals abstract this paper introduces a nonparametric copula based index for detecting the strength and monotonicity structure of linear and nonlinear statistical dependence between pairs of random variables or stochastic signals our index termed copula index for detecting dependence and monotonicity cim satisfies several desirable properties of measures of association including most of renyi s properties the data processing inequality dpi and consequently self equitability synthetic data simulations reveal that the statistical power of cim compares favorably to other state of the art measures of association that are proven to satisfy the dpi simulation results with real world data reveal cim s unique ability to detect the monotonicity structure among stochastic signals to find interesting dependencies in large datasets additionally simulations show that cim compares favorably to estimators of mutual information when discovering markov network structure

measuring dependence powerfully and equitably given a high dimensional data set we often wish to find the strongest relationships within it a common strategy is to evaluate a measure of dependence on every variable pair and retain the highest scoring pairs for follow up this strategy works well if the statistic used a has good power to detect non trivial relationships and b is equitable meaning that for some measure of noise it assigns similar scores to equally noisy relationships regardless of relationship type e g linear exponential periodic in this paper we define and theoretically characterize two new statistics that together yield an efficient approach for obtaining both power and equitability to do this we first introduce a new population measure of dependence and show three equivalent ways that it can be viewed including as a canonical smoothing of mutual information we then introduce an efficiently computable consistent estimator of our population measure of dependence and we empirically establish its equitability on a large class of noisy functional relationships this new statistic has better bias variance properties and better runtime complexity than a previous heuristic approach next we derive a second related statistic whose computation is a trivial side product of our algorithm and whose goal is powerful independence testing rather than equitability we prove that this statistic yields a consistent independence test and show in simulations that the test has good power against independence taken together our results suggest that these two statistics are a valuable pair of tools for exploratory data analysis

functional models for time varying random objects functional data analysis provides a popular toolbox of functional models for the analysis of samples of random functions that are real valued in recent years samples of time varying object data such as time varying networks that are not in a vector space have been increasingly collected these data can be viewed as elements of a general metric space that lacks local or global linear structure and therefore common approaches that have been used with great success for the analysis of functional data such as functional principal component analysis cannot be applied we propose metric covariance a novel association measure for paired object data lying in a metric space d that we use to define a metric autocovariance function for a sample of random valued curves where generally will not have a vector space or manifold structure the proposed metric autocovariance function is non negative definite when the squared semimetric d2 is of negative type then the eigenfunctions of the linear operator with the autocovariance function as kernel can be used as building blocks for an object functional principal component analysis for valued functional data including time varying probability distributions covariance matrices and time dynamic networks analogues of functional principal components for time varying objects are obtained by applying frechet means and projections of distance functions of the random object trajectories in the directions of the eigenfunctions leading to real valued frechet scores using the notion of generalized frechet integrals we construct object functional principal components that lie in the metric space we establish asymptotic consistency of the sample based estimators for the corresponding population targets under mild metric entropy conditions on and continuity of the valued random curves these concepts are illustrated with samples of time varying probability distributions for human mortality time varying covariance matrices derived from trading patterns and time varying networks that arise from new york taxi trips

time varying periodicity in intraday volatility abstractwe develop a nonparametric test for whether return volatility exhibits time varying intraday periodicity using a long time series of high frequency data our null hypothesis commonly adopt

approximating the null distribution of a class of statistics for testing independence abstract a class of tests for testing independence whose test statistic is an l 2 norm of the difference between the joint empirical characteristic function and the product of the marginal empirical characteristic functions associated with a sample is considered since the null distribution of these test statistics is unknown some approximations are investigated specifically the permutation bootstrap and weighted bootstrap estimators are examined all of them provide consistent estimators a simulation study analyzes the performance of these approximations for small and moderate sample sizes an application to a real data set for testing independence between positional errors in spatial data is included

rotational motion control of a spacecraft this note describes a systematic procedure for the control synthesis of a rigid spacecraft using the energy shaping method the geometric concept of a mechanical system in a coordinate independent form is used to derive a control algorithm for the euler poincare equations the main result of this note is a specialization of the method on the unit quaternion group this note is concluded with the examples of the potential functions and implementation for the three axis attitude control problem

a new coefficient of correlation abstract is it possible to define a coefficient of correlation which is a as simple as the classical coefficients like pearson s correlation or spearman s correlation and yet b consistently es

natural gradient works efficiently in learning when a parameter space has a certain underlying structure the ordinary gradient of a function does not represent its steepest direction but the natural gradient does information geometry is used for calculating the natural gradients in the parameter space of perceptrons the space of matrices for blind source separation and the space of linear dynamical systems for blind source deconvolution the dynamical behavior of natural gradient online learning is analyzed and is proved to be fisher efficient implying that it has asymptotically the same performance as the optimal batch estimation of parameters this suggests that the plateau phenomenon which appears in the backpropagation learning algorithm of multilayer perceptrons might disappear or might not be so serious when the natural gradient is used an adaptive method of updating the learning rate is proposed and analyzed

mean field approaches to independent component analysis we develop mean field approaches for probabilistic independent component analysis ica the sources are estimated from the mean of their posterior distribution and the mixing matrix and noise level is estimated by maximum a posteriori map the latter requires the computation of a good approximation to the correlations between sources for this purpose we investigate three increasingly advanced mean field methods the variational also known as naive mean field approach linear response corrections and an adaptive version of the thouless anderson and palmer 1977 tap mean field approach which is due to opper and winther 2001 the resulting algorithms are tested on a number of problems on synthetic data the advanced mean field approaches are able to recover the correct mixing matrix in cases where the variational mean field theory fails for handwritten digits sparse encoding is achieved using nonnegative source and mixing priors for speech the mean field method is able to separate in the underdetermined overcomplete case of two sensors and three sources one major advantage of the proposed method is its generality and algorithmic simplicity finally we point out several possible extensions of the approaches developed here

classification with a reject option using a hinge loss we consider the problem of binary classification where the classifier can for a particular cost choose not to classify an observation just as in the conventional classification problem minimization of the sample average of the cost is a difficult optimization problem as an alternative we propose the optimization of a certain convex loss function analogous to the hinge loss used in support vector machines svms its convexity ensures that the sample average of this surrogate loss can be efficiently minimized we study its statistical properties we show that minimizing the expected surrogate loss the risk also minimizes the risk we also study the rate at which the risk approaches its minimum value we show that fast rates are possible when the conditional probability p y 1 x is unlikely to be close to certain critical values

testing mutual independence in high dimension via distance covariance summary we introduce an l2 type test for testing mutual independence and banded dependence structure for high dimensional data the test is constructed on the basis of the pairwise distance covariance and it accounts for the non linear and non monotone dependences among the data which cannot be fully captured by the existing tests based on either pearson correlation or rank correlation our test can be conveniently implemented in practice as the limiting null distribution of the test statistic is shown to be standard normal it exhibits excellent finite sample performance in our simulation studies even when the sample size is small albeit the dimension is high and is shown to identify non linear dependence in empirical data analysis successfully on the theory side asymptotic normality of our test statistic is shown under quite mild moment assumptions and with little restriction on the growth rate of the dimension as a function of sample size as a demonstration of good power properties for our distance covariance based test we further show that an infeasible version of our test statistic has the rate optimality in the class of gaussian distributions with equal correlation

extremal probabilities for gaussian quadratic forms denote by q an arbitrary positive semidefinite quadratic form in centered gaussian random variables such that e q 1 we prove that for an arbitrary x 0 inf q p q x p 2 n n x where n 2 is a chi square distributed rv with n n x degrees of freedom n x is a non increasing function of x n 1 iff x x 1 1 5364 n 2 iff x x 2 x 1 where x 2 1 2989 etc n x rank q a similar statement is not true for the supremum if 1 x 2 and z 1 z 2 are independent standard gaussian rv s then sup0 1 2p z 1 2 1 z 2 2 x is taken not at 0 or at 1 2 but at 0 x 1 2 where x is a continuous increasing function from 1 0 to 2 1 2 e g 1 5 15 applications of our theorems include asymptotic quantiles of u and v statistics signal detection and stochastic orderings of integrals of squared gaussian processes

causal inference on discrete data via estimating distance correlations in this article we deal with the problem of inferring causal directions when the data are on discrete domain by considering the distribution of the cause and the conditional distribution mapping cause to effect as independent random variables we propose to infer the causal direction by comparing the distance correlation between and with the distance correlation between and we infer that x causes y if the dependence coefficient between and is smaller experiments are performed to show the performance of the proposed method

complexity pursuit separating interesting components from time series a generalization of projection pursuit for time series that is signals with time structure is introduced the goal is to find projections of time series that have interesting structure defined using criteria related to kolmogoroff complexity or coding length interesting signals are those that can be coded with a short code length we derive a simple approximation of coding length that takes into account both the nongaussianity and the autocorrelations of the time series also we derive a simple algorithm for its approximative optimization the resulting method is closely related to blind separation of nongaussian time dependent source signals

goodness of fit testing for time series models via distance covariance abstract in many statistical modeling frameworks goodness of fit tests are typically administered to the estimated residuals in the time series setting whiteness of the residuals is assessed using the sample autocorrelation function for many time series models especially those used for financial time series the key assumption on the residuals is that they are in fact independent and not just uncorrelated in this paper we apply the auto distance covariance function adcv to evaluate the serial dependence of the estimated residuals distance covariance can discriminate between dependence and independence of two random vectors the limit behavior of the test statistic based on the adcv is derived for a general class of time series models one of the key aspects in this theory is adjusting for the dependence that arises due to parameter estimation this adjustment has essentially the same form regardless of the model specification we illustrate the results in simulated examples

classification of gene expression data the manifold based metric learning way classification of microarray gene expression data can potentially help medical diagnosis and becomes an important topic in bioinformatics however microarray data sets are usually of small sample size relative to an overwhelming number of genes this makes the classification problem fairly challenging instance based learning ibl algorithms such as nearest neighbor k nn are usually the baseline algorithm due to their simplicity however practices show that k nn performs not very well in this field this paper introduces manifold based metric learning to improve the performance of ibl methods a novel metric learning algorithm is proposed by utilizing both local manifold structural information and local discriminant information in addition a random subspace extension is also presented we apply the proposed algorithm to the gene classification problem in three ways one in the original feature space another in the reduced feature space and the third via the random subspace extension statistical evaluation shows that the proposed algorithm can achieve promising results and gain significant performance improvement over traditional ibl algorithms

consistent testing for pairwise dependence in time series abstractwe consider the problem of testing pairwise dependence for stationary time series for this we suggest the use of a box ljung type test statistic that is formed after calculating the distance covariance function among pairs of observations the distance covariance function is a suitable measure for detecting dependencies between observations as it is based on the distance between the characteristic function of the joint distribution of the random variables and the product of the marginals we show that under the null hypothesis of independence and under mild regularity conditions the test statistic converges to a normal random variable the results are complemented by several examples this article has supplementary material online

using distance covariance for improved variable selection with application to learning genetic risk models variable selection is of increasing importance to address the difficulties of high dimensionality in many scientific areas in this paper we demonstrate a property for distance covariance which is incorporated in a novel feature screening procedure together with the use of distance correlation the approach makes no distributional assumptions for the variables and does not require the specification of a regression model and hence is especially attractive in variable selection given an enormous number of candidate attributes without much information about the true model with the response the method is applied to two genetic risk problems where issues including uncertainty of variable selection via cross validation subgroup of hard to classify cases and the application of a reject option are discussed

projection correlation between two random vectors we propose the use of projection correlation to characterize dependence between two random vectors projection correlation has several appealing properties it equals zero if and only if the two random vectors are independent it is not sensitive to the dimensions of the two random vectors it is invariant with respect to the group of orthogonal transformations and its estimation is free of tuning parameters and does not require moment conditions on the random vectors we show that the sample estimate of the projection correction is formula see text consistent if the two random vectors are independent and root formula see text consistent otherwise monte carlo simulation studies indicate that the projection correlation has higher power than the distance correlation and the ranks of distances in tests of independence especially when the dimensions are relatively large or the moment conditions required by the distance correlation are violated

distance multivariance new dependence measures for random vectors we introduce two new measures for the dependence of n ge2 random variables distance multivariance and total distance multivariance both measures are based on the weighted l 2 distance of quantities related to the characteristic functions of the underlying random variables these extend distance covariance introduced by szekely rizzo and bakirov from pairs of random variables to n tuplets of random variables we show that total distance multivariance can be used to detect the independence of n random variables and has a simple finite sample representation in terms of distance matrices of the sample points where distance is measured by a continuous negative definite function under some mild moment conditions this leads to a test for independence of multiple random vectors which is consistent against all alternatives

feature screening via distance correlation learning this article is concerned with screening features in ultrahigh dimensional data analysis which has become increasingly important in diverse scientific fields we develop a sure independence screening procedure based on the distance correlation dc sis the dc sis can be implemented as easily as the sure independence screening sis procedure based on the pearson correlation proposed by fan and lv however the dc sis can significantly improve the sis fan and lv established the sure screening property for the sis based on linear models but the sure screening property is valid for the dc sis under more general settings including linear models furthermore the implementation of the dc sis does not require model specification e g linear model or generalized linear model for responses or predictors this is a very appealing property in ultrahigh dimensional data analysis moreover the dc sis can be used directly to screen grouped predictor variables and multivariate response variables we establish

functional connectivity and structural covariance between regions of interest can be measured more accurately using multivariate distance correlation studies of brain wide functional connectivity or structural covariance typically use measures like the pearson correlation coefficient applied to data that have been averaged across voxels within regions of interest rois however averaging across voxels may result in biased connectivity estimates when there is inhomogeneity within those rois e g sub regions that exhibit different patterns of functional connectivity or structural covariance here we propose a new measure based on distance correlation a test of multivariate dependence of high dimensional vectors which allows for both linear and non linear dependencies we used simulations to show how distance correlation out performs pearson correlation in the face of inhomogeneous rois to evaluate this new measure on real data we use resting state fmri scans and t1 structural scans from 2 sessions on each of 214 participants from the cambridge centre for ageing neuroscience cam can project pearson correlation and distance correlation showed similar average connectivity patterns for both functional connectivity and structural covariance nevertheless distance correlation was shown to be 1 more reliable across sessions 2 more similar across participants and 3 more robust to different sets of rois moreover we found that the similarity between functional connectivity and structural covariance estimates was higher for distance correlation compared to pearson correlation we also explored the relative effects of different preprocessing options and motion artefacts on functional connectivity because distance correlation is easy to implement and fast to compute it is a promising alternative to pearson correlations for investigating roi based brain wide connectivity patterns for functional as well as structural data

equivariant adaptive source separation source separation consists of recovering a set of independent signals when only mixtures with unknown coefficients are observed this paper introduces a class of adaptive algorithms for source separation that implements an adaptive version of equivariant estimation and is henceforth called equivariant adaptive separation via independence easi the easi algorithms are based on the idea of serial updating this specific form of matrix updates systematically yields algorithms with a simple structure for both real and complex mixtures most importantly the performance of an easi algorithm does not depend on the mixing matrix in particular convergence rates stability conditions and interference rejection levels depend only on the normalized distributions of the source signals closed form expressions of these quantities are given via an asymptotic performance analysis the theme of equivariance is stressed throughout the paper the source separation problem has an underlying multiplicative structure the parameter space forms a matrix multiplicative group we explore the favorable consequences of this fact on implementation performance and optimization of easi algorithms

fundamentals of statistical signal processing estimation theory minimum variance unbiased estimation cramer rao lower bound linear models general minimum variance unbiased estimation best linear unbiased estimators maximum likelihood estimation least squares method of moments the bayesian philosophy general bayesian estimators linear bayesian estimators kalman filters summary of estimators extension for complex data and parameters

diagnostic studies in sufficient dimension reduction sufficient dimension reduction in regression aims to reduce the predictor dimension by replacing the original predictors with some set of linear combinations of them without loss of information numerous dimension reduction methods have been developed based on this paradigm however little effort has been devoted to diagnostic studies within the context of dimension reduction in this paper we introduce methods to check goodness of fit for a given dimension reduction subspace the key idea is to extend the so called distance correlation to measure the conditional dependence relationship between the covariates and the response given a reduction subspace our methods require minimal assumptions which are usually much less restrictive than the conditions needed to justify the original methods asymptotic properties of the test statistic are studied numerical examples demonstrate the effectiveness of the proposed approach

variational mixture of bayesian independent component analyzers there has been growing interest in subspace data modeling over the past few years methods such as principal component analysis factor analysis and independent component analysis have gained in popularity and have found many applications in image modeling signal processing and data compression to name just a few as applications and computing power grow more and more sophisticated analyses and meaningful representations are sought mixture modeling methods have been proposed for principal and factor analyzers that exploit local gaussian features in the subspace manifolds meaningful representations may be lost however if these local features are nongaussian or discontinuous in this article we propose extending the gaussian analyzers mixture model to an independent component analyzers mixture model we employ recent developments in variational bayesian inference and structure determination to construct a novel approach for modeling nongaussian discontinuous manifolds we automatically determine the local dimensionality of each manifold and use variational inference to calculate the optimum number of ica components needed in our mixture model we demonstrate our framework on complex synthetic data and illustrate its application to real data by decomposing functional magnetic resonance images into meaningful and medically useful features

distance covariance in metric spaces we extend the theory of distance brownian covariance from euclidean spaces where it was introduced by szekely rizzo and bakirov to general metric spaces we show that for testing independence it is necessary and sufficient that the metric space be of strong negative type in particular we show that this holds for separable hilbert spaces which answers a question of kosorok instead of the manipulations of fourier transforms used in the original work we use elementary inequalities for metric spaces and embeddings in hilbert spaces

distance correlation detecting lyapunov instabilities noise induced escape times and mixing abstract the properties of the statistical method of distance correlation between multivariate data are analysed in the context of nonlinear dynamical systems the distance correlation between the noisy and the noiseless quadratic maps are studied in periodic and chaotic regimes results are compared to the classical method of pearson s correlation while distance and pearson s correlations are affected by the lyapunov exponent from the noiseless orbits only the distance correlation is able to recognize the correct qualitative behaviour of noise induced escape times decays and the mixing of chaotic trajectories in addition the distance correlation is capable of detecting distinct size of attractors the main goal of this work is to establish the validity of the distance correlation as an method of correlation between multivariate data in dynamical systems

blind source separation by sparse decomposition in a signal dictionary the blind source separation problem is to extract the underlying source signals from a set of linear mixtures where the mixing matrix is unknown this situation is common in acoustics radio medical signal and image processing hyperspectral imaging and other areas we suggest a two stage separation process a priori selection of a possibly overcomplete signal dictionary for instance a wavelet frame or a learned dictionary in which the sources are assumed to be sparsely representable followed by unmixing the sources by exploiting the their sparse representability we consider the general case of more sources than mixtures but also derive a more efficient algorithm in the case of a nonovercomplete dictionary and an equal numbers of sources and mixtures experiments with artificial signals and musical sounds demonstrate significantly better separation than other known techniques

stabilization of hamiltonian systems 1 hamiltonian systems in this paper we will be concerned with the stabilization by feedback of hamiltonian systems in order to facilitate our discussions especially when applying lyapunov s second method we will restrict ourselves to a particular although natural subclass of hamiltonian systems given in the following way l let q be an n dimensional smooth manifold denoting the configuration space and let t q be the cotangent bundle denoting the phase space or srufe space furthermore there is a smooth m dimensional output manifold y m n and a smooth output map c q y smooth will mean c or ck with k sufficiently big although we shall restrict ourselves in the second part of section 2 to analytic data for simplicity we take c to be submersive so rank dc q m we assume that the system on t q has an internal energy which is the sum of a kinetic energy k and a porentiul energy v this means that there exists a riemannian metric on q in local coordinates q q j for q given by

state feedback spl hscr sub spl infin suboptimal control of a rigid spacecraft a state feedback spl hscr sub spl infin suboptimal control problem on tso 3 for a rigid spacecraft with three control torques and disturbances is addressed the hamilton jacobi inequality associated with a corresponding state feedback spl hscr sub spl infin suboptimal control problem on ts sup 3 is solved globally the spl hscr sub spl infin suboptimal state feedback on ts sup 3 involves the euler parameters and solves the original problem on tso 3 when the initial values of the euler parameters are appropriately chosen

the attitude control problem a general framework for the analysis of the attitude tracking control problem for a rigid body is presented a large family of globally stable control laws is obtained by using the globally nonsingular unit quaternion representation in a lyapunov function candidate whose form is motivated by the consideration of the total energy of the rigid body the controllers share the common structure of a proportional derivative feedback plus some feedforward which can be zero the model independent case the coriolis torque compensation or an adaptive compensation these controller structures are compared in terms of the requirement on the a priori model information guaranteed transient performance and robustness the global stability of the luh walker paul robot end effector controller is also analyzed in this framework

interaction pursuit in high dimensional multi response regression via distance correlation feature interactions can contribute to a large proportion of variation in many prediction models in the era of big data the coexistence of high dimensionality in both responses and covariates poses unprecedented challenges in identifying important interactions in this paper we suggest a two stage interaction identification method called the interaction pursuit via distance correlation ipdc in the setting of high dimensional multi response interaction models that exploits feature screening applied to transformed variables with distance correlation followed by feature selection such a procedure is computationally efficient generally applicable beyond the heredity assumption and effective even when the number of responses diverges with the sample size under mild regularity conditions we show that this method enjoys nice theoretical properties including the sure screening property support union recovery and oracle inequalities in prediction and estimation for both interactions and main effects the advantages of our method are supported by several simulation studies and real data analysis

composite coefficient of determination and its application in ultrahigh dimensional variable screening in this article we propose to measure the dependence between two random variables through a composite coefficient of determination ccd of a set of nonparametric regressions these regressions ta

an information maximization approach to blind separation and blind deconvolution we derive a new self organizing learning algorithm that maximizes the information transferred in a network of nonlinear units the algorithm does not assume any knowledge of the input distributions and is defined here for the zero noise limit under these conditions information maximization has extra properties not found in the linear case linsker 1989 the nonlinearities in the transfer function are able to pick up higher order moments of the input distributions and perform something akin to true redundancy reduction between units in the output representation this enables the network to separate statistically independent components in the inputs a higher order generalization of principal components analysis we apply the network to the source separation or cocktail party problem successfully separating unknown mixtures of up to 10 speakers we also show that a variant on the network architecture is able to perform blind deconvolution cancellation of unknown echoes and reverberation in a speech signal finally we derive dependencies of information transfer on time delays we suggest that information maximization provides a unifying framework for problems in blind signal processing