brief paper least squares based iterative identification for a class of multirate systems this paper studies modeling and identification problems for multi input multirate systems with colored noises the state space models are derived for the systems with different input updating periods and furthermore the corresponding transfer functions are obtained to solve the difficulty of identification models with unmeasurable noises terms the least squares based iterative algorithm is presented by replacing the unmeasurable variables with their iterative estimates finally the simulation results indicate that the proposed iterative algorithm has advantages over the recursive algorithms

ideal spatial adaptation by wavelet shrinkage summary with ideal spatial adaptation an oracle furnishes information about how best to adapt a spatially variable estimator whether piecewise constant piecewise polynomial variable knot spline or variable bandwidth kernel to the unknown function estimation with the aid of an oracle offers dramatic advantages over traditional linear estimation by nonadaptive kernels however it is a priori unclear whether such performance can be obtained by a procedure relying on the data alone we describe a new principle for spatially adaptive estimation selective wavelet reconstruction we show that variable knot spline fits and piecewise polynomial fits when equipped with an oracle to select the knots are not dramatically more powerful than selective wavelet reconstruction with an oracle we develop a practical spatially adaptive method riskshrink which works by shrinkage of empirical wavelet coefficients riskshrink mimics the performance of an oracle for selective wavelet reconstruction as well as it is possible to do so a new inequality in multivariate normal decision theory which we call the oracle inequality shows that attained performance differs from ideal performance by at most a factor of approximately 2 log n where n is the sample size moreover no estimator can give a better guarantee than this within the class of spatially adaptive procedures riskshrink is essentially optimal relying only on the data it comes within a factor log 2 n of the performance of piecewise polynomial and variableknot spline methods equipped with an oracle in contrast it is unknown how or if piecewise polynomial methods could be made to function this well when denied access to an oracle and forced to rely on data alone

parameter estimation with scarce measurements in this paper the problems of parameter estimation are addressed for systems with scarce measurements a gradient based algorithm is derived to estimate the parameters of the input output representation with scarce measurements and the convergence properties of the parameter estimation and unavailable output estimation are established using the kronecker lemma and the deterministic version of the martingale convergence theorem finally an example is provided to demonstrate the effectiveness of the proposed algorithm

scale space and edge detection using anisotropic diffusion a new definition of scale space is suggested and a class of algorithms used to realize a diffusion process is introduced the diffusion coefficient is chosen to vary spatially in such a way as to encourage intraregion smoothing rather than interregion smoothing it is shown that the no new maxima should be generated at coarse scales property of conventional scale space is preserved as the region boundaries in the approach remain sharp a high quality edge detector which successfully exploits global information is obtained experimental results are shown on a number of images parallel hardware implementations are made feasible because the algorithm involves elementary local operations replicated over the image

inverse regression estimation for censored data an inverse regression methodology for assessing predictor performance in the censored data setup is developed along with inference procedures and a computational algorithm the technique developed here allows for conditioning on the unobserved failure time along with a weighting mechanism that accounts for the censoring the implementation is nonparametric and computationally fast this provides an efficient methodological tool that can be used especially in cases where the usual modeling assumptions are not applicable to the data under consideration it can also be a good diagnostic tool that can be used in the model selection process we have provided theoretical justification of consistency and asymptotic normality of the methodology simulation studies and two data analyses are provided to illustrate the practical utility of the procedure

likelihood based sufficient dimension reduction we obtain the maximum likelihood estimator of the central subspace under conditional normality of the predictors given the response analytically and in simulations we found that our new estimator can preform much better than sliced inverse regression sliced average variance estimation and directional regression and that it seems quite robust to deviations from normality

graphical procedures for multiple comparisons under general dependence abstractit has been more than half a century since tukey first introduced graphical displays that relate nonoverlap of confidence intervals to statistically significant differences between parameter estimates in this article we show how tukey s graphical overlap procedure can be modified to accommodate general forms of dependence within and across samples we also develop a procedure that can be used to more effectively resolve rankings within the tails of the distributions of parameter values thereby generalizing existing methods for multiple comparisons with the best we show that these new procedures retain the simplicity of tukey s original procedure while maintaining asymptotic control of the familywise error rate under very general conditions simple examples are used throughout to illustrate the procedures supplementary materials for this article are available online

regularization theory and neural networks architectures we had previously shown that regularization principles lead to approximation schemes that are equivalent to networks with one layer of hidden units called regularization networks in particular s

confidence sets for a multivariate distribution on etudie une version multivariable alternative de la bande de confiance unidimensionnelle de kolmogorov smirnov qui preserve l invariance affine

identifying regression outliers and mixtures graphically regressions in practice can include outliers and other unknown subpopulation structures for example mixtures of regressions occur if there is an omitted categorical predictor like gender or location and different regressions occur within each category the theory of regression graphics based on central subspaces can be used to construct graphical solutions to long standing problems of this type it is argued that in practice the central subspace automatically expands to incorporate outliers and regression mixtures thus methods of estimating the central subspace can be used to identify these phenomena without specifying a model examples illustrating the power of the theory are presented

a new sure approach to image denoising interscale orthonormal wavelet thresholding this paper introduces a new approach to orthonormal wavelet image denoising instead of postulating a statistical model for the wavelet coefficients we directly parametrize the denoising process as a sum of elementary nonlinear processes with unknown weights we then minimize an estimate of the mean square error between the clean image and the denoised one the key point is that we have at our disposal a very accurate statistically unbiased mse estimate stein s unbiased risk estimate that depends on the noisy image alone not on the clean one like the mse this estimate is quadratic in the unknown weights and its minimization amounts to solving a linear system of equations the existence of this a priori estimate makes it unnecessary to devise a specific statistical model for the wavelet coefficients instead and contrary to the custom in the literature these coefficients are not considered random any more we describe an interscale orthonormal wavelet thresholding algorithm based on this new approach and show its near optimal performance both regarding quality and cpu requirement by comparing it with the results of three state of the art nonredundant denoising algorithms on a large set of test images an interesting fallout of this study is the development of a new group delay based parent child prediction in a wavelet dyadic tree

normalized cuts and image segmentation we propose a novel approach for solving the perceptual grouping problem in vision rather than focusing on local features and their consistencies in the image data our approach aims at extracting the global impression of an image we treat image segmentation as a graph partitioning problem and propose a novel global criterion the normalized cut for segmenting the graph the normalized cut criterion measures both the total dissimilarity between the different groups as well as the total similarity within the groups we show that an efficient computational technique based on a generalized eigenvalue problem can be used to optimize this criterion we applied this approach to segmenting static images as well as motion sequences and found the results to be very encouraging

image denoising via sparse and redundant representations over learned dictionaries we address the image denoising problem where zero mean white and homogeneous gaussian additive noise is to be removed from a given image the approach taken is based on sparse and redundant representations over trained dictionaries using the k svd algorithm we obtain a dictionary that describes the image content effectively two training options are considered using the corrupted image itself or training on a corpus of high quality image database since the k svd is limited in handling small image patches we extend its deployment to arbitrary image sizes by defining a global image prior that forces sparsity over patches in every location in the image we show how such bayesian treatment leads to a simple and effective denoising algorithm this leads to a state of the art denoising performance equivalent and sometimes surpassing recently published leading alternative denoising methods

single image super resolution using local geometric duality and non local similarity super resolution sr from a single image plays an important role in many computer vision applications it aims to estimate a high resolution hr image from an input low resolution lr image to ensure a reliable and robust estimation of the hr image we propose a novel single image sr method that exploits both the local geometric duality gd and the non local similarity of images the main principle is to formulate these two typically existing features of images as effective priors to constrain the super resolved results in consideration of this principle the robust soft decision interpolation method is generalized as an outstanding adaptive gd agd based local prior to adaptively design weights for the agd prior a local non smoothness detection method and a directional standard deviation based weights selection method are proposed after that the agd prior is combined with a variational framework based non local prior furthermore the proposed algorithm is speeded up by a fast gd matrices construction method which primarily relies on the selective pixel processing the extensive experimental results verify the effectiveness of the proposed method compared with several state of the art sr algorithms

on the conditional distributions of low dimensional projections from high dimensional data we study the conditional distribution of low dimensional projections from high dimensional data where the conditioning is on other low dimensional projections to fix ideas consider a random d vector z that has a lebesgue density and that is standardized so that ez 0 and ezz id moreover consider two projections defined by unit vectors and namely a response y z and an explanatory variable x z it has long been known that the conditional mean of y given x is approximately linear in x under some regularity conditions cf hall and li 1993 however a corresponding result for the conditional variance has not been available so far we here show that the conditional variance of y given x is approximately constant in x again under some regularity conditions these results hold uniformly in and for most s provided only that the dimension of z is large in that sense we see that most linear submodels of a high dimensional overall model are approximately correct our findings provide new insights in a variety of modeling scenarios we discuss several examples including sparse linear modeling generalized linear models under potential link violation sliced inverse regression sliced average variance estimation and kernel learning machines

hierarchical estimation algorithms for multivariable systems using measurement information abstract with the development of industry information technology many modelling methods have been focusing on the estimation problems of multivariable systems especially for the multivariable systems with output error autoregressive noises from input output measurement information since such a system includes both a parameter vector and a parameter matrix the conventional methods cannot be applied to parameter estimation and modelling in order to solve this difficulty a hierarchical least squares based iterative identification algorithm and a hierarchical generalized least squares identification algorithm are proposed the basic idea is to decompose the system into two fictitious subsystems to estimate the parameters of each subsystem and to coordinate the associated items between the two subsystems the simulation results indicate that the proposed algorithm is effective

a note on visualizing response transformations in regression a new graphical method for assessing parametric transformations of the response in linear regression is given we simply regress the response variable y on the predictors find the fitted values and then dynamically plot the transformed response y against those fitted values by varying the transformation parameter until the plot is linear the method can be used also to assess the success of numerical response transformation methods and to discover influential observations modifications using robust estimators can be used as well

sliced inverse regression for dimension reduction abstract modern advances in computing power have greatly widened scientists scope in gathering and investigating information from many variables information which might have been ignored in the past yet to effectively scan a large pool of variables is not an easy task although our ability to interact with data has been much enhanced by recent innovations in dynamic graphics in this article we propose a novel data analytic tool sliced inverse regression sir for reducing the dimension of the input variable x without going through any parametric or nonparametric model fitting process this method explores the simplicity of the inverse view of regression that is instead of regressing the univariate output variable y against the multivariate x we regress x against y forward regression and inverse regression are connected by a theorem that motivates this method the theoretical properties of sir are investigated under a model of the form y f 1 x k x e where the k s are the unknown

graph connection laplacian methods can be made robust to noise recently several data analytic techniques based on graph connection laplacian gcl ideas have appeared in the literature at this point the properties of these methods are starting to be understood in the setting where the data is observed without noise we study the impact of additive noise on these methods and show that they are remarkably robust as a by product of our analysis we propose modifications of the standard algorithms that increase their robustness to noise we illustrate our results in numerical simulations

robust kernel principal component analysis this letter discusses the robustness issue of kernel principal component analysis a class of new robust procedures is proposed based on eigenvalue decomposition of weighted covariance the proposed procedures will place less weight on deviant patterns and thus be more resistant to data contamination and model deviation theoretical influence functions are derived and numerical examples are presented as well both theoretical and numerical results indicate that the proposed robust method outperforms the conventional approach in the sense of being less sensitive to outliers our robust method and results also apply to functional principal component analysis

dimension reduction in regression without matrix inversion summary regressions in which the fixed number of predictors p exceeds the number of independent observational units n occur in a variety of scientific fields sufficient dimension reduction provides a promising approach to such problems by restricting attention to d n linear combinations of the original p predictors however standard methods of sufficient dimension reduction require inversion of the sample predictor covariance matrix we propose a method for estimating the central subspace that eliminates the need for such inversion and is applicable regardless of the n p relationship simulations show that our method compares favourably with standard large sample techniques when the latter are applicable we illustrate our method with a genomics application

behavioral analysis of anisotropic diffusion in image processing in this paper we analyze the behavior of the anisotropic diffusion model of perona and malik 1990 the main idea is to express the anisotropic diffusion equation as coming from a certain optimization problem so its behavior can be analyzed based on the shape of the corresponding energy surface we show that anisotropic diffusion is the steepest descent method for solving an energy minimization problem it is demonstrated that an anisotropic diffusion is well posed when there exists a unique global minimum for the energy functional and that the ill posedness of a certain anisotropic diffusion is caused by the fact that its energy functional has an infinite number of global minima that are dense in the image space we give a sufficient condition for an anisotropic diffusion to be well posed and a sufficient and necessary condition for it to be ill posed due to the dense global minima the mechanism of smoothing and edge enhancement of anisotropic diffusion is illustrated through a particular orthogonal decomposition of the diffusion operator into two parts one that diffuses tangentially to the edges and therefore acts as an anisotropic smoothing operator and the other that flows normally to the edges and thus acts as an enhancement operator

on distribution weighted partial least squares with diverging number of highly correlated predictors summary because highly correlated data arise from many scientific fields we investigate parameter estimation in a semiparametric regression model with diverging number of predictors that are highly correlated for this we first develop a distribution weighted least squares estimator that can recover directions in the central subspace then use the distribution weighted least squares estimator as a seed vector and project it onto a krylov space by partial least squares to avoid computing the inverse of the covariance of predictors thus distrbution weighted partial least squares can handle the cases with high dimensional and highly correlated predictors furthermore we also suggest an iterative algorithm for obtaining a better initial value before implementing partial least squares for theoretical investigation we obtain strong consistency and asymptotic normality when the dimension p of predictors is of convergence rate o n1 2 log n and o n1 3 respectively where n is the sample size when there are no other constraints on the covariance of predictors the rates n1 2 and n1 3 are optimal we also propose a bayesian information criterion type of criterion to estimate the dimension of the krylov space in the partial least squares procedure illustrative examples with a real data set and comprehensive simulations demonstrate that the method is robust to non ellipticity and works well even in small n large p problems

principal neighborhood dictionaries for nonlocal means image denoising we present an in depth analysis of a variation of the nonlocal means nlm image denoising algorithm that uses principal component analysis pca to achieve a higher accuracy while reducing computational load image neighborhood vectors are first projected onto a lower dimensional subspace using pca the dimensionality of this subspace is chosen automatically using parallel analysis consequently neighborhood similarity weights for denoising are computed using distances in this subspace rather than the full space the resulting algorithm is referred to as principal neighborhood dictionary pnd nonlocal means we investigate pnd s accuracy as a function of the dimensionality of the projection subspace and demonstrate that denoising accuracy peaks at a relatively low number of dimensions the accuracy of nlm and pnd are also examined with respect to the choice of image neighborhood and search window sizes finally we present a quantitative and qualitative comparison of pnd versus nlm and another image neighborhood pca based state of the art image denoising algorithm

self consistency and principal component analysis abstract i examine the self consistency of a principal component axis that is when a distribution is centered about a principal component axis a principal component axis of a random vector x is self consistent if each point on the axis corresponds to the mean of x given that x projects orthogonally onto that point a large class of symmetric multivariate distributions are examined in terms of self consistency of principal component subspaces elliptical distributions are characterized by the preservation of self consistency of principal component axes after arbitrary linear transformations a lack of fit test is proposed that tests for self consistency of a principal axis the test is applied to two real datasets

a probabilistic analysis of em for mixtures of separated spherical gaussians we show that given data from a mixture of k well separated spherical gaussians in d a simple two round variant of em will with high probability learn the parameters of the gaussians to near optimal precision if the dimension is high d ln k we relate this to previous theoretical and empirical work on the em algorithm

new families of skewed higher order kernel estimators to solve the bss ica problem for multimodal sources mixtures this letter suggests two new types of asymmetrical higher order kernels hok that are generated using the orthogonal polynomials laguerre positive or right skew and bessel negative or left skew these skewed hok are implemented in the blind source separation independent component analysis bss ica algorithm the tests for these proposed hok are accomplished using three scenarios to simulate a real environment using actual sound sources an environment of mixtures of multimodal fast changing probability density function pdf sources that represent a challenge to the symmetrical hok and an environment of an adverse case near gaussian the separation is performed by minimizing the mutual information mi among the mixed sources the performance of the skewed kernels is compared to the performance of the standard kernels such as epanechnikov bisquare trisquare and gaussian and the performance of the symmetrical hok generated using the polynomials chebyshev1 chebyshev2 gegenbauer jacobi and legendre to the tenth order the gaussian hok are generated using the hermite polynomial and the wand and schucany procedure the comparison among the 96 kernels is based on the average intersymbol interference ratio aisir and the time needed to complete the separation in terms of aisir the skewed kernels performance is better than that of the standard kernels and rivals most of the symmetrical kernels performance the importance of these new skewed hok is manifested in the environment of the multimodal pdf mixtures in such an environment the skewed hok come in first place compared with the symmetrical hok these new families can substitute for symmetrical hoks in such applications

sudakov s typical marginals random linear functionals and a conditional central limit theorem v n sudakov sud78 proved that the one dimensional marginals of a high dimensional second order measure are close to each other in most directions extending this and a related result in the context of projection pursuit of p diaconis and d freedman dia84 we give for a probability measure p and a random a s linear functional f on a hilbert space simple sufficient conditions under which most of the one dimensional images of p under f are close to their canonical mixture which turns out to be almost a mixed normal distribution using the concept of approximate conditioning we deduce a conditional central limit theorem theorem 3 for random averages of triangular arrays of random variables which satisfy only fairly weak asymptotic orthogonality conditions

multi innovation stochastic gradient algorithm for multiple input single output systems using the auxiliary model in order to reduce computational burden and improve the convergence rate of identification algorithms an auxiliary model based multi innovation stochastic gradient am misg algorithm is derived for the multiple input single output systems by means of the auxiliary model identification idea and multi innovation identification theory the basic idea is to replace the unknown outputs of the fictitious subsystems in the information vector with the outputs of the auxiliary models and to present an auxiliary model based stochastic gradient algorithm and then to derive the am misg algorithm by expanding the scalar innovation to innovation vector and introducing the innovation length the simulation example shows that the proposed algorithms work quite well

on surrogate dimension reduction for measurement error regression an invariance law we consider a general nonlinear regression problem where the predictors contain measurement error it has been recently discovered that several well known dimension reduction methods such as ols sir and phd can be performed on the surrogate regression problem to produce consistent estimates for the original regression problem involving the unobserved true predictor in this paper we establish a general invariance law between the surrogate and the original dimension reduction spaces which implies that at least at the population level the two dimension reduction problems are in fact equivalent consequently we can apply all existing dimension reduction methods to measurement error regression problems the equivalence holds exactly for multivariate normal predictors and approximately for arbitrary predictors we also characterize the rate of convergence for the surrogate dimension reduction estimators finally we apply several dimension reduction methods to real and simulated data sets involving measurement error to compare their performances

data filtering based recursive least squares algorithm for hammerstein systems using the key term separation principle this paper concerns parameter identification of hammerstein output error moving average systems with a two segment piecewise nonlinearity by combining the key term separation principle and the data filtering technique we transfer the hammerstein model into two regression identification models and present a data filtering based recursive least squares method to estimate the parameters of these two identification models the proposed algorithm achieves a higher computational efficiency than the standard approach by using covariance matrices of smaller dimensions from the two identification models instead of one identification model in the standard approach

sensitivity analysis for box cox power transformation model contrast parameters summary sensitivity analysis is an important tool for the evaluation of scientific models we consider the box cox power transformation model and study the sensitivity for the estimated slope vector when the transformation parameter is perturbed this is a key issue in the debate among bickel doksum 1981 box cox 1982 and hinkley runger 1984 on the application of the power transformation model hinkley runger 1984 conjectured that box cox s 1964 z transformation would eliminate asymptotically the sensitivity for the estimated slope vector we establish this conjecture under appropriate symmetry conditions on the joint distribution for the regressors x when the true transformation is logarithmic the conjecture holds if the distribution for x is axially symmetric for other power transformations the conjecture holds if x is multivariate normal we also consider a weaker version of hinkley runger s conjecture namely the z transformation would achieve the best possible reduction in the sensitivity we establish this weaker conjecture under less restrictive symmetry conditions which only involve the marginal distribution for xfl both conjectures might fail when those symmetry conditions are not satisfied two examples are given to illustrate the limitations of the conjectures sensitivity analysis is an important tool for the evaluation of scientific models many scientific models include some nuisance parameters which are not known precisely it is usually desirable for the model output to be stable with respect to those parameters the performance of the model can be questionable if the model output is highly sensitive to those parameters it is therefore useful to carry out a sensitivity analysis in which those parameters are perturbed the model output is derived under the perturbed parameters and compared with the model output prior to the perturbation we will study the sensitivity analysis for the box cox power transformation model with respect to the specific transformation taken let yi xi i 1 n be the observed data where yi denotes a positive scalar outcome and xi denotes a p dimensional row vector of regressors let the transformed outcome be

the central limit problem for convex bodies it is shown that every symmetric convex body which satisfies a kind of weak law of large numbers has the property that almost all its marginal distributions are approximately gaussian several quite broad classes of bodies are shown to satisfy the condition

exploratory projection pursuit abstract a new projection pursuit algorithm for exploring multivariate data is presented that has both statistical and computational advantages over previous methods a number of practical issues concerning its application are addressed a connection to multivariate density estimation is established and its properties are investigated through simulation studies and application to real data the goal of exploratory projection pursuit is to use the data to find low one two or three dimensional projections that provide the most revealing views of the full dimensional data with these views the human gift for pattern recognition can be applied to help discover effects that may not have been anticipated in advance since linear effects are directly captured by the covariance structure of the variable pairs which are straightforward to estimate the emphasis here is on the discovery of nonlinear effects such as clustering or other general nonlinear associations among the variables although arbitrary

classification of fmri time series in a low dimensional subspace with a spatial prior we propose a new method for detecting activation in functional magnetic resonance imaging fmri data we project the fmri time series on a low dimensional subspace spanned by wavelet packets in order to create projections that are as non gaussian as possible our approach achieves two goals it reduces the dimensionality of the problem by explicitly constructing a sparse approximation to the dataset and it also creates meaningful clusters allowing the separation of the activated regions from the clutter formed by the background time series we use a mixture of gaussian densities to model the distribution of the wavelet packet coefficients we expect activated areas that are connected and impose a spatial prior in the form of a markov random field our approach was validated with in vivo data and realistic synthetic data where it outperformed a linear model equipped with the knowledge of the true hemodynamic response

dimension reduction for conditional mean in regression in many situations regression analysis is mostly concerned with inferring about the conditional mean of the response given the predictors and less concerned with the other aspects of the conditional distribution in this paper we develop dimension reduction methods that incorporate this consideration we introduce the notion of the central mean subspace cms a natural inferential object for dimension reduction when the mean function is of interest we study properties of the cms and develop methods to estimate it these methods include a new class of estimators which requires fewer conditions than phd and which displays a clear advantage when one of the conditions for phd is violated cms also reveals a transparent distinction among the existing methods for dimension reduction ols phd sir and save we apply the new methods to a data set involving recumbent cows

determining the dimension of iterative hessian transformation the central mean subspace cms and iterative hessian transformation iht have been introduced recently for dimension reduction when the conditional mean is of interest suppose that x is a vector valued predictor and y is a scalar response the basic problem is to find a lower dimensional predictor t x such that e y x e y t x the cms defines the inferential object for this problem and iht provides an estimating procedure compared with other methods iht requires fewer assumptions and has been shown to perform well when the additional assumptions required by those methods fail in this paper we give an asymptotic analysis of iht and provide stepwise asymptotic hypothesis tests to determine the dimension of the cms as estimated by iht here the original iht method has been modified to be invariant under location and scale transformations to provide empirical support for our asymptotic results we will present a series of simulation studies these agree well with the theory the method is applied to analyze an ozone data set

multimodality exploration by an unsupervised projection pursuit neural network graphical inspection of multimodality is demonstrated using unsupervised lateral inhibition neural networks three projection pursuit indexes are compared on low dimensional simulated and real world data principal components legendre polynomial and projection pursuit network

a multi innovation generalized extended stochastic gradient algorithm for output nonlinear autoregressive moving average systems this paper proposes a generalized extended stochastic gradient gesg algorithm for estimating the parameters of a class of wiener nonlinear autoregressive moving average systems using the gradient search in order to improve the convergence rates of the gesg algorithm a multi innovation gesg algorithm is derived the simulation results indicate that the proposed algorithms can effectively estimate the parameters of a class of output nonlinear systems

novel speed up strategies for non local means denoising with patch and edge patch based dictionaries in this paper a novel technique to speed up a non local means nlm filter is proposed in the original nlm filter most of its computational time is spent on finding distances for all the patches in the search window here we build a dictionary in which patches with similar photometric structures are clustered together dictionary is built only once with high resolution images belonging to different scenes since the dictionary is well organized in terms of indexing its entries it is used to search similar patches very quickly for efficient nlm denoising we achieve a substantial reduction in computational cost compared with the original nlm method especially when the search window of nlm is large without much affecting the psnr second we show that by building a dictionary for edge patches as opposed to intensity patches it is possible to reduce the dictionary size thus further improving the computational speed and memory requirement the proposed method preclassifies similar patches with the same distance measure as used by nlm method the proposed algorithm is shown to outperform other prefiltering based fast nlm algorithms computationally as well as qualitatively

image approximation and modeling via least statistically dependent bases abstract statistical independence is one of the most desirable properties of a coordinate system for representing and modeling images in reality however truly independent coordinates may not exist for a given set of images or it may be too difficult to compute them in practice therefore we propose a new method to rapidly compute the least statistically dependent basis lsdb from a basis dictionary e g the local cosine or wavelet packet dictionaries containing a huge number of orthonormal or biorthogonal bases our new basis selection criterion is minimization of the mutual information of the distributions of the basis coefficients as a measure of statistical dependence which in turn is equivalent to minimization of the sum of the differential entropy of each coordinate in the basis dictionary in this sense we can view this lsdb algorithm as the best basis version of the independent component analysis ica which is increasingly gaining popularity this criterion is different from that of the joint best basis jbb proposed by wickerhauser which can be viewed as the best basis version of the karhunen loeve basis klb we demonstrate the usefulness of the lsdb for image approximation and modeling and compare its performance with that of klb and jbb using a collection of real geophysical acoustic waveforms and an image database of human faces

multiresolution bilateral filtering for image denoising the bilateral filter is a nonlinear filter that does spatial averaging without smoothing edges it has shown to be an effective image denoising technique an important issue with the application of the bilateral filter is the selection of the filter parameters which affect the results significantly there are two main contributions of this paper the first contribution is an empirical study of the optimal bilateral filter parameter selection in image denoising applications the second contribution is an extension of the bilateral filter multiresolution bilateral filter where bilateral filtering is applied to the approximation low frequency subbands of a signal decomposed using a wavelet filter bank the multiresolution bilateral filter is combined with wavelet thresholding to form a new image denoising framework which turns out to be very effective in eliminating noise in real noisy images experimental results with both simulated and real data are provided

a model free test for reduced rank in multivariate regression we propose a test of dimension in multivariate regression this test is in the spirit of tests on the rank of the coefficient matrix in a multivariate linear model but it does not require a prespecified model the test may be particularly useful at the outset of an analysis before a multivariate model is posited because it can lead to low dimensional summary plots that are inferred to contain all of the sample information on the multivariate mean function

a multivariate distribution for sub images a new method for obtaining multivariate distributions for sub images of natural images is described the information in each sub image is summarized by a measurement vector in a measurement space the dimension of the measurement space is reduced by applying a random projection to the truncated output of the discrete cosine transforms of the sub images the measurement space is then reparametrized such that a gaussian distribution is a good model for the measurement vectors in the reparametrized space an ornstein uhlenbeck process associated with the gaussian distribution is used to model the differences between measurement vectors obtained from matching sub images the probability of a false alarm and the probability of accepting a correct match are calculated the accuracy of the resulting statistical model for matching sub images is tested using images from the middlebury stereo database with promising results in particular if the probability of accepting a correct match is relatively large then there is good agreement between the calculated and the experimental probabilities of obtaining a unique match that is also a correct match

gradient based and least squares based iterative algorithms for matrix equations axb cxtd f abstract this paper develops a gradient based and a least squares based iterative algorithms for solving matrix equation axb cx t d f the basic idea is to decompose the matrix equation system under consideration into two subsystems by applying the hierarchical identification principle and to derive the iterative algorithms by extending the iterative methods for solving ax b and axb f the analysis shows that when the matrix equation has a unique solution under the sense of least squares the iterative solution converges to the exact solution for any initial values a numerical example verifies the proposed theorems

on the impact of predictor geometry on the performance on high dimensional ridge regularized generalized robust regression estimators we study ridge regularized generalized robust regression estimators i e begin aligned widehat beta mathrm argmin beta in mathbb r p frac 1 n sum i 1 n rho i y i x i beta frac tau 2 beta 2 quad text where quad y i epsilon i x i beta 0 end aligned in the situation where p n tends to a finite non zero limit our study here focuses on the situation where the errors epsilon i s are heavy tailed and x i s have an elliptical like distribution our assumptions are quite general and we do not require homoskedasticity of epsilon i s for instance we obtain a characterization of the limit of widehat beta beta 0 as well as several other results including central limit theorems for the entries of widehat beta

a statistical model of cluster stability in the current paper we present a method for assessing cluster stability this method combined with a clustering algorithm yields an estimate of the data partition namely the number of clusters we adopt the cluster stability standpoint where clusters are imagined as islands of high density in a sea of low density explicitly a cluster is associated with its high density core our approach offers to evaluate the goodness of a cluster by the similarity amongst the entire cluster and its core we propose to measure this resemblance by two sample tests or by probability distances between appropriate probability distributions the distances are calculated on clustered samples drawn from the source population according to two different distributions the first law is the underlying set distribution the second law is constructed so that it represents the clusters cores here a variant of the k nearest neighbor density estimation is applied so that items belonging to cores have a much higher chance to be selected as the sample distribution is unknown a distribution free two sample test is required to examine the mentioned correspondence for constructing such a test we use distance functions built on negative definite kernels in practice outliers in the samples and limitations of the clustering algorithm heavily contribute to the noise level as a result of this shortcoming the distance values have to be determined for many pairs of samples and therefore an empirical distance s distribution is obtained the distribution is dependent on the examined number of clusters to prevent this property for biasing the results we normalize the distances it is conjectured that the true number of clusters yields the most concentrated normalized distribution to measure the concentration we use the sample mean and the sample 25th percentile the paper exhibits the good performance of the proposed method on synthetic and real world data

on directional regression for dimension reduction we introduce directional regression dr as a method for dimension reduction like contour regression dr is derived from empirical directions but achieves higher accuracy and requires substantially less computation dr naturally synthesizes the dimension reduction estimators based on conditional moments such as sliced inverse regression and sliced average variance estimation and in doing so combines the advantages of these methods under mild conditions it provides exhaustive and consistent estimate of the dimension reduction space we develop the asymptotic distribution of the dr estimator and from that a sequential test procedure to determine the dimension of the central space we compare the performance of dr with that of existing methods by simulation and find strong evidence of its advantage over a wide range of models finally we apply dr to analyze a data set concerning the identification of hand written digits

a central limit theorem for convex sets we show that there exists a sequence varepsilon n searrow0 for which the following holds let k n be a compact convex set with a non empty interior let x be a random vector that is distributed uniformly in k then there exist a unit vector in n t0 and 0 such that sup a subset mathbb r left textit prob langle x theta rangle in a frac 1 sqrt 2 pi sigma int ae frac t t 0 2 2 sigma 2 dt right leq varepsilon n qquad ast where the supremum runs over all measurable sets a and where denotes the usual scalar product in n furthermore under the additional assumptions that the expectation of x is zero and that the covariance matrix of x is the identity matrix we may assert that most unit vectors satisfy with t0 0 and 1 corresponding principles also hold for multi dimensional marginal distributions of convex sets

adaptive wavelet thresholding for image denoising and compression the first part of this paper proposes an adaptive data driven threshold for image denoising via wavelet soft thresholding the threshold is derived in a bayesian framework and the prior used on the wavelet coefficients is the generalized gaussian distribution ggd widely used in image processing applications the proposed threshold is simple and closed form and it is adaptive to each subband because it depends on data driven estimates of the parameters experimental results show that the proposed method called bayesshrink is typically within 5 of the mse of the best soft thresholding benchmark with the image assumed known it also outperforms sureshrink donoho and johnstone 1994 1995 donoho 1995 most of the time the second part of the paper attempts to further validate claims that lossy compression can be used for denoising the bayesshrink threshold can aid in the parameter selection of a coder designed with the intention of denoising and thus achieving simultaneous denoising and compression specifically the zero zone in the quantization step of compression is analogous to the threshold value in the thresholding function the remaining coder design parameters are chosen based on a criterion derived from rissanen s minimum description length mdl principle experiments show that this compression method does indeed remove noise significantly especially for large noise power however it introduces quantization noise and should be used only if bitrate were an additional concern to denoising

on robust regression with high dimensional predictors we study regression m estimates in the setting where p the number of covariates and n the number of observations are both large but p n we find an exact stochastic representation for the distribution of argmin p i 1 n y i x i at fixed p and n under various assumptions on the objective function and our statistical model a scalar random variable whose deterministic limit r can be studied when p n 0 plays a central role in this representation we discover a nonlinear system of two deterministic equations that characterizes r interestingly the system shows that r depends on through proximal mappings of as well as various aspects of the statistical model underlying our study several surprising results emerge in particular we show that when p n is large enough least squares becomes preferable to least absolute deviations for double exponential errors

the discrimination power of projection pursuit with different density estimators summary we explore the properties of projection pursuit discriminant analysis this discriminant method is very powerful but relies heavily on a univariate density estimate we show that the procedure based on wavelets maintains the same rate of convergence as with univariate wavelet density estimation we also show the bayes risk strong consistency of both the kernel and wavelet based methods simulated data and real data concerning character recognition show that the method is effective and robust against the curse of dimensionality the wavelet alternative seems more likely than the kernel counterpart to find an interesting projection wavelets are often criticised for giving too wiggly an estimate and for being too localised to give good global properties in the above context these potential drawbacks do not weaken the method but the use of wavelets seems to enhance it a multiple projection generalisation is also considered

rates of growth and sample moduli for weighted empirical processes indexed by sets probability inequalities are obtained for the supremum of a weighted empirical process indexed by a vapnik cervonenkis class c of sets these inequalities are particularly useful under the assumption p c c p c t 0 as t 0 they are used to obtain almost sure bounds on the rate of growth of the process as the sample size approaches infinity to find an asymptotic sample modulus for the unweighted empirical process and to study the ratio pn p of the empirical measure to the actual measure

projection pursuit in high dimensions projection pursuit is a classical exploratory data analysis method to detect interesting low dimensional structures in multivariate data originally projection pursuit was applied mostly to data of moderately low dimension motivated by contemporary applications we here study its properties in high dimensional settings specifically we analyze the asymptotic properties of projection pursuit on structureless multivariate gaussian data with an identity covariance as both dimension p and sample size n tend to infinity with p n 0 our main results are that i if then there exist projections whose corresponding empirical cumulative distribution function can approximate any arbitrary distribution and ii if 0 not all limiting distributions are possible however depending on the value of various non gaussian distributions may still be approximated in contrast if we restrict to sparse projections involving only a few of the p variables then asymptotically all empirical cumulative distribution functions are gaussian and iii if 0 then asymptotically all projections are gaussian some of these results extend to mean centered sub gaussian data and to projections into k dimensions hence in the small n large p setting unless sparsity is enforced and regardless of the chosen projection index projection pursuit may detect an apparent structure that has no statistical significance furthermore our work reveals fundamental limitations on the ability to detect non gaussian signals in high dimensional data in particular through independent component analysis and related non gaussian component analysis

estimating the structural dimension of regressions via parametric inverse regression a new estimation method for the dimension of a regression at the outset of an analysis is proposed a linear subspace spanned by projections of the regressor vector x which contains part or all of the modelling information for the regression of a vector y on x and its dimension are estimated via the means of parametric inverse regression smooth parametric curves are fitted to the p inverse regressions via a multivariate linear model no restrictions are placed on the distribution of the regressors the estimate of the dimension of the regression is based on optimal estimation procedures a simulation study shows the method to be more powerful than sliced inverse regression in some situations

h control for discrete time markov jump systems with uncertain transition probabilities in this technical note the h control problem for a class of discrete time markov jump systems mjss with uncertain transition probabilities tps is investigated the uncertain information of transition probabilities is quantized by gaussian transition probability density function pdf in light of the proposed descriptions the mjss with gaussian pdf of tps cover the systems with precisely known and partially known tps as two special cases sufficient conditions for the existence of h controller of the underlying systems are derived in term of linear matrix inequalities a numerical example is presented to illustrate the effectiveness and potential of the developed theoretical results

linear dimensionality reduction survey insights and generalizations linear dimensionality reduction methods are a cornerstone of analyzing high dimensional data due to their simple geometric interpretations and typically attractive computational properties these methods capture many data features of interest such as covariance dynamical structure correlation between data sets input output relationships and margin between data classes methods have been developed with a variety of names and motivations in many fields and perhaps as a result the connections between all these methods have not been highlighted here we survey methods from this disparate literature as optimization programs over matrix manifolds we discuss principal component analysis factor analysis linear multidimensional scaling fisher s linear discriminant analysis canonical correlations analysis maximum autocorrelation factors slow feature analysis sufficient dimensionality reduction undercomplete independent component analysis linear regression distance metric learning and more this optimization framework gives insight to some rarely discussed shortcomings of well known methods such as the suboptimality of certain eigenvector solutions modern techniques for optimization over matrix manifolds enable a generic linear dimensionality reduction solver which accepts as input data and an objective to be optimized and returns as output an optimal low dimensional projection of the data this simple optimization framework further allows straightforward generalizations and novel variants of classical methods which we demonstrate here by creating an orthogonal projection canonical correlations analysis more broadly this survey and generic solver suggest that linear dimensionality reduction can move toward becoming a blackbox objective agnostic numerical technology

pattern classification using projection pursuit abstract this article discusses the adaptation of recently developed regression techniques to classifier design apart from finite sample effects projection pursuit pp regression may be used to model a desired response class as a sum of ridge functions according to a minimum expected squared error criterion this approach can be shown to furnish an optimal discriminant function which can satisfy the neyman pearson criterion over all possible thresholds basis function expansions are used instead of smoothed histograms to reduce computation since good approximation of a discriminant by a linear combination of moderate number of ridge functions may not be easy we introduce an improved method utilizing a nonlinear weighting function

dimension reduction for censored regression data without parametric assumptions high dimensional regression analysis is already complex this is made even harder when data are subject to censoring in this article we seek ways of reducing the dimensionality of the regressor before applying nonparametric smoothing techniques if the censoring time is independent of the lifetime then the method of sliced inverse regression can be applied directly otherwise modification is needed to adjust for the censoring bias a key identity leading to the bias correction is derived and the root n consistency of the modified estimate is established patterns of censoring can also be studied under a similar dimension reduction framework some simulation results and an application to a real data set are reported 1 introduction survival data are often subject to censoring when this occurs the incompleteness of the observed data may induce a substantial bias in the sample several approaches have been suggested to overcome the associated difficulties in regression including the accelerated failure time model censored linear regression the cox proportional hazard model and many others survival analysis becomes even more intricate when the dimension of the regressor increases to apply any of the aforementioned methods users are required to specify a functional form which relates the outcome variables to the input ones however in reality knowledge needed for an appropriate model specification is often inadequate as a matter of fact the acquisition of such information may well turn out to be one of the primary goals of the study itself under such circumstances it seems preferable to have exploratory tools that rely less on such model specification this is the issue to be addressed in this article the dimension reduction approach of li 1991 will be extended to settings which allow for censoring in the data we shall offer methods of finding low dimensional projections of the data for visually examining the censoring pattern we shall show how censored regression data can still be analyzed without assuming the functional form a priori

nonlocal means with dimensionality reduction and sure based parameter selection nonlocal means nlm is an effective denoising method that applies adaptive averaging based on similarity between neighborhoods in the image an attractive way to both improve and speed up nlm is by first performing a linear projection of the neighborhood one particular example is to use principal components analysis pca to perform dimensionality reduction here we derive stein s unbiased risk estimate sure for nlm with linear projection of the neighborhoods the sure can then be used to optimize the parameters by a search algorithm or we can consider a linear expansion of multiple nlms each with a fixed parameter set for which the optimal weights can be found by solving a linear system of equations the experimental results demonstrate the accuracy of the sure and its successful application to tune the parameters for nlm

dictionary learning algorithms for sparse representation algorithms for data driven learning of domain specific overcomplete dictionaries are developed to obtain maximum likelihood and maximum a posteriori dictionary estimates based on the use of bayesian models with concave schur concave csc negative log priors such priors are appropriate for obtaining sparse representations of environmental signals within an appropriately chosen environmentally matched dictionary the elements of the dictionary can be interpreted as concepts features or words capable of succinct expression of events encountered in the environment the source of the measured signals this is a generalization of vector quantization in that one is interested in a description involving a few dictionary entries the proverbial 25 words or less but not necessarily as succinct as one entry to learn an environmentally adapted dictionary capable of concise expression of signals generated by the environment we develop algorithms that iterate between a representative set of sparse representations found by variants of focuss and an update of the dictionary using these sparse representations experiments were performed using synthetic data and natural images for complete dictionaries we demonstrate that our algorithms have improved performance over other independent component analysis ica methods measured in terms of signal to noise ratios of separated sources in the overcomplete case we show that the true underlying dictionary and sparse sources can be accurately recovered in tests with natural images learned overcomplete dictionaries are shown to have higher coding efficiency than complete dictionaries that is images encoded with an overcomplete dictionary have both higher compression fewer bits per pixel and higher accuracy lower mean square error

measurement error regression with unknown link dimension reduction and data visualization abstract a general nonlinear regression problem is considered with measurement error in the predictors we assume that the response is related to an unknown linear combination of a multidimensional predictor through an unknown link function instead of observing the predictor we instead observe a surrogate with the property that its expectation is linearly related to the true predictor with constant variance we identify an important transformation of the surrogate variable using this transformed variable we show that if one proceeds with the usual analysis ignoring measurement error then both ordinary least squares and sliced inverse regression yield estimates which consistently estimate the true regression parameter up to a constant of proportionality we derive the asymptotic distribution of the estimates a simulation study is conducted applying sliced inverse regression in this context

wavelet based image denoising using a markov random field a priori model this paper describes a new method for the suppression of noise in images via the wavelet transform the method relies on two measures the first is a classic measure of smoothness of the image and is based on an approximation of the local holder exponent via the wavelet coefficients the second novel measure takes into account geometrical constraints which are generally valid for natural images the smoothness measure and the constraints are combined in a bayesian probabilistic formulation and are implemented as a markov random field mrf image model the manipulation of the wavelet coefficients is consequently based on the obtained probabilities a comparison of quantitative and qualitative results for test images demonstrates the improved noise suppression performance with respect to previous wavelet based image denoising methods

on principal hessian directions for data visualization and dimension reduction another application of stein s lemma abstract modern graphical tools have enhanced our ability to learn many things from data directly with much user friendly graphical software available we are encouraged to plot a lot more often than before the benefits from direct interaction with graphics have been enormous but trailing behind these high tech advances is the issue of appropriate guidance on what to plot there are too many directions to project a high dimensional data set and unguided plotting can be time consuming and fruitless in a recent article li set up a statistical framework for study on this issue based on a notion of effective dimension reduction edr directions they are the directions to project a high dimensional input variable for the purpose of effectively viewing and studying its relationship with an output variable a methodology sliced inverse regression was introduced and shown to be useful in finding edr directions this article introduces another method for finding edr directions it begins with the observat

sparse non gaussian component analysis non gaussian component analysis ngca introduced in offered a method for high dimensional data analysis allowing for identifying a low dimensional non gaussian component of the whole distribution in an iterative and structure adaptive way an important step of the ngca procedure is identification of the non gaussian subspace using principle component analysis pca method this article proposes a new approach to ngca called sparse ngca which replaces the pca based procedure with a new the algorithm we refer to as convex projection