nonparametric density estimation and detection in impulsive interference channels i estimators for pt i see ibid vol 42 no 2 4 p 1684 1697 1994 nonlinear processing significantly enhances detector performance in nongaussian noise relative to that of linear detectors several nonparametric detection schemes for impulsive noise channels are formulated using the nonparametric probability density estimators developed in part i the likelihood ratio test and the small signal locally optimum nonlinearity provide the basis for the formulation of these nonparametric detection schemes several modifications to these basic strategies are used to compensate for inaccuracies in the density estimates in particular for the problem of detecting a known signal in impulsive noise two modifications to the standard likelihood ratio test are considered the first is adapted from robust statistics whereas the second the l sub 1 error based detector is specifically formulated for use with density estimates both schemes are found to perform close to the optimum likelihood ratio detector for a wide variety of impulsive noise densities from the merits of these two tests a new detection scheme that approximates the locally optimum nonlinearity is then developed this detector which uses the nonparametric density estimators developed in part i is shown to perform very well for the wide variety of impulsive and heavy tailed densities considered in the study this nonparametric density estimate based detector is also shown to outperform more conventional nonparametric detectors in impulsive noise

a bayesian modeling approach for generalized semiparametric structural equation models in behavioral biomedical and psychological studies structural equation models sems have been widely used for assessing relationships between latent variables regression type structural models based on parametric functions are often used for such purposes in many applications however parametric sems are not adequate to capture subtle patterns in the functions over the entire range of the predictor variable a different but equally important limitation of traditional parametric sems is that they are not designed to handle mixed data types continuous count ordered and unordered categorical this paper develops a generalized semiparametric sem that is able to handle mixed data types and to simultaneously model different functional relationships among latent variables a structural equation of the proposed sem is formulated using a series of unspecified smooth functions the bayesian p splines approach and markov chain monte carlo methods are developed to estimate the smooth functions and the unknown parameters moreover we examine the relative benefits of semiparametric modeling over parametric modeling using a bayesian model comparison statistic called the complete deviance information criterion dic the performance of the developed methodology is evaluated using a simulation study to illustrate the method we used a data set derived from the national longitudinal survey of youth

feature extraction by non parametric mutual information maximization we present a method for learning discriminative feature transforms using as criterion the mutual information between class labels and transformed features instead of a commonly used mutual information measure based on kullback leibler divergence we use a quadratic divergence measure which allows us to make an efficient non parametric implementation and requires no prior assumptions about class densities in addition to linear transforms we also discuss nonlinear transforms that are implemented as radial basis function networks extensions to reduce the computational complexity are also presented and a comparison to greedy feature selection is made

model comparison of nonlinear structural equation models with fixed covariates recently it has been recognized that the commonly used linear structural equation model is inadequate to deal with some complicated substantive theory a new nonlinear structural equation model with fixed covariates is proposed in this article a procedure which utilizes the powerful path sampling for computing the bayes factor is developed for model comparison in the implementation the required random observations are simulated via a hybrid algorithm that combines the gibbs sampler and the metropolis hastings algorithm it is shown that the proposed procedure is efficient and flexible and it produces bayesian estimates of the parameters latent variables and their highest posterior density intervals as by products empirical performances of the proposed procedure such as sensitivity to prior inputs are illustrated by a simulation study and a real example

on estimating a density using hellinger distance and some other strange facts on s interesse ici aux possibles vitesses d estimation d une densite a support compact dans m sous des hypotheses de regularite lorsque la perte est mesuree par le carre de la distance de hellinger on regardera aussi le cas connu des normes mathbb l q pour 1 q 2 et le risque est le risque minimax sur la famille on donne une methode generale permettant de traiter les problemes dans le cadre de la theorie de l approximation sous des conditions concernant l entropie metrique et l e capacite des familles a estimer les rapports entre regularite et entropie metrique etant bien connus nous pourrons aussi traiter les cas classiques et d autres qui le sont moins sous des conditions de bornes inferieures les vitesses sont celles observees pour la norme mathbb l q mais elles different dans le cas general on montre aussi que les restrictions sur la compacite du support ou la regularite sont indispensables et que leur absence mene a l impossibilite d obtenir une estimation raisonnable en ce sens que n importe quelle suite d estimateurs sera arbitrairement mauvaise en un point au moins un resultat analogue est vrai sous des conditions de regularite

nonexistence of positive weak solutions of elliptic inequalities abstract in this paper we give sufficient conditions for the nonexistence of positive entire weak solutions of coercive and anticoercive elliptic inequalities both of the p laplacian and of the mean curvature type depending also on u and x inside the divergence term while a gradient factor is included on the right hand side in particular to prove our theorems we use a technique developed by mitidieri and pohozaev in e mitidieri s i pohozaev a priori estimates and the absence of solutions of nonlinear partial differential equations and inequalities proc steklov inst math 234 2001 1 362 which relies on the method of test functions without using comparison and maximum principles their approach is essentially based first on a priori estimates and on the derivation of an asymptotics for the a priori estimate finally nonexistence of a solution is proved by contradiction

on entire solutions of degenerate elliptic differential inequalities with nonlinear gradient terms in this paper we give sufficient conditions for the nonexistence of nonnegative nontrivial entire weak solutions of p laplacian elliptic inequalities with possibly singular weights and gradient terms of the form divfg jxj jduj p 2 dug h jxj f u jxj l jduj under the main request

nonlinear elliptic equations with singular boundary conditions and stochastic control with state constraints hamiltoniens sousquadratiques conditions aux limites infinies et donnees d explosion hamiltoniens superquadratiques formulation viscosite des conditions aux limites le probleme ergodique commande stochastique optimale avec contraintes d etat

a note on the strong maximum principle and the compact support principle abstract in this note we are concerned with the strong maximum principle smp and the compact support principle csp for non negative solutions to quasilinear elliptic inequalities of the form div a u u g u f u 0 in and div a u u g u f u 0 in r n b r 0 respectively we give new conditions on the data a g f to obtain smp and csp when these conditions are particularized to the m laplacian and pure power nonlinearities we completely classify the data according to the validity of the csp or the smp in doing so we clarify the general situation and we consider a case not covered in the literature

on the inequality f x d 2u ge f u g u du q we consider fully nonlinear degenerate elliptic equations with zero and first order terms we provide a priori upper bounds and characterize the existence of entire subsolutions under growth conditions on the lower order coefficients which extend the classical keller osserman condition for semilinear equations

the consistency of automatic kernel density estimates we consider the parzen rosenblatt kernel density estimate on rd with data dependent smoothing factor sufficient conditions on the asymptotic behavior of the smoothing factor are given under which the estimate is pointwise consistent almost everywhere for all densities f to be estimated when the smoothing factor is a function only of the sample size n it is shown that these conditions are also necessary a generalization of results by deheuvels the consistency of various automatic kernel density estimates is a simple consequence of these theorems

no empirical probability measure can converge in the total variation sense for all distributions for any sequence of empirical probability measures n on the borel sets of the real line and any 0 there exists a singular continuous probability measure such that inf sup n 9m n a a a 1 2 almost surely

consensus in ad hoc wsns with noisy links part ii distributed estimation and smoothing of random signals distributed algorithms are developed for optimal estimation of stationary random signals and smoothing of even nonstationary dynamical processes based on generally correlated observations collected by ad hoc wireless sensor networks wsns maximum a posteriori map and linear minimum mean square error lmmse schemes well appreciated for centralized estimation are shown possible to reformulate for distributed operation through the iterative alternating direction method of multipliers sensors communicate with single hop neighbors their individual estimates as well as multipliers measuring how far local estimates are from consensus when iterations reach consensus the resultant distributed d map and lmmse estimators converge to their centralized counterparts when inter sensor communication links are ideal the d map estimators do not require the desired estimator to be expressible in closed form the d lmmse ones are provably robust to communication or quantization noise and both are particularly simple to implement when the data model is linear gaussian for decentralized tracking applications distributed kalman filtering and smoothing algorithms are derived for any time mmse optimal consensus based state estimation using wsns analysis and corroborating numerical examples demonstrate the merits of the novel distributed estimators

lower bounds for the rate of convergence in nonparametric pattern recognition we show that there exist individual lower bounds corresponding to the upper bounds for the rate of convergence of nonparametric pattern recognition which are arbitrarily close to yang s minimax lower bounds for certain cubic classes of regression functions used by stone and others the rates are equal to the ones of the corresponding regression function estimation problem thus for these classes classification is not easier than regression function estimation

complexity reduced scheme for feature extraction with linear discriminant analysis owing to the singularity of the within class scatter linear discriminant analysis lda becomes ill posed for small sample size sss problems null space based lda nlda which is an extension of lda provides good discriminant performances for sss problems yet as the original scheme for the feature extractor fe of nlda suffers from a complexity burden a few modified schemes have since been proposed for complexity reduction in this brief by transforming the problem of finding the fe of nlda into a linear equation problem a novel scheme is derived offering a further reduction of the complexity

entire solutions of fully nonlinear elliptic equations with a superlinear gradient term in this paper we consider second order fully nonlinear operators with an additive superlinear gradient term like in the pioneering paper of brezis for the semilinear case we obtain the existence of entire viscosity solutions defined in all the space without assuming global bounds a uniqueness result is also obtained for special gradient terms subject to a convexity concavity type assumption where superlinearity is essential and has to be handled in a different way from the linear case

on the inequality of cover and hart in nearest neighbor discrimination when x1 1 xn n are independent identically distributed random vectors from ird x 0 1 distributed as x and when is estimated by its nearest neighbor estimate 1 then cover and hart have shown that p 1 n 2e x 1 x 2r 1 r where r is the bayes probability of error and x p 1 x x they have conditions on the distribution of x we give two proofs one due to stone and a short original one of the same result for all distributions of x if ties are carefully taken care of we also show that p 1 x1 1 xn n converges in probability to a constant for all distributions of x thereby strengthening results of wagner and fritz

online sparse system identification and signal reconstruction using projections onto weighted ell_ 1 balls this paper presents a novel projection based adaptive algorithm for sparse signal and system identification the sequentially observed data are used to generate an equivalent sequence of closed convex sets namely hyperslabs each hyperslab is the geometric equivalent of a cost criterion that quantifies data mismatch sparsity is imposed by the introduction of appropriately designed weighted l1 balls and the related projection operator is also derived the algorithm develops around projections onto the sequence of the generated hyperslabs as well as the weighted l1 balls the resulting scheme exhibits linear dependence with respect to the unknown system s order on the number of multiplications additions and an o llog2l dependence on sorting operations where l is the length of the system signal to be estimated numerical results are also given to validate the performance of the proposed method against the least absolute shrinkage and selection operator lasso algorithm and two very recently developed adaptive sparse schemes that fuse arguments from the lms rls adaptation mechanisms with those imposed by the lasso rational

an optimal global nearest neighbor metric a quadratic metric dao x y x y t ao x y is proposed which minimizes the mean squared error between the nearest neighbor asymptotic risk and the finite sample risk under linearity assumptions a heuristic argument is given which indicates that this metric produces lower mean squared error than the euclidean metric a nonparametric estimate of ao is developed if samples appear to come from a gaussian mixture an alternative parametrically directed distance measure is suggested for nearness decisions within a limited region of space examples of some two class gaussian mixture distributions are included

entire solutions of completely coercive quasilinear elliptic equations ii abstract a famous theorem of sergei bernstein says that every entire solution u u x x r 2 of the minimal surface equation div d u 1 d u 2 0 is an affine function no conditions being placed on the behavior of the solution u bernstein s theorem continues to hold up to dimension n 7 while it fails to be true in higher dimensions in fact if x r n with n 8 there exist entire non affine minimal graphs bombieri de giorgi and giusti our purpose is to consider an extensive family of quasilinear elliptic type equations which has the following strong bernstein liouville property that u 0 for any entire solution u no conditions whatsoever being placed on the behavior of the solution outside of appropriate regularity assumptions in many cases moreover no conditions need be placed even on the dimension n we also study the behavior of solutions when the parameters of the problem do not allow the bernstein liouville property and give a number of counterexamples showing that the results of the paper are in many cases best possible

the optimal distance measure for nearest neighbor classification a local distance measure is shown to optimize the performance of the nearest neighbor two class classifier for a finite number of samples the difference between the finite sample error and the asymptotic error is used as the criterion of improvement this new distance measure is compared to the well known euclidean distance an algorithm for practical implementation is introduced this algorithm is shown to be computationally competitive with the present nearest neighbor procedures and is illustrated experimentally a closed form for the corresponding second order moment of this criterion is found finally the above results are extended to

bandwidth choice for nonparametric classification it is shown that for kernel based classification with univariate distributions and two populations optimal bandwidth choice has a dichotomous character if the two densities cross at just one point where their curvatures have the same signs then minimum bayes risk is achieved using bandwidths which are an order of magnitude larger than those which minimize pointwise estimation error on the other hand if the curvature signs are different or if there are multiple crossing points then bandwidths of conventional size are generally appropriate the range of different modes of behavior is narrower in multivariate settings there the optimal size of bandwidth is generally the same as that which is appropriate for pointwise density estimation these properties motivate empirical rules for bandwidth choice

any discrimination rule can have an arbitrarily bad probability of error for finite sample size consider the basic discrimination problem based on a sample of size n drawn from the distribution of x y on the borel sets of rdx 0 1 if 0 thus any attempt to find a nontrivial distribution free upper bound for rn will fail and any results on the rate of convergence of rn to r must use assumptions about the distribution of x y

adaptive density estimation from data with small measurement errors in this paper we study the problem of density estimation from data that contains small measurement errors the only assumption on these errors is that the maximal measurement error is bounded by some real number converging to zero for sample size tending to infinity in particular we do not assume that the measurement errors are independent with expectation zero we estimate the density by a standard kernel density estimate applied to data with measurement errors and derive a data driven method to choose its bandwidth we derive an adaptation result for this estimate and analyze the expected l 1 error of our density estimate depending on the smoothness of the density and the size of the maximal measurement error the results are applied in a density estimation problem in a simulation model where we show under suitable assumptions that the l 1 error of our newly proposed estimate converges to zero much faster than the l 1 error of the standard kernel density estimate if both are based on the same number of observations in the simulation model the performance of the method in case of finite sample size is analyzed using simulated data

estimation for polynomial structural equation models abstract structural equation analysis is one of the most widely used statistical methods in social and behavioral science research and has become a popular tool in marketing subject matter needs for considering nonlinear structural models have been well documented but current fitting procedures are available only for a limited class of models in this article a systematic statistical approach is developed for the general polynomial structural equation model the new procedure applies a method of moments procedure similar to the one used in errors in variables regression to the factor score estimates from the measurement model fit the asymptotic properties of the estimator are derived and a modified estimator with better small sample properties is introduced simulation studies are reported to show the usefulness of the procedure and to compare its performance to other methods an example from a substance abuse prevention study is also discussed

on the relationship between stability of extreme order statistics and convergence of the maximum likelihood kernel density estimate by michel broniatowski paul deheuvels and luc devroyeiuniversite pierre et marie curie universite pierre et marie curie andmcgill universitylet f be a density on the real line and let f be the kernel estimate of fin which the smoothing factor is obtained by maximizing the cross validatedlikelihood product according to the method of duin and habbema hermansand vandenbroek under mild regularity conditions on the kernel and f weshow among other things that f jf f 0 almost surely if and only if thesample extremes of f are strongly stable

empirically estimable classification bounds based on a nonparametric divergence measure information divergence functions play a critical role in statistics and information theory in this paper we show that a nonparametric f divergence measure can be used to provide improved bounds on the minimum binary classification probability of error for the case when the training and test data are drawn from the same distribution and for the case where there exists some mismatch between training and test distributions we confirm these theoretical results by designing feature selection algorithms using the criteria from these bounds and by evaluating the algorithms on a series of pathological speech classification tasks

geometric linear discriminant analysis for pattern recognition we propose an innovative technique geometric linear discriminant analysis geometric lda to reduce the complexity of pattern recognition systems by using a linear transformation to lower the dimension of the observation space we experimentally compare geometric lda to other dimensionality reduction methods found in the literature we show that geometric lda produces the same and in many cases a significantly better linear transformation than other methods found in the literature

lower bounds for bayes error estimation we give a short proof of the following result let x y be any distribution on n spl times 0 1 and let x sub 1 y sub 1 x sub n y sub n be an i i d sample drawn from this distribution in discrimination the bayes error l inf sub g p g x spl ne y is of crucial importance here we show that without further conditions on the distribution of x y no rate of convergence results can be obtained let spl phi sub n x sub 1 y sub 1 x sub n y sub n be an estimate of the bayes error and let spl phi sub n be a sequence of such estimates for any sequence a sub n of positive numbers converging to zero a distribution of x y may be found such that e l spl phi sub n x sub 1 y sub 1 x sub n y sub n spl ges a sub n often converges infinitely

generalized harnack inequality for nonhomogeneous elliptic equations this paper is concerned with nonlinear elliptic equations in nondivergence form f d 2 u du x 0 where f has a drift term which is not lipschitz continuous under this condition the equations are nonhomogeneous and nonnegative solutions do not satisfy the classical harnack inequality this paper presents a new generalization of the harnack inequality for such equations as a corollary we obtain the optimal harnack type of inequality for p x harmonic functions which quantifies the strong minimum principle

a general structural equation model with dichotomous ordered categorical and continuous latent variable indicators a structural equation model is proposed with a generalized measurement part allowing for dichotomous and ordered categorical variables indicators in addition to continuous ones a computationally feasible three stage estimator is proposed for any combination of observed variable types this approach provides large sample chi square tests of fit and standard errors of estimates for situations not previously covered two multiple indicator modeling examples are given one is a simultaneous analysis of two groups with a structural equation model underlying skewed likert variables the second is a longitudinal model with a structural model for multivariate probit regressions

a unifying view of wiener and volterra theory and polynomial kernel regression volterra and wiener series are perhaps the best understood nonlinear system representations in signal processing although both approaches have enjoyed a certain popularity in the past their application has been limited to rather low dimensional and weakly nonlinear systems due to the exponential growth of the number of terms that have to be estimated we show that volterra and wiener series can be represented implicitly as elements of a reproducing kernel hilbert space by using polynomial kernels the estimation complexity of the implicit representation is linear in the input dimensionality and independent of the degree of nonlinearity experiments show performance advantages in terms of convergence interpretability and system sizes that can be handled

classification error for a very large number of classes classification error is analyzed for a situation where the number of possible classes may be on the order of a hundred or more the error associated with classifying to a single class is shown to depend mainly on average nearest neighbor distance between class means noise level and effective dimensionality of the class mean distribution and not much on other aspects of the distribution noise correlation or number of classes since single class error is large separation of classes into groups is also explored group classification error has the same properties as single class error but the size of the error is moderated by the bayes overlap between groups standard curves are provided to predict single class and group error also discussed are the effect of pattern blurring on classification error and the nearest neighbor distance statistics throughout a distribution

bayes error estimation using parzen and k nn procedures the use of k nearest neighbor k nn and parzen density estimates to obtain estimates of the bayes error is investigated under limited design set conditions by drawing analogies between the k nn and parzen procedures new procedures are suggested and experimental results are given which indicate that these procedures yield a significant improvement over the conventional k nn and parzen procedures we show that by varying the decision threshold many of the biases associated with the k nn or parzen density estimates may be compensated and successful error estimation may be performed in spite of these biases experimental results are given which demonstrate the effect of kernel size and shape parzen the size of k k nn and the number of samples in the design set

keller osserman type conditions for some elliptic problems with gradient terms in this paper we consider the elliptic boundary blow up problems u g u f u in u on where is a smooth bounded domain and the functions f and g are increasing and continuous our main concern will be to prove both existence and nonexistence of nonnegative solutions depending on new integral conditions of keller osserman type involving f and g we show in particular that the problem with a minus sign may have solutions inclusive for some functions g with slightly superquadratic growth at infinity that is somehow not expected we also obtain uniqueness of nonnegative solutions in some cases

estimation of a regression function by maxima of minima of linear functions in this paper estimation of a regression function from independent and identically distributed random variables is considered estimates are defined by minimization of the empirical l2 risk over a class of functions which are defined as maxima of minima of linear functions results concerning the rate of convergence of the estimates are derived in particular it is shown that for smooth regression functions satisfying the assumption of single index models the estimate is able to achieve up to some logarithmic factor the corresponding optimal one dimensional rate of convergence hence under these assumptions the estimate is able to circumvent the so called curse of dimensionality the small sample behavior of the estimates is illustrated by applying them to simulated data

classification of radial solutions for semilinear elliptic systems with nonlinear gradient terms abstract we are concerned with the classification of positive radial solutions for the system u v p v f u where p 0 and f c 1 0 is a nondecreasing function such that f t 0 for all t 0 we show that in the case where the system is posed in the whole space r n such solutions exist if and only if 1 0 s f t d t p 2 p 1 d s this is the counterpart of the keller osserman condition for the case of single semilinear equation similar optimal conditions are derived in case where the system is posed in a ball of r n if f t t q q 1 using dynamical system techniques we are able to describe the behaviour of solutions at infinity in case where the system is posed in the whole r n or around the boundary in case of a ball

nonparametric estimation via empirical risk minimization a general notion of universal consistency of nonparametric estimators is introduced that applies to regression estimation conditional median estimation curve fitting pattern recognition and learning concepts general methods for proving consistency of estimators based on minimizing the empirical error are shown in particular distribution free almost sure consistency of neural network estimates and generalized linear estimators is established

properties of bagged nearest neighbour classifiers summary it is shown that bagging a computationally intensive method asymptotically improves the performance of nearest neighbour classifiers provided that the resample size is less than 69 of the actual sample size in the case of with replacement bagging or less than 50 of the sample size for without replacement bagging however for larger sampling fractions there is no asymptotic difference between the risk of the regular nearest neighbour classifier and its bagged version in particular neither achieves the large sample performance of the bayes classifier in contrast when the sampling fractions converge to 0 but the resample sizes diverge to 1 the bagged classifier converges to the optimal bayes rule and its risk converges to the risk of the latter these results are most readily seen when the two populations have well defined densities but they may also be derived in other cases where densities exist in only a relative sense cross validation can be used effectively to choose the sampling fraction numerical calculation is used to illustrate these theoretical properties

nonparametric density estimation for classes of positive random variables a kernel based density estimator for positive random variables is proposed and analyzed in particular a nonparametric estimator is developed which takes advantage of the fact that positive random variables can be represented as the norms of random vectors by appropriately choosing the dimension of the assumed vector space the estimator can be structured to exploit a priori knowledge about the density to be estimated the asymptotic properties e g pointwise and l sub 1 consistency of this density estimator are investigated and found to be similar to the desirable features of the standard kernel estimator an upper bound on the expected value of the l sub 1 error is also derived which provides insight into the behavior of the estimator upon using this upper bound the optimal form for the estimator i e the kernel function the smoothing factor etc is selected via a minimax strategy in addition this upper bound is used to compare the asymptotic performance of the proposed estimator to that of the standard kernel estimator and to boundary corrected kernel estimators numerical examples illustrate that the proposed scheme outperforms the standard and boundary corrected estimators for a variety of density types

nonparametric estimation of conditional distributions estimation of conditional distributions is considered it is assumed that the conditional distribution is either discrete or that it has a density with respect to the lebesgue measure partitioning estimates of the conditional distribution are constructed and results concerning consistency and rate of convergence of the integrated total variation error of the estimates are presented

solvability of nonlinear elliptic equations with gradient terms abstract we study the solvability in the whole euclidean space of coercive quasi linear and fully nonlinear elliptic equations modeled on u g u f u u 0 where f and g are increasing continuous functions we give conditions on f and g which guarantee the availability or the absence of positive solutions of such equations in r n our results considerably improve the existing ones and are sharp or close to sharp in the model cases in particular we completely characterize the solvability of such equations when f and g have power growth at infinity we also derive a solvability statement for coercive equations in general form

kernel based structural equation models for topology identification of directed networks structural equation models sems have been widely adopted for inference of causal interactions in complex networks recent examples include unveiling topologies of hidden causal networks over which processes such as spreading diseases or rumors propagate the appeal of sems in these settings stems from their simplicity and tractability since they typically assume linear dependencies among observable variables acknowledging the limitations inherent to adopting linear models the present paper put forth nonlinear sems which account for possible nonlinear dependencies among network nodes the advocated approach leverages kernels as a powerful encompassing framework for nonlinear modeling and an efficient estimator with affordable tradeoffs is put forth interestingly pursuit of the novel kernel based approach yields a convex regularized estimator that promotes edge sparsity a property exhibited by most real world networks and the resulting optimization problem is amenable to proximal splitting optimization methods to this end solvers with complementary merits are developed by leveraging the alternating direction method of multipliers and proximal gradient iterations experiments conducted on simulated data demonstrate that the novel approach outperforms linear sems with respect to edge detection errors furthermore tests on a real gene expression dataset unveil interesting new edges that were not revealed by linear sems which could shed more light on regulatory behavior of human genes

optimal reference subset selection for nearest neighbor classification by tabu search this paper presents an approach to select the optimal reference subset ors for nearest neighbor classifier the optimal reference subset which has minimum sample size and satisfies a certain resubstitution error rate threshold is obtained through a tabu search ts algorithm when the error rate threshold is set to zero the algorithm obtains a near minimal consistent subset of a given training set while the threshold is set to a small appropriate value the obtained reference subset may have reasonably good generalization capacity a neighborhood exploration method and an aspiration criterion are proposed to improve the efficiency of ts experimental results based on a number of typical data sets are presented and analyzed to illustrate the benefits of the proposed method the performances of the result consistent and non consistent reference subsets are evaluated

bias of nearest neighbor error estimates the bias of the finite sample nearest neighbor nn error from its asymptotic value is examined expressions are obtained which relate the bias of the nn and 2 nn errors to sample size dimensionality metric and distributions these expressions isolate the effect of sample size from that of the distributions giving an explicit relation showing how the bias changes as the sample size is increased experimental results are given which suggest that the expressions accurately predict the bias it is shown that when the dimensionality of the data is high it may not be possible to estimate the asymptotic error simply by increasing the sample size a new procedure is suggested to alleviate this problem this procedure involves measuring the mean nn errors at several sample sizes and using our derived relationship between the bias and the sample size to extrapolate an estimate of the asymptotic nn error the results are extended to the multiclass problem the choice of an optimal metric to minimize the bias is also discussed

a note on robust hypothesis testing we introduce a simple new hypothesis testing procedure which based on an independent sample drawn from a certain density detects which of k nominal densities is the true density closest to under the total variation l sub 1 distance we obtain a density free uniform exponential bound for the probability of false detection

bounds on the bayes error given moments we show how to compute lower bounds for the supremum bayes error if the class conditional distributions must satisfy moment constraints where the supremum is with respect to the unknown class conditional distributions our approach makes use of curto and fialkow s solutions for the truncated moment problem the lower bound shows that the popular gaussian assumption is not robust in this regard we also construct an upper bound for the supremum bayes error by constraining the decision boundary to be linear

on arbitrarily slow rates of global convergence in density estimation let a density f on rd be estimated by fn x x1 xn where x rd fn is a borel measurable function of its arguments and x1 xn are independent random vectors with common density f let p 1 be a constant one of the main results of this note is that for every sequence fn and for every positive number sequence an satisfying lim an 0 there exists an f such that e left smallint f n left x right f left x right p dx right a n infinitely often

a bootstrap technique for nearest neighbor classifier design a bootstrap technique for nearest neighbor classifier design is proposed our primary interest in designing a classifier is in small training sample size situations conventional bootstrapping techniques sample the training samples with replacement on the other hand our technique generates bootstrap samples by locally combining original training samples the nearest neighbor classifier is designed on the bootstrap samples and is tested on the test samples independent of training samples the performance of the proposed classifier is demonstrated on three artificial data sets and one real data set experimental results show that the nearest neighbor classifier designed on the bootstrap samples outperforms the conventional k nn classifiers as well as the edited 1 nn classifiers particularly in high dimensions

pattern classification using projection pursuit abstract this article discusses the adaptation of recently developed regression techniques to classifier design apart from finite sample effects projection pursuit pp regression may be used to model a desired response class as a sum of ridge functions according to a minimum expected squared error criterion this approach can be shown to furnish an optimal discriminant function which can satisfy the neyman pearson criterion over all possible thresholds basis function expansions are used instead of smoothed histograms to reduce computation since good approximation of a discriminant by a linear combination of moderate number of ridge functions may not be easy we introduce an improved method utilizing a nonlinear weighting function

the rate of convergence of k_n nn regression estimates and classification rules corresp the rate of convergence of k n nn regression estimates and the corresponding multiple classification error are calculated without assuming the existence of the density of the observations

lower bounds in pattern recognition and learning lower bounds are derived for the performance of any pattern recognition algorithm which using training data selects a discrimination rule from a certain class of rules the bounds involve the vapnik chervonenkis dimension of the class and l the minimal error probability within the class we provide lower bounds when l 0 the usual assumption in valiant s theory of learning and l 0

global convergence of the recursive kernel regression estimates with applications in classification and nonlinear system estimation an improved exponential bound on the l sub 1 error for the recursive kernel regression estimates is derived it is shown using the martingale device that weak strong and complete l sub 1 consistencies are equivalent consequently the conditions on a certain smoothing sequence are necessary and sufficient for strong l sub 1 consistency of the recursive kernel regression estimate the rates of global convergence are also given obtained results are applied to recursive classification rules and to nonlinear time series estimation

tensor decompositions for identifying directed graph topologies and tracking dynamic networks directed networks are pervasive both in nature and engineered systems often underlying the complex behavior observed in biological systems microblogs and social interactions over the web as well as global financial markets since their structures are often unobservable in order to facilitate network analytics one generally resorts to approaches capitalizing on measurable nodal processes to infer the unknown topology structural equation models sems are capable of incorporating exogenous inputs to resolve inherent directional ambiguities however conventional sems assume full knowledge of exogenous inputs which may not be readily available in some practical settings this paper advocates a novel sem based topology inference approach that entails factorization of a three way tensor constructed from the observed nodal data using the well known parallel factor parafac decomposition it turns out that second order piecewise stationary statistics of exogenous variables suffice to identify the hidden topology capitalizing on the uniqueness properties inherent to high order tensor factorizations it is shown that topology identification is possible under reasonably mild conditions in addition to facilitate real time operation and inference of time varying networks an adaptive parafac tensor decomposition scheme that tracks the topology revealing tensor factors is developed extensive tests on simulated and real stock quote data demonstrate the merits of the novel tensor based approach

consensus in ad hoc wsns with noisy links part i distributed estimation of deterministic signals we deal with distributed estimation of deterministic vector parameters using ad hoc wireless sensor networks wsns we cast the decentralized estimation problem as the solution of multiple constrained convex optimization subproblems using the method of multipliers in conjunction with a block coordinate descent approach we demonstrate how the resultant algorithm can be decomposed into a set of simpler tasks suitable for distributed implementation different from existing alternatives our approach does not require the centralized estimator to be expressible in a separable closed form in terms of averages thus allowing for decentralized computation even of nonlinear estimators including maximum likelihood estimators mle in nonlinear and non gaussian data models we prove that these algorithms have guaranteed convergence to the desired estimator when the sensor links are assumed ideal furthermore our decentralized algorithms exhibit resilience in the presence of receiver and or quantization noise in particular we introduce a decentralized scheme for least squares and best linear unbiased estimation blue and establish its convergence in the presence of communication noise our algorithms also exhibit potential for higher convergence rate with respect to existing schemes corroborating simulations demonstrate the merits of the novel distributed estimation algorithms

a no panacea theorem for classifier combination we introduce the no panacea theorem npt for multiple classifier combination previously proved only in the case of two classifiers and two classes in this paper we extend the npt to cases of multiple classifiers and multiple classes we prove that if the combination function is continuous and diverse there exists a situation in which the combination algorithm will give very bad performance the proof relies on constructing pathological probability density distributions that have high densities in particular areas such that the combination functions give incorrect classification thus there is no optimal combination algorithm that is suitable in all situations it can be seen from this theorem that the probability density functions pdfs play an important role in the performance of combination algorithms so studying the pdfs becomes the first step of finding a good combination algorithm although devised for classifier combination the npt is also relevant to all supervised classification problems