concentration inequalities for the missing mass and for histogram rule error this paper gives distribution free concentration inequalities for the missing mass and the error rate of histogram rules negative association methods can be used to reduce these concentration problems to concentration questions about independent sums although the sums are independent they are highly heterogeneous such highly heterogeneous independent sums cannot be analyzed using standard concentration inequalities such as hoeffding s inequality the angluin valiant bound bernstein s inequality bennett s inequality or mcdiarmid s theorem the concentration inequality for histogram rule error is motivated by the desire to construct a new class of bounds on the generalization error of decision trees

probability inequalities for empirical processes and a law of the iterated logarithm given a class f of functions on x we can view vn as a stochastic process indexed by f and consider limit theorems for this process to prove such theorems it is often helpful to have bounds on the tail of the r v sup w i n f i see for example dudley 1978 kuelbs and dudley 1980 dudley and phillip 1983 or gine and zinn 1983 our main question of interest is for what classes f can best possible bounds be obtained so first we must ask what are these best possible bounds of course

sample spacings based density and entropy estimators for spherically invariant multidimensional data while the sample spacings based density estimation method is simple and efficient its applicability has been restricted to one dimensional data in this letter the method is generalized such that it can be extended to multiple dimensions in certain circumstances as a consequence a multidimensional entropy estimator of spherically invariant continuous random variables is derived partial bias of the estimator is analyzed and the estimator is further used to derive a nonparametric objective function for frequency domain independent component analysis the robustness and the effectiveness of the objective function are demonstrated with simulation results

an optimal global nearest neighbor metric a quadratic metric dao x y x y t ao x y is proposed which minimizes the mean squared error between the nearest neighbor asymptotic risk and the finite sample risk under linearity assumptions a heuristic argument is given which indicates that this metric produces lower mean squared error than the euclidean metric a nonparametric estimate of ao is developed if samples appear to come from a gaussian mixture an alternative parametrically directed distance measure is suggested for nearness decisions within a limited region of space examples of some two class gaussian mixture distributions are included

three dimensional unstructured mesh generation part 2 surface meshes abstract this paper deals with surface patches and surface meshing triangular and quadrilateral patches in linear and quadratic forms and non uniform rational b spline nurbs patch have been used to define surface geometry from the real application of view data conversion between the mesh generator and some existing cad packages has been considered as a result of this study converters from a graphics data standard and several cad data formats have been implemented the quality improvement of surface meshes has been discussed in terms of parametric plane stretching diagonal swapping and smoothing procedures furthermore a scheme of visual representation is introduced to utilize colour effect in validating the geometry and its surface meshes

termination and continuity of greedy growing for tree structured vector quantizers tree structured vector quantizers tsvq provide a computationally efficient variable rate method of compressing vector valued data in applications the problem of designing a tsvq from empirical training data is critical greedy growing algorithms are a common and effective approach to the design problem they are recursive procedures that produce a tsvq one node at a time by optimizing a simple splitting criterion at each step while unsupervised greedy growing algorithms are well understood from an experimental point of view there has been little theory to support their use or to examine their behavior on large training sets the authors present a rigorous analysis of a greedy growing algorithm proposed by riskin 1990 riskin and gray 1991 and balakrishnan 1991 the first part of the paper is a description of the algorithm and an examination of its asymptotic behavior as it applies to a fixed absolutely continuous distribution the second part of the paper establishes the structural consistency of the algorithm with respect to a convergent sequence of distributions as an application the authors obtain results concerning the large sample empirical behavior of the algorithm when it is applied to stationary ergodic training vectors

convergence properties of a class of learning vector quantization algorithms a mathematical analysis of a class of learning vector quantization lvq algorithms is presented using an appropriate time coordinate transformation we show that the lvq algorithms under consideration can be transformed into linear time varying stochastic difference equations using this fact we apply stochastic lyapunov stability arguments and we prove that the lvq algorithms under consideration do indeed converge provided that some appropriate conditions hold

combining image compression and classification using vector quantization we describe a method of combining classification and compression into a single vector quantizer by incorporating a bayes risk term into the distortion measure used in the quantizer design algorithm once trained the quantizer can operate to minimize the bayes risk weighted distortion measure if there is a model providing the required posterior probabilities or it can operate in a suboptimal fashion by minimizing the squared error only comparisons are made with other vector quantizer based classifiers including the independent design of quantization and minimum bayes risk classification and kohonen s lvq a variety of examples demonstrate that the proposed method can provide classification ability close to or superior to learning vq while simultaneously providing superior compression performance

consistency of data driven histogram methods for density estimation and classification we present general sufficient conditions for the almost surel1 consistency of histogram density estimates based on data dependent partitions analogous conditions guarantee the almost sure risk consistency of histogram classification schemes based on data dependent partitions multivariate data is considered throughout in each case the desired consistency requires shrinking cells subexponential growth of a combinatorial complexity measure and sub linear growth of the number of cells it is not required that the cells of every partition be rectangles with sides paralles to the coordinate axis or that each cell contain a minimum number of points no assumptions are made concerning the common distribution of the training vectors we apply the results to establish the consistency of several known partitioning estimates including the kn spacing density estimate classifiers based on statistically equivalent blocks and classifiers based on multivariate clustering schemes

minimax nonparametric classification i rates of convergence this paper studies minimax aspects of nonparametric classification we first study minimax estimation of the conditional probability of a class label given the feature variable this function say f is assumed to be in a general nonparametric class we show the minimax rate of convergence under square l sub 2 loss is determined by the massiveness of the class as measured by metric entropy the second part of the paper studies minimax classification the loss of interest is the difference between the probability of misclassification of a classifier and that of the bayes decision as is well known an upper bound on risk for estimating f gives an upper bound on the risk for classification but the rate is known to be suboptimal for the class of monotone functions this suggests that one does not have to estimate f well in order to classify well however we show that the two problems are in fact of the same difficulty in terms of rates of convergence under a sufficient condition which is satisfied by many function classes including besov sobolev lipschitz and bounded variation this is somewhat surprising in view of a result of devroye gorfi and lugosi see a probabilistic theory of pattern recognition new york springer verlag 1996

modelling the folding of paper into three dimensions using affine transformations abstract we model the folding of ordinary paper via piecewise isometries r 2 r 3 the collection of crease lines and vertices in the unfolded paper is called the crease pattern our results generalize the previously known necessity conditions from the more restrictive case of folding paper flat into r 2 if the crease pattern is foldable then the product in a non intuitive order of the associated rotational matrices is the identity matrix this condition holds locally in a multiple vertex crease pattern and can be adapted to a global condition sufficiency conditions are significantly harder and are not known except in the two dimensional single vertex case

the parameterization method in optimal control problems and differential algebraic equations the paper is devoted to the explanation of the numerical parameterization method pm for optimal control oc problems with intermediate phase constraint and to its circumstantiation for classical calculus of variation cv problems that arise in connection with singular odes or daes especially in cases of their essential degeneracy the pm is based on a finite parameterization of control functions and on derivation of the problem functional with respect to control parameters the first and the second derivatives are calculated with the help of adjoint vector and matrix impulses results of the solution to one phase constrained oc and two degenerate cv problems connected with singular daes nonreducible to the normal form are presented

nonparametric density estimation the l 1 view differentiation of integrals consistency lower bounds for rates of convergence rates of convergence in l1 the automatic kernel estimate l1 and pointwise convergence estimates related to the kernel estimate and the histogram estimate simulation inequalities and random variate generation the transformed kernel estimate applications in discrimination operations on density estimates estimators based on orthogonal series index

divergence estimation of continuous distributions based on data dependent partitions we present a universal estimator of the divergence d p spl par q for two arbitrary continuous distributions p and q satisfying certain regularity conditions this algorithm which observes independent and identically distributed i i d samples from both p and q is based on the estimation of the radon nikodym derivative dp dq via a data dependent partition of the observation space strong convergence of this estimator is proved with an empirically equivalent segmentation of the space this basic estimator is further improved by adaptive partitioning schemes and by bias correction the application of the algorithms to data with memory is also investigated in the simulations we compare our estimators with the direct plug in estimator and estimators based on other partitioning approaches experimental results show that our methods achieve the best convergence performance in most of the tested cases

vanishing distortion and shrinking cells we establish an asymptotic connection between vanishing rth power distortion and shrinking cell diameters for vector quantizers with convex cells as a consequence a number of shrinking cell conditions may be easily verified by showing that the quantizers in question have distortion tending to zero this also plays an important role in the asymptotic analysis of a common greedy growing scheme for tree structured vector quantizers

the optimal strategy of a tree classifier abstract this paper deals with the decision rules of a tree classifier for performing the classification at each nonterminal node under the assumption of complete probabilistic information for given tree structure and feature subsets to be used the optimal decision rules strategy are derived which minimize the overall probability of misclassification the primary result is illustrated by an example

multiscale poisson intensity and density estimation the nonparametric poisson intensity and density estimation methods studied in this paper offer near minimax convergence rates for broad classes of densities and intensities with arbitrary levels of smoothness the methods and theory presented here share many of the desirable features associated with wavelet based estimators computational speed spatial adaptivity and the capability of detecting discontinuities and singularities with high resolution unlike traditional wavelet based approaches which impose an upper bound on the degree of smoothness to which they can adapt the estimators studied here guarantee nonnegativity and do not require any a priori knowledge of the underlying signal s smoothness to guarantee near optimal performance at the heart of these methods lie multiscale decompositions based on free knot free degree piecewise polynomial functions and penalized likelihood estimation the degrees as well as the locations of the polynomial pieces can be adapted to the observed data resulting in near minimax optimal convergence rates for piecewise analytic signals in particular the error of this estimator converges at nearly the parametric rate these methods can be further refined in two dimensions and it is demonstrated that platelet based estimators in two dimensions exhibit similar near optimal error convergence rates for images consisting of smooth surfaces separated by smooth boundaries

consistent estimation for partition wise regression and classification models partition wise models offer a flexible approach for modeling complex and multidimensional data and are capable of producing interpretable results they are based on partitioning the observed data into regions each of which is modeled with a simple submodel the success of this approach highly depends on the quality of the partition as too large a region could lead to a nonsimple submodel while too small a region could inflate estimation variance this paper proposes an automatic procedure for choosing the partition i e the number of regions and the boundaries between regions as well as the submodels for the regions it is shown that under the assumption of the existence of a true partition as well as a set of known significant predictors the proposed partition estimator is statistically consistent features of the proposed methodology are highlighted for both regression and classification problems on synthetic and real data

optimal bandwidth selection in nonparametric regression function estimation kernel estimators of an unknown multivariate regression function are investigated a bandwidth selection rule is considered which can be formulated in terms of cross validation under mild assumptions on the kernel and the unknown regression function it is seen that this rule is asymptotically optimal

automatic pattern recognition a study of the probability of error a test sequence is used to select the best rule from a class of discrimination rules defined in terms of the training sequence the vapnik chervonenkis and related inequalities are used to obtain distribution free bounds on the difference between the probability of error of the selected rule and the probability of error of the best rule in the given class the bounds are used to prove the consistency and asymptotic optimality for several popular classes including linear discriminators nearest neighbor rules kernel based rules histogram rules binary tree classifiers and fourier series classifiers in particular the method can be used to choose the smoothing parameter in kernel based rules to choose k in the k nearest neighbor rule and to choose between parametric and nonparametric rules

analysis of a complexity based pruning scheme for classification trees a complexity based pruning procedure for classification trees is described and bounds on its finite sample performance are established the procedure selects a subtree of a possibly random initial tree in order to minimize a complexity penalized measure of empirical risk the complexity assigned to a subtree is proportional to the square root of its size two cases are considered in the first the growing and pruning data sets are identical and in the second they are independent using the performance bound the bayes risk consistency of pruned trees obtained via the procedure is established when the sequence of initial trees satisfies suitable geometric and structural constraints the pruning method and its analysis are motivated by work on adaptive model selection using complexity regularization

a classification procedure using the multiple fourier series abstract a nonparametric procedure using the multiple fourier series is presented and examined conditions for the weak and strong bayes risk consistency are given and the rates at which the probability of misclassification converges to the bayes probability of error are presented the rates for the convergence in mean as well as w p 1 are independent of the dimension

complexity regularized tree structured partition for mutual information estimation a new histogram based mutual information estimator using data driven tree structured partitions tsp is presented in this paper the derived tsp is a solution to a complexity regularized empirical information maximization with the objective of finding a good tradeoff between the known estimation and approximation errors a distribution free concentration in equality for this tree structured learning problem as well as finite sample performance bounds for the proposed histogram based solution is derived it is shown that this solution is density free strongly consistent and that it provides with an arbitrary high probability an optimal balance between the mentioned estimation and approximation errors finally for the emblematic scenario of independence i x y 0 it is shown that the tsp estimate converges to zero with o e n1 3 log log n

estimating the error rate of a prediction rule improvement on cross validation abstract we construct a prediction rule on the basis of some data and then wish to estimate the error rate of this rule in classifying future observations cross validation provides a nearly unbiased estimate using only the original data cross validation turns out to be related closely to the bootstrap estimate of the error rate this article has two purposes to understand better the theoretical basis of the prediction problem and to investigate some related estimators which seem to offer considerably improved estimation in small samples

properties of bagged nearest neighbour classifiers summary it is shown that bagging a computationally intensive method asymptotically improves the performance of nearest neighbour classifiers provided that the resample size is less than 69 of the actual sample size in the case of with replacement bagging or less than 50 of the sample size for without replacement bagging however for larger sampling fractions there is no asymptotic difference between the risk of the regular nearest neighbour classifier and its bagged version in particular neither achieves the large sample performance of the bayes classifier in contrast when the sampling fractions converge to 0 but the resample sizes diverge to 1 the bagged classifier converges to the optimal bayes rule and its risk converges to the risk of the latter these results are most readily seen when the two populations have well defined densities but they may also be derived in other cases where densities exist in only a relative sense cross validation can be used effectively to choose the sampling fraction numerical calculation is used to illustrate these theoretical properties

mixing coefficients between discrete and real random variables computation and properties in this paper we study the problem of estimating the alpha beta and phi mixing coefficients between two random variables that can either assume values in a finite set or the set of real numbers in either case explicit closed form formulas for the beta mixing coefficient are already known therefore for random variables assuming values in a finite set our contributions are twofold 1 in the case of the alpha mixing coefficient we show that determining whether or not it exceeds a prespecified threshold is np complete and provide efficiently computable upper and lower bounds 2 we derive an exact closed form formula for the phi mixing coefficient next we prove analogs of the data processing inequality from information theory for each of the three kinds of mixing coefficients then we move on to real valued random variables and show that by using percentile binning and allowing the number of bins to increase more slowly than the number of samples we can generate empirical estimates that are consistent i e converge to the true values as the number of samples approaches infinity

bayes risk weighted vector quantization with posterior estimation for image compression and classification classification and compression play important roles in communicating digital information their combination is useful in many applications including the detection of abnormalities in compressed medical images in view of the similarities of compression and low level classification it is not surprising that there are many similar methods for their design because some of these methods are useful for designing vector quantizers it seems natural that vector quantization vq is explored for the combined goal we investigate several vq based algorithms that seek to minimize both the distortion of compressed images and errors in classifying their pixel blocks these algorithms are investigated with both full search and tree structured codes we emphasize a nonparametric technique that minimizes both error measures simultaneously by incorporating a bayes risk component into the distortion measure used for the design and encoding we introduce a tree structured posterior estimator to produce the class posterior probabilities required for the bayes risk computation in this design for two different image sources we demonstrate that this system provides superior classification while maintaining compression close or superior to that of several other vq based designs including kohonen s 1992 learning vector quantizer and a sequential quantizer classifier design

pattern recognition for conditionally independent data in this work we consider the task of relaxing the i i d assumption in pattern recognition or classification aiming to make existing learning algorithms applicable to a wider range of tasks pattern recognition is guessing a discrete label of some object based on a set of given examples pairs of objects and labels we consider the case of deterministically defined labels traditionally this task is studied under the assumption that examples are independent and identically distributed however it turns out that many results of pattern recognition theory carry over a weaker assumption namely under the assumption of conditional independence and identical distribution of objects while the only assumption on the distribution of labels is that the rate of occurrence of each label should be above some positive threshold we find a broad class of learning algorithms for which estimations of the probability of the classification error achieved under the classical i i d assumption can be generalized to the similar estimates for case of conditionally i i d examples

optimal control for nonlinear dynamical system of microbial fed batch culture in fed batch culture of glycerol bio dissimilation to 1 3 propanediol 1 3 pd the aim of adding glycerol is to obtain as much 1 3 pd as possible so a proper feeding rate is required during the process taking the concentration of 1 3 pd at the terminal time as the performance index and the feeding rate of glycerol as the control function we propose an optimal control model subject to a nonlinear dynamical system and constraints of continuous state and non stationary control a computational approach is constructed to seek the solution of the above model in two aspects on the one hand we transcribe the optimal control model into an unconstrained one based on the penalty functions and an extension of the state space on the other hand by approximating the control function with simple functions we transform the unconstrained optimal control problem into a sequence of nonlinear programming problems which can be solved using gradient based optimization techniques the convergence analysis of this approximation is also investigated numerical results show that by employing the optimal control policy the concentration of 1 3 pd at the terminal time can be increased considerably

continuation method for computing certain singular points for index 1 differential algebraic equations we will use the arclength continuation method to compute certain singular points for index 1 daes backward difference method will be employed to descretize the problem we will show that the resulting numerical iterative matrix is nonsingular in the presence of the singularity some numerical examples will be given to show the efficiency of the resulting augmented system

multivariate density estimation by bayesian sequential partitioning consider a class of densities that are piecewise constant functions over partitions of the sample space defined by sequential coordinate partitioning we introduce a prior distribution for a density in this function class and derive in closed form the marginal posterior distribution of the corresponding partition a computationally efficient method based on sequential importance sampling is presented for the inference of the partition from this posterior distribution compared to traditional approaches such as the kernel method or the histogram the bayesian sequential partitioning bsp method proposed here is capable of providing much more accurate estimates when the sample space is of moderate to high dimension we illustrate this by simulated as well as real data examples the examples also demonstrate how bsp can be used to design new classification methods competitive with the state of the art

testing for covariate effects in the fully nonparametric analysis of covariance model traditional inference questions in the analysis of covariance mainly focus on comparing different factor levels by adjusting for the continuous covariates which are believed to also exert a significant effect on the outcome variable testing hypotheses about the covariate effects although of substantial interest in many applications has received relatively limited study in the semiparametric nonparametric setting in the context of the fully nonparametric analysis of covariance model of akritas et al we propose methods to test for covariate main effects and covariate factor interaction effects the idea underlying the proposed procedures is that covariates can be thought of as factors with many levels the test statistics are closely related to some recent developments in the asymptotic theory for analysis of variance when the number of factor levels is large the limiting normal distributions are established under the null hypotheses and local alternatives by asymptotically approximating a new class

nonproduct data dependent partitions for mutual information estimation strong consistency and applications a new framework for histogram based mutual information estimation of probability distributions equipped with density functions in rd b rd is presented in this work a general histogram based estimate is proposed considering nonproduct data dependent partitions and sufficient conditions are stipulated to guarantee a strongly consistent estimate for mutual information two emblematic families of density free strongly consistent estimates are derived from this result one based on statistically equivalent blocks the gessaman s partition and the other on a tree structured vector quantization scheme

estimation of entropy and mutual information we present some new results on the nonparametric estimation of entropy and mutual information first we use an exact local expansion of the entropy function to prove almost sure consistency and central limit theorems for three of the most commonly used discretized information estimators the setup is related to grenander s method of sieves and places no assumptions on the underlying probability measure generating the data second we prove a converse to these consistency theorems demonstrating that a misapplication of the most common estimation techniques leads to an arbitrarily poor estimate of the true information even given unlimited data this inconsistency theorem leads to an analytical approximation of the bias valid in surprisingly small sample regimes and more accurate than the usual 1 n formula of miller and madow over a large region of parameter space the two most practical implications of these results are negative 1 information estimates in a certain data regime are likely contaminated by bias even if bias corrected estimators are used and 2 confidence intervals calculated by standard techniques drastically underestimate the error of the most common estimation methods finally we note a very useful connection between the bias of entropy estimators and a certain polynomial approximation problem by casting bias calculation problems in this approximation theory framework we obtain the best possible generalization of known asymptotic bias results more interesting this framework leads to an estimator with some nice properties the estimator comes equipped with rigorous bounds on the maximum error over all possible underlying probability distributions and this maximum error turns out to be surprisingly small we demonstrate the application of this new estimator on both real and simulated data

tree based ranking methods this paper investigates how recursive partitioning methods can be adapted to the bipartite ranking problem in ranking the pursued goal is global based on past data define an order on the whole input space x so that positive instances take up the top ranks with maximum probability the most natural way to order all instances consists of projecting the input data onto the real line through a real valued scoring function s and use the natural order on r the accuracy of the ordering induced by a candidate s is classically measured in terms of the roc curve or the auc here we discuss the design of tree structured scoring functions obtained by recursively maximizing the auc criterion the connection with recursive piecewise linear approximation of the optimal roc curve both in the l1 sense and in the linfin sense is highlighted a novel tree based algorithm for ranking called treerank is proposed consistency results and generalization bounds of functional nature are established for this ranking method when considering either the l1 or linfin distance we also describe committee based learning procedures using treerank as a ldquobase ranker rdquo in order to overcome obvious drawbacks of such a top down partitioning technique simulation results on artificial data are also displayed

design of three dimensional origami structures based on a vertex approach origami geometric design is fundamental to many engineering applications of origami structures this paper presents a new method for the design of three dimensional 3d origami structures suitable

histogram regression estimation using data dependent partitions we establish general sufficient conditions for the l 2 conoistency of multivariate histogram regression estimates based on data dependent partitions these same conditions insure the consistency of partitioning regression estimates based on local polynomial fits and with an additional regularity assumption the consistency of histogram estimates for conditional medians our conditions require shrinking cells subexponential growth of a combinatorial complexity measure and sublinear growth of restricted cell counts it is not assumed that the cells of every partition be rectangles with sides parallel to the coordinate axis or that each cell contain a minimum number of points response variables are assumed to be bounded throughout our results may be applied to a variety of partitioning schemes we established the consistency of histograms regression estimates based on cubic partitions with data dependent offsets k thresholding in one dimension and empirically optimal nearest neighbor clustering schemes in addition it is shown that empirically optimal regression trees are consistent when the size of the trees grows with the number of samples at an appropriate rate

recursive partitioning to reduce distortion adaptive partitioning of a multidimensional feature space plays a fundamental role in the design of data compression schemes most partition based design methods operate in an iterative fashion seeking to reduce distortion at each stage of their operation by implementing a linear split of a selected cell the operation and eventual outcome of such methods is easily described in terms of binary tree structured vector quantizers this paper considers a class of simple growing procedures for tree structured vector quantizers of primary interest is the asymptotic distortion of quantizers produced by the unsupervised implementation of the procedures it is shown that application of the procedures to a convergent sequence of distributions with a suitable limit yields quantizers whose distortion tends to zero analogous results are established for tree structured vector quantizers produced from stationary ergodic training data the analysis is applicable to procedures employing both axis parallel and oblique splitting and a variety of distortion measures the results of the paper apply directly to unsupervised procedures that may be efficiently implemented on a digital computer

combining classifiers via discretization abstract i consider a method for combining different classifiers to develop more effective classification rules the proposed combined classifier which turns out to be strongly consistent is quite simple to use in real applications it is also shown that this combined classifier is strongly asymptotically at least as good as any one of the individual classifiers in addition if one of the individual classifiers is already bayes optimal asymptotically then so is the combined classifier

the rates of convergence of kernel regression estimates and classification rules both nonrecursive and recursive nonparametric regression estimates are studied the rates of weak and strong convergence of kernel estimates as well as corresponding multiple classification errors are derived without assuming the existence of the density of the measurements an application of the obtained results to a nonparametric bayes predication is presented

design and simulation of origami structures with smooth folds origami has enabled new approaches to the fabrication and functionality of multiple structures current methods for origami design are restricted to the idealization of folds as creases of zeroth o

automatic classification of cervical cells using a binary tree classifier abstract a k means clustering algorithm for designing binary tree classifiers is introduced for the classification of cervical cells at each nonterminal node of the designed binary tree classifier two sets of effective feature are selected one is based on the bhattacharyya distance a measure of separability between two classes the other is based on the merits of classification accuracy the classification result has shown the effectiveness of the features and the binary tree classifier used