optimal cycles in doubly weighted graphs and approximation of bivariate functions by univariate ones the problem of finding optimal cycles in a doubly weighted directed graph problem a is closely related to the problem of approximating bivariate functions by the sum of two univariate functions with respect to the supremum norm problem b the close relationship between problem a and problem b is detected by the characterization 7 4 of the distance dist f t of problem b in part 1 we construct an algorithm for problem a where the essential role is played by the minimal lengthsy j k defined by 2 3 if weight functiont 1 then the minimum of problem a is computed by equality 2 4 ift 1 then the minimum is obtained by a binary search procedure algorithm 3 in part 2 we construct our algorithms for solving problem b by following exactly the ideas of part 1 by algorithm 4 we compute the minimal pseudolengthsh k y m defined by 7 5 if weight functiont 1 then the infimum dist f t of problem b is obtained by equality 7 12 which is closely related to 2 4 ift 1 we compute the infimum dist f t by the binary search procedure algorithm 5 additionally algorithm 4 leads to a constructive proof of the existence of continuous optimal solutions of problem b see theorem 7 1e which is already known in caset 1 but unknown in caset 1 interesting applications to the steady state behaviour of industrial processes with interference sect 3 and the solution of integral equations problem c are included

pairwise local fisher and naive bayes improving two standard discriminants abstract the fisher discriminant is probably the best known likelihood discriminant for continuous data another benchmark discriminant is the naive bayes which is based on marginals only in this paper we extend both discriminants by modeling dependence between pairs of variables in the continuous case this is done by local gaussian versions of the fisher discriminant in the discrete case the naive bayes is extended by taking geometric averages of pairwise joint probabilities we also indicate how the two approaches can be combined for mixed continuous and discrete data the new discriminants show promising results in a number of simulation experiments and real data illustrations

fast rates for support vector machines using gaussian kernels for binary classification we establish learning rates up to the order of n 1 for support vector machines svms with hinge loss and gaussian rbf kernels these rates are in terms of two assumptions on the considered distributions tsybakov s noise assumption to establish a small estimation error and a new geometric noise condition which is used to bound the approximation error unlike previously proposed concepts for bounding the approximation error the geometric noise assumption does not employ any smoothness assumption 1 introduction in recent years support vector machines svms have been the subject of many theoretical considerations despite this effort their learning performance on restricted classes of distributions is still widely unknown in particular it is unknown under which nontrivial circumstances svms can guarantee fast learning rates the aim of this work is to use concepts like tsybakov s noise assumption and local rademacher averages to establish learning rates up to the order of n 1 for nontrivial distributions in addition to these concepts that are used to deal with the stochastic part of the analysis we also introduce a geometric assumption for distributions that allows us to estimate the approximation properties of gaussian rbf kernels unlike many other concepts introduced for bounding the approximation error our geometric assumption is not in terms of smoothness but describes the concentration and the noisiness of the data generating distribution near the decision boundary let us formally introduce the statistical classification problem to this end let us fix a subset x r d we write y 1 1 given a finite training set

product states and c dynamical system of product type abstract a study is made of a special type of c dynamical system consisting of a class n uniformly hyperfinite c algebra a compact lie group and a natural product action of g on by automorphisms in particular a simple class of representations of the fixed point algebra g is examined those induced by restrictions of pure product states of sufficient conditions are obtained for 1 irreducibility and 2 unitary equivalence of such irreducible representations in the case of an arbitrary compact lie group g with continuous unitary representation in c n these are used to produce necessary and sufficient conditions when g u 1 or g su 2 acts by a product type conjugation on the class 2 algebra

rates of convergence for minimum contrast estimators we shall present here a general study of minimum contrast estimators in a nonparametric setting although our results are also valid in the classical parametric case for independent observations these estimators include many of the most popular estimators in various situations such as maximum likelihood estimators least squares and other estimators of the regression function estimators for mixture models or deconvolution the main theorem relates the rate of convergence of those estimators to the entropy structure of the space of parameters optimal rates depending on entropy conditions are already known at least for some of the models involved and they agree with what we get for minimum contrast estimators as long as the entropy counts are not too large but under some circumstances large entropies or changes in the entropy structure due to local perturbations the resulting the rates are only suboptimal counterexamples are constructed which show that the phenomenon is real for non parametric maximum likelihood or regression this proves that under purely metric assumptions our theorem is optimal and that minimum contrast estimators happen to be suboptimal

pure states on some group invariant c algebras let w be a uhf algebra of glimm type n i e 9t 0 nk where n n n2 are n x n matrix algebras we define an af subalgebra w g of 9t consisting of those elements of 9 invariant under a group of automorphisms ag g e g su n of product type stg is shown to be generated by an embedding of s oo the discrete group of finite permutations on countably many symbols let co be a pure product state on 9t xc its restriction to c g let e e n be a one dimensional projection with corresponding projections e k e n then if both i kvlw ek oo and ii 0 ekliw ek w ek oo hold wg is not pure g is shown to be pure if there exist orthogonal one dimensional projections p i i s n of n with corresponding projections pk e nk such that w pk 0 or 1 i i s n k 1 and 0 ek2iw pk 00 for at most one i

a general approach to classification problems abstract in the classification problem the value of the discrete random variable i must be estimated from that of a random variable x given a random sample i1 x1 i2 x2 in xn a classification rule giving such an estimate can be based on most nonparametric density estimators which are based on delta sequences the rate at which the nonerror rate for this rule converges to the optimal is calculated it is found to be o n 1 2d 2 log d n where d is the dimension of the variable x under mild assumptions on the densities a procedure for choosing an associated parameter for small samples is given

boosting with the l2 loss this article investigates a computationally simple variant of boosting l2boost which is constructed from a functional gradient descent algorithm with the l2 loss function like other boosting algorithms l2boost uses many times in an iterative fashion a prechosen fitting method called the learner based on the explicit expression of refitting of residuals of l2boost the case with symmetric linear learners is studied in detail in both regression and classification in particular with the boosting iteration m working as the smoothing or regularization parameter a new exponential bias variance trade off is found with the variance complexity term increasing very slowly as m tends to infinity when the learner is a smoothing spline an optimal rate of convergence result holds for both regression and classification and the boosted smoothing spline even adapts to higher order unknown smoothness moreover a simple expansion of a smoothed 0 1 loss function is derived to reveal the importance of the d

smooth discrimination analysis ties f and g can be based on the estimation of the set g x fx gx we consider applications where it is appropriate to assume that the region g has a smooth boundary or belongs to another nonparametric class of sets in particular this assumption makes sense if discrimination is used as a data analytic tool decision rules based on minimization of empirical risk over the whole class of sets and over sieves are considered their rates of convergence are obtained we show that these rules achieve optimal rates for estimation of g and optimal rates of convergence for bayes risks an interesting conclusion is that the optimal rates for bayes risks can be very fast in particular faster than the parametric root n rate these fast rates cannot be guaranteed for plug in rules

a conforming decomposition theorem a piecewise linear theorem of the alternative and scalings of matrices satisfying lower and upper bounds a scaling of a nonnegative matrixa is a matrixxay 1 wherex andy are nonsingular nonnegative diagonal matrices some condition may be imposed on the scaling for example whena is square x y or detx dety we characterize matrices for which there exists a scaling that satisfies predetermined upper and lower bound our principal tools are a piecewise linear theorem of the alternative and a theorem decomposing a solution of a system of equations as a sum of minimal support solutions which conform with the given solutions

characters of the infinite symplectic group a riesz ring approach we classify both the finite and infinite characters of the inductive limit symplectic group g an important feature of our technique is the systematic use of a multiplicative structure on an ordered completion of the k0 group for the group c algebra a of g we also give explicit examples of the k theory for certain primitive quotients of a

convergence rate of sieve estimates in this paper we develop a general theory for the convergence rate of sieve estimates maximum likelihood estimates mle s and related estimates obtained by optimizing certain empirical criteria in general parameter spaces in many cases especially when the parameter space is infinite dimensional maximization over the whole parameter space is undesirable in such cases one has to perform maximization over an approximating space sieve of the original parameter space and allow the size of the approximating space to grow as the sample size increases this method is called the method of sieves in the case of the maximum likelihood estimation an mle based on a sieve is called a sieve mle we found that the convergence rate of a sieve estimate is governed by a the local expected values variances and l 2 entropy of the criterion differences and b the approximation error of the sieve a robust nonparametric regression problem a mixture problem and a nonparametric regression problem are discussed as illustrations of the theory we also found that when the underlying space is too large the estimate based on optimizing over the whole parameter space may not achieve the best possible rates of convergence whereas the sieve estimate typically does not suffer from this difficulty

misclassification among methods used for multiple group discrimination the effects of distributional properties methods of multiple group discriminant analysis have not been fully studied with respect to classification into more than two populations when the covariate distributions are normal or non normal the present study examines the classification performance of several multiple discrimination methods under a variety of simulated continuous normal and non normal covariate distributions the methods include polychotomous logistic regression multiple group linear discriminant analysis kernel density estimation and rank transformations of the data as input into the linear function the parameters of interest were distance among populations configuration of population mean vectors collinear or forming the vertices of a regular simplex skewness kurtosis and bimodality simulation of the last three parameters was by log normal sinh 1 normal and a two component mixture of normal distributions respectively results with three trivariate populations show that for all distributions logistic discrimination classifies close to the optimal under neyman pearson allocation these results suggest that logistic discrimination is preferable to other widely used methods for multiple group classification with non normal data and is comparable to classification by multiple linear discrimination with normal data

minimax nonparametric classification i rates of convergence this paper studies minimax aspects of nonparametric classification we first study minimax estimation of the conditional probability of a class label given the feature variable this function say f is assumed to be in a general nonparametric class we show the minimax rate of convergence under square l sub 2 loss is determined by the massiveness of the class as measured by metric entropy the second part of the paper studies minimax classification the loss of interest is the difference between the probability of misclassification of a classifier and that of the bayes decision as is well known an upper bound on risk for estimating f gives an upper bound on the risk for classification but the rate is known to be suboptimal for the class of monotone functions this suggests that one does not have to estimate f well in order to classify well however we show that the two problems are in fact of the same difficulty in terms of rates of convergence under a sufficient condition which is satisfied by many function classes including besov sobolev lipschitz and bounded variation this is somewhat surprising in view of a result of devroye gorfi and lugosi see a probabilistic theory of pattern recognition new york springer verlag 1996

asymptotic efficiency of classifying procedures using the hermite series estimate of multivariate probability densities corresp pattern recognition procedures derived from a nonparametric estimate of multivariate probability density functions using the orthogonal hermite system are examined for sufficiently regular densities the convergence rate of the mean integrated square error mise is o n l epsilon epsilon 0 where n is the number of observations and is independent of the dimension as a consequence the rate at which the probability of misclassification converges to the bayes probability of error as the length n of the learning sequence tends to infinity is also independent of the dimension of the class densities and equals o n 1 2 delta delta o

estimation of a multivariate density function using delta sequences this paper studies some asymptotic properties of density estimates hat f of f based on d variate delta sequences the mean square consistency almost sure consistency and asymptotic normality of hat f have been obtained as corollaries to the l 1 convergence properties of these delta sequences estimators based on kernel functions orthogonal series and some histogram methods can be obtained as special cases of hat f

minimax optimal level set estimation this paper describes a new methodology and associated theoretical analysis for rapid and accurate extraction of level sets of a multivariate function from noisy data the identification of the boundaries of such sets is an important theoretical problem with applications for digital elevation maps medical imaging and pattern recognition this problem is significantly different from classical segmentation because level set boundaries may not correspond to singularities or edges in the underlying function as a result segmentation methods which rely upon detecting boundaries would be potentially ineffective in this regime this issue is addressed in this paper through a novel error metric sensitive to both the error in the location of the level set estimate and the deviation of the function from the critical level hoeffding s inequality is used to derive a novel regularization term that is distinctly different from regularization methods used in conventional image denoising settings building upon this foundation it is possible to derive error performance bounds for the proposed estimator and demonstrate that it exhibits near minimax optimal error decay rates for large classes of level set problems the proposed method automatically adapts to the spatially varying regularity of both the boundary of the level set and the underlying function

applications of shortest path algorithms to matrix scalings a symmetric scaling of a nonnegative square matrixa is a matrixxax 1 wherex is a nonsingular nonnegative diagonal matrix by associating a family of weighted directed graphs with the matrixa we are able to adapt the shortest path algorithms to compute an optimal scaling ofa where we call a symmetric scalinga ofa optimal if it minimizes the maximum of the ratio of non zero elements

automatic pattern recognition a study of the probability of error a test sequence is used to select the best rule from a class of discrimination rules defined in terms of the training sequence the vapnik chervonenkis and related inequalities are used to obtain distribution free bounds on the difference between the probability of error of the selected rule and the probability of error of the best rule in the given class the bounds are used to prove the consistency and asymptotic optimality for several popular classes including linear discriminators nearest neighbor rules kernel based rules histogram rules binary tree classifiers and fourier series classifiers in particular the method can be used to choose the smoothing parameter in kernel based rules to choose k in the k nearest neighbor rule and to choose between parametric and nonparametric rules

probability inequalities for likelihood ratios and convergence rates of sieve mles let y 1 y n be independent identically distributed with density p 0 and let f be a space of densities we show that the supremum of the likelihood ratios i 1 n p y i p 0 y i where the supremum is over p f with p l 2 p 0 1 2 2 e is exponentially small with probability exponentially close to 1 the exponent is proportional to ne 2 the only condition required for this to hold is that e exceeds a value determined by the bracketing hellinger entropy of f a similar inequality also holds if we replace f by f n and p 0 by q n where q n is an approximation to p 0 in a suitable sense these results are applied to establish rates of convergence of sieve mles furthermore weak conditions are given under which the optimal rate e n defined by h e n f ne n 2 where h f is the hellinger entropy of f is nearly achievable by sieve estimators

universal donsker classes and metric entropy when mathfrak f is a universal donsker class then for independent indetically distributed i i d observation mathbf x 1 ldots mathbf x n with an unknown law p for any mathfrak f i in mathfrak f i 1 ldots m quad n 1 2 left mathfrak f 1 left mathbf x 1 right ldots mathfrak f i left mathbf x n right right 1 leq i leq m is asymptotically normal with mean vector n 1 2 left int mathfrak f i left mathbf x n right d mathbf p left x right right 1 leq i leq m and covariance matrix int mathfrak f i mathfrak f j d mathbf p int mathfrak f id mathbf p int mathfrak f jd mathbf p uniformly for mathfrak f i in mathfrak f then for certain statistics formed frome the mathfrak f i left mathbf x k right even where mathfrak f i may be chosen depending on the mathbf x k there will be asymptotic distribution as n rightarrow infty for example for mathbf x 2 statistics where f i are indicators of disjoint intervals depending suitably on mathbf x 1 ldots mathbf x n whose union is the real line mathbf x 2 quadratic forms have limiting distributions roy 1956 and watson 1958 which may however not be mathbf x 2 distributions and may depend on p chernoff and lehmann 1954 universal donsker classes of sets are up to mild measurability conditions just classes satisfying the vapnik cervonenkis comdinatorial conditions defined later in this section donsker the vapnik cervonenkis combinatorial conditions defined later in this section durst and dudley 1981 and dudley 1984 chapter 11 the use of such classes allows a variety of extensions of the roy watson results to general multidimensional sample spaces pollard 1979 and moore and subblebine 1981 vapnik and cervonenkis 1974 indicated application of their families of sets to classification pattern recognition problems more recently the classes have been applied to tree structured classifiacation breiman friedman olshen and stone 1984 chapter 12

a classification procedure using the multiple fourier series abstract a nonparametric procedure using the multiple fourier series is presented and examined conditions for the weak and strong bayes risk consistency are given and the rates at which the probability of misclassification converges to the bayes probability of error are presented the rates for the convergence in mean as well as w p 1 are independent of the dimension

nonparametric estimation via empirical risk minimization a general notion of universal consistency of nonparametric estimators is introduced that applies to regression estimation conditional median estimation curve fitting pattern recognition and learning concepts general methods for proving consistency of estimators based on minimizing the empirical error are shown in particular distribution free almost sure consistency of neural network estimates and generalized linear estimators is established

properties of bagged nearest neighbour classifiers summary it is shown that bagging a computationally intensive method asymptotically improves the performance of nearest neighbour classifiers provided that the resample size is less than 69 of the actual sample size in the case of with replacement bagging or less than 50 of the sample size for without replacement bagging however for larger sampling fractions there is no asymptotic difference between the risk of the regular nearest neighbour classifier and its bagged version in particular neither achieves the large sample performance of the bayes classifier in contrast when the sampling fractions converge to 0 but the resample sizes diverge to 1 the bagged classifier converges to the optimal bayes rule and its risk converges to the risk of the latter these results are most readily seen when the two populations have well defined densities but they may also be derived in other cases where densities exist in only a relative sense cross validation can be used effectively to choose the sampling fraction numerical calculation is used to illustrate these theoretical properties

kernel classification rules from missing data nonparametric kernel classification rules derived from incomplete missing data are studied a number of techniques of handling missing observation in the training set are taken into account in particular the straightforward approach of designing a classifier only from available data deleting missing values is considered the class of imputation techniques is also taken into consideration in the latter case one estimates missing values and then calculates classification rules from such a completed training set consistency and speed of convergence of proposed classification rules are established results of simulation studies are presented

approximation of least squares regression on nested subspaces on presente une methode generale pour approximer le biais et la variance dans une echelle de normes hilbertiennes

extending traces on fixed point c algebras under xerox product type actions of compact lie groups let g denote a compact connected lie group with maximal torus t of dimension d say let n g t m c be a unitary representation and form the njl uhf c algebra a 0 m c n is fixed this is the infinite tensor product of copies of the n on matrix ring we have an action tt g 3 aut a by setting a g 0 ad n g b m baker suggested the name xerox product type action because the representation 7c is duplicated over and over we may form the crossed product a x 1 g and the fixed point algebra a or simply ac if there is little likelihood of ambiguity by restriction of c to t we also obtain a xerox action of the d torus t also called ci clearly ac is a unital subalgebra of a the principal result asserts that every trace on ac extends to a trace on a in the course of the proof we also show that the grothendieck group of a k ag is finitely generated as a ring and that ko at which is also a ring is finitely generated as a k a module there are several consequences of these results the space of faithful pure traces on ag is a dense open subset of the pure trace space of a and moreover is homeomorphic to rd w the orbit space of the strictly positive d tuples under the natural action of the weyl group provided that o is faithful if b fl m c is a uhf algebra with corresponding product type action not necessarily xerox 0 ad nrr then the natural inclusion a 3 a b g301 b induces a bijection on faithful pure traces if c is faithful and the set of the latter is dense in the pure trace space of a b another result not proved here using the main result of the article asserts that a difference of characters of g divides an actual character within the representation ring of g if and only if its restriction to t divides a character of t

minimax optimal classification with dyadic decision trees decision trees are among the most popular types of classifiers with interpretability and ease of implementation being among their chief attributes despite the widespread use of decision trees theoretical analysis of their performance has only begun to emerge in recent years in this paper it is shown that a new family of decision trees dyadic decision trees ddts attain nearly optimal in a minimax sense rates of convergence for a broad range of classification problems furthermore ddts are surprisingly adaptive in three important respects they automatically 1 adapt to favorable conditions near the bayes decision boundary 2 focus on data distributed on lower dimensional manifolds and 3 reject irrelevant features ddts are constructed by penalized empirical risk minimization using a new data dependent penalty and may be computed exactly with computational complexity that is nearly linear in the training sample size ddts comprise the first classifiers known to achieve nearly optimal rates for the diverse class of distributions studied here while also being practical and implementable this is also the first study of which we are aware to consider rates for adaptation to intrinsic data dimension and relevant features

computing optimal scalings by parametric network algorithms asymmetric scaling of a square matrixa 0 is a matrix of the formxax 1 wherex is a nonnegative nonsingular diagonal matrix having the same dimension ofa anasymmetric scaling of a rectangular matrixb 0 is a matrix of the formxby 1 wherex andy are nonnegative nonsingular diagonal matrices having appropriate dimensions we consider two objectives in selecting a symmetric scaling of a given matrix the first is to select a scalinga of a given matrixa such that the maximal absolute value of the elements ofa is lesser or equal that of any other corresponding scaling ofa the second is to select a scalingb of a given matrixb such that the maximal absolute value of ratios of nonzero elements ofb is lesser or equal that of any other corresponding scaling ofb we also consider the problem of finding an optimal asymmetric scaling under the maximal ratio criterion the maximal element criterion is of course trivial in this case we show that these problems can be converted to parametric network problems which can be solved by corresponding algorithms

pattern recognition algorithms based on space filling curves and orthogonal expansions the classical problem of constructing a multidimensional pattern classifier in the bayesian framework is considered preprocessing of the learning sequence by a quasi inverse of a space filling curve is proposed and properties of space filling curves which are necessary to obtain bayes risk consistency are indicated the learning sequence transformed into the unit interval is used to estimate the coefficients in an orthogonal expansion of the bayes decision rule to transform a new observation into the unit interval requires o d elementary operations where d is the dimension of the observation space strong bayes risk consistency of the classifiers is proved when distributions of the random pair of the observation vector and its class are absolutely continuous with respect to the lebesgue measure attainable convergence rate of the bayes risk is discussed details of the classification algorithm based on the haar series and its properties are presented distribution free consistency of the classifier is established the performance of such a classifier is tested both on simulated data and on the standard benchmarks providing misclassification errors comparable to or even better than the k nearest neighbors k nn method

edge preserving smoothers for image processing abstract classical smoothers have limited usefulness in image processing because sharp edges tend to be blurred there is a literature on edge preserving smoothers but these all require moderately large smooth stretches here we discuss an approach to this problem called sigma filtering and propose an improvement based on running m estimation both computational and theoretical aspects are developed for image processing the methods have a niche between standard filtering approaches and bayes markov random field analysis

max balancing weighted directed graphs and matrix scaling a weighted directed graph g is a triple v a g where v a is a directed graph and g is an arbitrary real valued function defined on the arc set a let g be a strongly connected simple weighted directed graph we say that g is max balanced if for every nontrivial subset of the vertices w the maximum weight over arcs leaving w equals the maximum weight over arcs entering w we show that there exists a up to an additive constant unique potential pv for v v such that v a gp is max balanced where gap pu ga pv for a u v a we describe an o v 2 a algorithm for computing p using an algorithm for computing the maximum cycle mean of g finally we apply our principal result to the similarity scaling of nonnegative matrices

nonparametric maximum likelihood estimation by the method of sieves maximum likelihood estimation often fails when the parameter takes values in an infinite dimensional space for example the maximum likelihood method cannot be applied to the completely nonparametric estimation of a density function from an operatorname iid sample the maximum of the likelihood is not attained by any density in this example as in many other examples the parameter space positive functions with area one is too big but the likelihood method can often be salvaged if we first maximize over a constrained subspace of the parameter space and then relax the constraint as the sample size grows this is grenander s method of sieves application of the method sometimes leads to new estimators for familiar problems or to a new motivation for an already well studied technique we will establish some general consistency results for the method and then we will focus on three applications

optimal rates of convergence to bayes risk in nonparametric discrimination consider the multiclassification discrimination problem with known prior probabilities and a multi dimensional vector of observations assume the underlying densities corresponding to the various classes are unknown but a training sample of size n is available from each class rates of convergence to bayes risk are investigated under smoothness conditions on the underlying densities of the type often seen in nonparametric density estimation these rates can be drastically affected by a small change in the prior probabilities so the error criterion used here is bayes risk averaged uniformly over all prior probabilities then it is shown that a certain rate n r is optimal in the sense that no rule can do better uniformly over the class of smooth densities and a rule is exhibited which does that well the optimal value of r depends on the smoothness of the distributions and the dimensionality of the observations in the same way as for nonparametric density estimation with integrated square error loss

sur la classification des automorphismes periodiques des c algebres uhf abstract we obtain a complete classification up to conjugacy and up to outer conjugacy of finite tensor product type automorphisms of uhf c algebras which are periodic of period n n prime