regression modeling in back propagation and projection pursuit learning we study and compare two types of connectionist learning methods for model free regression problems 1 the backpropagation learning bpl and 2 the projection pursuit learning ppl emerged in recent years in the statistical estimation literature both the bpl and the ppl are based on projections of the data in directions determined from interconnection weights however unlike the use of fixed nonlinear activations usually sigmoidal for the hidden neurons in bpl the ppl systematically approximates the unknown nonlinear activations moreover the bpl estimates all the weights simultaneously at each iteration while the ppl estimates the weights cyclically neuron by neuron and layer by layer at each iteration although the bpl and the ppl have comparable training speed when based on a gauss newton optimization algorithm the ppl proves more parsimonious in that the ppl requires a fewer hidden neurons to approximate the true function to further improve the statistical performance of the ppl an orthogonal polynomial approximation is used in place of the supersmoother method originally proposed for nonlinear activation approximation in the ppl

representations of nets of c algebras over s 1 in recent times a new kind of representations has been used to describe superselection sectors of the observable net over a curved spacetime taking into account the effects of the fundamental group of the spacetime using this notion of representation we prove that any net of c algebras over s 1 admits faithful representations and when the net is covariant under diff s 1 it admits representations covariant under any amenable subgroup of diff s 1

a semiparametric approach to canonical analysis summary classical canonical correlation analysis is one of the fundamental tools in statistics to investigate the linear association between two sets of variables we propose a method called semiparametric canonical analysis to generalize canonical correlation analysis to incorporate the important non linear association semiparametric canonical analysis is easy to implement and interpret statistical properties are proved a consistent estimation method is developed selection of significant semiparametric canonical analysis components is discussed simulations suggest that the methods proposed have satisfactory performance in finite samples one environmental data set and one data set in social science are investigated in which non linear canonical associations are observed and interpreted

the generally covariant locality principle a new paradigm for local quantum field theory a new approach to the model independent description of quantum field theories will be introduced in the present work the main feature of this new approach is to incorporate in a local sense the principle of general covariance of general relativity thus giving rise to the concept of a locally covariant quantum field theory such locally covariant quantum field theories will be described mathematically in terms of covariant functors between the categories on one side of globally hyperbolic spacetimes with isometric embeddings as morphisms and on the other side of algebras with unital injective monomorphisms as morphisms moreover locally covariant quantum fields can be described in this framework as natural transformations between certain functors the usual haag kastler framework of nets of operator algebras over a fixed spacetime background manifold together with covariant automorphic actions of the isometry group of the background spacetime can be re gained from this new approach as a special case examples of this new approach are also outlined in case that a locally covariant quantum field theory obeys the time slice axiom one can naturally associate to it certain automorphic actions called relative cauchy evolutions which describe the dynamical reaction of the quantum field theory to a local change of spacetime background metrics the functional derivative of a relative cauchy evolution with respect to the spacetime metric is found to be a divergence free quantity which has as will be demonstrated in an example the significance of an energy momentum tensor up to addition of scalar functions for the locally covariant quantum field theory furthermore we discuss the functorial properties of state spaces of locally covariant quantum field theories that entail the validity of the principle of local definiteness

a simple lemma on greedy approximation in hilbert space and convergence rates for projection pursuit regression and neural network training a general convergence criterion for certain iterative sequences in hilbert space is presented for an important subclass of these sequences estimates of the rate of convergence are given under very mild assumptions these results establish an 0 1 f4n nonsampling convergence rate for projection pursuit regression and neural network training where n represents the number of ridge functions neurons or coefficients in a greedy basis expansion

use of bias term in projection pursuit learning improves approximation and convergence properties in a regression problem one is given a multidimensional random vector x the components of which are called predictor variables and a random variable y called response a regression surface describes a general relationship between x and y a nonparametric regression technique that has been successfully applied to high dimensional data is projection pursuit regression ppr the regression surface is approximated by a sum of empirically determined univariate functions of linear combinations of the predictors projection pursuit learning ppl formulates ppr using a 2 layer feedforward neural network the smoothers in ppr are nonparametric whereas those in ppl are based on hermite functions of some predefined highest order r we demonstrate that ppl networks in the original form do not have the universal approximation property for any finite r and thus cannot converge to the desired function even with an arbitrarily large number of hidden units but by including a bias term in each linear projection of the predictor variables ppl networks can regain these capabilities independent of the exact choice of r experimentally it is shown in this paper that this modification increases the rate of convergence with respect to the number of hidden units improves the generalization performance and makes it less sensitive to the setting of r finally we apply ppl to chaotic time series prediction and obtain superior results compared with the cascade correlation architecture

a piecewise single index model for dimension reduction in this article we propose to use different single index models for observations in different regions of the sample space this approach inherits the estimation efficiency of the single index model in each region and at the same time allows the global model to have multidimensionality in the sense of conventional dimension reduction on the other hand the model can be seen as an extension of cart and a piecewise linear model proposed modeling procedures including identifying the region for every single index model and estimation of the single index models are developed simulation studies and real data analysis are employed to demonstrate the usefulness of the approach computer code and technical details of the method are provided as supplementary material online

application of projection pursuit learning to boundary detection and deblurring in images abstract projection pursuit learning networks pplns have been used in many fields of research but have not been widely used in image processing in this paper we demonstrate how this highly promising technique may be used to connect edges and produce continuous boundaries we also propose the application of ppln to deblurring a degraded image when little or no a priori information about the blur is available the ppln was successful at developing an inverse blur filter to enhance blurry images theory and background information on projection pursuit regression ppr and ppln are also presented

the c0 x algebra of a net and index theory abstract given a connected and locally compact hausdorff space x with a good base k we assign in a functorial way a c 0 x algebra to any precosheaf of c algebras a defined over k afterwards we consider the representation theory and the kasparov k homology of a and interpret them in terms respectively of the representation theory and the k homology of the associated c 0 x algebra when a is an observable net over the spacetime x in the sense of algebraic quantum field theory this yields a geometric description of the recently discovered representations affected by the topology of x

presheaves of superselection structures in curved spacetimes we show that superselection structures on curved spacetimes that are expected to describe quantum charges affected by the underlying geometry are categories of sections of presheaves of symmetric tensor categories when an embedding functor is given the superselection structure is a tannaka type dual of a locally constant group bundle which hence becomes a natural candidate for the role of the gauge group indeed we show that any locally constant group bundle with suitable structure group acts on a net of c algebras fulfilling normal commutation relations on an arbitrary spacetime we also give examples of gerbes of c algebras defined by wightman fields and constructed using projective representations of the fundamental group of the spacetime which we propose as solutions for the problem that existence and uniqueness of the embedding functor are not guaranteed

a new light on nets of c algebras and their representations the present paper deals with the question of representability of nets of c algebras whose underlying poset indexing the net is not upward directed a particular class of nets called c net bundles is classified in terms of c dynamical systems having as group the fundamental group of the poset any net of c algebras has a canonical morphism into a c net bundle the enveloping net bundle which generalizes the notion of universal c algebra given by fredenhagen to nonsimply connected posets this allows a classification of nets in particular we call injective those nets such that the canonical morphism is faithful injectivity turns out to be equivalent to the existence of faithful representations we further relate injectivity to a generalized cech cocycle of the net and this allows us to give examples of nets exhausting the above classification

harmonic decomposition of audio signals with matching pursuit we introduce a dictionary of elementary waveforms called harmonic atoms that extends the gabor dictionary and fits well the natural harmonic structures of audio signals by modifying the standard matching pursuit we define a new pursuit along with a fast algorithm namely the fast harmonic matching pursuit to approximate n dimensional audio signals with a linear combination of m harmonic atoms our algorithm has a computational complexity of o mkn where k is the number of partials in a given harmonic atom the decomposition method is demonstrated on musical recordings and we describe a simple note detection algorithm that shows how one could use a harmonic matching pursuit to detect notes even in difficult situations e g very different note durations lots of reverberation and overlapping notes

sparse representations for image decompositions we are given an image i and a library of templates cal l such that cal l is an overcomplete basis for i the templates can represent objects faces features analytical functions or be single pixel templates canonical templates there are infinitely many ways to decompose i as a linear combination of the library templates each decomposition defines a representation for the image i given cal l what is an optimal representation for i given cal l and how to select it we are motivated to select a sparse compact representation for i and to account for occlusions and noise in the image we present a concave cost function criterion on the linear decomposition coefficients that satisfies our requirements more specifically we study a weighted l norm with 0 p 1 we prove a result that allows us to generate all local minima for the l norm and the global minimum is obtained by searching through the local ones due to the computational complexity i e the large number of local minima we also study a greedy and iterative weighted l matching pursuit strategy

a theory of bundles over posets in algebraic quantum field theory the spacetime manifold is replaced by a suitable base for its topology ordered under inclusion we explain how certain topological invariants of the manifold can be computed in terms of the base poset we develop a theory of connections and curvature for bundles over posets in search of a formulation of gauge theories in algebraic quantum field theory

a posteriori quantization of progressive matching pursuit streams this paper proposes a rate distortion optimal a posteriori quantization scheme for matching pursuit mp coefficients the a posteriori quantization applies to an mp expansion that has been generated offline and cannot benefit of any feedback loop to the encoder in order to compensate for the quantization noise the redundancy of the mp dictionary provides an indicator of the relative importance of coefficients and atom indices and subsequently on the quantization error it is used to define a universal upper bound on the decay of the coefficients sorted in decreasing order of magnitude a new quantization scheme is then derived where this bound is used as an oracle for the design of an optimal a posteriori quantizer the latter turns the exponentially distributed coefficient entropy constrained quantization problem into a simple uniform quantization problem using simulations with random dictionaries we show that the proposed exponentially upper bounded quantization euq clearly outperforms classical schemes stepping on the ideal oracle based approach a suboptimal adaptive scheme is then designed that approximates the euq but still outperforms competing quantization methods in terms of rate distortion characteristics finally the proposed quantization method is studied in the context of image coding it performs similarly to state of the art coding methods and even better at low rates while interestingly providing a progressive stream that is very easy to transcode and adapt to changing rate constraints

local minimax learning of functions with best finite sample estimation error bounds applications to ridge and lasso regression boosting tree learning kernel machines and inverse problems optimal local estimation is formulated in the minimax sense for inverse problems and nonlinear regression this theory provides best mean squared finite sample error bounds for some popular statistical learning algorithms and also for several optimal improvements of other existing learning algorithms such as smoothing splines and kernel regularization the bounds and improved algorithms are not based on asymptotics or bayesian assumptions and are truly local for each query not depending on cross validating estimates at other queries to optimize modeling parameters results are given for optimal local learning of approximately linear functions with side information context using real algebraic geometry in particular finite sample error bounds are given for ridge regression and for a local version of lasso regression the new regression methods require only quadratic programming with linear or quadratic inequality constraints for implementation greedy additive expansions are then combined with local minimax learning via a change in metric an optimal strategy is presented for fusing the local minimax estimators of a class of experts providing optimal finite sample prediction error bounds from random forests local minimax learning is extended to kernel machines best local prediction error bounds for finite samples are given for tikhonov regularization the geometry of reproducing kernel hilbert space is used to derive improved estimators with finite sample mean squared error mse bounds for class membership probability in two class pattern classification problems a purely local cross validation free algorithm is proposed which uses fisher information with these bounds to determine best local kernel shape in vector machine learning finally a locally quadratic solution to the finite fourier moments problem is presented after reading the first three sections the reader may proceed directly to any of the subsequent applications sections

model checking in regression via dimension reduction lack of fit checking for parametric and semiparametric models is essential in reducing misspecification the efficiency of most existing model checking methods drops rapidly as the dimension of the covariates increases we propose to check a model by projecting the fitted residuals along a direction that adapts to the systematic departure of the residuals from the desired pattern consistency of the method is proved for parametric and semiparametric regression models a bootstrap implementation is also discussed simulation comparisons with several existing methods are made suggesting that the proposed methods are more efficient than the existing methods when the dimension increases air pollution data from chicago are used to illustrate the procedure

pattern classification using projection pursuit abstract this article discusses the adaptation of recently developed regression techniques to classifier design apart from finite sample effects projection pursuit pp regression may be used to model a desired response class as a sum of ridge functions according to a minimum expected squared error criterion this approach can be shown to furnish an optimal discriminant function which can satisfy the neyman pearson criterion over all possible thresholds basis function expansions are used instead of smoothed histograms to reduce computation since good approximation of a discriminant by a linear combination of moderate number of ridge functions may not be easy we introduce an improved method utilizing a nonlinear weighting function

on a conjecture of huber concerning the convergence of projection pursuit regression we generalize the projection pursuit procedure of friedman and stuetzle abstract version and prove strong convergence this answers a question of huber

quantum charges and spacetime topology the emergence of new superselection sectors a new form of superselection sectors of topological origin is developed by that it is meant a new investigation that includes several extensions of the traditional framework of doplicher haag and roberts in local quantum theories at first we generalize the notion of representations of nets of c algebras then we provide a brand new view on selection criteria by adopting one with a strong topological flavour we prove that it is coherent with the older point of view hence a clue to a genuine extension in this light we extend roberts cohomological analysis to the case where 1 cocycles bear non trivial unitary representations of the fundamental group of the spacetime equivalently of its cauchy surface in the case of global hyperbolicity a crucial tool is a notion of group von neumann algebras generated by the 1 cocycles evaluated on loops over fixed regions one proves that these group von neumann algebras are localized at the bounded region where loops start and end and to be factorial of finite type i all that amounts to a new invariant in a topological sense which can be defined as the dimension of the factor we prove that any 1 cocycle can be factorized into a part that contains only the charge content and another where only the topological information is stored this second part much resembles what in literature is known as geometric phases indeed by the very geometrical origin of the 1 cocycles that we discuss in the paper they are essential tools in the theory of net bundles and the topological part is related to their holonomy content at the end we prove the existence of net representations

fast matching pursuit with a multiscale dictionary of gaussian chirps we introduce a modified matching pursuit algorithm called fast ridge pursuit to approximate n dimensional signals with m gaussian chirps at a computational cost o mn instead of the expected o mn sup 2 logn at each iteration of the pursuit the best gabor atom is first selected and then its scale and chirp rate are locally optimized so as to get a good chirp atom i e one for which the correlation with the residual is locally maximized a ridge theorem of the gaussian chirp dictionary is proved from which an estimate of the locally optimal scale and chirp is built the procedure is restricted to a sub dictionary of local maxima of the gaussian gabor dictionary to accelerate the pursuit further the efficiency and speed of the method is demonstrated on a sound signal