linear programming in linear time when the dimension is fixed it is demonstrated that the linear programming problem in d variables and n constraints can be solved in o n time when d is fixed this bound follows from a multidimensional search technique which is applicable for quadratic programming as well there is also developed an algorithm that is polynomial in both n and d provided d is bounded by a certain slowly growing function of n

ergodicity of the 2d navier stokes equations with degenerate stochastic forcing the stochastic 2d navier stokes equations on the torus driven by degenerate noise are studied we characterize the smallest closed invariant subspace for this model and show that the dynamics restricted to that subspace is ergodic in particular our results yield a purely geometric characterization of a class of noises for which the equation is ergodic in l 2 0 t 2 unlike previous works this class is independent of the viscosity and the strength of the noise the two main tools of our analysis are the asymptotic strong feller property introduced in this work and an approximate integration by parts formula the first when combined with a weak type of irreducibility is shown to ensure that the dynamics is ergodic the second is used to show that the first holds un

bounds on the minimax rate for estimating a prior over a vc class from independent learning tasks abstract we study the optimal rates of convergence for estimating a prior distribution over a vc class from a sequence of independent data sets respectively labeled by independent target functions sampled from the prior we specifically derive upper and lower bounds on the optimal rates under a smoothness condition on the correct prior with the number of samples per data set equal the vc dimension these results have implications for the improvements achievable via transfer learning we additionally extend this setting to real valued function where we establish consistency of an estimator for the prior and discuss an additional application to a preference elicitation problem in algorithmic economics

additive rank one preservers on block triangular matrix spaces in this note we address a description of the general form of additive rank one preservers on block triangular matrix spaces over an arbitrary field as an application we characterize additive mappings preserving rank additivity on block triangular matrix algebras

maximal pattern complexity dual system and pattern recognition for a family w of sets in r 2 and a finite subset s of r 2 let p w s be the number of distinct sets of the form s w for all w w the maximum pattern complexity p w k is the maximum of p w s among s with s k the s attaining the maximum is considered as the most effective sampling to distinguish the sets in w we obtain the exact values or at least the order of p w k in k for various classes w we also discuss the dual problem in the case that w that is consider the partition of r 2 generated by a finite family t w the number of elements in the partition is written as p r 2 t and p r 2 k is the maximum of p r 2 t among t with t k here p w k p r 2 k does not hold in general for the general setting that w is an infinite subset of a s where a is a finite alphabet s is an arbitrary infinite set and p w k max s k w s it is known that the entropy h w limk logp w k k exists and takes value in log1 log2 log a in this paper we prove that the entropy h s of the dual system coincides with h w

sample sizes for threshold networks with equivalences this paper applies the theory of probably approximately correct pac learning to multiple output feedforward threshold networks in which the weights conform to certain equivalences it is shown that the sample site for reliable learning can be bounded above by a formula similar to that required for single output networks with no equivalences the best previously obtained bounds are improved for all cases

ergodicity of stochastic 2d navier stokes equation with levy noise abstract in this paper we deal with the 2d navier stokes equation perturbed by a levy noise force whose white noise part is non degenerate and that the intensity measure of poisson measure is finite existence and uniqueness of invariant measure for this equation is obtained two main properties of the markov semigroup associated with this equation are proved in other words strong feller property and irreducibility hold in the same space

ergodic and mixing properties of the boussinesq equations with a degenerate random forcing we establish the existence uniqueness and attraction properties of an ergodic invariant measure for the boussinesq equations in the presence of a degenerate stochastic forcing acting only in the temperature equation and only at the largest spatial scales the central challenge is to establish time asymptotic smoothing properties of the markovian dynamics corresponding to this system towards this aim we encounter a lie bracket structure in the associated vector fields with a complicated dependence on solutions this leads us to develop a novel hormander type condition for infinite dimensional systems demonstrating the sufficiency of this condition requires new techniques for the spectral analysis of the malliavin covariance matrix

identification criteria and lower bounds for perceptron like learning rules perceptron like learning rules are known to require exponentially many correction steps in order to identify boolean threshold functions exactly we introduce criteria that are weaker than exact identification and investigate whether learning becomes significantly faster if exact identification is replaced by one of these criteria probably approximately correct pac identification order identification and sign identification pac identification is based on the learning paradigm introduced by valiant 1984 and is known to be easier than exact identification order identification uses the fact that each threshold function induces an ordering relation on the input variables that can be represented by weights of linear size sign identification is based on a property of threshold functions known as unateness and requires only weights of constant size we show that perceptron like learning rules cannot satisfy these criteria when the number of correction steps is to be bounded by a polynomial we also pre

how to construct random functions a constructive theory of randomness for functions based on computational complexity is developed and a pseudorandom function generator is presented this generator is a deterministic polynomial time algorithm that transforms pairs g r where g is any one way function and r is a random k bit string to polynomial time computable functions r 1 2 k 1 2 k these r s cannot be distinguished from random functions by any probabilistic polynomial time algorithm that asks and receives the value of a function at arguments of its choice the result has applications in cryptography random constructions and complexity theory

ergodicity of stochastic magneto hydrodynamic equations driven by α stable noise abstract the current paper is devoted to the ergodicity of stochastic magneto hydrodynamic equations driven by stable noise with 3 2 2 by the maximal inequality for the stochastic stable convolution and vorticity transformation the well posedness of the mild solution for stochastic magneto hydrodynamic equation is established due to the discontinuous trajectories the existence and uniqueness of the invariant measure for stochastic magneto hydrodynamic equation are obtained by the strong feller property and the accessibility to zero instead of the irreducibility

exact learning of discretized geometric concepts we first present an algorithm that uses membership and equivalence queries to exactly identify a discretized geometric concept defined by the union of m axis parallel boxes in d dimensional discretized euclidean space where each coordinate can have n discrete values this algorithm receives at most md counterexamples and uses time and membership queries polynomial in m and log n for any constant d furthermore all equivalence queries can be formulated as the union of o md log m axis parallel boxes next we show how to extend our algorithm to efficiently learn from only equivalence queries any discretized geometric concept generated from any number of halfspaces with any number of known to the learner slopes in a constant dimensional space in particular our algorithm exactly learns from equivalence queries only unions of discretized axis parallel boxes in constant dimensional space in polynomial time furthermore this equivalence query only algorithm can be modified to handle a polynomial number of lies in the counterexamples provided by the environment finally we introduce a new complexity measure that better captures the complexity of the union of m boxes than simply the number of boxes and the dimension our new measure sigma is the number of segments in the target where a segment is a maximum portion of one of the sides of the target that lies entirely inside or entirely outside each of the other halfspaces defining the target we present a modification of our first algorithm that uses time and queries polynomial in sigma and log n in fact the time and queries both membership and equivalence used by this single algorithm are polynomial for either m or d constant

bounds on the number of examples needed for learning functions we prove general lower bounds on the number of examples needed for learning function classes within different natural learning models which are related to pac learning and coincide with the pac learning model of valiant in the case of 0 1 valued functions the lower bounds are obtained by showing that all nontrivial function classes contain a hard binary valued subproblem although at first glance it seems to be likely that real valued function classes are much harder to learn than their hardest binary valued subproblem we show that these general lower bounds cannot be improved by more than a logarithmic factor this is done by discussing some natural function classes like nondecreasing functions or piecewise smooth functions the function classes that were discussed in m j kearns and r e schapire proc 31st annual symposium on the foundations of computer science ieee computer society press los alamitos ca 1990 pp 382 392 full version j comput system sci 48 1994 pp 464 497 d kimber and p m long proc 5th annual workshop on computational learning theory acm new york 1992 pp 153 160 with certain restrictions concerning their slope

a metric entropy bound is not sufficient for learnability the authors prove by means of a counterexample that it is not sufficient for probably approximately correct pac learning under a class of distributions to have a uniform bound on the metric entropy of the class of concepts to be learned this settles a conjecture of benedek and itai 1991

rank one preserving r linear maps on spaces of self adjoint operators on complex hilbert spaces abstract let h be a complex hilbert space and let s a h be the space of all self adjoint operators on h let denote the subset of s a h consisting of all rank one operators a map l from s a h to itself is called a rank one preserver on s a h if l and is said to be r linear if l a b l a l b and l ka kl a for any k r and a b s a h where r is the field of real numbers we give a complete classification of all r linear and weakly continuous rank one preservers on s a h moreover we also determine the general form of all r linear and weakly continuous rank preservers respectively rank one idempotence preservers on s a h

on the complexity of learning from drifting distributions abstract we consider two models of on line learning of binary valued functions from drifting distributions due to bartlett we show that if each example is drawn from a joint distribution which changes in total variation distance by at mosto e3 d log 1 e between trials then an algorithm can achieve a probability of a mistake at mosteworse than the best function in a class of vc dimensiond we prove a corresponding necessary condition ofo e3 d finally in the case that a fixed function is to be learned from noise free examples we show that if the distributions on the domain generating the examples change by at mosto e2 d log 1 e then any consistent algorithm learns to within accuracye

spectral gaps in wasserstein distances and the 2d stochastic navier stokes equations we develop a general method to prove the existence of spectral gaps for markov semigroups on banach spaces unlike most previous work the type of norm we consider for this analysis is neither a weighted supremum norm nor an lp type norm but involves the derivative of the observable as well and hence can be seen as a type of 1 wasserstein distance this turns out to be a suitable approach for infinite dimensional spaces where the usual harris or doeblin conditions which are geared toward total variation convergence often fail to hold in the first part of this paper we consider semigroups that have uniform behavior which one can view its the analog of doeblin s condition we then proceed to study situations where the behavior is not so uniform but the system has a suitable lyapunov structure leading to a type of harris condition we finally show that the latter condition is satisfied by the two dimensional stochastic navier stokes equations even in situations where the forcing is extremely de generate using the convergence result we show that the stochastic navier stokes equations invariant measures depend continuously on the viscosity and the structure of the forcing

neural networks with local receptive fields and superlinear vc dimension local receptive field neurons comprise such well known and widely used unit types as radial basis function rbf neurons and neurons with center surround receptive field we study the vapnik chervonenkis vc dimension of feedforward neural networks with one hidden layer of these units for several variants of local receptive field neurons we show that the vc dimension of these networks is superlinear in particular we establish the bound w log k for any reasonably sized network with w parameters and k hidden nodes this bound is shown to hold for discrete center surround receptive field neurons which are physiologically relevant models of cells in the mammalian visual system for neurons computing a difference of gaussians which are popular in computational vision and for standard rbf neurons a major alternative to sigmoidal neurons in artificial neural networks the resutlt for rbf neural networks is of particular interest since it answers a question that has been open for several years the results also give rise to lower bounds for networks with fixed input dimension regarding constants all bounds are larger than those known thus far for similar architectures with sigmoidal neurons the superlinear lower bounds contrast with linear upper bounds for single local receptive field neurons also derived here

a general lower bound on the number of examples needed for learning abstract we prove a lower bound of 1 ln 1 vcdim c on the number of random examples required for distribution free learning of a concept class c where vcdim c is the vapnik chervonenkis dimension and and are the accuracy and confidence parameters this improves the previous best lower bound of 1 ln 1 vcdim c and comes close to the known general upper bound of o 1 ln 1 vcdim c ln 1 for consistent algorithms we show that for many interesting concept classes including k cnf and k dnf our bound is actually tight to within a constant factor

learnability and the vapnik chervonenkis dimension valiant s learnability model is extended to learning classes of concepts defined by regions in euclidean space e n the methods in this paper lead to a unified treatment of some of valiant s results along with previous results on distribution free convergence of certain pattern recognition algorithms it is shown that the essential condition for distribution free learnability is finiteness of the vapnik chervonenkis dimension a simple combinatorial parameter of the class of concepts to be learned using this parameter the complexity and closure properties of learnable classes are analyzed and the necessary and sufficient conditions are provided for feasible learnability

exponential mixing for stochastic pdes the non additive case we establish a general criterion which ensures exponential mixing of parabolic stochastic partial differential equations spde driven by a non additive noise which is white in time and smooth in space we apply this criterion on two representative examples 2d navier stokes ns equations and complex ginzburg landau cgl equation with a locally lipschitz noise due to the possible degeneracy of the noise doob theorem cannot be applied hence a coupling method is used in the spirit of kuksin and shirikyan j math pures appl 1 567 602 2002 and mattingly commun math phys 230 421 462 2002

optimization approaches to supervised classification abstract the supervised classification problem one of the oldest and most recurrent problems in applied data analysis has always been analyzed from many different perspectives when the emphasis is placed on its overall goal of developing classification rules with minimal classification cost supervised classification can be understood as an optimization problem on the other hand when the focus is in modeling the uncertainty involved in the classification of future unknown entities it can be formulated as a statistical problem other perspectives that pay particular attention to pattern recognition and machine learning aspects of supervised classification have also a long history that has lead to influential insights and different methodologies in this review two approaches to supervised classification strongly related to optimization theory will be discussed and compared in particular we will review methodologies based on mathematical programming models that optimize observable criteria linked to the true objective of misclassification error or cost minimization and approaches derived from the minimization of known bounds on the true misclassification error the former approach is known as the mathematical programming approach to supervised classification while the latter is in the origin of the well known classification support vector machines throughout the review two group as well as general multi group problems will be considered and the review will conclude with a discussion of the most promising research directions in this area

automatic pattern recognition a study of the probability of error a test sequence is used to select the best rule from a class of discrimination rules defined in terms of the training sequence the vapnik chervonenkis and related inequalities are used to obtain distribution free bounds on the difference between the probability of error of the selected rule and the probability of error of the best rule in the given class the bounds are used to prove the consistency and asymptotic optimality for several popular classes including linear discriminators nearest neighbor rules kernel based rules histogram rules binary tree classifiers and fourier series classifiers in particular the method can be used to choose the smoothing parameter in kernel based rules to choose k in the k nearest neighbor rule and to choose between parametric and nonparametric rules

the dynamic phi 4_3 φ 3 4 model comes down from infinity we prove an a priori bound for the dynamic phi 4 3 model on the torus which is independent of the initial condition in particular this bound rules out the possibility of finite time blow up of the solution it also gives a uniform control over solutions at large times and thus allows one to construct invariant measures via the krylov bogoliubov method it thereby provides a new dynamic construction of the euclidean phi 4 3 field theory on finite volume our method is based on the local in time solution theory developed recently by gubinelli imkeller perkowski and catellier chouk the argument relies entirely on deterministic pde arguments such as embeddings of besov spaces and interpolation which are combined to derive energy inequalities

universal donsker classes and metric entropy when mathfrak f is a universal donsker class then for independent indetically distributed i i d observation mathbf x 1 ldots mathbf x n with an unknown law p for any mathfrak f i in mathfrak f i 1 ldots m quad n 1 2 left mathfrak f 1 left mathbf x 1 right ldots mathfrak f i left mathbf x n right right 1 leq i leq m is asymptotically normal with mean vector n 1 2 left int mathfrak f i left mathbf x n right d mathbf p left x right right 1 leq i leq m and covariance matrix int mathfrak f i mathfrak f j d mathbf p int mathfrak f id mathbf p int mathfrak f jd mathbf p uniformly for mathfrak f i in mathfrak f then for certain statistics formed frome the mathfrak f i left mathbf x k right even where mathfrak f i may be chosen depending on the mathbf x k there will be asymptotic distribution as n rightarrow infty for example for mathbf x 2 statistics where f i are indicators of disjoint intervals depending suitably on mathbf x 1 ldots mathbf x n whose union is the real line mathbf x 2 quadratic forms have limiting distributions roy 1956 and watson 1958 which may however not be mathbf x 2 distributions and may depend on p chernoff and lehmann 1954 universal donsker classes of sets are up to mild measurability conditions just classes satisfying the vapnik cervonenkis comdinatorial conditions defined later in this section donsker the vapnik cervonenkis combinatorial conditions defined later in this section durst and dudley 1981 and dudley 1984 chapter 11 the use of such classes allows a variety of extensions of the roy watson results to general multidimensional sample spaces pollard 1979 and moore and subblebine 1981 vapnik and cervonenkis 1974 indicated application of their families of sets to classification pattern recognition problems more recently the classes have been applied to tree structured classifiacation breiman friedman olshen and stone 1984 chapter 12

stability and generalization we define notions of stability for learning algorithms and show how to use these notions to derive generalization error bounds based on the empirical error and the leave one out error the methods we use can be applied in the regression framework as well as in the classification one when the classifier is obtained by thresholding a real valued function we study the stability properties of large classes of learning algorithms such as regularization based algorithms in particular we focus on hilbert space regularization and kullback leibler regularization we demonstrate how to apply the results to svm for regression and classification

exponential mixing properties of stochastic pdes through asymptotic coupling we consider parabolic stochastic partial differential equations driven by white noise in time we prove exponential convergence of the transition probabilities towards a unique invariant measure under suitable conditions these conditions amount essentially to the fact that the equation transmits the noise to all its determining modes several examples are investigated including some where the noise does not act on every determining mode directly

polynomial certificates for propositional classes this paper studies the complexity of learning classes of expressions in propositional logic from equivalence queries and membership queries in particular we focus on bounding the number of queries that are required to learn the class ignoring computational complexity this quantity is known to be captured by a combinatorial measure of concept classes known as the certificate complexity the paper gives new constructions of polynomial size certificates for monotone expressions in conjunctive normal form cnf for unate cnf functions where each variable affects the function either positively or negatively but not both ways and for horn cnf functions lower bounds on certificate size for these classes are derived showing that for some parameter settings the new certificate constructions are optimal finally the paper gives an exponential lower bound on the certificate size for a natural generalization of these classes known as renamable horn cnf functions thus implying that the class is not learnable from a polynomial number of queries

towards more practical average bounds on supervised learning in this paper we describe a method which enables us to study the average generalization performance of learning directly via hypothesis testing inequalities the resulting theory provides a unified viewpoint of average case learning curves of concept learning and regression in realistic learning problems not necessarily within the bayesian framework the advantages of the theory are that it alleviates the practical pessimism frequently claimed for the results of the vapnik chervonenkis vc theory and its alike and provides general insights into generalization besides the bounds on learning curves are directly related to the number of adjustable system weights although the theory is based on an approximation assumption and cannot apply to the worst case learning setting the precondition of the assumption is mild and the approximation itself is only a sufficient condition for the validity of the theory we illustrate the results with numerical simulations and apply the theory to examining the generalization ability of combination of neural networks

sample size lower bounds in pac learning by algorithmic complexity theory this paper focuses on a general setup for obtaining sample size lower bounds for learning concept classes under fixed distribution laws in an extended pac learning framework these bounds do not depend on the running time of learning procedures and are information theoretic in nature they are based on incompressibility methods drawn from kolmogorov complexity and algorithmic probability theories

a coupling approach to randomly forced nonlinear pde s ii we consider a class of discrete time random dynamical systems and establish the exponential convergence of its trajectories to a unique stationary measure the result obtained applies in particular to the 2d navier stokes system and multidimensional complex ginzburg landau equation with random kick force

exponential convergence for the stochastically forced navier stokes equations and other partially dissipative dynamics we prove that the two dimensional navier stokes equations possess an exponentially attracting invariant measure this result is in fact the consequence of a more general harris like ergodic theorem applicable to many dissipative stochastic pdes and stochastic processes with memory a simple iterated map example is also presented to help build intuition and showcase the central ideas in a less encumbered setting to analyze the iterated map a general doeblin like theorem is proven one of the main features of this paper is the novel coupling construction used to examine the ergodic theory of the non markovian processes

additive preservers of rank on alternate matrix spaces over fields and applications abstract suppose f is any field and n is an integer with n 4 let k n f be the set of all n n alternate matrices over f and let k n f be the non associative ring formed by k n f under the usual addition and the multiplication defined by x y xyx for all x y k n f a pair of n n matrices a b is said to be rank additive if rank a b rank a rank b and rank subtractive if rank a b rank a rank b we say that an operator k n f k n f is additive if x y x y for any x y k n f a preserver of rank additivity respectively rank subtractivity on k n f if it preserves the set of all rank additive respectively rank subtractive pairs a preserver of rank on k n f if rank x rank x for every x k n f and a ring endomorphism of k n f if it is additive and satisfies x y x y for any x y k n f we determine the general form of all additive preservers of rank respectively rank additivity and rank subtractivity on k n f and characterize all ring endomorphisms of k n f

ergodicity and exponential mixing of the real ginzburg landau equation with a degenerate noise abstract in this paper we establish the existence uniqueness and attraction properties of an invariant measure for the real ginzburg landau equation in the presence of a degenerate stochastic forcing acting only in four directions the main challenge is to establish time asymptotic smoothing properties of the markovian dynamics corresponding to this system to achieve this we propose a condition which only requires four noises

exponential ergodicity for stochastic burgers and 2d navier stokes equations abstract it is shown that transition measures of the stochastic navier stokes equation in 2 d converge exponentially fast to the corresponding invariant measures in the distance of total variation as a corollary we obtain the existence of spectral gap for a related semigroup obtained by a sort of ground state transformation analogous results are proved for the stochastic burgers equation

additive preservers of rank additivity on the spaces of symmetric and alternate matrices denote the set of n n symmetric matrices resp alternate matrices over a field f by sn f resp kn f we characterize the bijective additive preservers of rank additivity on sn f and kn f the set sn f resp kn f forms a non associative ring with respect to the addition a b a b and the multiplication a b a b aba where a b sn f resp kn f the forms of any automorphism of the two non associative rings are also given

learning regular sets from queries and counterexamples the problem of identifying an unknown regular set from examples of its members and nonmembers is addressed it is assumed that the regular set is presented by a minimamy adequate teacher which can answer membership queries about the set and can also test a conjecture and indicate whether it is equal to the unknown set and provide a counterexample if not a counterexample is a string in the symmetric difference of the correct set and the conjectured set a learning algorithm l is described that correctly learns any regular set from any minimally adequate teacher in time polynomial in the number of states of the minimum dfa for the set and the maximum length of any counterexample provided by the teacher it is shown that in a stochastic setting the ability of the teacher to test conjectures may be replaced by a random sampling oracle ex a polynomial time learning algorithm is shown for a particular problem of context free language identification

ergodic theory of infinite dimensional systems with applications to dissipative parabolic pdes we consider a class of randomly perturbed dynamical systems satisfying conditions which reflect the properties of general nonlinear dissipative parabolic pdes results on invariant measures and their exponential mixing properties are proved and applications to 2d navier stokes systems are included

inferring decision trees using the minimum description length principle abstract we explore the use of rissanen s minimum description length principle for the construction of decision trees empirical results comparing this approach to other methods are given

how to use expert advice we analyze algorithms that predict a binary value by combining the predictions of several prediction strategies called experts our analysis is for worst case situations i e we make no assumptions about the way the sequence of bits to be predicted is generated we measure the performance of the algorithm by the difference between the expected number of mistakes it makes on the bit sequence and the expected number of mistakes made by the best expert on this sequence where the expectation is taken with respect to the randomization in the predictins we show that the minimum achievable difference is on the order of the square root of the number of mistakes of the best expert and we give efficient algorithms that achieve this our upper and lower bounds have matching leading constants in most cases we then show how this leads to certain kinds of pattern recognition learning algorithms with performance bounds that improve on the best results currently know in this context we also compare our analysis to the case in which log loss is used instead of the expected number of mistakes

linear additive preservers of ranks 2 and 4 on alternate matrix spaces over fields abstract let kn f be the linear space of all n n alternate matrices over a field f an operator f kn f kn f is said to be additive if f a b f a f b for any a b kn f linear if f is additive and f aa af a for every a f and a kn f and a preserver of ranks 2 and 4 on kn f if rank f x rank x for every x kn f with rank x 2 or 4 when n 4 we characterize all linear respectively additive preservers of ranks 2 and 4 on kn f over any field respectively any field that is not isomorphic to a proper subfield of itself furthermore it is also shown that the condition f is not isomorphic to a proper subfield of itself is necessary for the obtained conclusion on additive preservers of ranks 2 and 4 on kn f

minimum polygonal separation abstract in this paper we study the problem of polygonal separation in the plane i e finding a convex polygon with minimum number k of sides separating two given finite point sets k separator if it exists we show that for k n n log n is a lower bound to the running time of any algorithm for this problem and exhibit two algorithms of distinctly different flavors the first relies on an o n log n time preprocessing task which constructs the convex hull of the internal set and a nested star shaped polygon determined by the external set the k separator is contained in the annulus between the boundaries of these two polygons and is constructed in additional linear time the second algorithm adapts the prune and search approach and constructs in each iteration one side of the separator its running time is o kn but the separator may have one more side than the minimum

quantifying inductive bias ai learning algorithms and valiant s learning framework abstract we show that the notion of inductive bias in concept learning can be quantified in a way that directly relates to learning performance in the framework recently introduced by valiant our measure of bias is based on the growth function introduced by vapnik and chervonenkis and on the vapnik chervonenkis dimension we measure some common language biases including restriction to conjunctive concepts conjunctive concepts with internal disjunction k dnf and k cnf concepts we also measure certain types of bias that result from a preference for simpler hypotheses using these bias measurements we analyze the performance of the classical learning algorithm for conjunctive concepts from the perspective of valiant s learning framework we then augment this algorithm with a hypothesis simplification routine that uses a greedy heuristic and show how this improves learning performance on simpler target concepts improved learning algorithms are also developed for conjunctive concepts with internal disjunction k dnf and k cnf concepts we show that all our algorithms are within a logarithmic factor of optimal in terms of the number of examples they require to achieve a given level of learning performance in the valiant framework our results hold for arbitrary attribute based instance spaces defined by either tree structured or linear attributes

computational limitations on learning from examples the computational complexity of learning boolean concepts from examples is investigated it is shown for various classes of concept representations that these cannot be learned feasibly in a distribution free sense unless r np these classes include a disjunctions of two monomials b boolean threshold functions and c boolean formulas in which each variable occurs at most once relationships between learning of heuristics and finding approximate solutions to np hard optimization problems are given

some general techniques on linear preserver problems abstract several general techniques on linear preserver problems are described the first one is based on a transfer principle in model theoretic algebra that allows one to extend linear preserver results on complex matrices to matrices over other algebraically closed fields of characteristic 0 the second one concerns the use of some simple geometric technique to reduce linear preserver problems to standard types so that known results can be applied the third one is about solving linear preserver problems on more general operator algebras by reducing the problems to idempotent preservers numerous examples will be given to demonstrate the proposed techniques

efficient learning with virtual threshold gates abstract we reduce learning simple geometric concept classes to learning disjunctions over exponentially many variables we then apply an online algorithm called winnow whose number of prediction mistakes grows only logarithmically with the number of variables the hypotheses of winnow are linear threshold functions with one weight per variable we find ways to keep the exponentially many weights of winnow implicitly so that the time for the algorithm to compute a prediction and update its virtual weights is polynomial our method can be used to learnd dimensional axis parallel boxes whendis variable and unions ofd dimensional axis parallel boxes whendis constant the worst case number of mistakes of our algorithms for the above classes is optimal to within a constant factor and our algorithms inherit the noise robustness of winnow we think that other online algorithms with multiplicative weight updates whose loss bounds grow logarithmically with the dimension are amenable to our methods