optimal 2 cyclic msor for bowtie spectra and the continuous manteuffel algorithm abstract for the numerical solution of a linear system whose matrix coefficient is block 2 cyclic consistently ordered with the eigenvalues of the associated block jacobi matrix lying in a bowtie region several efficient stationary iterative methods have been proposed among others by chin and manteuffel 1988 elman and golub 1990 de pillis 1991 and eiermann niethammer and varga 1992 we propose as an alternative the stationary modified successive overrelaxation msor method or an equivalent two step method applied to the cyclically reduced linear system it is shown both theoretically and experimentally that the application of a continuous version of manteuffel s algorithm to derive the optimal parameters of the two step method produces an iterative method that is asymptotically faster than all the aforementioned methods

learning by distances a model of learning by distances is presented in this model a concept is a point in a metric space at each step of the learning process the student guesses a hypothesis and receives from the teacher an approximation of its distance to the target a notion of a distance measuring the proximity of a hypothesis to the correct answer is common to many models of learnability by focusing on this fundamental aspect we discover some general and simple tools for the analysis of learnability tasks as a corollary we present new learning algorithms for valiant s pac scenario with any given distribution these algorithms can learn any pac learnable class and in some cases settle for significantly less information than the usual labeled examples insight gained by the new model is applied to show that every class of subsets c that has a finite vc dimension is pac learnable with respect to any fixed distribution previously known results of this nature were subject to complicated measurability constraints

the query complexity of finding local minima in the lattice in this paper we study the query complexity of finding local minimum points of a boolean function this task occurs frequently in exact learning algorithms for many natural classes such as monotone dnf o log n term dnf unate dnf and decision trees on the negative side we prove that any possibly randomized algorithm that produces a local minimum of a function f chosen from a sufficiently rich concept class using a membership oracle for f must ask n2 membership queries in the worst case in particular this lower bound applies to the class of decision trees a simple algorithm is known that achieves this lower bound on the positive side we show that for the class o log n term dnf finding local minimum points requires only n log n membership queries and more generally tn membership queries for t term dnf with tn this efficient procedure improves the time and query complexity of known learning algorithms for the class o log n term dnf 2001 elsevier science

optimal parameters for linear second degree stationary iterative methods in this paper we show that the optimal parameters for linear second degree stationary iterative methods applied to nonsymmetric linear systems can be found by solving the same minimax problem used to find optimal parameters for the chebyshev iteration in fact the chebyshev iteration is asymptotically equivalent to a linear seconnd degree stationary method the method of finding optimal parameters for the chebyshev iteration given in manteuffel numer math 28 1977 pp 307 327 can be used to find optimal parameters for the sitationary method as well

on learning read k satisfy j dnf we study the learnability of read k satisfy j rksj dnf formulas these are boolean formulas in disjunctive normal form dnf in which the maximum number of occurrences of a variable is bounded by k and the number of terms satisfied by any assignment is at most j after motivating the investigation of this class of dnf formulas we present an algorithm that for any unknown rksj dnf formula to be learned with high probability finds a logically equivalent dnf formula using the well studied protocol of equivalence and membership queries the algorithm runs in polynomial time for k cdot j o log n over log log n where n is the number of input variables

approximate minimization of weighted tree automata abstract this paper studies the following approximate minimization problem given a minimal weighted tree automaton a with n states recognizing a weighted tree language f can we construct a smaller automaton a with n n states recognizing a language f that is a good approximation of f the corresponding problem for weighted automata on strings was recently studied by balle et al 16 15 where the authors introduced a new canonical form for weighted automata called singular value automata inspired by spectral methods and showed that truncating such canonical form yields a solution for the problem satisfying a certain approximation criteria in this paper we take a similar approach and show that in the tree case one can obtain an analogous canonical form that we call singular value tree automata and use it to study the approximate minimization problem for weighted tree automata we first establish the existence of this canonical form for weighted tree automata and then provide bounds on the quality of the resulting approximation method based on truncation we also study the problem of computing the canonical form given a minimal weighted tree automaton and show that in the tree case this task is considerably more complicated than in the string case in particular computing the canonical form reduces to solving a system of polynomial equations by further reducing this problem to the computation of generalized partition functions for weighted tree automata we propose and analyze two methods for computing the canonical form based on iterative methods fixed point iteration and newton s method our analysis of newton s method unveils a connection between iterative methods and sequences of sets of trees satisfying a certain technical condition that might be of independent interest

preference elicitation and query learning in this paper we explore the relationship between preference elicitation a learning style problem that arises in combinatorial auctions and the problem of learning via queries studied in computational learning theory preference elicitation is the process of asking questions about the preferences of bidders so as to best divide some set of goods as a learning problem it can be thought of as a setting in which there are multiple target concepts that can each be queried separately but where the goal is not so much to learn each concept as it is to produce an optimal example in this work we prove a number of similarities and differences between two bidder preference elicitation and query learning giving both separation results and proving some connections between these problems

on the rate of convergence of regularized boosting classifiers a regularized boosting method is introduced for which regularization is obtained through a penalization function it is shown through oracle inequalities that this method is model adaptive the rate of convergence of the probability of misclassification is investigated it is shown that for quite a large class of distributions the probability of error converges to the bayes risk at a rate faster than n v 2 4 v 1 where v is the vc dimension of the base class whose elements are combined by boosting methods to obtain an aggregated classifier the dimension independent nature of the rates may partially explain the good behavior of these methods in practical problems under tsybakov s noise condition the rate of convergence is even faster we investigate the conditions necessary to obtain such rates for different base classes the special case of boosting using decision stumps is studied in detail we characterize the class of classifiers realizable by aggregating decision stumps it is shown that some versions of boosting work especially well in high dimensional logistic additive models it appears that adding a limited labelling noise to the training data may in certain cases improve the convergence as has been also suggested by other authors

polynomial inference of universal automata from membership and equivalence queries a mat learning algorithm is presented that infers the universal automaton ua for a regular target language using a polynomial number of queries with respect to that automaton the ua is one of several canonical characterizations for regular languages our learner is based on the concept of an observation table which seems to be particularly fitting for this computational model and the necessary definitions are adapted from the literature to the case of ua

inductive synthesis of recursive processes from logical properties this paper proposes an inductive synthesis algorithm for a recursive process to synthesize a process facts which must be satisfied by the target process are given to the algorithm one by one since such facts are infinitely many in general when n facts are input to the algorithm it outputs a process which satisfies the given n facts and this generating process is repeated infinitely many times to represent facts of a process we adopt a subcalculus of calculus first we introduce a new preorder d on recursive processes based on the subcalculus to discuss its properties p dq means that p f implies q f for all formulae f in the subcalculus then its discriminative power and relationship with other preorders are also discussed finally we present the synthesis algorithm the correctness of the algorithm can be stated that the output sequence of processes by the algorithm converges to a process which cannot be distinguished from the intended one if we could know it by a given enumeration of facts in the limit a prototype system based on the algorithm is stated as well

a comparison of numerical models for one dimensional stefan problems in this paper we present a critical comparison of the suitability of several numerical methods level set moving grid and phase field model to address two well known stefan problems in phase transformation studies melting of a pure phase and diffusional solid state phase transformations in a binary system similarity solutions are applied to verify the numerical results the comparison shows that the type of phase transformation considered determines the convenience of the numerical techniques finally it is shown both numerically and analytically that the solid solid phase transformation is a limiting case of the solid liquid transformation

regular ω languages with an informative right congruence abstract a regular language is almost fully characterized by its right congruence relation the same does not hold for regular languages the right congruence of a regular language may not be informative enough many regular languages have a trivial right congruence and in general it is not always possible to define an automaton recognizing a given language that is isomorphic to its right congruence the weak regular languages do have fully informative right congruences that is any weak regular language can always be recognized by a deterministic b u chi automaton that is isomorphic to its right congruence weak regular languages reside in the lower levels of the expressiveness hierarchy of regular languages are there more expressive sub classes of regular languages that have fully informative right congruences can we characterize the class of languages that have trivial right congruences in this paper we try to place some additional pieces of this big puzzle

polynomial time identification of very simple grammars from positive data this paper concerns a subclass of simple deterministic grammars called very simple grammars and studies the problem of identifying the subclass in the limit from positive data the class of very simple languages forms a proper subclass of simple deterministic languages and is incomparable to the class of regular languages this class of languages is also known as the class of left szilard languages of context free grammars after providing some properties of very simple languages we show that the class of very simple grammars is polynomial time identifiable in the limit from positive data in the following sense that is we show that there effectively exists an algorithm that given a target very simple grammar g over alphabet identifies a very simple grammar g equivalent to g in the limit from positive data satisfying the property that the time for updating a conjecture is bounded by o m and the total number of prediction errors made by the algorithm is bounded by o n where n is the size of g m max n 1 3 and n is the total length of all positive data provided

fuzzy perceptron neural networks for classifiers with numerical data and linguistic rules as inputs this paper presents a novel learning algorithm of fuzzy perceptron neural networks fpnns for classifiers that utilize expert knowledge represented by fuzzy if then rules as well as numerical data as inputs the conventional linear perceptron network is extended to a second order one which is much more flexible for defining a discriminant function in order to handle fuzzy numbers in neural networks level sets of fuzzy input vectors are incorporated into perceptron neural learning at different levels of the input fuzzy numbers updating the weight vector depends on the minimum of the output of the fuzzy perceptron neural network and the corresponding nonfuzzy target output that indicates the correct class of the fuzzy input vector this minimum is computed efficiently by employing the modified vertex method moreover the fuzzy pocket algorithm is introduced into our fuzzy perceptron learning scheme to solve the nonseparable problems simulation results demonstrate the effectiveness of the proposed fpnn model

fuzzy analysis of statistical evidence bayesian classifiers are effective methods for pattern classification although their assumptions on the belief structure among attributes are not always justified in this paper we introduce a new classification method based on the possibility measure which does not require a precise belief model and in a sense it includes the bayesian classifiers as special cases this new classification method uses the fuzzy operators to aggregate attributes information evidence and it is referred to as fuzzy analysis of statistical evidence fase fase has several nice properties it is noise tolerant it can handle missing values with ease and it can extract statistical patterns from the data and represent them by knowledge of beliefs which in turn are propositions for an expert system thus from pattern classification to expert systems fase provides a linkage from inductive reasoning to deductive reasoning

new horn revision algorithms a revision algorithm is a learning algorithm that identifies the target concept starting from an initial concept such an algorithm is considered efficient if its complexity in terms of the measured resource is polynomial in the syntactic distance between the initial and the target concept but only polylogarithmic in the number of variables in the universe we give efficient revision algorithms in the model of learning with equivalence and membership queries the algorithms work in a general revision model where both deletion and addition revision operators are allowed in this model one of the main open problems is the efficient revision of horn formulas two revision algorithms are presented for special cases of this problem for depth 1 acyclic horn formulas and for definite horn formulas with unique heads

parameterized learnability of juntas we study the parameterized complexity of learning k juntas and some variations of juntas we show the hardness of learning k juntas and subclasses of k juntas in the pac model by reductions from a w 2 complete problem on the other hand as a consequence of a more general result we show that k juntas are exactly learnable with improper equivalence queries and access to a w p oracle

discrete time model reference adaptive control of nonlinear dynamical systems using neural networks a multilayer discrete time neural net nn controller is presented for the direct model reference adaptive control of a class of multi input multi output mim0 nonlinear dynamical systems the nonlinear dynamical system is assumed to be controllable and its state vector is available for measurement the nn controller exhibits learning while functioning features instead of learning then control so that control is immediate with no explicit learning phase needed the tracking error between the output of a nonlinear plant and an ideal linear model converges within a very short time persistence of excitation pe is not needed linearity in the parameters is not required and certainty equivalence is not used this overcomes several limitations of standard adaptive control the novel weight tuning paradigm for the nn controller is based on the well known delta rule but includes a modification to the learning rate parameter plus a correction term it guarantees tracking as well as bounded nn weights in non i

read twice dnf formulas are properly learnable we show that read twice dnf formulas boolean formulas in disjunctive normal form in which each variable appears at most twice are exactly and properly learnable in polynomial time our algorithm uses membership queries and proper equivalence queries and is based on a simple new characterization of minimal read twice dnf formulas the algorithm improves on earlier results of hancock and aizenstein and pitt which showed that read twice dnf formulas are learnable using more powerful equivalence queries i e where the hypotheses could be arbitrary dnf formulas we also improve on the time complexity of these earlier algorithms other results which may be of independent interest outside of learning follow directly from this paper specifically we show that read twice dnf formulas can be tested for equivalence in polynomial time and that the smallest read twice formula equivalent to a given one can be found in polynomial time

four one shot learners for regular tree languages and their polynomial characterizability we survey two existing algorithms for the inference of finite state tree automata from membership queries and a finite positive sample or equivalence queries and we suggest a reformulation of one of them which we deem necessary to ensure its termination we present two algorithms for the same two settings when the underlying description is not a deterministic but a residual canonical finite state automaton to this end we adapt all necessary notions to the residual tree case from our completed perspective we discuss the terms on which those four algorithms can be considered to be of polynomial complexity and also where there may be hidden exponentiality

learning equilibria of games via payoff queries a recent body of experimental literature has studied empirical game theoretical analysis in which we have partial knowledge of a game consisting of observations of a subset of the pure strategy profiles and their associated payoffs to players the aim is to find an exact or approximate nash equilibrium of the game based on these observations it is usually assumed that the strategy profiles may be chosen in an on line manner by the algorithm we study a corresponding computational learning model and the query complexity of learning equilibria for various classes of games we give basic results for exact equilibria of bimatrix and graphical games we then study the query complexity of approximate equilibria in bimatrix games finally we study the query complexity of exact equilibria in symmetric network congestion games for directed acyclic networks we can learn the cost functions and hence compute an equilibrium while querying just a small fraction of pure strategy profiles for the special case of parallel links we have the stronger result that an equilibrium can be identified while only learning a small fraction of the cost values

automatic learning of subclasses of pattern languages automatic classes are classes of languages for which a finite automaton can decide the membership problem for the languages in the class in a uniform way given an index for the language for alphabet size of at least 4 every automatic class of erasing pattern languages is contained for some constant n in the class of all languages generated by patterns which contain 1 every variable only once and 2 at most n symbols after the first occurrence of a variable it is shown that such a class is automatically learnable using a learner with the length of the long term memory being bounded by the length of the first example seen the study is extended to show the learnability of related classes such as the class of unions of two pattern languages of the above type

learning expressions and programs over monoids we study the problem of learning an unknown function represented as an expression or a program over a known finite monoid as in other areas of computational complexity where programs over algebras have been used the goal is to relate the computational complexity of the learning problem with the algebraic complexity of the finite monoid indeed our results indicate a close connection between both kinds of complexity we focus on monoids which are either groups or aperiodic and on the learning model of exact learning from queries for a group g we prove that expressions over g are efficiently learnable if g is nilpotent and impossible to learn efficiently under cryptographic assumptions if g is nonsolvable we present some results for restricted classes of solvable groups and point out a connection between their efficient learnability and the existence of lower bounds on their computational power in the program model for aperiodic monoids our results seem to indicate that the monoid class known as da captures exactly learnability of expressions by polynomially many evaluation queries when using programs instead of expressions we show that our results for groups remain true while the situation is quite different for aperiodic monoids

the minimum consistent dfa problem cannot be approximated within any polynomial the minimum consistent dfa problem is that of finding a dfa with as few states as possible that is consistent with a given sample a finite collection of words each labeled as to whether the dfa found should accept or reject assuming that p np it is shown that for any constant k no polynomial time algorithm can be guaranteed to find a consistent dfa with fewer than opt k states where opt is the number of states in the minimum state dfa consistent with the sample this result holds even if the alphabet is of constant size two and if the algorithm is allowed to produce an nfa a regular expression or a regular grammar that is consistent with the sample a similar nonapproximability result is presented for the problem of finding small consistent linear grammars for the case of finding minimum consistent dfas when the alphabet is not of constant size but instead is allowed to vary with the problem specification the slightly stronger lower bound on approximability of opt 1 e log log opt is shown for any e 0

learning of event recording automata in regular inference a regular language is inferred from answers to a finite set of membership queries each of which asks whether the language contains a certain word one of the most well known regular inference algorithms is the l algorithm due to dana angluin however there are almost no extensions of these algorithms to the setting of timed systems we extend angluin s algorithm for on line learning of regular languages to the setting of timed systems since timed automata can freely use an arbitrary number of clocks we restrict our attention to systems that can be described by deterministic event recording automata deras we present three algorithms tl s g tl n s g and tl s for inference of deras in tl s g and tl n s g we further restrict event recording automata to be event deterministic in the sense that each state has at most one outgoing transition per action learning such an automaton becomes significantly more tractable the algorithm tl n s g builds on tl s g by attempts to construct a smaller in number of locations automaton finally tl s is a learning algorithm for a full class of deterministic event recording automata which infers a so called simple dera which is similar in spirit to the region graph

polynomial time recognition of 2 monotonic positive boolean functions given by an oracle we consider the problem of identifying an unknown boolean function f by asking an oracle the functional values f a for a selected set of test vectors a in 0 1 n furthermore we assume that f is a positive or monotone function of n variables it is not yet known whether or not the whole task of generating test vectors and checking if the identification is completed can be carried out in polynomial time in n and m where m min t f max f f and min t f respectively max f f denotes the set of minimal true respectively maximal false vectors of f to partially answer this question we propose here two polynomial time algorithms that given an unknown positive function f of n variables decide whether or not f is 2 monotonic and if f is 2 monotonic output both sets min t f and max f f the first algorithm uses o nm 2 n 2 m time and o nm queries while the second one uses o n 3 m time and o n 3 m queries

feedback stabilization using two hidden layer nets the representational capabilities of one hidden layer and two hidden layer nets consisting of feedforward interconnections of linear threshold units are compared it is remarked that for certain problems two hidden layers are required contrary to what might be in principle expected from the known approximation theorems the differences are not based on numerical accuracy or number of units needed nor on capabilities for feature extraction but rather on a much more basic classification into direct and inverse problems the former correspond to the approximation of continuous functions while the latter are concerned with approximating one sided inverses of continuous functions and are often encountered in the context of inverse kinematics determination or in control questions a general result is given showing that nonlinear control systems can be stabilized using two hidden layers but not in general using just one

map learning with uninterpreted sensors and effectors this dissertation presents a set of methods by which a learning agent called a critter can learn a sequence of increasingly abstract and powerful interfaces to control a robot whose sensorimotor apparatus and environment are initially unknown the result of the learning is a rich hierarchical model of the robot s world its sensorimotor apparatus and environment the learning methods rely on generic properties of the robot s world such as almost everywhere smooth effects of actions on sensory features at the lowest level of the hierarchy the critter analyzes the effects of its actions in order to define control signals one for each of the robot s degrees of freedom it uses a generate and test approach to define sensory features that capture important aspects of the environment it uses linear regression to learn action models that characterize context dependent effects of the control signals on the learned features it uses these models to define high level control laws for finding and following paths defined using constraints on the learned features the critter abstracts these control laws which interact with the continuous environment to a finite set of actions that implement discrete state transitions at this point the critter has abstracted the robot s world to a finite state machine and can use existing methods to learn its structure

approximation dimension reduction and nonconvex optimization using linear superpositions of gaussians this paper concerns neural network approaches to function approximation and optimization using linear superposition of gaussians or what are popularly known as radial basis function rbf networks the problem of function approximation is one of estimating an underlying function f given samples of the form y sub i x sub i i 1 2 n with y sub i f x sub i when the dimension of the input is high and the number of samples small estimation of the function becomes difficult due to the sparsity of samples in local regions the authors find that this problem of high dimensionality can be overcome to some extent by using linear transformations of the input in the gaussian kernels such transformations induce intrinsic dimension reduction and can be exploited for identifying key factors of the input and for the phase space reconstruction of dynamical systems without explicitly computing the dimension and delay they present a generalization that uses multiple linear projections onto scalars and successive rbf networks mlprbf that estimate the function based on these scaler values they derive some key properties of rbf networks that provide suitable grounds for implementing efficient search strategies for nonconvex optimization within the same framework

learnability and the vapnik chervonenkis dimension valiant s learnability model is extended to learning classes of concepts defined by regions in euclidean space e n the methods in this paper lead to a unified treatment of some of valiant s results along with previous results on distribution free convergence of certain pattern recognition algorithms it is shown that the essential condition for distribution free learnability is finiteness of the vapnik chervonenkis dimension a simple combinatorial parameter of the class of concepts to be learned using this parameter the complexity and closure properties of learnable classes are analyzed and the necessary and sufficient conditions are provided for feasible learnability

some recent results on the modified sor theory abstract for the solution of the linear system ax i t x c 1 where t is a weakly cyclic of index p 2 matrix of a special structure the block sor method with different relaxation factors associated with the row blocks of a is considered first a well known relationship connecting the eigenvalue spectra of the block jacobi matrix t of a and its associated modified sor msor matrix is proved for all p 3 via an approach due to varga niethammer and cai next it is shown that the matrix analogue of the eigenvalue relationship holds at least for p 3 this together with the facts that the matrix analogue holds true for p 2 and also for any p 3 provided all relaxation factors coincide suggests a more general validity of the aforementioned matrix identity then based on the matrix relationship an equivalence is established between the msor method and a stationary p parameter p step iterative one for the solution of 1 so one can study convergence properties and determine optimal parameters of both methods by studying the simpler of the two that is the p step one finally some applications of the theory developed when p 2 are presented and a brief discussion concerning comparisons of the optimum msor and aor methods in a specific case is given

polynomial characteristic sets for dfa identification we study the order in grammatical inference algorithms and its influence on the polynomial with respect to the data identification of languages this work is motivated by recent results on the polynomial convergence of data driven grammatical inference algorithms in this paper we prove a sufficient condition that assures the existence of a characteristic sample whose size is polynomial with respect to the minimum dfa of the target language

polynomial time learning of simple deterministic languages via queries and a representative sample we show that simple deterministic languages are polynomial time learnable via membership queries if the learner knows a special finite set of positive examples this finite set is called a representative sample and has been introduced by angluin inform control 51 1981 to show that regular languages are polynomial time learnable via membership queries if simple deterministic languages are learnable in polynomial time via membership and equivalence queries we can obtain a representative sample of a target language in polynomial time from a correct hypothesis thus our result implies that the polynomial time learning problem of simple deterministic languages via membership and equivalence queries is solvable if and only if we can find a representative sample in polynomial time via these queries we show the learnability of simple deterministic languages by giving a learning algorithm the algorithm at the first stage makes all possible candidate rules to generate the target language and a set of simple deterministic grammars which are little different each other then comparing them the algorithm eliminates inappropriate rules

observer design based on triangular form generated by injective map a procedure is provided for constructing an injective map as a generalized coordinate change for triangular form transformation based on the triangular form a method is developed for designing nonlinear state observers

learning dynamics system identification for perceptually challenged agents from the perspective of an agent the input output behavior of the environment in which it is embedded can be described as a dynamical system inputs correspond to the actions executable by the agent in making transitions between states of the environment outputs correspond to the perceptual information available to the agent in particular states of the environment we view dynamical system identification as inference of deterministic finite state automata from sequences of input output pairs the agent can influence the sequence of input output pairs it is presented by pursuing a strategy for exploring the environment we identify two sorts of perceptual errors errors in perceiving the output of a state and errors in perceiving the inputs actually carried out in making a transition from one state to another we present efficient high probability learning algorithms for a number of system identification problems involving such errors we also present the results of empirical investigations applying these algorithms to learning spatial representations

improved lower bounds for learning from noisy examples this paper presents a general information theoretic approach for obtaining lower bounds on the number of examples required for probably approximately correct pac learning in the presence of noise this approach deals directly with the fundamental information quantities avoiding a bayesian analysis the technique is applied to several different models illustrating its generality and power the resulting bounds add logarithmic factors to or improve the constants in previously known lower bounds

pricing general insurance in a reactive and competitive market a simple parameterisation is introduced which represents the insurance market s response to an insurer adopting a pricing strategy determined via optimal control theory claims are modelled using a lognormally distributed mean claim size rate and the market average premium is determined via the expected value principle if the insurer maximises its expected wealth then the resulting bellman equation has a moving boundary in state space that determines when it is optimal to stop selling insurance this stochastic optimisation problem is simplified by the introduction of a stopping time that prevents an insurer leaving and then re entering the insurance market three finite difference schemes are used to verify the existence of a solution to the resulting bellman equation when there is market reaction all of the schemes use a front fixing transformation if the market reacts then it is found that the optimal strategy is altered in that premiums are raised if the strategy is of loss leading type and lowered if it is optimal for the insurer to set a relatively high premium and sell little insurance

learning behaviors of automata from multiplicity and equivalence queries we consider the problem of identifying the behavior of an unknown automaton with multiplicity in the field ratviii of rational numbers ratviii automaton from multiplicity and equivalence queries we provide an algorithm which is polynomial in the size of the ratviii automaton and in the maximum length of the given counterexamples as a consequence we have that ratviii automata are probably approximately correctly learnable pac learnable in polynomial time when multiplicity queries are allowed a corollary of this result is that regular languages are polynomially predictable using membership queries with respect to the representation of unambiguous nondeterministic automata this is important since there are unambiguous automata such that the equivalent deterministic automaton has an exponentially larger number of states

a sufficient condition to polynomially compute a minimum separating dfa the computation of a minimal separating automaton msa for regular languages has been studied from many different points of view from synthesis of automata or grammatical inference to the minimization of incompletely specified machines or compositional verification in the general case the problem is np complete but this drawback does not prevent the problem from having a real application in the above mentioned fields in this paper we propose a sufficient condition that guarantees that the computation of the msa can be carried out with polynomial time complexity

an analysis of block successive overrelaxation for a class of matrices with complex spectra optimal block successive overrelaxation bsor parameters are derived for a special class of matrices having complex spectra these matrices arise from the central difference approximation to singularly perturbed convection diffusion equations using a coarse grid and thus have large cell reynolds number a simple problem is examined in detail and it is shown that bsor is an excellent method for cell reynolds numbers near 1 and that the rate of convergence is invariant to the cell reynolds number for large cell reynolds number

the analysis ofk step iterative methods for linear systems from summability theory using the theory of euler methods from summability theory we investigate general iterative methods for solving linear systems of equations in particular for a given euler method a regions of the complex plane is determined such that ak step iterative method converges if the eigenvalues of an iteration operatort are contained ins for a givens optimal methods are described and upper and lower bounds are derived for the associated asymptotic rate of convergence special attention is given to two step methods with complex parameters

the power of a pebble exploring and mapping an unknown environment is a fundamental problem that is studied in a variety of contexts many results have focused on finding efficient solutions to restricted versions of the problem in this paper we consider a model that makes very limited assumptions about the environment and solve the mapping problem in this general setting we model the environment by an unknown directed graph g and consider the problem of a robot exploring and mapping g the edges emanating from each vertex are numbered from 1 to d but we do not assume that the vertices of g are labeled since the robot has no way of distinguishing between vertices it has no hope of succeeding unless it is given some means of distinguishing between vertices for this reason we provide the robot with a pebble a device that it can place on a vertex and use to identify the vertex later in this paper we show 1 if the robot knows an upper bound on the number of vertices then it can learn the graph efficiently with only one pebble 2 if the robot does not know an upper bound on the number of vertices n then log log n pebbles are both necessary and sufficient in both cases our algorithms are deterministic

conjunctions of unate dnf formulas learning and structure abstract a central topic in query learning is to determine which classes of boolean formulas are efficiently learnable with membership and equivalence queries we consider the class r kconsisting of conjunctions ofkunate dnf formulas this class generalizes the class ofk clause cnf formulas and the class of unate dnf formulas both of which are known to be learnable in polynomial time with membership and equivalence queries we prove that r 2can be properly learned with a polynomial number of polynomial size membership and equivalence queries but can be properly learned in polynomial time with such queries if and only if p np thus the barrier to properly learning r 2with membership and equivalence queries is computational rather than informational few results of this type are known in our proofs we use recent results of hellersteinet al 1997 j assoc comput mach 43 5 840 862 characterizing the classes that are polynomial query learnable together with work of bshouty on the monotone dimension of boolean functions we extend some of our results to r kand pose open questions on learning dnf formulas of small monotone dimension we also prove structural results for r k we construct for any fixedk 2 a class of functionsfthat cannot be represented by any formula in r k but which cannot be easily shown to have this property more precisely for any functionfonnvariables in the class the value offon any polynomial size set of points in its domain is not a witness thatfcannot be represented by a formula in r k our construction is based on bch codes

discrete time neural net controller for a class of nonlinear dynamical systems a family of two layer discrete time neural net nn controllers is presented for the control of a class of mnth order mimo dynamical system no initial learning phase is needed so that the control action is immediate in other words the neural network nn controller exhibits a learning while functioning feature instead of a learning then control feature a two layer nn is used which is linear in the tunable weights the structure of the neural net controller is derived using a filtered error approach it is indicated that delta rule based tuning when employed for closed loop control can yield unbounded nn weights if 1 the net cannot exactly reconstruct a certain required function or 2 there are bounded unknown disturbances acting on the dynamical system certainty equivalence is not used overcoming a major problem in discrete time adaptive control in this paper new online tuning algorithms for discrete time systems are derived which are similar to spl epsiv modification for the case of continuous time systems that include a modification to the learning rate parameter and a correction term to the standard delta rule

monotone term decision lists we introduce a new representation class of boolean functions monotone term decision lists which combines compact representation size with tractability of essential operations we present many properties of the class which make it an attractive alternative to traditional universal representation classes such as dnf formulas or decision trees we study the learnability of monotone term decision lists in the exact model of equivalence and membership queries we show that for any constant k 0 k term monotone decision lists are exactly and properly learnable with no k membership queries in no k3 time we also show that n k membership queries are necessary for exact learning in contrast both k term monotone decision lists k 2 and general monotone term decision lists are not learnable with equivalence queries alone we also show that a subclass of monotone term decision lists disj mdl is learnable with equivalence and membership queries while neither type of query alone suffices

fast identification of geometric objects with membership queries we investigate the number of membership queries that are needed to identify polygons i e intersections of halfplanes over a two dimensional grid 0 n 1 2 we exhibit a learning algorithm that learns 100 correctly while requiring no random examples and not more membership queries than previous algorithms needed in addition to their random examples for probably almost correctly learning even for moderate values of furthermore the learning algorithm in this paper uses only grid points for its membership queries this appears to be appropriate in situations where the probing device has only a limited resolution and for applications to the set of pixels on a two dimensional screen the learning algorithm that is described in this paper overcomes a fundamental obstacle related to the zigzag borderline that is generated by a halfplane over a discrete grid this obstacle had thwarted previous attempts to show that in angluin s model for on line learning with equivalence and membership queries one can learn in an efficient manner even those classes of geometric objects which cannot be learnt fast by equivalence queries alone such as rectangles in general position

theory revision with queries horn read once and parity formulas a theory in this context is a boolean formula it is used to classify instances or truth assignments theories can model real world phenomena and can do so more or less correctly the theory revision or concept revision problem is to correct a given roughly correct concept this problem is considered here in the model of learning with equivalence and membership queries a revision algorithm is considered efficient if the number of queries it makes is polynomial in the revision distance between the initial theory and the target theory and polylogarithmic in the number of variables and the size of the initial theory the revision distance is the minimal number of syntactic revision operations such as the deletion or addition of literals needed to obtain the target theory from the initial theory efficient revision algorithms are given for horn formulas and read once formulas where revision operators are restricted to deletions of variables or clauses and for parity formulas where revision operators include both deletions and additions of variables we also show that the query complexity of the read once revision algorithm is near optimal

learning in the limit with lattice structured hypothesis spaces we define a collection of language classes which are txtex learnable learnable in the limit from positive data the learners map any data input to an element of a fixed lattice and keep the least upper bound of all lattice elements thus obtained as the current hypothesis each element of the lattice is a grammar for a language and the learner climbs the lattice searching for the right element we call these classes in our collection lattice classes we provide several characterizations of lattice classes and their learners which suggests they are very natural in particular we show that any class of languages is a lattice class iff it is txtex learnable consistently conservatively set drivenly and strongly monotonically we show several language classes previously discussed in the literature to be lattice classes including the locally k testable classes the piecewise k testable classes the k reversible languages and the pattern languages we also show that lattice classes contain three previously known collections of language classes string extension language classes function distinguishable language classes and closed set systems finally the lattice perspective helps analyze the learning of these classes illustrations include query learning results in dependence on the lattice structure characterizations of closure properties and the vc dimension of lattice classes in terms of lattice properties

probabilistic learnability of context free grammars with basic distributional properties from positive examples in recent years different interesting subclasses of cfls have been found to be learnable by techniques generically called distributional learning the theoretical study on the exact learning of cfls by those techniques under different learning schemes is now quite mature on the other hand positive results on the pac learnability of cfls are rather limited and quite weak this paper shows that several subclasses of context free languages that are known to be exactly learnable with positive data and membership queries by distributional learning techniques are pac learnable from positive data under some assumptions on the string distribution

efficient learning of context free grammars from positive structural examples in this paper we introduce a new normal form for context free grammars called reversible context free grammars for the problem of learning context free grammars from positive only examples a context free grammar g n s p s is said to be reversible if 1 a a and b a in p implies a b and 2 a email protected and a email protected in p implies b c we show that the class of reversible context free grammars can be identified in the limit from positive samples of structural descriptions and there exists an efficient algorithm to identify them from positive samples of structural descriptions where a structural description of a context free grammar is an unlabelled derivation tree of the grammar this implies that if positive structural examples of a reversible context free grammar for the target language are available to the learning algorithm the full class of context free languages can be learned efficiently from positive samples

nonlinear regulation the piecewise linear approach this paper approaches nonlinear control problems through the use of discrete time piecewise linear systems these are systems whose next state and output maps are both described by pl maps i e by maps which are affine on each of the components of a finite polyhedral partition various results on state and output feedback observers and inverses standard for linear systems are proved for pl systems many of these results are then used in the study of more general both discrete and continuous time systems using suitable approximations

learning from examples with unspecified attribute values a challenging problem within machine learning is how to make good inferences from data sets in which pieces of information are missing while it is valuable to have algorithms that perform well for specific domains to gain a fundamental understanding of the problem one needs a theory about how to learn with incomplete data the important contribution of such a theory is not so much the specific algorithmic results but rather that it provides good ways of thinking about the problem formally in this paper we introduce the unspecified attribute value uav learning model as a first step towards a theoretical framework for studying the problem of learning from incomplete data in the exact learning framework in the uav learning model an example x is classified positive resp negative if all possible assignments for the unspecified attributes result in a positive resp negative classification otherwise the classification given to x is for unknown given an example x in which some attributes are unspecified the oracle uav mq responds with the classification of x given a hypothesis h the oracle uav eq returns an example x that could have unspecified attributes for which h x is incorrect we show that any class of functions learnable in angluin s exact model using the mq and eq oracles is also learnable in the uav model using the mq and uav eq oracles as long as the counterexamples provided by the uav eq oracle have a logarithmic number of unspecified attributes we also show that any class learnable in the exact model using the mq and eq oracles is also learnable in the uav model using the uav mq and uav eq oracles as well as an oracle to evaluate a given boolean formula on an example with unspecified attributes for some hypothesis classes such as decision trees and unate formulas the evaluation can be done in polynomial time without an oracle we also study the learnability of a universal class of decision trees under the uav model and of dnf formulas under a representation dependent variation of the uav model

predicting neutron diffusion eigenvalues with a query based adaptive neural architecture a query based approach for adaptively retraining and restructuring a two hidden layer artificial neural network ann has been developed for the speedy prediction of the fundamental mode eigenvalue of the neutron diffusion equation a standard nuclear reactor core design calculation which normally requires the iterative solution of a large scale system of nonlinear partial differential equations pdes the approach developed focuses primarily upon the adaptive selection of training and cross validation data and on artificial neural network ann architecture adjustments with the objective of improving the accuracy and generalization properties of ann based neutron diffusion eigenvalue predictions for illustration the performance of a bare bones feedforward multilayer perceptron mlp is upgraded through a variety of techniques namely nonrandom initial training set selection adjoint function input weighting teacher student membership and equivalence queries for generation of appropriate training data and a dynamic node architecture dna implementation the global methodology is flexible in that it ran wrap around any specific training algorithm selected for the static calculations i e training iterations with a fixed training set and architecture finally the improvements obtained are carefully contrasted against past works reported in the literature

a note on the number of queries needed to identify regular languages we consider the number of queries required to identify a regular set given an oracle for the set and some auxiliary information about the set if the auxiliary information is n the number of states of the canonical finite state acceptor for the language then the upper and lower bounds on the number of queries are exponential in n if the auxiliary information consists of a set of strings guaranteed to reach every live state of the canonical acceptor for the language then the upper and lower bounds are polynomial in n and the size of the given set of strings as a corollary the problem considered by pao and carr 1978 is shown to be solvable in a polynomial number of queries

using contextual representations to efficiently learn context free languages we present a polynomial update time algorithm for the inductive inference of a large class of context free languages using the paradigm of positive data and a membership oracle we achieve this result by moving to a novel representation called contextual binary feature grammars cbfgs which are capable of representing richly structured context free languages as well as some context sensitive languages these representations explicitly model the lattice structure of the distribution of a set of substrings and can be inferred using a generalisation of distributional learning this formalism is an attempt to bridge the gap between simple learnable classes and the sorts of highly expressive representations necessary for linguistic representation it allows the learnability of a large class of context free languages that includes all regular languages and those context free languages that satisfy two simple constraints the formalism and the algorithm seem well suited to natural language and in particular to the modeling of first language acquisition preliminary experimental results confirm the effectiveness of this approach

the crystallizing substochastic sequential machine extractor cryssmex this letter presents an algorithm cryssmex for extracting minimal finite state machine descriptions of dynamic systems such as recurrent neural networks unlike previous algorithms cryssmex is parameter free and deterministic and it efficiently generates a series of increasingly refined models a novel finite stochastic model of dynamic systems and a novel vector quantization function have been developed to take into account the state space dynamics of the system the experiments show that 1 extraction from systems that can be described as regular grammars is trivial 2 extraction from high dimensional systems is feasible and 3 extraction of approximative models from chaotic systems is possible the results are promising and an analysis of shortcomings suggests some possible further improvements some largely overlooked connections of the field of rule extraction from recurrent neural networks to other fields are also identified

learning by switching type of information the present work is dedicated to the study of modes of data presentation in the range between text and informant within the framework of inductive inference in this study the learner alternatingly requests sequences of positive and negative data we define various formalizations of valid data presentations in such a scenario we resolve the relationships between these different formalizations and show that one of these is equivalent to learning from informant we also show a hierarchy formed for each of the formalizations studied by considering the number of switches between requests for positive and negative data

learning regular omega languages we provide an algorithm for learning an unknown regular set of infinite words using membership and equivalence queries three variations of the algorithm learn three different canonical representations of omega regular languages using the notion of families of dfas one is of size similar to l a dfa representation recently learned using l 7 the second is based on the syntactic forc introduced in 14 the third is introduced herein we show that the second can be exponentially smaller than the first and the third is at most as large as the first two with up to a quadratic saving with respect to the second

models of active learning in group structured state spaces we investigate the problem of learning the transition dynamics of deterministic discrete state environments we assume that an agent exploring such an environment is able to perform actions from a finite set of actions in the environment and to sense the state changes the question investigated is whether the agent can learn the dynamics without visiting all states such a goal is unrealistic in general hence we assume that the environment has structural properties an agent might exploit in particular we assume that the set of all action sequences forms an algebraic group we introduce a learning model in different variants and study under which circumstances the corresponding group structured environments can be learned efficiently by experimenting with group generators actions it turns out that for some classes of such environments the choice of actions given to the agent determines if efficient learning is possible negative results are presented even without efficiency constraints for rather general classes of groups showing that even with group structure learning an environment from partial information is far from trivial however positive results for special subclasses of abelian groups turn out to be a good starting point for the design of efficient learning algorithms based on structured representations

an efficient query learning algorithm for ordered binary decision diagrams in this paper we propose a new algorithm that exactly learns ordered binary decision diagrams obdds with a given variable ordering via equivalence and membership queries our algorithm uses at most n equivalence queries and at most 2n log2m 3n membership queries where n is the number of nodes in the target reduced obdd and m is the number of variables the upper bound on the number of membership queries is smaller by a factor of o m compared with that for the previous best known algorithm proposed by r gavalda and d guijarro learning ordered binary decision diagrams proceedings of the 6th international workshop on algorithmic learning theory 1995 pp 228 238

inference of regular languages using state merging algorithms with search state merging algorithms have emerged as the solution of choice for the problem of inferring regular grammars from labeled samples a known np complete problem of great importance in the grammatical inference area these methods derive a small deterministic finite automaton from a set of labeled strings the training set by merging parts of the acceptor that corresponds to this training set experimental and theoretical evidence have shown that the generalization ability exhibited by the resulting automata is highly correlated with the number of states in the final solution as originally proposed state merging algorithms do not perform search this means that they are fast but also means that they are limited by the quality of the heuristics they use to select the states to be merged sub optimal choices lead to automata that have many more states than needed and exhibit poor generalization ability in this work we survey the existing approaches that generalize state merging algorithms by using search to explore the tree that represents the space of possible sequences of state mergings by using heuristic guided search in this space many possible state merging sequences can be considered leading to smaller automata and improved generalization ability at the expense of increased computation time we present comparisons of existing algorithms that show that in widely accepted benchmarks the quality of the derived solutions is improved by applying this type of search however we also point out that existing algorithms are not powerful enough to solve the more complex instances of the problem leaving open the possibility that better and more powerful approaches need to be designed

learning regular sets from queries and counterexamples the problem of identifying an unknown regular set from examples of its members and nonmembers is addressed it is assumed that the regular set is presented by a minimamy adequate teacher which can answer membership queries about the set and can also test a conjecture and indicate whether it is equal to the unknown set and provide a counterexample if not a counterexample is a string in the symmetric difference of the correct set and the conjectured set a learning algorithm l is described that correctly learns any regular set from any minimally adequate teacher in time polynomial in the number of states of the minimum dfa for the set and the maximum length of any counterexample provided by the teacher it is shown that in a stochastic setting the ability of the teacher to test conjectures may be replaced by a random sampling oracle ex a polynomial time learning algorithm is shown for a particular problem of context free language identification

how many queries are needed to learn we investigate the query complexity of exact learning in the membership and proper equivalence query model we give a complete characterization of concept classes that are learnable with a polynomial number of polynomial sized queries in this model we give applications of this characterization including results on learning a natural subclass of dnf formulas and on learning with membership queries alone query complexity has previously been used to prove lower bounds on the time complexity of exact learning we show a new relationship between query complexity and time complexity in exact learning if any honest class is exactly and properly learnable with polynomial query complexity but not learnable in polynomial time then p np in particular we show that an honest class is exactly polynomial query learnable if and only if it is learnable using an oracle for p 4

learning read once formulas with queries a read once formula is a boolean formula in which each variable occurs at most once such formulas are also called formulas or boolean trees this paper treats the problem of exactly identifying an unknown read once formula using specific kinds of queries the main results are a polynomial time algorithm for exact identification of monotone read once formulas using only membership queries and a polynomial time algorithm for exact identification of general read once formulas using equivalence and membership queries a protocol based on the notion of a minimally adequate teacher 1 the results of the authors improve on valiant s previous results for read once formulas 26 it is also shown that no polynomial time algorithm using only membership queries or only equivalence queries can exactly identify all read once formulas

neural network approximation of piecewise continuous functions application to friction compensation one of the most important properties of neural nets nns for control purposes is the universal approximation property unfortunately this property is generally proven for continuous functions in most real industrial control systems there are nonsmooth functions e g piecewise continuous for which approximation results in the literature are sparse examples include friction deadzone backlash and so on it is found that attempts to approximate piecewise continuous functions using smooth activation functions require many nn nodes and many training iterations and still do not yield very good results therefore a novel neural network structure is given for approximation of piecewise continuous functions of the sort that appear in friction deadzone backlash and other motion control actuator nonlinearities the novel nn consists of neurons having standard sigmoid activation functions plus some additional neurons having a special class of nonsmooth activation functions termed jump approximation basis function two types of nonsmooth jump approximation basis functions are determined a polynomial like basis and a sigmoid like basis this modified nn with additional neurons having jump approximation activation functions can approximate any piecewise continuous function with discontinuities at a finite number of known points applications of the new nn structure are made to rigid link robotic systems with friction nonlinearities friction is a nonlinear effect that can limit the performance of industrial control systems it occurs in all mechanical systems and therefore is unavoidable in control systems it can cause tracking errors limit cycles and other undesirable effects often inexact friction compensation is used with standard adaptive techniques that require models that are linear in the unknown parameters it is shown here how a certain class of augmented nn capable of approximating piecewise continuous functions can be used for friction compensation

the power of random counterexamples abstract learning a target concept from a finite n m concept space requires n proper equivalence queries in the worst case we propose a variation of the usual equivalence query in which the teacher is constrained to choose counterexamples randomly from a known probability distribution on examples we present and analyze the max min learning algorithm which identifies an arbitrary target concept in an arbitrary finite n m concept space using at most an expected log 2 n proper equivalence queries with random counterexamples

iterative methods for cyclically reduced non self adjoint linear systems we study iterative methods for solving linear systems of the type arising from two cyclic discretizations of non self adjoint two dimensional ellip tic partial differential equations a prototype is the convection diffusion equa tion the methods consist of applying one step of cyclic reduction resulting in a reduced system of half the order of the original discrete problem com bined with a reordering and a block iterative technique for solving the reduced system for constant coefficient problems we present analytic bounds on the spectral radii of the iteration matrices in terms of cell reynolds numbers that show the methods to be rapidly convergent in addition we describe numerical experiments that supplement the analysis and that indicate that the methods compare favorably with methods for solving the unreduced system

asymptotic controllability and exponential stabilization of nonlinear control systems at singular points we discuss the relation between exponential stabilization and asymptotic controllability of nonlinear control systems with constrained control range at singular points using a discounted optimal control approach we construct discrete feedback laws minimizing the lyapunov exponent of the linearization thus we obtain an equivalence result between uniform exponential controllability and uniform exponential stabilizability by means of a discrete feedback law

fast parallel algorithms for sparse multivariate polynomial interpolation over finite fields the authors consider the problem of reconstructing i e interpolating a t sparse multivariate polynomial given a black box which will produce the value of the polynomial for any value of the arguments it is shown that if the polynomial has coefficients in a finite field gf q and the black box can evaluate the polynomial in the field gf q ulcorner 2 log q nt 3 urcorner where n is the number of variables then there is an algorithm to interpolate the polynomial in o log 3 nt boolean parallel time and o n 2 t 6 log 2 nt processors this algorithm yields the first efficient deterministic polynomial time algorithm and moreover boolean nc algorithm for interpolating t sparse polynomials over finite fields and should be contrasted with the fact that efficient interpolation using a black box that only evaluates the polynomial at points in gf q is not possible cf m clausen a dress j grabmeier and m karpinski theoret comput sci 1990 to appear this algorithm tog

real time identification of discrete event systems using petri nets the paper defines the identification problem for discrete event systems des as the problem of inferring a petri net pn model using the observation of the events and the available output vectors that correspond to the markings of the measurable places two cases are studied considering different levels of the system knowledge in the first case the place and transition sets are assumed known hence an integer linear programming problem is defined in order to determine a pn modelling the des in the second case the transition and place sets are assumed unknown and only an upper bound of the number of places is given hence the identification problem is solved by an identification algorithm that observes in real time the occurred events and the corresponding output vectors the integer linear programming problem is defined at each observation so that the pn can be recursively identified some results and examples characterize the identified pn systems and show the flexibility and simplicity of the proposed technique moreover an application to the synthesis of supervisory control of pn systems via monitor places is proposed

learning languages from positive data and a finite number of queries a computational model for learning languages in the limit from full positive data and a bounded number of queries to the teacher oracle is introduced and explored equivalence superset and subset queries are considered for the latter one we consider also a variant when the learner tests every conjecture but the number of negative answers is uniformly bounded if the answer is negative the teacher may provide a counterexample we consider several types of counterexamples arbitrary least counterexamples the ones whose size is bounded by the size of positive data seen so far and no counterexamples a number of hierarchies based on the number of queries answers and types of answers counterexamples is established capabilities of learning with different types of queries are compared in most cases one or two queries of one type can sometimes do more than any bounded number of queries of another type still surprisingly a finite number of subset queries is sufficient to simulate the same number of equivalence queries when behaviourally correct learners do not receive counterexamples and may have unbounded number of errors in almost all conjectures

a neural state estimator with bounded errors for nonlinear systems a neural state estimator is described acting on discrete time nonlinear systems with noisy measurement channels a sliding window quadratic estimation cost function is considered and the measurement noise is assumed to be additive no probabilistic assumptions are made on the measurement noise nor on the initial state novel theoretical convergence results are developed for the error bounds of both the optimal and the neural approximate estimators to ensure the convergence properties of the neural estimator a minimax tuning technique is used the approximate estimator can be designed offline in such a way as to enable it to process on line any possible measure pattern almost instantly

modified successive overrelaxation msor and equivalent 2 step iterative methods for collocation matrices abstract we consider a class of consistently ordered matrices which arise from the discretization of boundary value problems bvps when the finite element collocation method with hermite elements is used through a recently derived equivalence relationship for the asymptotic rates of convergence of the modified successive overrelaxation msor and a certain 2 step iterative method we determine the optimum values for the parameters of the msor method as it pertains to collocation matrices a geometrical algorithm which utilizes capturing ellipse arguments has been successfully used the fast convergence properties of the optimum msor method are revealed after its comparison to several well known iterative schemes numerical examples which include the solution of poisson s equation are used to verify our results

successive overrelaxation sor and related methods abstract covering the last half of the 20th century we present some of the basic and well known results for the sor theory and related methods as well as some that are not as well known most of the earlier results can be found in the excellent books by varga matrix iterative analysis prentice hall englewood cliffs nj 1962 young iterative solution of large linear systems academic press new york 1971 and berman and plemmons nonnegative matrices in the mathematical sciences siam philadelphia pa 1994 while some of the most recent ones are given in the bibliography of this paper in this survey both the point and the block sor methods are considered for the solution of a linear system of the form ax b where a c n n and b c n 0 some general results concerning the sor and related methods are given and also some more specific ones in cases where a happens to possess some further property e g positive definiteness l m h matrix property p cyclic consistently ordered property etc

learning functions represented as multiplicity automata we study the learnability of multiplicity automata in angluin s exact learning model and we investigate its applications our starting point is a known theorem from automata theory relating the number of states in a minimal multiplicity automaton for a function to the rank of its hankel matrix with this theorem in hand we present a new simple algorithm for learning multiplicity automata with improved time and query complexity and we prove the learnability of various concept classes these include among others the class of disjoint dnf and more generally satisfy o 1 dnf the class of polynomials over finite fields the class of bounded degree polynomials over infinite fields the class of xor of terms certain classes of boxes in high dimensions in addition we obtain the best query complexity for several classes known to be learnable by other methods such as decision trees and polynomials over gf 2 while multiplicity automata are shown to be useful to prove the learnability of some subclasses of dnf formulae and various other classes we study the limitations of this method we prove that this method cannot be used to resolve the learnability of some other open problems such as the learnability of general dnf formulas or even k term dnf for k log n or satisfy s dnf formulas for s 1 these results are proven by exhibiting functions in the above classes that require multiplicity automata with super polynomial number of states

computational limitations on learning from examples the computational complexity of learning boolean concepts from examples is investigated it is shown for various classes of concept representations that these cannot be learned feasibly in a distribution free sense unless r np these classes include a disjunctions of two monomials b boolean threshold functions and c boolean formulas in which each variable occurs at most once relationships between learning of heuristics and finding approximate solutions to np hard optimization problems are given