logical analysis of numerical data logical analysis of data lad is a methodology developed since the late eighties aimed at discovering hidden structural information in data sets lad was originally developed for analyzing binary data by using the theory of partially defined boolean functions an extension of lad for the analysis of numerical data sets is achieved through the process of binarization consisting in the replacement of each numerical variable by binary indicator variables each showing whether the value of the original variable is above or below a certain level binarization was successfully applied to the analysis of a variety of real life data sets this paper develops the theoretical foundations of the binarization process studying the combinatorial optimization problems related to the minimization of the number of binary variables to provide an algorithmic framework for the practical solution of such problems we construct compact linear integer programming formulations of them we develop polynomial time algorithms for some of these minimization problems and prove np hardness of others

generalization error bounds for threshold decision lists in this paper we consider the generalization accuracy of classification methods based on the iterative use of linear classifiers the resulting classifiers which we call threshold decision lists act as follows some points of the data set to be classified are given a particular classification according to a linear threshold function or hyperplane these are then removed from consideration and the procedure is iterated until all points are classified geometrically we can imagine that at each stage points of the same classification are successively chopped off from the data set by a hyperplane we analyse theoretically the generalization properties of data classification techniques that are based on the use of threshold decision lists and on the special subclass of multilevel threshold functions we present bounds on the generalization error in a standard probabilistic learning framework the primary focus in this paper is on obtaining generalization error bounds that depend on the levels of separation or margins achieved by the successive linear classifiers we also improve and extend previously published theoretical bounds on the generalization ability of perceptron decision trees

context weighting for general finite context sources context weighting procedures are presented for sources with models structures in four different classes although the procedures are designed for universal data compression purposes their generality allows application in the area of classification

results on learnability and the vapnik chervonenkis dimension abstract we consider the problem of learning a concept from examples in the distribution free model by valiant an essentially equivalent model if one ignores issues of computational difficulty was studied by vapnik and chervonenkis we introduce the notion of dynamic sampling wherein the number of examples examined may increase with the complexity of the target concept this method is used to establish the learnability of various concept classes with an infinite vapnik chervonenkis dimension we also discuss an important variation on the problem of learning from examples called approximating from examples here we do not assume that the target concept t is a member of the concept class c from which approximations are chosen this problem takes on particular interest when the vc dimension of c is infinite finally we discuss the problem of computing the vc dimension of a finite concept set defined on a finite domain and consider the structure of classes of a fixed small dimension

a universal prior for integers and estimation by minimum description length of the number of bits required to write down the observed data has been reformulated to extend the classical maximum likelihood principle the principle permits estimation of the number of the parameters in statistical models in addition to their values and even of the way the parameters appear in the models i e of the model structures the principle rests on a new way to interpret and construct a universal prior distribution for the integers which makes sense even when the parameter is an individual object truncated realvalued parameters are converted to integers by dividing them by their precision and their prior is determined from the universal prior for the integers by optimizing the precision 1 introduction in this paper we study estimation based upon the principle of minimizing the total number of binary digits required to rewrite the observed data when each observation is given with some precision instead of attempting at an absolutely shortest description which would be futile we look for the optimum relative to a class of parametrically given distributions this minimum description length mdl principle which we introduced in a less comprehensive form in 25 turns out to degenerate to the more familiar maximum likelihood ml principle in case the number of parameters in the models is fixed so that the description length of the parameters themselves can be ignored in another extreme case where the parameters determine the data it similarly degenerates to jaynes s principle of maximum entropy 14 but the main power of the new criterion is that it permits estimates of the entire model its parameters their number and even the way the parameters appear in the model i e the model structure hence there will be no need to supplement the estimated parameters with a separate hypothesis test to decide whether a model is adequately parameterized or perhaps over parameterized

determining the wind driven surface currents for prediction of movements of oil slicks in the arabian gulf abstract proposed is a computational algorithm for determination of surface currents under given wind conditions for prediction of oil slick trajectories in the arabian gulf the algorithm interpolates a prespecified set of current velocities obtained from the three dimensional hydrodynamic model of the arabian gulf the three dimensional hydrodynamic model of the arabian gulf computes surface currents for any given wind velocity the prespecified wind velocities are combinations of the eight cardinal wind directions and seven wind speeds ranging from 5 to 35 knots at 5 knot increments a database of surface current velocities for the selected wind conditions is created the interpolation algorithm developed in this study uses the database to compute the surface currents for various wind conditions the proposed algorithm provides the many prospective users an opportunity to utilize the three dimensional hydrodynamic models without the requirement for large computing resources numerical results indicate the feasibility of such an approach

applying mdl to learn best model granularity abstract the minimum description length mdl principle is solidly based on a provably ideal method of inference using kolmogorov complexity we test how the theory behaves in practice on a general problem in model selection that of learning the best model granularity the performance of a model depends critically on the granularity for example the choice of precision of the parameters too high precision generally involves modeling of accidental noise and too low precision may lead to confusion of models that should be distinguished this precision is often determined ad hoc in mdl the best model is the one that most compresses a two part code of the data set this embodies occam s razor in two quite different experimental settings the theoretical value determined using mdl coincides with the best value found experimentally in the first experiment the task is to recognize isolated handwritten characters in one subject s handwriting irrespective of size and orientation based on a new modification of elastic matching using multiple prototypes per character the optimal prediction rate is predicted for the learned parameter length of sampling interval considered most likely by mdl which is shown to coincide with the best value found experimentally in the second experiment the task is to model a robot arm with two degrees of freedom using a three layer feed forward neural network where we need to determine the number of nodes in the hidden layer giving best modeling performance the optimal model the one that extrapolizes best on unseen examples is predicted for the number of nodes in the hidden layer considered most likely by mdl which again is found to coincide with the best value found experimentally

hierarchical mixtures of experts and the em algorithm we present a tree structured architecture for supervised learning the statistical model underlying the architecture is a hierarchical mixture model in which both the mixture coefficients and the mixture components are generalized linear models glim s learning is treated as a maximum likelihood problem in particular we present an expectation maximization em algorithm for adjusting the parameters of the architecture we also develop an on line learning algorithm in which the parameters are updated incrementally comparative simulation results are presented in the robot dynamics domain

a vertical horizontal splitting algorithm for three dimensional tidal and storm surge computations an algorithm is described for the computation of the three dimensional velocity fields due to tides and storm surges the surface elevation and depth averaged velocity components are first computed from the shallow water equations these equations are then used as part of the input to the second part of the algorithm in which the velocity profiles are computed from the momentum equations the nonlinear terms in the momentum equations and both the advective terms and the bottom friction terms in the shallow water equations are fully included the shallow water equations are solved by a finite difference scheme that achieves third order local truncation errors in all the dominant terms and that permits correct parallel flow on coastal boundaries of any orientation two alternative algorithms are discussed for computation of the velocity profiles one based on finite elements in the vertical coordinate direction the other based on a generalized crank nicolson scheme the complete algorithm has been tested on several model problems and has been found to be accurate and fast

on the derivation of the herman kluk propagator following an earlier derivation by miller we give a concise derivation of the herman kluk propagator that fully exploits the symplectic structure of the problem we also show directly that the version of the thawed gaussian semiclassical propagator obtained by baranger and co workers is equivalent to the linearized herman kluk propagator

semiclassical approximations in phase space with coherent states we present a complete derivation of the semiclassical limit of the coherent state propagator in one dimension starting from path integrals in phase space we show that the arbitrariness in the path integral representation which follows from the overcompleteness of the coherent states results in many different semiclassical limits we explicitly derive two possible semiclassical formulae for the propagator we suggest a third one and we discuss their relationships we also derive an initial value representation for the semiclassical propagator based on an initial gaussian wavepacket it turns out to be related to but different from heller s thawed gaussian approximation it is very different from the herman kluk formula which is not a correct semiclassical limit we point out errors in two derivations of the latter finally we show how the semiclassical coherent state propagators lead to wkb type quantization rules and to approximations for the husimi distributions of stationary states

stochastic complexity and modeling on demontre un theoreme fondamental qui donne une borne inferieure pour la longueur de code et donc pour les erreurs de prediction on definit les notions d information a priori et d information utile dans les donnees

a general lower bound on the number of examples needed for learning abstract we prove a lower bound of 1 ln 1 vcdim c on the number of random examples required for distribution free learning of a concept class c where vcdim c is the vapnik chervonenkis dimension and and are the accuracy and confidence parameters this improves the previous best lower bound of 1 ln 1 vcdim c and comes close to the known general upper bound of o 1 ln 1 vcdim c ln 1 for consistent algorithms we show that for many interesting concept classes including k cnf and k dnf our bound is actually tight to within a constant factor

new approach to semiclassical analysis in mechanics a new method is proposed for constructing approximate solutions to the schrodinger equation in place of the wave function its gaussian windowed fourier transform is used as the fundamental entity this allows an intuitively attractive connection to be made with a family of classical trajectories and at all times the wave function is inferred from the present state of these trajectories the fact that the connection between the wave function and the classical trajectories is consistently constructed in phase space allows this method to be free of the limitations of other methods

path integral based transition probability evaluation in a quantum system with a diagonal density matrix of the final state the variant of transition probability evaluation in a quantum system based on transition probability representation by a path integral using a density matrix formalism is described to estimate an integral s value a set of variables is selected and the monte carlo method is used for integration over these variables the stationary phase method is applied to integration over the remaining variables in the case of the final state with a diagonal density matrix this scheme leads to a simple expression for transition probability

learnability and the vapnik chervonenkis dimension valiant s learnability model is extended to learning classes of concepts defined by regions in euclidean space e n the methods in this paper lead to a unified treatment of some of valiant s results along with previous results on distribution free convergence of certain pattern recognition algorithms it is shown that the essential condition for distribution free learnability is finiteness of the vapnik chervonenkis dimension a simple combinatorial parameter of the class of concepts to be learned using this parameter the complexity and closure properties of learnable classes are analyzed and the necessary and sufficient conditions are provided for feasible learnability

finite difference model for tidal flows in the arabian gulf abstract a two dimensional hydrodynamical model for tidal flows in the arabian gulf is developed the model uses variable mesh sizes to handle a more detailed tidal current filed in selected sub regions in this study a three block model scheme is developed in which the deep areas are covered by a relatively coarse grid and the shallow coastal areas by a finer grid the third block which is a super fine mesh in the area of dhahran bahrain and the west coast of qatar is included in the model to predict the more detailed tidal current field in this area a finite difference model is utilized to solve the partial differential equations considered in the model

numerical solution of the two dimensional tidal equations using the method of characteristics abstract two dimensional shallow water equations which are obtained from the full three dimensional equations by averaging over the vertical coordinate were solved using the method of characteristics the solution technique which utilizes a first order computational scheme is presented in detail and an analysis of the tidal currents in the arabian gulf and the gulf of salwa is given as an example of the method the advantages of using the method of characteristics are discussed

a special adi model for the laplace tidal equations abstract an efficient method for numerical integration of the time dependent laplace tidal equations for a flat or curved earth is described the method is implicit uses alternating directions in two space dimensions with time staggering and a split sea height variable some special auxiliary features of the method are described and stability is discussed

tree pruning with subadditive penalties in this paper we study the problem of pruning a binary tree by minimizing over all pruned subtrees of the given tree an objective function that combines an additive cost term with a penalty term that depends only on tree size we present algorithms for general size based penalties although our focus is on subadditive penalties roughly penalties that grow more slowly than linear penalties with increasing tree size such penalties are motivated by recent results in statistical learning theory for decision trees but may have wider application as well we show that the family of pruned subtrees induced by a subadditive penalty is a subset of the family induced by an additive penalty this implies by known results about additive penalties that the family induced by a subadditive penalty 1 is nested 2 is unique and 3 can be computed efficiently it also implies that when a single tree is to be selected by cross validation from the family of prunings subadditive penalties will never present a richer set of options than an additive penalty

universal modeling and coding the problems arising in the modeling and coding of strings for compression purposes are discussed the notion of an information source that simplifies and sharpens the traditional one is axiomatized and adaptive and nonadaptive models are defined with a measure of complexity assigned to the models a fundamental theorem is proved which states that models that use any kind of alphabet extension are inferior to the best models using no alphabet extensions at all a general class of so called first in first out fifo arithmetic codes is described which require no alphabet extension devices and which therefore can be used in conjunction with the best models because the coding parameters are the probabilities that define the model their design is easy and the application of the code is straightforward even with adaptively changing source models

bayesian cart model search abstract in this article we put forward a bayesian approach for finding classification and regression tree cart models the two basic components of this approach consist of prior specification and stochastic search the basic idea is to have the prior induce a posterior distribution that will guide the stochastic search toward more promising cart models as the search proceeds such models can then be selected with a variety of criteria such as posterior probability marginal likelihood residual sum of squares or misclassification rates examples are used to illustrate the potential superiority of this approach over alternative methods

inferring decision trees using the minimum description length principle abstract we explore the use of rissanen s minimum description length principle for the construction of decision trees empirical results comparing this approach to other methods are given

adaptation of the solution of the two dimensional tidal equations using the method of characteristics to wind induced currents and storm surges abstract the method of characteristics previously employed to predict two dimensional tidal flows is extended to examine wind driven currents and storm surges after the shallow water equations used to model flow in coastal regions are given modifications are made in two of the forcing functions to incorporate wind contributions two hypothetical storms are constructed and results are presented in a series of figures which show sea level heights and velocity fields at various times as the storm progresses

feature subset selection by bayesian network based optimization abstract a new method for feature subset selection in machine learning fss ebna feature subset selection by estimation of bayesian network algorithm is presented fss ebna is an evolutionary population based randomized search algorithm and it can be executed when domain knowledge is not available a wrapper approach over naive bayes and id3 learning algorithms is used to evaluate the goodness of each visited solution fss ebna based on the eda estimation of distribution algorithm paradigm avoids the use of crossover and mutation operators to evolve the populations in contrast to genetic algorithms in absence of these operators the evolution is guaranteed by the factorization of the probability distribution of the best solutions found in a generation of the search this factorization is carried out by means of bayesian networks promising results are achieved in a variety of tasks where domain knowledge is not available the paper explains the main ideas of feature subset selection estimation of distribution algorithm and bayesian networks presenting related work about each concept a study about the overfitting problem in the feature subset selection process is carried out obtaining a basis to define the stopping criteria of the new algorithm

learning decision trees from random examples needed for learning abstract we define the rank of a decision tree and show that for any fixed r the class of all decision trees of rank at most r on n boolean variables is learnable from random examples in time polynomial in n and linear in 1 and log 1 where is the accuracy parameter and is the confidence parameter using a suitable encoding of variables rivest s polynomial learnability result for decision lists can be interpreted as a special case of this result for rank 1 as another corollary we show that decision trees on n boolean variables of size polynomial in n are learnable from random examples in time linear in n o log n 1 and log 1 as a third corollary we show that boolean functions that have polynomial size dnf expressions for both their positive and their negative instances are learnable from random examples in time linear in n o log n 2 1 and log 1

on finding a minimum dominating set in a tournament note abstract the problem of finding a minimum dominating set in a tournament can be solved in n o log n time it is shown that if this problem has a polynomial time algorithm then for every constant c there is also a polynomial time algorithm for the satisfiability problem of boolean formulas in conjunctive normal form with m clauses and c log 2 m variables on the other hand the problem can be reduced in polynomial time to a general satisfiability problem of length l with o log 2 l variables another relation between the satisfiability problem and the minimum dominating set in a tournament says that the former can be solved in 2 o v n k time where v is the number of variables n is the length of the formula and k is a constant if and only if the latter has a polynomial time algorithm