geometric aspects of sturm liouville problems iii level surfaces of the nth eigenvalue for a given regular sturm liouville equation and any natural number n we explicitly describe the level surfaces of the nth eigenvalue of the equation regarded as a function on the space of self adjoint boundary conditions the description of each level surface uses only a particular value of the transfer matrix of the equation the shapes of these surfaces are also determined

a gibbs sampler for learning dags we propose a gibbs sampler for structure learning in directed acyclic graph dag models the standard markov chain monte carlo algorithms used for learning dags are random walk metropolis hastings samplers these samplers are guaranteed to converge asymptotically but often mix slowly when exploring the large graph spaces that arise in structure learning in each step the sampler we propose draws entire sets of parents for multiple nodes from the appropriate conditional distribution this provides an efficient way to make large moves in graph space permitting faster mixing whilst retaining asymptotic guarantees of convergence the conditional distribution is related to variable selection with candidate parents playing the role of covariates or inputs we empirically examine the performance of the sampler using several simulated and real data examples the proposed method gives robust results in diverse settings outperforming several existing bayesian and frequentist methods in addition our empirical results shed some light on the relative merits of bayesian and constraint based methods for structure learning

learning equivalence classes of bayesian network structures two bayesian network structures are said to be equivalent if the set of distributions that can be represented with one of those structures is identical to the set of distributions that can be represented with the other many scoring criteria that are used to learn bayesian network structures from data are score equivalent that is these criteria do not distinguish among networks that are equivalent in this paper we consider using a score equivalent criterion in conjunction with a heuristic search algorithm to perform model selection or model averaging we argue that it is often appropriate to search among equivalence classes of network structures as opposed to the more common approach of searching among individual bayesian network structures we describe a convenient graphical representation for an equivalence class of structures and introduce a set of operators that can be applied to that representation by a search algorithm to move among equivalence classes we show that our equivalence class operators can be scored locally and thus share the computational efficiency of traditional operators defined for individual structures we show experimentally that a greedy model selection algorithm using our representation yields slightly higher scoring structures than the traditional approach without any additional time overhead and we argue that more sophisticated search algorithms are likely to benefit much more

a characterization of markov equivalence classes for acyclic digraphs undirected graphs and acyclic digraphs adgs as well as their mutual extension to chain graphs are widely used to describe dependencies among variables in multivariate distributions in particular the likelihood functions of adg models admit convenient recursive factorizations that often allow explicit maximum likelihood estimates and that are well suited to building bayesian networks for expert systems whereas the undirected graph associated with a dependence model is uniquely determined there may however be many adgs that determine the same dependence markov model thus the family of all adgs with a given set of vertices is naturally partitioned into markov equivalence classes each class being associated with a unique statistical model statistical procedures such as model selection or model averaging that fail to take into account these equivalence classes may incur substantial computational or other inefficiencies here it is shown that each markov equivalence class is uniquely determined by a single chain graph the essential graph that is itself simultaneously markov equivalent to all adgs in the equivalence class essential graphs are characterized a polynomial time algorithm for their construction is given and their applications to model selection and other statistical questions are described

learning linear cyclic causal models with latent variables identifying cause effect relationships between variables of interest is a central problem in science given a set of experiments we describe a procedure that identifies linear models that may contain cycles and latent variables we provide a detailed description of the model family full proofs of the necessary and sufficient conditions for identifiability a search algorithm that is complete and a discussion of what can be done when the identifiability conditions are not satisfied the algorithm is comprehensively tested in simulations comparing it to competing algorithms in the literature furthermore we adapt the procedure to the problem of cellular network inference applying it to the biologically realistic data of the dreamchallenges the paper provides a full theoretical foundation for the causal discovery procedure first presented by eberhardt et al 2010 and hyttinen et al 2010

large sample learning of bayesian networks is np hard in this paper we provide new complexity results for algorithms that learn discrete variable bayesian networks from data our results apply whenever the learning algorithm uses a scoring criterion that favors the simplest structure for which the model is able to represent the generative distribution exactly our results therefore hold whenever the learning algorithm uses a consistent scoring criterion and is applied to a sufficiently large dataset we show that identifying high scoring structures is np hard even when any combination of one or more of the following hold the generative distribution is perfect with respect to some dag containing hidden variables we are given an independence oracle we are given an inference oracle we are given an information oracle we restrict potential solutions to structures in which each node has at most k parents for all k 3 our proof relies on a new technical result that we establish in the appendices in particular we provide a method for constructing the local distributions in a bayesian network such that the resulting joint distribution is provably perfect with respect to the structure of the network

estimation of directed acyclic graphs through two stage adaptive lasso for gene network inference abstractgraphical models are a popular approach to find dependence and conditional independence relationships between gene expressions directed acyclic graphs dags are a special class of directed graphical models where all the edges are directed edges and contain no directed cycles the dags are well known models for discovering causal relationships between genes in gene regulatory networks however estimating dags without assuming known ordering is challenging due to high dimensionality the acyclic constraints and the presence of equivalence class from observational data to overcome these challenges we propose a two stage adaptive lasso approach called ns dist which performs neighborhood selection ns in stage 1 and then estimates dags by the discrete improving search with tabu dist algorithm within the selected neighborhood simulation studies are presented to demonstrate the effectiveness of the method and its computational efficiency two real data examples are used to demonstrate the p

jointly interventional and observational data estimation of interventional markov equivalence classes of directed acyclic graphs summary in many applications we have both observational and randomized interventional data we propose a gaussian likelihood framework for joint modelling of such different data types based on global parameters consisting of a directed acyclic graph and corresponding edge weights and error variances thanks to the global nature of the parameters maximum likelihood estimation is reasonable with only one or few data points per intervention we prove consistency of the bayesian information criterion for estimating the interventional markov equivalence class of directed acyclic graphs which is smaller than the observational analogue owing to increased partial identifiability from interventional data such an improvement in identifiability has immediate implications for tighter bounds for inferring causal effects besides methodology and theoretical derivations we present empirical results from real and simulated data

a recursive method for structural learning of directed acyclic graphs in this paper we propose a recursive method for structural learning of directed acyclic graphs dags in which a problem of structural learning for a large dag is first decomposed into two problems of structural learning for two small vertex subsets each of which is then decomposed recursively into two problems of smaller subsets until none subset can be decomposed further in our approach search for separators of a pair of variables in a large dag is localized to small subsets and thus the approach can improve the efficiency of searches and the power of statistical tests for structural learning we show how the recent advances in the learning of undirected graphical models can be employed to facilitate the decomposition simulations are given to demonstrate the performance of the proposed method

dissipative non self adjoint sturm liouville operators and completeness of their eigenfunctions abstract in this paper non self adjoint sturm liouville operators in weyl s limit circle case are studied we first determine all the non self adjoint boundary conditions yielding dissipative operators for each allowed sturm liouville differential expression then using the characteristic determinant the completeness of the system of eigenfunctions and associated functions for these dissipative operators is proved

estimating high dimensional directed acyclic graphs with the pc algorithm we consider the pc algorithm spirtes et al 2000 for estimating the skeleton and equivalence class of a very high dimensional directed acyclic graph dag with corresponding gaussian distribution the pc algorithm is computationally feasible and often very fast for sparse problems with many nodes variables and it has the attractive property to automatically achieve high computational efficiency as a function of sparseness of the true underlying dag we prove uniform consistency of the algorithm for very high dimensional sparse dags where the number of nodes is allowed to quickly grow with sample size n as fast as o na for any 0 a the sparseness assumption is rather minimal requiring only that the neighborhoods in the dag are of lower order than sample size n we also demonstrate the pc algorithm for simulated data

a graphical representation of equivalence classes of amp chain graphs this paper deals with chain graph models under alternative amp interpretation a new representative of an amp markov equivalence class called the largest deflagged graph is proposed the representative is based on revealed internal structure of the amp markov equivalence class more specifically the amp markov equivalence class decomposes into finer strong equivalence classes and there exists a distinguished strong equivalence class among those forming the amp markov equivalence class the largest deflagged graph is the largest chain graph in that distinguished strong equivalence class a composed graphical procedure to get the largest deflagged graph on the basis of any amp markov equivalent chain graph is presented in general the largest deflagged graph differs from the amp essential graph which is another representative of the amp markov equivalence class

finding optimal bayesian network given a super structure classical approaches used to learn bayesian network structure from data have disadvantages in terms of complexity and lower accuracy of their results however a recent empirical study has shown that a hybrid algorithm improves sensitively accuracy and speed it learns a skeleton with an independency test it approach and constrains on the directed acyclic graphs dag considered during the search and score phase subsequently we theorize the structural constraint by introducing the concept of super structure s which is an undirected graph that restricts the search to networks whose skeleton is a subgraph of s we develop a super structure constrained optimal search cos its time complexity is upper bounded by o m n where m 2 depends on the maximal degree m of s empirically complexity depends on the average degree m and sparse structures allow larger graphs to be calculated our algorithm is faster than an optimal search by several orders and even finds more accurate results when given a sound super structure practically s can be approximated by it approaches significance level of the tests controls its sparseness enabling to control the trade off between speed and accuracy for incomplete super structures a greedily post processed version cos still enables to significantly outperform other heuristic searches

pc algorithm for nonparanormal graphical models the pc algorithm uses conditional independence tests for model selection in graphical modeling with acyclic directed graphs in gaussian models tests of conditional independence are typically based on pearson correlations and high dimensional consistency results have been obtained for the pc algorithm in this setting analyzing the error propagation from marginal to partial correlations we prove that high dimensional consistency carries over to a broader class of gaussian copula or nonparanormal models when using rank based measures of correlation for graph sequences with bounded degree our consistency result is as strong as prior gaussian results in simulations the rank pc algorithm works as well as the pearson pc algorithm for normal data and considerably better for non normal data all the while incurring a negligible increase of computation time while our interest is in the pc algorithm the presented analysis of error propagation could be applied to other algorithms that test the vanishing of low order partial correlations

left definite sturm liouville problems abstract left definite regular self adjoint sturm liouville problems with either separated or coupled boundary conditions are studied we give an elementary proof of the existence of eigenvalues for these problems for any fixed equation we establish a sequence of inequalities among the eigenvalues for different boundary conditions and estimate the range of each eigenvalue as a function on the space of boundary conditions some of our results here yield an algorithm for numerically computing the eigenvalues of a left definite problem with an arbitrary coupled boundary condition our inequalities imply that the well known asymptotic formula for the eigenvalues in the separated case also holds in the coupled case moreover we study the continuous and differentiable dependence of the eigenvalues of the general left definite problem on all the parameters in its differential equation and boundary condition

learning causal bayesian network structures from experimental data we propose a method for the computational inference of directed acyclic graphical structures given data from experimental interventions order space markov chain monte carlo equi energy sampling importance weighting and stream based computation are combined to create a fast algorithm for learning causal bayesian network structures

counting and exploring sizes of markov equivalence classes of directed acyclic graphs when learning a directed acyclic graph dag model via observational data one generally cannot identify the underlying dag but can potentially obtain a markov equivalence class the size the number of dags of a markov equivalence class is crucial to infer causal effects or to learn the exact causal dag via further interventions given a set of markov equivalence classes the distribution of their sizes is a key consideration in developing learning methods however counting the size of an equivalence class with many vertices is usually computationally infeasible and the existing literature reports the size distributions only for equivalence classes with ten or fewer vertices in this paper we develop a method to compute the size of a markov equivalence class we first show that there are five types of markov equivalence classes whose sizes can be formulated as five functions of the number of vertices respectively then we introduce a new concept of a rooted sub class the graph representations of rooted subclasses of a markov equivalence class are used to partition this class recursively until the sizes of all rooted subclasses can be computed via the five functions the proposed size counting is efficient for markov equivalence classes of sparse dags with hundreds of vertices finally we explore the size and edge distributions of markov equivalence classes and find experimentally that in general 1 most markov equivalence classes are half completed and their average sizes are small and 2 the sizes of sparse classes grow approximately exponentially with the numbers of vertices

fulfilling hard constraints in uncertain linear systems by reference managing a method based on conceptual tools of predictive control is described for tackling tracking problems of uncertain linear systems wherein pointwise in time input and or state inequality constraints are present the method consists of adding to a primal compensated system a nonlinear device called predictive reference filter which manipulates the desired reference in order to fulfill the prescribed constraints provided that an admissibility condition on the initial state is satisfied the control scheme is proved to fulfill the constraints as well as stability and set point tracking requirements for all systems whose impulse step responses lie within given uncertainty ranges

markov equivalence for ancestral graphs ancestral graphs can encode conditional independence relations that arise in directed acyclic graph dag models with latent and selection variables however for any ancestral graph there may be several other graphs to which it is markov equivalent we state and prove conditions under which two maximal ancestral graphs are markov equivalent to each other thereby extending analogous results for dags given by other authors these conditions lead to an algorithm for determining markov equivalence that runs in time that is polynomial in the number of vertices in the graph

model predictive control tuning by controller matching the effectiveness of model predictive control mpc in dealing with input and state constraints during transient operations is well known however in contrast with several linear control techniques closed loop frequency domain properties such as sensitivities and robustness to small perturbations are usually not taken into account in the mpc design this technical note considers the problem of tuning an mpc controller that behaves as a given linear controller when the constraints are not active e g for perturbations around the equilibrium that remain within the given input and state bounds therefore inheriting the small signal properties of the linear control design and that still optimally deals with constraints during transients we provide two methods for selecting the mpc weight matrices so that the resulting mpc controller behaves as the given linear controller therefore solving the posed inverse problem of controller matching and is globally asymptotically stable

updating schemes correlation structure blocking and parameterization for the gibbs sampler in this paper many convergence issues concerning the implementation of the gibbs sampler are investigated exact computable rates of convergence for gaussian target distributions are obtained different random and non random updating strategies and blocking combinations are compared using the rates the effect of dimensionality and correlation structure on the convergence rates are studied some examples are considered to demonstrate the results for a gaussian image analysis problem several updating strategies are described and compared for problems in bayesian linear models several possible parameterizations are analysed in terms of their convergence rates characterizing the optimal choice

estimation of sparse directed acyclic graphs for multivariate counts data the next generation sequencing data called high throughput sequencing data are recorded as count data which are generally far from normal distribution under the assumption that the count data follow the poisson log normal distribution this article provides an l1 penalized likelihood framework and an efficient search algorithm to estimate the structure of sparse directed acyclic graphs dags for multivariate counts data in searching for the solution we use iterative optimization procedures to estimate the adjacency matrix and the variance matrix of the latent variables the simulation result shows that our proposed method outperforms the approach which assumes multivariate normal distributions and the log transformation approach it also shows that the proposed method outperforms the rank based pc method under sparse network or hub network structures as a real data example we demonstrate the efficiency of the proposed method in estimating the gene regulatory networks of the ovarian cancer study

application of a smoothing technique to decomposition in convex optimization dual decomposition is a powerful technique for deriving decomposition schemes for convex optimization problems with separable structure although the augmented lagrangian is computationally more stable than the ordinary lagrangian the ldquoprox termrdquo destroys the separability of the given problem in this technical note we use another approach to obtain a smooth lagrangian based on a smoothing technique developed by nesterov which preserves separability of the problem with this approach we derive a new decomposition method called ldquoproximal center algorithm rdquo which from the viewpoint of efficiency estimates improves the bounds on the number of iterations of the classical dual gradient scheme by an order of magnitude

analyticity of the spectrum and dirichlet to neumann operator technique for quantum graphs in some previous works the analytic structure of the spectrum of a quantum graph operator as a function of the vertex conditions and other parameters of the graph were established however a specific local coordinate chart on the grassmannian of all possible vertex conditions was used thus creating an erroneous impression that something wrong can happen at the boundaries of the chart here we show that the analyticity of the corresponding dispersion relation holds over the whole grassmannian as well as over other parameter spaces we also address the dirichlet to neumann technique of relating quantum and discrete graph operators which allows one to transfer some results from the discrete to the quantum graph case but which has issues at the dirichlet spectrum we conclude that this difficulty as in the first part of the paper stems from the use of specific coordinates in a grassmannian and show how to avoid it to extend some of the consequent results to the general situation in some previous works the analytic structure of the spectrum of a quantum graph operator as a function of the vertex conditions and other parameters of the graph were established however a specific local coordinate chart on the grassmannian of all possible vertex conditions was used thus creating an erroneous impression that something wrong can happen at the boundaries of the chart here we show that the analyticity of the corresponding dispersion relation holds over the whole grassmannian as well as over other parameter spaces we also address the dirichlet to neumann technique of relating quantum and discrete graph operators which allows one to transfer some results from the discrete to the quantum graph case but which has issues at the dirichlet spectrum we conclude that this difficulty as in the first part of the paper stems from the use of specific coordinates in a grassmannian and show how to avoid it to extend some of the consequent results to the general situation

dependence of the nth sturm liouville eigenvalue on the problem abstract we consider the nth eigenvalue as a function on the space of self adjoint regular sturm liouville problems with positive leading coefficient and weight functions the discontinuity of the nth eigenvalue is completely characterized

stochastic complexity and modeling on demontre un theoreme fondamental qui donne une borne inferieure pour la longueur de code et donc pour les erreurs de prediction on definit les notions d information a priori et d information utile dans les donnees

trade offs for fully dynamic transitive closure on dags breaking through the o n 2 barrier we present an algorithm for directed acyclic graphs that breaks through the o n2 barrier on the single operation complexity of fully dynamic transitive closure where n is the number of edges in the graph we can answer queries in o ne worst case time and perform updates in o n 1 e 1 epn1pe worst case time for any e 0 1 where 1 e 1 is the exponent of the multiplication of an n ne matrix by an ne n matrix the current best bounds on 1 e 1 imply an o n0 575 query time and an o n1 575 update time in the worst case our subquadratic algorithm is randomized and has one sided error as an application of this result we show how to solve single source reachability in o n1 575 time per update and constant time per query

all pairs shortest paths using bridging sets and rectangular matrix multiplication we present two new algorithms for solving the all pairs shortest paths apsp problem for weighted directed graphs both algorithms use fast matrix multiplication algorithms the first algorithm solves the apsp problem for weighted directed graphs in which the edge weights are integers of small absolute value in o n2 time where satisfies the equation 1 1 1 2 and 1 1 is the exponent of the multiplication of an n n matrix by an n n matrix currently the best available bounds on 1 1 obtained by coppersmith imply that 0 is an error parameter and w is the largest edge weight in the graph after the edge weights are scaled so that the smallest non zero edge weight in the graph is 1 it returns estimates of all the distances in the graph with a stretch of at most 1 e corresponding paths can also be found efficiently

amortized efficiency of a path retrieval data structure a data structure is presented for the problem of maintaining a digraph under an arbitrary sequence of two kinds of operations an add operation that inserts an arc in the digraph and a searchpath operation that checks the presence of a path between a pair of nodes our data structure supports both operations in o n amortized time and requires o n2 space where n is the number of nodes in the digraph

simple linear time algorithms to test chordality of graphs test acyclicity of hypergraphs and selectively reduce acyclic hypergraphs chordal graphs arise naturally in the study of gaussian elimination on sparse symmetric matrices acyclic hypergraphs arise in the study of relational data bases rose tarjan and lueker siam j c

optimal structure identification with greedy search in this paper we prove the so called meek conjecture in particular we show that if a dag h is an independence map of another dag g then there exists a finite sequence of edge additions and covered edge reversals in g such that 1 after each edge modification h remains an independence map of g and 2 after all modifications g h as shown by meek 1997 this result has an important consequence for bayesian approaches to learning bayesian networks from data in the limit of large sample size there exists a two phase greedy search algorithm that when applied to a particular sparsely connected search space provably identifies a perfect map of the generative distribution if that perfect map is a dag we provide a new implementation of the search space using equivalence classes as states for which all operators used in the greedy search can be scored efficiently using local functions of the nodes in the domain finally using both synthetic and real world datasets we demonstrate that the two phase greedy approach leads to good solutions when learning with finite sample sizes

a scoring function for learning bayesian networks based on mutual information and conditional independence tests we propose a new scoring function for learning bayesian networks from data using score search algorithms this is based on the concept of mutual information and exploits some well known properties of this measure in a novel way essentially a statistical independence test based on the chi square distribution associated with the mutual information measure together with a property of additive decomposition of this measure are combined in order to measure the degree of interaction between each variable and its parent variables in the network the result is a non bayesian scoring function called mit mutual information tests which belongs to the family of scores based on information theory the mit score also represents a penalization of the kullback leibler divergence between the joint probability distributions associated with a candidate network and with the available data set detailed results of a complete experimental evaluation of the proposed scoring function and its comparison with the well known k2 bdeu and bic mdl scores are also presented

projection free parallel quadratic programming for linear model predictive control a key component in enabling the application of model predictive control mpc in fields such as automotive aerospace and factory automation is the availability of low complexity fast optimisation algorithms to solve the mpc finite horizon optimal control problem in architectures with reduced computational capabilities in this paper we introduce a projection free iterative optimisation algorithm and discuss its application to linear mpc the algorithm originally developed by brand for non negative quadratic programs is based on a multiplicative update rule and it is shown to converge to a fixed point which is the optimum an acceleration technique based on a projection free line search is also introduced to speed up the convergence to the optimum the algorithm is applied to mpc through the dual of the quadratic program qp formulated from the mpc finite time optimal control problem we discuss how termination conditions with guaranteed degree of suboptimality can be enforced and how the algorithm

exact bayesian structure discovery in bayesian networks learning a bayesian network structure from data is a well motivated but computationally hard task we present an algorithm that computes the exact posterior probability of a subnetwork e g a directed edge a modified version of the algorithm finds one of the most probable network structures this algorithm runs in time o n 2n nk 1c m where n is the number of network variables k is a constant maximum in degree and c m is the cost of computing a single local marginal conditional likelihood for m data instances this is the first algorithm with less than super exponential complexity with respect to n exact computation allows us to tackle complex cases where existing monte carlo methods and local search procedures potentially fail we show that also in domains with a large number of variables exact computation is feasible given suitable a priori restrictions on the structures combining exact and inexact methods is also possible we demonstrate the applicability of the presented algorithm on four synthetic data sets with 17 22 37 and 100 variables

equi energy sampler with applications in statistical inference and statistical mechanics we introduce a new sampling algorithm the equi energy sampler for efficient statistical sampling and estimation complementary to the widely used temperature domain methods the equi energy sampler utilizing the temperature energy duality targets the energy directly the focus on the energy function not only facilitates efficient sampling but also provides a powerful means for statistical estimation for example the calculation of the density of states and microcanonical averages in statistical mechanics the equi energy sampler is applied to a variety of problems including exponential regression in statistics motif sampling in computational biology and protein folding in biophysics we congratulate samuel kou qing zhou and wing wong referred to subsequently as kzw for this beautifully written paper which opens a new direction in monte carlo computation this discussion has two parts first we describe a very closely related method multicanonical sampling mcs and report a simulation example that compares the equi energy ee sampler with mcs overall we found the two algorithms to be of comparable efficiency for the simulation problem considered in the second part we develop some additional convergence results for the ee sampler novel sampling algorithms can significantly impact open questions in computational biology most notably the in silico protein folding problem by using computational methods protein folding aims to find the three dimensional structure of a protein chain given the sequence of its amino acid building blocks the complexity of the problem strongly depends on the protein representation and its energy function the more detailed the model the more complex its corresponding energy function and the more challenge it sets for sampling algorithms kou zhou and wong have introduced a novel sampling method which could contribute significantly to the field of structural prediction

probability inequalities for likelihood ratios and convergence rates of sieve mles let y 1 y n be independent identically distributed with density p 0 and let f be a space of densities we show that the supremum of the likelihood ratios i 1 n p y i p 0 y i where the supremum is over p f with p l 2 p 0 1 2 2 e is exponentially small with probability exponentially close to 1 the exponent is proportional to ne 2 the only condition required for this to hold is that e exceeds a value determined by the bracketing hellinger entropy of f a similar inequality also holds if we replace f by f n and p 0 by q n where q n is an approximation to p 0 in a suitable sense these results are applied to establish rates of convergence of sieve mles furthermore weak conditions are given under which the optimal rate e n defined by h e n f ne n 2 where h f is the hellinger entropy of f is nearly achievable by sieve estimators

bais a bayesian artificial immune system for the effective handling of building blocks significant progress has been made in theory and design of artificial immune systems aiss for solving hard problems accurately however an aspect not yet widely addressed by the research reported in the literature is the lack of ability of the aiss to deal effectively with building blocks partial high quality solutions coded in the antibody the available aiss present mechanisms for evolving the population that do not take into account the relationship among the variables of the problem potentially causing the disruption of high quality partial solutions this paper proposes a novel ais with abilities to identify and properly manipulate building blocks in optimization problems instead of using cloning and mutation to generate new individuals our algorithm builds a probabilistic model representing the joint probability distribution of the promising solutions and subsequently uses this model for sampling new solutions the probabilistic model used is a bayesian network due to its capability of properly capturing the most relevant interactions among the variables therefore our algorithm called bayesian artificial immune system bais represents a significant attempt to improve the performance of immune inspired algorithms when dealing with building blocks and hence to solve efficiently hard optimization problems with complex interactions among the variables the performance of bais compares favorably with that produced by contenders such as state of the art estimation of distribution algorithms

on inclusion driven learning of bayesian networks two or more bayesian network structures are markov equivalent when the corresponding acyclic digraphs encode the same set of conditional independencies therefore the search space of bayesian network structures may be organized in equivalence classes where each of them represents a different set of conditional independencies the collection of sets of conditional independencies obeys a partial order the so called inclusion order this paper discusses in depth the role that the inclusion order plays in learning the structure of bayesian networks in particular this role involves the way a learning algorithm traverses the search space we introduce a condition for traversal operators the inclusion boundary condition which when it is satisfied guarantees that the search strategy can avoid local maxima this is proved under the assumptions that the data is sampled from a probability distribution which is faithful to an acyclic digraph and the length of the sample is unbounded the previous discussion leads to the design of a new traversal operator and two new learning algorithms in the context of heuristic search and the markov chain monte carlo method we carry out a set of experiments with synthetic and real world data that show empirically the benefit of striving for the inclusion order when learning bayesian networks from data

a generalized back door criterion we generalize pearl s back door criterion for directed acyclic graphs dags to more general types of graphs that describe markov equivalence classes of dags and or allow for arbitrarily many hidden variables we also give easily checkable necessary and sufficient graphical criteria for the existence of a set of variables that satisfies our generalized back door criterion when considering a single intervention and a single outcome variable moreover if such a set exists we provide an explicit set that fulfills the criterion we illustrate the results in several examples r code is available in the r package pcalg

bnc pso structure learning is a very important problem in the field of bayesian networks bns it is also an active research area for more than 2 decades therefore many approaches have been proposed in order to find an optimal structure based on training samples in this paper a particle swarm optimization pso based algorithm is proposed to solve the bn structure learning problem named bnc pso bayesian network construction algorithm using pso edge inserting deleting is employed in the algorithm to make the particles have the ability to achieve the optimal solution while a cycle removing procedure is used to prevent the generation of invalid solutions then the theorem of markov chain is used to prove the global convergence of our proposed algorithm finally some experiments are designed to evaluate the performance of the proposed pso based algorithm experimental results indicate that bnc pso is worthy of being studied in the field of bns construction meanwhile it can significantly increase nearly 15 in the scoring metric values comparing with other optimization based algorithms

learning gaussian tree models analysis of error exponents and extremal structures the problem of learning tree structured gaussian graphical models from independent and identically distributed i i d samples is considered the influence of the tree structure and the parameters of the gaussian distribution on the learning rate as the number of samples increases is discussed specifically the error exponent corresponding to the event that the estimated tree structure differs from the actual unknown tree structure of the distribution is analyzed finding the error exponent reduces to a least squares problem in the very noisy learning regime in this regime it is shown that the extremal tree structure that minimizes the error exponent is the star for any fixed set of correlation coefficients on the edges of the tree if the magnitudes of all the correlation coefficients are less than 0 63 it is also shown that the tree structure that maximizes the error exponent is the markov chain in other words the star and the chain graphs represent the hardest and the easiest structures to learn in the class of tree structured gaussian graphical models this result can also be intuitively explained by correlation decay pairs of nodes which are far apart in terms of graph distance are unlikely to be mistaken as edges by the maximum likelihood estimator in the asymptotic regime

singular sturm liouville problems the friedrichs extension and comparison of eigenvalues a new characterization of singular self adjoint boundary conditions for sturm liouville problems is given these are an exact parallel of the regular case they are given explicitly in terms of principal and non principal solutions the special nature of the friedrichs extension is clearly apparent and highlighted inequalities among the eigenvalues of different boundary conditions separated and coupled are obtained most of all e want to stress the method of proof it is based on a very elementary transformation which transforms any singular non oscillatory limit circle endpoint into a regular one

search for additive nonlinear time series causal models pointwise consistent feasible procedures for estimating contemporaneous linear causal structure from time series data have been developed using multiple conditional independence tests but no such procedures are available for non linear systems we describe a feasible procedure for learning a class of non linear time series structures which we call additive non linear time series we show that for data generated from stationary models of this type two classes of conditional independence relations among time series variables and their lags can be tested efficiently and consistently using tests based on additive model regression combining results of statistical tests for these two classes of conditional independence relations and the temporal structure of time series data a new consistent model specification procedure is able to extract relatively detailed causal information we investigate the finite sample behavior of the procedure through simulation and illustrate the application of this method through analysis of the possible causal connections among four ocean indices several variants of the procedure are also discussed

optimal search on clustered structural constraint for learning bayesian network structure we study the problem of learning an optimal bayesian network in a constrained search space skeletons are compelled to be subgraphs of a given undirected graph called the super structure the previously derived constrained optimal search cos remains limited even for sparse super structures to extend its feasibility we propose to divide the super structure into several clusters and perform an optimal search on each of them further to ensure acyclicity we introduce the concept of ancestral constraints acs and derive an optimal algorithm satisfying a given set of acs finally we theoretically derive the necessary and sufficient sets of acs to be considered for finding an optimal constrained graph empirical evaluations demonstrate that our algorithm can learn optimal bayesian networks for some graphs containing several hundreds of vertices and even for super structures having a high average degree up to four which is a drastic improvement in feasibility over the previous optimal algorithm learnt networks are shown to largely outperform state of the art heuristic algorithms both in terms of score and structural hamming distance

model averaging for prediction with discrete bayesian networks in this paper we consider the problem of performing bayesian model averaging over a class of discrete bayesian network structures consistent with a partial ordering and with bounded in degree k we show that for n nodes this class contains in the worst case at least distinct network structures and yet model averaging over these structures can be performed using operations furthermore we show that there exists a single bayesian network that defines a joint distribution over the variables that is equivalent to model averaging over these structures although constructing this network is computationally prohibitive we show that it can be approximated by a tractable network allowing approximate model averaged probability calculations to be performed in o n time our result also leads to an exact and linear time solution to the problem of averaging over the 2n possible feature sets in a naive bayes model providing an exact bayesian solution to the troublesome feature selection problem for naive bayes classifiers we demonstrate the utility of these techniques in the context of supervised classification showing empirically that model averaging consistently beats other generative bayesian network based models even when the generating model is not guaranteed to be a member of the class being averaged over we characterize the performance over several parameters on simulated and real world data

finding optimal bayesian networks using precedence constraints we consider the problem of finding a directed acyclic graph dag that optimizes a decomposable bayesian network score while in a favorable case an optimal dag can be found in polynomial time in the worst case the fastest known algorithms rely on dynamic programming across the node subsets taking time and space 2n to within a factor polynomial in the number of nodes n in practice these algorithms are feasible to networks of at most around 30 nodes mainly due to the large space requirement here we generalize the dynamic programming approach to enhance its feasibility in three dimensions first the user may trade space against time second the proposed algorithms easily and efficiently parallelize onto thousands of processors third the algorithms can exploit any prior knowledge about the precedence relation on the nodes underlying all these results is the key observation that given a partial order p on the nodes an optimal dag compatible with p can be found in time and space roughly proportional to the number of ideals of p which can be significantly less than 2n considering sufficiently many carefully chosen partial orders guarantees that a globally optimal dag will be found aside from the generic scheme we present and analyze concrete tradeoff schemes based on parallel bucket orders

geometric aspects of sturm liouville problems ii space of boundary conditions for left definiteness for a given regular sturm liouville equation with an indefinite weight function we explicitly describe the space of left definite selfadjoint boundary conditions the description only uses one value of a fundamental solution of the matrix form of the equation as a consequence we show that this space has the shape of a solid consisting of two cones sharing a common base

on the properties of concept classes induced by multivalued bayesian networks the concept class c n induced by a bayesian network n can be embedded into some euclidean inner product space the vapnik chervonenkis vc dimension of the concept class and the minimum dimension of the inner product space are very important indicators for evaluating the classification capability of the bayesian network in this paper we investigate the properties of the concept class c n k induced by a multivalued bayesian network n k where each node x i of n k is a k valued variable we focus on the values of two dimensions i the vc dimension of the concept class c n k denoted as vcdim n k and ii the minimum dimension of the inner product space into which c n k can be embedded we show that the values of these two dimensions are k n 1 for fully connected k valued bayesian networks n f k with n variables for non fully connected k valued bayesian networks n k without v structure we prove that the two dimensional values are k 1 i 1 nk m i 1 where m i denotes the number of parents for the i t h variable we also derive the upper and lower bounds on the minimum dimension of the inner product space induced by non fully connected bayesian networks

learning high dimensional directed acyclic graphs with latent and selection variables we consider the problem of learning causal information between random variables in directed acyclic graphs dags when allowing arbitrarily many latent and selection variables the fci fast causal inference algorithm has been explicitly designed to infer conditional independence and causal information in such settings however fci is computationally infeasible for large graphs we therefore propose the new rfci algorithm which is much faster than fci in some situations the output of rfci is slightly less informative in particular with respect to conditional independence information however we prove that any causal information in the output of rfci is correct in the asymptotic limit we also define a class of graphs on which the outputs of fci and rfci are identical we prove consistency of fci and rfci in sparse high dimensional settings and demonstrate in simulations that the estimation performances of the algorithms are very similar all software is implemented in the r package pcalg

projected newton methods for optimization problems with simple constraints we consider the problem min f x x geqq 0 and propose algorithms of the form x k 1 x k alpha k d k nabla f x k where cdot denotes projection on the positive orthant alpha k is a stepsize chosen by an armijo like rule and d k is a positive definite symmetric matrix which is partly diagonal we show that d k can be calculated simply on the basis of second derivatives of f so that the resulting newton like algorithm has a typically superlinear rate of convergence with other choices of d k convergence at a typically linear rate is obtained the algorithms are almost as simple as their unconstrained counterparts they are well suited for problems of large dimension such as those arising in optimal control while being competitive with existing methods for low dimensional problems the effectiveness of the newton like algorithm is demonstrated via computational examples involving as many as 10 000 variables extensions to general linearly constrained pr

estimating high dimensional intervention effects from observational data we assume that we have observational data generated from an unknown underlying directed acyclic graph dag model a dag is typically not identifiable from observational data but it is possible to consistently estimate the equivalence class of a dag moreover for any given dag causal effects can be estimated using intervention calculus in this paper we combine these two parts for each dag in the estimated equivalence class we use intervention calculus to estimate the causal effects of the covariates on the response this yields a collection of estimated causal effects for each covariate we show that the distinct values in this set can be consistently estimated by an algorithm that uses only local information of the graph this local approach is computationally fast and feasible in high dimensional problems we propose to use summary measures of the set of possible causal effects to determine variable importance in particular we use the minimum absolute value of this set since that is a lower bound on the size of the causal effect we demonstrate the merits of our methods in a simulation study and on a data set about riboflavin production

the explicit linear quadratic regulator for constrained systems we present a technique to compute the explicit state feedback solution to both the finite and infinite horizon linear quadratic optimal control problem subject to state and input constraints we show that this closed form solution is piecewise linear and continuous as a practical consequence of the result constrained linear quadratic regulation becomes attractive also for systems with high sampling rates as on line quadratic programming solvers are no more required for the implementation

local causal and markov blanket induction for causal discovery and feature selection for classification part i algorithms and empirical evaluation we present an algorithmic framework for learning local causal structure around target variables of interest in the form of direct causes effects and markov blankets applicable to very large data sets with relatively small samples the selected feature sets can be used for causal discovery and classification the framework generalized local learning or gll can be instantiated in numerous ways giving rise to both existing state of the art as well as novel algorithms the resulting algorithms are sound under well defined sufficient conditions in a first set of experiments we evaluate several algorithms derived from this framework in terms of predictivity and feature set parsimony and compare to other local causal discovery methods and to state of the art non causal feature selection methods using real data a second set of experimental evaluations compares the algorithms in terms of ability to induce local causal neighborhoods using simulated and resimulated data and examines the relation of predictivity with causal induction performance our experiments demonstrate consistently with causal feature selection theory that local causal feature selection methods under broad assumptions encompassing appropriate family of distributions types of classifiers and loss functions exhibit strong feature set parsimony high predictivity and local causal interpretability although non causal feature selection methods are often used in practice to shed light on causal relationships we find that they cannot be interpreted causally even when they achieve excellent predictivity therefore we conclude that only local causal techniques should be used when insight into causal structure is sought in a companion paper we examine in depth the behavior of gll algorithms provide extensions and show how local techniques can be used for scalable and accurate global causal graph learning

multiplicative updates for nonnegative quadratic programming many problems in neural computation and statistical learning involve optimizations with nonnegativity constraints in this article we study convex problems in quadratic programming where the optimization is confined to an axis aligned region in the nonnegative orthant for these problems we derive multiplicative updates that improve the value of the objective function at each iteration and converge monotonically to the global minimum the updates have a simple closed form and do not involve any heuristics or free parameters that must be tuned to ensure convergence despite their simplicity they differ strikingly in form from other multiplicative updates used in machine learning we provide complete proofs of convergence for these updates and describe their application to problems in signal processing and pattern recognition

reversible mcmc on markov equivalence classes of sparse directed acyclic graphs author s he y jia j yu b abstract graphical models are popular statistical tools which are used to represent dependent or causal complex systems statistically equivalent causal or directed graphical models are said to belong to a markov equivalent class it is of great interest to describe and understand the space of such classes however with currently known algorithms sampling over such classes is only feasible for graphs with fewer than approximately 20 vertices in this paper we design reversible irreducible markov chains on the space of markov equivalent classes by proposing a perfect set of operators that determine the transitions of the markov chain the stationary distribution of a proposed markov chain has a closed form and can be computed easily specifically we construct a concrete perfect set of operators on sparse markov equivalence classes by introducing appropriate conditions on each possible operator algorithms and their accelerated versions are provided to efficiently generate markov chains and to explore properties of markov equivalence classes of sparse directed acyclic graphs dags with thousands of vertices we find experimentally that in most markov equivalence classes of sparse dags 1 most edges are directed 2 most undirected subgraphs are small and 3 the number of these undirected subgraphs grows approximately linearly with the number of vertices institute of mathematical statistics 2013

optimal predictive model selection often the goal of model selection is to choose a model for future prediction and it is natural to measure the accuracy of a future prediction by squared error loss under the bayesian approach it is commonly perceived that the optimal predictive model is the model with highest posterior probability but this is not necessarily the case in this paper we show that for selection among normal linear models the optimal predictive model is often the median probability model which is defined as the model consisting of those variables which have overall posterior probability greater than or equal to 1 2 of being in a model the median probability model often differs from the highest probability model

characterization and greedy learning of interventional markov equivalence classes of directed acyclic graphs the investigation of directed acyclic graphs dags encoding the same markov property that is the same conditional independence relations of multivariate observational distributions has a long tradition many algorithms exist for model selection and structure learning in markov equivalence classes in this paper we extend the notion of markov equivalence of dags to the case of interventional distributions arising from multiple intervention experiments we show that under reasonable assumptions on the intervention experiments interventionalmarkov equivalence defines a finer partitioning of dags than observational markov equivalence and hence improves the identifiability of causal models we give a graph theoretic criterion for two dags being markov equivalent under interventions and show that each interventional markov equivalence class can analogously to the observational case be uniquely represented by a chain graph called interventional essential graph also known as cpdag in the observational case these are key insights for deriving a generalization of the greedy equivalence search algorithm aimed at structure learning from interventional data this new algorithm is evaluated in a simulation study

rates of convergence of posterior distributions we compute the rate at which the posterior distribution concentrates around the true parameter value the spaces we work in are quite general and include infinite dimensional cases the rates are driven by two quantities the size of the space as measured by bracketing entropy and the degree to which the prior concentrates in a small ball around the true parameter we consider two examples