clustered model reduction of positive directed networks this paper proposes a clustered model reduction method for semistable positive linear systems evolving over directed networks in this method we construct a set of clusters i e disjoint sets of state variables based on a notion of cluster reducibility defined as the uncontrollability of local states by aggregating the reducible clusters with aggregation coefficients associated with the frobenius eigenvector we obtain an approximate model that preserves not only a network structure among clusters but also several fundamental properties such as semistability positivity and steady state characteristics furthermore it is found that the cluster reducibility can be characterized for semistable systems based on a projected controllability gramian that leads to an a priori h 2 error bound of the state discrepancy caused by aggregation the efficiency of the proposed method is demonstrated through an illustrative example of enzyme catalyzed reaction systems described by a chemical master equation this captures the time evolution of chemical reaction systems in terms of a set of ordinary differential equations

learning equivalence classes of bayesian network structures two bayesian network structures are said to be equivalent if the set of distributions that can be represented with one of those structures is identical to the set of distributions that can be represented with the other many scoring criteria that are used to learn bayesian network structures from data are score equivalent that is these criteria do not distinguish among networks that are equivalent in this paper we consider using a score equivalent criterion in conjunction with a heuristic search algorithm to perform model selection or model averaging we argue that it is often appropriate to search among equivalence classes of network structures as opposed to the more common approach of searching among individual bayesian network structures we describe a convenient graphical representation for an equivalence class of structures and introduce a set of operators that can be applied to that representation by a search algorithm to move among equivalence classes we show that our equivalence class operators can be scored locally and thus share the computational efficiency of traditional operators defined for individual structures we show experimentally that a greedy model selection algorithm using our representation yields slightly higher scoring structures than the traditional approach without any additional time overhead and we argue that more sophisticated search algorithms are likely to benefit much more

causal inference through a witness protection program one of the most fundamental problems in causal inference is the estimation of a causal effect when treatment and outcome are confounded this is difficult in an observational study because one has no direct evidence that all confounders have been adjusted for we introduce a novel approach for estimating causal effects that exploits observational conditional independencies to suggest weak paths in an unknown causal graph the widely used faithfulness condition of spirtes et al is relaxed to allow for varying degrees of path cancellations that imply conditional independencies but do not rule out the existence of confounding causal paths the output is a posterior distribution over bounds on the average causal effect via a linear programming approach and bayesian inference we claim this approach should be used in regular practice as a complement to other tools in observational studies

a characterization of markov equivalence classes for acyclic digraphs undirected graphs and acyclic digraphs adgs as well as their mutual extension to chain graphs are widely used to describe dependencies among variables in multivariate distributions in particular the likelihood functions of adg models admit convenient recursive factorizations that often allow explicit maximum likelihood estimates and that are well suited to building bayesian networks for expert systems whereas the undirected graph associated with a dependence model is uniquely determined there may however be many adgs that determine the same dependence markov model thus the family of all adgs with a given set of vertices is naturally partitioned into markov equivalence classes each class being associated with a unique statistical model statistical procedures such as model selection or model averaging that fail to take into account these equivalence classes may incur substantial computational or other inefficiencies here it is shown that each markov equivalence class is uniquely determined by a single chain graph the essential graph that is itself simultaneously markov equivalent to all adgs in the equivalence class essential graphs are characterized a polynomial time algorithm for their construction is given and their applications to model selection and other statistical questions are described

on modifications of newton s second law and linear continuum elastodynamics in this paper we suggest a new perspective where newton s second law of motion is replaced by a more general law which is a better approximation for describing the motion of seemingly rigid macroscopic bodies we confirm a finding of willis that the density of a body at a given frequency of oscillation can be anisotropic the relation between the force and the acceleration is non local but causal in time conversely for every response function satisfying these properties and having the appropriate high frequency limit there is a model which realizes that response function in many circumstances the differences between newton s second law and the new law are small but there are circumstances such as in specially designed composite materials where the difference is enormous for bodies which are not seemingly rigid the continuum equations of elastodynamics govern behaviour and also need to be modified the modified versions of these equations presented here are a generalization of the equations proposed by willis to describe elastodynamics in composite materials it is argued that these new sets of equations may apply to all physical materials not just composites the willis equations govern the behaviour of the average displacement field whereas one set of new equations governs the behaviour of the averageweighted displacement field where the weighted displacement field may attach zero weight to hidden areas in the material where the displacement may be unobservable or not defined from knowledge of the average weighted displacement field one obtains an approximate formula for the ensemble averaged energy density two other sets of new equations govern the behaviour when the microstructure has microinertia i e where there are internal spinning masses below the chosen scale of continuum modelling in the first set the average displacement field is assumed to be observable while in the second set an average weighted displacement field is assumed to be observable

right singular vector projection graphs fast high dimensional covariance matrix estimation under latent confounding we consider the problem of estimating a high dimensional p p covariance matrix given n observations of confounded data with covariance t where is an unknown p q matrix of latent factor loadings we propose a simple and scalable estimator based on the projection onto the right singular vectors of the observed data matrix which we call right singular vector projection rsvp our theoretical analysis of this method reveals that in contrast with approaches based on the removal of principal components rsvp can cope well with settings where the smallest eigenvalue of t is relatively close to the largest eigenvalue of as well as when the eigenvalues of t are diverging fast rsvp does not require knowledge or estimation of the number of latent factors q but it recovers only up to an unknown positive scale factor we argue that this suffices in many applications e g if an estimate of the correlation matrix is desired we also show that by using subsampling we can further improve the performance of the method we demonstrate the favourable performance of rsvp through simulation experiments and an analysis of gene expression data sets collated by the gtex consortium

parallel solutions of very large sparse lyapunov equations by balanced bbd decompositions in this paper a method for the parallel solution of large sparse matrix equations is presented the method is based on the balanced bordered block diagonal bbd decomposition which is applied in conjunction with the successive over relaxation sor iterative method to solve large lyapunov equations in the kronecker sum representation a variety of experimental results will be presented including solutions of lyapunov equations with matrices as large as 1993 spl times 1993

optimum parameters for the generalized adi method in the context of the generalized adi method we are concerned with the problem of finding in the set of rational functions r with numerator degree m and denominator degree n an element r that minimizes where e f are disjoint real intervals by extending a recent analysis by levin and saff we present an explicit formula for choosing the pair m n for given m n furthermore we provide a characterization of r and a remes type algorithm for its determination extensive numerical computations furnish some comparison of r with asymptotically optimal solutions based on fejer walsh and leja bagby points

systems of coupled generalized sylvester matrix equations this paper studies some systems of coupled generalized sylvester matrix equations we present some necessary and sufficient conditions for the solvability to these systems we give the expressions of the general solutions to the systems when their solvability conditions are satisfied

analytical formulation of three dimensional dynamic homogenization for periodic elastic systems homogenization of the equations of motion for a three dimensional periodic elastic system is considered expressions are obtained for the fully dynamic effective material parameters governing the spatially averaged fields by using the plane wave expansion method the effective equations are of willis form with coupling between momentum and stress and tensorial inertia the formulation demonstrates that the willis equations of elastodynamics are closed under homogenization the effective material parameters are obtained for arbitrary frequency and wavenumber combinations including but not restricted to bloch wave branches for wave propagation in the periodic medium numerical examples for a one dimensional system illustrate the frequency dependence of the parameters on bloch wave branches and provide a comparison with an alternative dynamic effective medium theory which also reduces to willis form but with different effective moduli

pc algorithm for nonparanormal graphical models the pc algorithm uses conditional independence tests for model selection in graphical modeling with acyclic directed graphs in gaussian models tests of conditional independence are typically based on pearson correlations and high dimensional consistency results have been obtained for the pc algorithm in this setting analyzing the error propagation from marginal to partial correlations we prove that high dimensional consistency carries over to a broader class of gaussian copula or nonparanormal models when using rank based measures of correlation for graph sequences with bounded degree our consistency result is as strong as prior gaussian results in simulations the rank pc algorithm works as well as the pearson pc algorithm for normal data and considerably better for non normal data all the while incurring a negligible increase of computation time while our interest is in the pc algorithm the presented analysis of error propagation could be applied to other algorithms that test the vanishing of low order partial correlations

an implicit preconditioning strategy for large scale generalized sylvester equations abstract large scale generalized sylvester equations appear in several important applications although the involved operator is linear solving them requires specialized techniques different numerical methods have been designed to solve them including direct factorization methods suitable for small size problems and krylov type iterative methods for large scale problems for these iterative schemes preconditioning is always a difficult task that deserves to be addressed we present and analyze an implicit preconditioning strategy specially designed for solving generalized sylvester equations that uses a preconditioned residual direction at every iteration the advantage is that the preconditioned direction is built implicitly avoiding the explicit knowledge of the given matrices only the effect of the matrix vector product with the given matrices is required we present encouraging numerical experiments for a set of different problems coming from several applications

computing real low rank solutions of sylvester equations by the factored adi method abstract we investigate the factored alternating directions implicit adi iteration for large and sparse sylvester equations a novel low rank expression for the associated sylvester residual is established which enables cheap computations of the residual norm along the iteration and which yields a reformulated factored adi iteration the application to generalized sylvester equations is considered as well we also discuss the efficient handling of complex shift parameters and reveal interconnections between the adi iterates w r t those complex shifts this yields a further modification of the factored adi iteration which employs only an absolutely necessary amount of complex arithmetic operations and storage and which produces low rank solution factors consisting of entirely real data certain linear matrix equations such as e g cross gramian sylvester and stein equations are in fact special cases of generalized sylvester equations and we show how specially tailored low rank adi iterations can be deduced from the generalized factored adi iteration

estimating high dimensional directed acyclic graphs with the pc algorithm we consider the pc algorithm spirtes et al 2000 for estimating the skeleton and equivalence class of a very high dimensional directed acyclic graph dag with corresponding gaussian distribution the pc algorithm is computationally feasible and often very fast for sparse problems with many nodes variables and it has the attractive property to automatically achieve high computational efficiency as a function of sparseness of the true underlying dag we prove uniform consistency of the algorithm for very high dimensional sparse dags where the number of nodes is allowed to quickly grow with sample size n as fast as o na for any 0 a the sparseness assumption is rather minimal requiring only that the neighborhoods in the dag are of lower order than sample size n we also demonstrate the pc algorithm for simulated data

stabilisation of large scale nonlinear systems by modifying the interconnection network in this article we develop a new strategy for the robust stabilisation of large scale singular systems the models that we will be concerned with consist of a large number of nonlinear subsystems which are linked through a linear interconnection network this approach does not require the availability of control inputs and focuses instead on variable parameters in the interconnection network it is shown that the determination of appropriate parameter values reduces to a constrained static output feedback problem which can be solved using linear matrix inequalities the proposed method is designed to minimise the number of optimisation variables and is therefore suitable for large scale applications it can also incorporate information structure constraints and uncertainties in the system model

a low rank krylov squared smith method for large scale discrete time lyapunov equations abstract the squared smith method is adapted to solve large scale discrete time lyapunov matrix equations the adaptation uses a krylov subspace to generate the squared smith iteration in a low rank form a restarting mechanism is employed to cope with the increase of memory storage of the krylov basis theoretical aspects of the algorithm are presented several numerical illustrations are reported

effective constitutive relations for waves in composites and metamaterials definitions of effective fields for a randomly inhomogeneous material are offered which guarantee automatic satisfaction of the equations of motion the important case of a medium with periodic microstructure is included in this special case the definitions are completely explicit and can be applied without reference to random media the presentation is mostly expressed in terms of electromagnetic waves the reasoning is applicable also to other types of waves and its realization for elastodynamics is briefly outlined towards the end some of the effective fields are defined directly as ensemble averages ensuring the exact satisfaction of the equations of motion but the effective kinematic fields to which they are related are defined more generally as weighted averages the main result of this work is an explicit formula for the tensor of effective properties the important issue of uniqueness or not of the effective properties is explained and resolved self adjointness of the original problem is not assumed an attractive feature of the formulation is that self adjointness at the local level implies self adjointness at the level of the effective medium

high dimensional causal discovery under non gaussianity we consider graphical models based on a recursive system of linear structural equations this implies that there is an ordering of the variables such that each observed variable yv is a linear function of a variable specific error term and the other observed variables yu with u v the causal relationships i e which other variables the linear functions depend on can be described using a directed graph it has previously been shown that when the variable specific error terms are non gaussian the exact causal graph as opposed to a markov equivalence class can be consistently estimated from observational data we propose an algorithm that yields consistent estimates of the graph also in high dimensional settings in which the number of variables may grow at a faster rate than the number of observations but in which the underlying causal structure features suitable sparsity specifically the maximum in degree of the graph is controlled our theoretical analysis is couched in the setting of log concave error distributions

krylov subspace methods for solving large unsymmetric linear systems some algorithms based upon a projection process onto the krylov subspace k sub m span r sub 0 ar sub 0 a sup m 1 r sub 0 are developed generalizing the method of conjugate gradients to unsymmetric systems these methods are extensions of arnoldi s algorithm for solving eigenvalue problems the convergence is analyzed in terms of the distance of the solution to the subspace k sub m and some error bounds are established showing in particular a similarity with the conjugate gradient method for symmetric matrices when the eigenvalues are real several numerical experiments are described and discussed

alternating direction implicit iteration for systems with complex spectra the alternating direction implicit adi iteration model problem has hitherto been the discretized dirichlet problem with an spd matrix splitting for which optimum iteration parameters are obtained as the solution due to w b jordan of a rational chebyshev minimax problem over the reals recently disclosed application to a class of sylvester equations bf ax bf xb bf c with bf a and bf b positive real matrices required generalization to complex spectra in the positive real half plane theory is developed here for applying jordan s parameters to complex domains which remain close to the real line and for predicting associated error reduction numerical studies are presented in support of this theory this analysis provided a basis for analysis of more general complex spectra which is reported in another paper

algorithms for model reduction of large dynamical systems three algorithms for the model reduction of large scale continuous time time invariant linear dynamical systems with a sparse or structured transition matrix and a small number of inputs and outputs are described they rely on low rank approximations to the controllability and observability gramians which can efficiently be computed by adi based iterative low rank methods the first two model reduction methods are closely related to the well known square root method and schur method which are balanced truncation techniques the third method is a heuristic balancing free technique the performance of the model reduction algorithms is studied in numerical experiments

a generalized adi iterative method the adi iterative method for the solution of sylvester s equationax xb c proceeds by strictly alternating between the solution of the two equations begin gathered left a delta k 1 i right x 2k 1 x 2k left b delta k 1 i right c hfill x 2k 2 left b tau k 1 i right left a tau k 1 i right x 2k 1 c hfill end gathered fork 0 1 2 herex o is a given initial approximate solution and the k and k are real or complex parameters chosen so that the computed approximate solutionsx k converge rapidly to the solutionx of the sylvester equation ask increases this paper discusses the possibility of solving one of the equations in the adi iterative method more often than the other one i e relaxing the strict alternation requirement in order to achieve a higher rate of convergence our analysis based on potential theory shows that this generalization of the adi iterative method can give faster convergence than when strict alternation is required

a generalization of the hermitian and skew hermitian splitting iteration method for solving sylvester equations this paper is concerned with a generalization of the hermitian and skew hermitian splitting hss iteration method for solving large sparse continuous sylvester equations the analysis shows that the ghss iteration method will converge under certain assumptions numerical results show that this new method is more efficient and robust than the existing ones

optimal quadrature for analytic functions abstract let g be a domain in the complex plane which is symmetric with respect to the real axis and contains 1 1 for a measure on 1 1 satisfying a regularity condition we determine the geometric rate of the error of integration measured uniformly on the class of functions analytic in g and bounded by 1 if the integrals are replaced by optimal interpolatory quadrature formulas with n nodes we show that this rate is obtained for modified gauss quadrature formulas with respect to certain varying weights

optimal alternating direction implicit parameters for nonsymmetric systems of linear equations the determination of optimal parameters for the well known alternating direction implicit adi method leads to a minimization problem for rational functions of given degree l on compact sets e and f that contain the eigenvalues of the matrices involved if e and f are real intervals the solution of this problem was given by zolotarev more than 100 years ago and by wachspress in the 1960s however in the complex case e g if the linear system arises from the discretization of non self adjoint elliptic differential equations little is known about this problem in this paper two approaches to the zolotarev problem in the complex plane are considered first generalized leja points are considered as an example for rational functions which are asymptotically minimal in addition for rectangular eigenvalue domains e and f the exact minimal solutions for l 2 are constructed and it is shown by numerical examples that the results obtained using the optimal parameters of degree 2 are comparable to tho

estimation of sparse directed acyclic graphs for multivariate counts data the next generation sequencing data called high throughput sequencing data are recorded as count data which are generally far from normal distribution under the assumption that the count data follow the poisson log normal distribution this article provides an l1 penalized likelihood framework and an efficient search algorithm to estimate the structure of sparse directed acyclic graphs dags for multivariate counts data in searching for the solution we use iterative optimization procedures to estimate the adjacency matrix and the variance matrix of the latent variables the simulation result shows that our proposed method outperforms the approach which assumes multivariate normal distributions and the log transformation approach it also shows that the proposed method outperforms the rank based pc method under sparse network or hub network structures as a real data example we demonstrate the efficiency of the proposed method in estimating the gene regulatory networks of the ovarian cancer study

methods for causal inference from gene perturbation experiments and validation inferring causal effects from observational and interventional data is a highly desirable but ambitious goal many of the computational and statistical methods are plagued by fundamental identifiability issues instability and unreliable performance especially for large scale systems with many measured variables we present software and provide some validation of a recently developed methodology based on an invariance principle called invariant causal prediction icp the icp method quantifies confidence probabilities for inferring causal structures and thus leads to more reliable and confirmatory statements for causal relations and predictions of external intervention effects we validate the icp method and some other procedures using large scale genome wide gene perturbation experiments in saccharomyces cerevisiae the results suggest that prediction and prioritization of future experimental interventions such as gene deletions can be improved by using our statistical inference techniques

preconditioned hss iteration method and its non alternating variant for continuous sylvester equations abstract bai 2010 proposed an efficient hermitian and skew hermitian splitting hss iteration method for solving a broad class of large sparse continuous sylvester equations to further improve the efficiency of the hss method in this paper we present a preconditioned hss phss iteration method and its non alternating variant nphss for this matrix equation the convergence properties of the phss and nphss methods are studied in depth and the quasi optimal values of the iteration parameters for the two methods are also derived moreover to reduce the computational cost we establish the inexact variants of the two iteration methods numerical experiments illustrate the efficiency and robustness of the two iteration methods and their inexact variants

extended bayesian information criteria for model selection with large model spaces summary the ordinary bayesian information criterion is too liberal for model selection when the model space is large in this paper we re examine the bayesian paradigm for model selection and propose an extended family of bayesian information criteria which take into account both the number of unknown parameters and the complexity of the model space their consistency is established in particular allowing the number of covariates to increase to infinity with the sample size their performance in various situations is evaluated by simulation studies it is demonstrated that the extended bayesian information criteria incur a small loss in the positive selection rate but tightly control the false discovery rate a desirable property in many applications the extended bayesian information criteria are extremely useful for variable selection in problems with a moderate sample size but with a huge number of covariates especially in genome wide association studies which are now an active area in genetics research

on an additive partial correlation operator and nonparametric estimation of graphical models we introduce an additive partial correlation operator as an extension of partial correlation to the nonlinear setting and use it to develop a new estimator for nonparametric graphical models our graphical models are based on additive conditional independence a statistical relation that captures the spirit of conditional independence without having to resort to high dimensional kernels for its estimation the additive partial correlation operator completely characterizes additive conditional independence and has the additional advantage of putting marginal variation on appropriate scales when evaluating interdependence which leads to more accurate statistical inference we establish the consistency of the proposed estimator through simulation experiments and analysis of the dream4 challenge dataset we demonstrate that our method performs better than existing methods in cases where the gaussian or copula gaussian assumption does not hold and that a more appropriate scaling for our method further enhances its performance

a hybrid arnoldi faber iterative method for nonsymmetric systems of linear equations we present here a new hybrid method for the iterative solution of large sparse nonsymmetric systems of linear equations say of the formax b wherea n n witha nonsingular andb n are given this hybrid method begins with a limited number of steps of the arnoldi method to obtain some information on the location of the spectrum ofa and then switches to a richardson iterative method based on faber polynomials for a polygonal domain the faber polynomials can be constructed recursively from the parameters in the schwarz christoffel mapping function in four specific numerical examples of non normal matrices we show that this hybrid algorithm converges quite well and is approximately as fast or faster than the hybrid gmres or restarted versions of the gmres algorithm it is however sensitive as other hybrid methods also are to the amount of information on the spectrum ofa acquired during the first arnoldi phase of this procedure

a new route to finding bounds on the generalized spectrum of many physical operators here we obtain bounds on the generalized spectrum of that operator whose inverse when it exists gives green s function we consider the wide range of physical problems that can be cast in a form where a constitutive equation j x l x e x h x with a source term h x holds for all x in some domain and relates fields e and j that satisfy appropriate differential constraints symbolized by e e 0 and j j where e 0 and j are orthogonal spaces that span the space h of square integrable fields in which h lies boundedness and coercivity conditions on the moduli l x ensure that there exists a unique e for any given h i e e g h which then establishes the existence of green s function g we show that the coercivity condition is guaranteed to hold if weaker conditions involving generalized quasiconvex functions are satisfied the advantage is that these weaker conditions are easier to verify and for multiphase materials they can be independent of the geometry of the phases for l x depending linearly on a vector of parameters z z1 z2 zn we obtain constraints on z that ensure that green s function exists and hence which provide bounds on the generalized spectrum here we obtain bounds on the generalized spectrum of that operator whose inverse when it exists gives green s function we consider the wide range of physical problems that can be cast in a form where a constitutive equation j x l x e x h x with a source term h x holds for all x in some domain and relates fields e and j that satisfy appropriate differential constraints symbolized by e e 0 and j j where e 0 and j are orthogonal spaces that span the space h of square integrable fields in which h lies boundedness and coercivity conditions on the moduli l x ensure that there exists a unique e for any given h i e e g h which then establishes the existence of green s function g we show that the coercivity condition is guaranteed to hold if weaker conditions involving generalized quasiconvex functions are satisfied the advantage is that these weaker conditions are easier to verify and for multiphase materials they can be independent of the geometry of the phases for l x depend

on the generalized adi method for the matrix equation x axb c the paper investigates a rational approximation problem in connection with the convergence analysis of the adi iterative method applied to the matrix equation x axb c in the real case a complete theory on the characterization and unicity of the optimal solution as well as on the convergence of a remes like algorithm for its computation is presented numerical experiments provide some comparison with asymptotically optimal solutions which have been previously reported particularly in applications for image restoration

a note on the numerical approximate solutions for generalized sylvester matrix equations with applications abstract in the present paper we propose a krylov subspace method for solving large and sparse generalized sylvester matrix equations the proposed method is an iterative projection method onto matrix krylov subspaces as a particular case we show how to adapt the ilu and the ssor preconditioners for solving large sylvester matrix equations numerical examples and applications to some pde s will be given

a new version of successive approximations method for solving sylvester matrix equations abstract this paper presents a new version of the successive approximations method for solving sylvester equations ax xb c where a and b are symmetric negative and positive definite matrices respectively this method is based on the block gmres sylvester method we also discuss the convergence of the new method some numerical experiments for obtaining the numerical solution of sylvester equations are given numerical experiments show that the solution of sylvester equations can be obtained with high accuracy and the new algorithm is a robust technique for solving sylvester equations

low rank approximate solutions to large sylvester matrix equations in the present paper we propose an arnoldi based method for solving large and sparse sylvester matrix equations with low rank right hand sides we will show how to extract low rank approximations via a matrix krylov subspace method we give some theoretical results such an expression of the exact solution and upper bounds for the norm of the error and for the residual numerical experiments will also be given to show the effectiveness of the proposed method

feje r walsh points for rational functions and their use in the adi iterative method abstract a new approach to the rational zolotarev problem in the complex plane is presented this minimization problem for rational functions arises from the determination of optimal parameters for the adi iterative method applied to nonsymmetric systems of linear equations based on a generalization due to walsh of the fejer points which are well known in connection to polynomial interpolation in the complex plane we are able to construct rational functions which are almost optimal for the rational zolotarev problem in an asymptotic sense for the construction of these fejer walsh points we make use of the schwarz christoffel mapping for doubly connected polygonal regions due to akhiezer and its recent implementation by dappen a numerical example illustrates the usefulness of the fejer walsh points as adi parameters

krylov subspace methods for solving large lyapunov equations this paper considers several methods for calculating low rank approximate solutions to large scale lyapunov equations of the form ap pa bb 0 the interest in this problem stems from model reduction where the task is to approximate high dimensional models by ones of lower order the two recently developed krylov subspace methods exploited in this paper are the arnoldi method saad math comput 37 1981 pp 105 126 and the generalised minimum residual method gmres saad and schultz siam j sci statist comput 7 1986 pp 856 869 exact expressions for the approximation errors incurred are derived in both cases the numerical solution of the low dimensional linear matrix equation arising from the gmres method is discussed and an algorithm for its solution is proposed low rank solutions of discrete time lyapunov equations and continuous time algebraic riccati equations are also considered throughout this paper the authors tackle problems in which b has more than one column with the

robust gaussian graphical modeling via l1 penalization gaussian graphical models have been widely used as an effective method for studying the conditional independency structure among genes and for constructing genetic networks however gene expression data typically have heavier tails or more outlying observations than the standard gaussian distribution such outliers in gene expression data can lead to wrong inference on the dependency structure among the genes we propose a l1 penalized estimation procedure for the sparse gaussian graphical models that is robustified against possible outliers the likelihood function is weighted according to how the observation is deviated where the deviation of the observation is measured based on its own likelihood an efficient computational algorithm based on the coordinate gradient descent method is developed to obtain the minimizer of the negative penalized robustified likelihood where nonzero elements of the concentration matrix represents the graphical links among the genes after the graphical structure is obtained we re estimate the positive definite concentration matrix using an iterative proportional fitting algorithm through simulations we demonstrate that the proposed robust method performs much better than the graphical lasso for the gaussian graphical models in terms of both graph structure selection and estimation when outliers are present we apply the robust estimation procedure to an analysis of yeast gene expression data and show that the resulting graph has better biological interpretation than that obtained from the graphical lasso

high dimensional semiparametric gaussian copula graphical models ing the spearman s rho and kendall s tau we prove that the nonparanormal skeptic achieves the optimal parametric rates of convergence for both graph recovery and parameter estimation this result suggests that the nonparanormal graphical models can be used as a safe replacement of the popular gaussian graphical models even when the data are truly gaussian besides theoretical analysis we also conduct thorough numerical simulations to compare the graph recovery performance of dierent estimators under both ideal and noisy settings the proposed methods are then applied on a largescale genomic dataset to illustrate their empirical usefulness the r package huge implementing the proposed methods is available on the comprehensive r archive network http cran r project org

theoretical framework and experimental procedure for modelling mesoscopic volume fraction stochastic fluctuations in fiber reinforced composites many engineering materials exhibit fluctuations and uncertainties on their macroscopic mechanical properties this randomness results from random fluctuations observed at a lower scale especially at the meso scale where microstructural uncertainties generally occur in the present paper we first propose a complete theoretical stochastic framework that is a relevant probabilistic model as well as a non intrusive stochastic solver in which the volume fraction at the microscale is modelled as a random field whose statistical reduction is performed using a karhunen loeve expansion then an experimental procedure dedicated to the identification of the parameters involved in the probabilistic model is presented and relies on a non destructive ultrasonic method the combination of the experimental results with a micromechanical analysis provides realizations of the volume fraction random field in particular it is shown that the volume fraction can be modelled by a homogeneous random field whose spatial correlation lengths are determined and may provide conditions on the size of the meso volumes to be considered

small sample statistical condition estimation of large scale generalized eigenvalue problems we consider the evaluation of the sensitivity or condition number of generalized eigenvalue problems for a large and sparse real matrix or matrix pair in r n n through some coupled sylvester equation using newton s method the technique of the statistical condition estimation has been adapted to the sensitivity of symmetric matrices as well as general matrices with special structures under some assumptions on various types of perturbations

a reflection on the implicitly restarted arnoldi method for computing eigenvalues near a vertical line abstract in this article we will study the link between a method for computing eigenvalues closest to the imaginary axis and the implicitly restarted arnoldi method the extension to computing eigenvalues closest to a vertical line is straightforward by incorporating a shift without loss of generality we will restrict ourselves here to computing eigenvalues closest to the imaginary axis in a recent publication meerbergen and spence discussed a new approach for detecting purely imaginary eigenvalues corresponding to hopf bifurcations which is of interest for the stability of dynamical systems the novel method is based on inverse iteration inverse power method applied on a lyapunov like eigenvalue problem to reduce the computational overhead significantly a projection was added this method can also be used for computing eigenvalues of a matrix pencil near a vertical line in the complex plane we will prove in this paper that the combination of inverse iteration with the projection step is equivalent to sorensen s implicitly restarted arnoldi method utilizing well chosen shifts

krylov subspace methods for the generalized sylvester equation abstract in the paper we propose galerkin and minimal residual methods for iteratively solving generalized sylvester equations of the form axb x c the algorithms use krylov subspace for which orthogonal basis are generated by the arnoldi process and reduce the storage space required by using the structure of the matrix we give some convergence results and present numerical experiments for large problems to show that our methods are efficient

sylvester tikhonov regularization methods in image restoration in this paper we consider large scale linear discrete ill posed problems where the right hand side contains noise regularization techniques such as tikhonov regularization are needed to control the effect of the noise on the solution in many applications such as in image restoration the coefficient matrix is given as a kronecker product of two matrices and then tikhonov regularization problem leads to the generalized sylvester matrix equation for large scale problems we use the global gmres method which is an orthogonal projection method onto a matrix krylov subspace we present some theoretical results and give numerical tests in image restoration

solution of underdetermined sylvester equations in sensor array signal processing abstract we propose efficient solution of an underdetermined matrix equation the matrix equations studied here appear in the study of subspace based high resolution multiparameter direction of arrival estimation the proposed gradient based technique is at least an order of magnitude computational improvement over the existing approaches simulation results are provided to illustrated the proposed technique

partitioned krylov subspace iteration in implicit runge kutta methods abstract application of the method of lines to partial differential equation leads to very large sparse systems of ordinary differential equations which are usually stiff we consider high order implicit runge kutta methods for the time integration of these systems and propose a partitioned krylov method for the solution of the resulting linear systems it is shown that this method allows for parallel computation in the construction of krylov subspaces and in a few examples it converges much faster than gmres

the sylvester equation and approximate balanced reduction abstract the purpose of this paper is to investigate the problem of iterative computation of approximately balanced reduced order systems the resulting approach is completely automatic once an error tolerance is specified and also yields an error bound this is to be contrasted with existing projection methods namely pade via lanczos pvl and rational krylov which do not satisfy these properties our approach is based on the computation and approximation of the cross gramian of the system the cross gramian is the solution of a sylvester equation and therefore some effort is dedicated to the study of this equation leading to some new insights our method produces a low rank approximation to this gramian in factored form and thus directly provides a reduced order model and a reduced basis for the original system it is well suited to large scale problems because there are no matrix factorizations of the large sparse system matrix only matrix vector products are required

an iterative svd krylov based method for model reduction of large scale dynamical systems abstract in this paper we propose a model reduction algorithm for approximation of large scale linear time invariant dynamical systems the method is a two sided projection combining features of the singular value decomposition svd based and the krylov based model reduction techniques while the svd side of the projection depends on the observability gramian the krylov side is obtained via iterative rational krylov steps the reduced model is asymptotically stable matches certain moments and solves a restricted h 2 minimization problem we present modifications to the proposed approach for employing low rank gramians in the reduction step and also for reducing discrete time systems several numerical examples from various disciplines verify the effectiveness of the proposed approach it performs significantly better than the q cover a yousouff r e skelton covariance equivalent realizations with applications to model reduction of large scale systems in c t leondes ed control and dynamic systems vol 22 academic press 1985 pp 273 348 a yousouff d a wagie r e skelton linear system approximation via covariance equivalent realizations journal of math anal and appl 196 1985 91 115 and the least squares s gugercin a c antoulas model reduction of large scale systems by least squares linear algebra appl 415 2 3 2006 290 321 methods that have a similar projection structure to the proposed method also in terms of both the h 2 and h error measures its performance is comparable to or sometimes better than that of balanced truncation moreover the method proves to be robust with respect to the perturbations due to usage of approximate gramians

sor for ax xb c we consider a new approach to the block sor method applied to linear systems of equations which can be written as a matrix equation ax xb c such systems arise for example from finite difference discretizations of seperable elliptic boundary value problems on rectangular domains on one hand this gives us an iterative method for the solution of such matrix equations e g lyapunov s matrix equation where b at and on the other hand the problem of choosing appropriate parameters for the block sor method can be written in a more compact form which may be helpful especially for non self adjoint problems i e if a and b are nonsymmetric using this technique we determine under more general assumptions than those of chin and manteuffel the optimal sor parameters for the model problem of a convection diffusion equation

projection methods for large lyapunov matrix equations abstract in the present paper we propose krylov subspace methods for solving large lyapunov matrix equations of the form ax xat bbt 0 where a and b are real n n and n s matrices respectively with s n such problems appear in many areas of control theory such as the computation of hankel singular values model reduction algorithms and others the proposed methods are based on the arnoldi process we show how to extract low rank approximate solutions to lyapunov matrix equations and give some theoretical results finally some numerical tests will be reported to illustrate the effectiveness of the proposed method

minimal residual methods augmented with eigenvectors for solving sylvester equations and generalized sylvester equations in this paper we propose minimal residual methods augmented with eigenvectors for solving large sylvester matrix equations ax xb c and generalized sylvester matrix equations axb x c the subspace from which the approximate solution is extracted is the kronecker product subspace of two augmented krylov subspaces numerical experiments report the effectiveness of these methods

an alternative extended block arnoldi method for solving low rank sylvester equations abstract projection methods based on krylov subspaces are an effective tool for solving low rank sylvester matrix equations in the large scale cases the initial problem is projected onto a sequence of nested subspaces at each step of the process a reduced sylvester equation whose coefficient matrices are restrictions of the original coefficient matrices to the projection subspaces are solved by a direct method in this paper after recalling some important properties about the extended block arnoldi process we propose an alternative approach that enables us to determine approximate solutions by solving projected and reduced sylvester equations whose coefficient matrices are restrictions of the inverse of the original matrices some numerical experiments are given in order to illustrate the performance of our approach

least squares symmetric and skew symmetric solutions of the generalized sylvester matrix equation σi 1 s a i x b i σj 1 t c j y d j e the generalized sylvester matrix equation i 1 s a i x b i j 1 t c j y d j e with unknown matrices x and y is encountered in many system and control applications in this paper a direct method is established to solve the least squares symmetric and skew symmetric solutions of the equation by using the kronecker product and the generalized inverses and the expression of the solution set s are provided moreover an optimal approximation between a given matrix pair and the affine subspace s is discussed and an explicit formula for the unique optimal approximation solution is presented finally two numerical examples are given which demonstrate that the introduced algorithm is quite efficient

iterative methods for x axb c alternating direction implicit iterative methods for the solution of matrix equations of the form x axb c are described a convergence analysis based on potential theory shows that iterating in one direction more than the other can give faster convergence than strict alternation of directions computed examples illustrate an application to the restoration of images degraded by noise

on short recurrence krylov type methods for linear systems with many right hand sides block and global krylov subspace methods have been proposed as methods adapted to the situation where one iteratively solves systems with the same matrix and several right hand sides these methods are advantageous since they allow to cast the major part of the arithmetic in terms of matrix block vector products and since in the block case they take their iterates from a potentially richer subspace in this paper we consider the most established krylov subspace methods which rely on short recurrences i e bicg qmr and bicgstab we propose modifications of their block variants which increase numerical stability thus at least partly curing a problem previously observed by several authors moreover we develop modifications of the global variants which almost halve the number of matrix vector multiplications we present a discussion as well as numerical evidence which both indicate that the additional work present in the block methods can be substantial and that the new economic versions of the global bicg and qmr method can be considered as good alternatives to the bicgstab variants

on positive definite and skew hermitian splitting iteration methods for continuous sylvester equation ax xb c in this paper we present a positive definite and skew hermitian splitting pss iteration method for continuous sylvester equations ax xb c with positive definite semi definite matrices the theoretical analysis shows that the pss iteration method will converge unconditionally and the optimal parameter of the new method is presented moreover to reduce the computing cost an inexact variant of the pss iteration method ipss and the analysis of its convergence property in detail have been established numerical results show that this new method and its inexact invariant are efficient and robust solvers for this class of continuous sylvester equations

learning high dimensional directed acyclic graphs with latent and selection variables we consider the problem of learning causal information between random variables in directed acyclic graphs dags when allowing arbitrarily many latent and selection variables the fci fast causal inference algorithm has been explicitly designed to infer conditional independence and causal information in such settings however fci is computationally infeasible for large graphs we therefore propose the new rfci algorithm which is much faster than fci in some situations the output of rfci is slightly less informative in particular with respect to conditional independence information however we prove that any causal information in the output of rfci is correct in the asymptotic limit we also define a class of graphs on which the outputs of fci and rfci are identical we prove consistency of fci and rfci in sparse high dimensional settings and demonstrate in simulations that the estimation performances of the algorithms are very similar all software is implemented in the r package pcalg

adi preconditioned krylov methods for large lyapunov matrix equations abstract in the present paper we propose preconditioned krylov methods for solving large lyapunov matrix equations ax xa t bb t 0 such problems appear in control theory model reduction circuit simulation and others using the alternating direction implicit adi iteration method we transform the original lyapunov equation to an equivalent symmetric stein equation depending on some adi parameters we then define the smith and the low rank adi preconditioners to solve the obtained stein matrix equation we apply the global arnoldi method and get low rank approximate solutions we give some theoretical results and report numerical tests to show the effectiveness of the proposed approaches

optimal rational functions for the generalized zolotarev problem in the complex plane it has been long recognized that the determination of optimal parameters for the classical alternating direction implicit adi method leads to the zolotarev problem min r in r nn frac max r z z in e min r z z in f for disjoint compact sets e f subseteq mathbb c where rnn is the collection of rational functions of order n in the case where e and f are real intervals it was more recently pointed out that if they have different lengths it is of interest to generalize the foregoing problem to the set rmn with unequal numerator degree m and denominator degree n the object of the paper is to investigate the generalized zolotarev problem in the complex plane a method is proposed to construct the optimal rational function which is then applied to the particular example of two line segments e on the real axis f parallel to the imaginary axis numerical experiments show that the improvement on the classical solution may be amazingly great explicit expressions are also provided for special values of data

krylov subspace methods for the sylvester equation abstract we describe galerkin and minimal residual algorithms for the solution of sylvester s equation ax xb c the algorithms use krylov subspaces forwhich orthogonal bases are generated by the arnoldi process for certain choices of krylov subspaces the computation of the solution splits into the solution of many independent subproblems this makes the algorithms suitable for implementation on parallel computers

a cyclic low rank smith method for large sparse lyapunov equations in this paper we present the cyclic low rank smith method which is an iterative method for the computation of low rank approximations to the solution of large sparse stable lyapunov equations it is based on a generalization of the classical smith method and profits by the usual low rank property of the right hand side matrix the requirements of the method are moderate with respect to both computational cost and memory furthermore we propose a heuristic for determining a set of suboptimal alternating direction implicit adi shift parameters this heuristic which is based on a pair of arnoldi processes does not require any a priori knowledge on the spectrum of the coefficient matrix of the lyapunov equation numerical experiments show the efficiency of the iterative scheme combined with the heuristic for the adi parameters

on normal and skew hermitian splitting iteration methods for large sparse continuous sylvester equations this paper is concerned with some generalizations of the hermitian and skew hermitian splitting hss iteration for solving continuous sylvester equations the main contents we will introduce are the normal and skew hermitian splitting nss iteration methods for the continuous sylvester equations it is shown that the new schemes can outperform the standard hss method in some situations theoretical analysis shows that the nss methods converge unconditionally to the exact solution of the continuous sylvester equations moreover we derive the upper bound of the contraction factor of the nss iterations numerical experiments further show the effectiveness of our new methods

numerical experience with a superfast real toeplitz solver abstract we briefly describe the generalized schur algorithm for the superfast solution of positive definite toeplitz systems of equations and its relationship with schur s algorithm and the szego recursions we then present some experimental results obtained with our fortran implementation of this superfast toeplitz solver the algorithm displays favorable behavior in that the growth rates of the resulting residuals and errors are comparable with those of the szego recursions

approximate inverse preconditioner by computing approximate solution of sylvester equation abstract this paper presents a new method for obtaining a matrix m which is an approximate inverse preconditioner for a given matrix a where the eigenvalues of a all either have negative real parts or all have positive real parts this method is based on the approximate solution of the special sylvester equation ax xa 2i we use a krylov subspace method for obtaining an approximate solution of this sylvester matrix equation which is based on the arnoldi algorithm and on an integral formula the computation of the preconditioner can be carried out in parallel and its implementation requires only the solution of very simple and small sylvester equations the sparsity of the preconditioner is preserved by using a proper dropping strategy some numerical experiments on test matrices from harwell boing collection for comparing the numerical performance of the new method with an available well known algorithm are presented

the nonlocal influence of density variations in a composite abstract in any randomly inhomogeneous elastic medium it is well known that the mean stress may be related to the mean strain through a nonlocal operator which reduces to an ordinary tensor of moduli when applied to a slowly varying mean field this study discusses a corresponding relation between the mean momentum density and the mean velocity attention is focused on this feature of overall behaviour by considering a model medium in which only the density varies when the medium is statistically uniform the overall mass density operator has the form of a convolution in time and space and it is convenient to discuss its combined laplace and fourier transform the operator is introduced via a brief consideration of its representation as a perturbation series but the bulk of the work is devoted to establishing bounds of hashin shtrikman type for the eigenvalues of its combined transform when the transform variables are real explicit bounds for a two phase medium with a particular form for its twopoint correlation function are given they define the eigenvalues within 0 5 percent when the ratio of the two densities is two and remain close enough to give a good indication of overall behaviour at density ratios as high as ten

a derivation for the acoustic material parameters in transformation domains abstract the concept of transformation acoustics offers a new strategy to design devices with different functionalities such as cloaks concentrators and others the material parameters of the transformed shell can be expressed in the form of a multiplication of jacobian matrix here we present a direct and different approach to analytically determine the material parameters of transformation shell the proof ensures that with the specified material parameters the field exterior to the shell will be the same as that of the original medium as that the governing equation remains unchanged and that the impedance match conditions are exactly fulfilled along the boundary we will present our algorithm for a general class of devices with two or three dimensional configurations described by general orthogonal coordinates the derivation is valid for any continuous transformation function corresponding to cloaks anti cloaks rotators concentrators etc in this work we focus on the mathematical framework of acoustic wave but the same approach can be applied to electromagnetic wave as well explicit formulae for the mass density tensor and bulk modulus are derived in terms of the transformation function

controllability observability and the solution of ax xb c a closed form finite series representation of the unique solution x of the matrix equation ax xb c is developed using this representation the image kernel and rank of x are related to the controllable and unobservable subspaces of the a c and c b pairs respectively bounds on the rank of x are obtainedin terms of the dimensions of these subspaces in the case that c has unitary rank an exact calculation of rank x is made the generic rank of x with a b fixed and c generic is evaluated

iterative methods for restoring noisy images an iterative image restoration algorithm is presented that is based on the best linear mean square estimate of the wiener filter technique and the iterative approach of the first order stationary linear jacobi method the iterative algorithm incorporates a priori knowledge concerning the image and noise statistics directly into the iterative procedure it also assumes the image covariance to be described by a separable first order markov field and exploits the resulting toeplitz structure the usefulness and validity of this algorithm are demonstrated by implementation and testing on real images the advantages offered by the algorithm are that it is computationally efficient since only a small number of computations per pixel per iteration is required it requires a small amount of memory it has a fast rate of convergence and it exhibits neither high noise sensitivity nor significant loss of resolution

the nonparanormal semiparametric estimation of high dimensional undirected graphs recent methods for estimating sparse undirected graphs for real valued data in high dimensional problems rely heavily on the assumption of normality we show how to use a semiparametric gaussian copula or nonparanormal for high dimensional inference just as additive models extend linear models by replacing linear functions with a set of one dimensional smooth functions the nonparanormal extends the normal by transforming the variables by smooth functions we derive a method for estimating the nonparanormal study the method s theoretical properties and show that it works well in many examples

on the third and fourth zolotarev problems in the complex plane in relation to the determination of optimal parameters for the alternating direction implicit adi method the authors consider the third zolotarev problem in the complex plane which has been the topic of several recent contributions in the real case zolotarev also posed a fourth problem that involves best rational approximants in the chebyshev sense and achieser proved later that the third and fourth zolotarev problems are actually equivalent by generalizing achieser s argument it is shown that the complex third zolotarev problem can be stated as a best chebyshev approximation by complex rational functions this enables the authors to use known optimality and uniqueness conditions as well as an iterative algorithm for computing the exact solutions some numerical results are reported for rectangular domains of the complex plane that are of practical importance in the adi method

fields of values and iterative methods abstract the performance of an iterative scheme to solve x t x c t c n n c c n is often judged by spectral properties of t if t is not normal it is however well known that only conclusions about the assymptotic behavior of an iterative method can be drawn from spectral information to anticipate the progress of the iteration after a finite number of steps the knowledge of the eigenvalues alone is often useless in addition the spectrum of t may be highly sensitive to perturbations if t is not normal an iterative method which on the basis of some spectral information is predicted to converge rapidly for t may well diverge if t is slightly perturbed in practice the convergence of the iteration x m t x m 1 c is therefore frequently measured by some norm verbar t verbar rather than by the spectral radius t but apart from the fact that norms lead to error estimates which are often too pessimistic they cannot be used to analyze more general schemes such as e g the chebyshev iterative methods here we discuss another tool to analyze the behavior of an iterative method namely the field of values w t the collection of all rayleigh quotients of t w t contains the eigenvalues of t and the numerical radius t max z w t z defines a norm on c n n the field of values represents therefore an intermediate concept to judge an iterative scheme by it is related to the spectral approach but has also certain norm properties

on the adi method for sylvester equations this paper is concerned with the numerical solution of large scale sylvester equations ax xb c lyapunov equations as a special case in particular included with c having very small rank for stable lyapunov equations penzl 2000 22 and li and white 2002 20 demonstrated that the so called cholesky factor adi method with decent shift parameters can be very effective in this paper we present a generalization of the cholesky factor adi method for sylvester equations an easily implementable extension of penz s shift strategy for the lyapunov equation is presented for the current case it is demonstrated that galerkin projection via adi subspaces often produces much more accurate solutions than adi solutions