text: 154,49,1418410337695506432,96779364,Arnab Bhattacharyya,"New paper: <LINK> (Learning Sparse Fixed-Structure Gaussian Bayesian Networks) with Davin Choo, @rrgajjala, Sutanu Gayen, and @Yohanna49592977. We look at a basic model used to specify causal dependencies among continuous variables. You have n variables that are ordered in some way, and each variable is generated as a linear combination of the previous variables plus an independent gaussian noise. Simple, right? E.g.: <LINK> These are called Gaussian Bayes nets. The dependency structure of the variables is naturally encoded by a DAG. For the example above: <LINK> Suppose you have a distribution P generated as a Gaussian Bayes net over a DAG G.  The distribution learning problem is: given samples from P, infer parameters of a distribution Q such that TV(P,Q)&lt;ε with good enough probability. There are actually two problems here. The first is the ""structure learning"" problem where G is not known (but maybe you only know that it is sparse). This problem is quite hard, and there are essentially no general algorithmic results. In this paper, we look at the easier ""fixed-structure"" problem where G is already given. Amazingly to us, we could say something new about this basic problem! The obvious thing to try is to learn the coefficients of each equation by lin regression at each node. If you run least squares with O~(n/ε) equations at each node, then you learn a Bayes net with KL div ε from P. But this isn't the only option! At each node, you can run several batches of least squares, where each batch is a ""small"" system of equations. Each batch solution gives you an estimate of the coefficients at that node, and then you can take the average across batch solutions. In the extreme case, if a node has p parents, you can solve several batches of pxp systems (with gaussian elim). Here, we show that each batch solution is distributed as Cauchy (!), not gaussian. It then makes more sense to take the median of the solutions rather than average. The advantage of these other algorithms is that they allow each batch to be processed parallelly. Also, in experiments (<LINK>), they perform better when there's noise or the DAG is mis-specified. #causalinference #Statistics #MachineLearning #Algorithms",https://arxiv.org/abs/2107.10450,"Gaussian Bayesian networks (a.k.a. linear Gaussian structural equation models) are widely used to model causal interactions among continuous variables. In this work, we study the problem of learning a fixed-structure Gaussian Bayesian network up to a bounded error in total variation distance. We analyze the commonly used node-wise least squares regression (LeastSquares) and prove that it has a near-optimal sample complexity. We also study a couple of new algorithms for the problem: - BatchAvgLeastSquares takes the average of several batches of least squares solutions at each node, so that one can interpolate between the batch size and the number of batches. We show that BatchAvgLeastSquares also has near-optimal sample complexity. - CauchyEst takes the median of solutions to several batches of linear systems at each node. We show that the algorithm specialized to polytrees, CauchyEstTree, has near-optimal sample complexity. Experimentally, we show that for uncontaminated, realizable data, the LeastSquares algorithm performs best, but in the presence of contamination or DAG misspecification, CauchyEst/CauchyEstTree and BatchAvgLeastSquares respectively perform better. ",Learning Sparse Fixed-Structure Gaussian Bayesian Networks,11,"['New paper: <LINK> (Learning Sparse Fixed-Structure Gaussian Bayesian Networks) with Davin Choo, @rrgajjala, Sutanu Gayen, and @Yohanna49592977.', 'We look at a basic model used to specify causal dependencies among continuous variables. You have n variables that are ordered in some way, and each variable is generated as a linear combination of the previous variables plus an independent gaussian noise. Simple, right? E.g.: https://t.co/MPLTH7fugA', 'These are called Gaussian Bayes nets. The dependency structure of the variables is naturally encoded by a DAG. For the example above: https://t.co/hO6gWyXvtX', 'Suppose you have a distribution P generated as a Gaussian Bayes net over a DAG G. \n\nThe distribution learning problem is: given samples from P, infer parameters of a distribution Q such that TV(P,Q)&lt;ε with good enough probability.', 'There are actually two problems here. The first is the ""structure learning"" problem where G is not known (but maybe you only know that it is sparse). This problem is quite hard, and there are essentially no general algorithmic results.', 'In this paper, we look at the easier ""fixed-structure"" problem where G is already given. Amazingly to us, we could say something new about this basic problem!', 'The obvious thing to try is to learn the coefficients of each equation by lin regression at each node. If you run least squares with O~(n/ε) equations at each node, then you learn a Bayes net with KL div ε from P.', 'But this isn\'t the only option! At each node, you can run several batches of least squares, where each batch is a ""small"" system of equations. Each batch solution gives you an estimate of the coefficients at that node, and then you can take the average across batch solutions.', 'In the extreme case, if a node has p parents, you can solve several batches of pxp systems (with gaussian elim). Here, we show that each batch solution is distributed as Cauchy (!), not gaussian. It then makes more sense to take the median of the solutions rather than average.', ""The advantage of these other algorithms is that they allow each batch to be processed parallelly. Also, in experiments (https://t.co/tZZoMYXnCE), they perform better when there's noise or the DAG is mis-specified."", '#causalinference #Statistics #MachineLearning #Algorithms']",21,07,2221