Marginal Causal Flows for Validation and Inference

Authors: daniel  de vassimon manela
laura  battaglia
robin j evans
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-02 15:01:46

Abstract: Investigating the marginal causal effect of an intervention on an outcome from complex data remains challenging due to the inflexibility of employed models and the lack of complexity in causal benchmark datasets, which often fail to reproduce intricate real-world data patterns. In this paper we introduce Frugal Flows, a likelihood-based machine learning model that uses normalising flows to flexibly learn the data-generating process, while also directly targeting the marginal causal quantities inferred from observational data. We provide a novel algorithm for fitting a model to observational data with a parametrically specified causal distribution, and propose that these models are exceptionally well suited for synthetic data generation to validate causal methods. Unlike existing data generation methods, Frugal Flows generate synthetic data that closely resembles the empirical dataset, while also automatically and exactly satisfying a user-defined average treatment effect. To our knowledge, Frugal Flows are the first generative model to both learn flexible data representations and also \textit{exactly} parameterise quantities such as the average treatment effect and the degree of unobserved confounding. We demonstrate the above with experiments on  both simulated and real-world datasets.

Main paper: 
 introduction: 
 Simulating realistic datasets such that the marginal causal effect is constrained to take a specific form is a significant challenge in causal inference. Many methods for inferring these effects exist, but simulating from them is a significant challenge (Young et al., 2008; Havercroft and Didelez, 2012; Keogh et al., 2021) . In particular, it is difficult to simulate complex benchmarks from generative models in such a way that a custom marginal effect exactly holds.The frugal parameterisation (Evans and Didelez, 2024) provides a solution to this problem by constructing a joint distribution that explicitly parameterises the marginal causal effect and builds the rest of the model around it. Frugal models typically represent the dependency between an outcome and pretreatment covariates using copulae. Standard multivariate copulae are parametric, leading to potential model misspecification.In this paper we show how one can construct frugally parameterised marginal causal models using normalising flows (NFs, Rezende and Mohamed, 2015; Dinh et al., 2016) to target the causal margin of the distribution (a conditional univariate marginal density of an outcome conditioned on a treatment). We name the resulting model a Frugal Flow (FF). To the best of our knowledge, FFs offer the first likelihood-based framework for learning a marginal causal effect while modelling the outcome and propensity nuisance parameters using flexible generative models.FFs are exceptionally well suited for generating benchmark datasets for causal method validation. Since FFs enable direct parameterisation of the causal margin, they provide a framework for generating causal benchmark datasets which resemble real-world datasets, but which also allow users to encode causal properties in order to validate novel inference models. FFs can be used to generate benchmarks with customisable degrees of unobserved confounding. This can aid in the validation of model robustness under conditions where the assumption of conditional ignorability does not hold. Here, conditional ignorability (or conditional exchangeability) means that marginal distribution of the potential outcomes is independent of the value of treatment, conditional on the observed covariates (Pearl, 2009) .FFs offer marked improvements over current benchmarking generation methods, which use soft constraint optimisation to enforce the desired causal restrictions (Kendall, 1975; Parikh et al., 2022) . As a result, post hoc checks are required to see whether these conditions are present in the synthetic data. FFs do not require this second step, as relevant conditions are explicitly encoded in the underlying likelihood. Finally, FFs allow for outcomes to be sampled from marginal logistic and probit models, making them the first generative benchmarking model to facilitate the simulation of binary outcomes with a choice of user specified risk differences, risk ratios, or odds ratios.Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?Answer: [Yes] Justification: The abstract outlines the a key problem in generating causal benchmarks for method validation, in particular that it is hard to train and sample from generative models which explicitly target the marginal causal effect. Our method (Frugal Flows) builds on statistical work by Evans and Didelez (2024) , and we develop a normalising flow framework which allows us to exactly specify the marginal causal effect and simulate from it exactly. We show that it does indeed target the causal effect by fitting FFs to simulated data with a known causal margin, and demonstrates that it infers the correct parameters despite the data being demonstrably confounded. Additionally, fitting popular causal inference models to synthetic data generated from trained Frugal Flows verifies that the samples are follow the desired causal margin when the data is confounded and unconfounded.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
background: 
 In this paper we consider a static treatment model with an outcome Y ∈ Y ⊆ R and T a binary treatment in T = {0, 1}. Let the set of measured pretreatment covariates be Z ∈ Z ⊆ R D . Additionally, we will use the notation of Pearl (2009) where intervened distributions are indicated by the presence of a "do(•)" operator, with its absence indicating that the distribution is from the observational regime.Causal inference methods are generally developed to estimate the average effect of a treatment (T ) on an outcome (Y ) for a population defined by a set of pretreatment covariates (Z) (Hernán and Robins, 2020) . Let the variables be distributed according to (Z, T, Y ) ∼ P ZTY with density p ZTY . We make the standard assumptions of a stable unit treatment value (commonly referred to as SUTVA), positivity, and conditional ignorability (equivalent to conditional exchangability) outlined in Pearl (2009) . Additionally, the covariate set Z must only include pretreatment covariates. The conditional distribution of Y and Z after an intervention on T is equal top ZY | do(T ) (z, y | t) = p Z (z) • p Y |Z,do(T ) (y | z, t).Causal practitioners are often interested in the marginal effect of T on Y on the intervened system, sometimes referred to as the marginal outcome distribution (MOD), p Y |do(T ) :p Y | do(T ) (y | t) = Z dz p Y |Z,do(T ) (y | z, t) p Z (z).(1)The difference between the means of Y under this margin between different values of T is called the average treatment effect (ATE), τ where, τ = E[Y | do(T = 1)] -E[Y | do(T = 0)]. Models which target this marginal quantity are known as marginal structural models (MSMs, Robins, 1998) and are frequently used in epidemiological and medical domains to account for time-varying confounding. In particular, they are effective at quantifying the effect of an intervention over a population, where the specific relationships between the outcome and (possibly high dimensional) pretreatment covariates are not relevant, and are modelled as nuisance parameters. The semiparametric question of estimating finite dimensional quantities in the presence of high dimensional nuisance parameters has a long history (Robins et al., 1995; Robins and Rotnitzky, 1995) , but has undergone a renaissance since the development of methods such as targeted maximum likelihood estimation (van der Laan and Rose, 2011) and double machine learning (Chernozhukov et al., 2018) , which allow for general machine learning algorithms to flexibly describe the nuisance models and still have valid inference on a low-dimensional treatment effect.Frugally parameterised distributions consist of three distinct components: the distribution of the 'past,' θ ZT ; the intervened causal quantity of interest, θ Y | do(T ) ; and an intervened dependency measure between Y and Z conditional on T , ϕ ZY | do(T ) . The key idea is to explicitly parameterise the marginal causal effect, and build the rest of the model around it. In this paper we encode all the dependence among covariates in the copula, so 'the past' is really just the propensity for treatment (also called the propensity score) and the product of the univariate margins (Evans and Didelez, 2024) .Figure 1 provides an illustrative summary of our framework, and outline which models are used to parameterise each component of a frugal model. , is modelled with a univariate normalising flow, which we denote by F (see Section 2.4). The intervened dependency measure, ϕ ZY | do(T ) , is modelled with a copula flow which we denote by C (see Section 2.5). The past, θ ZT , is modelled by the combination of univariate normalising flows (for the univariate pretreatment covariate distributions) and a copula flow (for the propensity of treatment).Z θ Z : F Z d = F -1 Z d D d=1 Y T θ T|Z : F T|Z = C -1 T|Z F -1 T | F -1 Z1 , . . . , F -1 Z D θ Y | do(T ) : F Y | do(T ) = F -1 Y | do(T ) ϕ ZY | do(T ) : C ZY | do(T ) = C -1 F -1 Y | do(T ) , F -1 Z1 , . . . , F -1 Z DVariation Independence Any smooth and regular 'dependency measure' can be chosen for parameterising ϕ ZY | do(T ) ; this is defined as a quantity which, when combined with the marginal distributions, smoothly parameterises the joint distribution. It is desirable that the three parameter sets(θ ZT , θ Y | do(T ) , ϕ ZY | do(T )) are variation independent (Barndorff-Nielsen, 2014) of each other; such parameterisations have the benefit of allowing the measure ϕ ZY | do(T ) to be freely specified without restricting the rest of the model. Copulae are an example of such a dependency measure, and are a natural choice for frugally modelling dependencies in continuous and mixed datasets. For further detail we refer the reader to Appendix A.A multivariate copula, denoted by C : [0, 1] d → [0, 1] is a multivariate cumulative distribution function (CDF) defined over a set of d uniform margins, with an associated density c(•) if it is continuous with respect to its arguments (Sklar, 1959; Joe, 2014) . Copulae are often used to parameterise the dependency structure of a joint distribution independent of its univariate margins. Large, complex dependency structures are often modelled by pair-copula constructions (PCCs) or vine copulae (Czado and Nagler, 2022; Joe and Kurowicka, 2011) . These methods factorise the dependency structure into a set of non-overlapping bivariate copulae. However, these approaches typically impose the constraints of a finite dimensional parameterisation on the dependency structure in the bivariate copulae used. A more comprehensive introduction to copulae can be found in Appendix B.Copulae in Machine Learning More complex ML models have been developed to more flexibly learn copula distributions. Several alternatives have been proposed, some targeting specific copula classes (Ling et al., 2020; Wilson and Ghahramani, 2010) , and others constraining a neural networkbased architecture to estimate valid copulae, though often with limited scalability (Zeng and Wang, 2022; Chilinski and Silva, 2020) or using variational approximations (Letizia and Tonello, 2022) . However, the most active research area in this field makes use of normalising flows, leveraging their likelihood-based, composable and invertible nature to chain transformations of marginal quantities to the fitting of the copula density.A key motivation for this paper is the search for a flexible parameterisation of the copulaEQUATIONbetween the probability integral transforms of the univariate pretreatment covariates and a conditional univariate quantity which parameterises the causal margin. Evans and Didelez (2024) show that this can be done using parametric copulae, and also prove that it targets the marginal causal rather than the conditional distribution when ϕ ZY | do(T ) is parameterised by a multivariate copula. Consider the multivariate copula for the distribution of Z and Y conditional on T :c(F Y |T , F Z1|T , . . . , F Z D |T ).For an intervened distribution, all pretreatment covariates Z are marginally independent of T , and so the intervened joint density becomesp Y |Z,do(T ) = p Y | do(T ) • c(F Y | do(T ) , F Z1 , . . . , F Z D ),where is estimated separately from the copula, the marginal conditional effect will be inferred rather than the marginal causal effect.p Y | do(T )Generative ML methods allow for estimating more flexible and general copulae, but have struggled so far to learn copulae together with conditional univariate quantities. We resolve this problem and design a NF-based copula inference method that allows for these quantities to be estimated jointly as required by the frugal parametrisation (see Section 2.5). The model is then trained on real-world data and used for generating customised causal benchmarks which closely resemble the original dataset.Normalising flows (NFs) (Tabak and Turner, 2013; Rezende and Mohamed, 2015; Dinh et al., 2016) allow for density estimation via learning a diffeomorphic transformation F that maps the unknown target distribution p X (x), x ∈ R D to a simple and known base distributionp U (u), u ∈ R D , so that when X ∼ p X and U ∼ p U then U = F -1 (X) .F is usually a composition of invertible and differentiable transformations F i parametrised by neural networks, and is often trained by maximising the log-likelihood of observed {x i } N i=1 . This can be conveniently done in closed form exploiting the change of variable formulaEQUATIONprovided that the chosen model for F allows for efficient computation of the Jacobian determinant det(∂(F -1 (x))/∂x). The implementation of F -1 then allows for density evaluation, whereas F can be used for sampling from the joint.As for the choice of F, the literature has explored a number of implementations that retain invertibility while allowing for computational tractability of the determinant. See Papamakarios et al. (2021) for an introduction and overview. Our implementation relies on neural spline flows (NSF, Durkan et al., 2019) , a particular type of autoregressive flows that will be further illustrated in Section 2.5.Our Frugal Flow approach builds upon the copula-based flow model proposed by Kamthe et al. (2021) for synthetic data generation. The authors start by considering a copula C(F X1 , . . . , F X D ) defined over the marginal probability integral transforms F X1 , . . . , F X D of a random vector X = [X 1 , . . . , X D ]. Assuming the copula density exists, the joint density of X can be written asEQUATIONwhere p X d is the marginal density of X d . This factorisation of the density can be similarly induced by a NF that composes D flows F 1 , . . . , F D for the marginal quantities and a flow C X for the copula.For the rest of this paper, we will let U ∼ Uniform[0, 1] D represent a vector of independent uniforms, and let V ∼ C represent a vector of dependent uniforms as a multivariate copula C. The generative procedure for this NF takes samples U from a base distribution of independent uniforms and first pushes them through the copula flow C X , obtaining correlated uniform samples V = C X (U ). Then V is mapped through the marginal flows F X = [F X1 , . . . , F X D ] to obtain the random vector X = F X (V ).The composed flow X = F X (C X (U )) is also a valid flow, and via the change of variable formula as in eq. ( 3) it induces a specific factorisation of the density of X Here, we quote the result from Kamthe et al. (2021) :EQUATION)As the univariate mapping from a uniform to a random variable is uniquely defined by the CDF, the flowsF -1 X = [F -1 X1 , . . . , F -1 X D ]target the marginal CDFs F X1 , . . . , F X D . Note how eq. ( 5) factorises the density of X into a copula density and a product of marginal densities as in eq. ( 4).C X is estimated with a NSF, a NF of the autoregressive flow class. Autoregressive flows (Papamakarios et al., 2017; Huang et al., 2018) factorise C X as a recursive sequence of univariate conditional flows:EQUATIONIn principle, since the input U is a vector of independent uniforms, the conditional flows would approximate the inverse of the Rosenblatt transform (Rosenblatt, 1952) and thus be universal approximators if the flows were sufficiently expressive (Papamakarios et al., 2021) . The Rosenblatt transform sequentially maps each component S d of any random vector S with strictly positive density through its corresponding conditional CDF F S d |S1,...,S d-1 , obtaining a vector U of independent uniforms. It is known to be a diffeomorphism, so its inverse bears the same structure as eq. ( 6)), but uses inverse conditional CDFs C -1 d|1,...,d-1 for each V d . We use the notation C -1 to emphasise that in the copula flow case we are dealing with inverse copula CDFs, whose codomain is also uniform.Autoregressive flows estimate each univariate conditional flow C d|1...d-1 with a strictly monotone function whose parameters are only allowed to depend on dimensions 1, . . . , d -1. The monotonicity of the function ensures invertibility, while the autoregressive structure in the function parameter dependence gives a triangular Jacobian whose determinant is tractable. Kamthe et al. (2021) use a NSF, where the monotone function is given by a monotone rational quadratic spline, whose knot parameters are provided by a neural network where weights are appropriately masked to ensure the autoregressive structure. The univariate marginal flows F X are estimated with separate NSFs before training the copula flow using the transformed data V .While a NSF can constrain the support of both the base and target distributions, it cannot control the form of the marginal distribution. If marginal and copula flows are learned simultaneously, neither will be correctly inferred due to the infinite possible combinations of (F X , C X ) which yield the same composite flow W = F X • C X . These flows must be learned sequentially if C is to model a copula.In our application, we wish to infer a multivariate copula which models the joint dependence between univariate pretreatment covariates and conditional univariate quantities such that the latter parameterises the causal margin. Inferring the MOD separately from the copula, as copula-based flows do, will target the conditional causal effect rather than the marginal causal effect. We propose a solution in the form of Frugal Flows, which we introduce in Section 3.1.1. Moreover, for discrete variables we use a dequantised form of the empirical CDF rather than a NSF adaptation (see Appendix B.2 for further details).Methods for validating causal models can be broadly categorised into two groups. The first comprises auxiliary analyses conducted after fitting a causal model and estimating a treatment effect. These include but are not limited to sensitivity analyses (Imai et al., 2010) , subgroup analyses (Cochran and Chambers, 1965) , placebo tests (Eggers et al., 2023) , and negative controls (Shi et al., 2020) .The second set of validation methods is where we see FFs having a significant impact. These methods are used to construct synthetic datasets while allowing the causal practitioner to customise specific features of the data-generating process. For example, when validating an inference method which estimates an ATE under certain confounding assumptions, it is crucial that generated data follow the "ground truth" ATE and confounding assumptions one wishes to measure. However, synthetic data risk being oversimplified and contrived, failing to reflect the complexity of real world datasets.To mitigate this, generative models are trained on real-world data and calibrated to generate samples with modifiable causal constraints. Such constraints include the average causal treatment effect, unobserved confounding, and positivity. To our knowledge, the FF framework proposed in this paper is the first method to allow all of these conditions to adjusted by the user. Existing methods (Neal et al., 2020; Athey et al., 2021; Parikh et al., 2022) encode these effects through soft optimisation constraints, hence there is no guarantee that the constraints are satisfied. Enforcing these constraints too strongly may negatively impact model optimisation, and may affect the reconstructive ability of the underlying model. Furthermore, since these approaches do not explicitly parameterise the causal effect, samples from trained models must be tested post hoc to ensure the desired constraints are present in the sampled data. A key benefit of frugal models is that the marginal causal effect is directly parameterised by the user through the likelihood. As a result, synthetic data samples will exactly satisfy these constraints.Accurately modelling the univariate marginal CDFs of pretreatment covariates is a crucial step in training Frugal Flows, particularly when the dataset includes discrete variables. For continuous covariates, the mapping between observations and ranks is unique, allowing for straightforward estimation of the marginal distribution. However, with discrete covariates, this mapping becomes one-to-many, as the same observation can be transformed from different ranks. This non-uniqueness introduces significant challenges when modelling the joint distribution via copulae, as the joint set of ranks for discrete covariates lacks a unique representation. As a result, estimating the dependency structure between variables becomes more complex and less reliable.To extend Frugal Flows to accommodate mixed data types, it is essential to generate empirical ranks for discrete covariates in a way that allows the model to capture their dependencies. Our goal is to obtain valid rank samples that can be used to train a Frugal Flow without introducing distortions in the copula structure. While this issue has been widely explored in the copula literature for parametric models, there remains a gap in effectively addressing it within more flexible, nonparametric frameworks. In this work, we implement a generalised distributional transform for discrete covariates (presented in Appendix B.2), which allows Frugal Flows to be trained effectively, maintaining the flexibility of the model while accurately capturing the relationships between variables.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: The limitations are discussed in Section 5.1, elaborating in particular on the drawbacks of Frugal Flows for inference purposes. In particular, while we believe Frugal Flows are powerful generative models for flexibly simulating causal benchmarks, their inference capabilities are fairly limited, and require large datasets to learn the correct causal margin.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.• If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
building the joint distribution: 
 In this section we parameterise the full observational joint using FFs. Section 3.1.1 outlines how the FF is constructed; we first learn the probability integral transforms of the pretreatment covariates, and then infer the causal margin jointly with an extended copula flow, the Frugal Flow. To infer the causal margin, this is sufficient. Nevertheless, the propensity score is needed to complete the joint in order to generate benchmarks which are confounded in a similar fashion to the original real-world dataset. We describe the fitting of the propensity score in Section 3.1.2The first step involves learning the margins for the pretreatment covariates Z. This is done in a similar fashion to that of Kamthe et al. (2021) 's copula-based flows, as described in Section 2.5. The outcome, treatment, and the inferred ranks V Z of the pretreatment covariates are then used to train the Frugal Flow (see bottom part of Figure 2 ) that models F -1 Y | do(T ) together with the copula flow. This is required in order to learn the causal marginal p Y | do(T ) rather than the conditional p Y |T .The Frugal Flow of dimension D + 1 transforms the joint input of (Y, V Z | do(T )) into a random vector U which we set to be distributed according to an independent uniform base distribution. In the first subflow of the composition, Y is pushed through a univariate flowF -1 Y | do(T ) conditioned on T to obtain V Y | do(T ) , while the V Z remain untransformed. Subsequently, V Y | do(T ) is kept fixed, while a copula is learnt over V Z conditional on V Y | do(T )via an NSF. Importantly, a specific ordering of the variables is imposed, such that the causal margin is ranked first. In this way, we ensure that U 1 and V Y | do(T ) have the same distribution, andV Y | do(T ) is therefore constrained to be uniform. The marginal flow F -1 Y | do(T ) thus targets the CDF of the marginal causal effect, F Y |do(T ) . In summary, we construct a flow Q -1 : (Y, V Z1 , . . . , V Z D | T ) → V as a composition of a marginal flow F -1 Y | do(T ) and conditional copula distribution C -1 (v Y | do(T ) , v Z1 , . . . , v Z D ) = C(v Z1 , . . . , v Z D | v Y | do(T )). More on the implementation details can be found in Appendix C.We constructed the conditional distribution of Y and Z after an intervention on T in Section 3.1.1: Inferring the above is sufficient for identifying the causal margin. However, to generate realistic samples for causal method validation, one also needs to learn the propensity score, p T |Z = p T • c T|Z . By decoupling the marginal treatment density p T from the conditional copula c T|Z , one can can modify the marginal treatments while retaining the dependence of the original data. We therefore learn an approximate probability integral transform of the discrete treatment T (see Appendix B.2.1 for further details), followed by the conditional copula flow of T on Z, C -1p ZY | do(T ) (z, y | t) = D i=1 p Zi (z i ) • p Y | do(T ) (y, t) • c ZY | do(T ) (v Y | do(T ) , v Z1 , . . . , v Z D ).    Z 1 . . . Z D       F Z1 (•) . . . F Z D (•)       V Z1 . . . V Z D    = V Z Y | do(T ) V Z F -1 Y | do(T ) (•) I(•) V Y | do(T ) V Z NSF c(v Y | do(T ) ) = 1 c(v Z | v Y | do(T ) ) U 1 U 2:(D+1)T |Z : V T → V T|Z | Z.One could directly model p T|Z using a normalising flow, which would also constitute a valid frugal model. We instead choose to model the conditional copula using a flow, C T|Z = C -1 T|Z , allowing users to encode a degree of unobserved confounding in the generated data by sampling the ranks V T|Z and V Y | do(T ) from a non-independence copula. Assuming ignorability, these ranks would be independent. However, unobserved confounders imply dependence between these ranks. Sampling them from a copula can replicate this effect, as demonstrated in the far-right plots in Figures 3 and 4 .The above section describes how one can estimate the propensity of treatment from a real-world dataset. However, we remark that one can choose any custom propensity score function to generate treatments conditional on the pretreatment covariates via inverse probability integral transforms on V T |Z . Hence, one can fully control the overlap/positivity of FF generated benchmark datasets.Data generated from a fitted FF can be customised with a range of properties, allowing for model validation against a range of customisable causal assumptions. We describe these below.The central output of the Frugal Flow is a method for sampling ranks for each of the margins in P Y | do(T ) , P Z1 , . . . , P Z d . Any causal marginal density q Y | do(T ) can be used to generate samples of Y via inverse probability integral transforms. Since the Frugal Flow returns ranks for the intervened causal effect, these can be inverse transformed by any valid CDF. Unlike other methods, this constraint is strictly enforced by the the frugal likelihood.Simulating from Discrete Outcomes Since FFs return V Y | do(T ) ranks, one can sample from any custom causal margin. This extends to both continuous and discrete causal margins. One can simulate from a logistic marginal effectY | do(T ) ∼ Bernoulli(p = expit(βT + c)) or probit model Y | do(T ) ∼ Bernoulli(p = Φ(βT + c))where Φ(•) is a univariate standard Gaussian CDF. This is non-trivial, because logistic regression is not collapsible, meaning that if (for example) Y | T = t, Z = z is a logistic regression, then Y | do(T = t) generally will not be. Hence it is infeasible for a fully conditional method of simulation to produce outcomes where the causal margin uses a logistic link. For experimental results see Appendix D.2.1.Modifying the Degree of Unobserved Confounding One can sample data from FFs as if the outcome is affected by unobserved confounding. The variables V Y | do(T ) and V T|Z are independent of each other if no unobserved confounding is assumed. Introducing a dependence between these ranks replicates the effect of unobserved confounding. This can be achieved by sampling (V Y | do(T ) , V T |Z ) from a Gaussian bivariate copula, c(v Y | do(T ) , v T |Z ; ρ),p Y |W,do(T ) (y | w, t) = W dw p Y |Z,do(T ) (y | w, w, t) p W |W (w | w) (7)We propose a method to exactly parameterise heterogeneous treatment effects using a subset of pretreatment covariates, W ⊂ Z. FFs offer exact parameterisation of p Y |W,do(T ) , allowing for customisation of heterogeneity while capturing complex dependencies between other covariates. Specifically, the model infers the conditional treatment margin, p Y |W,do(T ) (y | w, t), ensuring proper inference of the joint pretreatment covariate distribution, p Z (•). Thus, one may simulate data where causal effects are conditional on a selected subset of variables, offering flexible and precise control over treatment heterogeneity. Further details may be found in Appendix D.2.2.Customising the Propensity Score Since the propensity score is variation independent from the rest of the model, one has complete flexibility on how to parameterise the propensity score. Any distribution P T|Z can be used to generate treatments with varying degrees of overlap in a manner that is completely customisable by the user.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [NA] Justification: Our paper does not derive any new theoretical results, but quotes the relevant theorems if necessary, and the papers from which they are derived.Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced. • All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced. 
experiments: 
 The following section discusses our experiments and results, which aim to i) demonstrate that FFs accurately infer the true MOD for confounded data, and ii) show how a trained FF can generate synthetic datasets that meet user specified causal margins and unobserved confounding.We generate simulated data from three models. The first two are parameterised by four pretreatment covariates Z = {Z 1 , . . . , Z 4 } with a binary treatment T , a linear Gaussian causal margin Y | do(T ) ∼ N (µ = T + 1, σ = 1), and a copula dependence measure c(v Y |T , v Z1 , . . . , v Z4 ). In the first model M 1 , all four covariates follow a gamma distribution. In the second M 2 , the data is generated from an even split of gamma and binary covariates. Additionally, we generate data from model M 3 with ten pretreatment covariates comprising five gamma and five binary variables. A more quantitative description of the simulated data generating process and hyperparameter values are presented in Appendix D.1. We generated datasets with a sample size of N = 25,000 across B = 25 different runs. Frugal Flows (FFs) were compared against outcome regression (OR), traditional causal propensity score matching (Stuart, 2010) , and state-of-the-art causal normalising flows (CNFs) (Javaloy et al., 2024) . Further details on the methods can be found in Appendix D.3.3. A default set of hyperparameters was used for all models. The estimated ATEs are shown in Table 4 . OR models, which estimate the conditional rather than the marginal effect of T on Y , consistently exhibited bias, pulling the estimates away from the true value. In contrast, FFs achieved the lowest error in identifying the true ATE, outperforming both statistical matching and CNFs.Our results demonstrate that Frugal Flows can correctly identify causal relationships under ideal conditions, confirming that they are a valid, efficient way to parameterise a causal model using deep learning architectures. A drawback of FFs is that they need large datasets to accurately infer causal margins. Additionally, the complexity of data dependencies might require careful hyperparameter tuning to prevent the copula from overfitting, which could bias the inference of the causal relationships.Because of these challenges, we do not recommend using FFs on real-world datasets for statistically inferring treatment effect sizes, as causal benchmark datasets are usually small.In this section we present the results of multiple causal inference methods on data generated from FFs trained on two real-world datasets. The first is the Lalonde data, taken from a randomised control trial to study the effect of a temporary employment program in the US on post intervention income level (LaLonde, 1986) . The second is an observational dataset used to quantify the effect of individuals' 401(k) eligibility on their accumulated net assets, in the presence of several relevant covariates (Abadie, 2003) . Both datasets have a binary treatment and continuous outcome. Appendix D.3 can be referred to for a more comprehensive description of the data. In addition, we present diagnostics on the quality of the model fit in Appendix D.3.6, and the loss optimization for both datasets is presented in Appendix D.3.7. FFs were fitted to both datasets and used to simulate data with an ATE of 1000. We simulate 50 datasets of size N = 1000 from three different cases each: i) no confounding, ii) with confounding according to the propensity flow inferred in the model fitting, and iii) with propensity flow confounding and unobserved confounding introduced via a Gaussian copula. A variety of causal inference methods (see Appendix D.3.4 for a more detailed description) were fit to the data sets, including a difference of means (DoMs) estimate which is an unbiased estimator of the treatment effect for randomised data. The inferred ATEs across all runs are presented in Figure 3 and Figure 4 .In both cases, all inference methods demonstrate no bias when fitted to unconfounded data. With real-world confounding, most methods estimate the correct ATE in Figure 4 , whereas the DoMs shows a substantial bias from the ground truth. In Figure 3 however, all methods infer the correct ATE including DoMs. This is not surprising as the original data was randomised; the propensity flow here appears to simply add more noise to the outcomes. Finally, we note that all causal inference methods show confounding bias in the far right hand plots, demonstrating that FFs can simulate data with replicate the effects of hidden confounding. Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?Answer: [Yes] Justification: Information about the datasets used can be found in Appendix D.3. Appendix C elaborates on the model infrastructure. The code required to reproduce the results is attached to this paper. 
conclusions: 
 We introduce Frugal Flows, a novel likelihood-based model that leverages NFs to flexibly learn the data-generating process while directly targeting the marginal causal quantities inferred from observational data. Our proposed model addresses the limitations of existing methods by expliclitly parameterising the causal margin. FFs offer significant improvements in generating benchmark datasets for validating causal methods, particularly in scenarios with customizable degrees of unobserved confounding. To our knowledge, FFs are the first generative model that allows for exact parameterisation of causal margins, including binary outcomes from logistic and probit margins.Our experiments validated the empirical effectiveness of FFs, showing that they can infer the correct form of causal margins on confounded data simulations. Despite these promising results, FFs come with certain limitations that need to be addressed in future research. NFs require extensive hyperparameter tuning, which can be computationally intensive and time-consuming. Moreover, we see that FFs perform better in inference tasks with larger datasets. Future work could explore alternative ML copula methods and architectures that may be more effective for smaller datasets. Fortunately, this is less problematic for simulation as specification of the exact causal margin is left to the user. Additionally, the dequantising mechanism used by FFs implicitly shuffles the order of discrete samples, potentially losing some inherent structure in the data, making FFs less suitable for categorical datasets without implicit ordering.In summary, Frugal Flows offer a novel approach to causal inference and model validation that combines flexibility with exact parameterisation of causal effects. Future work will refine the inference capabilities and extend the applicability of FFs to a wider range of data types and sizes.Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [Yes] Justification: We provide the codebase as a ZIP attachment (and also as a anonymised link) for implementing our methods and experiments along with the required external packages.The process for generating the data is recorded in the appendix of the paper, and the code for doing so is also provided. We provide the code for processing the real-world data for the benchmarking experiments, and ensure that it is documented and referenced in the paper. Guidelines:• The answer NA means that paper does not include experiments requiring code. • The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.• The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes] Justification: The results presented in tables record the mean and 2σ error with a description on how these were estimated (either multiple runs, or standard deviations on estimated parameters inferred from robust linear regression). Other results are presented using boxplots which present the relevant quantile ranges, median, and range. The methods for generating these uncertainty bounds (multiple runs) is documented in the main body of the paper.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes] Justification: We name the compuational resources and runtime used in Appendix D.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?Answer: [Yes]• We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: For the relevant licenced assets and packages, we cite the paper corresponding to the package if possible, and if not, reference the URL and the year the package has been developed. Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 13. New Assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [Yes] Justification: We release our code with a README which points the user to the relevant notebooks/scripts used to generate a particular experiment in the paper. Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: The frugal parameterisation proposed by Evans and Didelez (2024) provides a method for simulating from a parametric marginal causal model, by starting with this distribution and building the rest of the model around it.Figure 5 : A generalised example of a static causal treatment model. The past P (T, Z) (black) can be freely specified separately from the causal effect (blue). However, the dependency measure between Z and Y (red), ϕ should be parameterised in such a way that the margins P (Z) and P (Y | do(T )) are invariant to changes in ϕ.We specify the notation used in this appendix. Functions labelled F i (•) are CDFs for the variable i. Apart from this, in general density functions will be labelled with a lower case letter, whereas CDFs will be named with the upper case (e.g. we contrast the copula density c(u 1 , u 2 ) with the distribution function C(u 1 , u 2 )).Consider firstly the case of a static treatment model with a single outcome Y , a single treatment T and an effective pretreatment covariate set Z. Assume that any of these covariates occur prior to treatment even if they do not causally affect the treatment directly. Evans and Didelez (2024) construct frugal models in three parts:• The causal distribution of interest P (Y | do(T ))• The past P (Z, T ) • The intervened variation independent dependency measure ϕ(Y, Z | do(T )).The three frugal components are variation independent in the sense that they characterise nonoverlapping components of the full observational joint. We quote the following definition from Evans and Didelez (2024):Definition 1 Take a set Θ and two functions defined on it ϕ, ψ. We say that ϕ and ψ are variation independent if (ϕ × ψ)(Θ) = ϕ(Θ) × ψ(Θ); i.e. the range of the pair of functions together is equal to the Cartesian product of the range of them individually.Variation independence (VI) is a highly desirable property for a parameterization, since it allows different components to be specified entirely separately. This is extremely useful if one is trying to use a link function in a GLM, or to specify independent priors for a Bayesian analysis. In addition, VI is important in semiparametric statistics. The definition simply states that the Cartesian product of the images is the same as the image of the joint map. For example, in a bivariate gamma-distribution with positive responses, thenµ 1 ∈ R + and µ 2 ∈ R + is a variation independent parameterization, since (µ 1 × µ 2 )(Θ) = R + × R + = µ 1 (Θ) × µ 2 (Θ). However, if we replace µ 2 with µ ′ 2 = µ 2 -µ 1 (for example), then although the range of this parameter is R, (µ 1 × µ ′ 2 )(Θ) = {(x, y) : x > 0, y > -x} ̸ = R + × R = µ 1 (Θ) × µ ′ 2 (Θ).Central to this is the choice of ϕ * . This dependency measure should encode dependencies between Z and Y | do(T ), but not provide information about their marginal distributions.Discrete frugal models can be parameterised by conditional odds ratios, while continuous variables typically use copulae. Both allow for variation independent parameterisation of the full joint distribution. The methodology facilitates the creation and simulation of models with parametrically determined causal distributions, enabling fitting using likelihood-based techniques, including fully Bayesian methods. Furthermore, this parameterisation covers a range of causal quantities, such as the average causal effect and the effect of treatment on the treated.Copulae present a powerful tool to model joint dependencies independent of the univariate margins. This aligns well with the requirements of the frugal parameterisation, where dependencies need to be varied without altering specified margins (the most critical being the specified causal effect). Understanding the constraints and limitations of copula models ensures that causal models remain accurate and consistent with the intended parameterisation.Sklar's theorem (Sklar, 1959; Czado, 2019) is the fundamental foundation for copula modelling, as it provides a bridge between multivariate joint distributions and their univariate margins. It allows one to separate the marginal behaviour of each variable from their joint dependence structure, with the latter being represented by the copula itself.Theorem 1 For a d-variate distribution function F 1:d ∈ F(F 1 , . . . , F d ), with j th univariate margin F j , the copula associated with F is a distribution function C : [0, 1] d → [0, 1] with uniform margins on (0, 1) that satisfiesF 1:d (y) = C(F 1 (y 1 ), . . . , F d (y d )), y ∈ R d .1. If F is a continuous d-variate distribution function with univariate margins F 1 , . . . , F d and rank functionsF -1 1 , . . . , F -1 d then C(u) = F 1:d (F -1 1 (u 1 ), . . . , F -1 d (u d )), u ∈ [0, 1] d .2. If F 1:d is a d-variate distribution function of discrete random variables (more generally, partly continuous and partly discrete), then the copula is unique only on the setRange(F 1 ) × • • • × Range(F d ).The copula distribution is associated with its density c(•)f (y) = c(F 1 (y 1 ), . . . , F d (y d )) • f 1 (y 1 ) . . . f d (y d )where f i (•) is the univariate density function of the i th variable.Note that Sklar's theorem explicitly refers to the univariate marginals of the variable set {Y 1 , . . . , Y d } to convert between the joint of univariate margins C(u) and the original distribution F (y). For absolutely continuous random variables, the copula function C is unique. This uniqueness no longer holds for discrete variables, but this does not severely limit the applicability of copulae to simulating from discrete distributions. The non-uniqueness does play a more problematic role in copula inference, however (Genest and Nevslehova, 2007 ).An equivalent definition (from an analytical purview) isC : [0, 1] d → [0, 1] is a d-dimensional copula if it has the following properties: 1. C(u 1 , . . . , 0, . . . , u d ) = 0 2. C(1, . . . , 1, u i , 1, . . . , 1) = u i . 3. C is d-non-decreasing. Definition 2 A copula C is d-non-decreasing if, for any hyperrectangle H = d i=1 [u i , y i ] ⊆ [0, 1] d , the C-volume of H is non-negative. H C(u) du ≥ 0 B.In addition to the above, copulae encode a degree of ordering in the joint as probability integral transforms are inherently ranked, and hence should only be used for variables that have an inherent ordering of their own (e.g. count or ordinal data models). While approaches to model discrete variables exist in parametric copulae models (Zilko and Kurowicka, 2016; Panagiotelis et al., 2017) , more flexible non-parametric copulae struggle to capture the dependencies of empirical copulae. Similar to Kamthe et al. (2021) we use the approach suggested by Rüschendorf (2009) . An outline of this method is presented in Appendix B.2.2. However, unlike Kamthe et al. we use the empirical CDF inferred from the discrete data as opposed to modelling the CDF with a marginal flow.In order to deal with discrete variables, we use a similar approach as taken by Kamthe et al. (2021) , who quote the generalised distributional transform of a random variable found originally proposed by Rüschendorf (2009) . We quote the main result from Rüschendorf (2009) below.Theorem 2 On a probability space (Ω, A, P ) let X be a real random variable with distribution function F and let V ∼ U (0, 1) be uniformly distributed on (0, 1) and independent of X. The modified distribution function F (x, λ) is defined by F (x, λ) := P (X < x) + λP (X = x).We define the (generalised) distributional transform of X by U := F (X, V ).U = F (X-) + V (F (X) -F (X-)).Rüschendorf (2009) makes a key remark about the generalised transform's lack of uniqueness for discrete variables. Such a dequantisation step may introduce artificial local dependence which may lead to an incorrect flow being inferred, and therefore hinder the inference of the causal margin.The FF software used for this paper can be found in the GitHub repository https://github.com/llaurabatt/frugal-flows.git. FF software builds upon FlowJax (Ward, 2024) , a Python package implementing normalising flows in JAX (Bradbury et al., 2018) . JAX is an open-source numerical computing library that extends NumPy functionality with automatic differentiation and GPU/TPU support, designed for high-performance machine learning research.The Frugal Flow component builds a flow of the form in the bottom part of Figure 2 . It allows us to implement F Y |T with either (i) a customised CDF conditioned on T , of a known parametric family;(ii) a univariate NSF on the [-1, 1] interval modified to allow a location translation parameter that represents the ATE for T , where the input is mapped from the real line via a tanh transform; or finally (iii) a univariate NSF on the [-1, 1] interval that is not conditional on T , where the input is mapped from [0, 1] via an affine transform. As for known parametric families, only the Gaussian CDF is currently implemented, but the architecture allows us to define any different parametric class provided that it constitutes a diffeomorphism. As for the univariate NSF in (iii), it does not explicitly learn an ATE in the training phase, but can be used for simulation of e.g. binary outcomes by applying a subsequent logistic transformation to its outcome.The multivariate NSF element is a composition of multiple modified NSF subflows. In each subflow the first transform is fixed to be an identity, while the other dimensions are transformed with a monotone rational quadratic spline whose knot parameters are produced by masked multilayer perceptrons (MLPs) implemented as in Germain et al. ( 2015), conditional on the first dimension. In order to increase expressivity, dimension permutation is usually applied between the different subflows in the NSF composition. We still allow this permutation but excluding the first dimension, that is fixed to be at the top in each subflow. The NSF acts on the on the [-1, 1] interval and is mapped from and back to the quantile space with affine transforms.Tunable hyperparameters to the Frugal Flow component are the number of subflows of the multivariate NSF, the width and depth of the MLPs and the number of spline knots, together with the specific hyperparameters of the chosen F Y |T .Marginal flows architecture for continuous variables.Each marginal flow for the continuous covariates maps each variable from the real line to the [-1, 1] interval with a tanh transform, then applies a univariate NSF on the [-1, 1] interval, and maps back to the standard uniform base distribution via an affine transform. Tunable hyperparameters are the number of subflows of the NSF, the width and depth of the MLPs and the number of spline knots.To map a discrete Z to the ranks V Z , we compute its empirical CDF and then apply the procedure outlined in Appendix B.2.2. We use the inverse of the same empirical CDF to map ranks back to the Z for sampling.Propensity score model architecture.To map a discrete T to the ranks V T , we compute its empirical CDF and then apply the procedure outlined in Appendix B.2.2. A univariate NSF flow with a uniform base distribution is then applied to learn the copula CDF of T on the [0, 1] support, conditioned on Z. The Z conditioning is obtained by adding Z as an (unmasked) input to the MLP that produces the knot parameters for the rational quadratic spline. This is standard in NF literature. Tunable hyperparameters are the number of subflows of the NSF, the width and depth of the MLPs and the number of spline knots.The propensity score model can be inverted to generate T samples conditioned on a given Z. Uniform samples are pushed through the trained univariate propensity score flow to obtain ranks, that are then mapped to the discrete space via the inverse of the empirical CDF of T .Training the Frugal Flow and the propensity score flow.In order to train a FF, one must fit the marginal flows first. The marginal flows are trained (for continuous variables) via maximising the log-likelihood with stochastic gradient descent, and/or the discrete Z are mapped to the rank space via the procedure in Appendix B.2.2. Next, the FF is trained via maximum likelihood estimation (MLE), taking as input the outcome Y together with the ranks V Z , and conditioning the flow for the causal margin on treatment T where required by the chosen method. For MLE optimisation, we take advantage of JAX automatic differentiation capabilities and use the Adam optimiser (Kingma and Ba, 2015) , whose hyperparameters can also be tuned. If required, the propensity flow is likewise trained on V T conditioning on Z via MLE with an Adam optimiser.One can use a trained FF for simulation of causal benchmarks. The general data simulation pipeline is:1. Generate a sample of U T|Z , V Y | do(T ) from a bivariate Gaussian copula, parameterised by correlation ρ, which quantifies the degree of unobserved confounding one wishes to encode in the benchmark. If no unobserved confounding is desired, set ρ = 0. All experiments were run on a MacBook (16-inch, 2021) with an M1 Max chip and 32GB memory using the CPU.The simulated data generated for the inference experiments was generated using the causl package written in R, which was called in Python via the rpy2 package (Evans, 2021; Evans and Didelez, 2024) .The covariates were either selected to be binary (marginally distributed according to Bernoulli(p = 0.5) or continuous (marginally distributed according to Gamma(µ = 1, ϕ = 1). The marginal causal effect was chosen to be a linear Gaussian; Y | do(T ) ∼ N (T, 1).The underlying data generating process uses a multivariate Gaussian copula to generate dependencies between the marginal covariates and the causal effect. The Spearman correlation matrix used to generate the data for models M 1 and M 2 isR 4 =     1.0 0.5 0.3 0.1 0.8 0.5 1.0 0.4 0.1 0.8 0.3 0.4 1.0 0.1 0.8 0.1 0.1 0.1 1.0 0.8 0.8 0.8 0.8 0.8 1.0    and the correlation matrix used to generate data for model M 3 isR 10 =                1.0 0.3 0.4 0.5 0.1 0.2 0.7 0.5 0.4 0.5 0.5 0.3 1.0 0.3 0.6 0.3 0.4 0.4 0.6 0.3 0.2 0.5 0.4 0.3 1.0 0.5 0.2 0.1 0.1 0.0 0.4 0.4 0.5 0.5 0.6 0.5 1.0 0.2 0.2 0.5 0.5 0.3 0.4 0.5 0.1 0.3 0.2 0.2 1.0 0.1 0.5 0.6 0.2 0.4 0.5 0.2 0.4 0.1 0.2 0.1 1.0 0.0 0.4 0.2 0.5 0.5 0.7 0.4 0.1 0.5 0.5 0.0 1.0 0.4 0.4 0.4 0.5 0.5 0.6 0.0 0.5 0.6 0.4 0.4 1.0 0.4 0.4 0.5 0.4 0.3 0.4 0.3 0.2 0.2 0.4 0.4 1.0 0.4 0.5 0.5 0.2 0.4 0.4 0.2 0.5 0.4 0.4 0.4 1.0 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 1.0               , where the later rows/columns are indexed by the causal effect ranks, V Y | do(T ) , and the earlier rows/columns correspond to the Spearman correlation matrix between the ranks of the covariates, V Z .The propensity model for M 1 and M 2 was a sigmoidp T|Z (t | z) = Sigmoid(-0.3 + 0.1z 1 + 0.2z 2 + 0.5z 1 z 2 -0.2z 3 + z 4 ),and M 3 was parameterised byp T|Z (t | z) = Sigmoid(-0.3 + 0.1z 1 + 0.2z 2 + 0.5z 3 -0.2z 4 + z 5+ 0.3z 6 -0.4z 7 + 0.7z 8 -0.1z 9 + 0.9z 10 ).The hyperparameters and runtime for the simulated inference datasets are presented in Table 4 . In these cases, the models were trained with a default set of hyperparameters. To demonstrate the FF ability to generate discrete outcomes from known marginal logistic models, we ran the following experiment. First, data of the following formZ ∼ N (µ = 0, σ = 2) V Z , V Y | do(T ) ∼ c Gaussian (ρ = 0.8) Y | do(T ) ∼ N (µ = 2X, σ = 1)was generated from the causl package. It was then fitted with a FF using the same hyperparameters as in Appendix D.1.1. We then generated samples from a custom logistic CDF such thatY | do(T ) ∼ Bernoulli(p = Sigmoid(2X -1)).A dataset size of N = 1000 was generated from the model, and fit using two models. The first is a Bernoulli outcome regression model, and the second uses inverse propensity weighting (IPW) to estimate the logistic parameters instead. The outcome regression (OR) estimates are biased indicating a clear confounding effect, whereas the IPW estimates comfortably contain the true parameters within their 2σ bounds. These results are presented in Table 3 . V W Y | W , do(T ) V W   I(•) F -1 Y |W,do(T ) (•) I(•)   V W V Y |W,do(T ) V W NSF   c(v W ) c(v Y |W,do(T ) ) = 1 c(v W | v Y |W,do(T ) , v W )   U 1:d U d+1 U d+2:(D+2)In the main paper, we comment on the ability of the model to simulate data from distributions where the causal effect is a marginal quantity taken over the entire covariate setEQUATIONHowever, one may wish to simulate from more complex heterogeneous treatment effect models. Interest may lie in the causal treatment margin conditional on the subset of variables W :EQUATIONwhich we call the conditional treatment margin. We infer this effect by constructing a Frugal Flow which ensures that the pretreatment covariate joint p Z (•) is correctly inferred and that the conditional treatment margin ranks are uniformly distributed. A modified version of the Frugal Flow illustrated Figure 2 is used to account for this change. The choice of F -1 Y |W,do(T ) (•) can be left to the user for inferring the Frugal Flow. For simulating benchmarks, the conditional treatment margin can be free set to any valid CDF, for example:Y | W , do(T ) ∼ N (µ = g(W , T ), 1)where g(•) : R d × T → R can be chosen to encode arbitrary heterogeneity in treatment effects.The Jobs dataset by LaLonde is a benchmark in causal inference studies, where job training serves as the treatment and the outcomes are post-training income and employment status. Originating from the National Support Work Demonstration (NSW), this randomized controlled trial (RCT) examines the impact of a temporary employment program (i.e. the treatment) in the US on participants' income levels (LaLonde, 1986) . Due to its design the treatment assignment is random, eliminating unobserved confounding. The measured features, all recorded in 1975, are:• an individual's age in years;• the number of years an individual spent in education;• whether an individual is black;• whether an individual is hispanic;• an individual's marital status (1 if married, 0 otherwise);• whether an individual has a high school degree.The outcome is the individual's real earnings in 1978.The 401(k) savings plans dataset has been analysed in a variety of studies. We use it to investigate the impact of eligibility to enroll on the increase in net assets.The dataset includes 9,915 individuals with the following variables measured:• age of the individual in years;• income of the individual;• years of education the individual has completed;• size of the individual's family;• indicator variable of whether the individual is married (1 for married, 0 otherwise);• indicator of whether there are two earners in the household (1 if two earners, 0 otherwise);• membership of a defined benefit pension scheme (1 if true, 0 otherwise);• eligibility for Individual Retirement Allowance (IRA) (1 if true, 0 otherwise);• homeownership status of the individual (1 if true, 0 otherwise).Section 4.1 reports FF ATE inference performance in a simulated setting comparing to a number of methods.• Outcome regression (OR) ATE results are obtained by regressing Y on the treatment T together with the covariates Z in the different scenarios and reporting the coefficient and confidence interval on T .• Propensity score matching is implemented using the R package MatchIt for estimating the ATE (Stuart et al., 2011) .• Causal normalising flow (CNF) (Javaloy et al., 2024) is trained using the causal abductive model with one layer, that the paper reports to be the best-performing model variation (Paragraph 6.1). We use the hyperparameter settings recommended in the package and do not perform hyperparameter tuning. For the flow architecture, we use neural spline flows, as the paper reports in Appendix D.3 that they yield a better performance than simple masked autoregressive flows, plus this resembles our architecture choice in Frugal Flows. We do not add uniform independent noise to the binary inputs as recommended in paragraph 3.1 as we find this worsens the ATE estimates for the model.Similar to Parikh et al. (2022) we use a variety of different causal inference methods to validate the generated benchmark samples of our model.• Propensity score matching is implemented using the R package MatchIt for estimating the ATE (Stuart et al., 2011) .• Causal BART is implemented via the R package dbarts using default hyperparameters (Dorie et al., 2024).• The double machine learning methods are implemeneted using the Python package EconML (Research, 2019) using the scikit-learn's machine learning API for the same. For GBT DML, we used the method with 100 trees, and the linear DML used ridge regression (Pedregosa et al., 2011) . • EconML was also used for implementing the S-, T-and X-learners also using scikit-learn's ML API to for gradient boosting trees and ridge regression. • TMLE was implemented using the zepid Python package (Zivich, 2020) .For both datasets, a random hyperparameter search was conducted by choosing the hyperparameter set which minimised the validation loss, given a train/test data split of 9/1. The total number of neural network of the hyperparameter tuned Frugal Flow for the Lalonde and e401(k) datasets are 485243 and 106969 respectively. We conducted additional validation of the proposed Frugal Flows method to enhance its robustness in comparison to current state-of-the-art methods such as Credence (Parikh et al., 2022) and RealCause.Credence allows for the exact specification of conditional average treatment effect (CATE) in generative samples, whereas RealCause adjusts the causal effect post hoc, preventing it from realistically modelling a null hypothesis where the average treatment effect (ATE) is zero. Additionally, Frugal Flows and Credence both model unobserved confounding, a feature absent in RealCause, making Credence the more suitable method for direct comparison.We ran the benchmarking simulations in Section 4.2 with Credence, using its default parameters, evaluating model performance on Lalonde and the e401(k) dataset. We compared the correlation matrices of the pretreatment covariates and outcomes for the original data, Frugal Flows-generated data, and two Credence-generated datasets with different causal constraints. The results, illustrated in Figure 7 and Figure 8 , show that Frugal Flows produce samples closely resembling the original data, especially for the larger e401(k) dataset.While Credence also performs well on the e401(k) dataset, altering its causal constraints significantly affects the covariate dependencies. In contrast, Frugal Flows optimise the model once, allowing for causal constraint modifications without altering the covariate joint distribution or propensity score. NeurIPS Paper Checklist• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 


Decision: Summary:

The paper introduces Frugal Flows, a generative modeling approach that learns the data generation process with explicit parametrization for the marginal causal effect of treatment on outcomes. 

Pros:

+ Addresses a significant gap by allowing for general constraints on quantities of interest during data generation for causal inference.

+ Novel approach combining frugal parametrization, normalizing flows, and copulas for causal effect estimation and synthetic data generation.

+ Well-written with clear explanations of complex concepts and potential for high impact in the causality research community.

Cons:

+ Limited empirical validation compared to prior work in generative modeling for causal inference.

Reviews: review: paper_summary: paper_summary: This work proposes to leverage existing neural density estimators (specifically, normalizing flows) to exploit a newly-proposed "frugal parametrization" that can capture the causal marginal distribution of an underlying causal model. Under this parametrization, the authors show how to specify and train each component of the model, and thus train the proposed Frugal-Flows to match the observational distribution as closely as possible, while being able to tune the marginal causal effect present in the generated data.

This way, frugal flows can be used to generate synthetic causal benchmarks that closely represent the _observational_ data while having more difficult-to-estimate causal effects, putting existing approaches to the test.
strength_weakness: strength_weakness: - **W1.** I find the frugal parametrization to be extremely under-explained, relying too much on the reader having full knowledge of the referenced work. Similarly, there is little to no explanation/intuition on why the frugal parametrization would properly capture the marginal causal distributions.
- **W2.** The lack of explanations also applies to other concepts, e.g., "conditional ignorability" (line 39) "variation independence" (line 82, and I know the definition is later in App. A), or why copula-based flows would target conditional causal effects instead of marginal causal ones (line 182). (similar with lines 221 and 229)
- **W3.** There are no mention to related works that propose similar ways of constructing causal benchmarks. From a 1-min search in google scholar, I already found some likely relevant works: [Work 1](https://arxiv.org/abs/2406.08311), [Work 2](https://arxiv.org/abs/2011.15007).
- **W4.** I find the experiments a bit underwhelming, specially those from Section 4.1. The authors should at least show how is the fitting of the observational likelihood, and if they want to show the capabilities of frugal flows for causal inference (and not only causal-benchmark generation), they should compare with other methods like [Causal Normalizing Flows](https://arxiv.org/abs/2306.05415).
questions: questions: - **Q1.** I am not sure that I understand what does the dotted red line represent in the boxplots.
- **Q2.** Doesn't the statement in lines 291-294 directly contradict what you say later in lines 296-297?
- **Q3.** Why is Figure 1 placed there?
limitations: limitations: I think limitations are properly discussed.
paper_summary: paper_summary: This paper proposed a generative model called Frugal Flows making use of copula flows to infer about marginal causal effects by simulating the data generating process.
strength_weakness: strength_weakness: See questions.
questions: questions: I have two question --

1. A highly related (and I suspect might be viewed as a "dual" appproach to your Frugal Flow) is the statistical matching (e.g., bipartite matching) to estimate the average treatment effect. It would be very informative to compare this as one of the baselines in your benchmarking and validation, as this also shed lights on how these two different schools of causal inference may (or may not) converge on ATE estimation.

2. I think it is good to use real data for benchmarking/validation (but perhaps benchmarking is a bit strong here since only two datasets were used), but usually it is unclear how to interpret the results since the ground truth is unknown. Can you design and run some controlled synthetic experiment to verify the model?
limitations: limitations: See questions.
paper_summary: paper_summary: The paper introduces a generative modeling approach called Frugal Flows, designed to learn the data generation process with an explicit parametrization for the marginal causal effect of treatment on outcomes. Inspired by the frugal parametrization of marginal structural models, this approach models the marginal intervention distribution $p(Y|do(X))$ directly, rather than the joint distribution $p(Y|Z, do(X))$. This helps in preserving any constraints on the average treatment effect while flexibly modeling the data generation process. Frugal Flows employs copula flows to parameterize the model, accommodating constraints on the average causal effect and handling unobserved confounding during data generation. The authors validate the proposed method through experiments on both synthetic and real-world datasets, demonstrating its ability to generate realistic datasets with user-specified constraints.
strength_weakness: strength_weakness: My main concern with the work is the limited empirical validation of the proposed approach. Given that the primary contribution is the learning methodology rather than theoretical analysis, I would expect a more extensive set of experiments to validate its effectiveness. For example, prior research on generative modeling for causal inference, such as the work by [1], includes comprehensive experiments with various statistical tests to assess the realism of generated samples and a broader benchmarking of causal estimators. This paper would benefit from similar depth in its empirical evaluation.

It would nice if the authors can conduct similar experiments to asses whether learned generative model generates realistic samples and evaluate it on more datasets. Also, the authors should compare with the prior works [1, 2] as baselines to establish which approach is the best at capturing the underlying data generation process, and empirically validate their claim (Section 2.6) that the proposed approach would be better than prior works at capturing used-specified constraints on the average causal effect.

References

[1] Neal, Brady, Chin-Wei Huang, and Sunand Raghupathi. "Realcause: Realistic causal inference benchmarking." arXiv preprint arXiv:2011.15007 (2020).

[2] Harsh Parikh, Carlos Varjao, Louise Xu, and Eric Tchetgen Tchetgen. Validating causal inference methods. In International conference on machine learning, pages 17346–17358. PMLR, 2022.
questions: questions: * A suggestion for the notation is that the authors could use $T$ instead of $X$ to denote the treatment variables in the paper. This way the notation would be less confusing, as $X$ represents a general random variable in Section 2.5 and Section 2.6

* Maybe there is a typo in Figure 2? The top row should be $\mathcal{F_{Z_i}}^{-1}(.)$   as we transforming the covariate $\{ Z_i \}$ to the correlated uniform variables $\{ V_{i} \}$

* I don't understand the Section 3.1.1 on copula flow for $X$ on $Z$. Why don't we directly model $p(X|Z)$ using a normalizing flow and why do we need to parametrize using copula flow as $p(X|Z)= p(X).c(X|Z)$?
limitations: limitations: Yes, the authors have adequately addressed the limitations of their work.
paper_summary: paper_summary: This paper introduces _Frugal Flows_ a method that learns the data distribution of data for causal effect estimation; namely outcome $Y$, binary treatment $X$ and pretreatment covariates $\mathbf{Z}$. 
Through a combination of frugal parametrisation, normalizing flows and copulas, separate components for the marginal causal effect $p_{Y| do(X)}$, the probability integral transforms of $\mathbf{Z}$ and the propensity score are leaned.
(The components of) the learned model, can be used for (i) estimating the marginal effect and (ii) to generate synthetic data with a fixed marginal effect for benchmarking other causal inference methods.
In the second application the component for the marginal effect is switched out for another with desired properties.
(i) is demonstrated on small synthetic datasets.
(ii) is demonstrated by fitting FFs to two real-world datasets and generating synthetic data with adjusted properties.
strength_weakness: strength_weakness: - Normalizing Flows have been used in the causal modelling context before (see [1, 2]). While prior works solve different problems (the inferred latents correspond to exogenous variables of an SCM, not directly applied to causal effect estimation), I think it would still be valuable to contrast this work to what has been done before for future reference in the literature.
- L59: The basic causal assumptions aren't explicitly stated. What are the causal assumptions on $X$, $Y$ and $\mathbf{Z}$? It seems like the method wouldn't hold if $\mathbf{Z}$ was a mediator (I suppose the equation after L60 wouldn't hold). A reference to a 500+ page book is given for the assumptions, which feels like a slap in the face for the reader.
- The notation for interventional distributions is confusing: what's the difference between using an asterisk and explicitly using the do-notation? In the equation after L60, the LHS seems to be an interventional quantitiy (asterisk, but no do-notation), whereas Equation (1) has the do-notation, but no asterisk. Do the two notation elements mean different things?
- I think this paper would greatly benefit from a visual abstract showing how the different flows and distributions come together. Maybe this is something that could be added for the camera-ready.


Minor:

- L201: typo

[1] Javaloy et al. "Causal normalizing flows: from theory to practice." NeurIPS 2023

[2] Wendong et al. "Causal component analysis" NeurIPS 2023
questions: questions: - Fig. 1: What's the meaning of the red undirected edges. Does it mean they could go either way and/or they could be confounded? Please specify this somewhere in the paper or appepndix.
- Def. 1, App. A: I struggle to understand this definition. The cartesian product on the RHS makes sense to me, but what does the LHS of the equation mean? What is the "x" operation between functions? Do the two functions map to the same space?
- Fig. 2: Why is the first line needed? If I understand correctly, this just makes all pretreatment variables uniform. Couldn't you put $\mathbf{Z}$ directly in the second line?
- You show synthetic data generation based on two datasets in Sec. 4.2. Why couldn't you use the same datasets to test causal effect estimation in Sec. 4.1?
- How many datapoints are in the Lalonde dataset?
- In training, how did you check whether the training has succeeded? I suppose you minimize the log-likelihood, how did you define "good enough"? I'm asking because the training seems pretty fast (App. D2.4). What's the total number of parameters for each of the datasets?
limitations: limitations: - The biggest limitation of the work seems to be the requirements for the dataset size, needed to train normalizing flows. The authors mention this in Sec. 4.1. There could be some applications that have enough data for using Frugal Flows for causal effect estimation (e.g. online businesses with many customers, or well-curated medical datasets like the ones from healthcare providers in Isreal). However, for most applications the data won't be enough.

Comments: comment: Thank you for providing a detailed response to my questions. I will keep my score of acceptance.
Thank you for addressing my concerns.
Thanks for the detailed response! My questions have been addressed and I have increased my rating accordingly!
I thank the authors for their super detailed responses and the additional experiments. I do think they add quite some value, and if the authors use the feedback to improve the readability of the manuscript (and thus how welcoming it is for newer audiences), I think it has potential to be quite a good paper.

With respect to the new results, I am a bit surprised by the results of CNF on model 1. Just to make sure, I'd encourage the authors to check that they used a model different from MAF for the base model, as this is not a universal density approximator.

In any case, I am happy with the response from the authors and I will update my score.