Amortized Inference of Causal Models via Conditional Fixed-Point Iterations
Authors: Divyat Mahajan
Jannes Gladrow
Agrin Hilmkil
Cheng Zhang
Meyer Scetbon
Venue: Accepted by TMLR ()

Abstract: Structural Causal Models (SCMs) offer a principled framework to reason about interventions and support out-of-distribution generalization, which are key goals in scientific discovery. However, the task of learning SCMs from observed data poses formidable challenges, and often requires training a separate model for each dataset. In this work, we propose an amortized inference framework that trains a single model to predict the causal mechanisms of SCMs conditioned on their observational data and causal graph. We first use a transformer-based architecture for amortized learning of dataset embeddings, and then extend the Fixed-Point Approach (FiP) to infer the causal mechanisms conditionally on their dataset embeddings. As a byproduct, our method can generate observational and interventional data from novel SCMs at inference time, without updating parameters. Empirical results show that our amortized procedure performs on par with baselines trained specifically for each dataset on both in and out-of-distribution problems, and also outperforms them in scare data regimes.

Published in Transactions on Machine Learning Research (12/2025)
Amortized Inference of Causal Models via Conditional Fixed- Point Iterations
Divyat Mahajan',, Jannes Gladrow?, Agrin Hilmkil?, Cheng Zhang?, Meyer Scetbon Mila, Université de Montréal, 2 Microsoot Research
Reviewed on OpenReview: https: // openreview. net/ formmf id D9pg2556e5
Abstract
Structural Causal Models (SCMs) offer principled framework to reason about interventions and support out-of-distribution generalization, which are key goals in scientific discovery. However, the task of learning SCMs from observed data poses formidable challenges, and often requires training a separate model for each dataset. In this work, we propose an amortized inference framework that trains a single model to predict the causal mechanisms of SCMs conditioned on their obsorvational data and causal graph. We first use transformer- based architecture for amortized learning of dalaset embeddings, and then exlend the Fixed-Point Approach (FiP) to infer the causal mechanisms conditionally their dataset embeddings. As a byproduct, our method can generate observational and interventional data from novel SCMs at inference time, without updating parameters. Empirical results show that our amortized procedure performs on par with baselines trained specifically for each dataset on both in and out-of-distribution problems, and also outperforms them in scarce data regimes
1 Introduction
Learning structural causal models (SCMs) from observations is core problem in many scientific do- mains (Sachs et al. 2005 Foster et al.
2011 Xie et al. 2012), as SCMs provide a principled way to model the data generation process. They enable sinulation of controlled interventions, offering the potential to accelerate
scientific discovery by predicting the outcomes of unseen experiments without requiring sotlyy/iim-consuneee
lab trials (e et al., 2023: Zhang al.] 2024). However, solving this inverse problem of Icarning SCMs from observed data challenging both the causal graph and the causal mechanisms are unknown a priori. Recovering causal graphs is an NP-hard combinatorial optimization problem the space of causal graphs is super-exponential (Chickering et al. 2004). This subsequently complicates the estimation of causal mechanisms via maximum likelihood estimation per node (Blöbaum et al. 2022). To address these challenges, recent, approaches have focused on learning causal mechanisms with partial causal struct.ure, using techniques
such as autoregressive flows (Khemakhem et al. 2021 Geffner et al. 2022 Javaloy et al. 2023 or modeling
SCMs as fixed-point iterations via transformers (Scetbon et al. 2024).
Despite these advances, major limitation remains: each new dataset requires training specific model, that prevents the transfer of causal knowledge across datascts. Amortized inference offers solution by lcarning single model that can generalize across instances of the same optimization problem by exploiting their shared structure (Andrychowicz et al.] 2016 Gordon et al.) 2019). This results in models that can quickly adapt to new instances at test time (Finn et al.) 2017). Amortized inference has shown success in several
challenging tasks, like bayesian posterior estimation GGarneoo et al.] 2018 Müller et al., 2021], sampling from unnormalized densities (Akhound-Sadegee
al. 2024 Sendera et al. 2024), as well as causal structure learning Lorch et al. 2022 Ke al. 2022), which is more aligned with our paper.
Equal Contribution. Correspondence to: divyatmahajanngmaill This work was done when DM was an intern Microsoft Research. JG, AH, CZ, and MS worked this project when they affiliated with Mierosoft. Research.

---

Published in Transactions on Machine Learning Research (12/2025)
IR4
X, X2 X
d
p
Conditional Embed
Cond-FiP
d
72
DAG Attn.
d
Attention node
sample
D
Encoder
Projection
d
T (z, Dx. E Rd
Figure Sketch of the approach proposed in this work. Given a dataset of observations Dx and causal graph g obtained from an unknown SCM S(P,, 9, F), the encoder produces dataset embedding u(Dx; 9), which serves condition to instantiate Cond-FiP. Then for any point z Rd, T(z, Dx,9) 9) aims at replicating the functional mechanism Fz) of the generative SCM.
In this work, tackle the novel problem of amortized inference of causal mechanisms for additive noise SCMs. While prior research has primarily focused on amortized approaches for causal discovery (Lorch et al. 2022 Dhir et al.] 2024) or treatment effect estimation (Nilforoshan et al. 2023 Zhang et al. 2023). our goal instead to train a single model capable of inferring the causal mechanisms novel SCMs, given their observational data and associated causal graph. We propose a two-step approach where we first. learn datasot emboddings via in-context learning (Garg et al. 2022) to represent the task-specific information. These embeddings are then used condition the fixed-point (FiP) approach (Scetbon el. al. 2024) for modeling causal mechanisms. This conditional modification, termed Cond-FiP, enables the model to adapt the causal mechanism for each specific instance (Figure 1 Our key contributions are highlighted below.
We propose Cond-FiP, a novel extension of FiP approach that enables amortized inference by training a single model across different instances from the functional class of SCMs.
For novel SCMs at inference, Cond-FiP can recover the causal mechanisms from the input observations without updating any parameters, thereby allowing us to generate observational and interventional data on the fly.
We show empirically 1 that Cond-FiP achieves similar performances the state-oftthe-ar. (SOTA) approaches trained from scratch for each dataset both in and out-of-distribution (OOD) problems. Further, Cond-FiP obtains better results than basclines in scaree data regimes, due to its amortized inference procedure.
1.1 Related Works
Amortized Causal Learning. Amortized inference has gained traction in causality research, particularly for structure learning. Early works by Lorch et al. (2022) and Ke et al. (2022) introduced transformer- based models trained on multiple synthetic datasets using supervised objectives for amortized inference of causal structure. Their approach aligns with recent works on in-context learning of function classes via
transformers (Müller al. 2021 Akyürek et al. 2022
Garg al. 2022 Von Oswald al. 2023). Subsequent
improvements targeted OOD generalization Wu et al. 2024).
bayesian causal discovery DDir et al. 2024).
and applications to gene regulatory networks
Ke et al.
2023
Kim ct al. 2025
Beyond structure learning,
The code available on Github: microsoft/causica
2

---

Published in Transactions on Machine Learning Research (12/2025)
amortized methods have been developed for ATE/ estimation Nilforoshan et al. 2023 Zhang et al.
2023:
Sauter et al.) 2025 Bynum et al. 2025 Balazadel et al.) 2025; Robertson et al.) 2025 Ma et al. 2025
Dhir et al. 2025), model selection (Gupta et al. 2023 and partial causal discovery tasks such as learning
topological order (Scetbon et al. 2024). However, amortized inference of causal mechanisms in SCMs remains
unaddressed, which is the central focus of our work.
Autoregressive Causal Learning. Most causal discovery methods focus first on structure learning Chick Zheng et al., 2018), followed by per-node maximum likelihood estimation to [Blbbuum et al.) 2022) In constrast, recent works causal autoregressive
ering 2002 Peters et al. 2014
recover the causal mechanisms
Geffner et al. 2022: Javaloy et al. 2023) focus on SOTA normalizing flow based generative models to infer causal mechanisms. Further, FiP Scetbon et al., 2024) modeled SCMs as fixed-point problems over causal (topological) ordering of nodes using transformer based architectures. These approaches efficiently learn SCMs but require training separate model per dataset. In this work, we remove this limitation by extending FiP to enable amortizod inference of causal mechanisms across different SCMs.
flows [Khemakhem et al.) 2021
2
Amortized Causal Learning
2.1 Brief Overview of Amortized Inference
Amortized inference aims to learn shared inference mechanism across multiple tasks that enables fast adaptation to new tasks at test time. We motivate amortized inference through the following example, consider the task of predicting the motion of objects on different planets. While trajectories vary across planets across due to differences in gravitational constants, the underlying physical laws remain the same. Rather than training a new model from scratch for each planet, we can exploit this shared structure to rapidly adapt our predictions to new settings. This is the core idea behind amortized inference, leveraging common patterns across training tasks to enable fast and efficient adaptation novel ones.
Consider task T that defines distribution over imputs (Z) and targets (Y); Z,Y ~ Pr. Given collection of tasks (T())) K k=1 and some objective function L, the goal is to learn a shared model To across tasks:
arg min Σ Ez, EZ L(Y, To(Z,
(1)
where (() denotes additional context for task (()), such as dataset with samples [Z,. ...... Zn)- Instead of retraining from scratch, the model should leverage the context I' to adapt to the task T'.
A classic approach towards this is meta-learning (Andrychowicz et al. 2016 Finn et al. 2017 Hospedales et al.) 2021), which leverages the context I by task-specific finetuning. These methods typically learn a shared initialization that is adapted for a new task via few gradient stops in an inner optimization loop. In contrast, in-context learning (ICL.) (Müller et al., 2021 Xie et al. 2021 Garg al. 2022 Akyürek al., 2024) avoids this inner loop by using transformer-bast the forward pass, ICL methods adapt to new task without any parameter updates. This capability is often attributed to transformers' ability to implicitly approximate learning algorithms such as gradient descent within their activations (Akyürek et al. 2022 Von Oswald et al. 2023
architectures. By attending to the context I during
While ICL is often used to describe "emergent" test-time adaptation in large language models (Brown et al., 2020), where the training objective does not explicitly involve those tasks. Here, we focus on its formulation prior-fitted networks (PFNs) (Hollmann et al. 2022 Robertson et al. 2024). where transformers are pretrained on synthetic datasets generated from simulator that implicitly defines a prior over tasks.
2.2 Problem Setup
We start by formally defining structural causal models (SCMs). An SCM defines the causal generative process of set of d endogenous (causal) random variables V = {XI,... Xa), where each causal variable X; is defined as a function of a subset of other causal variables (V \ (X+}) and an exogenous noise variable Nj:
X; (((P((X.), N.) s.t. PA(X.) C CV, X₂ & PA(X.)
22)
3

---

Published in Transactions on Machine Learning Research (12/2025)
Hence, an SCM S(PN, 9, F) describes the data-generation process of X [X1, .............................. .X]] noise variables N = [N1.... Na) ~ PN via. the function F: [Fi, Fal, and graph g E {0, 1] 1
Px from the
indicating the parents of each X3, that is [g == if E PA(X.). We make the following assumptions about SCMs.
g is a directed and acyelic graph (DAG), and noise variables are mutually independent (Markovian SCM).
SCMs are restricted to be addilive noise models (ANM), ie.. X; = F:(PA(X.)) + N.
While the first assumption is pretty standard, we make the ANM assumption for training the proposed dataset encoder in Scction 3.1
Consider a distribution over SCMs S(PN N,,,, 9, F) ~ Ps. Then the goal with amortized inference of causal mechanisms is to learn a single model To that can approximate the true causal mechanism F(z) for any input 2 RR.. With task specific context as g) equation 1 we have
arg min Es., Ez-rx (F)(2), Dx. Dx, 9))
Note that we consider access to causal graph g as part of the input context, which is available when training on synthetic SCMs. Even if we don't 't have access to g, we can use prior works on amortized causal learning (Lorch et al.) 2022 Ke et al. 2022) to infer the causal graphs from observations Dx This justifies our setup where the causal graphs are provided as part of the context to the model.
3 Methodology: Conditional FiP
We present methodology for learning the model T(., 9) that consists of two components: (1) dataset encoder that generates dataset embeddings n(Dx. 9) from the input context, and (2) conditional variant of FiP (Scetbon et al. 2024), termed Cond-FiP that allows it to leverage the task-specific context for amortized inference via the learned dataset embeddings u(Dx, ()) We first present our dataset encoder, then Cin,-FiP, and conclude with data generation via Cond-Fip.
3.1 Dataset Encoder
The objective this section is to develop a method capable of producing efficient latent representations of datasets. To achieve this, we propose to train an encoder that predicts the noise samples from their associatod observations given the causal structures via in-context learning (pseudo code in Algorithm [
Appendix A.4
Training Setting. We consider empirical representations of K SCMs (S(P))) (k)), ()),)))))
k= each sampled N (k) F(() ~ Pss. Each empirical representation, denoted (D() ( contains obscrvations D : [x|1 ...., xx" E Raxd and the causal graph g(k) E {0. 1d For training, also need the associated noise samples D(&) [N(()) ....., !* " e gaxd which play the role of the target variable in our prediction task. For simplicity, we drop the index & in our
K
independently from a distribution over SCMs ((P(k)
notation and assume access to the full distribution Ps. The objective is to recover the true noise DN from a dataset observations Dx and the causal graph g, which provide us with dataset embeddings as detailed below.
Encoder Architecture. Following (Lorch al. 2021 Scetbon et al. 2024 we encode datasets using transformer-based architecture that alternates attention over both sample and node dimension. Given a dataset Dx, we first apply a linear embedding L(Dx) E Raxdxd, where d, is the hidden dimension. The encoder E then applies transformer blocks, each comprising self-attention followed by an MLP (Vaswani ct al. 2017), where the attention mechanism is applicd either across the samples n or the nodes alternately. Recall the standard self-attention is defined as
exp((QKT M)//dn)
Am Q K)
exp( -M)/ 1a
4

---

Published in Transactions on Machine Learning Research (12/2025)
where Q. K E Rd denote the keys and queries for single attention head, and M {0, +oo]axd is (potential) mask. When attending over samples, the encoder uses standard self-attention without masking (M = [0]mxm) But for node-wise attention, we incorporate causal structure by masking invalid dependencies using mask M = +00 x (1 - g) in standard self-attention, with the convention that 0x (+00) =0. Finally, the embeddings E(L(Dx): 9) E Raxdxda are passed a prediction network H Raxdxds - Band implemented as 2-hidden layers MLP to project back to the original data space.
Training Procedure. We minimize the mean squared error (MSE) of predicting the target DN from the input (Dx,9) g) over the distribution of SCMs Ps available during training:
ES |D H (
Further, as we restrict ourselves to the case of ANMs, we can equivalently reformulate our training objective in order to predict the causal mechanism rather than the noise samples, as F(Dx) Dx DN- Therefore, instead propose to train encoder as follows:
EswPs [F(xx) - H E(L(x)))
Note that ANM assumption provides a simplified true mapping from data noise as - F(x), which is difficult to obtain in general SCMs. Please check Appendix A.2 for more details on justification for ANMs and why recovering noise is equivalent to lcarning the inverse SCM.
Given new dataset Dx and its causal graph G, encoder provides us with the dataset embedding p(Dx 9) == L(L(Dx), 9) E Baxddxda
Inference.
3.2 Cond-FiP: Conditional Fixed-Poin Decoder
We now present the modification of FiP that uses the learned dataset embeddings s(Dx, 9) for amortized inference of causal mechanisms (pscudo code in Algorithm 2 Appendix A.4].
Training Setting. Analogous the encoder training setup, we assume that we have access to a distribution of SCMs S(PN, 9, F) ~ Ps at training time, from which we can extract empirical representations (Dx 9). Our goal is to train T such that given the context g) from SCM S(PN, F) ~ Ps, the induced conditional function IRd T(z, Dx,99 g) e Rd approximates the true causal mechanisms F P:a. E Rd
F(z) E Rd (E.q.)
Decoder Architecture. The design of our decoder is based on the FiP architecture for fixed-point SCM learning, with two major differences: (1) we use the dataset embeddings p(Dx. (DDx.9) 9) as high dimensional codebook to embed the nodes, and (2) we leverage adaptive layer norm operators (Peebles & Xie 2023 in the transformer blocks of FiP to enable conditional attention mechanisms.
Conditional Embedding. The key change of our decoder compared to the original FiP is in the embedding of the input. FiP proposes to embed a data point z (ZI,..., za] E IRd into a high dimensional space using a learnable codebook := = [C,...., Ca) E ddd,d, and positional embedding P == PP.....,PP PaJ E Iddxd, from which they define:
Zemb := [21 C1...., 2d Cal + dxd,
This ensures that the embedded samples preserve the original causal structure. However, this embedding layer is only adapted if the samples considered are all drawn from the same observational distribution, as the representation of the nodes given by the codebook C, is fixed. In order to generalize their embedding consider conditional codebooks and positional
strategy to the case where multiple SCMs are considered, embeddings adapted for each datasct. Given datasct Dx and causal graph G, we propose to define the conditional codebook and positional embedding as
C(Dx, 9) :- : 9WW
P(Dx,9) 9) == -
5

---

Published in Transactions on Machine Learning Research (12/2025)
where (DDx, 9) == (axxPoo(E((((X))] Rdxda is obtained by max-pooling w.r.t the sample dimension the dataset embedding F.(T(Dx), 9) E RRxxdxds produced by our trained encoder, and Wc. Wp E Rdixaa Xds are learnable parameters. Then we propose to embed any point 2 E R conditionally on the context (Dx,9) 9)
as follows:
Zemb = * C 22 Cd(Dx. ...,, + P(Dx.9) E Rddda
Adaptive Transfomer Block. Once an input z E Rd has been embedded as Zemb E Rdxdd FiP models SCMs by simulating the reconstruction of the data from noise. Starting from no E Rdxd, a learnable parameter, they propose to update the current noise L 2 1 times by computing:
ne+1 h(DAM (ng, zemb) Zemb nee)
where h refers to the MLP block, and for clarity, omit both the layer's dependence its parameters and the inclusion of layer normalization in the notation. Note that here FiP considers the DAG-Attention mechanism (dctails in Appendix A.1 in order to correctly model the root nodes of the SCM. To obtain a conaitional formulation, we first replace the starting noise ng with no == p(Dx, g) E RRd where W E Rd, Xd, is a learnable parameter. Then we add adaptive layer normalization operators (Peebles Xie 2023 to both attention and MLP blocks, where each scale or shift obtained by applying a hidden-layer MLP the embedding g)
Projection. To project back the latent reprosentation of obtained from previous stages, n. E Rddd,, use a linear operation lo get 2 nr, oont E RR,, where Won. E Rds is learnable.
Training Procedure. The result of forward pass can be summarized as = T where we omit the dependence of on context (Dx,9) for simplicity. We train the model T by minimizing the reconstruction error of the true causal mechanisms estimated by our model over the distribution of SCMs Ps, as shown below.
Eswpy Be-FF (x, 9) F22)
(4)
where Fx is chosen independent of the random dataset Dx To compute propose to sample n independent samples Xi...., X from xx, leading to new dataset Dx independent of Dx and obtain the following optimzation problem:
EsaP, IT(x -F(xx:)
Remark on ANM assumption. Though our method relics on the ANM assumption for encoder training (Appendix A we do not need this assumption for decoder training! An interesting direction for future work is to develop more general encoder training strategies, such as using self-supervised learning for dataset. encoding. Another option is to pursue end-to-end training of Cond-FiP, adopt curriculum learning by first training the encoder to predict noise variables (as "pretraining' step), and then fine-tume tue-tuning it using the decoder's reconstruction loss on more realistic SCMs. However, we consider these extensions beyond the scope of the current work.
3.3 Inference with Cond-FiP
We provide summary of inference procedure with Cond-FiP, with details in Appendix A.3
Observational Generation. Cond-FiP capable of generating new data samples: given a random vector noise n ~ IPv, we can estimate the observational sample associated according to an unknown SCM S( F) Fs as long as we have access to its empirical representation (Dx, 9). Formally, starting from ni n, we infer the associated observation by computing for É = 1,...,d:
n₂ T-n--1, 9) nn.
(5)
6

---

Published in Transactions on Machine Learning Research (12/2025)
After (at most) d iterations, na corresponds to the observational sample associated to the original noise n according to our conditional SCM T(:: 9). To sample noise from IN,, we leverage cond-FiP that can estimates noise samples under the ANM assumption by computing DN := DN= these estimated noise samples, can efficiently estimate the joint distribution of the noise by computing the inverse edfs of the marginals proposed in FiP.
Dx - T(xx, ((Dx,u(Dx) ((Dxx)))). From
Interventional Generation. Cond-FiP also enables the estimation of interventions given an empirical representation g) of an unkown SCM S(PN. F) ~ Pg. To achieve this, we start from a noise sample n, and we generate the associated intervened sample 2do by directly modifying the conditional SCM provided by Cond-FiP. More specifically, we modify in place the SCM obtained by Cond-FiP, leading to its interventional version (- Dx 9). Now, generating an intervened sample can be done by applying the loop defined in 5 starting from and using the intervened SCM (. Dx, () rather than the original one.
4 Experiments
4.1 Setup
Data Generation Process. We use the synthetic data generation procedure proposed by Lorch et al. (2022) to generate SCMs as this framework supports a wide variety of SCMs, making it well-suited for amortized training. It allows sampling of graphs from different schemes and noise variables from diverse distributions. Further, we can also control the complexity of causal mechanisms, choosing between linear (LIN) functions or random fourier features (RFF) for non-linear causal mechanisms. We construct two distribution of SCMs, FIN, and Pour, which vary based on the choice for sampling causal graphs, noisc variables, and causal relationships, see Appendix B.1 for more details.
Training Datasets. We randomly sample 4e6 SCMs from the FIN distribution, each with d = 20 total nodes. From each SCM, we extract the causal graph g and generate Attrinn = 400 observations to obtain Dx. This procedure is used to generate training data both the dataset encoder and Cond-FiP, with each epoch containing = 400 randomly generated datasets.
Test Datasets. We evaluate the model's generalization both in-distribution and -istof-distribuution by sampling test datasets from FIN and POUT, respectively. The test datasets are categorized as follows: LIN IN and RFF IN where the SCM are sampled from PIN with linear and non-linear causal mechanisms respectively. Similarly, we define LIN OUT and RFF OUT where the SCMs are sampled from POUT instead. For each category, we vary the total nodes d E [10, 20, 50, 100] and sample 6 or 9 SCMs per d, based on the available schemes for sampling the causal graphs (check Appendix B.1 for details). This results in a total of 120 test datascts, supporting a comprehensive evaluation of the methods. For each SCM we generate Titsst = 800 samples, split equally into lask context Dx and queries Dx for evaluation. An interesting aspect of our test setup is we assess the model's ability to generalize to larger graphs (d = 50, 100), despite training only with d = 20 node graphs.
Model Architecture. For both the dataset encoder and cond-FiP, we set the embedding dimension to d₁ = 256 and the hidden dimension of MLP blocks to 512. Both of our transformer-based models contains 4 attention layers and each attention consists of 8 attention heads. Please check Appendix B.3 for further details and Cond-FiP's memory and compute requirements.
, and DoWhy (Blöbaum et al., 2022). Since the baselines do not have any amortization procedure, they are trained from scratch on each test setting. For fair comparison with our method, we use the same context set Dx with 400 samples to train the basclines, which was used to obtain the dataset embeddings in Cond-FiP. All the methods are then evaluated on the remaining 400 samples in query set Dx Also, we provide the true graph G to all the baselines to ensure consistency with Cond-FiP.
Basclines. We compare Cond-FiP against FiP (Sectbon et al. 2024 DECI (Geffner et al., 2022)
To avoid potential confusion, we clarify that the notion of distribution shift is defined w.rt Cond-FiP's training setup. For the baselines, there is no distribution shift as they are trained on the context (Dx) drawn from the specific test distribution. The most important comparison is with the baseline FiP, Cond-FiP is
7

---

Published in Transactions on Machine Learning Research (12/2025)
0.5
0.25
0.4
0.20
0.15
0.3
0.3
0.10-
0.2
DoWny DECI
Cond-FiP
DoWhy DECI
FiP Cond-FiP
DoWhy DECi
Cond-FiP
(a) Noise Prediction
(b) Sample Generation
(c) Interventional Generation
Figure In-Distribution Results. Benchmarking Cond-FiP for various evaluation tasks, with datasets sampled from RFF IN with d 20. The y-axis denotes the RMSE, with mean and standard error over the respective test datasets. Results indicate Cond-FiP can generalize to novel in-distribution instances, with
detailed results in Appendix C
0.8
0.8
0.30
0.7
0.25
0.6
0.5
0.15
DoWny DECI
FiP Cond-FiP
DoWhy DÉCI
FiP Cond-FiP
DoWhy DECI
FiP Cond-FiP
(a) Noise Prediction
(b) Sample Generation
(c) Interventional Generation
Figure 3: OOD Results. Benchmarking Cond-FiP for various evaluation tasks, with datasets sampled from RFF OUT with d = 100 to test for OOD generalization. The y-axis denotes the RMSE, with mean and standard error over the respective test datasets. Results indicate Cond-FiP can generalize to novel OOD instances and larger graphs, with detailed results Appendix C
amortized counterpart. Further, we do not report detailed comparisons with CausalNF (Javaloy al., 2023) as its performance was consistently weaker than other baselines, check Appendix for details.
Evaluation Tasks. We evaluate the methods on the following three tasks. Noise Prediction: given the obscrvations Dx and the true graph G, infer the noise variables DN Sample Generation: given the noise samples DN and the true graph 9, generate the causal variables Dx Interventional Generation: generate intervened samples from noise samples DN and the true graph G.
Metric. Let us denote predicted & true target as Ý E Rie and Y E Xd Then RMSE is computed as Neset Eiiet HIll), 1 Note that we scale RMSE by dimension d, which allows to compare results across different graph sizes.
4.2 Results
Genoralization to OOD data and larger graphs. In Figure 2 we first present results for in-distribution generalization using test datasets sampled from RFF IN for graphs with = 20 nodes. Cond-FiP performs competitively with basclines trained from scratch on each test instance, hence successfully generalizes novel in-distribution instances. Notably, Cond-FiP was never explicitly trained to generate interventional data, and its strong performance this task further supports that it captures the underlying causal mechanisms.
Next we consider the more challenging case of OOD generalization using test datascts sampled from RFF OUT and graphs with = 100 nodes, while the Cond-FiP was trained only with d = 20 node graphs. As shown in Figure 3 Cond-FiP contimues to perform well, indicating successful generalization to OOD instances and significantly larger graphs! Due to space constraints, we report results for SCMs with non-lincar mechanisms the more challenging setting. Full results for both in-distribution and OOD scenarios are available in Appendix
C where findings remain consistent.
8

---

Published in Transactions on Machine Learning Research (12/2025)
0.5
Cond
0.8
0.4
0.7
0.7
0.5
50
100
400
50
100
400
50
100
400
Test Dataset Size
Test Dataset Size
Test Dataset Size
(a) Noise Prediction
(b) Sample Generation
(c) Interventional Generation
Figure 4: Scarce Data Regime Results. Benchmarking Cond-FiP on the various evaluation tasks (RFF OUT and d 100) reduce the test dataset size. The y-axis denotes the RMSE, with mean and standard error over the respective test datasets. Cond-FiP generalizes much better than the baselines in the low-data regime, with detailed results in Appendix 3
0.8
0.8
0.60
0.55
0.7
0.7
0.50
0.6
0.6
DoWny DÉCI
Cond-
DoWhy DÉCI
Cond-FiP
DoWhy Cond-FiP
(a) Noise Prediction
(b) Sample Generation
(c) Interventional Generation
Figure OOD Results without True Graph. Benchmarking Cond-FiP for various evaluation tasks, with datasets sampled from RFF OUT with 100 where the true graph is not present input context, rather its inferred via AVICI. The y-axis denotes the RMSE, with mean and standard error over the respective test datasets. Results indicate Cond-FiP can generalize novel instances even in the absence of true graph, with detailed results in Appendix
F
We also assess Cond-FiP? sensitivity to distribution shifts by varying the magnitude distribution shift (details in Appendix D). We consider two cases, where we control the severity distribution shift by controlling the causal mechanisms or the noise variables. We find that Cond-FiP is more robust to shifts in causal mechanisms, with minimal performance degradation. However, its performance is more sensitive to shifts in noise distributions, deteriorating the magnitude of shift increases.
Better Generalization in Scarce Data Regimes. An advantage of amortized inference methods their ability to generalize well when context Dx for lest instances small. As the context size decreases, baselines often suffer significant performance drops they require training from scratch. In contrast, Cond-FiP is less impacted its parameters remain unchanged at inference time, and the inductive bias learned during training enables effective generalization even with limited context. In Figure | we demonstrate this in the challenging OOD setting (RFF OUT d = 100), where Cond-FiP outperforms the baselines .Please check Appendix for further details.
Gencralization without Truc Causal Graph. So far, our results assume access the true causal graph (9) as part of the input context to Cond-FiP. However, Cond-FiP can be extended operate without this information by first inferring the graph using amortized structure learning methods (Lorch et al. 2022 Ke ct al. 2022). We demonstrate this in Figure for the RFF OUT; setting with d = 100 nodes, using graphs inferred via AVICI (oorch et al.) 2022) for both Cond-FiP and the baselines. The results show that Cond-FiP remains competitive, ivve supporting its ability to capture underlying causal mechanisms (details in Appendix
Further, to assess whether Cond-FiP genuinely learns causal mechanisms tied to the input graph structure, we perform systematic sensitivity analysis by perturbing the causal graph. Starting from the true graph, randomly remove a proportion (p) of the true edges, such that on average px (total edges) are missing.

---

Published in Transactions on Machine Learning Research (12/2025)
0.8
0.8
0.6
0.4
0.2
DoWhy Cond-FiP
DoWhy DECI
FiP Cond-FiP
DoWhy DECI
Cond-FiP
(a) Noise Prediction
(b) Sample Generation
(c) Interventional Generation
Figure CSuite Results. Benchmarking Cond-FiP on the various evaluation tasks on the CSuite benchmark, which uses different data simulator than the Cond-FiP's training data simulator. The y-axis denotes the RMSE, with mean and standard error across the test datascts.
2.0
2.0
1.25
1.00
1.5
0.75
1.0
0.50
0.5
0.5
DoWhy DECI
FIP Cond-FiP
DoWhy DÉCI
FiP
Cond-Fip
DoWhy DÉCI
FiP
Cond-FiP
(a) Noise Prediction
(b) Sample Generation
(c) Interventional Generation
Figure/ CSuite GMM Results, Benchmarking Cond-FiP on the Large Backdoor and Weak Arrow datasets from the CSuite benchmark, where the noise distribution is modified to be multi-modal gaussian mixture model. The y-axis denotes the RMSE, with mean and standard error across the 12 lest scenarios. Results indicate that Cond-FiP can generalize to instances with more complex noise distributions like GMMs.
As shown in Table (Appendix F.1
Cond-FiP remains robust under moderate graph perturbations and performs competitively with FiP, which is retrained from scratch for each setting. These results further support that Cond-FiP learns transferable causal mechanisms and can adapt at test time by inferring functions consistent with the available (and potentially imperfect) graph and observational context.
Ablation Study. Since the method relies large-scale pretraining over synthetic SCMs, we analyze how Cond-FiP's performance scales with the number of pretraining SCMs. We conduct experiments at smaller scales, with total of 1c5, 4e5, 1e6 SCMs, as opposed to using 4e6 SCMs in our main results above. Our results in Table 22 (Appendix G.1 show that Cond-FiP benefits consistently from additional pretraining data, though the returns gradually diminish as scale increases. The most pronounced gains appear when increasing the pretraining size from 1e5 to 4e5, while improvements become more incremental beyond 1e6.
Further, conduct ablation studies both the encoder (Appendix G.2) and decoder (Appendix better understand how the training data affects generalization performance. We find that Cond-FiP remains mixture of both.
to
competitive even when the encoder is trained on only RFF data, compared to training In contrast, decoder performance benefits more noticeably from training on the combined dataset.
Generalization to novel data simulators. We further evaluate Cond-FiP on test datasets generated using C-Suite Geefner et al., 2022), a synthetic data simulator distinct from the training simulator. As shown in Figure G Cond-FiP generalizes well to these novel instances. Additionally, to conduct more OOD evaluations, we modify the noise distribution of the Large Backdoor and Woak Arrow datasets from the Csuite benchmark such that the noise variables are sampled from a gaussian mixture model (GMM) (details in Appendix B.2). Results Figure demonstrate that Cond-FiP can generalize to more complex noise distributions as well. Importantly, while baselines were trained from scratch for each spocific gaussian mixture noise distribution, CiPd-FiP was pretrained only on gaussian noise and generalizes effectively settings with GMM noise distribution.
10

---

Published in Transactions on Machine Learning Research (12/2025)
Method
MMD(DR) guery, D
MMD(
( Doontext D X
(MDDDxeaae D
Dowhy
0.015
0.014
0.005
DECI
0.014
0.005
0.005
FiP 0.015
0.005
0.005
Cond-FiP 0.013
0.005
0.005
Table 1: Results for Flow Cytometry (Sachs) dataset. We benchmark Cond-FiP against the basclines for the task of generating observations data on the real world Sachs benchmark. Each cell reports the MMD, and we also report the reconstruction error for all of the methods. Results indicate that Cond-Fit FiP matches the performance of baselines trained from scratch.
Method
MM D
MMD (D Deontexs, Diuery)
MMD Dronteext D
DoWhy 0.020
0.014
0.005
DECI
0.016
0.005
0.005
Fip 0.017
0.005
0.005
Cond-FiP 0.019
0.005
0.005
Table 2: Results for Ecoli dataset. We benchmark Cond-FiP against the baselines for the task of generating obscrvational data on the real world Ecoli benchmark from the bulearn repository. Each cell reports the MMD, and we also report the reconstruction error for all of the methods. Resulls indicale that Cond-FiP matches the performance of baselines trained from scratch.
Experiments on real-world benchmarks. Finally, we show that Cond-FiP can generalize to the real- world instances using the flow cytometry dataset (Sachs et al., 2005) and ecoli dataset (Scutari) 2010). Although Cond-FiP cannot be trained on real-world datasets since the encoder requires access to true noise variables, can still be used for inference. Both datasets contains n = 800 observational samples expressed in d = 11 dimensional space for the flow cytometry datasot and d = 46 dimensional space for the ocoli datasct, and the corresponding reference (true) causal graph. We split this into context Deontext E Re xd and queries Douery E Ru xd each of size Reontext Rquery = 400. Note that the context dataset is to used to train the basclines and obtain dataset embedding for Cond-FiP, while the query datasct is used for evaluation of all the methods.
Since we don't have access to the true causal mechanisms, we cannot compute RMSE for noise prediction or sample generation like we did our experiments with synthetic benchmarks. Instead for cach method, obtain the noise predictions Doontext on the context, and use it to fit gaussian distribution for each component (node). Then use the learned gaussian distribution to sample new noise variables, D which are mapped to the observations per the causal mechanisms learned by each method, D Finally, we compute the maximum mean discrepancy (MMD) distance between Dquery and Diuery as metric to determine whether the method has captured the true causal mechanisms. For consistency, we also evaluate the reconstruction performances by directly using the inferred noise from context D from the models, and then compute MMD between their reconstructed data (Deontext) and the query data (D
Table D 2 presents our results for the flow cytometry and the ecoli dataset, where for reference we also report the MMD distance between samples from the context and query split, which should serve the gold standard since both the datascts are sampled from the same distribution. We find that Cond-FiP is competitive with the baselines that were trained from scratch. Except DoWhy, the MMD distance with reconstructed samples from the methods are close to oracle performance.
Note that Cond-FiP (as well as the other basclines) only supports hard interventions while the interventional data available for Sachs are soft interventions. Hence, we are unable provide comprehensive evaluation of Cond-FiP (and the baselines) for interventional predictions Sachs.
11

---

Published in Transactions on Machine Learning Research (12/2025)
Computational Efficiency. Like other amortized approaches, Cond-FiP has a higher training cost than the baselines, as it is trained across multiple datasets. While the cost of each forward-pass is comparable to FiP, we trained Cond-Fil over approximately 4M datasets in an amortized manner. However, Cond-FiP offers a significant advantage at inference time since requires only a single forward pass to generate predictions, whereas the baselines must be retrained from scratch for each new dataset. Thus, while Cond-FiP incurs a higher one-time training cost, its substantially faster at inference.
For instance, Cond-FiP can infer causal mechanisms for novel task in under minute, while FiP takes average 30 minutes per task. For concrete comparison, it took 30 hours to train Cond-FiP but we can solve each inference task in max 1 minute. Therefore, to compute our main results ( (Appendix C we evaluated 360 different lasks, implying a total cost of 30 + 3606600 = 36 hours with Cond-FiP. In contrast, evaluating FiP on the same 360 tasks would require retraining from scratch each time, taking approximately 180 hours!
Thus, while Cond-FiP has higher one-time training cost, it offers 5x speedup over FiP in total runtime when evaluating across multiple tasks.
5 Conclusion
In this work, propose novel methodology for training a single model for amortized inference of causal mechanisms in SCMs. Cond-FiP not only generalizes to unseen in-distribution instances, but also wide range of OOD instances, including larger graphs, unknown causal graphs, complex noise distributions, and real-world data. To the best of our knowledge, this is the first approach to demonstrate the feasibility of learning causal mochanisms in a reusable, foundational mannor-paving the way for a paradigmatic shift towards the assimilation of causal knowledge across datasets.
Limitations. Our training is limited to synthetic additive noise SCMs due to the requirement of true noise variables for learning the dataset encoder. However, the conditional FiP decoder (sce Scction 3.2 does not rely on this assumption and can be applied to general SCMs given pretrained dataset embeddings. A promising direction for future work is to explore more general encoding schemes, such self-supervised learning, or design an implicit in-context learning approach to remove the need for dataset embeddings via direct attention over the context (Mittal et al. 2024). We believe our framework can serve as a good motivation for future works that incorporate real-world datasets during training as well.
While Cond-FiP generalizes to larger graphs, does not yet benefit from larger context sizes at inference (Appendix [.p, suggesting the need to scale both the model and training data for richer contexts. Addi- tionally, although Cond-FiP performs well on generating interventional samples, it doesn't 't perform well on counterfactual generation (Appendix
I.2 Future work will explore scaling Cond-FiP to larger problem inst ances and application for more complex tasks (counterfactue ual generation) in real-world scenarios.
12

---

Published in Transactions on Machine Learning Research (12/2025)
Acknowledgements
We thank the members of the Machine Intelligence team at Microsoft Research for helpful discussions. We also thank Sarthak Mittal for their suggestion to benchmark Cond-FiP in the scarce data regime. Further, we thank Moksh Jain for their feedback on the draft. Part of the experiments were enabled by the Digital Research Alliance of Canada (attps://allaanceeaa ca/en) and Mila cluster (https:// docs. qwwee//.eeee//ideee.. httmm Divyat Mahajan acknowledgee support via FRQNT doctoral scholarship (https://doi 69777//5769777 for his graduate studies.
Broader Impact Statement
We propose novel methodology for amortized inference of causal mechanisms in structural causal models, representing an initial step toward the development of causal foundational models. Iniegrating causal principles into machine learning has been widely suggested to improve robustness and reliability, an important property for high-stakes domains such as healthcare, policy, and scientific discovery. By advancing core methodology in causal inference, our work may indirectly support the creation of machine learning systems that are more transparent and trustworthy. However, our research currently does not target any socictal application, and does not pose foreseeable risks or negative consequences.
References
Tara Akhound-Sadegh, Jarrid Rector-Brooks, Avishek Joey Bose, Sarthak Mittal, Pablo Lemos, Cheng-Hao Liu, Marcin Sendera, Siamak Ravanbakhsh, Gauthier Gidel, Yoshua Bengio, et al. Iterated denoising energy matching for sampling from boltzmann densities. arXiv preprint #xXi:2442.00122) 2024.
Ekin Akyürek, Dale Schuurmans, Jacob Andreas, Tengyu Ma, and Denny Zhou. What learning algorithm is in-context learning? investigations with linear models. arXiv preprint arXiv:2211.16611 2022.
Ekin Akyürek, Bailin Wang, Yoon Kim, and Jacob Andreas. In-context language learning: Architectures and algorithms. Xiv preprint 7Xiu:2401.12973 2024.
Marcin Andrychowiez, Misha Denil, Sergio Gomez, Matthew W Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, and Nando De Freitas. Learning learn by gradient descent by gradient descent. Advances in neural information processing systems, 29, 2016.
Vahid Balazadeh, Hamidreza Kamkari, Valentin Thomas, Benson Li, Junwei Ma, Jesse C Cresswell, and Rahul G Krishnan. Causalpfn: Amortized causal effect estimation via in-context learning. arXiv preprint arXiw:2506.07911 2025.
Albert-Lászle Barabási and Réka Albert. Emergence of scaling in random networks. science, 286(5439): 0099-112,17 1999.
Patrick Blöbaum, Peter Götz, Kailash Budhathoki, Atalanti A. Mastakouri, and Dominik Janzing. Dowhy- gem: An extension of dowhy for causal inference in graphical causal models. arxiv preprinl arXiv:2206.0683 2022.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877 1901, 2020.
Lucius EJ Bynum, Aahlad Manas Puli, Diego Merrero-Quevedee Nhi Nguyen, Carlos Fernandez-Granda, Kyunghyun Cho, and Rajesh Ranganath. Black box causal inference: Effect estimation via meta prediction. arXiv preprial ar arXic:2503.0500 2025.
David Maxwell Chickering. Optimal structure identification with greedy search. Journal of machine learningg research, 3(Nov):507-55) 554, 2002.
Max Chickering, David Heckerman, and Chris Meek. Largo-sample learning of bayesian networks np-hard. Journal of Machine Learning Research, 5:1287-13300 2004.
13

---

Published in Transactions on Machine Learning Research (12/2025)
Anish Dhir, Matthew Ashman, James Requeima, and Mark van der Wilk. A meta-learning approach to bayesian causal discovery. arXie preprint arXiv:24 12. 16577, 2024.
Anish Dhir, Cristiana Diaconu, Valentinian Mihai Lungu, James Requeima, Richard E Turner, and Mark van der Wilk. Estimating interventional distributions with uncertain causal graphs through meta-learning. arviv preprinl arXi::2507.00 05526,, 2025.
P Erdos and A Renyi. On random graphs i. Publ. math. debrecen, 6(290-297):18, 1959.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning, pp. 1126 1136-115. PMLR, 2017.
Jared C Foster, Jeremy MG Taylor, and Stephen J Ruberg. Subgroup identification from randomized clinical trial data. Statistics in medicine, 30(24):2867 2880, 2011.
Shivam Garg, Dimitris Tsiprss, Percy Liang, and Gregory Valiant. What can transformers learn in-context? a case study of simple function classes. Advances in Neural Information Processing Systems, 35:30583 33598, 2022.
Marta Garnelo, Dan Rosenbaum, Christopher Maddison, Tiago Ramalho, David Saxton, Murray Shanahan, Yee Whye Tch, Danilo Rezende, and SM Ali Eslami. Conditional neural processes. In International conference on machine learning, pp. 1704-1713. PMLR, 2018.
Tomas Geffner, Javier Antoran, Adam Foster, Wenbo Gong, Chao Ma, Emre Kiciman, Amit Sharma, Angus Lamb, Martin Kukla, Nick Pawlowski, et al. Deep end-to-end causal inference. arXiv preprint or arXiv:2202.021 2022.
Jonathan Gordon, John Bronskill, Matthias Bauer, Sebastian Nowozin, and Richard Turner. Meta-learning probabilistic inference for prediction. In International Conference Learning Representations, 2019. URL
Shantanu Gupta, Cheng Zhang, and Agrin Hilmkil. Learned causal method prediction. arxiv preprint arXi::2311.03989, 2023.
Paul W Holland, Kathryn Blackmond Laskey, and Samuel Leinhardt. Stochastic blockmodels: First steps. Social networks, 5(2):109 137, 1983.
Noah Hollmann, Samuel Müller, Katharina Eggensperger, and Frank Hutter. TabPFN: A transformer that solves small labular classification problems in second. In NewrIPS 2022 First Table Representation Workshop, 2022. URL http:///w.ggggeeeeeeeeeeee/eeeea
Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. Meta-learning in neural networks: A survey. IEEE transactions on pattern analysis and machine intelligence, 44(9):5149 5169, 2021.
Adrián Javaloy, Pablo Sanchez-Martin, and Isabel Valora. Cansal normalizing flows: from theory to practice. Advances in Neural Information Processing Systems, volume 36, 2023.
Tero Karras, Mika Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. Analyzing and improving the training dynamics of diffusion models. ArXiu, abs/2312.026666, 2023. URL https:
//api semanticschola orgg/orrpusDD:26555533 265659032
Nan Rosemary Ke, Silvia Chiappa, Jane Wang, Anirudh Goyal, Jorg Bornschein, Melanic Rey, Theophane Weber, Matthew Botvinic, Michael Mozer, and Danilo Jimenez Rezende. Learning to induce causal structure. arXio preprint vxXi:2224.04877 2022.
Nan Rosemary Ke, Sara-Janc Dunn, Jorg Bornschein, Silvia Chiappa, Mclanic Rey, Jean-Baptiste Lespiau, Albin Cassirer, Jane Wang, Theophane Weber, David Barrett, Matthew Botvinick, Anirudh Goyal, Mike Mozer, and Danilo Rezende. Discogen: Learning to discover gene regulatory networks, 2023.
14

---

Published in Transactions on Machine Learning Research (12/2025)
llees Khemakhem, Ricardo Monti, Robert Leech, and Aapo Hyvarinen. Causal autoregressive flows. In International Conference on Arlificial Intelligence and Stalistics, pp. 3520-330000 35200-358...0 PMT.R, 2021.
Jang-Hyun Kim, Claudia Skok Gibbs, Sangdoo Yun, Hyun Oh Song, and Kyunghyun Cho. Large-scale targeted cause discovery via learning from simulated data. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. URL httts:// http://////pppeereeeeee/e///ee/a
Lars Lorch, Jonas Rothfuss, Bernhard Schölkopf, and Andreas Krause. Dibs: Differentiable bayesian structure learning. Advances in Neural Information Processing Systems, 34:24111-24123, 2021.
Lars Lorch, Scott Sussex, Jonas Rothfuss, Andreas Krause, and Bernhard Schölkopf. Amortized inference for causal structure learning. Advances in Neural Information Processing Systems, 35:13104 13118, 2022.
Yuchen Ma, Dennis Frauen, Emil Javurek, and Stefan Feuerriegel. Foundation models for causal inference via prior-data fitted networks. arviv preprint vrXiv:2506.1091, 2025.
Sarthak Mittal, Eric Elmoznino, Leo Gagnon, Sangnie Bhardwaj, Dhanya Sridhar, and Guillaume Lajoie. Does learning the right latent variables necessarily improve in-context learning? arXiv preprint arXiv:2405.19162, 2024.
Samuel Müller, Noah Hollmann, Sebastian Pineda Arango, Josif Grabocka, and Frank Hutter. Transformers can do bayesian inference. arXiv preprint orXiv:21 12. 1001000, 2021.
Hamed Nilforoshan, Michael Moor, Yusuf Roohani, Yining Chen, Anja Surina, Michihiro Yasunaga, Sara Oblak, and Jure Leskovec. Zero-shot causal learning. Advances in Neural Information Processing Systems, 36:6862-6901, 2023.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
William Peebles and Saining Xic. Scalable diffusion models with transformers. In Proccedings of the IEEE/CVF International Conference Compuler Vision, pp. 4195 4205, 2023.
Jonas Peters, Joris M Mooij, Dominik Janzing, and Bernhard Schölkopf. Causal discovery with continuous additive noise models. Journal of Machine Learning Research, 2014.
Jake Robertson, Noah Hollmann, Noor Awad, and Frank Hutter. Fairpfn: Transformers can do counterfactual fairness. Xix preprint orXiv:2407.05732 2024.
Jake Robertson, Arik Reuter, Siyuan Guo, Noah Hollmann, Frank Hutter, and Bernhard Schölkopf. Do-pfi: In-context learning for causal effect eslimation. arXiv prepriul V1v:25060060334 2025.
Karen Sachs, Omar Perez, Dana Pe'er, Douglas A Lauffenburger, and Garry P Nolan. Causal protein-signaling networks derived from multiparameter single-cell data. Science, 308(5721) ):523 -299, 2005.
Andreas Sauter, Saber Salehkaleybar, Aske Plaat, and Erman Acar. Activa: Amortized causal effect estimation without graphs via transformer- based variational autoencoder. orXiv preprint arXiv:2503. 01290, 2025.
Meyer Scetbon, Joel Jennings, Agrin Hilmkil, Cheng Zhang, and Chao Ma. Fip: fixed-point approach for causal generative modeling, 2024.
Marco Scutari. Learning bayesian networks with the bulearn package. Journal of statistical sofbware, 35: 1-22, 2010.
Marcin Sendera, Minsu Kim, Sarthak Mittal, Pablo Lemos, Luca Scimeca, Jarrid Rector-Brooks, Alexandre Adam, Yoshua Bengio, and Nikolay Malkin. Improved off-policy training of diffusion samplers. Advances in Neural Information Processing Systems, 37:81066-81055, 2024.
Ashish Vaswani, Noam Shazoer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
15

---

Published in Transactions on Machine Learning Research (12/2025)
Johannes Von Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento, Alexander Mordvintsev, Andrey Zhmoginov, and Max Vladymyrov. Transformers learn in-context by gradient descent. In International Conference on Machine Learning, pp. 35151-330 -5114. PMLR, 2023.
Duncan J Watts and Steven H Strogatz. Collective dynamics of 'small-world networrs..... nature, 393(6684): 440-442, 1998.
Menghua Wu, Yujia Bao, Regina Barzilay, and Tommi Jaakkola. Sample, estimate, aggregate: recipe for causal discovery foundation models. arXiv preprint arXiv:2402.01929 2024.
Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. An explanation of in-context learning as implicit bayesian inference. arXiv preprinl arxi::21 11.111..000 11.00000, 2021.
Yu Xie, Jennie E Brand, and Ben Jann. Estimating heterogeneous treatment effects with observational data. Sociological methodology, 424333314-347 2012.
Jiaqi Zhang, Joel Jennings, Cheng Zhang, and Chao Ma. Towards causal loundation model: on duality between causal inference and attention. arXiv preprint arXio:2310.0000 008009 2023.
Jiaqi Zhang, Kristjan Greenewald, Chandler Squires, Akash Srivastava, Karthikeyan Shanmugam, and Caroline Uhler. Identifiability guarantees for causal disentanglement from soft interventions. Advances in Neural Information Processing Systems, 36, 2024.
Xun Zheng, Bryon Aragam, Pradeep K Ravikumar, and Eric P Xing. Dags with no tears: Contin- uous optimization for structure learning. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. BiancBi and R. Garnett (eds.), Advances in Neural Information Processing Systems, vol- ume 31. Curran Associates, Inc., 2018. URL https://prooceeding nwwrips..c/peeeeee searrppp.ccpapprr/008ff...... 637755449FFF23333566602222222222 pdf.
16

---

Published in Transactions on Machine Learning Research (12/2025)
Appendix
Table of Contents
A Additional Details on Cond-FiP
18
A.1 DAG-Antention Mechanism
18
A.2 Details on Encoder Training
18
Inference with FiP
19
A.4
Pseduo Code
20
Details on Experiment Setup
B
AVICI Benchmark
................................
22
B.2
CSuile Benchmark
23
B.3 Model Architecture and Training Details
24
Complete Results Cond-FiP on AVICI Benchmark
D
Experiments Sensitivity to Distribution Shifts AVICI benchmark
26
Experiment on Generalization in Scarce Data Regime on AVICI benchmark E
30
Experiments with Wp..
100
30
E.2 Experiments with n...
50
32
Experiments without True Causal Graph on AVICI Benchmark
34
F 11 Analyzing Cond-FiP's Sensitivity to Causal Graph Structure
36
Ablation Study on AVICI benchmark
37
G.1
Analyzing the Effect of Pretraining Scale
37
C Ablation Study of Encoder
37
G.3 Ablation Study of Decoder
40
H
Comparing Cond-FiP with NF
42
Limitations of Cond-FiP
43
Evaluating Generalization of Cond-Fip to Larger Sample Size
Counterfactual Generation with Cond-FiP
45
17

---

A Additional Details on Cond-FiP
A.1 DAG-Attention Mechanism
In FiP Scetbon et al. 2024) the authors propose to leverage the transformer architecture to learn SCMs from observations. By reparameterizing an SCM according to a topological ordering induced by its graph, the authors show that any SCM can be reformulated as fixed-point problem of the form X = H N) where H admits a simple triangular structure:
[Jac (ar, m)las 0, if j2i [Jac,, H (xx n)lss 0, if i i
where Jac, H, Jacn H denote the Jacobian of H w.r.t the first and second variables respectively. Motivated by this fixed-point reformulation, FiP considers a transformer- based architecture to model the functional relationships of SCMs and propose a new attention mechanism to represent DAGs in differentiable manner. Recall that the standard attention matrix is defined as:
expl(QKT - MM) (m (Q, K)= eXp([QKT - MM//vd/) 1a
where Q, K E Rdxda denote the keys and queries for a single attention head, and M E {0. +00 is a (potential) mask. When M chosen to be a triangular mask, the attention mechanism 1 enables to parameterize the effects of previous nodes on the current one However, the normalization inherent to the softmax operator in standard attention mechanisms prevents effective modeling of root nodes, which should not be influenced by any other node in the graph. To alleviate this issuc, FiP proposes to consider the following formulation instead:
exp((QKT M)/ DAM(Q, V( exp((QKT M)/ /d.) 1a)
where (i = 12 u, 1, else V.(v) 1 for any E Rd. While softmax forces the coefficients along cach row of the attention matrix to sum to one, the attention mechanism described in 7 allows the rows to sum in [0, 1], thus enabling to model root nodes in attention.
A.2 Details on Encoder Training
Additive Noise Model Assumption. Our method relies on the ANM assumption only for the training the encoder. This is because we require the encoder to predict the noise from data in order to obtain emboddings, and under the ANM assumption, the mapping from data to noise can be casily expressed as x -- x F(x) where the generative functional mechanism of the generative ANM. However, we were to consider general SCMs, i.e. of the form X FV,, we would need access to the mapping - F- (x,)) (assuming this function is invertible), which for general functions is not tractable. Also, note that the ANM assumption by default ensures invertibility since the jacobian w.r.t noise is triangular matrix with nonzero diagonal elements. An interesting future work would be to consider more general dataset encoding (using self-supervised techniques) that do not require the ANM assupttion, but we believe this is out of the scope of this work.
We now provide further details on training the encoder and show how recovering the noise equivalent learn the inverse causal generative process. Recall that an SCM an implicit generative model that, given a noise sample N, generates the corresponding observation according to the following fixed-point equation in X
X F(X.N)
More preciscly, generate the associated observation, one must solve the above fixed-point equation in X given the noise N. Let us now introduce the following notation that will be instrumental for the subsequent discussion: we denote Fx(2): N).

---

Published in Transactions on Machine Learning Research (12/2025)
Due to the specific structure of F (determined by the DAG G associated with the SCM), the fixed-point equation mentioned above can be efficiently solved by iteratively applying the function FN to the noise (see Eq. (5) in the manuscript). As direct consequence, the observation X can be expressed as a function of the noise:
F (N)
where Fgen (N) (FiN)od(N) is the number of nodes, and denotes the composition operation. In the following refer to Fgen as the explicit generative model induced by the SCM.
Conversely, assuming that the mapping- 2 + F Fgen(2) (2) is invertible, then one can express the noise as a function of the data:
gen
Therefore, learning to recover the noise from observation is equivalent learn the function Fgen which is exactly the inverse of the explicit generative model Fgen- It is also worth noting that under the ANM assumption (i.e. F(X,N) f(X) + N), Fgen is in fact always invertible and its inverse admits simple expression which is
Fgen(2 (2) = =2- f(2)
Therefore, in this specific case, learning the inverse generative model Fge is exactly equivalent to learning the causal mechanism function f.
A.3 Inference with Cond-Fip
Sample Generation. Given a dataset Dx and its causal graph 9, we denote & - T(z, Dx,9) the function infered by Cond-FiP. This function defines the predicted SCM obtained by our model, and we can directly use it to generate new points. More precisely, given noise sample n, we can generate the associated observational sample by solving the following equation in
T(x, Dx 9)
To solve this fixed-point equation, rely on the fact that DAG, which enables solve the fixed-point problem using the following simple iterative procedure. Starting with 2y n, compute for & = 1.....d d where d is the number of nodes
= 9x +n
After d iterations we obtain the following,
zd = T(za, Dx,9x +
Therefore, za is the solution of the fixed-point problem above, which corresponds to the obsorvational sample associated to n according to our predicted SCM - T(2, ().
Intorventional Prediction. Recall that given a dataset Dx and its causal graph 9, 2 E Ed T(z, Dx,9) Rd denotes the SCM infered by Cond-FiP. Let also denote the coordinate-wise for- mulation of our SCM defined for any z E IRd T(z, Dx 9) [xx ())...,,,,,,ITT Dx M)aa,, where for all i E (I....,d). .d, Rd - [2 Dx, G)]; E R is real-valued function.
In order to intervene on this predicted SCM, we simply have to modify in place the predicted function. For example, assume that we want to perform the following intervention do(X;) = a. Then, to obtain the intervened SCM, we define a new function = + Tdo(X.)=a (z, Dx.9) defined for any z Rd as: [7do(X))==(z, (x, := Dx, G)ly if j # and [odo(X;) ())o(x))=00) Dx 9]], := a.
Now, using this intervened SCM 2 dd) Dx, 9), apply the exact same generation procedure the one introduced above to generate intervened samples according to our intervened SCM.
19

---

Published in Transactions on Machine Learning Research (12/2025)
A.4 Pseduo Code
Algorithm Cond-FiP Part 1: Dataset Encoder p(Dx.9) g)
Input:
Observational dataset Dx E IRadd causal graph g E {0, 13
Output: Dataset embedding ,()x,9)
1: Linear Embedding: Apply a learned lincar map
Rd to each sample:
L(Dx) I,, X, W E R Raxdxd,
2: Alternating Attention: Pass L(Dx) through a stack of transformer blocks that alternate between:
Sample-wise attention (unmasked) across the n observations, capturing global context.
Node-wise attention with mask defined as the causal graph.
0,
if GG==
Mij 100, otherwise.
3: Contextual Representation: After alternating attention blocks, obtain dataset embeddings
m(Dx 9) = E(L(Dx). E R Baxdxda
4: Prediction Head: Apply a two-layer MLP H to map embeddings to estimated noise function values:
Ý H (p(((((()))))))) 9)) E Rnxd
Training Objective: Under the additive noise model assumption, minimize the MSE loss:
LE = Es-Pa [F(Dxx 813
Output: Return the dataset embedding p(Dx,9) 9)
20

---

Published in Transactions on Machine Learning Research (12/2025)
Algorithm Cond-FiP Part 2: Conditional Fixed-Point Decoder Dx, 9)
Input: Input point 2 E Rd, dataset Dx E Raxd, causal graph g {0, 1 1]dxd encoder output p(Dx.9
Output: Learned causal model: T(2, Dx. g)
1: Pooling:
Aggrogate the encoder representations across samples:
(Dxx 9) (BxP)00, E Rddd,
Context-Specific Codebooks: Compute dataset-conditionee codebooks and positional embeddings:
9) = =
3: Input Embedding:
Embed the input via element-wise modulation:
..... (
Zemb
Initialization: Set initial latent state (which will be transformed into the estimated causal variable)
using the dataset embedding:
= 5: Fixed-Point Iterations: For 8= ..... L - 1, apply DAG-Atttntion and adaptive normalization:
n++1 h(DAM(n,, Zemb) Zemb + nn2).
where fi(-) includes adaptive LayerNorm whose scale and shift depend on x',,,9)) 9).
6: Output Projection: After L iterations, project the final state:
Dx, = n2, Wont E Rdd
7: Training Objective: Minimize reconstruction loss over random samples 2 ~ Px:
Lr = Es =w [ T(z, 9) - F(z)
21

---

Published in Transactions on Machine Learning Research (12/2025)
B Details on Experiment Setup
B.1 AVICI Benchmark
We use the synthetic data generation procedure proposed by Lorch et al. 2022 to generate SCMs in our empirical study. It provides access to a wide variely of SCMs, hence making it an excellent setting for amortized training.
Graphs: We have the option to sample graphs as per the following schemes: Grods-Reny' (Erdos & Renyi 1959), scale-free models (Barabási & Albert 1999), Watts-Strogatz (Watts & Strogatz 1998).
and stochastic block models (Holland et al. 1983).
Noise Variables: To sample noise variables, distribution where variances are sampled randomly.
choose from either the gaussian laplace
Functional Mechanisms: We can control the complexity of causal relationships: either we set them to be linear (LIN) functions randomly sampled, or use random fourier features (RFF) for generating random non-lincar causal relationships.
We construct two distribution of SCMs PIN, and POUT, which vary based on the choice for sampling causal graphs, noise variables, and causal relationships. The classification aids in understanding the creation of train and test datasets.
In-Distribution (FIN): We sample causal graphs using the Erods- Erods-Renyi and scale-free models schemes. Noise variables are sampled from the gaussian distribution, and we allow for both LIN and RFF causal relationships.
Out-of-Distribution (PoUT): Causal graphs are drawn from Watts-Strogatz and stochastic block models schemes. Noise variables follow the laplace distribution, and both the LIN and RFF cases are used to sample functions. However, the parameters of these distributions are sampled from a different range compared to FIN to create distribution shift.
We provide further details on the shift in the support of parameters for functional mochanisms below. For complete details please refer to Table 3, Appendix in Lorch et al. 2022
Linear Functional Mechanism.
In- Dn-Distribution (PIN)
Weights: ~ (1,3)) Bias U(-333).
Soatrf--Dittiibution (POUT)
Weights: (0.5, 2) (2,4), Bias ~ U(-3,3).
RFF Functional Mechanism.
In- Dn-Diitribution (PN Length Scale: U7, 10), Output Scale: U(5,8)UUU( 12), Bias ~ U+(-3,3).
(PoTT):
Length Scale: U10, 20), Output Scale: UU) U ( 22), Bias U+ (=(33,3).
22

---

Published in Transactions on Machine Learning Research (12/2025)
Test Datasets.
SCMs sampled from PIN with linear causal mechanisms. We have different options for sampling graphs in this case, and we randomly sample 3 different SCMs for each scenario, leading total of 9 instances.
LIN IN:
RFF IN:
SCMs sampled from FIN with non-lineat causal mechanisms. We have 3 different options for sampling graphs in this case, and we randomly sample 3 different SCMs for each scenario, leading to a total of 9 instances.
LIN OUT: SCMs sampled from POUT with linear causal mechanisms. We have 2 different options for sampling graphs in this case, and we randomly sample 3 different SCMs for each scenario, leading to a total of instances.
SCMs sampled from Pour with non-linear causal mechanisms. We have 2 different options for sampling graphs in this case, and we randomly sample 3 different SCMs for each scenario, leading to a total of instances.
RFF OUT:
B.2 CSuite Benchmark
CSuite /Geffner et al. 2022 is collection of synthetic structural causal models (SCMs) designed to evaluate causal discovery and effect estimation methods. The benchmark covers a diverse set of settings by varying graph structurcs, functional forms, and noise distributions, thereby testing models under both linear and nonlinear, Gaussian and non-Gaussian conditions, etc. We use the lingauss, linexp, nonlinguuss, nonlin-simpson, symprod-simpson, large-backdoor, and weak-arrou tasks from their paper.
To conduct more OOD evaluations, modify the noise distribution of the Large Backdoor and Weak Arrow dalasets from the Csuite benchmark such that the noise variables are sampled from guassian mixture model (GMM). We considered the following cases for the GMM noise distribution.
Noise is sampled with equal probability from either NV-22,1) and N2, 1).
Noise is sampled with equal probability from either N(-2,2) and N( 2).
Noise is sampled with equal probability from either 2N and [2 2).
Noise is sampled with equal probability from either N(-5, and [(, 1).
Noise is sampled with equal probability from either (N-5,2) and N55, 2).
Noise is sampled with equal probability from either N(-5,, 1) and N(5, 2).
This leads to a total of 12 experimental setting with 6 different GMM noise distribution for both the Large Backdoor and Weak Arrow datasets from the CSuite benchmark.
23

---

Published in Transactions on Machine Learning Research (12/2025)
B.3 Model Architecture and Training Details
For both the dataset encoder and cond-FiP, we set the embedding dimension to da - 256 and the hidden dimension of MLP blocks to 512. Both of our transformer- based models contains 4 attention layers and each attention consists of attention heads. The models were trained for a total of 10k epochs with the Adam optimizer (Paszke et al.) 2017), where we used a learning rate 1e - and a weight decay of 5e - 9. Each epoch contains = 400 randomly generated datasets from the distribution PIN. We also use the FMA implementation of (Karras al. 2023) to train our models.
Memory Requirements. We trained Cond-FiP single T.40 GPU with 48GB of memory, using effective batch size of 8 with gradient accumulation. We outline the detailed memory computation follows:
Each batch consists of 400 samples with dimension d = 20 requiring less than MiB of data FP32 precision. Also, storing the model on the GPU requires under 100 MiB.
Our transformer architecture has 4 attention layers, a 256-dimensional embedding space, and a 512-dimensional feedforward network. Using a standard (non-flash) attention implementation, a forward pass consumes approximately 30 GiB of GPU memory.
Compared to the baselines, Cond-FiP has similar memory requirements to DECI (Geffner et al. 2022) and FiP (Scetbon et al.j 2024). as all three train neural networks of comparable sizc. The main exception is DoWhy (Blöbaum et al. 2022), which fits simpler models for each node, but this approach does not scale well as the graph size increases.
C Complete Results for Cond-FiP on AVICI Benchmark
Method
Total Nodes
LIN IN
RFF IN
LIN OUT
RFF OUT
DoWhy
10
0.03 (0.0)
0.13 (0.02)
0.04 (0.01)
0.11 (0.01)
DECI
10
0.09 (0.01)
0.23 (0.03)
0.12 (0.01)
0.23 (0.03)
FiP
10
0.04 (0.0)
0.09 (0.01)
0.06 (0.01)
0.08 (0.01)
Cond-FiP
10
0.06 (0.01)
0.10 (0.01)
0.07 (0.01)
0.10 (0.01)
DoWhy
20
0.03 (0.01)
0.15 (0.02)
0.03 (0.0)
0.23 (0.01)
DECI
0.10 (0.02)
0.21 (0.03)
0.08 (0.02)
0.23 (0.02)
Fip
0.04 (0.0)
0.12 (0.02)
0.05 (0.0)
0.15 (0.02)
Cond-FiP
20
0.06 (0.01)
0.09 (0.01)
0.07 (0.0)
0.12 (0.0)
DoWhy
50
0.03 (0.0)
0.18 (0.03)
0.03 (0.0)
0.29 (0.03)
DECI
50
0.09 (0.01)
0.24 (0.02)
0.07 (0.01)
0.29 (0.02)
Fip
0.04 (0.0)
0.14 (0.03)
0.04 (0.0)
0.23 (0.04)
Cond-FiP
50
0.06 (0.01)
0.10 (0.01)
0.07 (0.01)
0.14 (0.01)
DoWhy
100
0.03 (0.0)
0.20 (0.03)
0.03 (0.0)
0.31 (0.02)
DECI
100
0.08 (0.02)
0.26 (0.03)
0.07 (0.01)
0.30 (0.02)
FiP
100
0.04 (0.0)
0.16 (0.03)
0.04 (0.0)
0.24 (0.02)
Cond-FiP
100
0.05 (0.0)
0.10 (0.01)
0.07 (0.01)
0.16 (0.01)
Table 3: Results for Noise Prediction. We compare Cond-FiP against the baselines for the task of predicting noise variables from the input observations. Each cell reports the mean (standard error) RMSE over the multiple test datasets for each scenario. Shaded rows denote the case where the graph size is larger than the train graph sizes (d = 20) for Cond-FiP. Results show that Cond-Fit FiP generalizes to both in-distribuutinn and OOD inslances.
24

---

Published in Transactions on Machine Learning Research (12/2025)
Method
Total Nodes
LIN IN
RFF IN
LIN OUT
RFF OUT
DoWhy
10
0.05 (0.0)
0.18 (0.03)
0.06 (0.01)
0.12 (0.02)
DECI
10
0.15 (0.02)
0.33 (0.04)
0.16 (0.02)
0.27 (0.03)
FiP
10
0.07 (0.0)
0.13 (0.02)
0.08 (0.01)
0.11 (0.02)
Cond-FiP
10
0.06 (0.01)
0.14 (0.02)
0.05 (0.01)
0.08 (0.01)
DoWhy
20
0.06 (0.01)
0.27 (0.05)
0.05 (0.0)
0.39 (0.04)
DECI
20
0.16 (0.02)
0.39 (0.05)
0.13 (0.02)
0.44 (0.04)
FiP
20
0.08 (0.01)
0.23 (0.05)
0.08 (0.01)
0.27 (0.04)
Cond-FiP
20
0.05 (0.01)
0.24 (0.06)
0.07 (0.01)
0.30 (0.08)
DoWhy
50
0.08 (0.01)
0.35 (0.09)
0.06 (0.01)
0.54 (0.06)
DECI
50
0,13 (0.01)
0.46 (0.06)
0.13 (0.02)
0.67 (0.06)
FiP
0.09 (0.01)
0.26 (0.05)
0.08 (0.01)
0.48 (0.06)
Cond-FiP
50
0.08 (0.01)
0.25 (0.05)
0.07 (0.0)
0.48 (0.07)
DoWhy
100
0.06 (0.0)
0.33 (0.07)
0.06 (0.01)
0.63 (0.07)
DECI
100
0.14 (0.02)
0.50 (0.09)
0.14 (0.02)
0.71 (0.08)
FiP
100
0.08 (0.01)
0.3 (0.06)
0.09 (0.01)
0.55 (0.08)
Cond-FiP
100
0.07 (0.01)
0.29 (0.07)
0.09 (0.01)
0.57 (0.07)
Table 4: Results for Sample Generation. We compare Cond-FiP against the baselines for the task of generating samples from the input noise variables. Each cell reports the mean (standard error) RMSE over the multiple test datasets for each scenario. Shaded rows denote the case where the graph size is larger than the train graph sizes (d = 20) for Cond-FiP. Results show that Cond-FiP generalizes to both in-distributton and OOD inslances.
Method
Total Nodes
LIN IN
RFF IN
LIN OUT
RFF OUT
DoWhy
10
0.08 (0.08)
0.19 (0.04)
0.05 (0.01)
0.12 (0.02)
DECI
10
0.17 (0.02)
0.34 (0.04)
0.13 (0.02)
0.25 (0.03)
FiP
10
0.08 (0.01)
0.15 (0.02)
0.07 (0.01)
0.09 (0.01)
Cond-FiP
10
0.10 (0.03)
0.21 (0.03)
0.07 (0.01)
0.11 (0.01)
DoWhy
20
0.06 (0.01)
0.27 (0.06)
0.05 (0.0)
0.36 (0.03)
DECI
0.16 (0.02)
0.38 (0.05)
0.15 (0.04)
0.42 (0.03)
FiP
20
0.09 (0.01)
0.23 (0.05)
0.12 (0.04)
0.25 (0.03)
Cond-FiP
20
0.09 (0.01)
0.24 (0.05)
0.14 (0.03)
0.31 (0.03)
DoWhy
50
0.08 (0.01)
0.29 (0.05)
0.06 (0.01)
0.53 (0.06)
DECI
50
0.17 (0.02)
0.44 (0.06)
0.13 (0.02)
0.64 (0.06)
FiP
50
0.11 (0.02)
0.25 (0.05)
0.09 (0.01)
0.46 (0.06)
Cond-FiP
50
0.13 (0.02)
0.27 (0.04)
0.12 (0.02)
0.48 (0.07)
DoWhy
100
0.05 (0.0)
0.33 (0.07)
0.06 (0.01)
0.60 (0.07)
DECI
100
0.14 (0.02)
0.49 (0.08)
0.15 (0.02)
0.70 (0.08)
Fip
100
0.08 (0.01)
0.29 (0.07)
0.10 (0.01)
0.54 (0.08)
Cond-FiP
100 0.10 (0.01)
0.30 (0.06)
0.14 (0.02)
0.56 (0.07)
Table 5: Results for Interventional Generation. We compare Cond-FiP against. the baselines for the task of generating interventional data from the input noise variables. Each cell reports the mean (standard error) RMSE over the multiple test datasets for each scenario. Shaded rows denote the case where the graph size is larger than the train graph sizes (d= = 20) for Cond-FiP. Results show that Cond FiP generalizes to both in-distribution and OOD instances.
25

---

Published in Transactions on Machine Learning Research (12/2025)
D Experiments on Sensitivity to Distribution Shifts on AVICI benchmark
In Appendix C (Table Table 4 Table 50, we tested OOD genrcalization with datascts sampled from SCM following a different distribution (LIN OUT, RFF OUT) than the datasets used for training Cond-FiP (LIN IN, RFF IN). We now analyze how sensitive is Cond-FiP to distribution shifts by comparing its performance acTOSs scenarios as the severity of the distribution shift increased.
To illustrate how we control the magnitude of distribution shift, we discuss the difference in the distribution of causal mechanisms across PIN and PoTT- The distribution shift arises because the support of the parameters of causal mechanisms changes from IP to POUT. For example, for linear causal mechanism case, the weights in PIN are sampled uniformly from (-3, -1) U (1, while in Pour they are sampled from uniformly from (0.5, 4). We now change the support set of the parameters in PoUT to (0.5a, 4a), so that by increasing a we make the distribution shift more severe. We follow this procedure for the support set of all the parameters associated with functional mechanisms and generate distributions (Fout(a)) with varying shift w.r.t PIN by changing a. Note that a corresponds to the same FOUT as the one used for sampling datasets in our main results.
We conduct two experiments for evaluating the robustness of Cond-FiP to distribution shifts, described ahead.
Controlling Shift in Causal Mechanisms. We start with the parameter configuration of PoUT from the setup in main results; and then control the magnitude of shift by changing the support set. of parameters of causal mechanisms.
Controlling Shift in Noise Variables. We start with the parameter configuration of FoUT from the sctup in main results; and then control the magnitude of shift by changing the support set of parameters of noise distribution.
Tables and S provide results for the case of controlling shift via causal mechanisms, for the task of noise prediction, sample generation, and interventional generation respectively. We find that the performance of Cond-FiP does not change much as we incrcase a, indicating that Cond-FiP robust to the varying levels of distribution shits in causal mechanisms.
However, for the case of controlling shift via noise variables (Table g quite sensitive to the varying levels of distribution shift in noise variables. The performance of Cond-FiP degrades with increasing magnitude of the shift (a) for all the lasks.
10 and 11 find that Cond-FiP is
26

---

Published in Transactions on Machine Learning Research (12/2025)
Total Nodes
Shift Level (a)
LIN OUT
RFF OUT
10
0.07 (0.01)
0.10 (0.01)
10
0.06 (0.01)
0.10 (0.01)
10
0.05 (0.01)
0.10 (0.01)
10
10
0.05 (0.01)
0.10 (0.01)
20
0.07 (0.0)
0.12 (0.0)
0.06 (0.0)
0.13 (0.01)
20
0.05 (0.0)
0.11 (0.01)
20
10
0.05 (0.0)
0.10 (0.01)
50
0.07 (0.01)
0.14 (0.01)
50
0.05 (0.01)
0.17 (0.01)
50
0.05 (0.01)
0.14 (0.01)
50
10
0.04 (0.0)
0.14 (0.01)
100
0.07 (0.01)
0.16 (0.01)
100
0.05 (0.01)
0.18 (0.0)
100
0.05 (0.0)
0.17 (0.01)
100
10
0.05 (0.0)
0.16 (0.01)
Table 6: Results for Noise Prediction under Distribution Shifts in Causal Mechanisms. We evaluate the robustness of Cond-FiP to distribution shifts in the parametrization of causal mechanisms, We vary the distribution shift controlled by a, where = 1 corresponds to the results in Table 3 Each cell reports the mean (standard error) RMSE over the multiple test datasets for each scenario. We find that Cond-Fil is robust to varying levels of distribulion shift in causal mechanisms.
Total Nodes
Shift Level (a)
LIN OUT
RFF OUT
10
0.05 (0.01)
0.08 (0.01)
10
0.05 (0.0)
0.07 (0.01)
10
0.05 (0.0)
0.07 (0.01)
10
10
0.06 (0.0)
0.06 (0.01)
20
0.07 (0.01)
0.30 (0.03)
20
0.06 (0.01)
0.34 (0.05)
20
0.06 (0.01)
0.35 (0.05)
20
10
0.06 (0.01)
0.29 (0.07)
50
0.07 (0.0)
0.48 (0.07)
50
0.07 (0.0)
0.47 (0.07)
50
0.07 (0.01)
0.38 (0.06)
50
10
0.07 (0.01)
0.32 (0.06)
100
0.09 (0.01)
0.57 (0.07)
100
0.09 (0.01)
0.60 (0.05)
100
0.09 (0.01)
0.58 (0.05)
100
10
0.12 (0.02)
0.56 (0.06)
Table 7: Results for Sample Generation under Distribution Shifts in Causal Mechanisms. We evaluate the robustness of Cond-FiP to distribution shifts in the parametrization of causal mechanisms. We vary the distribution shift controlled by a, where = 1 corresponds to the results in Table 4 Each cell reports the mean (standard error) RMSE over the multiple test datasets for each scenario. We find that Cond-Fil FiP is robust to varying levels of distribution shift in causal mechanisms.
27

---

Published in Transactions on Machine Learning Research (12/2025)
Total Nodes
Shift Level (a)
LIN OUT
RFF OUT
10
0.07 (0.01)
0.11 (0.01)
10
0.07 (0.01)
0.11 (0.01)
10
0.07 (0.01)
0.10 (0.01)
10
10
0.06 (0.01)
0.10 (0.01)
20
0.14 (0.03)
0.31 (0.08)
20
0.10 (0.02)
0.33 (0.04)
20
0.17 (0.1)
0.34 (0.04)
20
10
0.10 (0.03)
0.28 (0.05)
50
0.12 (0.02)
0.48 (0.07)
50
0.12 (0.03)
0.47 (0.07)
0.11 (0.01)
0.39 (0.06)
50
10
0.11 (0.02)
0.32 (0.06)
100
0.14 (0.02)
0.58 (0.07)
100
0.13 (0.02)
0.60 (0.06)
100
0.14 (0.03)
0.58 (0.05)
100
10
0.18 (0.04)
0.55 (0.06)
Table 8: Results for Interventional Generation under Distribution Shifts in Causal Mechanisms. We evaluate the robustness of Cond-FiP to distribution shifts in the parametrization of causal mechanisms. We vary the distribution shift controlled by where 1 corresponds to the results in Table 5 Each cell reports the mean (standard error) RMSE over the multiple test datasets for each scenario. We find that Cond-Fil FiP is robust to varying levels of distribution shift in causal mechanisms.
Total Nodes
Shift Level (a)
LIN OUT
RFF OUT
10
0.07 (0.01)
0.10 (0.01)
10
0.07 (0.01)
0.11 (0.01)
10
0.07 (0.01)
0.18 (0.02)
10
10
0.08 (0.01)
0.26 (0.04)
20
0.07 (0.0)
0.12 (0.0)
20
0.07 (0.0)
0.16 (0.01)
0.07 (0.0)
0.30 (0.01)
20
10
0.07 (0.0)
0.41 (0.02)
50
0.07 (0.01)
0.14 (0.01)
50
0.07 (0.01)
0.19 (0.01)
50
0.07 (0.01)
0.33 (0.02)
50
10
0.07 (0.01)
0.44 (0.02)
100
0.07 (0.01)
0.16 (0.01)
100
0.07 (0.01)
0.22 (0.0)
100
0.07 (0.01)
0.35 (0.01)
100
10
0.07 (0.01)
0.44 (0.01)
Table 9: Results for Noise Prediction under Distribution Shifts in Noise Variables. We evaluate the robustness of Cond-FiP to distribution shifts in the parametrization of noise distribution. We vary the distribution shift controlled by a, where mean (standard error) RMSE over the multiple test datasets for each scenario. We find that Cond- FiP is sensitive to varying levels of distribution shift in noise variables, its performance decrenses with increasing magnitude of the shift.
corresponds to the results in Table 3 Each cell reports the
28

---

Published in Transactions on Machine Learning Research (12/2025)
Total Nodes
Shift Level (a)
LIN OUT
RFF OUT
10
0.05 (0.01)
0.08 (0.01)
10
0.05 (0.0)
0.13 (0.03)
10
0.05 (0.01)
0.28 (0.06)
10
10
0.05 (0.01)
0.36 (0.08)
20
0.07 (0.01)
0.30 (0.03)
20
0.07 (0.01)
0.45 (0.04)
20
0.07 (0.01)
0.59 (0.03)
20
10
0.07 (0.01)
0.58 (0.02)
50
0.07 (0.0)
0.48 (0.07)
50
0.07 (0.0)
0.59 (0.06)
50
0.07 (0.0)
0.64 (0.03)
50
10
0.07 (0.0)
0.58 (0.02)
100
0.09 (0.01)
0.57 (0.07)
100
0.09 (0.01)
0.63 (0.05)
100
0.09 (0.01)
0.65 (0.03)
100
10
0.09 (0.01)
0.59 (0.02)
Table 10: Results for Sample Generation under Distribution Shifts in Noise Variables. We evaluate the nobustness of Cond-FiP to distribution shifts in the paramelrization of noise distribution. We vary the distribution shift controlled by a, where mean (standard error) RMSE over the multiple test datasets for each scenario. We find that Cond- FiP is sensilive to varying levels of distribution shift in noise variables, ils performance decrenses with increasing magnitude of the shift.
corresponds to the results in Table 1 Each cell reports the
Total Nodes
Shift Level (a)
LIN OUT
RFF OUT
10
0.07 (0.01)
0.11 (0.01)
10
0.07 (0.01)
0.14 (0.02)
10
0.07 (0.01)
0.25 (0.05)
10
10
0.07 (0.01)
0.32 (0.06)
20
0.14 (0.03)
0.31 (0.03)
20
0.14 (0.03)
0.42 (0.03)
20
0.14 (0.03)
0.57 (0.03)
20
10
0.14 (0.03)
0.56 (0.02)
50
0.12 (0.02)
0.48 (0.07)
50
0.12 (0.01)
0.58 (0.06)
50
0.12 (0.01)
0.65 (0.04)
50
10
0.12 (0.01)
0.59 (0.02)
100
0.14 (0.02)
0.58 (0.07)
100
0.14 (0.02)
0.65 (0.06)
100
0.14 (0.02)
0.67 (0.04)
100
10
0.14 (0.02)
0.60 (0.03)
Table 11: Results for Interventional Generation under Distribution Shifts in Noise Variables. We evaluate the robustness of Cond-FiP to distribution shifts in the parametrization of noise distribution. We vary the distribution shift controlled by where reports the mean (standard error) RMSE over the multiple test datasets for each scenario. We find that Cond- FiP is sensitive to varying levels of distribution shift in noise variables, its performance decreases with increasing magnitude of the shift.
corresponds to the results in Table 5 Each cell
29

---

Published in Transactions on Machine Learning Research (12/2025)
E Experiment on Generalization in Scarce Data Regime on AVICI benchmark
E.1 Experiments with DeDes = 100
In this section we benchmark Cond-FiP against the basclines for the scenario when test datascts in the input context have smaller sample size (ND 100) as compared to the train datasets (ID 400) in Appendix C
We report the results for the task of noise prediction, sample generation, and interventional generation Table 12 Table 13 and Table 14 respectively. We find that Cond-FiP exhibits superior generalization compared to baselines. For example, in the case of RFF IN, Cond-FiP is even better than FiP for all the tasks! This can be attributed to the advantage of amortized inference; as the sample size in test dataset decreases, the generalization of baselines would be affected a lot since they require training from scratch on these datasets. However, amortized inference methods would be impacted less as they do not have to trained from scratch, and the inductive bias learned by them can help them generalize even with smaller input context.
Method Total Nodes
LIN IN
RFF IN
LIN OUT
RFF OUT
DoWhy DECI
10
0.06 (0.01)
0.22 (0.03)
0.09 (0.01)
0.16 (0.03)
10
0.15 (0.01)
0.3 (0.02)
0.22 (0.01)
0.3 (0.03)
FiP
10
0.07 (0.01)
0.18 (0.01)
0.12 (0.01)
0.11 (0.01)
Cond-FiP
10
0.07 (0.01)
0.14 (0.01)
0.09 (0.01)
0.14 (0.01)
DoWhy
20
0.06 (0.01)
0.27 (0.05)
0.07 (0.01)
0.37 (0.01)
DECI
20
0.15 (0.02)
0.33 (0.02)
0.17 (0.02)
0.35 (0.03)
FiP
20
0.09 (0.01)
0.21 (0.03)
0.1 (0.01)
0.27 (0.03)
Cond-FiP
20
0.08 (0.01)
0.12 (0.01)
0.1 (0.01)
0.15 (0.01)
DoWhy
0.06 (0.01)
0.29 (0.04)
0.05 (0.01)
0.47 (0.04)
DECI
50
0.14 (0.01)
0.33 (0.02)
0.14 (0.02)
0.4 (0.03)
FiP
50
0.08 (0.01)
0.23 (0.03)
0.08 (0.01)
0.37 (0.04)
Cond-FiP
50
0.08 (0.0)
0.12 (0.01)
0.08 (0.01)
0.15 (0.01)
DoWhy
100
0.06 (0.01)
0.31 (0.04)
0.06 (0.01)
0.5 (0.03)
DECI
100
0.13 (0.01)
0.36 (0.03)
0.12 (0.02)
0.44 (0.02)
FiP
100
0.08 (0.01)
0.25 (0.04)
0.1 (0.01)
0.39 (0.03)
Cond-FiP
100
0.07 (0.0)
0.13 (0.01) 0.08 (0.01)
0.17 (0.01)
Table 12: Results for Noise Prediction with Smaller Sample Size (RD... = 100). We compare Cond-FiP against the baselines for the task of predicting noise variable from input observations. Each test dataset contains 100 samples, as opposed to 400 samples in Table 3 Each cell reports the mean (standard error) RMSE over the multiple test datasets for each scenario. Shaded rows deonte the case where the graph size is larger than the train graph sizes (d = 20) for Cond-FiP. Results show that Cond-Fit FiP generalizes much beller than the boselines in this low-dola regime.
30

---

Published in Transactions on Machine Learning Research (12/2025)
Method
Total Nodes
LIN IN
RFF IN
LIN OUT
RFF OUT
DoWhy
10
0.1 (0.01)
0.3 (0.06)
0.12 (0.02)
0.19 (0.03)
DECI
10
0.23 (0.01)
0.45 (0.04)
0.31 (0.02)
0.38 (0.04)
Fip
10
0.13 (0.01)
0.29 (0.04)
0.18 (0.02)
0.15 (0.03)
Cond-FiP
10
0.09 (0.01)
0.2 (0.03)
0.09 (0.02)
0.14 (0.02)
DoWhy
20
0.11 (0.01)
0.47 (0.15)
0.11 (0.02)
0.5 (0.03)
DECI
20
0.26 (0.02)
0.53 (0.05)
0.26 (0.03)
0.57 (0.04)
FiP
0.17 (0.02)
0.34 (0.06)
0.17 (0.02)
0.39 (0.03)
Cond-FiP
20
0.08 (0.0)
0.31 (0.06)
0.13 (0.01)
0.37 (0.02)
DoWhy
50
0.11 (0.01)
0.42 (0.08)
0.09 (0.01)
0.66 (0.06)
DECI
50
0.23 (0.02)
0.59 (0.08)
0.27 (0.04)
0.73 (0.06)
FiP
50
0,13 (0.01)
0.38 (0.07)
0.14 (0.01)
0.58 (0.06)
Cond-FiP
0.1 (0.01)
0.32 (0.05)
0.12 (0.01)
0.54 (0.05)
DoWhy
100
0.11 (0.01)
0.44 (0.08)
0.11 (0.01)
0.74 (0.05)
DECI
100
0.25 (0.02)
0.62 (0.08)
0.25 (0.01)
0.78 (0.07)
FiP
100
0.15 (0.01)
0.4 (0.07)
0.19 (0.02)
0.67 (0.07)
Cond-FiP
100 0.11 (0.01)
0.35 (0.07)
0.14 (0.02)
0.63 (0.07)
Table 13: Results for Sample Generation with Smaller Sample Size (R'ii 100). We compare Cond-FiP against the baselines for the task of generating samples from the input noise variable. Each test. dataset contains 100 samples, opposed to 400 samples in Table 4 Each cell reports the mean (standard error) RMSE over the multiple test datascts for cach scenario. Shaded rows deonte the case where the graph size is larger than the train graph sizes (d= = 20) for Cond-FiP. Results show that Cond-FiP generalizes much better than the baselines in this low- data regime.
Method
Total Nodes
LIN IN
RFF IN
LIN OUT
RFF OUT
DoWhy
10
0.09 (0.01)
0.34 (0.08)
0.11 (0.01)
0.2 (0.04)
DECI
10
0.24 (0.02)
0.43 (0.04)
0.26 (0.03)
0.35 (0.04)
FiP
10
0.13 (0.01)
0.29 (0.04)
0.14 (0.02)
0.14 (0.03)
Cond-FiP
10
0.09 (0.02)
0.21 (0.03)
0.09 (0.01)
0.12 (0.02)
DoWhy
20
0.1 (0.01)
0.37 (0.08)
0.11 (0.02)
0.49 (0.04)
DECI
20
0.25 (0.03)
0.5 (0.05)
0.28 (0.03)
0.54 (0.04)
Fip
0.16 (0.01)
0.33 (0.06)
0.2 (0.03)
0.38 (0.03)
Cond-FiP
20
0.1 (0.01)
0.27 (0.05)
0.15 (0.02)
0.29 (0.03)
DoWhy
50
0.12 (0.02)
0.49 (0.14)
0.09 (0.01)
0.64 (0.07)
DECI
50
0.26 (0.08)
0.56 (0.07)
0.26 (0.03)
0.72 (0.06)
Fip
50
0.16 (0.02)
0.36 (0.06)
0.15 (0.01)
0.57 (0.06)
Cond-FiP
50
0.13 (0.02)
0.29 (0.04)
0.12 (0.01)
0.49 (0.07)
DoWhy
100
0.11 (0.01)
0.46 (0.07)
0.11 (0.01)
1.16 (0.38)
DECI
100
0.24 (0.02)
0.62 (0.08)
0.26 (0.01)
0.78 (0.07)
FiP
100
0.16 (0.02)
0.39 (0.07)
0.2 (0.02)
0.66 (0.07)
Cond-FiP
100
0.12 (0.02)
0.32 (0.07)
0.13 (0.01)
0.58 (0.07)
Table 14: Results for Interventional Generation with Smaller Sample Size (TD 100). We compare Cond-FiP against the baselines for the task of generating interventional data from the input noise variable. Each test dataset contains 100 samples, as opposed to 400 samples in Table 5 Each cell reports the mean (standard error) RMSE over the multiple test datasets for each scenario. Shaded rows deonte the case where the graph size is larger than the train graph sizes (d= = 20) for Cond-FiP. Results show that Cond-FiP generolizes much better than the baselines in this tow-data regime.
31

---

Published in Transactions on Machine Learning Research (12/2025)
E.2 Experiments with DeDros 50
We conduct more experiments for the smaller sample size scenarios, where decrease the sample size even further to Ru = 50 samples. We report the results for the task of noise prediction, sample generation, and interventional generation in Table [5] Table 16, and Table [7 respectivoly. We find that baselines perform much worse than Cond-FiP for the all different SCM distributions, highlighting the efficacy of Cond-FiP for inferring causal mechanisms when the input context has smaller sample size. Note that there were issues with training DoWhy for such a small dataset, hence we do not consider them for this scenario.
Method
Total Nodes
LIN IN
RFF IN
LIN OUT
RFF OUT
DECI
10
0.19 (0.02)
0.41 (0.03)
0.2 (0.02)
0.42 (0.04)
FiP
10
0.13 (0.03)
0.27 (0.03)
0.15 (0.02)
0.21 (0.03)
Cond-FiP
10
0.09 (0.01)
0.17 (0.01)
0.11 (0.01)
0.16 (0.01)
DECI
20
0.2 (0.01)
0.42 (0.03)
0.25 (0.04)
0.45 (0.05)
FiP
20
0.12 (0.01)
0.33 (0.04)
0.15 (0.02)
0.35 (0.04)
Cond-FiP
20
0.1 (0.01)
0.16 (0.01)
0.11 (0.01)
0.17 (0.01)
DECI
50
0.2 (0.02)
0.43 (0.02)
0.2 (0.03)
0.5 (0.05)
FiP
0.13 (0.01)
0.32 (0.03)
0.13 (0.01)
0.49 (0.05)
Cond-FiP
50
0.1 (0.01)
0.16 (0.0)
0.1 (0.01)
0.17 (0.01)
DECI
100
0.19 (0.02)
0.43 (0.03)
0.21 (0.01)
0.53 (0.02)
FiP
100
0.11 (0.01)
0.32 (0.04)
0.13 (0.01)
0.48 (0.02)
Cond-FiP
100
0.09 (0.01)
0.16 (0.01)
0.09 (0.01)
0.18 (0.01)
Table 15: Results for Noise Prediction with Smaller Sample Size (RD,... = 50). We compare Cond-FiP against. the baselines for the task of predicting noise variable from input observations. Fach test dataset. contains 50 samples, as opposed to 400 samples in Table 3 Each cell reports the mean (standard error) RMSE over the multiple test datasets for each scenario. Shaded rows denote the case where the graph size is larger than the train graph sizes (d = 20) for Cond-FiP, Results show that Cond- FiP generalizes much better than the baselines in this low-data regime.
32

---

Published in Transactions on Machine Learning Research (12/2025)
Method
Total Nodes
LIN IN
RFF IN
LIN OUT
RFF OUT
DECI
10
0.31 (0.02)
0.58 (0.05)
0.27 (0.04)
0.49 (0.07)
FiP
10
0.2 (0.03)
0.4 (0.05)
0.21 (0.03)
0.25 (0.04)
Cond-FiP
10
0.12 (0.02)
0.28 (0.03)
0.12 (0.01)
0.18 (0.03)
DECI
20
0.34 (0.02)
0.66 (0.08)
0.39 (0.07)
0.68 (0.05)
FiP
0.2 (0.01)
0.51 (0.08)
0.25 (0.04)
0.51 (0.02)
Cond-FiP
20
0.13 (0.01)
0.4 (0.06)
0.19 (0.02)
0.43 (0.02)
DECI
50
0.32 (0.02)
0.66 (0.06)
0.36 (0.02)
0.8 (0.06)
FiP
50
0.2 (0.01)
0.48 (0.07)
0.22 (0.02)
0.69 (0.06)
Cond-FiP
50
0,15 (0.02)
0.4 (0.05)
0.16 (0.01)
0.59 (0.06)
DECI
100
0.36 (0.04)
0.68 (0.08)
0.39 (0.03)
0.84 (0.06)
Fip
100
0.2 (0.02)
0.49 (0.09)
0.28 (0.03)
0.73 (0.07)
Cond-FiP
100 0.16 (0.01)
0.42 (0.07)
0.22 (0.01)
0.65 (0.06)
Table 16: Results for Sample Generation with Smaller Sample Size (nD,.... = 50). We compare Cond-FiP against the basclines for the task of generating samples from the input noise variable. Each test. dataset contains 50 samples, as opposed to 400 samples in Table | Each cell reports the mean (standard error) RMSE over the multiple test datasets for each scenario. Shaded rows denote the case where the graph size is larger than the train graph sizes (d = 20) for Cond-FiP. Results show that Cond-Fit FiP generalizes much better than the boselines in this low-data regime.
Method
Total Nodes
LIN IN
RFF IN
LIN OUT
RFF OUT
DECI
10
0.3 (0.03)
0.53 (0.05)
0.26 (0.04)
0.42 (0.05)
FiP
10
0.21 (0.04)
0.35 (0.04)
0.2 (0.03)
0.22 (0.03)
Cond-FiP
10
0.12 (0.01)
0.19 (0.03)
0.07 (0.01)
0.14 (0.02)
DECI
20
0.33 (0.02)
0.6 (0.06)
0.43 (0.07)
0.63 (0.04)
FiP
0.21 (0.02)
0.46 (0.07)
0.29 (0.04)
0.49 (0.02)
Cond-FiP
20
0.11 (0.01)
0.29 (0.06)
0.15 (0.02)
0.32 (0.03)
DECI
50
0.34 (0.02)
0.66 (0.07)
0.34 (0.02)
0.78 (0.06)
FiP
50
0.21 (0.02)
0.46 (0.07)
0.23 (0.02)
0.68 (0.06)
Cond-FiP
50
0.13 (0.02)
0.31 (0.05)
0.12 (0.02)
0.51 (0.07)
DECI
100
0.37 (0.04)
0.67 (0.08)
0.4 (0.04)
0.84 (0.06)
FiP
100
0.21 (0.02)
0.49 (0.08)
0.28 (0.03)
0.73 (0.07)
Cond-FiP
100
0.12 (0.01)
0.33 (0.07)
0.14 (0.01)
0.58 (0.07)
Table 17: Results for Interventional Generation with Smaller Sample Size (ND = 50). We compare Cond-FiP against the baselines for the task of generating interventional data from the input noise variable. Each test dataset contains 50 samples, as opposed to 400 samples in Table 5 Each cell reports the mean (standard error) RMSE over the multiple test datasets for each scenario. Shaded rows deonte the case where the graph size is larger than the train graph sizes (d = 20) for Cond-FiP. Results show that Cond- FiP generolizes much better than the baselines in this tow-data regime.
33

---

Published in Transactions on Machine Learning Research (12/2025)
F
Experiments without True Causal Graph on AVICI Benchmark
Results in Appendix C (Table 3 Table Table 5 require the knowledge of truc graph (g) as part of the input context to Cond-FiP. In this section we conduct where we don't provide the true graph in the input context, rather we infer the graph 9 using an amortized causal discovery approach (AVICI (Lorch et al. 2022 from the observational data Dx We chose AVICI for this task since it can enable to amortized inference of causal graphs, hence allowing the combined pipeline of AVICI + Cond-FiP can perform amortizod inference of SCMs. More precisely, AVTCI infers the graph from a novel instance g from input context Dx without updating any parameters, and we pass (9, Dx) as the input context for Cond-FiP. Therefore, for any E R4, Cond-FiP ( (T(z, Dx.9) aims to replicate the functional mechanism F()) of the underlying SCM.
The results for benchmarking Cond-FiP with inferred graphs using AVICI for the task of noise prediction, sample generation, and interventional generation are provided in Table I8 Table 19 and Table 20 respectively. For a fair comparison, the baselines FiP, DECI, and DoWhy also use the inferred graph (인) by AVICI instead of the true graph (9). We find that Cond-Fit remains competitive baselines even for the scenario of unknown true causal graph. Hence, our training procedure can be extended for amortized inference of both causal graphs and causal mechanisms of the SCM.
Method
Total Nodes
LIN IN
RFF IN
LIN OUT
RFF OUT
DoWhy
10
0.16 (0.05)
0.24 (0.04)
0.12 (0.03)
0.12 (0.02)
DECI
10
0.21 (0.05)
0.29 (0.04)
0.16 (0.03)
0.19 (0.04)
FiP
10
0.16 (0.05)
0.2 (0.04)
0.13 (0.03)
0.09 (0.01)
Cond-FiP
10
0.15 (0.05)
0.2 (0.04)
0.13 (0.03)
0.11 (0.01)
DoWhy
20
0.19 (0.05)
0.22 (0.03)
0.2 (0.03)
0.26 (0.01)
DECI
20
0.23 (0.05)
0.28 (0.03)
0.24 (0.04)
0.28 (0.02)
FiP
20
0.2 (0.05)
0.2 (0.03)
0.21 (0.03)
0.21 (0.02)
Cond-FiP
20
0,18 (0.05)
0.17 (0.02)
0.21 (0.03)
0.16 (0.02)
DoWhy
50
0.44 (0.05)
0.3 (0.03)
0.51 (0.03)
0.38 (0.04)
DECI
50
0.46 (0.05)
0.33 (0.04)
0.52 (0.03)
0.42 (0.05)
FiP
50
0.44 (0.05)
0.28 (0.04)
0.51 (0.03)
0.35 (0.05)
Cond-FiP
50
0.43 (0.05)
0.24 (0.03)
0.53 (0.03)
0.29 (0.04)
DoWhy
100
0.49 (0.06)
0.38 (0.03)
0.64 (0.03)
0.53 (0.04)
DECI
100
0.5 (0.06)
0.41 (0.03)
0.64 (0.03)
0.55 (0.03)
FiP
100
0.49 (0.06)
0.37 (0.03)
0.64 (0.03)
0.51 (0.04)
Cond-FiP
100
0.48 (0.06) 0.34 (0.03)
0.64 (0.03)
0.49 (0.04)
Table 18: Results for Noise Prediction without True Graph. We compare Cond-FiP against the basclines for the task of prodicting noise variable from input observations. Unlike experiments Table 3 the true graph G not present in input context, rather its inferred via AVICI (Lorch al.] 2022). Each cell reports the mean (standard error) RMSE over the multiple test datasets for each scenario. Shaded rows deonte the case where the graph size is larger than the train graph sizes (d 20) for Cond-FiP. Results indicate Cond-Fit FiP can generalize to novel instances even in the absence of true graph.
34

---

Published in Transactions on Machine Learning Research (12/2025)
Method
Total Nodes
LIN IN
RFF IN
LIN OUT
RFF OUT
DoWhy
10
0.22 (0.07)
0.29 (0.05)
0.13 (0.01)
0.14 (0.02)
DECI
10
0.29 (0.06)
0.39 (0.05)
0.18 (0.04)
0.22 (0.05)
Fip
10
0.23 (0.06)
0.26 (0.05)
0.15 (0.04)
0.12 (0.02)
Cond-FiP
10
0.22 (0.07)
0.26 (0.05)
0.13 (0.04)
0.11 (0.02)
DoWhy
20
0.25 (0.05)
0.38 (0.06)
0.29 (0.06)
0.42 (0.03)
DECI
20
0.3 (0.06)
0.52 (0.07)
0.34 (0.06)
0.47 (0.04)
FiP
0.26 (0.05)
0.37 (0.07)
0.3 (0.06)
0.33 (0.04)
Cond-FiP
20
0.24 (0.05)
0.36 (0.06)
0.29 (0.06)
0.35 (0.03)
DoWhy
50
0.53 (0.07)
0.46 (0.06)
0.58 (0.03)
0.59 (0.07)
DECI
50
0.55 (0.07)
0.54 (0.07)
0.59 (0.02)
0.66 (0.06)
FiP
50
0.53 (0.07)
0.44 (0.05)
0.58 (0.02)
0.53 (0.07)
Cond-FiP
0.52 (0.07)
0.43 (0.05)
0.58 (0.02)
0.53 (0.07)
DoWhy
100
0.67 (0.07)
0.52 (0.06)
0.69 (0.02)
0.68 (0.04)
DECI
100
0.69 (0.08)
0.57 (0.08)
0.69 (0.02)
0.71 (0.04)
FiP
100
0.66 (0.07)
0.5 (0.07)
0.68 (0.02)
0.64 (0.05)
Cond-FiP
100
0.64 (0.06)
0.49 (0.06) 0.68 (0.02)
0.63 (0.05)
Table 19: Results for Sample Generation without True Graph. We compare Cond-FiP against the baselines for the lask of generating samples from the input noise variable. Unlike experiments in Table 4 the true graph G is not present in imput context, rather its inferred via AVICI (oocc et al. 2022).. Each cell reports the mean (standard error) RMSE over the multiple test datascts for each scenario. Shaded rows deonte the case where the graph size is larger than the train graph sizes (d 20) for Cond-FiP. Results indicate Cond-Fit FiP can generalize to novel instances even in the absence of true graph.
Method
Total Nodes
LIN IN
RFF IN
LIN OUT
RFF OUT
DoWhy
10
0.32 (0.09)
0.3 (0.05)
0.13 (0.04)
0.13 (0.02)
DECI
10
0.37 (0.08)
0.39 (0.05)
0.17 (0.03)
0.21 (0.04)
FiP
10
0.32 (0.08)
0.27 (0.05)
0.14 (0.04)
0.1 (0.02)
Cond-FiP
10
0.31 (0.08)
0.3 (0.05)
0.14 (0.04)
0.13 (0.02)
DoWhy
20
0.29 (0.06)
0.38 (0.07)
0.37 (0.05)
0.4 (0.03)
DECI
20
0.34 (0.06)
0.51 (0.07)
0.41 (0.05)
0.43 (0.03)
Fip
0.3 (0.06)
0.37 (0.07)
0.38 (0.05)
0.31 (0.03)
Cond-FiP
20
0.29 (0.06)
0.37 (0.06)
0.37 (0.05)
0.33 (0.03)
DoWhy
50
0.54 (0.08)
0.45 (0.06)
0.62 (0.04)
0.57 (0.06)
DECI
50
0.57 (0.08)
0.52 (0.07)
0.63 (0.03)
0.64 (0.06)
Fip
50
0.55 (0.08)
0.43 (0.05)
0.62 (0.03)
0.51 (0.07)
Cond-FiP
50
0.54 (0.08)
0.43 (0.05)
0.62 (0.03)
0.51 (0.06)
DoWhy
100
0.66
(0.06)
0.52 (0.07)
0.71 (0.05)
0.65 (0.05)
DECI
100
0.68 (0.07)
0.58 (0.09)
0.71 (0.05)
0.7 (0.04)
FiP
100
0.65 (0.06)
0.51 (0.07)
0.71 (0.05)
0.62 (0.05)
Cond-FiP
100
0.64 (0.06) 0.49 (0.06)
0.7 (0.04)
0.62 (0.05)
Table 20: Results for Interventional Generation without True Graph. We compare Cond-FiP against. the baselines for the task of interventional data. from the input noise variable. Unlike experiments in Table 5 the true graph g is not present in input context, rather its inferred via AVICI (Lorch et al.) 2022 Each cell reports the mean (standard error) RMSE over the multiple test datasets for each scenario. Shaded rows deonte the case where the graph size is larger than the train graph sizes (d = 20) for Cond-FiP. Results indicate Cond-Fit can generalize to novel instances even in the absence of true graph.
35

---

Published in Transactions on Machine Learning Research (12/2025)
F.1 Analyzing Cond-FiP's Sensitivity to Causal Graph Structure
To better understand the ability of Cond-FiP to capture graph-specific structure, conduct systematic analysis by introducing random perturbations (p) to the true causal graph. Specifically, we randomly remove a proportion p of the true edges, such that on average x total odges are missing. We report results for sample generation in the challenging OOD setting (d=100 & RFF OUT).
Our findings (Table 21 show that Cond-FiP's performance shows significant difference after we remove edges (p 0.05). Across all tested levels of perturbation, Cond-FiP is competitive with FiP, baseline trained from scratch for each scenario. These results demonstrate that Cond-FiP exhibits robustness to moderate errors in the input causal graph, comparable to baseline trained from scratch. This provides more evidence to our claim that Cond-FiP learns causal mechanisms during training and can adapt to new contexts at test time by inferring functions that best explain the available information in the context (input graph and observations), even when the input causal graph is inaccurate.
p
p 0.01
p 0.02
0.05
0.1
FiP
0.55 (0.08)
0.55 (0.08)
0.57 (0.08)
0.62 (0.08)
0.68 (0.08)
Cond-FiP
0.57 (0.07)
0.58 (0.07)
0.59 (0.07) 0.62 (0.07)
0.67 (0.07)
Table 21: Robustness to causal graph errors. We compare Cond-FiP for varying levels of corruption (p) to the true causal graph by randomly removing proportion p of edges. Results show that Cond-FiP remains competitive with FiP, even the imput graph becomes increasingly inaccurate, demonstrating robustness to moderate structural errors and its ability to utilize the given imperfect causal information.
36

---

Published in Transactions on Machine Learning Research (12/2025)
G Ablation Study on AVICI benchmark
G.1 Analyzing the Effect of Pretraining Scale
To better understand the scaling of Cond-FiP w.r.t the pretraining data, we conduct experiments at smaller scales, with a total of le5, 4e5, le6 SCMs, as opposed to using 4e6 SCMs in our main body results.
Table 22 reports sample-generation accuracy for the = 20 setting, As expected, Cond-FiP benefits consistently from additional pretraining data: all four metrics (LIN IN, RFF IN, LIN OUT, RFF OUT) improve monotonically with scale. The most pronounced gains appear when increasing the pretraining size from Ie5 to 4e5, while improvements become more incremental beyond 1e6. These results highlight that Cond-FiP continues to leverage larger synthetic datasots, though the returns gradually diminish as scale
increases.
Method
Total Nodes
LIN IN
RFF IN
LIN OUT
RFF OUT
Cond-FiP(1e5)
20
0.11 (0.01)
0.32 (0.06)
0.16 (0.03)
0.44 (0.04)
Cond-FiP(4et)
20
0.07 (0.01)
0.28 (0.06)
0.11 (0.01)
0.37 (0.03)
Cond-FiP(le6)
20
0.07 (0.01)
0.25 (0.06)
0.09 (0.01)
0.33 (0.03)
Cond-FiP(4e6)
20
0.05 (0.01)
0.24 (0.06) 0.07 (0.01) 0.30 (0.03)
Table 22: Effect of Pretraining Scale. Sample generation performance of Cond-FiP (d= 20) under varying amounts pretraining data. Larger synthetic SCM pretraining datasets lead consistent improvements across all scenarios, with diminishing but still positive gains beyond 1e6 SCMs.
G.2 Ablation Study of Encoder
We conduct an ablation study where we train two variants of the encoder Cond-FiP described as follows:
Cond-FiP (LIN): We sample SCMs with linear causal mechanisms during training of the encoder.
Cond-FiP (RFF): We sample SCMs with non- con-linear causal mechanisms during training of the encoder.
Note that for the training the su subsequent decoder, we sample SCMs with both linear and rff causal mechanisms as in the main results Table 3 Table 4 and Table 5). Note that in the main results, the encoder was trained by sampling SCMs with both lincar and rff functional relationships. Hence, this ablation helps us to understand whether the strategy of training encoder on mixed functional relationships can bring more generalization to the amortization process, or if we should have trained encoders specialized for linear and non-linear functional relationships.
We present. our results of the ablation study for the lask of noise prediction, sample generation, and interventional generation in Table 23 Table 24 Table 25 respectively. Our findings indicate that Cond-FiP is robust to the choice of encoder training strategy! Even though the encoder for Cond-FiP (RFF) was only trained on data from non-linear SCMs, its generalization performance is similar to Cond-Fij where the encoder trained data from both linear and non-linear SCMs.
37

---

Published in Transactions on Machine Learning Research (12/2025)
Method
Total Nodes
LIN IN
RFF IN
LIN OUT
RFF OUT
Cond-FiP(LIN)
10
0.07 (0.01)
0.21 (0.02)
0.08 (0.01)
0.2 (0.03)
Cond-FiP(RFF)
10
0.06 (0.01)
0.11 (0.01)
0.07 (0.01)
0.09 (0.01)
Cond-FiP
10
0.06 (0.01)
0.1 (0.01)
0.07 (0.01)
0.1 (0.01)
(LLd-FiP(LIN)
20
0.07 (0.01)
0.19 (0.02)
0.09 (0.01)
0.21 (0.01)
Cond-FiP(RFF)
20
0.06 (0.01)
0.09 (0.01)
0.1 (0.02)
0.11 (0.01)
Cond-FiP
20
0.06 (0.01)
0.09 (0.01)
0.07 (0.0)
0.12 (0.0)
(Ond-FiP(LIN)
50
0.07 (0.01)
0.21 (0.02)
0.07 (0.01)
0.24 (0.01)
Cond-FiP(RFF)
50
0.07 (0.01)
0.09 (0.01)
0.07 (0.0)
0.14 (0.01)
Cond-FiP
50
0.06 (0.01)
0.1 (0.01)
0.07 (0.01)
0.14 (0.01)
Cond-FiP(LIN)
100
0.06 (0.0)
0.22 (0.02)
0.07 (0.01)
0.26 (0.01)
(RFFF
100
0.06 (0.01)
0.09 (0.01)
0.07 (0.01)
0.14 (0.01)
Cond-FiP
100
0.05 (0.0)
0.1 (0.01)
0.07 (0.01)
0.16 (0.01)
Table 23: Encoder Ablation for Noise Prediction. We compare Cond-FiP against the baselines for the task of predicting noise variable from input observations against two variants. One variant corresponds to the encoder trained on SCMs with only lincar functional relationships, Cond-F1P(LIIN). Similarly, we have another variant where the decoder was trained on SCMs with only rff functional relationships, Cond-FiP(RFT), Each cell reports the mean (standard error) RMSE over the multiple test datasets for each scenario. Results show that training on only non-linear SCMs (Cond-PAP(RFF)) gives similor performance as training on both linear and non-linear SCMs (Cond-FiP).
Method
Total Nodes
LIN IN
RFF IN
LIN OUT
RFF OUT
Cond-FiP(LIN)
10
0.05 (0.01)
0.14 (0.02)
0.06 (0.0)
0.08 (0.01)
CoRF-FiP(RFFF)
10
0.08 (0.01)
0.18 (0.06)
0.06 (0.0)
0.07 (0.01)
Cond-FiP
10
0.06 (0.01)
0.14 (0.02)
0.05 (0.01)
0.08 (0.01)
Cond-FiP(LIN)
20
0.05 (0.01)
0.25 (0.06)
0.07 (0.01)
0.3 (0.03)
Cond-F1P(RFF)
20
0.08 (0.01)
0.22 (0.05)
0.11 (0.01)
0.29 (0.03)
Cond-FiP
20
0.05 (0.01)
0.24 (0.06)
0.07 (0.01)
0.3 (0.03)
Cond-FiP(LIN)
50
0.08 (0.01)
0.26 (0.05)
0.11 (0.04)
0.52 (0.08)
Cond-F1P(RFF)
50
0.11 (0.01)
0.26 (0.05)
0.15 (0.02)
0.48 (0.07)
Cond-FiP
50
0.08 (0.01)
0.25 (0.05)
0.07 (0.0)
0.48 (0.07)
Cond -ond-FiP(LIN)
100
0.07 (0.01)
0.27 (0.06)
0.08 (0.0)
0.57 (0.07)
Cond-FiP(RFF)
100
0.11 (0.01)
0.29 (0.08)
0.18 (0.03)
0.61 (0.08)
Cond-FiP
100
0.07 (0.01)
0.29 (0.07) 0.09 (0.01)
0.57 (0.07)
Table 24: Encoder Ablation for Sample Generation. We compare Cond-FiP against the baselines for the task of generating samples from input noise variables against two variants. One variant corresponds to the encoder trained on SCMs with only linear functional relationships Cond-FiP(LIN). Similarly, we have another variant where the decoder was trained SCMs with only rff functional relationships, Cond-FiP(RFF). Fach cell reports the mean (standard error) RMSE over the multiple test datasets for each scenario. Results show that training on only non-linear SCMs (Cond-FiP(RFF)) gives similar performance as training on both linear and non-linear SCMs (Cond-FiP).
38

---

Published in Transactions on Machine Learning Research (12/2025)
Method
Total Nodes
LIN IN
RFF IN
LIN OUT
RFF OUT
Cond-F1P(LIN)
10
0.09 (0.02)
0.2 (0.03)
0.06 (0.01)
0.1 (0.01)
Cond-FiP(RFT)
10
0.13 (0.04)
0.23 (0.08)
0.08 (0.01)
0.1 (0.01)
Cond-FiP
10
0.1 (0.03)
0.21 (0.03)
0.07 (0.01)
0.11 (0.01)
Cond-F1P(LIN)
20
0.08 (0.01)
0.24 (0.05)
0.12 (0.04)
0.3 (0.03)
Cond-FiP(RFT)
20
0.13 (0.02)
0.23 (0.05)
0.13 (0.08)
0.31 (0.02)
Cond-FiP
20
0.09 (0.01)
0.24 (0.05)
0.14 (0.03)
0.31 (0.03)
FLLNN))
50
0.12 (0.02)
0.29 (0.05)
0.1 (0.01)
0.51 (0.07)
Cond-FiP(RFF)
50
0.14 (0.02)
0.29 (0.05)
0.18 (0.03)
0.47 (0.06)
Cond-FiP
50
0.13 (0.02)
0.27 (0.04)
0.12 (0.02)
0.48 (0.07)
Cond-FiP(LIN)
100
0.1 (0.01)
0.3 (0.06)
0.12 (0.01)
0.56 (0.07)
ConF-F1P(RFF)
100
0.12 (0.01)
0.31 (0.07)
0.2 (0.04)
0.6 (0.09)
Cond-FiP
100
0.1 (0.01)
0.3 (0.06)
0.14 (0.02)
0.58 (0.07)
Table 25: Encoder Ablation for Intorventional Generation. We compare Cond-FiP against the baselines for the lask of generating interventional data from input noise variables against two variants. One variant corresponds to the encoder trained on SCMs with only linear functional relationships, Cond-FiP(LIN), Similarly, have another variant where the decoder trained SCMs with only rff functional relationships, Cond-FiP (RFFF. Each cell reports the mean (standard error) RMSE over the multiple test datasets for each scenario. Resulls show that training on only mon-lineur SCMs (Cond-FiP(RFF)) gives similar performance training on both linear and non linear SCMs (Cond-FiP).
39

---

Published in Transactions on Machine Learning Research (12/2025)
G.3 Ablation Study of Decoder
We conduct an ablation study where we train two variants of the decoder Cond-FiP described as follows:
Cond-FiP (LIN): We sample SCMs with linear functional relationships during training.
Cond-FiP (RFF): We sample SCMs with non-linear functional relationships for training.
Note that in the main results (Table Table show the performances of Cond-FiP trained by sampling SCMs with both linear and non-linear causal mechanisms. Hence, this ablations helps us to understand whether the strategy of training mixed causal mechanisms can bring more generalization to the amortization process, or if we should have trained decoders specialized for linear and non-linear functional relationships.
We present the results of our ablation study Table 26 and Table 27 for the task sample generation and interventional generation respectively. Our findings indicate that Cond-FiP decoder trained for both lincar and non-linear functional relationships is able to specialize for both the scenarios. While Cond-FiP (LIN) is only able to perform well for linear benchmarks, and similarly Cond-FiP (RFF) can only achieve decent predictions for non-linear benchmarks, Cond-FiP is achieve the best performances on both the linear and non-linear benchmarks.
Method
Total Nodes
LIN IN
RFF IN
LIN OUT
RFF OUT
Cond-FiP
10
0.07 (0.02)
0.4 (0.06)
0.07 (0.01)
0.25 (0.06)
Cond-FiP(RFT)
10
0.1 (0.02)
0.15 (0.02)
0.08 (0.01)
0.09 (0.01)
Cond-FiP
10
0.06 (0.01)
0.14 (0.02)
0.05 (0.01)
0.08 (0.01)
Cond-FiP(LIN)
20
0.07 (0.01)
0.44 (0.07)
0.10 (0.01)
0.58 (0.02)
Cond-FiP(RFF)
20
0.11 (0.01)
0.26 (0.06)
0.14 (0.01)
0.31 (0.03)
Cond-FiP
20
0.05 (0.01)
0.24 (0.06)
0.07 (0.01)
0.3 (0.03)
Cond-FLP(LIN)
50
0.10 (0.01)
0.5 (0.07)
0.14 (0.02)
0.69 (0.04)
Cond-FiP(RFT)
50
0.15 (0.02)
0.27 (0.05)
0.19 (0.02)
0.5 (0.07)
Cond-FiP
50
0.08 (0.01)
0.25 (0.05)
0.07 (0.0)
0.48 (0.07)
Cond-FiPP (LIIIFF
100
0.1 (0.01)
0.51 (0.07)
0.15 (0.02)
0.72 (0.04)
Cond-F1P(RFF)
100
0.16 (0.08)
0.29 (0.07)
0.27 (0.04)
0.59 (0.06)
Cond-FiP
100
0.07 (0.01)
0.29 (0.07)
0.09 (0.01)
0.57 (0.07)
Table 26: Decoder Ablation for Sample Generation. We compare Cond-FiP for the task of generating samples from input noise variables against two variants. One variant corresponds to decoder trained on SCMs with only linear functional relationships, Cond-FAP(LIN), Similarly, we have another variant where the decoder was trained on SCMs with only rff functional relationships, Cond-F1P(RFF), Each cell reports the mean (standard error) RMSE over the multiple test datasets for each scenario. Results indicale that training on both linear and non non-linen linear SCMs is crucial to generalize effectively in all scenorios.
40

---

Published in Transactions on Machine Learning Research (12/2025)
Method
Total Nodes
LIN IN
RFF IN
LIN OUT
RFF OUT
(Ond-FIP(LIN)
10
0.09 (0.02)
0.40 (0.07)
0.06 (0.01)
0.22 (0.04)
Cond-FiP(RFF)
10
0.16 (0.05)
0.22 (0.08)
0.08 (0.01)
0.11 (0.01)
Cond-FiP
10
0.10 (0.03)
0.21 (0.03)
0.07 (0.01)
0.11 (0.01)
Cond-FiP(LIN)
20
0.10 (0.01)
0.45 (0.07)
0.16 (0.03)
0.57 (0.02)
Cond-FiP(RFF)
20
0.14 (0.02)
0.26 (0.05)
0.21 (0.03)
0.32 (0.02)
Cond-FiP
20
0.09 (0.01)
0.24 (0.05)
0.14 (0.03)
0.31 (0.03)
Cond-FiP(LIN)
50
0.14 (0.02)
0.49 (0.07)
0.14 (0.02)
0.68 (0.04)
CRFFFF
50
0.19 (0.03)
0.28 (0.05)
0.21 (0.03)
0.49 (0.06)
Cond-FiP
50
0.13 (0.02)
0.27 (0.04)
0.12 (0.02)
0.48 (0.07)
Cond-FiP(LLIN)
100
0.12 (0.02)
0.52 (0.07)
0.18 (0.03)
0.71 (0.01)
(ond-F)P(RFF)
100
0.18 (0.03)
0.32 (0.07)
0.24 (0.04)
0.59 (0.07)
Cond-FiP
100
0.10 (0.01)
0.30 (0.06)
0.14 (0.02)
0.58 (0.07)
Table 27: Decoder Ablation for Interventional Generation. We compare Cond-FiP against two variants for the task of interventional data from input noise variables. One variant corresponds to a decoder trained on SCMs with only linear functional relationships, Cond-FiP(LIN), Similarly, we have another variant where the decoder was trained SCMs with only rff functional relationships, Cond-FiP(RFF), Each cell reports the mean (standard error) RMSE over the multiple test datasets for each scenario. Resulls indicale that training on both linear and lineer SCMs is crucial to generalize effectively in all scenarios.
41

---

Published in Transactions on Machine Learning Research (12/2025)
H Comparing Cond-FiP with CausalNF
We also compare Cond-FiP with CausalNF Javaloy et al. 2023) for the task of noise prediction (Table 28 and sample generation (Table 29 The test datasets consist of Nhest = 400 samples, exact same selup as in and Table 5). To ensure a fair comparison; we provided CausalNF with
our main results (Table 3 Table the true causal graph.
Our analysis reveals that CausalNF underperforms compared to Cond-Fip in both tasks, and it also a weaker baseline relative to FiP. Note also the authors did not experiment with large graphs for CausalNF; the largest graph they used contained approximately 10 nodes. Also, they trained CausalNF on much larger datasets with sample size of 20k, while our setup has datasets with 400 samples only.
Method
Total Nodes
LIN IN
RFF IN
LIN OUT
RFF OUT
CausalNF
10
0.16 (0.02)
0.41 (0.09)
0.38 (0.04)
0.35 (0.02)
Cond-FiP
10
0.06 (0.01)
0.10 (0.01)
0.07 (0.01)
0.10 (0.01)
CausalNF
20
0.18 (0.03)
0.45 (0.12)
0.29 (0.05)
0.36 (0.03)
Cond-FiP
20
0.06 (0.01)
0.09 (0.01)
0.07 (0.00)
0.12 (0.00)
CansalNF
50
0.25 (0.03)
0.56 (0.09)
0.45 (0.06)
0.38 (0.04)
Cond-FiP
50
0.06 (0.01)
0.10 (0.01)
0.07 (0.01)
0.14 (0.01)
CausalNF
100
0.24 (0.02)
0.80 (0.1)
0.37 (0.06)
0.49 (0.05)
Cond-FiP
100
0.05 (0.0) 0.10 (0.01)
0.07 (0.01)
0.16 (0.01)
Table 28: Results for Noise Prediction with CausalNF. We compare Cond-FiP against CausalNF for the task of predicting noise variables from input observations. We find that CausolNF underperforms compared to Cond- FiP by signéficant margin.
Method
Total Nodes
LIN IN
RFF IN
LIN OUT
RFF OUT
CausalNF
10
0.27 (0.07)
0.29 (0.04)
0.20 (0.03)
0.20 (0.03)
Cond-FiP
10
0.06 (0.01)
0.14 (0.02)
0.05 (0.01)
0.08 (0.01)
CausalNF
20
0.23 (0.02)
0.36 (0.05)
0.22 (0.02)
0.45 (0.02)
Cond-FiP
20
0.05 (0.01)
0.24 (0.06)
0.07 (0.01)
0.30 (0.03)
CausalNF
50
1.5 (0.26)
0.93 (0.13)
3.09 (0.55)
0.95 (0.04)
Cond-FiP
50
0.08 (0.01)
0.25 (0.05)
0.07 (0.00)
0.48 (0.07)
CausalNF
100
1.23 (0.13)
0.85 (0.08)
1.67 (0.13)
0.96 (0.04)
Cond-FiP
100
0.07 (0.01)
0.29 (0.07)
0.09 (0.01)
0.57 (0.07)
Table 29: Results for Sample Generation with CausalNF. We compare Cond-FiP against CausalNF for the task of generating samples from input noise variables. We find that CausalNF underperforms compared to Cond-Fil by significant margin.
42

---

Published in Transactions on Machine Learning Research (12/2025)
Limitations of Cond-FiP
1.1 Evaluating Generalization of Cond-Fip to Larger Sample Size
In the main results (Table 3 Table 4 and Table 5), we evaluated Cond-FiP's generalization capabilities to larger graphs (d = 50, d = 100) than those used for training (d = 20). In this section, we carry a similar experiment where instead increasing the total nodes in the graph, we test Cond-FiP on datasets with more samples TeD... 1000, while Cond-FiP was only trained for datasets with sample size np 400.
The results for the experiments are presented in Table 30 Table 31 and Table 32 for the task of noise prediction, sample generation, and interventional generation respectively. Our findings indicate that Cond-FiP is still able to compete with other bascline in this regime. However, we observe that the performances of Cip did not improve by increasing the sample size compared to the results obtained for the 400 samples case, meaning that the performance of our models depends exclusively on the setting used at training time. We leave for future works the learning of a larger instance of Cond-FiP trained on larger sample size problems.
Method
Total Nodes
LIN IN
RFF IN
LIN OUT
RFF OUT
DoWhy
10
0.02 (0.0)
0.10 (0.01)
0.21 (0.04)
0.23 (0.02)
DECI
10
0.05 (0.01)
0.12 (0.01)
0.21 (0.04)
0.27 (0.03)
FiP
10
0.03 (0.0)
0.06 (0.0)
0.21 (0.04)
0.23 (0.02)
Cond-FiP
10
0.05 (0.01)
0.11 (0.01)
0.21 (0.04)
0.25 (0.02)
DoWhy
0.02 (0.0)
0.11 (0.02)
0.16 (0.01)
0.3 (0.02)
DECI
20
0.04 (0.01)
0.11 (0.02)
0.16 (0.01)
0.29 (0.02)
FiP
20
0.03 (0.0)
0.08 (0.02)
0.16 (0.01)
0.26 (0.02)
Cond-FiP
20 0.06 (0.01)
0.09 (0.01)
0.18 (0.01)
0.26 (0.01)
Table 30: Results for Noise Prediction with Larger Sample Size (N.D.. 1000). We compare Cond-FiP against the baselines for the task of predicting noise variables from the input observations. Each cell reports the mean (standard error) RMSE over the multiple test datasets for each scenario. Results indicate that CoP does not yet benefit from larger context sizes at inference, suggesting the need to scale both the model and training data for richer conterts.
43

---

Published in Transactions on Machine Learning Research (12/2025)
Method
Total Nodes
LIN IN
RFF IN
LIN OUT
RFF OUT
DoWhy
10
0.04 (0.0)
0.14 (0.02)
0.29 (0.04)
0.3 (0.03)
DECI
0.07 (0.01)
0.17 (0.02)
0.29 (0.04)
0.33 (0.04)
FiP
10
0.05 (0.0)
0.09 (0.01)
0.29 (0.04)
0.29 (0.03)
Cond-FiP
10
0.05 (0.01)
0.14 (0.02)
0.29 (0.04)
0.29 (0.08)
DoWhy
20
0.04 (0.01)
0.21 (0.05)
0.28 (0.01)
0.55 (0.06)
DECI
20
0.07 (0.01)
0.21 (0.04)
0.29 (0.01)
0.59 (0.06)
FiP
0.05 (0.0)
0.17 (0.04)
0.28 (0.01)
0.53 (0.06)
Cond-FiP
20 0.05 (0.0)
0.24 (0.05)
0.28 (0.01)
0.53 (0.06)
Table 31: Results for Sample Generation with Larger Sample Size (ND... 1000). We compare Cond-FiP against the baselines for the task of generaling samples from the input noise variables. Each cell reports the mean (standard error) RMSE over the multiple test datasets for each scenario. Results indicate that Cond-PiP does not yet benefit from larger contert sizes at inference, suggesting the need to scale both the model and training data for richer conterts.
Method
Total Nodes
LIN IN
RFF IN
LIN OUT
RFF OUT
DoWhy
10
0.04 (0.01)
0.16 (0.03)
0.26 (0.03)
0.27 (0.03)
DECI
0.09 (0.01)
0.19 (0.02)
0.26 (0.03)
0.31 (0.04)
FiP
10
0.05 (0.01)
0.12 (0.02)
0.26 (0.03)
0.27 (0.03)
Cond-FiP
10
0.09 (0.02)
0.19 (0.03)
0.27 (0.03)
0.3 (0.03)
DoWhy
20
0.04 (0.0)
0.20 (0.04)
0.26 (0.01)
0.53 (0.06)
DECI
0.08 (0.01)
0.20 (0.03)
0.29 (0.02)
0.54 (0.05)
FiP
0.06 (0.01)
0.16 (0.04)
0.28 (0.02)
0.48 (0.06)
Cond-FiP
20
0.07 (0.01)
0.27 (0.05) 0.30 (0.02)
0.51 (0.06)
Table 32: Results for Interventional Generation with Larger Sample Size (N 1000). We compare Cond-FiP against the baselines for the task of generating interventional data from the input noise variables. Each cell reports the mean (standard error) RMSE over the multiple test datascts for cach scenario. Results indicate that Cond-FiP does not yet benefil from larger context sizes al inference, suggesling the need to scale both the model and training data for richer contezts.
44

---

Published in Transactions on Machine Learning Research (12/2025)
1.2 Counterfactual Generation with Cond-FiP
We provide results (Table 33) for bechmarking Cond-FiP against baselines for the lask of counterfactual generation. We operate in the same setup as the one in our main results (T..... = 400) Appendix C and all the methods are provided with the truc casual graph. We observe that Unlike the tasks of noise prodiction, sample & interventional generation, we find that Cond-FiP is worse than the baselines for the task of counterfactual generation. This can be explained as the training of Cond-FiP decoder relies on the true noise variables, and the model struggles to generalize the learned functional mechanisms when provided with inferred noise variables. We leave the improvement of Cond-FiP for counterfactual generation future work.
Method
Total Nodes
LIN IN
RFF IN
LIN OUT
RFF OUT
DoWhy
10
0.03 (0.03)
0.13 (0.03)
0.0 (0.0)
0.04 (0.01)
DECI
10
0.1 (0.02)
0.2 (0.03)
0.04 (0.01)
0.11 (0.02)
FiP
10
0.03 (0.01)
0.09 (0.02)
0.02 (0.0)
0.03 (0.01)
Cond-FiP
10
0.09 (0.03)
0.21 (0.03)
0.05 (0.01)
0.11 (0.01)
DoWhy
20
0.01 (0.0)
0.12 (0.03)
0.0 (0.0)
0.13 (0.02)
DECI
20
0.06 (0.01)
0.15 (0.03)
0.07 (0.03)
0.15 (0.02)
FiP
20
0.03 (0.01)
0.1 (0.03)
0.06 (0.04)
0.09 (0.02)
Cond-FiP
20
0.09 (0.02)
0.26 (0.05)
0.13 (0.02)
0.3 (0.03)
DoWhy
50
0.0 (0.0)
0.09 (0.02)
0.0 (0.0)
0.17 (0.04)
DECI
50
0.04 (0.01)
0.11 (0.02)
0.03 (0.01)
0.18 (0.04)
FiP
50
0.03 (0.01)
0.08 (0.02)
0.03 (0.01)
0.14 (0.04)
Cond-FiP
50
0.1 (0.02)
0.26 (0.04)
0.1 (0.01)
0.46 (0.06)
DoWhy
100
0.0 (0.0)
0.08 (0.02)
0.0 (0.0)
0.2 (0.05)
DECI
100
0.02 (0.01)
0.1 (0.02)
0.02 (0.01)
0.22 (0.05)
FiP
100
0.01 (0.01)
0.07 (0.02)
0.02 (0.01)
0.19 (0.05)
Cond-FiP
100 0.09 (0.02)
0.29 (0.06)
0.13 (0.02)
0.56 (0.08)
Table 33: Results for Counterfactual Generation. We compare Cond-FiP against the baselines for the task of generating counterfactual data from the input noise variables. Each cell reports the mean (standard error) RMSE over the multiple test datasets for each scenario. Shaded rows denote the case where the graph size is larger than the train graph sizes (d = 20) for Cond-FiP. Results indicate that Cond-Fit FiP struggles with counterfactual generation and cannot always match the performance of baselines trained from scratch.
45


Reviews:
[{"review_id": "TsUdmLxwkP", "reviewer": "Reviewer_n8pZ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes an amortized inference framework for predicting the causal mechanisms of a Structural Causal Model (SCM) given observed data and a known causal graph.\nUnlike existing methods that require training a separate model for each new dataset, the proposed approach introduces a Transformer-based dataset embedding module, enabling inference of causal mechanisms for unseen graphs and datasets without re-training model parameters.\nThe method is evaluated on three tasks (noise prediction, data generation, and interventional generation) and achieves performance that is comparable to or better than state-of-the-art baselines in both in-distribution and out-of-distribution (OOD) settings.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The claims of the paper are well supported by comprehensive experiments (by section 4.2))\nThe authors demonstrate that the proposed amortized inference framework achieves competitive or superior performance to baselines such as FiP, DECI, and DoWhy across multiple benchmarks and data regimes.\n\n---\n[In-distribution performance]\n\nCond-FiP matches or slightly surpasses baselines when both models are evaluated on graphs drawn from the same distribution as training.\n\n---\n[Out-of-distribution (OOD) generalization]\n\nThe model is evaluated on graphs and data distributions differing from those seen during training including different graph topologies, noise families, and functional parameter shifts. The results show that Cond-FiP maintains stable performance. \n\n---\n[Scarce data regimes]\n\nWhen the number of observational samples is reduced to 50 or 100, Cond-FiP outperforms the baselines trained from scratch.", "audience_yn": "Yes", "audience_explanation": "Researchers interested in causal inference, structural causal models (SCMs), and amortized inference would find this work valuable.", "requested_changes": "I believe the authors’ claims are well supported and the novelty of the work is clear.\nNone of the following suggestions are critical to secure my recommendation; rather, they would further strengthen the paper.\n\n---\n## [Clarifying the benefit of amortization versus additional data]\n\nThe proposed model is persuasive, particularly in demonstrating both efficient inference-time performance and competitive predictive performance.\n\nHowever, it remains somewhat unclear whether the model’s effectiveness truly stems from its ability to capture graph-specific structure, or rather from extensive pretraining over a large number of SCMs.\nTo strengthen this point, it might be informative to include an experiment that tests the model’s sensitivity to graph structure—for instance, by perturbing the input graph or substituting it with mismatched structures during dataset embedding.\nIf such perturbations lead to a notable degradation in performance, it would more convincingly demonstrate that the model genuinely relies on graph-specific information rather than benefiting merely from data scale.\n\n---\n## [Clarifying the motivation for this specific design]\n\nIt might be helpful to elaborate on why this particular amortization structure was chosen over other possible designs. A short conceptual discussion or comparison to plausible alternatives could make the design choice more transparent to readers. Also, it might also be worth discussing whether there exist alternative amortization strategies (beyond FiP-style approaches) that could achieve similar goals. \n\n---\n## [Exploring the effect of pretraining scale]\n\nSince the method relies on large-scale pretraining over synthetic SCMs, it could be informative to examine how performance scales with the number of pretraining SCMs (e.g., from 1e5 to 4e6).\nPresenting this sensitivity analysis (perhaps even as a brief ablation) would help readers understand how the model’s generalization depends on the scale and diversity of the pretraining corpus.", "broader_impact_concerns": "N/A. No concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "LqQHA18J69", "reviewer": "Reviewer_EpFg", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose  an amortized inference framework that trains a single transformer-based model to predict the causal mechanisms of Structural Causal Models conditioned on their observational data and causal graph (using Transformer's in-context learning capabilities)", "claims_and_evidence_yn": "Yes", "claims_explanation": "The results clearly show the capability of the model to learn SCMs using the datasets and causal graphs, in general matching the traditional Fixed-Point Approach (FiP) and being superior to the other baselines, while being more general than them.", "audience_yn": "Yes", "audience_explanation": "The method should allow one to train a single model that can easily adapt to new Structural Causal Models, as long as their data obeys the  Additive Noise Model Assumption. Besides the potential applications, the ease of learning and manipulation of causal models is likely an important feature of general intelligence, and this paper shows that for at least some kinds of causal models that is possible.", "requested_changes": "Some experiments on datasets where the Additive Noise Model Assumption is false to show the impact on performance from a practical perspective (and whether the model can still perform reasonably when ANM is false but an ok approximation). Another important detail missing is a comparison of the computational cost for training and inference of the Cond-FiP model vs the baselines.", "broader_impact_concerns": "None a priori", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "nsGIdyTa1z", "reviewer": "Reviewer_1ybH", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper tackles the problem of learning SCMs in an amortized fashion, aiming to infer causal mechanisms across multiple datasets using a single, reusable model. The authors introduce Cond-FiP, a conditional extension of the recently proposed Fixed-Point (FiP) approach for causal generative modeling. While FiP must be trained separately for each dataset, Cond-FiP incorporates dataset-conditioned embeddings that enable amortized inference of causal mechanisms, allowing the same model to adapt to unseen SCMs without further training.\n\nThe proposed framework consists of two components:\n1) a dataset encoder that learns context embeddings from observational data and causal graphs using transformer-based in-context learning\n2) a conditional decoder that models causal mechanisms as conditional fixed-point iterations, integrating the dataset embeddings through adaptive layer normalization and context-dependent codebooks.\nThis setup allows Cond-FiP to reconstruct noise variables, generate both observational and interventional data, and generalize to unseen causal models\n\nEmpirically, the authors evaluate Cond-FiP on a large suite of synthetic and real-world benchmarks (including AVICI, C-Suite, and the Sachs dataset). The results demonstrate that Cond-FiP achieves performance comparable to or better than per-dataset baselines (FiP, DECI, and DoWhy) while generalizing across out-of-distribution settings, varying graph sizes, unknown causal graphs, and limited data scenarios. The model also shows robustness when applied to unseen data simulators and complex noise distributions.\n\nKey strengths:\n* Introduces a novel amortized inference framework for SCMs, addressing a significant limitation of existing methods that require retraining per dataset.\n* Conceptually elegant integration of dataset conditioning into the FiP architecture through adaptive normalization and context-dependent embeddings.\n* Thorough experimental evaluation across diverse and challenging settings, including tests of generalization, robustness to distribution shifts, and transfer to real-world data.\n* Clear presentation and strong empirical evidence supporting the claims of improved data efficiency and generalization.\n\nKey weaknesses (minor at this stage):\n* The approach relies on the additive noise model assumption during training, limiting applicability to more general classes of SCMs.\n* The encoder requires access to true noise variables, which restricts end-to-end training on real-world data.\n* Experiments focus mostly on synthetic data, and real-world validation remains limited in scope.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The paper’s central assertion of Cond-FiP enabling amortized inference of causal mechanisms across different SCMs without retraining is validated through a comprehensive set of experiments. The authors systematically compare Cond-FiP against strong baselines (FiP, DECI, DoWhy) across multiple benchmarks and evaluation tasks (noise prediction, sample generation, and interventional generation).\n\nThe results consistently show that Cond-FiP performs on par with or better than models trained from scratch, both in-distribution and under substantial distribution shifts, such as unseen causal mechanisms, noise types, and larger graph sizes. Additional experiments on scarce-data settings, unknown causal graphs, and different simulators (C-Suite) provide further support for the generalization claims. The real-world Sachs dataset evaluation (while limited) also confirms that Cond-FiP matches baseline performance even without retraining.", "audience_yn": "Yes", "audience_explanation": "The paper's topic is highly relevant to the TMLR audience, particularly those interested in causal machine learning, meta-learning, and generative modeling. The proposed method addresses a key limitation in current causal generative modeling: the need to train separate models for each dataset. By introducing a conditional, amortized approach, the paper connects ideas from causal inference, transformer-based in-context learning, and meta-learning, three areas that are currently of significant interest in the community.\n\nMoreover, the work contributes to an emerging research direction on “causal foundation models,” which aim to generalize causal reasoning across tasks. The presented method offers a concrete, well-evaluated step in that direction, supported by extensive empirical evidence.", "requested_changes": "Requested Changes\n1) Clarify the role and limitations of the ANM assumption: The training of the dataset encoder relies on the ANM assumption and access to true noise variables, which substantially limits applicability to real-world data. The authors should clearly state this limitation in the main text (not only in the appendix or conclusion) and discuss potential strategies for relaxing this assumption, such as self-supervised or implicit encoding methods.\n2) Improve the presentation of the methodology section: The description of the Cond-FiP architecture is technically detailed but dense, making it difficult to follow the flow from dataset encoding to conditional decoding. Adding a small schematic or pseudo-code block summarizing the data flow (input -> embedding -> conditional codebook -> adaptive transformer -> output) would make the method more accessible to readers less familiar with FiP.\n3) Expand the real-world evaluation: The Sachs dataset experiment is a good starting point but currently limited in scope. Even if only partially feasible, including one or two additional small-scale real datasets (e.g., synthetic biology, causal discovery benchmarks) or reporting qualitative results would demonstrate the model’s broader applicability and enhance the paper’s impact.\n4) Discuss computational efficiency and scaling behavior: The amortized nature of Cond-FiP is claimed to bring computational savings, but no quantitative evidence (e.g., runtime or training cost per dataset) is provided. A small comparison table summarizing training and inference time relative to FiP or DECI would highlight the practical benefits of amortization more.\n5) Minor textual and formatting revisions: A few small typographical inconsistencies (e.g., “scare data regime” instead of “scarce”) should be corrected. Figures 2–4 would benefit from increasing the width (emoving the whitespace) and from using a larger font and clearer legends.", "broader_impact_concerns": "The paper includes a Broader Impact Statement, and it is generally appropriate for a technical contribution of this nature. The work focuses on a methodological advance without targeting any specific downstream application domain. The authors correctly state that the research does not pose foreseeable societal risks in its current form.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)